Have a question for our experts? Ask it here!

Expert Connection

Q:
"I'm a big fan of using curriculum-based measures for evaluating progress in reading and other skill areas, but I need help with using the same ideas for evaluating students' learning in middle- and high-school content. How can I use CBM for mastering science or social studies?" Ken, Denver, CO

A:
Dr. Christine Espin of the College of Education at the University of Minnesota, an expert on many uses of formative evaluation, addresses this question. After she provides background about monitoring progress, she explains exactly how to create and use measures for assessing progress in content areas as well as providing cautions about appropriate use of them.

Brief Description of CBM

For those who may not be familiar with curriculum-based measurement (CBM), it is a system for on-going monitoring of students' progress through a curriculum. It reflects the success of student's instructional programs. If the instructional programs are working well, CBM measures will show that students are making progress. If the programs are not working well, the measures will show little or no gain over time. A Current Practice Alert on formative evaluation recommends that teachers use these sorts of measures for monitoring progress.

CBM assessments, which are often called "probes," are brief and scored objectively. Because they can be administered quickly, it is possible to assess students' performance frequently. When plotted on a graph, the results provide an easy-to-interpret way of determining whether students are making progress.

The early research on CBM focused on the development and use of measures at the elementary-school level in basic skill areas such as reading, written expression, and mathematics. Because of their value in assessing reading, most state education agencies have recommended use of one or another CBM system for schools participating in the Reading First program.

CBM In the Content Areas

Recently, use of CBM methods for monitoring progress has been extended to students at the secondary-school level, not only in basic skill areas, but also in content-learning areas such as social studies and science. Results of this research show that a 5-minute vocabulary-matching measure is a valid and reliable indicator of students' performance and progress in the content areas.

It is relatively easy to develop and use a vocabulary-matching measure. Let me describe how to develop and administer the measures, as well as how we know these measures are effective. Finally, I will identify some cautions about what we still must learn about them.

Creating Vocabulary-matching Probes

It is important that CBM assessments reflect the actual curriculum used with students. Therefore, teachers should develop measures based on their particular curriculum. Fortunately, they can do this easily. Use the following steps to create vocabulary-matching probes.

1. Create a pool of potential vocabulary terms. Develop a pool of important vocabulary terms from the content to be covered over the entire school year (or semester if the class is offered on a semester basis.) Terms can be selected from the classroom textbook, from teacher notes and lectures, or from both sources. Selected terms should be germane to the content being covered. If the textbook is fairly representative of the content being covered, the terms can be created from the glossary of the textbook or from terms in the text that are highlighted or italicized.

2. Develop definitions for each term. For each term, develop a short definition. The easiest method for developing definitions is to use the glossary of the textbook. Other methods are to rely on teachers' notes and lectures or to use a school-based dictionary. Limit the length of each definition to approximately 15 words. Make them clear and unambiguous.

3. Create weekly measures that are similar. For each measure, randomly select 20 terms and definitions from the pool created in steps 1 and 2. In addition, select two definitions that do not match any of the terms. Thus, each probe will have 20 terms and 22 definitions.

One practical way to develop the measures is to write each vocabulary term on the front of an index card with its definition on the back. For each measure, shuffle all of the cards, and randomly select terms and definitions. Place the terms on the left-hand side of the page and the definitions in random order on the right-hand side. Number the terms, leaving a blank space by each term; put letters by each definition. The students write the letter for the correct definition in the blank next to each term.

Administering Probes

Administer a measure once a week. Administer the measure to students one time per week. Give students 5 minutes to match terms with definitions. Make a different version of the measure each time you administer it, following the same procedure of selecting terms randomly and putting the terms on the left of the probe and the definitions on the right.

Students should not be able to finish the measure within the 5-minute time limit. If they finish, they will have no room to grow over the year. (If you think students will finish the measures within the 5 minutes, create longer measures. However, be sure to make this decision at the beginning of the year and keep the measure length consistent throughout the year.)

Tell students that they are not expected to finish the measure, and that they will see words they have not yet learned. Here are sample instructions you can use when administering the measure.

"When I say begin, match the words on the left-hand side of the page with their definitions. Write the letter of the correct definition in the blank next to each word. Do as many as you can. I do not expect you to finish. You will see terms you have not yet learned. Do not worry about not knowing all of the words. Just do your best work. Ready? Begin."

The teacher simply records the number of items matched correctly by each student. As students acquire knowledge of the content being taught during the year, they will be able to match more terms with definitions correctly. You can record their progress on a graph with scores ranging from 0 to 20 or 0% to 100%. Many students find it rewarding to see their scores rise.

Research Behind the Vocabulary-matching Measures

Our research team at the University of Minnesota has conducted a series of studies to examine the reliability and validity of vocabulary matching as an indicator of content-area learning. The results of this research show that the vocabulary-matching measure is a valid and reliable indicator of performance and progress in social studies and science. Performance on the vocabulary-matching measure is related to performance on other content-area tasks, including research-made content tests, content-area subtests of standardized achievement tests, and teacher-made content measures. In addition, students who grow more on the vocabulary-matching measures score higher on criterion measures of content-area performance. As an aside, our research also shows that students must read the measures themselves (as opposed to having the measures read to them by the examiner) to obtain reliable and valid growth rates.

What We Still Do Not Know

Although the results of the early research about monitoring progress in the content areas are promising, there is still much to be learned. (If someone is looking for research opportunities, these would be fruitful areas for study.)

• First, our research has focused only on the development of measures. We have not yet examined the effects of teacher implementation of these measures. That is, we do not know if teachers who implement progress monitoring in content-area classes are able and willing to use the data to change their instructional programs and, further, whether such changes result in improved learning for students.

• Second, we have found that the growth rates produced by the vocabulary-matching measure are small--only about 1/2 a word a week. Such small growth rates may discourage teachers from using the measures. These slow rates of growth may also make the task less rewarding to students than we might hope.

• Third, our research has included students across a range of ability levels. We have not focused specifically on the development and use of the measures for students with learning or language difficulties.

In future research, we plan to investigate the effects of teacher implementation of progress monitoring, compare the growth rates produced by different formats of the measure, and examine the validity and reliability of the measure for students at different achievement levels. In conclusion, our research to date in the content areas is promising, but there is still much to be done. I think teachers can have confidence that a vocabulary-matching measure will reflect general performance and progress; however, whether use of the measures will easily drive instruction and improve achievement for students with LD has yet to be determined.

Sources for Additional Information

Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 49, 36-45.

Espin, C. A., Busch, T. W., Shin, J., & Kruschwitz, R. (2001). Curriculum-based measurement in content areas: Validity of vocabulary-matching as an indicator of performance in social studies. Learning Disabilities Research and Practice, 16, 142-151.

Espin, C. A., & Deno, S. L. (1993). Performance in reading from content-area text as an indicator of achievement. Remedial and Special Education 14(6), 47-59.

Espin, C. A., & Deno, S. L. (1995). Curriculum-based measures for secondary students: Utility and task specificity of text-based reading and vocabulary measures for predicting performance on content-area tasks. Diagnostique, 20, 121-142.

Espin, C. A., & Foegen, A. (1996). Validity of three general outcome measures for predicting secondary students' performance on content-area tasks. Exceptional Children, 62, 497-514.

Espin, C., Shin, J., & Busch, T. (2000). Formative evaluation: Go for it. Current Practice Alerts, 3. Available from http://www.teachingld.org/ld_resources/alerts/.

Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.), Advanced applications of curriculum-based measurement (pp. 214-253). New York: Guilford.

Shinn, M. R. (1989). Curriculum-based measurement: Assessing special children. New York: Guilford.

Tindal, G., & Nolet. V. (1995). Curriculum-based measurement in middle and high schools: Critical thinking skills in content areas. Focus on Exceptional Children, 27(7), 1-22.

Back to Expert Connection