Assessing Effectiveness
Reading Recovery may be judged as highly effective if data show that the intervention is significantly reducing the numbers of children with severe reading problems, there is clear evidence of strong support for Reading Recovery, and Reading Recovery is perceived as an integral part of the educational system
Assessing the effectiveness of Reading Recovery is a complex issue for several reasons. Implementing any high-quality educational change such as Reading Recovery is a slow process that involves changes in people’s beliefs and allegiances, organizational patterns, allocations of resources, roles, and communication patterns.
Therefore, any assessment of Reading Recovery effectiveness
- must recognize the time required for change to occur. Teachers and schools will not easily realize the full potential of the intervention in the early years of implementation.
- must be evidence-based, but the evidence may need to be understood in relation to incomplete or partial implementation and in relation to classroom practice.
- must seek to understand how the system is supporting the implementation as well as how Reading Recovery teaching is effective.
Following are more specifics for assessing Reading Recovery implementation.
Look for evidence from these groups:
- children whose series of lessons were successfully discontinued (children who are now able to continue their learning in classroom situations)
- complete intervention children (children who have had ample opportunity to learn)
- incomplete intervention children (children for whom time ran out at the end of the year)
- children recommended for action (children who have had a complete intervention but whose lessons were not discontinued)
- all Reading Recovery children (every child who received any service at all)
Look for evidence on these points:
- relative size of each group (percentage of all children served)
- entrance- and exit-level performance on all tests, but especially on text reading
- comparison of each group to a random sample control group identified by the National Data Evaluation Center
- change in reading group placement for children in each group
For evidence of program efficiency, check these items:
- the number of children served per teacher
- the number of weeks and progress of children with an incomplete intervention
- the number of lessons per week for teachers (evidence of daily consistent lessons)
- the number of weeks in the intervention for children whose lessons were discontinued.
- Assess the percent of coverage — the degree to which this intervention is available to all the children in each first-grade cohort that need this intervention. Entry-level data from Observation Survey testing will help determine the extent of need for this intervention.
- Examine how Reading Recovery has changed the paths of progress for these children within the system. Look for evidence of reduction in the numbers of children retained as well as reductions in the numbers referred for testing and classified as qualifying for exceptional services.
- Examine longitudinal data on Reading Recovery children beyond the first-grade year using tests given to all children in Grades 3 and 4. Examine this in three groups: (1) Reading Recovery children whose lessons were discontinued, (2) children who had a complete intervention, and (3) all Reading Recovery children. It is best to collect data from a number of sources (standardized tests, informal reading inventories, teacher perception, and letter grades), since no one assessment provides a full depiction of the child’s competencies.
- Compare the progress of various majority and minority groups to determine the degree that Reading Recovery closes the achievement gap.
- Watch for signs of improvement over time in the entering scores of first-grade children (evidence that kindergarten programs build early literacy skills).
- Look for change over time in average reading levels of grade-level cohorts.
- Collect data on classroom teachers’ perceptions of the performance of Reading Recovery children.
The second positive outcome of Reading Recovery is that the children who need long-term learning support are identified and referred for further evaluation. These children are referred to as recommended for further action. Some of these children have made good learning progress and with some supplementary small-group assistance will be able to make adequate school progress in the future. Others will need the long-term intensive support of special education services. The Reading Recovery intervention reduces the number of students who need longer services, especially those who will receive special education services, saving the system time and expense.
Even though the children whose lessons are not successfully discontinued may be viewed positively, the continuing goals of the system and of Reading Recovery teaching should include these:
- reducing the percentage of children recommended for further action
- providing efficient and timely evaluation of those who need referral
- providing additional support for the children whose lessons were not discontinued who can learn in small-group contexts or with other additional tutoring
- the program is implemented according to Standards and Guidelines for Reading Recovery in the United States, published by the Reading Recovery Council of North America
- resources are committed to achieve full implementation (Reading Recovery teaching time is available to all children who need it)
- administrators lead, encourage, and develop strong classroom teaching and close collaboration and communication among Reading Recovery and classroom teachers
- coordination and collaboration is established among all basic and support educational services
- established expectations about what is possible for low-achieving children
- expectations for children with variant social and behavioral characteristics
- traditional curriculum expectations
- conflicting ideas about how children learn to read and write
Beliefs and values can change as people perceive a need to change and as they confront strong, contradictory evidence. To facilitate this change administrators and the teacher leader will
- share entrance data and progress data with classroom teachers
- establish mutual interest (with classroom teachers) in children’s performance
- arrange to have classroom teachers observe Reading Recovery lessons
- invite teachers to bring children to the training class and to come as observers
- arrange for children to read books to their teachers and to their class
- work with the teacher leader and principals to establish school teams
- share data and arrange for direct contact to help upper-grade teachers to see how an early intervention can make a significant contribution to their work
Positive social and intellectual interactions also contribute to belief and value changes. These can be fostered in the following ways:
- Reading Recovery teachers must take responsibility for establishing relationships.
- Principals can arrange positive contexts and opportunities for sharing expertise.
- Reading Recovery teachers can exchange helpful suggestions with classroom teachers.
- Reading Recovery teachers can ask specialist educators for information or advice.
- Reading Recovery teachers can enlist parent participation and visitation from families that have not previously engaged with school staff.
Administrators who have embedded Reading Recovery in their literacy plans call it an ideal fit in the RTI framework. Find out why.
THE JOURNAL OF READING RECOVERY
Spring 2024
Constructing a More Complex Neural Network for Working on Written Language That Learns to Extend Itself by Carol A. Lyons
Reading Recovery IS the Science(s) of Reading and the Art of Teaching by Debra Semm Rich
Predictions of Progress: Charting, Adjusting, and Shaping Individual Lessons by Janice Van Dyke and Melissa Wilde
Teachers Designing for Context: Using Integrity Principles to Design Early Literacy Support in Aotearoa New Zealand by Rebecca Jesson, Judy Aitken, and Yu Liu