BLOG

Back to Blog

Intervention Essentials: The Annual National Evaluation of Reading Recovery

Published On: January 25th, 2023 | Categories: Latest News |

Mary Anne Doyle, University of Connecticut

Originally published in The Journal of Reading Recovery. Interested in full access to the journal? Learn more about becoming a member of RRCNA.

Why do all Reading Recovery® implementations submit data annually?
Each nation offering Reading Recovery has established processes for the annual, national evaluation of their implementations, and this entails collection and analyses of data on all Reading Recovery children served, including those participating in redevelopments of Reading Recovery in Spanish (Descubriendo la Lectura or DLL) and French (Intervention preventive en lectureécriture or IPLÉ). Procedures for the annual evaluations of DLL and IPLÉ parallel those of Reading Recovery and are not discussed separately.

In the U.S., centralized procedures are conducted by the International Data Evaluation Center (IDEC) at The Ohio State University. In Canada, the Canadian Institute of Reading Recovery (CIRR) directs this work with technical assistance as needed. In each case, procedures adhere to formal research practices and are designed to answer questions allowing evaluation of how well the national implementation is meeting its goals. What level of success is achieved by the learners struggling to acquire beginning literacy? What do the analyses reveal in terms of both (a) strengths of the implementation and (b) areas in need of attention and improvement?

Reports generated by IDEC and CIRR additionally provide local data (e.g., state/provincial data, site data, school data) to allow assessments by individual implementations. The data are examined to assure that Reading Recovery teachers are meeting the expectations of this trademark program for learners and schools and to identify any implementation issues.

Design of the evaluation of the Reading Recovery implementation research
The national data evaluation questions and processes are based on Marie Clay’s earliest studies of the implementation of Reading Recovery in the U.S. (i.e., beginning in 1984). As Clay initiated the implementation of Reading Recovery in Columbus, OH, she designed a replication study to examine the effectiveness of this new venture. Her initial inquiry was, Can the Reading Recovery innovation, with its impressive record of proven results in New Zealand, be replicated in the U.S. demonstrating success for children, teachers, and schools?

Clay was an astute researcher and theorist and asserted that, “Implementation and dissemination have their own bodies of theory and their own evaluation criteria and innovations do not last unless due attention is paid to these aspects of an innovation” (Clay, 1994, p. 139). Thus, her replication studies established formal procedures for the evaluation of implementation factors. In designing this research, two realities observed by Clay were of concern:

  • establishing quality control over implementation factors in order to prevent changes (creative and uncreative) that would be detrimental to the effectiveness of Reading Recovery (Clay, 2009b), and
  • establishing a system for confirming that the innovation is being implemented as designed by her in order to optimize results (Clay, 2009a, 2009b).

Only if implemented as designed could Marie Clay, and Reading Recovery trainers, offer specific guarantees for positive outcomes for Reading Recovery. These concerns remain today.

Annual data collection helps ensure quality of the implementation
Very early in her replication research, Marie Clay concluded that she needed to establish strong ‘guard rails’ for Reading Recovery, and she therefore turned to trademark law. In the U.S. she granted the royalty-free trademark for Reading Recovery to The Ohio State University with the understanding that the university would establish and maintain a center for the ongoing collection and reporting of annual data. CIRR fulfills this responsibility in Canada. The trademark is the guarantee to all participants of a quality intervention supported by research and monitored annually for effectiveness. With the collection and reporting of the annual data by all participants, the holders of the trademark confirm compliance with the trademark assurances and standards by each university training center and by all participating teachers and schools. These standards are presented in formal documents published by each nation (e.g., Standards and Guidelines of Reading Recovery in the United States; CIRR Standards and Guidelines).

The annual, national data evaluation provides an ongoing check on the implementation of Reading Recovery, and the data are examined to reveal implementation strengths and concerns. Any concerns are addressed by trainers who monitor implementation effectiveness and problem solve challenges with trainer colleagues, as well as with their respective sites’ teacher leaders, and site coordinators, as appropriate. The system for data collection, initiated by Marie Clay, entails a methodology that allows for outcome and process evaluations (Gómez-Bellengé, 2009) and the identification of factors that are key to answering the research questions in an objective and standard way (i.e., used in a consistent way by all participants). This is the research process that is applied to establish assumptions of reliability and replicability.

In conclusion
Continuing Clay’s evaluation research processes ensures that Reading Recovery is maintained with fidelity to Marie Clay’s theoretical perspectives. These include her theories of research designed to evaluate an intervention as well as her theories of literacy, teaching, and learning as they pertain to both training teachers and instructing learners struggling to acquire beginning literacy. In doing so, we are continuing critically important practices that have sustained the Reading Recovery innovation in North America for nearly 38 years. Without taking this position, the royalty-free trademark for Reading Recovery cannot be used to describe the intervention. It remains the responsibility of each trainer, teacher leader, and teacher to uphold and preserve the integrity of Reading Recovery nationally and within their respective sites.

References
Clay, M. M. (1994). Reading Recovery: The wider implications of an educational intervention. Literacy, Teaching and Learning: An International Journal of Early Literacy, 1(1), 121–141.

Clay, M. M. (2009a). The Reading Recovery research reports. In B. Watson & B. Askew (Eds.), Boundless horizons, Marie Clay’s search for the possible in children’s literacy (pp. 35–100). Pearson Education New Zealand.

Clay, M. M. (2009b). Implementing Reading Recovery internationally. In B. Watson & B. Askew (Eds.), Boundless horizons, Marie Clay’s search for the possible in children’s literacy (pp. 221–249).

Pearson Education New Zealand. Gómez-Bellengé, F. (2009). Monitoring Reading Recovery in the United States. In B. Watson & B. Askew (Eds.), Boundless horizons, Marie Clay’s search for the possible in children’s literacy. Pearson Education New Zealand.


Dr. Mary Anne Doyle is chair of the International Reading Recovery Trainers Organization Executive Board and has served as consulting editor for the Marie Clay Literacy Trust. She is a member of the North American Trainers Group, a professor emeritus at the University of Connecticut, and former director of Reading Recovery in Connecticut.


The Process of Letting Go: Exiting a Child from Reading Recovery
Teaching Reading and the Goldilocks' Dilemma: A Case for Purposeful Literacy
Search Journal Archive

THE JOURNAL OF READING RECOVERY

Fall 2023