Executive Summary: Reach Out and Read Assessment

Jacqueline Gramann, Ed. M.
Principal Investigator, RORA Project
Family Literacy Specialist, Texas Center for the Advancement of Literacy & Learning
June 2007

The Reach Out and Read Assessment (RORA) project was a quasi-experimental study designed to evaluate the effectiveness of a Reach Out and Read program. Reach Out and Read (ROR) is a non-profit, national organization that features giving parenting and literacy information to families. Books are given to their children at well-child visits between the ages of six months and five years by pediatricians. Reading techniques are modeled by the pediatrician and by volunteers reading in print-rich clinic waiting rooms (ROR, 2003-2006). The evaluation consisted of an oral (bilingual) survey and a child assessment using the Preschool Language Scale, 4th Edition (Zimmerman, 2002) instrument. Twenty-four families in the final sample participated at their child’s six-month-old well-baby visit with a follow-up at the 12-month well-child visit. Impact of the ROR program on the families was measured against a control population. The study process, findings, and best practices are transferable information for practitioners in family literacy programs (Gramann, 2007).

Background
An objective of early literacy interventions is to increase the school readiness abilities of young children at an age when parent input is significant (National Scientific Council on the Developing Child, 2004; Dickinson, 2006). Literacy abilities at kindergarten have been shown to be related with reading abilities in the later elementary years (National Center for Family Literacy, 2005; Dickinson, 2006).

In the past 20 years, early literacy has become a frequent message directed toward parents in our society. ROR and other family literacy programs empower young parents by helping families to understand the connection between activities when the child is six months of age and learning to read during the school years. Shared book reading has been shown to enhance language, not early literacy skills directly. Increasing a child’s vocabulary before learning to read can impact phonological awareness and listening comprehension. Learning vocabulary indirectly assists in learning to read (Dickinson, 2006). The National Early Literacy Panel has compiled a synthesis of early literacy research (National Center for Family Literacy, 2005). The RORA study supported the ROR model and added to the early literacy research base.

Method
The survey and assessment sites consisted of the experimental site, or Reach Out and Read treatment, and a control site, without a Reach Out and Read program. At the time of the study, both sites were medical facilities serving young children, including those from underinsured families. Children and their parents came to the medical sites for routine well-child visits and health care. The sample population came from a small city and nearby, surrounding communities. The city population make-up included 17.72% African American, 27.8% Hispanic, and 64.65 % White, according to the 2000 census. A total of 14,873 families were in the community with 15.5% of the families falling below the poverty line (Wikipedia, 2007).

In the final sample of participant families for the RORA project, 60.9% were from the treatment site and 39.1% were from the control site. Of the 24 adult participants, 22 were Hispanic, one was African American, and one was White. All of the adult participants who primarily answered the parent survey questions were the mothers of the children. In the final sample, 78.3% of the participant families chose to have the parent survey and child assessment conducted in Spanish language, and 21.7% chose English language. The control group reported 8.4 years of school completed, while the ROR group reported an average of 7.6 years. The sample reflected parent participants born in Mexico and the United States. Final statistics reflected the ROR site as having 85.7% of participants from Mexico and 14.3% from the United States. At the control site, 55.6% were from Mexico and 44.4% from the United States. This difference in the number of recent immigrants, contributed to contrasts in the data results.

Data was collected at the child’s 6 month well-baby visit and a follow-up at the 12 month well-child visit. At both visits and both sites, a parent survey and a child assessment were completed. The RORA parent survey was an updated and lengthened version of the Before and After Books and Reading (BABAR) parent survey (ROR, 2001). The PLS-4: Preschool Language Scale, Fourth Edition (Zimmerman, 2002) was used for the child assessment at both visits. Subscales from the standardized test included auditory comprehension and expressive communication. A total language score was also provided. Both the Spanish and English language versions were used.

Limitations arose when it was found that the control site medical staff did encourage families to read to their children at the six-month check-up. In addition, at least three of the control site parents were enrolled in high school programs designed to assist teenage mothers with infants. These factors may have influenced the control group. In a way the control group did provide a contrast by showing how important ROR may be with families that have less opportunities and education.

Findings
Key findings about differences between the ROR and control groups were tested using an ANOVA of gain scores between 6 and 12 months. Because of the small sample sizes, all tests were evaluated against an alpha level of .10. The survey points not directly influenced by the control site limitations were those concerning receiving books and observing reading volunteers. The number of books reported at home for one child was completely out of line with the rest of the sample. The outlier was removed for the data analysis. Analysis of the data was conducted on 14 treatment site participant families and 9 control site participant families.

The main question that looked at book reading, “Do you ever read children’s books to [name],” reflected the ROR group gaining more. An ANOVA of the gain scores showed a significantly greater change in the ROR group than in the control group parents reading to their children (F = 2.69, df = 1,21, p = .10). At 6 months, 50% of ROR parents said they read children’s books to their infants. By 12 months this increased to 93% of ROR parents reading. By comparison, 78% of control parents read to their children at 6 months, increasing to 89% at 12 months. This is illustrated in Figure 1. In other words, at 6 months more parents in the control group were reading to their infants, but by 12 months this pattern had reversed: a greater proportion of ROR parents read to their children.

D
Figure 1: Reads to Child

When the parent replied that they did not read to their child, “is he/she too young for that” was asked. Five parents responded that their child “was too young for that” in the ROR group, at 6 months. In the control group two parents replied the same way at 6 months. At 12 months, only one parent in the whole sample replied that they did not read to the child because they thought they were too young. This participant was at the ROR site and had only been at the clinic once. Parent perceptions are critical when educating about why it is important to read to the very young child.

As expected, ROR parents received significantly more children’s books from a doctor (mean = 2.4) than did parents at the control site (mean = 0; F  = 18.15, df = 1, 21, p < .001). When asked if they had ever seen volunteers reading to children in the waiting room, 57.1% of ROR parents indicated they had, compared to none of the control-site parents (chi-square = 7.89, df = 1, p = .005). ROR parents were significantly more likely to say their children had listened to volunteers reading (35.7%) than was the case with control-site parents (0%; chi-square = 4.12, df  = 1, p = .043). Given that 12 months is young enough to often be sleeping or easily distracted, this number may be regarded positively.

The PLS-4 instrument (Zimmerman, 2002) assesses children for an auditory comprehension score, an expressive communication score, and a total language score. All of the child assessment scores were insignificant. The total sample mean scores for auditory comprehension were 90.2% for the 6 month-olds, increasing to 92% at the age of 12 months. At 6 months of age, the expressive communication means were 86.1%, and a very close score of 85.9% at 12 months of age. All of the twenty-three subjects had a total language score of 87% at 6 months of age, and an 88% at 12 months of age. Looking at just the ROR site, the mean score for the auditory comprehension assessment for the 6 month-olds was 89.2%, increasing to 92.8% at 12 months.

In calculating the age equivalents for two assessment visits, using the PLS-4, the total sample of children were 5.5 months and 11.7 months, respectively. The children were on target developmentally as the children had just reached the 6-8 month and 12-17 month assessment age ranges. This may be viewed as a positive result. Further research question: what happens between 12 months of age and entering kindergarten?

Summary
Research with a similar questionnaire, the Before and After Books and Reading parent survey by Robert Needlman, MD (ROR, 2001), has shown positive results for the early literacy intervention. In comparing published data results from the BABAR parent survey (ROR, 2001), one of the significant findings from both studies was identical. Results with the BABAR instrument in ROR sites, with 1,006 children from ten sites, indicated that 93% (from 88%, p< .02) of the parents reported reading to their child. The RORA project had the same 93% response (from 50%, F = 2.69, df = 1,21, p = .10) with the same question and a very small sample. The BABAR study differed by including children, aged 6 months to five years, and a follow-up interval of one year. The RORA sample included only six-month and twelve-month-old children. Because parents are more likely to read more to older children, the findings are more interesting than analogous.

Another ROR research paper by Weitzman (2004) also found 93% parents reporting to be reading to children. The study used a waiting room interview and a home visit to collect data using the Home Observation for Measurement of the Environment (Caldwell, 1984) instrument.

Key to the RORA findings was the positive trend in physician and volunteer readers modeling; and the giving of a book to the child. These were contributing factors for the significant gain in reading to their children by the ROR group when compared to the control group. ROR training video resources mention an inappropriate “lollipop” strategy of giving books at the end of the visit as a “reward,” with no modeling (ROR, 2003-2006). The RORA data emphasizes how significant the pediatrician modeling with a book is to success in parents reading more to their children. The reading volunteers reinforced this crucial aspect. The physicians modeled at the end of the visit so it would be the last thing the parent hears. Other than the slight modification of the end of the visit modeling, the study Reach Out and Read program used the ROR model (ROR, 2003-2006).

Even with a small sample due mostly to the transient population and limited age range, RORA results found that the ROR program did have a positive impact on the community, reaching the families most in need of literacy assistance. The sample reflected an underinsured, predominately Hispanic population. The Reach Out and Read Assessment project evaluated and found the following key points:

• The strongest finding was that of the ROR parents reading to their children in a greater proportion by the time the children were 12 months of age when compared to the control parents.
• The number of books given out by the program, the volunteers reading in the waiting room, and the very young children listening to stories being read in the waiting room were all significantly greater at the ROR location.
• The local ROR program was shown to be evidence-based. Overall, trends in the data were positive, with some findings being statistically significant. Results supported the ROR program model.
• The ROR intervention appeared to have no effect on PLS-4 (Zimmerman, 2002) scores from six months to 12 months of age. At this youngest age group, it may be too early for the PLS-4 to be a valid measure for an invention impact, or with assessing just a 6 month interval. The children as a group assessed at their appropriate age equivalents, a positive outcome for the families.

Conclusion
Next steps for the program included expanding collaborations and information about other community resources for families in order to provide more literacy support for families (Harvard Family Research Project, 2005). ROR books that are bilingual would be a better choice to provide. Locating English Language Learner classes would be important in the sample community. Increasing documentation of volunteer hours would allow a monetary value to be assigned for reporting and recognition.

A revised and redesigned literacy survey, Family Literacy—Short Survey (FL-SS) (Gramann, 2007), is suggested and can be found in the appendix of Reach Out and Read Assessment: The Final Report for program evaluation. FL-SS is a revision in answer to the call (ROR, 2001) to develop workable methods for monitoring ROR, and with modification, other family literacy programs, in an on-going basis.

Best practices for application in family literacy programs were developed out of the study findings. The best practices are:

• To encourage parents to read early and often to their child.
• To model ways parents can read a book to their child.
• To repeat the reading message multiple times to the parents.
• To provide and assist the parents in locating children’s books for the child.

Parent perceptions are vital when educating about why it is important to read to the very young child. Early literacy begins at birth and is a lifelong endeavor (Neuman, 1998). Reinforcing that the very young child is ready to learn and methods to engage the child in literacy and learning activities should be repeated messages. Literacy is a family affair, with modeling of reading being the critical element.

References

Barone, Diane M., Morrow, Lesley Mandel, Editors (2003). Literacy and Young Children: Research-based Practices. New York, NY: Guilford.

Bowman, Barbara T., Donovan, M. Suzanne, Burns, M. Susan, Editors (2001). Eager to Learn: Educating Our Preschoolers. Washington, DC: National Academy Press.

Butterfield, Perry McArthur, Martin, Carol A., and Prairie, Arleen Pratt (2004). Emotional Connections: How Relationships Guide Early Learning. Washington, DC: Zero To Three.

Caldwell and Bradley (1984). Home Observation for Measurement of the Environment (HOME). Eau Claire, WI: HOME Inventory, LLC.

Clay, Marie M. (1991). Becoming Literate: The Construction of Inner Control. Portsmouth, NH: Heinemann.

Cook, Thomas D., Campbell Donald T. (1979). Quasi-Experimentation: Design & Analysis Issues for Field Settings. Boston, MA: Houghton Mifflin Co.

DeBruin-Parecki, Andrea and Krol-Sinclair, Barbara, Editors (2003). Family Literacy: From Theory to Practice. Newark, DE: International Reading Association.

Dickinson, David K. and Tabor, Patten O., Editors (2001). Beginning Literacy With Language: Young Children Learning at Home and School. Baltimore, MD: Brookes.

Dickinson, David K. and Neuman, Susan B., Editors (2003). Handbook of Early Literacy Research, Volume 1. New York, NY: Guilford.

Dickinson, David K. and Neuman, Susan B., Editors (2006). Handbook of Early Literacy Research, Volume 2. New York, NY: Guilford Press.

Duke, Nell K. Mallette, Marla H., Editors (2004). Literacy Research Methodologies. New York, NY: Guilford.

Epstein, Joyce L. (2001). School, Family, and Community Partnerships: Preparing Educators and Improving Schools. Boulder, CO: Westview Press.

Gall, Meredith D., Gall, Joyce P., Borg, Walter R. (2006). Educational Research, An Introduction, 8th Edition. Boston, MA: Allyn & Bacon.

Gramann, Jacqueline (2007). Family Literacy-Short Survey (FL-SS). College Station, TX: Texas Center for the Advancement of Literacy & Learning.

Gramann, Jacqueline (2007). NCFL Poster: Looking for Early Literacy: The Reach Out and Read Assessment Project. Orlando, FL: 2007 National Center for Family Literacy Annual Conference.

Gramann, Jacqueline (2007). Reach Out and Read Assessment: The Final Report. Retrieved April 26, 2007 from http://www-tcall.tamu.edu/research/rora/cover.html

Hall, Nigel, Larson, Joanne, and Marsh, Jackie, Editors (2003). Handbook of Early Childhood Literacy. Thousand Oaks, CA: Sage.

Hart, Betty, Risley, Todd R. (1995, 2002). Meaningful Differences in the Everyday Experience of Young American Children. Baltimore, MD: Brookes.

Hart, Betty, Risley, Todd R. (1999). The Social World of Children Learning to Talk. Baltimore, MD: Brookes.

Harvard Family Research Project (2005). Complementary Learning. Retrieved February 26, 2007 from http://www.gse.harvard.edu/hfrp/projects/complementary-learning.html

Harvard Family Research Project (2006). Family Involvement Storybook Corner. Retrieved February 26, 2007 from http://www.gse.harvard.edu/hfrp/projects/fine/resources/storybook/index.html

McCardle, Peggy, Chabra, Vinita, Editors (2004). The Voice of Evidence in Reading Research. Baltimore, MD: Brookes.

Mendoza, Jean, Katz, Lilian, Robertson, Anne S., and Rothenberg, Dianne (2003). Connecting with Parents in the Early Years. Champaign, IL: University of Illinois. Retrieved February 14, 2005 from http://ceep.crc.uiuc.edu/pubs/connecting.html.

Morrow, Lesley Mandel (2005). Literacy Development in the Early Years: Helping Children Read and Write, Fifth Edition. Boston, MA: Pearson.

National Association for the Education Young Children (2005). Whatever Happened to Developmentally Appropriate Practice in Early Literacy? Retrieved February 26, 2007 from http://www.journal.naeyc.org/btj/200507/02Neuman.asp

National Center for Family Literacy (2006). National Early Literacy Panel: A Synthesis of Scientific Research on Young Children’s Early Literacy Development. Retrieved February 26, 2007 from http://www.famlit.org/site/c.gtJWJdMQIsE/b.2133427/k.2623/National_Early_Literacy_Panel.htm

National Center for Family Literacy (2006). Tips for Parents: The Sounds of Reading. Retrieved February 26, 2007 from http://www.famlit.org/site/apps/nl/content2.asp?c=gtJWJdMQIsE&b=1988675&ct=2084527

National Research Council (2000). How People Learn: Brain, Mind, Experience, and School. National Academy Press: Washington, DC.

National Research Council (1999). Starting Out Right: A Guide to Promoting Children’s Reading Success. Washington, DC: National Academy Press.

National Scientific Council on the Developing Child (2004). Young Children Develop in an Environment of Relationships, Working Paper No. 1. Retrieved February 26, 2007 from
http://www.developingchild.net/pubs/wp/Young_Children_Environment_Relationships.pdf

Needlman, R., Klass, Perri, Zuckerman, Barry (2002). Reach Out and Get Your Patients to Read. Contemporary Pediatrics: Vol. 19, No. 1, pp. 51-69.

Needlman, R. and Silverstein, M. (2004). Pediatric Interventions to Support Reading Aloud: How Good is the Evidence? Developmental and Behavioral Pediatrics: Vol. 25, No. 5, pp. 352-363.

Needlman, R., Toker, Karen H., Dreyer, Benard P., Klass, Perri, and Mendelsohn, Alan L. (2005). Effectiveness of a Primary Care Intervention to Support Reading Aloud: A Multicenter Evaluation. Ambulatory Pediatrics: Vol. 5, No. 4, pp. 209-215.

Neuman, Susan B. and Roskos, Kathleen A. (1998). Children Achieving: Best Practices in Early Literacy. Newark, DE: International Reading Association.

Neuman, Susan B., Copple, Carol, and Bredekamp, Sue (2000). Learning To Read and Write: Developmentally Appropriate Practices for Young Children. National Association for the Education of Young Children: Washington, DC.

Reach Out and Read (2001). Before and After Books and Reading (BABAR) Study. Retrieved February 26, 2007 from http://www.reachoutandread.org/FileRepository/newsletter_fall2001.pdf

Reach Out and Read (2003-2006). ROR National Center: How ROR Works: ROR Model. Retrieved February 26, 2007 from http://www.reachoutandread.org/about_how.html

Reach Out and Read (2003-2006). ROR National Center: Research Summary. Retrieved February 26, 2007 from http://www.reachoutandread.org/about_summary.html

Reach Out and Read (2003-2006). ROR National Center: ROR Program Description. Retrieved February 26, 2007 from http://www.reachoutandread.org/FileRepository/One_Pager_English.pdf

Reach Out and Read (2003-2006). Reading, Ready for School, Ready for Life: Preschool Health Visit Training Video. Somerville, MA: ROR.

Rosenkoetter, Sharon E. and Knapp-Philo, Joanne, Editors (2006). Learning to Read the World: Language and Literacy in the First Three Years. Washington, DC: Zero To Three Press.

Shipler, David K. (2004). The Working Poor: Invisible America. New York, NY: Vintage Books.

Shonkoff, Jack P., Meisels, Samuel J. (2000). Handbook of Early Childhood Intervention, Second Edition. Cambridge University Press: Cambridge, U.K.

Shonkoff, Jack P. and Phillips, Deborah A., Editors (2000). From Neurons to Neighborhoods: The Science of Early Childhood Development. National Academy Press: Washington, DC.

Siegler, R. S., DeLoache, J. S., and Eisenberg, N. (2003). How Children Learn. New York, NY: Worth.

Snow, Catherine E., Burns, Susan M., Griffin, Peg, Editors (1998). Preventing Reading Difficulties in Young Children. Washington, DC: National Academy Press.

Spodek, Bernard and Saracho, Olivia N., Editors (2006). Handbook of Research on the Education of Young Children. Mahwah, NJ: Erlbaum.

Stechuk, Robert A., Burns, M. Susan, Yandian, Sharon E. (2006). Bilingual Infant/Toddler Environments: A Guide for Migrant and Seasonal Head Start Programs. Retrieved February 26, 2007 from http://www.aed.org/ToolsandPublications/upload/BITE_web1106.pdf

Tabors, Patton O (1997). One Child, Two Languages. Baltimore, MD: Brookes.

Washington Research Institute (2004). Language is the Key: Talking and Play & Talking and Books. Seattle, WA: Washington Learning Systems.

Wasik, Barbara Hanna, Editor (2004). Handbook of Family Literacy. Mahwah, NJ: Erlbaum.

Weitzman, Carol Cohen, Roy, Lisa, Walls, Theodore, and Tomlin, Ricarda (2004). More Evidence for Reach Out and Read: A Home-Based Study. Pediatrics: Vol. 113, No. 5, pp. 1248-1253.

What Works Clearinghouse, Institute of Educational Sciences (2006). Early Childhood Education: Interventions for Improving Preschool Children’s School Readiness; Shared Book Reading; Interactive Shared Book Reading; Dialogic Reading; other titles. Retrieved February 26, 2007 from http://www.whatworks.ed.gov/Topic.asp?tid=13&ReturnPage=default.asp

Wikipedia (2007). Census data. Retrieved February 26, 2007 from http://en.wikipedia.org

Zimmerman, Irla Lee, Steiner, Violette G., and Pond, Roberta Evatt (2002). PLS-4: Preschool Language Scale, Fourth Edition. The Psychological Corporation: San Antonio, TX.

Zimmerman, Irla Lee, Steiner, Violette G., and Pond, Roberta Evatt (2002). PLS-4: Preschool Language Scale, Fourth Edition, Spanish. The Psychological Corporation: San Antonio, TX.