Assessment

LEARNER ASSESSMENT

Learner assessment is complex because it cannot be discussed apart from other program issues. Since curriculum and assessment are linked, a program's choice and implementation of assessment reflect its view of language, literacy, and learning, whether these ideas are consciously articulated or not. Which assessments are chosen and how the assessment process is carried out not only illustrates what "counts as success" but also reveals something about the roles that learners and teachers play in the program. In essence, assessment decisions are based on pedagogical (and in some cases political) concerns that reflect the philosophies, theories, and approaches that a program supports. If assessment is to be an effective part of a literacy program, it must fit into the overall framework the program has chosen for itself.

Given the many perspectives on the roles, functions, and uses of literacy, it is not surprising that approaches to literacy assessment vary widely. Some programs focus on evaluating overall communicative competence through integrated tests, while others focus on one or more particular areas, such as reading, writing, speaking, and listening.

While most literacy assessments focus on knowledge and performance, as measured by choosing the right answer on multiple choice tests, there now is increasing support for assessments that try to capture the reading and writing process through interviews, surveys of literacy behaviors and practices, and the use of portfolios.

A major summary of evaluation in adult literacy divides assessment into four areas (Lytle & Wolfe, 1989):

• standardized testing, which may be either norm referenced (students are compared to each other, most often across programs) or criterion referenced (student achievement is compared to an externally derived standard)
• materials-based assessment, in which assessment is based on a particular set of teaching materials, often commercially packaged and sold
• competency-based assessment, in which learner performance is compared to pre-established competencies that need to be achieved
• participatory assessment, in which learners play a significant role in deciding both the content and process of assessment.

For program purposes, we can combine these four assessment areas into two broad categories:

• general assessments, such as standardized tests, that are designed to measure achievement, knowledge, and skills of large groups of students across programs and are said to have content validity
• program-based assessments, that reflect the educational approach and literacy curriculum of a particular program and thus have both content validity and curriculum validity.

When it comes to systematic assessment across programs in the general assessments category, standardized tests dominate adult education. One reason for the popularity of standardized assessments may lie in the history of testing in the United States, which has emphasized the need for program accountability above the need for quality teaching (Resnick & Resnick, 1985).

Most larger adult literacy programs use some form of commercially available standardized test to report student data to their funding source. The most common tests are group-administered adult basic skills tests such as the Tests of Adult Basic Education (TABE), the Adult Basic Learning Examination (ABLE), or selected portions of the Comprehensive Adult Student Assessment System (CASAS).

While some literacy programs use the TABE or the ABLE for both their native English speakers and their second language learners, others use tests specifically designed for ESL students. These include the Basic English Skills Test, (BEST) which includes an oral interview section, and the Basic Inventory of National Language (BINL), which provides for a grammatical analysis of spoken language samples. Another individually administered test is the Reading Evaluation Adult Diagnosis (READ), used by the Literacy Volunteers of America (Sticht, 1990).

Standardized tests are popular because they offer certain advantages: (1) their construct validity and scoring reliability have been tested; (2) they are cost-effective and don't require a great deal of training to administer; (3) funding sources accept them as part of the documentation of program accountability; (4) they allow for comparisons of learner progress across programs; and (5) they give learners a sense of where they stand compared to students in other programs (Brindley, 1989).

In spite of these apparent advantages, standardized tests have a number of disadvantages.

Standardized tests

• fail to distinguish between language, literacy, and culture. In other words, they don't tell us whether a learner has trouble with an item because (1) he or she is unfamiliar with the cultural notion underlying the task, (2) lacks the requisite knowledge of English vocabulary or sentence structure, or (3) does not have enough experience with reading and writing to complete the task
• reduce the complexity of language and literacy learning to a set of skills
• don't reflect what has been taught or capture all the learning that has taken place
• don't capture changes in language use and literacy practices beyond the classroom and don't provide data on the socio-linguistic and affective dimensions of language and literacy
• don't discriminate well at the lower end of literacy achievement, failing to capture experience with environmental print or provide information on the different levels of "initial literacy," such as being able to write the names of one's children but not those of strangers
• focus on pencil and paper tasks, the very things that literacy students have trouble with
• don't provide opportunities for literacy students to show what they can do in "real-life"
• fail to show how learners deal with literacy in the mother tongue, treating English literacy as "the only literacy that counts" (Macias, 1990).

While these shortcomings hold true for any standardized language or literacy test, there are additional concerns if these instruments are used to assess ESL literacy students.

ESL literacy students may not be familiar with the background knowledge and cultural conventions underlying these tests. Standardized tests, in an effort to be "fair" to all, either discount the unique background knowledge that learners bring to school or assume that cultural conventions are shared across countries.

Standardized tests often fail to distinguish between language problems, in which the learner is unfamiliar with the language or concepts of the test item, and literacy problems, in which the learner lacks the requisite reading and writing skills but could easily respond to similar items presented as part of a conversation.

Standardized tests treat language and literacy as isolated from the social content of the learner. Students are assessed individually and no help may be given or received. ESL literacy students, as a rule, work together to help each other solve problems that require English or reading and writing and often develop strong coping skills and social networks that allow them to deal with problems that require literacy. By disallowing access to resources, peer assistance or group work, standardized tests tail to measure the very strengths that literacy students bring to class (Wolfe et al., 1991).

Most standardized tests fail to take into account the wide range of literacy practices in which learners engage in their mother tongues. By disregarding the biliteracy aspect of ESL literacy, they give the impression that literacy in English is the only literacy that counts and deprive programs of valuable information about learners' past experiences with literacy (Macias, 1990).

In an effort to make assessment more responsive to the concerns of learners and teachers, many programs are developing alternatives to standardized tests. While some of these tests are commercially available, others are program-based or "home grown."

Commercially available tests used in adult ESL literacy programs include the following:

• the Henderson/Moriarty ESL Placement Test, designed to assess literacy and oral skills of both literate and non-literate adults
• the English Language Skills Assessment (ELSA, now available as CELSA), a test that measures reading comprehension and grammatical proficiency
• the California Adult Learner Progress Evaluation Process (CALPEP), a reading and writing inventory that seeks to identify reading and writing habits, learner goals with respect to literacy, and estimated proficiency levels in reading and writing
• the English as a Second Language Oral Assessment (ESLOA), a test designed to help tutors measure a learner's ability to speak and understand English.

Examples of commonly used program-based assessments include the following:

• checklists that provide a record of the skills and competencies a learner has attained
• true/false or multiple choice tests that allow programs to assess the knowledge that students have gained or the skills that they have developed while in the program
• anecdotal evidence that serves as examples of success; such evidence is often culled from discussions with other adults who interact with the literacy learners (e.g., employers or supervisors in workplace programs, children's teachers in family literacy programs, or counselors in general programs)
• daily or weekly charts that list learner accomplishments or document significant literacy events that have taken place (e.g., a supervisor notices that an immigrant worker has started to speak up in meetings).

Integrated assessments that provide evidence of the overall proficiency of a second language learner are also common. In this category, we find holistic writing assessments, as well as close tests and dictations. However, most of the programs we studied had not yet fully incorporated these program-based assessments into an overall framework.

Many literacy programs use a combination of assessments, including standardized and other commercially available tests, for placement and program-based assessments to measure learner progress. Many, if not most, innovative programs see great promise in alternative assessments that are program-based. In the eyes of many literacy educators, program-based assessments offer the following advantages. Program-based assessments

• reflect the local curriculum and provide information that is helpful to the program
• are developed by the individual program, sometimes with the help of evaluators or other researchers, and thus are responsive to the program context
• focus on learning processes, not just outcomes, allowing for trial and error instead of giving learners just one chance to answer each question; they do not insist on a "cold start" response (Wolfe et al, 1991)
• actively involve learners by giving them the opportunity to (1) discuss their goals and interests in literacy, (2) choose the kind of reading and writing they want to be evaluated on, and (3) talk about what they have learned. In other words, they are part of a process in which assessment is done with adults, not to them.

Most importantly, perhaps, alternative assessments go beyond conventional skills-based notions of language and literacy. When carried out as part of an initial intake process and repeated at regular intervals during the teaching cycle, they can provide information that can be used for curriculum development.

Increasingly, alternative assessments also focus on non-linguistic factors, such as learners' changing perceptions of what it means to be literate, how to help one's children to enjoy literacy, increased confidence in one's ability to deal with tasks that require literacy, and a stronger voice in presenting one's own ideas.

Source: Bringing Literacy to Life: Issues and Options in Adult ESL Literacy. Wrigley, Heide Spruck & Guth, Gloria. (1992). Dominie Press, San Diego. Reprinted with permission from copyright owner: Aguirre International.

Brindley, G. (1989). Assessing achievement in the learner-center curriculum. Sydney, Australia: Macquarie University, National Centre for English Language Teaching and Research.

Lytle, S. L. & Wolfe, M. (1989). Adult literacy education: Program evaluation and learner assessment. Information Series No. 338. Columbus, OH: ERIC Clearinghouse on Adult, Career, & Vocational Education.

Macias, R. (1990). In C. Simich-Dudgeon (Ed.). Proceedings of the first research symposium on limited English proficient students' issues (pp. 406-412). Washington, DC: U.S. Dept. of Ed., Office of Bilingual Education & Minority Languages Affairs.

Resnick, D. P., & Resnick, L. B. (1985, April). Standards, curriculum, and performance: A historical and comparative perspective. Educational Researcher, pp. 5-20.

Sticht, T. G. (1990). Measuring adult literacy: A response. In R. L. Venezky. A. Wagner, and B. S. Ciliberti (Eds.). Toward defining literacy (pp. 48-53). Newark, DE: International Reading Association.

Wolf, D., Bixby, J., Glenn III J., and Gardner, H. (1991). To use their minds well: Investigating new forms of students assessment. In G. Grant (Ed.). Review of research in education. Washington, DC: American Educational Research Association.

Heide Spruck Wrigley is a senior researcher for Aguirre International where she specializes in issues related to language, literacy and learning. She was content specialist for the National Study on Promising Practices in ESL literacy, carried out for the U.S. Department of Education. Gloria J. Guth is a senior researcher for Far West Laboratories. She was the project director for the study on Promising Practices.