Evaluating Phonological Skills in Adult ESOL Learners

Part Five: Criteria for the test: Met or not met?

An evaluation of the criteria for the APS shows the following:

Criterion #1: Measuring core skills
The APS seems to be at least adequate for measuring the core phonological skills. The trend towards stronger skills in the educated and less strong in the less educated indicates that the test is probably measuring what it should. Individual items, however, are less satisfactory. The phonological awareness tasks have been arranged so that syllable counting comes after word counting; however, some learners are easily confused by syllables versus words in English. While on one level, that is the point of having both tasks, the confusion could be seen as indicating the need for one more phonological awareness task where that kind of confusion would be avoided. The deletion task appears to be effective (See below for problems explaining it). If deletion is as accurate a measure of phoneme awareness as the experts suggest, then only one of these learners actually has a weakness that might need to be considered in light of dyslexia, a finding that would indicate that probably fewer ELLs have August 24, 2007asks were also not uniformly effective. The sentence counting produced such poor scores overall that clearly it needs to be studied much more. Possibly balancing sentences in terms of number of syllables in words and words in sentences will produce a more balanced set of scores. The other tasks of repetition—real words and nonsense words, proved easy for adults as predicted. These will probably be eliminated. The rhyme production was not effective because of the difficulty in explaining how to do it and of participants’ efforts to make it a vocabulary test. Participants were clearly influenced by their perception that they needed to find a real English word to rhyme and many could not (See below for more discussion of this.)

Criterion #2: Brevity of administration
Since administration of the test took about 20-25 minutes, the APS is too long to be comfortably administered in adult education sites without being burdensome. Mostly the time beyond the estimation was due to the need to explain certain rhyme and deletion items. A different choice of rhyming activities, clearer instructions on the deletion task and elimination of the word-repetition tasks will shorten administration time significantly.

Criterion # 3: Reducing the advantage a native speaker would have
Native speakers might still have an advantage in the rhyme production task and the deletion task. The rhyme production caused participants to seek real words, where a native speaker would have been more easily able to understand that real words were not necessary. Indeed, those who spoke English extremely well-e.g. the Azerbaijani who went to high school for four years in this country—found the task easy. Others found it extremely hard to understand.

Though it involved relatively common words (grasp, winter, snail), the deletion task did not always result in words the ELLs recognized. In fact, many were stuck on the last item until the word “snail” was explained to them. This task requires more study as well. One other task with some advantage for native English speakers was the word counting task where participants sometimes thought one very long word, “extraordinarily,” was more than one word. Native speakers would be unlikely to make that error. On the other hand, the word was deliberately chosen because in speech, English speakers generally recognize very long words as opposed to short ones by the rhythm of speech, itself a feature of phonological awareness.

Pronounceability of words was an issue for some language groups where it obviously would not have been for nearly all native speakers. Vietnamese had trouble with certain consonant clusters, and Spanish speakers could not hear endings of some words. The words need to be more carefully screened for those possiblities in order to keep the test broadly applicable to learners from as many language backgrounds as possible. One word that was very puzzling because of the errors in syllable counting was “house,” which was more often counted as having two syllables than one no matter what the language background of the participant!

Criterion #4: Simplicity of the test for low literate learners
Literacy level did not seem to influence the difficulty of items at all. Learners of all literacy levels managed the items more or less the same.

Criterion # 5: Ease of administration for those with limited English
Many of the tasks were simple enough for all levels of learners but a few were not. The rhyme recognition task was often difficult to explain, as the word rhyme was used instead of “the same.” Even though explanations were given and several models provided, it was still difficult for many to understand adequately. The same was true for rhyme production, which proved to be very hard to explain and as mentioned earlier, ended up being a vocabulary test despite all effort to explain that it was not. This difficulty was very likely due in large part to the unfamiliarity of many participants with the idea of rhyme. Other tasks, where the concept of rhyme does not have to be understood but the phonological aspects can be tested, would probably be more manageable.

The deletion task was sometimes hard to explain but when compound words were used as models, participants could understand more quickly. The task perhaps needs to be extended so that compound word items are included. Also, when the participants were warned that the item would involve removing a sound from a specific place in a word, they did much better. Directions can be altered to include that coaching.

Criterion #6: Measuring phonological skills, not vocabulary
This criterion was also met fairly well. Only the rhyme production task was subject to the problem of participants searching for a word. Some were hindered by wanting to know meanings before they would do the deletion task, but in general they could still do the tasks if they did not understand. None asked for explanations of the sentences or words to be repeated or words in which syllables had to be counted.

Conclusions

Clearly, the test needs more tweaking and studying, but overall, it seems to measure the phonological skills as generally intended. More importantly, the participants liked doing it very much—they said so—and loved learning what specific things they could do to help themselves learn English.

It is hoped that this effort will spur more research and experiments with testing and teaching phonological skills to adult ELLS.

Next | Table of Contents | Previous