Skip to Content | Contact Us | Search our Library | Home

Rider 82 Curriculum
2007 Pilot Report Findings

---

Quantitative Findings

To determine the effectiveness of the industry-related curricula, both during and after the pilot, various evaluation instruments were given to teachers and students at the pilot sites. The findings discussed here reflect the quantitative findings from these evaluations.

Teachers

Piloting the industry clusters
As mentioned in the method section of this report, to collect data regarding each industry-related curriculum (Healthcare, Manufacturing, and Sales and Service), the TCALL research staff requested that after the conclusion of each lesson, the teachers were asked to complete a lesson evaluation. Data collected from these evaluations were analyzed and reports for each of the five pilot sites were generated and sent to EPCC. These reports were to be used by the curriculum writers to make changes to the curriculum. Please review Appendices H, I, J, K, and L for these reports.

Table 2 displays how many of the pilot sites completed all of the lessons within a module. For every module, there were five lessons consisting of five ESL, five employability, five math, and five technology components (one component per lesson). If a pilot site completed all of the lessons in a module, it is denoted ‘Complete’ in Table 2, likewise if not all of the lessons were completed, then it is denoted ‘Incomplete’. When no lessons of the module were piloted, ‘n/a’ is denoted.

As indicated in Table 2, only one of the pilot sites (McAllen) completed all four modules. The Houston and El Paso pilot sites completed Modules 1, 2, and 3 and the Seguin and Mt. Pleasant pilot sites completed Modules 1 and 2.

Table 3 displays how many of the components were completed by site. Only one of the pilot sites completed the entire curriculum (McAllen). The remaining sites attempted to complete as much of the curriculum as possible during the duration of their summer or fall semester.

When asked if each of the teachers had completed 200 hours of instruction (100 hours for the Mt. Pleasant pilot site) all of the teachers indicated that they had completed 200 of instruction with their students. However, when reviewing the lesson-by-lesson evaluations, not one teacher taught for a total of more than 181 hours.

Table 4 displays how much time it took each site to teach each component of the modules. The symbol ‘*’ in Table 4 indicates when all of the lessons within the module were not completed, while ‘n/a’ indicates if none of the lessons in the module were piloted.

The Houston pilot site took the most amount of time to teach the curriculum, 181 hours and 15 minutes, however this site did not complete all of the lessons. The McAllen site, which completed all of the lessons, took 149 hours and 15 minutes to teach the curriculum. The Mt. Pleasant pilot site, which completed all the components for Modules 1 and 2 took 73 hours and 45 minutes to teach the curriculum.

The average time it took to complete Module 1 was 53 hours and 15 minutes. The average time it took to complete Module 2 was 39 hours. The average time it took the teach Modules 3 and 4 cannot be determined since only three sites completed all of the components in Module 3 and only one pilot site completed all of the components in Module 4.

To collect data regarding general information about the curriculum, after the conclusion of the pilot the teachers were asked to complete an overall evaluation. The questions on this evaluation dealt with the teachers’ perceptions of the students. Each teacher at the Houston, McAllen, El Paso, and Seguin pilot sites completed the evaluation. Two teachers from the Mt. Pleasant site completed this evaluation because they taught different components of the curriculum. Table 5 presents the number of teacher responses to questions asked on the overall evaluation form.

As indicated in Table 5, most of the teachers responded that they agreed that they perceived their students being motivated, improving their English, and that the students knowledge of industry-related skills increased. The teachers also agreed that the industry specific content in each lesson generally was aligned across the ESL, math, technology, and employability components and that the 5-Step Model used in the ESL components was consistent across lessons. Lastly, four of the six teachers agreed that the curriculum included activities that addressed diverse learning styles.

Five out of the six teachers marked that they felt the overall lessons were adequately challenging for the students. One teacher marked that the lessons were too difficult for the students. The teacher explained that “The students had a difficult time understanding the English terms in several of the lessons”.

Teacher lesson plans
When asked ‘How did you use the teacher lesson plans?’ four of the six teachers piloting the curriculum reported that they had used all of the lesson plans. One teacher had used some. The reason the teacher did not use the entire teacher lesson plans was because “the lessons were not given in a timely manner.” One teacher modified the lesson plans by adding extension activities to make the lessons “more interactive and student centered”. Some examples of these activities were mock interviews and a style show that demonstrated how to dress for an interview.

Instructor Manual
Five of the six teachers had looked through the Instructor Manual handed out at the one-day professional development workshop. The one teacher who had not looked through the Instructor Manual reported that she was not at the training session so she had never been given a copy of the Instructor Manual.

The five teachers who had looked through the Instructor Manual found all of the sections to be useful because it gave them an idea of how to conduct the class and provided additional information helpful throughout the piloting process. Not one of the teachers could think of a section that could have been added to the manual.

• I think the Instructor's Manual is excellent. It is a good resource for instructors.

• I frequently referred to the manual because there was additional information (eg. from the O*NET, additional evaluative checklists, including student motivation and language use, setting priorities, student use of computer observations, a set of pictures power point style).

Other useful sections in the Instructor Manual mentioned by the teachers were:

• Student assessments and data collection
• The ESL and math components
• Orientation materials

Assessments
When asked if each of the teachers had used the assessments found in each one of the lessons, they were asked to pick which responses best matched how they used the assessments. The three possible responses were:

• I used all the assessment in the teacher lesson plans
• I used some of the assessments in the teacher lesson plans
• I used few or none of the assessments in the teacher lesson plans

After choosing a response, the teachers were asked to provide additional information as to why they used all, some, few/none of the assessments for that given lesson. Table 6 displays how each pilot site’s teacher used the majority of assessments in each module. The symbol ‘*’ in Table 6 indicates when not all of the lessons within the module were completed, while ‘n/a’ indicates when none of the lessons in the module were piloted.

Pilot site	Industry cluster	Component	Module 1	Module 2	Module 3	Module 4
Houston	Sales and Service	ESL	All assessments	All assessments	* Some assessments	* Some assessments
Houston	Sales and Service	Technology	All assessments	All assessments	All assessments	* All assessments
Houston	Sales and Service	Employability	All assessments	All assessments	All assessments	* All assessments
Houston	Sales and Service	Math	All assessments	All assessments	All assessments	* All assessments
Mt. Pleasant	Sales and Service	ESL	All assessments	All assessments	n/a	n/a
Mt. Pleasant	Sales and Service	Technology	All assessments	* All assessments	n/a	n/a
Mt. Pleasant	Sales and Service	Employability	All assessments	All assessments	n/a	n/a
Mt. Pleasant	Sales and Service	Math	All assessments	All assessments	n/a	n/a
El Paso	Healthcare	ESL	Some assessments	Some assessments	Some assessments	Some assessments
El Paso	Healthcare	Technology	None/a few assessments	Some assessments	Some assessments	n/a
El Paso	Healthcare	Employability	Some assessments	Some assessments	Some assessments	* Some assessments
El Paso	Healthcare	Math	Some assessments	Some assessments	Some assessments	Some assessments
McAllen	Healthcare	ESL	All assessments	All assessments	All assessments	All assessments
McAllen	Healthcare	Technology	All assessments	All assessments	All assessments	All assessments
McAllen	Healthcare	Employability	All assessments	All assessments	All assessments	All assessments
McAllen	Healthcare	Math	All assessments	All assessments	All assessments	All assessments
Seguin	Manufacturing	ESL	Some assessments	All assessments	* All assessments	n/a
Seguin	Manufacturing	Technology	All assessments	All Assessments	* All assessments	n/a
Seguin	Manufacturing	Employability	All assessments	All assessments	* All assessments	n/a
Seguin	Manufacturing	Math	All assessments	All assessments	* All assessments	n/a

On the whole, most of the sites used all of the assessments. Two reasons why the El Paso pilot sites used some or none of the assessments in the teacher lessons plans were:

• Informal assessments were done, such as informal listening and observation of students reading and participation to determine whether lesson was understood
• The lesson was not challenging for the student so the student completed the lessons very quickly

When asked, ‘What type of data did they reveal?’, the teachers reported that the assessments revealed a lot of things about the students and about the material covered in class. A few revelations were:

• The students comprehended the materials
• The students had fun with the lessons
• The students did not understand the lesson
• The students could not relate to the lesson
• The students already knew the material that was covered in the lesson
• The lessons provided helpful information
• There needed to be more exercises in the lesson

Students
To collect data from the students regarding the Rider 82 curriculum, the TCALL research staff requested that after the conclusion of each set of lesson components (ESL, employability, math, and technology), the students were asked to complete a survey. In total, 1,159 surveys were collected from all five of the pilot sites (657 surveys from the Healthcare pilot sites, 46 from the Manufacturing pilot site, and 456 from the Sales and Service pilot sites). Nine questions asked on this survey. The responses from each industry cluster and the overall responses to each of the questions can be found below in Table 7.

Table 7: Student responses to lesson evaluation questions

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
The lessons helped me learn about this industry.
Sales and Service	0%	5%	28%	28%	38%
Manufacturing	0%	0%	0%	11%	89%
Healthcare	0%	2%	10%	26%	61%
Overall	0%	4%	17%	26%	53%
The lessons helped me with math.
Sales and Service	1%	5%	13%	24%	56%
Manufacturing	0%	0%	0%	9%	91%
Healthcare	0%	1%	12%	31%	56%
Overall	0%	3%	12%	28%	57%
The lessons helped me with pre-employment skills.
Sales and Service	0%	7%	24%	25%	43%
Manufacturing	0%	0%	0%	9%	91%
Healthcare	0%	1%	12%	28%	58%
Overall	0%	4%	16%	26%	54%
The lessons helped me with reading English.
Sales and Service	0%	2%	15%	28%	55%
Manufacturing	0%	0%	2%	13%	84%
Healthcare	0%	1%	9%	33%	57%
Overall	0%	1%	11%	31%	57%
The lessons helped me with speaking English.
Sales and Service	0%	3%	18%	28%	50%
Manufacturing	0%	0%	4%	11%	85%
Healthcare	0%	1%	12%	33%	54%
Overall	0%	2%	14%	30%	54%
The lessons helped me with technology.
Sales and Service	0%	1%	18%	24%	57%
Manufacturing	0%	0%	2%	7%	91%
Healthcare	1%	1%	10%	29%	59%
Overall	1%	4%	16%	27%	52%
The lessons helped me with understanding English.
Sales and Service	0%	0%	18%	28%	54%
Manufacturing	0%	0%	0%	13%	87%
Healthcare	0%	0%	11%	26%	62%
Overall	0%	1%	13%	25%	61%
The lessons helped me with writing English.
Sales and Service	0%	0%	13%	35%	52%
Manufacturing	0%	0%	6%	9%	85%
Healthcare	0%	0%	9%	34%	57%
Overall	0%	0%	12%	31%	57%
The pictures, tables, and graphs in the lessons helped me to learn.
Sales and Service	0%	6%	13%	24%	56%
Manufacturing	0%	0%	0%	13%	87%
Healthcare	0%	1%	11%	28%	60%
Overall	0%	1%	11%	30%	58%

When reviewing Tables 7, most of the students found that these lessons helped them to improve their understanding, writing, reading, and speaking of English; technology; math; pre-employment skills, and to learn about the industry. Well over half of the students (58%) responded that they strongly agreed that the pictures, tables, and graphs in the lesson helped them learn.

---