4.0 Annual Evaluation of Adult Education and Literacy Activities
(Section 224(b)(3) Section 224(b)(3) requires a description of how the Texas Education Agency will evaluate annually the effectiveness of the adult education and literacy activities based on the performance measures described in section 212 of AEFLA.
4.1 Annual Evaluations
Evaluation of local adult education and family literacy program will be conducted quarterly through the Adult Education Guidance Information System (AEGIS) with a final report due annually to ascertain the extent to which both the Texas statewide program and each local provider have met projected levels of performance for each core indicator established in the State Plan. AEGIS will perform automatic quarterly desk reviews for each grantee based on the performance measures and core indicators negotiated with USDOE and set by the program in their application thus comparing targets to actual student data entered into the management information system. Quarterly, programs will submit a Data Sign Off report to the agency to verify the integrity of data and that data has been entered into the web-enabled Texas Educating Adult Management System (TEAMS), i.e., management information system AEGIS will raise issues and electronically notify programs if satisfactory progress toward performance measures is not being made. The two major focuses of the evaluation will be the effectiveness of Texas statewide and local providers in attaining the core indicators negotiated with the USDOE and established in the state and to identify areas where state leadership activities, technical assistance, and staff development may be directed for program improvement. Applicants are required to maintain an acceptable rating on their performance measures as negotiated by TEA and the program division. Programs that do not continue to achieve acceptable improvements or are not complying with program established improvement plans are subject to termination of their contract and terms for non-renewal of a continuation grant. Acceptable performance is defined as meeting or exceeding at least 50% of the 11 Educational Functioning Level Performance Targets. In addition, the percentage of students setting goals for obtain employment, retain employment, obtain GED and enroll in postsecondary or other training must increase annually or programs are subject to program improvement plans. (State Requirement)
Management Information System
Adult education data that will be collected and used in the performance accountability system will originate from the Texas Educating Adult Management System (TEAMS) management information system. TEAMS is TEA’s state-of-the-art, web-enabled system that maintains student-by-student data including demographic, assessment data, outcomes, as well as information about class, site, program, and provider information. TEAMS users can analyze his/her class, site, or program data through use of the standard reports that can be generated at the user level. TEAMS provides a wide range of information about adult education, including snapshots of student and program performance, personnel qualifications, staff development activities, and evaluation of staff development.Real-time reports generated through the TEAMS will be used by local providers to conduct their own analysis and comparisons of their program’s performance and develop action plans for continuous improvement. These reports will also be useful to local workforce boards and one-stop career center partners in identifying local providers for referral of workforce clients. TEAMS will serve as a warehouse of information for AEGIS (described above) to compare actual data to compliance with target measures.
Desk Review and Self-Evaluation
On-site visits are completed on a minimum of 20% of programs each year. In addition, the Adult Education Guidance Information System (AEGIS) will assist programs across the state in improving program quality and to perform self evaluation. AEGIS gives the needed tools for program managers to know the program status at any given point of time, and it also provides a mechanism to assist programs in need of assistance. Desk reviews and notifications are conducted electronically.
AEGIS performs the following functions:
- Collects data entered by local service providers/fiscal agents from TEAMS;
- Compares data reported by fiscal agents to thresholds for acceptable performance on indicators mandated by law;
- Assesses risk in program performance;
- Identifies discrepancies in data;
- Displays desk review results quarterly for grantees and administrative staff;
- Flags programs for investigation;
- Alerts staff to potential problems;
- Tracks status on corrective actions; and
- Stores information entered by grantees and administrative staff including issues, notifications, and improvement plans.
The issues raised electronically through AEGIS are copied to the state office for adult education and monitored by a manager, who works with each program to develop program improvement and professional development plans. All correspondence between programs and the state office that is generated by the system is kept in a database for historical purposes and future reference.
A final evaluation report is submitted by each program to the state agency and the state agency utilizes the information to provide technical assistance, professional development, and to create mentoring partnerships and networking for program improvement state-wide.
An Academic Excellence Indicator System for Adult Education
The Texas Education Agency will submit the required Annual Performance Report to the USDOE. Other state performance measures may be submitted to the Texas Legislative Budget Board and Texas Workforce Investment Council (TWIC). The Council was created by the Governor and the Legislature to carry out certain strategic planning and evaluation functions in order to promote the development of a well-educated, highly skilled workforce for Texas and to advocate the development of an integrated workforce development system that provides quality services to address the needs of businesses and workers. The Council is also charged by the Legislature and designated by the Governor to carry out the federal and state duties and responsibilities of advisory councils required by federal law or regulation, including serving as the State Workforce Investment Board under the federal Workforce Investment Act of 1998. [See executive summary for TWIC in Section 3.3]
The performance accountability system for adult education includes the Indicators of Program Quality (Appendix I) and core indicators established in the State Plan. Additional indicators may be added to the system as it is refined. The level of detail necessary for such an accountability system will be possible thanks to the amount of adult education data resident in the adult education management information system, ACES. ACES assessment data will be used to determine demonstrated improvement in literacy skill levels.
To calculate the numbers and percentage of Adult Secondary Education students who achieve a GED or diploma, Texas will use an automated record match of the TEAMS student records with the Agency’s GED database and K-12 database (PEIMS) nightly to give programs daily updated information about GED attainment.
To calculate the numbers and percentages of adult education students who enter postsecondary education, employment, or job training, Texas will use data reported in TEAMS as well as an automated record match of the TEAMS student records with the Texas Workforce Commission databases of participants in job training and Unemployment Insurance (UI) and the Texas Higher Education Coordinating Board database of public colleges and universities at annually. A memorandum of understanding exists between the Texas Education Agency and the Texas Higher Education Coordinating Board, to match data in the Higher Education database and for Higher Education to match for TEA with the database of Texas Workforce Commission.
During the next two years, TEA with the assistance of HCDE will gather and analyze data and explore the possibilities for recognizing programs through an approved incentive system that demonstrate performance above the state standard. Programs whose performance falls below the adopted state standard will be provided technical assistance and may have an approved system of monetary sanctions applied.
Based on the results of program evaluations, programs that employ scientifically-based or evidence-based research for the design and implementation of programs which exceed target performance measures thus exhibiting excellence in providing services to special populations and show continued promise, may be considered for additional resources to further the investigation of the effectiveness of the strategy or dissemination of information about its initial findings to other local programs.
Indicators of Program Quality are the basis of program planning, academic excellence, teacher proficiencies, learner outcomes, program planning, curriculum and instruction, support services for educationally disadvantaged adults and other undereducated adults, professional development, and student retention.