6005 Westview Dr. | Houston, Texas | 77055-5419 | 713-696-0700 | 1-866-696-4233

Texas Even Start Administrative Manual

June 2004, (Revised January 2008, September 2008)

Monitoring & Technical Assistance

Planning and Program Evaluation

Program Planning

---

Planning for adult education and literacy programs is an on-going process, and the Indicators of Program Quality stress the collaborative nature of program planning. Adult education and literacy eligible applicants who intend to apply to TEA for funds must gather the necessary information to conduct comprehensive needs assessments; meet with advisory committees and service providers in the community; identify resources; establish goals and performance objectives; develop recruitment strategies; develop instructional programs based on identified needs, resources, and Indicators of Program Quality; develop process (formative) and product (summative) evaluation plans; develop professional development plans; develop coordination plans and agreements; and budget funds to various line items.

The adult education program planning process must be guided by an extensive needs assessment of the target population to be served in order to reduce barriers to access to adult education and literacy programs through a collaboratively designed plan of action.

In regard to the delivery of adult education and literacy instruction, the Indicators of Program Quality, in the area of Program Planning, say that the adult education program must reflect broad-based, collaborative input from public and private representatives of the public sector.

In addition, the indicators stress collaboratively developed evaluation procedures that determine the accomplishment of goals and objectives and provide managers with feedback and implementation and the need for “mid-course corrections.”

Collaboration requires joint planning, implementation, and process and product evaluation, as well as a sharing of resources, power, and authority. The result of this activity is comprehensive services for adults and their families.

Program Evaluation

The overall purpose of a program evaluation plan is to determine though formative and summative evaluation processes whether the program met its stated goals and achieved the desired results based on the objectives and performance targets established by the grantee.

Grantees selected for funding must provide a final evaluation which addresses, in general, the following:

• The extent to which the activities of the project where implemented as planned
• The effectiveness of the activities in achieving the goals and objectives of the project
• The impact of the project activities on the participants
• The extent to which the performance targets were met

The purpose of process evaluation is to identify, during implementation, any defects in the design or potential sources of failure of a project. Process evaluation provides information about whether a project was implemented as designed. Process evaluation also provides project management with timely feedback about the implementation procedures of the project allowing changes during implementation rather than waiting for an end-of-project analysis.

Process evaluation is planned by management, staff, and the evaluators. Process evaluation may be conducted by internal evaluators, including advisory committees, or by external evaluators. A plan for process evaluation should include:

1. a list of process evaluation questions to be investigated. These questions focus on the areas of interest to project management and participating service providers and include questions about the factors that influence the success of the project. Identifying the process evaluation questions is a critical task and should be thoughtfully undertaken;
2. a plan and procedures for collecting information and data to answer the questions;
3. a time frame for collecting the information; and
4. a description of the reporting system and the intended audience.

The purpose of conducting a summative or process evaluation is to assess a program after development for the purpose of determining its merit and effectiveness. Product evaluation refers to measuring achieved results, as opposed to desired results. A plan for product evaluation should include:

1. how the accomplishment of each performance objective will be determined;
2. the data sources that will be used; and
3. how the data will be analyzed.

Conducting the process or formative evaluation involves two components:

1. the program receives quarterly information from the AEGIS system as to performance, and the program engages in a process of collaborative ongoing self-evaluation in order to identify and enhance areas of strength and to be able to address areas of concern identified by AEGIS.
2. AEGIS is used to perform regular desk reviews of program performance. The system will generate information regarding compliance with program requirements, performance measures, and progress completion of program objectives.

Compliance

• Every program has to be in compliance with program requirements (Compliance Indicators)
• Federal requirements
• Goals set by Fiscal Agent in grants application
• Desk reviews will be performed at the end of every quarter to check the program status and to identify any issues or difficulties with program implementation
• Scoring mechanism has been developed to identify the level of compliance with the program requirements
• Program issues will be worked out by Fiscal Agent and Grants Services Manager by communication through AEGIS

Desk Review Follow Up

After AEGIS generates the desk review, the following steps should take place:

• Desk review may raise some program related issues

• It is Fiscal Agent’s responsibility to provide an explanation or improvement plan

• Submission of improvement plan triggers a notification to Grant Services Manager

• Grant Services Manager reviews FA’s response to the issue, and Closes or Reopens the issue

• Issue resolution interactions will be stored in the system for future reference

Notifications

• Fiscal Agent will be notified on completion of the Desk Review

• Notifications can also be viewed by recipients using AEGIS Portal

• All the notifications sent through AEGIS system will be saved for historic purpose

Engaging in an ongoing process of self-evaluation will help programs to provide immediate answers to any issues generated by the desk review, and to design an appropriate program improvement plan.

---

Previous | Next | Table of Contents