Time Management

National Reporting System: State Data Quality Standards Checklist The Latest News from the Texas Education Agency Your State Adult Education Office by Sheila Rosenberg, Ph.D. Senior Director, Division of Adult and Community Education

The State Office has received the following information regarding Certification of the Local and State National Reporting System (NRS) data.

The quality of NRS data submitted by states varies widely and this problem has been of great concern within states and to the U.S. Department of Education. The Office of Vocational and Adult Education (OVAE) has been working on developing a data certification process whereby states would attest to the level of NRS data quality. This attestation will include a formal data quality checklist that describes data procedures for each main area of NRS data collection within the state. The State Data Quality Checklist includes content areas and scoring levels for the checklist and the general content that the checklist will cover in each area. Scoring levels range from "Level D" - unacceptable, to "Level C" - minimum acceptable procedures, to "Level A" - the highest level of quality. The U.S. Department of Education (USDE) will use the checklist to describe the general level of quality of each state's data and as a tool for improvement of data quality within the states. State offices are to return the completed Checklist by January 31, 2003.

The Checklist Completed by the State Office
The NRS state data quality standards identify the policies, processes and materials that states and local programs should have in place to collect valid and reliable data for the NRS. The Department developed the standards to define the characteristics of high quality state and local data collection systems for the NRS. The standards provide an organized way for the Department to understand the quality of NRS data collection within the states and also provide guidance to states on how to improve their systems. States are to complete this checklist, which incorporates the standards, with their annual NRS data submission to rate their level of implementation of the standards.

General Content Areas. The standards are organized into four content areas that define high quality data collection systems.

Data Foundation and Structure. This content area addresses whether the State has in place the foundation and structures for collecting quality data that meet NRS guidelines. Standards measure whether the state has policies for assessment, follow-up and goal setting; whether local programs know those policies and whether the state conducts validity studies to ensure processes are working to produce accurate and reliable data.

Data Collection and Verification. This area measures whether the state collects measures according to NRS guidelines, using procedures that are likely to result in high reliability and validity. Standards also address whether data are collected in a timely manner, are systematically checked for errors, and whether the state also has processes for verifying the validity of data.

Data Analysis and Reporting. The quality standards in this content area include whether the state has systems for analyzing and reporting data, including appropriate databases and software. The standards also address whether analyses and reports are produced regularly, are used to check for errors and missing data, meet NRS and state needs, and are useful to state and local staff for program management and improvement.

Staff Development. The standards under this area address whether the state has systems for professional development of state and local staff on the NRS, including whether the state has provided training on data collection, measures, assessment, goal setting and follow-up procedures. Standards also focus on whether the training is ongoing and continuous, meets the needs of state and local staff, and is designed to improve data quality.

Levels of Quality and Quality Improvement
Within each area there are three levels of quality that reflect whether the state has policies and procedures likely to improve the reliability and validity of data. Based on the checklist, the Department will classify states' NRS data procedures into one of these levels each year. Texas, of course, anticipates being able to be graded in the Exemplary Quality category - if not this first year, then by next year!

Acceptable Quality - State policies and procedures for implementing the NRS meet the essential requirements for NRS implementation, as described in the NRS Implementation Guidelines and Guide for Improving NRS Data Quality. This level reflects the minimum acceptable for the December 2002 data submission, according to the Department policy.

Superior Quality - State procedures go beyond the minimum to promote higher levels of data validity and reliability through more rigorous definitions, regular oversight of data collection methods, ongoing assistance to local programs on NRS data issues and procedures for verifying the accuracy of data. In future years, states will be required to have NRS policies and procedures conforming to this level.

Exemplary Quality - The state has procedures and systems that promote the highest levels of data validity and reliability, including systems for verifying data accuracy from local programs, systems for monitoring data collection and analyses, and has corrective systems to improve data on an ongoing basis. State procedures indicate a focus on continuous improvement of the quality and accuracy of data.

States will have to meet all of the standards within a quality level to be considered at that level. In addition, the scoring will be cumulative, so that to score at the superior level, a state would have to meet all of the standards for that level AND all standards for the acceptable quality level. To rank at the exemplary level, a state would have to meet all of the standards for ALL quality levels. The State Office continues to make improvements to our management information systems (ACES and TESPIRS). We have a list of improvements from many of you that we are working to implement. In summation, as we stated above, the State Office expects to attain the highest quality - Exemplary quality - in the NRS data quality standards checklist for Texas - if not this first year, then by next year - 2004!!!"

We are ready to go into phase II for the GED 2002, Instructional Strategies Training.

Co-op directors can e-mail Myrna Saucedo-Schwarz, mschwarz@tea.tetn.net, to schedule for 6-hour, 9-hour, 12-hour or 15-hour trainings of a compilation of Language Arts-Writing, Language Arts-Reading, Social Studies, Science or Mathematics. Each subject is presented in 3-hour increments.

Directors should specify the following information in their e-mails: number of hours for trainings: 6 hours, 9 hours, 12 hours or 15 hours; subject or subjects to be presented; and number of participants expected.

Attendance at these trainings is a minimum of 8 and a maximum of 20 participants only.

Additional trainings can be scheduled to accommodate participants who were unable to attend training because of the limitation of space.

We look forward to working with you in helping our students improve their lives.

The funds are available through the 21st Century Community Learning Center Grant program. The program provides funding to public and private groups that offer academic and enrichment services to children. Those services could include tutoring, particularly for those children who attend low-performing schools and who need help with math or reading. The grant funds could also be used to offer a broad array of additional services and programs, such as drug and violence prevention programs; counseling programs; art, music and physical education and fitness programs; and technology programs that are designed to complement regular academic programs, as well as family literacy programs.

Eligible to receive these grants are school districts, charter schools, education service centers, community-based organizations and other public or private entities. One application may cover as many as five community learning centers.

The grants, which will be awarded later this spring, will range from $50,000 to $175,000 per community center. A copy of the Request for Application (RFA) 701-02-035 may be obtained by writing the Texas Education Agency, Document Control Center, Room 6-108, 1701 N. Congress Avenue, Austin, Texas 78701; by calling (512) 463-9304; by faxing (512) 463-9811 or by sending an email to: dcc@tea.state.tx.us.

Applications must be received by 5 p.m. April 1 to be considered for funding. Programs funded by the grant are to begin in the summer of 2003.