Workforce Investment Act: Labor and States Have Taken Actions to Improve Data Quality, but Additional Steps Are Needed
GAO-06-82
Published: Nov 14, 2005. Publicly Released: Nov 14, 2005.
Skip to Highlights
Highlights
Federal programs carried out in partnership with states and localities continually balance the competing objectives of collecting uniform performance data with giving program implementers the flexibility they need. Our previous work identified limitations in the quality of performance data for the key employment and training program--the Workforce Investment Act (WIA). WIA relies on states and localities to work together to track and report on participant outcomes, and it changed the way outcomes are measured. Given the magnitude of changes and the impact such changes can have on data quality, we examined (1) the data quality issues that affected states' efforts to collect and report WIA performance data; (2) states' actions to address them; and (3) the actions the Department of Labor (Labor) is taking to address data quality issues, and the issues that remain.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Labor | To address the inconsistencies in determining when participants should be registered and counted in the performance measures, the Secretary of Labor should determine a standard point of registration and monitor states to ensure that the policy is consistently applied. |
Closed – Not Implemented
In FY08, Labor stated that it provided guidance on who is considered a participant and counted in the performance measures in its Training and Employment Guidance Letter 17-05, Common Measures Policy for the Employment and Training Administration's (ETA) Performance Accountability System and Related Performance Issues. GAO believes, however that this revised policy guidance does not address the issue. Labor also noted that it provided technical assistance to states and local areas on performance-related issues, including participation. According to Labor, its Performance Enhancement Project (PEP) has been able to engage in a performance dialogue with state and local areas. PEP outreach and targeted technical assistance has taken many forms, such as webinars, conference calls, and face-to-face meetings. Labor adds that these and other technical support efforts (for example, the performance webinar series) have supported the communication of consistent and ongoing performance guidance. As needed, Labor used these opportunities to clarify policy. Technical assistance was also provided on how to best handle program flow and modify Management Information Systems to meet TEGL 17-05 requirements. In FY 2010, Labor reported that it continues to provide technical assistance to states on performance accountability and has convened a workgroup of state representatives to help further clarify the guidance on how to report self-service participants. While these steps may lead to improvements in performance data quality, Labor has not yet set a standard point of registration. GAO believes that actions taken do not fully address our recommendation.
|
| Department of Labor | To enhance the data validation requirements, the Secretary of Labor should conduct its own review of the WIA participant files validated by states to ensure that states did this correctly. |
Closed – Implemented
In 2006, Labor officials told us they planned to modify the data validation software to allow federal staff to pull a sub-sample of states' data validation records and conduct a federal review of these records. In 2008, Labor reported that it continues to be interested in making this enhancement to the software. However, due to resource constraints, priority has been given to enhancements to allow the states to continue using the software for data validation and to make it easier for the states to use in carrying out their data validation responsibilities. The regional staff continues to review state validation files and provide ongoing recommendations and suggestions to the states for improving their record-keeping. In FY2010, Labor reported that its regional office staff now conduct reviews of WIA participant files validated by states to ensure accuracy, and provide ongoing recommendations and suggestions to the states, helping them to improve their recordkeeping.
|
| Department of Labor | To enhance the data validation requirements, the Secretary of Labor should ensure that steps are taken to hold states accountable to both the report validation and data element validation requirements. |
Closed – Implemented
Labor originally planned to have standard error rates for report validation in place by program year 2004. However, Labor revised reporting requirements to implement a set of common measures across Employment and Training Administration programs. To allow states a sufficient transition period, Labor delayed establishing acceptable error rates and explored the possibility of including states' data validation results as part of the criteria for determining eligibility for incentive awards. In 2008, Labor reported that it included data validation results as one of the criteria for determining eligibility for WIA incentive awards (see TEGL 9-07). The results will be considered in determination of eligibility for awards made based on Program Year 2007 outcomes. In FY2010, Labor reported that it now factors the results of data validation into the determination of eligibility for WIA incentive grant awards. These results, which are determined by analysis of overall error rates, were used for the eligibility determination of the incentive grant awards based on Program Year 2008 (July 2008-June 2009) performance.
|
| Department of Labor | To address variations in federal monitoring practices, the Secretary of Labor should develop a standard comprehensive monitoring tool for WIA performance data that is used across all regions, including monitoring the new guidelines for determining when participants end services. |
Closed – Implemented
In 2006, Labor agreed with this recommendation and told us they were taking steps to develop a comprehensive monitoring guide for all regions that will include a section on the review of performance information. In 2008, Labor reported that it has used the Core Monitoring Guide since 2005, which has one chapter devoted to performance management, as its standard tool for monitoring performance-related subjects with our grantees. The agency also developed a draft data validation review guide as a supplemental tool that is being used to monitor the validity of reported data. This draft supplemental guide directs the reviewer to verify that state policy accurately reflects Labor's guidelines on program exiters, and to test a sample of participant files comparing the exit date with the last service date for validity. These reviews have been and continue to be conducted by each of Labor's regional offices. Labor also conducted 3 national training sessions for grantees in 2006 which included training on the provisions in TEGL 17-05 relating to the 90-day soft exit guidelines now in effect under the common measures. In FY2010, Labor reported that it is finalizing its data validation monitoring guide and expects to have the guide completed by fall 2010. The guide will supplement the overall Core Monitoring Guide.
|
Full Report
Office of Public Affairs
Topics
Data collectionData integrityInformation systemsInformation technologyIntergovernmental relationsIT standardsPerformance measuresReporting requirementsData qualityLabor force