Workforce Investment Act:
DOL Should Do More to Improve the Quality of Participant Data
GAO-14-4: Published: Dec 2, 2013. Publicly Released: Dec 20, 2013.
What GAO Found
Flexibility in the Department of Labor's (DOL) data reporting guidance and limitations in some state information systems continue to impair the quality of the data on participants in the Workforce Investment Act (WIA) Adult and Dislocated Worker Programs. The flexibility in the guidance stems from the inherent nature of WIA, which allows states and local areas to tailor program design and service delivery to their needs. As a result, DOL's guidance on collecting and reporting the data allows variation in how some WIA data elements are defined, collected, and reported. Specifically, an American Job Center--formerly known as a one-stop center--can choose to provide certain basic services exclusively through WIA programs, exclusively through a partner program, or through a blend of both WIA and partner programs. However, this flexibility involves variations in data reporting that have contributed to inconsistencies among states regarding when job seekers are counted as WIA participants. Moreover, some aspects of DOL's guidance are open to interpretation, leaving it to states to define variables such as type of training service received, further contributing to data inconsistencies. In addition, some state information systems used to collect and report WIA participant data have limitations that hamper the affected states' ability to report uniform and complete data. For example, data are incomplete to the extent that states may not have information systems that can track participants who access services online without significant staff assistance. Having inconsistent and incomplete data makes it difficult for DOL to compare data on program participants across states or to aggregate the data at a national level.
DOL engages in various oversight activities designed to ensure the accuracy of states' data on participants in the WIA Adult and Dislocated Worker Programs and has taken steps to improve data consistency across states. However, DOL does not consistently use the results of its oversight to identify and resolve systemic data issues nor has it evaluated the effect of oversight on the quality of WIA participant data. Specifically, DOL requires states to validate the data they collect and report on participants in the WIA Adult and Dislocated Worker Programs on an annual basis, but it does not strategically use the findings from this effort to identify systemic data issues or improve the quality of the data. Similarly, although DOL's regional offices review a sample of each state's WIA participant files every few years to assess states' compliance with data reporting and validation requirements, DOL officials said they have not analyzed the findings from the most recent reviews to identify nation-wide reporting issues. DOL has taken steps to improve the consistency of the data by providing general technical assistance to states and local areas and through standardizing the way DOL collects WIA data. For example, since 2007, two states have been piloting a unified reporting system developed by DOL that uses standardized data definitions and is integrated across certain American Job Center programs administered by DOL. However, DOL officials said they have no plans to evaluate the system before expanding it to other states. Without an evaluation, DOL does not know what impact the pilot has had on the quality of WIA participant data.
Why GAO Did This Study
Having reliable program data is important in effectively managing a program. However, there have been longstanding concerns about the quality of data on job seekers enrolled in the WIA Adult and Dislocated Worker Programs, which rely on states and local areas to track participants and the services they receive. Given these concerns and WIA's anticipated reauthorization, GAO was asked to examine the data on these WIA participants. This report addresses: (1) the factors that have affected the ability to report consistent and complete data on participants in the WIA Adult and Dislocated Worker Programs, and (2) actions that DOL has taken to improve the quality of these data. To conduct this work, GAO reviewed relevant federal laws, regulations, guidance, and documentation from DOL. GAO interviewed officials from DOL's national and regional offices and state and local workforce officials from a nongeneralizable sample of eight states. GAO also analyzed WIA data from program year 2011 to determine the number of, characteristics of, and services provided to WIA participants.
What GAO Recommends
GAO recommends that DOL take steps to improve the consistency and completeness of data reported across states and to promote a continuous process for improving the data's quality. DOL officials did not agree or disagree with GAO's overall recommendations and detailed how data quality is being addressed primarily through existing efforts. However, GAO believes that the recommendations remain valid as discussed in the report.
For more information, contact Revae E. Moran at (202) 512-7215 or firstname.lastname@example.org.
Recommendations for Executive Action
Comments: While the Department of Labor (DOL) believes its guidance is clear on reporting services provided to Workforce Investment Act Adult and Dislocated Worker Program participants by partner programs, it plans to develop additional guidance to clarify areas that are open to interpretation and work with states on related issues in the future. In addition, DOL stated that the Workforce Investment Act Standardized Record Data system is not explicitly designed to improve data quality but rather, to ease the reporting burden on states. As such, DOL does not believe that expanding the use of the system will improve data collection and reporting but will, nevertheless, conduct an assessment of the system along with state and local data collection procedures. We commend the Department for taking these steps and will monitor the development of additional guidance and assessment.
Recommendation: To improve the consistency and completeness of national data on participants in the WIA Adult and Dislocated Worker Programs, the Secretary of Labor should take additional steps to improve the uniformity of participant data reported by states. These steps could include the following: (a) providing additional guidance to states on data reporting, such as how core and intensive services should be recorded for WIA participants who receive these services through partner programs; and (b) conducting an evaluation or review of Workforce Investment Streamlined Performance Reporting System (WISPR) to determine if it has resulted in more complete and consistent data collection and reporting for participants in the WIA Adult and Dislocated Worker Programs and placing a high priority on the implementation of WISPR if it is shown to improve data consistency and completeness.
Agency Affected: Department of Labor
Comments: DOL plans to work with its Regional Offices to analyze data validation results to identify and address areas of concern across multiple states. DOL is also soliciting input from states on streamlining data validation methods to increase efficiency and accountability. In addition, DOL will examine the effect of data validation on program error rates and continue to provide targeted guidance to states regarding systematic data issues. Finally, DOL will continue to regularly monitor Social Policy Research Associates' corrections of state participant data and disseminate promising practices on data collection and reporting through its Workforce Systems Strategies website. We will monitor the development of DOL's plans to promote a continuous process for improving data quality.
Recommendation: To improve the consistency and completeness of national data on participants in the WIA Adult and Dislocated Worker Programs, the Secretary of Labor should take additional steps to improve the uniformity of participant data reported by states. The Secretary of Labor should promote a formal, continuous process for improving the quality of data on participants in the WIA Adult and Dislocated Worker Programs through such measures as the following: (a) consistently sharing the results of all oversight activities with states and local areas, including findings from validation of participant data; (b) reviewing the methods used for data validation, such as its scope and error rate threshold, to identify opportunities to increase efficiencies and accountability in the process. This could include implementing, if appropriate, recommendations from the Regions' review of data validation procedures; (c) evaluating data validation efforts to determine their effects on data quality, particularly on systemic errors, and providing targeted guidance and assistance to states and local areas to address such errors; (d) regularly monitoring Social Policy Research Associates' corrections and analyses of state WIA participant data, sharing this information with states, and coordinating with states to ensure that any corrections are appropriate and accurate; and (e) collecting and disseminating promising practices to states and local areas on data collection and reporting on a regular basis.
Agency Affected: Department of Labor