Managing for Results:

Greater Transparency Needed in Public Reporting on the Quality of Performance Information for Selected Agencies' Priority Goals

GAO-15-788: Published: Sep 10, 2015. Publicly Released: Sep 10, 2015.

Additional Materials:

Contact:

J. Christopher Mihm
(202) 512-6806
mihmj@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The six agencies GAO reviewed generally did not publicly report on how they ensured the accuracy and reliability of performance information used to measure progress on their highest priority performance goals, referred to as agency priority goals (APGs). The GPRA Modernization Act of 2010 (GPRAMA) requires agencies to identify the following when publicly reporting on their APGs: 1) how performance information was verified and validated; 2) data sources; 3) level of accuracy required for intended use; 4) any limitations at the required level of accuracy; and 5) how the agency will compensate for such limitations (if needed) to reach the required level of accuracy. GPRAMA requires agencies to provide this information to the Office of Management and Budget (OMB) for publication on Performance.gov. GPRAMA also directs agencies to provide this information for performance goals, which include APGs, in their annual performance plans and reports. While all six agencies described how they ensured the quality of their performance information overall, GAO found discussions about performance information quality addressing all five GPRAMA requirements in only the Department of Homeland Security's (DHS) performance plans and reports.

Agency Priority Goal (APG) Performance Information Quality Discussion in Selected Agencies' Performance Plans and Reports for FY13-FY16

Agency

Description of how agency ensured performance information quality overall

Number of APGs for FY14 and FY15

Description of how each APG met GPRAMA performance information quality requirements

Agriculture

Yes

3

0

Defense

Yes

4

0

Homeland Security

Yes

3

3

Interior

Yes

6

0

Labor

Yes

3

0

National Aeronautics and Space Administration (NASA)

Yes

4

0

Total

6 yes

23

3

Source: GAO analysis of selected agencies' performance plans and reports. |GAO-15-788

OMB and the Performance Improvement Council (PIC)–a cross-agency council of agency performance improvement officers–established the Data Quality Cross-Agency Working Group in February 2015. The group has identified several goals, such as improving the reliability and quality of performance information, and could serve as a vehicle for disseminating good practices in public reporting on data quality.

Why GAO Did This Study

Federal agencies have not always clearly and transparently explained to Congress and the public how they ensure the quality of their performance information. GPRAMA requires agencies to publicly explain how they ensure the accuracy and reliability of their performance information used to assess progress for their APGs.

This is one of a series of GAO reports examining the implementation of GPRAMA, as required by the act. This report assesses how well selected agencies publicly reported on the quality of performance information used to measure progress on APGs. GAO selected six agencies–the Departments of Agriculture, Defense, Interior, and Labor, and NASA and DHS– based on GAO's 2013 federal managers survey on their agency's use of performance information. GAO reviewed information concerning these agencies' APGs published on Performance.gov and in their annual performance plans and reports.

What GAO Recommends

GAO recommends that all six of the agencies work with OMB to describe on Performance.gov how they are ensuring the quality of their APGs's performance information, and that all agencies, except for DHS, also describe this information in their annual performance plans and reports. GAO also recommends that OMB, working with the PIC, focus on ways the PIC's data quality working group can improve public reporting for APGs. OMB did not comment on the recommendations, but the six agencies generally concurred or identified actions they planned to take to implement them.

For more information, contact J. Christopher Mihm at (202) 512-6806 or mihmj@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In response to our recommendation, DHS added an overall description on Performance.gov of how the agency ensures reliable performance information is reported to external audiences. DHS's explanation refers to the data quality appendix in its performance plan and report covering fiscal years 2015-2017 and a hyperlink is provided from Performance.gov to the DHS website where this appendix can be found. Further, DHS also made it easier to locate information concerning its APGs in this data quality appendix by consolidating the relevant discussion in one section.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Homeland Security

  2. Status: Open

    Comments: The Department of Agriculture (USDA) has identified two priority goals (APGs) for fiscal years 2018 and 2019 and has posted information describing its progress through the first two quarters of fiscal year 2019 on Performance.gov. For the improving customer experience at USDA goal, USDA reports that data sources and collection methodologies are being developed for the majority of the key performance indicators. For the modernize USDA's Information Technology goal, USDA provides no data quality explanation. In a November 2018 letter to the Comptroller General, USDA's Inspector General acknowledged that additional efforts are needed to better describe the quality of the data supporting the APGs and said that the Department will increase the amount of information provided in the quarterly APG updates. In May 2019, we requested updated information from USDA officials and as of June 25, 2019 have not received the requested information. We will continue to monitor USDA's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Agriculture

  3. Status: Open

    Comments: The Department of Defense (DOD) has identified three priority goals (APGs) for fiscal years 2018 and 2019 and we found information describing its progress through December 2018 on Performance.gov when we checked the website on June 25, 2019. For the reduce regulatory burden APG, DOD provides a data accuracy and reliability statement describing verification and validation procedures and identifying data sources, but no explanation is provided of the other three requirements such as potential limitations. For the financial management APG, no data quality explanation is provided and the document summarizing progress made on the force readiness APG was not available at the time of our review. In July 2019, DOD officials reported that they continue to work to address our recommendation. We will continue to monitor DOD's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Defense

  4. Status: Open

    Comments: The Department of the Interior (Interior) has identified six priority goals (APGs) for fiscal years 2018 and 2019. Interior provides specific data quality explanation addressing the five GPRA Modernization Act of 2010 (GPRAMA) requirements for three APGs: Improve Access to Leasing Public Lands for Oil, Gas, and Mineral Extraction, Improve Processing of Drilling Applications (the last available update is from December 2018), and Water Conservation and Supply Enhancement. For two additional APGs--the Species Status Assessments for Listings with State Input (the last available update is from December 2018) and Improve Condition of Priority Assets Necessary for Mission Delivery Across the Department of the Interior--Interior partly addresses GPRAMA's requirements. For the remaining APG--the Ensure Clear, Quantitative Criteria for Threatened Endangered Species' Recovery Plans--Interior provides a general statement on data accuracy and reliability, but not more specific data quality explanation. In June 2019, we requested updated information from Interior officials, but as of June 25, 2019 have not received the requested information. We will continue to monitor Interior's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of the Interior

  5. Status: Closed - Implemented

    Comments: In response to our recommendation, the Department of Labor (Labor) added data quality descriptions to Performance.gov for four of its five agency priority goals (APGs) for fiscal years 2018 and 2019 that explain how Labor ensures the accuracy and reliability of the performance information used to measure progress toward each APG. For Labor's remaining APG to begin implementation of a shared services model for administrative functions, the agency identifies data sources on Performance.gov and its performance report for fiscal year 2018 (which can be accessed through Performance.gov ) provides additional data quality discussion addressing the remaining four requirements

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Labor

  6. Status: Closed - Implemented

    Comments: In response to our recommendation, NASA added data quality descriptions to Performance.gov for each of its four APGs for fiscal years 2016 and 2017 that explain how NASA tracks progress toward each APG, consistent with GPRAMA requirements.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: National Aeronautics and Space Administration

  7. Status: Open

    Comments: The Department of Agriculture's (USDA) Fiscal Year 2020 Annual Performance Plan and Fiscal Year 2018 Annual Performance Report provides a general statement on how the agency uses a standardized methodology to measure its performance and that agency officials attest to the quality of the performance information. USDA also identifies its priority goals, but specific data quality explanation is not provided for these APGs. In a November 2018 letter to the Comptroller General, USDA's Inspector General stated that the Department agrees with the recommendation and will begin providing data quality explanation for the APGs in its next annual performance plan and report to be published in February 2019, but as noted above the plan and report do not provide the required explanation. In May 2019, we requested updated information from USDA officials and as of June 25, 2019 have not received the requested information. We will continue to monitor USDA's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Agriculture

  8. Status: Open

    Comments: The Department of Defense's (DOD) Fiscal Year (FY) 2020 Annual Performance Plan and FY2018 Annual Performance Report states that, "each goal owner has attested the performance results and narrative information included in this report is complete, accurate, and reliable; and that data validation and verification procedures are documented and available upon request," and DOD refers readers to Performance.gov for more information about its priority goals (APGs). However, the GPRA Modernization Act of 2010 requires agencies to publish more specific data quality explanation for each APG in performance plans and reports and on Performance.gov. DOD's performance plan and report does not contain the more specific explanation required. Nor did our review of Performance.gov find the required explanation. In July 2019, DOD officials reported that they continue to work to address our recommendation. We will continue to monitor DOD's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Defense

  9. Status: Open

    Comments: The Department of the Interior's (Interior) 2020/2019 Annual Performance Plan & 2018 Report (APP&R) includes a section concerning data accuracy and reliability and describes in general terms how Interior ensures the accuracy and reliability of performance information and how it addresses the five data quality requirements in the GPRA Modernization Act of 2010 (GPRAMA). Interior states in this section that measurement procedures for agency performance goals are described on Performance.gov. As of June 2019, our review found that Interior has provided specific data quality explanation addressing the five GPRAMA requirements for three of its six APGs on Performance.gov. In June 2019, we requested updated information from Interior officials, but as of June 25, 2019 have not received the requested information. We will continue to monitor Interior's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of the Interior

  10. Status: Closed - Implemented

    Comments: The Department of Labor's (Labor) performance report for fiscal year 2018 (which also serves as its performance plan for fiscal year 2020) refers readers interested in its agency priority goals (APGs) to Performance.gov. In response to our recommendation, Labor added on Performance.gov data quality descriptions for four of its five APGs for fiscal years 2018 and 2019. For Labor's remaining APG to begin implementation of a shared services model for administrative functions , the agency identifies data sources on Performance.gov and its performance report for fiscal year 2018 provides additional data quality discussion for specific measures related to this APG which address the remaining requirements

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Labor

  11. Status: Closed - Implemented

    Comments: In response to our recommendation, NASA added a data quality explanation to the section of its FY2015 Annual Performance Report/FY2017 Annual Performance Plan that describes how NASA tracks progress toward each of its four APGs for fiscal years 2016 and 2017, consistent with GPRAMA requirements.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: National Aeronautics and Space Administration

  12. Status: Closed - Implemented

    Comments: In July 2018, OMB staff shared with us a reporting template for Performance.gov for the fiscal years 2018 and 2019 APGs. This template allows agencies to provide information on data accuracy and reliability for individual APGs and agencies may provide hyperlinks to more detailed data quality discussions published in their annual performance plans and reports.

    Recommendation: To help participating agencies improve their public reporting, the Director of OMB, working with the PIC Executive Director, should identify additional changes that need to be made in OMB's guidance to agencies related to ensuring the quality of performance information for APGs on Performance.gov.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  13. Status: Closed - Implemented

    Comments: In addition to highlighting the need for better public reporting, our review also focused more attention on agencies' internal processes for verifying and validating performance information. In July 2015, we briefed the Performance Improvement Council's (PIC) Data Quality Cross-Agency Working Group on leading practices identified in our prior work for ensuring agencies are accurately verifying, validating, and reporting performance information, and shared GAO's performance and accountability report which describes our own data quality processes. In February 2016, the PIC Executive Director directed Performance Improvement Officers (PIOs) and their deputies to complete a self-assessment of their agency's data quality policies and procedures to ensure these were consistent with OMB guidance. PIC staff summarized the agencies' self-assessments at the March 2016 meeting of the PIOs for the Chief Financial Officers Act agencies. We reviewed presentation notes which confirmed that PIC staff had addressed our recommendation by summarizing for the PIOs the results of these self-assessments, identifying aspects of data quality in which agencies had generally rated their performance highest, and other aspects of data quality in which agencies had rated their performance lowest.

    Recommendation: To help participating agencies improve their public reporting, the Director of OMB, working with the PIC Executive Director, should identify practices participating agencies can use to improve their public reporting in their performance plans and reports of how they are ensuring the quality of performance information used to measure progress towards APGs.

    Agency Affected: Executive Office of the President: Office of Management and Budget

 

Explore the full database of GAO's Open Recommendations »

Aug 1, 2019

Jul 26, 2019

Jul 24, 2019

Jul 23, 2019

Jul 19, 2019

Jul 18, 2019

Jul 16, 2019

Jul 2, 2019

Jun 14, 2019

Looking for more? Browse all our products here