Managing for Results:

Greater Transparency Needed in Public Reporting on the Quality of Performance Information for Selected Agencies' Priority Goals

GAO-15-788: Published: Sep 10, 2015. Publicly Released: Sep 10, 2015.

Additional Materials:

Contact:

J. Christopher Mihm
(202) 512-6806
mihmj@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The six agencies GAO reviewed generally did not publicly report on how they ensured the accuracy and reliability of performance information used to measure progress on their highest priority performance goals, referred to as agency priority goals (APGs). The GPRA Modernization Act of 2010 (GPRAMA) requires agencies to identify the following when publicly reporting on their APGs: 1) how performance information was verified and validated; 2) data sources; 3) level of accuracy required for intended use; 4) any limitations at the required level of accuracy; and 5) how the agency will compensate for such limitations (if needed) to reach the required level of accuracy. GPRAMA requires agencies to provide this information to the Office of Management and Budget (OMB) for publication on Performance.gov. GPRAMA also directs agencies to provide this information for performance goals, which include APGs, in their annual performance plans and reports. While all six agencies described how they ensured the quality of their performance information overall, GAO found discussions about performance information quality addressing all five GPRAMA requirements in only the Department of Homeland Security's (DHS) performance plans and reports.

Agency Priority Goal (APG) Performance Information Quality Discussion in Selected Agencies' Performance Plans and Reports for FY13-FY16

Agency

Description of how agency ensured performance information quality overall

Number of APGs for FY14 and FY15

Description of how each APG met GPRAMA performance information quality requirements

Agriculture

Yes

3

0

Defense

Yes

4

0

Homeland Security

Yes

3

3

Interior

Yes

6

0

Labor

Yes

3

0

National Aeronautics and Space Administration (NASA)

Yes

4

0

Total

6 yes

23

3

Source: GAO analysis of selected agencies' performance plans and reports. |GAO-15-788

OMB and the Performance Improvement Council (PIC)–a cross-agency council of agency performance improvement officers–established the Data Quality Cross-Agency Working Group in February 2015. The group has identified several goals, such as improving the reliability and quality of performance information, and could serve as a vehicle for disseminating good practices in public reporting on data quality.

Why GAO Did This Study

Federal agencies have not always clearly and transparently explained to Congress and the public how they ensure the quality of their performance information. GPRAMA requires agencies to publicly explain how they ensure the accuracy and reliability of their performance information used to assess progress for their APGs.

This is one of a series of GAO reports examining the implementation of GPRAMA, as required by the act. This report assesses how well selected agencies publicly reported on the quality of performance information used to measure progress on APGs. GAO selected six agencies–the Departments of Agriculture, Defense, Interior, and Labor, and NASA and DHS– based on GAO's 2013 federal managers survey on their agency's use of performance information. GAO reviewed information concerning these agencies' APGs published on Performance.gov and in their annual performance plans and reports.

What GAO Recommends

GAO recommends that all six of the agencies work with OMB to describe on Performance.gov how they are ensuring the quality of their APGs's performance information, and that all agencies, except for DHS, also describe this information in their annual performance plans and reports. GAO also recommends that OMB, working with the PIC, focus on ways the PIC's data quality working group can improve public reporting for APGs. OMB did not comment on the recommendations, but the six agencies generally concurred or identified actions they planned to take to implement them.

For more information, contact J. Christopher Mihm at (202) 512-6806 or mihmj@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In response to our recommendation, DHS added an overall description on Performance.gov of how the agency ensures reliable performance information is reported to external audiences. DHS's explanation refers to the data quality appendix in its performance plan and report covering fiscal years 2015-2017 and a hyperlink is provided from Performance.gov to the DHS website where this appendix can be found. Further, DHS also made it easier to locate information concerning its APGs in this data quality appendix by consolidating the relevant discussion in one section.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Homeland Security

  2. Status: Open

    Comments: We reviewed the information published on Performance.gov as of April 2016 for USDA's five APGs for fiscal years 2016-2017 and found that USDA had not added data quality explanation that addressed the five GPRAMA requirements. In May 2016, we requested additional information from a senior USDA official, but as of June 2016 have not received the requested information. We will continue to monitor USDA's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Agriculture

  3. Status: Open

    Comments: We reviewed the information published on Performance.gov as of April 2016 for DOD's five APGs for fiscal years 2016-2017. While we found some examples of DOD discussing data quality, such as challenges in improving the quality of the data used to measure progress on its Transition to Veterans APG, we did not find discussion clearly identified as addressing the five specific GPRAMA requirements for each APG on Performance.gov. However, our review of DOD's most recent performance report and discussions with agency officials indicate that DOD is making improvements to its internal processes that may help improve its public reporting on Performance.gov. We will continue to monitor DOD's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Defense

  4. Status: Open

    Comments: We reviewed the information published on Performance.gov as of April 2016 for Interior's eight APGs for fiscal years 2016-2017. For the performance measures used to measure progress on these APGs, Interior added an overall data quality statement that describes in general terms how the agency addresses the five GPRAMA requirements. However, repeating the same information across Performance.gov is not consistent with the GPRAMA requirement to report specific data quality information for each APG. For example, Interior does not provide more specific explanation of the data sources for different APGs and different performance measures. We will continue to monitor Interior's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of the Interior

  5. Status: Open

    Comments: We reviewed the information published on Performance.gov as of April 2016 for Labor's three APGs for fiscal years 2016-2017. Labor notes on Performance.gov that it is piloting performance data quality self-assessments with component agencies that support APGs. Labor officials reported in June 2016 that when the data quality self-assessments are fully implemented, Labor will discuss on Performance.gov the steps component agencies are taking to ensure data quality. We will continue to monitor Labor's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Labor

  6. Status: Closed - Implemented

    Comments: In response to our recommendation, NASA added data quality descriptions to Performance.gov for each of its four APGs for fiscal years 2016 and 2017 that explain how NASA tracks progress toward each APG, consistent with GPRAMA requirements.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: National Aeronautics and Space Administration

  7. Status: Open

    Comments: We reviewed USDA's FY 2015 Annual Performance Report/FY 2017 Annual Performance Plan, which contains a section describing the agency's five APGs for fiscal years 2016 and 2017. However, this section does not address the five GPRAMA data quality requirements. Instead, USDA refers readers to Performance.gov for explanation on data quality, but our review of the information published on Performance.gov as of April 2016 found that USDA had not added the required data quality discussion. In May 2016, we requested additional information from a senior USDA official, but as of June 2016 have not received the requested information. We will continue to monitor USDA's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Agriculture

  8. Status: Open

    Comments: We reviewed the FY 2015 Department of Defense Annual Performance Report. The report states that the agency's Performance Improvement Officer conducted a review of performance data verification and validation processes during fiscal year 2015 and refined the performance goal action plans to include a data verification and validation section for each performance measure. In addition, goal owners are expected to attest that their report is complete, accurate, and reliable. However, a DOD official in June 2016 confirmed that these are internal documents and the performance report indicates that external audiences must request this information from DOD. While the internal reviews DOD has been conducting have the potential to improve the quality of the agency's performance data, GPRAMA's transparency provisions require this information to be published in performance plans and reports. We will continue to monitor DOD's efforts to address this recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Defense

  9. Status: Open

    Comments: We reviewed the Department of the Interior's 2016/2017 Annual Performance Plan & 2015 Report. Interior added a data accuracy and reliability statement that describes in general terms how the agency addresses the five GPRAMA requirements. However, this does not provide more specific explanation for individual APGs. We will continue to monitor Interior's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of the Interior

  10. Status: Open

    Comments: We reviewed Labor's FY2015 Annual Performance Report (Labor officials said their performance report also serves as their performance plan). Labor reports that it is piloting performance data quality self-assessments with component agencies that support APGs. Labor officials reported in June 2016 that when the data quality self-assessments are fully implemented, Labor will discuss in the performance report the steps component agencies are taking to ensure data quality. We will continue to monitor Labor's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Labor

  11. Status: Closed - Implemented

    Comments: In response to our recommendation, NASA added a data quality explanation to the section of its FY2015 Annual Performance Report/FY2017 Annual Performance Plan that describes how NASA tracks progress toward each of its four APGs for fiscal years 2016 and 2017, consistent with GPRAMA requirements.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: National Aeronautics and Space Administration

  12. Status: Open

    Comments: We reviewed updates OMB published to Circular A-11 in July 2016. Circular A-11 continues to direct agencies to provide data quality information for publication on Performance.gov or to provide a hyperlink from Performance.gov to relevant explanation in agencies' performance reports, which was a requirement OMB added in June 2015 in response to our preliminary findings. However, our review found that OMB also needed to update the template agencies complete for Performance.gov updates to make it easier for agencies to publish this information on the website. The July 2016 update of Circular A-11 does not indicate whether this template has been updated or whether additional changes to A-11 are needed. We will continue to monitor OMB and the PIC's efforts to address our recommendation.

    Recommendation: To help participating agencies improve their public reporting, the Director of OMB, working with the PIC Executive Director, should identify additional changes that need to be made in OMB's guidance to agencies related to ensuring the quality of performance information for APGs on Performance.gov.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  13. Status: Closed - Implemented

    Comments: In addition to highlighting the need for better public reporting, our review also focused more attention on agencies' internal processes for verifying and validating performance information. In July 2015, we briefed the Performance Improvement Council's (PIC) Data Quality Cross-Agency Working Group on leading practices identified in our prior work for ensuring agencies are accurately verifying, validating, and reporting performance information, and shared GAO's performance and accountability report which describes our own data quality processes. In February 2016, the PIC Executive Director directed Performance Improvement Officers (PIOs) and their deputies to complete a self-assessment of their agency's data quality policies and procedures to ensure these were consistent with OMB guidance. PIC staff summarized the agencies' self-assessments at the March 2016 meeting of the PIOs for the Chief Financial Officers Act agencies. We reviewed presentation notes which confirmed that PIC staff had addressed our recommendation by summarizing for the PIOs the results of these self-assessments, identifying aspects of data quality in which agencies had generally rated their performance highest, and other aspects of data quality in which agencies had rated their performance lowest.

    Recommendation: To help participating agencies improve their public reporting, the Director of OMB, working with the PIC Executive Director, should identify practices participating agencies can use to improve their public reporting in their performance plans and reports of how they are ensuring the quality of performance information used to measure progress towards APGs.

    Agency Affected: Executive Office of the President: Office of Management and Budget

 

Explore the full database of GAO's Open Recommendations »

Dec 1, 2016

Nov 28, 2016

Oct 31, 2016

Oct 26, 2016

Oct 13, 2016

Oct 12, 2016

Sep 30, 2016

Sep 29, 2016

Sep 27, 2016

Looking for more? Browse all our products here