Managing for Results:

Greater Transparency Needed in Public Reporting on the Quality of Performance Information for Selected Agencies' Priority Goals

GAO-15-788: Published: Sep 10, 2015. Publicly Released: Sep 10, 2015.

Additional Materials:

Contact:

J. Christopher Mihm
(202) 512-6806
mihmj@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The six agencies GAO reviewed generally did not publicly report on how they ensured the accuracy and reliability of performance information used to measure progress on their highest priority performance goals, referred to as agency priority goals (APGs). The GPRA Modernization Act of 2010 (GPRAMA) requires agencies to identify the following when publicly reporting on their APGs: 1) how performance information was verified and validated; 2) data sources; 3) level of accuracy required for intended use; 4) any limitations at the required level of accuracy; and 5) how the agency will compensate for such limitations (if needed) to reach the required level of accuracy. GPRAMA requires agencies to provide this information to the Office of Management and Budget (OMB) for publication on Performance.gov. GPRAMA also directs agencies to provide this information for performance goals, which include APGs, in their annual performance plans and reports. While all six agencies described how they ensured the quality of their performance information overall, GAO found discussions about performance information quality addressing all five GPRAMA requirements in only the Department of Homeland Security's (DHS) performance plans and reports.

Agency Priority Goal (APG) Performance Information Quality Discussion in Selected Agencies' Performance Plans and Reports for FY13-FY16

Agency

Description of how agency ensured performance information quality overall

Number of APGs for FY14 and FY15

Description of how each APG met GPRAMA performance information quality requirements

Agriculture

Yes

3

0

Defense

Yes

4

0

Homeland Security

Yes

3

3

Interior

Yes

6

0

Labor

Yes

3

0

National Aeronautics and Space Administration (NASA)

Yes

4

0

Total

6 yes

23

3

Source: GAO analysis of selected agencies' performance plans and reports. |GAO-15-788

OMB and the Performance Improvement Council (PIC)–a cross-agency council of agency performance improvement officers–established the Data Quality Cross-Agency Working Group in February 2015. The group has identified several goals, such as improving the reliability and quality of performance information, and could serve as a vehicle for disseminating good practices in public reporting on data quality.

Why GAO Did This Study

Federal agencies have not always clearly and transparently explained to Congress and the public how they ensure the quality of their performance information. GPRAMA requires agencies to publicly explain how they ensure the accuracy and reliability of their performance information used to assess progress for their APGs.

This is one of a series of GAO reports examining the implementation of GPRAMA, as required by the act. This report assesses how well selected agencies publicly reported on the quality of performance information used to measure progress on APGs. GAO selected six agencies–the Departments of Agriculture, Defense, Interior, and Labor, and NASA and DHS– based on GAO's 2013 federal managers survey on their agency's use of performance information. GAO reviewed information concerning these agencies' APGs published on Performance.gov and in their annual performance plans and reports.

What GAO Recommends

GAO recommends that all six of the agencies work with OMB to describe on Performance.gov how they are ensuring the quality of their APGs's performance information, and that all agencies, except for DHS, also describe this information in their annual performance plans and reports. GAO also recommends that OMB, working with the PIC, focus on ways the PIC's data quality working group can improve public reporting for APGs. OMB did not comment on the recommendations, but the six agencies generally concurred or identified actions they planned to take to implement them.

For more information, contact J. Christopher Mihm at (202) 512-6806 or mihmj@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In response to our recommendation, DHS added an overall description on Performance.gov of how the agency ensures reliable performance information is reported to external audiences. DHS's explanation refers to the data quality appendix in its performance plan and report covering fiscal years 2015-2017 and a hyperlink is provided from Performance.gov to the DHS website where this appendix can be found. Further, DHS also made it easier to locate information concerning its APGs in this data quality appendix by consolidating the relevant discussion in one section.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Homeland Security

  2. Status: Open

    Comments: The Department of Agriculture (USDA) has identified two priority goals (APGs) for fiscal years 2018 and 2019. For the improving customer experience at USDA goal, USDA reports on Performance.gov that data sources and collection methodologies are being developed for the majority of the key performance indicators. For the modernize USDA's Information Technology goal, USDA provides no data quality explanation. In a November 2018 letter to the Comptroller General, USDA's Inspector General acknowledged that additional efforts are needed to better describe the quality of the data supporting the APGs and said that the Department will increase the amount of information provided in the quarterly APG updates. We will continue to monitor USDA's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Agriculture

  3. Status: Open

    Comments: The Department of Defense (DOD) partially concurred with the recommendation. DOD stated that it had ongoing actions to improve the quality of performance information. However, DOD stated that it did not believe our recommended action had any major management value. We disagreed. As of August 2018, DOD has identified three priority goals for fiscal years 2018 and 2019. DOD states that implementation action plans for these three goals will be available for the next quarterly update but provides no specific information regarding these three goals. In June 2018, we contacted DOD officials to request updated information, but as of August 30, 2018 have not received the requested information. We will continue to monitor DOD's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Defense

  4. Status: Open

    Comments: As of August 2018, the Department of the Interior (Interior) has identified six priority goals (APGs) for fiscal years 2018 and 2019. Interior provides specific data quality explanation addressing the five GPRA Modernization Act of 2010 (GPRAMA) requirements for three of these APGs: Improve Access to Leasing Public Lands for Oil, Gas, and Mineral Extraction, Improve Processing of Drilling Applications, and Water Conservation and Supply Enhancement. For two additional APGs-the Species Status Assessments for Listings with State Input and Improve Condition of Priority Assets Necessary for Mission Delivery Across the Department of the Interior-Interior partly addresses GPRAMA's requirements. In July 2018, we contacted Interior officials to request updated information, but as of August 22, 2018 have not received the requested information. We will continue to monitor Interior's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of the Interior

  5. Status: Open

    Comments: As of August 2018, Labor has identified five priority goals (APGs) for fiscal years 2018 and 2019. Labor addresses one of the five relevant requirements in the GPRA Modernization Act of 2010 by identifying data sources for all APGs. However, Labor did not address other requirements, such as potential limitations with the performance data. In August 2018, Labor officials reported to us that they plan to address the remaining four requirements in the fourth quarter updates for fiscal year 2018. We will continue to monitor Labor's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Labor

  6. Status: Closed - Implemented

    Comments: In response to our recommendation, NASA added data quality descriptions to Performance.gov for each of its four APGs for fiscal years 2016 and 2017 that explain how NASA tracks progress toward each APG, consistent with GPRAMA requirements.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: National Aeronautics and Space Administration

  7. Status: Open

    Comments: The Department of Agriculture's (USDA) Fiscal Year 2019 Annual Performance Plan and Fiscal Year 2017 Annual Performance Report continues to provide a general statement on how the agency uses a standardized methodology to measure its performance and that agency officials attest to the quality of the performance information. Further, USDA provides more specific data quality explanations for individual key performance measures discussed in the plan and report. However, USDA does not discuss its priority goals in the plan and report. Thus, it is not clear for external audiences the extent to which these more specific data quality explanations relate to the two priority goals identified on Performance.gov. In a November 2018 letter to the Comptroller General, USDA's Inspector General stated that the Department agrees with the recommendation and will begin providing data quality explanation for the APGs in its next annual performance plan and report to be published in February 2019. We will continue to monitor USDA's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Agriculture

  8. Status: Open

    Comments: The Department of Defense (DOD) partially concurred with the recommendation. DOD stated that it had ongoing actions to improve the quality of performance information. However, DOD stated that it did not believe our recommended action had any major management value. We disagreed. DOD's Fiscal Year 2017 Annual Performance Report continues to state that performance goal owners are expected to attest that their performance data is complete, accurate, and reliable. Further, DOD states that additional improvements will be addressed to ensure the reliability of performance information as the agency strengthens its Enterprise Performance capability. While the performance report asserts that verification and validation procedures have been documented and are available upon request, the GPRA Modernization Act of 2010 requires that agencies make this information accessible to external audiences by publishing an explanation in the agency's annual performance plans and reports. Further, DOD's performance report does not discuss priority goals for fiscal years 2018 and 2019. In June 2018, we contacted DOD officials to request updated information, but as of August 30, 2018 have not received the requested information. We will continue to monitor DOD's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Defense

  9. Status: Open

    Comments: The Department of the Interior's (Interior) 2018/2019 Annual Performance Plan & 2017 Report (APP&R) includes a section concerning data accuracy and reliability and describes in general terms how Interior ensures the accuracy and reliability of performance information and how it addresses the five data quality requirements in the GPRA Modernization Act of 2010 (GPRAMA). Interior states in this section that measurement procedures for agency performance goals are described on Performance.gov. As of July 2018, our review found that Interior has provided specific data quality explanation addressing the five GPRAMA requirements for three of its six APGs on Performance.gov. In July 2018, we contacted Interior officials to request updated information, but as of August 22, 2018 have not received the requested information. We will continue to monitor Interior's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of the Interior

  10. Status: Open

    Comments: The Department of Labor's Fiscal Year 2017 Annual Performance Report, which also serves as the performance plan for fiscal year 2019, identifies the agency's priority goals as part of broader discussion of the agency's strategic goals and objectives. However, there is no discussion of the five data quality requirements spelled out in the GPRA Modernization Act of 2010. In August 2018, Labor officials reported to us that they plan to expand the data quality explanation they provide on Performance.gov to address all five requirements for their APGs. Providing more comprehensive data quality explanation on Performance.gov should also help Labor address the requirements in its performance report by allowing Labor to refer readers interested in data quality explanation to Performance.gov. We will continue to monitor Labor's efforts to address our recommendation.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: Department of Labor

  11. Status: Closed - Implemented

    Comments: In response to our recommendation, NASA added a data quality explanation to the section of its FY2015 Annual Performance Report/FY2017 Annual Performance Plan that describes how NASA tracks progress toward each of its four APGs for fiscal years 2016 and 2017, consistent with GPRAMA requirements.

    Recommendation: To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs.

    Agency Affected: National Aeronautics and Space Administration

  12. Status: Closed - Implemented

    Comments: In July 2018, OMB staff shared with us a reporting template for Performance.gov for the fiscal years 2018 and 2019 APGs. This template allows agencies to provide information on data accuracy and reliability for individual APGs and agencies may provide hyperlinks to more detailed data quality discussions published in their annual performance plans and reports.

    Recommendation: To help participating agencies improve their public reporting, the Director of OMB, working with the PIC Executive Director, should identify additional changes that need to be made in OMB's guidance to agencies related to ensuring the quality of performance information for APGs on Performance.gov.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  13. Status: Closed - Implemented

    Comments: In addition to highlighting the need for better public reporting, our review also focused more attention on agencies' internal processes for verifying and validating performance information. In July 2015, we briefed the Performance Improvement Council's (PIC) Data Quality Cross-Agency Working Group on leading practices identified in our prior work for ensuring agencies are accurately verifying, validating, and reporting performance information, and shared GAO's performance and accountability report which describes our own data quality processes. In February 2016, the PIC Executive Director directed Performance Improvement Officers (PIOs) and their deputies to complete a self-assessment of their agency's data quality policies and procedures to ensure these were consistent with OMB guidance. PIC staff summarized the agencies' self-assessments at the March 2016 meeting of the PIOs for the Chief Financial Officers Act agencies. We reviewed presentation notes which confirmed that PIC staff had addressed our recommendation by summarizing for the PIOs the results of these self-assessments, identifying aspects of data quality in which agencies had generally rated their performance highest, and other aspects of data quality in which agencies had rated their performance lowest.

    Recommendation: To help participating agencies improve their public reporting, the Director of OMB, working with the PIC Executive Director, should identify practices participating agencies can use to improve their public reporting in their performance plans and reports of how they are ensuring the quality of performance information used to measure progress towards APGs.

    Agency Affected: Executive Office of the President: Office of Management and Budget

 

Explore the full database of GAO's Open Recommendations »

May 21, 2019

May 6, 2019

Apr 30, 2019

Apr 17, 2019

Apr 10, 2019

Looking for more? Browse all our products here