Defense Management:

Tools for Measuring and Managing Defense Agency Performance Could Be Strengthened

GAO-04-919: Published: Sep 13, 2004. Publicly Released: Sep 13, 2004.

Additional Materials:

Contact:

Sharon L. Pickup
(202) 512-9619
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

GAO was mandated to assess the effectiveness of defense agency performance contracts as management tools. As agreed, GAO also reviewed other tools (performance plans and balanced scorecards) and focused on three defense agencies--the Defense Logistics Agency (DLA), the Defense Information Systems Agency (DISA), and the Department of Defense Education Activity (DODEA). GAO addressed (1) the extent that the defense agencies initially used performance contracts, including whether this tool addressed attributes associated with results-oriented management; (2) defense agencies' efforts to implement performance plans using lessons learned from the initial contracts; and (3) the extent DOD established mechanisms to share lessons learned. GAO reviewed the content of these tools, but not the actual or reported performance. DISA has not yet finalized its scorecard, thus this report discusses only DISA's plans for its scorecard.

Since fiscal year 1998, the Department of Defense (DOD) has implemented various tools to help manage and oversee the performance of defense agencies. Between fiscal year 1999 and 2003, DLA, DISA, and DODEA initially used "performance contracts"--internal management agreements--to bring specific problems to the attention of senior DOD and agency leadership. While the contracts produced some useful information for decision makers, this tool would have been more effective for assessing performance, making resource allocation decisions, and taking corrective actions if DOD had required the agencies to include certain attributes associated with results-oriented management. Such attributes include aligning agency performance goals and measures with agency strategic plans and departmentwide goals; identifying individuals accountable for achieving results; providing a comprehensive view of organizational performance; linking resource needs to performance; discussing data quality; and providing contextual information, including external factors that affect reported performance. Beginning in fiscal year 2003, DOD renamed the performance contracts as "performance plans" and gave the defense agencies the option to use a "balanced scorecard" approach, a tool used in the public and private sectors to assess organizational performance. Based on experiences using the initial contracts, DOD took steps to strengthen performance plans and scorecards by revising the oversight and review process, requiring performance measures to align with agency and departmentwide goals, and requiring measures to provide a more comprehensive view of agency performance. DLA's scorecard, DODEA's performance plan, and DISA's plans for the agency's scorecard incorporated these changes and other attributes to varying degrees. While these tools have the potential to provide information useful to decision makers, they would be strengthened if DOD had required the agencies to include additional attributes such as designating specific individuals responsible for achieving results; identifying the relationship between resource needs and performance; reporting on data quality; and providing contextual information to allow top leaders to understand the extent of progress made, take corrective actions to achieve goals, and establish realistic performance goals for future years. With these attributes, decision makers would potentially gain additional insights into agency performance and areas needing greater management attention. DOD has developed mechanisms, such as a performance management Web site and roundtables, to help agencies share lessons learned from implementing performance plans and scorecards. In response to GAO's suggestions during this review, DOD recognized the need to continue to hold roundtables more frequently. DLA and DISA have also proactively shared their experiences with each other.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To make performance plans and scorecards more informative and useful for decision making, and further strengthen these tools' potential for measuring and managing defense agency performance, the Secretary of Defense should direct the Director, Program Analysis and Evaluation, to revise guidance to require all defense agencies to discuss data quality, including the reliability, validity, and limitations of performance measures as well as data sources.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Director of PA&E agreed that any Defense Agency that does not have a discussion of data quality (to include the reliability, validity, and limitation of performance measures as well as data sources) in its performance plans or balanced scorecards should do so in accordance with DOD guidance to the Defense Agencies dated August 22, 2003. The Director of PA&E issued a guidance memorandum, "Implementing Instructions for FY 2005 Defense Agency Performance Plans." dated October 25, 2004, that implemented this recommendation.

    Recommendation: To make performance plans and scorecards more informative and useful for decision making, and further strengthen these tools' potential for measuring and managing defense agency performance, the Secretary of Defense should direct the Director, Program Analysis and Evaluation, to revise guidance to require all defense agencies to identify resources needed to achieve performance goals and inform budget decisions.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Director of PA&E stated that given the complex nature of defense organizations, it might not be possible for Defense Agencies to fully cost their performance goals. The Director added that Defense Agency performance plans should be in synchronization with the Planning, Programming, Budgeting, and Execution System by demonstrating how PPBES decisions are implemented and by providing the performance information and metrics that can be used in the PPBES review process. The Director of PA&E issued a guidance memorandum, "Implementing Instructions for FY 2005 Defense Agency Performance Plans," dated October 25, 2004, that implemented this recommendation.

    Recommendation: To make performance plans and scorecards more informative and useful for decision making, and further strengthen these tools' potential for measuring and managing defense agency performance, the Secretary of Defense should direct the Director, Program Analysis and Evaluation, to revise guidance to require all defense agencies to include measures that are clearly defined and include trend data for at least the past fiscal year's performance to help assess progress.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Director of PA&E agreed that any Defense Agency that does not have clearly defined measures that include trend data in its performance plans or balanced scorecards should do so in accordance with DOD guidance to the Defense Agencies dated August 22, 2003. The Director of PA&E issued a guidance memorandum, "Implementing Instructions for FY 2005 Defense Agency Performance Plans," dated October 25, 2004, that implemented this recommendation.

    Recommendation: To make performance plans and scorecards more informative and useful for decision making, and further strengthen these tools' potential for measuring and managing defense agency performance, the Secretary of Defense should direct the Director, Program Analysis and Evaluation, to revise guidance to require all defense agencies to identify individuals accountable for achieving results at lower organizational levels.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Director of PA&E agreed that any Defense Agency not currently including in its performance plan the identification of individuals accountable for achieving results at lower organizational levels should do so in accordance with DOD guidance to the Defense Agencies dates August 22, 2005. The Director of PA&E subsequently issued a guidance memorandum, "Implementing Instructions for FY 2005 Defense Agency Performance Plans," dated October 25, 2004, that implemented this recommendation.

    Recommendation: To make performance plans and scorecards more informative and useful for decision making, and further strengthen these tools' potential for measuring and managing defense agency performance, the Secretary of Defense should direct the Director, Program Analysis and Evaluation, to revise guidance to require all defense agencies to provide contextual information to better understand how performance measures support the agency's mission.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Director of PA&E agreed that any Defense Agency that does not have contextual information that enables better understanding of how performance measures support the agency's mission in its performance plans or balanced scorecards should do so in accordance with DOD guidance to the Defense Agencies dated August 22, 2003. The Director of PA&E issued a guidance memorandum, "Implementing Instructions for FY 2005 Defense Agency Performance Plans," dated October 25, 2004, that implemented this recommendation.

    Apr 11, 2014

    Apr 10, 2014

    Apr 9, 2014

    Apr 8, 2014

    Apr 3, 2014

    Apr 2, 2014

    Apr 1, 2014

    Mar 31, 2014

    Looking for more? Browse all our products here