This is the accessible text file for GAO report number GAO-06-13 
entitled 'Defense Management: Additional Actions Needed to Enhance 
DOD's Risk-Based Approach for Making Resource Decisions' which was 
released on November 16, 2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Subcommittee on Readiness and Management Support, 
Committee on Armed Services, U.S. Senate: 

United States Government Accountability Office: 

GAO: 

November 2005: 

Defense Management: 

Additional Actions Needed to Enhance DOD's Risk-Based Approach for 
Making Resource Decisions: 

GAO-06-13: 

GAO Highlights: 

Highlights of GAO-06-13, a report to the Subcommittee on Readiness and 
Management Support, Committee on Armed Services, U.S. Senate: 

Why GAO Did This Study: 

The Department of Defense (DOD) is simultaneously conducting costly 
military operations and transforming its forces and business practices 
while it is also competing for resources in an increasingly constrained 
fiscal environment. As a result, GAO has advocated that DOD adopt a 
comprehensive threat or risk management approach as a framework for 
decision making. In its 2001 strategic plan, the Quadrennial Defense 
Review (QDR), DOD stated its intent to establish an approach—the risk 
management framework—to balance priorities against risk over time and 
monitor results against its strategic goals. 

GAO was asked to (1) assess the extent to which DOD has implemented the 
framework, including using it to make investment decisions, and (2) 
identify the most significant challenges DOD faces in implementing the 
framework, or a similar approach. 

What GAO Found: 

DOD has taken some positive steps to implement the framework, but 
additional actions are needed before DOD can show real and sustainable 
progress in using a risk-based and results-oriented approach to 
strategically allocate resources across the spectrum of its investment 
priorities. For example, DOD defined four risk areas, and developed 
performance goals and department-level measures, but it needs to, among 
other things, further develop and refine the measures so that they 
clearly demonstrate results and provide a well-rounded depiction of 
departmental performance. DOD’s current strategic plan and goals also 
are not clearly linked to the framework’s performance goals and 
measures, and linkages between the framework and budget are also 
unclear. While DOD officials stated that risk was considered during the 
fiscal year 2006 budget cycle, DOD’s budget submission does not 
specifically discuss how DOD identified or assessed risks to establish 
DOD-wide investment priorities. Without better measures, clear 
linkages, and greater transparency, DOD will be unable to fully measure 
progress in achieving strategic goals or demonstrate to Congress and 
others how it considered risks, and made trade-off decisions, balancing 
needs and costs for weapon programs and other investment priorities. 

DOD’s Risk Management Framework: 

Force Management Risk; 
Definition: Challenge of sustaining personnel, infrastructure, and 
equipment. 

Operational Risk; 
Definition: Challenge of deterring or defeating near-term threats. 

Future Challenges Risk; 
Definition: Challenge of dissuading, deterring, defeating longer-term 
threats. 

Institutional Risk; 
Definition: Challenge of improving efficiency (includes financial 
management). 

Source: DOD. 

[End of table] 

DOD faces four challenges that have affected the implementation of the 
framework. First, DOD’s organizational culture resists department-level 
approaches to priority setting and investment decisions. Second, 
sustained leadership, adequate transparency, and appropriate 
accountability are lacking. Further, no one individual or office has 
been assigned overall responsibility or sufficient authority for the 
framework’s implementation. DOD also has not developed implementation 
goals or timelines with which to establish accountability, or measure 
progress. Finally, integrating the risk management framework with 
decision support processes and related reform initiatives into a 
coherent, unified management approach for the department is a challenge 
that DOD plans to address during the 2005 QDR. However, GAO has 
concerns about DOD’s ability to follow through on this integration, 
because of its limited success in implementing other management 
reforms. Unless DOD successfully addresses these challenges and 
effectively implements the framework, or a similar approach, it will 
likely continue to experience (1) a mismatch between programs and 
budgets, and (2) a proportional, rather than strategic, allocation of 
resources to the services. 

What GAO Recommends: 

GAO recommends that DOD take various actions to increase its chances of 
successfully implementing a risk-based approach for investment decision 
making, such as developing results-oriented measures and assigning 
clear leadership with appropriate accountability and authority to 
implement the framework. DOD partially concurred with our 
recommendations. 

www.gao.gov/cgi-bin/getrpt?GAO-06-13. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Sharon Pickup at (202) 
512-9619 or pickups@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Despite Positive Steps, Additional Actions Needed to Fully Implement 
the Risk Management Framework: 

Cultural Resistance, Combined with the Lack of Leadership, 
Implementation Goals, and Process Integration, Affects DOD's 
Implementation of the Risk Management Framework: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Defense: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Definitions and Examples of DOD Department-Level Measures (as 
of November 2004): 

Table 2: The Number of Activity and Performance Measures for Each 
Quadrant: 

Table 3: Military Service and Defense-Wide Percentage of the 2005 and 
2006 Future Years Defense Programs: 

Table 4: Select Initiatives to Improve Investment Decision Making: 

Figures: 

Figure 1: The Risk Management Cycle: 

Figure 2: Comparison of the Balanced Scorecard and the Risk Management 
Framework: 

Abbreviations: 

CBO: Congressional Budget Office: 

CMO: chief management official: 

DOD: Department of Defense: 

FYDP: Future Years Defense Program: 

GAO: Government Accountability Office: 

GPRA: Government Performance and Results Act: 

JCIDS: Joint Capabilities Integration and Development System: 

OSD: Office of the Secretary of Defense: 

PA&E: Program Analysis and Evaluation (PA&E): 

PPBE: Planning, Programming, Budgeting, and Execution: 

P&R: Personnel and Readiness: 

PART: Program Assessment Rating Tool: 

QDR: Quadrennial Defense Review: 

United States Government Accountability Office: 

Washington, DC 20548: 

November 15, 2005: 

The Honorable John Ensign: 
Chairman: 
The Honorable Daniel K. Akaka 
Ranking Minority Member: 
Subcommittee on Readiness and Management Support: 
Committee on Armed Services: 
United States Senate: 

Among the 21st century challenges facing the Department of Defense 
(DOD) and the nation as a whole are difficult decisions concerning how 
to strike an affordable balance between current and future national 
security needs and between national security and domestic 
needs.[Footnote 1] For example, DOD is simultaneously maintaining a 
high pace of military operations for combating terrorism and 
transforming its military forces and business operations for the 21st 
century while it is also competing for federal resources in an 
increasingly fiscally constrained environment. We have advocated that 
DOD--as well as the rest of the federal government--adopt a 
comprehensive threat or risk management approach as a framework for 
decision making.[Footnote 2] This approach would fully link strategic 
goals to plans and budgets; assess the values and risks of various 
courses of actions as a tool for reexamining defense programs, setting 
priorities, and allocating resources; and use performance measures to 
assess outcomes. 

To its credit, DOD introduced a balanced scorecard for risk management, 
commonly known as the risk management framework, in its strategic plan, 
the 2001 Quadrennial Defense Review (QDR) report. The 2001 strategic 
plan articulated the new administration's emphasis on transforming 
military forces and defense business practices to meet the emerging 
challenges facing our nation. DOD intended the framework to be used as 
a management tool to focus DOD's efforts on implementing the defense 
program as outlined in the strategic plan. In particular, DOD's senior 
leadership intended the risk management framework to assist decision 
makers in formulating top-down strategy, balancing investment 
priorities against risk over time, measuring near-and midterm outputs 
against strategic goals, and focusing on actual performance results. 
According to DOD officials, the risk management framework also was 
intended to increase transparency within the department over the 
decision-making process. During the ongoing 2005 QDR, DOD plans to 
refine the risk management framework. 

You asked us to examine the status of DOD's efforts to adopt a risk- 
based approach to decision making, given the emphasis that DOD was 
placing on the risk management framework. In response, we (1) assessed 
the extent to which DOD has implemented its risk management framework, 
including the extent to which DOD has used the framework to make 
investment decisions; and (2) identified the most significant 
challenges DOD faced in implementing the risk management framework or a 
similar risk-based and results-oriented management approach. 

To assess the extent to which DOD has implemented the risk management 
framework, we analyzed key documents, policy guidance, data, and 
interview results, and compared the analysis to the principles for 
managing risk and results identified in prior GAO reports. In addition, 
we conducted interviews with DOD and service officials, and members of 
the Joint Staff. We discussed the department's progress in implementing 
the risk management framework with members of the Defense Business 
Board. We also analyzed DOD's department-level performance goals and 
measures that are associated with the risk management framework and 
assessed how DOD reported that information externally. We did not 
validate the appropriateness of the risk management framework's risk 
quadrants or the procedures that DOD has in place to ascertain the 
reliability of performance data and we also did not assess the basis 
for DOD's investment decisions. To identify the most significant 
challenges DOD faced in implementing the risk management framework, we 
analyzed documents, data, and interview results, and compared the 
results of this analysis to the key practices to assist mergers and 
organizational transformation identified in prior GAO reports.[Footnote 
3] A more detailed discussion of our scope and methodology is presented 
in appendix I. 

Our work was performed from October 2004 through September 2005 in 
accordance with generally accepted government auditing standards. 

Results in Brief: 

DOD has taken positive steps toward implementing the risk management 
framework; however, additional actions are needed before the framework 
is fully implemented and DOD can demonstrate real and sustainable 
progress in using a risk-based and results-oriented approach to 
strategically allocate resources across the spectrum of its investment 
priorities. For example, while DOD established four risk areas, or 
quadrants, and developed performance goals and measures of two types-- 
activity measures (measures to track initiatives) and performance 
measures--the majority of these measures do not provide sufficient 
information to monitor performance against the risk quadrants' goals. 
Specifically, and contrary to results-oriented management principles, 
the risk management framework's measures (1) do not clearly demonstrate 
results, (2) do not provide a well-rounded depiction of performance 
across the department, and (3) are not being systemically monitored 
across all quadrants, except for the force management quadrant. In 
addition, the framework's performance goals and measures are not 
clearly linked to DOD's current strategic plan and strategic goals. 
Lacking measures that follow results-oriented management principles and 
clear linkages to strategic goals, DOD may be unable to provide a clear 
roadmap of how its activities at all levels contribute to meeting DOD's 
strategic goals. Finally, although DOD officials stated that risk was 
considered in the fiscal year 2006 budget cycle, the fiscal year 2006 
budget submission does not include any specific information on how DOD 
systematically identified or assessed departmental risks to establish 
DOD-wide investment priorities. Therefore, the linkages between the 
risk management framework and the budget are unclear. Without better 
measures, clear linkages, and greater transparency, DOD will be unable 
to fully measure progress in achieving strategic goals or demonstrate 
to Congress and others how it considered risks and made trade-off 
decisions, balancing needs and costs for weapon system programs and 
other investment priorities. 

DOD faces four key challenges that affect its ability to fully 
implement the risk management framework, or a similar risk-based and 
results-oriented management approach: (1) overcoming cultural 
resistance to the transformational change represented by such an 
approach in a department as massive, complex, and decentralized as DOD; 
(2) maintaining sustained leadership and clear accountability for this 
cultural transformation; (3) providing implementation goals and 
timelines to gauge progress in transforming the culture; and (4) 
integrating the risk management framework with decision support 
processes and related reform initiatives into a coherent, unified 
management approach for the department. Our prior work on results- 
oriented management and organizational transformation and mergers has 
shown that addressing these challenges is at the center of successful 
change management efforts in leading organizations. DOD is having 
difficulties implementing the framework because it has not addressed 
these four challenges. With respect to the first challenge, DOD's size 
and complexity result in a culture that makes developing department- 
level approaches to priority setting and investment decision making 
difficult. For example, the allocation of budgets on a proportional, 
rather than a strategic, basis among the services is a long-standing 
budgetary problem that we have reported about for years. Second, the 
lack of sustained leadership and clear accountability for the 
framework's implementation has resulted in a lack of emphasis and 
understanding of its status and purpose within the department. Because 
of the lack of sustained leadership for other management reform 
efforts, we have supported legislation to create a chief management 
official (CMO) at DOD to provide this leadership.[Footnote 4] Third, 
DOD did not establish implementation goals or timelines with which to 
establish accountability, measure progress, and build momentum. 
Finally, integrating the risk management framework with other decision 
support processes and related reform initiatives into a coherent, 
unified management approach is a challenge that DOD intends to address 
in the ongoing 2005 QDR. Illustrating this challenge, DOD is attempting 
to implement the risk management framework while it is also shifting to 
biennial budgeting and reforming defense planning. Our work has shown 
that if risk-based and results-oriented management approaches are to be 
successfully implemented, they must be integrated into the usual cycle 
of agency decision making. Unless DOD addresses these challenges and 
successfully implements the risk management framework, or a similar 
approach, it may continue to experience (1) a mismatch between programs 
and budgets, and (2) the proportional, rather than strategic, 
allocation of resources to the services. Therefore, Congress may have 
insufficient transparency into how DOD has identified and assessed 
risks and made trade-offs in its investment decision making. 

In this report, we recommend that DOD take various actions to increase 
its chances of successfully implementing a risk-based approach for 
investment decision making. This includes developing results-oriented 
measures and assigning clear leadership with appropriate accountability 
and authority to implement and sustain the risk management framework, 
or a similar approach. In written comments on a draft of this report, 
DOD partially concurred with our recommendations. DOD's comments and 
our evaluation of them are on page 25 of this report. 

Background: 

In our report, High-Risk Series: An Update,[Footnote 5] we identified 
agencies' lack of comprehensive risk management strategies as an 
emerging challenge for the federal government. Increasingly limited 
fiscal resources across the federal government, coupled with the 
emerging requirements from the changing security environment, emphasize 
the need for DOD to develop a risk-based strategic investment approach. 
For this reason, we have advocated that DOD adopt a comprehensive risk 
management approach for decision making.[Footnote 6] Furthermore, DOD 
and other federal agencies are required by statute to develop a results-
oriented management approach to strategically allocate resources on the 
basis of performance.[Footnote 7] The balanced scorecard--a concept to 
balance an organization's focus across financial, customer, internal 
business, and learning and growth management areas--is one approach for 
developing results-oriented management that government agencies have 
recently started to adopt.[Footnote 8] At the direction of the 
Secretary of Defense, DOD developed a risk management framework that 
DOD later aligned with its results-oriented management activities 
through a DOD balanced scorecard. 

Risk Management Is an Emerging 21st Century Challenge: 

An emerging challenge for the federal government involves the need for 
the completion of comprehensive national threat and risk assessments in 
a variety of areas. For example, emerging requirements from the 
changing security environment, coupled with increasingly limited fiscal 
resources across the federal government, emphasize the need for 
agencies to adopt a sound approach to establishing resource 
decisions.[Footnote 9] We have advocated that the federal government, 
including DOD, adopt a comprehensive threat or risk management approach 
as a framework for decision making that fully links strategic goals to 
plans and budgets, assesses values and risks of various courses of 
actions as a tool for setting priorities and allocating resources, and 
provides for the use of performance measures to assess outcomes. Based 
on our review of the literature,[Footnote 10] as shown in figure 1, the 
goal of risk management is to integrate systematic concern for risk 
into the usual cycle of agency decision making and implementation. 

Figure 1: The Risk Management Cycle: 

[See PDF for image] 

[End of figure] 

A risk management cycle represents a series of analytical and 
managerial steps, basically sequential, that can be used to assess 
risk, evaluate alternatives for reducing risks, choose among those 
alternatives, implement the alternatives, monitor their implementation, 
and continually use new information to adjust and revise the 
assessments and actions, as needed. Adoption of a risk management cycle 
such as this can aid in assessing risk by determining which 
vulnerabilities should be addressed, and how they should be addressed, 
within available resources. For the purposes of this report, we focused 
on the stages of the risk management cycle that involve DOD's actions 
to set strategic goals and objectives, establish investment priorities 
based on risk assessments, and implementation and monitoring. 

Risk management's objectives are essentially the same as those of good 
management, and they are consistent with the broad economy and 
efficiency objectives of good government--namely, to provide better 
outcomes for the same amount of money, or to provide the same outcomes 
with less money. Therefore, risk management's objectives are also 
compatible with those of the federal government's results-oriented 
management approach, which was enacted in the Government Performance 
and Results Act (GPRA) of 1993,[Footnote 11] and the balanced scorecard 
approach. Congress enacted GPRA to focus the federal government on 
achieving results through the creation of clear links between the 
process of allocating scarce resources and an agency's strategic goals, 
or the expected results to be achieved with those resources. Building 
on GPRA's foundation, the current administration has taken steps to 
strengthen the integration of budget, cost, and performance information 
by including budget and performance integration as one of its 
management initiatives under the umbrella of the President's Management 
Agenda.[Footnote 12] The Budget and Performance Integration initiative 
includes efforts such as the Program Assessment Rating Tool (PART), 
improving outcome measures, and improving monitoring of program 
performance.[Footnote 13] The balanced scorecard approach is a 
management tool that some federal agencies have adopted to help them 
translate the strategy set forth in a results-oriented management 
approach into the operational objectives that drive both behavior and 
performance. The balanced scorecard consists of four management areas 
that organizations should focus on--financial, customer, internal 
business, and learning and growth. 

DOD's 2001 Strategic Plan Outlines a New Risk Management Framework: 

DOD introduced the risk management framework in its strategic plan, the 
2001 QDR report. The 2001 strategic plan articulated the new 
administration's emphasis on transforming military forces and defense 
business practices to meet the changing threats facing our nation. In 
his guidance to the department for the 2001 QDR strategic planning 
process, the Secretary of Defense stated the need for DOD to use a risk 
mitigation approach for balancing force, resource, and modernization 
requirements across defense planning timelines. This guidance also 
stated that DOD must include the identification of output-based 
measures to reduce inefficiencies through the department in any 
approach to risk management. Building on the guidance, the 2001 QDR 
outlined DOD's risk management framework. According to the QDR, the 
framework would enable DOD to address the tension between preparing for 
future threats and meeting the demands of the present with finite 
resources. It was also intended to ensure that DOD was sized, shaped, 
postured, committed, and managed with a view toward accomplishing the 
strategic plan's defense policy goals. 

DOD adapted the balanced scorecard concept to the risk management 
framework by substituting the four dimensions of risk--force 
management, operational, future challenges, and institutional--for the 
scorecard's four management areas. The risk management framework was to 
be a transformational tool that would provide a balanced perspective of 
the organization's execution of strategy and ensure a top-down 
approach. The 2002 policy guidance also designated four preliminary 
performance goals for each of the four risk quadrants. In addition, the 
guidance required that performance goals and measures were to be 
cascaded to the services and defense agencies. Figure 2 shows a 
comparison, as provided by DOD. 

Figure 2: Comparison of the Balanced Scorecard and the Risk Management 
Framework: 

[See PDF for image] 

[End of figure] 

Despite Positive Steps, Additional Actions Needed to Fully Implement 
the Risk Management Framework: 

Despite positive steps, DOD needs to take additional actions before the 
risk management framework is fully implemented and DOD can demonstrate 
real and sustainable progress in using a risk-based and results- 
oriented approach to strategically allocate resources across the 
spectrum of its investment priorities. For example, DOD is still in the 
process of developing department-level measures for the framework that 
address results-based management principles, such as linking 
performance information to strategic goals so that this information can 
be used to monitor performance results and determine how well the 
department is doing in achieving its strategy. Without more results- 
oriented performance measures, DOD may be unable to provide the 
services and other defense components with clear roadmaps of how their 
activities contribute to meeting DOD's strategic goals. In addition, 
the framework's performance goals and measures are not clearly linked 
to DOD's current strategic plan and strategic goals. Furthermore, the 
extent to which the risk management framework is linked to the budget 
cycle is unclear. Without better measures, clear linkages, and greater 
transparency, DOD will be unable to fully measure progress in achieving 
strategic goals or demonstrate to Congress and others how it considered 
risks and made trade-offs in making investment decisions. 

Developing a Set of Measures That Can Be Used to Monitor Performance Is 
a Work in Progress: 

DOD has taken positive steps toward developing measures for each of the 
performance goals under the framework's four risk quadrants; however, 
developing a set of measures that can be used to monitor performance 
results is still a work in progress. Based on GAO's prior work on 
results-based management principles, we found that leading 
organizations' performance measures are: (1) designed to demonstrate 
results, or provide information on how well the organization is 
achieving its goals; (2) limited to a vital few, and balanced across 
priorities; and (3) used by management to improve performance.[Footnote 
14] However, the set of measures DOD has developed for the risk 
management framework do not adequately address these principles. While 
DOD established four risk quadrants and developed performance goals and 
measures of two types--activity measures (measures to track 
initiatives) and performance measures--the majority of its measures do 
not provide sufficient information to monitor performance against the 
risk quadrants' goals. 

First, DOD officials acknowledge that establishing department-level 
measures for the framework that demonstrate results is still a work in 
progress, as the majority of the risk management framework's measures 
require further development or refinement. In fact, as shown in table 
1, 44 of the 77 department-level measures for all four quadrants, or 
over 50 percent, are activity measures. According to DOD sources, 
activity measures are to result in a new performance measure, a new 
baseline or benchmark, or define a new capability, rather than monitor 
a specific annual performance target. Once these activities are 
completed, DOD officials stated that the department will be better able 
to monitor department-level performance against strategic goals. 
However, our analysis found that the activity measures, as defined in 
DOD's external reports, typically do not provide sufficient information 
to monitor the department's progress in achieving the stated goal they 
are to measure, such as developing a new performance measure or 
baseline. The desired outcomes for activity measures generally state 
that a task was or will be completed by a certain date but they do not 
provide sufficient information on whether the activity is on schedule, 
the interdependencies among tasks, or the contribution toward enhancing 
the department's performance. Therefore, Congress and other external 
stakeholders lack information and adequate assurances that DOD is 
making progress in implementing a risk-based and results-oriented 
management approach to making investment decisions. 

Table 1: Definitions and Examples of DOD Department-Level Measures (as 
of November 2004): 

Type: Activity measures; 
Number: 44[A]; 
Definition: Activity measures track developmental activities, are 
usually qualitative, and track key milestones or events in lieu of a 
specific annual performance target; 
Examples: Deny enemy advantages and exploit weaknesses; 
Description of desired outcome monitored by measure: Roadmap will be 
complete by the end of fiscal year 2005. 

Examples: Enhance homeland defense and consequence management; 
Description of desired outcome monitored by measure: Strategy will be 
complete by the first quarter of fiscal year 2005. 

Type: Performance measures; 
Number: 33[A]; 
Definition: Performance measures track current outputs and set 
quantitative annual targets for performance that are measurable; 
Examples: Reserve component enlisted recruiting quality; 
Description of desired outcome monitored by measure: Target > 90% of 
recruits holding high school diplomas; Actual 88% of recruits holding 
high school diplomas. 

Examples: Reduce customer wait time (in days); 
Description of desired outcome monitored by measure: Target 15 days 
from order to receipt for material goods; Actual 24 days from order to 
receipt for material goods. 

Source: GAO analysis of the Risk Management Framework's performance 
measures. 

[A] We have recoded five performance measures as activity measures as 
these measures tracked milestones and events, which corresponds with 
DOD's definition of an activity measure. 

[End of table] 

Second, DOD's department-level performance measures are still a work in 
progress in that these measures do not provide a well-rounded depiction 
of DOD's performance. In our previous work, we have found that 
performance measurement efforts that are not balanced across priorities 
may skew an agency's performance and keep its senior leadership from 
seeing the whole picture.[Footnote 15] For example, in developing 
department-level measures for the risk management framework, DOD 
appears to have overemphasized its force management priorities at the 
expense of operational risk. As illustrated in table 2, the operational 
risk quadrant has no performance measures, while the force management 
risk quadrant has a total of 36 measures, including 15 activity 
measures and 21 performance measures. 

Table 2: The Number of Activity and Performance Measures for Each 
Quadrant: 

Activity measures: Operational: 15; 
Performance measures: Operational: 21; 
Total measures: Operational: 36[A]. 

Force Management: Operational; 
Activity measures: 9; 
Performance measures: 0; 
Total measures: 9. 

Activity measures: 11; 
Performance measures: 10; 
Total measures: 21[B]. 

Force Management; 
Activity measures: 9; 
Performance measures: 2; 
Total measures: 11[C]. 

Source: GAO analysis of DOD data. 

[A] We have recoded two performance measures as activity measures as 
these measures tracked milestones and events, which corresponds with 
DOD's definition of an activity measure. 

[B] We have recoded one performance measure as an activity measure as 
this measure tracked milestones and events, which corresponds with 
DOD's definition of an activity measure. 

[C] We have recoded two performance measures as activity measures as 
these measures tracked milestones and events, which corresponds with 
DOD's definition of an activity measure. 

[End of table] 

In providing technical comments to a draft of this report, DOD objected 
to our recoding of five department-level performance measures as 
activity measures. We recoded these measures because they tracked 
milestones and events, which corresponded to DOD's definition of an 
activity measure. The measures we recoded addressed the following: 

* a civilian human resources strategic plan, 

* a military human resources strategic plan, 

* monitor the status of defense technology objectives, 

* strategic transformation appraisal, and: 

* support acquisition excellence goals. 

Finally, DOD officials indicated that DOD is systematically using 
performance measures to monitor progress and improve performance for 
only one risk quadrant, although individual measures under the other 
three risk quadrants may be monitored. We have found that leading 
organizations use performance information to improve organizational 
performance and identify performance gaps, and to provide incentives 
that reinforce a results-oriented management approach.[Footnote 16] 
According to DOD officials, the force management quadrant is the only 
quadrant that is managed by one individual and one office--the Under 
Secretary of Defense for Personnel and Readiness and his office. These 
officials stated that this situation is a critical factor in the 
progress DOD has made in systematically monitoring performance across 
the force management quadrant on a routine basis. For example, 
officials stated that the Under Secretary of Defense personally leads 
quarterly monitoring sessions on the force management quadrant's 
performance. DOD officials also told us that the Under Secretary of 
Defense for Personnel and Readiness has greatly facilitated this 
monitoring by developing a centralized database to capture the 
performance data used to track DOD's performance in meeting the 
quadrant's goals. Unless all of the risk management framework's 
quadrants are systematically monitored, implementation of the framework 
may be hindered and the framework risks becoming a paper-driven, 
compliance exercise. Indeed, one DOD official told us that he views the 
risk management framework and its measures as a "reporting drill" and, 
in addition, his office would not change its processes if DOD was to no 
longer use the framework. 

Cascading the Risk Management Framework's Goals and Measures Is an 
Ongoing Effort: 

DOD is still in the process of cascading the risk management 
framework's goals and measures to the services. We have found that 
leading organizations seek to establish clear hierarchies of goals and 
measures that cascade down so that subordinate units have 
straightforward roadmaps to demonstrate how their activities contribute 
to meeting the organization's strategy.[Footnote 17] According to DOD 
officials, all of the services are attempting to align their existing 
performance measures with the department-level performance goals and 
measures. However, service officials said that it is challenging to 
cascade the department-level activity measures, because these measures 
represent very broad initiatives that may not be applicable at all DOD 
levels. Officials from one service said they have had to develop new 
measures to align with the department-level measures, because they had 
been assessing performance with fewer measures than the Office of the 
Secretary of Defense had developed. 

Developing a Strategic Plan with Clear Linkages between the Risk 
Management Framework and Strategic Goals Is a Critical Next Step: 

The risk management framework's performance goals and measures are not 
clearly linked--a key principle of results-oriented management--to a 
coherent strategic plan.[Footnote 18] The development of such a 
strategic plan is a critical next step in using a risk-based and 
results-oriented approach to making investment decisions. Without these 
linkages, DOD cannot easily demonstrate how achievement of a 
performance goal or measure contributes to the achievement of strategic 
goals and ultimately the organization's mission. Our previous work 
indicated that DOD's strategic plan, the 2001 QDR, did not provide a 
sound foundation for the risk management framework.[Footnote 19] We 
reported that the usefulness of the 2001 QDR was limited by the lack of 
focus on longer-term threats and requirements for critical support 
capabilities, and provided few insights into how future threats and 
planned technical advances could affect future force requirements. In 
turn, this lack of focus and insight limited the QDR's usefulness as a 
foundation for fundamentally reassessing U.S. defense plans and 
programs and for balancing resources across near-and midterm risks. 

DOD officials indicated that DOD has not yet defined the linkages 
between the risk management framework's performance goals and the 
strategic goals in the 2001 QDR. Furthermore, the Defense Business 
Board's official minutes for its July 28, 2005, meeting contained a 
recommendation that the Secretary of Defense define department-level 
objectives, which should then be cascaded down the department.[Footnote 
20] In discussing the ongoing 2005 QDR, DOD stated that although the 
department would continue its efforts to do so, establishing these 
linkages was very challenging because of the size and scope of DOD's 
operations. However, as suggested by the Defense Business Board and our 
previous work, if DOD's strategic plan is to drive the department's 
operations, a straightforward linkage is needed among strategic goals, 
annual performance goals, and day-to-day activities.[Footnote 21] The 
ongoing 2005 QDR offers DOD the opportunity to strengthen its strategic 
planning. 

Although Risk Considered, Linkages Between the Risk Management 
Framework and Budget Are Unclear: 

According to DOD officials, the department has begun to consider risk 
in its investment decision making; however, the full extent to which 
the framework's risk-based and results-oriented approach has been 
linked to the fiscal year 2006 budget cycle is unclear. Our work 
indicates that leading organizations link strategy to the budget 
process through results-oriented management to evaluate potential 
investments or initiatives.[Footnote 22] 

DOD sources indicated that the department has begun to consider risk 
during its usual cycle of investment decision making. For example, 
according to DOD sources, the Secretary of Defense articulated broad 
areas for increasing or decreasing risk under each quadrant in the 
fiscal years 2006-2011 planning guidance, leaving it up to the defense 
components to decide how to structure their investment decisions within 
those broad areas consistent with the Secretary's risk guidance. In 
addition, DOD officials stated that the framework has increased 
awareness within the department on the need to balance risk over time. 
For example, when DOD reduced the fiscal years 2006-2011 defense 
program by $30 billion, DOD officials stated that the department did 
not take the traditional budgetary approach of cutting each defense 
component's budget by a certain percentage. Instead, DOD officials 
stated that the Secretary of Defense used a collaborative approach with 
service participation to discuss where to take the budget reductions 
and how these cuts would affect risk, although DOD officials offered 
various views on how extensively the framework was used to make those 
decisions. 

Second, DOD required that the services and other defense components 
offset any funding increase in one area with a funding decrease in 
another area for the fiscal years 2006-2007 budget submission. 
According to DOD officials, risk--whether on the basis of "professional 
judgment" or analysis--was considered in these deliberations. For 
example, the Army's plan for fiscal years 2006-2023 articulated areas 
for increasing risks so that it could decrease risk in the operational 
risk dimension by investing in current capacity. 

However, the fiscal year 2006 budget submission does not include any 
specific information on how DOD systematically identified or assessed 
departmental risks to establish DOD-wide investment priorities. For 
example, the military services' share of the Future Years Defense 
Program (FYDP) remained relatively unchanged from fiscal year 2005 to 
fiscal year 2006 (see table 3),[Footnote 23] providing one indication 
that the risk management framework may not yet be a useful tool for 
balancing departmental risks across the services. 

Table 3: Military Service and Defense-Wide Percentage of the 2005 and 
2006 Future Years Defense Programs: 

Department of the Army; 
2005 Percentage of FYDP: 24.23; 
2006 Percentage of FYDP: 24.63; 
Percentage change by department: 0.40. 

Department of the Navy; 
2005 Percentage of FYDP: 29.75; 
2006 Percentage of FYDP: 29.47; 
Percentage change by department: -0.28. 

Department of the Air Force; 
2005 Percentage of FYDP: 29.80; 
2006 Percentage of FYDP: 29.82; 
Percentage change by department: 0.02. 

Defense-wide; 
2005 Percentage of FYDP: 16.22; 
2006 Percentage of FYDP: 16.08; 
Percentage change by department: -0.14. 

Total; 
2005 Percentage of FYDP: 100.00; 
2006 Percentage of FYDP: 100.00. 

Source: GAO analysis of DOD FYDP data. 

Note: Totals may not add due to rounding. 

[End of table] 

DOD has reported on the risk management framework in the department's 
GPRA and other reporting requirements. For example, the fiscal year 
2004 Performance and Accountability Report describes what DOD is doing, 
or plans to do, to define, measure, and monitor performance goals in 
the four risk quadrants but does not discuss the implementation status 
of the risk management framework. Furthermore, the fiscal year 2004 
report, the most recent available, provided insufficient information to 
assist Congress in overseeing how DOD plans to prioritize investment 
decisions within or across the risk quadrants. Without more detailed 
information, Congress may have insufficient transparency into how DOD 
has identified and assessed risks and made trade-offs in its investment 
decision making. In addition, we reported in May 2004 that 
congressional visibility over investment decision making also was 
limited by the absence of linkages between the risk management 
framework and military capabilities planning and the FYDP.[Footnote 24] 
Because the FYDP lacked these linkages, we concluded that decision 
makers could not use it to determine how a proposed increase in 
capability would affect the risk management framework. 

Our work also has shown that the FYDP may understate the costs of 
weapon system programs; therefore, DOD may be starting more programs 
than it can afford. For example, our assessment of 54 major programs, 
representing an investment of over $800 billion, found that the 
majority of these programs were costing more and taking longer to 
develop than planned.[Footnote 25] Problems occurred because of DOD's 
overly optimistic planning assumptions about the long-term costs of 
weapon system programs and its failure to capture early on the 
requisite knowledge that is needed to efficiently and effectively 
manage program risks. When DOD has too many programs competing for 
funding and approves programs with low levels of knowledge, it is 
accepting the attendant likely adverse cost and schedule risks. As a 
result, it will probably get fewer quantities for the same investment 
or face difficult choices on which investments it cannot afford to 
pursue. The findings of our work suggest that having a departmentwide 
investment strategy for weapon systems, to allocate resources across 
investment priorities, would help reduce these risks. 

Cultural Resistance, Combined with the Lack of Leadership, 
Implementation Goals, and Process Integration, Affects DOD's 
Implementation of the Risk Management Framework: 

Four key challenges impede DOD's progress toward implementing the risk 
management framework. The first implementation challenge facing DOD is 
overcoming cultural resistance to change in a department as massive, 
complex, and decentralized as DOD. The second challenge is the lack of 
sustained leadership, and the third challenge is the absence of 
implementation goals and timelines. These challenges relate to DOD's 
failure to follow crucial transformational steps. The fourth challenge-
-integrating the risk management framework with decision support 
processes and related reform initiatives, into a coherent, unified 
management approach for the department--relates to key results-oriented 
management practices. Unless DOD addresses these challenges and 
successfully implements the risk management framework, or a similar 
approach, it may continue to experience (1) a mismatch between programs 
and budgets, and (2) the proportional, rather than strategic, 
allocation of resources to the services. 

Transforming DOD's Organizational Culture Is a Significant Challenge: 

Transforming DOD's organizational culture--from a focus on inputs and 
programs to strategically balancing investment risks and monitoring 
outcomes across the department--through the implementation of the risk 
management framework is a significant challenge for the department for 
several reasons. First, as we noted in our 21st Century Challenges 
report, to successfully transform, DOD needs to overcome the inertia of 
various organizations, policies, and practices that became rooted in 
the Cold War era.[Footnote 26] The department's expense, size, and 
complexity, however, make overcoming this resistance and inertia 
difficult. In fiscal year 2004, DOD reported that its operations 
involved $1.2 trillion in assets, $1.7 trillion in liabilities, over 
$605 billion in net cost of operations, and over 3.3 million military 
and civilian personnel. For fiscal year 2005, DOD received 
appropriations of about $417 billion. Moreover, execution of its 
operations spans a wide range of defense organizations, including the 
military services and their respective major commands and functional 
activities, numerous large defense agencies and field activities, and 
various combatant and joint operation commands, which are responsible 
for military operations for specific geographic regions or theaters of 
operations. 

Second, DOD's highly decentralized management structure is another 
contributing factor that makes cultural change difficult. Although 
under the authority, direction, and control of the Secretary of 
Defense, the military services have the legislative authority to 
organize, equip, and train the nation's armed forces for combat under 
Title 10 of the U.S. Code. Furthermore, Congress directly appropriates 
funds to the services for programs and activities that support these 
purposes. In the opinion of knowledgeable DOD officials, this 
legislative authority has resulted in a culture that makes it difficult 
to develop department-level, or joint, management approaches. For 
example, the allocation of budgets on a proportional, rather than a 
strategic basis, among the military services is a long-standing 
budgetary problem that we have identified as a major management 
challenge for the department.[Footnote 27] In addition, the Joint 
Defense Capabilities Study, chartered by the Secretary of Defense in 
March 2003, made the following observations on how DOD's organizational 
culture does not reinforce a departmental or joint approach to 
investment decision making and results management:[Footnote 28] 

* DOD's bottom-up strategic planning process did not support early 
senior leadership involvement and did not provide integrated 
departmentwide objectives, priorities, and roles as a framework for 
planning joint capabilities. 

* Service-centric focus on programs and weapons platforms resulted in a 
process that did not provide an accurate picture of joint needs, nor 
did it provide a consistent view of priorities and acceptable risks 
across the department. 

* The resulting budget did not optimize capabilities at either the 
department or the service level. 

* Accountability and feedback focused on monetary input rather than 
output; therefore, much of the information provided did not support the 
senior leaders' decision making as it did not tell how well the 
department was being resourced to meet current and future mission 
requirements. 

Lack of Sustained Leadership and Appropriate Accountability Has 
Challenged DOD's Implementation of the Risk Management Framework: 

The lack of sustained leadership attention and appropriate 
accountability has challenged DOD's progress in implementing the risk 
management framework. Our work has indicated that sustained leadership 
is a key transformational, or change management, practice.[Footnote 29] 
However, knowledgeable DOD officials indicated that DOD's senior 
leadership did not provide sustained attention to the framework's 
implementation. For example, a DOD official actively involved in the 
framework's implementation stated that meetings with senior leadership 
that were to provide oversight of the framework's implementation have 
not been regularly scheduled. DOD officials indicated that as a result 
of this lack of sustained leadership, DOD has not placed much emphasis 
on implementing the risk management framework at the department level. 
In addition, other DOD officials stated that changes in leadership have 
made it difficult to implement the risk management framework or develop 
performance measures. For example, since October 2004, DOD has 
experienced turnover in the following senior level positions, including 
the Deputy Secretary of Defense; the Under Secretary of Defense for 
Acquisition, Technology and Logistics; and the Director of Program 
Analysis and Evaluation (PA&E). Lacking sustained leadership attention, 
DOD officials offered conflicting perspectives on the status of the 
risk management framework with some officials suggesting that the 
framework had been overtaken by other performance-based or risk-based 
management initiatives while another suggested that the framework was 
primarily a compliance exercise. DOD officials also held differing 
perspectives on the purpose of the framework, including the beliefs 
that it was developed to monitor the Secretary of Defense's priority 
areas or that it was a programming and budgeting tool. 

Implementation of the risk management framework has also been 
challenged by the lack of clear lines of authority and appropriate 
accountability. No single individual or organization has been given 
overarching leadership responsibilities, authority, or the 
accountability for achieving the framework's implementation. Instead, 
the responsibility for various tasks and performance measures have been 
spread among several organizations, including the Director, PA&E; the 
Under Secretary of Defense for Personnel and Readiness (P&R); and the 
Under Secretary of Defense, Comptroller/Chief Financial Officer. 

We testified in April 2005 that as DOD embarks on large-scale change 
initiatives, the complexity and long-term nature of these initiatives 
require the development of an executive position capable of providing 
strong and sustained leadership--over a number of years and various 
administrations.[Footnote 30] For this reason, we have supported 
legislation to create a CMO at DOD to provide such sustained 
leadership.[Footnote 31] A CMO could also provide the leadership needed 
to successfully develop a risk-based and results-oriented management 
approach at DOD, such as the risk management framework. 

Lack of Implementation Goals and Timelines Further Challenges DOD's 
Implementation of Risk Management Framework: 

Accountability for implementation of the risk management framework also 
has been hindered by the absence of implementation goals and timelines 
with which to gauge progress. As we have previously reported, 
successful change management efforts use implementation goals and 
timelines to pinpoint performance shortfalls and gaps, suggest 
midcourse corrections, and build momentum by demonstrating 
progress.[Footnote 32] However, DOD's limited guidance on the risk 
management framework did not establish implementation goals and 
timelines, nor did it require that implementation goals and timelines 
be developed. According to knowledgeable DOD officials, DOD did not see 
the need for implementation goals or timelines because the framework 
was not meant to change processes or create new ones, but rather was a 
management tool to improve upon investment decision-making processes. 
Regardless of how DOD classifies the risk management framework, we have 
found that implementation goals and timelines are essential to any 
transformational change, such as that envisioned by the Secretary of 
Defense with the risk management framework, because of the number of 
years it can take to complete the change.[Footnote 33] Moreover, the 
absence of implementation goals and timelines makes it difficult to 
determine whether progress has been made in implementing the framework 
over the last 2 ½ years, and whether DOD's revisiting of the framework 
during the 2005 QDR represents an evolutionary progression or 
implementation delays. 

Integrating the Risk Management Framework with Decision Support 
Processes and Related Reform Initiatives Is a Significant Challenge: 

DOD faces a significant challenge integrating the risk management 
framework with decision support processes for planning, programming, 
and budgeting and with related reform initiatives into a coherent, 
unified management approach. The goal of both risk management and 
results-oriented management is to integrate the systematic concern for 
risk and performance into the usual cycle of agency decision making and 
implementation. DOD's challenge in meeting these goals is demonstrated 
by the number of initiatives, as shown in table 4, that DOD has put in 
place to improve investment decision making and manage performance 
results. For example, both capabilities planning and the risk 
management framework are to define risks and develop performance 
measures but, according to DOD officials, the department is still 
determining how to align capabilities planning with the risk management 
framework. Other initiatives, including GPRA and PART, are also to 
develop performance measures and DOD is still working on integrating 
these initiatives with the risk management framework and individual 
performance monitoring approaches of the services and other defense 
components into a single, integrated system. In December 2002, the 
Deputy Secretary of Defense issued a memorandum to correct this 
situation by requiring the alignment of the risk management framework 
and the President's Management Agenda with DOD's results-oriented 
management activities, including those associated with GPRA. 

Table 4: Select Initiatives to Improve Investment Decision Making: 

Initiative: Two-Year Planning, Programming, Budgeting, and Execution 
Process (PPBE); 
Description: In 2003, DOD implemented a 2-year cycle for its strategic 
planning, program development, and resource determination process. DOD 
stated that this change was needed to integrate DOD's processes for 
strategic planning, identification of needs for military capabilities, 
systems development and acquisition, and program and budget 
development. During the second year of the biennial budget, DOD is to 
focus on budget execution and program performance. 

Initiative: Enhanced Planning Process; 
Description: In fiscal year 2004, DOD initiated a reform of defense 
planning to make it more responsive and adaptive to the needs of senior 
decision makers. The process is to result in fiscally constrained 
guidance and priorities-- for military forces, modernization, readiness 
and sustainability, and supporting business processes and 
infrastructure activities--for program development. The enhanced 
planning process is to integrate the outcomes of operational, 
enterprise, and capabilities planning efforts in a document called the 
Joint Programming Guidance. The Joint Programming Guidance is to 
provide a link between planning and programming, and it is to provide 
guidance to the DOD components for the development of their program 
proposals. 

Initiative: Capabilities Planning; 
Description: The 2001 QDR announced a defense strategy built around the 
concept of shifting to a "capabilities-based" approach to defense. 
According to the 2001 QDR, while DOD cannot know with confidence what 
nation, group of nations, or nonstate actor might pose a threat to U.S. 
vital interests, it is possible to anticipate the capabilities an 
adversary might employ. Capabilities planning is to provide a top-down, 
competitive approach to weigh options against resource constraints 
across a spectrum of challenges and to apportion risk against those 
challenges. It is also to enable risk assessments and trade-off 
decisions across DOD organizational stovepipes. The new concept 
stresses joint solutions to problems, requires the identification of 
risk trade-offs within and across mission areas, and treats uncertainty 
explicitly. 

Initiative: Program/Budget Framework Initiative; 
Description: As part of the financial management enterprise initiatives 
of the Business Management Modernization Program, this initiative is to 
provide a foundation for a new program and budget data structure using 
a common language that enables senior level DOD decision makers to 
weigh options versus resource constraints across a spectrum of 
challenges. The framework is to consist of a number of related data 
transparency initiatives that span across all portions of the PPBE 
process, including creating department-level definitions for the four 
risk quadrants. One of the stated benefits is establishing an ability 
to view programs and resources based on the risk management framework. 

Initiative: Joint Capabilities Integration and Development System 
(JCIDS); 
Description: A system for the Joint Staff to assess gaps in military 
joint warfighting capabilities and recommend solutions to resolve those 
gaps. This system is replacing DOD's requirements- generation process 
for major acquisitions in an effort to shift the focus to a more 
capabilities-based approach for determining joint warfighting needs 
rather than a threat-based approach focused on individual systems and 
platforms. Under this system, boards comprised of high-level DOD 
civilians and military officials are to identify future capabilities 
needed around key functional concepts and areas, such as command and 
control, force application, and battlespace awareness, and to make 
trade-offs among air, space, land, and sea platforms in doing so. 

Initiative: President's Management Agenda; 
Description: The President's Management Agenda contains five 
initiatives aimed at improving federal agency management and 
performance: (1) strategic human capital management, (2) competitive 
sourcing, (3) improved financial performance, (4) expand electronic 
government, and (5) budget and performance integration. The President 
cited our work on high-risk areas and major management challenges in 
developing his initiatives, and implementation of the agenda has 
reinforced the need to focus agencies' efforts on achieving key 
management and performance improvements. 

Initiative: Budget and Performance Integration; 
Description: The budget and performance integration initiatives of the 
President's Management Agenda include elements such as the PART used to 
review programs, an emphasis on improving outcome measures, and 
improving monitoring of program performance. PART is the central 
element in the performance budgeting piece of the President's 
Management Agenda. PART builds on GPRA by actively promoting the use of 
results-oriented information to assess programs in the budget. 

Source: GAO analysis. 

[End of table] 

We note that these reform initiatives address key business processes 
within the department and that we have placed DOD's overall business 
transformation on our list of federal programs and activities at high 
risk of waste, fraud, abuse, and mismanagement.[Footnote 34] 

The Under Secretary of Defense for Acquisition, Technology and 
Logistics indicated that DOD plans to address the challenge associated 
with the integration of DOD's planning, resourcing, and execution 
processes and initiatives, including the risk management framework. The 
Under Secretary stated that one task of the ongoing 2005 QDR was 
"strategic process integration." The Under Secretary also stated that 
the department is planning to provide a roadmap with performance goals 
and timelines on how it will implement initiatives to improve strategic 
process integration. This roadmap is to be submitted with the 2005 QDR 
report to Congress in early 2006 with the fiscal year 2007 budget. 

Conclusions: 

DOD has made some progress in implementing the risk management 
framework, including establishing risk quadrants and performance goals. 
However, more work will be required for DOD to be able to put in place 
a management tool, such as the risk management framework, to 
strategically balance the allocation of resources across the spectrum 
of its investment priorities against risk over time and to monitor 
performance. The development of performance measures that clearly 
demonstrate results and that are cascaded down throughout the 
department would enable DOD to provide a clear roadmap of how its 
activities at all levels contribute to meeting its strategic goals and 
would assist the department in aligning the core processes and 
resources of its four military services and multiple defense agencies 
to better support a departmental or joint approach to national 
security. Furthermore, the risk management framework cannot be fully 
implemented until its performance goals are clearly linked to DOD's 
strategic planning goals. Unless a cause and effect relationship can be 
demonstrated between the department's performance measures and 
strategic goals, the framework's usefulness as a tool for monitoring 
DOD's execution of its strategic plan and identifying performance goals 
will be severely restricted, if not eliminated. Furthermore, the fiscal 
year 2006 budget submission does not provide sufficient information on 
how DOD identified or assessed departmental risks to establish DOD-wide 
investment priorities; thus, the linkages between the framework and the 
budget are unclear. Without better measures, clear linkages, and 
greater transparency, DOD will be unable to fully measure progress in 
achieving strategic goals or demonstrate to Congress and others how it 
considered risks and made trade-off decisions, balancing needs and 
costs for weapon programs and other investment priorities. 

The efforts of DOD's senior leadership to establish a risk-based and 
results-oriented management approach have been impeded by some key 
challenges. The lack of sustained leadership and clear lines of 
accountability has hampered implementation of the risk management 
framework and the establishment and achievement of implementation goals 
and timelines. Strong and sustained leadership could enable DOD to 
overcome resistance to change that exists in a department as massive 
and complex as DOD. In addition, the establishment of implementation 
goals and timelines could enable DOD to determine what progress has 
been made in implementing the risk management framework. Furthermore, 
the successful integration of the risk management framework into DOD's 
investment decision-making processes, including recent reform 
initiatives, could assist DOD in its overall transformation efforts. 
Until DOD develops a risk-based and results-oriented management 
approach for making investment decisions, it will likely continue to 
experience a mismatch between programs and budgets, and the 
proportional, rather than strategic, allocation of resources to the 
services. 

Recommendations for Executive Action: 

To address the challenges associated with implementing the risk 
management framework, or a similar risk-based management approach, we 
recommend that the Secretary of Defense take the following four 
actions: 

* develop or refine department-level performance measures so that they 
clearly demonstrate performance results and cascade those measures down 
throughout the department, 

* assign clear leadership with accountability and authority to 
implement and sustain the risk management framework, 

* develop implementation goals and timelines, and: 

* demonstrate the integration of the risk management framework with 
DOD's decision support processes and related reform initiatives to 
improve investment decision making and manage performance results. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, DOD partially concurred 
with our four recommendations. DOD's written comments are reprinted in 
their entirety in appendix II. DOD also provided technical comments, 
which we incorporated as appropriate. 

DOD partially concurred with our first recommendation. DOD stated that 
it concurred with our recommendation that the Secretary of Defense 
refine department-level performance measures so that they clearly 
demonstrate results, but that it did not concur with the notion that 
effectively cascading the risk management framework has been inhibited 
by the current suite of performance measures. DOD noted that that a 
number of defense components--including the Army, DOD Comptroller, the 
Defense Logistics Agency, and the Defense Information Systems Agency-- 
have successfully cascaded departmentwide strategic goals and 
implemented frameworks to measure their organization's performance. DOD 
also believes that empowering the leadership at the component level to 
develop measures, while ensuring strategic alignment, is the most 
effective way of encouraging performance management and increasing its 
utility. In our report, we acknowledge that DOD has taken positive 
steps toward developing a performance monitoring system and cascading 
the framework's goals and measures to defense components. However, our 
recommendation addresses limitations in those measures that currently 
hinder DOD's ability to use the risk management framework as a 
management tool for aligning the components' performance goals and 
measures with the risk management framework, or for strategic balancing 
investment decisions across the risk quadrants. For example, the 
majority of the risk management framework's measures are activity 
measures, or initiatives, that do not monitor a specific annual 
performance target, nor do these measures provide sufficient 
information to determine whether the activity is on schedule or 
contributes to enhancing the department's overall performance. Finally, 
our recommendation is not intended to suggest that DOD not empower the 
components to develop performance measures, but rather that DOD 
establish a clear hierarchy of goals and measures that provide 
straightforward roadmaps to demonstrate how the components' activities 
contribute to meeting DOD's strategic goals. 

DOD partially concurred with our second recommendation that the 
Secretary of Defense assign clear leadership with accountability and 
authority to implement and sustain the risk management framework. DOD 
stated that, although it agrees that such leadership is key to any 
successful performance management system, the department's senior 
executives provide sufficient leadership and accountability for 
implementing and sustaining the risk management framework. DOD also 
stated that it did not agree that a new organization or bureaucratic 
structure is needed to ensure successful implementation and sustainment 
of the risk management framework. We agree that DOD has assigned 
specific roles and responsibilities for goals and measures associated 
with the risk management framework to various high-level DOD officials. 
However, we based our recommendation on the fact that no single 
individual, with appropriate authority, was held responsible for 
ensuring that the risk management framework was implemented across the 
department. Further, our recommendation does not propose that DOD set 
up a new organization or bureaucratic structure, but, as stated in this 
report, we continue to believe that one way to provide strong and 
sustained leadership for change initiatives, such as the risk 
management framework, over a number of years and various 
administrations is to legislatively establish a CMO. 

In partially concurring with our third recommendation to develop 
implementation goals and timelines, DOD agreed that tracking progress 
in implementing the risk management framework is a good management 
practice. DOD stated that it has established goals and timelines for 
the risk management framework that are unique to the individual 
metrics, or measures, and that because the risk management framework 
continually evolves over time, new metrics will be developed while 
others may be retired. As we stated in the report, successful change 
management efforts use implementation goals--such as, for example, 
linking the risk management framework to the budget--and timelines for 
meeting those goals, to pinpoint shortfalls and gaps, suggest midcourse 
corrections, and build momentum by demonstrating progress. Therefore, 
while DOD may continually refine the individual goals and measures 
associated with the framework's risk quadrants, we believe that goals 
and timelines for the overall implementation of the framework across 
the department are essential for keeping this reform initiative on 
track. 

DOD partially concurred with our fourth recommendation that the 
Secretary of Defense demonstrate the integration of the risk management 
framework with DOD's decision support processes and related reform 
initiatives to improve investment decision making and manage 
performance results. DOD stated that the department is currently 
studying ways to further integrate the risk management framework with 
other decision support processes, but no single framework or decision 
model can provide all the necessary information or flexibility needed 
by the Secretary of Defense and his senior leadership team. We 
recognize that DOD's senior leadership needs reliable information from 
a variety of sources and flexibility to make decisions among 
alternative actions or solutions. However, if the risk management 
framework is to successfully serve as a management tool to assist 
decision makers in formulating top-down strategy, balancing investment 
priorities against risk over time, measuring near-and midterm outputs 
against strategic goals, and focusing on actual performance results--as 
intended by DOD's senior leadership--it is crucial that it be 
successfully integrated with DOD's investment decision-making 
processes, including recent reform initiatives. 

We are sending copies of this report to interested congressional 
committees; the Secretaries of Defense, Army, Navy, and Air Force; the 
Commandant of the Marine Corps; and the Director, Office of Management 
and Budget. We will also make copies available to others upon request. 
In addition, the report will be available at no charge on GAO's Web 
site at http://www/gao.gov. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-9619 or pickups@gao.gov. Contact points for our 
offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made major contributions to 
this report are listed in appendix III. 

Signed by: 

Sharon L. Pickup: 
Director, Defense Capabilities and Management: 

[End of section] 

Appendix I: Scope and Methodology: 

To assess to what extent the Department of Defense (DOD) has 
implemented the risk management framework, we obtained and analyzed DOD 
directives, briefings, and other documents that described the risk 
management framework's purpose, implementation status, and performance 
measures. We also obtained and analyzed DOD's 2001 Quadrennial Defense 
Review and annual strategic planning and budget documents. Moreover, we 
interviewed knowledgeable DOD and service officials involved with the 
implementation of the risk management framework. Specifically, we 
obtained testimonial evidence from officials representing the Office of 
the Secretary of Defense (OSD) offices--such as Program Analysis and 
Evaluation; Comptroller; Policy; Acquisition, Technology and Logistics; 
and Personnel and Readiness--the Joint Staff, the military services, 
and the Defense Business Board. To identify key risk-based and results-
oriented management principles, we reviewed our prior reports and other 
relevant literature, including information on the balanced scorecard 
concept. For example, we identified characteristics of results-oriented 
performance measures. These characteristics focused on performance 
measures that are (1) designed to demonstrate results by providing 
information on how well the organization is achieving its goals; (2) 
limited to a vital few, and balanced across priorities; and (3) used by 
management to improve performance. As another example, risk-based and 
results-oriented management principles indicate that leading 
organizations seek to establish clear hierarchies of goals and measures 
that cascade down so that subordinate units have straightforward 
roadmaps to demonstrate how their activities contribute to meeting the 
organization's strategy. We systematically analyzed and compared the 
risk management framework's department-level performance measures with 
these characteristics. However, we did not validate the procedures that 
DOD has in place to ascertain the reliability of the data used to 
support the performance measures. Regarding strategic planning, these 
principles focused on (1) establishing clear linkages among strategic 
planning goals, resources, performance goals and measures and (2) 
integrating the consideration of risk into the usual cycle of agency 
decision making and implementation. While these principles do not cover 
all attributes associated with risk-based and results-oriented 
management approaches, we believe that they are the most important ones 
for assessing DOD's progress in implementing the risk management 
framework. 

To identify the most significant challenges, we reviewed our previous 
work on change management principles. We then compared DOD's 
implementation of the risk management framework to sound change 
management principles and interviewed knowledgeable DOD officials about 
the challenges that faced the department in implementing the risk 
management framework. In addition, we reviewed our previous work to 
determine to what extent deficiencies in DOD's overall business 
transformation efforts might influence the implementation of the risk 
management framework. 

Our work was performed from October 2004 through September 2005 in 
accordance with generally accepted government auditing standards. 

[End of section] 

Appendix II: Comments from the Department of Defense: 

OFFICE OF THE SECRETARY OF DEFENSE: 
PROGRAM ANALYSIS AND EVALUATION: 
1600 DEFENSE PENTAGON: 
WASHINGTON, DC 20301-1600: 

NOV - 2 2005: 

Ms. Sharon Pickup: 
Director, Defense Capabilities and Management: 
U.S. Government Accountability Office: 
441 G Street, N.W.: 
Washington, DC 20548: 

Dear Ms. Pickup: 

This is the Department of Defense (DoD) response to the GAO draft 
report "DEFENSE MANAGEMENT: Additional Actions Needed to Enhance DoD's 
Risk-Based Approach for Making Resource Decisions," dated October 4, 
2005 (GAO Code 350611/GAO-06-13). 

DoD has reviewed the report for technical accuracy and content and 
partially concurs with the GAO's recommendations. The Department agrees 
that it is vitally important to have a coherent management process to 
set goals and objectives, measure performance, and respond rapidly to 
changing world events. Furthermore, DoD is committed to continuously 
refining its risk management framework to improve the linkages between 
strategy, performance goals, and performance results. While GAO 
recommends a number of actions to strengthen the risk management 
framework, the Department believes that its approach to date reflects a 
deliberate and tailored plan that has proven effective for managing 
this enterprise. 

Attached are specific comments on each recommendation, as well as 
technical comments. 

We appreciate the opportunity to comment on the draft report. 

Signed by: 

Stanley R. Szemborski: 
VADM, USN: 
Principal Deputy Director: 

Enclosures: As Stated: 

GAO DRAFT REPORT - DATED OCTOBER 4,2005: 
GAO CODE 350611/GAO-06-13: 

"DEFENSE MANAGEMENT: Additional Actions Needed to Enhance DoD's Risk- 
Based Approach for Making Resource Decisions" 

DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS: 

RECOMMENDATION 1: The GAO recommended that the Secretary of Defense 
develop or refine department-level performance measures so that they 
clearly demonstrate performance results. The GAO further recommended 
that the measures be cascaded through the Department. (p. 30/GAO Draft 
Report): 

DoD RESPONSE: Partially Concur. The Department concurs with the 
recommendation to refine department-level performance measures. DoD has 
worked continuously to refine both its performance measures and risk 
management framework in order to clearly demonstrate performance 
results and better link strategy to outcomes. In fact, one of the 
primary outputs of many activity metrics are new, refined measures that 
include the data necessary for quantitative evaluation. The Department 
believes that the process of developing better, more refined 
performance measures should continue, as no performance management 
system is ever truly "complete." 

However, DoD nonconcurs with the notion that effectively cascading the 
risk framework has been inhibited by the current suite of performance 
measures. DoD has demonstrated substantial progress in cascading 
measures throughout the Department. Upon introducing the risk 
management framework, the Department set out to ensure alignment across 
components and Defense agencies by requiring that each organization's 
performance plan or balanced scorecard: (t) reflect the balanced 
scorecard risk quadrants; (2) reflect the Department's overall 
performance objectives; and (3) align with and support the outcomes and 
performance metrics of the next higher organization. A number of 
Defense components-including the Army, DoD Comptroller, Defense 
Logistics Agency (DLA), and Defense Information Systems Agency (DISA)- 
have successfully cascaded Department-wide strategic goals and 
implemented frameworks to measure their organization's performance. DoD 
believes that empowering the leadership at the component level to 
develop measures that are meaningful and relevant to one's own 
organization, while ensuring strategic alignment, is the most effective 
way of encouraging performance management and increasing its utility. 

RECOMMENDATION 2: The GAO recommended that the Secretary of Defense 
assign clear leadership with accountability and authority to implement 
and sustain the risk management framework. (p. 30/GAO Draft Report): 

DOD RESPONSE: Partially Concur. The Department agrees-that clear 
leadership with accountability and authority are key characteristics of 
any successful performance management system. However, contrary to the 
GAO's assessment, the Department has taken a number of deliberate 
actions to ensure proper accountability and authority. Therefore, DoD 
does not agree that a new organization or bureaucratic structure is 
needed to ensure successful implementation and/or sustainment of the 
risk management framework. 

DoD's senior executives, including Principal Staff Assistants and their 
Performance Management Coordinators, provide sufficient leadership and 
accountability for implementing and sustaining the risk management 
framework. Specific roles and responsibilities include: 

(1) the development of annual performance targets to support the risk 
management framework; (2) data -collection; (3) verification and 
validation of performance data; and (4) management of activities to 
improve performance and achieve targets during, and following, the year 
of execution. By empowering senior leaders throughout the Department to 
measure organizational performance, and manage accordingly, the 
Department has made great progress in ensuring accountability. 

RECOMMENDATION 3: The GAO recommended that the Secretary of Defense 
develop implementation goals and timelines. (p. 30/GAO Draft Report): 

DoD RESPONSE: Partially Concur. DoD agrees that tracking progress in 
implementing the framework is a good management practice. However, DoD 
has taken the necessary steps to track implementation of the risk 
management framework. For example, DoD has established goals and 
timelines for the risk management framework that are unique to the 
individual metrics. The Department has also established either a 
performance metric or activity metric for every relevant area. Since 
the risk management framework continually evolves over time to reflect 
changes in strategy, priorities, and senior leadership interest, new 
metrics will be developed while others may retired. Because individual 
metric development occurs at different points in time, there is not one 
single milestone or event that will mark completion of the risk 
management framework as a whole. 

RECOMMENDATION 4: The GAO recommended that the Secretary of Defense 
demonstrate the integration of the risk management framework with DoD's 
decision support processes and related reform initiatives to improve 
investment decisionmaking and manage performance results. (p. 30/GAO 
Draft Report): 

DOD RESPONSE: Partially Concur. The Department agrees that integrating 
the risk management framework with other decision support processes and 
related reform initiatives is both a natural evolution of the risk 
management framework's maturation effort and a beneficial output of the 
process. As such, the Department is currently studying ways to further 
integrate the risk management framework with other decision support 
processes. However, the Secretary of Defense and his senior leadership 
team must be able to make well-informed decisions in a variety of 
circumstances. No single framework or decision model can provide all of 
the necessary information and/or flexibility. 

Technical Comments Draft GAO Report GAO-06-13, GAO Code 350611: 

Issue: Page 2, paragraph 2. Sentence reads: "..we conducted interviews 
with DoD and service officials, members of the Joint Staff, and members 
of the Defense Business Board." 

The Defense Business Board does not officially speak on the 
Department's behalf. 

Issue: Page 12, paragraph 2. The sentence reads: "44 of the 77 
departmental-level measures .. are activity measures." Page 13, Table 
1. Footnote describes a recoding of five performance measures to 
activity measures. 

DoD objects to recoding these metrics from performance metrics to 
activity metrics. 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Sharon Pickup, (202) 512-9619 or pickups@gao.gov: 

Acknowledgments: 

In addition to the contact named above, David Moser, Assistant 
Director; Donna Byers; Gina Flacco; and Renee S. Brown made key 
contributions to this report. 

FOOTNOTES 

[1] See GAO, 21st Century Challenges: Reexamining the Base of the 
Federal Government, GAO-05-325SP (Washington, D.C.: February 2005) for 
a comprehensive compendium of areas throughout the federal government 
that could be considered for reexamination and review by Congress. 

[2] GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.: 
January 2005). 

[3] GAO, Highlights of a GAO Forum: Mergers and Transformation: Lessons 
Learned for a Department of Homeland Security and Other Federal 
Agencies, GAO-03-293SP (Washington, D.C.: Nov. 14, 2002), and Results- 
Oriented Cultures: Implementation Steps to Assist Mergers and 
Organizational Transformations, GAO-03-669 (Washington, D.C.: July 2, 
2003). 

[4] S. 780, 109th Cong. §1 (2005). 

[5] GAO-05-207. 

[6] GAO-05-207. 

[7] The Government Performance and Results Act of 1993 (Pub. L. No. 103-
62). 

[8] The balanced scorecard approach was advocated by Professor Robert 
Kaplan and Dr. David Norton in the November/December 1992 Harvard 
Business Review. 

[9] GAO-05-325SP. 

[10] See for example, Committee of Sponsoring Organizations of the 
Treadway Commission, Enterprise Risk Management--Integrated Framework: 
Executive Summary (New York, N.Y.: September 2004). 

[11] Pub. L. No. 103-62 (1993). 

[12] The President's Management Agenda, by focusing on a number of 
targeted areas, seeks to improve the performance management of the 
federal government. 

[13] For further information see: GAO, Performance Budgeting: PART 
Focuses Attention on Program Performance, but More Can Be Done to 
Engage Congress, GAO-06-28 (Washington, D.C.: Oct. 28, 2005); 
Management Reform: Assessing the President's Management Agenda, GAO-05-
574T (Washington, D.C.: Apr. 21, 2005); Results-Oriented Government: 
GPRA Has Established a Solid Foundation for Achieving Greater Results, 
GAO-04-38 (Washington, D.C.: Mar. 10, 2004); and Performance Budgeting: 
Observations on the Use of OMB's Program Assessment Rating Tool for the 
Fiscal Year 2004 Budget, GAO-04-174 (Washington, D.C.: Jan. 30, 2004). 

[14] GAO, Executive Guide: Effectively Implementing the Government 
Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June 
1996) and Managing for Results: Enhancing Agency Use of Performance 
Information for Management Decision Making, GAO-05-927 (Washington, 
D.C.: Sept. 9, 2005). 

[15] GAO/GGD-96-118. 

[16] GAO/GGD-96-118 and GAO-05-927. 

[17] GAO/GGD-96-118. 

[18] GAO/GGD-96-118. 

[19] GAO, Quadrennial Defense Review: Future Reviews Can Benefit from 
Better Analysis and Changes in Timing and Scope, GAO-03-13 (Washington, 
D.C.: Nov. 4, 2002). 

[20] The Defense Business Board was established in 2001 by the 
Secretary of Defense to provide DOD's senior leadership with leading- 
edge, actionable advice on management improvements. 

[21] GAO, Managing for Results: Critical Issues for Improving Agencies' 
Strategic Plans, GAO/GGD-97-180 (Washington, D.C.: Sept. 16, 1997). 

[22] See GAO/GGD-96-118. 

[23] The Future Years Defense Program provides information on DOD's 
current and planned outyear budget requests. 

[24] GAO, Future Years Defense Program: Actions Needed to Improve 
Transparency of DOD's Projected Resource Needs, GAO-04-514 (Washington, 
D.C.: May 7, 2004). 

[25] GAO, Defense Acquisitions: Assessments of Selected Major Weapon 
Programs, GAO-05-301 (Washington, D.C.: Mar. 31, 2005). 

[26] GAO-05-325SP. 

[27] GAO-05-325SP. 

[28] Joint Defense Capabilities Study Team, Joint Defense Capabilities 
Study: Final Report (Washington, D.C.: December 2003). 

[29] GAO, Management Reform: Elements of Successful Improvement 
Initiatives, GAO/T-GGD-00-26 (Washington, D.C.: Oct. 15, 1999). 

[30] GAO-05-520T and GAO-05-629T. 

[31] S. 780, 109th Cong. §1 (2005). 

[32] GAO-03-669. 

[33] GAO-03-669. 

[34] GAO-05-207. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, 

NelliganJ@gao.gov 

(202) 512-4800 

U.S. Government Accountability Office, 

441 G Street NW, Room 7149 

Washington, D.C. 20548: