GAO-08-298, Defense Acquisitions: DOD's Practices and Processes for Multiyear Procurement Should Be Improved


This is the accessible text file for GAO report number GAO-08-298 
entitled 'Defense Acquisitions: DOD's Practices and Processes for 
Multiyear Procurement Should Be Improved' which was released on 
February 8, 2008.

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: GAO: 

Report to the Committee on Armed Services, U.S. Senate: 

February 2008: 

Defense Acquisitions: 

DOD's Practices and Processes for Multiyear Procurement Should Be 
Improved: 

GAO-08-298: 

GAO Highlights: 

Highlights of GAO-08-298, a report to the Committee on Armed Services, 
U.S. Senate. 

Why GAO Did This Study: 

DOD spends $10 billion annually on multiyear procurement (MYP) 
contracts for weapons systems. MYPs may save money through more 
efficient relationships with suppliers and producers, but may also 
suffer losses if canceled and can limit future budget flexibility. 
Recently, Congress has been concerned about DOD’s management of the 
process and savings realized by MYPs. GAO was asked to evaluate DOD’s 
review process for MYP candidates; examine MYP program outcomes; 
identify the impact of changes to MYP savings threshold guidance, and 
determine how much DOD validates MYP performance. To do this, GAO 
reviewed statutes and other guidance, held discussions with relevant 
officials, examined DOD budget justifications and contracts, and 
conducted limited case studies. 

What GAO Found: 

DOD’s process for justifying multiyear programs leaves questions about 
the appropriateness of some approved MYPs and the cost effectiveness of 
investments made for the risks assumed, as indicated by recent 
submissions for the F-22A and V-22. Although the law has clear 
requirements for stable, low risk programs with realistic cost and 
savings estimates, lack of guidance and a rigorous process is not 
achieving this. 

It is difficult to precisely determine the impact of multiyear 
contracting on procurement costs. GAO studies of three recent MYPs 
identified unit cost growth ranging from 10 to 30 percent compared to 
original estimates, due to changes in labor and material costs, 
requirements and funding, and other factors. In some cases, actual MYP 
costs were higher than estimates for annual contracts. Although annual 
contracts also have unit cost growth, it is arguably more problematic 
for MYP’s because of the up-front investments and the government’s 
exposure to risk over multiple years. 

MYP savings were on average higher before changes in law called for 
“substantial savings” rather than a specific quantitative standard. 
Other factors—lower quantities of modern systems procured, stricter 
cancellation liability allowances, and contraction in the defense 
industrial base—may have also impacted savings by lessening 
opportunities for more efficient purchases, a key attribute of MYPs. 

DOD does not track multiyear results against original expectations and 
makes little effort to validate if actual savings were achieved. GAO’s 
case studies indicate that evaluating actual MYP results provides 
valuable information on the veracity of original estimates in the 
justification packages, the impacts on costs and risks from internal 
and external events, and lessons learned that can be used to improve 
future multiyear candidates and savings opportunities. 

Multiyear Contract Dollars and Percentage of Total Defense Procurement 
Obligation Authority: 

[See PDF for image] 

This figure is a combination line and vertical bar graph. The following 
approximated data is depicted: 

Fiscal year: 2000; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately $50 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 18%. 

Fiscal year: 2001; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately $56 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 16%. 

Fiscal year: 2002; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately$58 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 15%. 

Fiscal year: 2003; 
Total service obligation authority for mutiyear contracts: 
approximately $11 billion; 
Total DOD procurement budget: approximately $65 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
14%. 

Fiscal year: 2004; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately $80 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 12%. 

Fiscal year: 2005; 
Total service obligation authority for mutiyear contracts: 
approximately $12 billion; 
Total DOD procurement budget: approximately $78 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 14%. 

Fiscal year: 2006; 
Total service obligation authority for mutiyear contracts: 
approximately $8 billion; 
Total DOD procurement budget: approximately $74 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 12%. 

Fiscal year: 2007; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately $100 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 11%. 

Source: GAO analysis of DOD annual reports to the Congress. 

[End of figure] 

What GAO Recommends: 

GAO is making recommendations that DOD improve the outcomes of 
multiyear justification reviews by improving guidance, providing third 
party validation of MYP estimates, implementing a centralized database 
for MYP information, and conducting after-action assessments of 
completed MYP contracts. DOD concurred with two of the recommendations 
and partially concurred with the other two, stating that it will 
consider whether the delays and expense of third party validation are 
warranted by the benefits and that several factors limit the usefulness 
of after-action assessments. GAO believes its recommendations remain 
valid and can improve MYP outcomes. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.GAO-08-298]. For more information, contact 
Michael Sullivan at (202) 512-4841 or sullivanm@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

DOD's Practices and Processes for Justifying and Approving Multiyear 
Programs Could Be Improved: 

Actual Results on Some Completed Contracts Were Much Different Than 
Predicted in Budget Justifications: 

Estimated Savings for Candidate Programs Are Trending Downward: 

Multiyear Contracting Results Are Uncertain: 

Conclusions: 

Recommendations: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Defense: 

Appendix III: GAO Contacts and Staff Acknowledgments: 

Tables: 

Table 1: Statutory Requirements for Multiyear Procurement Candidates: 

Table 2: Unit Cost Growth for Three Multiyear Contracts: 

Table 3: Characteristics of Aircraft MYP Candidate Programs: 

Figures: 

Figure 1: Multiyear Contract Dollars and Percentage Share of Total 
Defense Procurement Obligation Authority: 

Figure 2: Contract Provisions That Impacted Case Study Multiyear 
Procurements: 

Figure 3: Legislative History Timeline for Multiyear Savings 
Requirement: 

Figure 4: Range of Estimated Savings for Multiyear Programs: 

Abbreviations: 

DFARS: Defense Federal Acquisition Regulation Supplement: 

DOD: Department of Defense: 

EOQ: economical order quantities: 

FAR: Federal Acquisition Regulation: 

FMR: Financial Management Regulation: 

FPDS-NG: Federal Procurement Data System-Next Generation: 

ICAR: Individual Contract Action Report: 

IDA: Institute for Defense Analyses: 

MYP: multiyear procurement: 

OSD: Office of the Secretary of Defense: 

USD: Under Secretary of Defense: 

[End of section] 

United States Government Accountability Office: Washington, DC 20548: 

February 7, 2008: 

The Honorable Carl Levin: 
Chairman: 
The Honorable John McCain: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

For the past 25 years, the Department of Defense (DOD) has used 
multiyear contracts to procure thousands of major weapon systems, 
investing about $10 billion annually in recent years. Multiyear 
procurement is a special authority to contract for up to 5 program 
years of requirements in one year. When used appropriately, multiyear 
contracting can save money compared to a series of annual contracts by 
allowing more economic procurement from suppliers and more efficient 
production. Multiyear contracting can also entail some risks of 
substantial losses if a program is reduced or a contract is canceled 
early and can limit DOD's and the Congress' future budget flexibility. 
To identify multiyear candidates that demonstrate sufficient benefits 
and manageable risks, the law requires certain criteria be met before 
an agency can enter into a multiyear contract. 

During discussions on the President's Budget for fiscal year 2007, 
committee members were concerned about the amount of projected savings 
realized on multiyear contracts and had questions about the DOD's 
management and oversight of programs with multiyear contracts. 
Accordingly, you asked us to (1) evaluate DOD's review process for 
multiyear procurement candidates submitted to Congress for approval; 
(2) examine cost and program outcomes on selected multiyear programs; 
(3) identify impacts from changes in guidance and interpretation of the 
savings requirement; and (4) determine the extent to which DOD tracks 
and validates multiyear performance. 

In conducting our work, we identified statutory criteria and 
implementing policies and procedures used by DOD to prepare and 
evaluate multiyear justification proposal packages. We discussed 
management oversight, practices, and results of the justification 
process with cognizant officials from each of the military departments 
and the Office of the Secretary of Defense (OSD). We examined DOD 
historical budgetary and contractual records to compile and summarize 
data on prior and current multiyear programs. We looked in more detail 
at nine major programs representing each military department: three had 
completed multiyear contracts, two were beginning multiyear contracts, 
and four programs were just recently authorized by the Congress. 
Several programs have been approved for more than one multiyear 
contract and, collectively, these programs comprised almost one-third 
of approvals granted since 1996. We researched the legislative history 
on the required savings criterion and DOD's efforts to interpret and 
apply it. We reviewed and discussed two recent multiyear studies 
conducted by the defense research organizations, RAND National Defense 
Research Institute and the Institute for Defense Analyses (IDA). 
Appendix I further explains the report's scope and methodology. We 
conducted this performance audit from June 2007 to February 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

Results in Brief: 

Multiyear contracting is big business and promises savings at some risk 
to the government, yet DOD's management direction and process for 
justifying multiyear programs to the Congress is limited, raising 
questions about the appropriateness of some approved multiyear programs 
and the cost effectiveness of investments made for the risks assumed. 
We identified concerns about the relative stability and savings 
potential of two recently approved programs, the F-22A Raptor and V-22 
Osprey. We found differences in how officials interpreted and applied 
the statutory criteria and in the methods and data used to compute 
contract cost and savings. Further, few records are kept to document 
decisions and supporting evidence. The statutory criteria establish 
requirements for stable, low risk programs with realistic cost and 
savings estimates, but DOD has not provided sufficient guidance and a 
rigorous, disciplined process to ensure high quality, consistent 
decisions supported by strong empirical evidence. 

Although it is difficult to precisely determine the impact of multiyear 
contracting on actual procurement costs, our case studies of completed 
multiyear contracts for the C-17A Globemaster, F/A-18E/F Super Hornet, 
and Apache Longbow Helicopter identified significant unit cost growth, 
ranging from 10 to 30 percent compared to the original estimates 
provided to Congress. All three programs--presumably approved based on 
their stability--were significantly impacted during contract execution 
by labor and material cost increases, changes in requirements and 
funding, and other factors that helped drive up total contract costs 
much beyond original projections. Savings also do not appear to have 
materialized as expected in the budget justifications submitted to the 
Congress and ultimately more funding was needed to buy the systems. In 
two of the three cases, actual costs for multiyear procurement exceeded 
original estimates for annual contracts. While both annual and 
multiyear contracting are prone to the underestimation of costs and 
overstatement of benefits as we have noted in our prior body of work on 
defense acquisitions, the stakes are arguably higher for multiyear 
programs because of the increased up-front investment required, 
considerable cost increases if a program is significantly restructured, 
and the greater liabilities incurred if multiyear programs are 
canceled. 

The meaning and application of the savings requirement for multiyear 
contracts have evolved over time. Expectations of 10 percent savings or 
more were emphasized during the 1980s, but replaced with a "substantial 
savings" requirement in fiscal year 1991. While a direct cause and 
effect relationship cannot be demonstrated, our analysis of multiyear 
programs approved by Congress shows that estimated savings were on 
average higher in the years before the "substantial savings" 
requirement was established than after, although there were wide ranges 
below and above 10 percent during both periods. Other factors--lower 
quantities of modern systems being procured, stricter cancellation 
liability allowances, and contraction in the defense industrial base-- 
may also decrease estimated savings for current and future systems by 
lessening the opportunities for achieving benefits from economic 
quantity buys, a key attribute of multiyear contracts. 

DOD does not have a formal mechanism for tracking multiyear results 
against original expectations and makes few efforts to validate whether 
actual savings were achieved by multiyear procurement. It does not 
maintain comprehensive central records and historical information that 
could be used to enhance oversight and knowledge about multiyear 
performance to inform and improve future multiyear procurement (MYP) 
candidates. DOD and defense research centers officials said it is 
difficult to assess results because of the lack of historical 
information on multiyear contracts, comparable annual costs, and the 
dynamic acquisition environment. Despite these limitations, our case 
studies indicate that evaluating the actual results from multiyear 
contracting provide valuable information regarding the veracity of 
original estimates in the justification packages, the impacts on costs 
and risks from internal and external events, and lessons learned that 
can be used to improve future MYP candidates and savings opportunities. 

GAO is making four recommendations to enhance the multiyear procurement 
approval process for major DOD weapon systems that include improving 
guidance related to the multiyear procurement decision criteria, 
establishing a third party validation process for multiyear candidate 
programs, maintaining a central database for monitoring major DOD 
weapon system multiyear procurements, and conducting after-action 
assessments of completed multiyear contracts used to procure major DOD 
weapon systems. In written comments on our draft report, DOD concurred 
with our two recommendations on improving guidance and maintaining a 
central database. DOD partially concurred with our two recommendations 
for third party validations and after-action assessments, stating that 
they may be of value for selected, but not all, programs. We believe 
that third party validations can improve the consistency and quality of 
cost and savings estimates that are integral to congressional and DOD 
decisions on multiyear proposals. Our review identified inconsistent 
practices and varying degrees of quality and completeness in the 
preparation and internal review of initial cost and savings estimates 
made by the weapon system program offices. After-action assessments can 
provide a continuing database of lessons learned that can benefit 
future programs. 

Background: 

Multiyear contracting is a special authority for acquiring more than 
one year's requirements --including weapon systems--under a single 
contract award without having to exercise an option for each program 
year after the first. It is an exception to the full-funding policy 
that requires the entire procurement cost of a weapon or piece of 
equipment to be funded in the year in which the item is procured. Under 
a multiyear procurement, DOD can contract for up to 5 years of 
quantities, although funding is still appropriated on an annual basis. 

Multiyear procurement can potentially save money and improve the 
defense industrial base by permitting the more efficient use of a 
contractor's resources. Multiyear contracts are expected to achieve 
lower unit costs compared to annual contracts through one or more of 
the following sources: (1) purchase of parts and materials in economic 
order quantities[Footnote 1] (EOQ), (2) improved production processes 
and efficiencies, (3) better utilized industrial facilities, (4) 
limited engineering changes due to design stability during the 
multiyear period, and (5) cost avoidance by reducing the burden of 
placing and administering annual contracts. Multiyear procurement also 
offers opportunities to enhance the industrial base by providing 
defense contractors a longer and more stable time horizon for planning 
and investing in production and by attracting subcontractors, vendors, 
and suppliers. However, multiyear procurement also entails certain 
risks that must be balanced against potential benefits, such as the 
increased costs to the government should the multiyear contract be 
changed or canceled and decreased annual budget flexibility for the 
program and across DOD's portfolio of weapon systems. Additionally, 
multiyear contracts often require greater budgetary authority in the 
earlier years of the procurement to economically buy parts and 
materials for multiple years of production than under a series of 
annual buys. 

Although DOD had been entering into multiyear contracts on a limited 
basis prior to the 1980s, the Department of Defense Authorization Act, 
1982,[Footnote 2] codified the authority for DOD to procure on a 
multiyear basis major weapon systems that meet certain criteria. Since 
that time, DOD has annually submitted various weapon systems as 
multiyear procurement candidates for congressional authorization. Over 
the past 25 years, Congress has authorized the use of multiyear 
procurement for approximately 140 acquisition programs, including some 
systems approved more than once. Section 2306b of title 10, United 
States Code, governs the use of multiyear contracting authority for the 
procurement of property by DOD. It specifies six statutory 
requirements, or criteria, that an acquisition program is expected to 
meet in order to be considered for multiyear contracting. These 
criteria are listed in table 1 below. 

Table 1: Statutory Requirements for Multiyear Procurement Candidates: 

Criteria: Substantial savings; 
Description: Candidate programs should demonstrate that use of a 
multiyear contract will result in substantial savings in the total 
estimated costs when compared to the use of a series of annual 
contracts for the same procurement. 

Criteria: Stability of the requirement; Description: Candidate programs 
should demonstrate that the minimum need to be purchased in terms of 
total quantity, production rate, and procurement rate is expected to be 
substantially unchanged during the multiyear contract period. 

Criteria: Stability of funding; 
Description: Candidate programs should have a reasonable expectation 
that sufficient funding will be requested by DOD to carry out the 
contract and avoid cancellation over the proposed multiyear contract 
period. 

Criteria: Stable design; 
Description: Candidate programs should be able to demonstrate that they 
have technical risks that are not excessive over the multiyear period 
and that the items procured should be substantially unchanged during 
the multiyear period. 

Criteria: Realistic cost estimates; 
Description: Candidate programs should be able to demonstrate realistic 
estimates of contract cost and projected multiyear savings/cost 
avoidance through the use of a multiyear contract strategy. 

Criteria: National security; 
Description: Candidate programs should be able to show that the use of 
a multiyear contracting strategy will promote the national security 
interests of the United States government. 

Source: GAO analysis and 10 U.S.C. 2306b. 

[End of table] 

Expected costs to be avoided should be sufficient to offset the added 
risk the government assumes with a multiyear contract in the form of a 
cancellation liability, decreased flexibility in future funding 
decisions and any erroneous assumptions in the estimates. Immature, 
volatile programs and those at risk of future changes should not be 
proposed as MYP candidates because such instability puts the savings 
attributed to efficiencies of production and EOQ buying at risk. The 
multiyear approach should be reserved for established production 
operations and low risk technology. 

In submitting candidates for multiyear authorization by the Congress, 
the heads of the respective military departments vouch that each 
program complies with the criteria in table 1. Additionally, the 
Secretary of Defense is required to certify to Congress that the 
current future years defense program fully funds the support costs 
associated with the multiyear contract and that planned production will 
not be less than the minimum economic rates given the existing tooling 
and facilities. Multiyear contracts historically account for a 
substantial share of the defense procurement dollar. Figure 1 shows 
that DOD has budgeted about $10 million annually for multiyear 
contracts since fiscal year 2000, accounting for more than 13 percent 
of DOD's total budget for procurement over this time frame. Over the 
period, the general trend shows an increase in total defense 
procurement, but multiyear obligations holding fairly steady, resulting 
in a downward trend of the percentage obligated on multiyear contracts. 
For 2007, the large increase in total defense procurement caused a drop 
below 12 percent obligated on multiyear. 

Figure 1: Multiyear Contract Dollars and Percentage Share of Total 
Defense Procurement Obligation Authority: 

[See PDF for image] 

This figure is a combination line and vertical bar graph. The following 
approximated data is depicted: 

Fiscal year: 2000; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately $50 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 18%. 

Fiscal year: 2001; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately $56 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 16%. 

Fiscal year: 2002; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately$58 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 15%. 

Fiscal year: 2003; 
Total service obligation authority for mutiyear contracts: 
approximately $11 billion; 
Total DOD procurement budget: approximately $65 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
14%. 

Fiscal year: 2004; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately $80 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 12%. 

Fiscal year: 2005; 
Total service obligation authority for mutiyear contracts: 
approximately $12 billion; 
Total DOD procurement budget: approximately $78 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 14%. 

Fiscal year: 2006; 
Total service obligation authority for mutiyear contracts: 
approximately $8 billion; 
Total DOD procurement budget: approximately $74 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 12%. 

Fiscal year: 2007; 
Total service obligation authority for mutiyear contracts: 
approximately $10 billion; 
Total DOD procurement budget: approximately $100 billion; 
Multiyear contracts as a percentage of total DOD procurement budget: 
approximately 11%. 

Source: GAO analysis of DOD annual reports to the Congress. 

[End of figure] 

DOD's Practices and Processes for Justifying and Approving Multiyear 
Programs Could Be Improved: 

The statutory criteria for a multiyear procurement require that a 
candidate program make realistic cost estimates, expect to achieve 
substantial savings, and provide adequate evidence that the program is 
stable in terms of funding, requirements, and design. Some recent 
programs of questionable stability and savings submitted to the 
Congress for multiyear authorization raise concerns about DOD's 
management and controls for justifying multiyear candidates. We found 
that DOD does not provide sufficient guidance and direction to ensure a 
rigorous, disciplined process supported by adequate empirical data for 
preparing and reviewing multiyear candidates. This increases the risk 
of poor outcomes and inappropriate, unstable programs approved for 
multiyear procurement. 

Questions about the Appropriateness of Recently Approved Programs: 

We reviewed DOD's multiyear justification data submitted in recent 
defense budgets and, in particular, examined two newly approved 
programs--the Air Force's F-22A fighter and the joint V-22 tilt rotor 
aircraft. The F-22A acquisition has had a turbulent history with a 
lengthy development period, major cost increases and quantity 
decreases, changes in mission, and disagreements within DOD about the 
total number required. The Air Force's submission of the F-22A for 
multiyear procurement generated considerable debate over its merits and 
whether it met the legal and business conditions conducive to success. 
We also examined the V-22 tilt rotor aircraft multiyear proposal, 
another acquisition program with a turbulent history and continuing 
challenges. 

Funding and Requirements Issues on F-22A Raptor Program: 

The latest restructure of the F-22A acquisition occurred in December 
2005. DOD extended production 2 years, added four aircraft and $1 
billion in procurement funds, and proposed to buy the final 60 aircraft 
under two separate 3-year multiyear contracts for the airframes and 
engines. Multiyear costs and savings estimates were not completed in 
time for submission with the fiscal year 2007 defense budget. The Air 
Force later submitted the completed MYP justification package with 
estimated total multiyear costs of $8.7 billion and projected savings 
of $235 million, or 2.6 percent, compared to estimated annual 
contracts. Multiyear proponents cited the projected total dollar 
savings as substantial and believed there was little risk that the 
remaining 60 aircraft would not be procured. On the other hand, 
multiyear critics argued that the low percent of savings predicted, the 
short time frame for accruing savings, and the program's relatively 
unstable past made it an inappropriate multiyear candidate. 

In prior work, we determined that the restructured F-22A program was 
underfunded and questioned whether the proposed multiyear strategy met 
statutory criteria.[Footnote 3] We identified concerns about savings, 
funding, requirements, and design stability that we believed needed to 
be addressed before the multiyear plan could be justified. For example, 
the Air Force did not fully fund the multiyear proposal and asked for 
incremental funding. Also, a major development program to add new 
capabilities and improve reliability of the F-22A has begun; these 
efforts could result in future design modifications which may require 
retrofit onto aircraft purchased under the multiyear contract. We also 
noted that having only a 3-year period of performance at the end of 
production limited the ability to achieve savings normally expected 
under multiyear authority such as EOQ buys and cost reduction 
initiatives to improve manufacturing efficiency. 

To provide for EOQ buys, Congress subsequently added $210 million to 
the F-22A advance procurement budget. In authorizing a multiyear 
contract for the F-22A, Congress specified certain conditions to be met 
and prohibited the use of incremental funding.[Footnote 4] The 
Secretary of Defense was required to certify that all statutory 
requirements have been met, including the determination that the 
contract will result in substantial savings of the total anticipated 
costs of carrying out the program through annual contracts. DOD 
submitted its certifications to the Congress in June 2007 and 
subsequently awarded the F-22A multiyear contract in August 2007. For 
the fiscal year 2009 budget cycle, the Air Force continues to propose 
buying more aircraft than the 183 in the current defense plan. 

Stability and Production Concerns about the V-22 Osprey Program: 

DOD submitted the V-22 Osprey for multiyear procurement authorization 
in the fiscal year 2007 President's Budget. Officials proposed a 5-year 
multiyear contract to acquire 185 aircraft for about $10.1 billion. 
Multiyear savings were projected at $435 million, or 4.2 percent, 
compared to the estimated costs for annual contracts. 

Ongoing changes in quantity, funding, design, and concerns about 
production raise questions about the stability of this program and its 
appropriateness for multiyear contracting. Subsequent to congressional 
authorization for multiyear contracting, DOD reduced its planned 
procurements quantity from 185 to 167 due to service funding 
limitations with DOD cutting the proposed procurement request for 
fiscal year 2009 by $234 million. 

Development and test efforts continue with a number of design changes 
under review to address serious safety, reliability, and performance 
problems. The program office has aggressively prioritized these issues 
and is making improvements to the V-22 platform by funding engineering 
design changes for the correction of deficiencies. One such deficiency 
is leaking hydraulic fluid causing engine compartment fires. Design 
changes to fix this deficiency are being studied and implemented. In 
comments on a draft of this report the program office stated it is 
confident that these engineering design changes will address the 
hydraulic leak problems. 

To date, DOD has procured 111 aircraft in 11 years. Production aircraft 
continue to be conditionally accepted with deviations and waivers. 
Engineering investigations to fix these issues are not complete as the 
program continues to work to minimize these deviations and waivers. 
Even so, the planned production rate for the multiyear period is twice 
the current fiscal year 2007 production rate of 17 V-22s. This 
increase, coupled with design and production problems, raises concerns 
over the contractor's ability to meet such a demand. DOD reviews and 
assessments of the V-22 production ramp up have endorsed the increase 
with known risks that require continued management. Officials told us 
that the supplier base should be able to meet the elevated production 
rate, but expressed concerns about the availability of spare parts and 
the challenges in managing manufacturing and installation at three 
different and dispersed facilities. 

Insufficient Guidance and Management Direction by DOD to Ensure High 
Quality Decisions: 

The statutory criteria in section 2306b of title 10, United States 
Code, establish a framework for limiting multiyear contracts to very 
stable programs that appropriately balance risks with anticipated 
savings. DOD's process and practices for justifying multiyear 
candidates, however, provide little specific guidance on the meaning 
and application of the criteria, a process that allows for subjective 
interpretations about how well a particular program meets the criteria. 
Cost and savings estimating techniques and data also vary considerably 
and inadequate records are kept to document decisions and supporting 
reasons. As a result, costs, savings, and evidence supporting stability 
is not consistently prepared, reviewed, and documented. 

Concerns about DOD's Processes and Practices for Justifying Multiyear 
Candidates: 

The statutory criteria are general in tone and qualitative by nature to 
provide application over a wide range of programs. The regulations DOD 
uses to implement the criteria provide contract policies and establish 
a process for developing and justifying multiyear candidates.[Footnote 
5] Each candidate program prepares a budget justification package, 
normally for inclusion with the annual defense budget submission that 
provides funding requirements and estimated cost savings expected under 
a multiyear contract compared to the estimated costs for a series of 
annual contracts. The justification package also includes short 
statements about how programs are believed to meet each of the six 
statutory criteria. Weapon system acquisition program officials prepare 
the package for subsequent reviews by military service acquisition 
commands, service headquarters offices, and OSD offices responsible for 
program policy and budget oversight. Approved candidates are submitted 
to the Congress for authorization. 

We discussed DOD's justification process, multiyear contracting 
policies, and management practices with OSD and military service 
officials at each organizational level. We collected historical and 
budget data on approved multiyear programs since fiscal year 1982 and 
tracked more recent multiyear candidates through the budget process. We 
specifically examined nine major programs representing each military 
department: three had completed multiyear contracts, two were beginning 
multiyear contracts, and four programs were just recently authorized by 
the Congress. Several programs have been approved for more than one 
multiyear contract and, collectively, these programs accounted for 17 
of the 59 approvals granted since 1996. 

We found that DOD provides little in the way of supplemental guidance 
to operationalize the statutory criteria by amplifying terms such as 
"reasonable," "substantial," and "stable" and quantifying where 
possible to provide more objectivity and rigor to the multiyear review 
process. Guidance for the most part restates the statutory criteria and 
establishes formats for submitting budget justification materials, but 
does not provide much elucidation for interpreting and applying the 
criteria and establishing internal evidence standards for demonstrating 
criteria are met.[Footnote 6] From our review of justification packages 
and our discussions with DOD officials responsible for generating and 
reviewing multiyear justification packages, we determined that 
reviewers interpret and apply criteria differently and that the methods 
and data used to compute contract costs and savings and providing 
evidence to document program stability vary in quality and 
sophistication. 

For example, officials we talked to at all levels of the review process 
had different ideas and perspectives on what constituted substantial 
savings when applying the criterion. An official in the Navy, for 
example, expected programs to project at least 10 percent savings, but 
would consider candidates under that level. Some Army officials wanted 
to see 10 percent savings or hundreds of millions of dollars. An OSD 
official applied a "rule of thumb" of 4 to 5 percent. He said that 
programs under that amount would typically be more closely evaluated to 
ensure they were viable, but that programs over that amount would 
generally receive a less detailed check of reasonableness and to ensure 
paperwork requirements were met. An Air Force official told us that a 5 
percent savings level should be considered the floor for a genuinely 
viable candidate for multiyear and a 10 percent savings level 
achievable, although he cautioned that recent statutory changes to 
eligible cancellation ceiling costs will likely have a negative impact 
on future multiyear savings. 

Further, review of the justification packages for the F-22A and the V- 
22, both submitted in the same fiscal year for approval, indicated 
differences in how the design stability criterion was applied. Initial 
operational capability, an important milestone for stability, had been 
declared 2 years prior to requesting multiyear authority on the F-22A, 
while initial operational capabilities for both variants of the V-22 
are not scheduled to be achieved until after multiyear approval was 
granted, at least 2 years later for the Navy's variant. 

Comparing multiyear savings and judging reasonableness of the estimates 
is complicated because the techniques and data used to estimate cost 
savings can vary substantially in form, sophistication, and detail. For 
example, for its first multiyear contract proposal on the C-17A, the 
Air Force simply used the prime contractor's offer to save 5.5 percent, 
a figure that was considered a "management challenge" the contractor 
believed it could meet. We found other instances where it was unclear 
what data was used in formulating the savings estimates. For example, 
the Army's budget justifications for the Bradley Fighting Vehicle and 
Abrams Tank estimated savings at 5 percent and 10 percent, 
respectively, and provided very sketchy details to document how the 
savings estimates were derived and compliance with the stability 
criteria. 

On the other hand, some analyses can be very involved, provide a range 
of estimated savings, and use independent third parties to validate 
data. For example, the Navy's F-18E/F cost estimates and methodologies 
were independently verified by OSD's Cost Analysis Improvement Group. 
For the F-22A multiyear proposal, two defense research centers 
developed contract costs and savings estimates. Several very different 
methodologies were employed, including cost improvement curves based on 
historical actual production costs, production cost estimate models for 
single year contracts less reductions for expected savings, and 
summation of savings initiatives from the prime and subcontractors. 
These studies produced a wide range of potential savings, from $235 
million to $643 million. 

Also, cost and savings estimates in general may be subject to biases 
and other factors that impact their fidelity and reliability. Our 
extensive past body of work on DOD's major acquisitions suggests that 
to gain approval and continued funding for a weapon system, the 
acquisition environment encourages programs to submit overoptimistic 
estimates about a weapon system's readiness for production and to 
underestimate its costs. Systems therefore appear more affordable from 
an investment perspective and can fit within forecasts of available 
funds. These circumstances invariably lead to acquisition programs 
costing substantially more than originally estimated.[Footnote 7] 
Furthermore, prior reports have discussed the importance of using 
present value analysis to account for the time value of money when 
evaluating and comparing costs of alternative annual and multiyear 
contracts. The timing of government expenditures is expected to differ 
with a multiyear contract expected to have relatively more up-front 
costs (to fund EOQ for example) and lower costs in the outyears 
compared to a series of annual contracts.[Footnote 8] Although the 
justification packages are required to have a present value analysis of 
the savings estimates, according to an OSD official and a defense 
research study, the cost savings estimates in then-year dollars are the 
primary estimates used in making cost decisions. Our review of the 
justification packages appear to confirm this because the section of 
the multiyear exhibit highlighting the benefit to the government 
contained only the then-year dollar estimate rather than the present 
value estimate. 

Documentation and Record-Keeping Deficiencies: 

Through our discussions with officials and inspection of records, we 
determined that DOD's review process for the multiyear justification 
budget packages does not adequately capture important information and 
events to document decisions and help ensure that consistent and 
reliable determinations are made regarding multiyear criteria. Once 
approved, OSD officials stated few records are kept on multiyear 
programs regarding how they determined whether multiyear candidates met 
the six statutory criteria. According to OSD and service officials, 
much of the discussion on a program proposed for multiyear should have 
already taken place during regular executive-level reviews of major 
weapon systems and been agreed upon before the multiyear justification 
package is reviewed for submission in the budget request. Review of the 
justification package then essentially becomes a paperwork formality 
rarely involving any surprises. Also, we found programs can be proposed 
"out of cycle" with the President's Budget submission--as in the case 
of the F-22A, and may not be included in the budget details that could 
affect the review path the multiyear candidate takes to obtain 
approval. Without maintaining records that document decisions and the 
data supporting them, it is difficult to ensure the quality and 
comprehensiveness of stakeholder reviews based on the criteria, 
fidelity of the data used, and supporting rationales for decisions. 

Finally, officials at every level of the multiyear justification 
process--from program offices, through higher headquarters, and on to 
primary OSD action offices--indicated that they recently were appointed 
to their current positions or the person responsible during the 
multiyear justification process was no longer in that position. We 
believe this contributed to "knowledge gaps," historical record-keeping 
deficiencies, and differences in interpretation and application of 
multiyear decision criteria. Turnover implies loss of experience and 
corporate knowledge; in this environment, improved and more definitive 
guidance and retention of comprehensive historical records is even more 
important. 

Actual Results on Some Completed Contracts Were Much Different Than 
Predicted in Budget Justifications: 

Implementing the statutory criteria requires realistic estimates of 
multiyear and annual contract costs. This requirement provides fidelity 
to savings projections and allows for accurate estimates of funding 
requirements over the life of the multiyear contract. We reviewed cost 
performance and results on completed multiyear contracts for the Air 
Force's C-17A Globemaster transport, the Navy's F/A-18E/F Super Hornet 
fighter, and the Army's Apache Longbow helicopter.[Footnote 9] Although 
the precise impact of multiyear contracting is difficult to determine, 
our analysis shows that these programs--presumably approved based on 
their demonstrated stability--experienced substantial changes during 
contract execution. These changes significantly increased unit costs 
and drove up total funding requirements much beyond the estimates 
submitted to Congress in the budget justification materials. Each was 
also impacted by contract provisions and changes in business 
conditions. 

Cost Increases Lead to Questions about Realism of Budget Estimates Used 
to Justify Multiyear: 

We found that unit cost growth on these programs ranged from 10 to 30 
percent more than projected by the budget justification data. Table 2 
shows the growth in unit and total contract costs. We also found that, 
for two of the three programs, actual multiyear contract costs exceeded 
the original estimates for annual contract costs. The third program, 
the F/A-18E/F, came in below annual estimates, but also bought fewer 
systems than planned. This reduction in quantity would have also likely 
decreased annual costs had that alternative been selected. 

Table 2: Unit Cost Growth for Three Multiyear Contracts: 

Weapon system: C-17A; 
Original savings projection: 5.5%; 
Original quantity: 80; 
Final quantity: 80; 
Original MYP contract estimated: $14,354.0; 
Original unit cost: $179.43; 
Final MYP contract value: $16,614.0; 
Final unit cost: $207.68; 
Unit cost increase (percent): 15.7%. 

Weapon system: F/A-18E/F; 
Original savings projection: 7.4%; 
Original quantity: 222; 
Final quantity: 210; 
Original MYP contract estimated: 8,840.8; 
Original unit cost: 39.82; 
Final MYP contract value: 9,221.8; 
Final unit cost: 43.91; 
Unit cost increase (percent): 10.3%. 

Weapon system: Apache; 
Original savings projection: 7.6%; 
Original quantity: 232; 
Final quantity: 232; 
Original MYP contract estimated: 1,596.4; 
Original unit cost: 6.88; 
Final MYP contract value: 2,078.8; 
Final unit cost: 8.96; 
Unit cost increase (percent): 30.2%. 

Source: GAO analysis of MYP justification packages and selected 
acquisition reports. 

[End of table] 

Substantial cost increases for completed multiyear contracts on the 
three programs meant that Congress had to eventually provide 
considerably more funding than originally budgeted. We do not know how 
cost growth affected the level of savings achieved, if any, because we 
do not know how an alternative series of annual contracts would have 
fared. In comments on a draft of this report, DOD officials stressed 
that cost growth due to labor and material price escalation under a 
multiyear contract would likely have also occurred under an alternative 
series of annual contracts. The final MYP contract values in table 3 
also include price increases resulting from engineering design changes 
made to the baseline weapon system. Although these factors may limit 
the ability to make inferences about the level of savings achieved, a 
case could be made that multiyear savings and costs did not materialize 
as presented in the multiyear justification materials. 

As discussed earlier, our past body work suggests that defense 
acquisition programs are prone to underestimating costs and overstating 
readiness. While this tendency would apply to annual as well as 
multiyear contracts, it is arguably more problematic for multiyear 
contracts because of the government's increased exposure to risk over 
multiple years. DOD officials agreed with us that multiyear contracting 
should be held to a high standard because of its special requirements, 
funding commitments, and risks. 

Impacts on Cost and Performance from Contract Provisions and Other 
Factors: 

We also collected information on multiyear contract provisions for the 
three programs with completed multiyear contracts and for the recently 
awarded F-22A contract. Each of these programs awarded a fixed-price 
contract for the multiyear procurement, but they were not always firm- 
fixed-price contracts, which typically entail the least risk to the 
government. The multiyear contracts contained standard provisions that 
provided flexibility to increase or decrease costs based on inflation, 
labor rate changes, and/or material cost fluctuations.[Footnote 10] The 
multiyear contracts also included provisions for early cancellation, 
quantity variations, and/or design changes. In some cases, the 
government waived provisions for cost and pricing data, which according 
to officials decreased the government's insight. Figure 2 below shows 
the frequency of the various contract provisions in our case studies. 

Figure 2: Contract Provisions That Impacted Case Study Multiyear 
Procurements: 

[See PDF for image] 

This figure is a table depicting the following data: 

Economic price adjustments: 
C-17A MYP I: [Check]; 
C-17A MYP II: [Check]; 
F/A-18E/F MYP I: [Check]; 
F/A-18E/F MYP II: [Check]; 
Apache MYP I: [Check]; 
Apache MYP II: [Check]; 
F-22A MYP: [Check]. 

Variation in quantity: 
C-17A MYP I: [Empty]; 
C-17A MYP II: [Empty]; 
F/A-18E/F MYP I: [Check]; 
F/A-18E/F MYP II: [Check]; 
Apache MYP I: [Empty]; 
Apache MYP II: [Empty]; 
F-22A MYP: [Empty]. 

Engineering changes: 
C-17A MYP I: [Check]; 
C-17A MYP II: [Check]; 
F/A-18E/F MYP I: [Check]; 
F/A-18E/F MYP II: [Check]; 
Apache MYP I: [Check]; 
Apache MYP II: [Check]; 
F-22A MYP: [Check]. 

Cost/pricing data waivers: 
C-17A MYP I: [Check]; 
C-17A MYP II: [Check]; 
F/A-18E/F MYP I: [Empty]; 
F/A-18E/F MYP II: [Check]; 
Apache MYP I: [Check]; 
Apache MYP II: [Check]; 
F-22A MYP: [Empty]. 

Cancellation ceiling (unfunded): 
C-17A MYP I: [Check]; 
C-17A MYP II: [Check]; 
F/A-18E/F MYP I: [Check]; 
F/A-18E/F MYP II: [Empty]; 
Apache MYP I: [Check]; 
Apache MYP II: [Check]; 
F-22A MYP: [Check]. 

Source: DOD contract file information and officials. 

[End of figure] 

Case Studies Provide Details on Meeting Cost and Program Expectations: 

The three case studies provide insight into multiyear contracting 
expectations and realities and the internal and external factors that 
affected actual execution. Each case raises some questions about the 
accuracy of cost and savings estimates used to justify multiyear 
procurement and the degree to which requirements, design, and funding 
were stable. 

C-17A Globemaster: Risky Strategy Affected Multiyear Funding Stability: 

The estimated savings for the first C-17A Globemaster multiyear 
contract was simply a percentage amount submitted by the contractor. 
For the second multiyear contract, Air Force officials assumed the same 
percent of savings and added additional savings based on the use of a 
controversial funding strategy. This strategy relied on incremental 
funding, advanced buys of parts, and large potential cancellation 
liability to maintain a production schedule of 15 aircraft per year 
even though all of these aircraft had not been fully funded. The 
unfunded liability to the Air Force had the contract been canceled 
eventually grew to $1.5 billion. Concerned that this incremental 
funding strategy violated DOD's full-funding policy and could 
potentially violate the Anti-Deficiency Act, Congress increased C-17A 
procurement funding a total of $745 million in fiscal years 2003 and 
2005 to fully fund all aircraft. In annual DOD appropriations acts, 
Congress also has prohibited incremental funding of multiyear 
contracts.[Footnote 11] 

In addition, the costs for both C-17A multiyear contracts were affected 
by economic price adjustments. On the first multiyear, overhead costs 
were significantly increased as a result of a merger between two major 
defense contractors. The Air Force subsequently paid $150 million to 
cover cost increases resulting from the merger and another $50 million 
to remove the clause from the contract. By the end of the multiyear 
contract, unit costs had increased 15 percent. The multiyear 
justification materials submitted to Congress supported a plan to buy 
80 aircraft at an average cost of $179 million; instead the Air Force 
eventually paid about $207 million per aircraft. During the second 
multiyear contract, the contractor made large contributions to its 
pension fund, which triggered the price adjustment clause and resulted 
in a potential cost increase of over $530 million. The Air Force is in 
the process of restructuring the contract to reduce this amount. 

The Air Force also awarded two multiyear contracts for the C-17A 
engine, the F117. These procurements appear to have been successful 
with demonstrated stability during the multiyear period and price 
breaks based on the multiyear contract. The F117 engine is a 
commercially available engine with a stable design and manufacturing 
process. There were no engineering or design changes; no advanced 
procurement or EOQ requirements; and no cancellation ceilings 
associated with either contract. The only potential cost risk was the 
economic price adjustment clause, but officials stated that the actual 
adjustments were not exclusively in the contractor's favor. According 
to the program office, the first multiyear contract resulted in a 
savings of 10 percent, more than it had originally expected. Savings 
from the second multiyear contract are consistent with the original 
estimate. 

F/A-18E/F Super Hornet: Design and Requirement Changes Impact Costs: 

For its first F/A-18E/F multiyear contract, the Navy did not award a 
firm-fixed-price contract because the program was early in the 
production phase and there were still ongoing design development 
efforts on the airframe. During the contract period, the economic price 
adjustment clause resulted in the Navy paying an additional $378 
million because of labor rate and material cost increases. The first 
multiyear contract also included a variation-in-quantity clause that 
permitted an upward or downward adjustment of six aircraft per year. 
Annual quantities and the specific mix of buys between the two models 
changed more than once during the multiyear period. By the end of the 
multiyear contract, the number of aircraft procured had dropped from 
222 to 210 aircraft and the average unit costs had increased by 10 
percent, compared to the budget estimates. 

In a May 2000 report, we had questioned whether the Navy was ready to 
enter into its first multiyear contract for full-rate production. 
Deficiencies identified during operational testing had not been 
corrected, and to avoid costly retrofitting and redesign, we believed 
that corrections should be made and tested before entering into the 
contract.[Footnote 12] 

The Navy proposed buying another 210 Super Hornets on a second 
multiyear contract, but later changed the requirement to procure 154 
Super Hornets and 56 of the new E/A-18G Growler, an electronic attack 
variant still in product development. The Navy's total requirement for 
the Super Hornet had been reduced, and the new Growler was needed to 
replace aging EA-6B aircraft in the electronic attack mission. The 
follow-on multiyear also included a variation-in-quantity clause, but 
this time it only covered upward adjustments. Multiyear costs and 
funding were further impacted by the economic price adjustment clauses. 
As in the C-17A's case, the F/A-18E/F multiyear contract was affected 
by the contractor's large pension fund contribution. The Navy estimated 
that it could have been obligated to pay over $1 billion, which is 
nearly the same as the amount of cost savings originally estimated to 
justify the multiyear contract award. However, the Navy renegotiated 
the terms of the clause and restructured the contract to bring the 
price adjustment down to about $152 million. 

Longbow Apache: Modifications and Unplanned Work Increased Unit Costs: 

The Army's Longbow Apache helicopter experienced significant cost 
increases during both its multiyear procurements. Army officials stated 
that increases were largely because of aircraft modifications and 
unplanned work. These modifications included a voice data recorder and 
an improved rotor blade assembly that would enhance operational safety. 
Contract costs were also increased by additional unplanned work. 
Program officials explained that it is very difficult to predict the 
condition of fielded aircraft when they return to be upgraded or 
remanufactured. Along with normal wear and tear, many operational 
aircraft were returned with extensive corrosion and battle damage; 
others had been cannibalized for parts. Remanufacturing these aircraft 
required significantly more effort and funding than originally planned. 
By the end of the first multiyear contract, the Apache's average unit 
cost had increased by 30 percent; at the end of the second multiyear, 
these costs had increased by 25 percent. 

While it may be difficult to predict unusual wear and tear on a system 
and it is common to incorporate new modifications over time, it is 
especially problematic to roll these costs into the multiyear contract 
that had been assumed stable and that had been justified based on 
initial cost estimates without these new add-ons. OSD cost analysts are 
studying this issue to determine the proper accounting for 
modifications under multiyear contracting. 

Estimated Savings for Candidate Programs Are Trending Downward: 

The amount and percentage of savings expected from a multiyear contract 
compared to a series of estimated annual contracts is the most visible 
and perhaps the most critical criteria in the eyes of many 
stakeholders. The savings requirement in definition and application has 
evolved over the years. A threshold level of 10 percent savings 
emphasized during the 1980s was eliminated and replaced with a 
nonquantifiable requirement for "substantial savings" since fiscal year 
1991, allowing wide flexibility in its interpretation.[Footnote 13] 
Although a direct causal link is not demonstrated, our analysis of 
multiyear programs approved by Congress shows that estimated savings 
were on average higher before the substantial savings requirement than 
after. Other factors--lower quantities of modern systems being 
procured, stricter termination liability allowances, and contraction in 
the defense industrial base--may also contribute to decreased estimated 
savings for current and future systems by lessening the benefits from 
large quantity buys and efficient production rates, key attributes of 
multiyear contracts. 

Multiyear Savings Percentages Were Generally Lower after Changes in the 
Statutory Savings Criteria: 

When Congress codified the authority for multiyear procurement 
contracting in December 1981,[Footnote 14] there was no specific 
savings criterion in the law that candidate programs had to meet. 
However, the impetus behind multiyear was provided by DOD studies at 
the time predicting savings averaging 10 to 20 percent. The 10 percent 
figure became a savings benchmark for decision-makers in the early 
1980s when judging the merits of candidate programs. In the late 1980s, 
this benchmark became a threshold requirement for many candidate 
programs as Congress began stipulating a 10 to 12 percent savings 
amount in annual defense authorization acts for selected programs. 

In November 1989, Congress decided to codify the 10 percent savings 
requirement and other conditions and limitations previously imposed on 
an annual basis.[Footnote 15] However, a year later, that threshold was 
struck from the U.S. Code, and the requirement for "substantial 
savings" was substituted after DOD had argued that a rigid threshold 
limited the potential for savings on stable, low risk programs 
projecting lesser savings amounts.[Footnote 16] This substantial 
savings requirement has remained unchanged since November 1990, with 
specific savings requirements stipulated in annual legislation for two 
candidate programs approved by Congress during this time. Figure 3 
illustrates the evolution of the savings requirement. 

Figure 3: Legislative History Timeline for Multiyear Savings 
Requirement: 

[See PDF for image] 

This figure is an illustration of the Legislative History Timeline for 
Multiyear Savings Requirement. The following data is depicted: 

April 1981: 
The “Carlucci Memo” encourages the use of multiyear procurement because 
it could result in average dollar savings of 10 to 20 percent, becoming 
an unofficial MYP savings benchmark. 

December 1981: 
Statutory authority for multiyear procurement codified by Congress. But 
the legislation did not provide any specific savings criteria requiring 
instead that the savings estimates be realistic. 

November 1985: 
Authorization act including a requirement for a specific savings 
percentage is passed by Congress. It stipulated for certain Army 
programs that multiyear cost should be no more than 90 percent of 
annual estimated costs. 

1985 – 1988: 
Defense authorization acts passed during this time each require some 
sort of a savings percentage for multiyear contracts on certain major 
weapon system programs. 

November 1989: 
Congress passes legislation that amends provisions governing the use of 
multiyear contracts by adding a 10 percent savings requirement for all 
multiyear procurement contracts proposed for major weapon systems. 

November 1990: 
Congress passes legislation that strikes out the 10 percent savings 
requirement put in place the year before and replaces it with a 
“substantial savings” requirement. 

1991 - Present: 
The substantial savings requirement remains unchanged since it was 
first codified in November 1990. 

Source: GAO analysis. 

[End of figure] 

Our analysis of estimated savings for approved multiyear programs 
determined that the average savings level trended lower after the 
substantial savings criterion was adopted. Although programs have been 
approved during both eras over a wide range of savings below and above 
10 percent, the change in the law provided decision makers the 
flexibility to propose and approve candidate programs since fiscal year 
1991 with lower savings estimates on average compared to the 1980s. As 
previously discussed, an unofficial rule of thumb for savings normally 
expected is now down to 4 to 5 percent. 

Figure 4 illustrates the range of savings and general trend as the 
savings requirement evolved. It suggests that a shift did occur since 
the change to substantial savings in 1991. A larger proportion of 
multiyear programs with estimated savings of less than 10 percent were 
approved after 1991. On the other hand, candidate programs approved 
prior to 1991 show a larger proportion of savings of 10 percent or 
more. This finding is supported by a defense consultant study that 
calculated estimated savings averaged 13 percent for candidate programs 
from 1982 to 1989.[Footnote 17] 

Figure 4: Range of Estimated Savings for Multiyear Programs: 

[See PDF for image] 

This figure is a stacked bar graph depicting the following data: 

Estimated savings percentage: less than 5; 
1991-present (after the "substantial savings" requirement): 7 MYP 
programs; 
1982-1990 before the change in the savings requirement): 3 MYP 
programs; 
Total: 10 MYP programs. 

Estimated savings percentage: 5-10; 
1991-present (after the "substantial savings" requirement): 19 MYP 
programs; 
1982-1990 before the change in the savings requirement): 16 MYP 
programs; 
Total: 35 MYP programs. 

Estimated savings percentage: 10-15[A]; 
1991-present (after the "substantial savings" requirement): 8 MYP 
programs; 
1982-1990 before the change in the savings requirement): 27 MYP 
programs; 
Total: 35 MYP programs. 

Estimated savings percentage: greater than 15; 
1991-present (after the "substantial savings" requirement): 4 MYP 
programs; 
1982-1990 before the change in the savings requirement): 10 MYP 
programs; 
Total: 14 MYP programs. 

Source: GAO analysis of DOD budget and prior GAO work. 

[A] The '10-15' category includes those MYP candidates with estimated 
savings of exactly 10 percent. 

[End of figure] 

Other Factors Affecting Estimated Savings: 

In addition to the revised savings requirement, other factors may also 
be impacting the level of savings for current and future multiyear 
programs: 

* Smaller quantities procured. The higher cost for modern weapon 
systems and changes in required force structure has resulted in 
generally smaller procurement quantities for new systems compared to 
predecessor systems. Smaller quantities of systems bought under 
multiyear contracts may not provide the same opportunity to achieve 
savings through such mechanisms as EOQ buys of parts and materials. 
Past multiyears, on the F-16 Falcon aircraft and Black Hawk 
helicopters, for example, procured hundreds of systems. In contrast, 
the more recent multiyear procurement of the F/A-18E/F Super Hornet 
procured less than half the F-16 quantity. Analysis of the data on MYP 
candidate programs that we collected and summarized in table 3 below 
showed that the median multiyear procurement quantity for aircraft MYP 
candidate programs declined over 40 percent from the 1980s to the 
present era with a concurrent decline in average savings. With fewer 
aircraft procured during the multiyear period, savings from economic 
buys and optimized production is typically smaller. 

Table 3: Characteristics of Aircraft MYP Candidate Programs: 

Median MYP quantity; 
1980s: 252 units; 
1990s-2000s: 140 units. 

Median MYP savings percentage; 
1980s: 10.7%; 
1990s-2000s: 7.2%. 

Source: GAO analysis of DOD budget justification materials and prior 
GAO work. 

[End of table] 

* Cancellation liability changes. If a multiyear contract is canceled, 
the government is liable for reimbursing the contractor for its 
incurred costs up to a negotiated cancellation ceiling typically in the 
contract. Until recently, DOD has been able to include both recurring 
and nonrecurring costs.[Footnote 18] Multiyear contracts awarded during 
the past 25 years have included a cancellation liability, but the 
cancellation ceiling for the C-17A multiyear contract awarded in 2002 
was considered very large with a potential liability of more than $1.5 
billion. Some members of Congress were concerned by its size and 
concluded that this large liability inappropriately committed the 
government to a production schedule for which funding had not been 
appropriated. As a result, for the past several years Congress has 
limited the cancellation liability for multiyear contracts to 
nonrecurring costs only.[Footnote 19] Some DOD officials expect this 
change to limit savings on current and future multiyear contracts as 
contractors choose not to bear financial risks previously borne by the 
government. A major source of multiyear savings is the EOQ buys, and 
without including these kinds of recurring costs in the cancellation 
ceiling, fewer contractors may buy supplies and materials up front in 
bulk to limit their risks should the multiyear contract be canceled 
early. 

* Declining competition. Some DOD and defense research center officials 
believe that the consolidation and contraction of the defense industry 
and resulting decline in competition and contraction among vendors and 
suppliers, make it harder to wring savings from EOQ buys. For example, 
an OSD official stated that because the F-22A was originally designed 
well over a decade ago, it is now experiencing diminishing 
manufacturing sources on many components as well as parts and equipment 
obsolescence. Similarly, a defense research center official believes 
that diminishing manufacturing sources negatively impacted the 
multiyear savings potential for the F-22A. 

Multiyear Contracting Results Are Uncertain: 

DOD does not perform post contract assessments of completed multiyear 
contracts to validate actual results and determine cost effectiveness 
of its investments. Some prior studies, including GAO work, provide 
some limited, but inconclusive insights into multiyear results and 
benefits. OSD and the military services do not maintain adequate, 
comprehensive records on historical and current multiyear contracts 
that could facilitate better tracking and provide perspective for 
future multiyear efforts. 

DOD Does Not Have a Formal Process for Assessing and Tracking Multiyear 
Contract Results: 

DOD does not have a formal process for assessing the cost and 
performance of completed multiyear contracts. According to DOD 
officials, once a program is approved for multiyear, they do not track 
subsequent performance nor validate actual results against expectations 
established in the budget justification submissions. Also, they do not 
have an official method to capture and share lessons learned with other 
programs considering multiyear contracts that could improve prospects 
for successful outcomes. 

However, we did find that individual program offices may make efforts 
to ascertain benefits and learn lessons that can be applied to future 
multiyear submissions, but it appears that these efforts are isolated. 
Based on prior multiyear experiences, F/A-18E/F Super Hornet program 
officials decided to require cost and pricing data to better inform 
cost estimates for their next multiyear application. Also, they based 
expected savings on cost reduction initiatives rather than EOQ buys 
because they believed these initiatives had a better return on 
investment. Similarly, the results of our case studies discussed 
earlier demonstrate that assessing actual results can glean valuable 
information about contract costs and performance that can be used to 
improve future multiyear outcomes. 

Prior Studies of Multiyear Cost Effectiveness Show Limited and 
Inconclusive Results: 

Some attempts to assess historical multiyear performance have been 
made, but validating actual savings is elusive. According to DOD and 
defense research center officials as well as the studies they 
conducted, calculating the actual cost savings from the use of a 
multiyear contract and comparing results to original expectations is 
very problematic for several reasons: (1) multiyear cost and other 
program data is unavailable; (2) lack of comparable data on costs of 
annual contracts; and (3) original assumptions change from the 
justification package, such as design modifications and variations in 
buy quantities, labor, and material rates. 

Recent studies by two defense research centers attempted to gain a 
historical perspective on actual multiyear savings achieved for past 
contracts in order to provide context as to the relative level of 
expected savings for the F-22A multiyear proposal. In reports, both 
centers noted that the government does not collect or save data needed 
to do a detailed analysis and that, once programs are approved and 
implemented, important assumptions on which original savings estimates 
were based often changed. 

In particular, since a series of annual contracts are not executed for 
the same procurement once a multiyear strategy has been adopted, 
comparisons of actual multiyear costs can only be compared to 
hypothetical estimates of annual contracts. Further, changes that occur 
in a multiyear contract environment can also occur in an annual 
contract environment, and the exact effect on the actual costs for the 
annual contracts is unknown. In reviewing the literature, one center 
noted that it found "very few examples of serious and methodologically 
credible attempts to validate claimed savings and savings percentages 
after the fact once programs had been completed," and none that 
produced, in the center's opinion, definitive findings.[Footnote 20] 

In the case of the F/A-18E/F program, researchers from one center felt 
they had more data available on this multiyear contract than for 
others, and they attempted to validate actual savings. They concluded 
that while the available data supported savings in the neighborhood of 
original estimates, a definitive answer would require a more detailed 
analysis of EOQ data and cost reduction initiatives. Similarly, in our 
1988 report on the first F-16 multiyear procurement, we concluded that 
savings were likely achieved from EOQ buys in the order of magnitude 
expected, but we were unable to make definitive conclusions about total 
savings achieved.[Footnote 21] 

Inadequate Records Maintained on Current and Historical Multiyear 
Programs: 

Undergirding attempts to track and assess multiyear performance is the 
quality and sufficiency of data. As discussed earlier in this report, 
we found that DOD at all organizational levels does not keep adequate 
records on multiyear programs to document stakeholder reviews and the 
empirical evidence used to justify multiyear contracts. OSD Comptroller 
officials told us that, while they are the final authority on approving 
multiyear candidates and the ultimate owner of the review process, they 
do not track multiyear packages through the approval process and after 
the final decision has been made to submit it to the Congress. DOD has 
no comprehensive, central information system that records the status of 
multiyear candidates in-process, candidates that have been approved or 
disapproved, detailed information on how multiyear criteria were 
applied, or information on specific criteria contained in the final 
justification package. 

DOD is not required to, nor does it maintain a central database of 
historical or ongoing multiyear contracts. In response to our 
inquiries, the OSD office ultimately responsible for the multiyear 
procurement review process was unable to provide us the justification 
packages for over half of the programs approved by Congress since 1992, 
including very recent submissions. To obtain more complete and accurate 
data that could be used to track performance and conduct trend 
analyses, information must be compiled from many different sources, 
including budgets, program office files, contractor studies, and 
contracting databases. For example, our efforts to identify and track 
multiyear contract information contained in two major federal 
databases[Footnote 22] were only partially successful. Some valuable 
multiyear contract information was readily available through these 
sources. However, the type of data stored and storage format--as well 
as issues pertaining to reliability, consistency, and comparability-- 
limited their current usefulness in tracking and evaluating multiyear 
contracts. However, with some improvements particularly with 
reliability, these databases could support future studies. 

Conclusions: 

The statutory criteria for approving the multiyear procurement of major 
DOD weapon systems clearly establish requirements to limit multiyear 
authorization to stable, low risk programs with realistic cost 
estimates and reasonable expectation for savings. To move forward 
otherwise is to accept significant risks with little chance of reward. 
However, DOD does not have an adequate process with controls in place 
to ensure multiyear candidates meet all the criteria and are supported 
by sufficient empirical evidence. Inconsistent application of criteria, 
questionable cost and savings estimates, and inadequate documentation 
increase potential for approving inappropriate, unstable multiyear 
programs and incurring costly, poor outcomes when plans go awry and 
conditions change. Improving guidance, ensuring decisions are informed 
by knowledge, and maintaining better records are critical needs, as 
well as important tools for retaining corporate memory given frequent 
turnover of personnel at all levels of the justification review 
process. It is not possible to calculate accurately the cost to 
taxpayers that has resulted from these conditions, but the lack of a 
disciplined and rigorous process that demands knowledge about stability 
and costs provides potential for significant waste of taxpayer dollars. 

Furthermore, DOD does not track and evaluate actual performance on 
multiyear contracts for major DOD weapon systems. Once a contract is 
awarded for a multiyear program, little effort is made to collect data 
and assess actual results to compare performance against original 
expectations and to validate savings and other benefits achieved. 
Assessing results could provide valuable insights and lessons learned 
on prior experience and identify opportunities to improve future 
multiyear procurements. Not having a clear picture of actual 
performance further emphasizes the criticality of getting it right up 
front by ensuring only appropriate programs go to Congress for 
approval. 

Therefore, despite a long history and substantial funding for major DOD 
weapons system multiyear contracts, DOD does not know whether it has 
gotten a reasonable return on its investments for the extra risks 
incurred. Some concerns noted in this report, such as the practice of 
understating costs and overselling benefits, apply also to annual 
contracts, but the standards should be higher for multiyear contracts 
because of the larger up-front investments required and the 
government's exposure to risk should the program fail or be 
substantially changed. Strengthening both the front end of the process-
-identifying and justifying good candidates--and the back end-- 
assessing results and gleaning lessons learned from completed 
contracts--can help ensure costs and risks are adequately balanced for 
new multiyear programs and improve future outcomes. Underpinning both 
ends, it is important to capture and make available essential data on 
multiyear decisions and subsequent performance that can be readily 
accessed by stakeholders and prospective users of multiyear procurement 
authority. 

Recommendations: 

To improve the outcomes of the multiyear justification reviews of major 
DOD weapon systems, the Secretary of Defense should direct appropriate 
offices within the Under Secretary of Defense (USD) (Comptroller) and 
USD (Acquisition, Technology & Logistics) to: (1) improve and expand 
guidance provided to military services to better define multiyear 
decision criteria for major DOD weapon systems and to facilitate more 
consistent, objective, and knowledge-based evaluations of these 
multiyear candidates within DOD; (2) establish a process for third 
party validation of the costs and savings data submitted for such 
candidate programs; and (3) implement a central database for 
maintaining historical records and for effectively monitoring and 
tracking major DOD weapon system multiyear procurements, to include 
documenting the specific decisions made by stakeholders and their 
rationales for decisions. 

To provide lessons learned for informing and improving future major DOD 
weapon system multiyear candidate programs and to ensure DOD is earning 
a sufficient return on its investments in multiyear contracts for major 
DOD weapon systems, the Secretary of Defense should direct that the 
responsible military service, in conjunction with appropriate elements 
within OSD, conduct after-action assessments at the conclusion of all 
multiyear contracts used to procure major DOD weapon systems to 
determine their effectiveness in achieving predicted benefits while 
managing associated risks. These assessments should identify major 
deviations, if any, between the unit costs predicted in the multiyear 
justification package and the unit costs actually incurred. The 
assessments should also substantiate--to the extent practicable-- 
savings achieved and identify reasons and causes contributing to 
overall performance results and attempt to isolate those issues 
peculiar to the multiyear contract from those that would likely have 
also affected annual contracts if a multiyear strategy had not been 
employed. Internally, DOD should use the results of these assessments 
to provide lessons learned to both industry and the government that can 
help inform and lead to better and more supportable decisions on future 
multiyear candidate programs. 

Agency Comments and Our Evaluation: 

DOD provided us with written comments on a draft of this report. The 
comments appear in appendix II. DOD also separately provided technical 
comments which we reviewed and incorporated as appropriate. In written 
comments, DOD concurred with our two recommendations to improve 
guidance and to implement a central database for maintaining records 
and tracking multiyear programs. DOD partially concurred with the other 
two recommendations. 

DOD partially concurred with our recommendation to establish a process 
for third party validation of the costs and savings data submitted for 
candidate programs. In its comments, DOD stated that independent third 
party validations of cost and savings are done on selected programs 
and, in developing new guidance in response to our first 
recommendation, would consider whether the benefits of requiring 
validation on all programs warrant the delays and costs of validation. 
Our review of five new proposals and six approved multiyear contracts 
found only one such instance, and our discussions with service and OSD 
officials show that third party validations are rare. We believe that 
independent third party validations would result in more accurate and 
comprehensive cost and savings information critical to congressional 
and DOD decision making on multiyear candidates. Our review identified 
inconsistent practices in preparing and reviewing multiyear proposals 
and varying degrees of quality and completeness in the initial cost and 
savings estimates made by the weapon system program offices. An 
independent third party check would help ensure that appropriate 
multiyear candidates are submitted to the Congress for approval. 

DOD also partially concurred with our recommendation that after-action 
assessments be conducted to provide lessons learned for informing and 
improving future multiyear candidate programs. In its comments, DOD 
agreed that after-action reports may be of value for certain multiyear 
programs, but questioned the value for all programs because (1) the 
extensive time before the assessment results are known and can be 
applied; and (2) the difficulty in determining actual savings. The 
intent of the recommendation is to learn lessons that can be applied to 
strengthen future multiyear proposals and improve their prospects for 
success. This in formulation is not time-bounded as DOD has contracted 
for studies that drew useful lessons from programs many years earlier. 
Collecting and distributing data on lessons learned would provide a 
continuing database of knowledge for future programs. Furthermore, 
while we recognize difficulties and constraints in calculating actual 
savings, this does not preclude the department from making good faith 
efforts that can provide valuable, albeit imperfect information. Also, 
after-action assessments include more than savings calculations. As 
pointed out in the report, one program assessed itself and identified 
important contractual features and other factors that it used to 
improve subsequent multiyear proposals. We note that the practice of 
doing after-action reports is widespread in the department and used for 
many different kinds of activities, including military contingency 
operations and logistics functions, and that these efforts provide 
planners and decision makers with critical lessons learned for applying 
to and improving future actions. 

We are sending copies of this report to the Secretary of Defense; and 
the Secretaries of the Air Force, Army, and Navy; and the Director of 
the Office of Management and Budget. We will provide copies to others 
on request. This report will also be available at no charge on GAO's 
Web site at [hyperlink, http://www.gao.gov]. Key contributors to this 
report were Bruce Fairbairn, Assistant Director; Noah Bleicher; Matthew 
Drerup; Mary Jo Lewnard; Rae Ann Sapp; and Bob Swierczek. 

If you have any questions about this report or need additional 
information, please call me at (202) 512-4841 (sullivanm@gao.gov). 
Contact points for the offices of Congressional Relations and Public 
Affairs are located on the last page of this report. 

Signed by: 

Michael J. Sullivan: 
Director, Acquisition and Sourcing Management: 

[End of section] 

Appendix I: Scope and Methodology: 

Our review was limited to major DOD weapon systems that have received 
congressional approval in annual defense authorization and/or 
appropriations acts to award a multiyear contract and meet the 
statutory requirements identified in 10 U.S.C. § 2306b. Work was 
performed at the Offices of the Secretary of Defense and the three 
military service headquarters (Navy, Army, and Air Force), in 
Washington, D.C.; Naval Air Systems Command, in Patuxent River, 
Maryland; Aviation Missile Command, in Huntsville, Alabama; 
Aeronautical Systems Center, at Wright-Patterson Air Force Base, Ohio; 
Institute of Defense Analysis (IDA), Alexandria, Virginia; and RAND 
National Defense Research Institute, Arlington, Virginia. 

To evaluate DOD's multiyear review process, we compiled a list of the 
all the candidate programs approved by Congress to use a multiyear 
contract by examining DOD authorization and appropriations acts going 
back to fiscal year 1982 when the statutory language in 10 U.S.C § 
2306b was first enacted. We then reviewed DOD multiyear justification 
packages submitted to Congress in recent defense budgets, identified 
statutory criteria authorizing the use of multiyear procurement, and 
considered regulatory policies and procedures used within the Services 
and at OSD to prepare and evaluate multiyear justification packages. We 
discussed with officials at acquisition program offices and at higher 
command review levels how they interpreted and applied the statutory 
criteria and guidance to evaluate the appropriateness and feasibility 
of multiyear candidates. We reviewed DOD and congressional actions on 
recent multiyear candidates and examined specifically two major 
programs recently approved for multiyear contracting--the F-22 Raptor 
and V-22 Osprey--to illustrate how this process works and address some 
questions raised about the appropriateness of these candidates and data 
used in the justification packages. For these systems, we extensively 
drew upon GAO's work in prior and ongoing engagements. 

We conducted limited case studies for selected multiyear aircraft 
contracts to assess outcomes and the internal and external events 
affecting performance. Because our sample of DOD aircraft multiyear 
contracts was not randomly selected, we cannot project our findings to 
other programs. These case studies included three major DOD weapon 
systems--C-17A Globemaster, F/A-18E/F Super Hornet, and the Apache 
Longbow Helicopter. These aircraft programs have fully executed at 
least one multiyear contract in the recent past and also have ongoing 
follow-on multiyear contracts. We reviewed the multiyear proposal 
packages submitted to Congress, annual budget information, Selected 
Acquisition Reports, contract file documentation, and information in 
DOD contract databases for multiyear contracts awarded by these 
programs. We calculated unit cost changes and identified key 
programmatic and environmental changes impacting the execution of the 
multiyear programs and compared these to the original projections in 
their multiyear justification packages. To identify the characteristics 
of multiyear contracts and how they affect the costs, risks, and 
savings for selected multiyear aircraft contracts, we reviewed contract 
file documentation and information on the DOD DD 350 Individual 
Contract Action Report (ICAR) database and the Federal Procurement Data 
System-Next Generation (FPDS-NG). As part of the case study approach, 
we reviewed the types of contracts, contract clauses, and other 
contract modifications to determine how they affected the unit costs 
under a multiyear contract. Further, we discussed the multiyear 
contracts included in our limited case study approach and the F-22A 
program's August 2007 multiyear contract award with program officials 
to help us assess the effects that specific contract provisions have on 
unit costs during contract execution. 

To research the legislative history underlying the adoption and 
subsequent repeal of the 10-percent savings requirement and the current 
requirement that defense multiyear procurement contracts achieve 
"substantial savings," we reviewed the evolution of multiyear savings 
criteria in early defense initiatives, the fiscal year 1982 
codification, subsequent amendments, and the savings criteria contained 
in annual authorization and appropriations acts subsequent to the 
granting of multiyear authority. To make comparisons in average and 
median levels of estimated savings on multiyear candidate programs as 
the criteria for awarding a multiyear contract changed, we reviewed 
savings estimates in the budget justification packages for multiyear 
candidates submitted to Congress since 1982 and past GAO reviews of 
multiyear candidates conducted in the 1980s. We were able to obtain 
savings estimates for approximately 94 of the 141 approvals granted by 
Congress to award a multiyear contract. 

To determine the extent to which DOD tracks performance and validates 
savings and other benefits actually achieved by multiyear contracts, we 
evaluated the kind and extent of cost data and program information 
maintained at the services and OSD, and how they use these data to 
determine whether predicted savings and other benefits were actually 
achieved by multiyear contracting. We also reviewed DOD policies and 
guidance for estimating and validating multiyear savings, and discussed 
with DOD officials practices used to estimate and monitor multiyear 
savings. We discussed with DOD officials record-keeping and management 
oversight requirements and reasons why they do not have a formal 
process for assessing multiyear results. We reviewed two recent major 
studies done by defense research firms that summarized estimated 
savings on historical programs and performed case studies on selected 
programs to identify key events and practices that affected ultimate 
performance. We discussed with IDA and RAND officials their cost 
estimating techniques and assumptions used in their F-22A multiyear 
studies that supported the planned multiyear contract. We reviewed and 
summarized their attempts to validate savings and other benefits from 
prior multiyear programs. We also reviewed prior GAO work on selected 
weapon systems and the results of our work during the 1980s when 
Congress regularly asked us to review DOD's multiyear candidates. 

We conducted this performance audit from June 2007 to February 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Defense: 

Office Of The Under Secretary Of Defense: 
Acquisition Technology And Logistics: 
3000 Defense Pentagon: 
Washington DC 20301-3000: 

February 1, 2008: 

Mr. Michael Sullivan: 
Director, Acquisition and Sourcing Management: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Mr. Sullivan: 

This is the Department of Defense (DoD) response to the GAO Draft 
Report, GAO-08-298, "Defense Acquisitions: DoD's Practices and 
Processes for Multiyear Procurement Should Be Improved," dated December 
21, 2007, (GAO Code 120649). 

The Department acknowledges receipt of the draft report. DoD's written 
comments are enclosed. 

Signed by: 

Shay D. Assad: 
Director, Defense Procurement and Acquisition Policy: 

Enclosure: As stated: 

GAO Draft Report Dated December 21, 2007: 
GAO-08-298 (GAO Code 120649): 

"Defense Acquisitions: DOD's Practices And Processes For
Multiyear Procurement Should Be Improved" 

Department Of Defense Comments To The GAO Recommendation: 

DOD General Comments. The department is strongly committed to multiyear 
(MY) procurements which (1) offer substantial savings through improved 
economies and efficiencies in production processes, and better 
utilization of industrial facilities; (2) enhance the attractiveness of 
and competition for Government requirements at the subcontracting 
level; and (3) reduce the administrative burden in the placement and 
administration of contracts. A key factor in the successful use of MY 
procurements is the intelligent selection of the programs. 

In Appendix I: Scope & Methodology, GAO indicated that their review was 
limited to major DOD weapon systems. Accordingly, the department 
suggests that the recommendations only be applied to this category of 
MY procurements, and not to other categories, e.g., procurements of 
energy commodities and energy-related services. 

The description of the "stability of funding" criteria in Table 1 of 
the report should be revised to more accurately reflect the intent of 
statute (10 USC 2306b(a)(3)). The phrase "the head of the agency will 
request funding" rather than the phrase "sufficient funding will be 
provided by DOD" better reflects the statute. Likewise, the description 
of the "stable design" criteria in the same table should be revised by 
substituting the phrase "technical risks that are not excessive" for 
the phrase "low technical risk" (10 USC 2306b(a)(4)). 

The department will also comply with the recent amendments to the 
multiyear statute, 10 U.S.C. 2306b, made by section 811 of the National 
Defense Authorization Act for Fiscal Year, 2008. 

Recommendation 1: The GAO recommended that the Secretary of Defense 
should direct appropriate offices within the Under Secretary of Defense 
(USD)(Comptroller) and USD (Acquisition, Technology and Logistics) to 
improve and expand guidance provided to military services to better 
define MY decision criteria and to facilitate more consistent, 
objective, and knowledge-based evaluations of MY candidates within the 
department. (p.28/GAO Draft Report) 

DOD Response: Concur. To facilitate evaluations of MY candidates at the 
time they are proposed and prior to a request for MY contract 
authority, guidance is needed to better define the Office of the 
Secretary of Defense's review process. The guidance will result in an 
enhanced deliberative process for the evaluation of multiyear 
candidates, and improved historical records that document the reviews 
and the supporting rationales for the decisions. The department plans 
to issue this guidance within one year after the final GAO report is 
published. 

Recommendation 2: The GAO recommended that the Secretary of Defense 
should direct appropriate offices within the USD (Comptroller) and USD 
(Acquisition, Technology and Logistics) to establish a process for 
third party validation of the costs and savings data submitted for 
candidate programs. (p. 28/GAO Draft Report) 

DOD Response. Partially concur. Costs and savings are reviewed twice; 
once when MY authority is proposed and again at the conclusion of the 
contract negotiations when it is determined that the costs and savings 
meet the requirements of the statute. 

OSD's Cost Analysis Improvement Group or one of several Federally 
Funded Research and Development Centers (FFRDC) perform an independent 
cost assessment on selected programs. Also, the procedures for 
negotiating a major contract include consideration of independent cost 
audits and assessments from the Defense Contract Management Agency and 
the Defense Contract Audit Agency. 

When developing OSD's guidance described in response to recommendation 
1, the Department will consider whether the benefits of requiring third 
party validation of preliminary estimates of costs and savings data for 
all programs warrant the delays and the expense of validation, since a 
final determination of statutory compliance must be made as part of the 
contract negotiation and award process. 

Recommendation 3: The GAO recommended that the Secretary of Defense 
should direct appropriate offices within the USD (Comptroller) and USD 
(Acquisition, Technology and Logistics) to implement a central database 
for maintaining historical records and for effective monitoring and 
tracking multiyear programs, to include documenting the specific 
decisions made by stakeholders and their rationales for decisions. (p. 
28/GAO Draft Report) 

DOD Response. Concur. DOD will implement this action concurrent with 
implementation of the guidance described in DOD response to 
recommendation 1. 

Recommendation 4: The GAO recommended that the Secretary of Defense 
direct that the responsible military service, in conjunction with 
appropriate elements within OSD, conduct an after action assessment at 
the conclusion of each multiyear contract to determine its 
effectiveness in achieving predicted benefits while managing associated 
risks. (p. 28/GAO Draft Report) 

DOD Response. Partially concur. The department concurs that an after 
action assessment may be of value for certain programs. However, in 
general, it is not known whether the value in performing an assessment 
(e.g., lessons learned) on all MY programs outweighs (1) the extensive 
time before the assessment results are known and can be applied to 
similar programs; and (2) the difficulty in determining actual savings. 
First, the assessment can not begin until after performance is complete 
and data on actual costs is available, e.g., 7-8 years after contract 
award. It is not apparent how valuable the results would be to the 
department after such a long period has transpired. Second, as 
indicated in the report, savings are difficult to determine due to too 
many factors that may affect costs during the life of a MY contract, 
e.g., changes in quantity, economic conditions, or configuration (e.g., 
to address safety, reliability, or performance problems); the impact of 
cost reduction initiatives, subsequent changes in statutory and 
regulatory rules that may impact costs, etc. As recognized in the 
report, calculating the cost of a MY contract if these unanticipated 
changes had not occurred would be especially challenging and difficult 
to validate. Likewise, it is difficult to determine how an alternative 
series of annual contracts would have fared under these conditions. 

[End of section] 

Appendix III: GAO Contacts and Staff Acknowledgments: 

GAO Contact: 

Bruce Fairbairn, (937) 258-7952, fairbairnb@gao.gov: 

Staff Acknowledgments: 

Noah Bleicher; Matthew Drerup; Mary Jo Lewnard; Rae Ann Sapp; and Bob 
Swierczek: 

[End of section] 

Footnotes: 

[1] The purchase of certain materials or parts by the prime contractor 
from vendors in quantities greater than those needed for any single 
year of production under the multiyear contract. The goal is to 
minimize the costs of these items by buying in larger more economically 
efficient quantities and avoid the expenses related to additional 
production line set-ups and terminations that an annual buy approach 
would necessitate. 

[2] Pub. L. No. 97-86, § 909 (1981). 

[3] GAO, Tactical Aircraft: Questions Concerning the F-22A's Business 
Case, GAO-06-991T (Washington, D.C.: July 25, 2006) and GAO, Tactical 
Aircraft: DOD Should Present a New F-22A Business Case before Making 
Further Investments, GAO-06-455R (Washington, D.C.: June 20, 2006). 

[4] John Warner National Defense Authorization Act for Fiscal Year 
2007, Pub. L. No. 109-365 §134 (2006). 

[5] Principally, the Federal Acquisition Regulation (FAR), and the 
Defense Federal Acquisition Regulation Supplement (DFARS), and DOD 
Financial Management Regulation (FMR) 7000.14-R. 

[6] At the Air Force, we identified a guidebook developed in the 1980s 
that provides some details and examples that could aid officials in 
deciding whether a multiyear contract would be appropriate and in 
developing justification materials. We found, however, that some 
officials were either unaware of this or mainly used the FAR and DFAR 
guidance. Officials knew of no comparable document in the Army, Navy, 
and OSD. 

[7] GAO, DOD Acquisition Outcomes: A Case for Change, GAO-06-257T 
(Washington, D.C.: Nov. 15, 2005) and GAO, Weapons Acquisition: A Rare 
Opportunity for Lasting Change, GAO/NSIAD-93-15 (Washington, D.C.: Dec. 
1, 1992). 

[8] GAO, An Assessment of the Army's Multiple Launch Rocket System 
Multiyear Contract, (GAO/NSIAD-86-5 Oct. 28, 1985); GAO Procurement: An 
Assessment of the Air Force's F-16 Aircraft Multiyear Contract (GAO/ 
NSAID-86-38, February 1986); and GAO Procurement: Assessment of DOD' 
Multiyear Contract Candidates (GAO/NSAID-87-202-BR August 1987). 

[9] For comparability purposes and availability of data, we selected 
these three aircraft programs, each with recently completed multiyear 
contracts and ongoing follow-on multiyear contracts. Our findings 
cannot be extended to multiyear contracting in total, but do provide 
illustrations of what can happen during execution of a multiyear 
contract. 

[10] The Federal Acquisition Regulation (FAR) contains examples of 
standard economic price adjustment clauses at FAR § 52.216-2 through 4 
and value engineering at 52.248-1 through 3. 

[11] See Department of Defense Appropriations Act, 2005, Pub. L. No. 
108-287, § 8008 (2004), and similar provisions in subsequent DOD 
appropriations acts. 

[12] GAO, Defense Acquisitions: F/A-18E/F Aircraft Does Not Meet All 
Criteria for Multiyear Procurement, GAO/NSIAD-00-158 (Washington, D.C.: 
May 26, 2000). 

[13] The Conference Report for the National Defense Authorization Act 
for Fiscal Year 2008 indicates a preference for a 10 percent estimated 
savings level for future multiyear candidates. See H.R. Conf. Rep. No. 
110-447, at 951 (2007). 

[14] Department of Defense Authorization Act, 1982, Pub. L. No. 97-86, 
§ 909 (1981). 

[15] National Defense Authorization Act for Fiscal Years 1990 and 1991, 
Pub. L. No. 101-189, § 805 (1989). 

[16] National Defense Authorization Act for Fiscal Year 1991, Pub. L. 
No. 101-510, § 808 (1990). 

[17] Institute for Defense Analyses, F/A-22 Independent Cost Estimate 
(August 2005). 

[18] Recurring costs are production costs, such as labor and materials 
that vary with the quantity being produced. Nonrecurring costs are 
fixed expenses that do not vary with the quantity, such as capital 
investments in facilities, equipment, and tooling. 

[19] See Department of Defense Appropriations Act, 2005, Pub. L. No. 
108-287, § 8008 (2004) and similar provisions in subsequent DOD 
appropriations acts. 

[20] RAND Corporation, National Defense Research Institute, F-22A 
Multiyear Procurement Program: An Assessment of Cost Savings, 2007. 

[21] GAO, Procurement: An Assessment of the Air Force's F-16 Aircraft 
Multiyear Contract, GAO/NSIAD-86-39 (Washington, D.C.: Feb. 20, 1986). 

[22] The two databases were the DOD DD350 Individual Contract Action 
Report, and the Federal Procurement Data System-Next Generation (FPDS- 
NG). 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: