This is the accessible text file for GAO report number GAO-07-566T 
entitled 'Defense Acquisitions: Issues Concerning Airlift and Tanker 
Programs' which was released on March 8, 2007. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Testimony: 

Before the House Committee on Armed Services, Air and Land Forces 
Subcommittee: 

United States Government Accountability Office: 

GAO: 

For Release on Delivery Expected at 3:00 p.m. EST: 

Wednesday, March 7, 2007: 

Defense Acquisitions: 

Issues Concerning Airlift and Tanker Programs: 

Statement of William M. Solis, Director: 
Defense Capabilities and Management Issues and: 
Michael J. Sullivan, Director: 
Acquisition and Sourcing Management Issues: 

GAO-07-566T: 

GAO Highlights: 

Highlights of GAO-07-566T, a testimony to House Committee on Armed 
Services, Air and Land Forces Subcommittee 

Why GAO Did This Study: 

The Department of Defense (DOD) has continuing efforts to modernize its 
airlift and tanker fleets by investing billions of dollars to modify 
legacy airlift systems, such as the C-5 and C-130, and procure new 
aircraft, such as a tanker replacement. Acquisition has been on GAO’s 
list as a high risk area since 1990. GAO has reported that elements 
contributing to a sound business case for an acquisition are missing or 
incomplete as DOD and the services attempt to acquire new capabilities. 
Those elements include firm requirements, mature technologies, a 
knowledge-based acquisition strategy, a realistic cost estimate, and 
sufficient funding. Acquisition problems that include failure to limit 
cost growth, schedule delays, and quantity reductions persist, but 
fiscal realities will not allow budgets to accommodate these problems 
any longer. 

Today’s testimony addresses 
(1) the analyses supporting the Department of Defense’s (DOD) mobility 
capabilities and requirements and (2) actions that are needed to 
improve the outcomes of weapon system acquisitions. 

For this testimony, GAO drew from issued reports, containing statements 
of the scope and methodology used, as well as recently completed work 
not yet reported. GAO’s work was performed in accordance with generally 
accepted government auditing standards. 

What GAO Found: 

Past GAO reports, including two recently issued, raise concerns about 
the quality of analyses underpinning the programmatic decision-making 
surrounding DOD’s airlift requirements. In September 2006, GAO issued 
our report (GAO-06-938) on DOD’s Mobility Capabilities Study (MCS). The 
MCS determined that the projected mobility capabilities are adequate to 
achieve U.S. objectives with an acceptable level of risk during the 
period from fiscal years 2007 through 2013; that is, the current U.S. 
inventory of aircraft, ships, prepositioned assets, and other 
capabilities are sufficient, in conjunction with host nation support. 
GAO’s report stated that conclusions of the MCS were based on 
incomplete data and inadequate modeling and metrics that did not fully 
measure stress on the transportation system. GAO further observed that 
the MCS results were incomplete, unclear, or contingent on further 
study, making it difficult to identify findings and evaluate evidence. 
It was not clear how the analyses done for the study support DOD’s 
conclusions and GAO suggested that decision makers exercise caution in 
using the results of this study to make programmatic decisions. In 
March 2007, GAO reported (GAO-07-367R) on the lack of mandatory 
analyses to support a passenger and cargo capability for the new 
replacement refueling aircraft, the KC-X tanker. Contrary to mandatory 
Air Force implementing guidance, the Air Force proposed a capability 
without analyses identifying an associated gap, shortfall, or 
redundancy. GAO believes that without sound analyses, the Air Force may 
be at risk of spending several billion dollars unnecessarily for a 
capability that may not be needed to meet a gap or shortfall and made 
recommendations to the Secretary of Defense that included conducting 
the requiring analyses necessary to establish capabilities. 

Successful acquisition programs make sound decisions based on critical 
product knowledge to ensure that program investments are getting 
promised returns--on time delivery, within estimated costs, and with 
expected capabilities. However, GAO has shown in its work that DOD 
practices diverge from best development practices intended to produce 
good outcomes and, as a result, have experienced significant cost 
growth and schedule delays. DOD expects to invest over $12 billion in 
new and improved capabilities in four airlift programs discussed in 
this testimony between now and 2013—C-5 Avionics Modernization Program, 
C-5 Reliability Enhancement and Reengining Program, C-130 Avionics 
Modernization Program, and the C-130J acquisition program. GAO found 
that all four programs failed at basic systems engineering practices to 
1) fully analyze the resources needed to integrate proven commercial 
technologies, 2) achieve a stable design before beginning system 
demonstration, and 3) demonstrate the aircraft would work as required 
before making large production investments. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-566T]. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact William M. Solis at (202) 
512-8365 or solisw@gao.gov and Michael J. Sullivan at (202)-512-4841 or 
sullivanm@gao.gov. 

[End of section] 

Mr. Chairman and Members of the Subcommittee: 

We are pleased to be here to discuss the development of aircraft 
program requirements and issues related to the acquisition process. We 
recently issued our report on high risk areas in the federal 
government, which lists DOD's acquisition process as one longstanding 
area of concern.[Footnote 1] Acquisition has been on this list since 
1990. As we have reported, DOD knows what to do to achieve more 
successful outcomes, but finds it difficult to apply the necessary 
discipline and controls or assign much needed accountability. DOD has 
not been employing a knowledge-based development approach and business 
cases have not measured up. DOD has just begun piloting some corrective 
actions, so the proof of actual implementation may be years away. These 
initiatives also may not necessarily be applied to programs already 
under way. 

DOD has continuing efforts to modernize its airlift and tanker fleets 
by investing billions of dollars to modify legacy airlift systems, such 
as the C-5 and C-130, and procure new aircraft, such as the KC-X 
replacement tanker. We have reported in the past that a sound business 
case for an acquisition contains firm requirements, mature 
technologies, a knowledge-based acquisition strategy, a realistic cost 
estimate, and sufficient funding. However, we have found many of these 
elements are missing or incomplete as DOD and the services attempt to 
acquire new capabilities. Persistent acquisition problems include 
failure to identify needs versus wants and to limit cost growth, 
schedule delays, and quantity reductions, but fiscal realities will not 
allow budgets to accommodate these problems any longer. 

Today I will highlight for you some issues related to the analyses 
supporting the Department of Defense's (DOD) mobility capabilities and 
requirements and Mike Sullivan will discuss actions that are needed to 
improve the outcomes of weapon system acquisitions. For this testimony, 
we drew from issued reports, which contain statements of the scope and 
methodology used, as well as recently completed work not yet reported. 
Our work was performed in accordance with generally accepted government 
auditing standards. 

Summary: 

DOD has an mandate to deliver high-quality products to warfighters, 
when they need them and at a price the country can afford. However, our 
work shows that acquisition problems will likely persist until DOD 
provides a better foundation for buying the right things, the right 
way. This foundation begins with setting requirements that are based on 
adequate and complete analyses, using current operational data and 
updated, effective models. For the past several years, we have reported 
our concerns with the analyses done to support requirements and have 
recently issued two reports that raise concerns about the quality of 
analyses underpinning the programmatic decision-making surrounding 
DOD's airlift requirements. In September 2006, we issued our report on 
DOD's Mobility Capabilities Study (MCS). The MCS determined that the 
projected mobility capabilities are adequate to achieve U.S. objectives 
with an acceptable level of risk during the period from fiscal years 
2007 through 2013; that is, the current U.S. inventory of aircraft, 
ships, prepositioned assets, and other capabilities are sufficient, in 
conjunction with host nation support. In our report, we stated that 
conclusions of the MCS were based on incomplete data and inadequate 
modeling and metrics that did not fully measure stress on the 
transportation system.[Footnote 2] We further observed that, in some 
cases, the MCS results were incomplete, unclear, or contingent on 
further study, making it difficult to identify findings and evaluate 
evidence. It is not clear how the analyses done for the study support 
DOD's conclusions and we suggested that decision makers exercise 
caution in using the results of this study to make programmatic 
decisions. This week, we issued a report on the lack of mandatory 
analyses to support a passenger and cargo capability for the new 
replacement refueling aircraft, the KC-X tanker.[Footnote 3] Contrary 
to mandatory Air Force implementing guidance, the Air Force proposed a 
capability without an analyses identifying an associated gap, 
shortfall, or redundancy. Air Force officials could not provide 
supporting information sufficient to explain this discrepancy between 
the required analyses and their proposal. 

Successful acquisition programs make sound decisions based on critical 
product knowledge to ensure that program investments are getting 
promised returns--on time delivery, within estimated costs, and with 
expected capabilities. This is important because DOD expects to invest 
over $12 billion in new and improved capabilities in four airlift 
programs discussed in this testimony between now and 2013--the C-5 
Avionics Modernization Program, C-5 Reliability Enhancement and 
Reengining Program, C-130 Avionics Modernization Program, and the C- 
130J acquisition program. These four programs have diverged from the 
best development practices intended to produce good outcomes and as a 
result have experienced significant cost growth and schedule delays. We 
found that all four programs failed at basic systems engineering 
practices to 1) fully understand the resources needed to integrate 
proven commercial technologies, 2) achieve a stable design before 
beginning system demonstration, and 3) demonstrate the aircraft would 
work as required before making large production investments. As a 
result, each has encountered significant delays in delivering 
capability to the field and as a group have spent $962.3 million 
(fiscal year 2007 dollars) more than planned for development. 

Background: 

DOD must be capable of rapidly deploying armed forces to respond to 
contingency and humanitarian operations around the world. Airlift and 
tanker aircraft play a vital role in providing this capability. Over 
the past 25 years, DOD has invested almost $141 billion to develop, 
procure, and modify its airlift and tanker forces with an additional 
investment planned for fiscal years 2007 through 2011 of $32 billion. 
Recent annual funding levels are at the highest levels in two decades. 
(See figure 1.) 

Figure 1: Planned and Future DOD Investments in Airlift and Tanker 
Fleets (1982 to 2011): 

[See PDF for image] 

Source: GAO analysis of DOD data. 

Note: Based on DOD's Fiscal Years Defense Program (2007). 

[End of figure] 

In December 2005, DOD issued a report on the study of its mobility 
capabilities. The goal of this Mobility Capabilities Study was to 
identify and quantify the mobility capabilities needed to support U.S. 
strategic objectives into the next decade. The MCS determined that the 
projected mobility capabilities are adequate to achieve U.S. objectives 
with an acceptable level of risk during the period from fiscal years 
2007 through 2013; that is, the current U.S. inventory of aircraft, 
ships, prepositioned assets, and other capabilities are sufficient, in 
conjunction with host nation support. The MCS emphasized that continued 
investment in the mobility system, in line with current departmental 
priorities and planned spending, is required to maintain these 
capabilities in the future. This includes, for example, fully funding 
Army prepositioned assets as planned and completing a planned 
reengineering of the C-5 aircraft. 

In our previous reports concerning acquisition outcomes and best 
practices, we have noted the importance of matching warfighter 
requirements with available resources, a responsibility shared by the 
requirements and acquisition communities in DOD. As described in Air 
Force implementing guidance, there is within DOD a distinct separation 
between the requirements authority and acquisition authority.[Footnote 
4] Under this guidance, this separation requires early and continued 
collaboration between both communities. 

Analyses Used to Determine Mobility and Tanker Capabilities Were 
Inadequate: 

Analyses done for the MCS contained methodological limitations that 
create concerns about the adequacy and completeness of the study and 
decision makers approving the KC-X tanker proposal lacked required 
analyses identifying need and associated risk for a passenger and cargo 
capability. 

Mobility Capabilities Study Limitations Raise Questions about Adequacy 
and Completeness: 

While DOD used an innovative approach in conducting the study and 
acknowledged some methodological limitations in its report, it did not 
fully disclose how these limitations could affect the MCS conclusions 
and recommendations. In September 2006, we reported that DOD's 
conclusions were based, in some instances, on incomplete data and 
inadequate modeling and metrics that did not fully measure stress on 
the transportation system, and that, in some cases, MCS results were 
incomplete, unclear, or contingent on further study, making it 
difficult to identify findings and evaluate evidence. It is not clear 
how the analyses done for the study supported DOD's conclusions, and we 
suggested that decision makers exercise caution in using the results of 
this study to make programmatic decisions. 

As measured against relevant generally accepted research standards, we 
identified limitations in the MCS study and report that raise 
questions. Among our findings: 

* Aspects of modeling and data were inadequate in some areas because 
data were lacking and some of the models used could not simulate all 
relevant aspects of the missions. The report did not explain how these 
limitations could affect the study results or what the effect on the 
projected mobility capabilities might be. Relevant research standards 
require that models used are adequate for the intended purpose and 
represent a complete range of conditions, and also that data used are 
properly generated and complete. For example, the MCS modeled 
hypothetical homeland defense missions rather than missions for 
homeland defense demands from a well-defined and approved concept of 
operations for homeland defense because the specific details of the 
missions were still being determined, and DOD acknowledged that the 
data used may be incomplete. The MCS also was unable to model the 
flexible deterrent options/deployment order process to move units and 
equipment into theater due to lack of data, but the study assumed a 
robust use of this process, which in one scenario accounted for 
approximately 60 percent of the airlift prior to beginning combat 
operations.[Footnote 5] In addition, the MCS report contains more than 
80 references to the need for improved modeling, and 12 of these 
references call for additional data or other refinements. Additionally, 
the MCS modeled the year 2012 to determine the transportation 
capabilities needed for the years 2007 through 2013. The year 2012 did 
not place as much demand for mobility assets in support of smaller 
military operations, such as peacekeeping, as other years. However, DOD 
officials considered 2012--the year modeled--as "most likely" to occur 
and stated that statistically it was not different from other years in 
the 2007 to 2013 period even though the number of smaller military 
operations is the least of any of the years reviewed. 

As I mentioned, we have reported before on the lack of data available 
for analysis that could benefit decision makers. In September 2005, we 
reported that the Air Force captured data on short tons transported but 
did not systematically collect and analyze information on operational 
factors, such as weather and runway length, that impact how much can be 
loaded on individual missions.[Footnote 6] Therefore, Air Force 
officials could not know how often it met its secondary goal to use 
aircraft capacity as efficiently as possible. Without this information, 
Air Mobility Command officials do not know the extent to which 
opportunities exist to use aircraft more efficiently and whether 
operational tempo, cost, and wear and tear on aircraft could be 
reduced. In addition, DOD officials do not have the benefit of such 
analysis to determine future airlift requirements for planning 
purposes. 

* While the MCS concluded that combined U.S. and host nation 
transportation assets were adequate to meet U.S. objectives with 
acceptable risk, the report, in describing the use of warfighting 
metrics in its analyses, does not provide a clear understanding of the 
direct relationship of warfighting objectives to transportation 
capabilities. Acknowledging this point, the report stated that further 
analysis is required to understand the operational impact of increased 
or decreased strategic lift on achieving warfighting objectives. 
Relevant generally accepted research standards require that conclusions 
be supported by analyses. The use of warfighting metrics is a measure 
to determine whether combat tasks, such as achieving air superiority, 
are achieved. However, they do not measure whether appropriate 
personnel, supplies, and equipment arrived in accordance with 
timelines. As a result, we could not determine how the study concluded 
that planned transportation assets were adequate because the study did 
not contain a transparent analysis to support its conclusion or a clear 
roadmap in the report to help decision makers understand what that 
conclusion meant in terms of type and number of mobility assets needed. 
Previous DOD mobility studies primarily used mobility metrics, which 
measured success in terms of tons of equipment and personnel moved per 
day to accomplish military objectives. The use of both warfighting and 
mobility metrics to measure success would allow decision makers to know 
whether combat tasks were achieved and how much strategic 
transportation is needed to accomplish those tasks. 

* In some cases, the MCS results were incomplete, unclear, or 
contingent on further study, making it difficult to identify findings 
and evaluate evidence. Relevant research standards require results to 
be presented in a complete, accurate, and relevant manner. For example, 
the report contains several recommendations for further studies and 
assessments, five of which are under way. However, at the time of our 
report, DOD had no plans to report the effect of these studies on the 
MCS results after the studies are complete. In addition, the report 
contains qualified information that is not presented clearly, such as 
varying assessments of intratheater assets in three different places in 
the report. The lack of clarity and conciseness of the reported results 
can limit the study's usefulness to decision makers and stakeholders. 

The MCS report also made recommendations to conduct further studies, 
develop plans and strategies, and improve data collection and mobility 
models. In fact, DOD officials told us at the time that a Mobility 
Capabilities Study-2006 was underway, as well as studies on 
intratheater lift, aerial refueling, and other mobility issues. 
However, unless DOD addresses the concerns I just outlined for you, 
decision makers may be unable to clearly understand the operational 
implications of the study results and make fully informed programmatic 
investment decisions concerning mobility capabilities. Also, some of 
the underlying assumptions used in the MCS have now changed 
significantly, such as the assumption that Army prepositioned equipment 
is in place and fully funded, which will no longer be the case. 
Therefore, the MCS analyses and results, which would be the starting 
point for any new studies, may no longer be relevant. 

Mandatory Analyses Done to Support Passenger and Cargo Capabilities in 
the KC-X Tanker Were Incomplete: 

Mandatory Air Force policy requires Air Force organizations to use a 
formal capabilities-based approach to identify, evaluate, develop, 
field, and sustain capabilities that compete for limited resources. 
Contrary to mandatory Air Force implementing guidance, however, the Air 
Force proposal for a replacement refueling aircraft, the KC-X tanker, 
included a passenger and cargo capability without analyses identifying 
an associated gap, shortfall, or redundant capability. According to 
mandatory Air Force implementing guidance, analyses supporting the 
decision-making process should assess a capability based on the effects 
it seeks to generate and the associated operational risk of not having 
it. In this case, the supporting analyses determined neither need nor 
risk with regard to a passenger and cargo capability. Air Force 
officials could not provide supporting information sufficient to 
explain this discrepancy between the analyses and their proposal. 
Without sound analyses, the Air Force may be at risk of spending 
several billion dollars unnecessarily for a capability that may not be 
needed to meet a gap or shortfall. 

Military decision makers approved the passenger and cargo capability as 
a requirement although supporting analyses identified no need or 
associated risk. Mandatory Air Force implementing guidance states that 
senior leaders must use the documented results of analyses to confirm 
the identified capability requirement. The Air Force Requirements for 
Operational Capabilities Council validated, and the Chairman of the 
Joint Chiefs of Staff's Joint Requirements Oversight Council validated 
and approved, KCX tanker proposal with a passenger and cargo 
capability. Following the approvals of the oversight councils, DOD 
plans to solicit proposals and award a contract for the KC-X tanker 
late in fiscal year 2007. At this time, the Under Secretary of Defense 
for Acquisition, Technology and Logistics, who supervises DOD 
acquisition,[Footnote 7] must certify, as Milestone Decision Authority 
for the proposed tanker acquisition, that, among other things, the 
Joint Requirements Oversight Council has accomplished its statutory 
duties and that the proposed program is in compliance with DOD policies 
and regulations.[Footnote 8] However, the absence of analyses 
identifying a capability gap, shortfall, or redundancy, and the Joint 
Requirements Oversight Council approval of the program without these 
analyses is contrary to policy and implementing guidance and could 
preclude certification of the program by the Under Secretary. Absent 
this certification, the acquisition program for the KC-X tanker cannot 
begin.[Footnote 9] 

In this report, we recommended that the Secretary of Defense direct the 
Secretary of the Air Force to accomplish the required analyses to 
evaluate the proposed passenger and cargo capability so as to determine 
if there is a gap, shortfall, or redundancy, assess the associated 
risk, and then submit such documentation to the Joint Requirements 
Oversight Council for validation. We also recommended that, once these 
analyses are completed, the Secretary of Defense direct the Chairman, 
Joint Chiefs of Staff, to formally notify the Under Secretary of 
Defense for Acquisition, Technology and Logistics that such analyses 
have been completed as required prior to certification of the program 
to Congress. 

DOD disagreed with our first recommendation to accomplish the required 
analyses. In its comments, DOD stated that through the Joint 
Capabilities Integration and Development System process, the Air Force 
presented analysis and rationale for the passenger and cargo 
capability. DOD further stated that its Joint Requirements Oversight 
Council and the Air Force concluded that the analysis was sufficient 
justification for the capability and the Joint Requirements Oversight 
Council validated the requirement. However, as our report points out, 
DOD did not perform the required analyses and failed to identify a gap, 
shortfall, or redundancy for the passenger and cargo capability. 
Considering the requirement for analyses that separate needs from wants 
and the risk of unnecessary expenditures in this multi-year multi- 
billion dollar acquisition program, we continue to believe that our 
recommendation has merit and that the analyses required by mandatory 
guidance are necessary to inform the decision that begins the 
acquisition. 

DOD agreed with our recommendation to formally notify the Under 
Secretary of Defense for Acquisition, Technology and Logistics once the 
required analyses have been completed. However, DOD did not offer 
assurance that the Air Force would accomplish the required analyses 
that evaluate the proposed passenger and cargo capability as we 
recommended, and then submit such documentation to the Joint 
Requirements Oversight Council for validation. We believe that the time 
it could take to accomplish the required analyses and submit the 
analyses for revalidation by the Joint Requirements Oversight Council, 
could delay the Under Secretary's certification until just prior to the 
Milestone B decision, and may frustrate the congressional oversight 
that would otherwise be permitted under section 2366a.[Footnote 10] We 
believe that in a program committing $120 billion over several decades, 
the review confirming that needs are justified should occur as far in 
advance of program initiation as possible. 

In light of the DOD comments on our report, we have put forward a 
matter for congressional consideration. Specifically, we are suggesting 
that Congress consider requiring: 

* in addition to the certification described by section 2366a of title 
10, United States Code, the Under Secretary of Defense for 
Acquisitions, Technology and Logistics make a specific certification 
that the Air Force employed a sound, traceable, and repeatable process 
producing analyses that determined if there is a gap, shortfall, or 
redundancy and assessed the associated risk with regard to passenger 
and cargo capability for the KC-135 Recapitalization, and: 

* consistent with service policy, these analyses are made available to 
the Joint Requirements Oversight Council prior to the Under Secretary's 
certification of the program pursuant to section 2366a of title 10, 
United States Code. 

The Air Force intends to replace the fleet of more than 500 tankers and 
the Mobility Capabilities Study of 2005 set the requirement for tankers 
at a range of between 520 to 640 aircraft. Replacement of this fleet is 
estimated to cost a minimum of $72 billion. Compared to a refueling 
aircraft without a passenger and cargo capability, the inclusion of the 
capability is estimated, according to the Analysis of Alternatives done 
for the KC-X tanker, to increase costs by 6 percent. The Joint 
Requirements Oversight Council approval of the proposal of a 
replacement tanker aircraft with the passenger and cargo capability, 
without an established need supported by analyses and without an 
analysis of risk, could result in an unnecessary expenditure of at 
least $4.3 billion by our estimates. 

In our August 1996 report, U.S. Combat Air Power: Aging Refueling 
Aircraft Are Costly to Maintain and Operate, we recommended 
consideration of a dual-use aircraft that could conduct both aerial 
refueling and airlift operations as a replacement for the KC- 
135.[Footnote 11] We recommended that the Secretary of Defense require 
that future studies and analyses of replacement airlift and tanker 
aircraft consider accomplishing the missions with a dual-use aircraft. 
DOD only partially concurred with this recommendation, expressing 
concern at that time about how a dual-use aircraft would be used and 
whether one mission area might be degraded to accomplish the second 
mission. The lack of analyses done to support the current proposal 
still does not give DOD officials information about how a dual-use 
aircraft would be used or whether the primary mission of aerial 
refueling would be degraded. 

Employing Best Practices Can Shape Better Program Outcomes for DOD's 
Airlift Acquisitions: 

Over the past 25 years, DOD has invested more than $140 billion on its 
airlift and tanker forces. Success for acquisitions requires sound 
decisions to ensure that program investments are getting promised 
returns--on time deliveries to the field, predictable costs, and 
sufficient capability. We have reviewed four major airlift programs and 
found they did not meet delivery schedules and were over cost. These 
programs did not involve huge technological leaps but presented 
significant design challenges to integrate new systems into the older 
aircraft. A consistent problem plaguing the programs was an 
insufficient job of analyzing the requirements and resources at the 
programs' outset, a key systems engineering activity. The divergence 
between these programs' experience and best product development 
practices are contributing factors to their outcomes. 

Outcomes of Certain Airlift Programs: 

We assessed four airlift programs as part of our annual assessment of 
DOD's major acquisition programs and each has experienced cost growth 
and schedule delays. Despite being based largely on low technological 
risks involving mature systems, these programs have failed to deliver 
on the business cases that justified their initial investment. DOD 
estimates it will need over $12 billion between 2007 and 2013 to 
develop, modify, or procure these aircraft. The specific airlift 
programs include: 

* The Air Force's C-5 Avionics Modernization Program (AMP) is intended 
to improve the mission capability rate and transport capabilities, as 
well as reduce ownership costs by incorporating global air traffic 
management, navigation and safety equipment, modern digital equipment, 
and an all-weather flight control system. 

* The Air Force's C-5 Reliability Enhancement and Reengining Program 
(RERP) is intended to enhance the reliability, maintainability, and 
availability of the C-5 through engine replacements and modifications 
to subsystems such as the electrical and fuel subsystems. The C-5 
aircraft will require installation of the AMP capabilities before the 
aircraft engines can be replaced. 

* The Air Force's C-130 Avionics Modernization Program (AMP) is 
intended to standardize the cockpit configurations and avionics of 
different models of C-130 aircraft by providing such things as 
communication and navigational system upgrades, terrain avoidance and 
warning system, dual flight management systems, and new data links. 

* The C-130J, the latest model of the C-130 aircraft series, is 
designed primarily for the transport of cargo and personnel within a 
theater of operation. Variants of the C-130J are being acquired by the 
Air Force (e.g., Air Mobility Command and Special Operations Command), 
Marine Corps, and Coast Guard. 

Each of these programs has experienced problems that have impacted cost 
and schedule (see table 1). The net effect of the outcomes to date is 
that DOD is now paying more to modify or acquire these systems and the 
warfighter has had to wait longer than initially planned before new 
capability is delivered. For example, the Air Force now expects by 2011 
to have completed the modification of about 135 fewer C-130 airlift 
aircraft when compared to its plan 2 years ago. 

Table 1: Outcomes of Selected Airlift Programs (as of fiscal year 2006) 
(FY 2007 dollars in millions): 

Weapon Systems: C-5 AMP; 
Latest Development Cost: $432.1; 
Development Cost Change Since Start: 17 percent; 
Program Acquisition Unit Cost Increases: 86 percent; 
Program schedule delays: Initial operational capability delayed about 1 
year. 

Weapon Systems: C-5 RERP; 
Latest Development Cost: $1,342.9; 
Development Cost Change Since Start: (16 percent); 
Program Acquisition Unit Cost Increases: 10 percent; 
Program schedule delays: Initial operational capability delayed over 2 
years. 

Weapon Systems: C-130 AMP; 
Latest Development Cost: $1,627.8; 
Development Cost Change Since Start: 128 percent; 
Program Acquisition Unit Cost Increases: 43 percent; 
Program schedule delays: First production delivery delayed over 2 
years. 

Weapon Systems: C-130J; 
Latest Development Cost: $262.9; 
Development Cost Change Since Start: Not applicable; 
Program Acquisition Unit Cost Increases: 26 percent; 
Program schedule delays: First production delivery delayed about 1 ˝ 
years. 

Source: GAO analysis of DOD data: 

Note: C-130J development costs have increased by 2,347 percent but this 
includes costs to correct deficiencies and add new capabilities. 

[End of table] 

We anticipate there could be additional cost increases and schedule 
delays reported in the future. For example, the C-130 AMP fiscal year 
2008 budget indicates that the total program costs have increased 
almost $700 million and planned quantities have been reduced from 434 
units to 268 units--nearly doubling the program acquisition unit costs 
since December 2005. The program recently notified Congress of a 
critical Nunn-McCurdy breach concerning its unit cost 
increases.[Footnote 12] The budget also shows the Air Force plans to 
fund the modification of 110 C-5 aircraft with AMP improvements instead 
of 59 aircraft as stated in last year's budget. According to C-5 RERP 
program officials, total program costs are expected to increase due to 
costs with the engine, pylons, and labor. 

Airlift Programs Have Not Captured Critical Product Knowledge at Key 
Decision Points: 

Over the last several years, we have undertaken a body of work that 
examines weapon acquisition issues from the perspective that draws upon 
lessons learned from best commercial practices for product development. 
We have found that a key to successful product development is the 
formulation of a business case that provides demonstrated evidence that 
(1) the warfighter need exists and that it can best be met with the 
chosen concept and (2) the concept can be developed and produced within 
existing resources--including proven technologies, design knowledge, 
adequate funding, and adequate time to deliver the product when needed. 
The business case is then executed through an acquisition process that 
is anchored in knowledge. Leading firms ensure a high level of 
knowledge is achieved at key junctures in development, characterized as 
knowledge points described below: 

* Knowledge point 1: A match must be made between the customer's needs 
and the developer's available resources--technology, engineering 
knowledge, time, and funding--before a program starts. 

* Knowledge point 2: The product's design must be stable and must meet 
performance requirements before beginning system demonstration. This is 
primarily evidenced by the release of 90 percent of the design drawings 
by the critical design review and successful system integration. 

* Knowledge point 3: The product must be producible within cost, 
schedule, and quality targets and demonstrated to work as intended 
before production begins. 

There is a synergy in this process, as the attainment of each 
successive knowledge point builds on the preceding one. We have found 
that if the knowledge based acquisition concept is not applied, a 
cascade of negative effects becomes magnified in the product 
development and production phases of an acquisition program leading to 
cost increases and schedule delays, poor product quality and 
reliability, and delays in getting new capability to the warfighter 
(see figure 2). 

Figure 2: Illustration of Cascading Negative Effects of Failing to 
Follow a Knowledge-Based Acquisition Approach: 

[See PDF for image] 

Source: GAO. 

[End of figure] 

DOD programs often do not capture sufficient knowledge by critical 
junctures but decide to move forward regardless. The airlift systems we 
reviewed were not immune to this condition and have experienced 
unnecessary cost growth and schedule delays as a result. While we do 
not have in-depth knowledge on the specific details for these programs, 
we do have a broad understanding of the basic underpinnings that led to 
the problems. All of the programs were considered low technological 
risks by DOD because they planned to rely extensively on proven 
commercial and modified off the shelf technology for its new 
capabilities. However, these acquisitions have turned out to be more 
difficult than expected. The programs did not follow sound systems 
engineering practices for analyzing requirements and for ensuring a 
well integrated design at the right time. As a result, each program has 
encountered some difficulty in achieving design and production maturity 
as the program moved forward. Some of the causes to problems 
encountered include: 

* Failing to fully analyze the resources needed to integrate proven 
commercial technologies and subsystems into a military system before 
initiating development. 

* Not achieving a stable design before beginning system demonstration 
phase resulting in costly design changes and rework. 

* Failing to demonstrate the aircraft would work as required before 
making large production investments. 

In all these instances where appropriate knowledge was not captured 
before moving forward, the impact has resulted in a predictable need 
for additional resources as shown below in specific airlift programs. 

C-5 AMP: 

The C-5 AMP entered production without demonstrating that the system 
worked as intended and was reliable. The program entered production 
just 2 months after flight testing started and ran into significant 
design problems while trying to complete development. Problems 
uncovered after flight test began required modifications to the 
aircraft design which increased by 50 percent the number of engineering 
drawings needed for the system. Addressing these problems delayed the 
initial operational capability by a year and contributed to the 
significant growth in the program's unit costs. Even today, 4 years 
after production was initiated, performance concerns remain with the C- 
5 AMP. The Director of Operational Test and Evaluation recently 
reported that the C-5 AMP is not operationally suitable because of high 
component failure rates, inadequate diagnostics systems, and low 
reliability rates. 

C-5 RERP: 

The C-5 RERP did not demonstrate design stability before entering the 
system demonstration phase which resulted in rework and schedule 
delays. At the time the program entered system demonstration, program 
officials believed that they had released 90 percent of the design 
drawings but had not successfully demonstrated that the subsystems 
could be integrated onto the C-5 aircraft. During system integration 
activities the program found that the "pylon/thrust reverser" had to be 
redesigned to address overweight conditions and safety concerns. The 
program's design efforts have also been hampered by the fact that its 
success is dependent upon the success of the C-5 AMP program. 
Presently, according to test officials, the C-5 AMP design is not 
mature enough to provide a baseline design for the RERP efforts. These 
design issues have contributed to an increase in costs and a 2-year 
delay in delivering an initial operational capability. 

C-130 AMP: 

The C-130 AMP began development in 2001 without a clear understanding 
of the resources needed to integrate proven commercial technologies 
into a military system. According to the program office, they clearly 
underestimated the complexity of the engineering efforts that were 
needed to modify the different models of the C-130. At the critical 
design review held in 2005--the point that the design is expected to be 
stable and ready to begin the system demonstration phase--the program 
had not proven that the subsystems and components could be successfully 
integrated into the product.Upon integrating the new avionics into the 
test aircraft, program officials realized that it had significantly 
underestimated (by 400 percent) the amount of wiring and the number of 
harnesses and brackets needed for the installation. As a result, the 
design had to be reworked, delaying the delivery of the test aircraft 
and increasing costs. 

C-130J: 

The Air Force procured the C-130J without assurances that the aircraft 
would work as intended. Program officials believed the design was 
mature when procurement began in 1996, largely because the C-130J 
evolved from earlier models and was offered as a commercial 
item[Footnote 13]. However, the C-130J has encountered numerous 
deficiencies that had to be corrected in order to meet the minimum 
warfighter requirements delaying the initial aircraft delivery to the 
warfighter by about 1.5 years. DOD testing officials still report 
performance issues with the aircraft resulting in it being rated as 
partially mission capable. The performance issues involve the 
aircraft's ability to meet its airdrop operations requirements, its 
effectiveness in non-permissive threat environments, and 
maintainability issues. Program officials plan to address the 
deficiencies as part of a C-130J modernization effort. 

Conclusion: 

As we said at the beginning, our work shows that acquisition problems 
will likely persist until DOD provides a better foundation for buying 
the right things the right way. This involves making tough tradeoff 
decisions as to which programs should be pursued, and, more 
importantly, not pursued, making sure programs are executable, 
establishing and locking in needed requirements before programs are 
ever started, and making it clear who is responsible for what and 
holding people accountable when these responsibilities are not 
fulfilled. Recognizing this, DOD has tried to embrace best practices in 
its policies, as well as taking many other actions. However, DOD still 
has trouble distinguishing between wants and needs. Because of our 
concerns about the analyses done for both the MCS, which has broad 
implications for DOD's mobility needs, and the KC-X tanker 
requirements, we would urge Congress and other decision makers to 
exercise caution when making airlift and tanker investment decisions. 

DOD will continue to face challenges in modernizing its forces with new 
demands on the federal dollar created by changing world conditions. 
Consequently, it is incumbent upon DOD to find and adopt best product 
development practices that can allow it to manage its weapon system 
program in the most efficient and effective way. Success over the long 
term will depend on following knowledge-based acquisition practices as 
well as DOD leadership's commitment to improving outcomes. 

The four acquisition cases we cite in this testimony are not atypical 
for all programs. Even with no major technological invention necessary 
to meet the warfighters needs in these cases, acquisition outcomes are 
not good. There are consequences to these outcomes. The warfighter does 
not receive needed capability on time and the Department and Congress 
must spend additional unplanned money to correct mistakes--an expense 
they can ill afford. A knowledge-based product development process 
steeped in best practices from systems engineering can solve many of 
these problems before they start. DOD knows how to do this and, in 
fact, informs its acquisition policy with systems engineering rules. It 
should redouble its efforts to drive these policies into practice. 

Mr. Chairman and members of the Subcommittee, this concludes our 
prepared statement. We would be pleased to answer any questions you may 
have. 

Contact and Staff Acknowledgements: 

For further information about this statement, please contact William M. 
Solis at 202-512-8365 or Solisw@gao.gov or Michael J. Sullivan at 202- 
512-4841 or Sullivanm@gao.gov. Contact points for Offices of 
Congressional Relations and Public Affairs may be found on the last 
page of this statement. 

GAO staff who made major contributions to this testimony include Marie 
Ahearn, Ann Borseth, Cheryl Andrew, Claudia Dickey, Mike Hazard, 
Matthew Lea, Oscar Mardis, Sean Merrill, Karen Thornton, and Steve 
Woods. 

[End of section] 

Related GAO Products: 

Defense Acquisitions: Major Weapon Systems Continue to Experience Cost 
and Schedule Problems under DOD's Revised Policy. GAO-06-368. 
Washington, D.C.: April 13, 2006: 

Defense Acquisitions: Assessments of Selected Major Weapon Programs. 
GAO-06-391. Washington, D.C.:March 31, 2006. 

DOD Acquisition Outcomes: A Case for Change. GAO-06-257T. Washington, 
D.C.: November 15, 2005. 

Best Practices: Capturing Design and Manufacturing Knowledge Early 
Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15, 
2002. 

Defense Acquisitions: DOD Faces Challenges in Implementing Best 
Practices. GAO-02-469T. Washington, D.C.: February 27, 2002. 

Best Practices: Better Matching of Needs and Resources Will Lead to 
Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: March 8, 
2001. 

Defense Acquisition: Employing Best Practices Can Shape Better Weapon 
System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 26, 2000. 

Best Practices: Better Management of Technology Development Can Improve 
Weapon System Outcomes. GAO/NSIAD-99-162. Washington, D.C.: July 30, 
1999. 

Best Practices: Successful Application to Weapon Acquisition Requires 
Changes in DOD's Environment. GAO/NSIAD-98-56. Washington, D.C.: 
February 24, 1998. 

FOOTNOTES 

[1] GAO, High Risk Series: An Update, GAO-07-310 (Washington, D.C.: 
Jan. 2007). 

[2] GAO, Defense Transportation: Study Limitations Raise Questions 
about the Adequacy and Completeness of the Mobility Capabilities Study 
and Report, GAO-06-938 (Washington, D.C.: Sept. 20, 2006). 

[3] GAO, Defense Acquisitions: Air Force Decision to Include a 
Passenger and Cargo Capability in Its Replacement Refueling Aircraft 
Was Made without Required Analyses, GAO-07-367R (Washington, D.C.: Mar. 
6, 2007). 

[4] Air Force Instruction 10-601, Capabilities-Based Requirements 
Development, ¶ 1.2 (July 31, 2006). 

[5] Deployment orders are issued to deploy specific capabilities as 
commitment decisions are made, rather than a deploying unit's full set 
of equipment or capabilities. Flexible Deterrent Options (FDOs) provide 
escalation options during the initial stages of a conflict. FDOs are 
employed under certain conditions to deter adversarial actions contrary 
to U.S. interests. 

[6] GAO, Defense Transportation: Air Mobility Command Needs to Collect 
and Analyze Better Data to Assess Aircraft Utilization. GAO-05-819 
(Washington, D.C.: Sept. 29, 2005). 

[7] 10 U.S.C. §133 (b)(1) (2006). 

[8] 10 U.S.C. § 2366a § (a)(7) and (a)(10) (2006). 

[9] 10 U.S.C. § 2366a (a) (2006) and Department of Defense Instruction 
5000.2, Operation of the Defense Acquisition System, ¶ 3.7.1.2 (May 12, 
2003). 

[10] In respect to acquisition programs, milestones are established in 
DODI 5000.2 and are the points where a recommendation is made and 
approval is sought regarding starting or continuing a program into the 
next phase. In this instance, the decision at Milestone B is to enter 
into the system development and demonstration phase pursuant to 
guidance prescribed by the Secretary of Defense and to begin the 
acquisition program. 

[11] GAO, U.S. Combat Air Power: Aging Refueling Aircraft Are Costly to 
Maintain and Operate. GAO / NSIAD-96-160, Washington D.C.: (August 8, 
1996). 

[12] 10 U.S.C. § 2433 establishes the requirement for unit cost reports 
if certain thresholds for program costs are exceeded (known as unit 
cost or Nunn-McCurdy breaches). DOD is required to report to Congress 
and, if applicable, certify the program to Congress. 

[13] In October 2006, the program completed the transition to a non- 
commercial negotiated contract. 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: