This is the accessible text file for GAO report number GAO-11-112 entitled 'Defense Acquisitions: DOD Needs to Reassess Joint Cruise Missile Costs before Starting New Production Phase' which was released on October 13, 2010. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: United States Government Accountability Office: GAO: October 2010: Defense Acquisitions: DOD Needs to Reassess Joint Cruise Missile Costs before Starting New Production Phase: GAO-11-112: GAO Highlights: Highlights of GAO-11-112, a report to congressional committees. Why GAO Did This Study: Over the past two and a half decades, the Department of Defense (DOD) has invested heavily to acquire a cruise missile capable of attacking ground targets stealthily, reliably, and affordably. After abandoning an earlier, more expensive missile and a joint service effort, the Air Force began producing the Joint Air-to-Surface Standoff Missile (JASSM) in 2001. After that, the program (1) encountered many flight test failures, (2) decided to develop an extended range version, and (3) recognized significant cost growth. The production decision for the JASSM-ER is planned for November 2010. Also, the Secretary of Defense has recently announced a major initiative to restore affordability and productivity in defense spending. This initiative is expected to, among other things, identify savings by conducting needed programs more efficiently. As DOD faces the initial production decision on JASSM-ER, GAO was asked to assess (1) most recent test results, correction of causes of previous flight test failures, and efforts to improve JASSM’s reliability; and (2) JASSM cost changes, efforts to control costs, and additional cost risks for the program. What GAO Found: Since 2007, design changes and other corrective actions by the Air Force have improved the baseline JASSM’s test results significantly— the missile has now demonstrated 85 percent success versus 58 percent achieved previously and before the corrections. The JASSM-ER variant has done well thus far, with no failures during the first seven flight tests. These results reflect the Air Force’s enhanced oversight of the program and significant investments made to improve reliability. These efforts also identified many of the root causes for flight test failures. While baseline JASSM missile reliability has improved, it is not expected to achieve the Under Secretary of Defense for Acquisition, Technology and Logistics’ required level of 90 percent until 2013. Tests conducted thus far of the improved baseline JASSM and the JASSM-ER variants have been developmental—or controlled—in nature. Neither the improved JASSM baseline missile nor the JASSM-ER has been demonstrated in operationally realistic testing or in a combat operation. Table: JASSM Estimated Costs and Quantities Through 2025: Quantities to be produced: December 1998: 2,400; May 2008: 4,900; December 2009: 4,900. Total Program Cost (millions): December 1998: $2,232.5; May 2008: $6,641.4; December 2009: $7,129.4. Program Unit Cost (thousands): December 1998: $907.0; May 2008: $1,327.0; December 2009: $1,421.0. Procurement Unit Cost (thousands): December 1998: $515.0; May 2008: $1,073.0; December 2009: $1,160.0. Source: GAO analysis of Department of Defense data. Note: Fiscal year 2010 dollars. [End of table] JASSM costs have increased by over seven percent since the program was restructured in 2008. As the table shows, since 1998, JASSM quantities have more than doubled and estimated program costs have grown from $2.2 billion to a $7.1 billion. The Air Force has taken several steps to control JASSM costs, but options to reduce costs at this point appear limited. In fact, several factors suggest additional cost growth is likely. First, the Air Force has not been able to provide enough funding to produce the missiles at planned rates. That has led to a less efficient production process, a longer production period, and higher costs that have not yet been reflected in the $7.1 billion estimate. Second, the Air Force’s potential plans to retrofit existing missiles with the reliability improvements may not be feasible, given the missile’s sensitivity to being reopened. If retrofits prove infeasible, new replacements may have to be purchased; if they are feasible, the Air Force may have to provide additional funding to retrofit all existing missiles. Finally, since the Air Force last compared JASSM to possible alternatives, the unit cost was assumed to be about 40 percent less than currently expected and that now could make alternatives more competitive in terms of cost and/or capabilities. A reevaluation of the JASSM program, given that most of its costs have yet to be incurred, is warranted before the decision to produce the JASSM-ER is made. What GAO Recommends: GAO recommends that the Secretary of Defense reevaluate the JASSM program’s affordability and cost-effectiveness before making the decision to produce the JASSM-ER. DOD partially concurred with GAO’s assessment, but believes the JASSM-ER should begin production in November 2010. GAO believes that it is incumbent upon the department to reexamine JASSM before making the production decision to ensure that the program is structured as efficiently as possible and is still a good investment given the other demands DOD faces. View [hyperlink, http://www.gao.gov/products/GAO-11-112] or key components. For more information, contact Mike Sullivan at (202) 512- 4841 or sullivanm@gao.gov. [End of section] Contents: Letter: Background: Corrective Actions Have Led to Improved JASSM Test Results but Operational Effectiveness Is Still to Be Demonstrated: Several Areas of Potential Cost Risk Could Further Increase JASSM Program Costs: Conclusions: Recommendation for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: Comments from the Department of Defense: Appendix III: JASSM Baseline and JASSM-ER Cost Estimates Met Most Best Practices, but the Risk Analyses Did Not Consider Reliability or Extending the Production Schedule: Appendix IV: GAO Contact and Staff Acknowledgments: Tables: Table 1: Changes Since 2008 in JASSM Estimated Cost, Schedule, and Quantities through 2025: Table 2: Flight Test Cost to Increasing Reliability: Table 3: Extent to Which the Cost Estimating Practices for JASSM Baseline and JASSM-ER Reflect GAO Best Practices: Figures: Figure 1: JASSM Missile Reliability Assessment Flight Test: Figure 2: Initial JASSM Program Schedule: Figure 3: JASSM's Original Acquisition Strategy: Figure 4: JASSM Subsystems Involved in Flight Test Failures: Figure 5: JASSM Baseline Missile Test Results: Figure 6: Comparison of JASSM's Original and 2009 Total Acquisition Funding Profiles: Figure 7: JASSM Baseline and JASSM-ER Cost Estimate Trend Analysis 1997 through 2009: Abbreviations: CARD: Cost Analysis Requirements Document: CAIG: Cost Analysis Improvement Group: DOD: Department of Defense: JASSM: Joint Air-to-Surface Standoff Missile: JASSM-ER: Joint Air-to-Surface Standoff Missile Extended Range: OSD: Office of Secretary of Defense: TSPR: Total System Performance Responsibility: TSSAM: Tri-Service Standoff Attack Missile: U.S.C. United States Code: USD/ATL: Under Secretary of Defense for Acquisition, Technology, and Logistics: WBS: Work Breakdown Structure: [End of section] United States Government Accountability Office: Washington, DC 20548: October 13, 2010: Congressional Committees: Over the past two and a half decades, the Department of Defense (DOD) has invested heavily to acquire an Air Force air-to-ground cruise missile capable of stealthy flight and reliable performance at affordable costs. The Air Force invested in the Tri-Service Standoff Attack Missile (TSSAM) from 1986 through 1994 but ended the program once the unit costs exceeded $2 million per missile and as testing issues surfaced. The Air Force is currently in production for the Joint Air-to-Surface Standoff Missile (JASSM). This program is intended to provide a next-generation cruise missile capable of striking high-value, highly defended targets from outside the enemy's air defenses launched from a variety of aircraft, including the B-1, B- 2, B-52, and F-16. Over the first several years of the program, the Air Force (1) encountered many flight test failures, (2) decided to develop an extended range version, and (3) recognized significant cost growth which led to a critical Nunn-McCurdy unit cost breach in 2006. [Footnote 1] The Air Force is about to begin producing the extended range version (JASSM-ER) that will more than double the range of the baseline missile. The two variants of JASSM are approximately 70 percent common in hardware and 95 percent common in software. As DOD approaches the production decision on JASSM-ER, you asked us (1) to assess the results of testing, whether the causes of previous flight test failures have been corrected, and Air Force initiatives to improve JASSM's reliability; and (2) to identify the extent JASSM costs have changed over time, the steps the Air Force is taking to control costs, and additional cost risks, if any, for the program. We note that the production decision on JASSM-ER is being made shortly after the Secretary of Defense announced a major initiative to improve the cost efficiency of weapon systems acquisition. To determine JASSM's current costs and the extent they have changed, we analyzed JASSM's contracts, budgets, and selected acquisition reports. We compared the cost estimating practices of Office of Secretary of Defense (OSD) cost analysts in the development of the life-cycle cost estimates for JASSM and JASSM-ER to our Cost Estimating and Assessment Guide.[Footnote 2] To assess the results of the most recent tests and to determine if corrective actions have been implemented, we analyzed JASSM flight test results and the results of the failure review boards to determine scoring criteria and results and what corrective actions were implemented. We interviewed officials with the JASSM joint program office, Lockheed Martin--the prime contractor, the Air Force, and Office of Secretary of Defense. We interviewed a former Air Force official who was an early JASSM program manager to better understand the program's objectives and original acquisition strategy. We also reviewed our prior work on best acquisition practices which established a set of evaluation criteria for a knowledge-based acquisition approach.[Footnote 3] We conducted this performance audit from November 2009 through October 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: The JASSM program began in 1995 and was to be an affordable, joint program between the Air Force and the Navy to meet an urgent need with a streamlined acquisition strategy. JASSM predecessor TSSAM was also planned to be a low-cost cruise missile able to deliver several different munitions. However, after several unsuccessful flight tests, the lead contractor for TSSAM initiated a reliability improvement program to address higher reliability requirements, but demonstration of whether problems had been resolved would have taken several years and cost more than $300 million. As costs for TSSAM increased, the Army ended its participation in the program and after a period of declining budgets and changes to threat scenarios, a cost and operational effectiveness analysis was completed, which showed that other options might be adequate to meet national security requirements. In 2004, the Navy left the JASSM program citing it as a redundant capability to other systems in its inventory. JASSM was expected to require minimal maintenance while in storage and life- cycle cost was to be controlled through improved reliability and supportability achieved during development. Figure 1: JASSM Missile Reliability Assessment Flight Test: [Refer to PDF for image: photograph] Source: JASSM program office. [End of figure] To execute the acquisition strategy and meet cost and schedule goals, the Air Force used Total System Performance Responsibility (TSPR). [Footnote 4] TSPR generally gives the contractor total responsibility for the entire weapon system and for meeting DOD requirements, with minimum government oversight. The Air Force made initial JASSM requirements flexible to allow Lockheed Martin to have clear control of the design and product baseline. Program officials stated this strategy was based on other successful programs, such as the Joint Direct Attack Munition program, and would allow the contractor flexibility to make changes to meet cost and schedule deadlines without having to consult with the government. An example of this flexibility was the mission missile effectiveness requirement. The effectiveness requirement is the minimum number of missiles required to kill specified targets and was named as a key performance parameter, allowing trades between reliability, survivability, and lethality. In other words, if the program was successful at achieving high levels of survivability and lethality, reliability could remain low, even fluctuate, and still meet the stated parameters. Quantities for JASSM were established by reviewing the threshold targets and determining the number of missiles necessary to meet operational damage criteria, based on missile performance using the effectiveness requirement. Therefore, changes to reliability would affect the quantities necessary to meet requirements. Figure 2: Initial JASSM Program Schedule: [Refer to PDF for image: illustrated schedule] FY 1994, Q4: TSSAM termination. FY 1995, Q4-FY 1996 Q3: Proposal. FY 1995, Q3: Milestone A. FY 1995, Q4-FY 1998 Q4: Risk reduction (24 months). FY 1998, Q3: MIlestone B. FY 1998, Q4-FY 2001, Q2: Development (32 months). FY 1999, Q4-FY 2000, Q4: 4 flight tests. FY 2000, Q3: Low-rate production. FY 2001, Q3: Milestone C. FY 2001, Q3: Lot 1, 87. FY 2002, Q3: Lot 2, 91. FY 2003, Q3: Lot 3, 242. FY 2004, Q3: Lot 4, 340. FY 2005, Q3: Lot 5, 346. FY 2006, Q3: Lot 6, 360. FY 2007, Q3: Lot 7, 360. FY 2008, Q3: Lot 8, 360. FY 2009, Q3: Lot 9, 214. Source: GAO analysis of DOD data. [End of figure] As part of the program's 1995 acquisition strategy, the Air Force received five proposals for JASSM and in 1996 selected Lockheed Martin and McDonnell Douglas to begin a 24-month risk-reduction phase. Following the risk-reduction phase, the Air Force planned 32 months for development and a total of 56 months from program start to full- rate production in 2001. The program planned for concurrent developmental and operational testing and evaluation with four flight tests planned before initial production. The Air Force planned to have nine fixed-price production lots from 2001 through 2009 totaling 2,400 baseline missiles with an initial program cost estimate of $2.2 billion (fiscal year 2010 dollars). A former Air Force official who was an early JASSM program manager stated the Air Force accepted Lockheed Martin's proposal, which included favorable fixed-price contract prices for production lots 1 through 5 with the understanding that the prices would increase after Lot 5. JASSM's acquisition strategy planned for a 74 percent unit cost increase between Lots 5 and 6. The cost increase between Lots 5 and 6 was to occur at a time when quantities were increasing. Despite this planned cost increase, the production unit costs would have remained within the Air Force's acceptable range established before the competition and at much less cost than TSSAM. Further, the prices offered by Lockheed Martin for the first five production lots were below the Air Force's desired cost range for the system. Air Force officials said the low costs contributed to Lockheed Martin's selection. However, to maintain the benefits of this pricing, the quantities purchased by the Air Force had to remain within a certain range for each of the first 5 years. Figure 3: JASSM's Original Acquisition Strategy: [Refer to PDF for image: illustrated line graph] Air Force planned range for unit cost projection: $500,000-$900,000. Fiscal year: 2001 Unit cost projection: $395,000; Lot: 1; Quantity: 87. Fiscal year: 2002 Unit cost projection: $397,000; Lot: 2; Quantity: 91. Fiscal year: 2003 Unit cost projection: $354,000; Lot: 3; Quantity: 242. Fiscal year: 2004 Unit cost projection: $355,000; Lot: 4; Quantity: 340. Fiscal year: 2005 Unit cost projection: $357,000; Lot: 5; Quantity: 346. Fiscal year: 2006 Unit cost projection: $620,000; Lot: 6; Quantity: 360. Fiscal year: 2007 Unit cost projection: $606,000; Lot: 7; Quantity: 360. Fiscal year: 2008 Unit cost projection: $606,000; Lot: 8; Quantity: 360. Fiscal year: 2009 Unit cost projection: $594,000; Lot: 9; Quantity: 214. Source: GAO analysis of DOD data. Notes: The increase between Lot 5 and Lot 6 represents a 74 percent increase in unit costs. Fiscal year 2010 dollars. Quantities are all Air Force. [End of figure] While the Air Force planned for some cost growth in the original acquisition strategy, the program's cost grew much more than expected. For the first four production lots, the Air Force benefited from the favorable prices in the original contract. However, because of funding limitations, it was not able to procure the minimum missile purchase in Lot 5 and had to renegotiate this lot with Lockheed Martin. In doing so, Lockheed Martin was able to renegotiate Lot 5 prices based on its actual production costs--at over $1 million per missile. Air Force documentation indicates that previously negotiated unit prices for Lot 1 through Lot 5 were as much as 45 percent less than Lockheed Martin's actual costs. Subsequent lots that had not been negotiated under the original contract similarly reflected an increase in price. Most of this cost growth took place prior to 2006, culminating in a critical Nunn-McCurdy unit cost breach late in 2006. According to program documents, several causes have been cited for the critical Nunn-McCurdy unit cost breach: an unrealistic cost estimate resulting from a flawed acquisition strategy; the addition of 2,500 more expensive JASSM-ER variants; the costly efforts to overcome reliability problems; and reduced annual production rates for a longer period. Following the critical Nunn-McCurdy unit cost breach, the Air Force halted production of the missiles until DOD certified that the program should continue. The Under Secretary of Defense for Acquisition, Technology, and Logistics (USD/ATL) found no lower cost alternatives and certified the program in 2008, despite the missile's higher than projected production costs. Since operational testing began in 2001, the reliability of the JASSM missile has been inconsistent. The Air Force flight-tested 62 baseline missiles from January 2001 through May 2007, resulting in 25 failures and 3 "no tests," which was a 58 percent reliability success rate. However, because the program's strategy allowed for the contractor to manage to mission effectiveness by combining reliability with other factors, the 58 percent reliability rate was sufficient to meet mission effectiveness criteria. The Air Force tracked reasons for flight test failures, but was not part of the failure review boards until production Lot 5, 5 years after the start of operational testing. Air Force officials stated that until 2006, Lockheed Martin handled all flight test failure review determinations and made corrective actions internally and the government was not heavily involved. During the Nunn-McCurdy certification process, USD/ATL directed the JASSM program to develop a reliability growth plan that would achieve 90 percent reliability for the baseline missile. The program set a goal of achieving this reliability rate by Lot 11, or fiscal year 2013. In addition, the JASSM-ER program set a reliability goal of 85 percent by Lot 4, or fiscal year 2014. In our 2000 report on JASSM, we recommended the Secretary of Defense revise its acquisition strategy for the JASSM program to be more closely linked to demonstrating that the missile design is stable and can meet performance requirements before making the production decision.[Footnote 5] DOD partially concurred with our recommendation stating that its acquisition strategy is directly linked to knowledge points, that it is linked to specific criteria established for making the low-rate initial production decision, and that the contractor is required to meet these criteria. We concluded that, while the Air Force had taken steps to link production decisions for JASSM to knowledge, we did not believe that the specific criteria established to support a production decision were sufficient to minimize cost and schedule risk. In June 2010, the Secretary of Defense announced an initiative to restore affordability and productivity in defense spending. He stated that there is a need to abandon inefficient practices accumulated in a period of budget growth and learn to manage defense dollars in a manner that is "respectful of the American taxpayer at a time of economic and fiscal stress." He set a goal to save $100 billion over the course of the 5 year defense planning period. Subsequently, USD/ATL has issued guidance on delivering better value to the taxpayer and improving the way DOD does business. That guidance indicated that budget savings could be found by eliminating unneeded and costly programs and activities as well as by conducting needed programs and activities more efficiently, such as by stabilizing production rates. Subsequently, in a September 14, 2010, memorandum, USD/ATL provided specific guidance to acquisition professionals to achieve this mandate. That guidance included 23 principle actions to improve efficiency, including "Mandate affordability as a requirement" and "Drive productivity growth through Will Cost/Should Cost management." Corrective Actions Have Led to Improved JASSM Test Results but Operational Effectiveness Is Still to Be Demonstrated: Since 2007, the Air Force has enhanced its oversight of the JASSM program and made significant investments to improve its reliability as directed by USD/ATL. As a result of increased reliability testing and investments in reliability initiatives, the Air Force has identified many of the root causes for flight test failures. Since then, design changes and other corrective actions have improved JASSM baseline's test results significantly--now demonstrating 85 percent success. The JASSM-ER variant has done well thus far, with no scored failures during the first seven flight tests. However, while JASSM baseline missile reliability has improved, it is not expected to achieve the USD/ATL-required level of 90 percent until 2013, and its operational effectiveness has not yet been demonstrated either through operational testing or use in a combat operation. Enhanced Oversight and Increased Investments Have Led to Corrective Actions and Improved JASSM Reliability: In 2004, after two back-to-back flight test failures, the Air Force formed a reliability enhancement team to address what it considered the loss of confidence in JASSM's performance, OSD's concerns about the program, and budget reductions. The team's report stated that while JASSM's development and reliability were within acceptable ranges when compared to other cruise missiles, the JASSM program should increase testing to discover additional weaknesses in design or production as well as increase confidence in the level of reliability achieved and tie those results to contractor incentives. In 2007, after direction from USD/ATL and the Air Force during the Nunn-McCurdy certification process, the program office updated the Joint Reliability and Maintainability Evaluation Team and Test Data Scoring Board charters to significantly expand their role in management of system development, manufacturing, configuration changes, and testing. Since 2007, as a result of the Air Force's increased attention to reliability testing and investments in reliability initiatives, the program has identified many of the root causes for reliability failures. While there is no single cause behind JASSM flight test failures, common failures occurred across JASSM subsystems including navigation, flight control, and propulsion. Figure 4: JASSM Subsystems Involved in Flight Test Failures: [Refer to PDF for image: pie-chart] Payload: 21.0%; Navigation: 16.1%; Flight Control: 14.0%; Propulsion: 14.0%; Other: 9.8%; Unverified: 6.3%. Missile: 4.9%. Source: GAO analysis of DOD data. Notes: Other includes airplane, power, and missile control unit. Unverified is due to ongoing investigations. [End of figure] Most of the corrective actions to address the causes of the flight test failures affect missile hardware and many have been implemented in the current configuration for new production missiles. However, some flight test failure investigations are still ongoing. Those investigations are often difficult because of the lack of physical evidence after the flight test missile detonates on the White Sands missile range. As a result, identifying the root causes for failures were based on very extensive component testing at supplier facilities. Additionally, the root causes for several test failures were never conclusively determined as the failures may have resulted from aircraft or user malfunctions. Efforts to address significant reliability problems found during testing have contributed greatly to JASSM's cost growth and schedule delays since the beginning of development. The Air Force has estimated that it may ultimately spend about $400 million through fiscal year 2025 on its reliability improvement initiatives. The Air Force has also increased lot acceptance testing of the fuses and implemented high-speed photography and screening improvements. In addition to forensic evaluation of the missile impact area, the Air Force also employs visual inspections and a built-in-test.[Footnote 6] The Air Force has also taken a variety of actions, in addition to flight testing, to improve JASSM's reliability, including the following initiatives. * Increased Oversight: The Air Force and Lockheed Martin have begun a process verification program to ensure suppliers follow prime contractor specifications. According to Air Force officials, the process verification program has allowed the Air Force to avoid unforeseen costs as some missile parts have become obsolete. Further, officials stated that it allows the JASSM program to catch problems earlier and plan on how to replace parts sooner. According to a program official, one process verification program team caught an obsolescence issue with a global positioning satellite receiver and was able to minimize the cost and production effect on the program. * Missile Redesign: Program officials state that while wholesale missile redesign is not considered a cost-effective option, they are considering design changes and improvements at the component level. * Increased Personnel: The program office has increased the number of government personnel supporting the process verification program and corrective action efforts. During Lot 1, the program had two staff members with production and manufacturing engineering expertise--by Lot 7, 22 staff members had such expertise. * Improved Quality Assurance: In August 2006, the Air Force and Lockheed Martin implemented a quality assurance program. Lockheed Martin has implemented tests and improvement programs to increase user confidence in reliability and control costs. For example, Lockheed Martin officials stated that, to improve reliability, they have begun using a test that exposes electrical connections to higher voltages than they usually encounter during flight to make sure the wiring can handle a surge. Additionally, Lockheed Martin has increased the sample sizes of certain components they inspect and test. Recent JASSM Testing Has Shown Improved Results: Recent tests of JASSM have demonstrated increased reliability. Since the Air Force's reliability initiatives began in fiscal year 2007, the JASSM program has conducted 48 missile flight tests and 39 have been successful (2 were characterized as "no-test") for a reliability rate of 85 percent. Figure 5: JASSM Baseline Missile Test Results: [Refer to PDF for image: stacked vertical bar graph] Number of tests: Fiscal year: 2001; Successful: 4; Failures: 2. Fiscal year: 2002; Successful: 6; Failures: 1. Fiscal year: 2003; Successful: 7; Failures: 7. Fiscal year: 2004; Successful: 3; Failures: 4. Fiscal year: 2005; Successful: 8; Failures: 3. Fiscal year: 2006; Successful: 6; Failures: 3. Fiscal year: 2007; Successful: 0; Failures: 5. Fiscal year: 2008; Successful: 17; Failures: 2. Fiscal year: 2009; Successful: 17; Failures: 5. Fiscal year: 2010; Successful: 5v Failures: 0. Source: GAO analysis of DOD data. [End of figure] The current focus of JASSM baseline testing has been on improving the reliability of the missile. In the most recent tests of the JASSM baseline missiles produced in 2008, 15 of 16 flight tests were considered successful. In the one failure, the warhead did not detonate and the program is awaiting fuse recovery to make a determination of the root cause. The JASSM-ER is in developmental testing. Developmental testing of JASSM-ER is primarily addressing the differences of JASSM-ER from the baseline system (i.e., larger engine and fuel tanks) and will verify integration on the B-1 aircraft. All seven test flights of JASSM-ER have been successful.[Footnote 7] The program office is planning three additional integrated JASSM-ER tests to be flown before a production decision is made. Uncertain If Improved Missile Reliability Will Result in Improved JASSM Operational Effectiveness: In 2007, after USD/ATL's decision to enhance JASSM reliability, the Air Force and Lockheed Martin agreed to focus on the inherent reliability of the missile and not take into account user error or platform malfunctions (i.e., carrier aircraft, aircrew instrumentation, range safety, etc.). Whereas operational testing is designed to evaluate the ability of JASSM to execute a mission, reliability testing is more narrowly focused on evaluating the missile's performance during the mission. While mission failures were counted against the program during initial testing, more recent mission test failures have been declared "no tests." For example, in early testing when a B-52 software issue resulted in an aborted mission and it was scored a test failure, this event would have been declared a no test under current missile reliability definitions. While recent flight testing of the baseline missile has shown improved missile reliability, the Air Force has not yet evaluated the operational effectiveness and suitability of the baseline JASSM with all corrective actions implemented.[Footnote 8] The JASSM program assesses operational effectiveness through operational testing, follow- on testing, and the weapon system evaluation program (routine tests of inventory assets). These flight test scenarios assess operational effectiveness in realistic combat scenarios against targets by determining reliability, evaluating capability and limitations by identifying deficiencies, and recommending corrective actions. In operational testing, the JASSM baseline program flight tested 38 missiles from June 2002 through May 2007 resulting in 19 failures and two no tests. While these tests identified issues with missile reliability, they also identified issues related to the B-52 aircraft, aircraft software, and fuse issues which negatively affected the operational effectiveness of the missile. This led to a 9-month suspension of testing in 2004 to address these issues. The improved JASSM baseline missile's suitability was assessed by Air Force testers in 2008 and it was characterized as suitable and likely to meet reliability goals; however, operational testing of the effectiveness of the improved missile has not yet been scheduled. Several Areas of Potential Cost Risk Could Further Increase JASSM Program Costs: Current projections of JASSM costs have increased by over 7 percent since the Nunn-McCurdy certification in 2008. When taking into consideration the pre-2008 cost growth, which included the cost of adding the JASSM-ER variant, JASSM has grown from a $2.2 billion to a $7.1 billion program. In addition, while it has initiated several cost control measures, the Air Force appears to have limited options to reduce JASSM costs. Moreover, several areas of risk could add to those costs. First, the Air Force has not been able to provide enough annual funding to support the annual procurement levels used as the basis for its 2008 program cost estimate. That has led to a less efficient production process and a longer production period (most recently extended 5 years to 2025). Second, until the Air Force evaluates the effectiveness of the inventory JASSM baseline missiles with corrective actions for previously identified hardware and software issues, [Footnote 9] their viability and military utility is in question. If inventory missiles are found not to have utility, they may need to be replaced. If retrofitted missiles are found to be effective, the Air Force may still have to find additional funding to complete the retrofit process. Third, the Air Force plans to conduct many more flight tests to improve JASSM reliability from 85 to 90 percent. Finally, in comparing the capabilities and cost of JASSM to several domestic and international missile systems in 2008, the Air Force assumed that JASSM would cost about $1 million per unit, which is about 40 percent less than currently expected. JASSM Costs Are Much Higher Than Originally Anticipated: Compared to original program estimates, JASSM's currently projected costs are much higher because of (1) higher than anticipated production costs, (2) longer production period, (3) the addition of the JASSM-ER variant, and (4) reliability improvement efforts. Through fiscal year 2010, about 75 percent of the planned JASSM quantities have yet to be procured and, as a result, most of the program costs have yet to be incurred. Figure 6: Comparison of JASSM's Original and 2009 Total Acquisition Funding Profiles: [Refer to PDF for image: multiple line graph] Fiscal year: 1996; December 1988 selected acquisition report: $34.4 million; December 2009 selected acquisition report: $34.4 million. Fiscal year: 2000; December 1988 selected acquisition report: $200.7 million; December 2009 selected acquisition report: $188.2 million. Fiscal year: 2004; December 1988 selected acquisition report: $166.9 million; December 2009 selected acquisition report: $164 million. Fiscal year: 2008; December 1988 selected acquisition report: $192.3 million; December 2009 selected acquisition report: $173.4 million. Fiscal year: 2012; December 1988 selected acquisition report: $0; December 2009 selected acquisition report: $231.2 million. Fiscal year: 2016; December 2009 selected acquisition report: $380.8 million. Fiscal year: 2020; December 2009 selected acquisition report: $335.2 million. Fiscal year: 2024; December 2009 selected acquisition report: $326.5 million. Fiscal year: 2028; December 2009 selected acquisition report: $0. Source: GAO analysis of DOD data. Notes: Fiscal year 2010 dollars. The $7.1 billion includes 2,500 JASSM- ER missiles. [End of figure] Following the critical Nunn-McCurdy unit cost breach, the Air Force halted production of the missiles until USD/ATL certified that the program should continue. USD/ATL found no lower cost alternatives and certified the program in 2008, despite the missile's higher than projected production costs. As a part of our review, we examined the cost estimates used by OSD to certify the program following the critical Nunn-McCurdy unit cost breach. This estimate used the actual costs of the missile since JASSM was well into the production phase at the time. Overall, the Air Force's cost estimate substantially met our best practice standards in our Cost Guide. For a more in-depth discussion of our review of this JASSM cost estimate, see appendix III. Since the Nunn-McCurdy certification in 2008, the growth in JASSM's projected program cost has moderated, rising about $500 million (from $6.6 billion to $7.1 billion) through 2025. Reliability enhancements to the JASSM missile instituted in 2007 and additional reliability testing have added the majority of the increase in program costs. These enhancements were implemented to meet USD/ATL's 90 percent reliability goal which was set during the Nunn-McCurdy certification. Also, the Air Force decided to lengthen the program's procurement schedule by another 5 years, buying the same number of missiles over a longer time period. That reduces the efficiency of the production processes and adds inflation to the cost estimate. Currently, on a per unit basis, the average procurement unit cost of a JASSM missile is projected to be about $1.2 million. JASSM-ER is expected to cost about $200,000 more than the average, about $1.4 million per unit.[Footnote 10] Table 1: Changes Since 2008 in JASSM Estimated Cost, Schedule, and Quantities through 2025: Expected quantities: Development quantities; May 2008 (Nunn-McCurdy certification): 106; December 2009 (selected acquisition report): 118; Percentage Change (2008 to 2009): 11.3%. Expected quantities: Procurement quantities; May 2008 (Nunn-McCurdy certification): 4,900; December 2009 (selected acquisition report): 4,900; Percentage Change (2008 to 2009): 0.0. Expected quantities: Total quantities; May 2008 (Nunn-McCurdy certification): 5,006; December 2009 (selected acquisition report): 5,018; Percentage Change (2008 to 2009): 0.2%. Cost estimates (dollars in millions): Development; May 2008 (Nunn-McCurdy certification): $1,383.5; December 2009 (selected acquisition report): $1,447.8; Percentage Change (2008 to 2009): 4.6%. Cost estimates (dollars in millions): Procurement; May 2008 (Nunn-McCurdy certification): $5,257.9; December 2009 (selected acquisition report): $5,681.6; Percentage Change (2008 to 2009): 8.1%. Cost estimates (dollars in millions): Total program acquisition; May 2008 (Nunn-McCurdy certification): $6,641.4; December 2009 (selected acquisition report): $7,129.4; Percentage Change (2008 to 2009): 7.3%. Unit cost estimates (dollars in thousands): Program acquisition; May 2008 (Nunn-McCurdy certification): $1,327; December 2009 (selected acquisition report): $1,421; Percentage Change (2008 to 2009): 7.1%. Unit cost estimates (dollars in thousands): Average procurement; May 2008 (Nunn-McCurdy certification): $1,073; December 2009 (selected acquisition report): $1,160; Percentage Change (2008 to 2009): 8.1%. Estimated delivery dates: Production length (years); May 2008 (Nunn-McCurdy certification): 20; December 2009 (selected acquisition report): 25; Percentage Change (2008 to 2009): 25.0%. Source: GAO analysis of DOD data. Note: Fiscal year 2010 dollars. [End of table] Since 2008, the Air Force has added several measures to control costs in the JASSM program, but the effect of these measures is not yet clear. Examples of these measures include the following. * Contract Incentives: The Air Force has begun using fixed-price incentive (firm target) contracts for each lot to produce the missiles for less than projected.[Footnote 11] * Greater Insight into Actual Costs: Air Force officials have increased insight into Lockheed Martin's actual costs, which may make them more informed when negotiating new contracts. Program officials stated that, for example, they now know how many engineers are needed to perform a certain task and the number of hours it takes to assemble a missile. The Air Force can directly verify the costs charged by subcontractors. * Increased Authority over Design: In recent contract negotiations, the Air Force gained approval authority over certain design changes that may affect current and future lots, including those that may increase cost, require retrofit, or affect safety. Previously, Lockheed Martin had full authority over most design changes. While the effectiveness of the cost control measures is not yet known, the Air Force appears to have limited options to actually reduce those costs. For example, annual production rates are expected to remain well below the levels projected at the start of the program. The 2008 Air Force cost estimate was based on an annual production rate of 280 missiles per year. However, that cost estimate may now be understated because the program has not produced that many missiles in a single year since 2005. For example, the Air Force's procurement quantities for production Lot 7 and Lot 8 were 111 and 80 units, respectively, well below the economic order quantity of 175 missiles per year. Program officials stated that annual quantities below the economic order quantity will result in an increasingly inefficient production process and some key suppliers may shift from continuous to limited production. Further, Lockheed Martin officials stated that low production rates could cause skilled labor to look elsewhere for work and JASSM reliability could be adversely affected. The contractor has been able to maintain some level of production efficiency because of foreign military sales that make up for the reduced Air Force procurements. However, lower than projected annual procurement levels will increase production costs. A further challenge is the fact that JASSM's design is mostly complete and there may be few opportunities to reduce production costs through redesign. As a result, average JASSM unit costs may remain in excess of $1.2 million indefinitely. Effectiveness of Retrofitting Corrective Actions into JASSM Inventory Is Uncertain: The Air Force has plans to address the low reliability of missiles in its inventory by retrofitting some of its 942 missiles with hardware and software corrective actions. Program officials state the retrofit costs will be shared between Lockheed Martin and the Air Force, but the total cost to retrofit the missiles in inventory has not been calculated. However, previous efforts to retrofit JASSM missiles have proven to be problematic. An example of challenges associated with retrofitting missiles is adding telemetry instrumentation kits after the missiles have been produced and are in the inventory. Those kits are added to all missiles to be flight tested.[Footnote 12] This requires opening up the missile to insert telemetry after the stealth coating has been applied and increasing the number of electrical connections as compared to a production missile.[Footnote 13] Air Force officials stated the kit could add some reliability concerns when it is added to test missiles because the missiles were not designed to be opened after they were completed. Air Force officials also stated that workers have to reroute wires and remove the engine so that the self-destruct mechanism can be installed and all of this rework inside the missile has the potential to lead to more errors and cause additional reliability issues. The impact of retrofitting missiles has become evident in the weapon system evaluation program, which is operationally representative flight testing run by users of the system and focuses on the performance of missiles in the inventory. JASSM's performance in this evaluation program has not been good, with 7 failures in 12 tests from 2006 through 2007, and with at least some of the failures attributable to the retrofit process. The addition of telemetry kits has also contributed to 3 no tests during other JASSM flight testing. The Air Force has not yet flight tested any of the JASSM inventory missiles that have been retrofitted with all of the corrective actions to address reliability issues. This type of test would be important in determining the viability of the current inventory of JASSM missiles and would be a key input in the Air Force deciding whether or not to retrofit the entire inventory of missiles. Additional Flight Testing to Further Improve Reliability Will Be Costly: The Air Force plans more flight tests in the next few years of new production missiles to meet missile reliability goals. For the JASSM baseline missile to meet its reliability requirement, the Air Force is planning to conduct up to 48 additional flight tests, at a cost of about $120 million. In addition, most reliability issues with the baseline variant will directly affect the progress of the JASSM-ER variant as the missiles are at least 70 percent common in hardware and 95 percent common in software. Anything learned during these flight tests about the baseline applies to JASSM-ER. According to the Air Force, as many as 20 additional flight tests may be needed to fully demonstrate JASSM-ER's reliability goal of 85 percent.[Footnote 14] Table 2: Flight Test Cost to Increasing Reliability: Variant of JASSM: Baseline; Additional flight tests required to reach reliability requirement: 48[A]; Current cost per flight test: $2.5 million; Additional flight test costs to reach reliability requirement: $120 million. Variant of JASSM: Extended range; Additional flight tests required to reach reliability requirement: 20; Current cost per flight test: $3.5 million; Additional flight test costs to reach reliability requirement: $70 million. Source: GAO analysis of DOD data. [A] The JASSM baseline program can achieve its missile reliability goal with either (1) 21 successful reliability test shots, or (2) 3 more sets of reliability tests with 16 test shots each--with no more than two failures in 1 set of reliability tests and not more than 1 failure in each of the other 2 sets of reliability tests. [End of table] The $190 million cost to achieve the final percentages of missile reliability reflects the fact that problems or weaknesses become harder to find and correct as the more obvious issues are corrected. Program officials are considering alternative means to meet user needs for a more reliable missile while reducing the cost of JASSM flight testing. Previous Analysis of Alternatives Assumed Much Lower Costs of JASSM Missile: As part of the Nunn-McCurdy certification process in fiscal years 2007 through 2008, DOD assessed whether there were readily available alternatives that provided as much or more capability as JASSM at lower cost. DOD assessed programs ranging from direct attack munitions to intercontinental range missiles. For the JASSM baseline missile, all of DOD's existing programs were found to be less effective in terms of lethality, survivability, or capacity. The Navy's Tomahawk missile was the closest alternative to meeting JASSM's capability but it is not as lethal as JASSM. Also, the Tomahawk is launched from ships and not from aircraft, as the Air Force plans to use the capability. DOD also evaluated new or modified programs as possible alternatives to JASSM and JASSM-ER. The Air Force evaluated 12 domestic and international missile systems with projected unit production costs ranging from $600,000 to $2.8 million. JASSM-ER, estimated at the time of this evaluation to cost $1 million per unit, was more expensive than 5 alternative systems under consideration. In terms of performance, some alternatives were more capable than JASSM and some were not. All of the alternative systems were expected to require some up-front investment. Based on this analysis, no other alternative was found to provide greater or equal military capability at less cost than JASSM-ER. Later in the Nunn-McCurdy process, however, OSD cost analysts found that the costs of JASSM-ER would likely be at least $1.4 million per missile. That continues to be the projected unit cost of JASSM-ER and, as we discussed earlier in this report, there are cost risks that may drive that unit cost higher. Despite the higher production unit costs for JASSM-ER, the Air Force has not revisited the results of its assessment of alternatives. In light of the current cost projections, which are 40 percent higher than assumed in the previous assessment, JASSM-ER would be equal to the cost of an additional alternative. Further, the unit cost differential between JASSM-ER and the lower- priced alternatives may now be large enough to make those alternatives more competitive in terms of cost or capabilities. Conclusions: DOD's 25-year history to acquire and field an affordable air-to- ground cruise missile has been a difficult one. After abandoning the expensive TSSAM program, the Air Force conceived the JASSM program using an acquisition strategy that minimized government oversight. After restructuring the program in 2008 and after considerable effort to improve reliability, the JASSM program as it exists today is much different than originally envisioned. A $2.2 billion, 11-year program to produce 2,400 missiles has become a $7.1 billion, 28-year program to produce 4,900 missiles. From a technical and capability standpoint, the program offers more now than the baseline missile did before 2008. Yet, the effectiveness of the new missiles remains to be demonstrated in operational testing, and low production rates, retrofit costs, and additional reliability testing could drive program costs higher. At this point, about 70 percent of the projected JASSM costs have not yet been incurred. In November 2011, DOD will decide whether to approve the Air Force's request to start low-rate initial production of the JASSM-ER variant. Low-rate initial production is normally the last major milestone decision for an acquisition program. With the JASSM program now expected to extend through 2025 and about $5 billion yet to be spent, a reevaluation of its cost-effectiveness is warranted before such a commitment is made. This is particularly true given the Secretary of Defense's recent initiative to improve the cost- efficiency of defense acquisition programs. At this juncture, the JASSM program would seem to be an excellent opportunity for DOD and Air Force leadership to take a hard look at the cost-effectiveness and efficiency of this important but costly defense program. Recommendation for Executive Action: We recommend that the Secretary of Defense defer the production decision for JASSM-ER until (1) the program's likely costs and affordability are reassessed to take into account the feasibility and cost of retrofitting JASSM baseline missiles or replacing them, the cost of additional reliability testing against the likely improvement, and the effect of sustained low production rates; and (2) the results of the previous analysis of alternatives are reassessed in light of the likely costs of the JASSM program. Agency Comments and Our Evaluation: In its comments on our draft report, DOD partially concurred with our recommendation. DOD stated that JASSM-ER is on track for a Milestone C low-rate initial production decision in November 2010. DOD also agreed that the rate of JASSM production has not been optimum and that it plans to address efficient production rates as part of the JASSM-ER Milestone C decision. DOD also stated that (1) there are no additional plans (nor is there a need) to retrofit fielded JASSMs above what has already been accomplished or is under way; (2) it has revisited various alternatives and reaffirms the continued validity of its 2008 conclusion that none of the alternative concepts provide comparable operational utility at or near a similar cost or schedule to JASSM; and (3) in the absence of viable alternatives, delaying the program further will increase costs and further postpone delivering a vital capability to the warfighter. DOD's response is reprinted in appendix II. In concluding that retrofits to the inventory missiles may not be necessary, DOD does not address the viability of the current inventory of JASSM baseline missiles or the need to replace some or all of them. Until the Air Force evaluates the effectiveness of the inventory of JASSM baseline missiles with corrective actions for previously identified hardware and software issues, their viability and military utility will still be in question. In addition, DOD states that the Air Force has revisited its earlier assessment of alternatives to JASSM and again found that there are none with comparable utility, cost, or schedule. This is new information and DOD did not provide details for us to assess, including whether the Air Force factored in the higher current projections of JASSM costs. Finally, DOD did not address the part of our recommendation dealing with the cost of additional reliability testing against the likely improvement. To the extent DOD has made decisions on retrofits and reconsideration of alternatives, these are positive signs, as is its agreement to address the efficiency of production rates. At this point, it is not clear whether the reliability of the existing baseline inventory missiles is acceptable or whether additional reliability testing is warranted. These determinations are necessary to establish the full value and cost of the JASSM program. Beyond these steps, it is incumbent upon the department to reexamine JASSM before making the production decision to ensure that the program is structured as efficiently as possible and is still a good investment given the other demands DOD faces. DOD's agreement to address the efficiency of JASSM production rates is a positive step. This is particularly important given the Secretary's current efficiency and affordability initiative. DOD needs to ensure that it has the information available to fully assess the JASSM investment before making the production decision. If DOD needs more time, then we believe the decision could be delayed. We also received several technical comments from DOD and the Air Force and have made other changes to our report. We are sending copies of this report to the Secretary of Defense and interested congressional committees. In addition, this report will be made available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report or need additional information, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV. Signed by: Michael J. Sullivan, Director: Acquisition and Sourcing Management: List of Congressional Committees: The Honorable Joseph Lieberman: Chairman: The Honorable John Thune: Ranking Member: Subcommittee on Airland: Committee on Armed Services: United States Senate: The Honorable Daniel K. Inouye: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Adam Smith: Chairman: The Honorable Roscoe Bartlett: Ranking Member: Subcommittee on Air and Land Forces: Committee on Armed Services: House of Representatives: The Honorable Norm Dicks: Chairman: The Honorable C.W. Bill Young: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Appendix I: Objectives, Scope, and Methodology: To determine Joint Air-to-Surface Standoff Missile's (JASSM) current production unit costs and the extent they have grown, we analyzed JASSM's contracts, budgets, and compared the program's selected acquisition reports. We analyzed Nunn-McCurdy documentation and certification criteria as well as Air Force and Lockheed Martin data to determine the causes for the cost growth and critical breaches. We interviewed officials with the JASSM joint program office; Lockheed Martin; the Office of Secretary of Defense Cost Assessment and Program Evaluation; and a former program official. To determine the extent to which Department of Defense's (DOD) cost estimating policies and guidance support the development of high- quality cost estimates, we analyzed the cost estimating practices of the Cost Analysis Improvement Group (CAIG), now known as the Cost Analysis and Process Evaluation (CAPE), in the development of life- cycle cost estimates for the Air Force's Joint Air-to-Surface Standoff Missile Program (baseline and JASSM-ER variants), against the 12 best practices of a high-quality cost estimate as defined in our Cost Estimating and Assessment Guide.[Footnote 15] We assessed each cost estimate, used in support of the critical Nunn- McCurdy unit cost breach, against these 12 key practices associated with four characteristics of a reliable estimate. As defined in the guide, these four characteristics are comprehensive, well-documented, accurate, and credible, and the practices address, for example, the methodologies, assumptions, and source data used. We also interviewed program officials responsible for the cost estimate about the estimate's derivation. We then characterized the extent to which each of the four characteristics was met; that is, we rated each characteristic as being either Not Met, Minimally Met, Partially Met, Substantially Met, or Fully Met. To do so, we scored each of the 12 individual key practices associated with the four characteristics on a scale of 1-5 (Not Met = 1, Minimally Met = 2, Partially Met = 3, Substantially Met = 4, and Fully Met = 5), and then averaged the individual practice scores associated with a given characteristic to determine the score for that characteristic. To determine the results of the most recent tests and if corrective actions have been implemented for previous test failures, we analyzed JASSM flight test results and failure review board findings to determine scoring criteria and results to determine what corrective actions were implemented. We determined whether recent flight test results are representative of the entire fleet by comparing and evaluating the lot-by-lot missile configuration changes and retrofit activities. We interviewed officials with the JASSM joint program office; Lockheed Martin; Office of Under Secretary of Defense for Acquisition, Technology, and Logistics; Director, Operational Test and Evaluation; Air Combat Command; Secretary of the Air Force for Acquisition; Office of Secretary of Defense Cost Assessment and Program Evaluation; Joint Staff; Air Force Directorate of Test and Evaluation; and a former Air Force official who was an early JASSM program manager to better understand the program's objectives and original acquisition strategy. We discussed recent DOD reliability initiatives with Office of the Director, Operational Test and Evaluation officials. To determine what the Air Force has done to control and reduce production costs while improving reliability, we examined JASSM's contracts to see what provisions have been added as well as Air Force and Lockheed Martin data. We interviewed officials with the JASSM joint program office; Lockheed Martin; and Office of Secretary of Defense (OSD) test organizations to determine if testing reflects the current effectiveness of the missile. We reviewed our prior work on best practices for a knowledge-based approach for acquisition programs in determining if the Air Force's approach to beginning the JASSM-ER program meets best practices. We compared requirements and other documents to see if the JASSM-ER missile reflects lessons learned from the baseline variant as well as increased knowledge and oversight from the government. We compared the baseline design with JASSM-ER to determine commonality. We interviewed officials with the JASSM joint program office; Lockheed Martin; Office of Under Secretary of Defense for Acquisition, Technology, and Logistics; Office of Secretary of Defense Cost Assessment and Program Evaluation; Director of Operational Test and Evaluation; Air Combat Command; Secretary of the Air Force for Acquisition; Joint Staff; and a former program official to the determine acquisition planning leading up to JASSM-ER's production decision and what initiatives have been taken to control costs. [End of section] Appendix II: Comments from the Department of Defense: Office Of The Under Secretary Of Defense: Acquisition, Technology And Logistics: 3000 Defense Pentagon: Washington, DC 20301-3000: September 30, 2010: Mr. Michael J. Sullivan: Director, Acquisition and Sourcing Management: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Sullivan: This is the Department of Defense (DoD) response to the GAO draft report, GAO-10-573, "Defense Acquisitions: DoD Needs to Reassess Joint Cruise Missile Costs Before Starting New Production Phase," dated August 25, 2010 (GAO Code 120868). The DoD partially concurs with the recommendation. The rationale for our position is enclosed. We appreciate the opportunity to comment on the draft report. My point of contact for this effort is Mr. Ron Woods, Ronald.Woods@osd.mil, 703- 697-8183. Sincerely, Signed by: David G. Ahern: Director: Portfolio Systems Acquisition: Enclosure: As stated: [End of letter] GAO Draft Report Dated August 25, 2010: GAO-10-573 (GAO 120868): "Defense Acquisitions: DoD Needs to Reassess Joint Cruise Missile Costs Before Starting New Production Phase" Department Of Defense Comments To The GAO Recommendation: Recommendation 1: We recommend that the Secretary of Defense defer the production decision for JASSM-ER until (1) the program's likely costs and affordability are reassessed to take into account the feasibility and cost of retrofitting JASSM baseline missiles or replacing them, the cost of additional reliability testing against the likely improvement, and the effect of sustained low production rates; and (2) the results of the previous analysis of alternatives are reassessed in light of the likely costs of the JASSM program. DOD Response: Partial-concur. Having passed all qualification testing and production readiness reviews, JASSM-ER is on track for a Milestone C Low Rate Initial Production decision in November 2010. JASSM-ER Flight test results have met expectations. The Department agrees with GAO's assessment that the rate of JASSM production has not been optimum. Lower than optimum production rates in Lots 5 through 8 were driven by the need to restructure the program to address reliability concerns resulting from Weapon System Evaluation Program failures in 2006-2007 and Lot 5 test results. The Department accepted the near-term cost impacts of the production delays to lay a solid foundation for production of a more reliable baseline JASSM and future JASSMER low-rate production. Baseline JASSM has been in production since December 2001. JASSM-ER has demonstrated readiness for a production decision. The Department plans to address efficient production rates as part of the JASSM-ER Milestone C decision. Lot 5 and 6 missiles have successfully been retrofitted with new fuzes, cabling and other corrective actions to increase their reliability. Below a minimum acceptable level, the prime contractor continues to bear responsibility for determining, validating and implementing all fixes to return the missile to a specified reliability level. The Air Force has also accomplished retrofits and additional screening on previous blocks of missiles (Lot 1 and part of Lot 2) since 2007. There are no additional plans (nor need) to retrofit currently fielded JASSMs above what has already been accomplished or is under way. The Air Force issued a Request for Information (RFI) on alternate cruise missile concepts and completed an associated analysis in 2007- 2008. The Air Force recently reviewed this analysis and affirmed the continued validity of the 2008 conclusion that none of the alternative concepts provide comparable operational utility at or near a similar cost or schedule. In the absence of viable alternatives (i.e., no other concepts offer similar capability at or near same cost), delaying the program further will increase costs and further postpone delivering a vital stand-off capability to the warfighter. [End of section] Appendix III: JASSM Baseline and JASSM-ER Cost Estimates Met Most Best Practices, but the Risk Analyses Did Not Consider Reliability or Extending the Production Schedule: After reviewing documentation submitted by the JASSM program office, conducting interviews, and reviewing relevant sources, we determined the CAIG's life-cycle cost estimate's totaling $7.1 billion for both programs--the JASSM baseline cost estimate was $3.4 million while the JASSM-ER variant cost estimate of $3.7 million--Fully Met one and Substantially Met the other three characteristics of a reliable cost estimate, as shown in Table 3 below. We assessed 12 measures consistently applied by cost estimating organizations throughout the federal government and industry and considered best practices for the development of reliable cost estimates. We analyzed the cost estimating practices used by CAIG in developing the life-cycle cost estimates for both programs against these 12 best practices and the findings are documented in table 3 below. Table 3: Extent to Which the Cost Estimating Practices for JASSM Baseline and JASSM-ER Reflect GAO Best Practices: Best practice: Comprehensive; Explanation: The cost estimates should include both government and contractor costs of the program over its full life-cycle, from inception of the program through design, development, deployment, and operation and maintenance to retirement of the program. They should also provide a level of detail appropriate to ensure that cost elements are neither omitted nor double-counted, and they should document all cost-influencing ground rules and assumptions; Satisfied?: Substantially Met. Best practice: Well documented; Explanation: The documentation should addresses the purpose of the estimate, the program background and system description, its schedule, the scope of the estimate, the ground rules and assumptions, all data sources, estimating methodology and rationale, the results of the risk analysis, and a conclusion about whether the cost estimate is reasonable. Therefore, a good cost estimate--while taking the form of a single number--is supported by detailed documentation that describes how it was derived and how the expected funding will be spent in order to achieve a given objective. Finally, the cost estimate should be reviewed and accepted by management to ensure that there is a high level of confidence in the estimating process and the estimate itself; Satisfied?: Fully Met. Best practice: Accurate; Explanation: The cost estimate should provide for results that are unbiased and should not be overly conservative or optimistic. Among other things, the estimate should be grounded in a historical record of cost estimating and actual experiences on comparable programs. Estimates are accurate when they are based on an assessment of most likely costs, adjusted properly for inflation, updated regularly, and contain few, if any, minor mistakes; Satisfied?: Substantially Met. Best practice: Credible; Explanation: The cost estimates should discuss any limitations of the analysis because of uncertainty or biases surrounding data or assumptions. Major assumptions should be varied, and other outcomes recomputed to determine how sensitive they are to changes in the assumptions. Risk and uncertainty analysis should be performed to determine the level of risk associated with the estimate. Further, the estimate's results should be crosschecked, and an independent cost estimate conducted by a group outside the acquiring organization should be developed to determine whether other estimating results produce similar results; Satisfied?: Substantially Met. Source: GAO analysis of DOD data. [End of table] The following explains the definitions we used in assessing CAIG's cost estimating methods used in support of the critical Nunn-McCurdy unit cost breach: * Fully Met--JASSM program office provided complete evidence that satisfies the entire criterion; * Substantially Met--JASSM program office provided evidence that satisfies a large portion of the criterion; * Partially Met--JASSM program office provided evidence that satisfies about half of the criterion; * Minimally Met--JASSM program office provided evidence that satisfies a small portion of the criterion; and: * Not Met--JASSM program office provided no evidence that satisfies any of the criterion. The sections that follow highlight the key findings of our assessment. JASSM Baseline and JASSM-ER Estimates Substantially Met Characteristics for Comprehensiveness: Though the cost estimates accounted for all possible costs and were structured in such a manner that would ensure that cost elements were omitted or double-counted, neither the JASSM baseline nor JASSM-ER had a Work Break Down Structure (WBS) dictionary that defined each element.[Footnote 16] In addition, the JASSM baseline variant provided no evidence that risks associated with the ground rules and assumptions were traced back to specific cost elements. * All applicable costs including government and contractor costs were included in the estimates--The cost estimates included sunk costs such as contractor program management, overhead, system design, and development and testing. In addition, the program office outlined the cost estimating methodology, basis of the costs, as well as development costs for JASSM-ER and other government costs. * The cost estimates' level of detail ensure that no costs were omitted or double-counted--The cost estimates are based on a product- oriented WBS which is in line with best practices. For example, the cost estimate is broken down into various components such as the propulsion, payload, airframe, and guidance and control and also includes supporting cost elements such as systems engineering, program management, and system test and evaluation. As a result, all of the system products are visible at lower levels of WBS providing us with confidence that no costs were omitted or double-counted. WBS has been updated as the JASSM baseline and JASSM-ER programs have evolved; however, there is not an accompanying dictionary that defines each element and how it relates to others in the hierarchy. * Ground rules and assumptions were largely identified and documented-- The JASSM baseline cost estimate documentation included a list of risk model inputs based on WBS elements. Although WBS elements such as engineering support, subcontractor, and warranty were identified, there was no discussion of risk upon assumptions that drive costs such as product reliability, sustainability of subcontractors, or schedule variability. Like the JASSM baseline cost estimate documentation, the JASSM-ER cost estimate documentation also included a list of ground rules and assumptions; however, there was evidence that risk associated with the fuel tank assumption was traceable to a specific WBS element. In separate documentation, we were able to identify where the program office considered risks for the JASSM baseline estimate. JASSM Baseline and JASSM-ER Estimates Fully Met Characteristics for Being Well Documented: Both cost estimates were documented in enough detail that would allow an analyst unfamiliar with the program to recreate the estimate and get the same result. In addition, the briefing to management was detailed enough to show that the estimates were credible and well documented. * The cost estimate is fully documented--For the JASSM baseline and JASSM-ER, the cost estimate documentation included a report documentation page identifying the report date, title, contract number, report authors, and other information. The documentation also included a table of contents, introduction, purpose, and structure of the document as well as the scope of the estimate, a list of team members, the cost methodology, and a system description. The documentation discussed a risk and sensitivity analysis, costs broken out by WBS elements including data sources and estimating method and rationale, and provided evidence that the estimates were updated using actual costs. In a separate briefing, the program office outlined the cost estimating methodology, basis of the costs, as well as development costs for JASSM-ER and other government costs. The program office also provided a copy of the cost sufficiency review of the estimate, which included the estimate's purpose and scope, technical description and schedule, ground rules and assumptions, data sources and analysis, and methodology. For both programs, the estimate documentation and the cost analysis requirements document (CARD) addressed best practices and the 12 steps of a high-quality estimate. Contingency reserves and the associated level of confidence for the risk-adjusted cost estimate were also documented. Electronic versions of the cost estimates were also provided. * The estimate documentation describes how the estimate was derived-- The point estimate was developed primarily using actual costs, with a few cost elements estimated based on learning curves method.[Footnote 17] Actual sunk costs for prior years were presented and remaining production lot costs were based on a labor staffing assessment and the latest contractor labor rates. Cross-checks were performed and no instances of double-counting were visible. A separate document was provided that showed in detail how the cost estimate was developed, what data were used to create the cost estimate, and how risks were quantified to determine a level of confidence in the cost estimate. * The estimates were reviewed and approved by management--The estimates were presented by OSD CAIG to the OSD overarching integrated product team for consideration as the new acquisition program baseline. In November 2009, the team provided a detailed overview of the JASSM program which addressed the major cost growth factors, such as the addition of the JASSM-ER variant, reliability enhancements, and the reduction in missile purchases. JASSM Baseline and JASSM-ER Variant Estimates Substantially Met Characteristics for Accuracy: Both cost estimates were unbiased and represented most likely costs. For example, the estimates were adjusted to reflect risks and the program office also included requirements in the new Lot 8 contract that would allow them to update the cost estimates with actual data. * The cost estimates were adjusted for inflation--The JASSM program office used the February 2009 version of the OSD inflation rates provided by the Secretary of the Air Force/Financial Management Cost and Economics. The estimates were developed and documented in base year 1995 dollars and inflated using the weighted rates applicable to the appropriations in the estimate. Base year 1995 is the program's designated base year. * The cost estimates included most likely costs--Per the Nunn-McCurdy certification process, the CAIG developed independent cost estimates for the JASSM baseline and JASSM-ER development and procurement costs as well as future-year resource requirements for the baseline and JASSM-ER variants. Operating and support costs as well as software costs were also included in the estimates. The JASSM baseline life- cycle cost estimate of $3.4 billion, which spans a period of time from 2001 through 2015, was estimated at the 77 percent confidence level, while the JASSM-ER life-cycle cost estimate of $3.7 billion spans the period from 2011 through 2025 and was estimated at the 73 percent confidence level. * The cost estimates have not been updated to reflect current costs-- Though the JASSM baseline estimate dated April 2008 was updated to reflect new program changes, the CARD has not been updated since May 2003. Examples of JASSM baseline changes include additional reliability enhancement team improvements and additional testing, which are not reflected in the 2003 CARD. However, when comparing the JASSM WBS dated January 1999 and the Lot 8 contract dated January 2010, it is evident that WBS has been updated as changes have occurred. On the other hand, the CARD for the JASSM-ER was updated as of August 2009. Updates to the JASSM-ER CARD include a new, more powerful engine than the baseline variant. As part of the Milestone C process, work is currently under way by CAPE to update the JASSM-ER cost estimate. The program office said that the JASSM-ER cost model will include updated costs based on the Lot 8 proposal data, updated quantity profiles, and January 2010 revised inflation rates. The program office is in the process of updating labor rates and overhead rates and is reexamining all component prices. * The cost estimates were based on historical costs--The JASSM baseline and JASSM-ER share 70 percent building materials and 95 percent software design; therefore, the WBS is virtually the same. As such, the JASSM baseline and JASSM-ER share many of the same costs. As outlined in both the JASSM baseline and JASSM-ER cost estimate documentation, the cost estimates were based on actual labor costs from JASSM baseline production Lot 1 through Lot 4, actual material and subcontractor costs from JASSM production Lot 1 through Lot 4, and estimated labor costs for JASSM production Lot 5 and Lot 6. Also, costs were estimated for the total JASSM program by fiscal year, and then allocated to both programs based on the associated number of missiles produced. The program office also used data from similar programs such as the HARPOON Missile, Advanced Medium Range Air-to-Air Missile, and Joint Standoff Weapon. JASSM Baseline and JASSM-ER Estimates Substantially Met Criteria for Credibility: While the cost estimates addressed risk and uncertainty as well as sensitivity, the estimates failed to address the risks regarding reliability and changes to the production schedule. By not doing so, the program office may not have a full understanding of the future effects to the overall cost position of these two programs. * The estimates were assessed for risk and uncertainty--Both programs identified engineering and test support, subcontractors, and warranty as major risk elements. However, the analysis did not identify reliability or an increase in the production schedule as possible risk factors. During the Nunn-McCurdy certification process, the DOD's analyses found that the cost breach was driven by four primary factors, two of which focused on reliability. As a result, the program office instituted a reliability enhancement program directed to address reliability concerns. An indirect effect of the enhancement program was an increase in the overall missile costs. The December 2009 selected acquisition report identified increases to the missile hardware cost due to reduced annual quantities, missile production breaks, and increased test requirements and reliability programs. The combined cost estimate, for the JASSM baseline and JASSM-ER variants, has grown significantly over time. By not including reliability and the extension of the production schedule as possible risk factors, cost growth could continue to occur in future production lots. As a result, the programs' calculated point estimate confidence level of 77 percent for baseline and 73 percent for the JASSM and JASSM-ER variants may be overstated. Figure 7: JASSM Baseline and JASSM-ER Cost Estimate Trend Analysis 1997 through 2009: [Refer to PDF for image: multiple line graph] Fiscal year: 1997 Current estimate: $602 million; Selected acquisition report for the JASSM baseline production estimate: $3.03 billion. Fiscal year: 1998 Current estimate: $201 billion; Selected acquisition report for the JASSM baseline production estimate: $3.03 billion. Fiscal year: 1999 Current estimate: $2.10 billion; Selected acquisition report for the JASSM baseline production estimate: $2.07 billion. Fiscal year: 2001 Current estimate: $3.12 billion; Selected acquisition report for the JASSM baseline production estimate: $2.07 billion. Fiscal year: 2002 Current estimate: $4.05 billion; Selected acquisition report for the JASSM baseline production estimate: $2.07 billion. Fiscal year: 2003 Current estimate: $4.0 billion; Selected acquisition report for the JASSM baseline production estimate: $2.07 billion. Fiscal year: 2004 Current estimate: $4.64 billion; Selected acquisition report for the JASSM baseline production estimate: $4.98 billion. Fiscal year: 2005 Current estimate: $4.91 billion; Selected acquisition report for the JASSM baseline production estimate: $4.98 billion. Fiscal year: 2006 Current estimate: $5.80 billion; Selected acquisition report for the JASSM baseline production estimate: $4.98 billion. Fiscal year: 2007 Current estimate: $6.07 billion; Selected acquisition report for the JASSM baseline production estimate: $4.98 billion. Fiscal year: 2009 Current estimate: $7.71 billion; Selected acquisition report for the JASSM baseline production estimate: $4.98 billion. Source: GAO analysis of DOD data. Note: DOD did not issue selected acquisition reports in December for fiscal years 2000 and 2008. [End of figure] * The estimates were assessed for sensitivity--For both the JASSM baseline and JASSM-ER estimates, key cost drivers were identified. The cost estimators examined eight cost factors for the JASSM baseline estimate and 13 cost factors for the JASSM-ER estimate. For the JASSM analysis, engineering support, testing support, other subcontractors, and Teledyne propulsion had the greatest impact on the total variance in the estimate. Engineering support showed a 14 percent impact, followed by test support with a 13 percent impact, other subcontractors with a 12 percent impact, and Teledyne propulsion with a 9 percent impact. These four elements account for 68 percent of the total cost before risk was applied. For the JASSM-ER analysis, the Williams propulsion, other subcontractors, engineering support, and testing support showed the greatest impact on the total cost variance in the estimate. The Williams propulsion had a 15 percent impact, followed by a 15 percent impact for other subcontractors, an 11 percent impact for engineering support, and a 9 percent impact for test support. These four elements account for 76 percent of the total cost before risk was applied. * The cost estimates were checked for errors--Cross-checks were performed and no instances of double-counting were visible. The Lot 5 and Lot 6 estimates were compared back to Lot 1 through Lot 4 for consistency and reasonableness. Also, multiple row and column summation cross-checks were performed to avoid duplication and omission errors. Upon review of the electronic cost model, GAO found no instances of double-counting and the spreadsheet calculations are accurate given the input parameters and assumptions. * The cost estimates were validated against an independent cost estimate--The CAIG estimate is the independent cost estimate. As part of the Nunn-McCurdy certification process, the CAIG developed an independent cost estimate for the development and procurement costs as well as future-year resource requirements for the baseline and JASSM- ER variants. This new independent estimate was a joint effort by the OSD CAIG, the program office, and the Financial Management Center of Expertise, so there was no other estimate for comparison. Per the Nunn- McCurdy JASSM certification package dated April 30, 2008, the CAIG estimate of the acquisition costs for the restructured JASSM program is $7.1 billion, which is directly comparable to the $6 billion estimate reported in the quarterly selected acquisition report dated December 2007. [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov. In addition to the contact name above, the following individuals made key contributions to this report: William Graveline (Assistant Director), John Crawford, Morgan DelaneyRamaker, Tisha Derricotte, Michael J. Hesse, Karen Richey, Hai Tran, and Alyssa Weir. [End of section] Footnotes: [1] 10 U.S.C. § 2433 establishes the requirement for DOD to prepare unit cost reports on major defense acquisition programs or designated major defense subprograms. If a program exceeds cost growth thresholds specified in the law, this is known as a Nunn-McCurdy breach. The law describes two types of cost growth thresholds that can be breached; the "significant cost growth threshold" and the "critical cost growth threshold." A breach of the critical cost growth threshold occurs when a major defense acquisition program or designated major defense subprogram experiences at least a 25 percent increase over either the program acquisition unit cost (total cost of development, procurement, and system-specific military construction divided by the number of fully configured end items to be procured) or over the procurement unit cost (total funds programmed for procurement divided by the total number of fully configured end items to be procured) in the current baseline estimate or at least a 50 percent increase over the program acquisition unit cost or the procurement unit cost in the original baseline estimate. DOD is also required to report breaches to Congress. At the time of JASSM's breach, section 2433 also required DOD to, in certain circumstances, reassess the program and submit a certification to Congress in order to continue the program. Currently, these and other requirements are found in 10 U.S.C. § 2433a. [2] GAO, GAO Cost Estimating and Assessment Guide, [hyperlink, http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). [3] GAO, Defense Acquisitions: Strong Leadership is Key to Planning and Executing Stable Weapon Programs, [hyperlink, http://www.gao.gov/products/GAO-10-522] (Washington, D.C.: May 6, 2010). [4] In May 2003, a report of the Defense Science Board/Air Force Scientific Advisory Board Joint Task Force stated that the TSPR policy marginalized the government program management role and replaced traditional government "oversight" with "insight." It further stated that the authority of program managers and other working-level acquisition officials subsequently eroded to the point where it reduced their ability to succeed on development programs. We have also found that the use of TSPR in government oversight and involvement led to major reductions in various government capabilities, including cost- estimating and systems-engineering staff. The loss of cost-estimating and systems-engineering staff in turn led to a lack of technical data needed to develop sound cost estimates. GAO, Space Acquisitions: Actions Needed to Expand and Sustain Use of Best Practices, GAO-07- 730T (Washington, D.C.: Apr. 19, 2007). [5] GAO, Defense Acquisitions: Need to Revise Acquisition Strategy to Reduce Risk for Joint Air-to-Surface Standoff Missile, [hyperlink, http://www.gao.gov/products/GAO/NSIAD-00-75] (Washington, D.C.: Apr. 26, 2000). [6] A built-in-test provides fault detection to assess operational availability of the missile. The test is conducted on the ground or via aircraft interface and evaluates at least 95 percent of the missile components. It does not assess one-shot devices such as fuse, wing retention, and warhead. [7] OSD and Lockheed Martin officials have not officially scored the two most recent tests of JASSM-ER conducted in September 2010. Contractor officials stated that one test was fully successful while the other experienced an in-flight anomaly. Contractor officials stated that issues with the electronic control unit for the new engine are being evaluated as a possible root cause for the anomaly. [8] The program office provided technical comments stating that while operational effectiveness has not been evaluated since 2007, testing has been more operationally oriented since 2008. [9] Currently, the Air Force has about 942 JASSM baseline missiles in its inventory. [10] In May 2008, USD/ATL required the program to prepare a new acquisition program baseline within 90 days that separates JASSM and JASSM-ER costs. The certification of a new acquisition program baseline was originally delayed pending the analysis of flight test failures and recently delayed until the JASSM-ER requirements had been validated by the Joint Requirements Oversight Council. Program officials stated that the acquisition program baseline will be completed to support an initial production decision for JASSM-ER in November 2011. [11] Under this type of contract, there is a target cost, a target profit, and a price ceiling. After performance, the government and the contractor negotiate the final cost. If the final negotiated cost is over the target cost, then the contractor may realize a profit less than the target profit, or even a net loss. If the negotiated final cost is less than the target cost, then the contractor may realize a profit greater than the target profit. If the final cost exceeds the price ceiling, the contractor absorbs the difference. [12] The telemetry kit is added during the production process for newly produced missiles to be tested. [13] Air Force officials state that because of the cost of the telemetry kits, about $350,000, or 30 percent to 40 percent of the total production unit cost, it is cost prohibitive to add telemetry kits to every missile produced. Therefore, the Air Force has decided against changing the design of the missile to be more testable, and adds telemetry after fabrication. However, this requires additional retrofitting if the missiles do not have telemetry kits installed while still on the production line. [14] OSD and Air Force officials were not able to provide an analytical explanation for why the JASSM baseline and JASSM-ER variants have different reliability goals--90 percent and 85 percent, respectively. [15] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, [hyperlink, http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009), p. 71). [16] A work breakdown structure is the cornerstone of every program because it defines in detail the work necessary to accomplish a program's objectives. WBS reflects the requirements, resources, and tasks that must be accomplished to develop a program. [17] Learning curve theory is based on the premise that people and organizations learn to do things better and more efficiently when they perform repetitive tasks. Learning curves assume that as the quantity of units to be produced doubles the amount of effort declines by a constant percentage. GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, [hyperlink, http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009), p. 137. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: