This is the accessible text file for GAO report number GAO-03-431 entitled 'Tactical Aircraft: DOD Should Reconsider Decision to Increase F/A-22 Production Rates While Development Risks Continue' which was released on March 14, 2003. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products’ accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. Report to Congressional Committees: United States General Accounting Office: GAO: March 2003: TACTICAL AIRCRAFT: DOD Should Reconsider Decision to Increase F/A-22 Production Rates While Development Risks Continue: F/A-22 Aircraft: GAO-03-431: GAO Highlights: Highlights of GAO-03-431, a report to Congressional Committees Why GAO Did This Study: The Air Force is developing the F/A-22 aircraft to fly at higher speeds for longer distances, be less detectable, and improve the pilot’s awareness of the surrounding situation. The F/A-22 will replace the Air Force’s existing fleet of F-15 aircraft. Over the past several years the program has experienced significant cost overruns and schedule delays. Congress mandated that GAO assess the development program and determine whether the Air Force is meeting key performance, schedule, and cost goals. GAO also assessed the implications of the progress of the development program on production. What GAO Found: The F/A-22 development program did not meet key performance, schedule, and cost goals in fiscal year 2002, and delays in the flight test program have led to an increase in the development cost estimate of $876 million. In response to this increase, DOD restructured the development program and reduced production aircraft by 27. If additional delays occur, further changes may be required. The program also continues to address technical problems that have limited the performance of test aircraft, including violent movement or “buffeting” of the vertical fins, overheating in portions of the aircraft, weakening of materials in the horizontal tail, and instability of avionics software. Air Force officials cannot predict when they will resolve these problems. These technical problems, along with the late delivery of aircraft to the flight test center, have delayed the development program. Based on F/A-22 flight test accomplishment data and current flight test plans, we believe that operational testing will likely be delayed several months beyond the planned August 2003 start date. The F/A-22 program is in its final stages of development, and low-rate initial production has begun. Since fiscal year 1997, funds have been appropriated to acquire production aircraft, and the F/A-22 acquisition plan calls for steadily increasing annual production rates. However, GAO considers the Air Force’s acquisition strategy at high risk for increases in production costs. In past reports, GAO has reported that acquiring aircraft while significant technical challenges remain does not allow for adequate testing of the aircraft. The uncertainties regarding performance capabilities of the F/A-22 aircraft and its development schedule will persist until technical problems have been addressed, including testing of modifications or fixes necessary to potentially alleviate these problems. In light of those uncertainties, steadily increasing annual production rates could result in the Air Force having to modify a larger quantity of aircraft after they are built. What GAO Recommends: To help minimize the risks of producing large quantities of aircraft that may require costly modifications, GAO recommends that the Secretary of Defense (1) reconsider the decision to increase the annual production rate beyond 16 aircraft until greater knowledge on any need for modifications is established through completion of operational testing and (2) update the 2002 risk assessment and certification with sufficient detail to allow verification of the conclusions. In comments on a draft of this report, the Department of Defense (DOD) stated that it agreed, for the most part, with our description of the current state of the F/A-22 program’s content, schedule and cost. However, DOD did not concur with our recommendation. www.gao.gov/cgi-bin/getrpt?GAO-03-431. To view the full report, including the scope and methodology, click on the link above. For more information, contact Allen Li at (202) 512-4841 or lia@gao.gov. March 2003 Contents: Letter: Results in Brief: Background: F/A-22 Technical Problems Continue to Affect Performance: Flight Test Schedules Have Been Extended and May Slip Further: Development and Modernization Costs Have Increased: Risks in the F/A-22 Acquisition Plan: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Scope and Methodology: Appendix I: Estimates of Performance for Key Parameters: Appendix II: Comments from the Department of Defense: Appendix III: GAO Staff Acknowledgments: Related GAO Products: Table: Table 1: Schedule Changes for Key F/A-22 Test Program Events: Figures: Figure 1: F/A-22 Flight Test Schedule Changes: Figure 2: Planned Modernization Funding Increases, President’s Budgets for Fiscal Years 2001-2004: Figure 3: Number of Production Aircraft on Contract Prior to Completion of Operational Testing: This is a work of the U.S. Government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. It may contain copyrighted graphics, images or other materials. Permission from the copyright holder may be necessary should you wish to reproduce copyrighted materials separately from GAO’s product. Abbreviations: DIOT&E: Dedicated Initial Operational Test and Evaluation: DOD: Department of Defense: OSD: Office of the Secretary of Defense: United States General Accounting Office: Washington, DC 20548: March 14, 2003: Congressional Committees: The Air Force is developing the F/A-22[Footnote 1] aircraft to replace its fleet of F-15 air superiority aircraft. The F/A-22 is designed to be superior to the F-15 because it is capable of flying at higher speeds for longer distances, more difficult to detect, and able to provide the pilot with substantially improved awareness of the surrounding situation. The Air Force began the F/A-22 development program in 1991. During the past several years, the program has experienced repeated and significant cost overruns and schedule delays. Congressional concern about the aircraft’s development program cost and progress is long-standing, and it continues. The National Defense Authorization Act for Fiscal Year 1998[Footnote 2] requires us to assess the Air Force’s F/A-22 development program annually and determine whether the Air Force is meeting key performance, schedule, and cost goals.[Footnote 3] This is our fifth report. Specifically, we determined (1) the progress of F/A-22 development in terms of performance, schedule, and cost and (2) implications of this progress on the Department of Defense’s (DOD) acquisition plans. Results in Brief: The F/A-22 development program did not meet its key performance, schedule, and cost goals for fiscal year 2002. The program continues to address technical problems that have limited the performance of test aircraft. These problems include unexpected shutdowns of the aviation electronics (avionics) and excessive movement of the vertical tails. Air Force officials stated they do not yet understand the problems associated with the avionics instability well enough to predict when they would be able to resolve them. Aircraft also have been unable to meet maintenance requirements and are spending more time than planned on the ground undergoing maintenance. In addition, the program has experienced schedule delays. These delays are the result of technical problems and the late delivery of developmental aircraft to the flight test center. Many tasks originally scheduled for 2002 have been rescheduled for 2003, and the Air Force now plans to conduct more developmental flight testing concurrent with operational testing.[Footnote 4] Moreover, we believe it is unlikely the Air Force will complete all necessary flight testing prior to the planned start of operational testing. Therefore, the start of operational testing may need to be delayed several months beyond the planned August 2003 start date. Delays in the flight test program have significant consequences. Most recently, they have led to an increase of $876 million in the development cost estimate. In December 2002, in response to this increase, DOD restructured the program using funds from production and modernization upgrades to cover the cost increases. As a result, DOD reduced the number of production aircraft by 27, which decreased the total number of aircraft to be acquired from 303 to 276.[Footnote 5] If additional delays occur, further changes may be required. Despite continuing development problems and challenges, the Air Force plans to continue to acquire aircraft during low-rate production at increasing yearly rates. For example, the Air Force plans to acquire 20 aircraft in 2003, rather than the maximum of 16 Congress allowed without DOD submittal of a risk assessment and certification. However, as we have previously reported, acquiring aircraft before adequate testing is a high-risk strategy that could serve to further increase production costs. The performance capabilities of the F/A-22 and the aircraft’s development schedule will remain uncertain until technical problems have been addressed, including testing of modifications or fixes necessary to potentially alleviate these problems. We are providing recommendations aimed at reducing the risk of increasing the production rate of F/A-22 aircraft before technical challenges have been addressed through operational testing. In its comments on a draft of this report, DOD indicated it did not concur with our recommendation that it not exceed an annual production rate of 16 aircraft until operational testing is complete. DOD stated that the acquisition of more than 16 aircraft in fiscal year 2003 involves lower risk and lower total program cost than staying at 16. Background: The F/A-22 is to be an air superiority and ground attack aircraft with advanced features to make it less detectable to adversaries (stealth characteristics) and capable of high speeds for long ranges.[Footnote 6] It is designed to have integrated avionics that greatly improve pilots’ awareness of the situation surrounding them. The objectives of the F/A-22 development program are to (1) design, fabricate, test, and deliver nine F/A-22 development test aircraft, two non-flying structural test aircraft, six production representative test aircraft, and 37 flight-qualified engines; (2) design, fabricate, integrate, and test the avionics; and (3) design, develop, and test the support and training systems. The F/A-22 is being developed under contracts with Lockheed Martin Corporation, the prime contractor (for the aircraft), and Pratt & Whitney Corporation (for the engine). Following a history of increasing cost estimates to complete the development phase of the F/A-22 program, the National Defense Authorization Act for Fiscal Year 1998 established a cost limitation for both the development and production.[Footnote 7] Subsequently, the National Defense Authorization Act of 2002 eliminated the cost limitation for the development, but left the cost limit for production cost in place.[Footnote 8] The production program is now limited to $36.8 billion.[Footnote 9] The current cost estimate of the development program is $21.9 billion. Currently, the F/A-22 program is in both development and production. Development is in its final stages, and low rate initial production has begun. Since fiscal year 1997, funds have been appropriated to acquire production aircraft, and the F/A-22 acquisition plan calls for steadily increasing annual production rates. The aircraft’s development problems and schedule delays have caused congressional concerns, particularly in light of DOD’s planned increase in production rates. The National Defense Appropriations Act for Fiscal Year 2003[Footnote 10] prohibited the obligation of funds for the acquisition of more than 16 production aircraft in fiscal year 2003, until the Under Secretary of Defense for Acquisition, Technology, and Logistics submits the following to the congressional defense committees: (1) a formal risk assessment that identifies and characterizes the potential cost, technical, schedule, or other significant risks resulting from increasing the F/A-22 production quantities prior to the Dedicated Initial Operational Test and Evaluation (DIOT&E)[Footnote 11] of the aircraft and (2) either a certification that increasing the F/A-22 production quantity for fiscal year 2003 beyond 16 aircraft involves lower risk and lower total program cost than staying at that quantity or implementing a revised production plan, funding, and test schedule. In December 2002, DOD submitted the risk assessment and certification to Congress. F/A-22 Technical Problems Continue to Affect Performance: The F/A-22 developmental program did not meet key performance goals established for fiscal year 2002 and continues to confront numerous technical challenges. Major technical problems include instability of the avionics software, violent movement, or “buffeting,” of vertical fins, overheating in portions of the aircraft, weakening of materials in the horizontal tail, and the inability to meet airlift support and maintenance requirements. Modifications are being made to some test aircraft to address some of these problems in preparation for operational testing. Nevertheless, these problems continue to restrict the performance and testing of the F/A-22. Avionics Instability: Software instability has hampered efforts to integrate advanced avionics capabilities into the F/A-22 system. Avionics control and integrated airborne electronics and sensors provide an increased awareness of the situation around the pilot. The Air Force told us that the avionics have failed or shut down during numerous tests of F/A-22 aircraft due to software problems. The shutdowns occur when the pilot attempts to use the radar, communication, navigation, identification, and electronic warfare systems concurrently. Although the plane can still be flown after the avionics have failed, the pilot is unable to successfully demonstrate the performance of the avionics. Therefore, the Air Force has had to extend the test program schedule. The Air Force recognized that the avionics problems pose a high technical risk to the F/A-22 program, and in June 2002 the Air Force convened a special team to address the problem. According to the team, the unpredictable nature of the shutdowns was not surprising considering the complexity of the avionics system. The team recommended that the software be stabilized in the laboratory before releasing it to flight testing. The team further recommended conducting a stress test on the software system architecture to reduce problems and ensure that it is operating properly. The Air Force implemented these recommendations. Further, the Air Force extended the avionics schedule to accommodate avionics stability testing and now plans to complete avionics testing in the first quarter of 2005. However, Air Force officials stated that they do not yet understand the problems associated with the instability of the avionics software well enough to predict when they will be able to resolve this problem. Vertical Fin Buffeting: Under some circumstances, the F/A-22 experiences violent movement, or buffeting, of the vertical fins in the tail section of the aircraft. This occurs as air, moving first over the body and the wings of the aircraft, places unequal pressures on the vertical fins and rudders. Unless the violent movement is resolved or the fins strengthened, the vertical fins will break over time because the pressures experienced exceed the strength limits of the fins. In addition, the buffeting problem has restricted the testing of aerial maneuvers of the aircraft. Lockheed Martin has developed several modifications to strengthen the vertical fins and has performed an analysis to test the structural strength of the aircraft. It concluded that no flight restrictions above 10,000 feet are necessary as a result of buffeting. Currently, the Air Force has not begun testing to verify flight operations at or below 10,000 feet; operational limitations at altitudes below 10,000 feet remain in effect, with testing scheduled to begin in June 2003. Overheating Concerns: Overheating in the rear portions of the aircraft has significantly restricted the duration of high-speed flight testing. As the F/A-22 flies, heat builds up inside several areas in of the rear of the aircraft. Continued exposure to high temperatures would weaken these parts of the aircraft. For example, a portion of the airframe that sits between the engines’ exhausts[Footnote 12] experiences the highest temperatures. This intense heat could weaken or damage the airframe. To prevent this heat buildup during flight testing, the aircraft is restricted to flying just over 500 miles per hour, about the same speed as a modern jet liner, and significantly below the supercruise[Footnote 13] requirement. Currently, the F/A-22 flies with temperature sensors in those areas of the aircraft, and it slows down whenever the temperature approaches a certain level. The Air Force may add copper sheets to the rear of the aircraft to alleviate the problem. The Air Force began these modifications in January 2003 and plans to complete them by July 2003. Horizontal Tail Material Separations: F/A-22 aircraft have experienced separations of materials in the horizontal tail and the shaft, which allows the tail to pivot. Because the separations reduce tail strength, the Air Force restricted flight testing of some aircraft until it determined that this problem would not affect flight safety during testing. The Air Force and the contractor initially believed that improvements to the aircraft’s manufacturing process would solve this problem. However, the Air Force has determined that it could only solve this problem by redesigning the tail of the aircraft. The Air Force plans to conduct flight testing of the redesigned tail between February 2004 and April 2004. Meeting Airlift Support Requirements: The Air Force estimates it will not meet the F/A-22 airlift support requirement despite last year’s estimate that it would meet all identified key performance parameters.[Footnote 14] (Appendix I contains a list of key performance parameters.) The airlift support requirement is that 8 C-141 aircraft or their equivalents would be sufficient to deploy a squadron of 24 F/A-22s for thirty days without resupply. Today the Air Force estimates that 8.8 C-141 equivalents will be necessary. Impact of Maintenance Needs on Performance: The F/A-22’s performance may also be affected by maintenance needs that exceed established objectives. The Air Force estimates that the F/ A-22 should, at this point in its development, be able to complete 1.67 flying hours between maintenance actions and 1.95 flying hours by the end of development. However, aircraft are requiring five times the maintenance actions expected at this point in development. As of November 2002, the development test aircraft have been completing only .29 flying hours between maintenance actions. Therefore, the development test aircraft are spending more time than planned on the ground undergoing maintenance. In addition, the F/A-22 program has not completed the testing required to prove the aircraft can be maintained worldwide without unique support equipment. For example, the Air Force planned to fly the F/A-22 a minimum of 650 hours prior to the start of operational testing to establish that special support equipment is not necessary to maintain the materials on the exterior of the aircraft. These materials are critical to the aircraft’s low observable, or stealthy, nature. However, as of December 2002, the program has only accomplished 191.6 hours. According to the Air Force, the program will not complete testing for this requirement until the completion of the development program, currently planned for July 2004. Modifications to Improve Performance: In 2002, the F/A-22 development program implemented several modifications to development aircraft to improve performance. The majority of modifications were related to installing the necessary upgrades to complete operational testing. The last three development test aircraft have required an average of 63 modifications. The first two production aircraft have required an average of 50 of these upgrades. In addition, the program repaired problems in the aircraft’s arresting gear system that were discovered during development testing. Further, the Air Force has scheduled modifications to address the previously cited problems found with the vertical tail of the aircraft (fin- buffeting). The Air Force included these repairs in its 2002 modification schedule, but did not begin them in 2002. The modifications will begin during fiscal year 2003. Flight Test Schedules Have Been Extended and May Slip Further: Progress in F/A-22 flight testing was slower than expected in 2002 in all test areas, according to Office of the Secretary of Defense (OSD) testing officials. Consequently, the Air Force extended flight test schedules and reduced the number of flight tests. Many tasks originally planned for 2002 were rescheduled for 2003. Further, the Air Force now plans to conduct more developmental flight testing concurrently with operational testing. Continuing technical problems were the primary reasons for the delays in flight testing. In addition, late delivery of development aircraft to the flight test center was a contributing problem; three developmental aircraft were delivered from 9 to 12 months late. Late deliveries were due not only to technical problems, but also to continuing problems associated with the manufacture and assembly of development aircraft by the prime contractor. With the new schedule, the Air Force delayed the beginning of operational testing for 4 months, until the portion of developmental testing required to begin operational testing could be completed. Operational testing is now planned to begin in August 2003. Figure 1 and table 1 show the changes in the FA/-22 flight test schedules. Figure 1: F/A-22 Flight Test Schedule Changes: [See PDF for image] [End of figure] Table 1: Schedule Changes for Key F/A-22 Test Program Events: Key Events: Completion of development flight testing necessary prior to operational testing; Prior schedule: April 2003; Revised schedule: August 2003; Change in months: 4. Key Events: Start of operational testing; Prior schedule: April 2003; Revised schedule: August 2003; Change in months: 4. Key Events: Completion of operational testing; Prior schedule: December 2003; Revised schedule: July 2004; Change in months: 7. Key Events: High-rate production decision; Prior schedule: March 2004; Revised schedule: March 2004; Change in months: 0. Source: U.S. Air Force. [End of table] However, according to OSD officials involved in operational testing, there is a high risk of not completing an adequate amount of development flight testing before operational testing is scheduled to begin. Indeed, we believe that it is unlikely that the Air Force will be able to complete all necessary avionics flight testing prior to the planned start of operational testing. Based on F/A-22 flight test accomplishment data and current flight test plans, we project that the start of operational testing might be delayed until January 2004. As a result, operational testing could be delayed by several months beyond the current planned date of August 2003. Development and Modernization Costs Have Increased: In December 2002, the Air Force estimated that development costs had increased by $876 million, bringing total development costs to $21.9 billion. This increase was due to the technical problems and schedule delays discussed earlier. In addition, since fiscal year 2001, there have been dramatic increases in planned funding for modernization upgrades[Footnote 15] that enhance the operational capabilities of the F/A-22, as shown in figure 2. Currently, the Air Force has almost $3.0 billion in funding for modernization projects, which it plans to spend through fiscal year 2009. Most of the recent increase in modernization funding is necessary to provide increased ground attack capability. Other modernization projects include upgrading avionics software, adding an improved short- range missile capability, upgrading instrumentation for testing, and incorporating a classified project. Figure 2: Planned Modernization Funding Increases, President’s Budgets for Fiscal Years 2001-2004: [See PDF for image] [End of figure] In December 2002, in response to the increase in development costs, the Under Secretary of Defense, Comptroller, approved the restructuring of the F/A-22 program. According to the Comptroller, the cost increase will not require increased funds from Congress. Rather, the estimated $876 million increase for development will be met by a $763 million decrease in production funding and a transfer of $113 million from modernization funds. This restructure eliminates 27 aircraft from the current production program, reducing the total number of aircraft to be acquired from 303 to 276.[Footnote 16] Risks in the F/A-22 Acquisition Plan: Despite continuing development problems and challenges, the Air Force plans to continue acquiring production aircraft at increasing annual rates. This is a very risky strategy, because, as we have previously reported, the Air Force may encounter higher production costs as a result of acquiring significant quantities of aircraft before adequate testing. Late testing could identify problems that require costly modifications in order to achieve satisfactory performance. For example, as shown in figure 3, the Air Force plans to acquire 20 aircraft during 2003, rather than the maximum of 16 Congress allowed without DOD’s submittal of a risk assessment and certification. DOD justified this strategy in the December 2002 risk assessment and certification it submitted to Congress.[Footnote 17] In this document, DOD certified that acquiring more than 16 aircraft involved lower risk and lower total program cost than acquiring only 16. DOD identified the costs associated with acquiring more than 16 aircraft per year as between $7 million and $221 million, depending on the number of aircraft in excess of 16. DOD concluded that this additional cost would be less than the potential cost of modifying production aircraft once operational testing has been completed. Figure 3 shows the Air Force’s acquisition plan. Figure 3: Number of Production Aircraft on Contract Prior to Completion of Operational Testing: [See PDF for image] [End of figure] [A] Operational testing is scheduled to be completed in July 2004. However, DOD’s risk assessment may be overly optimistic because it is grounded in the conclusion that there is a low risk that remaining development and operational testing will identify needs for expensive modifications. The performance capabilities of the F/A-22 and its schedule will remain uncertain until technical problems have been addressed, including testing of modifications or fixes necessary to potentially alleviate these problems. Furthermore, we believe that the amount of development and operational testing and the remaining uncertainties increase the possibility that modifications considered unlikely in DOD’s analysis will, indeed, need to be made. For example, the Air Force has still not completely defined the fin-buffet problem described earlier in this report. The remaining 15 percent of flight testing to help characterize the problem is not scheduled to begin until June 2003. Consequently, there is still the possibility that additional modifications and costs may be necessary to correct this problem on production aircraft. DOD’s risk assessment acknowledges that additional fin buffet testing is needed, but concludes that modifications are not expected. The optimism of DOD’s risk assessment is reflected in the Air Force’s general acquisition strategy. As also shown by figure 3, the Air Force is currently committed to acquiring 73 production aircraft (26 percent) before operational and development testing is complete. We believe that--like the fiscal year 2003 decision to acquire more than 16 aircraft--this is an overly optimistic strategy given the remaining F/A-22 technical problems and the current status of testing. As we have noted, acquiring aircraft before completing adequate testing to resolve significant technical problems increases the risk of costly modifications later. If F/A-22 testing schedules slip further--as we believe is likely--even more aircraft will be acquired before development and operational testing is complete, and the risk of costly modifications will increase still more. Conclusions: Continuing the acquisition of aircraft in increasing quantities when significant development testing and technical problems remain is an acquisition strategy that relies on overly optimistic assumptions regarding the outcome and timing of the remaining testing events. By employing such a strategy, major problems are more likely to be discovered after production has begun when it is either too late or very costly to correct them. At the very least, key decisions are being made without adequate information about the weapon system’s demonstrated operational test results. In its certification, DOD quantified the estimated costs associated with a higher production rate. However, the potential advantage was predicated on the assumption that the risks of modifications are low. As we stated last year, by limiting F/A-22 production quantities and completing development testing, the Air Force could gain information that would reduce uncertainties and the risks of increased costs and delays before committing to additional production aircraft. As we discussed earlier in this report, DOD recently decided to reduce production quantities as part of a program restructure to address F/A-22 development problems and associated cost increases. Based on uncertainties about the resolution of problems found in the past year, we continue to maintain the position that production quantities should be limited. Recommendations for Executive Action: In light of continued uncertainties regarding the resolution of problems found in the past year and notwithstanding the December 2, 2002 certification provided by DOD, we recommend that the Secretary of Defense: * reconsider the Department’s decision to increase the annual production rate beyond 16 aircraft until greater knowledge on any need for modifications is established through completion of operational testing, and: * update the 2002 risk assessment and certification with sufficient detail to allow for verification of the conclusions following the completion of operational testing. Agency Comments and Our Evaluation: In written comments on a draft of this report, DOD stated that it agreed, for the most part, with our description of the current state of the F/A-22 program’s content, schedule, and cost. However, DOD did not concur with our recommendation that it not increase its production rate beyond the maximum of 16 aircraft Congress allowed without DOD submitting a risk assessment and certification. DOD said that our recommendation does not sufficiently account for the costs of termination associated with the approval given to funding long-lead items, the manufacturing inefficiencies associated with a reduction in aircraft quantities, or the effects of inflation on the cost of acquiring aircraft at a lower rate. DOD also noted that we had not provided a quantitative assessment to justify limiting production, and it reiterated its reliance on the risk assessment and certification it submitted to Congress in December 2002. DOD also asserted, incorrectly, that our report concludes that minimal cost risk would be realized by slowing production. Following review of DOD’s comments, we clarified the recommendation in our draft report by establishing two recommendations. These recommendations are based on the current state of the program-- including the challenges and risks it faces--and on our examination of DOD’s risk assessment and certification. DOD acknowledges the challenges faced by the program but believes the risk of modification is low. As we discussed in this report, until testing has been completed and technical problems have been addressed, the performance capabilities of the F/A-22 and its schedule will remain uncertain; thus, it is not possible to predict that expensive modifications will not be required. For example, as we stated earlier in this report, DOD’s risk assessment concludes that significant costs associated with a more extensive modification to resolve the fin buffet problem may be required, but the probability is low. DOD arrives at this conclusion even though the last phase of testing to help characterize the fin buffet problem has not yet begun. Furthermore, we continue to believe there is still significant risk that the F/A-22 program will not be able to begin operational testing as scheduled in August 2003. Subsequent to our providing the draft of this report to DOD for comment, OSD’s operational test and evaluation office issued a report stating that F/A-22 technical and schedule risk are still high, as is the risk that operational testing will be further delayed. While DOD’s December 2002 risk assessment and certification did provide an indication that manufacturing inefficiencies and inflation as a result of lower production rates would increase costs, sufficient detail was not provided in its risk assessment for us to verify DOD’s conclusion. We requested additional detailed information to help us evaluate and verify the conclusions. However, the information provided to us was not adequate to verify the conclusions contained in the risk assessment. Regardless, even with such verification, still needing to be resolved are the uncertainties to date regarding when development problems can be fixed and the possibility of finding additional problems prior to the completion of operational testing. As a result, we have little confidence that existing problems can be quickly resolved and will not result in further delays. Our work has shown that continuing the acquisition of aircraft in increasing quantities when significant development testing and technical problems remain is risky. By employing such a strategy, major problems are more likely to be discovered after the program has begun production when it is either too late or very costly to correct them. DOD also provided various technical comments, which we have incorporated as appropriate. One of these comments related to the total number of production aircraft to be acquired. The projected number of production aircraft the Air Force plans to or can actually acquire has historically been fluid and elusive. For example, the President’s budget for fiscal year 2003 reflected plans to acquire 333 production aircraft, even though the approved program at the time called for acquiring 295 production aircraft. In its technical comments, DOD stated that the approved program plan is to acquire 295 aircraft. As a result of the recent F/A-22 restructuring to cover development cost increases, the Air Force says that it now plans to acquire 276 aircraft. However, DOD estimates that the cost of production to acquire these 276 aircraft will be $42.2 billion, which exceeds the current production cost limit by $5.4 billion.[Footnote 18] Consequently, unless the production cost limit is raised or substantial cost reduction plans are achieved, it appears that the number of aircraft that can actually be purchased will have to be lowered from the 276 planned. This is particularly true if production or development costs- -or both--continue to rise and no additional funds are provided by the Congress. Last month, we recommended in another report that DOD provide Congress with documentation reflecting the quantity of aircraft that DOD believes can be procured within the existing production cost limit.[Footnote 19] DOD’s explanation in its technical comments to a draft of this report identifies the likelihood that F/A-22 aircraft quantities will continue to fluctuate. This makes our recent recommendation that much more compelling. Scope and Methodology: To determine whether the development program is likely to meet performance goals, we analyzed information on the status of key performance parameters. We compared performance goals established by the Under Secretary of Defense for Acquisition, Technology, and Logistics with the Air Force’s estimates of performance for completion of development made in December 2002. To identify the status of F/A-22 modifications, we collected updated information on the status of existing aircraft structural problems that have required aircraft modifications. To determine whether the program is expected to meet schedule goals, we reviewed program and avionics schedules and discussed potential changes to these schedules with F/ A-22 program officials. We tracked progress in the flight test program and evaluated schedule variances in the contractors’ performance management system and compared planned milestone accomplishment dates with actual dates. We tracked technical problems in manufacturing and assembling the development test aircraft. To determine whether the program is likely to meet the cost goal, we examined (1) the extent to which the development program is likely to be completed within the current cost estimate, (2) the Air Force’s plans to fund the program for fiscal year 2003, and (3) the program’s funding plan compared to the current cost estimate. In examining DOD’s risk assessment, we discussed the various DOD assumptions and approaches used in the assessment with a program official who conducted the assessment. We then analyzed the various DOD assumptions and approaches used to make the assessment conclusions. In making these determinations, assessments, and identifications, we required access to current information about test results, performance estimates, schedule achievements and revisions, costs being incurred, aircraft modifications, and the program’s plans for continued development and initial production. The Air Force and contractors gave us access to sufficient information to make informed judgments on the matters covered in this report. In performing our work, we obtained information or interviewed officials from the Office of the Secretary of Defense, Washington, D.C., and the F/A-22 System Program Office, Wright-Patterson Air Force Base, Ohio. We performed our work from September 2002 through December 2002 in accordance with generally accepted government auditing standards. We are sending copies of this report to interested congressional committees; the Secretary of Defense; the Secretary of the Air Force; and the Director, Office of Management and Budget. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. Please contact me at (202) 512-4841 or Catherine Baltzell at (202) 512- 8001 if you or your staff have any questions concerning this report. Major contributors to this report are listed in appendix III. Allen Li Director, Acquisition and Sourcing Management: Signed by Allen Li List of Congressional Committees: The Honorable John Warner Chairman The Honorable Carl Levin Ranking Minority Member Committee on Armed Services United States Senate: The Honorable Ted Stevens Chairman The Honorable Daniel K. Inouye Ranking Minority Member Subcommittee on Defense Committee on Appropriations United States Senate: The Honorable Duncan Hunter Chairman The Honorable Ike Skelton Ranking Minority Member Committee on Armed Services House of Representatives: The Honorable Jerry Lewis Chairman The Honorable John P. Murtha Ranking Minority Member Subcommittee on Defense Committee on Appropriations House of Representatives: [End of section] Appendix I: Estimates of Performance for Key Parameters: Key performance parameter: Radar cross section; Requirement: Classified; Current Estimate: Favorable; Margin: Favorable. Key performance parameter: Supercruise; Requirement: 1.5 Mach; Current Estimate: 1.68 Mach; Margin: 12% favorable. Key performance parameter: Acceleration (<100% is favorable)[A]; Requirement: 54 seconds; Current Estimate: 52.3 seconds; Margin: 3% favorable. Key performance parameter: Maneuverability; Requirement: 3.7 g; Current Estimate: 3.7 g; Margin: 0. Key performance parameter: Payload (missiles); Requirement: Four medium-range, two short-range; Current Estimate: Six medium range, two short-range; Margin: NA. Key performance parameter: Combat radius[B] sub + super; Requirement: 260 + 100; Current Estimate: 315 + 100; Margin: 15% favorable. Key performance parameter: Radar detection range; Requirement: Classified; Current Estimate: 105%; Margin: 5% favorable. Key performance parameter: Independent airlift support (C-141 equivalents); Requirement: 8; Current Estimate: 8.8; Margin: (0.8) unfavorable. Key performance parameter: Sortie generation rate; Requirement: Classified; Current Estimate: 100%; Margin: 0. Key performance parameter: Average flight test hours between maintenance; Requirement: 3.0; Current Estimate: 3.0; Margin: 0. Key performance parameter: Interoperability; Requirement: 100% of IERs[C]; Current Estimate: 100% of IERs[C]; Margin: 0. Source: U.S. Air Force. [A] The acceleration parameter is a measure of the time it takes the aircraft to increase speed to a certain level. If the aircraft is able to increase speed to a certain level in less time than expected, this is considered favorable. Therefore, a measure of less than 100 percent is favorable. [B] Subsonic is below speed of sound, and supersonic is above speed of sound. [C] IERs are information exchange requirements. [End of table] [End of section] Appendix II: Comments from the Department of Defense: ACQUISITION, TECHNOLOGY AND LOGISTICS: OFFICE OF THE UNDER SECRETARY OF DEFENSE: 3000 DEFENSE PENTAGON WASHINGTON, DC 20301-3000: 27 FEB 2003: Mr. Allen Li: Director, Acquisition and Sourcing Management U.S. General Accounting Office: 441 G Street, NW Washington, D.C. 20548: Dear Mr. Li: This is the Department of Defense’s (DoD’s) response to the GAO Draft Report, GAO-03-431, “TACTICAL AIRCRAFT: Continuing F/A-22 Development Risks Warrant Limiting Low-Rate Production to the Rate Initially Approved by Congress,” dated February 7, 2003 (GAO Code 120179). The Department appreciates the opportunity to comment on the draft report. Delays in the delivery of development aircraft during the past two years, delays in flight-testing, and software stability issues, have contributed to cost and schedule pressures on the program. The recently announced increase in the estimate at completion for the Engineering and Manufacturing Development (EMD) program has led the Department to re-baseline the overall EMD and production programs for F/A-22. This new baseline is reflected in the FY04 President’s Budget submission. The Department does not concur with the GAO’s recommendation that until operational testing is completed, the Secretary of Defense not increase the annual production rate beyond the 16 aircraft per year initially approved by Congress for fiscal year 2003. The GAO’s recommendation does not account sufficiently for the termination liability and manufacturing inefficiencies associated with the reduction of quantities in Lots 3 and 4, or for the effects of inflation on the cost to acquire aircraft at a lower rate. Based on the cost analyses performed in support of the Department’s certification to the congressional defense committees, in December 2002, we believe that the costs associated with reducing the annual production rate to 16 aircraft would exceed the retrofit costs for these aircraft. This is due, in part, to the fact that only a small number of additional aircraft would be subject to retrofit under DoD’s current program. The FY04 President’s Budget submission re-baselines the production program and reduces the low-rate initial production ramp rate. The differences between the GAO’s recommended quantities and the quantities in DoD’s re-baselined program are 4 aircraft in FY 2003, and 6 aircraft in FY 2004, or delivery deltas of 4 in FY 2005, and 6 in FY 2006. Therefore, DoD’s plan would require the retrofit of, at most, 10 additional aircraft if problems were not found until FY 2007. The Department continues to monitor program costs closely, and maintains the flexibility, under the buy-to-budget acquisition strategy, to adjust the production rate further if warranted. As it has in the past, the Department will review the program before approval of Lot 5 long-lead procurement and Lot 4 full funding, in October/November 2003, and determine if any changes need to be made at that time. Comments regarding the recommendation are enclosed. The Department also has provided more detailed comments on the report under separate cover. Sincerely, Glenn F. Lamartin Director, Defense Systems: Signed for Glenn F. Lamartin Enclosure: GAO DRAFT REPORT DATED FEBRUARY 7, 2003 (GAO-03-431/GAO Code 120179): “TACTICAL AIRCRAFT: Continuing F/A-22 Development Risks Warrant Limiting Low-Rate Production to the Rate Initially Approved by Congress,”: DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATION: RECOMMENDATION: To help minimize the risks of producing large quantities of aircraft that may require costly modifications, the GAO recommended that, until operational testing is completed, the Secretary of Defense not increase the annual production rate beyond the: 16 aircraft per year initially approved by Congress for fiscal year 2003. (p. 19/GAO Draft Report): DOD RESPONSE: Non-concur. While we agree, for the most part, with the GAO’s description of the current state of the F/A-22 program (content, schedule and cost), the GAO has not provided a quantitative assessment to justify its recommendation to limit the production rate for the F/A- 22 to 16 aircraft until operational testing is complete. The GAO report concludes that, based on the current set of circumstances in the F/A-22 program, minimal cost risk would be realized by slowing production. This conclusion does not account sufficiently for the termination liability and manufacturing inefficiencies associated with the reduction of quantities in Lots 3 and 4, or for the effects of inflation on the cost to acquire aircraft at a lower rate. The milestone decision authority approved long-lead procurement for Lot 4 in December 2002, and authorized bridge funding, beyond long-lead procurement, for Lot 3. In both cases, the Department funded more than 16 aircraft. If the Department were now to limit Lots 3 and 4 to 16 aircraft apiece, the Department would incur some amount of termination liability for those lots. The cost of the remaining aircraft would increase, as a function of inefficiencies and the reduced benefits of learning curves. Finally, the additional cost to acquire those aircraft deferred, either across subsequent lots or in another lot at the end of production, would be substantial. The Under Secretary of Defense (Acquisition, Technology and Logistics), in compliance with section 8119 of the Department of Defense Appropriations Act, 2003, Public Law 107-248, has certified to the congressional defense committees that the acquisition of more than 16 aircraft in fiscal year (FY) 2003 involves lower risk and lower total program cost than staying at that quantity. The likely cost to retrofit aircraft is not as great as the cost to reduce the rate of production. This is due, in part, to the fact that only a small number of additional aircraft would be subject to retrofit under DoD’s current program. The FY 2004 President’s Budget submission re-baselines the production program and reduces the low-rate initial production ramp rate. The differences between GAO’s recommended quantities and those in DoD’s re-baselined program are 4 in FY 2003, and 6 in FY 2004, or delivery deltas of 4 in FY 2005, and 6 in FY 2006. Therefore, DoD’s plan would require the retrofit of, at most, 10 additional aircraft, if problems were not found until FY 2007. Retrofit costs will depend on the nature of the required modifications. Most of the current areas of concern, such as avionics and software, are relatively inexpensive to address. If airframe retrofits were required, costs would likely be much more expensive. On balance, it continues to be less costly to pursue the production profile in the FY04 President’s Budget than to reduce production to 16 aircraft in FY 2003 and FY 2004. The Department and the Air Force are committed to the “buy-to-budget” strategy, which requires continuous monitoring of the potential cost growth within the program and appropriate program adjustments when warranted. [End of section] Appendix III: GAO Staff Acknowledgments: Acknowledgments: Catherine Baltzell, Marvin E. Bonner, Edward Browning, Gary Middleton, Sameena Nooruddin, Madhav Panwar, Karen A. Richey, Don M. Springman, and Ralph White made key contributions to this report. [End of section] Related GAO Products: Tactical Aircraft: F-22 Delays Indicate Initial Production Rates Should Be Lower to Reduce Risks. GAO-02-298. Washington, D.C.: March 5, 2002. Tactical Aircraft: Continuing Difficulty Keeping F-22 Production Costs Within the Congressional Limitation. GAO-01-782. Washington, D.C.: July 16, 2001. Tactical Aircraft: F-22 Development and Testing Delays Indicate Need for Limit on Low-Rate Production. GAO-01-310. Washington, D.C.: March 15, 2001. Defense Acquisitions: Recent F-22 Production Cost Estimates Exceeded Congressional Limitation. GAO/NSIAD-00-178. Washington, D.C.: August 15, 2000. Defense Acquisitions: Use of Cost Reduction Plans in Estimating F-22 Total Production Costs. GAO/T-NSIAD-00-200. Washington, D.C.: June 15, 2000. Budget Issues: Budgetary Implications of Selected GAO Work for Fiscal Year 2001. GAO/OCG-00-8. Washington, D.C.: March 31, 2000. F-22 Aircraft: Development Cost Goal Achievable If Major Problems Are Avoided. GAO/NSIAD-00-68. Washington, D.C.: March 14, 2000. Defense Acquisitions: Progress in Meeting F-22 Cost and Schedule Goals. GAO/T-NSIAD-00-58. Washington, D.C.: December 7, 1999. Fiscal Year 2000 Budget: DOD’s Production and RDT&E Programs. GAO/ NSIAD-99-233R. Washington, D.C.: September 23, 1999. Budget Issues: Budgetary Implications of Selected GAO Work for Fiscal Year 2000. GAO/OCG-99-26. Washington, D.C.: April 16, 1999. Defense Acquisitions: Progress of the F-22 and F/A-18E/F Engineering and Manufacturing Development Programs. GAO/T-NSIAD-99-113. Washington, D.C.: March 17, 1999. F-22 Aircraft: Issues in Achieving Engineering and Manufacturing Development Goals. GAO/NSIAD-99-55. Washington, D.C.: March 15, 1999. F-22 Aircraft: Progress of the Engineering and Manufacturing Development Program. GAO/T-NSIAD-98-137. Washington, D.C.: March 25, 1998. F-22 Aircraft: Progress in Achieving Engineering and Manufacturing Development Goals. GAO/NSIAD-98-67. Washington, D.C.: March 10, 1998. Tactical Aircraft: Restructuring of the Air Force F-22 Fighter Program. GAO/NSIAD-97-156. Washington, D.C.: June 4, 1997. Defense Aircraft Investments: Major Program Commitments Based on Optimistic Budget Projections. GAO/T-NSIAD-97-103. Washington, D.C.: March 5, 1997. F-22 Restructuring. GAO/NSIAD-97-100BR. Washington, D.C.: February 28, 1997. Tactical Aircraft: Concurrency in Development and Production of F-22 Aircraft Should Be Reduced. GAO/NSIAD-95-59. Washington, D.C.: April 19, 1995. Tactical Aircraft: F-15 Replacement Issues. GAO/T-NSIAD-94-176. Washington, D.C.: May 5, 1994. Tactical Aircraft: F-15 Replacement Is Premature as Currently Planned. GAO/NSIAD-94-118. Washington, D.C.: March 25, 1994. FOOTNOTES [1] “F/A” stands for fighter/attack aircraft. The Air Force changed the designation from F-22 to F/A-22 in September 2002 to reflect the aircraft’s air-to-surface attack capability. [2] P.L. 105-85, Section 217, Nov. 18, 1997. [3] Section 217 of the act also requires us to assess whether we had access to sufficient information to make informed judgments on matters covered by our report. [4] Short of war, operational testing is the most realistic way of assessing weapon system performance. It puts a weapon through the rigors of combat conditions to determine its operational effectiveness and suitability. [5] The 27 aircraft are a reduction from the approved program quantity of 303 aircraft, however, the Air Force had hoped to acquire as many as 339 aircraft by achieving cost reductions. [6] Air superiority is the degree of air dominance that allows the conduct of operations by land, sea, and air forces without prohibitive interference by the enemy. [7] P.L. 105-85, Section 217, Nov. 18, 1997. [8] P.L. 107-107, Section 213, Dec. 28, 2001. [9] The cost limitation, before adjustment under the act’s provisions, was $43.4 billon. [10] P.L. 107-248, Section 8119, Oct. 23, 2002. [11] Short of war, operational testing is the most realistic way of assessing a weapon system. [12] The technical term for this section of the airframe is called the “stinger.” [13] Supercruise is the aircraft’s ability to travel at high speeds for long ranges. The F/A-22’s supercruise requirement is approximately 1,000 miles per hour. [14] U.S. General Accounting Office, Tactical Aircraft: F-22 Delays Indicate Initial Production Rates Should Be Lower to Reduce Risks, GAO- 02-298 (Washington, D.C.: Mar. 5, 2002). [15] The Air Force considers modernizations outside the scope of the development and production programs. [16] The 27 aircraft are a reduction from the approved program quantity of 303 aircraft; however, the Air Force had hoped to acquire as many as 339 aircraft by achieving cost reductions. [17] The National Defense Appropriations Act for Fiscal Year 2003 (P.L. 107-248, Oct. 23, 2002) required DOD to submit a certification to Congress to justify the acquisition of more than 16 aircraft in fiscal year 2003. In December 2002, DOD submitted the certification to Congress. [18] The current production cap, as adjusted, is $36.8 billion. [19] U.S. General Accounting Office, Tactical Aircraft: DOD Needs to Better Inform Congress about Implications of Continuing F/A-22 Cost Growth, GAO-03-280 (Washington, D.C.: Feb. 28, 2003). GAO’s Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: