F-35 Sustainment:

Need for Affordable Strategy, Greater Attention to Risks, and Improved Cost Estimates

GAO-14-778: Published: Sep 23, 2014. Publicly Released: Sep 23, 2014.

Multimedia:

Additional Materials:

Contact:

Cary Russell
(202) 512-5431
russellc@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Department of Defense (DOD) currently has or is developing several plans and analyses that will make up its overall F-35 sustainment strategy, which is expected to be complete in fiscal year 2019. The annual F-35 operating and support (O&S) costs are estimated to be considerably higher than the combined annual costs of several legacy aircraft (see fig.). DOD has begun some cost-savings efforts and established sustainment affordability targets for the F-35 program, but DOD did not use the military services' budgets to set these targets. Therefore, these targets may not be representative of what the services can afford and do not provide a clear benchmark for DOD's cost-savings efforts. In addition, DOD has not fully addressed several issues that have an effect on affordability and operational readiness, including aircraft reliability and technical-data rights, which could affect the development of the sustainment strategy.

Comparison of the Annual Estimated F-35 Operating and Support (O&S) Cost at Steady State to Actual Legacy Aircraft O&S Costs in Fiscal Year 2010

Comparison of the Annual Estimated F-35 Operating and Support (O&S) Cost at Steady State to Actual Legacy Aircraft O&S Costs in Fiscal Year 2010

Notes: For the purposes of this report, GAO defines steady-state operations as the period from 2036 to 2040, when, according to the services' plans, the number of F-35 aircraft and flying hours reaches its highest point and plateaus.

aThe F-35 cost presented is Cost Assessment and Program Evaluation's (CAPE) estimated total annual operating and support (O&S) cost for 2040 in base year 2012 dollars.

bLegacy aircraft cost is based on a CAPE analysis of 2010 cost data, representing a high point for aircraft O&S budgets due to contingency operations at that time.

It is unclear whether DOD's O&S cost estimates for the F-35 program reflect the most likely costs that the F-35 program will incur. DOD has two primary F-35 O&S estimates that each total around $1 trillion over a 56-year life cycle. These cost estimates are comprehensive in that they include all DOD-required program elements and are organized according to a standard O&S cost-estimating structure; however, weaknesses exist with respect to a few of the assumptions, and the estimates did not include all analyses necessary to make them fully reliable. For example, the estimates did not use reasonable fuel burn rate assumptions that reflect the likely future F-35 fuel usage. Further, one of the estimates did not use reasonable assumptions about part replacement rates and depot maintenance. Finally, while DOD took some steps to mitigate the uncertainties inherent in cost estimates, DOD officials did not conduct key analyses to determine the level of risk associated with the estimates.

Why GAO Did This Study

The F-35 Lightning II is intended to replace a variety of existing aircraft in the Air Force, Navy, and Marine Corps, while providing the most supportable, technologically advanced, lethal, and survivable aircraft to date. The F-35 is DOD's most expensive weapon system, with estimated sustainment costs of about $1 trillion. With the military services planning for the ability to deploy and maintain the F-35 within 4 years, DOD is working to develop a sustainment strategy that will be both affordable and executable for the program's life cycle.

GAO was mandated to review DOD's F-35 sustainment planning efforts. This report addresses the extent to which DOD has (1) developed an F-35 sustainment strategy and addressed potential risks related to affordability and operational readiness and (2) developed a reliable O&S cost estimate for the program's life cycle. GAO analyzed documented plans and cost estimates and interviewed DOD and contractor officials.

What GAO Recommends

GAO recommends that DOD develop better informed affordability constraints; address three risks that could affect sustainment, affordability, and operational readiness; and take steps to improve the reliability of its cost estimates. DOD concurred with all but one recommendation and partially concurred with the recommendation to conduct uncertainty analysis on one of its cost estimates, stating it already conducts a form of uncertainty analysis. GAO continues to believe that the recommended analysis would provide a more comprehensive sense of the uncertainty in the estimates.

For more information, contact Cary Russell at (202) 512-5431 or russellc@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: According to DOD officials, subsequent to the release of GAO-14-778, the F-35 Program Executive Officer (PEO) expanded its analysis of the Future Support Construct through a Sustainment Campaign Plan (SCP), which will define and implement a Global Support Solution (GSS) that delivers affordable readiness to the F-35 fleet. In September 2015, the Joint Strike Fighter Executive Steering Board (JESB) approved the GSS framework and strategy as presented. To identify areas for cost savings within the GSS, the PEO also defined the Service/Partner and Joint Program Office managed O&S costs, based on the 2014 program office estimate cost categories. According to DOD officials, doing so revealed high costs areas that the SCP will target as the F-35 system and the GSS continue to mature. Although DOD remains focused on making informed sustainment decisions, the military services have yet to define affordability constraints for the F-35 program based on their individual budgets. Until this measure is taken, this recommendation will remain open. According to DOD officials, subsequent to the release of GAO-14-778, the PEO expanded its analysis of the Future Support Construct through a Sustainment Campaign Plan (SCP), which will define and implement a Global Support Solution (GSS) that delivers affordable readiness to the F-35 fleet. Notable aspects of GSS are the decision-making authorities among the Services, Partners, Product Support Manager (PSM) and Product Support Integrator (PSI), criteria for the PSM organizational design and the scope of PSI functions. In September 2015, the Joint Strike Fighter Executive Steering Board (JESB) approved the GSS framework and strategy as presented. Ongoing SCP analysis due to the JESB in December 2015 will define other GSS aspects including the technical data strategy, cost allocation principles and contractual framework for sustainment over the 2016-2023 timeframe. To identify areas for cost savings within the GSS, the PEO also defined the Service/Partner and Joint Program Office managed O&S costs, based on the 2014 program office estimate cost categories. Doing so revealed high costs areas that the SCP will target as F-35 system and the GSS continue to mature. Although the services are involved in meetings to determine budget decisions in the F-35 program, the F-35 program has not yet established affordability constraints specifically based on service budgets. Until the program develops these constraints, this recommendation will remain open.

    Recommendation: To help DOD develop an affordable sustainment strategy for the F-35, the Secretary of Defense should direct the Under Secretary of Defense for Acquisitions, Technology and Logistics to direct the F-35 Program Executive Officer to establish affordability constraints linked to, and informed by, military service budgets that will help guide sustainment decisions, prioritize requirements, and identify additional areas for savings by March 2015, at which point the Future Support Construct decision will be approved.

    Agency Affected: Department of Defense

  2. Status: Open

    Comments: According to DOD officials, the Cost Assessment and Program Evaluation (CAPE) has not updated its F-35 estimate subsequent to the release of GAO-14-778. Pending a major program change, CAPE will update the F-35 O&S estimate for the full-rate production decision point in the second quarter of fiscal year 2019. Until CAPE updates its F-35 estimate, we will not be able to determine if it clearly documents assumptions related to intermediate-level maintenance and fuel burn; therefore, this recommendation will remain open.

    Recommendation: To improve the reliability of the Cost Assessment and Program Evaluation (CAPE) F-35 O&S cost estimate, the Secretary of Defense should direct the Director of CAPE, for future F-35 O&S cost estimates, to clearly document assumptions related to intermediate-level maintenance and revise fuel burn assumptions to better reflect the current and future state of the F-35 program.

    Agency Affected: Department of Defense

  3. Status: Open

    Comments: According to DOD officials, the ALIS Integrated Product Team (IPT) is continuing to work with the Joint Program Office's Performance Based Logistics (PBL) team to further develop and refine appropriate metrics for inclusion into future sustainment contracts. An initial set of metrics are identified in the ALIS Fleet Management Plan that will include System Health Indicators and Strategic Performance Measures. According to DOD officials, an ALIS Sustainment planning meeting in Feb 2016 was intended to define the way ahead for documenting and reporting these metrics as well as lower tier metrics. Further, an ALIS Readiness Check (ARC) capability will measure specific internal ALIS performance and will be available in ALIS 2.0.2. Also, Technical Performance Parameters (TPPs) of the actual system performance from the user perspective, such as response times to execute ALIS functions, are evaluated during the functional testing of each ALIS release. Although DOD has made progress in developing performance metrics for ALIS, DOD has yet to develop metrics that are based on intended behavior of the system and tie system performance to user requirements. Until this progression is made, this recommendation will remain open. On August 1, 2016, according to DOD officials, the ALIS Integrated Product Team (IPT) is continuing to work with the JPO Performance Based Logistics (PBL) team to further develop and refine appropriate metrics for inclusion into future sustainment contracts. An initial set of customer facing metrics are identified in the ALIS Fleet Management Plan that, at a very high level, and are being formalized into the F-35 FY-18A Sustainment Performance Work Statement. Initial ALIS metrics include System Health Indicators and Strategic Performance Measures. Specifically, those items included repair turn around times, number of Action Requests (ARs), AR response time, number of Sustainment Product Information Record (SPIR) processing times, and ALIS System Availability. An ALIS Sustainment planning meeting in Feb 2016 will define the way ahead for documenting and reporting these metrics as well as lower tier metrics such as ALIS Administrator Head Counts for each location. Further, an ALIS Readiness Check (ARC) capability will measure specific internal ALIS performance such as system CPU usage, amount of system memory, the operating state of software application, and the volume in message queues. The ARC will be available in ALIS 2.0.2. Also, Technical Performance Parameters (TPPs) of the actual system performance from the user perspective, such as response times to execute ALIS functions, are evaluated during the functional testing of each ALIS release. Although DOD has made progress in developing performance metrics for ALIS, DOD has yet to develop metrics that are based on intended behavior of the system and tie system performance to user requirements. Until this progression is made, this recommendation will remain open.

    Recommendation: To help DOD address key risks to F-35 affordability and operational readiness, and to improve the reliability of its O&S cost estimates for the life cycle of the program, the Secretary of Defense should direct the F-35 Program Executive Officer, to enable DOD to better identify, address, and mitigate performance issues with the Autonomic Logistics Information System (ALIS) that could have an effect on affordability, as well as readiness, to establish a performance-measurement process for ALIS that includes, but is not limited to, performance metrics and targets that (1) are based on intended behavior of the system in actual operations and (2) tie system performance to user requirements.

    Agency Affected: Department of Defense

  4. Status: Open

    Comments: According to DOD officials, the program's overall Reliability and Maintainability (R&M) assessment process includes a software maintainability and product assurance effort that relies on the Joint Program Office (JPO) and Lockheed Martin (LM) analysis of the F-35 Failure Reporting and Corrective Action System (FRACAS) database. Lockheed Martin and its suppliers will continue using the process to inform future R&M improvement projects requiring a functional failure mode fix, including software fixes, along with electrical and mechanical fixes. Although DOD has an R&M assessment process in place, DOD has still not developed a process that would focus directly on software reliability and maintainability. Until DOD develops a process more focused on software and its effects on overall R&M issues, this recommendation will remain open. On August 1, 2016, according to DOD officials, the program's overall R&M assessment process includes a software maintainability and product assurance effort that relies on the Joint Program Office (JPO) and Lockheed Martin (LM) analysis of the F-35 Failure Reporting and Corrective Action System (FRACAS) database. FRACAS analysis continues to identify poor performing systems, sub-systems and line replaceable components (LRC), and informs R&M improvement projects. LM and its suppliers will continue using the process to inform future R&M improvement projects requiring a functional failure mode fix, including software fixes, along with electrical and mechanical fixes. The July 8, 2015 R&M Improvement Program Executive Review reported that of the 88 already implemented R&M improvement projects, the software maintainability and product assurance process was integral to seven LRC. These included the Auxiliary Power Unit Valve, 270V Battery Charger Control Unit interim firmware, Display Management Unit Helmet, Data Transfer Cartridge, General Purpose Input Output, Signal Processor and General Purpose Processor. Although DOD has an R&M assessment process in place, DOD has still not developed a process that would focus directly on software reliability and maintainability. Until DOD develops a process more focused on software and its effects on overall R&M issues, this recommendation will remain open.

    Recommendation: To help DOD address key risks to F-35 affordability and operational readiness, and to improve the reliability of its O&S cost estimates for the life cycle of the program, the Secretary of Defense should direct the F-35 Program Executive Officer, to develop a high level of confidence that the aircraft will achieve its R+M goals, to develop a software reliability and maintainability (R+M) assessment process, with metrics, by which the program can monitor and determine the effect that software issues may have on overall F-35 R+M issues.

    Agency Affected: Department of Defense

  5. Status: Open

    Comments: According to DOD officials, the F-35 Program Executive Officer (PEO) compiled a Data Assertions List (DAL) for the program. The DAL includes system development and demonstration (SDD) and production data assertions, and post-SDD technical data strategy steps that include: 1. Analysis of the technical data and computer software required to sustain the F-35, and to support competition. 2. Documenting how the program will provide data rights, access and delivery to sustainment organizations over the life cycle. 3. Cost-benefit analysis to inform optimal data acquisition and contracting approaches. 4. Risk and limitations analysis to inform the Government's use and release of data. Although progress is being made in the area of intellectual property, DOD has still not developed an overall strategy that would identify data rights ownership, needs, and costs. Until this strategy is developed, this recommendation will remain open. On August 1, 2016, according to DOD officials, the F-35 PEO compiled a Data Assertions List (DAL) for the program. The DAL includes system development and demonstration (SDD) and production data assertions through low rate initial production (LRIP) Lot 7. An LRIP 8 release of additional assertions is in work. The PEO also defined post-SDD technical data strategy steps. These include: 1. Analysis of the technical data and computer software required to sustain the F-35, and to support competition. 2. Documenting how the program will provide data rights, access and delivery to sustainment organizations over the life cycle. 3. Cost-benefit analysis to inform optimal data acquisition and contracting approaches. 4. Risk and limitations analysis to inform the Government's use and release of data. Although progress is being made in the area of intellectual property, DOD has still not developed an overall strategy that would identify data rights ownership, needs, and costs. Until this strategy is developed, this recommendation will remain open.

    Recommendation: To help DOD address key risks to F-35 affordability and operational readiness, and to improve the reliability of its O&S cost estimates for the life cycle of the program, the Secretary of Defense should direct the F-35 Program Executive Officer, to promote competition, address affordability, and inform its overarching sustainment strategy, to develop a long-term Intellectual Property (IP) Strategy to include, but not be limited to, the identification of (1) current levels of technical data rights ownership by the federal government and (2) all critical technical data needs and their associated costs.

    Agency Affected: Department of Defense

  6. Status: Closed - Implemented

    Comments: In response to our recommendation, the PEO updated the 2014 F-35 Ground Rules and Assumptions (GR&A) documentation for intermediate-level maintenance assumptions to more explicitly define the current 2-level (organizational and depot) maintenance strategy. Additionally, the PEO revised the 2014 fuel burn rate GR&A documentation for the F-35A according to a US Air Force modeled direct fuel consumption approach inclusive of future growth and the F-35B and F-35C according to Department of the Navy recommended fuel consumption rates and adjustment factors. Furthermore, the PEO reviewed and updated the 2014 part replacement GR&A documentation for reliability, condemnation assumptions, and component pricing for maintenance-significant items per aircraft variant. Finally, the PEO revised the 2014 depot maintenance GR&A documentation for the discrete tasks associated with Low Observable scuff and refresh, analytical condition inspection, and unscheduled maintenance. These actions will allow the JPO to provide more reliable O&S cost information in its future estimates, improving decision-makers' ability to make more informed F-35 sustainment decisions. We believe the actions taken have met the intent of our recommendation.

    Recommendation: To help DOD address key risks to F-35 affordability and operational readiness, and to improve the reliability of its O&S cost estimates for the life cycle of the program, the Secretary of Defense should direct the F-35 Program Executive Officer, to improve the reliability of the Joint Program Office (JPO) F-35 O&S cost estimate, to clearly document assumptions related to intermediate-level maintenance costs and revise assumptions related to fuel burn rates, part replacement, and depot-maintenance induction in its future F-35 O&S cost estimates to better reflect the current and future state of the F-35 program.

    Agency Affected: Department of Defense

  7. Status: Open

    Comments: According to DOD officials, during the September 2014 F-35 technical review, the F-35 Program Executive Officer (PEO) briefed the USD (AT&L) on the results of a sensitivity analysis of the key Operating and Support (O&S) cost drivers based on the 2013 O&S cost estimate. The analysis also depicted the impact to total O&S costs if each key driver increased by 10 percent. According to DOD officials, the PEO was to include a similar sensitivity analysis based on the 2014 estimate in the fourth quarter fiscal year 2015 update to the F-35 Life Cycle Sustainment Plan. Although the use of sensitivity analyses in DOD's cost estimating is a positive step, a sensitivity analysis is fundamentally different than a risk/uncertainty analysis, which is what we recommended that DOD conduct. Since DOD has not yet applied risk/uncertainty analyses to its cost estimates, this recommendation will remain open.

    Recommendation: To help DOD address key risks to F-35 affordability and operational readiness, and to improve the reliability of its O&S cost estimates for the life cycle of the program, the Secretary of Defense should direct the F-35 Program Executive Officer, to understand the potential range of costs associated with the JPO F-35 O&S cost estimate, to conduct uncertainty analyses on future JPO estimates.

    Agency Affected: Department of Defense

  8. Status: Open

    Comments: According to DOD officials, the Cost Assessment and Program Evaluation (CAPE) has not updated its F-35 estimate subsequent to the release of GAO-14-778. Pending a major program change, CAPE will update the F-35 O&S estimate for the full-rate production decision point in the second quarter of fiscal year 2019. Until CAPE updates its F-35 estimate, we will not be able to determine if they will perform any uncertainty analyses on its cost estimate; therefore, this recommendation will remain open.

    Recommendation: To improve the reliability of the CAPE F-35 O&S cost estimate, the Secretary of Defense should direct the Director of CAPE, for future F-35 O&S cost estimates, to conduct uncertainty analyses to understand the potential range of costs associated with its estimates to reflect the most likely costs associated with the program.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Dec 5, 2016

Dec 1, 2016

Nov 30, 2016

Nov 28, 2016

Nov 21, 2016

Nov 17, 2016

Nov 14, 2016

Nov 1, 2016

Looking for more? Browse all our products here