The F-35 program continues to face operational testing delays. The Department of Defense has not finished developing the aircraft simulator, which delayed testing and again delayed the decision on when to start full production.
We also found the program's schedule for its hardware and software modernization effort, known as "Block 4," was not realistic. The DOD routinely underestimated the amount of work needed to develop "Block 4" capabilities, resulting in additional schedule delays.
We made 3 recommendations, including one to help the DOD update its schedule to reflect achievable time frames.
What GAO Found
The Department of Defense (DOD) delayed the completion of key testing until problems with the F-35 aircraft simulator are resolved, which GAO also reported last year, and will again delay its full-rate production decision. In August 2020, the program office determined the aircraft simulator—to be used to replicate complex test scenarios that could not be accomplished in real-world environment testing—did not fully represent F-35 capabilities and could not be used for further testing until fixed. Since then, program officials have been developing a new plan to ensure the simulator works as intended. Until they finalize the plan and fix the simulator, the next production milestone date—which would formally authorize DOD's transition from development to full production—remains undetermined (see figure).
F-35 Operational Test Schedule and Key Events through 2021, as of November 2020
DOD is now in its third year of its modernization effort, known as Block 4, to upgrade the hardware and software of the aircraft. While DOD added another year to the schedule, GAO found the remaining development time frame is not achievable. The program routinely underestimated the amount of work needed to develop Block 4 capabilities, which has resulted in delays, and has not reflected historical performance into its remaining work schedule. Unless the F-35 program accounts for historical performance in the schedule estimates, the Block 4 schedule will continue to exceed estimated time frames and stakeholders will lack reliable information on when capabilities will be delivered.
GAO found the F-35 program office collects data on many Block 4 software development metrics, a key practice from GAO's Agile Assessment Guide, but has not met two other key practices for monitoring software development progress. Specifically, the F-35 program office has not implemented tools to enable automated data collection on software development performance, a key practice. The program's primary reliance on the contractor's monthly reports, often based on older data, has hindered program officials' timely decision-making. The program office has also not set software quality performance targets, inconsistent with another key practice. Without these targets, the program office is less able to assess whether the contractor has met acceptable quality performance levels.
Why GAO Did This Study
The F-35 Lightning II Joint Strike Fighter program remains DOD's most expensive weapon system program. DOD is 3 years into a development effort that is loosely based on Agile software development processes to modernize the F-35 aircraft's capabilities. With this approach, DOD intends to incrementally develop, test, and deliver small groups of new capabilities every 6 months. Congress included provisions in two statutes for GAO to review the F-35 program.
This report addresses the F-35 operational testing status, DOD's Block 4 modernization development schedule, and how the F-35 program office implements key practices for evaluating Agile software development progress. To assess cost and schedule concerns identified in prior years, GAO selected three key practices that focus on evaluating Agile software development progress. GAO reviewed DOD and contractor documentation and interviewed DOD officials and contractor representatives.
GAO is making three recommendations to DOD, including that it update its modernization schedule to reflect achievable time frames, identify and implement tools to enable automated data collection on software development performance, and set software quality performance targets. DOD agreed with GAO's recommendations.
Recommendations for Executive Action
|Department of Defense||1. The Undersecretary of Defense for Acquisition and Sustainment should direct the F-35 program office to update its Block 4 schedule to reflect historical performance, to develop more achievable time frames for Block 4 modernization capability development and delivery, and to provide an accurate baseline for comparing future cost estimates. (Recommendation 1)|
|Department of Defense||2. The Undersecretary of Defense for Acquisition and Sustainment should direct the F-35 program office to identify and implement automated tools to enable access to real-time data for software development metrics to inform program decisions and ensure the quality of data is reliable. (Recommendation 2)|
|Department of Defense||3. The Undersecretary of Defense for Acquisition and Sustainment should direct the F-35 program office to set software performance target values for critical software quality metrics as it takes steps to identify additional software development metrics. (Recommendation 3)|