Recommendation: The Director, MDA, should ensure an independent assessment is conducted of the agency's process for developing and executing its annual BMDS flight test plan. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense: Missile Defense Agency
Status: Open

Comments: DOD concurred with the recommendation stating that MDA will conduct an independent assessment as recommended.

Recommendation: The Secretary of the Army should ensure that the Commander, Army Materiel Command, develops procedures to help ensure that it will incorporate depot stakeholder input into the new metrics framework for the Army's organic industrial base through iterative and ongoing processes. (Recommendation 1)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The agency concurred with this recommendation.

Recommendation: The Commandant of the Marine Corps should ensure that the Marine Corps Logistics Command establishes a complete baseline schedule, which includes all planned depot maintenance work for the fiscal year, against which to measure performance. (Recommendation 2)

Agency: Department of Defense: United States Marine Corps
Status: Open

Comments: The agency concurred with this recommendation.

Recommendation: The Secretary of the Army should ensure that the Commander, Army Materiel Command, develops guidance that synchronizes the Army's timelines for required inputs from Army depot maintenance customers who use funds from the Procurement; Research, Development, Test, and Evaluation; and Operations and Maintenance budgets with the depots' timelines for development of their finalized budget estimate submissions to AMC. (Recommendation 3)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The agency concurred with this recommendation.

Recommendation: The Secretary of the Army should ensure that the Commander, Army Materiel Command, provides its non-Army customers with guidance that will help ensure that the depots have all updated maintenance needs in sufficient detail from non-Army customers prior to the depots' finalized budget estimate submissions to AMC. (Recommendation 4)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The agency concurred with this recommendation.

Recommendation: The Secretary of the Army should ensure that the Army Organic Industrial Base Corporate Board oversees a study that includes a recurring, comprehensive, and systematic analysis of Army depot data to identify trends and causes behind changes in depot maintenance schedules; and that it uses this analysis to recommend actions to reduce unplanned maintenance work, as appropriate and necessary. (Recommendation 5)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The agency concurred with this recommendation.

Recommendation: The Under Secretary of Defense for Acquisition and Sustainment should establish procedures requiring the military departments to collect and share findings from their peer reviews of MDAP contracting approaches—including choice of contract type—such as by updating the existing online compendium of best practices and lessons learned as they complete their reviews.

Agency: Department of Defense
Status: Open

Comments: DOD agreed with this recommendation, but as of August 2020 has not taken any action to implement it.

Recommendation: The Assistant Secretary of the Air Force for Acquisition, Technology and Logistics should direct the Chief Architect to develop a plan to attain mature technologies when needed for each ABMS development area, which includes an initial list of technologies and an assessment of their maturity that is updated to reflect changes, and update Congress quarterly. (Recommendation 1)

Agency: Department of Defense: Department of the Air Force: Office of the Assistant Secretary of the Air Force (Acquisition)
Status: Open

Comments: DOD agreed with this recommendation but, as of August 2020, has not yet taken any action to implement it. Officials indicated that in the first quarter of fiscal year 2021 the department expects to provide Congress with an initial list of technologies and an assessment of their maturity, and a plan to attain mature technologies for each development area. We will continue to monitor DOD's actions to implement this recommendation.

Recommendation: The Assistant Secretary of the Air Force for Acquisition, Technology and Logistics should direct the Chief Architect to prepare a cost estimate that is developed in accordance with cost estimating leading practices, to include regularly updating the estimate to reflect ABMS changes and actual costs, and update Congress quarterly. (Recommendation 2)

Agency: Department of Defense: Department of the Air Force: Office of the Assistant Secretary of the Air Force (Acquisition)
Status: Open

Comments: DOD agreed with this recommendation but, as of August 2020, has not yet taken any action to implement it. Officials indicated that in the first quarter of fiscal year 2021 the department expects to provide Congress with a cost estimate of ABMS development activities. We will continue to monitor DOD's actions to implement this recommendation.

Recommendation: The Assistant Secretary of the Air Force for Acquisition, Technology and Logistics should direct the Chief Architect to prepare an affordability analysis that should be regularly updated, and update Congress quarterly. (Recommendation 3)

Agency: Department of Defense: Department of the Air Force: Office of the Assistant Secretary of the Air Force (Acquisition)
Status: Open

Comments: DOD agreed with this recommendation but, as of August 2020, has not yet taken any action to implement it. Officials indicated that in the fourth quarter of fiscal year 2020 the department expects to provide Congress with the affordability analysis for the fiscal year 2021 budget request. In addition, the department plans to provide an affordability analysis as part of the fiscal year 2022 budget request submission to Congress. We will continue to monitor DOD's actions to implement this recommendation.

Recommendation: The Assistant Secretary of the Air Force for Acquisition, Technology and Logistics should formalize and document acquisition authority and decision-making responsibilities of the Air Force offices involved in the planning and execution of ABMS, to include the Chief Architect. This document should be included as part of the submission to Congress in June 2020 and communicated to the Air Force offices that support ABMS. (Recommendation 4)

Agency: Department of Defense: Department of the Air Force: Office of the Assistant Secretary of the Air Force (Acquisition)
Status: Open

Comments: DOD agreed with this recommendation but, as of August 2020, has not yet taken any action to implement it. Officials indicated that in the fourth quarter of fiscal year 2020 the department expects to provide Congress with documentation of decision-making authorities. We will continue to monitor DOD's actions to implement this recommendation.

Recommendation: The Secretary of Defense should ensure that the Director of CAPE, in coordination with the Deputy Assistant Secretary of Defense for Nuclear Matters, the Deputy Assistant Secretary of Defense for Nuclear and Missile Defense Policy, and the Joint Staff Deputy Director for Strategic Stability, as co-chairs of the Nuclear Deterrent Senior Oversight Group, update the applicable guidance for methods of tracking and evaluating progress on implementation of the recommendations from the 2014 nuclear enterprise reviews, requiring DOD components to keep information—including any revised time frames—current. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: DOD concurred with our recommendation and stated that the Nuclear Deterrent Senior Oversight Group co-chairs or, as necessary, the Deputy Secretary of Defense as the chair of Nuclear Deterrent Enterprise Review Group (NDERG), will update the applicable guidance to ensure that time frames and other information associated with planned actions are kept up to date. In April 2020, the acting Deputy Assistant Secretary of Defense for Nuclear Matters issued a memo requesting updates to information that is included in the 2014 tracker by June 1, 2020; however, no additional guidance requiring continuing updates beyond June 1 has been issued as of September 2020.

Recommendation: The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment updates the applicable guidance for methods of tracking and evaluating progress on implementation of the recommendations of the 2015 NC3 report, requiring DOD components to keep information—including metrics for measuring progress and outcomes as well as any revised time frames that may extend out more than 1 year—complete and current. 49 (Recommendation 2)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: DOD concurred with our recommendation and stated that the DOD Chief Information Officer and, as appropriate, the Under Secretary of Defense for Acquisition and Sustainment as the NC3 capability portfolio manager, will update the applicable guidance to ensure that metrics, time frames, and other information associated with planned actions are kept up to date and complete.

Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops a program-wide process for measuring, collecting, and tracking information on how ALIS is affecting the performance of the F-35 fleet to include, but not be limited to, its effects on mission capability rates. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops and implements a strategy for the re-design of ALIS. The strategy should be detailed enough to clearly identify and assess the goals, key risks or uncertainties, and costs of re-designing the system. (Recommendation 2)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Under Secretary of Defense for Acquisition and Sustainment should ensure that the Deputy Assistant Secretary of Defense for Materiel Readiness create, share, and maintain a comprehensive and up-to-date list of all DOD sharing venues (i.e., working groups), including points of contact, related to depot maintenance. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense: Office of the Under Secretary of Defense for Acquisition and Sustainment
Status: Open

Comments: DOD concurred with this recommendation. In response to our report, the Deputy Assistant Secretary of Defense for Materiel Readiness will create, share, and maintain a comprehensive and up-to-date list of all DOD sharing venues related to depot maintenance with associated points of contact. The estimated completion date is August 2020. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Army should ensure that Army Materiel Command reestablish and maintain organizations dedicated to sharing materiel best practices and lessons learned, as required by Army regulations. (Recommendation 2)

Agency: Department of Defense: Department of the Army: Office of the Secretary
Status: Open

Comments: DOD concurred with this recommendation. In response to our report, the Army is working to update policies to accurately reflect current activities for capturing, preserving, and distributing lessons learned and best practices throughout the organic industrial base. The estimated completion date is no later than December 2022. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: We recommend the Secretary of the Air Force issue policy emphasizing the following three key reliability practices when planning and executing acquisition programs: (1) leveraging reliability engineers early and often, (2) establishing realistic reliability requirements, and (3) employing reliability engineering activities to improve a system's design throughout development. (Recommendation 1)

Agency: Department of Defense: Department of the Air Force: Office of the Secretary of the Air Force
Status: Open

Comments: The Air Force agreed with this recommendation and told us they will be updating Air Force Instruction AFI63-101/20-101 to ensure the inclusion of key practices to improve weapon system reliability, including leveraging reliability engineers early and often, establishing realistic reliability requirements, and employing reliability engineering activities to improve a system's design throughout development. The Air Force said it plans to complete this effort by December 31, 2020.

Recommendation: We recommend the Secretary of the Navy issue policy emphasizing the following three key reliability practices when planning and executing acquisition programs: (1) leveraging reliability engineers early and often, (2) establishing realistic reliability requirements, and (3) employing reliability engineering activities to improve a system's design throughout development. (Recommendation 3)

Agency: Department of Defense: Department of the Navy: Office of the Secretary
Status: Open

Comments: The Navy agreed with this recommendation and told us they plan to address GAO's recommendation in two ways. First, they are developing a guidebook for program managers and leadership to emphasize the importance of leveraging reliability engineers early and often, establishing realistic reliability requirements, and employing reliability engineering activities to improve a system's design throughout development. The Navy anticipates the guidebook will be completed by the end of fiscal year 2021. Second, the Navy plans to update the SECNAV 5000.2 when it is open for changes. They noted, this is dependent on the Office of the Secretary of Defense (OSD) completing its ongoing update to DOD-wide acquisition guidance. The Navy anticipates OSD's efforts will be completed by end of calendar year 2020.

Recommendation: The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment develop and implement a plan to guide and coordinate efforts to implement the Wideband AOA recommendations to support timely, informed decisions on its next wideband satellite communications architecture. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: DOD concurred with our recommendation, but as of July 2020 is still working to implement its corrective action plan.

Recommendation: The Director, MDA should coordinate with the defense intelligence community on the agency's collective priorities for threat assessments and work with the defense intelligence community to determine if additional resources are needed to support the agency's threat assessment needs. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense: Missile Defense Agency
Status: Open

Comments: DOD concurred with our recommendation, stating that MDA will continue to follow established processes to identify threat assessment needs and to determine if additional resources are required. However, we have yet to see sufficient evidence that MDA is collectively prioritizing its threat assessment requests. We understand that MDA prioritizes its threat assessment requests within the distinct lanes of intelligence product types. We have yet to see evidence that shows MDA has taken steps to also prioritize amongst those lanes. For example, MDA could coordinate with the intelligence community to establish a formal process or venue through which such macro-level prioritization could be conveyed and discussed. In April 2020, MDA told us that it did not plan to transfer funds to the intelligence community in fiscal year 2021 for any unique MDA intelligence needs. By not taking actions to collectively prioritize its threat assessment needs or providing the intelligence community with resources, MDA continues to run the risk of not receiving the threat assessments it needs when it needs them.

Recommendation: The Director, MDA should fully engage the defense intelligence community on key threat-related missile defense acquisition processes and decisions, including providing insight into and obtaining input from the defense intelligence community on the threat space MDA establishes for the BMDS and the threat parameters and threat models MDA assigns to BMDS elements as design requirements and test cases. (Recommendation 2)

Agency: Department of Defense: Office of the Secretary of Defense: Missile Defense Agency
Status: Open

Comments: DOD concurred with our recommendation, stating that MDA will continue to fully engage the intelligence community on key threat-related Ballistic Missile Defense System (BMDS) acquisition processes and decisions. We have observed improvements in MDA including the intelligence community in some of these key threat-related processes and decisions, some of which were discussed in our report. Also, the National Defense Authorization Act for Fiscal Year 2020 included a provision requiring the Secretary of Defense to submit a report to the congressional defense committees on the NGI and include in the report, among several items, updated threat assessments by the intelligence community informing system threshold and objective requirements. To this end, we are aware that MDA consulted with the intelligence community on the threat space and threat-related requirements that are being considered for the Next Generation Interceptor (NGI). The Office of the Under Secretary of Defense for Research and Engineering stated in a September 2019 memo to GAO that MDA also coordinates in weekly Technical Interchange Meetings with the intelligence community on the threat space bounds for parameters that have high uncertainty. We are also aware of ongoing efforts between MDA and the intelligence community to jointly model missile threats that could directly be used in MDA ground tests. These efforts address much of our recommendation; however, we have yet to see whether MDA will coordinate with the intelligence community on the threat parameters assigned to BMDS elements in the BMD System Specification. We intend to follow up with MDA to determine the extent to which MDA has implemented our recommendation.

Recommendation: The Secretary of Defense should require the Director, MDA and the Director, DIA to coordinate on establishing a process for MDA to obtain validation of its threat models. (Recommendation 3)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: DOD concurred with our recommendation, stating that the department will re-examine the most cost-effective approach to meet the intent of DIA validation to support development and fielding of effective Ballistic Missile Defense System (BMDS) elements. We have observed significant progress on this recommendation, primarily through the joint coordination occurring through the Defense Intelligence Ballistic Missile Analysis Committee (DIBMAC) Joint Modeling Tiger Team, where multiple pathways are being explored. Our recommendation calls for the intelligence community and MDA to coordinate on establishing a process for MDA to obtain validation of its threat models. We are also open to other pathways, such as intelligence community and MDA jointly producing threat models or MDA making direct use of threat models built by the intelligence community. Our objective is that MDA use threat models that are validated by the intelligence community when such models are necessary to inform formal BMDS processes, products, and decisions. Any pathway that MDA and the intelligence community agree upon that yields this result meets the intent of our recommendation. We believe that through the tiger team initiative, such coordination is occurring and therefore the closure of this recommendation as implemented in imminent. We are waiting to see: (1) whether MDA and intelligence community establish a formal process and/or jointly sign a memorandum of agreement to codify the process; and (2) the production and use of an intelligence community-validated threat model by MDA in a ground test or other Models and Simulation application.

Recommendation: The Secretary of the Army should direct the Commanding General of Army Futures Command, in coordination with relevant Army entities, to establish command-wide performance measures and develop a plan to use these measures to systematically assess the effectiveness of small business engagement in support of research and development. (Recommendation 3)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: In fiscal year 2020, AFC developed both short-term and long-term performance measures to capture and monitor information on small business engagements across the command, but are still developing a database to systematically track the progression of engagements and outcomes. Officials expect to establish a database by fiscal year 2021.

Recommendation: We recommend that the Acting Secretary of Defense direct the Under Secretary for Acquisition and Sustainment to ensure it conducts a comprehensive assessment of the effect that its contract financing and profit policies have on the defense industry and update that assessment on a recurring basis. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with the recommendation. The Department stated that it would seek fiscal year 2020 funds to contract a study on DOD contract financing policies and their effect on the defense industry. In September 2020, DOD stated that this action would be completed by December 31, 2021. The first phase of the report is estimated to be completed September 30, 2021. The department secured funds to complete the study and a contract was awarded on April 23, 2020.

Recommendation: The Director, MDA, should utilize additional schedule margin afforded by the EPAA Phase 3 delay to conduct additional testing necessary to thoroughly assess the capabilities and limitations of Phase 3 against IRBMs and a raid scenario prior to delivery. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense: Missile Defense Agency
Status: Open

Comments: The Department of Defense partially concurred with this recommendation, stating that all EPAA Phase 3 BMDS functions requiring a flight test environment were already successfully demonstrated and that MDA has addressed the intent of our recommendation by adding ground tests to further assess EPAA Phase 3 Capabilities. However, in order for the agency to meet the full intent of our recommendation, additional flight testing to demonstrate capability against EPAA Phase 3 threats, including intermediate-range threats and raid scenarios, is necessary.

Recommendation: The Secretary of Defense should direct the Under Secretary of Defense for Acquisition and Sustainment to determine and identify in final guidance for middle-tier acquisition programs the metrics that will be used to assess the performance of middle-tier acquisition programs across the military departments, including whether programs are meeting statutory objectives. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation, and stated that it planned to determine performance metrics in coordination with its release of its final guidance on middle-tier programs, which DOD expected to release in late 2019. In December 2019, Congress passed the fiscal year 2020 National Defense Authorization Act. Section 837 in the Conference Report accompanying the Act requires DOD to submit a report to the congressional defense committees no later than December 15, 2019, that includes guidance on the use of middle-tier acquisition authority and the metrics required to assess the performance of such a program, among other topics. DOD's report, provided to Congress in January 2020, identified metrics that DOD planned to use to assess the performance of these programs. The Office of the Under Secretary of Defense for Acquisition and Sustainment subsequently leveraged these metrics in its March 2020 bi-annual review of middle-tier acquisition programs to assess program execution. However, DOD has yet to identify these metrics in guidance as we recommended, which we continue to believe is important to facilitate consistent reporting across the military departments and DOD components.

Recommendation: The Secretary of Defense should develop a plan for how the department will assess the effect of recent acquisition reforms, including identifying who will be responsible for the assessment and what data will be needed. (Recommendation 4)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation, and stated that it has included a division in the Office of the Assistant Secretary of Defense for Acquisition to analyze the effect of recent acquisition reforms and other high-level oversight and policy issues. In December 2019, Congress took additional action. The Conference Report accompanying the fiscal year 2020 National Defense Authorization Act required DOD to submit a report with the budget for fiscal year 2021 on the progress of implementing acquisition reform initiatives. In response, DOD provided a report to Congress in March 2020 that includes how the Secretary will identify, quantify, assess and manage program risk, describes changes to DOD's data collection and sharing processes, and describes new acquisition frameworks to be implemented. However, the report does not address how DOD will assess the acquisition reforms, what data is needed, or who is responsible. Additionally, in August 2020, the Assistant Secretary of Defense approved a plan for assessing the implementation of new acquisition pathways at DOD. However, DOD is still identifying the specific data needed to assess each acquisition pathway, as well as determining how to assess the remaining acquisition reforms we covered in our June 2019 report. Therefore, we will continue to monitor DOD efforts to implement this recommendation.

Recommendation: The Secretary of the Army should ensure that Army Materiel Command establishes measures for its depots to track facility or equipment conditions that lead to maintenance delays. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Army should ensure that Army Materiel Command implements tracking of the measures for identifying when facility or equipment conditions lead to maintenance delays at each Army depot. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Navy should ensure that Naval Sea Systems Command and the Commander, Fleet Readiness Centers establish measures for their depots to track facility or equipment conditions that lead to maintenance delays. (Recommendation 3)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Navy should ensure that Naval Sea Systems Command and the Commander, Fleet Readiness Centers implement tracking of the measures for identifying when facility or equipment conditions lead to maintenance delays at each Navy depot. (Recommendation 4)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Air Force should ensure that Air Force Materiel Command establishes measures for its depots to track facility or equipment conditions that lead to maintenance delays. (Recommendation 5)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Air Force should ensure that Air Force Materiel Command implements tracking of the measures for identifying when facility or equipment conditions lead to maintenance delays at each Air Force depot. (Recommendation 6)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commandant of the Marine Corps should ensure that Marine Corps Logistics Command establishes measures for its depots to track facility or equipment conditions that lead to maintenance delays. (Recommendation 7)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commandant of the Marine Corps should ensure that Marine Corps Logistics Command implements tracking of the measures for identifying when facility or equipment conditions lead to maintenance delays at each Marine Corp depot. (Recommendation 8)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Army should ensure that Army Materiel Command incorporates in its depot optimization plan, key results-oriented elements including analytically-based goals, results-oriented metrics, identification of required resources, risks, and stakeholders, and regular reporting to decision makers on progress. (Recommendation 9)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Navy should ensure that Commander, Fleet Readiness Centers incorporates in its depot optimization plan, key results-oriented elements including analytically-based goals, results-oriented metrics, identification of required resources, risks, and stakeholders, and regular reporting to decision makers on progress. (Recommendation 10)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Air Force should ensure that Air Force Materiel Command incorporates in its depot optimization plan, key results-oriented elements including analytically-based goals, results-oriented metrics, identification of required resources, risks, and stakeholders, and regular reporting to decision makers on progress. (Recommendation 11)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commandant of the Marine Corps should ensure that Marine Corps Logistics Command incorporates in its depot optimization plan, key results-oriented elements including analytically-based goals, results-oriented metrics, identification of required resources, risks, and stakeholders, and regular reporting to decision makers on progress. (Recommendation 12)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment develops an approach for managing service depot investments that includes management monitoring and regular reporting to decision makers and Congress on progress. (Recommendation 13)

Agency: Department of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: We recommend that the Secretary of the Navy categorize and report the Navy's software sustainment costs, in accordance with DOD policy on the Depot Maintenance Core Capabilities Determination Process. [Recommendation 1]

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: The PAO reported that OSD has established a cross-Service team to develop a framework for accounting and reporting core requirements and capabilities specific to weapon system software that recognizes Service-unique attributes, and is applicable across all of DoD. When developed, the framework will clarify how to identify software sustainment costs, and data sources in accordance with DOD policy. When we confirm what additional actions DOD has taken in response to this recommendation, we will provide updated information.

Recommendation: We recommend that the Secretary of Defense ensure that the Director for Cost Assessment and Program Evaluation complete its evaluation and select the most effective system to obtain cost and software data from DOD software centers, and develop an implementation plan that includes time frames for key milestones to execute and monitor the centers' submission of required data. [Recommendation 2]

Agency: Department of Defense
Status: Open

Comments: PAO reported contacting the office of the Director for Cost Assessment and Program Evaluation and provided a template for reporting progress implementing recommendations 2 and 4 with a suspense of 31 July 2020. When we confirm what actions DOD has taken in response to this recommendation, we will provide updated information.

Recommendation: We recommend that the Secretary of Defense ensure that the Director for Cost Assessment and Program Evaluation takes steps to prioritize the respective military departments' obtaining and reporting of complete operating and support costs for software sustainment through its Visibility and Management of Operating and Support Costs systems. [Recommendation 3]

Agency: Department of Defense
Status: Open

Comments: PAO reported contacting the office of the Director for Cost Assessment and Program Evaluation and provided a template for reporting progress implementing this recommendation with a suspense of 31 July 2020. When we confirm what actions DOD has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Army should ensure that the Commanding General of Army Futures Command applies leading practices as they relate to technology development, particularly that of demonstrating technology in an operational environment prior to starting system development. (Recommendation 1)

Agency: Department of Defense: Department of the Army
Status: Open
Priority recommendation

Comments: The Army concurred with this recommendation. Army officials acknowledged the importance of demonstrating technology in an operation environment prior to starting system development and stated that they have taken steps to assist in the identification and removal of infeasible or immature technologies. For example, development of the Integrated Visual Augmentation System includes a series of user engagements where soldiers interact with the system in a field environment and provide feedback to guide development and design. To fully implement this recommendation the Army will need to demonstrate that all of the technologies it is developing are matured in accordance with leading practices. Including soldier engagement in the development of the Integrated Visual Augmentation System is a positive step in that it uses operational experience to determine the utility of technologies on the battlefield. However, demonstrating that those technologies function as expected in an operational environment is equally important. Our past work has shown that demonstrations in an operational environment reduce the risk that technologies will not operate as intended or desired. It is important that the Army continue and expand its efforts to eliminate infeasible, or immature technologies across all of its development programs.

Recommendation: The Secretary of the Army should ensure that the Commanding General of Army Futures Command takes steps to incorporate the experiences of the cross-functional teams in applying leading practices for effective cross-functional teams. (Recommendation 2)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The Army concurred with this recommendation. According to Army officials, cross-functional teams routinely meet to discuss both successes and challenges. Additionally, senior Army leadership hosts weekly meetings with cross-functional teams to discuss decision points and concerns. Army Futures Command plans to include leadership from all cross-functional teams at future meetings to enhance knowledge sharing and ensure leadership viewpoints are communicated across the enterprise. These actions do incorporate elements of leading practices for cross-functional teams such as open and regular communication and senior management support. To fully implement this recommendation the Army should continue and expand this knowledge sharing as planned as well as ensure other elements of leading practices for effective cross-functional teams are implemented. These include the establishment of well-defined team operations with project-specific rules and procedures as well as appropriate training and learning environments for all cross-functional team members.

Recommendation: The Secretary of the Army should ensure that the Commanding General of Army Futures Command executes a process for identifying and incorporating lessons learned from cross-functional team pilots into the new command. (Recommendation 3)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The Army concurred with this recommendation. According to Army officials an analyst from the Center for Army Lessons Learned has embedded a permanent analyst at Army Futures Command who is capturing lessons learned and best practices from across the cross-functional teams. The results of this analysis are provided across Army Futures Command and to other stakeholders. Additionally, Army Futures Command has established the Directorate of Systems Integration to act as a repository and action office for lessons learned. This office hosts recurring coordination events to further the communication of lessons learned .

Recommendation: The Secretary of the Army should ensure that the Commanding General of Army Futures Command fully applies leading practices for mergers and organizational transformations as roles, responsibilities, policies and procedures are finalized for the new command. (Recommendation 4)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The Army concurred with this recommendation. According to Army officials Army Futures Command is applying leading practices for mergers and organizational transformations as it continues to establish the command. These practices include total employee involvement in the transformation process with senior leadership championing the overarching transformation. In addition, the command has worked to identify processes that need improvement as it incorporates the organizations transferred from other commands. It also continues to evolve its staff and functions as needs are identified. According to officials, Army Futures Command continues to select high-performing team members to guide its transformation through selective recruitment and talent management as well as augmenting its growing staff with other Army personnel selected for temporary assignments. The command is also involving employees to obtain their ideas and gain their ownership for the transformation by encouraging them to share their thoughts and ideas. Importantly, according to officials, the command recognizes the need to have an enduring change model to consistently reassess and reorganize the command to consistently deliver products in relevant timeframes. These actions do demonstrate elements of leading practices for mergers and organizational transformations. However, to fully implement this recommendation, Army Futures Command should formalize and institutionalize its authorities, responsibilities, policies and procedures as they relate to these leading practices, especially as leadership within the command and the Army change.

Recommendation: The Secretary of the Army, in conjunction with the U.S. Army Materiel Command, should assess the effectiveness of the Army depots' hiring, training, and retention programs. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: The Army concurred with our recommendation. In February 2020 the Army provided GAO with an outline of a corrective action plan that identified how the Army plans to address our recommendation. Some of the planned milestone dates for these action items occur in fiscal year 2021. As of June 30, 2020 we consider the actions the Army taking to address our rec as ongoing, and will follow up with the Army later this year or early 2021 to obtain a status update on this rec and attempt to close the rec.

Recommendation: The Secretary of the Navy, in conjunction with the Naval Sea Systems Command and Naval Air Systems Command, should assess the effectiveness of the Navy's shipyards' and fleet readiness centers' hiring, training, and retention programs. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: The Navy concurred with our recommendation. In response to our recommendation that the Secretary of the Navy, in conjunction with the Naval Sea Systems Command and Naval Air Systems Command, should assess the effectiveness of the Navy's shipyards' and fleet readiness centers' hiring, training, and retention programs; the Navy is in the process of collecting information to assess the effectiveness of these programs and considers these efforts ongoing.

Recommendation: The Commandant of the Marine Corps, in conjunction with the Marine Corps Logistics Command, should assess the effectiveness of the Marine Corps depots' hiring, training, and retention programs. (Recommendation 3)

Agency: Department of Defense
Status: Open

Comments: The Marine Corps concurred with our recommendation. In December of 2019 we began working with the Department of Defense, including each of the Military Service Components to determine how each component would address our recommendation. We successfully heard back from each component except the Marine Corps. As of June 30, 2020 GAO is still engaging with the Department of Defense and the Marine Corps to obtain a status update on the actions taken by the Marine Corps to address our recommendation. GAO will continue following up on this recommendation in 2020 and during the 2021 fiscal year to determine how the Marine Corps has addressed our recommendation so we can close this recommendation if possible.

Recommendation: The Secretary of the Air Force, in conjunction with the Air Force Materiel Command, should assess the effectiveness of the Air Force air logistics complexes' hiring, training, and retention programs. (Recommendation 4)

Agency: Department of Defense
Status: Open

Comments: The Air Force concurred with our recommendation. In December 2019 the Air Force provided GAO with an outline of a corrective action plan that identified how the Air Force plans to address our recommendation. Many of the planned milestone dates for these action items occur in fiscal year 2021. As of June 30, 2020 we consider the actions the Air Force is taking to address our rec as ongoing and will follow up with the Air Force later this year or early 2021 to obtain a status update on this rec and attempt to close the rec.

Recommendation: The Secretary of Defense should ensure that the Director of the Corrosion Office, in coordination with the military departments, issues guidance to require a standard process for identifying and documenting the rationale for the annual funding levels needed to perform the duties of each Corrosion Executive. The guidance should, at a minimum, identify and define the cost elements and the methods that may be used to estimate the funding levels and describe the supporting documentation that should be maintained. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation. The Corrosion Office plans to develop a standardized process for identifying annual funding levels to perform the duties of each Corrosion Executive. It plans to include this process in a new DOD manual on corrosion that it has a goal of creating by the end of calendar year 2020. We will monitor the extent to which DOD implements this recommendation.

Recommendation: The Secretary of Defense should ensure that the Director of the Corrosion Office documents, within existing or new guidance, its process for reviewing the Military Departments' Corrosion Executive Reports prior to submitting the Corrosion Office Annual Report to Congress, including a process to consistently maintain documentation of its reviews. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation. The Corrosion Office plans to develop and implement a standardized operating procedure for processing and documenting its review of the Corrosion Executive Reports. It plans to complete and implement this standardized operating procedure by November 1, 2019 . We will monitor the extent to which DOD implements this recommendation.

Recommendation: The Secretary of Defense should ensure that the Director of the Corrosion Office develops a process to consistently maintain documentation of its reviews and evaluations of corrosion planning for major weapon system programs. These records, at a minimum, should show what comments were made by the Corrosion Office in its reviews and evaluations, and track the actions taken to resolve those comments. (Recommendation 3)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation. The Corrosion Office plans to develop and maintain a process for documenting the Office's reviews, evaluations, and comments, and to track the weapon system programs' actions on corrosion planning. It plans to include this process in a new DOD manual on corrosion that will include information on considering corrosion during the weapon system program-planning evaluation process, and to develop an internal data system for tracking purposes. Its goal is to create this new manual and internal data system is by the end of calendar year 2020. We will monitor the extent to which DOD implements this recommendation.

Recommendation: The Secretary of the Army should ensure that its Corrosion Executive issues clear guidance establishing a process for reviewing, evaluating, and documenting the adequacy of corrosion planning and oversight efforts for major weapon systems throughout the systems' life cycle, including during the early phases of the acquisition life cycle, and maintain documentation of the oversight actions. (Recommendation 4)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation. When we confirm what actions the Army has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Air Force should ensure that its Corrosion Executive issues clear guidance establishing a process for reviewing, evaluating, and documenting the adequacy of corrosion planning and oversight efforts for major weapon systems throughout the systems' life cycle, including during the early phases of the acquisition life cycle, and maintain documentation of oversight actions. (Recommendation 5)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation. When we confirm what actions the Army has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment updates or issues new policy clarifying the requirements for documenting sustainment strategies for legacy weapon systems, including fixed-wing aircraft. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with the recommendation. In August 2019, DOD told us that draft language reinforcing and clarifying the requirement that all weapon system programs - including legacy weapon systems - have a current Life Cycle Sustainment Plan that is updated every five years had been drafted and submitted as part of the internal DOD coordination process for an update to DOD Instruction 5000.02. DOD estimates that the DOD Instruction 5000.02 will be updated by December 2019.

Recommendation: The Secretary of the Navy should update or issue new guidance clarifying the requirements for documenting sustainment strategies for legacy weapon systems, including fixed-wing aircraft. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: The Navy concurred with the recommendation. In August 2019, DOD told us that the Navy is working to update its policy to reinforce and clarify the requirement that all weapon system programs - including legacy weapon systems - have a current Life Cycle Sustainment Plan that is updated every five years. The Navy estimates that the policy will be updated by December 2019.

Recommendation: As MIBP moves forward with its plans to improve data collection and analysis, the Deputy Assistant Secretary for Defense, Manufacturing and Industrial Base Policy should determine a solution to make better use of existing lower-tier supplier information from program offices. (Recommendation 1)

Agency: Department of Defense: Office of the Under Secretary of Defense for Acquisition, Technology and Logistics: Office of Manufacturing and Industrial Base Policy: Deputy Assistant Secretary of Manufacturing and Industrial Base Policy
Status: Open

Comments: DOD partially concurred with this recommendation, noting that it plans to takes steps to identify and incorporate available supplier data from across the department into its defense industrial base data system, but noted that they only planned to rely on one data system, DIBNow, instead of continuing to develop its second data system, the Defense Planning Guidance Input and Retrieval System. The Industrial Policy Office, formerly Manufacturing and Industrial Base Policy, has taken some steps to improve sharing of industrial base analysis information across the Department, such as establishing a web repository of industrial base assessment information and subject matter expert contacts. However, these sharing tools continue to rely primarily on summary narratives and Industrial Policy has not incorporated program office lower-tier supplier data into this web repository or DIBnow. Industrial Policy has incorporated sub-tier supplier information into DIBnow from commercial industry sources, but these sources do not have the same level of insight into the supply chain as program offices. An Industrial Policy official said there are no plans to incorporate supply chain information from program offices because this information is proprietary and therefore cannot be incorporated into DIBnow, which is built and maintained by a contractor. DOD General Counsel has determined that contractors cannot have access to this proprietary information for the purposes of DIBnow. We continue to believe that program offices have the most complete insight into sub-tier suppliers, which is essential for DOD to achieve its goal of proactive industrial base risk analysis. Industrial Policy should continue to pursue risk reductions solutions, such as non-disclosure agreements and/or data masking, to make better use of existing lower-tier supplier information from program offices.

Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA revises MDA policies to require that all integrated capabilities--capabilities that require integration of two or more elements--be included in a Technical Capability Declaration. (Recommendation 1)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: DOD partially concurred with this recommendation. While DOD agreed with the intent of this recommendation, DOD stated that the MDA Director will determine which major integrated capabilities should be delivered via the Technical Capability Declaration (TCD) process. MDA also noted that it is updating MDA Directive 5000.17 (estimated release in 4Q FY2020), which is intended to address this recommendation. GAO will assess the revisions when they are available. We continue to believe that in order for the agency to meet the full intent of our recommendation, it should establish in policy, a clear, definitive standard for which capabilities require a TCD delivery.

Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA clarifies, in written policy, the exact requirements, process, and key milestones necessary to issue a Technical Capability Declaration, including a requirement that the Assessment Requirements Review be held in such a time frame that it can provide meaningful input to MDA's test plans. (Recommendation 2)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: DOD concurred with this recommendation and contends that its new Policy Memorandum 90 meets the intent of our recommendation. However, this new Policy Memorandum leaves open the possibility of continued inconsistent application of the Technical Capability Declaration (TCD) process. In order for the agency to meet the full intent of our recommendation, MDA should establish in policy, a clear, definitive standard for which capabilities require a TCD for delivery that should be tracked over several years to ensure that the policy is followed. MDA officials noted that revisions to MDA Directive 5000.17 expected at the end of Fiscal Year 2020 will address the recommendation. We will assess the new parameters set out in the revised policy directive, when it is available. We will also continue to track how MDA implements any new policies on TCDs and whether the information is transparent.

Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA includes information on current undefinitized contract actions in the BMDS Accountability Report, including the not-to-exceed price, the definitization schedule, the amount of funds obligated for the undefinitized period, and any changes since the contract action was entered into. (Recommendation 3)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: DOD concurred with this recommendation and has taken steps to implement it. While we noted in our report that the BMDS Accountability Report for 2018, approved by the MDA Director on March 9, 2018 reflected that the agency had taken steps to implement this recommendation, we plan to leave this recommendation open to ensure that the BMDS Accountability Report accurately contains all undefinitized contract actions that the agency enters into through the fiscal year and whether the reporting contains the information necessary to satisfy our recommendation.

Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA, ensures that models used for operational tests are validated and accredited for such assessments. To help achieve this, MDA should provide the BMDS Operational Test Agency all evidence previously agreed to and needed to accredit models before ground testing events, as specified in the BMDS OTA accreditation plan. (Recommendation 4)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: DOD concurred with this recommendation and stated that MDA is actively working with the BMDS Operational Test Agency (BMDS OTA) to resolve any issues associated with, and the reporting of, modeling limitations. However, according to BMDS OTA officials, while the number of accredited models has risen in recent years, some remain unaccredited. We, therefore, believe it is premature to closeout this recommendation but we will continue to track MDA's progress on taking the necessary steps to implement this recommendation.

Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA includes in capability delivery packages, such as the Technical Capability Declaration memos and Operational Capability Baseline change packages, the following: a. The verification, validation, and accreditation status of the models used in operational ground tests; and b. Modeling and simulation limitations that affect operational ground test results. (Recommendation 5)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: DOD concurred with this recommendation and stated that MDA has made significant progress over the last year in these efforts. MDA will provide the Technical Capability Declaration (TCD) and Operational Capability Baseline (OCB) packages for us to assess. Until we are able to do so, we believe it is premature to closeout this recommendation. We will continue to monitor MDA's progress in taking the necessary steps.

Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA and the Secretaries of the Armed Services responsible for operating BMDS elements develop written agreements as soon as feasible for modeling and simulations technical and funding requirements for any BMDS elements that are service-operated but represented in BMDS performance assessments. (Recommendation 6)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: DOD concurred with this recommendation and are taking steps to implement this recommendation. We will continue to monitor the efforts necessary including finalizing and executing written agreements between MDA and any BMDS elements that are Service-operated but represented in BMDS performance assessments.

Recommendation: The Administrator of NNSA should require an independent entity to validate that contractor EVM systems used for LEPs meet the EVM national standard. (Recommendation 1)

Agency: Department of Energy: National Nuclear Security Administration
Status: Open

Comments: In October 2018, NNSA stated that the integrated baseline reviews, internal reviews of earned value management (EVM) systems used by life extension programs (LEP), and other controls over data integration provide a practical and cost-beneficial approach to the validation of contractor EVM systems. However, in early 2019, NNSA conducted a lessons learned study to more fully assess the cost, benefit, mission impacts, and feasibility of implementing our recommendation for possible application to future LEPs. According to NNSA documentation, based on the results of the lessons learned study, NNSA concluded that the effort and expense needed to validate contractor EVM systems against the EVM national standard could be better used providing resources to improve the agency's ability to establish work breakdown structures, schedules, and integration of scope, cost, and schedule. As a result, as of September 2019, NNSA stated that its actions meet the intent of our recommendation. We disagree. As we stated in our report, without requiring an independent entity to validate that contractor EVM systems meet the EVM national standard, NNSA may not have assurance that its LEPs are obtaining reliable EVM data for managing their programs and reporting their status. We will continue to monitor NNSA's activities to address this recommendation.

Recommendation: The Administrator of NNSA should require an independent entity to conduct surveillance reviews of contractor EVM systems used for LEPs to ensure that they maintain compliance with the EVM national standard through program completion. (Recommendation 2)

Agency: Department of Energy: National Nuclear Security Administration
Status: Open

Comments: In October 2018, NNSA stated that it conducts ongoing surveillance through integrated baseline reviews, internal reviews of EVM systems used by LEPs, and other assessments, which consider national standards. However, in early 2019, NNSA conducted a lessons learned study to more fully assess the cost, benefit, mission impacts, and feasibility of implementing our recommendation for possible application to future LEPs. According to NNSA documentation, based on the results of the lessons learned study, NNSA concluded that the effort and expense needed to conduct surveillance reviews of contractor EVM systems to ensure compliance with the EVM national standard could be better used providing resources to improve the agency's ability to establish work breakdown structures, schedules, and integration of scope, cost, and schedule. As a result, as of September 2019, NNSA stated that its actions meet the intent of our recommendation. We disagree. As we stated in our report, without requiring an independent entity to conduct surveillance reviews of contractor EVM systems through program completion, NNSA may not have assurance that its LEPs are obtaining reliable EVM data for managing their programs and reporting their status. We will continue to monitor NNSA's activities to address this recommendation.

Recommendation: The Administrator of NNSA should require its programs to ensure that LEP critical technologies meet specific technology readiness level benchmarks at decision points, or otherwise document with program executive approval their rationale for not meeting these benchmarks. (Recommendation 3)

Agency: Department of Energy: National Nuclear Security Administration
Status: Open

Comments: In October 2018, NNSA stated that it has already incorporated specific benchmarks for technology readiness levels at decision points. As an example, it stated that it recommends a technology readiness level of 5 at the beginning of phase 6.3 for an LEP. As a result, NNSA stated that its actions meet the intent of our recommendation. We disagree. As we stated in our report, it is important for NNSA to establish a requirement, not just a recommendation, that LEP critical technologies meet specific technology readiness level benchmarks at decision points. We will continue to monitor NNSA's activities to address this recommendation.

Recommendation: The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the F-35 Program Executive Officer, should re-examine the metrics that it will use to hold the contractor accountable under the fixed-price, performance-based contracts to ensure that such metrics are objectively measurable, are fully reflective of processes over which the contractor has control, and drive desired behaviors by all stakeholders. (Recommendation 2)

Agency: Department of Defense
Status: Open
Priority recommendation

Comments: DOD concurred with this recommendation. As of December 2019, documentation provided by DOD stated that it had established a joint working group to assess the feasibility of and examine options for DOD's planned 5-year performance-based logistics contract. A DOD official said that this group is also working to identify appropriate metrics to hold the contractor accountable under a potential long-term performance-based contract. DOD has not provided us with a timeline of when the working group's assessment will be complete. We will continue to monitor DOD's efforts to re-examine metrics to ensure that they are objectively measureable, fully reflective of processes over which the contractor has control, and drive desired behaviors by all stakeholders.

Recommendation: The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the F-35 Program Executive Officer, should, prior to entering into multi-year, fixed-price, performance-based contracts, ensure that DOD has sufficient knowledge of the actual costs of sustainment and technical characteristics of the aircraft after baseline development is complete and the system reaches maturity. (Recommendation 3)

Agency: Department of Defense
Status: Open
Priority recommendation

Comments: DOD concurred with this recommendation. As of December 2019, documentation provided by DOD stated that it had established a joint working group to assess the feasibility of and examine options for DOD's planned 5-year performance-based logistics (PBL) contract. As a product of this assessment, DOD expects that the department will be able to outline what level of knowledge is required of the actual costs of sustainment and technical characteristics of the aircraft in order to enter into a PBL sustainment construct. DOD has not provided us with a timeline of when the working group's assessment will be complete. We will continue to monitor DOD's efforts in this area.

Recommendation: To help ensure DOD takes a strategic approach for its prototyping and innovation initiatives and overcomes funding and cultural barriers, the Secretary of Defense should direct the Assistant Secretary of Defense for Research and Engineering to develop a high-level DOD-wide strategy, in collaboration with the military services and other appropriate DOD components, to communicate strategic goals and priorities and delineate roles and responsibilities among DOD's prototyping and innovation initiatives.

Agency: Department of Defense
Status: Open

Comments: DOD concurred with our recommendation, but its progress on addressing it has stalled. In November 2019, DOD's Prototyping Guidebook indicated that the department was developing policy and strategy documents pertaining to prototyping. As part of this effort, the Office of the Undersecretary of Defense for Research and Engineering was drafting a broad DOD Research and Engineering strategy that would include strategies pertaining to prototyping and innovation and address this recommendation. As of August 2020, DOD had not published this strategy.

Recommendation: The Secretary of the Army should conduct a comprehensive assessment to better understand the resources necessary for the requirements development process and determine the extent to which the shortfalls can be addressed given other funding priorities.

Agency: Department of Defense: Department of the Army
Status: Open
Priority recommendation

Comments: The Army concurred with our recommendation. Previously, Army officials told GAO that they planned to implement this recommendation after the new Army Futures Command-which currently manages capabilities development for the Army-became fully operational. Although this command became fully operational in July 2019, Army officials stated that more time is needed to fully coordinate and implement this recommendation. The Army Futures Command will work with the U.S. Army Manpower Analysis Agency to assess the capabilities development workforce focused on requirements prior to programs entering the system development phase. Army officials estimate that this assessment will be completed in March 2021. In June 2020, GAO staff received a brief status update from the Army PAO who confirmed that this is still the plan.

Recommendation: To strengthen MDA's acquisition efforts and strengthen oversight, and to increase traceability and insight into MDA's test program, the Secretary of Defense should require MDA to (a) include a detailed crosswalk of changes to each test, such as names, planned execution dates, test types, targets, and other modifications, in each iteration of its Integrated Master Test Plan; (b) address deficiencies in its test scheduling policy by better aligning it with best practices for scheduling, including the use of a schedule work breakdown structure (WBS) that clearly traces each activity to the cost WBS, properly assigning resources to schedules, and clarifying guidance on when and how to conduct schedule risk analysis; (c) rectify deficiencies in its element and test level cost estimates by requiring the use of the common test WBS, documenting the traceability of source data, and codifying the processes and associated information for the software application Test Resource Mission Planning-Tool used to create the test level cost estimates in policy; and (d) break out funding requests by test in the BMDS Accountability Report and other budget documentation submitted during the annual budget submission.

Agency: Department of Defense
Status: Open

Comments: The department partially concurred with our recommendation, agreeing to include a detailed crosswalk of changes to each test in the Ballistic Missile Defense System (BMDS) Integrated Master Test Plan. However, DOD did not concur with the remaining three parts of our recommendation that include steps related to improving scheduling, cost, and reporting on MDA's test program. In August 2020, we observed that MDA had taken actions on our recommendation, such as including more detailed information on changes to the test schedule in its 2018 and 2019 versions of the BMDS Integrated Master Test Plan. We have an ongoing review to assess MDA's program and test cost estimates and plan to review the BMDS Integrated Master Test Plan to determine if the full intent of our recommendation is being met.

Recommendation: To strengthen MDA's acquisition efforts and strengthen oversight, and to improve MDA's requirement-setting process and ensure it includes an appropriate balance between MDA and warfighter priorities, the Secretary of Defense should require MDA to develop a plan to transition operational requirements analysis currently performed within MDA's Achievable Capabilities List to the U.S. Combatant Commanders, with U.S. Strategic Command as the lead entity and, in the interim, require MDA to obtain their concurrence of the Achievable Capabilities List prior to its release.

Agency: Department of Defense
Status: Open

Comments: DOD did not concur with our recommendation to allow the warfighter to determine operational-level requirements for the Ballistic Missile Defense System (BMDS). Although the department disagreed with our recommendation, the Director, MDA informed us in March 2018 that he supported the warfighter providing MDA with operational-level BMDS requirements, provided they are approved by the Combatant Commander for U.S. Strategic Command. The Director, MDA also agreed to obtain U.S. Strategic Command's concurrence on the Achievable Capabilities List prior to its release. Moreover, the January 2019 Missile Defense Review clarified that missile defense requirements are established through the Warfighter Involvement Process, which is governed by U.S. Strategic Command. The review also directed DOD components to evaluate the current missile defense requirements process to ensure that Service and Combatant Commanders' involvement occurs as early as possible in the capabilities development process. According to a U.S. Strategic Command official, in July 2019, the Missile Defense Executive Board agreed with a working group's finding that improvements to the process were needed. In August 2020, U.S. Strategic Command released an update to its instruction that articulates the BMDS Warfighter Involvement Process to address issues identified by the working group. The Director, MDA has also stated in a March 2020 congressional hearing that requirements for the Next Generation Interceptor were coordinated with combatant commanders and the Joint Requirements Oversight Council, which is a significant and positive development. We intend to evaluate these efforts, as they may potentially satisfy the actions we included in our recommendation.

Recommendation: To enhance program oversight and provide more robust input to budget deliberations, Congress should consider requiring DOD to report on each major acquisition program's systems engineering status in the department's annual budget request, beginning with the budget requesting funds to start development. The information could be presented on a simple timeline--as done for the case studies in this report--and at a minimum should reflect the status of a program's functional and allocated baselines as contained in the most current version of the program's systems engineering plan.

Agency: Congress
Status: Open

Comments: As of August 2020, Congress has not yet taken action on the matter for consideration. The Department of Defense (DOD) non-concurred and as of March 16, 2018, DOD officials reported they had closed this recommendation because they did not agree that the systems engineering plan was the most effective means to provide Congress insight into program risk. DOD officials stated that the timing of the systems engineering plan and any updates are not aligned to inform a budget decision that could occur as much as 18 months prior to program initiation; and existing statutory certifications and reports, such as 2366a and 2366b requirements, submitted to Congress contain adequate information regarding program risk and technical maturity. GAO initiated work in 2020 to examine recent congressionally mandated changes in DOD's acquisition and requirements processes and will assess whether those changes meet the intent of this recommendation.

Recommendation: To enhance DOD's efforts to better integrate and improve intelligence support to major defense acquisition programs, and to better enable personnel to provide intelligence inputs to their portfolios of acquisition programs, the Secretary of Defense should direct--as appropriate--the Under Secretary of Defense for Acquisition, Technology, and Logistics; the Under Secretary of Defense for Intelligence; and/or the Secretaries of the military departments, in coordination with one another, to establish certifications that include having these personnel complete required training.

Agency: Department of Defense
Status: Open

Comments: The Department of Defense (DOD) concurred with the recommendation. DOD provided corrective action plans that show they expect to complete efforts that will address this recommendation in 2019. We will continue to monitor the status of DOD's actions and whether these actions address the intent of our recommendations.

Recommendation: To enhance DOD's efforts to better integrate and improve intelligence support to major defense acquisition programs, and to facilitate implementation of improved processes and procedures developed by the Acquisition Intelligence Requirements Task Force and by the Air Force for the integration of intelligence into major defense acquisition programs, the Secretary of Defense should direct--as appropriate--the Under Secretary of Defense for Acquisition, Technology, and Logistics; the Under Secretary of Defense for Intelligence; and/or the Secretaries of the military departments, in coordination with one another, to revise relevant guidance and procedures--including DOD Instruction 5000.02 and DOD Directive 5250.01-- require that intelligence mission data at the acquisition program, service, and department levels be prioritized.

Agency: Department of Defense
Status: Open

Comments: The Department of Defense (DOD) concurred with the recommendation. DOD provided corrective action plans that show they expect to complete efforts that will address this recommendation in 2019. We will continue to monitor the status of DOD's actions and whether these actions address the intent of our recommendations.

Recommendation: To enhance DOD's efforts to better integrate and improve intelligence support to major defense acquisition programs, and to better ensure that DOD obtains useful feedback from stakeholders and the intended users of the Validated Online Lifecycle Threat tool, the Secretary of Defense should direct--as appropriate--the Under Secretary of Defense for Acquisition, Technology, and Logistics; the Under Secretary of Defense for Intelligence; and/or the Secretaries of the military departments, in coordination with one another, to instruct the Director of the Defense Intelligence Agency to develop a communication plan for the tool that includes plans for communicating with and obtaining feedback from stakeholders and intended users such as acquisition program offices and personnel providing intelligence support to acquisition programs.

Agency: Department of Defense
Status: Open

Comments: The Department of Defense (DOD) concurred with the recommendation. DOD provided corrective action plans that show they expect to complete efforts that will address this recommendation by the end of 2018. We will continue to monitor the status of DOD's actions and whether these actions address the intent of our recommendations.

Recommendation: To enhance DOD's efforts to better integrate and improve intelligence support to major defense acquisition programs, and to ensure that it fulfills the needs of acquisition programs and the intelligence community and works as intended, the Secretary of Defense should direct--as appropriate--the Under Secretary of Defense for Acquisition, Technology, and Logistics; the Under Secretary of Defense for Intelligence; and/or the Secretaries of the military departments, in coordination with one another, to assess the need for the Acquisition Intelligence Support Assessment tool and, if validated by this assessment, define this tool's requirements for development and identify the entity responsible for providing oversight and funding for its continued development, implementation, and operation.

Agency: Department of Defense
Status: Open

Comments: The Department of Defense (DOD) concurred with the recommendation. DOD provided corrective action plans that show they expect to complete efforts that will address this recommendation sometime in the future. We will continue to monitor the status of DOD's actions and whether these actions address the intent of our recommendations.

Recommendation: To ensure that risks associated with ALIS are addressed expediently and holistically, the Secretary of Defense should direct the F-35 Program Executive Officer to improve the reliability of its cost estimates, conduct uncertainty and sensitivity analyses consistent with cost-estimating best practices identified in GAO's Cost Estimating and Assessment Guide.

Agency: Department of Defense
Status: Open

Comments: DOD partially concurred with this recommendation. According to DOD officials, as of July 2018, this recommendation conflicts with established Office of the Under Secretary of Defense/Cost Estimation and Program Evaluation guidance for cost estimation and uncertainty analysis. Absent a change in policy at that level, the Joint Program Office will continue to follow Office of the Under Secretary of Defense/Cost Estimation and Program Evaluation policy on this issue. We continue to believe that in order for any risks associated with ALIS to be addressed expediently and holistically, uncertainty and sensitivity analysis must be used on the F-35s cost estimates to improve its overall reliability. Thus, this recommendation will remain open.

Recommendation: To ensure that risks associated with ALIS are addressed expediently and holistically, the Secretary of Defense should direct the F-35 Program Executive Officer to improve the reliability of its cost estimates, ensure that future estimates of ALIS costs use historical data as available and reflect significant program changes consistent with cost-estimating best practices identified in GAO's Cost Estimating and Assessment Guide.

Agency: Department of Defense
Status: Open

Comments: DOD partially concurred with this recommendation. According to DOD officials, since April 2016, the F-35 program has continued to update the ALIS estimate with the latest available cost data, based on recent contracts. Until more reliable actual costs become available, the program utilizes negotiated contract costs, incorporates program initiatives, and ensures the estimate reflects the latest technical baseline and requirements. Until actual costs associated with ALIS historical data are incorporated in the F-35 cost estimate, we believe that the estimate will not be as reliable as it could be. For this reason, this recommendation will remain open.

Recommendation: To enhance DOD's ability to make consistent and informed decisions in its management of the TCC program in accordance with internal control standards, the Under Secretary of Defense for Acquisition, Technology and Logistics should require the Director, Corrosion Policy and Oversight Office, to document the procedures for approving projects within the TCC program for civilian institutions.

Agency: Department of Defense: Office of the Under Secretary of Defense for Acquisition, Technology and Logistics
Status: Open

Comments: DOD did not agree with the recommendation. In 2016, DOD's Corrosion Office consistently maintained that its existing process is adequately documented in the DOD Corrosion Prevention and Mitigation Strategic Plan and the Technical Corrosion Collaboration (TCC) Definitions Document. However, GAO maintained that DOD could enhance its oversight of corrosion projects by documenting how it approves projects for civilian institutions. As of March 2019, DOD has since decided to take action to implement this recommendation. Specifically, the Corrosion Office plans to include information on documenting procedures for approving projects in a new DOD manual on corrosion that it has a goal of creating by the end of calendar year 2020. We will monitor the extent to which DOD implements this recommendation

Recommendation: To enhance DOD's ability to make consistent and informed decisions in its management of the TCC program in accordance with internal control standards, the Under Secretary of Defense for Acquisition, Technology and Logistics should require the Director, Corrosion Policy and Oversight Office, to document the procedures for selecting and approving projects within the TCC program for military academic institutions.

Agency: Department of Defense: Office of the Under Secretary of Defense for Acquisition, Technology and Logistics
Status: Open

Comments: DOD did not agree with the recommendation. In 2016, DOD's Corrosion Office had consistently maintained that its existing process is adequately documented in the DOD Corrosion Prevention and Mitigation Strategic Plan and the Technical Corrosion Collaboration (TCC) Definitions Document. However, GAO maintained that DOD could enhance its oversight of corrosion projects by documenting how it selects and approves TCC projects for military academic institutions. As of March 2019, DOD has since decided to take action to implement this recommendation. The Corrosion Office plans to include information on documenting procedures for selecting and approving projects in a new DOD manual on corrosion that it has a goal of creating by the end of calendar year 2020. We will monitor the extent to which DOD implements this recommendation.

Recommendation: To enhance DOD's ability to make consistent and informed decisions in its management of the TCC program in accordance with internal control standards, the Under Secretary of Defense for Acquisition, Technology and Logistics should require the Director, Corrosion Policy and Oversight Office, to document the procedures for selecting and approving military research labs supporting civilian and military institutions in conducting projects within the TCC program.

Agency: Department of Defense: Office of the Under Secretary of Defense for Acquisition, Technology and Logistics
Status: Open

Comments: DOD partially concurred with our recommendation. As of August 2018, the Corrosion Policy and Oversight office is currently re-writing Appendix A of the "Technical Corrosion Collaboration (TCC)" document to include steps and grading criteria for decision makers when selecting and approving military research labs supporting civilian and military institutions conducting projects with the TCC program. The Corrosion Policy and Oversight office will complete this re-write and the post procedures to their web site by November 30, 2018. As of March 2019, the Corrosion Policy and Oversight office plans to include procedures for selecting and approving labs to support institutions in a new DOD manual on corrosion. Its goal to create this new manual is by the end of calendar year 2020. We will monitor the extent to which DOD implements this recommendation.