Recommendation: The Secretary of the Navy should ensure that Naval Air Systems Command and Commander, Fleet Readiness Centers effectively use historical data to analyze turnaround time and establish accurate turnaround time targets for fixed-wing aircraft depot maintenance packages. (Recommendation 1)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: The Department of Defense concurred with this recommendation. When we confirm what actions the Navy has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Navy should ensure that Commander, Naval Air Forces and Commander, Naval Air Force, Pacific provide depot planners regular reporting on fixed-wing aircraft maintenance performed outside the Navy aviation depots by an operational unit or at an intermediate maintenance facility to ensure they have information on the current condition and depot maintenance needs of individual aircraft. (Recommendation 2)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: The Department of Defense concurred with this recommendation. When we confirm what actions the Navy has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Navy should ensure that Naval Air Systems Command and Commander, Fleet Readiness Centers establish formal processes and related guidance for communication and coordination between depot stakeholders to inform maintenance requirements planning. (Recommendation 3)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: The Department of Defense concurred with this recommendation. When we confirm what actions the Navy has taken in response to this recommendation, we will provide updated information.

Recommendation: The Under Secretary of Defense for Acquisition and Sustainment should establish procedures requiring the military departments to collect and share findings from their peer reviews of MDAP contracting approaches—including choice of contract type—such as by updating the existing online compendium of best practices and lessons learned as they complete their reviews.

Agency: Department of Defense
Status: Open

Comments: DOD agreed with this recommendation, but as of August 2020 has not taken any action to implement it.

Recommendation: The Secretary of the Navy should establish an analysis plan for the evaluation of OPN-funded pilot program availabilities, based on the leading practices for pilot programs. This analysis plan should identify opportunities to evaluate schedule outcomes of pilot program availabilities as compared to non-pilot program availabilities and document a process for evaluating lessons learned from the pilot program (Recommendation 1).

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: The Department of the Navy concurred with our recommendation in May 2020, stating that it would establish an analysis plan for evaluating Other Procurement, Navy-funded pilot program availabilities. In July 2020, the Director, Maintenance & Modernization, within the Office of the Deputy Assistant Secretary of the Navy for Ship Programs, stated that the Navy has drafted this analysis plan, which is now undergoing required Department of Defense (DOD) reviews. The Director stated that he expects this review process to be completed sometime in late summer 2020. Once complete, he stated that his office will provide the analysis plan to GAO for review to confirm that it satisfies our recommendation.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations directs the Advanced Exploration Systems division to define and determine a schedule for synchronization reviews, including the role of the proposed Lunar Exploration Control Board, to help ensure that requirements between mission and program levels are reconciled. (Recommendation 1)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation and stated that the Advanced Exploration Systems division will review program life-cycle review plans to ensure enterprise and program requirements are reconciled across the mission. NASA is in the process of determining the organizational structure of the Human Exploration and Operations Mission Directorate. Following this completion, NASA officials stated that the appropriate control board and division structures for review and program direction will become active.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations directs the Gateway program to conduct a joint cost and schedule confidence level at the program level for the Artemis III mission. (Recommendation 2)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation and stated it would conduct a joint cost and schedule confidence level analysis or equivalent. The Gateway program is planning to conduct a series of project- and program-level reviews and assessments aligned with key decision point reviews. This includes conducting a joint cost and schedule confidence level analysis or equivalent of the Gateway initial configuration to support a program key decision point planned for fall 2021. NASA has not yet taken action on this recommendation.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations directs the Gateway program to update its overall schedule for 2024 to add a KDP II to occur before system integration. (Recommendation 3)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation, but has not yet taken action on it. NASA stated that it would provide a schedule for future reviews, including whether there will be a Key Decision Point (KDP) II, at the KDP-I review currently scheduled for fall 2021.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations creates a life-cycle cost estimate for the Artemis III mission. (Recommendation 4)

Agency: National Aeronautics and Space Administration
Status: Open
Priority recommendation

Comments: NASA agreed with the recommendation and stated that the agency will provide a preliminary cost estimate for the Artemis III mission by the end of 2020. Further, NASA stated that it will provide an updated cost estimate for the Artemis III mission after it establishes cost and schedule commitments for some of the projects that compose the lunar mission, currently planned for the Spring of 2021. To fully implement this recommendation, NASA will need to provide a cost estimate.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations directs the Advanced Exploration Systems division to commit to a completion date and finalize a cohesive document outlining the rationale for selecting its current lunar architecture and lunar programs. (Recommendation 5)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation and stated that it is developing a document that will summarize the trades and architectural studies, but the document is not yet complete.

Recommendation: The NASA Administrator should ensure that the Office of the Chief Engineer determines under what conditions it is appropriate to complete an analysis of alternatives, particularly when there are multiple pathways—including architectures or programs—that NASA could pursue in the future, and document the justification for not completing an analysis. (Recommendation 6)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation, but has not yet taken any action on it. NASA stated that it will provide additional clarifying guidance for conducting analyses of alternatives for new programs in the next update to NASA Procedural Requirements 7120.5E, "NASA Space Flight Program and Project Management Requirements." NASA plans to complete the update of the procedural requirement in September 2021.

Recommendation: The Secretary of the Navy should direct NAVSEA to incorporate current cost and program data and an updated cost risk analysis in its planned update of the Columbia class lead submarine cost estimate. (Recommendation 1)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: DOD concurred with this recommendation, stating that the Columbia Class Program cost estimate will be updated in 2019 to support the lead ship authorization Decision Acquisition Board in 2020. As of September 2020, we have yet to receive an update on the status of this recommendation.

Recommendation: The Secretary of the Navy should direct NAVSEA to develop a realistic and well-documented estimate of savings from use of the authorities associated with the Fund and incorporate the savings associated with the lead submarine into the Columbia lead submarine cost estimate. (Recommendation 2)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: DOD agreed with this recommendation, stating that the updated Columbia Class Program cost estimate would incorporate estimated savings from use of the authorities associated with the fund and savings associated with the Columbia lead submarine cost estimate. In August 2020, Navy officials indicated that NAVSEA updated the Columbia lead submarine cost estimate to include updates to the estimate of savings from the use of the authorities associated with the Fund.

Recommendation: The Secretary of the Navy should direct the Columbia class program office to update the lead submarine cost estimate and cost risk analysis prior to requesting funds for lead submarine construction. (Recommendation 3)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: DOD concurred with this recommendation, stating that the lead submarine cost estimate and cost risk analysis will be updated to support the lead ship authorization Decision Acquisition Board in 2020. In August 2020, Navy officials indicated that NAVSEA updated the Columbia lead submarine cost estimate. However, this estimate was completed after funding was requested for lead submarine construction. While the Office of Cost Assessment and Program Evaluation plans to conduct an assessment of this estimate in the summer of 2020, the assessment will also be too late to inform the Navy's funding request.

Recommendation: The Director of NSF should assess the agency's large facilities oversight workforce to identify any project management competency gaps, develop a plan to address any gaps and time frames for doing so, and monitor progress in closing them. (Recommendation 1)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of September 2019, had awarded a contract for a proficiency assessment and workforce gap analysis. NSF anticipated finishing the proficiency assessment and workforce gap analysis by the second quarter calendar year 2020 and planned to consider options to address any identified gaps, such as identifying workforce development training opportunities. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Director of NSF should establish criteria for the project management expertise of award recipients for large facilities projects and incorporate the criteria in project requirements and external panel reviews. (Recommendation 2)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of November 2019, had drafted but not finalized new language for the Major Facilities Guide and related supplemental award terms and conditions for major facilities. The new language would require award recipients to document their plans for meeting project management competencies. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Director of NSF should ensure, through a requirement or other means, that award recipients for large facilities projects provide information to NSF on any lessons learned or best practices. (Recommendation 3)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of November 2019, had drafted but not finalized supplemental award terms and conditions for major facilities to require recipients to participate in NSF's process for identifying and sharing lessons learned on projects - for example, by sending appropriate staff to the annual major facilities workshop that NSF hosts to provide a collaborative forum for continuous learning and information-sharing or by presenting lessons learned or good practices at the workshop. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Secretary of Energy should direct the Director of the Office of Project Management and the Assistant Secretary of the Office of Environmental Management to work together to establish requirements for classifying cleanup work as capital asset projects or operations activities. (Recommendation 1)

Agency: Department of Energy
Status: Open

Comments: DOE partially concurred with this recommendation. In August 2019, in a letter to GAO on how EM will address our recommendation, the Senior Advisor for Environmental Management to the Under Secretary for Science stated that DOE developed a draft Cleanup Project Management Policy to clarify the expectations for EM's management of discrete work, such as operations activities that could be considered projects. In February 2020, EM officials stated that they are addressing this recommendation in two steps. First, EM is working with DOE's Office of Project Management to draft a Cleanup Project Management Policy (expected to be completed in spring 2020) to address the Decommissioning and Deactivation (D&D) phase of cleanup. This policy will become an appendix in DOE's Order 413.3B. EM officials further stated that EM plans to develop an additional Cleanup Program Management Policy that would classify the remaining types of activities not covered by the Cleanup Project Management Policy, including what EM currently classifies as operations activities. EM plans to implement this policy by the end of fiscal year 2020.

Recommendation: The Secretary of Energy should direct the Director of the Office of Project Management and the Assistant Secretary of the Office of Environmental Management to work together to asses EM's ongoing operations activities to determine if they should be reclassified as capital asset projects based on the newly established requirements. (Recommendation 2)

Agency: Department of Energy
Status: Open

Comments: DOE partially concurred with this recommendation. In August 2019, in a letter to GAO on how EM will address our recommendation, the Senior Advisor for Environmental Management to the Under Secretary for Science stated that DOE developed a draft Cleanup Project Management Policy to clarify the expectations for EM's management of discrete work, such as operations activities that could be considered projects. In February 2020, EM officials stated that they are addressing this recommendation in two steps. First, EM is working with DOE's Office of Project Management to draft a Cleanup Project Management Policy (expected to be completed in spring 2020) to address the Decommissioning and Deactivation (D&D) phase of cleanup. This policy will become an appendix in DOE's Order 413.3B. EM officials further stated that EM plans to develop an additional Cleanup Program Management Policy that would classify the remaining types of activities not covered by the Cleanup Project Management Policy, including what EM currently classifies as operations activities. EM plans to implement this policy by the end of fiscal year 2020.

Recommendation: The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to review and revise EM's 2017 cleanup policy to include program management leading practices related to scope, cost, schedule performance, and independent reviews. (Recommendation 3)

Agency: Department of Energy
Status: Open
Priority recommendation

Comments: DOE concurred with this recommendation. In August 2019, in a letter to GAO on how EM will address our recommendation, the Senior Advisor for Environmental Management to the Under Secretary for Science stated that EM intends to replace its current EM Cleanup Policy with two separate project and program management policies that will incorporate leading practices related to scope, cost, and schedule, and independent reviews, as appropriate. In February 2020, EM officials stated that EM is working on a Cleanup Project Management Policy (expected to be completed in spring 2020) that will cover the Decommissioning and Deactivation (D&D) phase of cleanup. EM plans to develop an additional Cleanup Program Management Policy (expected to be completed by the end of fiscal year 2020), which will supersede EM's current cleanup policy and include leading program management practices-and possibly also project management practices-related to scope, cost, schedule performance, and independent reviews.

Recommendation: The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to review and revise EM's 2017 cleanup policy to include project management leading practices related to scope, cost, schedule performance, and independent reviews. (Recommendation 4)

Agency: Department of Energy
Status: Open
Priority recommendation

Comments: DOE concurred with this recommendation. In August 2019, in a letter to GAO on how EM will address our recommendation, the Senior Advisor for Environmental Management to the Under Secretary for Science stated that EM intends to replace its current EM Cleanup Policy with two separate project and program management policies that will incorporate leading practices related to scope, cost, and schedule, and independent reviews, as appropriate. In February 2020, EM officials stated that EM is working on a Cleanup Project Management Policy (expected to be completed in spring 2020) that will cover the Decommissioning and Deactivation (D&D) phase of cleanup. EM plans to develop an additional Cleanup Program Management Policy (expected to be completed by the end of fiscal year 2020), which will supersede EM's current cleanup policy and include leading program management practices-and possibly also project management practices-related to scope, cost, schedule performance, and independent reviews.

Recommendation: The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to update its cleanup policy to require that earned value management (EVM) systems be maintained and used in a way that follows EVM best practices. (Recommendation 5)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with this recommendation. In August 2019, in a letter to GAO on how EM will address our recommendation, the Senior Advisor for Environmental Management to the Under Secretary for Science stated that EM intends to replace its current EM Cleanup Policy with two separate project and program management policies that will incorporate earned value management best practices, as appropriate. In February 2020, EM officials stated that EM's forthcoming Cleanup Project Management Policy (expected to be completed in spring 2020) will require capital asset projects covered under the policy to use earned value management (EVM) systems that have been independently certified, which is an EVM best practice. However, EM officials told us that this policy will not require operations activities to follow EVM best practices. EM officials stated that EM is also considering including this requirement in its forthcoming Cleanup Program Management Policy (expected to be completed by the end of fiscal year 2020). In addition, EM officials told is that EM plans to require future end-state contracts to use EVM systems that have either been certified by the DOE Office of Project Management or been subject to a "compliance review" conducted by EM itself, depending on the dollar value of the contract.

Recommendation: The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to develop a policy to ensure that work is categorized as level of effort only in appropriate, specified circumstances, such as when work is not measurable or when measurement is impractical. (Recommendation 6)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with this recommendation. In August 2019, in a letter to GAO on how EM will address our recommendation, the Senior Advisor for Environmental Management to the Under Secretary for Science stated that EM intends to include requirements related to the categorization of level of effort work in the project and program management policies EM is developing.

Recommendation: The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to integrate EVM data into EM's performance metrics for operations activities. (Recommendation 7)

Agency: Department of Energy
Status: Open

Comments: DOE partially concurred with this recommendation. In August 2019, in a letter to GAO on how EM will address our recommendation, the Senior Advisor for Environmental Management to the Under Secretary for Science stated that DOE is in the process of developing a Cleanup Program Management Policy that will, among other things, incorporate changes relative to the integration of EVM data into performance metrics as appropriate. As of February 2020, EM officials told us that the new policy is expected to be completed by the end of fiscal year 2020. In February 2020, EM officials stated that EM plans to begin a new performance review process for operations activities that will include EVM-based performance metrics as part of annual reporting to the Assistant Secretary. The new process is expected to begin in October 2020, but EM officials have not determined whether this will be a requirement included in EM's forthcoming Cleanup Program Management Policy.

Recommendation: The Director of NSF should revise the agency's policies for developing schedules for large facilities projects, and for reviewing those schedules, to better incorporate the best practices in GAO's schedule guide. (Recommendation 2)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of November 2019, had updated its internal guidance to include a new section related to schedule reviews. The updated guidance states that the NSF Large Facilities Office will lead analysis of the schedule for each proposed major facilities project, which will include a technical evaluation by the sponsoring office. As further steps to implement this recommendation, NSF planned to develop (1) a new section of the Major Facilities Guide on schedule development, estimating, and analysis and (2) new internal guidance on including project schedules as part of external panels' oversight reviews. NSF anticipated completing these actions by mid-fiscal year 2020. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Administrator of NNSA should require an independent entity to validate that contractor EVM systems used for LEPs meet the EVM national standard. (Recommendation 1)

Agency: Department of Energy: National Nuclear Security Administration
Status: Open

Comments: In October 2018, NNSA stated that the integrated baseline reviews, internal reviews of earned value management (EVM) systems used by life extension programs (LEP), and other controls over data integration provide a practical and cost-beneficial approach to the validation of contractor EVM systems. However, in early 2019, NNSA conducted a lessons learned study to more fully assess the cost, benefit, mission impacts, and feasibility of implementing our recommendation for possible application to future LEPs. According to NNSA documentation, based on the results of the lessons learned study, NNSA concluded that the effort and expense needed to validate contractor EVM systems against the EVM national standard could be better used providing resources to improve the agency's ability to establish work breakdown structures, schedules, and integration of scope, cost, and schedule. As a result, as of September 2019, NNSA stated that its actions meet the intent of our recommendation. We disagree. As we stated in our report, without requiring an independent entity to validate that contractor EVM systems meet the EVM national standard, NNSA may not have assurance that its LEPs are obtaining reliable EVM data for managing their programs and reporting their status. We will continue to monitor NNSA's activities to address this recommendation.

Recommendation: The Administrator of NNSA should require an independent entity to conduct surveillance reviews of contractor EVM systems used for LEPs to ensure that they maintain compliance with the EVM national standard through program completion. (Recommendation 2)

Agency: Department of Energy: National Nuclear Security Administration
Status: Open

Comments: In October 2018, NNSA stated that it conducts ongoing surveillance through integrated baseline reviews, internal reviews of EVM systems used by LEPs, and other assessments, which consider national standards. However, in early 2019, NNSA conducted a lessons learned study to more fully assess the cost, benefit, mission impacts, and feasibility of implementing our recommendation for possible application to future LEPs. According to NNSA documentation, based on the results of the lessons learned study, NNSA concluded that the effort and expense needed to conduct surveillance reviews of contractor EVM systems to ensure compliance with the EVM national standard could be better used providing resources to improve the agency's ability to establish work breakdown structures, schedules, and integration of scope, cost, and schedule. As a result, as of September 2019, NNSA stated that its actions meet the intent of our recommendation. We disagree. As we stated in our report, without requiring an independent entity to conduct surveillance reviews of contractor EVM systems through program completion, NNSA may not have assurance that its LEPs are obtaining reliable EVM data for managing their programs and reporting their status. We will continue to monitor NNSA's activities to address this recommendation.

Recommendation: The Administrator of NNSA should require its programs to ensure that LEP critical technologies meet specific technology readiness level benchmarks at decision points, or otherwise document with program executive approval their rationale for not meeting these benchmarks. (Recommendation 3)

Agency: Department of Energy: National Nuclear Security Administration
Status: Open

Comments: In October 2018, NNSA stated that it has already incorporated specific benchmarks for technology readiness levels at decision points. As an example, it stated that it recommends a technology readiness level of 5 at the beginning of phase 6.3 for an LEP. As a result, NNSA stated that its actions meet the intent of our recommendation. We disagree. As we stated in our report, it is important for NNSA to establish a requirement, not just a recommendation, that LEP critical technologies meet specific technology readiness level benchmarks at decision points. We will continue to monitor NNSA's activities to address this recommendation.

Recommendation: The Under Secretary of Defense for Acquisition, Technology and Logistics should identify the specific types of information that would best meet the department's needs and, based on that determination, collect and analyze relevant data after contract performance is sufficiently complete to determine the extent to which contracts with incentives achieved their desired outcomes.

Agency: Department of Defense: Office of the Secretary of Defense: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics
Status: Open

Comments: DOD concurred with our recommendation and has taken steps to address it. For example, in April 2018, the department developed a template for the military departments to use to identify specific types of information to collect. Since then, each of the military departments has initiated or planned to initiate efforts to collect and analyze information about outcomes of incentive contracts. In addition, in July 2020 DOD provided examples of selected DOD, Army, and Navy incentive contracts documented in the template previously noted. The department did not provide additional information about the Air Force's efforts, or about how DOD is analyzing the information to determine whether incentives can achieve desired outcomes. GAO has ongoing work to review DOD's use of fixed-price type contracts--including fixed-price incentive contracts--for major DOD systems, which may provide additional insights related to this recommendation.

Recommendation: To improve insight into cost changes for individual ships in the Ford Class, the program office should prepare cost summary and funding summary sections for each individual ship in the class as part of the SAR for the overall Ford-Class program.

Agency: Department of Defense
Status: Open
Priority recommendation

Comments: In its comments on our draft report DOD disagreed with our recommendation, stating that it currently provides progress reports to Congress on costs for CVN 78 and CVN 79. In July 2020, Navy officials stated that the department continues to disagree with this recommendation. We continue to maintain that Selected Acquisition Reports represent the primary statutorily required means for DOD to report on program status. Grouping average unit costs for all Ford-class ships obscures individual ship cost growth and does not provide Congress with an adequate level of insight to monitor this over $55 billion program. Our recommendation would ensure that Congress receives insight into the costs of each existing and planned Ford-class ship. To fully implement this recommendation, the report should include cost information on individual ships.

Recommendation: To enhance program oversight and provide more robust input to budget deliberations, Congress should consider requiring DOD to report on each major acquisition program's systems engineering status in the department's annual budget request, beginning with the budget requesting funds to start development. The information could be presented on a simple timeline--as done for the case studies in this report--and at a minimum should reflect the status of a program's functional and allocated baselines as contained in the most current version of the program's systems engineering plan.

Agency: Congress
Status: Open

Comments: As of August 2020, Congress has not yet taken action on the matter for consideration. The Department of Defense (DOD) non-concurred and as of March 16, 2018, DOD officials reported they had closed this recommendation because they did not agree that the systems engineering plan was the most effective means to provide Congress insight into program risk. DOD officials stated that the timing of the systems engineering plan and any updates are not aligned to inform a budget decision that could occur as much as 18 months prior to program initiation; and existing statutory certifications and reports, such as 2366a and 2366b requirements, submitted to Congress contain adequate information regarding program risk and technical maturity. GAO initiated work in 2020 to examine recent congressionally mandated changes in DOD's acquisition and requirements processes and will assess whether those changes meet the intent of this recommendation.

Recommendation: To ensure that major steady state IT investments are being adequately analyzed, the Secretaries of Defense, Veterans Affairs, and the Treasury should direct appropriate officials to develop an OA policy, annually perform OAs on all investments, and ensure the assessments include all key factors.

Agency: Department of Veterans Affairs
Status: Open

Comments: For fiscal years 2013, 2014, and 2016, the Department of Veterans Affairs provided its operational analyses to GAO for its major information technology investments. These operational analyses addressed a majority of the key factors identified in Office of Management and Budget guidance. Nevertheless, in February 2019, the department was still in the process of finalizing its operational analysis policy and identified a target completion date of September 2019 for when the policy would be complete and ready for publication.