Recommendation: The FTA Administrator should take steps to provide information to Congress and sponsors to better clarify how FTA applies the methods and factors it considers when performing its review of sponsors' projects. Clarification could include how FTA uses documents, such as project roadmaps, to help sponsors advance through the development process; estimated time frames for reviewing submissions and responding to sponsors' requests; and how FTA considers geographic diversity of projects in its decision-making. (Recommendation 1)

Agency: Department of Transportation
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The FTA Administrator should take steps to review all agency guidance pertaining to the Capital Investment Grants program's development process and identify aspects that may be outdated or confusing for sponsors, and communicate to Congress and sponsors the steps needed to update FTA's guidance or otherwise provide sponsors with more current information on the Capital Investment Grants program. As part of FTA's efforts, FTA could obtain feedback from sponsors and the industry on these issues as appropriate. (Recommendation 2)

Agency: Department of Transportation
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The FTA Administrator should take steps to ensure its staff communicates information, such as the reason why a project is not advancing, to sponsors in a timely manner and in writing upon the sponsor's request. (Recommendation 3)

Agency: Department of Transportation
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments, provide updated guidance for the oversight of privatized military housing, to include oversight objectives for each service to monitor the physical condition of privatized homes over the remaining duration of the ground leases. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: The Department of Defense (DOD) concurred with this recommendation. In its August 2020 response, DOD noted that the Assistant Secretary of Defense for Sustainment (ASD (S)), as the Chief Housing Officer, issued guidance requiring the military departments to monitor work order completion for housing privatized under the Military Housing Privatization Initiative based on a combination of resident input, timeliness of work order completion, and number of repeat work orders for the same repair. The guidance also required increased tracking of MHPI project work orders by installation staff. Moving forward, the ASD(S) plans to issue quarterly program review guidance that establishes oversight objectives for the military departments to monitor the physical condition of MHPI housing over the duration of their project ground leases, formalizing the requirement that the data be monitored by the Chief Housing Officer. DOD expects this to be completed by December 2020. We will continue to monitor the status of this recommendation.

Recommendation: The Secretary of the Army should take steps, in collaboration with the Army's private housing partners, to review the indicators underlying the privatized housing project performance metrics to ensure they provide an accurate reflection of the condition and quality of the homes. (Recommendation 2

Agency: Department of Defense: Department of the Army: Office of the Secretary
Status: Open

Comments: The Department of Defense (DOD) concurred with this recommendation. In its August 2020 response, DOD noted that the Secretary of the Army has taken several steps toward addressing this recommendation. For example, the Army published the Portfolio and Asset Management Handbook creating a multi-tiered assessment approach of performance metrics to measure the health of each privatized home through inspection, assessment, satisfaction, and feedback. The Army and the private housing partners revised the Incentive Fee Performance Management Plan, placing increased emphasis on resident satisfaction and work order/maintenance management. The Army also put Commanders in charge, ensuring Army leadership at every Army installation is tracking housing quality and safety. In late 2020, the Army plans to review and evaluate these actions and make a determination by 31 Jan 2021 if any changes or revisions are needed to best implement the recommendation. As such, we will continue to monitor the status of this recommendation.

Recommendation: The Secretary of the Air Force should take steps, in collaboration with the Air Force's private housing partners, to review the indicators underlying the privatized housing project performance metrics to ensure they provide an accurate reflection of the condition and quality of the homes. (Recommendation 3)

Agency: Department of Defense: Department of the Air Force: Office of the Secretary of the Air Force
Status: Open

Comments: The Department of Defense (DOD) concurred with this recommendation. In its August 2020 response, DOD noted that the Air Force is engaging in several steps to address this recommendation. Specifically, in March 2020, the Air Force tasked each of the Military Housing Offices to inspect all move-in, move-out, and change of occupancy maintenance events and all emergency, urgent, and life, health, and safety work orders, which is outlined in Air Force guidance. The Air Force is also engaging in several ongoing actions. In response to a memo to the military departments to provide consistency of performance incentive fees, the Air Force was negotiating with the privatized housing project owners to update performance incentive fee metrics in accordance with ASD directed categories and weightings. As of August 2020, agreements had been finalized with 2 partners and work was ongoing with the remaining partners. In addition, the Air Force was working with the project owners to deploy Satisfacts, a survey tool to independently measure resident satisfaction with projects' work order performance, across all Air Force projects with an expected completion by December 2020. We will continue to monitor the status of these recommendations.

Recommendation: The Secretary of the Navy should take steps, in collaboration with the Navy and Marine Corps' private housing partners, to review the indicators underlying the privatized housing project performance metrics to ensure they provide an accurate reflection of the condition and quality of the homes. (Recommendation 4)

Agency: Department of Defense: Department of the Navy: Office of the Secretary
Status: Open

Comments: The Department of Defense (DOD) concurred with this recommendation. In its August 2020 response, DOD noted that the Navy and Marine Corps are engaging in several steps to address this recommendation. Specifically, the Navy and Marine Corps have developed a centralized electronic data warehouse, which receives data from privatized housing partner maintenance systems to display work order and survey performance dashboards. By February 2021, the Navy expects to complete the development of metrics displayed by the data warehouse to include key service call performance metrics and resident feedback data. The Navy and Marine Corps are also developing a web-based monitoring matrix tool housing officials can use to evaluate the performance of privatized housing partners. The tool is intended to provide improved tracking capabilities and improved accessibility to information, thus providing more consistent oversight and improved advocacy service members and their families. The Navy is also working to hire 247 additional Navy and Marine Corps housing staff to review and analyze private partner provided recurring maintenance and customer satisfaction reports in an effort to strengthen oversight and monitoring, with an estimated completion of September 2020. Moving forward, we will continue to monitor the status of these and other efforts.

Recommendation: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments and private housing partners, establish minimum data requirements and consistent terminology and practices for work order data collection for comparability across installations and projects and to track trends over time. (Recommendation 5)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: e Department of Defense (DOD) partially concurred with this recommendation. In its August 2020 response, DOD stated that the Assistant Secretary of Defense for Sustainment (ASD(S)), as the Chief Housing Officer, plans to issue a policy directing the military departments to establish, to the maximum extent practical, minimum data requirements and consistent terminology and practices for MHPI housing unit work order collection to aid in comparability across installations and projects, and for tracking trends over time. However, DOD noted that the department cannot mandate changes to existing MHPI project legal documents. DOD estimates that this effort will be completed by December 2021. We will continue to monitor the status of this recommendation.

Recommendation: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment require the military departments to establish a process to validate data collected by the private housing partners to better ensure the reliability and validity of work order data and to allow for more effective use of these data for monitoring and tracking purposes. (Recommendation 6)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: The Department of Defense (DOD) concurred with this recommendation. In its August 2020 response, DOD noted that the Assistant Secretary of Defense for Sustainment (ASD(S)), as the Chief Housing Officer, issued guidance directing the military departments to exercise proper oversight to ensure Military Housing Privatization Initiative (MHPI) projects perform in accordance with legal agreements, to include due diligence in monitoring and auditing project maintenance records and other project performance data. The guidance also required military departments to review their entire portfolios of MHPI projects to ensure accurate and appropriate work order management processes. In response to the new guidance, DOD noted that the military departments put in place appropriate oversight measures and undertook the required reviews, though the investigations of project business practices were ongoing in some cases. As another step, the ASD(S) plans to issue guidance directing the military departments to establish a process to validate data collected by their respective MHPI Project Owners to better ensure the reliability and validity of work order data and to allow for more effective use of these data for monitoring and tracking purposes. DOD expects this to be completed by the end of September 2020. We will continue to monitor the status of this recommendation.

Recommendation: The Secretary of Defense should ensure the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments, develop a process for collecting and calculating resident satisfaction data from the military departments to ensure that the data are compiled and calculated in a standardized and accurate way. (Recommendation 7)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: The Department of Defense (DOD) partially concurred with this recommendation based on the fact that the draft report listed the incorrect office as the source for addressing the deficiency, but subsequently changed its response to concur after the recommendation was directed to the appropriate office in the final report. In its August 2020 response, DOD noted that the Assistant Secretary of Defense for Sustainment (ASD(S)) plans to issue guidance establishing a department-wide process for collecting and calculating resident satisfaction data to ensure that the data are compiled and calculated in a standardized and accurate way effective with the survey collection effort in Fiscal Year 2021. The department expects this effort to be completed by October 2020. We will continue to monitor the status of this recommendation.

Recommendation: The Secretary of Defense should ensure the Assistant Secretary of Defense for Sustainment provides additional explanation of the data collected and reported in future reports to Congress, such as explaining the limitations of available survey data, how resident satisfaction was calculated, and reasons for any missing data, among other things. (Recommendation 8)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: The Department of Defense (DOD) partially concurred with this recommendation. In its August 2020 response, DOD noted that the Assistant Secretary of Defense for Sustainment (ASD(S)) would provide additional explanation of the MHPI resident satisfaction data collected and reported in future annual Military Housing Privatization Initiative (MHPI) reports to Congress, effective with the annual report covering fiscal year 2019. DOD noted that the additional information will include, among other things, an explanation of the limitations of available survey data, how resident satisfaction was calculated, and reasons for any missing data. As of August 2020, the annual MHPI report covering fiscal year 2018 was in final coordination and the department noted that the report would addresses a vast majority, but not all, of the requirements identified in our recommendation. DOD noted that the additional information would be provided in the next annual MHPI report. We will continue to monitor the status of this recommendation.

Recommendation: The Secretary of the Army should develop and implement a plan to clearly and systematically communicate to residents the difference between the military housing office and the private partner. At a minimum, these plans should include the Army housing office's roles, responsibilities, locations, and contact information and should ensure that all residents are aware that they can directly contact Army housing office officials. (Recommendation 9)

Agency: Department of Defense: Department of the Army: Office of the Secretary
Status: Open

Comments: The Department of Defense (DOD) concurred with this recommendation. In its response, DOD noted that the Army developed a "Plain Language" briefing as required by the 2020 National Defense Authorization Act that included the Army Housing Office's roles, responsibilities, location, and contact information at each privatized housing project site. DOD noted that the intent of the briefing was to ensure that all residents were aware of their ability to directly contact Army Housing Office and/or the Garrison Commanders. DOD stated that the briefing was disseminated to all of the Military Housing Offices, who are using it in newcomer briefings, and stated that the briefing would be provided to all current residents of privatized military housing, but that measure would not be tracked due to attrition. In addition, DOD noted that Headquarters, Department of the Army was tasking Army Materiel Command to develop a more detailed plan to communicate to residents the difference between the Army Housing Office and the private housing partner. The Army's intent is to not only capture residents upon their arrival at an installation, but making the services of the MHO known over the duration of a resident's time on at installation. We will continue to monitor the status of this recommendation.

Recommendation: The Secretary of the Navy should develop and implement a plan to clearly and systematically communicate to residents the difference between the military housing office and the private partner. At a minimum, these plans should include the Navy housing office's roles, responsibilities, locations, and contact information and should ensure that all residents are aware that they can directly contact Navy housing office officials. (Recommendation 11)

Agency: Department of Defense: Department of the Navy: Office of the Secretary
Status: Open

Comments: The Department of Defense (DOD) concurred with this recommendation. In its August 2020 response, DOD noted that the Navy has taken various steps to address this recommendation, with additional steps planned. For example, the Navy has ensured that each installation has a specific issue resolution process description marketing flyer available, both in hard copy and on the public housing websites, with a reminder that residents can contact both the privatized housing property manager and the Navy housing office with any issues. Moreover, every housing unit has been provided with a refrigerator magnet reminding residents that they can and should contact the Navy housing office if they have any issues with their home. In addition, the Navy and Marine Corps have established a requirement to contact each privatized housing resident not later than 15 days after move-in and again 60 days after move-in to provide an opportunity to request assistance and remind them of available support. Moving forward, the Navy has an ongoing effort to require private housing companies to market the same messaging as the service issue resolution processes for the MHOs that they support, for consistent advocacy messaging to the tenants. The information will be added to PPV partner websites, printed material and resident handbooks. The Navy also plans to use its annual survey to tracks resident satisfaction and awareness of the Navy's issue resolution process, with expected completion by October 2020. In addition, the Marine Corps has identified a near-term initiative to procure name tags for all MHO employees to wear, identifying themselves as distinct and separate from privatized housing property management company, which will be standardized across all USMC installations. The Marine Corps also plans to develop a standard welcome aboard package to include magnets and other items with key point of contact information. The Marine Corps expects these efforts to be completed by the end of September 2020.

Recommendation: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments, assess the risks of proposed initiatives aimed at improving the privatized military housing program on the financial viability of the projects. (Recommendation 12)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: The Department of Defense (DOD) concurred with this recommendation. In its August 2020 response, DOD noted that the Assistant Secretary of Defense for Sustainment, as the Chief Housing Officer, planned to issue a policy establishing the assessment of Military Housing Privatization Initiative (MHPI) project financial viability as part of quarterly program reviews as a long-term requirement. The department noted that the program review data would be augmented by input from the MHPI companies, who are assessing the likely impact of proposed initiatives in conjunction with their third party lenders. The department expected this effort to be completed by December 2020. We will continue to monitor the status of this recommendation.

Recommendation: The Chairman of the Board should require the Safety Recommendation and Communication team to fully document its evaluations when assessing items to propose for Most Wanted List consideration (Recommendation 1).

Agency: National Transportation Safety Board
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Chairman of the Board should take steps to publicly and fully communicate the selection rationale, such as including why NTSB believes an issue is "ripe for action" to its documentation on its website (Recommendation 2).

Agency: National Transportation Safety Board
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops a program-wide process for measuring, collecting, and tracking information on how ALIS is affecting the performance of the F-35 fleet to include, but not be limited to, its effects on mission capability rates. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops and implements a strategy for the re-design of ALIS. The strategy should be detailed enough to clearly identify and assess the goals, key risks or uncertainties, and costs of re-designing the system. (Recommendation 2)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to take steps to ensure that the details of the pilot program's data protections are incorporated into the existing agreements. (Recommendation 1)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will require the Primary Sponsor for each applicable federally funded research and development center (FFRDC) to provide documentation that shows that the details of the pilot program's data protections have been incorporated into existing sponsoring agreements and contracts, and estimated these actions would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to take steps to ensure that the FFRDCs and sponsors are implementing the pilot program's protections for sensitive data. (Recommendation 2)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will task the Primary Sponsor for each applicable federally funded research and development center (FFRDC) to provide a report detailing the FFRDC's plan for implementing the pilot program's protections for sensitive data and confirming that the FFRDC is adhering to that plan. OUSD(R&E)/L&PO estimated these actions would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to establish a monitoring and oversight mechanism to ensure that primary sponsors submit complete information on pilot projects, as required by DOD's guidance for the pilot program. (Recommendation 3)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will develop a monitoring and oversight plan, and estimated it would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan that outlines the methodology by which DOD will assess the pilot and how and when information collected will be analyzed to evaluate the pilot program. (Recommendation 4)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will develop and coordinate a plan that outlines the methodology by which DOD will assess the pilot as well as the methodology for collecting and analyzing information to evaluate the pilot program. OUSD(R&E)/L&PO estimated these actions would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan to identify and evaluate lessons learned from the pilot program. (Recommendation 5)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will develop a plan to identify and evaluate lessons learned from the pilot program, and estimated the plan would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan for obtaining input from stakeholders on the pilot program. (Recommendation 6)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will develop a plan to obtain stakeholder inputs for use in evaluating the pilot program, and estimated the plan would be completed in early Fall 2020.

Recommendation: We recommend the Secretary of the Air Force issue policy emphasizing the following three key reliability practices when planning and executing acquisition programs: (1) leveraging reliability engineers early and often, (2) establishing realistic reliability requirements, and (3) employing reliability engineering activities to improve a system's design throughout development. (Recommendation 1)

Agency: Department of Defense: Department of the Air Force: Office of the Secretary of the Air Force
Status: Open

Comments: The Air Force agreed with this recommendation and told us they will be updating Air Force Instruction AFI63-101/20-101 to ensure the inclusion of key practices to improve weapon system reliability, including leveraging reliability engineers early and often, establishing realistic reliability requirements, and employing reliability engineering activities to improve a system's design throughout development. The Air Force said it plans to complete this effort by December 31, 2020.

Recommendation: We recommend the Secretary of the Navy issue policy emphasizing the following three key reliability practices when planning and executing acquisition programs: (1) leveraging reliability engineers early and often, (2) establishing realistic reliability requirements, and (3) employing reliability engineering activities to improve a system's design throughout development. (Recommendation 3)

Agency: Department of Defense: Department of the Navy: Office of the Secretary
Status: Open

Comments: The Navy agreed with this recommendation and told us they plan to address GAO's recommendation in two ways. First, they are developing a guidebook for program managers and leadership to emphasize the importance of leveraging reliability engineers early and often, establishing realistic reliability requirements, and employing reliability engineering activities to improve a system's design throughout development. The Navy anticipates the guidebook will be completed by the end of fiscal year 2021. Second, the Navy plans to update the SECNAV 5000.2 when it is open for changes. They noted, this is dependent on the Office of the Secretary of Defense (OSD) completing its ongoing update to DOD-wide acquisition guidance. The Navy anticipates OSD's efforts will be completed by end of calendar year 2020.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations directs the Advanced Exploration Systems division to define and determine a schedule for synchronization reviews, including the role of the proposed Lunar Exploration Control Board, to help ensure that requirements between mission and program levels are reconciled. (Recommendation 1)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation and stated that the Advanced Exploration Systems division will review program life-cycle review plans to ensure enterprise and program requirements are reconciled across the mission. NASA is in the process of determining the organizational structure of the Human Exploration and Operations Mission Directorate. Following this completion, NASA officials stated that the appropriate control board and division structures for review and program direction will become active.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations directs the Gateway program to conduct a joint cost and schedule confidence level at the program level for the Artemis III mission. (Recommendation 2)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation and stated it would conduct a joint cost and schedule confidence level analysis or equivalent. The Gateway program is planning to conduct a series of project- and program-level reviews and assessments aligned with key decision point reviews. This includes conducting a joint cost and schedule confidence level analysis or equivalent of the Gateway initial configuration to support a program key decision point planned for fall 2021. NASA has not yet taken action on this recommendation.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations directs the Gateway program to update its overall schedule for 2024 to add a KDP II to occur before system integration. (Recommendation 3)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation, but has not yet taken action on it. NASA stated that it would provide a schedule for future reviews, including whether there will be a Key Decision Point (KDP) II, at the KDP-I review currently scheduled for fall 2021.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations creates a life-cycle cost estimate for the Artemis III mission. (Recommendation 4)

Agency: National Aeronautics and Space Administration
Status: Open
Priority recommendation

Comments: NASA agreed with the recommendation and stated that the agency will provide a preliminary cost estimate for the Artemis III mission by the end of 2020. Further, NASA stated that it will provide an updated cost estimate for the Artemis III mission after it establishes cost and schedule commitments for some of the projects that compose the lunar mission, currently planned for the Spring of 2021. To fully implement this recommendation, NASA will need to provide a cost estimate.

Recommendation: The NASA Administrator should ensure that the NASA Associate Administrator for Human Exploration and Operations directs the Advanced Exploration Systems division to commit to a completion date and finalize a cohesive document outlining the rationale for selecting its current lunar architecture and lunar programs. (Recommendation 5)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation and stated that it is developing a document that will summarize the trades and architectural studies, but the document is not yet complete.

Recommendation: The NASA Administrator should ensure that the Office of the Chief Engineer determines under what conditions it is appropriate to complete an analysis of alternatives, particularly when there are multiple pathways—including architectures or programs—that NASA could pursue in the future, and document the justification for not completing an analysis. (Recommendation 6)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with this recommendation, but has not yet taken any action on it. NASA stated that it will provide additional clarifying guidance for conducting analyses of alternatives for new programs in the next update to NASA Procedural Requirements 7120.5E, "NASA Space Flight Program and Project Management Requirements." NASA plans to complete the update of the procedural requirement in September 2021.

Recommendation: The CBP Commissioner, in conjunction with the DHS Office of the Chief Readiness Support Officer, should develop and implement a plan to ensure that CBP executes its Facility Condition Assessments (FCA) program by conducting FCAs at each CBP-owned land border crossing consistent with DHS Directive 119-02-004. (Recommendation 1)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: CBP concurred with the recommendation and, in March 2020, provided a plan for conducting FCAs at some, but not all, CBP-owned land border crossings. According to officials, CBP will update this plan to include all CBP-owned land border crossings and may coordinate with the DHS Office of the Chief Readiness Support Officer to ensure the plan is consistent with DHS Directive 119-02-004. To fully address this recommendation, CBP should complete the FCA plan to include all CBP-owned land border crossings.

Recommendation: The CBP Commissioner should share FCA reports with GSA and use facility condition information in GSA's Building Assessment Tool to inform FCAs. (Recommendation 2)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: CBP concurred with the recommendation and, in February 2020, GSA confirmed that it received FCA reports from CBP for GSA-owned land border crossings. GSA also confirmed that it provided CBP with a spreadsheet containing data from GSA Building Assessment Tools. To fully address this recommendation, CBP should demonstrate that it is using GSA Building Assessment Tool information to inform its FCAs at GSA-owned land border crossings.

Recommendation: The GSA Administrator should share Building Assessment Tool reports with CBP and use facility condition information in CBP's FCAs to inform its assessments through the Building Assessment Tool. (Recommendation 3)

Agency: General Services Administration
Status: Open

Comments: GSA concurred with the recommendation and, in April 2020, provided documentation that it received CBP FCAs and provided Building Assessment Tool information to CBP. GSA further provided updated guidance describing how it will share this information with CBP going forward. To fully address this recommendation, GSA should demonstrate that it using CBP FCA reports to inform its Building Assessment Tool assessments.

Recommendation: The GSA Administrator, in conjunction with CBP, should share with CBP information on GSA maintenance and repair work at GSA-owned land border crossings at the level of detail necessary to inform CBP's data in TRIRIGA. (Recommendation 4)

Agency: General Services Administration
Status: Open

Comments: GSA concurred with the recommendation and, in December 2019, stated that it would share this information with CBP on a monthly basis until it is available to CBP for self-service on a shared data system. To fully address this recommendation, GSA should provide documentation that CBP has access to the shared data system.

Recommendation: The CBP Commissioner should use information on maintenance and repair work conducted by GSA at GSA-owned land border crossings to update facility condition information in TRIRIGA on an ongoing basis. (Recommendation 5)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: CBP concurred with the recommendation and, in December 2019, reported that it began documenting processes for validating and correcting existing records in TRIRIGA using information on maintenance and repair work conducted by GSA. In addition, CBP reported that it is developing a standard operating procedure for TRIRIGA data entry. To fully address this recommendation, CBP should demonstrate that it is regularly receiving information on maintenance and repair work performed by GSA and that it is using this information to update data in TRIRIGA.

Recommendation: The CBP Commissioner should establish review time frames for stakeholders involved in its Five-year Capital Investment Plan review and approval process. (Recommendation 6)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: CBP concurred with the recommendation and, in April 2020, provided a document establishing time frames for stakeholders involved in the five-year plan review and approval process. To fully address this recommendation, CBP should demonstrate that it has communicated these time frame expectations to stakeholders.

Recommendation: The CBP Commissioner should establish and document a methodology for its annual land border crossing capital prioritization process that includes procedures and time frames for each step. (Recommendation 7)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: CBP concurred with the recommendation and, in April 2020, provided a document that formally establishes and documents a methodology for its land border crossing prioritization process, including procedures and time frames for each step. To fully address this recommendation, CBP should demonstrate that it is implementing this methodology during its annual five-year capital investment planning process.

Recommendation: The Secretary of Defense should issue guidance on incorporating climate projections into installation master planning, including—at a minimum—what sources of climate projections to use, how to use projections involving multiple future scenarios, and what future time periods to consider. (Recommendation 4)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation, and stated that it is in the process of developing guidance to incorporate projections for sea level change into DOD's Unified Facilities Criteria standard for installation master planning, using a DOD-vetted source of data. DOD also stated that it will continue to tailor additional sources of climate projections data to other planning requirements and integrate these into departmental criteria as appropriate. In February 2020, the Under Secretary of Defense for Acquisition and Sustainment issued a memorandum with guidance on incorporating sea level change projections in installation master planning, and as of April 2020, was researching additional sources of climate projection data to tailor to other planning requirements. DOD estimated that it would complete implementation of this recommendation in the fourth quarter of fiscal year 2020. We will continue to monitor the status of these efforts.

Recommendation: The Secretary of Defense should issue guidance on incorporating climate projections into facilities project designs, including—at a minimum—what sources of climate projections to use, how to use projections involving multiple future scenarios, and what future time periods to consider. (Recommendation 5)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation, and stated that it is in the process of developing guidance to incorporate projections for sea level change into Unified Facilities Criteria for facilities design, using a DOD-vetted source of data. DOD also stated that it will continue to tailor additional sources of climate projections data to other engineering requirements and integrate these projections into its criteria as appropriate. In February 2020, the Under Secretary of Defense for Acquisition and Sustainment issued a memorandum with guidance on incorporating sea level change projections into facilities designs, and as of April 2020, was researching additional sources of climate projection data to tailor to other planning requirements. DOD estimated that it would complete implementation of this recommendation in the fourth quarter of fiscal year 2020. We will continue to monitor the status of these efforts.

Recommendation: The Secretary of the Army should ensure that the Chief of Engineers and Commanding General of the U.S. Army Corps of Engineers works with the Assistant Secretary of Defense for Sustainment; the Chief of Civil Engineers and Commander, Naval Facilities Engineering Command; and the Director of the Air Force Civil Engineer Center to update relevant Unified Facilities Criteria to require that installations consider climate projections in designing facilities projects and incorporate, as appropriate, DOD guidance on the use of climate projections in facilities project designs—including identification of authoritative sources of such projections, use of projections involving multiple future scenarios, and what future time periods to consider. (Recommendation 6)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: DOD concurred with this recommendation. As of April 2020, DOD stated that the update of Unified Facilities Criteria to implement this recommendation was pending the issuance of additional guidance by DOD on the use of such projections in project designs, as recommended by GAO. DOD estimated it would complete the actions to implement this recommendation in the second quarter of fiscal year 2021. We will continue to monitor the status of this effort.

Recommendation: The Secretary of the Navy should ensure that the Chief of Civil Engineers and Commander, Naval Facilities Engineering Command works with the Assistant Secretary of Defense for Sustainment, the Chief of Engineers and Commanding General of the U.S. Army Corps of Engineers, and the Director of the Air Force Civil Engineer Center to update relevant Unified Facilities Criteria to require that installations consider climate projections in designing facilities projects and incorporate, as appropriate, DOD guidance on the use of climate projections in facilities project designs—including identification of authoritative sources of such projections, use of projections involving multiple future scenarios, and what future time periods to consider. (Recommendation 7)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: DOD concurred with this recommendation. As of April 2020, DOD stated that the update of Unified Facilities Criteria to implement this recommendation was pending the issuance of additional guidance by DOD on the use of such projections in project designs, as recommended by GAO. DOD estimated it would complete the actions to implement this recommendation in the second quarter of fiscal year 2021. We will continue to monitor the status of this effort.

Recommendation: The Secretary of the Air Force should ensure that the Director of the Air Force Civil Engineer Center works with the Assistant Secretary of Defense for Sustainment; the Chief of Engineers and Commanding General of the U.S. Army Corps of Engineers; and the Chief of Civil Engineers and Commander, Naval Facilities Engineering Command to update relevant Unified Facilities Criteria to require that installations consider climate projections in designing facilities projects and incorporate, as appropriate, DOD guidance on the use of climate projections in facilities project designs—including identification of authoritative sources of such projections, use of projections involving multiple future scenarios, and what future time periods to consider. (Recommendation 8)

Agency: Department of Defense: Department of the Air Force
Status: Open

Comments: DOD concurred with this recommendation. As of April 2020, DOD stated that the update of Unified Facilities Criteria to implement this recommendation was pending the issuance of additional guidance by DOD on the use of such projections in project designs, as recommended by GAO. DOD estimated it would complete the actions to implement this recommendation in the second quarter of fiscal year 2021. We will continue to monitor the status of this effort.

Recommendation: The Secretary of the Navy should clarify whether and how to use the laboratory initiated research authority within the Capital Investment Program. (Recommendation 2)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: In December 2018, DOD agreed with our recommendation. In March 2019, DOD reported that the Navy was working on establishing a process for its laboratories to use the funds made available to them through the laboratory initiated research authority and planned to have this new policy in place by September 1, 2019 but was subsequently changed to a new date of October 1, 2019. However, the Navy was unable to finalize its policy prior to the start of the 2020 fiscal year. In February 2020, a senior USD(R&E) official stated that internal discussions between Navy acquisition officials and Navy financial management officials were ongoing. This official further noted that the Navy planned to finalize its policy by December 1, 2020, in time to influence the Navy's fiscal year 2022 budget request.

Recommendation: FERC should provide standard language and procedures to its staff on how to record information collected during inspections, including how and where to record information about safety deficiencies, in order to facilitate analysis of safety deficiencies across FERC's portfolio of regulated dams. (Recommendation 1)

Agency: Federal Energy Regulatory Commission
Status: Open

Comments: In February 2020, GAO confirmed that FERC had developed new standard operating procedures related for tracking deficiencies and follow-up items arising from dam safety inspections and other dam safety reviews. In addition, FERC told GAO that they plan to update their tracking system beginning in fiscal year 2021, which will facilitate the complete recording and subsequent analysis of safety deficiencies from inspections across FERC's portfolio of regulated dams. GAO will continue to monitor FERC's efforts to implement this recommendation.

Recommendation: FERC should use information from its inspections to assess safety risks across its portfolio of regulated dams to identify and prioritize safety risks at a national level. (Recommendation 2)

Agency: Federal Energy Regulatory Commission
Status: Open

Comments: In January 2019, FERC told GAO that it had begun developing a screening-level risk-assessment program to assess safety risks across the inventory of regulated dams and to help guide safety decisions. In February 2020, GAO confirmed that FERC had completed this screening-level risk assessment, and conducted some preliminary analysis of the results. In addition, FERC told GAO that the results of the screening level risk analyses will be used to revise the potential timing, frequency, and technical disciplines represented on dam safety inspections; to confirm or revise the urgency of existing and potential new follow up dam safety actions; and to identify previously unrecognized dam safety concerns and issues; to identify new dam safety priorities. GAO will continue to monitor FERC's efforts to implement this program.

Recommendation: As the Subcommittee on Quantum Information Science (QIS Subcommittee) moves forward, the Office of Science and Technology Policy co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 1)

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: The Office of Science and Technology Policy (OSTP) agreed with GAO's September 2018 recommendation despite expressing some concerns about required resources and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019, the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the Department of Commerce co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 2)

Agency: Department of Commerce
Status: Open

Comments: The Department of Commerce agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the Department of Energy co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 3)

Agency: Department of Energy
Status: Open

Comments: The Department of Energy agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the National Science Foundation co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 4)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the Interagency Working Group on Synthetic Biology moves forward, the Director of the National Science Foundation, in coordination with participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 5)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation (NSF) agreed with GAO's September 2018 recommendation and, as of January 2020, had taken some steps to implement it. In November 2018, the Interagency Working Group on Synthetic Biology was formally established under the Biological Sciences Subcommittee of the National Science and Technology Council. The co-chairs of the Interagency Working Group on Synthetic Biology are officials from the National Institutes of Health, the National Institute of Standards and Technology, and NSF. The charter for the working group states that the group is to facilitate coordination and collaboration across 16 federal agencies. In October 2019, the working group hosted an Interagency Synthetic Biology Workshop to examine a roadmap that included basic science, enabling technologies, infrastructure and workforce needs in the area of synthetic biology. The workshop included 100 participants across the federal government, academia and industry, according to NSF officials. On the final day of the workshop participants from federal agencies used the input from the workshop to prepare a list of priority areas for investment along with agencies interested in participating in those priority areas. In January 2020, NSF officials reported that among the next steps for the working group was to develop a federal strategic roadmap for synthetic biology. Officials also reported that the working group is actively preparing a memorandum of understanding to create policies that will enable more sharing of information and collaboration. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in synthetic biology. When we confirm what additional actions the working group has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: The Secretary of Energy should direct ORP to require the WTP contractor to determine the full extent to which problems exist in all WTP structures, systems, and components.

Agency: Department of Energy
Status: Open

Comments: DOE agreed with the recommendation. As of June 15, 2020, DOE has started to take action to implement this recommendation, but actions are not sufficient to close it yet. DOE has directed the Waste Treatment and Immobilization Plant (WTP) construction contractor to determine the full extent to which problems exist in all WTP structures, systems, and components and plans to complete full implementation of this recommendation by February 15, 2021.

Recommendation: The Secretary of Energy should direct ORP to use its authorities to stop work in areas where quality assurance problems are recurring until ORP's Quality Assurance Division can verify that the problems are corrected and will not recur.

Agency: Department of Energy
Status: Open

Comments: DOE agreed with the recommendation. As of July 1, 2020, DOE has taken some action to implement this recommendation, but actions are not sufficient to close it yet. To implement this recommendation, DOE strengthened and evaluated the existing Office of River Protection (ORP) Stop Work process to ensure it is sufficiently rigorous. In June 2020, ORP activated this new process. However, ORP has not yet used its authority to stop work in areas in areas where quality assurance problems are recurring.

Recommendation: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Energy, Installations, and the Environment work with DOD's construction agents, military departments, and other offices to improve DOD's MILCON cost estimating guidance (i.e., DOD's Unified Facilities Criteria) by fully incorporating all the steps needed for developing high-quality reliable cost estimates. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD partially concurred with our recommendation to improve its cost estimating guidance by fully incorporating all 12 steps needed for developing high-quality, reliable estimates. DOD stated that it did not believe that it is suitable to fully apply all 12 steps to any construction project due to characteristics of the military construction program that DOD believes differ from those of major system or weapon acquisition programs. However, DOD also stated that it concurred with the intent and general applicability of the twelve steps to military construction and that DOD cost estimating guidance lacks specificity in several of these areas. DOD acknowledged that expanding its cost guidance to more fully incorporate these steps would benefit the military construction program, and told us that it is planning to address this by revising its cost guidance. In our report, we recognize that it may not be appropriate to fully apply all 12 steps to each construction project. For example, it may not be realistic or to the military departments' benefit to conduct a sensitivity and uncertainty analysis or develop an independent cost estimate for all the construction projects they initiate every year, especially for low-cost projects. Accordingly, we did not recommend that DOD fully apply all 12 steps to each construction project, but rather that it fully incorporate the 12 steps into the Unified Facilities Criteria so that, at least, each step is considered for each project. DOD could then choose to establish thresholds-based on, for example, the dollar values of the projects-to determine for which the 12 steps should be fully applied or other circumstances in which some steps might not be applicable. We believe DOD's planned revisions once completed will meet the general intent of our recommendation.

Recommendation: The leadership of the Committee on STEM Education should review performance assessments of federal STEM education programs and then take appropriate steps to enhance effectiveness of the portfolio, such as by sharing promising practices that agencies could use in designing or revising their programs. (Recommendation 1)

Agency: Committee on Science, Technology, Engineering, and Math Education
Status: Open

Comments: The Committee agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. The Committee's plan states that to enhance the effectiveness of the STEM education portfolio, federal agencies that comprise the Committee are to perform a systematic review of evidence from current programs (e.g., performance assessments). Committee agencies are to also identify promising, evidence-based STEM education programs, practices, and policies in order to learn from and implement them more broadly. The strategic plan commits federal agencies to develop an implementation plan. According to agency officials, the implementation plan will be finalized in the Fall of 2019. GAO will review future efforts of the Committee to determine if they address the recommendation and could lead to improved management of the federal STEM education portfolio.

Recommendation: The leadership of the Committee on STEM Education should improve public awareness of information on programs' performance assessments by documenting program-level information on performance assessments in its federal STEM education inventory. (Recommendation 2)

Agency: Committee on Science, Technology, Engineering, and Math Education
Status: Open

Comments: The Committee agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. One of the plan's objectives focuses on making program performance and outcomes publically available. To achieve this objective, federal agencies that comprise the Committee are to document and share programs' performance assessment information for both internal strategic planning purposes and to inform public audiences. According to the strategic plan, Committee agencies are to provide content for the inventory of STEM education programs by the end of 2019. GAO will review the results of the Committee's future efforts, including its inventory of programs, to determine if they adequately address the recommendation. The plan acknowledges GAO's finding on the importance of collecting, organizing, and reporting federal performance data to increase public awareness of the impact of federal STEM education programs.

Recommendation: The leadership of the Committee on STEM Education should report required information on the participation rates of women, underrepresented minorities, and persons from rural areas in federal STEM education programs that collect this information. (Recommendation 3)

Agency: Committee on Science, Technology, Engineering, and Math Education
Status: Open

Comments: The Committee agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. According to the strategic plan, the Committee will develop a common reporting framework that federal agencies with STEM education programs can use to provide the rates of program participation by women, underrepresented minorities, and persons in rural areas. Agencies will begin reporting this information by the end of 2019. According to an official from the Office of Science and Technology Policy, the agency anticipates reporting programs' participation rates in its 2020 annual report to Congress. The plan also states that agencies that conduct valid and reliable tracking of such data are to share their promising practices across agencies so that those strategies can be replicated where and when appropriate. GAO will follow the Committee's progress developing and implementing the reporting framework to determine if it addresses GAO's recommendation. As noted in the strategic plan, tracking and reporting on the participation of underrepresented groups in federal STEM education programs provides agencies with information to gauge their effectiveness at fostering diversity and inclusion.

Recommendation: The Director of the Office of Science and Technology Policy should report the outcomes of programs' performance assessments completed in the previous year in its annual report. (Recommendation 4)

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: The Office of Science and Technology Policy agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. The strategic plan states that the Office of Science and Technology Policy will include in its annual reports descriptions of the outcomes of any program assessments conducted on federal STEM education programs in the previous year. According to an Office of Science and Technology Policy official, this information will be included in annual reports starting in 2020. GAO will review these annual reports to determine if they address the recommendation. As noted in the strategic plan, collecting and reporting federal performance data supports efficient and effective use of public funding, increases public awareness of federal program outcomes, and promotes the use of high-quality performance assessment and evaluation methods.

Recommendation: To enhance program oversight and provide more robust input to budget deliberations, Congress should consider requiring DOD to report on each major acquisition program's systems engineering status in the department's annual budget request, beginning with the budget requesting funds to start development. The information could be presented on a simple timeline--as done for the case studies in this report--and at a minimum should reflect the status of a program's functional and allocated baselines as contained in the most current version of the program's systems engineering plan.

Agency: Congress
Status: Open

Comments: As of August 2020, Congress has not yet taken action on the matter for consideration. The Department of Defense (DOD) non-concurred and as of March 16, 2018, DOD officials reported they had closed this recommendation because they did not agree that the systems engineering plan was the most effective means to provide Congress insight into program risk. DOD officials stated that the timing of the systems engineering plan and any updates are not aligned to inform a budget decision that could occur as much as 18 months prior to program initiation; and existing statutory certifications and reports, such as 2366a and 2366b requirements, submitted to Congress contain adequate information regarding program risk and technical maturity. GAO initiated work in 2020 to examine recent congressionally mandated changes in DOD's acquisition and requirements processes and will assess whether those changes meet the intent of this recommendation.

Recommendation: To ensure that risks associated with ALIS are addressed expediently and holistically, the Secretary of Defense should direct the F-35 Program Executive Officer to improve the reliability of its cost estimates, conduct uncertainty and sensitivity analyses consistent with cost-estimating best practices identified in GAO's Cost Estimating and Assessment Guide.

Agency: Department of Defense
Status: Open

Comments: DOD partially concurred with this recommendation. According to DOD officials, as of July 2018, this recommendation conflicts with established Office of the Under Secretary of Defense/Cost Estimation and Program Evaluation guidance for cost estimation and uncertainty analysis. Absent a change in policy at that level, the Joint Program Office will continue to follow Office of the Under Secretary of Defense/Cost Estimation and Program Evaluation policy on this issue. We continue to believe that in order for any risks associated with ALIS to be addressed expediently and holistically, uncertainty and sensitivity analysis must be used on the F-35s cost estimates to improve its overall reliability. Thus, this recommendation will remain open.

Recommendation: To ensure that risks associated with ALIS are addressed expediently and holistically, the Secretary of Defense should direct the F-35 Program Executive Officer to improve the reliability of its cost estimates, ensure that future estimates of ALIS costs use historical data as available and reflect significant program changes consistent with cost-estimating best practices identified in GAO's Cost Estimating and Assessment Guide.

Agency: Department of Defense
Status: Open

Comments: DOD partially concurred with this recommendation. According to DOD officials, since April 2016, the F-35 program has continued to update the ALIS estimate with the latest available cost data, based on recent contracts. Until more reliable actual costs become available, the program utilizes negotiated contract costs, incorporates program initiatives, and ensures the estimate reflects the latest technical baseline and requirements. Until actual costs associated with ALIS historical data are incorporated in the F-35 cost estimate, we believe that the estimate will not be as reliable as it could be. For this reason, this recommendation will remain open.

Recommendation: To determine the viability and cost-effectiveness of reducing transmission times for biometrics data, the Secretary of Defense should direct the Assistant Secretary of Defense for Research and Engineering, through the Under Secretary of Defense for Acquisition, Technology, and Logistics, to comprehensively assess and then address, as appropriate, the factors that contribute to transmission time for biometrics data.

Agency: Department of Defense
Status: Open

Comments: In January 2016, DOD published Directive 8521.01E, Defense Biometrics, which directs the Secretary of the Army to measure the health and performance of the DOD Biometrics Enterprise and generate results for the Biometrics Principal Staff Assistant and the DOD Biometrics Executive Committee. OUSD(AT&L) and Army officials also noted that the department is required to obtain a favorable evaluation from the Director, Operational Test and Evaluation (DOT&E) and the Army Test and Evaluation Command in order to obtain approval for extending the service life of DOD's authoritative biometric system. These officials note that the tests and evaluations required for such approval will include an assessment of transmission and response times against approved requirements for the biometrics system. However, Marine Corps officials highlighted continued biometrics data transmission and synchronization issues with a currently fielded biometric capability that uses some of the same technology we identified issues with during the course of our review. In Summer 2017, DOD informed GAO that the department will soon issue a report to address these issues, so GAO is keeping this recommendation open until such time as DOD's report becomes available for GAO review.