Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops a program-wide process for measuring, collecting, and tracking information on how ALIS is affecting the performance of the F-35 fleet to include, but not be limited to, its effects on mission capability rates. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops and implements a strategy for the re-design of ALIS. The strategy should be detailed enough to clearly identify and assess the goals, key risks or uncertainties, and costs of re-designing the system. (Recommendation 2)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Homeland Security should ensure that the Undersecretary for Management develops an oversight process to confirm that programs' schedule goals are developed and updated in accordance with GAO's Schedule Assessment Guide, to include ensuring traceability between APB schedule goals and IMSs. (Recommendation 1)

Agency: Department of Homeland Security: Office of the Secretary
Status: Open

Comments: In providing comments on this report, DHS concurred with our recommendation and stated that the Management Directorate's Office of Program Accountability and Risk Management (PARM) developed a checklist based on GAO's Schedule Assessment Guide, among other things, to evaluate programs' IMSs. PARM also plans to develop guidance on schedules which is intended to assist the Component Acquisition Executives (CAE) and acquisition program staff responsible for building IMSs and APBs. In July 2020, PARM officials reported that the schedule checklist was already being used to evaluate and ensure program IMSs adhered to GAO's Schedule Assessment Guide. In addition, PARM officials are updating and drafting guidance on schedules to assist CAEs and program staff when building IMSs and APBs. As of July 2020, PARM was still in the process of executing these efforts.

Recommendation: The Secretary of Homeland Security should ensure that the Undersecretary for Management revises the schedule development guidance in the Systems Engineering Life Cycle Guidebook to state clearly that an IMS should be used as the basis for APB schedule goals. (Recommendation 2)

Agency: Department of Homeland Security: Office of the Secretary
Status: Open

Comments: DHS concurred with our recommendation and stated that Management Directorate's Office of Program Accountability and Risk Management (PARM) was in the process of revising the Systems Engineering Life Cycle Guidebook and would clarify the language relating to IMSs to ensure guidance is consistent. As of July 2020, PARM was still in the process of revising the Systems Engineering Life Cycle Guidebook.

Recommendation: The Secretary for Homeland Security should direct the Director of OTE to revise T&E policy or guidance as necessary to fully meet the key practice for programs to test that components and subsystems work together as a system in a controlled setting before finalizing a system's design. (Recommendation 1)

Agency: Department of Homeland Security
Status: Open

Comments: In providing comments on this report DHS concurred with our recommendation and stated that it planned to update its T&E policy to specify that acquisition programs demonstrate that components and subsystems work together before finalizing a system's design. In July 2020, DHS Test and Evaluation Division (TED) officials said they were in the process of updating the policy and that it was undergoing management review with an anticipated completion in fall 2020. Once finalized, GAO will evaluate the revised policy to determine whether DHS has met the intent of this recommendation.

Recommendation: The Secretary for Homeland Security should direct the Director of OTE, in coordination with OCPO, to develop an assessment process-including establishing performance measures-to help ensure T&E training achieves desired results. (Recommendation 2)

Agency: Department of Homeland Security
Status: Open

Comments: In providing comments on this report, DHS concurred with our recommendation and stated that it planned to assess the knowledge and skill requirements for the T&E workforce and establish performance goals for the training. DHS Test and Evaluation Division (TED), in coordination with OCPO, also plan to develop strategies to address any deficiencies with the current training that do not meet the identified requirements. In April 2020, TED officials said that they developed a new survey process to obtain recurring feedback from participants on the training's impact on their ability to perform T&E duties as assigned over time to inform the annual review of the T&E curriculum. However, this effort is still in a piloting stage so the extent to which this information is used to assess the training is still unknown at this time. As of July 2020, TED was still in the process of executing these efforts.

Recommendation: The Secretary for Homeland Security should direct the Director of OTE to revise T&E policy or guidance as necessary to specify when in the acquisition life cycle a major acquisition program manager should designate a level III certified T&E manager. (Recommendation 3)

Agency: Department of Homeland Security
Status: Open

Comments: In providing comments on this report DHS concurred with our recommendation and stated that it planned to update its T&E policy to specify when in the acquisition lifecycle acquisition program managers should designate a qualified T&E manager. In July 2020, DHS Test and Evaluation Division (TED) officials said they were in the process of revising the policy to include this specification and that it was undergoing management review with an anticipated completion in fall 2020. Once finalized, GAO will evaluate the revised policy to determine whether DHS has met the intent of this recommendation.

Recommendation: The Secretary for Homeland Security should direct the Director of OTE to establish an internal control process to ensure that data collected and maintained on major acquisition programs' T&E managers are reliable. (Recommendation 4)

Agency: Department of Homeland Security
Status: Open

Comments: In providing comments on this report, DHS concurred with our recommendation and stated that it planned to establish an internal control process to reliably collect and maintain data on acquisition programs' assigned test and evaluation managers. In April 2020, DHS Test and Evaluation Division (TED) reported taking steps to ensure the validity of this data including establishing points of contacts within each component to cross-check collected information for accuracy and having the Director review collected data on a quarterly basis beginning in third quarter fiscal year 2020. As of July 2020, TED was still in the process of improving its internal collection process, but had not completed these efforts.

Recommendation: The Secretary for Homeland Security should direct the Under Secretary for Science and Technology to assess OTE's workforce to ascertain the extent to which it has the appropriate number of staff with the necessary skills to fulfill its responsibilities. (Recommendation 5)

Agency: Department of Homeland Security
Status: Open

Comments: In providing comments on this report, DHS concurred with our recommendation and stated that it planned to assess the Test and Evaluation Division's (TED) workforce by reviewing current staffing levels and vacancies against the division's roles and responsibilities. The Senior Official Performing the Duties of the Under Secretary for Science and Technology plans to use the results of this review to inform strategic hiring in future years, if needed. In February 2020, DHS released its fiscal year 2020 strategic guidance memorandum for the Science and Technology (S&T) Directorate which included a statement pertaining to resourcing S&T's test and evaluation capabilities. However, as of July 2020, S&T had not yet conducted its review of TED's workforce.

Recommendation: The Director, MDA, should utilize additional schedule margin afforded by the EPAA Phase 3 delay to conduct additional testing necessary to thoroughly assess the capabilities and limitations of Phase 3 against IRBMs and a raid scenario prior to delivery. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense: Missile Defense Agency
Status: Open

Comments: The Department of Defense partially concurred with this recommendation, stating that all EPAA Phase 3 BMDS functions requiring a flight test environment were already successfully demonstrated and that MDA has addressed the intent of our recommendation by adding ground tests to further assess EPAA Phase 3 Capabilities. However, in order for the agency to meet the full intent of our recommendation, additional flight testing to demonstrate capability against EPAA Phase 3 threats, including intermediate-range threats and raid scenarios, is necessary.

Recommendation: The Secretary of Defense should direct the Director, Office of Cost Assessment and Program Evaluation to conduct an independent schedule assessment of the full program schedule for the Global Positioning System's next generation operational control system based on progress made through the end of calendar year 2019. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD did not concur with our recommendation to conduct an independent assessment of the full OCX program schedule based on progress made through the end of calendar year 2019, citing an independent cost and schedule estimate conducted in September 2018 and other ongoing program assessment and monitoring efforts. GAO continues to affirm that the recommendation is necessary given that DOD has not conducted an assessment of the full schedule since June 2018, since which time program risks have evolved. Additionally, ongoing oversight efforts are limited in scope and do not include the developmental test period after product delivery. As of August 2020, the Air Force position remains a non-concur, based on the same rationale previously noted.

Recommendation: The Commissioner of CBP should analyze the costs associated with future barrier segments and include cost as a factor in the Impedance and Denial Prioritization Strategy. (Recommendation 1)

Agency: Department of Homeland Security
Status: Open
Priority recommendation

Comments: In July 2018, we reported on U.S. Customs and Border Protection's (CBP) efforts to construct new physical barriers along the southwest border. We found that the Impedance and Denial Prioritization Strategy--CBP's decision support tool for prioritizing locations for barrier construction projects--did not include analysis of the costs of deploying barriers in each location, which can vary depending on topography, land ownership, and other factors. We recommended that CBP analyze the costs associated with future barrier segments and include cost as a factor in the Impedance and Denial Prioritization Strategy. CBP agreed with this recommendation. CBP officials stated that, after prioritizing locations, CBP conducts detailed cost estimates as part of the acquisitions process. As of January 2020, CBP officials stated that this cost information may affect how the construction projects are executed, but that it would not influence how CBP prioritizes barrier construction projects across various locations. As we have previously reported, organizations should use an integrated approach to the requirements, acquisitions, and budget processes to prioritize needs and allocate resources, so they can optimize return on investment, and maintain program affordability. To fully address our recommendation, Border Patrol needs to incorporate its analysis of the costs of barrier projects into its process for prioritizing locations for construction of barriers.

Recommendation: The Secretary of Defense should direct the F-35 program office to resolve all critical deficiencies before making a full-rate production decision. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with our recommendation and stated that critical deficiencies would be resolved before full-rate production, due in December 2019. In May 2020, we reported that DOD delayed its full-rate production decision from December 2019 to between September 2020 and March 2021.

Recommendation: The Secretary of Defense should direct the F-35 program office to identify what steps are needed to ensure the F-35 meets reliability and maintainability requirements before each variant reaches maturity and update the Reliability and Maintainability Improvement Program with these steps. (Recommendation 2)

Agency: Department of Defense
Status: Open
Priority recommendation

Comments: DOD concurred with our recommendation. According to a DOD official, as of July 2020, the F-35 program office is in the process of revising its plan for improving the F-35's reliability and maintainability.

Recommendation: The Secretary of the Navy should take steps to ensure that the Marine Corps not enter full-rate production until the Marine Corps has determined that the contractor has achieved an MRL of at least 9 for all risk areas. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: DOD partially concurred with this recommendation, stating that it agreed that manufacturing readiness levels (MRL) for the Amphibious Combat Vehicle 1.1 should be assessed prior to the decision to award the option for the second lot of low-rate production, but disagreed that an MRL assessment of any individual risk area, in itself, should delay the contract award. We maintained that achieving an overall MRL-9 by the by the start of full-rate production represents best practice to minimize production risk. According the Marine Corp, the most recent MRL Assessment as of August 2020 assessed the program at an overall MRL-8. The Marine Corps indicated that another MRL assessment is estimated to be completed in November 2020 to support the decision to enter full-rate production, which is planned for December 2020.

Recommendation: In addition to implementing our previous recommendations, to more effectively implement security-related policies and plans, the Commissioner of Internal Revenue should update system and application audit plans based on the current version of referenced policies and guidelines and when significant changes are made to a system or application.

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: During GAO's audit of IRS' FY 2019 financial statements, IRS indicated that it had not yet implemented this recommendation. When the agency indicates that it has implemented this recommendation, we will review its actions.

Recommendation: To help ensure that biosurveillance-related funding is directed to programs that can demonstrate their intended capabilities, and to help ensure sufficient information is known about the current Gen-2 system to make informed cost-benefit decisions about possible upgrades and enhancements to the system, the Secretary of Homeland Security should direct the Assistant Secretary for Health Affairs and other relevant officials within the Department to not pursue upgrades or enhancements to the current BioWatch system until the Office of Health Affairs (OHA): (1) establishes technical performance requirements, including limits of detection, necessary for a biodetection system to meet a clearly defined operational objective for the BioWatch program by detecting attacks of defined types and sizes with specified probabilities; (2) assesses the Gen-2 system against these performance requirements to reliably establish its capabilities; and (3) produces a full accounting of statistical and other uncertainties and limitations in what is known about the system's capability to meet its operational objectives.

Agency: Department of Homeland Security
Status: Open

Comments: DHS told us in June, 2019 that it terminated the Biodetection Technology Enhancements program to enhance BioWatch. Additionally, DHS is transitioning to a new acquisition program called BioDetection 21 intended to replace BioWatch. We will update the status of this recommendation as we obtain more information about DHS's BioDetection 21 acquisition efforts and the extent to which these efforts are consistent with our recommendation.

Recommendation: To help reduce the risk of acquiring immature detection technologies, the Secretary of Homeland Security should direct the Assistant Secretary for Health Affairs, in coordination with the Under Secretary for Science and Technology, to use the best practices outlined in this report to inform test and evaluation actions for any future upgrades or changes to technology for BioWatch.

Agency: Department of Homeland Security
Status: Open

Comments: DHS told us in June, 2019 that it terminated the Biodetection Technology Enhancements program to enhance BioWatch. Additionally, DHS is transitioning to a new acquisition program called BioDetection 21 intended to replace BioWatch. We will update the status of this recommendation as we obtain more information about DHS's BioDetection 21 acquisition efforts and the extent to which these efforts are consistent with our recommendation.