Recommendation: The Commissioner of FDA should, as part of the agency's efforts to update OLS's strategic plan for overseeing agency-wide laboratory safety, resolve agency-wide disagreements on the roles and responsibilities for the centers and OLS in implementing laboratory safety reforms. (Recommendation 1)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commissioner of FDA should, as part of the agency's efforts to update OLS's strategic plan for overseeing agency-wide laboratory safety, address issues of duplication, overlap, and fragmentation within the safety program. (Recommendation 2)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commissioner of FDA should, as part of the agency's efforts to update OLS's strategic plan for overseeing agency-wide laboratory safety, identify how FDA leadership will communicate agency-wide on a sustained basis about the importance of laboratory safety and OLS's role in ensuring successful implementation of laboratory safety reforms. (Recommendation 3)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commissioner of FDA should provide OLS—as FDA's laboratory oversight body—with the necessary oversight authority and access to laboratories to oversee FDA's laboratory safety program and ensure compliance with the agency's laboratory safety policies. (Recommendation 4)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commissioner of FDA should take steps to assess and mitigate any risks to independence posed by funding OLS—as FDA's laboratory safety oversight body—through the working capital fund. In conducting its risk assessment, FDA should specifically focus on ensuring the decision-making processes of the working capital fund supports OLS's independence and ability to hire sufficient staff to implement its laboratory safety oversight priorities. (Recommendation 5)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure that the Under Secretary of Defense for Research and Engineering revise its IR&D instruction to require USD (R&E) personnel to annually review defense industry IR&D investments to inform DOD's science and technology strategic planning efforts. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation stating that USD(R&E) will investigate and revise its IR&D Instruction to require annual review of defense industry IR&D investments.

Recommendation: The Secretary of Defense should ensure that the Administrator, Defense Technical Information Center, assess and determine whether the DTIC IR&D database should require contractors to include additional information on IR&D projects, such as: (a) The IR&D project's linkage, if any, to DOD's modernization priorities; (b) The allowable category (basic research, applied research, technology development, or concept study) to which the IR&D project belongs; (c) The nature of the project as either potentially disruptive or potentially incremental research and development; and (d) The actual IR&D project costs when the project is completed. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with the recommendation stating that the DTIC Administrator will assess whether the DOD IR&D database should require contractors to include additional information on IR&D projects, and make his recommendation to the Director of Defense Research and Engineering for Research and Technology for its decision.

Recommendation: The Commissioner of CBP should ensure that the Biometric Entry-Exit Program's privacy notices contain complete and current information, including all of the locations where facial recognition is used and how travelers can request to opt out as appropriate. (Recommendation 1)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commissioner of CBP should ensure that the Biometric Entry-Exit Program's privacy signage is consistently available at all locations where CBP is using facial recognition. (Recommendation 2)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commissioner of CBP should direct the Biometric Entry-Exit Program to develop and implement a plan to conduct privacy audits of its commercial partners', contractors', and vendors' use of personally identifiable information. (Recommendation 3)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commissioner of CBP should develop and implement a plan to ensure that the biometric air exit capability meets its established photo capture requirement. (Recommendation 4)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commissioner of CBP should develop a process by which Biometric Entry-Exit program officials are alerted when the performance of air exit facial recognition falls below established thresholds. (Recommendation 5)

Agency: Department of Homeland Security: United States Customs and Border Protection
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Director of CDC should take steps to determine participation rates and distribution needed in the AR Option of the National Healthcare Safety Network for conducting regional and national assessments of antibiotic resistance of public health importance. (Recommendation 1)

Agency: Department of Health and Human Services: Public Health Service: Centers for Disease Control and Prevention
Status: Open

Comments: The Department of Health and Human Services concurred with this recommendation. When we confirm any actions the agency has taken to implement the recommendation, we will provide updated information.

Recommendation: The Director of CDC should ensure that CDC's evaluation of its surveillance system for antibiotic-resistant gonorrhea includes measures of its representativeness, such as comparison of the trends in the sample population with those in the overall U.S. population, using specially designed studies if needed. (Recommendation 2)

Agency: Department of Health and Human Services: Public Health Service: Centers for Disease Control and Prevention
Status: Open

Comments: The Department of Health and Human Services and the Centers for Disease Control and Prevention concurred with this recommendation. When we confirm any actions the agency has taken to implement the recommendation, we will provide updated information.

Recommendation: The Director of CDC should provide information on uncertainties for antibiotic resistance estimates in its consolidated Threats Reports, including standard errors or confidence intervals, as appropriate. (Recommendation 3)

Agency: Department of Health and Human Services: Public Health Service: Centers for Disease Control and Prevention
Status: Open

Comments: The Department of Health and Human Services generally concurred with this recommendation. When we confirm any actions the agency has taken to implement the recommendation, we will provide updated information.

Recommendation: The Director of CDC should develop a plan for timely, consolidated reports of antibiotic resistance in priority pathogens at regular intervals. (Recommendation 4)

Agency: Department of Health and Human Services: Public Health Service: Centers for Disease Control and Prevention
Status: Open

Comments: The Department of Health and Human services concurred with this recommendation. When we confirm any actions the agency has taken to implement the recommendation, we will provide updated information.

Recommendation: The Secretary of HHS should identify leadership and clarify roles and responsibilities among HHS agencies to assess the clinical outcomes of diagnostic testing for identifying antibiotic-resistant bacteria. (Recommendation 5)

Agency: Department of Health and Human Services: Office of the Secretary
Status: Open

Comments: The Department of Health and Human Services concurred with GAO's recommendation. When we confirm any actions the agency has taken to implement the recommendation, we will provide updated information.

Recommendation: The Commissioner of FDA should direct the Center for Devices and Radiological Health to conduct additional monitoring and evaluation of the status of FDA-authorized tests that rely on breakpoints, on a regular basis, to determine whether test manufacturers are updating breakpoints, seeking additional resources as needed. (Recommendation 6)

Agency: Department of Health and Human Services: Public Health Service: Food and Drug Administration: Office of the Commissioner
Status: Open

Comments: The Department of Health and Human Services and the Food and Drug Administration concurred with this recommendation. When we confirm any actions the agency has taken to implement the recommendation, we will provide updated information.

Recommendation: The Secretary of HHS should develop a strategic framework to further incentivize the development of new treatments for antibiotic-resistant infections, including through the use of postmarket financial incentives, and, if appropriate, make recommendations to Congress for necessary authority. (Recommendation 7)

Agency: Department of Health and Human Services: Office of the Secretary
Status: Open

Comments: The Department of Health and Human Services did not concur with GAO's recommendation. In commenting on our report, the department noted that it has convened a workgroup to develop a strategic framework that includes proposals to address a variety of challenges facing antibiotic product developers and agreed that additional incentives are needed. However, the department stated it is still analyzing whether postmarket financial incentives should be included in this framework. We believe our recommendation is still warranted, given the importance of antibiotic resistance to public health and the importance of sustaining the antibiotic pipeline, including after antibiotics are brought to market. When we confirm any actions the agency has taken to implement the recommendation, we will provide updated information.

Recommendation: The Secretary of HHS should direct the CARB Task Force to include in its annual updates to the President plans for addressing any barriers preventing full implementation of the National Action Plan and, as appropriate, make recommendations for new or modified actions. Specifically, the CARB Task Force should identify plans to address barriers, such as those related to expanding (1) a CDC program designed to strengthen the U.S. response to resistant gonorrhea; (2) antibiotic stewardship programs across health care settings; and (3) antibiotic use data collection across health care settings, to the extent feasible. (Recommendation 8)

Agency: Department of Health and Human Services: Office of the Secretary
Status: Open

Comments: The Department of Health and Human Services concurred with this recommendation, and stated that beginning in 2020 and continuing annually thereafter, the CARB Task Force's progress reports will include discussion of any barriers preventing full implementation of the National Action Plan, including, as appropriate, barriers that GAO has identified. When we confirm any actions the agency has taken to implement the recommendation, we will provide updated information.

Recommendation: The Secretary of the Department of Agriculture should direct the Assistant Secretary for Civil Rights to publicize promising practices for Title IX compliance on its websites for their university grantees. (Recommendation 1)

Agency: Department of Agriculture
Status: Open

Comments: USDA agreed with the recommendation. In its March 2020 formal comment letter, the department stated that it will ensure promising practices information is posted to websites for the Office of the Assistant Secretary for Civil Rights (OASCR) and the National Institute of Food and Agriculture. Additionally, a notice will be sent to universities to ensure they are aware the information is available, along with specifics regarding access to the same.

Recommendation: The Secretary of Energy should direct the Director of the Office of Economic Impact and Diversity to publicize promising practices for Title IX compliance on its websites for their university grantees. (Recommendation 2)

Agency: Department of Energy: Office of the Secretary
Status: Open

Comments: DOE agreed with the recommendation. In its February 2020 formal comment letter, the department stated that the Office of Civil Rights and Diversity (OCRD) has identified promising practices on an ongoing basis in its Title IX compliance review reports. OCRD will develop and publicize a promising practices guide on its website for reference by university grantees. The estimated completion date is June 1, 2020. GAO will review this action once it receives documentation from DOE.

Recommendation: The Secretary of the Department of Health and Human Services should direct the Director for the Office for Civil Rights to publicize a stand-alone list of promising practices for Title IX compliance on its websites for their university grantees. (Recommendation 3)

Agency: Department of Health and Human Services: Office of the Secretary
Status: Open

Comments: HHS agreed with the recommendation. In its February 2020 formal comment letter, the department stated the Office for Civil Rights will incorporate effective practices from materials posted on its website and external sources in a stand-alone list of effective practices to help universities comply with Title IX. The Office for Civil Rights will also draw from its Title IX enforcement work to update the effective practices list on an ongoing basis.

Recommendation: The Secretary of Energy should direct the Director of the Office of Economic Impact and Diversity to finalize and publish Title IX complaint procedures, consistent with DOJ's regulations. (Recommendation 4)

Agency: Department of Energy: Office of the Secretary
Status: Open

Comments: DOE agreed with the recommendation. In its February 2020 formal comment letter, the department stated that the Office of Civil Rights and Diversity (OCRD) has developed a draft outline of complaint procedures. OCRD is benchmarking these procedures against NASA and NSF external civil rights complaint procedures. OCRD will finalize and publish Title IX complaint procedures, consistent with the Department of Justice's regulations. The estimated completion date is October 31, 2020.

Recommendation: The Secretary of the Department of Agriculture should direct the Assistant Secretary for Civil Rights to finalize and publish revised Title IX complaint procedures. (Recommendation 5)

Agency: Department of Agriculture
Status: Open

Comments: USDA agreed with the recommendation to finalize and publish Title IX complaint procedures. According to the department's March 2020 formal comment letter, USDA's Departmental Regulation (DR) 4330-002, Nondiscrimination in Program and Activities Receiving Federal Financial Assistance from USDA, addresses processing Title IX administrative complaints filed with the Department in any program or activity receiving financial assistance from USDA. The Departmental Regulation (DR) 4330-002 was revised in 2019 and is currently in the clearance process for publication.

Recommendation: The Secretary of the Department of Agriculture should direct the Assistant Secretary for Civil Rights to clarify on its website that individuals on USDA-funded grants can file Title IX complaints through the Assistant Secretary for Civil Rights—including clarifying who is considered "customers." (Recommendation 6)

Agency: Department of Agriculture
Status: Open

Comments: USDA agreed with this recommendation. According to the department's March 2020 formal comment letter, USDA's Office of the Assistant Secretary for Civil Rights (OASCR) will add language on its website to make clear recipients of USDA-funded grants are included in the definition of a customer and, as such, may file Title IX complaints through OASCR. Additionally, the requested information will be added to the Frequently Asked Questions section on OASCR' s website.

Recommendation: The Secretary of the Department of Health and Human Services should direct the Director for the Office for Civil Rights to assess the feasibility of receiving and reviewing concerns of sex discrimination—including sexual harassment—and communicating to individuals on agency-funded grants the option to notify the agency of these concerns, outside of the Title IX complaint process. (Recommendation 7)

Agency: Department of Health and Human Services: Office of the Secretary
Status: Open

Comments: HHS agreed with the recommendation. In its February 2020 formal comment letter, the department stated that outside of the Title IX enforcement process, the Office for Civil Rights updated its sex discrimination web pages to inform persons on how to file complaints of discrimination directly to the Office for Civil Rights, including complaints of sexual harassment. The Office for Civil Rights has also incorporated information about Title IX rights and obligations in outreach material used on university campuses, including information addressing sexual harassment.

Recommendation: The Secretary of Energy should direct the Director of the Office of Economic Impact and Diversity to assess the feasibility of receiving and reviewing concerns of sex discrimination—including sexual harassment—and communicating to individuals on agency-funded grants the option to notify the agency of these concerns, outside of the Title IX complaint process. (Recommendation 8)

Agency: Department of Energy: Office of the Secretary
Status: Open

Comments: DOE agreed with the recommendation. In its February 2020 formal comment letter, the department stated that the Office of Civil Rights and Diversity (OCRD) will evaluate the feasibility of receiving and reviewing concerns of discrimination, including sexual harassment, outside of the Title IX complaint process described in DOE's Title IX regulations, as well as communicating this option to individuals on DOE funded grants. The estimated completion date is December 31, 2020.

Recommendation: The Administrator of NASA should assess the feasibility of receiving and reviewing concerns of sex discrimination—including sexual harassment—and communicating to individuals on agency-funded grants the option to notify the agency of these concerns, outside of the Title IX complaint process. (Recommendation 9)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with the recommendation. According to the agency's February 2020 formal comment letter, NASA plans to assess the feasibility of receiving and reviewing concerns of sex discrimination and harassment and communicating to individuals on agency-funded grants the option to notify the agency of these concerns. In conducting the feasibility assessment, NASA will benchmark with agencies that currently have the capability to receive and review concerns of sex discrimination, including sexual harassment. NASA will also examine internal resources to ensure successful implementation. NASA estimates completion by September 20, 2020.

Recommendation: The Secretary of the Department of Agriculture should direct the Assistant Secretary for Civil Rights to assess the feasibility of receiving and reviewing concerns of sex discrimination—including sexual harassment—and communicating to individuals on agency-funded grants the option to notify the agency of these concerns, outside of the Title IX complaint process. (Recommendation 10)

Agency: Department of Agriculture
Status: Open

Comments: USDA agreed with this recommendation. According to the department's March 2020 formal comment letter, USDA is to direct the Office of the Assistant Secretary for Civil Rights (OASCR) to assess the feasibility of receiving and reviewing concerns of sex discrimination (including sexual harassment) and including language in agency-funded grants on the option to notify the agency of harassment issues outside of the Title IX complaint process. The Office of the Mission Area Liaison within OASCR will assess the feasibility of creating a tracking system to capture inquiries and concerns outside of the Title IX complaint process. If feasible, the system will serve as a risk assessment tool to identify issues, potential violations, trends, risks, and areas prime for focused compliance review, according to USDA's formal comment letter.

Recommendation: The Secretary of the Department of Health and Human Services should direct the Director for the Office for Civil Rights and the Director of NIH to develop and implement formal procedures for sharing relevant information about Title IX (compliance reviews, violations, and complaints) and sex discrimination concerns, including sexual harassment. For example, HHS components should internally share information regarding findings of Title IX violations, concerns of sex discrimination, including sexual harassment, and Title IX compliance review reports. (Recommendation 11)

Agency: Department of Health and Human Services: Office of the Secretary
Status: Open

Comments: HHS agreed with the recommendation. In its February 2020 formal comment letter, the department stated that the Office for Civil Rights and NIH will review the current procedure for sharing information, and development and implement (as necessary) formal procedures for sharing relevant information about Title IX and sexual harassment concerns. Furthermore, an action plan to address the recommendation will be provided in the department's 180-day letter response to Congress.

Recommendation: The Secretary of Energy should establish goals and an overall plan to assess all of the agency's sexual harassment prevention efforts for their university grantees, including methods to regularly monitor and evaluate its sexual harassment prevention policies and communication mechanisms (e.g. Title IX or sex discrimination websites). (Recommendation 12)

Agency: Department of Energy: Office of the Secretary
Status: Open

Comments: DOE agreed with the recommendation. In its February 2020 formal comment letter, the department stated that the Office of Civil Rights and Diversity will conduct research into best practices for harassment prevention efforts for university grantees and benchmark with other federal agencies. The estimated completion date for establishment of goals is January 31, 2021 and the estimated completion date for establishment of an overall plan is August 31, 2021.

Recommendation: The Secretary of the Department of Health and Human Services should establish goals and an overall plan to assess all of the agency's sexual harassment prevention efforts for their university grantees, including methods to regularly monitor and evaluate its sexual harassment prevention policies and communication mechanisms (e.g. Title IX or sex discrimination websites). (Recommendation 13)

Agency: Department of Health and Human Services: Office of the Secretary
Status: Open

Comments: HHS agreed with the recommendation. In its February 2020 formal comment letter, the department stated that NIH has created goals for all NIH grantee sexual harassment prevention efforts, as noted in GAO's report. NIH will develop a plan to assess progress towards achieving those goals, including methods to regularly monitor and evaluate policies and communication methods. To the extent Title IX enforcement and outreach efforts may prevent sexual harassment, the Office for Civil Rights will develop a plan for leveraging enforcement efforts and outreach communications to help HHS grantees implement prevention efforts. HHS will submit an action plan that fully address department efforts regarding this recommendation in its 180-day letter response to Congress.

Recommendation: The Administrator of NASA should establish goals and an overall plan to assess all of the agency's sexual harassment prevention efforts for their university grantees, including methods to regularly monitor and evaluate its sexual harassment prevention policies and communication mechanisms (e.g. Title IX or sex discrimination websites). (Recommendation 14)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA agreed with the recommendation. According to the agency's February 2020 formal comment letter, NASA accepts its responsibility to establish policies and communication mechanisms to help university grantees to prevent sexual harassment, and the agency is committed to ongoing monitoring and evaluation of these efforts. At the same time, NASA judges that the most effective way to discharge these responsibilities is to adopt the approach now being developed by the National Science and Technology Council. NASA will establish goals, plans, and methods pursuant to this recommendation by adopting the policies and approaches developed through the interagency process. The estimated completion is to be determined based on timelines and milestones established by the Office of Science and Technology Policy's Safe and Inclusive Research Environments Subcommittee Subcommittee.

Recommendation: The Director of NSF should establish goals and an overall plan to assess all of the agency's sexual harassment prevention efforts for their university grantees, including methods to regularly monitor and evaluate its sexual harassment prevention policies and communication mechanisms (e.g. Title IX or sex discrimination websites). (Recommendation 15)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with the recommendation. According to the agency's February 2020 formal comment letter, NSF is embarking on an assessment process to improve its policies and practices continually in order to achieve the goal of safe and inclusive research environments.

Recommendation: The Secretary of the Department of Agriculture should establish goals and an overall plan to assess all of the agency's sexual harassment prevention efforts for their university grantees, including methods to regularly monitor and evaluate its sexual harassment prevention policies and communication mechanisms (e.g. Title IX or sex discrimination websites). (Recommendation 16)

Agency: Department of Agriculture
Status: Open

Comments: USDA agreed with this recommendation. According to the department's March 2020 formal comment letter, USDA will consider developing language to include in university grantees' terms and conditions regarding allegations of sexual harassment. Additionally, the Office of the Assistant Secretary for Civil Rights (OASCR) will reach out to other agencies within the Title IX Quarterly Working Group to assess best practices for monitoring and evaluating sexual harassment prevention policies and communication mechanisms. Finally, USDA will increase its communication with university grantees and increase oversight through compliance reviews.

Recommendation: In consultation with DOE, HHS, NASA, NSF, and USDA, the Assistant Attorney General for the Department of Justice should direct the responsible Civil Rights Division sections to fully adopt two federal interagency leading practices—agree on agency's roles and responsibilities and develop mechanisms to monitor, evaluate, and report results of collaborative efforts, for its Quarterly Title IX STEM discussion group. (Recommendation 17)

Agency: Department of Justice: Office of the Assistant Attorney General for Administration
Status: Open

Comments: DOJ generally agreed with the recommendation. According to the department's February 2020 formal comment letter, the Civil Rights Division (CRT) is prepared to delineate agencies' roles and responsibilities within the Quarterly Title IX STEM Discussion Group and is willing to develop an enhanced process for evaluating, monitoring, and reporting on the group's collaborations in enforcing Title IX that is achievable within DOJ's current resource allocation, or if more resources become available.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to take steps to ensure that the details of the pilot program's data protections are incorporated into the existing agreements. (Recommendation 1)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will require the Primary Sponsor for each applicable federally funded research and development center (FFRDC) to provide documentation that shows that the details of the pilot program's data protections have been incorporated into existing sponsoring agreements and contracts, and estimated these actions would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to take steps to ensure that the FFRDCs and sponsors are implementing the pilot program's protections for sensitive data. (Recommendation 2)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will task the Primary Sponsor for each applicable federally funded research and development center (FFRDC) to provide a report detailing the FFRDC's plan for implementing the pilot program's protections for sensitive data and confirming that the FFRDC is adhering to that plan. OUSD(R&E)/L&PO estimated these actions would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to establish a monitoring and oversight mechanism to ensure that primary sponsors submit complete information on pilot projects, as required by DOD's guidance for the pilot program. (Recommendation 3)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will develop a monitoring and oversight plan, and estimated it would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan that outlines the methodology by which DOD will assess the pilot and how and when information collected will be analyzed to evaluate the pilot program. (Recommendation 4)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will develop and coordinate a plan that outlines the methodology by which DOD will assess the pilot as well as the methodology for collecting and analyzing information to evaluate the pilot program. OUSD(R&E)/L&PO estimated these actions would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan to identify and evaluate lessons learned from the pilot program. (Recommendation 5)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will develop a plan to identify and evaluate lessons learned from the pilot program, and estimated the plan would be completed in early Fall 2020.

Recommendation: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan for obtaining input from stakeholders on the pilot program. (Recommendation 6)

Agency: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering
Status: Open

Comments: The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it will develop a plan to obtain stakeholder inputs for use in evaluating the pilot program, and estimated the plan would be completed in early Fall 2020.

Recommendation: The Secretary of Defense should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense partially concurred with this recommendation noting the challenge with balancing ensuring public access to research data with considerations of national security and personally identifiable information. As discussed in our report, balancing these considerations is a challenge that agency officials and stakeholders identified during our work. Accordingly, our recommendation to DOD regarding findability and accessibility of agency-funded research data was qualified to pertain to appropriate agency-funded research data--recognizing that it might not be appropriate to make certain datasets publically available because of national security or other concerns. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Education should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 2)

Agency: Department of Education
Status: Open

Comments: The Department of Education concurred with this recommendation. According to its response to our report, the Department awarded a contract to support enhancements to its Education Resources Information Center (ERIC) to link scholarly research publications supported by the Department to its publicly accessible datasets. The Department indicated it expects to complete this work by September 30, 2020. When we confirm what actions the Department has taken to implement this recommendation we will provide additional information.

Recommendation: The Director of the Agency for Healthcare Research and Quality should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 3)

Agency: Department of Health and Human Services: Public Health Service: Agency for Healthcare Research and Quality
Status: Open

Comments: The Agency for Healthcare Research and Quality concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Commissioner of the Food and Drug Administration should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 4)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: The Food and Drug Administration concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Homeland Security should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 5)

Agency: Department of Homeland Security
Status: Open

Comments: The Department of Homeland Security concurred with this recommendation and noted that it was in the process of establishing a portal on its website to increase public access to agency-funded research. The department estimated these efforts would be completed by June 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Veterans Affairs should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 6)

Agency: Department of Veterans Affairs
Status: Open

Comments: The Department of Veterans Affairs concurred with this recommendation but indicated in its comments on the report that had already taken steps to implement it. As discussed in the report, the department's efforts to ensure research data availability pertained to a portion of the agency's federally-funded research data. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Director of the National Science Foundation should fully implement plans to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 7)

Agency: National Science Foundation
Status: Open

Comments: NSF concurred with this recommendation. According to its response to our report, NSF is expanding its public access repository to include metadata records about data that support publications resulting from NSF-funded research. NSF's response stated that, by storing metadata records for supporting datasets alongside metadata records for publications, the public will be able to more easily find and access appropriate agency funded research data. We will provide an update when we obtain additional information about the status of implementing this recommendation.

Recommendation: The Secretary of Defense should complete development of data management plan requirements for extramural researchers. (Recommendation 8)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense concurred with this recommendation noting planned steps to complete development of data management plan requirements. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Director of the Agency for Healthcare Research and Quality should complete development of data management plan requirements. (Recommendation 9)

Agency: Department of Health and Human Services: Public Health Service: Agency for Healthcare Research and Quality
Status: Open

Comments: The Agency for Healthcare Research and Quality concurred with this recommendation. When we confirm what actions the agency has taken to implement the recommendation we will provide updated information.

Recommendation: The Secretary of Homeland Security should complete development of data management plan requirements. (Recommendation 10)

Agency: Department of Homeland Security
Status: Open

Comments: The Department of Homeland Security concurred with this recommendation, stating that it would develop a departmentwide management directive for research and development data as well as data management plan guidance and a template to document requirements. The department estimated that these efforts would be completed by June 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The U.S. Agency for International Development Administrator should complete development of data management plan requirements for extramural researchers. (Recommendation 11)

Agency: United States Agency for International Development
Status: Open

Comments: The U.S. Agency for International Development agreed with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Agriculture should complete development of guidance and provide training to agency officials or others involved in reviewing the merits of researchers' data management plans. (Recommendation 12)

Agency: Department of Agriculture
Status: Open

Comments: The U.S. Department of Agriculture agreed with the findings of our report. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The U.S. Agency for International Development Administrator should complete development of and provide training for agency officials or others involved in reviewing the merits of researchers' data management plans. (Recommendation 13)

Agency: United States Agency for International Development
Status: Open

Comments: The U.S. Agency for International Development agreed with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Commissioner of the Food and Drug Administration should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 14)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: The Food and Drug Administration concurred with this recommendation. When we confirm what actions the agency has taken to implement the recommendation we will provide updated information.

Recommendation: The Secretary of Homeland Security should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 15)

Agency: Department of Homeland Security
Status: Open

Comments: The Department of Homeland Security concurred with this recommendation, and indicated it would evaluate training needs for data management plan reviews and develop plans to fulfill any additional training needs identified. The department estimated these efforts would be completed by September 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Veterans Affairs should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 16)

Agency: Department of Veterans Affairs
Status: Open

Comments: The Department of Veterans Affairs concurred with this recommendation and identified several planned steps to identify and meet training needs for those involved in reviewing researchers' data management plans. The department estimated these efforts would be completed by September 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Director of the Agency for Healthcare Research and Quality should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 17)

Agency: Department of Health and Human Services: Public Health Service: Agency for Healthcare Research and Quality
Status: Open

Comments: The Agency for Healthcare Research and Quality concurred with this recommendation stating that it would assess training needs for agency officials and others involved in reviewing the merits of researchers' data management plans and would develop and provide additional training if warranted. When we confirm what actions the agency has taken to implement the recommendation we will provide updated information.

Recommendation: The Secretary of Defense should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 18)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Energy should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 19)

Agency: Department of Energy
Status: Open

Comments: The Department of Energy concurred with this recommendation and stated it would assess and develop a plan to meet the training needs of internal DOE staff and external peer reviewers of data management plans. DOE estimated these efforts would be completed by December 31, 2020. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Environmental Protection Agency Administrator should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 20)

Agency: Environmental Protection Agency
Status: Open

Comments: The Environmental Protection Agency agreed with this recommendation stating that, during fiscal year 2020, the agency will evaluate training needs for agency officials who review researchers' data management plans. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Agriculture should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 21)

Agency: Department of Agriculture
Status: Open

Comments: The U.S. Department of Agriculture agreed with the findings of our report. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 22)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense concurred with this recommendation When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Director of the Agency for Healthcare Research and Quality should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements.(Recommendation 23)

Agency: Department of Health and Human Services: Public Health Service: Agency for Healthcare Research and Quality
Status: Open

Comments: The Agency for Healthcare Research and Quality concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Commissioner of the Food and Drug Administration should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 24)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: The Food and Drug Administration concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Director of the National Institutes of Health should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 25)

Agency: Department of Health and Human Services: Public Health Service: National Institutes of Health
Status: Open

Comments: The National Institutes of Health concurred with this recommendation but stated that the agency already had mechanisms in place to ensure compliance with public access plan and associated requirements for publications and data. As discussed in the report, we believe our recommendation, as worded, appropriately reflected the extent to which the agency had implemented researcher compliance mechanisms at the time of issuance of our report. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Homeland Security should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 26)

Agency: Department of Homeland Security
Status: Open

Comments: The Department of Homeland Security concurred with this recommendation and stated it would develop a mechanism to ensure researcher compliance with the department's public access plan and data management plan requirements. The department estimated these efforts would be completed by September 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The National Oceanic and Atmospheric Administration Administrator should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 27)

Agency: Department of Commerce: National Oceanic and Atmospheric Administration
Status: Open

Comments: The Department of Commerce agreed with this recommendation and the National Oceanic and Atmospheric Administration indicated it was pursuing multiple mechanisms to implement it. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Energy should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 28)

Agency: Department of Energy
Status: Open

Comments: The Department of Energy concurred with this recommendation stating that it would develop a compliance mechanism to identify researchers receiving funding from DOE financial assistance awards who are not compliant with DOE's public access plan for publications. DOE estimated these efforts would be completed by December 31, 2020. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Veterans Affairs should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 29)

Agency: Department of Veterans Affairs
Status: Open

Comments: The Department of Veterans Affairs concurred with this recommendation but indicated in its comments on the report that had already taken steps to implement it. As discussed in the report, the department's public access compliance mechanism covered a portion of the agency's federally-funded research data.. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Transportation should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 30)

Agency: Department of Transportation
Status: Open

Comments: The Department of Transportation concurred with this recommendation. According to its response to our report, the Department will build upon existing compliance mechanisms to ensure researcher compliance with its public access plan and associated requirements. As part of this process, the Department reported that it plans to update its public access plan, and expects to complete these actions by December 31, 2020. When we confirm what actions the Department has taken to implement this recommendation we will provide additional information.

Recommendation: The National Institute of Standards and Technology Director should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 31)

Agency: Department of Commerce: National Institute of Standards and Technology
Status: Open

Comments: The Department of Commerce agreed with this recommendation and the National Institute of Standards and Technology identified several planned steps to implement it with full completion anticipated by December 31, 2020. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: As the Subcommittee on Open Science moves forward, the Office of Science and Technology Policy co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 32)

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: OSTP disagreed with GAO's November 2019 recommendation, stating that the subcommittee had already taken steps to implement the leading practices GAO identified. However, OSTP officials did not provide documentation of these efforts and GAO continued to believe the recommendation was warranted. In March 2020, OSTP provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the Department of Defense co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 33)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with GAO's November 2019 recommendation, noting that DOD participates in subcommittee initiatives, including a working group on disclosure risk management, which is a topic of great importance to DOD. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the Department of Energy co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 34)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with GAO's November 2019 recommendation. DOE noted that, as a co-chair of the subcommittee, it is actively identifying areas of collaboration across agencies in implementing open science practices. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the National Institutes of Health co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 35)

Agency: Department of Health and Human Services: Public Health Service: National Institutes of Health
Status: Open

Comments: NIH concurred with GAO's November 2019 recommendation. NIH stated that the subcommittee and its working groups are actively coordinating and building consensus on issues and processes to implement leading practices that enhance and sustain collaboration across federal agencies. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the National Oceanic and Atmospheric Administration co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 36)

Agency: Department of Commerce: National Oceanic and Atmospheric Administration
Status: Open

Comments: NOAA concurred with GAO's November 2019 recommendation, noting that NOAA would work with the subcommittee to identify more opportunities for collaboration to promote access to research results. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the National Science Foundation co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 37)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation concurred with GAO's November 2019 recommendation. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: The Secretary of the Army should direct the Commanding General of Army Futures Command, in coordination with relevant Army entities, to establish command-wide performance measures and develop a plan to use these measures to systematically assess the effectiveness of small business engagement in support of research and development. (Recommendation 3)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: In fiscal year 2020, AFC developed both short-term and long-term performance measures to capture and monitor information on small business engagements across the command, but are still developing a database to systematically track the progression of engagements and outcomes. Officials expect to establish a database by fiscal year 2021.

Recommendation: The Secretary of Commerce should direct the National Institute of Standards and Technology (NIST) Director to work with other sponsoring federal agencies to develop and implement network-wide performance goals for the Manufacturing USA program with measurable targets and time frames. (Recommendation 1)

Agency: Department of Commerce
Status: Open

Comments: Commerce partially concurred with this recommendation according to its response to our report. Commerce stated that it lacks the legal authority to compel action by other federal agencies, and that legal constraints aside, it believes it is bad management practice to ask institutes to respond to performance goals issued by different bodies. As of March 2020, Commerce stated that it plans to report on performance goals for Commerce-sponsored institutes effective with the 2019 annual report, which it expects to issue by September 2020. Commerce also agreed to continue working closely with other sponsoring federal agencies on program and network goals for the Manufacturing USA institutes, but did not commit to working with sponsoring federal agencies to develop and implement network-wide performance goals with measurable targets and time frames. We recognize that Commerce does not have management authority over other the institutes sponsored by other agencies. We believe our report sufficiently characterizes the development of network-wide performance goals, targets, and time frames as a collaborative effort between Commerce and sponsoring agencies that is in keeping with Commerce's network-wide coordination functions under the RAMI Act. Moreover, our recommendation specifically pertained to developing performance goals for the Manufacturing USA program, not individual institutes. As we stated in our report, this would not necessarily entail new performance measures but, rather, could consist of measurable near-term performance goals corresponding to program performance measures already in place. Further, as stated in our report, GAO's prior work has shown that systems of performance measures benefit from certain key practices, such as creating a hierarchy that breaks down broad, long-term goals and objectives into more specific, near-term performance goals with measurable targets and time frames. Our recommendation was designed to ensure that the Manufacturing USA program performance measurement structure that Commerce has already worked with the other sponsoring agencies to develop more fully aligns with these key practices. We continue to believe that by working with other sponsoring federal agencies to develop and implement network-wide performance goals with targets and time frames, Commerce would be better able to observe and report on progress toward long-term Manufacturing USA program goals and objectives.

Recommendation: The Secretary of Commerce should direct the NIST Director to work with other sponsoring federal agencies to ensure that the Manufacturing USA network-wide performance measures are directly aligned with the network-wide performance goals, the Manufacturing USA strategic objectives and program goals, and the statutory purposes of the Revitalize American Manufacturing and Innovation (RAMI) Act. (Recommendation 2)

Agency: Department of Commerce
Status: Open

Comments: Commerce partially concurred with this recommendation according to its response to our report. In March 2020, Commerce stated that it supports the alignment of performance measures with performance goals only for Commerce-sponsored institutes. Commerce stated that it is unable to commit to this recommendation as 13 of the 14 existing institutes were authorized under authorities other than the RAMI Act and are sponsored by agencies other than Commerce. Commerce also stated that, until additional institutes authorized by the RAMI Act are in place, it does not support additional performance measures for the single Commerce-sponsored institute beyond the RAMI Act requirements, as doing so would impose an unfair level of scrutiny. Commerce agreed to report on performance metrics for department-sponsored institutes effective with the current annual report, which is expected by September 2020. We recognize that Commerce does not have management authority over other the institutes sponsored by other agencies. We believe our report sufficiently characterizes the effort to align the network-wide performance measures with network-wide performance goals and Manufacturing USA program goals as a collaborative effort between Commerce and sponsoring agencies that is in keeping with Commerce's coordination functions under the RAMI Act. Our recommendation does not ask Commerce to compel actions by other agencies, nor to develop any additional performance measures. As noted in our report, the Manufacturing USA program's performance measurement structure aligns near-term performance measures directly to the program's long-term goals. This structure bypasses connecting the performance measures with the program's objectives that have been developed to break down the long-term goals more specifically. GAO's prior work has shown that systems of performance measures benefit from certain key practices, such as creating a hierarchy that breaks down broad, long-term goals and objectives into more specific, near-term performance goals with measurable targets and time frames. Our recommendation was designed to ensure that the Manufacturing USA program performance measurement structure that Commerce has already worked with the other sponsoring agencies to develop more fully aligns with these key practices. We continue to believe that by working with other sponsoring federal agencies to ensure that the Manufacturing USA network-wide performance measures are directly aligned with the Manufacturing USA strategic program goals and objectives and the statutory purposes of the RAMI Act, Commerce would be better able to observe and report on progress made toward achieving the statutory purposes of the Manufacturing USA program.

Recommendation: The Secretary of Commerce should direct the NIST Director to develop criteria to evaluate whether the Commerce-sponsored institute can sustain its operations without additional federal financial assistance after its initial agreement. If an analysis based on such criteria indicates that additional federal financial assistance is needed to help the institute sustain its operations, then the Secretary of Commerce should consider a legislative proposal to amend relevant provisions of the RAMI Act. (Recommendation 3)

Agency: Department of Commerce
Status: Open

Comments: Commerce partially concurred with this recommendation according to its response to our report. In March 2020, Commerce stated that it will develop criteria to evaluate the sufficiency of the Commerce-sponsored institute's sustainability plan based on the anticipated operating costs of the institute at fully operational steady state, and the likelihood of sustaining those operations through the specific efforts outlined in the sustainability plan. Commerce stated that it plans to develop the evaluation criteria by April 2020. We will update this recommendation after we learn more about these efforts.

Recommendation: The Secretary of Defense should direct the Director of DOD's Manufacturing USA institutes to develop criteria to evaluate whether DOD-sponsored institutes can sustain their operations without additional federal financial assistance after their initial agreements. (Recommendation 4)

Agency: Department of Defense
Status: Open

Comments: DOD partially concurred with this recommendation. According to information DOD provided in March 2020, DOD developed criteria to evaluate whether each DOD-sponsored institute is effectively executing its mission, providing value to the department, and transitioning advanced manufacturing to U.S. manufacturers, while demonstrating progress toward business viability (diversified revenue, controlled costs, etc.). As of March 2020, DOD plans to incorporate these criteria into a strategic management plan, under which the department will review institutes' progress at the end of their agreements to determine the type and level of DOD's continued participation. DOD expects to complete work on the strategic management plan by the end of September 2020.

Recommendation: The Secretary of Energy should direct the Director of DOE's Manufacturing USA institutes to develop criteria to evaluate whether DOE-sponsored institutes can sustain their operations without additional federal financial assistance after their initial agreements. (Recommendation 5)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with this recommendation according to its response to our report. In September 2019, DOE stated that it will direct the Directors of DOE-sponsored institutes to collectively work toward updating institute sustainability plans and activities and collaboratively develop criteria and metrics to assess the institutes' progress toward financial sustainability. After development of the metrics, DOE will track as appropriate. As of March 2020, DOE reported agreement with its institutes on an initial set of criteria and metrics to assess progress toward financial sustainability. We will update this recommendation as we collect more information about these efforts.

Recommendation: The Secretary of Energy should take action to educate and communicate the agency's scientific integrity polices to staff through, for example, regular training. (Recommendation 1)

Agency: Department of Energy
Status: Open

Comments: In July 2020, DOE provided an update on the status of this recommendation. In that update, DOE said the scientific integrity official will be responsible for leading and coordinating with other elements of the Department, in the development of measures to educate and communicate DOE's scientific integrity policies to staff.

Recommendation: The Secretary of Energy should establish steps and a time frame for designating a scientific integrity official to oversee the department's scientific integrity activities. (Recommendation 3)

Agency: Department of Energy
Status: Open

Comments: In July 2020, DOE provided an update on the status of this recommendation. In that update, DOE said that it was still in the process of identifying an individual to serve as a scientific integrity official.

Recommendation: The Secretary of Energy should develop mechanisms to regularly monitor and evaluate implementation of the department's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 4)

Agency: Department of Energy
Status: Open

Comments: In July 2020, DOE provided an update on the status of this recommendation. In that update, DOE stated that the Department's scientific integrity official will have the responsibility to lead and coordinate with other elements of the Department in developing procedures to monitor and evaluate the implementation of DOE's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary.

Recommendation: The Secretary of Transportation should develop mechanisms to regularly monitor and evaluate implementation of the department's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 5)

Agency: Department of Transportation
Status: Open

Comments: In September 2020, the Department of Transportation provided an update on the status of this recommendation. In that update, officials said that DOT was still working on this action. According to officials, the department will implement several mechanisms to address the recommendation, including conducting annual reviews of the scientific integrity policy and making the policy available to all relevant employees. DOT estimated it would complete these actions by the end of March 2021.

Recommendation: The Director of NIST should develop mechanisms to regularly monitor and evaluate implementation of the agency's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 6)

Agency: Department of Commerce: National Institute of Standards and Technology
Status: Open

Comments: In Commerce's written comments, NIST had stated that, beginning in fiscal year 2019, the agency would review implementation of its policy at least annually and make recommendations to the Director of NIST as to whether any improvements were needed. In a September 2020 update, NIST provided a memorandum reporting that in fiscal year 2019 the Scientific Integrity Officer and General Counsel had discussed an allegation related to scientific integrity, and that it was determined that programmatic factors, not a lapse in scientific integrity, had occurred. According to NIST, it monitors the effectiveness of the implementation of its documented scientific integrity policy, order, and procedure by means of informal feedback from NIST researchers and supervisors, as documented in the memorandum.

Recommendation: The NOAA Administrator should develop mechanisms to regularly monitor and evaluate implementation of the agency's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 7)

Agency: Department of Commerce: National Oceanic and Atmospheric Administration
Status: Open

Comments: In Commerce's written comments, NOAA stated that it will identify additional metrics for monitoring and evaluating its policy. As of October 2020, we have requested an update from the agency but have not yet received information to evaluate implementation of this recommendation.

Recommendation: The Director of USGS should develop mechanisms to regularly monitor and evaluate implementation of the agency's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 8)

Agency: U.S. Geological Survey
Status: Open

Comments: In September 2020, the Department of the Interior provided an update on this recommendation. In that update, officials stated that the expected completion date is the end of November 2022.

Recommendation: The Secretary of Energy should develop documented procedures for identifying and addressing alleged violations of its scientific integrity policy. (Recommendation 9)

Agency: Department of Energy
Status: Open

Comments: In July 2020, DOE provided an update on the status of this recommendation. In that update, DOE stated that the department's scientific integrity official will be responsible for leading and coordinating with other elements of the Department in developing procedures for identifying and addressing alleged violations of DOE's scientific integrity policy.

Recommendation: The Administrator of NASA should develop documented procedures for identifying and addressing alleged violations of its scientific integrity policy. (Recommendation 10)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA officials stated that the agency is undergoing additional internal and external reviews-with NASA officials working through the National Science and Technology Council's Joint Committee on Research Environments, which supports scientific security, rigor, and integrity-and that completion of the implementation of the recommendation is due October 31, 2020.

Recommendation: The Director of NSF should assess the agency's large facilities oversight workforce to identify any project management competency gaps, develop a plan to address any gaps and time frames for doing so, and monitor progress in closing them. (Recommendation 1)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of September 2019, had awarded a contract for a proficiency assessment and workforce gap analysis. NSF anticipated finishing the proficiency assessment and workforce gap analysis by the second quarter calendar year 2020 and planned to consider options to address any identified gaps, such as identifying workforce development training opportunities. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Director of NSF should establish criteria for the project management expertise of award recipients for large facilities projects and incorporate the criteria in project requirements and external panel reviews. (Recommendation 2)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of November 2019, had drafted but not finalized new language for the Major Facilities Guide and related supplemental award terms and conditions for major facilities. The new language would require award recipients to document their plans for meeting project management competencies. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Director of NSF should ensure, through a requirement or other means, that award recipients for large facilities projects provide information to NSF on any lessons learned or best practices. (Recommendation 3)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of November 2019, had drafted but not finalized supplemental award terms and conditions for major facilities to require recipients to participate in NSF's process for identifying and sharing lessons learned on projects - for example, by sending appropriate staff to the annual major facilities workshop that NSF hosts to provide a collaborative forum for continuous learning and information-sharing or by presenting lessons learned or good practices at the workshop. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Secretary of Labor should direct the Bureau of Labor Statistics (BLS) and the Employment and Training Administration (ETA) to develop ways to use existing or new data collection efforts to identify and systematically track the workforce effects of advanced technologies. For example, the Secretary could select any of the following possibilities, or could identify others.
• BLS could expand existing worker or firm surveys to ask respondents whether advanced technologies have resulted in worker displacements, work hour reductions, or substantial adjustments to work tasks.
• BLS could expand its employment projections work to regularly identify occupations projected to change over time due to advanced technologies.
• ETA could expand the O*NET data system to identify changes to skills, tasks, and tools associated with occupations, as the information is updated on its rotational basis, and consider how this could be used to track the spread of advanced technologies.
(Recommendation 1)

Agency: Department of Labor
Status: Open
Priority recommendation

Comments: DOL agreed with this recommendation. DOL stated that it will continue coordinating with the Census Bureau on research activities in this area, and plans to identify and recommend data collection options to fill gaps in existing information about how the workplace is affected by new technologies, automation, and AI. In February 2020, DOL's Bureau of Labor Statistics (BLS) issued a public report evaluating data gaps and providing recommendations for data collection options. In June 2020, DOL reported that BLS plans to host a seminar to discuss the report findings and potential pilot data collection options. DOL also plans to release its first annual employment projections data in September 2020 (previously released every 2 years). In addition, DOL reported that the Employment and Training Administration has undertaken three research efforts, which are still underway, to test ways to analyze O*NET data elements for their potential to track changes in occupations over time and to flag areas for further study on the workforce effects of automation. This recommendation will be implemented when DOL completes more of its activities.

Recommendation: The Secretary of the Navy should clarify whether and how to use the laboratory initiated research authority within the Capital Investment Program. (Recommendation 2)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: In December 2018, DOD agreed with our recommendation. In March 2019, DOD reported that the Navy was working on establishing a process for its laboratories to use the funds made available to them through the laboratory initiated research authority and planned to have this new policy in place by September 1, 2019 but was subsequently changed to a new date of October 1, 2019. However, the Navy was unable to finalize its policy prior to the start of the 2020 fiscal year. In February 2020, a senior USD(R&E) official stated that internal discussions between Navy acquisition officials and Navy financial management officials were ongoing. This official further noted that the Navy planned to finalize its policy by December 1, 2020, in time to influence the Navy's fiscal year 2022 budget request.

Recommendation: As the Subcommittee on Quantum Information Science (QIS Subcommittee) moves forward, the Office of Science and Technology Policy co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 1)

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: The Office of Science and Technology Policy (OSTP) agreed with GAO's September 2018 recommendation despite expressing some concerns about required resources and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019, the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the Department of Commerce co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 2)

Agency: Department of Commerce
Status: Open

Comments: The Department of Commerce agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the Department of Energy co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 3)

Agency: Department of Energy
Status: Open

Comments: The Department of Energy agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the National Science Foundation co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 4)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the Interagency Working Group on Synthetic Biology moves forward, the Director of the National Science Foundation, in coordination with participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 5)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation (NSF) agreed with GAO's September 2018 recommendation and, as of January 2020, had taken some steps to implement it. In November 2018, the Interagency Working Group on Synthetic Biology was formally established under the Biological Sciences Subcommittee of the National Science and Technology Council. The co-chairs of the Interagency Working Group on Synthetic Biology are officials from the National Institutes of Health, the National Institute of Standards and Technology, and NSF. The charter for the working group states that the group is to facilitate coordination and collaboration across 16 federal agencies. In October 2019, the working group hosted an Interagency Synthetic Biology Workshop to examine a roadmap that included basic science, enabling technologies, infrastructure and workforce needs in the area of synthetic biology. The workshop included 100 participants across the federal government, academia and industry, according to NSF officials. On the final day of the workshop participants from federal agencies used the input from the workshop to prepare a list of priority areas for investment along with agencies interested in participating in those priority areas. In January 2020, NSF officials reported that among the next steps for the working group was to develop a federal strategic roadmap for synthetic biology. Officials also reported that the working group is actively preparing a memorandum of understanding to create policies that will enable more sharing of information and collaboration. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in synthetic biology. When we confirm what additional actions the working group has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: The NSF Director of Human Resource Management should complete the development of an agency-wide workforce strategy for balancing the agency's use of IPA and VSEE rotators with permanent staff as part of NSF's current agency reform planning efforts or updates to its human capital operating plan. (Recommendation 1)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation but, as of March 2020, had not completed its plans for implementing it. Specifically, in November 2018, NSF indicated that it planned to initiate an effort to help identify current and future workforce needs. NSF stated that this effort would include identifying specific competencies needed for the future and how different hiring authorities-including permanent and temporary appointments-can be used to meet those needs. In addition, the information gained from this effort would contribute to an agency-wide workforce strategy. We will continue to monitor NSF's efforts and provide updated information when it becomes available.

Recommendation: The NSF Director of Human Resource Management should evaluate the contributions of the IPA and VSEE rotator programs toward NSF's human capital goals and the contributions the programs have made toward achieving programmatic results. (Recommendation 2)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation but, as of March 2020, had not completed its plans for implementing it. In November 2018, indicated that the agency planned to build on existing reviews to evaluate the contributions of the rotator program towards advancing the progress of science. We will continue to monitor NSF's efforts and provide updated information when it becomes available.

Recommendation: The NIST Associate Director for Laboratory Programs should update NIST policy to include periodic comprehensive management review of the agency's measurement services to assess gaps and ensure alignment with stakeholders' needs, and take steps to ensure that the Associate Director completes the review of NIST's standards development activities (Recommendation 1).

Agency: Department of Commerce: National Institute of Standards and Technology
Status: Open

Comments: NIST concurred with this recommendation and, as of May 2020, had taken some steps to implement it. Specifically, NIST provided information indicating it uses multiple information sources to identify potential measurement service gaps. For example, the Associate Director for Laboratory Programs (ADLP) reviews quarterly reports from NIST's laboratory divisions that include information on measurement services. The ADLP may identify measurement service gaps as part of this review. Also, the NIST Measurement Services Council serves the ADLP in an advisory role to identify and address NIST-wide issues related to the quality, relevance, performance, operations, and resources allocated to the health and improvement of NIST measurement services. The Council produces an annual report that addresses the health of NIST's measurement services, including potential changes needed to meet future customer needs. Further, NIST employees may suggest new services through processes established in suborder 5901.01. Many of these efforts also include consideration of potential gaps in NIST's participation in standards development activities. Additionally, NIST Order 5301.00 delegates responsibility to review standards activities and participation across several levels of NIST management. Although these actions may help identify gaps in NIST's participation in standards development activities as well as identify gaps in the measurement services it provides, it is not clear how or whether they fulfill the periodic review of the effectiveness of NIST's participation in documentary standards activities that the ADLP is to conduct under NIST's standards participation policy. We will update our evaluation of NIST's implementation of this recommendation when the agency provides additional information on how the activities described above fulfill the effectiveness review called for by NIST's policy, or provides information documenting that the ADLP has conducted such a review.

Recommendation: The Secretary of Commerce should instruct NIST to fully report the range of challenges in federal patent licensing, such as those outlined in this report, by, for example, leveraging its survey of practices at federal technology transfer offices, past Federal Laboratory Consortium studies, and agency reports and including that information in its summary reports to Congress. (Recommendation 1)

Agency: Department of Commerce
Status: Open

Comments: According to the Department of Commerce, NIST actions in response to the recommendation are in process. In August 2020, Commerce officials stated that the next annual report to the President and Congress was in the review process and is expected to be published this fall.

Recommendation: The Secretary of Commerce should instruct NIST to clarify the link between establishing patent license financial terms and the goal of promoting commercial use, through appropriate means, such as the upcoming rule-making process and updating relevant guidance. (Recommendation 2)

Agency: Department of Commerce
Status: Open

Comments: According to the Department of Commerce, NIST actions in response to the recommendation are in process and the purpose of licensing royalties will be addressed through a related initiative. The expected completion date for this action has been revised to November 2020.

Recommendation: The Secretary of Commerce should instruct NIST to facilitate formal information sharing among the agencies to provide federal labs with information on financial terms in comparable patent licenses, as appropriate. (Recommendation 3)

Agency: Department of Commerce
Status: Open

Comments: According to the Department of Commerce, NIST actions in response to the recommendation are in process. The actions to address it have been discussed by the Interagency Workgroup for Technology Transfer and the FLC, including adding a licensing guide and community of practice for licensing terms to the FLC website. In August 2020, Commerce stated that the expected completion date for this action has been revised to September 2020.

Recommendation: The Secretary of Defense should ensure that the agency or its labs document processes for establishing license financial terms, while maintaining flexibility to tailor the specific financial terms of each license. (Recommendation 4)

Agency: Department of Defense
Status: Open

Comments: DOD officials stated that DoDI 5538.08, which would require military departments and defense agencies to instruct their laboratories to document their licensing processes as appropriate, is in the review process as of August 2020.

Recommendation: The Director of NSF should revise the agency's policies for developing schedules for large facilities projects, and for reviewing those schedules, to better incorporate the best practices in GAO's schedule guide. (Recommendation 2)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of November 2019, had updated its internal guidance to include a new section related to schedule reviews. The updated guidance states that the NSF Large Facilities Office will lead analysis of the schedule for each proposed major facilities project, which will include a technical evaluation by the sponsoring office. As further steps to implement this recommendation, NSF planned to develop (1) a new section of the Major Facilities Guide on schedule development, estimating, and analysis and (2) new internal guidance on including project schedules as part of external panels' oversight reviews. NSF anticipated completing these actions by mid-fiscal year 2020. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The leadership of the Committee on STEM Education should review performance assessments of federal STEM education programs and then take appropriate steps to enhance effectiveness of the portfolio, such as by sharing promising practices that agencies could use in designing or revising their programs. (Recommendation 1)

Agency: Committee on Science, Technology, Engineering, and Math Education
Status: Open

Comments: The Committee agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. The Committee's plan states that to enhance the effectiveness of the STEM education portfolio, federal agencies that comprise the Committee are to perform a systematic review of evidence from current programs (e.g., performance assessments). Committee agencies are to also identify promising, evidence-based STEM education programs, practices, and policies in order to learn from and implement them more broadly. The strategic plan commits federal agencies to develop an implementation plan. According to agency officials, the implementation plan will be finalized in the Fall of 2019. GAO will review future efforts of the Committee to determine if they address the recommendation and could lead to improved management of the federal STEM education portfolio.

Recommendation: The leadership of the Committee on STEM Education should improve public awareness of information on programs' performance assessments by documenting program-level information on performance assessments in its federal STEM education inventory. (Recommendation 2)

Agency: Committee on Science, Technology, Engineering, and Math Education
Status: Open

Comments: The Committee agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. One of the plan's objectives focuses on making program performance and outcomes publically available. To achieve this objective, federal agencies that comprise the Committee are to document and share programs' performance assessment information for both internal strategic planning purposes and to inform public audiences. According to the strategic plan, Committee agencies are to provide content for the inventory of STEM education programs by the end of 2019. GAO will review the results of the Committee's future efforts, including its inventory of programs, to determine if they adequately address the recommendation. The plan acknowledges GAO's finding on the importance of collecting, organizing, and reporting federal performance data to increase public awareness of the impact of federal STEM education programs.

Recommendation: The leadership of the Committee on STEM Education should report required information on the participation rates of women, underrepresented minorities, and persons from rural areas in federal STEM education programs that collect this information. (Recommendation 3)

Agency: Committee on Science, Technology, Engineering, and Math Education
Status: Open

Comments: The Committee agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. According to the strategic plan, the Committee will develop a common reporting framework that federal agencies with STEM education programs can use to provide the rates of program participation by women, underrepresented minorities, and persons in rural areas. Agencies will begin reporting this information by the end of 2019. According to an official from the Office of Science and Technology Policy, the agency anticipates reporting programs' participation rates in its 2020 annual report to Congress. The plan also states that agencies that conduct valid and reliable tracking of such data are to share their promising practices across agencies so that those strategies can be replicated where and when appropriate. GAO will follow the Committee's progress developing and implementing the reporting framework to determine if it addresses GAO's recommendation. As noted in the strategic plan, tracking and reporting on the participation of underrepresented groups in federal STEM education programs provides agencies with information to gauge their effectiveness at fostering diversity and inclusion.

Recommendation: The Director of the Office of Science and Technology Policy should report the outcomes of programs' performance assessments completed in the previous year in its annual report. (Recommendation 4)

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: The Office of Science and Technology Policy agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. The strategic plan states that the Office of Science and Technology Policy will include in its annual reports descriptions of the outcomes of any program assessments conducted on federal STEM education programs in the previous year. According to an Office of Science and Technology Policy official, this information will be included in annual reports starting in 2020. GAO will review these annual reports to determine if they address the recommendation. As noted in the strategic plan, collecting and reporting federal performance data supports efficient and effective use of public funding, increases public awareness of federal program outcomes, and promotes the use of high-quality performance assessment and evaluation methods.

Recommendation: The Under Secretary of Defense for Intelligence, in coordination with the DOD Chief Information Officer, the Under Secretaries of Defense for Policy; Acquisition, Technology, and Logistics; and Personnel and Readiness; and with military service and agency stakeholders, should conduct operations security surveys that identify IoT security risks and protect DOD information and operations, in accordance with DOD guidance, or address operations security risks posed by IoT devices through other DOD risk assessments.

Agency: Department of Defense: Office of the Under Secretary of Defense for Intelligence
Status: Open

Comments: DOD concurred with this recommendation. We reached out to DOD in August 2018 on this recommendation and are awaiting their response.

Recommendation: The Principal Cyber Advisor, in coordination with the DOD Chief Information Officer; the Under Secretaries of Defense for Policy; Intelligence; Acquisition, Technology, and Logistics; and Personnel and Readiness; and with military service and agency stakeholders, should (1) review and assess existing departmental security policies and guidance--on cybersecurity, operations security, physical security, and information security--that may affect IoT devices; and (2) identify areas where new DOD policies and guidance may be needed--including for specific IoT devices, applications, or procedures--and where existing security policies and guidance can be updated to address IoT security concerns.

Agency: Department of Defense: Office of the Principal Cyber Advisor to the Secretary of Defense
Status: Open

Comments: DOD concurred with this recommendation. DOD has implemented one geo-location policy in 2018 relating to operations security that addresses a portion of this recommendation.

Recommendation: To ensure that DOD is positioned to counter both near and far term threats, consistent with its S&T framework, the Secretary of Defense should direct the new Under Secretary of Defense for Research and Engineering to annually define the mix of incremental and disruptive innovation investments for each military department.

Agency: Department of Defense
Status: Open
Priority recommendation

Comments: In July 2017, the Department of Defense (DOD) disagreed with our recommendation, stating that it would be premature to get ahead of the Secretary of Defense's final decisions on the role of the new Under Secretary of Defense for Research and Engineering (USD(R&E)) until that position is established, as required by law. In July 2018, the Department issued a memorandum finalizing the organizational structures, and roles and responsibilities for USD(R&E), but did not include a requirement for that office to annually define the mix of incremental and disruptive innovation investments for each military department. In September 2019, the Office of the USD(R&E) released an updated science & technology strategy. While the updated strategy acknowledges the need to invest in both incremental and disruptive innovation, the strategy does not define what an appropriate investment mix should be. In lieu of a DOD-wide defined mix set by USD(R&E), in April 2019, the Air Force issued its own science and technology strategy that acknowledged the need for both incremental and disruptive investments and defined what that mix should be. However, recent Army (2019) and Navy (2017) science and technology strategies do not define those military departments' desired mixes of incremental and disruptive innovation investments. In the Conference Report accompanying the National Defense Authorization Act for Fiscal Year 2020, Congress directed USD(R&E) to report further on this issue to the Committees on Armed Services of the Senate and House of Representatives by December 31, 2019.

Recommendation: To ensure that DOD is positioned to counter both near and far term threats, consistent with its S&T framework, the Secretary of Defense should direct the new Under Secretary of Defense for Research and Engineering to annually assess whether that mix is achieved.

Agency: Department of Defense
Status: Open
Priority recommendation

Comments: In July 2017, the Department of Defense (DOD) disagreed with our recommendation, stating that it would be premature to get ahead of the Secretary of Defense's final decisions on the role of the new Under Secretary of Defense for Research and Engineering (USD(R&E)) until that position is established, as required by law. In July 2018, the Department issued a memorandum finalizing the organizational structures, and roles and responsibilities for USD(R&E), but did not require that office to annually assess whether a desired mix of incremental and disruptive innovation investments mix had been achieved. In December 2019, a senior official within the Office of the USD(R&E) stated that DOD's Communities of Interest -- a component of DOD's overarching Reliance 21 framework for science and technology coordination -- are required to plan short- and long-term research and assess that research for an appropriate mix and balance between research priorities. However, as of December 2019, USD(R&E) has not yet articulated what the appropriate mix of incremental and disruptive innovation investments should be for DOD. Therefore, it is unknown the criteria the Communities use to evaluate whether an appropriate balance exists between research priorities, including incremental and disruptive innovation. In the Conference Report accompanying the National Defense Authorization Act for Fiscal Year 2020, Congress directed USD(R&E) to report further on this issue to the Committees on Armed Services of the Senate and House of Representatives by December 31, 2019.

Recommendation: To ensure that DOD is positioned to more comprehensively implement leading practices for managing science and technology programs, the Secretary of Defense should direct the new Under Secretary of Defense for Research and Engineering to define, in policy or guidance, an S&T management framework that includes emphasizing greater use of existing flexibilities to more quickly initiate and discontinue projects to respond to the rapid pace of innovation.

Agency: Department of Defense
Status: Open
Priority recommendation

Comments: In July 2017, the Department of Defense (DOD) disagreed with our recommendation, stating that it would be premature to get ahead of the Secretary of Defense's final decisions on the role of the new Under Secretary of Defense for Research and Engineering (USD(R&E)) until that position is established, as required by law. In July 2018, the Department issued a memorandum finalizing the organizational structures, and roles and responsibilities for USD(R&E), but did not require it to define a science and technology management framework that includes a process for discontinuing projects. In December 2019, a senior official with USD(R&E) reported that DOD has successfully implemented flexible funding vehicles such as the Defense Modernization Account that allowed funds to be rapidly moved to promising prototype projects within DOD's science and technology enterprise. In addition, this senior official reported an increased use of Other Transaction Authority by the Defense Innovation Unit and Defense Advanced Research Projects Agency. Nonetheless, the Office of the USD(R&E) has not yet developed policy or guidance that military departments could use to emphasize greater use of existing flexibilities for initiating and discontinuing science and technology projects. Consequently, DOD's processes for initiating and terminating science and technology projects largely remain linked to the annual federal budgeting process, which is not responsive to the rapid pace of innovation. In the Conference Report accompanying the National Defense Authorization Act for Fiscal Year 2020, Congress directed USD(R&E) to report further on this issue to the Committees on Armed Services of the Senate and House of Representatives by December 31, 2019.

Recommendation: To ensure that DOD is positioned to more comprehensively implement leading practices for managing science and technology programs, the Secretary of Defense should direct the new Under Secretary of Defense for Research and Engineering to define, in policy or guidance, an S&T management framework that includes incorporating acquisition stakeholders into technology development programs to ensure they are relevant to customers.

Agency: Department of Defense
Status: Open
Priority recommendation

Comments: In July 2017, the Department of Defense (DOD) disagreed with our recommendation, stating that it would be premature to get ahead of the Secretary of Defense's final decisions on the role of the new Under Secretary of Defense for Research and Engineering (USD(R&E)) until that position is established, as required by law. In July 2018, the Department issued a memorandum finalizing the organizational structures, roles, and responsibilities for USD(R&E), but did not require that office to define, in policy or guidance, a science and technology framework that includes incorporating acquisition stakeholders into technology development programs. In December 2019, a senior official within the Office of the USD(R&E) reported that USD(R&E) actively partners with acquisition stakeholders to ensure technology development programs are relevant to customers. The official cited Rapid Prototyping Programs (RPPs), Joint Capability Technology Demonstrations (JCTDs), and Emerging Capability Technology Development (ECTD) programs as examples where management frameworks in which technology managers actively partner with (1) operational managers from the Combatant Commands or military departments and (2) technology transition managers from the military departments to ensure programs are relevant to customers. However, these efforts are narrow in scope and do not constitute the majority of science and technology investments DOD makes. In addition, the senior official reported that the Army and the Air Force are taking steps to incorporate and integrate acquisition stakeholders into their science and technology projects, but these efforts are in their infancy. In the Conference Report accompanying the National Defense Authorization Act for Fiscal Year 2020, Congress directed USD(R&E) to report further on this issue to the Committees on Armed Services of the Senate and House of Representatives by December 31, 2019.

Recommendation: To ensure that DOD is positioned to more comprehensively implement leading practices for managing science and technology programs, the Secretary of Defense should direct the new Under Secretary of Defense for Research and Engineering to define, in policy or guidance, an S&T management framework that includes promoting advanced prototyping of disruptive technologies within the labs so the S&T community can prove these technologies work to generate demand from future acquisition programs.

Agency: Department of Defense
Status: Open
Priority recommendation

Comments: In July 2017, the Department of Defense disagreed with our recommendation, stating that it would be premature to get ahead of the Secretary of Defense's final decisions on the role of the new Under Secretary of Defense for Research and Engineering (USD(R&E)) until that position is established, as required by law. In July 2018, the Department issued a memorandum finalizing the organizational structures, and roles and responsibilities for USD(R&E), but did not require it to define, in policy or guidance, a science and technology framework that includes promoting advanced prototyping of disruptive technologies within the labs. In December 2019, a senior official within the Office of the USD(R&E) reported that the Emerging Capability Technology Development (ECTD) program is one framework USD(R&E) uses to promote the prototyping of disruptive technologies within the labs. Under this framework, USD(R&E) co-funds and co-sponsors projects with Federally Funded Research and Development Centers (FFRDCs), University Affiliated Research Centers (UARCs), and the military department laboratories. An integrated management team leads the evaluation and demonstration of technologies and connects technology managers with acquisition programs in the Combatant Commands and the military departments. The senior official further reported that USD(R&E) leverages Rapid Prototyping Funds (RPFs) and Rapid Prototyping Programs (RPPs) to promote and prove advanced demonstrations in military department laboratories. In the Conference Report accompanying the National Defense Authorization Act for Fiscal Year 2020, Congress directed USD(R&E) to report further on this issue to the Committees on Armed Services of the Senate and House of Representatives by December 31, 2019.

Recommendation: To help ensure that DOD is implementing the fraud, waste, and abuse prevention requirements to the OIGs, the Inspectors General of the Army, Navy, and Air Force should implement the requirements themselves or delegate the implementation of the requirements to the investigative services.

Agency: Department of Defense: Department of the Navy: Naval Inspector General
Status: Open

Comments: DOD concurred with the recommendation in its comments on the draft report. In June 2020, we requested information on the status of the recommendation. When we confirm the actions the Navy has taken, we will provide updated information.

Recommendation: To help ensure that DOD is implementing the fraud, waste, and abuse prevention requirements to the OIGs, the Inspectors General of the Army, Navy, and Air Force should implement the requirements themselves or delegate the implementation of the requirements to the investigative services.

Agency: Department of Defense: Department of the Army: Office of the Inspector General
Status: Open

Comments: DOD concurred with the recommendation in its comments on the draft report. As of July 2020, the Army had drafted a memorandum that would delegate the requirements to the investigative services or program office. We will update the status of this recommendation after the Army issues the final memorandum.

Recommendation: To enhance interagency collaboration in the Manufacturing USA program, the Secretary of Commerce should direct the Director of the National Institute of Standards and Technology to work with all non-sponsoring agencies whose missions contribute to or are affected by advanced manufacturing to revise the Manufacturing USA governance system to ensure the roles and responsibilities for how these agencies could contribute to the Manufacturing USA program are fully identified.

Agency: Department of Commerce
Status: Open

Comments: In October 2019, the National Institute of Standards and Technology (NIST) reported that, working through the Manufacturing USA interagency team and the National Science and Technology Subcommittee on Advanced Manufacturing, it had revised the Manufacturing USA governance document to include a section defining roles related to facilitating information sharing for agencies who are not sponsoring Manufacturing USA institutes. We are seeking clarification from NIST on which non-sponsoring agencies are covered by the new section. We will revisit the status of this recommendation once we receive clarification.

Recommendation: To enhance the ability of the Executive Office of the President to implement the Revitalize American Manufacturing and Innovation Act of 2014 requirements related to reporting on advanced manufacturing, the Director of the Office of Science and Technology Policy, working through the National Science and Technology Council and agency leadership, as appropriate, should identify the information they will collect from federal agencies to determine the extent to which the objectives outlined in the National Strategic Plan for Advanced Manufacturing are being achieved.

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: In October, 2018, the Subcommittee on Advanced Manufacturing, Committee on Technology of the National Science Technology Council released a Strategy for American Leadership in Advanced Manufacturing. This strategy provided some information on progress toward achieving the objectives of the prior National Strategic Plan for Advanced Manufacturing; however, it is unclear what information is to be collected from agencies and likewise how progress toward achieving the goals of the current strategy will be measured. We will update the status of this recommendation when the Office of Science and Technology Policy identifies the information to be collected from federal agencies and how this information will be used to assess progress in achieving the current goals, objectives, and priorities.

Recommendation: To fulfill the role assigned to it under the 1980 Act, the Secretary of Commerce should engage with industry stakeholders and continually identify and assess critical materials needs across a broad range of industrial sectors.

Agency: Department of Commerce
Status: Open

Comments: In December 2016, Commerce provided information on its implementation of the recommendation from GAO-16-699. Commerce stated that it had developed an action plan consisting of the following steps: (1) consulting with relevant offices and agencies, including: OSTP, DOD, the U.S. Geological Survey, DOE, the U.S. International Trade Commission, the Bureau of Industry and Security, the National Institute of Standards and Technology, and the National Oceanic and Atmospheric Administration; (2) determining criteria to be used when it is necessary to collect information to identify and assess critical materials needs; (3) determining appropriate steps, which might include: (a) developing a summary of information that federal agencies currently collect on the domestic and international supply of critical raw materials; (b) soliciting input from a broad range of industries through a Federal Register notice; (c) assessing aggregate information, as allowable under law, that is submitted through the Miscellaneous Tariff Bill process over the course of fiscal year 2017; and (d) consulting with federal advisory groups for advice; (4) determining the audience for collected information and methodology for information dissemination; (5) determining the process for identifying further information collection needs and methodology for disseminating collected information; and (6) determining the timeline and responsibilities for information collection and distribution. In an April 2017 update, Commerce stated that it had identified points of contacts in seven of the eight agencies listed in its action plan and is in the process of contacting them for input. Commerce stated that it hoped to identify an appropriate contact in the eighth agency in the near future. Commerce stated that it had also drafted questions to ask the agencies in order to implement the action plan. Commerce did not provide a timeframe for when it expected to complete implementation of the action plan. In a June 2018 update, Commerce stated that since the change in Administration, Commerce has not been able to identify staff in all agencies to work with, but that Commerce is now in contact with several agencies who are aware of industry needs. Commerce did not provide a timeframe for when it expected to complete execution of its action plan. We requested additional information on Commerce's efforts to implement this recommendation, including plans to solicit industry input, and will update the status of the recommendation based on additional information received.

Recommendation: To enhance the ability of the Executive Office of the President to coordinate federal agencies to carry out the national materials policy outlined in the 1980 Act, and to broaden future applications of the early warning screening methodology, the Subcommittee should take the steps necessary to include potentially critical materials beyond minerals, such as developing a plan or strategy for prioritizing additional materials for which actions are needed to address data limitations.

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: In September 2017, OSTP provided updated information on its efforts to implement recommendations from GAO-16-699. OSTP stated that "the Subcommittee shares GAO's interest in improving data availability and granularity. However, in some cases, private entities and foreign governments may be unwilling or unable to provide (or even collect) such data. Additionally, the Subcommittee member agencies' financial and personnel resources are limited, and significant additional resources would be required to prioritize and pursue the data for additional materials and critical materials beyond minerals. Without the appropriation of additional resources, the Subcommittee's work on these additional items will be necessarily circumscribed." In its February 2018 report on the updated application of the early warning screening methodology, the Subcommittee stated that it saw the value in analyzing more minerals and non-minerals to help inform policy decisions, but that fulfilling this need will require additional dedicated personnel and financial resources for data collection, analysis, and distribution. In March 2020, OSTP stated that the Subcommittee has explored the possibility of expanding the scope of the early warning screening methodology to include critical materials beyond minerals. According to OSTP, possible expansion candidates include carbon fiber and critical chemicals. OSTP stated that it has initiated a discussion with the Department of Interior (U.S. Geological Survey), who has been leading the methodology development, and the Department of Commerce (Bureau of Economic Analysis) with regard to possible data that would be needed for such an expansion. In August 2020, OSTP stated that the expertise to expand data collection to additional materials of interest exists in the National Minerals Information Center (NMIC) at the U.S. Geological Survey; however, the capacity to expand beyond the current portfolio is not available due to budgetary constraints. We will update this recommendation when we obtain additional information on these efforts.

Recommendation: To understand the extent to which incomplete inactivation occurs and whether incidents are being properly identified, analyzed, and addressed, the Secretary of Health and Human Services should direct the Centers for Disease Control and Prevention (CDC) and the National Institutes of Health (NIH) to develop clear definitions of inactivation for use within their respective guidance documents that are consistent across the Select Agent Program, NIH's oversight of recombinant pathogens, and the Biosafety in Microbiological and Biomedical Laboratories manual.

Agency: Department of Health and Human Services
Status: Open

Comments: As of April 2020, the Centers for Disease Control and Prevention (CDC) and National Institutes of Health (NIH) within the Department of Health and Human Service (HHS) are taking steps to address this recommendation. Specifically, in January and March 2017, HHS, in collaboration with the United States Department of Agriculture (USDA), issued updated select agent regulations and guidance that included clear definitions of inactivation and a validated inactivation procedure that are consistent across the Federal Select Agent Program. Additionally, HHS stated in December 2016 that NIH will consider providing clear and consistent definitions of inactivation in future guidance that is harmonized with the select agent regulations. Moreover, NIH and CDC told us they plan to include a new appendix in the revised Biosafety in Microbiological and Biomedical Laboratories manual that specifically addresses the development, validation, and implementation of inactivation protocols, which they anticipate releasing in June 2020, according to a CDC official.

Recommendation: To help ensure that inactivation protocols are scientifically sound and are effectively implemented, the Secretary of Health and Human Services should direct CDC and NIH to create comprehensive and consistent guidance for the development, validation, and implementation of inactivation protocols--to include the application of safeguards--across the Select Agent Program, NIH's oversight of recombinant pathogens, and the Biosafety in Microbiological and Biomedical Laboratories manual.

Agency: Department of Health and Human Services
Status: Open

Comments: In March 2017, the Department of Health and Human Services (HHS), in collaboration with the United States Department of Agriculture (USDA), issued Federal Select Agent Program guidance on the inactivation of select agents and toxins. According to HHS, this guidance is intended to provide additional information to regulated entities to assist them in meeting new requirements for rendering samples with select agents as non-viable. HHS also stated that the Federal Select Agent Program will continue to work with other federal agencies to ensure that the federal government is addressing inactivation in a consistent manner. In addition, according to HHS, the National Institutes of Health (NIH) will consider providing clear and consistent guidance related to inactivation that is harmonized with the Federal Select Agent Program as appropriate. As of April 2020, NIH and the Centers for Disease Control and Prevention (CDC) were in the process of revising the Biosafety in Microbiological and Biomedical Laboratories manual to include a new appendix that addresses the development, validation, and implementation of inactivation protocols. HHS plans to release the updated manual in June 2020, according to a CDC official.

Recommendation: To help ensure that dangerous pathogens can be located in the event there is an incident involving incomplete inactivation, the Secretary of Health and Human Services should direct the Directors of CDC and NIH, when updating the Biosafety in Microbiological and Biomedical Laboratories manual, to include guidance on documenting the shipment of inactivated material.

Agency: Department of Health and Human Services
Status: Open

Comments: The Department of Health and Human Services (HHS) stated in March 2017 that the National Institutes of Health (NIH) and Centers for Disease Control and Prevention (CDC) are in the process of revising the Biosafety in Microbiological and Biomedical Laboratories manual to include a new appendix that addresses inactivation methods, including guidance on documenting the shipment of inactivated material. HHS plans to release the updated manual in June 2020, according to a CDC official.

Recommendation: To improve USDA's ability to better understand the economic impacts of unintended mixing of GE and other crops, the Secretary of Agriculture should direct the Administrator of the National Agricultural Statistics Service (NASS) to work with all relevant USDA stakeholders, including APHIS and the Organic Working Group, to determine what additional information should be sought in future organic surveys, such as the costs of reshipping and re-storing shipments rejected because of unintended GE presence, as well as the costs associated with finding new buyers for such shipments.

Agency: Department of Agriculture
Status: Open

Comments: In its August 2016 Statement of Action on our report, USDA did not provide any new information on actions it has taken, if any, to implement this recommendation. For example, there was no indication whether stakeholders internal to the department had continued to meet to discuss the 2014 Organic Survey results and how to move forward with future survey questions to obtain additional data, such as data needed to better understand the economic impacts of unintended mixing with GE crops. As of September 2020, USDA did not have an update regarding the implementation of this recommendation. We will continue to follow up on USDA's efforts.

Recommendation: To improve USDA's ability to better understand the economic impacts of unintended mixing of GE and other crops, the Secretary of Agriculture should direct the Administrator of NASS to include producers, growing identity-preserved crops, in addition to organic producers in USDA's survey efforts.

Agency: Department of Agriculture
Status: Open

Comments: In its August 2016 Statement of Action on our report, USDA did not provide any new information on actions it has taken, if any, to implement this recommendation. We continue to believe that USDA should survey producers growing identity-preserved crops regarding their potential economic losses from unintended GE presence, as is being done for organic producers. As we previously reported, U.S. acreage planted to identity-preserved crops is significantly greater than that planted to organic crops; yet, little is known about the economic costs to identity-preserved farmers of unintended mixing. As of September 2020, USDA did not have an update regarding the implementation of this recommendation. We will continue to follow up on USDA's efforts.

Recommendation: In order to ensure complete, analyzable records regarding research grant award decisions are available for management and analysis, the Secretary of Energy should direct DOE's grant-making agencies to implement additional data collection efforts, which should include: (1) Retaining complete records of pre-proposal, proposal, and award data, including a record of proposal disposition, in linked electronic files to facilitate aggregate, statistical analysis of the grant-making process, including the calculation of success rates. (2) Collecting demographic, education, and career information from applicants, on a voluntary basis, that is not available to proposal reviewers but is used for analysis of success rates.

Agency: Department of Energy
Status: Open

Comments: DOE generally agreed with this recommendation. DOE's Office of Science began collecting investigator demographics during the second quarter of fiscal year 2015 and already retained complete records that enabled the calculation of success rates. According to DOE officials, since September 2017, the other three grant-making components included in our audit have been taking various actions to implement the recommendation. These three components, the Office of Nuclear Energy (NE), the Advanced Research Projects Agency-Energy (ARPA-E), and the Office of Energy Efficiency and Renewable Energy (EERE), all complete steps to ensure that they retain complete grant life cycle information for each individual award, including complete records of pre-proposal, proposal, and award data in linked electronic files, thus fulfilling the first part of our recommendation. In 2019, EERE coordinated the development of a Federal Register Notice (FRN) regarding data collection jointly with NE and ARPA-E. The FRN drafted by EERE was finalized and published on February 11, 2020 (Vol. 85, Issue 28, Pages 7759-7760 [FR DOC# 2020-02674]). Under the proposed information collection request, an interface will be implemented in DOE grant application systems to allow external users to voluntarily provide a minimal amount of demographic information to comply with this recommendation, pursuant to review by the Office of Management and Budget. On June 30th, DOE officials shared with GAO the draft FRN for the 30-day notice and request for public comment prior to implementation.

Recommendation: To comply with Title IX enforcement requirements, the Secretary of the Department of Defense, which funds STEM research at universities, should direct the Director of the Office of Diversity Management and Equal Opportunity to ensure that Title IX compliance reviews of DOD's grantees are periodically conducted.

Agency: Department of Defense
Status: Open

Comments: DOD agreed with this recommendation and noted it is in the process of revising current DOD guidance which will address its Title IX enforcement requirements. In September 2017, a DOD official stated that the agency is in the process of formulating instructions related to both Title IX and Title VI that they believe will address the recommendation regarding Title IX enforcement. In a memorandum issued in December 2017, a DOD official described the agency's corrective action plan (CAP), including drafting an updated rule for the Code of Federal Regulations and the development and issuance of internal DOD policy documents regarding Title IX enforcement requirements. Both of these activities were expected to be completed by June 2019. DOD reported in March 2020 that it is continuing to revise current DOD guidance to address its Title IX enforcement requirements. The Director of DOD's Office of Diversity, Equity, and Inclusion (ODEI) stated in a letter to GAO that the revised Titile IX policy is in the early stages of DOD's policy process. ODEI expressed its commitment to developing the revised policy and ensuring Title IX compliance reviews are conducted as per the revised policy and GAO's recommendation. In September of 2020, DoD sent an update indicating that they will issue a policy memorandum, specifically outlining the requirement for DoD Components who provide financial assistance to educational programs or activities conduct periodic compliance reviews. This is expected to be complete by December 31, 2020.

Recommendation: To reduce uncertainty about the expected cost and schedule of the U.S. ITER Project and its potential impact on the U.S. fusion program, once the ITER Organization completes its reassessment of the international project schedule, the Secretary of Energy should direct the Associate Director of the Office of Fusion Energy Sciences to use that schedule, if reliable, to propose a final, stable funding plan for the U.S. ITER Project, approve a performance baseline with finalized cost and schedule estimates, and communicate this information to Congress.

Agency: Department of Energy
Status: Open
Priority recommendation

Comments: DOE approved a performance baseline for the first plasma portion of the U.S. ITER Project in January 2017 and communicated that performance baseline to Congress. However, as of June 2020, DOE has not set a performance baseline for the post-first plasma portion of the project. DOE officials told us the Administration was reviewing U.S. participation in ITER. They said that if, at the end of the review, the Administration decided to continue U.S. participation in ITER, then DOE would set a performance baseline for the post-first plasma portion of the U.S. ITER Project.

Recommendation: To reduce uncertainty about the expected cost and schedule of the U.S. ITER Project and its potential impact on the U.S. fusion program, the Secretary of Energy should direct the Associate Director of the Office of Fusion Energy Sciences to direct the U.S. ITER Project Office to revise and update the project's cost estimate to meet all characteristics of high-quality, reliable cost estimates. Specifically, the U.S. ITER Project Office should revise the project's cost estimate to ensure it is credible by including a comprehensive sensitivity analysis that includes all significant cost elements and conducting an independent cost estimate.

Agency: Department of Energy
Status: Open

Comments: DOE revised and updated the cost estimate for the first plasma portion of the U.S. ITER Project in November 2016. Officials reported that, as part of that update, the U.S. ITER Project Office completed a comprehensive sensitivity analysis and that the Office of Science's Office of Project Assessment had conducted a review of the revised cost estimate. However, as of June 2020, DOE had yet to revise and update the cost estimate for the post-first plasma portion of the U.S. ITER Project. DOE officials told us they planned to do so when they set a performance baseline for that portion of the project, which they expected to do if, at the end of an ongoing review, the Administration decided to continue U.S. participation in ITER.

Recommendation: To enhance DOD's ability to make consistent and informed decisions in its management of the TCC program in accordance with internal control standards, the Under Secretary of Defense for Acquisition, Technology and Logistics should require the Director, Corrosion Policy and Oversight Office, to document the procedures for approving projects within the TCC program for civilian institutions.

Agency: Department of Defense: Office of the Under Secretary of Defense for Acquisition, Technology and Logistics
Status: Open

Comments: DOD did not agree with the recommendation. In 2016, DOD's Corrosion Office consistently maintained that its existing process is adequately documented in the DOD Corrosion Prevention and Mitigation Strategic Plan and the Technical Corrosion Collaboration (TCC) Definitions Document. However, GAO maintained that DOD could enhance its oversight of corrosion projects by documenting how it approves projects for civilian institutions. As of March 2019, DOD has since decided to take action to implement this recommendation. Specifically, the Corrosion Office plans to include information on documenting procedures for approving projects in a new DOD manual on corrosion that it has a goal of creating by the end of calendar year 2020. We will monitor the extent to which DOD implements this recommendation

Recommendation: To enhance DOD's ability to make consistent and informed decisions in its management of the TCC program in accordance with internal control standards, the Under Secretary of Defense for Acquisition, Technology and Logistics should require the Director, Corrosion Policy and Oversight Office, to document the procedures for selecting and approving projects within the TCC program for military academic institutions.

Agency: Department of Defense: Office of the Under Secretary of Defense for Acquisition, Technology and Logistics
Status: Open

Comments: DOD did not agree with the recommendation. In 2016, DOD's Corrosion Office had consistently maintained that its existing process is adequately documented in the DOD Corrosion Prevention and Mitigation Strategic Plan and the Technical Corrosion Collaboration (TCC) Definitions Document. However, GAO maintained that DOD could enhance its oversight of corrosion projects by documenting how it selects and approves TCC projects for military academic institutions. As of March 2019, DOD has since decided to take action to implement this recommendation. The Corrosion Office plans to include information on documenting procedures for selecting and approving projects in a new DOD manual on corrosion that it has a goal of creating by the end of calendar year 2020. We will monitor the extent to which DOD implements this recommendation.

Recommendation: To enhance DOD's ability to make consistent and informed decisions in its management of the TCC program in accordance with internal control standards, the Under Secretary of Defense for Acquisition, Technology and Logistics should require the Director, Corrosion Policy and Oversight Office, to document the procedures for selecting and approving military research labs supporting civilian and military institutions in conducting projects within the TCC program.

Agency: Department of Defense: Office of the Under Secretary of Defense for Acquisition, Technology and Logistics
Status: Open

Comments: DOD partially concurred with our recommendation. As of August 2018, the Corrosion Policy and Oversight office is currently re-writing Appendix A of the "Technical Corrosion Collaboration (TCC)" document to include steps and grading criteria for decision makers when selecting and approving military research labs supporting civilian and military institutions conducting projects with the TCC program. The Corrosion Policy and Oversight office will complete this re-write and the post procedures to their web site by November 30, 2018. As of March 2019, the Corrosion Policy and Oversight office plans to include procedures for selecting and approving labs to support institutions in a new DOD manual on corrosion. Its goal to create this new manual is by the end of calendar year 2020. We will monitor the extent to which DOD implements this recommendation.

Recommendation: To ensure that the increasing risks of GPS disruptions to the nation's critical infrastructure are effectively managed, the Secretary of Homeland Security should increase the reliability and usefulness of the GPS risk assessment by developing a plan and time frame to collect relevant threat, vulnerability, and consequence data for the various critical infrastructure sectors, and periodically review the readiness of data to conduct a more data-driven risk assessment while ensuring that DHS's assessment approach is more consistent with the National Infrastructure Protection Plan (NIPP).

Agency: Department of Homeland Security
Status: Open

Comments: DHS officials had previously indicated that DHS's Office of Infrastructure Protection (IP) and Office of Cyber and Infrastructure Analysis (OCIA) have discussed an update of the GPS risk assessment. Additionally, information from DHS shows that DHS has continued other efforts to collect potentially relevant threat, vulnerability, and consequence data for various GPS equipment in use. For example, according to DHS officials, DHS has conducted visits to major maritime, finance, wireless communications, and electricity firms to gauge their understanding of GPS vulnerabilities and of technology- and strategy-based efforts to improve GPS resilience, and DHS documentation shows that DHS has held events to test GPS receivers as part of assessing vulnerabilities. In August 2020, DHS officials provided GAO with additional information regarding their progress on implementing the recommendation. We will update the status of this recommendation after we review the additional information from DHS.

Recommendation: Congress should consider strengthening the current consumer privacy framework to reflect the effects of changes in technology and the marketplace--particularly in relation to consumer data used for marketing purposes--while also ensuring that any limitations on data collection and sharing do not unduly inhibit the economic and other benefits to industry and consumers that data sharing can accord. Among the issues that should be considered are: (1) the adequacy of consumers' ability to access, correct, and control their personal information in circumstances beyond those currently accorded under FCRA; (2) whether there should be additional controls on the types of personal or sensitive information that may or may not be collected and shared; (3) changes needed, if any, in the permitted sources and methods for data collection; and (4) privacy controls related to new technologies, such as web tracking and mobile devices.

Agency: Congress
Status: Open

Comments: As of July 2020, Congress has not taken action on this matter.

Recommendation: To help address the uncertainty NIH faces, related to the potential impact of increasing indirect costs on its funding of future research, the Director of NIH should assess the impact of growth in indirect costs on its research mission, including, as necessary, planning for how to deal with potential future increases in indirect costs that could limit the amount of funding available for total research, including the direct costs of research projects.

Agency: Department of Health and Human Services: Public Health Service: National Institutes of Health
Status: Open

Comments: As of July 7, 2015, NIH provided some information indicating that it had taken action to address our recommendation by tracking the size of indirect costs as a proportion of NIH's overall budget as part of the agency's annual budget planning process and risk assessment program. However, we determined that the actions did not fully address the recommendation because they focus on the agency's overall budget and do not assess the potential ongoing impact of indirect costs for universities on its mission. As of September 2018, NIH officials have not informed us of any additional actions taken to implement this recommendation. We will update the status of this recommendation when we receive additional information.

Recommendation: To better ensure the credibility of IRIS assessments by enhancing their timeliness and certainty, the EPA Administrator should require the Office of Research and Development, should different time frames be necessary, to establish a written policy that clearly describes the applicability of the time frames for each type of IRIS assessment and ensures that the time frames are realistic and provide greater predictability to stakeholders.

Agency: Environmental Protection Agency
Status: Open
Priority recommendation

Comments: As of March 2020, we have not seen a formal written memo from the IRIS program laying out this information - in detail - publicly, or how timelines for assessments are influenced by various criteria. While IRIS program staff have discussed this issue, no written guidance has been created. Such communication from the IRIS Program, as well as more frequent updates of the timelines for chemicals currently in assessment and projected starting dates for every chemical listed as "under assessment" is needed.

Recommendation: To ensure that current and accurate information on chemicals that EPA plans to assess through IRIS is available to IRIS users--including stakeholders such as EPA program and regional offices, other federal agencies, and the public--the EPA Administrator should direct the Office of Research and Development to annually publish the IRIS agenda in the Federal Register each fiscal year.

Agency: Environmental Protection Agency
Status: Open

Comments: As of March 2020, EPA's Integrated Risk Information System (IRIS) Program has established the priority chemicals it is working on, and has published some timelines via the IRIS Program Outlook document. However, this information has not been published as an agenda in the Federal Register.