Recommendation: The Secretary of Defense should ensure that the Under Secretary of Defense for Research and Engineering revise its IR&D instruction to require USD (R&E) personnel to annually review defense industry IR&D investments to inform DOD's science and technology strategic planning efforts. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation stating that USD(R&E) will investigate and revise its IR&D Instruction to require annual review of defense industry IR&D investments.

Recommendation: The Secretary of Defense should ensure that the Administrator, Defense Technical Information Center, assess and determine whether the DTIC IR&D database should require contractors to include additional information on IR&D projects, such as: (a) The IR&D project's linkage, if any, to DOD's modernization priorities; (b) The allowable category (basic research, applied research, technology development, or concept study) to which the IR&D project belongs; (c) The nature of the project as either potentially disruptive or potentially incremental research and development; and (d) The actual IR&D project costs when the project is completed. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with the recommendation stating that the DTIC Administrator will assess whether the DOD IR&D database should require contractors to include additional information on IR&D projects, and make his recommendation to the Director of Defense Research and Engineering for Research and Technology for its decision.

Recommendation: The Secretary of Defense should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense partially concurred with this recommendation noting the challenge with balancing ensuring public access to research data with considerations of national security and personally identifiable information. As discussed in our report, balancing these considerations is a challenge that agency officials and stakeholders identified during our work. Accordingly, our recommendation to DOD regarding findability and accessibility of agency-funded research data was qualified to pertain to appropriate agency-funded research data--recognizing that it might not be appropriate to make certain datasets publically available because of national security or other concerns. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Education should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 2)

Agency: Department of Education
Status: Open

Comments: The Department of Education concurred with this recommendation. According to its response to our report, the Department awarded a contract to support enhancements to its Education Resources Information Center (ERIC) to link scholarly research publications supported by the Department to its publicly accessible datasets. The Department indicated it expects to complete this work by September 30, 2020. When we confirm what actions the Department has taken to implement this recommendation we will provide additional information.

Recommendation: The Director of the Agency for Healthcare Research and Quality should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 3)

Agency: Department of Health and Human Services: Public Health Service: Agency for Healthcare Research and Quality
Status: Open

Comments: The Agency for Healthcare Research and Quality concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Commissioner of the Food and Drug Administration should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 4)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: The Food and Drug Administration concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Homeland Security should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 5)

Agency: Department of Homeland Security
Status: Open

Comments: The Department of Homeland Security concurred with this recommendation and noted that it was in the process of establishing a portal on its website to increase public access to agency-funded research. The department estimated these efforts would be completed by June 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Veterans Affairs should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 6)

Agency: Department of Veterans Affairs
Status: Open

Comments: The Department of Veterans Affairs concurred with this recommendation but indicated in its comments on the report that had already taken steps to implement it. As discussed in the report, the department's efforts to ensure research data availability pertained to a portion of the agency's federally-funded research data. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Director of the National Science Foundation should fully implement plans to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 7)

Agency: National Science Foundation
Status: Open

Comments: NSF concurred with this recommendation. According to its response to our report, NSF is expanding its public access repository to include metadata records about data that support publications resulting from NSF-funded research. NSF's response stated that, by storing metadata records for supporting datasets alongside metadata records for publications, the public will be able to more easily find and access appropriate agency funded research data. We will provide an update when we obtain additional information about the status of implementing this recommendation.

Recommendation: The Secretary of Defense should complete development of data management plan requirements for extramural researchers. (Recommendation 8)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense concurred with this recommendation noting planned steps to complete development of data management plan requirements. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Director of the Agency for Healthcare Research and Quality should complete development of data management plan requirements. (Recommendation 9)

Agency: Department of Health and Human Services: Public Health Service: Agency for Healthcare Research and Quality
Status: Open

Comments: The Agency for Healthcare Research and Quality concurred with this recommendation. When we confirm what actions the agency has taken to implement the recommendation we will provide updated information.

Recommendation: The Secretary of Homeland Security should complete development of data management plan requirements. (Recommendation 10)

Agency: Department of Homeland Security
Status: Open

Comments: The Department of Homeland Security concurred with this recommendation, stating that it would develop a departmentwide management directive for research and development data as well as data management plan guidance and a template to document requirements. The department estimated that these efforts would be completed by June 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The U.S. Agency for International Development Administrator should complete development of data management plan requirements for extramural researchers. (Recommendation 11)

Agency: United States Agency for International Development
Status: Open

Comments: The U.S. Agency for International Development agreed with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Agriculture should complete development of guidance and provide training to agency officials or others involved in reviewing the merits of researchers' data management plans. (Recommendation 12)

Agency: Department of Agriculture
Status: Open

Comments: The U.S. Department of Agriculture agreed with the findings of our report. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The U.S. Agency for International Development Administrator should complete development of and provide training for agency officials or others involved in reviewing the merits of researchers' data management plans. (Recommendation 13)

Agency: United States Agency for International Development
Status: Open

Comments: The U.S. Agency for International Development agreed with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Commissioner of the Food and Drug Administration should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 14)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: The Food and Drug Administration concurred with this recommendation. When we confirm what actions the agency has taken to implement the recommendation we will provide updated information.

Recommendation: The Secretary of Homeland Security should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 15)

Agency: Department of Homeland Security
Status: Open

Comments: The Department of Homeland Security concurred with this recommendation, and indicated it would evaluate training needs for data management plan reviews and develop plans to fulfill any additional training needs identified. The department estimated these efforts would be completed by September 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Veterans Affairs should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 16)

Agency: Department of Veterans Affairs
Status: Open

Comments: The Department of Veterans Affairs concurred with this recommendation and identified several planned steps to identify and meet training needs for those involved in reviewing researchers' data management plans. The department estimated these efforts would be completed by September 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Director of the Agency for Healthcare Research and Quality should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 17)

Agency: Department of Health and Human Services: Public Health Service: Agency for Healthcare Research and Quality
Status: Open

Comments: The Agency for Healthcare Research and Quality concurred with this recommendation stating that it would assess training needs for agency officials and others involved in reviewing the merits of researchers' data management plans and would develop and provide additional training if warranted. When we confirm what actions the agency has taken to implement the recommendation we will provide updated information.

Recommendation: The Secretary of Defense should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 18)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Energy should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 19)

Agency: Department of Energy
Status: Open

Comments: The Department of Energy concurred with this recommendation and stated it would assess and develop a plan to meet the training needs of internal DOE staff and external peer reviewers of data management plans. DOE estimated these efforts would be completed by December 31, 2020. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Environmental Protection Agency Administrator should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 20)

Agency: Environmental Protection Agency
Status: Open

Comments: The Environmental Protection Agency agreed with this recommendation stating that, during fiscal year 2020, the agency will evaluate training needs for agency officials who review researchers' data management plans. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Agriculture should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 21)

Agency: Department of Agriculture
Status: Open

Comments: The U.S. Department of Agriculture agreed with the findings of our report. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 22)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense concurred with this recommendation When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Director of the Agency for Healthcare Research and Quality should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements.(Recommendation 23)

Agency: Department of Health and Human Services: Public Health Service: Agency for Healthcare Research and Quality
Status: Open

Comments: The Agency for Healthcare Research and Quality concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Commissioner of the Food and Drug Administration should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 24)

Agency: Department of Health and Human Services: Food and Drug Administration
Status: Open

Comments: The Food and Drug Administration concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Director of the National Institutes of Health should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 25)

Agency: Department of Health and Human Services: Public Health Service: National Institutes of Health
Status: Open

Comments: The National Institutes of Health concurred with this recommendation but stated that the agency already had mechanisms in place to ensure compliance with public access plan and associated requirements for publications and data. As discussed in the report, we believe our recommendation, as worded, appropriately reflected the extent to which the agency had implemented researcher compliance mechanisms at the time of issuance of our report. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Homeland Security should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 26)

Agency: Department of Homeland Security
Status: Open

Comments: The Department of Homeland Security concurred with this recommendation and stated it would develop a mechanism to ensure researcher compliance with the department's public access plan and data management plan requirements. The department estimated these efforts would be completed by September 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The National Oceanic and Atmospheric Administration Administrator should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 27)

Agency: Department of Commerce: National Oceanic and Atmospheric Administration
Status: Open

Comments: The Department of Commerce agreed with this recommendation and the National Oceanic and Atmospheric Administration indicated it was pursuing multiple mechanisms to implement it. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Energy should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 28)

Agency: Department of Energy
Status: Open

Comments: The Department of Energy concurred with this recommendation stating that it would develop a compliance mechanism to identify researchers receiving funding from DOE financial assistance awards who are not compliant with DOE's public access plan for publications. DOE estimated these efforts would be completed by December 31, 2020. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Veterans Affairs should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 29)

Agency: Department of Veterans Affairs
Status: Open

Comments: The Department of Veterans Affairs concurred with this recommendation but indicated in its comments on the report that had already taken steps to implement it. As discussed in the report, the department's public access compliance mechanism covered a portion of the agency's federally-funded research data.. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.

Recommendation: The Secretary of Transportation should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 30)

Agency: Department of Transportation
Status: Open

Comments: The Department of Transportation concurred with this recommendation. According to its response to our report, the Department will build upon existing compliance mechanisms to ensure researcher compliance with its public access plan and associated requirements. As part of this process, the Department reported that it plans to update its public access plan, and expects to complete these actions by December 31, 2020. When we confirm what actions the Department has taken to implement this recommendation we will provide additional information.

Recommendation: The National Institute of Standards and Technology Director should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 31)

Agency: Department of Commerce: National Institute of Standards and Technology
Status: Open

Comments: The Department of Commerce agreed with this recommendation and the National Institute of Standards and Technology identified several planned steps to implement it with full completion anticipated by December 31, 2020. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: As the Subcommittee on Open Science moves forward, the Office of Science and Technology Policy co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 32)

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: OSTP disagreed with GAO's November 2019 recommendation, stating that the subcommittee had already taken steps to implement the leading practices GAO identified. However, OSTP officials did not provide documentation of these efforts and GAO continued to believe the recommendation was warranted. In March 2020, OSTP provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the Department of Defense co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 33)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with GAO's November 2019 recommendation, noting that DOD participates in subcommittee initiatives, including a working group on disclosure risk management, which is a topic of great importance to DOD. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the Department of Energy co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 34)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with GAO's November 2019 recommendation. DOE noted that, as a co-chair of the subcommittee, it is actively identifying areas of collaboration across agencies in implementing open science practices. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the National Institutes of Health co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 35)

Agency: Department of Health and Human Services: Public Health Service: National Institutes of Health
Status: Open

Comments: NIH concurred with GAO's November 2019 recommendation. NIH stated that the subcommittee and its working groups are actively coordinating and building consensus on issues and processes to implement leading practices that enhance and sustain collaboration across federal agencies. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the National Oceanic and Atmospheric Administration co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 36)

Agency: Department of Commerce: National Oceanic and Atmospheric Administration
Status: Open

Comments: NOAA concurred with GAO's November 2019 recommendation, noting that NOAA would work with the subcommittee to identify more opportunities for collaboration to promote access to research results. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: As the Subcommittee on Open Science moves forward, the National Science Foundation co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 37)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation concurred with GAO's November 2019 recommendation. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Recommendation: The Deputy Secretary of Homeland Security should ensure that S&T take steps to more fully incorporate practices for developing milestones within DHS's budget preparation guidance, into the Surface Transportation Explosive Threat Detection program. (Recommendation 1)

Agency: Department of Homeland Security
Status: Open

Comments: In September 2019, GAO reported on a Department of Homeland Security's (DHS) Science and Technology Directorate (S&T) research and development (R&D) program to develop technologies to secure mass transit systems. DHS budget guidance requires S&T to develop results-oriented milestones to track program progress. GAO found that the S&T program's milestones did not clearly link to key activities described in program plans, and thus, were not results oriented. Therefore, we recommended that DHS develop milestones to track its progress developing the technologies that fully adhered to guidance. DHS concurred with our recommendation, and in February 2020, reported that S&T's Finance and Budget Division validated that milestones for the program were compliant with DHS guidance. GAO is currently working with DHS S&T to review documentation related to the validation process in order to close the recommendation..

Recommendation: The Administrator of TSA should develop a mechanism to more routinely and comprehensively share appropriate information on the performance of mass transit security technologies (such as the annual sensor catalog and security technology assessments) with mass transit operators and stakeholders until DHS completes work on a more permanent information sharing resource. (Recommendation 2)

Agency: Department of Homeland Security: Transportation Security Administration
Status: Open

Comments: In September 2019, GAO reported on key mechanisms that TSA uses to collaborate and share information on identifying capability gaps and security technologies with stakeholders, including mass transit operators. We found that TSA regularly assesses commercially available technologies, but does not routinely or comprehensively share its results with mass transit operators. Therefore, we recommended that TSA develop a mechanism to routinely and comprehensively share security technology information with mass transit operators. TSA concurred with our recommendation, and in February 2020, reported implementing two of three planned efforts to better share security technology information, including steps to increase distribution of its annual publication on security technologies and to provide regular updates on assessed technologies at routine stakeholder meetings. We will continue to monitor TSA efforts with a third effort in order to close this recommendation.

Recommendation: The Secretary of Commerce should direct the National Institute of Standards and Technology (NIST) Director to work with other sponsoring federal agencies to develop and implement network-wide performance goals for the Manufacturing USA program with measurable targets and time frames. (Recommendation 1)

Agency: Department of Commerce
Status: Open

Comments: Commerce partially concurred with this recommendation according to its response to our report. Commerce stated that it lacks the legal authority to compel action by other federal agencies, and that legal constraints aside, it believes it is bad management practice to ask institutes to respond to performance goals issued by different bodies. As of March 2020, Commerce stated that it plans to report on performance goals for Commerce-sponsored institutes effective with the 2019 annual report, which it expects to issue by September 2020. Commerce also agreed to continue working closely with other sponsoring federal agencies on program and network goals for the Manufacturing USA institutes, but did not commit to working with sponsoring federal agencies to develop and implement network-wide performance goals with measurable targets and time frames. We recognize that Commerce does not have management authority over other the institutes sponsored by other agencies. We believe our report sufficiently characterizes the development of network-wide performance goals, targets, and time frames as a collaborative effort between Commerce and sponsoring agencies that is in keeping with Commerce's network-wide coordination functions under the RAMI Act. Moreover, our recommendation specifically pertained to developing performance goals for the Manufacturing USA program, not individual institutes. As we stated in our report, this would not necessarily entail new performance measures but, rather, could consist of measurable near-term performance goals corresponding to program performance measures already in place. Further, as stated in our report, GAO's prior work has shown that systems of performance measures benefit from certain key practices, such as creating a hierarchy that breaks down broad, long-term goals and objectives into more specific, near-term performance goals with measurable targets and time frames. Our recommendation was designed to ensure that the Manufacturing USA program performance measurement structure that Commerce has already worked with the other sponsoring agencies to develop more fully aligns with these key practices. We continue to believe that by working with other sponsoring federal agencies to develop and implement network-wide performance goals with targets and time frames, Commerce would be better able to observe and report on progress toward long-term Manufacturing USA program goals and objectives.

Recommendation: The Secretary of Commerce should direct the NIST Director to work with other sponsoring federal agencies to ensure that the Manufacturing USA network-wide performance measures are directly aligned with the network-wide performance goals, the Manufacturing USA strategic objectives and program goals, and the statutory purposes of the Revitalize American Manufacturing and Innovation (RAMI) Act. (Recommendation 2)

Agency: Department of Commerce
Status: Open

Comments: Commerce partially concurred with this recommendation according to its response to our report. In March 2020, Commerce stated that it supports the alignment of performance measures with performance goals only for Commerce-sponsored institutes. Commerce stated that it is unable to commit to this recommendation as 13 of the 14 existing institutes were authorized under authorities other than the RAMI Act and are sponsored by agencies other than Commerce. Commerce also stated that, until additional institutes authorized by the RAMI Act are in place, it does not support additional performance measures for the single Commerce-sponsored institute beyond the RAMI Act requirements, as doing so would impose an unfair level of scrutiny. Commerce agreed to report on performance metrics for department-sponsored institutes effective with the current annual report, which is expected by September 2020. We recognize that Commerce does not have management authority over other the institutes sponsored by other agencies. We believe our report sufficiently characterizes the effort to align the network-wide performance measures with network-wide performance goals and Manufacturing USA program goals as a collaborative effort between Commerce and sponsoring agencies that is in keeping with Commerce's coordination functions under the RAMI Act. Our recommendation does not ask Commerce to compel actions by other agencies, nor to develop any additional performance measures. As noted in our report, the Manufacturing USA program's performance measurement structure aligns near-term performance measures directly to the program's long-term goals. This structure bypasses connecting the performance measures with the program's objectives that have been developed to break down the long-term goals more specifically. GAO's prior work has shown that systems of performance measures benefit from certain key practices, such as creating a hierarchy that breaks down broad, long-term goals and objectives into more specific, near-term performance goals with measurable targets and time frames. Our recommendation was designed to ensure that the Manufacturing USA program performance measurement structure that Commerce has already worked with the other sponsoring agencies to develop more fully aligns with these key practices. We continue to believe that by working with other sponsoring federal agencies to ensure that the Manufacturing USA network-wide performance measures are directly aligned with the Manufacturing USA strategic program goals and objectives and the statutory purposes of the RAMI Act, Commerce would be better able to observe and report on progress made toward achieving the statutory purposes of the Manufacturing USA program.

Recommendation: The Secretary of Commerce should direct the NIST Director to develop criteria to evaluate whether the Commerce-sponsored institute can sustain its operations without additional federal financial assistance after its initial agreement. If an analysis based on such criteria indicates that additional federal financial assistance is needed to help the institute sustain its operations, then the Secretary of Commerce should consider a legislative proposal to amend relevant provisions of the RAMI Act. (Recommendation 3)

Agency: Department of Commerce
Status: Open

Comments: Commerce partially concurred with this recommendation according to its response to our report. In March 2020, Commerce stated that it will develop criteria to evaluate the sufficiency of the Commerce-sponsored institute's sustainability plan based on the anticipated operating costs of the institute at fully operational steady state, and the likelihood of sustaining those operations through the specific efforts outlined in the sustainability plan. Commerce stated that it plans to develop the evaluation criteria by April 2020. We will update this recommendation after we learn more about these efforts.

Recommendation: The Secretary of Defense should direct the Director of DOD's Manufacturing USA institutes to develop criteria to evaluate whether DOD-sponsored institutes can sustain their operations without additional federal financial assistance after their initial agreements. (Recommendation 4)

Agency: Department of Defense
Status: Open

Comments: DOD partially concurred with this recommendation. According to information DOD provided in March 2020, DOD developed criteria to evaluate whether each DOD-sponsored institute is effectively executing its mission, providing value to the department, and transitioning advanced manufacturing to U.S. manufacturers, while demonstrating progress toward business viability (diversified revenue, controlled costs, etc.). As of March 2020, DOD plans to incorporate these criteria into a strategic management plan, under which the department will review institutes' progress at the end of their agreements to determine the type and level of DOD's continued participation. DOD expects to complete work on the strategic management plan by the end of September 2020.

Recommendation: The Secretary of Energy should direct the Director of DOE's Manufacturing USA institutes to develop criteria to evaluate whether DOE-sponsored institutes can sustain their operations without additional federal financial assistance after their initial agreements. (Recommendation 5)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with this recommendation according to its response to our report. In September 2019, DOE stated that it will direct the Directors of DOE-sponsored institutes to collectively work toward updating institute sustainability plans and activities and collaboratively develop criteria and metrics to assess the institutes' progress toward financial sustainability. After development of the metrics, DOE will track as appropriate. As of March 2020, DOE reported agreement with its institutes on an initial set of criteria and metrics to assess progress toward financial sustainability. We will update this recommendation as we collect more information about these efforts.

Recommendation: The Secretary of Energy should take action to educate and communicate the agency's scientific integrity polices to staff through, for example, regular training. (Recommendation 1)

Agency: Department of Energy
Status: Open

Comments: In July 2020, DOE provided an update on the status of this recommendation. In that update, DOE said the scientific integrity official will be responsible for leading and coordinating with other elements of the Department, in the development of measures to educate and communicate DOE's scientific integrity policies to staff.

Recommendation: The Secretary of Energy should establish steps and a time frame for designating a scientific integrity official to oversee the department's scientific integrity activities. (Recommendation 3)

Agency: Department of Energy
Status: Open

Comments: In July 2020, DOE provided an update on the status of this recommendation. In that update, DOE said that it was still in the process of identifying an individual to serve as a scientific integrity official.

Recommendation: The Secretary of Energy should develop mechanisms to regularly monitor and evaluate implementation of the department's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 4)

Agency: Department of Energy
Status: Open

Comments: In July 2020, DOE provided an update on the status of this recommendation. In that update, DOE stated that the Department's scientific integrity official will have the responsibility to lead and coordinate with other elements of the Department in developing procedures to monitor and evaluate the implementation of DOE's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary.

Recommendation: The Secretary of Transportation should develop mechanisms to regularly monitor and evaluate implementation of the department's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 5)

Agency: Department of Transportation
Status: Open

Comments: In September 2020, the Department of Transportation provided an update on the status of this recommendation. In that update, officials said that DOT was still working on this action. According to officials, the department will implement several mechanisms to address the recommendation, including conducting annual reviews of the scientific integrity policy and making the policy available to all relevant employees. DOT estimated it would complete these actions by the end of March 2021.

Recommendation: The Director of NIST should develop mechanisms to regularly monitor and evaluate implementation of the agency's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 6)

Agency: Department of Commerce: National Institute of Standards and Technology
Status: Open

Comments: In Commerce's written comments, NIST had stated that, beginning in fiscal year 2019, the agency would review implementation of its policy at least annually and make recommendations to the Director of NIST as to whether any improvements were needed. In a September 2020 update, NIST provided a memorandum reporting that in fiscal year 2019 the Scientific Integrity Officer and General Counsel had discussed an allegation related to scientific integrity, and that it was determined that programmatic factors, not a lapse in scientific integrity, had occurred. According to NIST, it monitors the effectiveness of the implementation of its documented scientific integrity policy, order, and procedure by means of informal feedback from NIST researchers and supervisors, as documented in the memorandum.

Recommendation: The NOAA Administrator should develop mechanisms to regularly monitor and evaluate implementation of the agency's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 7)

Agency: Department of Commerce: National Oceanic and Atmospheric Administration
Status: Open

Comments: In Commerce's written comments, NOAA stated that it will identify additional metrics for monitoring and evaluating its policy. As of October 2020, we have requested an update from the agency but have not yet received information to evaluate implementation of this recommendation.

Recommendation: The Director of USGS should develop mechanisms to regularly monitor and evaluate implementation of the agency's scientific integrity policy, including mechanisms to remediate identified deficiencies and make improvements where necessary. (Recommendation 8)

Agency: U.S. Geological Survey
Status: Open

Comments: In September 2020, the Department of the Interior provided an update on this recommendation. In that update, officials stated that the expected completion date is the end of November 2022.

Recommendation: The Secretary of Energy should develop documented procedures for identifying and addressing alleged violations of its scientific integrity policy. (Recommendation 9)

Agency: Department of Energy
Status: Open

Comments: In July 2020, DOE provided an update on the status of this recommendation. In that update, DOE stated that the department's scientific integrity official will be responsible for leading and coordinating with other elements of the Department in developing procedures for identifying and addressing alleged violations of DOE's scientific integrity policy.

Recommendation: The Administrator of NASA should develop documented procedures for identifying and addressing alleged violations of its scientific integrity policy. (Recommendation 10)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: NASA officials stated that the agency is undergoing additional internal and external reviews-with NASA officials working through the National Science and Technology Council's Joint Committee on Research Environments, which supports scientific security, rigor, and integrity-and that completion of the implementation of the recommendation is due October 31, 2020.

Recommendation: The Deputy Secretary of the Department of Homeland Security should develop a mechanism that aligns processes and information sources for collecting R&D project data from DHS components to ensure that the information can be collected, integrated and result in a comprehensive accounting of R&D projects DHS-wide. (Recommendation 2)

Agency: Department of Homeland Security: Office of the Secretary: Office of the Deputy Secretary
Status: Open

Comments: As of September 2020, DHS reported that S&T was revising the related DHS Directive to improve R&D project data collection and that detailed procedures for the data collection would be described in an associated guidance document. DHS further reported that S&T would be coordinating with the Undersecretary for Management, and meeting with DHS components regarding the data collection efforts. We will continue to monitor DHS's efforts to implement this recommendation.

Recommendation: The Deputy Secretary of the Department of Homeland Security should direct Office of the Chief Financial Officer program officials to ensure that S&T take steps to more fully incorporate leading practices, such as those included in DHS's budget preparation guidance, into R&D milestones. (Recommendation 3)

Agency: Department of Homeland Security: Office of the Secretary: Office of the Deputy Secretary
Status: Open

Comments: As of September 2020, S&T reported preparing Budget Justification documents, which will include key milestones for R&D projects, and indicated that the documents will be reviewed by the DHS Office of the Chief Financial Officer. We will continue to monitor DHS's efforts related to this recommendation.

Recommendation: The Deputy Secretary of the Department of Homeland Security should develop standard processes and procedures for collecting and analyzing customer feedback, applicable to components conducting R&D, for improving the usefulness of existing customer feedback mechanisms to assess R&D efforts and for implementing such mechanisms where absent. (Recommendation 4)

Agency: Department of Homeland Security: Office of the Secretary: Office of the Deputy Secretary
Status: Open

Comments: As of September 2020, DHS stated that a new DHS Directive and Instruction is being developed related to R&D customer and program feedback. This recommendation remains open, and we will continue to monitor DHS's efforts to address it.

Recommendation: The Secretary of the Army should ensure that the Commanding General of Army Futures Command applies leading practices as they relate to technology development, particularly that of demonstrating technology in an operational environment prior to starting system development. (Recommendation 1)

Agency: Department of Defense: Department of the Army
Status: Open
Priority recommendation

Comments: The Army concurred with this recommendation. Army officials acknowledged the importance of demonstrating technology in an operation environment prior to starting system development and stated that they have taken steps to assist in the identification and removal of infeasible or immature technologies. For example, development of the Integrated Visual Augmentation System includes a series of user engagements where soldiers interact with the system in a field environment and provide feedback to guide development and design. To fully implement this recommendation the Army will need to demonstrate that all of the technologies it is developing are matured in accordance with leading practices. Including soldier engagement in the development of the Integrated Visual Augmentation System is a positive step in that it uses operational experience to determine the utility of technologies on the battlefield. However, demonstrating that those technologies function as expected in an operational environment is equally important. Our past work has shown that demonstrations in an operational environment reduce the risk that technologies will not operate as intended or desired. It is important that the Army continue and expand its efforts to eliminate infeasible, or immature technologies across all of its development programs.

Recommendation: The Secretary of the Army should ensure that the Commanding General of Army Futures Command takes steps to incorporate the experiences of the cross-functional teams in applying leading practices for effective cross-functional teams. (Recommendation 2)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The Army concurred with this recommendation. According to Army officials, cross-functional teams routinely meet to discuss both successes and challenges. Additionally, senior Army leadership hosts weekly meetings with cross-functional teams to discuss decision points and concerns. Army Futures Command plans to include leadership from all cross-functional teams at future meetings to enhance knowledge sharing and ensure leadership viewpoints are communicated across the enterprise. These actions do incorporate elements of leading practices for cross-functional teams such as open and regular communication and senior management support. To fully implement this recommendation the Army should continue and expand this knowledge sharing as planned as well as ensure other elements of leading practices for effective cross-functional teams are implemented. These include the establishment of well-defined team operations with project-specific rules and procedures as well as appropriate training and learning environments for all cross-functional team members.

Recommendation: The Secretary of the Army should ensure that the Commanding General of Army Futures Command executes a process for identifying and incorporating lessons learned from cross-functional team pilots into the new command. (Recommendation 3)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The Army concurred with this recommendation. According to Army officials an analyst from the Center for Army Lessons Learned has embedded a permanent analyst at Army Futures Command who is capturing lessons learned and best practices from across the cross-functional teams. The results of this analysis are provided across Army Futures Command and to other stakeholders. Additionally, Army Futures Command has established the Directorate of Systems Integration to act as a repository and action office for lessons learned. This office hosts recurring coordination events to further the communication of lessons learned .

Recommendation: The Secretary of the Army should ensure that the Commanding General of Army Futures Command fully applies leading practices for mergers and organizational transformations as roles, responsibilities, policies and procedures are finalized for the new command. (Recommendation 4)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: The Army concurred with this recommendation. According to Army officials Army Futures Command is applying leading practices for mergers and organizational transformations as it continues to establish the command. These practices include total employee involvement in the transformation process with senior leadership championing the overarching transformation. In addition, the command has worked to identify processes that need improvement as it incorporates the organizations transferred from other commands. It also continues to evolve its staff and functions as needs are identified. According to officials, Army Futures Command continues to select high-performing team members to guide its transformation through selective recruitment and talent management as well as augmenting its growing staff with other Army personnel selected for temporary assignments. The command is also involving employees to obtain their ideas and gain their ownership for the transformation by encouraging them to share their thoughts and ideas. Importantly, according to officials, the command recognizes the need to have an enduring change model to consistently reassess and reorganize the command to consistently deliver products in relevant timeframes. These actions do demonstrate elements of leading practices for mergers and organizational transformations. However, to fully implement this recommendation, Army Futures Command should formalize and institutionalize its authorities, responsibilities, policies and procedures as they relate to these leading practices, especially as leadership within the command and the Army change.

Recommendation: The Secretary of the Navy should clarify whether and how to use the laboratory initiated research authority within the Capital Investment Program. (Recommendation 2)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: In December 2018, DOD agreed with our recommendation. In March 2019, DOD reported that the Navy was working on establishing a process for its laboratories to use the funds made available to them through the laboratory initiated research authority and planned to have this new policy in place by September 1, 2019 but was subsequently changed to a new date of October 1, 2019. However, the Navy was unable to finalize its policy prior to the start of the 2020 fiscal year. In February 2020, a senior USD(R&E) official stated that internal discussions between Navy acquisition officials and Navy financial management officials were ongoing. This official further noted that the Navy planned to finalize its policy by December 1, 2020, in time to influence the Navy's fiscal year 2022 budget request.

Recommendation: As the Subcommittee on Quantum Information Science (QIS Subcommittee) moves forward, the Office of Science and Technology Policy co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 1)

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: The Office of Science and Technology Policy (OSTP) agreed with GAO's September 2018 recommendation despite expressing some concerns about required resources and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019, the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the Department of Commerce co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 2)

Agency: Department of Commerce
Status: Open

Comments: The Department of Commerce agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the Department of Energy co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 3)

Agency: Department of Energy
Status: Open

Comments: The Department of Energy agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the National Science Foundation co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 4)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the Interagency Working Group on Synthetic Biology moves forward, the Director of the National Science Foundation, in coordination with participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 5)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation (NSF) agreed with GAO's September 2018 recommendation and, as of January 2020, had taken some steps to implement it. In November 2018, the Interagency Working Group on Synthetic Biology was formally established under the Biological Sciences Subcommittee of the National Science and Technology Council. The co-chairs of the Interagency Working Group on Synthetic Biology are officials from the National Institutes of Health, the National Institute of Standards and Technology, and NSF. The charter for the working group states that the group is to facilitate coordination and collaboration across 16 federal agencies. In October 2019, the working group hosted an Interagency Synthetic Biology Workshop to examine a roadmap that included basic science, enabling technologies, infrastructure and workforce needs in the area of synthetic biology. The workshop included 100 participants across the federal government, academia and industry, according to NSF officials. On the final day of the workshop participants from federal agencies used the input from the workshop to prepare a list of priority areas for investment along with agencies interested in participating in those priority areas. In January 2020, NSF officials reported that among the next steps for the working group was to develop a federal strategic roadmap for synthetic biology. Officials also reported that the working group is actively preparing a memorandum of understanding to create policies that will enable more sharing of information and collaboration. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in synthetic biology. When we confirm what additional actions the working group has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: The leadership of the Committee on STEM Education should review performance assessments of federal STEM education programs and then take appropriate steps to enhance effectiveness of the portfolio, such as by sharing promising practices that agencies could use in designing or revising their programs. (Recommendation 1)

Agency: Committee on Science, Technology, Engineering, and Math Education
Status: Open

Comments: The Committee agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. The Committee's plan states that to enhance the effectiveness of the STEM education portfolio, federal agencies that comprise the Committee are to perform a systematic review of evidence from current programs (e.g., performance assessments). Committee agencies are to also identify promising, evidence-based STEM education programs, practices, and policies in order to learn from and implement them more broadly. The strategic plan commits federal agencies to develop an implementation plan. According to agency officials, the implementation plan will be finalized in the Fall of 2019. GAO will review future efforts of the Committee to determine if they address the recommendation and could lead to improved management of the federal STEM education portfolio.

Recommendation: The leadership of the Committee on STEM Education should improve public awareness of information on programs' performance assessments by documenting program-level information on performance assessments in its federal STEM education inventory. (Recommendation 2)

Agency: Committee on Science, Technology, Engineering, and Math Education
Status: Open

Comments: The Committee agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. One of the plan's objectives focuses on making program performance and outcomes publically available. To achieve this objective, federal agencies that comprise the Committee are to document and share programs' performance assessment information for both internal strategic planning purposes and to inform public audiences. According to the strategic plan, Committee agencies are to provide content for the inventory of STEM education programs by the end of 2019. GAO will review the results of the Committee's future efforts, including its inventory of programs, to determine if they adequately address the recommendation. The plan acknowledges GAO's finding on the importance of collecting, organizing, and reporting federal performance data to increase public awareness of the impact of federal STEM education programs.

Recommendation: The leadership of the Committee on STEM Education should report required information on the participation rates of women, underrepresented minorities, and persons from rural areas in federal STEM education programs that collect this information. (Recommendation 3)

Agency: Committee on Science, Technology, Engineering, and Math Education
Status: Open

Comments: The Committee agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. According to the strategic plan, the Committee will develop a common reporting framework that federal agencies with STEM education programs can use to provide the rates of program participation by women, underrepresented minorities, and persons in rural areas. Agencies will begin reporting this information by the end of 2019. According to an official from the Office of Science and Technology Policy, the agency anticipates reporting programs' participation rates in its 2020 annual report to Congress. The plan also states that agencies that conduct valid and reliable tracking of such data are to share their promising practices across agencies so that those strategies can be replicated where and when appropriate. GAO will follow the Committee's progress developing and implementing the reporting framework to determine if it addresses GAO's recommendation. As noted in the strategic plan, tracking and reporting on the participation of underrepresented groups in federal STEM education programs provides agencies with information to gauge their effectiveness at fostering diversity and inclusion.

Recommendation: The Director of the Office of Science and Technology Policy should report the outcomes of programs' performance assessments completed in the previous year in its annual report. (Recommendation 4)

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: The Office of Science and Technology Policy agreed with this action, as recommended by GAO in its March 2018 report, and took some steps toward implementing it. In December 2018, the Committee issued a new 5-year STEM education strategic plan. The strategic plan states that the Office of Science and Technology Policy will include in its annual reports descriptions of the outcomes of any program assessments conducted on federal STEM education programs in the previous year. According to an Office of Science and Technology Policy official, this information will be included in annual reports starting in 2020. GAO will review these annual reports to determine if they address the recommendation. As noted in the strategic plan, collecting and reporting federal performance data supports efficient and effective use of public funding, increases public awareness of federal program outcomes, and promotes the use of high-quality performance assessment and evaluation methods.

Recommendation: To enhance interagency collaboration in the Manufacturing USA program, the Secretary of Commerce should direct the Director of the National Institute of Standards and Technology to work with all non-sponsoring agencies whose missions contribute to or are affected by advanced manufacturing to revise the Manufacturing USA governance system to ensure the roles and responsibilities for how these agencies could contribute to the Manufacturing USA program are fully identified.

Agency: Department of Commerce
Status: Open

Comments: In October 2019, the National Institute of Standards and Technology (NIST) reported that, working through the Manufacturing USA interagency team and the National Science and Technology Subcommittee on Advanced Manufacturing, it had revised the Manufacturing USA governance document to include a section defining roles related to facilitating information sharing for agencies who are not sponsoring Manufacturing USA institutes. We are seeking clarification from NIST on which non-sponsoring agencies are covered by the new section. We will revisit the status of this recommendation once we receive clarification.

Recommendation: In order to improve FDA's strategic planning for regulatory science efforts, the Secretary of Health and Human Services should direct the Commissioner of FDA to develop and document measurable goals, such as targets and time frames, for its regulatory science efforts so it can consistently assess and report on the agency's progress in regulatory science efforts.

Agency: Department of Health and Human Services
Status: Open

Comments: In September 2018, the agency described the actions of committees developed by each of the centers to oversee their regulatory science activities. In July 2019, the agency indicated that it was revisiting its strategic regulatory science priorities as part of its cyclical strategic planning process, and the centers and various offices have taken steps to address the recommendation. In an August 2020 written response, the agency reported that a committee had undertaken a review its 2011 regulatory science strategic plan that will result in the issuance of an accountability framework--an internal document outlining the type of information that FDA centers and offices will provide to the agency's Chief Scientist to demonstrate progress made in addressing relevant focus areas of regulatory science. FDA reported that its goal was to complete this work by the end of December 2020. However, FDA still needs to document steps taken by these centers to develop measurable goals, and the agency indicated that the recommendation should remain open. GAO will continue to monitor the implementation of this recommendation.

Recommendation: In order to improve FDA's strategic planning for regulatory science efforts, the Secretary of Health and Human Services should direct the Commissioner of FDA to systematically track funding of regulatory science projects across each of its priority areas.

Agency: Department of Health and Human Services
Status: Open

Comments: In September 2018 and July 2019, the agency described actions taken by each center to better track its regulatory science funding. However, FDA still needed to complete these activities and document that funds are systematically tracked across each of the priority areas. In an August 2020 written response, the agency reported that there continue to be efforts to improve tracking of funds for regulatory science projects in priority areas. Specifically, it described the development of an accountability framework-an internal document outlining the type of information that FDA centers and offices will provide to the agency's Chief Scientist to demonstrate progress made in addressing relevant focus areas of regulatory science. According to FDA, as part of the internal accountability framework, it will track regulatory science projects. FDA reported that its goal was to complete this work by the end of December 2020. The agency indicated that the recommendation should remain open, and GAO will continue to monitor the implementation of this recommendation.