Recommendation: The Secretary of Commerce should direct the National Institute of Standards and Technology (NIST) Director to work with other sponsoring federal agencies to develop and implement network-wide performance goals for the Manufacturing USA program with measurable targets and time frames. (Recommendation 1)

Agency: Department of Commerce
Status: Open

Comments: Commerce partially concurred with this recommendation according to its response to our report. Commerce stated that it lacks the legal authority to compel action by other federal agencies, and that legal constraints aside, it believes it is bad management practice to ask institutes to respond to performance goals issued by different bodies. As of March 2020, Commerce stated that it plans to report on performance goals for Commerce-sponsored institutes effective with the 2019 annual report, which it expects to issue by September 2020. Commerce also agreed to continue working closely with other sponsoring federal agencies on program and network goals for the Manufacturing USA institutes, but did not commit to working with sponsoring federal agencies to develop and implement network-wide performance goals with measurable targets and time frames. We recognize that Commerce does not have management authority over other the institutes sponsored by other agencies. We believe our report sufficiently characterizes the development of network-wide performance goals, targets, and time frames as a collaborative effort between Commerce and sponsoring agencies that is in keeping with Commerce's network-wide coordination functions under the RAMI Act. Moreover, our recommendation specifically pertained to developing performance goals for the Manufacturing USA program, not individual institutes. As we stated in our report, this would not necessarily entail new performance measures but, rather, could consist of measurable near-term performance goals corresponding to program performance measures already in place. Further, as stated in our report, GAO's prior work has shown that systems of performance measures benefit from certain key practices, such as creating a hierarchy that breaks down broad, long-term goals and objectives into more specific, near-term performance goals with measurable targets and time frames. Our recommendation was designed to ensure that the Manufacturing USA program performance measurement structure that Commerce has already worked with the other sponsoring agencies to develop more fully aligns with these key practices. We continue to believe that by working with other sponsoring federal agencies to develop and implement network-wide performance goals with targets and time frames, Commerce would be better able to observe and report on progress toward long-term Manufacturing USA program goals and objectives.

Recommendation: The Secretary of Commerce should direct the NIST Director to work with other sponsoring federal agencies to ensure that the Manufacturing USA network-wide performance measures are directly aligned with the network-wide performance goals, the Manufacturing USA strategic objectives and program goals, and the statutory purposes of the Revitalize American Manufacturing and Innovation (RAMI) Act. (Recommendation 2)

Agency: Department of Commerce
Status: Open

Comments: Commerce partially concurred with this recommendation according to its response to our report. In March 2020, Commerce stated that it supports the alignment of performance measures with performance goals only for Commerce-sponsored institutes. Commerce stated that it is unable to commit to this recommendation as 13 of the 14 existing institutes were authorized under authorities other than the RAMI Act and are sponsored by agencies other than Commerce. Commerce also stated that, until additional institutes authorized by the RAMI Act are in place, it does not support additional performance measures for the single Commerce-sponsored institute beyond the RAMI Act requirements, as doing so would impose an unfair level of scrutiny. Commerce agreed to report on performance metrics for department-sponsored institutes effective with the current annual report, which is expected by September 2020. We recognize that Commerce does not have management authority over other the institutes sponsored by other agencies. We believe our report sufficiently characterizes the effort to align the network-wide performance measures with network-wide performance goals and Manufacturing USA program goals as a collaborative effort between Commerce and sponsoring agencies that is in keeping with Commerce's coordination functions under the RAMI Act. Our recommendation does not ask Commerce to compel actions by other agencies, nor to develop any additional performance measures. As noted in our report, the Manufacturing USA program's performance measurement structure aligns near-term performance measures directly to the program's long-term goals. This structure bypasses connecting the performance measures with the program's objectives that have been developed to break down the long-term goals more specifically. GAO's prior work has shown that systems of performance measures benefit from certain key practices, such as creating a hierarchy that breaks down broad, long-term goals and objectives into more specific, near-term performance goals with measurable targets and time frames. Our recommendation was designed to ensure that the Manufacturing USA program performance measurement structure that Commerce has already worked with the other sponsoring agencies to develop more fully aligns with these key practices. We continue to believe that by working with other sponsoring federal agencies to ensure that the Manufacturing USA network-wide performance measures are directly aligned with the Manufacturing USA strategic program goals and objectives and the statutory purposes of the RAMI Act, Commerce would be better able to observe and report on progress made toward achieving the statutory purposes of the Manufacturing USA program.

Recommendation: The Secretary of Commerce should direct the NIST Director to develop criteria to evaluate whether the Commerce-sponsored institute can sustain its operations without additional federal financial assistance after its initial agreement. If an analysis based on such criteria indicates that additional federal financial assistance is needed to help the institute sustain its operations, then the Secretary of Commerce should consider a legislative proposal to amend relevant provisions of the RAMI Act. (Recommendation 3)

Agency: Department of Commerce
Status: Open

Comments: Commerce partially concurred with this recommendation according to its response to our report. In March 2020, Commerce stated that it will develop criteria to evaluate the sufficiency of the Commerce-sponsored institute's sustainability plan based on the anticipated operating costs of the institute at fully operational steady state, and the likelihood of sustaining those operations through the specific efforts outlined in the sustainability plan. Commerce stated that it plans to develop the evaluation criteria by April 2020. We will update this recommendation after we learn more about these efforts.

Recommendation: The Secretary of Defense should direct the Director of DOD's Manufacturing USA institutes to develop criteria to evaluate whether DOD-sponsored institutes can sustain their operations without additional federal financial assistance after their initial agreements. (Recommendation 4)

Agency: Department of Defense
Status: Open

Comments: DOD partially concurred with this recommendation. According to information DOD provided in March 2020, DOD developed criteria to evaluate whether each DOD-sponsored institute is effectively executing its mission, providing value to the department, and transitioning advanced manufacturing to U.S. manufacturers, while demonstrating progress toward business viability (diversified revenue, controlled costs, etc.). As of March 2020, DOD plans to incorporate these criteria into a strategic management plan, under which the department will review institutes' progress at the end of their agreements to determine the type and level of DOD's continued participation. DOD expects to complete work on the strategic management plan by the end of September 2020.

Recommendation: The Secretary of Energy should direct the Director of DOE's Manufacturing USA institutes to develop criteria to evaluate whether DOE-sponsored institutes can sustain their operations without additional federal financial assistance after their initial agreements. (Recommendation 5)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with this recommendation according to its response to our report. In September 2019, DOE stated that it will direct the Directors of DOE-sponsored institutes to collectively work toward updating institute sustainability plans and activities and collaboratively develop criteria and metrics to assess the institutes' progress toward financial sustainability. After development of the metrics, DOE will track as appropriate. As of March 2020, DOE reported agreement with its institutes on an initial set of criteria and metrics to assess progress toward financial sustainability. We will update this recommendation as we collect more information about these efforts.

Recommendation: As the Subcommittee on Quantum Information Science (QIS Subcommittee) moves forward, the Office of Science and Technology Policy co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 1)

Agency: Executive Office of the President: Office of Science and Technology Policy
Status: Open

Comments: The Office of Science and Technology Policy (OSTP) agreed with GAO's September 2018 recommendation despite expressing some concerns about required resources and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019, the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the Department of Commerce co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 2)

Agency: Department of Commerce
Status: Open

Comments: The Department of Commerce agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the Department of Energy co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 3)

Agency: Department of Energy
Status: Open

Comments: The Department of Energy agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the QIS Subcommittee moves forward, the National Science Foundation co-chair, in coordination with other co-chairs and participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 4)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation agreed with GAO's September 2018 recommendation and, as of January 2020, had taken steps to work with the other co-chairs of the National Science and Technology Council's Quantum Information Science (QIS) Subcommittee to begin implementing it. The QIS Subcommittee, created pursuant to the National Quantum Initiative Act, enacted in 2018, continues to be led by four co-chairs from the Department of Commerce's National Institute of Standards and Technology, Department of Energy, National Science Foundation (NSF), and OSTP. The law requires, among other things, that the QIS Subcommittee develop a 5-year Strategic Plan by December 21, 2019. In January 2020, an NSF official and OSTP staff reported that a draft strategic plan was under review. According to the NSF official, the strategic plan will include an assessment of actions the agencies are taking in support of QIS, and, in particular, the degree to which the agencies have developed mechanisms that enhance and sustain collaboration. The official said the draft plan will be submitted to the National Quantum Initiative Advisory Committee, which conducts independent assessments of and advises the President and QIS Subcommittee on matters related to the National Quantum Initiative. The NSF official reported that the membership of the Advisory Committee will be announced in the spring 2020 timeframe, at which time it will begin its review of the draft strategic plan. In addition to the development of a strategic plan, the National Quantum Initiative Act called for the establishment of a National Quantum Coordination Office to support the QIS Subcommittee, which OSTP formed in March 2019. Following this, in Oct. 2019 the QIS Subcommittee created three interagency working groups: (1) the science working group is working to coordinate the scientific and technical aspects of programs; (2) the workforce, infrastructure, and industry working group is working to identify workforce and technology needs; and (3) the end-user group is working to connect the nation's research and development community, including academics and industry players, to potential early adopters in the federal government. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in quantum computing. When the strategic plan is finalized and we confirm what additional actions the QIS Subcommittee has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: As the Interagency Working Group on Synthetic Biology moves forward, the Director of the National Science Foundation, in coordination with participating agency officials, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 5)

Agency: National Science Foundation
Status: Open

Comments: The National Science Foundation (NSF) agreed with GAO's September 2018 recommendation and, as of January 2020, had taken some steps to implement it. In November 2018, the Interagency Working Group on Synthetic Biology was formally established under the Biological Sciences Subcommittee of the National Science and Technology Council. The co-chairs of the Interagency Working Group on Synthetic Biology are officials from the National Institutes of Health, the National Institute of Standards and Technology, and NSF. The charter for the working group states that the group is to facilitate coordination and collaboration across 16 federal agencies. In October 2019, the working group hosted an Interagency Synthetic Biology Workshop to examine a roadmap that included basic science, enabling technologies, infrastructure and workforce needs in the area of synthetic biology. The workshop included 100 participants across the federal government, academia and industry, according to NSF officials. On the final day of the workshop participants from federal agencies used the input from the workshop to prepare a list of priority areas for investment along with agencies interested in participating in those priority areas. In January 2020, NSF officials reported that among the next steps for the working group was to develop a federal strategic roadmap for synthetic biology. Officials also reported that the working group is actively preparing a memorandum of understanding to create policies that will enable more sharing of information and collaboration. Taking this action will help to enhance and strengthen interagency collaboration and could help ensure that agencies effectively marshal their efforts to maintain U.S. competitiveness in synthetic biology. When we confirm what additional actions the working group has taken to fully implement leading practices that enhance and sustain collaboration, we will provide updated information.

Recommendation: The NIST Associate Director for Laboratory Programs should update NIST policy to include periodic comprehensive management review of the agency's measurement services to assess gaps and ensure alignment with stakeholders' needs, and take steps to ensure that the Associate Director completes the review of NIST's standards development activities (Recommendation 1).

Agency: Department of Commerce: National Institute of Standards and Technology
Status: Open

Comments: NIST concurred with this recommendation and, as of May 2020, had taken some steps to implement it. Specifically, NIST provided information indicating it uses multiple information sources to identify potential measurement service gaps. For example, the Associate Director for Laboratory Programs (ADLP) reviews quarterly reports from NIST's laboratory divisions that include information on measurement services. The ADLP may identify measurement service gaps as part of this review. Also, the NIST Measurement Services Council serves the ADLP in an advisory role to identify and address NIST-wide issues related to the quality, relevance, performance, operations, and resources allocated to the health and improvement of NIST measurement services. The Council produces an annual report that addresses the health of NIST's measurement services, including potential changes needed to meet future customer needs. Further, NIST employees may suggest new services through processes established in suborder 5901.01. Many of these efforts also include consideration of potential gaps in NIST's participation in standards development activities. Additionally, NIST Order 5301.00 delegates responsibility to review standards activities and participation across several levels of NIST management. Although these actions may help identify gaps in NIST's participation in standards development activities as well as identify gaps in the measurement services it provides, it is not clear how or whether they fulfill the periodic review of the effectiveness of NIST's participation in documentary standards activities that the ADLP is to conduct under NIST's standards participation policy. We will update our evaluation of NIST's implementation of this recommendation when the agency provides additional information on how the activities described above fulfill the effectiveness review called for by NIST's policy, or provides information documenting that the ADLP has conducted such a review.