Measuring Performance:
Challenges in Evaluating Research and Development
T-RCED-97-130: Published: Apr 10, 1997. Publicly Released: Apr 10, 1997.
Additional Materials:
- Full Report:
Contact:
(202) 512-6082
contact@gao.gov
Office of Public Affairs
(202) 512-4800
youngc1@gao.gov
GAO discussed its work related to the National Institute of Standards and Technology's Advanced Technology Program (ATP), Manufacturing Extension Partnership (MEP) Program, and research and development (R&D) performance measures.
GAO noted: (1) ATP has funded research projects that would have been funded by the private sector as well as those that would not; (2) the award recipients were nearly evenly divided when asked if they would have pursued their projects if they had not received such funding; (3) GAO also found that in most cases, the participants in its survey did not look for funding from other sources, private or public, before trying to obtain funding from ATP; (4) about half of the 45 applicants that tried to find funding elsewhere before turning to ATP were told by prospective funders that their projects were either too risky or "precompetitive", characteristics that fulfill the aims of ATP; (5) manufacturers viewed the manufacturing extension programs' services positively, as was demonstrated in GAO's national survey of manufacturers who had received substantive services from the programs in 1993; (6) most manufacturers responding to GAO's questionnaire, about 73 percent, reported that they believed that the type of assistance they had received from these programs had positively affected their overall business performance; (7) about 15 percent of the respondents reported that they believed the programs' assistance had not affected their overall business performance; (8) the amount of money spent on R&D, the primary indicator of research investment, is useful as an input measure of how much research is being performed; (9) however, the level of spending is not a reliable indicator of research results; (10) GAO found that there is no primary indicator of R&D results; (11) the companies that GAO spoke with collect data on various output indicators, such as return on investment and patents granted, but in general make limited use of them in their investment decisions; (12) instead, they emphasized that R&D contributes directly to the bottom line; (13) because companies are profit oriented, many of the indicators tracked by the private sector cannot be directly applied to the federal government; (14) determining the specific outcomes resulting from federal R&D is a challenge that will not be easily resolved; and (15) however, in response to recent legislation requiring agencies to report on program results, some progress is being made in measuring the impacts of research.
Dec 21, 2020
-
Science & Tech Spotlight:
Tracing the Source of Chemical WeaponsGAO-21-271SP: Published: Dec 21, 2020. Publicly Released: Dec 21, 2020.
Dec 17, 2020
-
Federal Research:
Agencies Need to Enhance Policies to Address Foreign InfluenceGAO-21-130: Published: Dec 17, 2020. Publicly Released: Dec 17, 2020.
Dec 7, 2020
-
Science & Tech Spotlight:
Air Quality SensorsGAO-21-189SP: Published: Dec 7, 2020. Publicly Released: Dec 7, 2020.
Nov 30, 2020
-
Artificial Intelligence in Health Care:
Benefits and Challenges of Technologies to Augment Patient CareGAO-21-7SP: Published: Nov 30, 2020. Publicly Released: Nov 30, 2020.
Nov 20, 2020
-
Biomedical Research:
NIH Should Publicly Report More Information about the Licensing of Its Intellectual PropertyGAO-21-52: Published: Oct 22, 2020. Publicly Released: Nov 20, 2020.
Sep 3, 2020
-
Defense Science and Technology:
Opportunities to Better Integrate Industry Independent Research and Development into DOD PlanningGAO-20-578: Published: Sep 3, 2020. Publicly Released: Sep 3, 2020.
Aug 31, 2020
-
Science & Tech Spotlight:
Consumer Electronics RecyclingGAO-20-712SP: Published: Aug 31, 2020. Publicly Released: Aug 31, 2020.
Aug 11, 2020
-
Facial Recognition Technology:
Privacy and Accuracy Issues Related to Commercial UsesGAO-20-522: Published: Jul 13, 2020. Publicly Released: Aug 11, 2020.
Jul 30, 2020
-
Covid-19:
Data Quality and Considerations for Modeling and AnalysisGAO-20-635SP: Published: Jul 30, 2020. Publicly Released: Jul 30, 2020.
Jul 28, 2020
-
Science & Tech Spotlight:
Contact Tracing AppsGAO-20-666SP: Published: Jul 28, 2020. Publicly Released: Jul 28, 2020.
Looking for more? Browse all our products here