Measuring Performance:

Challenges in Evaluating Research and Development

T-RCED-97-130: Published: Apr 10, 1997. Publicly Released: Apr 10, 1997.

Additional Materials:

Contact:

Victor S. Rezendes
(202) 512-6082
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

GAO discussed its work related to the National Institute of Standards and Technology's Advanced Technology Program (ATP), Manufacturing Extension Partnership (MEP) Program, and research and development (R&D) performance measures.

GAO noted: (1) ATP has funded research projects that would have been funded by the private sector as well as those that would not; (2) the award recipients were nearly evenly divided when asked if they would have pursued their projects if they had not received such funding; (3) GAO also found that in most cases, the participants in its survey did not look for funding from other sources, private or public, before trying to obtain funding from ATP; (4) about half of the 45 applicants that tried to find funding elsewhere before turning to ATP were told by prospective funders that their projects were either too risky or "precompetitive", characteristics that fulfill the aims of ATP; (5) manufacturers viewed the manufacturing extension programs' services positively, as was demonstrated in GAO's national survey of manufacturers who had received substantive services from the programs in 1993; (6) most manufacturers responding to GAO's questionnaire, about 73 percent, reported that they believed that the type of assistance they had received from these programs had positively affected their overall business performance; (7) about 15 percent of the respondents reported that they believed the programs' assistance had not affected their overall business performance; (8) the amount of money spent on R&D, the primary indicator of research investment, is useful as an input measure of how much research is being performed; (9) however, the level of spending is not a reliable indicator of research results; (10) GAO found that there is no primary indicator of R&D results; (11) the companies that GAO spoke with collect data on various output indicators, such as return on investment and patents granted, but in general make limited use of them in their investment decisions; (12) instead, they emphasized that R&D contributes directly to the bottom line; (13) because companies are profit oriented, many of the indicators tracked by the private sector cannot be directly applied to the federal government; (14) determining the specific outcomes resulting from federal R&D is a challenge that will not be easily resolved; and (15) however, in response to recent legislation requiring agencies to report on program results, some progress is being made in measuring the impacts of research.

Dec 21, 2020

Dec 17, 2020

Dec 7, 2020

  • science icon, source: National Cancer Institute

    Science & Tech Spotlight:

    Air Quality Sensors
    GAO-21-189SP: Published: Dec 7, 2020. Publicly Released: Dec 7, 2020.

Nov 30, 2020

Nov 20, 2020

Sep 3, 2020

Aug 31, 2020

Aug 11, 2020

Jul 30, 2020

Jul 28, 2020

  • science icon, source: National Cancer Institute

    Science & Tech Spotlight:

    Contact Tracing Apps
    GAO-20-666SP: Published: Jul 28, 2020. Publicly Released: Jul 28, 2020.

Looking for more? Browse all our products here