Strengths and Limitations of Research Indicators
RCED-97-91: Published: Mar 21, 1997. Publicly Released: Apr 10, 1997.
Pursuant to a congressional request, GAO provided information on the indicators used to evaluate the results of research and development (R&D), focusing on the relative strengths and limitations of the input and output indicators used by the federal and private sectors to measure the results of R&D.
GAO noted that: (1) the amount of money spent on research and development, the primary indicator of the investment in research, is useful as a measure of how much research is being performed; (2) having been refined over many years, these data are generally available for the research efforts in both the public and private sectors; (3) however, the level of spending is not a reliable indicator of the level of results achieved by research; (4) unlike the situation with the input measures of R&D, there is no primary indicator of the outputs; (5) output indicators include quantitative analyses of return on investment, patents granted, and other outputs as well as qualitative assessments based on peer review; (6) the companies that GAO spoke with collect data on various output indicators but, in general, make limited used of them in their investment decisions; (7) instead, the companies emphasized that R&D contribute directly to their "bottom line"; (8) because companies are profit-oriented, many of the indicators tracked by the private sector cannot be directly applied to the federal government; and (9) experiences from pilot efforts made under the Government Performance and Results Act have reinforced the finding that output measures are highly specific to the management and mission of each federal agency and that no single indicator exists to measure the results of research.