Evaluation of Small Business Innovation Research Can Be Strengthened
T-RCED-99-198: Published: Jun 17, 1999. Publicly Released: Jun 17, 1999.
Pursuant to a congressional request, GAO discussed the results of its review of the Small Business Innovation Research (SBIR) program, focusing on: (1) the distribution of awards by company and geographic area, with special emphasis on the share of awards received by the 25 most frequent winners; (2) the extent to which federal agencies are considering commercial potential and the program's other goals in making their awards; and (3) previous evaluations of the SBIR program to identify an opportunity to improve measurements of the program's outcomes.
GAO noted that: (1) the 25 most frequent winners, which represent fewer than 1 percent of the companies in the program, received about 11 percent of the program's awards from fiscal year (FY) 1983 through FY 1997; (2) these companies accumulated over $900 million in total awards; the leading frequent winner received over $108 million; (3) however, one-third of the companies receiving awards from FY 1993 through FY 1997 were first-time winners, indicating that the program is attracting an average of 750 new companies annually; (4) in GAO's view, this level of participation by first-time winners is indicative of a substantial capacity to attract new participants each year; (5) in response to the 1992 reauthorization, agencies are considering commercial potential as a criterion when evaluating proposals and collecting data on commercialization by frequent winners; (6) however, the reauthorization does not clarify how much weight should be given to the commercialization record and how much weight should be given to the program's other goals; (7) this lack of clarity has led to differences in agencies' evaluation approaches; (8) for example, using an approach shared by none of the other agencies, the Department of Defense planned to give significantly lower scores to companies perceived as poor commercializers, but GAO found that this approach would penalize companies with relatively few awards and no sales but would not penalize frequent winners with limited sales; (9) Defense has revised its approach to avoid these unintended consequences; (10) GAO's report raises as a matter for congressional consideration how the commercialization record as part of the goal of commercialization should be balanced against the program's other goals in evaluations of proposals; (11) federal agencies and others have relied on various methods to evaluate the program's commercial outcomes; (12) these methods have used snapshots of sales, data on additional developmental funding for projects, success stories, and other indicators of success; (13) however, they become quickly outdated and do not provide an ongoing, consistent, and programwide record; (14) the use of a single method with uniform criteria for success focusing on commercial and other outcomes would help to satisfy the requirements of the Government Performance and Results Act; and (15) the Small Business Administration is developing a new database called Tech-Net, which affords an opportunity to maintain consistent information on commercial outcomes and other indicators of success.