Evaluation of Small Business Innovation Research Can Be Strengthened
RCED-99-114, Jun 4, 1999
Pursuant to a congressional request, GAO provided information on the Small Business Innovation Research (SBIR) Program, focusing on: (1) the distribution of awards by company and geographic area, with special emphasis on the share of awards received by the 25 most frequent winners; (2) the extent to which federal agencies are considering commercial potential and the program's other goals in making their awards; and (3) previous evaluations of the SBIR program to identify opportunities to improve measurements of the program's outcomes.
GAO noted that: (1) from fiscal year (FY) 1983 through FY 1997, the 25 most frequent winners received over $900 million, or about 11 percent of the program's awards; (2) these companies represent fewer than 1 percent of all the companies that have received awards; (3) the program has a high number of first-time participants; (4) one-third of the companies receiving awards from FY 1993 through FY 1997 were first-time winners, indicating that the program is attracting hundreds of new companies annually; (5) SBIR awards are concentrated in certain states, largely because companies in these states submitted the most proposals; (6) to broaden the geographic distribution of awards, agencies have made efforts to encourage the submission of proposals from companies in states with fewer awards; (7) in response to the 1992 reauthorization, agencies are considering commercial potential as an explicit criterion when evaluating proposals; (8) the reauthorization does not clarify how a company's commercialization record and the program's other goals should be used in evaluations of proposals; (9) this lack of clarity has led to differences across agencies in how they evaluate proposals; (10) early tests of the Department of Defense's (DOD) plan indicated that some of the most frequent winners that have been relatively unsuccessful in commercializing their research results would not have been penalized if only a few of their awards had resulted in sales; (11) at the same time, companies with far fewer awards and no previous sales might have been subject to penalties; (12) although DOD has revised its plan to avoid this problem, the lack of clarity in the legislation remains a concern; (13) federal agencies and others have used various methods to evaluate the program's commercial outcomes; (14) these methods have used snapshots of sales, data on additional developmental funding for the projects, success stories, and other indications of commercial success; (15) however, they become quickly outdated and do not provide an ongoing, consistent, and programwide record; (16) the use of a single method with uniform criteria for success focusing on commercial outcomes and other indicators of success would help to satisfy the requirements of the Government Performance and Results Act of 1993; (17) the Small Business Administration is developing a new database called Tech-Net; and (18) Tech-Net affords an opportunity to maintain current, consistent information about commercial and other outcomes and responds to the Results Act.
- Closed - implemented
- Closed - not implemented
Matter for Congressional Consideration
Matter: When Congress considers the reauthorization of this program, it may wish to clarify the relative emphasis that agencies, in evaluating companies' proposals, should give to a company's commercialization record as part of the goal of commercialization and to the program's other goals. This clarification would help ensure uniformity in the program and a clear set of standards by which to determine whether, and to what extent, commercialization and the program's other goals should be considered in evaluations of proposals.
Status: Closed - Implemented
Comments: As of September 1, 1999, congressional action on reauthorization legislation had not begun. As of September 2000, the House and Senate have agreed on language to increase the emphasis on commercialization by requiring companies to include a business plan with their proposals. This partially responds to GAO's suggestion that Congress consider clarifying the relative emphasis on SBIR goals, but does not focus specifically on GAO's concern about the commercialization record. The recommendation should be closed.
Recommendation for Executive Action
Recommendation: To respond to the Government Performance and Results Act, the Administrator, Small Business Administration, should develop standard criteria for measuring commercial and other outcomes of the SBIR program and incorporate these criteria into the new Tech-Net database. The criteria should include uniform measures of sales, developmental funding, and other indicators of success.
Agency Affected: Small Business Administration
Status: Closed - Implemented
Comments: In the spring of 2001, the Small Business Administration began implementing a new SBIR reporting system that will measure the program's commercialization success. In fiscal year 2002, SBA further enhanced the reporting system to include commercialization results that will establish an initial baseline commercialization rate. Small business firms participating in the SBIR program are providing information annually on sales and investments associated with their SBIR projects.