Small Business Innovation Research:
Agencies Need to Strengthen Efforts to Improve the Completeness, Consistency, and Accuracy of Awards Data
GAO-07-38, Oct 19, 2006
The Small Business Innovation Research (SBIR) program was created to increase the use of small businesses to meet federal research needs and commercialize the results of this research. To monitor the program, the Small Business Administration (SBA) requires participating agencies to provide, in a standard format, specific data on all SBIR awards they make. SBA then compiles these data into a database known as Tech-Net. Congress also required SBA to create, by 2001, a restricted and more comprehensive database that would provide information for government agencies to use in evaluating the program. GAO was asked to identify the (1) types of data that agencies report to SBA for inclusion in the Tech-Net database, (2) extent to which these data are provided in a standard format, (3) extent to which SBA has established the government-use database, and (4) extent to which SBIR agencies have developed and implemented techniques to track commercialization of SBIR projects. GAO reviewed 8 of the 11 agencies participating in SBIR.
Federal agencies participating in the SBIR program annually submit over 40 data elements to SBA for each award they make. These data include information on the award, such as value and a descriptive abstract; information on the recipient, such as name and gender; and information about the firm receiving the award, such as number of employees and location. Participating agencies submit most of the information required by SBA, but they are not consistently providing all required data elements, including the number of employees in the firm, and the gender and socio-economic status of the award recipient, resulting in incomplete sections of the database. Agencies stated that this happens because they do not collect all of the information that SBA wants and because SBA's requirements change regularly. Some participating agencies are not submitting SBIR award data in the standard format required by SBA, and although SBA's quality assurance processes correct most of these problems, they do not correct all of them. In 2004 and 2005, about 25 percent of the data provided by five participating agencies did not comply with SBA's format. Formatting inconsistencies occur because the template SBA has provided agencies for reporting data can be edited. According to SBA, identifying and correcting inconsistently formatted data involves considerable resources, therefore the agency has focused its quality assurance efforts only on key data elements needed to track awards; other fields, such as those containing demographic data, are generally not corrected. As a result, comprehensive program evaluations may be limited by the quality of the data in these fields. SBA officials expect this problem to be resolved by fiscal year 2007, when all data will be submitted via an Internet interface that will not allow changes in the format. SBA is 5 years behind schedule in meeting the congressional mandate to implement a restricted government-use database for the SBIR program. SBA had planned to meet this requirement by expanding its Tech-Net database to include a restricted government-use section. SBA officials attributed the delay in meeting the 2001 deadline primarily to increased security requirements needed for the database, agency management changes, and budgetary constraints. SBA officials expect the government-use section of Tech-Net to be operational by October 1, 2006, when safeguards to protect the proprietary commercialization information in the database are in place. Most agencies GAO reviewed systematically gather data on the commercialization success of SBIR-funded projects. Five of these eight agencies regularly survey all awardees to gather information on program participation, including commercial success, and one agency is about to start a similar survey. In contrast, two agencies only gather anecdotal success stories from a small sample of SBIR awardees. SBA and agency officials generally agree that despite their best efforts, obtaining commercialization information from awardees remains a major challenge.
- Closed - implemented
- Closed - not implemented
Recommendation for Executive Action
Recommendation: The Administrator, SBA, and the SBIR participating agencies should work together to strengthen efforts to ensure that the data collected for SBA's Tech-Net database are complete, consistent, and accurate.
Agency Affected: Small Business Administration
Status: Closed - Not Implemented
Comments: This recommendation is closed but not implemented. By May 2009, SBA, EPA, USDA, and NSF had implemented the recommendation. However, to close the recommendation as implemented, needed responses from at least 6 agencies that account for about 97 percent of the data. DOD, DOE, NASA, NIH, and NIST have not responded with information on their actions in response to this recommendation.