Science, Math & Engineering, Inc.

Decision

Matter of: Science, Math & Engineering, Inc.

File: B-410509

Date: January 7, 2015

DIGEST

Protester’s challenge to agency’s decision not to fund protester’s application under Phase I of a Solicitation issued pursuant to the Department of Education Small Business Innovation Research program is denied where protester’s application was reasonably found to have weaknesses that resulted in it being ranked 51st of the 63 applications received.

Science, Math & Engineering, Inc. (SME), of Billerica, Massachusetts, protests the determination of the Department of Education[1] not to fund SME’s Phase I proposal under Small Business Innovation Research (SBIR) program solicitation No. 84.133S-1, which invited offers for research projects to address the needs of individuals with disabilities.

We deny the protest.

BACKGROUND

The SBIR program is designed to increase the participation of small business concerns in federally funded research or research and development (R&D). See Small Business Innovation Research Program Act of 1982, 15 U.S.C. § 638 (2006). Pursuant to this authority, certain federal agencies, including the Department of Education, are required to provide a program under which a portion of the agency’s research or R&D effort is reserved for award to small business concerns through a three-phased process. See 15 U.S.C. §§ 638(e)(4), (f).

Under Phase I, small businesses are invited to test the scientific, technical, and commercial merit and feasibility of a certain concept. If Phase I is successful, the firm may apply for a Phase II award to further develop the concept. During Phase III, firms are expected to obtain funding from non-SBIR sources to develop the concept into a product for commercial and/or military markets. See Solicitation at 5.

The solicitation here sought Phase I applications for projects that would address the needs of individuals with disabilities. Project activities could include conducting manufacturing-related research and development that encompasses improvements in existing methods or processes, or wholly new processes, machines, or systems, that would benefit individuals with disabilities. The solicitation indicated that the agency was particularly interested in applications that addressed one or more of the following:

(1) Increased independence of individuals with disabilities in community settings, including educational settings, through the development of technology to support access to these settings and promote integration of individuals with disabilities;

(2) Enhanced sensory or motor function of individuals with disabilities through the development of technology to support improved functional capacity;

(3) Enhanced workforce participation through the development of technology to increase access to employment, and support employment advancement for individuals with disabilities;

(4) Enhanced community living and participation for individuals with disabilities through the development of accessible information technology including cloud computing, software, systems, and devices that promote access to information in educational, employment, and community settings, and voting technology that improves access for individuals with disabilities; and

(5) Improved health-care interventions and increased use of related resources through development of technology to support independent access to community health-care services for individuals with disabilities.

Solicitation at 40-41.

The Solicitation provided that applications would be evaluated on a 100-point scale, allocating points as follows: (1) importance of the problem (20 points); (2) design of development activities (50 points);[2] (3) project staff (15 points); (4) adequacy and reasonableness of budget (5 points); and (5) adequacy and accessibility of resources (10 points). Solicitation at 22-23. The Solicitation noted that the agency planned to make 10 awards. Id. at 24.

Applicants were to “describe the approaches they expect[ed] to use to collect empirical evidence demonstrating the effectiveness of the technology they [were] proposing.” Id. at 41. The Solicitation further noted that “[t]his empirical evidence should facilitate the efficacy and usefulness of the technology.” Id. For example, with respect to the most important criterion--design of development activities--applicants were informed that the agency would consider the proposed plan for development, clinical testing, and evaluation of proposed devices, as well as whether the proposed product would use the most effective technical approach, with a demonstrated awareness of the state-of-the-art in technology. Id. at 22.

The agency received 87 Phase I applications, of which it evaluated 63,[3] including SME’s application. SME’s application was entitled, “Hazard Warning and Obstacle Avoidance System for Wheelchair Users.” AR, Tab C, attach. 3, SME Application, at 76-148.[4] SME’s proposal included a technical abstract that described its proposed project as follows:

This project will conduct research and development to provide individuals who use wheelchairs an effective and affordable universally designed hazard warning and obstacle avoidance system that is adaptable to the user’s needs and medical condition. The goal is to provide a safer environment of travel for the wheelchair user.

AR, Tab C, SME’s Application, at 1. SME identified as its anticipated Phase I result “design of a hazard warning and obstacle avoidance system that is unobtrusive to the user and does not encumber normal wheelchair travel.” Id.

Applications were reviewed by six evaluation panels, each consisting of five experts from the disability field. GASF at 4. Each panel, as a group, reviewed at least ten applications. Id. The agency reviewers found that SME’s application was vague and broadly written concerning the methods it would use and how they would be applied; did not provide any narrative detail of clinical testing; proposed technologies that already existed; and seemed to ignore significant similar types of products already being used in other fields. AR, Tab C, attach. 4, Review Panel Evaluations (RPEs), at 153, 159, 166, 172, and 178.

SME’s application contained very general concepts, without explaining their applicability to the program. For example, the application listed various sensors and stimulators based upon existing technology that SME would consider for use with wheelchairs, but did not provide any detail of how these products would be tested and applied to the problem being addressed. AR, Tab C, SME Application, at 107‑113. As an example of the lack of depth or specificity, SME generally stated that it would implement its efforts “using a recurrent process of incremental design followed by testing and characterization. . . .,” and cited a series of very general milestones. Id.[5]

The evaluation panel members gave SME’s application an average of 27.6 out of a possible 50 points under the most important factor, design of development activities (factor 2). AR, Tab C, attach. 4, RPEs, at 150, 156, 169, and 175. SME’s application received a total score of 63, and was ranked 51st among the 63 reviewed applications. GASF at 4. The 10 highest-scoring applications were recommended for funding. Among these applications, the lowest average score received was 89.9 points. Id.

DISCUSSION

SME disputes the agency’s evaluation, asserting that its application should have been selected because it met or exceeded all of the agency’s requirements. According to SME, the agency’s evaluation was “not objective, did not acknowledge the actual content of SME’s application and was not based on merit.” Protest at 2. In its comments, SME also contends that the agency evaluated its application disparately from those recommended for award. Comments at 4-6.

In reviewing protests against allegedly improper evaluations, it is not our role to reevaluate proposals. Rather, our Office examines the record to determine whether the agency’s judgment was reasonable and in accord with the evaluation criteria. Abt Assocs. Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD ¶ 223 at 4. Such judgments are by their nature often subjective; nevertheless, the exercise of these judgments in the evaluation of proposals must be reasonable and bear a rational relationship to their announced criteria upon which competing offers are to be selected. Southwest Marine, Inc.; Am. Sys. Eng'g Corp., B-265865.3, B-265865.4, Jan. 23, 1996, 96-1 CPD ¶ 56 at 10. The protester’s disagreement with the agency’s judgment, by itself, does not establish that an evaluation was unreasonable. UNICCO Gov't Servs., Inc., B-277658, Nov. 7, 1997, 97-2 CPD ¶ 134 at 7. Moreover, the agency is accorded considerable discretion to determine which proposals will be funded under the SBIR program. Virginia Accelerators Corp., B-271066, May 20, 1996, 97-2 CPD ¶ 13 at 2.

On this record, we find no basis to conclude that the agency acted improperly in deciding not to select SME’s application for funding. As noted above, the solicitation stated that applications were to provide empirical evidence demonstrating the effectiveness of the proposed techniques. Solicitation at 40-41. Here, as illustrated by the example above, the agency reasonably concluded that SME provided very general concepts of sensors and stimulators, all of which were based on existing technology, and did not discuss in any detail any proposed testing.

The agency identified and documented its assessment of various weaknesses in SME’s application, consistent with the solicitation’s stated evaluation criteria. The fact that the protester disagrees with the agency’s technical experts’ conclusions regarding the technical merit of SME’s application does not invalidate the reasonableness of the agency’s evaluation. This is particularly true under an SBIR procurement, which is not based on design or performance specifications for existing equipment, but rather emphasizes scientific and technical innovation and has as its objective the development of new technology. It is precisely because of the experimental and creative nature of this type of procurement that the contracting agency is given substantial discretion in determining which proposals it will fund. See, e.g., Noise Cancellation Technologies, Inc., B-246476, B-246476.2, Mar. 9, 1992, 92‑1 CPD ¶ 269.

Given the reasonably assessed weaknesses in SME’s application, there is no basis to conclude that the agency acted unreasonably in choosing not to award Phase I funding to SME.

The protest is denied.[6]

Susan A. Poling
General Counsel