Quimba Software

Decision

Matter of: Quimba Software

File: B-299000

Date: January 18, 2007

DIGEST

Protest that agency improperly evaluated proposals under Small Business Innovation Research program on basis of undisclosed evaluation factors is denied where record shows that evaluation was consistent with and encompassed by the solicitation evaluation criteria.

Quimba Software protests the Department of the Army's determination not to fund its phase I proposal under Department of Defense (DOD) Small Business Innovation Research (SBIR) program solicitation No. 2006.2. Quimba alleges that the Army failed to evaluate proposals and make award in accordance with the solicitation's stated evaluation factors.

We deny the protest.

The SBIR program is conducted pursuant to the Small Business Innovation Development Act, 15 U.S.C. sect. 638 (2000), which requires certain federal agencies to reserve a portion of their research and development funds for awards to small businesses. As part of its SBIR program, DOD issues an SBIR solicitation twice a year listing the research topics for which it will consider SBIR program admission. Firms first apply for a 6-month phase I award to test the scientific, technical, and commercial merit and feasibility of a certain concept. If the firm's phase I performance is deemed successful, it may be invited to apply for a 2-year phase II award to further develop the concept. After the completion of phase II, firms are expected to obtain funding from the private sector and/or non-SBIR government sources to develop the concept into a product for sale in private sector and/or military markets. DOD's SBIR Website, http://www.acq.osd.mil/sadbu/sbir/ overview/index.htm>.

Solicitation No. 2006.2 included Army topic No. A06-079, –Identification of Cultural Demographics to Predict Community Responses to Military Operations.— The objective of the topic is to predict community responses to military operations in Iraq based on the interactions among cultural demographics obtained through electronic and/or textual population data sets, and to develop display technologies that can help a command staff rapidly develop a holistic understanding of the region. Solicitation at 111.

The solicitation contemplated the award of one or more fixed-price phase I contracts to those offerors whose proposals represented the –best value— to the government, and provided that proposals would be evaluated by government scientific or technical personnel knowledgeable in the topic area under three criteria: soundness, technical merit, and innovation of the proposed approach and its incremental progress toward topic or subtopic solution; qualifications of the proposed principal/key investigators, supporting staff, and consultants, including not only the ability to perform the research and development but also the ability to commercialize the results; and potential for commercial (government or private sector) application and the benefits expected to accrue from this commercialization.

The Army received 23 proposals, and an evaluation team evaluated each. The evaluators noted weaknesses in Quimba's proposal under all three factors, assigning scores of 42 (of 50 possible) points under the soundness/technical merit factor, 20 (of 30) points under the qualifications of personnel factor, and 12 (of 20) points under the commercialization application factor. For example, under the soundness/ technical merit factor, the evaluators noted that Quimba failed to address using data from recent military operations, provided no prototype decision support tool, and proposed to hand annotate geographic boundaries to produce demographic and socioeconomic variables, which seemed laborious and time consuming. Under the qualifications of personnel factor, the evaluators found that Quimba lacked researchers/staff in human-factors/behavioral sciences and that, while Quimba proposed an intern to support the proposed hand annotation, no intern was currently on staff and the proposed time commitment for this task might have been understated. Under the commercial application factor, the evaluators found that Quimba had no letters of support showing that commercialization was feasible. AR, Tab D, Quimba Proposal Evaluation, at 1-3. Based on this evaluation, Quimba's proposal was ranked sixth overall, with a total of 74 of a possible 100 points. The agency made award to Aptima, Inc., whose proposal was the highest'ranked (100 points). AR, Tab H, Ranking of Proposals, at 1.

Quimba challenges the rejection of its proposal on several grounds. We have considered all of Quimba's arguments and find them to be without merit. We discuss the principal arguments below. Quimba argues that, because the solicitation did not specifically require the features for which its proposal was downgraded--for example, the solicitation did not require that offerors use military data, have personnel with a social science background, or provide letters of support--these weaknesses reflect the improper application of undisclosed evaluation factors. Quimba concludes that there was no proper basis for downgrading its proposal.

Where an agency is conducting an SBIR procurement, it has the discretion to determine which proposals it will fund. R & D Dynamics Corp., B-285979.3, Dec. 11, 2000, 2000 CPD para. 201 at 4. In light of this discretion, our review of an SBIR procurement is limited to determining whether the agency violated any applicable regulations or solicitation provisions, or acted in bad faith. U S Positioning Group, LLC, B-294027, June 21, 2004, 2004 CPD para. 133 at 3. While procuring agencies are required to identify significant evaluation factors and subfactors in a solicitation, they are not required to identify every aspect of each factor that might be taken into account; rather, agencies reasonably may take into account considerations, even if unstated, that are reasonably related to or encompassed by the stated evaluation criteria. Client Network Servs., Inc., B-297994, Apr. 28, 2006, 2006 CPD para. 79 at 6; Nicholson/ Soletanche Joint Venture, B-297011.3, B-297011.4, Apr. 20, 2006, 2006 CPD para. 71 at 8.

Here, we have no basis to question the agency's decision not to select Quimba's proposal for a phase I award. We have reviewed the entire record and find that the challenged weaknesses fell within the scope of the three evaluation factors.

For example, with respect to the agency's evaluation under the soundness/technical merit factor, Quimba's proposal was downgraded for failing to offer as resourceful and innovative a means of gathering demographic information as other proposals and for failing to address the availability of military sources of information. In this regard, Quimba's proposal specifically stated that there –is no database of demographic information available for Iraq or Bagdad,— AR, Quimba Proposal, Tab B, at 5, while other proposals identified websites that provided relevant demographic information. The evaluators determined that using available current data increased the likelihood of producing a relevant and useful final product. AR, Tab 3, Technical Response to Protest, at 1. Further, while Quimba generally proposed using –military sources— of demographic information (AR, Quimba Proposal, Tab B, at 5), it identified only one potential source, the Center for Army Lessons Learned. The agency viewed Quimba's failure to fully address using military data as a weakness. AR, Tab D, Quimba Proposal Evaluation, at 1; Tab 3, Technical Response to Protest, at 1. While the solicitation did not explicitly require offerors to identify databases of demographic information, or to address military sources, as noted above, the stated purpose of the program was to predict community responses to military operations based on the interactions among cultural demographics obtained through electronic and/or textual population data sets. Since use of demographic information clearly was contemplated by the solicitation, we think the agency reasonably could consider, consistent with the stated evaluation factors, the extent to which offerors identified relevant websites and other sources of demographic information in evaluating the soundness and technical merit of their proposed approach to the topic problem.

Similarly, we think Quimba's failure to propose human factors/behavioral scientists reasonably could be considered by the agency under the qualifications of personnel factor, given that the goal of this project is to provide on-site military personnel with relevant demographic information to develop an understanding of the region. A human factors/behavioral scientist reasonably could be viewed as beneficial in that such an expert would be able to assist military personnel in gaining the desired understanding. Although Quimba proposed to provide a computer scientist, an expert in text mining, and Arabic language and cultural consultants, it did not specifically offer a human factors/behavioral scientist. AR, Tab 3, Technical Response to Protest, at 3. Quimba suggests that its response to two visualization tasks and one optional task implicitly demonstrated its human factors expertise. However, we have reviewed Quimba's responses to these tasks and find nothing (there or elsewhere in its proposal) that references human factors expertise or suggests that Quimba would use human factor/behavioral scientists in the planning and design of its visualization tools. Quimba Proposal at 14-16; Protester's Response to Agency Report, Tab 3, Quimba Spreadsheet at D11. We conclude that the evaluation was unobjectionable.[1]

Quimba alleges that the evaluation team failed to include experts knowledgeable in marketing and commercialization or with practical experience in the development and deployment of demographic information systems. Protester's Response to Agency Report, Tab 3, Quimba Spreadsheet, at D7. However, the selection of individuals to serve as proposal evaluators is a matter within the discretion of the agency; accordingly, we will not review allegations concerning the qualifications of evaluators or the composition of evaluation panels absent a showing of possible fraud, conflict of interest, or actual bias on the part of evaluation officials. Glatz Aeronautical Corp., B-293968.2, Aug. 10, 2004, 2004 CPD para. 160 at 3 n.1. Quimba has made no such showing.

Finally, Quimba argues that the agency's concerns regarding the use of an intern to perform hand annotations and the hours required to perform such annotations was essentially a nonresponsibility determination that should have been referred to the Small Business Administration (SBA) for review under the certificate of competency program. See Federal Acquisition Regulation sect. 19.602-1(a). This argument is without merit. An agency may use traditional responsibility factors, such as personnel competencies and capabilities, as technical evaluation factors where, as here, a comparative evaluation of those areas is to be performed. Advanced Resources Int'l, Inc.--Recon., B-249679.2, Apr. 29, 1993, 93-1 CPD para. 348 at 2. A comparative evaluation means that competing proposals will be rated on a scale relative to each other rather than on a pass/fail basis. Dynamic Aviation-Helicopters, B-274122, Nov. 1, 1996, 96-2 CPD para. 166 at 3. We continue to hold that no SBA referral is required where, while the small business offeror's proposal is evaluated as acceptable, it is not selected for award because another offeror's proposal is evaluated as superior under a comparative analysis or because of a cost/technical tradeoff analysis. Capitol CREAG LLC, B-294958.4, Jan. 31, 2005, 2005 CPD para. 31 at 6'8. The assessment of proposed staff that occurred here was part of a comparative, best value evaluation, not a responsibility determination; Quimba's proposed staff was not evaluated on a pass/fail basis. Accordingly, no referral to SBA was required.

The protest is denied.

Gary L. Kepplinger

General Counsel