Noble Solutions
Highlights
Noble Solutions protests the evaluation of its proposal under solicitation No. FY04.1, issued by the Department of Defense (DOD) for the Small Business Innovation Research (SBIR) program. Noble, whose proposal was not selected for award, challenges the agency's evaluation.
B-294393, Noble Solutions, September 10, 2004
Decision
Henry Noble for the protester.
Craig S. McCaa, Esq., Department of the Air Force, for the agency.
Linda S. Lebowitz, Esq., and Michael R. Golden, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Agency reasonably determined not to select protester's proposal for a Phase I SmallBusiness Innovation Research contract where the agency reasonably evaluated the protester's proposal as technically unacceptable and the protester has not meaningfully challenged the agency's evaluation.
DECISION
Noble Solutions protests the evaluation of its proposal under solicitation No. FY04.1, issued by the Department of Defense (DOD) for the Small Business Innovation Research (SBIR) program. Noble, whose proposal was not selected for award, challenges the agency's evaluation.
The SBIR program is conducted pursuant to the Small Business Innovation Development Act, 15 U.S.C. 638 (2000), which requires certain federal agencies to reserve a portion of their research and development funds for awards to small businesses. In addition to advancing the role of small businesses and the participation of minority and disadvantaged persons in research and development, the objectives of DOD's SBIR program include stimulating technological innovation in DOD's critical technology area and increasing the commercial application of DODsupported research and development results. The program has the following three phases: Phase I is to determine the scientific, technical, and commercial merit of ideas; Phase II is the principal research and development effort resulting in a well-defined, deliverable prototype; and in Phase III, the small business seeks to obtain private and public funding to develop the prototype into a viable commercial product for sale to military and/or private sector markets. Solicitation 1.1, 1.2.
This solicitation sought proposals for Phase I awards to be made by a number of participating DOD component agencies and offices, including the Department of the Air Force. Each component agency identified technical topics. Phase I awards under Air Force topic No. AF04-087, entitled Expert Intelligent Match of Requirements and Solutions, are at issue here. As explained in the solicitation, the objective under this topic is to develop the ability to match descriptions of warfighter requirements to relevant descriptions of proposed solutions. More specifically, the solicitation explained that the government is developing a database that will include both information warfare (IW) warfighter requirements and IW solutions. The referenced database will use a set of multifaceted IW requirements that cut across 11 distinct areas, as follows: counterintelligence; computer network attack; computer network defense; electronic warfare; information assurance; integration; military deception; operations security; physical attack; public affairs operations; and psychological operations. The solicitation noted that the total number of requirements at any one time is usually between 100and 150, and that complex IWsolutions that number in the hundreds also would be included in the database. The solicitation explained that proposed IW solutions could be relevant to multiple requirements in multiple IW areas. Noting that matching warfighter requirements to solutions could be a daunting and timeconsuming effort when done by individuals, the solicitation stated that what was needed was a computer-based means of comparing IW requirements to the database collection of solutions to find those that were relevant to the requirements. The solicitation pointed out that simple, literal, wordbased searches would not be adequate for this effort because the keywords used might not be found in every IW solution document. Solicitation, Topic Document AF04-087.
The solicitation outlined the following four stated purposes for Phase I: (1) to investigate emerging and existing relevant methods of intelligent search; (2) to define the approach that would be used to intelligently search for matches between requirements and solutions; (3)to define a method of measuring performance of the search; and (4) to develop an interface prototype. Id.
The solicitation contemplated multiple awards of fixed-price Phase I contracts (generally not to exceed100,000 each) to those offerors whose proposals represented the best values to the government. The solicitation provided that proposals would be evaluated by scientists or engineers knowledgeable in the topic area. The solicitation listed the following three technical evaluation factors: (1) the soundness, technical merit, and innovation of the proposed approach and its incremental progress toward topic or subtopic solution; (2) the qualifications of the proposed principal/key investigators, supporting staff, and consultants, including not only the ability to perform the research and development, but also the ability to commercialize the results; and (3)the potential for commercial (government or private sector) application and the benefits expected to accrue from this commercialization. The solicitation also stated that where proposals were essentially equal in technical merit, cost to the government would be considered in determining the successful offerors. The solicitation advised offerors that the technical evaluations would be based only on the information contained in the proposals. Solicitation 4.1, 4.2.
In response to the solicitation, the Air Force received 26 proposals, including proposals from Stottler Henke Associates, Inc., 21st Century Technologies, Inc., and Noble. The proposals were evaluated by the Air Force Research Laboratory's Information Directorate, which assigned point scores for each of the technical evaluation factors; these point scores were supported by narratives of the strengths and weaknesses in each proposal. (The evaluation document for each proposal was prepared by the lead technical evaluator who was also the author of Air Force topic No. AF04-087.) Noble's proposal, which was based in part on developing IW warfighter search scenarios, was ranked fifteenth overall, with a total score of 77out of a possible 100 points, and was determined to be technically unacceptable essentially because Noble failed to demonstrate in its proposal a clear understanding of the referenced Air Force topic. The proposals of Stottler (ranked first overall) and 21stCentury (ranked second overall) received 100points and 95points, respectively, and both of these proposals were determined to be technically acceptable. (The price differences between these three proposals were de minimis . In this respect, Noble's price was9 higher than Stottler's price and $282 lower than 21st Century's price.) Legal Memorandum at 3; Contracting Officer's Statement at 2.
After being notified that its proposal was not selected for a Phase I award, Noble submitted an agencylevel protest in which it requested that the agency address four issues related to the evaluation of its technical proposal. In preparing its response to Noble's agency-level protest, the agency gave Noble's proposal to another member of its technical staff, an individual who did not participate in the initial evaluation of Noble's proposal, for the purpose of independently reevaluating Noble's proposal. This individual, who has a doctorate degree in computer science, a masters degree in electrical engineering, and a bachelors degree in mathematics/physics, reevaluated Noble's proposal, assigning the proposal a score of 30 points. In describing why he believed that Noble's proposal was technically unacceptable, this individual concluded that [o]verall[,] the proposal lacks sufficient details to understand the proposed approach and whether the approach is viable. Agency Report (AR), Tab13, Independent Reevaluation Report for Noble's Proposal, at 3. As relevant to this protest, this individual basically commented that Noble failed to clearly describe those aspects of its proposal involving the development of IW warfighter search scenarios. Id. at 2-3.
The agency subsequently denied Noble's agency-level protest. This protest followed. In responding to this protest, the agency included in its administrative report a copy of Noble's proposal, as well as the narrative evaluation reports for the initial evaluation and the independent reevaluation of that proposal. In addition, in response to a request from our Office, the agency furnished for our incamera review the proposals of Stottler and 21st Century and the narrative evaluation documentation for these two proposals.
Where an agency is conducting an SBIR procurement, the agency has the discretion to determine which proposals it will fund. The agency must act reasonably in deciding which proposals to fund. Glatz Aeronautical Corp. , B-293968.2,
Aug. 10, 2004, 2004 CPD ___ at 3; R&D Dynamics Corp. , B-285979.3, Dec. 11, 2000, 2000 CPD 201 at 4. Here, we have no basis to question the agency's decision not to select Noble's proposal for a Phase I award.
As a threshold matter, we point out that in its comments on the agency's administrative report, Noble failed to meaningfully respond to any position taken by the agency regarding the technical merit of its proposal. Rather, Noble merely expressed disagreement with the agency's technical evaluation, thereby failing to show that the agency's evaluation was unreasonable or otherwise not in accordance with the terms of the solicitation. Mere disagreement is not sufficient to render an evaluation unreasonable. See Bevilacqua Research Corp. , B-293051, Jan.12, 2004, 2004 CPD 15 at 8 n.8.
In its comments, Noble challenge[s] the Air Force to explain the reasonableness of failing to mention even once a most critical component of [its] proposal for AF04-87, which is not only shown pictorially (Figure 4-2b of AF04-087 Proposal), but mentioned more frequently than the lesser component (Focused Crawling), which the evaluation elected to address. Protester's Comments at 1. The record shows that the referenced graphic is titled IW War-Fighter Search Scenarios (IWSS) The Power and Secret and contains various technical terms, for example, Focused Crawling, IW War-Fighter Scenarios, and Search Relevance, and a number of directional arrows. Noble's Proposal at 7. However, it was the agency's view, and we concur based on our review of Noble's proposal, that Noble failed to meaningfully explain in its proposal its proposed technical approach. While Noble is correct that the documentation for the initial evaluation did not expressly refer to IW War-Fighter Search Scenarios, we do not view this omission as negating the agency's evaluation that Noble failed to demonstrate in its proposal a clear understanding of the Air Force topic since, for example, Noble did not meaningfully explain the referenced graphic. Noble also ignores the fact that the documentation for the independent reevaluation of its proposal specifically addressed the search scenarios aspect of its proposal. In this regard, the independent evaluator [1] recognized that Noble proposed to develop search scenarios, but, in the absence of any meaningful explanation, he concluded that it was not clear from Noble's proposal what these scenarios even were. Noble does not dispute the conclusion contained in the documentation of the independent reevaluation that [o]verall[,] the proposal lacks sufficient details to understand the proposed approach and whether the approach is viable. AR, Tab 13, Independent Reevaluation Report for Noble's Proposal, at 3. In sum, an offeror, like Noble, is responsible for providing a full discussion of its technical approach and methodology within the four corners of its proposal and it is not unreasonable for an agency to downgrade a proposal because the proposal lacks a detailed discussion of an offeror's proposed approach. See , e.g. , Wyle Labs., Inc. , B260815.2, Sept. 11, 1995, 95-2 CPD 187 at 5.
In contrast to Noble's proposal, as discussed above, the record shows that the proposals of the two firms awarded Phase I contracts contained detailed descriptions of their proposed approaches for identifying potential solutions for warfighter requirements. In their respective proposals, which we reviewed incamera , Stottler and 21st Century provided detailed descriptions of their Phase I work plans and comprehensively discussed, for example, their proposed methodologies, tasks, and milestones. In assigning the maximum of 100points to Stottler's proposal, the agency concluded that Stottler's proposal was innovative yet pragmatic, and contained an [e]xcellent technology review, corporate technical history, technical approach, and presentation of expected product performance. AR, Supplemental Documentation, Evaluation Report for Stottler's Proposal, at 2. In assigning 95 points to 21st Century's proposal, the agency concluded that while there were some weaknesses and risks associated with this firm's proposal, 21st Century nevertheless proposed very good technology. AR, Supplemental Documentation, Evaluation Report for 21st Century's Proposal, at 1.
On this record, we have no basis to question the reasonableness of the agency's evaluation of Noble's proposal as technically unacceptable and its decision not to select Noble's proposal for the award of a Phase I contract.
The protest is denied.
Anthony H. Gamboa
General Counsel
[1] Noble alleges in its comments that, in responding to its agency-level protest, the agency could not have performed an independent reevaluation of its proposal because the individual who performed the reevaluation worked in the same research laboratory as the lead technical evaluator for the initial evaluation.
To the extent Noble's allegation can be read as one challenging the selection of individuals to serve as proposal evaluators, this is a matter within the discretion of the agency; accordingly, we will not review allegations concerning the qualifications of evaluators or the composition of evaluation panels absent a showing of possible fraud, conflict of interest, or actual bias on the part of evaluation officials. Glatz Aeronautical Corp. , supra , at 3 n.1; CAE USA, Inc. , B-293002, B293002.2, Jan. 12, 2004, 2004 CPD 25 at 10-11 n.8; Solid Waste Integrated Sys. Corp. , B-258544, Jan.17, 1995, 95-1 CPD 23 at 6. Here, Noble has not made the required showing. In this respect, the solicitation stated that proposals would be evaluated by scientists or engineers knowledgeable in the topic area. Noble does not challenge the technical qualifications or expertise of the evaluators. Moreover, the fact that the individual selected by the agency to independently reevaluate Noble's proposal was located at the same research laboratory as the lead technical evaluator for the initial evaluation does not constitute evidence of fraud, conflict of interest, or actual bias.