Matter of: Orion Research, Inc. File: B-253786 Date: October 21, 1993

Matter of: Orion Research, Inc. File: B-253786 Date: October 21, 1993

PROCUREMENT Competitive Negotiation Offers Evaluation errors Non-prejudicial allegation Protest of agency evaluation of proposal is denied where protester has not demonstrated that evaluation was unreasonable or inconsistent with evaluation criteria set forth in the solicitation. PROCUREMENT Competitive Negotiation Offers Evaluation Organizational experience Where a solicitation lists experience as an evaluation factor, an agency may reasonably consider an offeror's experience in the particular areas to be addressed under the solicitation since such specific experience is related to and encompassed by a general experience factor. PROCUREMENT Competitive Negotiation Contract awards Administrative discretion Cost/technical tradeoffs Technical superiority Where agency reasonably determined that technical superiority of awardee's proposal outweighed its higher cost, selection of awardee's proposal as most advantageous to the government is not objectionable.

Attorneys

DECISION Orion Research, Inc. protests the rejection of its proposal and the award of a contract to ICF Inc. under request for proposals (RFP) No. RS-NRR-93-029, issued by the Nuclear Regulatory Commission for a study to assess the feasibility of recentralizing the nuclear reactor operator licensing function. Orion objects to the agency's evaluation of its proposal and to the selection of ICF's more costly proposal as most advantageous to the government.

We deny the protest.

The RFP requested proposals for the performance of an "integrated management analysis study" to investigate the feasibility and impact of recentralizing the nuclear reactor operator licensing function. By way of background, the solicitation explained that operator licensing responsibilities had been decentralized to the regional level during the early 1980s, and that several adverse consequences, relating principally to inconsistencies among the regions in the administration of the licensing examinations, were alleged to have occurred as a result. The objective of the requested study is to identify existing problems and to determine whether or not they are the result of decentralization; to assess the advantages and disadvantages of centralized and decentralized approaches to licensing, including the impact of organization structure on examiner career paths; and to determine how recentralization can best be accomplished, if that is the course of action decided upon. Tasks to be accomplished pursuant to the effort include development and administration of a study questionnaire, follow-up interviews with a percentage of the respondents, analysis of the data collected, and preparation of both preliminary and final reports.

The RFP contemplated the award of a cost-plus-fixed-fee contract to the offeror whose technical/cost relationship was judged most advantageous to the government, with technical merit carrying greater weight than cost in the evaluation. The solicitation provided for consideration of the following factors in the evaluation of technical proposals: technical quality of staff (worth 30 points) and availability of staff resources (10 points), organizational experience (30 points), understanding of contract objectives (20 points), and program management (10 points).

Six offerors submitted proposals by the January 21, 1993, closing date. Upon completion of its evaluation, the source evaluation panel (SEP) recommended that ICF, which had received a technical score of 90.8, be the sole firm included within the competitive range. According to the SEP, although four other proposals, including Orion's, had been rated as either technically marginal or satisfactory, each would require substantial clarifications "before a contract could be confidently awarded." The contracting officer accepted the SEP's recommendation and conducted discussions with ICF.

On February 16, after submission of ICF's best and final offer (BAFO), but prior to award of a contract, the contracting officer notified Orion and the other unsuccessful offerors that their proposals would not receive further consideration. Both Orion and another of the unsuccessful offerors protested that decision to the agency. Upon review of the protests, the agency decided to take corrective action, expanding the competitive range to include the four offerors that had received marginal or satisfactory technical scores. One of the offerors subsequently requested that its proposal be withdrawn from consideration, leaving a competitive range of four.

The agency conducted discussions with all offerors in the competitive range on April 26, and, on April 28, requested BAFOs with a due date of May 4. The evaluators again rated ICF's technical proposal substantially higher than any of the other offerors'--i.e., 93.7 versus 78.7, 76.5, and 75.2--and noted that its proposed cost of $312,588 was second low. Orion, which received the third highest technical score of 76.5, proposed a cost of $203,800. Although Orion's proposed cost was significantly lower than ICF's, the evaluators concluded that ICF's technical superiority outweighed the difference in cost and recommended award to ICF. On May 27, the agency awarded a contract to ICF. Shortly thereafter, the contracting officer notified unsuccessful offerors of the award, whereupon Orion protested to both the agency and our Office.[1]

Technical Evaluation

Orion protests the evaluation of its proposal, arguing that the agency applied evaluation criteria not set forth in the solicitation and that it ignored relevant information in the proposal.

The evaluation of proposals is within the discretion of the procuring agency, since it is responsible for defining its needs and the best method of accommodating them, and must bear the burden resulting from a defective evaluation. RAI, Inc.; The Endmark Corp., B-250663 et al., Feb. 16, 1993, 93-1 CPD Para. 140. In cases challenging an agency's technical evaluation, our Office will not independently weigh the merits of offers; rather, we will examine the agency's evaluation to ensure that it was reasonable and consistent with the stated evaluation factors. Id. As discussed below, we have examined the agency's evaluation here and conclude that it was both reasonable and consistent with the stated evaluation criteria.

Orion objects first to the evaluation of its proposal under the subfactor concerning the technical qualifications of its proposed staff. The RFP provided, with regard to staff qualifications, that:

"The contractors shall have key personnel whose training, experience, [and] overall qualifications [permit] the conduct of an integrated management analysis study. The contractors shall also have personnel with methodological skills to design studies, develop and deliver data collection instruments, tabulate and statistically present qualitative and quantitative findings, and analyze and interpret such findings into acceptable written report formats. An example of these qualifications may include a contractor staff team who holds advanced degrees (e.g., M.S., Ph.D.s in Industrial/Organizational Development; Management; and Psychology and Statistics) and who can perform an integrative approach to the tasks in this [statement of work]."

The evaluators awarded the protester a score of 25 (out of 30) under this subfactor, noting as a weakness that the educational backgrounds of many of the individuals proposed by Orion were not in the areas required by the RFP (i.e., industrial/organizational development, management, psychology and/or statistics).[2] Orion contends that although its proposed personnel did not hold degrees in the fields mentioned in the RFP, they had acquired the required educational background in pursuing related degrees.

With regard to the evaluators' criticism that the educational backgrounds of many of Orion's proposed personnel were not in the fields required by the RFP, the solicitation did not require degrees in certain fields; rather, it required particular skills in data gathering and analysis and listed degrees in specified fields as examples of acceptable qualifications. Thus, we do not think that it would have been appropriate for the agency to downgrade Orion's proposal simply because the protester did not propose personnel with degrees in the enumerated areas. It is clear from the agency report and comments, however, that the criticism of Orion's proposal was not simply that its key personnel did not hold degrees in the specified fields, but rather, that these individuals did not have degrees in fields in which they would necessarily have gained competence in the required methodological skills. For example, the protester asserts that it met the requirement for a background in statistics by proposing individuals with degrees in economics and mathematics. The agency states in response that the areas of expertise of an economist or mathematician are not the same as those of a statistician and are less relevant to the tasks to be performed under the solicitation. We agree. Although individuals who have attained degrees in economics or mathematics may well have received significant exposure to statistics while pursuing their degrees and thus possess the methodological skills sought by the agency, it cannot be assumed, based on their degrees alone, that they would have expertise in statistics. Thus, the evaluators reasonably awarded Orion a less than perfect score under the staff qualifications criterion based on their judgment that the educational qualifications of its proposed personnel did not necessarily imply expertise in the skills necessary for successful performance of the contract tasks.

Orion also objects to the agency's criticism of its proposal for failing to furnish sufficient detail regarding its interview methodology. The agency cited as a weakness under the staff availability evaluation subfactor (on which Orion received a score of 8.3 of 10) Orion's failure to provide information concerning its "staff's availability, hours or duration to conduct [the] interviews, to gather the qualitative data required in the study." Similarly, the agency cited as a weakness under the "Understanding of Contract Objectives" criterion (on which Orion received a score of 15 out of a possible 20) the protester's failure to provide "a discussion of the logistics of conducting interviews" in its BAFO. Orion contends that it did spell out the number of days that its staff members would devote to interviewing in the Integrated Project Schedule that it furnished with its BAFO and that it discussed the "logistics" of conducting interviews (i.e., how it would arrange interviewer travel schedules and seek, at utility sites, to conduct all interviews at a central location) in its initial proposal.

In responding to Orion's protest, the evaluators noted that the information that they perceived to be lacking in the protester's proposed interview methodology concerned the duration and content of an individual interview. The evaluators explained that Orion's proposal had failed to address the type of question that would be asked (i.e., whether open-ended or eliciting a rating, ranking, or yes/no answer) and had failed to specify the number of individuals who would be interviewed at a time, leading them to conclude that in conducting and analyzing the interviews, Orion would place greater emphasis on quantitative, rather than the desired qualitative, data. In our view, the agency reasonably downgraded Orion under the "Understanding of Contract Objectives" criterion based on its failure to furnish sufficient detail concerning its approach to interviewing.[3]

The protester also complains that the evaluators unreasonably lowered its score under the "Understanding of Contract Objectives" evaluation factor (from 16.7 to 15) after its submission of information responding to the agency's discussion questions. The protester contends that it was unfair for the agency to criticize it for failing to furnish information regarding interview logistics in its BAFO when it had furnished such information in its initial proposal and had not indicated that this information had been superseded.

The protester misunderstands the SEP's stated rationale for lowering its score under this evaluation factor, due in part to the SEP's choice of wording. As previously noted, when the evaluators observed that Orion had failed to discuss the "logistics" for conducting interviews, they meant that the protester had failed to furnish information concerning such matters as the duration of interviews, the type of questions to be asked, etc.--and not that Orion had failed to furnish information concerning the movement of interviewers and interviewees to the interview sites.

Furthermore, it is apparent from the individual evaluator worksheets and the agency report that the primary reason that the evaluators lowered Orion's score under this evaluation factor was that in its BAFO, Orion dropped its initially proposed methodology of surveying non-plant personnel by telephone rather than with written questionnaires. In initially proposing this methodology, the protester had emphasized that one advantage to contacting a segment of the targeted population by telephone prior to disseminating a written questionnaire would be that the responses would provide insights that could be applied in the development of the written questionnaire. Once Orion dropped this approach, it was unclear to the evaluators how the protester intended to develop the written questionnaire. We think that the fact that the protester changed its methodology in its BAFO without proposing an alternative approach provided the evaluators with a reasonable basis for lowering its score under this evaluation factor.

Next, Orion takes issue with the agency's evaluation of its proposal under the Organizational Experience criterion. The solicitation instructed offerors that their technical/management proposals should include a discussion of the offeror's organizational experience in performing integrated management systems evaluations. The evaluators assigned the protester's proposal an average score of 19 (out of a possible 30) under this factor and noted as a weakness that the subcontractor that Orion intended to rely on for much of the interviewing and analysis, Tim D. Martin Associates, did not appear to have experience in analyzing "issues such as morale, career paths, relocation costs and logistics, etc." The protester contends that since these items were not identified as evaluation criteria in the RFP, the agency should not have considered them in the evaluation of proposals.

Where a solicitation lists experience as an evaluation factor, an agency may reasonably consider an offeror's experience in addressing the particular issues to be addressed under the solicitation since such specific experience is intrinsically related to and encompassed by a general experience factor. Building Servs. Unlimited, Inc., B-252791.2, Aug. 25, 1993, 93-2 CPD Para. . Here, it was clear from the solicitation that among the issues to be addressed in the requested study were the potential impact of recentralization on examiner morale and career paths and the hardships to individuals and costs to the agency that a decision to recentralize would engender. Thus, although the solicitation did not specifically state that consideration would be given to an offeror's experience in performing integrated management systems evaluations addressing issues such as morale, career paths, and relocation costs and logistics, given the stated purpose of the study, we think that it was reasonable for the agency to consider the offeror's lack of such experience in evaluating its proposal.

Finally, the protester objects to the evaluation of its proposal under both of the subfactors relating to Program Management. Under the first subfactor, which concerned offerors' project management and quality assurance capabilities, offerors were required to discuss their experience in planning, scheduling, and control of personnel and workflow. Orion received a score of 4.5 out of a possible 5. Under the second subfactor, which concerned the offeror's system for monitoring and reporting detailed status and cost information for both the basic contract and individual task orders, the protester received a score of 4.7 out of 5. Orion also objects to the fact that, after submission of its BAFO, the agency lowered its score under the first subfactor from 4.8 to 4.5. We will not consider these arguments because even if the protester had received the maximum possible score under each subfactor, the total increase (.8 of a point) would have a de minimis impact on its overall score and therefore no effect on the selection decision.

Cost/Technical Tradeoff

The protester argues that the agency's selection of ICF's proposal as most advantageous to the government was unreasonable given that its own proposal received a satisfactory technical rating and was substantially lower in cost.

In a negotiated procurement, there is no requirement that award be made on the basis of lowest cost or price unless the RFP so specifies. Sabreliner Corp., B-242023; B-242023.2, Mar. 25, 1991, 91-1 CPD Para. 326. Agency officials have broad discretion in determining the manner and extent to which they will make use of technical and cost evaluation results. Cost/technical tradeoffs may be made; the extent to which one may be sacrificed for the other is governed only by the test of rationality and consistency with the established evaluation factors. Grey Advertising, Inc., 55 Comp.Gen. 1111 (1976), 76-1 CPD Para. 325. Award may be made to a higher-rated, higher-cost offeror where the decision is consistent with the evaluation factors and the agency reasonably determines that the technical superiority of the higher-priced offer outweighs the cost difference. Sabreliner Corp., supra.

Since the RFP here provided for selection of the proposal most advantageous to the government, with technical merit carrying greater weight than cost, there was no requirement that award be made to Orion simply because it received a satisfactory technical score and proposed lower costs. On the contrary, it was reasonable and consistent with the RFP for the evaluators to conclude that ICF's technical superiority-- reflected in its 17.2 point advantage--warranted its higher cost.[4]

The protester also objects to the agency's failure to quantify the trade- off to be made between cost and technical merit in determining which proposal was most advantageous to the government. Orion contends that the most logical method for determining which combination of technical merit and cost is most advantageous would be to divide each offeror's cost by its technical score and to compare the resulting ratios.

To the extent that the protester is arguing that the trade-off to be made between technical merit and cost ought to have been quantified in the solicitation itself, its protest is untimely. Protests based on alleged solicitation improprieties which are apparent on the face of the solicitation must be filed prior to the closing date for receipt of proposals. Bid Protest Regulations, 4 C.F.R. 21.2(a)(1) (1993). In any event, the solicitation did adequately express the relative weights of estimated cost versus technical merit, stating that estimated cost would receive less consideration than technical merit, but that its importance as an evaluation factor would increase as the degree of equality of technical merit between proposals increases.

Orion further complains that throughout the course of this procurement, the agency demonstrated a bias against it as a small business. As evidence of this bias, the protester cites the agency's failure to furnish it with a copy of its technical evaluation prior to intervention by our Office; the agency's reference to the fact that it had not retained counsel to pursue the protest; and the agency's expression of concern during discussions that Orion's designated project manager, who was also president of the firm, would be capable of committing 90 percent of his time to the project, as proposed. The protester also argues that the agency demonstrated a bias against it by inaccurately characterizing its technical score of 76.5, which was actually in the middle of the satisfactory range, as being in the low satisfactory range.

The actions cited by Orion do not demonstrate bias against it as a small business. We do not see how the agency's initial failure to provide the protester with a copy of its technical evaluation during the protest process shows that the agency was biased against it at the time it performed the evaluation; moreover, the agency promptly complied when we asked it to furnish a copy of the document to Orion. Further, the agency's reference in the cover letter of its report to the fact that the protester had not retained an attorney was simply an explanation of why the protester was being furnished with a redacted version of the report. Finally, we do not think that the agency demonstrated a bias against small business by asking whether an individual identified as the president of a company would have sufficiently few other duties that he would be available to devote 90 percent of his time to this contract.

Finally, in its original letter of protest, Orion complained that the agency had failed to notify unsuccessful offerors of its intention to award to ICF prior to actually making the award. The Federal Acquisition Regulation (FAR) requires preaward notification of unsuccessful offerors only under small business set-asides, FAR 15.1001(c); under other types of solicitations, contracting officers are required to notify unsuccessful offerors promptly after award, which the contracting officer did here.

The protest is denied.

1. Orion filed a protest with the agency on June 7. On June 16, prior to receiving a response to the agency-level protest, it protested to our Office.

2. The evaluators noted that only one of the individuals proposed by Orion had a degree in one of the specified fields (i.e., psychology), and that the others had degrees in areas such as systems analysis, computer systems management, economics, aeronautical engineering, and personnel management.

3. We fail to see why this lack of detail should have been cited as a weakness under the staff availability subcriterion since there is nothing in the record to show that the deficiency concerned the availability of Orion's proposed staff. The other weakness cited by the evaluators under the availability of staff resources subcriterion--that in its BAFO, Orion states that it will drop the telephone option as a proposed methodology for developing survey questionnaires--does not appear to concern staff availability. We cannot conclude that the agency's rating of Orion under this subcriterion was unreasonable, however, given that it is apparent from the individual evaluator worksheets that the evaluators were also concerned that Orion's proposal did not contain letters of commitment from non-Orion personnel, an omission which reasonably could have resulted in a downgrading of the proposal under this subcriterion.

4. The awardee's proposal was rated high in all evaluation areas. For example, the agency found that the awardee had strong organizational experience and had proposed an excellent interviewing methodology.