B-236961, Feb 19, 1990, 90-1 CPD ***

B-236961, Feb 19, 1990, 90-1 CPD ***

PROCUREMENT - Competitive Negotiation - Discussion - Adequacy - Criteria DIGEST: 1. Agency is not obligated, under a solicitation for engineering services to be provided under a time and materials contract, to conduct discussions on, and give offerors the opportunity to revise, responses to a sample problem included in the solicitation, which was intended to provide a measure of an offeror's ability to independently size up a problem and come up with a viable, efficient solution, where the solicitation made clear what information was to be submitted in response to the sample problem and that no changes to the sample problem responses would be permitted. PROCUREMENT - Competitive Negotiation - Offers - Evaluation errors - Non- prejudicial allegation 2. Protesters are not competitively prejudiced by an agency's failure to advise them during discussions that they failed a sample problem included in the solicitation, where they were precluded by the solicitation from revising the sample problem responses and they could not have sufficiently improved the rest of their proposals to be in line for award, given that the sample problem was the primary technical evaluation criterion. 3. While agency's use of arithmetic mean of actual scores of sample problem, included in solicitation to determine whether responses to the problem passed or failed, is questionable, the protesters, who failed the problem with scores which were the lowest of any offeror's included in the competitive range and which were significantly less than the awardees' scores, were not competitively prejudiced where they have not shown their responses were misevaluated. PROCUREMENT - Competitive Negotiation - Offers - Evaluation - Technical acceptability 4. Where weaknesses in the cost proposal and corporate experience are part of a proper technical evaluation, there is no requirement for referral of the matter to the Small Business Administration for a certificate of competency. PROCUREMENT - Competitive Negotiation - Contract awards - Administrative discretion - Cost/technical tradeoffs - Technical superiority 5. Where sample problem is the primary technical evaluation factor in a solicitation that weights technical factors more than price, agency has made reasonable cost/technical tradeoff in selecting higher priced offerors who passed the sample problem over lower priced offerors who failed.

Attorneys

Modern Technologies Corporation; Scientific Systems Company:

Modern Technologies Corporation and Scientific Systems Company protest the award of five indefinite quantity, time and materials contracts by the Air Force Electronic Systems Division (ESD), Hanscom Air Force Base, Massachusetts, under request for proposals (RFP) No. F19628-88-R-0059. /1/

Both protesters contend that the Air Force improperly evaluated proposals and failed to conduct meaningful discussions.

We deny the protests.

The RFP solicited technical and cost proposals to provide technical and engineering management support services for 3 years to ESD for development and acquisition of command, control, communications, and intelligence systems. The RFP provided for two awards to small disadvantaged businesses (SDBs); two awards for small businesses; and one award on an unrestricted basis. The two protesters are both SDB concerns.

The RFP listed two areas of evaluation criteria in descending order of importance: technical/management and price. Offeror's were also advised that "the proposed prices will be significant criteria for award, as part of an integrated assessment with the technical/management Area." Within the technical/management area, the following items were listed in descending order of importance: (1) sample problem, (2) corporate/personnel experience, (3) management/organization, (4) delivery order management, and (5) administrative support, facilities and technical support. For each evaluation area, the technical factors were rated according to a color coded system and assessed for proposal risk. rating of blue was exceptional, green was acceptable, yellow was marginal, and red was unacceptable. Offeror's were advised that selections would be based on an integrated assessment of proposals as rated under the evaluation criteria to determine which proposals were the most advantageous to the government.

A primary focus of the protests involves the first and most important evaluation criterion, under which offeror's were requested to respond to several sample tasks, which were representative of the type of work that could be assigned under the contract. The RFP advised that:

"The sample problem will be evaluated as an indicator of the quality of the offeror's technical and management skills. The offeror's choice of contractor team personnel, level of expertise, and amount of labor hours needed to respond to the Sample Problem will be evaluated. The evaluation will also include how well the offeror addresses and understands the key technical and management risks of the problem, critical issues, detailing and scheduling of appropriate tasks, necessary lines of communication between the offeror and appropriate government agencies, and coordination among technical disciplines. The offeror will be advised if their response to the Sample Problem has failed. However, the offeror will not be given the opportunity to correct the response. A poor response to the sample problem will not, in itself, preclude the offeror from being eligible for award."

Eighteen proposals were submitted in response to the RFP. The record shows that the task order responses were point scored, but there was some uncertainty on the part of the evaluators as to how to decide whether offeror's passed the sample problem. Consequently, the evaluators computed a mean point score and determined those offeror's with point scores above this mean figure passed and those below that figure failed. Nine offeror's, including all awardees, passed the sample problem and nine, including both protesters, were rated as failing the sample problem. Fourteen proposals, including the protesters', were included in the competitive range. After conducting written discussions through the use of deficiency reports and clarification requests, the agency requested best and final offers (BAFOs) and received 13 BAFOs by August 2, 1989.

The source selection authority (SSA) selected System Resources Corporation and Sumaria Systems, Inc., for award under the SDB set aside portion. According to the SSA, the awardees' proposals were clearly superior to the other SDB offeror's in the technical/management area. /2/ The SSA noted that while certain SDB offeror's proposed prices were lower than those of the selected sources, the selected sources were judged to offer the best value to the government. The small business and unrestricted awards were made to Analytical Systems Engineering Corp., Horizons Technology, Inc., and Dynamics Research Corp., each of which received significantly higher technical ratings than the SDB offeror's.

These protests followed. The protesters contend the agency did not conduct meaningful discussions, the proposals were misevaluated and the source selections were not in accordance with the RFP evaluation criteria.

The protesters' contention that meaningful discussions were not conducted is based on the fact that no offeror's were informed of any weaknesses or deficiencies regarding the sample task responses during discussions. They state that they were not informed until after award that they failed the sample problem, even though this was the most important technical factor to be evaluated, and that they therefore assumed that they had passed the sample problem, since the RFP stated an offeror would be advised if its response to the sample problem had failed. They further state that they would have revised their proposals by either improving their sample problem responses or the rest of the proposals if they had been so notified.

The Air Force responds that it is untimely for the protesters to argue that the agency has a duty to conduct meaningful discussions with the offeror's about their solutions to the sample problem since, among other reasons, the RFP stated that "the offeror will not be given the opportunity to correct the response." The Air Force argues that the protest concerns an alleged solicitation impropriety and should have been filed prior to the closing date for receipt of proposals.

Regarding the merits, the Air Force states it was concerned that, if correction of the proposed solution to the sample problem were allowed, "the resultant technical leveling would significantly denigrate the efficiency of the evaluation process." According to the Air Force, the import of a sample problem in a source selection requiring technical, engineering and management support for a wide variety of highly sophisticated program offices is for an offeror to be able to demonstrate that it has the ability to perform the contemplated contract. The Air Force states that pointing out weaknesses in the proposed solution to the sample problem would tend to bring all offeror's up to an equal level in the evaluation of the most important item under the technical/management area. Thus, the Air Force asserts that the significance of the item as a discriminator of the offeror's ability to perform would virtually be eliminated if discussions on perceived shortcomings in the proposed solution were allowed and offeror's given an opportunity to revise their proposed solution.

First, we feel the issue to be timely. Although the RFP stated that offeror's would not be given an opportunity to correct their responses to the sample problem, the protesters' point is that the agency nonetheless has a duty to inform them if their responses were deemed to be failures. Thus, the issue is not simply whether there was a solicitation defect, but whether the Air Force, despite the RFP language, was required to do more than it did here.

With regard to the merits, the Competition in Contracting Act of 1984, 10 U.S.C. Sec. 2305(b)(4)(B) (1988), as implemented in Federal Acquisition Regulation (FAR) 15.610(b), requires that meaningful written or oral discussions be held with all responsible offeror's whose proposals are in the competitive range. For discussions to be meaningful, agencies must furnish information to all offeror's in the competitive range as to the areas in their proposals which are believed to be deficient, so that offeror's may have an opportunity to revise their proposals to fully satisfy government requirements. Syscon Servs., Inc., B-235647, Sept. 21, 1989, 68 Comp.Gen. ***, 89-2 CPD Para. 258. On the other hand, in certain situations an agency need not point out proposal weaknesses that would not be subject to change through discussions. See, e.g., Saturn Constr. Co., Inc., B-236209, Nov. 16, 1989, 89-2 CPD Para. 467 and Cosmos Eng'rs, Inc., B-220000.3, Feb. 24, 1986, 86-1 CPD Para. 186 (weaknesses in actual corporate experience need not be discussed, except for informational deficiencies, since they are matters of historical information not subject to change).

Here, the Air Force used the sample problem to obtain a measure of the offeror's technical/management skills as reflected in their proposed solutions to the test problems.

These skills obviously are important to the Air Force since the contractor will be called upon to deal with various technical and engineering problems and will be reimbursed, under this time and materials contract, for every hour of labor expended. It is also apparent that the Air Force wanted to gauge the offeror's independent management and technical abilities and expertise to propose, on their own, solutions to a variety of complex engineering tasks. While the pointing out of deficiencies in the proposed solutions might well have produced improvements in the offeror's approaches, what was to be evaluated here was not how well an offeror could improve the problem areas, but rather how well an offeror could independently size up a problem and come up with a viable, efficient solution. In these circumstances, we agree with the Air Force that there would be negligible value in allowing revisions to the sample problem responses since the offeror's initial responses would represent the most accurate example of the offeror's abilities. Consequently, and given that the RFP made clear what information was to be submitted in response to the sample problem and that no changes to the sample problem responses would be permitted, we find that the Air Force was not obligated to conduct discussions on the sample problem responses and give offeror's the opportunity to revise their responses. Cf. E.H. Pechan & Assocs., Inc., B-221058, Mar. 20, 1986, 86-1 CPD Para. 278 (agency was obligated to point out deficiencies in sample task responses where agency dissatisfaction with responses involved the protesters' failure to provide detailed approaches and there was an arbitrary page limitation on the responses) and Syscon Servs., Inc., 68 Comp.Gen., supra (involving an RFP with no warning that discussions would not be conducted on the sample tasks, where limited discussions in this area were properly conducted).

However, we agree the Air Force was obligated to at least apprise offeror's during discussions whether they failed to pass the sample problem, particularly since this was promised by the RFP. Although the Air Force asserts that the protesters could have asked during discussions whether they passed the problem, we believe that, given the Air Force's decision not to mention the sample problem responses during discussions, it would have been inappropriate for the Air Force only to supply this information to those who asked. Moreover, the Air Force assertion that it satisfied this RFP promise by advising offeror's of their failure of the sample problem in their debriefings and in its reports on the protests is likewise not meritorious, since this promise is stated in conjunction with its limitation on the conduct of discussions. Nonetheless, given that offeror's could not modify their sample problem responses, we find the protesters were not competitively prejudiced by the agency's decision not to advise the protesters of the failures of the sample problem, since they could not have sufficiently improved the rest of their proposals to be in line for award. In this regard, since the protesters' BAFO prices were significantly lower than those of the awardees, we do not perceive how they would have gained any advantage by cutting their prices. Indeed, as discussed below, Scientific's BAFO prices were considered unrealistically low. Moreover, while the protesters assert they could have significantly improved the rest of their technical proposals if they had been told of their failure of the sample problem, they have provided no details in this regard. Furthermore, as discussed below, since the sample problem was listed and treated as the most important evaluation factor, we find it unlikely that either protester could have come in line for award, given their failure of the sample problem and the relative strengths of the awardees.

The protesters also contest the methodology used to evaluate the sample problem responses. The protesters argue that the Air Force's use of a mean score as a criterion for passing/failing the sample problem is arbitrary and inconsistent with the source selection plan and Air Force Regulation 70-30, Streamlined Source Selection Procedures. That regulation requires the technical team to establish objective standards at the lowest level of subdivision of specific evaluation criteria. The protesters argue the Air Force established a minimum score of 100 points to pass the problem prior to release of the RFP, and that no offeror's received this score. The protesters assert there is no rational relationship between that minimum 100 point score and the arithmetic mean of the actual scores awarded that was used to determine whether offeror's passed the sample problem. The protesters argue the use of a mean, for any symmetric distribution, must result in half the offeror's passing and the other half failing, irrespective of their technical and management skills, and that this is irrational. The record shows the source selection evaluation board (SSEB) scored sample problem responses individually and separately, according to predetermined evaluation standards. The sample problem had 10 areas (seven technical, three organizational) which were each assigned 10 points. An offeror could earn additional points by elaborating or providing additional information on elements above those required by the standard. A total of 122 points were available. The SSEB recommended 100 points be called the minimum and 122 points the maximum. After computing a raw score for each offeror, the SSEB displayed the evaluation results in the form of raw scores, which ranged from 58 to 90 for offeror's in the competitive range. The Source Selection Advisory Council (SSAC) did not approve the designation of 100 points as a minimum score, but decided to use the mean of all raw scores (65.33) as the pass/fail line. Nine proposals passed and nine proposals failed.

Initially, we note that alleged deficiencies in an agency's application of the source selection plan do not themselves provide a basis for questioning the validity of the award selection. Source selection plans are internal agency instructions and as such do not give outside parties any rights. Quality Sys., Inc., B-235344, B-235344.2, Aug. 31, 1989, 89-2 CPD Para. 197. It is the evaluation scheme in the RFP, not internal documents, to which the agency is required to adhere in evaluating proposals and making the source selection. Pan Am World Servs., Inc., B-235976, Sept. 28, 1989, 89-2 CPD Para. 283.

The record indicates that, consistent with the RFP, the Air Force prepared a detailed evaluation of offerors' proposed solutions to the sample problem, choice of contractor team personnel, level of expertise, and amount of labor hours needed to respond to the sample problem, as well as how well offeror's addressed and understood the key technical and management risks of the problem, critical issues, detailing and scheduling of appropriate tasks, necessary lines of communication between the offeror and appropriate government agencies, and coordination among technical disciplines. When the SSEB had completed its evaluation of proposals against the predetermined evaluation standards, it briefed the SSAC as to the evaluation results (raw numerical scores) when measured against the standards. Though both protesters argue that all offeror's in effect failed the sample problem because all scored below 100 points, the record indicates that no pass/fail line was established prior to receipt of offers, and 100 points was never approved by the SSAC as a pass/fail line.

While we find questionable the Air Force's use here of the arithmetic mean of the actual scores received on the sample problem as a pass/fail criterion, /3/ we do not think that the protesters were competitively prejudiced in any case. The source selection authority was presented with the offeror's raw scores on the sample problem for consideration in making the award decision. The protesters' scores of 58 on the sample problem were the lowest of all those given to the 14 offerors in the competitive range, and significantly lower than the scores of the SDB awardees (89 and 81) and other awardees (88, 87, and 77). Moreover, as discussed below, the protesters have not shown that their responses to the sample problem were misevaluated. Therefore, we find no prejudice to the protesters in the Air Force's scoring of the sample problem criterion. See IDG Architects, B-235487 et al., Sept. 18, 1989, 68 Comp.Gen. ***, 89-2 CPD Para. 236.

In a supplement to its initial protest, Scientific alleges that the Air Force made four errors in the evaluation of its response to the sample problem. /4/ Modern has not specifically contested the evaluation of its sample task responses. The Air Force responded in detail to show Scientific's allegations were not true. In its comments in response to the Air Force report on this issue, Scientific furnished no evidence or argument to rebut the Air Force position. Under these circumstances, we have no basis to question the agency's position. See TM Sys., Inc., B-228220, Dec. 10, 1987, 87-2 CPD Para. 573. In any case, from our review of the record, it appears the evaluation of the sample task responses was reasonable.

Scientific also contests other aspects of the evaluation of its proposal. For example, Scientific contends that it should not have received a yellow (marginal) rating for the corporate/personnel experience criterion since it resolved all questions raised on this matter during discussions. However, the record shows that Scientific was considered as having limited corporate experience in integrating the work of many specialists and subcontractor employees to form an interdisciplinary team, and this concern remained after receipt of BAFOs. Thus, it appears the yellow rating for this criterion was justified.

Scientific also suggests that the Air Force's concerns with the management aspects and cost realism of its proposal constituted a nonresponsibility finding which should have been forwarded to the Small Business Administration (SBA) for certificate of competency (COC) proceedings. We disagree.

The RFP here provided for a cost realism assessment of proposed labor rates, and provided that unrealistic rates would be "considered in the risk assessment and may result in a less favorable technical evaluation." The record indicates that Scientific in its BAFO proposed lower labor rates based on an extended work week, and, as a rationale, explained that it had instituted a stock compensation and option plan. No details on the plan were provided. The Air Force questioned Scientific's ability to hire and retain personnel at the labor rates proposed, and considered that the extended work week created substantial risk in terms of Scientific's ability to hire and retain a stable, experienced work force.

Since the evaluation of Scientific's cost proposal and corporate experience occurred in the context of an otherwise proper technical evaluation in consonance with the factors set forth in the RFP, even assuming the factors are traditionally viewed as bearing on responsibility, there was no requirement for a referral to SBA for a COC. See Sys. Processes Eng'g Corp., B-234142, May 10, 1989, 89-1 CPD Para. 441.

The protesters also assert that the SDB awardees do not have existing staff or experience to be selected instead of the protesters, who have been performing this ESD work. However, the record shows that the SDB awardees' relative lack of corporate and personnel experience was taken into account since they were assigned yellow (marginal) ratings with moderate risk for the evaluation factor associated with these weaknesses. The source selection was made with full knowledge of the SDB awardees' relative weaknesses in this area. Although the protesters assert that it is therefore inconceivable that the SDB awardees could receive higher technical ratings, the record shows that while the awardees resoundingly passed the sample problem, the protesters failed the problem; this accounts for the awardees' significant technical advantage.

Finally, both protesters challenge the award selections. /5/ They argue that too much weight was given to the sample problem and too little weight was given to cost. Agency officials have broad discretion in determining the manner and extent to which they will make use of the technical and cost evaluation results, and, therefore, cost/technical tradeoffs may be made, subject only to the tests of rationality and consistency with established evaluation factors. Babcock & Wilcox Co., B-235502, Sept. 18, 1989, 89-2 CPD Para. 237. In this case, technical factors were clearly stated to be of more weight than cost and the sample problem was listed as the most important technical evaluation factor. Based on our review of the record, we find the award selections were reasonable and made in accordance with the evaluation criteria.

Contrary to the protesters' arguments, we have consistently recognized that listing technical factors in descending order of importance is sufficient to apprise offeror's of their relative Syscon Servs., Inc., 68 Comp.Gen., supra. Even though the SDB awardees (and Scientific) received yellow (marginal) ratings with moderate risk in the second most important evaluation criterion, "corporate/personnel experience," the awardees passed the most heavily weighted sample problem criterion with relatively high scores, whereas the protesters failed the problem, with the lowest scores for this criterion of the offeror's included in the competitive range. Indeed, the source selection document highlights the SDB awardees' success in passing the sample problem as demonstrating their "strong technical expertise" in accomplishing the contract work and their relative superiority over the other SDB offeror's.

It is apparent that the protesters had an "uphill battle" to persuade the agency that they should be selected for award, since they could not correct their sample problem responses. /6/ Nevertheless, given that the sample problem was the most highly valued technical factor, we think the Air Force reasonably found the awardees significantly technically superior to the protesters such as to offset the protesters' proposed lower prices. In this regard, we have consistently upheld awards to higher rated offeror's with higher proposed prices or costs where the agency reasonably determines that the cost premium involved is justified considering the technical superiority of the selected offeror's proposal, particularly where cost is of lesser weight than technical factors. Norden Serv. Co., Inc., B-235526, Aug. 22, 1989, 89-2 CPD Para. 167; TRW, Inc., 68 Comp.Gen. 511 (1989), 89-1 CPD Para. 584.

The protests are denied.

/1/ Scientific protests all five awards; Modern's protest is limited to the two awards to small disadvantaged businesses.

/2/ A fifth SDB firm, offering an evaluated price higher than the awardees', also passed the sample problem and received a technical/management rating similar to the awardees', but was not selected because of its higher price.

/3/ In this regard, the Air Force drew the pass/fail line at 65.3 points, which meant one offeror with 66 points passed the problem and another with 64 points failed the problem, even though there appears to have been no substantial difference in quality between the sample problem responses.

/4/ Since Scientific marked these allegations proprietary, we provide no details of this portion of the evaluation, although we have considered the entire record in reviewing them.

/5/ In its initial proposal letter, Modern contended that it was unusual that four of the five awardees were Massachusetts firms. The Air Force in its report responded in detail to this allegation, and Modern in its comments did not rebut the Air Force's response. We therefore consider the issue to have been abandoned by the protester and will not consider it. See OptiMetrics, Inc., et al., B-235646, B-235646.2, Sept. 22, 1989, 89-2 CPD Para. 266.

/6/ The protesters suggest that perhaps their proposals should not have been included in the competitive range, after they failed the sample problem. However, the contracting officer reports that including the protesters in the competitive range is consistent with FAR Sec. 15.609(a) (FAC 84-16), which requires the inclusion in the competitive range of all proposals that have a reasonable chance of being selected for award, and provides that "when there is doubt as to whether a proposals is in the competitive range, the proposal should be included." Here, given the protesters' low prices and the yellow rating received by all the SDB offeror's for various evaluation criteria, we think the agency acted reasonably in including the protesters' proposals in the competitive range. See Monarch Enters., Inc., B-233303 et al., Mar. 2, 1989, 89-1 CPD Para. 222, at 10.

Robert B. Shearer, Esq., for Modern Technologies Corporation, the protester.