Skip to main content

B-245458, Jan 9, 1992

B-245458 Jan 09, 1992
Jump To:
Skip to Highlights

Highlights

Alleged failure to provide more exact identification of perceived weaknesses in proposal is not objectionable. Where agency's evaluation that protester's proposed approach was acceptable with low to moderate risk was reasonably based and protester's arguments to the contrary essentially constitute disagreement with the judgment of the evaluators. Protester has not shown that its proposal was evaluated inequitably. 3.Protest against award to higher cost. Higher technically rated offeror is denied where the solicitation evaluation scheme gave greater weight to technical merit than to cost. GSEI contends that it should have received the award since it submitted the lowest cost. BACKGROUND The MMB to be produced under this research and development effort is to be a small.

View Decision

B-245458, Jan 9, 1992

DIGEST: 1. Agency satisfied obligation to conduct meaningful discussions with offeror where it reasonably led offeror into areas of its proposal requiring clarification and offeror's responses resulted in rating of technical acceptability. Alleged failure to provide more exact identification of perceived weaknesses in proposal is not objectionable, since requirement to conduct discussions does not obligate agency to identify every aspect of a technically acceptable proposal that receives less than a maximum score. 2. Where agency's evaluation that protester's proposed approach was acceptable with low to moderate risk was reasonably based and protester's arguments to the contrary essentially constitute disagreement with the judgment of the evaluators, protester has not shown that its proposal was evaluated inequitably. 3.Protest against award to higher cost, higher technically rated offeror is denied where the solicitation evaluation scheme gave greater weight to technical merit than to cost, and the agency reasonably concluded that protester's lower proposed cost did not outweigh the technical advantages demonstrated in the awardee's higher cost proposal.

Attorneys

General Services Engineering, Inc.:

General Services Engineering, Inc. (GSEI) protests the award of a cost reimbursement contract to Motorola Inc. under request for proposals (RFP) No. DAAB07-91-R-J513, issued by the United States Army Communications Electronics Command for prototype models of the AN/PPN 20 Miniature Multiband Beacon (MMB) and accompanying test set (AN/PPM 4). GSEI contends that it should have received the award since it submitted the lowest cost, technically acceptable proposal.

We deny the protest.

BACKGROUND

The MMB to be produced under this research and development effort is to be a small, lightweight, manportable radar transponder beacon to be used by Special Operations Forces for en-route navigation, drop zone location, and ordnance delivery. The prototypes are to be designed and developed to meet all performance requirements provided in the RFP's statement of work (SOW). Minimal data is to be procured.

Award was to be made to the best overall (best value) proposal determined to be the most beneficial to the government under three evaluation factors -- technical, cost, and management-- in descending order of importance. /1/ The RFP provided that the technical factor was weighted more than the other factors combined and the cost factor was more important than management. The technical factor was divided into four subfactors: Engineering Approach (greater weight than other subfactors); Personnel; Facilities and Equipment; Engineering and Technical Man-hours; and Schedule (all of equal weight).

"Engineering Approach" was to be evaluated in three areas. understanding the problem; feasibility of approach; and completeness. With regard to feasibility, the RFP stated in pertinent part:

"The extent to which the proposed approach is workable and the end results achievable. The extent to which successful performance is contingent upon untried unproven devices and techniques which may require excessive development. The proposal will be evaluated to determine whether the offeror's methods and approach in meeting the requirements in a timely manner provide the Government with a high level of confidence to ensure successful performance. The offeror's approach to design of components, their maturity and performance pertaining to ... requirements of the specification will be evaluated."

The various factors and subfactors, as well as the proposals, were to be rated in one of four possible categories: "outstanding;" "acceptable;" "susceptible to being made acceptable;" and "unacceptable."

Seven offerors, including GSEI and Motorola, submitted proposals by the April 5, 1991, closing date and preliminary technical evaluations were conducted by the agency. One proposal was eliminated as technically unacceptable while the other six were rated as susceptible to being made acceptable in both the management and technical factors. The evaluators prepared items for negotiation (IFNs) specific to each of the six proposals in the competitive range and obtained responses from the offerors. The proposals were then reevaluated, resulting in a rating of acceptable for GSEI in both the technical and management factors. Motorola's proposal and that of a third offeror were rated outstanding in the technical factor and acceptable in the management factor.

A separate cost review was conducted by the agency and the Defense Contracting and Audit Agency (DCAA) and both Motorola's and GSEI's proposed costs were adjusted downward as a result. The third offeror's proposed cost was adjusted upward. As adjusted, Motorola's cost exceeded GSEI's cost by more than $1 million. Discussions were conducted on cost proposals, performance period, terms, and conditions, and a best and final offer (BAFO) was requested from each of the six offerors in the competitive range. The difference in Motorola's and GSEI's costs remained approximately the same.

Although both Motorola's proposal and the third offeror's proposal were rated outstanding, since Motorola's proposed cost was lower, the contracting officer conducted a cost/technical tradeoff between the Motorola and GSEI proposals prior to making her award decision. While finding that GSEI had provided analysis and discussions of its proposed approach, the contracting officer also found that the proposal lacked measured data on one of its transmitters. /2/ Consequently, she considered GSEI's approach unproven and of moderate risk. Motorola's design was considered unique and, because Motorola had furnished measured data on its approach for transmitters in both bands, the contracting officer considered it a proven design with a low risk. Based upon the evaluation criteria, which made the technical factor more important than the cost and management factors combined, the contracting officer determined that although Motorola's estimated cost was approximately 125 percent higher than GSEI's, the Motorola proposal represented the best value to the government. She awarded Motorola the contract on August 22.

When GSEI received a debriefing, the agency identified certain weaknesses in the proposal: one of GSEI's transmitters and its receiver design possessed moderate risk; there was a lack of actual measured data and technical uncertainties for that transmitter; and GSEI proposed only one "alternative/approach" using its proposed diode. In addition, the agency noted that while GSEI's man-hours were considered acceptable, it believed that the hours were a low estimate overall and risked an overrun. After its debriefing, GSEI filed a protest with our Office. Performance of the contract has been stayed pending our decision.

MEANINGFUL DISCUSSIONS

GSEI first contends that the IFNs it received during discussions failed to adequately address the weaknesses identified at the debriefing. Generally, the requirement for discussions with offerors is satisfied by advising them of weaknesses, excesses, or deficiencies in their proposals, unless doing so would result either in disclosure of one offeror's technical approach to another or in technical leveling, and by affording them the opportunity to satisfy the government's requirements through the submission of revised proposals. Federal Acquisition Regulation (FAR) Sec. 15.610(c)(2), (5); Miller Building Corp., B-245488, Jan. 3, 1992, 92-1 CPD Para. ***; The Scientex Corp., B-238689, June 29, 1990, 90-1 CPD Para. 597. Agencies are not, however, obligated to afford offerors all- encompassing discussions, id., or to discuss every element of a technically acceptable, competitive range proposal that has received less than the maximum possible score. See Associated Chem. and Envtl. Servs., et al., 67 Comp.Gen. 314 (1988), 88-1 CPD Para. 248.

Here the agency provided 12 IFNs to GSEI regarding various aspects of its technical, management, and cost proposals. In general, the IFNs requested additional information concerning which aspects of the design were "frozen" (i.e., no longer requiring changes); receiver performance, response curves, and satisfaction of requirements] temperature compensation, with references to the proposed diodes; range reply pulse requirements; and proposed level of effort. We find that the agency fulfilled its obligation to provide meaningful discussions on weaknesses in the GSEI approach.

With regard to GSEI's transmitter and receiver design, as well as the lack of measured data, the IFNs were adequate to lead it into the areas of weakness identified at the debriefing. One IFN asked which aspects of the design (including transmitter and receiver) were "frozen" and requested the submission of any necessary, additional test information. Other IFNs concerned aspects of the design involving the transmitter in question and the GSEI approach. Questions regarding the GSEI receiver and requests for an indication whether certain data were actual measured data or computer prediction were included in other IFNs. Further, according to the notes of the debriefing, although GSEI disputed the lack of measured data on its transmitter, it was unable to locate such data in either its original proposal or IFN responses.

We do not believe that the agency was required to specifically highlight the perceived weakness in GSEI's use of one "alternative/approach." /3/ In its proposal, GSEI included a tradeoff analysis which addressed positive and negative aspects of various approaches. GSEI's proposal made plain that although it could use different approaches, this was unnecessary, because its proposed approach was the best. This belief, echoed in its protest submissions, and coupled with its assessment that Motorola's and other approaches are inferior, leads us to the conclusion that advising GSEI of such a weakness would have resulted in prohibited technical leveling. See FAR Sec. 15.610(d)(1); Virginia Technology Assocs., B-241167, Jan. 29, 1991, 91-1 CPD Para. 80.

With regard to its proposed man-hours, one IFN requested a breakout of subcontractors and responsibilities, and questioned GSEI's proposed number of key personnel hours versus the total man-hours proposed. GSEI disagrees that this IFN was adequate to indicate an agency concern that its proposed hours were low. GSEI also notes that the agency's business clearance memorandum specifically accounts for GSEI's fewer proposed hours due to its approach and compressed schedule. The contracting officer found that GSEI's proposed program hours, as explained in the IFN response, appeared low and possibly unrealistic. Although the business clearance memorandum reflects the agency's understanding of the basis for GSEI's low proposed man-hours, we believe it was reasonable to conclude that the low number of man-hours represented a proposal weakness because it represented a moderate to high risk.

GSEI argues that its responses fully addressed all technical matters raised in the IFNs. However, it is apparent that although the agency found GSEI's proposal acceptable, it still perceived weaknesses. That the agency did not seek further discussions in these areas does not establish a lack of meaningful discussions. The agency was not obligated to continue discussions on weaknesses remaining in GSEI's technically acceptable, competitive range proposal simply because it received less than the maximum possible score. See Associated Chem. and Envtl. Servs., et al., supra. Likewise, as noted above, the agency was not permitted to advise the protester how to bring its proposal up to the outstanding level of Motorola's proposal, since such coaching would amount to prohibited technical leveling. See FAR Sec. 15.610(d)(1); Virginia Technology Assocs., supra.

UNFAIR EVALUATIONS

GSEI next argues that the evaluations were incorrect and unfair, providing detailed technical arguments why the Motorola proposal was higher risk than its own and, in fact, was unacceptable as to some specifications. In support, GSEI has submitted data from tests of diodes obtained from Motorola's supplier. However, it is not the function of our Office to evaluate proposals de novo. Rather, we will examine an agency's evaluation to ensure that it was reasonable and consistent with the stated evaluation criteria and applicable statutes and regulations, since the relative merit of competing proposals is primarily a matter of administrative discretion. Information Sys. Networks Corp., 69 Comp.Gen. 284 (1990), 90-1 CPD Para. 203. The protester's mere disagreement with the agency's judgment does not establish that an evaluation was unreasonable. Litton Sys., Inc., B-237596.3, Aug. 8, 1990, 90-2 CPD Para. 115.

We have considered GSEI's arguments and test data concerning the alleged incorrect evaluations of both Motorola and GSEI, and conclude that they provide no grounds for overturning the agency's evaluations. In general, GSEI's criticisms concern its disagreement with Motorola's approach and the agency's evaluation. For example, GSEI contends that Motorola's diode technology has only been developed within the last 1 to 2 years; that the diodes will be furnished by a single, untrustworthy supplier; and that Motorola's diodes, as tested by GSEI, fail to meet power output and specified operating temperature requirements. GSEI also relies heavily on a technical paper authored by a Motorola engineer which discusses the diode technology and acknowledges at least one risk.

Motorola responds that it has worked with the diodes in "stacked" and "unstacked" configurations since 1981. Motorola's proposal also lists a second supplier of these diodes. Further, although GSEI states that it terminated two contracts with Motorola's supplier, the record reflects that these were terminations for convenience effected 2 years before the submission of proposals. The supplier reports that GSEI continues to do business with it and recently inquired about developing a teaming arrangement.

Motorola questions GSEI's reliance on its test of Motorola's diodes and its reliance on the Motorola technical article. The firm denies experiencing any of the problems identified by GSEI and questions the reliability of tests performed on 4 diodes by its competitor, pointing out that Motorola has tested 80 such diodes. In support, Motorola relies upon test data in its proposal showing compliance with the temperature requirements and additional test data showing operation at temperatures well above those specified.

The record supports Motorola's position. First, Motorola is correct that the article referenced is of limited scope, does not contain its entire proposed approach, and was not considered by the evaluators. Second, Motorola's proposal contains an explicit agreement to meet the specifications and a number of references to test data on operation at various temperatures in the specified range. Since this is a research and development contract for prototypes, we do not believe the absence of actual test data showing that every specification has been met at the time of proposal is inconsistent with an outstanding evaluation on engineering approach. Rather, the promise to meet the specifications, coupled with detailed designs and test data indicating that success was feasible, supports the agency's evaluation. GSEI also describes several technical advantages of its approach and argues that it was not given appropriate credit. Among other matters, GSEI notes that it reported its tradeoff studies concerning the best approach and that it too submitted test data. We find that these and the other examples cited by GSEI essentially constitute further disagreement with the agency's evaluation and do not show that the agency's evaluation was unreasonable. For example, the agency's evaluation work sheets reveal that the evaluators considered GSEI's tradeoff studies. The protester was advised at the debriefing that the agency considered these tradeoff studies to be a strength of the proposal. That the evaluators did not agree that GSEI's conclusions represented the best approach does not show that the evaluations were unreasonable or unfair.

As to the issue of test data, the evaluators concluded that GSEI's design was of moderate risk because of a lack of measured data on one of its two transmitters. As pointed out by GSEI, while it did submit measured data for this transmitter, the data was based upon a transmitter with a frequency less than that specified. The data submitted by GSEI does not establish that the MMB's requirements would be met. In view of the RFP's discussion of feasibility in terms of "unproven" and "untried" devices, it was reasonable for the evaluators to find the lack of specifically applicable test data to be a weakness.

We also do not agree with GSEI's assessment that the agency discarded its superior approach simply because of its erroneous reliance on Motorola's approach. The third offeror received an outstanding proposal rating, comparable to Motorola's, by proposing the same type diodes as did GSEI. In support of this outstanding rating, the evaluators noted that the offeror's actual measured data supported its proposed design. Thus, it was not the inherent superiority of Motorola's approach that caused GSEI's lower rating-- the cause was GSEI's failure to establish a high level of confidence in the evaluators that GSEI's approach would ensure successful performance. GSEI failed to establish superiority in its proposal.

GSEI also contends that it was treated unfairly since certain lower level evaluations in which GSEI's proposal was highly evaluated and, in one instance, higher than Motorola's, were not carried through in the final evaluation. From our review of these evaluations and the final evaluations leading to the award decision, we conclude that GSEI was treated fairly. First, in instances where GSEI claims that the narrative portions seem identical, yet Motorola received a higher score, in fact the evaluators' narratives generally contained more positive comments about Motorola than is acknowledged by GSEI. For example, although one evaluator made comparable positive comments about both GSEI's and Motorola's signal processing, Motorola received the higher score because of its sound design, field programmable attributes, and ease of debugging. Second, in the one instance ("producibility") where GSEI received an outstanding rating and Motorola an acceptable rating, this was one of six subelements making up the element of "MMB and test set design." Overall, GSEI's proposal received four acceptable and two outstanding ratings for this element, while Motorola's proposal received four outstanding and two acceptable ratings. Thus, there was a reasonable basis for the evaluation of Motorola's proposal as outstanding.

With respect to the lower level evaluations, made at the subelement and element level of the subfactor, "engineering approach," these concerned one of five subfactors under the technical factor. According to the agency, these lower level evaluations were used by evaluators at the subfactor level in arriving at the final evaluations. In such an evaluation scheme it is not unusual for individual evaluators to have different subjective judgments on the relative strengths and weaknesses of the proposal. Mounts Eng'g, 65 Comp.Gen. 476 (1986), 86-1 CPD Para. 358. Consequently, differences in evaluator ratings do not establish that the evaluation process was flawed or otherwise irrational. Unisys Corp., B-232634, Jan. 25, 1989, 89-1 CPD Para. 75. It is proper for technical evaluators to discuss the relative strengths and weaknesses of proposals in order to reach a consensus rating, so that the consensus rating often differs from the ratings given by individual evaluators. The Cadmus Group, Inc., B-241372.3, Sept. 25, 1991, 91-2 CPD Para. 271. In sum, we find nothing objectionable about the fact that GSEI's final rating at the subfactor level was not as high as some subelement evaluation scores.

COST/TECHNICAL TRADEOFF

Finally, GSEI argues that the incorrect evaluations make the cost/technical tradeoff performed by the contracting officer unreasonable. GSEI also contends that the tradeoff was not sufficiently detailed to justify award to Motorola at its significantly higher proposed cost. We disagree.

In a negotiated procurement, there is no requirement that award be made on the basis of lowest cost or price unless the RFP so specifies. Henry H. Hackett Sons, B-237181, Feb. 1, 1990, 90-1 CPD Para. 136. Agency officials have broad discretion in determining the manner and extent to which they will make use of technical and cost evaluation results. Cost/technical tradeoffs may be made; the extent to which one may be sacrificed for the other is governed by the test of rationality and consistency with the established evaluation factors. Grey Advertising, Inc., 55 Comp.Gen. 1111 (1976), 76-1 CPD Para. 325. Award may be made to a higher rated, higher cost offeror where the decision is consistent with the evaluation factors and the agency reasonably determines that the technical superiority of the higher cost offer outweighs the cost difference. See Oklahoma Aerotronics, Inc.-- Recon., B-237705.2, Mar. 28, 1990, 90-1 CPD Para. 337.

As discussed above, we find that the evaluation of Motorola's proposal as outstanding and GSEI's proposal as acceptable was reasonably based. also find that the contracting officer's cost/technical tradeoff was rational and consistent with the evaluation factors.

In her source selection memorandum, the contracting officer detailed the evaluations of all offerors within the competitive range. She stated that GSEI's acceptable technical rating was based on acceptable ratings in the five technical subfactors and acknowledged that GSEI's rating under engineering approach was based upon an outstanding rating for "completeness" and acceptable ratings for "feasibility" and "understanding of the problem." She further noted that Motorola's outstanding rating was based upon outstanding ratings in "engineering approach," the most important technical subfactor, including "understanding of the problem," "feasibility," and "completeness," and an outstanding rating for "facilities and equipment."

In describing the evaluation of Motorola's proposal, the contracting officer specifically addressed Motorola's unique design and display of a thorough understanding through its comprehensive analysis, tradeoff studies, photographs, schematics, diagrams, concise narrative, and actual measured data. She also noted Motorola's engineering lab which contained the latest technology computer facilities, its specialized support software, transponder factory, and test equipment.

The contracting officer also addressed the cost difference between Motorola and GSEI, noting that Motorola proposed more than twice the man- hours proposed by GSEI. She found that GSEI was evaluated at moderate to high risk with regard to its man-hour proposal since the hours appeared low and GSEI's response to an IFN on the subject was considered incomplete. Motorola's proposed hours were found to be acceptable with adequate justification for all hours proposed. While DCAA found GSEI's costs, as adjusted downward, to be realistic for its particular proposed effort, this assessment is not inconsistent with the technical evaluators' assessment that GSEI's proposed hours were too low.

Overall, the contracting officer found that Motorola had conveyed superior knowledge and understanding of the problems associated with the design and fabrication of the MMB and test set prototypes. Since the contracting officer viewed Motorola's design as proven, while GSEI's was unproven due to a lack of measured data, and in light of the importance of technical ability and engineering approach, the contracting officer found it essential that the heaviest emphasis be placed on technical capabilities. Consequently, she determined that Motorola's proposal offered the best value to the government. Under the circumstances, we find her determination reasonable and sufficiently detailed.

Our conclusion is not changed by the cost difference between protester's and Motorola's proposals. Implicit in the contracting officer's cost/technical tradeoff is the determination that Motorola's outstanding approach is worth the difference in price. See Virginia Technology Assocs., supra. While the amount is great, we do not believe that that alone is reason to question the determination. See Dynamics Research Corp., B-240809, Dec. 10, 1990, 90-2 CPD Para. 471.

The protest is denied.

/1/ The RFP provided that up to two awards could be made for the effort. However, there were insufficient funds available to award a second contract.

/2/ A key difference between the approaches proposed by Motorola and GSEI concerns the type of diode used to power the two transmitters incorporated in the MMB. The specific name of the diodes used by GSEI, Motorola, and the third offeror, as well as the type band transmitter proposed by GSEI that was found "weak" by the agency, are not identified in this decision because the offerors' approaches are considered proprietary.

/3/ GSEI also contends that it was improper for the agency to consider one alternative/approach to be a weakness since an alternative to its proposed approach was unnecessary. Since this is a research and development contract to produce prototypes, we believe the agency was reasonable in finding that a proposal relying on a single, unsupported (by actual measured data) approach was weak.

GAO Contacts

Office of Public Affairs