Skip to main content

[Protests of Navy Contract Award for Engineering, Technical, and Logistics Support Services]

B-265865.3,B-265865.4 Jan 23, 1996
Jump To:
Skip to Highlights

Highlights

Two firms protested a Navy contract award for engineering, technical, and logistics support services relating to the sale of decommissioned ships, contending that the Navy: (1) unreasonably evaluated the technical proposals and failed to document its evaluation; (2) conducted an unreasonable cost-technical tradeoff analysis; and (3) failed to hold discussions. GAO held that the Navy: (1) reasonably evaluated the technical proposals despite its improper destruction of supporting documentation; (2) made a proper cost-technical tradeoff in selecting the awardee's technically superior, higher-priced proposal; and (3) was not required to hold discussions prior to award, since the solicitation advised bidders that award would be made on the basis of initial proposals. Accordingly, the protests were denied.

View Decision

Matter of: Southwest Marine, Inc.; American Systems Engineering Corporation File: B-265865.3; B-265865.4 Date: January 23, 1996

The destruction of individual evaluators' workpapers and scoring sheets by the procuring agency in a negotiated procurement, although improper, did not make the record inadequate for review by the General Accounting Office (GAO) where the remainder of the record, including the parties' arguments, explanations, and hearing testimony, sufficiently explained the agency's evaluation and source selection decision to allow the parties to present their arguments concerning the reasonableness of the agency's actions and for GAO to review the procurement. In a negotiated, best value procurement in which technical merit was stated to be significantly more important than cost/price, award was properly made to the higher-rated, higher-cost offeror where the agency reasonably evaluated the awardee's proposal as being significantly technically superior to that of the other offerors consistent with the stated evaluation criteria and the source selection official reasonably determined that technical merit of the awardee's proposal outweighed the cost advantages offered by the protesters' lower-rated, lower-cost proposals. Award may properly be made on the basis of initial proposals without conducting discussions where the solicitation incorporated Federal Acquisition Regulation section 52.215-16, Alternate III, which informed offerors that the agency intended to make award on the basis of initial proposals, and the contracting officer reasonably determined that discussions were not necessary to select the offer that represented the best value to the government, given the technical superiority of the awardee's proposal and the contracting officer's reasonable determination that no other offeror could improve its proposal to the level of the awardee's.

Attorneys

DECISION

Southwest Marine, Inc. (SWM) and American Systems Engineering Corporation (AMSEC) protest the award of a contract to BAV, a newly-formed joint venture of VSE Corporation and Booz, Allen & Hamilton, Inc., under request for proposals (RFP) No. N00140-95-R-F021, issued by the Department of the Navy, for engineering, technical, and logistics support services for various classes of decommissioned Navy ships to be sold or otherwise transferred to foreign governments. SWM and AMSEC protest the Navy's technical evaluation, cost/technical tradeoff analysis, and failure to conduct discussions.

We deny the protests.

The United States is selling, leasing, or otherwise transferring decommissioned Navy ships to foreign governments under the Foreign Military Sales (FMS) program; these ships are from various classes of ships, such as FF-1052 and FFG-7 frigates and DDG-2 cruisers, that will no longer be within the Navy's active inventory. The purpose of this procurement is to provide a single contract under which foreign recipients of the Navy's decommissioned ships can order a broad range of reactivation, overhaul, maintenance, training, and life-cycle support services for the ships and their systems.

A cost-plus-award-fee, indefinite delivery, indefinite quantity contract for 1 base year with 9 option years is contemplated by the RFP. The RFP estimated a total labor effort of approximately $24M [1] man-hours. The RFP's statement of work (SOW) detailed the services that could be ordered under the contract, including ship reactivation and overhaul, maintenance and maintenance planning, design, configuration management, field engineering, spare parts management and procurement, and training. Among other things, the offeror (or an offered subcontractor) was required to have a current master ship repair agreement (MSRA) and a valid Naval Sea Systems Command (NAVSEA) master ordinance repair qualification.

Instructions for the preparation of proposals were provided, detailing the information required to allow for the evaluation of the technical and cost proposals. The RFP provided that award would be made on a best value basis and stated that technical merit was "significantly more important than" price in the overall evaluation of proposals. The following technical evaluation factors and subfactors were stated:

1. Technical Approach a. Sample tasks b. Understanding and approach c. Specific technical capability d. Unique ideas and capabilities

2. Corporate Past Experience

3. Personnel Qualifications

4. Management Plan/Approach

5. Facilities

Offerors were informed that the technical evaluation factors were listed in descending order of importance, and that within the technical approach factor the subfactors were listed in descending order of importance. In addressing the technical approach evaluation factor, offerors were instructed to demonstrate their capability to perform and their understanding of the SOW requirements. Under the sample tasks subfactor, the RFP identified 12 sample tasks that offerors were to address, and informed offerors that their sample task responses should demonstrate their "understanding of the needs, programs, and objectives contained in the SOW" and their "understanding of FMS procedural and administrative requirements and soundness of approach to the specific aspects of the Sample Tasks." Offerors were also informed that they (or their subcontractors) should demonstrate prior experience "in efforts of similar complexity and difficulty, and in fields applicable to the services requested in the SOW," and that "particular emphasis should be placed on both recent (past three to five years) or current on-going efforts and on [Department of Defense], particularly Navy and FMS efforts."

The RFP provided for the evaluation of proposed costs on the basis of cost realism plus proposed award fee. In this regard, offerors were required to provide a complete and detailed cost breakdown to support their proposals. Subcontracts in excess of $500,000 were required to be supported by cost and price information prepared and signed by the subcontractor, and all subcontracts regardless of dollar value were required to be adequately documented to allow a cost realism and reasonableness evaluation.

The RFP incorporated Federal Acquisition Regulation (FAR) section 52.215-16, Alternate III, which states that the government intends to make award without conducting discussions and encourages offerors to submit their best offers in their initial proposals. This clause also reserves the agency's right to conduct discussions if necessary.

The Navy received proposals from eight offerors, including BAV, AMSEC, and SWM. The technical proposals were evaluated by a nine member technical evaluation committee (TEC), which used an adjectival rating scheme to evaluate proposals under each technical evaluation factor and subfactor. [2] Each proposal was in turn evaluated by all the evaluators, with the order of review determined by random selection. Hearing Transcript (Tr.) at 506-508. [3] Each evaluator, after reviewing a proposal, prepared narrative comments and assigned an adjectival rating under each evaluation factor and subfactor. Tr. at 35-36. The evaluators then met in committee to discuss their initial written reviews of the proposal; the individual evaluators made adjustments, if any, to their own comments and adjectival ratings as a result of their committee discussions. Tr. at 44-45, 60-61, 415-417, 510. After the individual adjustments were made, the evaluators' comments and ratings were collated into a single evaluation document for each proposal. This document was subject to further discussions in committee; narrative comments and ratings were amended, changed or deleted based upon the evaluators' evolving consensus view. [4] Tr. at 240-245, 287-288.

Ultimately, as a result of this iterative process, under which each proposal was subject to at least three reviews, Tr. at 532, the TEC arrived at a single consensus view for each proposal. A final evaluation report was prepared that documented the TEC's consensus evaluation ratings for each proposal; this document provided an adjectival rating and narrative comments for each evaluation factor and subfactor. The report, however, did not document any weaknesses or "drawbacks" in SWM's or AMSEC's proposals that resulted in the protesters' proposals being evaluated as weaker than BAV's. Tr. at 872.

After the TEC accepted the final report, the underlying evaluators' notes, comments, and ratings, and the TEC's preliminary consensus documents were destroyed. Tr. at 833, 840-841. Subsequent to the protests, the Navy discovered four computer diskettes containing some individual evaluator notes and ratings and some of the TEC's preliminary consensus reports for SWM and AMSEC. Tr. at 841-842. These documents were provided to counsel for the parties under the protective order.

BAV's proposal received the highest rating from the TEC, while AMSEC's and SWM's proposals were evaluated as being technically equivalent and second highest rated. The TEC assigned the following adjectival ratings to the parties' proposals in its report: [5]

BAV AMSEC SWM

1. Technical Approach HA A A a. Sample tasks HA A A b. Understanding HA A A c. Technical capability HA A A d. Unique ideas HA HA A

2. Corporate past experience HA A A

3. Personnel qualification A A A

4. Management plan HA HA A

5. Facilities HA HA HA

OVERALL RATING HA A A

BAV's proposal's overall highly acceptable technical rating reflected the evaluators' judgment that BAV had demonstrated a superior level of knowledge, experience, and proven capabilities for all the SOW and RFP requirements. For example, under the sample tasks subfactor, BAV provided comprehensive responses that not only provided a workable solution to each task, but demonstrated understanding of the interrelationship of all the SOW elements and the unique FMS environment under which the contract services will be rendered. E.g., Tr. at 70-71. BAV's proposal also demonstrated a number of unique and outstanding approaches to the SOW requirements, such as, for example, BAV's proposed use of [deleted] to address the problems of material/parts management and locating parts for systems that are no longer in the Navy's inventory, Tr. at 72-75, and its comprehensive training program for [deleted]. Tr. at 71, 74-76. Under the technical approach subfactor, BAV's proposal was judged to provide

"a comprehensive description of how to meet the requirements of the statement of work in an FMS environment and how the interrelationships worked between the different parts of that statement of work. Each of these [SOW elements] is not a stand alone item. [BAV] demonstrated a thorough understanding of what the requirements were, how to implement plans and programs in those areas that were tailored to the FMS environment, not just telling us how the U.S. Navy does it, and gave a detailed, thorough and it was a very straightforward treatment." Tr. at 110.

As another example, under the corporate past performance factor, BAV's proposed contract team was judged to offer a breadth of experience that addressed all elements of the SOW and a substantial amount of direct FMS experience. Tr. at 127-128. This experience was demonstrated and documented by a large number of contract citations. Tr. at 128. Overall, in the evaluators' view, BAV's highly acceptable proposal was vastly superior to that of any of the other acceptable proposals, including SWM's and AMSEC's. Tr. at 109, 146.

In contrast to BAV's overall highly acceptable rating, the evaluators determined that SWM's and AMSEC's proposals evidenced an overall acceptable approach, understanding, capability, and experience to satisfy the contract requirements. Each of the protesters' proposals was assessed as having some strengths, but neither protester provided, in the evaluators' judgment, as strong or comprehensive a proposal as BAV's. In particular, the Navy found that SWM's and AMSEC's proposals did not evidence the same level of understanding of the interrelationship of the various SOW elements and of the unique requirements presented by performing these services in an FMS environment. In addition, neither protester's experience was judged to be as complete or strong as BAV's.

Concurrent with the TEC's evaluation of proposals, the SSA independently reviewed the technical proposals, Tr. at 955, 1117-1118, had a number of conversations with the TEC chair regarding the process of the TEC's initial evaluation and its initial findings, and reviewed an early draft of the TEC's evaluation report. Tr. at 955, 957. After the TEC's final evaluation report was provided to the SSA, he spent a week comparing and validating the evaluation findings against his own review of the technical proposals. Tr. at 958. While the SSA found some "minor" points on which he disagreed with the evaluation findings, he agreed with the assigned adjectival ratings and that BAV's proposal was "clearly and vastly superior [to that] of the other offerors"; the SSA "adopted" the evaluation findings as his own. Tr. at 958-961, 967, 969, 974.

A cost evaluation was also performed to assess the realism and reasonableness of the offerors' cost proposals. AMSEC's, SWM's, and BAV's total proposed cost-plus-award-fees were evaluated as follows:

Proposed Evaluated

AMSEC $853,124,360 $853,124,360 SWM $945,007,862 $945,007,862 BAV $1,060,156,697 $1,079,770,912

The SSA requested that the TEC chair perform a cost/technical tradeoff analysis of the offers and "advise the [SSA] whether award to the technically superior offeror warrants the higher costs." The TEC chair was provided with cost evaluation information and met with the SSA to discuss the cost/technical tradeoff evaluation. Tr. at 764. After receiving input from the TEC members, the TEC chair provided the SSA with a written cost/technical tradeoff analysis that concluded that the evaluated superiority of BAV's proposal outweighed AMSEC's or SWM's substantial cost advantages. The TEC chair's cost/technical tradeoff identified a number of advantages in BAV's proposal that the TEC chair believed evidenced technical superiority which outweighed the cost advantages offered by the protesters' proposals. Generally, these advantages were (1) BAV's offer of more comprehensive and intensive training that would allow for "a faster and more comprehensive level of self-sufficiency for the FMS customer"; (2) BAV's superior understanding of the interrelationship of the SOW elements; awareness of the roles of and coordination required for the contractor, the agency, and the FMS customer; and systematic approach to accomplishing the SOW services--all of which would result in efficient level of effort usage, elimination of rework, and reduction of government oversight; (3) BAV's offer of "proven preventative solutions to anticipated problem areas" that would result in substantial cost avoidance to the FMS customer; and (4) BAV's proposed material support system that would manage the government's spare parts inventory, which had an approximate value of $400M. Tr. at 800-801.

The TEC chair's cost/technical tradeoff analysis was provided to the agency's source selection advisory council (SSAC), which also received copies of the proposals, the TEC's evaluation report, and the cost evaluation. Tr. at 988-989. The SSA was present for the SSAC's review and discussion of the proposals and evaluation. The SSAC's discussion of the value of BAV's technically superior proposal focused on the need to provide high quality services to FMS customers; specifically, the SSAC "elaborate[d] on a credibility gap that . . . existed between NAVSEA and their [FMS] customers," stressed the importance of training FMS users to attain self-sufficiency, and the need to manage the spare parts inventory and support the "FMS-unique items." Tr. at 990-991. The SSAC concurred with the TEC's cost/technical tradeoff analysis and also recommended award to BAV.

On the basis of the TEC's evaluations, the TEC chair's cost/technical tradeoff analysis, the SSAC's recommendation, as well as his own review, the SSA selected BAV's proposal for award. In the written source selection decision, the SSA noted that:

"The inherent quality of the BAV proposal will result in less rework on the part of the contractor, and less required Government oversight. The technical superiority of the BAV proposal is not overcome by the simple expenditure of additional hours on behalf of any lower cost/technically inferior offerors. Given the higher quantity and quality of contract deliverables, and given the inherent quality of the BAV proposal, unmatched by any other offer, which will result in less required rework and less required Government oversight, the SSA has determined the benefits provided by the BAV proposal outweigh the differences between the BAV cost realistic position and the cost realistic positions of the other offerors . . . ."

In the SSA's judgment, BAV would present FMS customers with a level of support that would generate the highest level of assurance in the Navy's capability and commitment, which would enhance the marketability of decommissioned Navy ships and increase the volume of work that FMS customers would choose to order under the contract. [6] The SSA also testified that there was a "real value" in the sale of government spare parts through this support services contract. Tr. at 1126-1127.

The SSA also decided that there was no need to conduct discussions before making award. Specifically, the SSA noted that the SOW invited creative and innovative approaches, but that only BAV had "maximized this opportunity by providing the most creative and innovative technical approach." In the SSA's view, discussions in this area would not be appropriate. The SSA also noted that there were no deficiencies or significant weaknesses in any of the acceptable proposals that required discussions and was convinced that the other offerors could not improve their proposals to the level of BAV's. Tr. at 1127. Finally, the SSA noted that offerors were cautioned by the RFP that the government intended to make award on the basis of initial proposals and that therefore offerors should have presented their best offer in their initial proposals.

Award was made to BAV, and these protests followed.

SWM and AMSEC first protest the Navy's evaluation of technical proposals. Specifically, SWM argues that the Navy gave undue weight to the experience of BAV's proposed team in evaluating BAV's proposal under the sample tasks and specific technical capability subfactors; SWM contends that since the RFP provided for the evaluation of corporate past experience under a separate evaluation factor, the Navy should not have considered BAV's experience under any other factor or subfactor. SWM also disagrees with the adjectival ratings assigned by the TEC for its proposal; SWM contends that since the TEC in its final report identified some areas in which SWM's proposal exceeded the RFP requirements, under the procurement's source selection plan, its proposal should have been rated highly acceptable. AMSEC argues that the Navy evaluated its and BAV's proposals unequally, finding that strengths in BAV's proposal merited a highly acceptable rating while similar strengths in AMSEC's proposal merited only an acceptable rating. Also, AMSEC argues that the Navy misevaluated its corporate past experience, which the protester believes should have been assessed as highly acceptable. Both protesters argue that because the Navy destroyed the evaluators' and TEC's workpapers and comments, which formed the basis for the final evaluation report, the agency did not adequately document its evaluation and source selection such that we can judge the rationality of the agency's actions.

In determining whether a particular evaluation conclusion is rational, we examine the record to determine whether the judgment was reasonable and in accord with the evaluation criteria listed in the solicitation. Abt Assocs., Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD para. 223. Such judgments are by their nature often subjective; nevertheless, the exercise of these judgments in the evaluation of proposals must be reasonable and must bear a rational relationship to the announced criteria upon which competing offers are to be selected. Bunker Ramo Corp., 56 Comp.Gen. 712 (1977), 77-1 CPD para. 427; Hydraudyne Sys. and Eng'g B.V., B-241236; B-241236.2, Jan. 30, 1991, 91-1 CPD para. 88.

Implicit in the foregoing is that these judgments must be documented in sufficient detail to show that they are not arbitrary. See FAR section 4.801(b); 15.608(a)(3); 15.612(d)(2); Wadell Eng'g Corp., 60 Comp.Gen. 11 (1980), 80-2 CPD para. 269; Benchmark section, Inc., B-247655; B-247655.2, Feb. 4, 1993, 93-1 CPD para. 133. In particular, the agency's technical evaluation documentation is required to include "[a]n analysis of the technically acceptable and unacceptable proposals, including an assessment of each offeror's ability to accomplish the technical requirements." FAR section 15.608(a)(3)(ii). While an agency is not required to retain every document or worksheet generated during its evaluation of proposals, the agency's evaluation must be sufficiently documented to allow review of the merits of a protest. KMS Fusion, Inc., B--242529, May 8, 1991, 91-1 CPD para. 447. In this regard, evaluators' notes and workpapers may or may not be necessary to determine the reasonableness of the agency's evaluation. See id.; Department of the Army--Recon., B-240647.2, Feb. 26, 1991, 91-1 CPD para. 211. Where an agency fails to document or retain evaluation materials, it bears the risk that there is inadequate supporting rationale in the record for the source selection decision and that we will not conclude that the agency had a reasonable basis for the decision. Engineering and Computation, Inc., B-261658, Oct. 16, 1995, 95-2 CPD para. 176; American President Lines, Ltd., B-236834.3, July 20, 1990, 90-2 CPD para. 53.

In determining the rationality of an agency's evaluation and award decision, we do not limit our review to contemporaneous evidence, but consider all the information provided, including the parties' arguments, explanations, and hearing testimony. Benchmark section, Inc., supra; KMS Fusion, Inc., supra; Hydraudyne Sys. and Eng'g B.V., supra. While we consider the entire record, including the parties' later explanations and arguments, we accord greater weight to contemporaneous evaluation and source selection material than to arguments and documentation prepared in response to protest contentions. DynCorp, 71 Comp.Gen. 129 (1991), 91-2 CPD para. 575.

Here, the Navy's destruction of the evaluators' notes and workpapers left a written record that did not meet the requirements of FAR section 15.608(a)(3)(ii). The contemporaneous documentation retained by the Navy--the TEC's final report, the TEC chair's cost/technical tradeoff document, and source selection decision--did not adequately explain why BAV's proposal was judged to be technically superior to SWM's and AMSEC's. Specifically, while the TEC report described BAV's proposal's strengths and advantages to justify its "highly acceptable" rating, it did not explain why the protesters' proposals, which appeared from this document to have similar strengths, were not essentially technically equivalent. Accordingly, the retained documentation was not sufficient to support the evaluation results. Indeed, a hearing was convened at our request, in part, because of the inadequacy of the evaluation documentation in the record, which did not fully explain the agency's evaluation, Tr. at 286-287, and the hearing testimony revealed that a number of identified strengths and weaknesses for all offerors' proposals were not recorded in the TEC's final evaluation report. Tr. at 336-338, 397-398.

While the Navy's document destruction was improper, we will not disrupt an agency's procurement on this basis alone where the protest record is otherwise adequate for our review. See Hydraudyne Sys. and Eng'g B.V., supra. SWM and AMSEC assert that the evaluators' notes and workpapers are necessary here to allow for our review of the procurement; they correctly note that the evaluators could not recall in their hearing testimony all the details of the lengthy evaluation process and that there were some minor inconsistencies in the evaluators' testimony. [7] This, however, does not establish the inadequacy of the protest record. The testimony of the agency's evaluators, TEC chair, and SSA explained the basis of the agency's evaluation conclusions, and, more specifically, why BAV's proposal was determined to be technically superior to SWM's and AMSEC's; they agreed on the major aspects of the evaluation and the evaluated differences among the parties' proposals. Looking at the record as a whole, we believe that it adequately explains the agency's actions so as to allow the parties to present their arguments concerning the reasonableness of the agency's source selection and to allow our Office to effectively review the matter. Id.

From our detailed review of this record, we conclude that the agency's evaluation of technical proposals and its cost/technical tradeoff decision were reasonable.

SWM argues that the Navy gave inappropriate weight to experience in the evaluation of BAV's proposal under the sample tasks and specific technical capability subfactors. The record does not support this allegation. The RFP provided that an offeror's overall corporate experience would be evaluated under the corporate past experience factor, and the evaluation documentation reflects that BAV's overall experience was assessed under this evaluation factor. The RFP also instructed offerors to demonstrate their understanding and capabilities under the technical approach evaluation factor, which includes the sample tasks and technical capabilities subfactors. The record shows that BAV's specific understanding, approach, and capabilities were assessed under these subfactors. For example, sample task three asked offerors to describe how they would replenish the repair part inventory for equipment that is obsolete to the Navy infrastructure. The Navy found that, for this sample task: "[deleted]."

While SWM complains that the Navy in evaluating BAV's response to this sample task considered the experience of BAV's team member in performing a similar task for the United Kingdom, the agency's evaluation comments demonstrate that the citation to the experience of BAV's team member was for the purpose of documenting BAV's capabilities, which is a matter that the RFP provided would be evaluated under these subfactors. [8]

SWM also complained prior to the hearing that the TEC report identifies a number of areas in which SWM's proposal exceeded the RFP requirements, but does not explain why these "excesses" were not considered sufficiently beneficial to the government to warrant receiving highly acceptable ratings. Hearing testimony was elicited from a number of the agency's evaluators explaining why, in their judgment, the evaluated "excesses" in SWM's proposal did not merit highly acceptable ratings under the evaluation factors. See, e.g., Tr. at 144-161, 550-554, 561-564, 743-752. SWM has not addressed this testimony or the evaluators' explanations in its post-hearing comments. Instead, SWM continues to argue that the contemporaneous TEC report does not adequately explain the agency's evaluation. This, however, ignores the hearing testimony that a "major difference" between BAV's overall highly acceptable proposal and SWM's acceptable proposal was that BAV demonstrated comprehensive understanding of the SOW requirements and how they related to executing tasks in an FMS environment, while SWM's proposal merely described the standard Navy approach to accomplishing the required tasks without demonstrating an understanding of the "uniqueness of doing things in an FMS environment" or how things would have to be tailored to satisfy an FMS customer. Tr. at 144-145. In other words, the Navy found that while SWM exceeded RFP requirements in various respects (particularly with regard to shipyard services), it did not in its proposal reflect an appreciation of the FMS nature of the procurement to the extent that BAV did. The record supports the Navy's findings, and given SWM's failure to rebut the agency's explanations in the hearing testimony regarding its evaluation of SWM's proposal, we conclude that the ratings of that proposal were reasonable.

AMSEC also challenged the agency's evaluation of its technical proposal, arguing that strengths identified in BAV's proposal were also present in AMSEC's and that in particular AMSEC's corporate experience should have been assessed as highly acceptable. In this regard, AMSEC points to some of the initial, pre-consensus evaluator comments concerning AMSEC's proposal, complains that its proposal initially received a higher rating than was presented in the TEC's consensus report, and suggests that the TEC report does not reflect the evaluators' conclusions.

First, regarding the pre-consensus evaluation documents to which AMSEC draws our attention, the record shows, as noted above, that the agency's evaluation was an iterative process which began with the individual evaluators' initial comments on the proposals, continued through committee discussion which resulted in changes to the evaluators' individual views, and ended with a consensus panel finding for each proposal that was documented in the TEC's evaluation report. [9] E.g.,Tr. at 414-417. Thus, the fact that an evaluator's preliminary evaluation comments or ratings changed does not indicate, as AMSEC asserts, that the TEC report does not reflect the agency's evaluation conclusions. Moreover, the record indicates that all the offers initially received high evaluation scores prior to the committee discussions after which individual evaluators adjusted their ratings. Tr. at 430.

Regarding the evaluation of AMSEC's proposal, we received extensive hearing testimony concerning the evaluation of AMSEC's proposal under the technical approach and corporate past experience factors, which highlighted the evaluated differences between BAV's and AMSEC's proposals. In general, BAV'S proposal under the sample tasks and understanding and approaches subfactors was found to provide a much more comprehensive response to the SOW requirements than AMSEC's, and to demonstrate BAV's understanding of the interrelationship of all the SOW elements and the unique aspects of accomplishing the required services in an FMS environment. Tr. at 70-71, 109-110. Also, a number of strengths or beneficial excesses were identified in BAV's proposal that supported its evaluated superiority. For example, BAV offered an existing material management system and database to address the problem of managing and acquiring spare parts for equipment that is no longer supported in the Navy's inventory. [10] Tr. at 71-74. As another example, BAV offered MSRA contractors on [deleted] to allow for flexibility in scheduling ship reactivation and repairs. Tr. at 113.

AMSEC does not dispute the Navy's contentions concerning the comprehensiveness of BAV's proposal or that BAV proposal demonstrated a better understanding of the interrelationship of SOW elements and of the FMS environment; in this regard, AMSEC's expert witness/consultant testified that he found the Navy's evaluation of BAV's and AMSEC's proposals under the technical approach factor to be generally reasonable. Tr. at 1183, 1222-1223. Nevertheless, AMSEC complains that some features of its proposal are equivalent to strengths identified for BAV, but its proposal was not given the same credit.

Specifically, AMSEC complains that just as BAV received credit for its material management system and database as a beneficial excess, AMSEC should have received credit for its proposed material management system--the [deleted]. The record shows that the Navy considered the [deleted] system, which AMSEC's proposal stated was currently being used to [deleted], but concluded that AMSEC's proposal provided insufficient information to allow the agency to assess this system as a beneficial excess. Tr. at 83, 173, 331-332, 558-559, 1082. In contrast, BAV fully explained its proposed material management system; in addition, the TEC and SSA [deleted]. Tr. at 992-992, 1079-1082. AMSEC has not shown the agency's conclusions regarding AMSEC's failure to explain the [deleted] system to be unreasonable.

AMSEC complains that the Navy assessed BAV's offer of MSRA contractors on [deleted] to be a proposal strength but AMSEC did not receive appropriate credit for its offer of MSRA contractors on [deleted]. The record reflects that the Navy recognized that AMSEC had offered MSRA contractors on [deleted], but concluded that because AMSEC's [deleted]. Tr. at 118. AMSEC has also not shown this conclusion to be unreasonable.

AMSEC also complains that the Navy assessed BAV's proposed training as a proposal strength, but that AMSEC's proposed equivalent training was not so assessed. While it is true that some aspects of AMSEC's proposed training approaches appear similar to that offered by BAV, the record does not support AMSEC's contention that the training was equivalent. Rather, the record supports the reasonableness of the Navy's evaluation that BAV's proposal demonstrated a more comprehensive overall training program; in this regard, the Navy found that, unlike the other offers, BAV's proposal did not simply provide training where requested by the RFP, but demonstrated "a philosophy that maximized the transfer of knowledge throughout the various tasks called for in the RFP." In accordance with this approach, BAV proposed integrated training for [deleted] (where pertinent) throughout the performance of contract tasks. Also, only BAV proposed providing [deleted] training.

AMSEC also disputes the agency's assessment of BAV's proposed "preventative solutions" as a unique proposal strength; AMSEC argues that its proposal offered the same preventative solutions. We find from our review of the record that while both firms offered preventative solutions to potential maintenance problems, BAV's proposal was clearer and more comprehensive in this area than AMSEC's. We find reasonable this aspect of the agency's evaluation.

AMSEC also challenges the Navy's evaluation of its corporate past experience, asserting that its proposal should have received at least the rating received by BAV's proposal. As noted above, BAV's proposal was assessed as highly acceptable under the corporate past experience factor because BAV was found to offer directly relevant experience in all SOW elements and this experience was supported by a large number of contract citations [deleted]. AMSEC's corporate past experience was evaluated as being on the high end of acceptable; the record indicates that AMSEC's proposal did not receive a higher rating for this factor because, in the Navy's view, AMSEC did not demonstrate the level of comprehensive and recent experience that BAV had provided, Tr. at 351, and the Navy had a number of concerns with the past experience demonstrated in AMSEC's proposal. These concerns were that AMSEC proposed an extremely large team of subcontractors to perform this contract, but that many of AMSEC's proposed subcontractors were not priced in its cost proposal; the list of AMSEC's "unpriced" companies included some companies that were identified as significant to AMSEC's contract performance. Tr. at 129-130. Also, AMSEC did not provide contract references for a number of its proposed subcontractors and the contract references that were provided were viewed as being dated in a number of instances. Tr. at 130-135.

AMSEC disagrees with the Navy's assessment of its corporate past experience and argues that it was unreasonable for the agency to view the unpriced companies it proposed as less committed than companies that were priced in its cost proposal. [11] In this respect, AMSEC contends that it was not required to price every firm it proposed to have available to perform task orders under the contract. Here, contrary to AMSEC's arguments, the Navy did not disregard the unpriced firms in assessing AMSEC's proposed technical capabilities and experience, but considered the risk associated with an offer of unpriced firms--AMSEC's offer of firms that were not priced in its proposal was only one element the Navy weighed in assessing AMSEC's experience to be at the high end of the acceptable range. We think that the agency was reasonably concerned by AMSEC's failure to price all its proposed team members given AMSEC's failure to explain what arrangements it had with these firms. Because the RFP required offerors to provide cost information for all subcontracts regardless of dollar amount, the agency could reasonably find that these firms were proposed not as a subcontractors to AMSEC's contract but as potential suppliers. [12]

AMSEC also disputes the agency's evaluation of the breadth and currency of AMSEC's identified contract references. In this regard, AMSEC's expert witness/consultant testified that in his opinion AMSEC's overall experience should have at least been rated equal to that of BAV's because AMSEC offered "ship repair oriented shipyards" with more direct experience in the building or repairing of the classes of ships to be supported under this contract. Tr. at 1190-92. The Navy responds, however, that the RFP required more than simply direct ship repair experience with the specific classes of ships identified in the RFP [13] and that, in any event, AMSEC failed to provide contract references for a number of the companies for which it asserted specific experience and that some of the contract references given for specific experience with the classes of ships identified in the RFP were dated (i.e., more than 5 years old and in one case more than 20 years old). In contrast, the Navy evaluators found that BAV presented contract references for all its proposed team that were more recent and showed experience with the same or similar ship systems presented by the classes of ships to be supported by this contract. See, e.g., Tr. at 188-207. While AMSEC disagrees with the Navy's judgment concerning the respective merit of the two firms' past experience, it has not shown the Navy's judgment to be unreasonable.

AMSEC also argues that the Navy evaluated its and BAV's proposal disparately in assessing the two firms' past experience because the SSA stated that he was concerned that two of the large contracts AMSEC represented as demonstrating experience with large, complex contracts were actually performed by other corporate subsidiaries of Science Applications International Corporation, while the Navy accepted as relevant experience contracts that were performed by the corporate joint venturers of BAV, even though BAV is a newly organized entity without any past experience. The SSA, however, testified that these matters were not significant to the SSA, see Tr. at 964, and we find that they had little impact on the evaluation of AMSEC's overall experience. [14]

As for the Navy's cost/technical tradeoff, AMSEC asserts that the SSA's determination does not demonstrate that the technical merit of BAV's proposal, vis-a-vis AMSEC's proposal, warrants the payment of a more than $200M cost premium. [15]

In a negotiated procurement, the government is not required to make award to the lowest-cost, technically acceptable offeror unless the RFP specifies that cost will be determinative. General Servs. Eng'g, Inc., B-245458, Jan. 9, 1992, 92-1 CPD para. 44. Source selection officials have broad discretion to determine the manner and extent to which they will make use of the technical and cost evaluation results in negotiated procurements. Grey Advertising, Inc., 55 Comp.Gen. 1111 (1976), 76-1 CPD para. 325. In deciding between competing proposals, cost/technical tradeoffs may be made, the propriety of which turns not on the difference in technical scores or ratings, per se, but on whether the source selection officials judgment concerning the significance of that difference was reasonable and adequately justified in light of the RFP evaluation scheme. DynCorp, B-245289.3, July 30, 1992, 93-1 CPD para. 69. Award may be made to a higher-rated, higher-cost offeror where the decision is consistent with the evaluation criteria and the agency reasonably determines that the technical superiority of the higher-priced offer outweighs the cost difference. Sabreliner Corp., B-242023; B-242023.2, Mar. 25, 1991, 91-1 CPD para. 326.

Here, the record supports the SSA's selection of BAV's proposal for award on the basis of BAV's evaluated technical superiority, notwithstanding AMSEC's proposal's much lower evaluated cost. The RFP informed offerors that technical merit was "significantly more important than" cost/price, and BAV's proposal was reasonably judged by the agency's evaluators and the SSA to be significantly superior to that of AMSEC. As noted above, four significant attributes of BAV's superior proposal were specifically identified by the agency as outweighing AMSEC's 26-percent evaluated cost advantage: (1) BAV's proposed training; (2) BAV's superior understanding of the interrelationship of the SOW elements; awareness of the roles of and coordination required for the contractor, the agency, and the FMS customer; and systematic approach to accomplishing the SOW services; (3) BAV's offer of "proven preventative solutions to anticipated problem areas"; and (4) BAV's proposed material support system. In addition, the SSA believed that instilling confidence of the FMS customer in the Navy's ability to deliver quality ship repair, maintenance, and life-cycle support services was crucial to enticing FMS customers to purchase or lease decommissioned Navy ships, each of which would sell for between $100 to $130M. Tr. at 1000, 1013. Likewise, the SSA believed that providing a high quality contract, which would encourage FMS customers to order services, would allow the government to sell its spare parts inventory at market value; the failure to sell these parts that are no longer needed by the Navy would result in the parts being scrapped. Tr. at 1002-03.

AMSEC attacks the Navy's cost/technical tradeoff analysis on a number of bases, none of which we find demonstrates that the agency's judgment was unreasonable. AMSEC first asserts that some of the superior features identified by the agency in BAV's proposal--specifically, BAV's proposed training, material management system, preventative solutions, and corporate past experience--are matched in AMSEC's proposal. These allegations essentially challenge the agency's technical evaluation, which, as noted above, we found reasonable in its conclusion that BAV's proposal was far superior to AMSEC's in this areas.

AMSEC next asserts that the SSA relied upon the TEC's evaluation report in performing his cost/technical tradeoff and that this report did not document all the TEC's findings regarding the proposals. While the TEC report did not document all of the TEC's evaluation findings, AMSEC's argument ignores the fact that the SSA performed an independent review of the proposals, which validated the TEC's adjectival evaluation ratings and, along with the TEC chair's cost/technical tradeoff recommendation, provided the SSA with the basis to make his source selection decision.

Finally, AMSEC asserts the SSA failed to provide sufficient weight to cost in his cost/technical tradeoff analysis. We disagree and find that the SSA carefully considered the respective costs of the offerors in assessing which proposal was most advantageous to the government; in this regard, the SSA properly recognized that technical merit was stated by the RFP to be significantly more important than cost. While AMSEC believes that the Navy was required to, but did not, "quantify" the value of BAV's evaluated superiority to demonstrate that the merit of BAV's proposal outweighed AMSEC's cost advantage, there is no requirement that an agency quantify the value of technical superiority vis-a-vis low cost/price to determine the best value to the government. Picker Int'l, Inc., B-249699.3, Mar. 30, 1993, 93-1 CPD para. 275. Rather, the relative value to be assigned respectively to technical merit and cost/price is a matter accorded to the subjective business judgment of the source selection official, whose judgment we will question only where that judgment is shown to be unreasonable or not in accord with the stated evaluation criteria. Id.

In sum, we find that SSA reasonably determined that the superior technical merit evaluated in BAV's proposal outweighed the substantial cost advantage offered by AMSEC. In so concluding, the SSA gave appropriate weight to cost within the evaluation scheme stated by the RFP.

SWM and AMSEC also protest the agency's determination to make award on the basis of initial proposals without conducting discussions. The protesters assert that the conduct of discussions would have allowed them to improve their technical proposals such that, considering the firms' much lower costs, the best value outcome in this procurement could have been different.

Where, as here, an RFP sets forth the provisions of FAR section 52.215-16, Alternate III, advising offerors of the agency's intent to award without conducting discussions, the agency may properly do so, provided the contracting officer determines that discussions are unnecessary. FAR section 15.610(a)(4); Infotec Dev., Inc., B-258198 et al., Dec. 27, 1994, 95-1 CPD para. 52. The discretion of the contracting officer to determine whether or not to hold discussions is not unfettered; we will review the exercise of that discretion to ensure that it was reasonably based on the particular circumstances of the procurement, including consideration of the proposals received and the basis for the selection decision. See Facilities Management Co., Inc., B-259731.2, May 23, 1995, 95-1 CPD para. 274; The Jonathan Corp.; Metro Mach. Corp., B-251698.3; B-251698.4, May 17, 1993, 93-2 CPD para. 174.

Here, the Navy reasonably determined that BAV's proposal contained no deficiencies and was clearly technically superior to SWM's and AMSEC's, and that this superiority clearly outweighed SWM's and AMSEC's cost advantage. Furthermore, the record supports the SSA's conclusion that while SWM and AMSEC could improve their proposals to some extent there was no reasonable possibility that either protester could improve its proposal to the level of BAV's proposal, given BAV's innovative approach which the SSA properly determined should not be disclosed to the protesters through discussions. Thus, the agency reasonably had no doubt as to which offer represented the best value to the government. Under these circumstances, the Navy could properly make award to BAV on the basis of initial proposals. Compare Information Spectrum, Inc., B-256609.3; B-256609.5, Sept. 1, 1994, 94-2 CPD para. 251 (discussions were not necessary where the agency reasonably could determine which offer represented the best value to the government) with The Jonathan Corp.; Metro Mach. Corp., supra (discussions were necessary where the agency could not reasonably determine which proposal represented the best value to the government, given the significant discrepancy between the agency's cost realism estimate and the cost proposals received and the closeness of the competition); see also TRW, Inc., B-254045.2, Jan. 10, 1994, 94-1 CPD para. 18.

The protests are denied. [16]

Comptroller General of the United States

1. "M" equals a million.

2. During some part of the evaluation process, evaluators also used a numerical evaluation scheme. This rating scheme was not used in the TEC's consensus report and was not provided to the source selection authority (SSA) as a part of the TEC's final evaluation report.

3. A hearing was held in this protest pursuant to 4 C.F.R. Sec. 21.5 (1995) to receive testimony from a number of the Navy's evaluators, the TEC chair, the SSA, and consultant/expert witnesses for AMSEC and BAV, regarding the evaluation of technical proposals, source selection decision, and decision to make award on initial proposals without conducting discussions.

4. Individual evaluators changed their views throughout this process. The evaluators had different areas of expertise, and evaluators would change their views after the group discussions where an evaluator became convinced that initial perceived strengths or weaknesses were not valid. Tr. at 35-36; 428-430; 489-490.

5. "HA" refers to highly acceptable, which the source selection plan defines as a highly responsive proposal that meets and exceeds the desired performance and the excess is beneficial to the Navy; "A" refers to acceptable, which was defined as a responsive proposal that demonstrates an understanding of, and satisfies, the Navy's requirements. Proposals could also be evaluated as unacceptable but capable of being made acceptable and unacceptable but not capable of correction without a major proposal revision.

6. The SSA testified that the Navy competes with other countries (for example, England, Germany, and the Netherlands) to sell decommissioned ships. Tr. at 1126.

7. Given the lengthy and iterative evaluation process that entailed the review of each proposal in total by each evaluator, the numerous committee discussions, and the rescoring of proposals by the evaluators, and given the passage of 6 months since these evaluations were performed, the evaluators' lack of total recall of all details of their evolving evaluation of proposals, and the minor inconsistencies in their testimony, are understandable, particularly since some individual evaluators' notes and ratings, and the preliminary consensus reports were destroyed.

8. SWM provides numerous other examples where the agency mentioned BAV's specific experience in evaluating BAV's proposal under the sample tasks and specific technical capability subfactors. We find that in each instance the Navy was properly assessing BAV'S capabilities as provided for by the RFP.

9. Both protesters challenge the TEC evaluation report as representing a "coerced" consensus that they imply does not reflect the views of the individual evaluators. As AMSEC acknowledges, a consensus approach to evaluating proposals is not unreasonable. Moreover, the record shows that all the evaluators agreed with the consensus opinion expressed in the TEC report.

10. Spare parts management and acquisition was seen as one of the more important and difficult aspects of the contract work. Tr. at 173, 1004. As one evaluator noted "if you don't have parts, you can't fix things and if you can't fix them, they don't work." Tr. at 84.

11. {Deleted].

12. AMSEC also complains that the Navy's judgment concerning the unpriced companies in AMSEC's proposal is based upon erroneous information because two firms that the Navy identified as unpriced were actually priced in AMSEC's cost proposal. We find that this error does not vitiate the Navy's reasonable concerns regarding AMSEC's failure to explain its arrangements with the more than [deleted] firms that were offered in its technical proposal but not priced in its cost proposal, especially since some of the unpriced firms were presented as performing significant roles in AMSEC's performance.

13. AMSEC and SWM both argue that the RFP indicated that most of the services to be performed under the contract involved shipyard repair, and therefore this element of the SOW should have been considered as more important than other SOW elements in the agency's evaluation of proposals. As the Navy and BAV point out, however, the RFP did not state that any element of the SOW was more important than any other; rather, the RFP required offerors to demonstrate how they would satisfy all the SOW requirements.

14. There is no citation to these concerns in the TEC report, cost/technical tradeoff analysis, or source selection decision documents.

15. SWM did not specifically protest the SSA's cost/technical tradeoff. In any event, given our decision that the agency reasonably found that the technical superiority of BAV's proposal outweighed AMSEC's cost advantage, SWM would have no basis to challenge the advantageousness of BAV's proposal vis-a-vis SWM's proposal which was found to be technically equivalent to, but with a higher cost than, AMSEC's.

16. SWM also protests that BAV has an organizational conflict of interest because of an ongoing contract VSE has with the Navy's security assistance program office, PMS-380. However, SWM did not identify which contract formed the basis for this complaint, and the record shows that VSE had no ongoing contracts with PMS-380.

Downloads

GAO Contacts

Office of Public Affairs