L-3 Communications, L-3 Link Simulations and Training

B-410644.2: Jan 20, 2016

Additional Materials:

Contact:

Ralph O. White
(202) 512-8278
WhiteRO@gao.gov

Kenneth E. Patton
(202) 512-8205
PattonK@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

L-3 Communications, L-3 Link Simulation and Training (L-3), of Arlington, Texas, protests the issuance of a delivery order to CAE USA, Inc., of Tampa, Florida, under request for proposals (RFP) No. N61340-14-R-0079, issued by the Department of the Navy for technology refresh and upgrades to existing helicopter flight training devices, as well as for the purchase of additional helicopter training devices. L-3 maintains that the agency misevaluated its proposal and made an unreasonable source selection decision.

We deny the protest.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of:  L-3 Communications, L-3 Link Simulations and Training

File:  B-410644.2

Date:  January 20, 2016

Craig A. Holman, Esq., Stuart W. Turner, Esq., and Nicole B. Neuman, Esq., Arnold & Porter LLP, for the protester.
Joseph P. Hornyak, Esq., Megan Mocho Jeschke, Esq., and Elizabeth N. Jochum, Esq., Holland & Knight LLP, for CAE USA, Inc., an intervenor.
R. Montana Erickson, Esq., and Sharon H. Sachs, Esq., Department of the Navy, for the agency.
Scott H. Riback, Esq., and Tania Calhoun, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Protest challenging agency’s evaluation of proposals and source selection decision is denied where record shows that evaluation and source selection were reasonable, rational, and consistent with the terms of the solicitation and applicable statutes and regulations. 

DECISION

L-3 Communications, L-3 Link Simulation and Training (L-3), of Arlington, Texas, protests the issuance of a delivery order to CAE USA, Inc., of Tampa, Florida, under request for proposals (RFP) No. N61340-14-R-0079, issued by the Department of the Navy for technology refresh and upgrades to existing helicopter flight training devices, as well as for the purchase of additional helicopter training devices.[1]  L-3 maintains that the agency misevaluated its proposal and made an unreasonable source selection decision.

We deny the protest.

BACKGROUND

The RFP contemplates the issuance, on a best-value basis, of a delivery order to perform technology refresh and upgrades to MH-60 Romeo (MH-60R) Tactical Operational Flight Trainers, MH-60 Sierra (MH-60S) Operational Flight Trainers and MH-60S Weapons Tactical Trainers, as well as the procurement of three new MH-60R Tactical Operational Flight Trainers, along with ancillary equipment, associated provisioning and logistics support.  Firms were advised that the agency would evaluate proposals considering price and one technical factor which was more important than price.  RFP at 132.[2]  The technical factor was divided into three subfactors:  technical approach, logistics approach (of approximately equal importance), and management approach (deemed less important than the other two subfactors).  RFP at 131-132.[3]

The RFP advised that the Navy would assign adjectival technical ratings of outstanding, good, acceptable, marginal or unacceptable under each subfactor, and an overall technical adjectival rating to the proposals.  RFP at 132, 134.  The RFP also advised offerors that the agency would assign technical risk ratings of low, moderate or high for each subfactor and for the proposal overall.  Id.  For price evaluation purposes, the agency would review prices for completeness and reasonableness, and to ensure that the proposed prices were balanced.  RFP at 133-134.[4]

The agency received several proposals in response to the RFP and evaluated them; engaged in discussions; and solicited, obtained, and evaluated revised proposals.  After completing that evaluation, the agency issued a delivery order to L-3, finding that its proposal represented the best value to the government.  After being advised of that award decision, CAE filed a protest in our Office challenging the agency’s selection decision.  In response to that protest, the agency informed our Office that it intended to reevaluate proposals and make a new source selection decision.  Based on the agency’s proposed corrective action, we dismissed CAE’s protest as academic.  (B-410644, Oct. 23, 2014, unpublished decision.)

Subsequent to CAE’s protest, the agency issued several amendments to the RFP and solicited revised proposals.  The agency again engaged in discussions with the offerors and solicited, obtained, and evaluated revised proposals.  The agency assigned the following ratings to the CAE and L-3 proposals:

Offeror

Overall

Technical Factor

Technical Approach Subfactor

Logistics Approach Subfactor

Management Approach Subfactor

Price

L-3

Outstanding

Low Risk

Outstanding

Low Risk

Outstanding

Low Risk

Acceptable

Low Risk

$116,999,996

CAE

Good

Low Risk

Outstanding

Low Risk

Good

Moderate Risk

Acceptable

Low Risk

$90,000,025


AR, exh. 35, Source Selection Decision Document (SSDD) at 3.  On the basis of these evaluation results, the agency selected CAE for issuance of the delivery order.  The record shows that the agency understood that L-3’s proposal had been deemed technically superior, but the agency concluded that that technical superiority was not worth the price premium associated with selecting L-3.  Id.  at 8-10.  After being advised of the agency’s selection decision and requesting and receiving a debriefing, L-3 filed the instant protest. 

PROTEST

L-3 has raised a large number of allegations, principally relating to the evaluation of its proposal and the propriety of the agency’s source selection decision.  We have considered all of L-3’s allegations and find no merit to any of them.  We discuss L‑3’s principal contentions below. 

Evaluation of the L-3 Proposal

With respect to L-3’s allegations concerning the propriety of the agency’s evaluation of its proposal, we note that, in reviewing protests that challenge an agency’s evaluation of proposals, our Office does not independently evaluate proposals; rather, we review the agency's evaluation to ensure that it is reasonable and consistent with the terms of the solicitation and applicable statutes and regulations.  Lowe Campbell Ewald, B-411614, B-411614.2, Sept. 11, 2015, 2015 CPD ¶ 296 at 4. 

By way of background, the record shows that, in its previous evaluation of proposals, the agency assigned strengths, weaknesses and deficiencies to the proposals in connection with assigning adjectival ratings, and also identified risks (where an aspect of a proposal was deemed to increase the risk of successful contract performance) before assigning adjectival risk ratings to the proposals.  Agency Report at 13.  However, the agency amended the RFP after CAE’s protest to revise the solicitation’s various definitions.  As is pertinent to the protest, the agency amended the RFP to include a definition of the term “risk reducer.”  In reevaluating proposals, the agency explains that it used the following process:

As part of the re-evaluation, all existing strengths were reviewed to determine if they were valid.  In this process, strengths were added, removed, and revised.  Additional weaknesses and deficiencies also resulted from this process.  In addition, the SSEB [source selection evaluation board] considered whether the benefit associated with a strength was risk reduction.  If it was, the SSEB identified it as a “risk reducer” rather than a strength.  The SSEB knew that Section M of the RFP would be amended to reflect “risk reducers” when the re-evaluation was complete.  As discussed below, NAWCTSD [Naval Air Warfare Center Training Systems Division] did, in fact, issue Amendment 0007 approximately one month later to incorporate and explain “risk reducers” to offerors.

Agency Report at 14. 

Although L-3 takes issue with virtually every aspect of the agency’s evaluation of its proposal, its numerous allegations essentially may be characterized as falling into one of three basic categories.  The first category of complaint advanced by L-3 relates to assigned strengths that L-3 maintains actually should have been identified as numerous, separate strengths.  For example, the record shows that the agency assigned a strength to the L-3 proposal under the technical approach subfactor because L-3 proposed to provide [deleted] to two particular software elements (the protester’s [deleted], and its [deleted] software solution), and also proposed to provide the agency with [deleted] in a third software element (L-3’s [deleted]).  AR, exh. 31, Final Proposal Evaluation Report (PER) at 60.  L‑3 maintains that the agency should have assigned a separate strength for each of these three [deleted] offerings. 

The second category of complaint advanced by L-3 relates to instances where the agency previously had assigned a strength to its proposal during the first evaluation, but that strength was identified as a risk reducer--rather than a strength--in the agency’s subsequent evaluation.  For example, the record shows that during its first evaluation, the agency assigned the L-3 proposal a strength under the logistics approach subfactor for offering to [deleted] its facilities design solution prior to actually [deleted] that solution.  AR, exh. 9, Initial Evaluation Notices, at 12.  During its reevalaution, the agency converted this strength to a risk reducer.  Id.  L-3 maintains that, rather than assigning its proposal only risk reducers, the agency also separately should have assigned strengths to its proposal for these features. 

The third category of complaint advanced by L-3 relates to instances where the agency assigned the L-3 proposal strengths under one evaluation subfactor, but L-3 maintains that the agency also should have assigned its proposal strengths for the same features under another subfactor.  For example, the record shows that the agency assigned the L-3 proposal a strength under the technical approach subfactor for its [deleted].  AR, exh. 31, Final PER, at 59.  L-3 maintains that it should have been assigned a separate strength for this same proposal feature under the logistics approach subfactor. 

We find no merit to these allegations.  As an initial matter, we note that all of L-3’s arguments rely on a faulty premise, namely, that, by assigning these additional strengths, the L-3 proposal will be, by some unstated calculus, found to be more technically superior than it already was found to be.  However, as we have long held, evaluation scores--whether they are numeric or adjectival ratings, or whether they involve the assignment of strengths (or in this case risk reducers)--are merely guides to intelligent decision making.  enrGies, Inc., B‑408609.9, May 21, 2014, 2014 CPD ¶ 158 at 8;  The scores or the number of strengths assigned to proposals are not dispositive metrics for an agency to express a proposal’s merit.  What is important is not the scores themselves, but the underlying substantive merits of the proposals as embodied in, or reflected by, the scores, along with the underlying narrative description that supports the assignment of those scores.  Id.

Here, the record shows that the agency neither assigned any weaknesses or deficiencies to the L-3 proposal, nor identified any elements of the proposal that would elevate potential risk during contract performance.  See AR, exh. 31 Final PER; exhibit 34, Final Proposal Analysis Report (PAR).  L-3 does not allege that the agency improperly identified weaknesses, deficiencies, or risks in its proposal, but, rather, that the agency did not--numerically speaking--assign enough strengths to its proposal. 

L-3’s protest also does not identify a single proposal element that was improperly overlooked or unrecognized by the agency.  None of L-3’s evaluation challenges involve a strength or risk reducer that was not expressly identified by the agency.  Instead, its protest focuses solely on the numeric count of strengths that were assigned to its proposal for the elements actually identified, and understood by the agency to be desirable features of the L-3 proposal.  L-3’s protest essentially misses the point.  It simply is immaterial how many strengths were assigned to L-3’s proposal; what matters is whether or not the agency recognized the qualitative features of L-3’s proposal that were desirable, and rendered it technically superior to the other proposals received.[5]  On this record, we have no basis to find the agency’s evaluation objectionable or unreasonable.  We therefore deny these protest allegations.

Risk Rating Assigned to the CAE Proposal

L-3 also alleges that the agency improperly “rolled up” the risk ratings assigned to the CAE proposal.  In this connection, the record shows that the CAE proposal was assigned a moderate risk rating under the logistics approach subfactor based on the following weakness identified in its proposal:

As a weakness, CAE proposed to develop documentation similar to that of an aircraft [deleted], but did not discuss how this approach would be applied to or affect other documents in the Common Documents initiative.  The SSAC believes CAE’s proposed approach to the Common Documents requirement to be a weakness as it will likely increase schedule and performance risk and increase the need for Government oversight.

AR, exh. 34, PAR, at 37.  L-3 maintains that the agency improperly assigned the CAE proposal an overall risk rating of low, notwithstanding that the agency had identified this as a weakness and assigned the CAE proposal a moderate risk rating under the logistics approach subfactor. 

We find no merit to this aspect of L-3’s protest.  As with its allegations relating to the evaluation of its own proposal, this aspect of L-3’s protest attempts to elevate form over substance in connection with the agency’s evaluation of the CAE proposal. 

As noted, the agency identified the weakness above in evaluating the CAE proposal.  In this regard, the record shows that the agency understood the nature of that weakness, and also understood that it was based on a feature of the CAE proposal that the agency considered less desirable.  The record also shows that the agency weighed the importance of that weakness in arriving at the overall risk rating assigned to the CAE proposal, and concluded that it was not significant enough to raise CAE’s overall risk rating to moderate.  AR, exh. 31, Final PER, at 112.  More significantly, the record shows that the agency expressly considered the comparative merits of the CAE and L-3 proposals in this area, and found L-3’s proposed approach to be of significantly greater value to the agency than the approach proposed by CAE.  AR, exh. 34, Final PAR, at 37

As with L-3’s other evaluation challenges, the record shows that, in this area, the agency substantively understood the comparative merits of the competing proposals, and also understood that L-3’s proposed approach was technically superior to the approach proposed by CAE.  While the protester once again takes issue with the adjectival rating assigned to the CAE proposal, it has not shown that the agency failed to understand the nature of the weakness; that the agency did not appropriately weigh the importance of the weakness in evaluating the CAE proposal; or that the agency somehow failed to appreciate the relative merits of the two proposals in comparing them to each other.  We therefore deny this aspect of L-3’s protest.

Source Selection Decision

L-3 contends that the agency’s source selection decision is unreasonable.  According to the protester, the agency improperly failed meaningfully to compare the relative technical superiority of the L-3 proposal against the price advantage enjoyed by CAE.  Specifically, L-3 alleges that the agency was required to weigh each of the features of its proposals that were found to offer technical advantages against the cost premium associated with issuing the delivery order to L-3.  L-3 therefore maintains that the agency’s source selection decision is unreasonable.

We deny this aspect of L-3’s protest.  Source selection officials in negotiated procurements have broad discretion in making cost/technical tradeoffs; the extent to which one may be sacrificed for the other is governed only by the tests of rationality and consistency with the evaluation criteria.  Vigor Shipyards, Inc., B-409635, June 5, 2014, 2014 CPD ¶ 170 at 9.  Where a cost/technical tradeoff is made, the source selection decision must be documented, and the documentation must include the rationale for any tradeoffs made.  Id.; FAR § 15.308.  For a proper tradeoff, the record must show that the SSA was aware of the relative technical merits of competing proposals, and specifically considered whether the advantage of a technically superior proposal is, or is not, worth any associated cost premium.  Vigor Shipyards, Inc., supra at 10.

Here, the record shows that the Navy conducted what can only be described as an exhaustive comparison of the proposals under not only each technical evaluation subfactor, but also under each of the numerous elements identified under each subfactor.[6]  AR, exh. 34, PAR, at 25-42.  For every element, the agency identified the strengths and risk reducers (along with any weaknesses) assigned to each proposal; compared the relative value to the agency of those strengths, risk reducers and weaknesses; and made a specific finding regarding the relative ranking of the proposals under each element.  Id.  The SSAC, while unanimously finding the L-3 proposal technically superior to the CAE proposal overall, nonetheless also presented the source selection authority (SSA) with majority and minority positions regarding which firm should receive award; the minority recommended award to L-3 and the majority recommended award to CAE.  Id. at 43-46.

The SSA, in turn, gave detailed consideration to the SSAC’s final proposal analysis report  findings and recommendations and incorporated that report, along with the SSEB’s final proposal evaluation report, into the SSDD.  AR, exh. 35, SSDD, at 2.  The SSA then went on to detail the strengths and risk reducers found in the L-3 proposal in order of importance for each of the evaluation subfactors,  id. at 5-7, and specifically concurred with the SSAC concerning its finding that the L-3 proposal was the technically superior proposal among all proposals.  Id. at 8.  He then went on to perform a comparative analysis of the proposals considering each of the elements under the three technical subfactors, and again acknowledged the superiority of the L-3 proposal from a technical standpoint.  Id. at 8-9.  Finally, he acknowledged the L-3 proposal features identified by the SSAC minority as forming the basis for their recommendation of award to L-3.  Id. at 9.  Nonetheless, he concluded, in the final analysis, as follows:

I believe that paying some premium is appropriate to attain a technical solution that is superior and/or offers less risk over a lower price.  The Government could have specified the Technical Factor as significantly more important than the Price Factor, but it did not.  The RFP specifically provided that the Technical Factor is more important than the Price Factor.  This indicates that the government is willing to pay some price premium, but not any price premium, for a technically superior offer.  After considering the stated evaluation scheme with the Technical Factor being more important than the price factor (not significantly more important), I do not find any one of L-3’s proposed enhancing features, benefits or risk reducers; or all of the enhancing features, benefits and risk reducers combined (including those that were not specifically addressed in the SSAC majority and minority positions) to be sufficient to overcome the $26,999,971 or 30% price premium over the CAE price.  Therefore, in accordance with the evaluation criteria and my independent analysis, I have determined that CAE’s proposal provides the best value to the government, all Factors and Subfactors considered, and select CAE for award of the MH-60R/S TPRS contract. 

Id. at 10.

We conclude that the agency’s source selection here is detailed and comprehensive, and presents a thorough explanation for the agency’s ultimate conclusion that award to CAE presented the best value to the government.  While L‑3 is correct that the agency did not separately “monetize” each and every feature of its proposal that the agency identified as superior to the other proposals, we are aware of no requirement for an agency to prepare such an analysis.  In view of the foregoing, we deny this aspect of L-3’s protest. 

The protest is denied.[7]

Susan A. Poling
General Counsel



[1] The RFP contemplates the issuance of a delivery order to the successful contractor under a multiple award indefinite-delivery, indefinite-quantity (IDIQ) program administered by the Navy called the Training Systems Contract III program.  The value of the delivery order here exceeds $10 million; accordingly, this procurement is within our jurisdiction to hear protests related to the issuance of delivery orders under multiple-award IDIQ contracts. 10 U.S.C. § 2304c(e)(1)(B).

[2] All RFP citations are to the conformed RFP included in the agency report (AR) at exhibit 24. 

[3] The RFP also identified numerous elements that were included under the subfactors.  The technical approach subfactor identified nine elements, the logistics approach subfactor identified three elements and the management approach subfactor identified two elements.  RFP at 131.

[4] In its initial protest, L-3 made several arguments that essentially amounted to challenges to the agency’s alleged failure to perform a price realism evaluation of the proposals.  In response to those allegations, the agency filed a request for dismissal, maintaining that those allegations failed to state a basis for protest; although an earlier version of the RFP included a requirement for the agency to perform a price realism evaluation, that requirement was removed from the RFP by amendment.  We agreed with the agency that those allegations failed to state a basis for protest, and advised the parties that we would not further consider them.  E-Mail From GAO to the Parties, Oct. 26, 2015.

[5] We also point out that there is an underlying logic to the agency’s assignment of proposal risk reducers in lieu of strengths in those instances where that occurred.  For example, in assigning the L-3 proposal a risk reducer under the logistics approach subfactor for offering to [deleted] its facilities design solution prior to [deleted], the agency concluded that L-3’s approach would allow it to ensure that its design [deleted], and that this would reduce any schedule risks associated with facility modifications and trainer installation.  AR, exh. 31, Final PER, at 68.  It was reasonable for the agency to have assigned a risk reducer to the L-3 proposal in this area--as opposed to a strength--in light of the fact that the benefit to be derived by the agency was a reduction in the risk that L-3 would be unable to meet the required schedule for trainer installation.  In the final analysis, regardless of the fact that this feature was identified as a risk reducer, the record shows that the agency identified this element of the L-3 proposal as a desirable feature that rendered it technically superior, notwithstanding the fact that the agency did not separately label it a strength.

[6] As noted, the RFP identified nine elements under technical approach subfactor, three elements under the logistics approach subfactor, and two elements under the management approach subfactor.  RFP at 131.

[7] As a final matter, L-3 argues that the agency did not engage in meaningful discussions with it because the agency did not advise it during discussions that its price was too high.  However, there is nothing in the record showing that the agency considered L-3’s price to be unreasonably high.  Accordingly, the agency was under no obligation to discuss L-3’s price with the firm.  Joint Logistics Managers, Inc., B‑410465.2, B-410465.3, May 5, 2015 2015 CPD ¶ 152 at 3-4. 

Oct 13, 2017

Oct 12, 2017

Oct 11, 2017

  • Daekee Global Company, Ltd.
    We dismiss the protest because the protester has not established that it is an interested party.
    B-414899,B-414899.2

Oct 10, 2017

Oct 6, 2017

Oct 5, 2017

Looking for more? Browse all our products here