Matter of: AT&T Corporation, Advanced Technology Systems File: B-261154.4 Date: October 16, 1995 * Redacted Decision

Matter of: AT&T Corporation, Advanced Technology Systems File: B-261154.4 Date: October 16, 1995 * Redacted Decision

Protest against source selection is denied where (1) agency reasonably determined that awardee had proposed a fundamentally less complex approach to deploying an underseas surveillance system which was more likely to successfully and timely deploy a survivable underwater segment than was the protester's, and (2) the record provides no basis for concluding that the awardee's contract cost was likely to be so substantially higher than the protester's as to offset the awardee's superiority under the substantially more important technical factors.

Attorneys

DECISION

AT&T Corporation, Advanced Technology Systems, protests the award of a contract to Loral Federal Systems under request for proposals (RFP) No. N00039-94-R-0020, issued by Department of the Navy, Space and Naval Warfare Systems Command (SPAWAR), for the demonstration/validation (DEM/VAL) phase of an underseas surveillance system. AT&T challenges the evaluation of cost and technical proposals. [1]

We deny the protest.

The RFP contemplated the award of a cost-plus-award-fee contract to design, develop, document, fabricate, assemble, inspect, integrate, and support the development of a DEM/VAL phase prototype of the Advanced Deployable System (ADS), a passive underseas surveillance system providing tactical warfare information to commanders in littoral (shallow) water areas. The solicitation required offerors to submit a technical proposal and system design documentation package which:

"in combination shall be sufficiently specific, detailed, and complete so as to clearly and fully demonstrate to the Government that the prospective offeror has a thorough understanding of the requirements for and problems inherent in performing, as well as the capability to perform the Demonstration and Validation Phase and delivery of ADS."

The solicitation further required offerors to submit cost proposals which explained:

"in whatever detail is required to demonstrate cost reasonableness and supportability, the methodology used to estimate each element of cost (e.g., labor, material, etc.). Enough data shall be provided so that an independent cost analysis verification can be performed."

In this regard, offerors were required to submit their cost estimates using a work breakdown structure (WBS) furnished with the solicitation and to provide:

"[a]dequate information . . . to allow Government evaluation of proposed labor hours, material costs, subcontractor costs, other direct costs and related overhead costs by the fourth level of the WBS. Major subcontractor estimates (i.e., exceeding Five Million Dollars ($5M)) shall be submitted using the same formats and degree of rationale."

The solicitation generally provided for award to be made to the offeror whose proposal is "considered most likely to satisfy the requirements of the government and to be in the best interest of the Government, cost and other factors considered." The RFP provided for the evaluation of proposals on the basis of the following four specific evaluation factors (in descending order of importance): (1) overall system design/development and (2) prototype development, implementation and demonstration, which were "substantially more important than" (3) management and (4) cost. Cost proposals were to be evaluated for (1) contract cost affordability/reasonableness and (2) life-cycle cost. With respect to the proposed contract costs, the RFP stated that:

"[t]he Government will conduct its own evaluation of these costs and the associated ranges, and will rely on this evaluation when determining the award. The Government's cost evaluation will be affected by the quality of the supporting data provided and on the basis of traceability of the proposed costs to the technical/management proposal, as well as the offeror's demonstrated ability to deliver large, technically complex development programs within budget. The primary purpose of this criterion is to determine if the cost proposal for the prototype development, demonstration/validation is complete, realistic and reasonable."

The RFP likewise provided for the government to conduct its own evaluation of the proposed life-cycle costs for use in selecting a proposal for award.

Four proposals were received by the closing time. The proposals submitted by Loral and AT&T were based on [DELETED]; two other proposals were based, entirely or primarily, on [DELETED]. Subsequent to the receipt of initial proposals, SPAWAR amended the solicitation to delete provisions for potential installation by air drop. Offerors were requested to submit updated proposals, but were not afforded the opportunity for discussions.

Since an autonomous architecture was not achievable in time for prototype demonstration, resulting in both of the proposals that emphasized an autonomous approach receiving an overall "below adequate" rating under the technical factors, and the life-cycle costs of the autonomous approach were expected to be substantially higher than those for the cabled approach, the source selection decision focused on the cable-connected proposals submitted by Loral and AT& T. The evaluation results are shown below.

AT& T Loral

TECHNICAL

System Design Adequate Above Adequate

Prototype Demonstration Below Adequate Adequate

Management Below Adequate Adequate

Overall Technical Adequate Above Adequate

COST

Proposed DEM/VAL $30.5 million $42.4 million

Evaluated DEM/VAL $30.5-62.6 million $42.4-56.2 million

Evaluated Life-Cycle $[DELETED] million $[DELETED] million

The record establishes that cost considerations were secondary in the source selection decision. As noted above, cost and management were "substantially" less important than the system design and prototype demonstration evaluation factors. Further, although the evaluated life- cycle cost of AT& T's approach, as determined by application of a parametric cost model, was slightly higher than Loral's, the source selection advisory council (SSAC) evaluated the life-cycle costs of the two proposals as being "virtually the same" and concluded that they "provide[d] no conclusive basis for selection of one over the other."

Offerors' proposed DEM/VAL costs were not determinative because the cost evaluation team (CET) found the task of developing an independent estimate of the most probable cost of each offeror's DEM/VAL approach to be "extremely difficult." The CET concluded that the proposals lacked sufficient detail to permit it to undertake the detailed, "bottoms-up" cost realism analysis recommended in the Navy's Cost Realism Handbook and traditionally used at SPAWAR -- that is, a detailed build-up of contract cost from the lowest WBS levels based on a detailed review of proposed labor and hardware costs. In this regard, although the solicitation generally required offerors to submit cost proposals which included "whatever detail is required to demonstrate cost reasonableness and . . . the methodology used to estimate each element of cost . . . so that an independent cost analysis verification can be performed," and specifically extended the obligation to furnish detailed WBS costs and an explanation of the underlying cost methodology to major subcontractors, AT&T failed to furnish a cost proposal for a subcontractor accounting for [DELETED] percent of its proposed cost. (In addition, the source selection evaluation board found that AT& T had proposed inadequate staffing levels to support its technical approach.) As for Loral's proposal, the CET expressed concern that the cost proposal of its principal subcontractor was based only on "engineering judgment" and that no historical data supported the subcontractor's proposed efficiency and learning curves. Since the CET was unable to develop independent estimates of offerors' DEM/VAL costs, the panel recommended that SPAWAR open discussions with offerors to acquire detailed cost backup data.

Instead of commencing discussions, however, SPAWAR determined that DEM/VAL phase cost realism could be evaluated by using the parametric cost model used to develop the life-cycle cost estimates, as modified to take into account certain proposal-unique information and eliminate costs associated with the engineering and manufacturing development (EMD) and production phases. Although the CET had found there were "serious weaknesses inherent in the model . . . which . . . prevent it from calculating other than rough approximations of contractor-specific [DEM/VAL] cost estimates," the SSAC concluded that the parametric cost model contained sufficient proposal-and-item-unique cost drivers to permit its use to define the upper limit of the cost estimate, with the lower limit of the range consisting of the offerors' proposed costs. [2]

Viewing the wide range between AT& T's proposed cost ($30.5 million) and the cost derived from the modified parametric cost model ($62.6 million) as calling into question the reliability of AT& T's cost data, and given AT& T's failure to furnish a cost proposal for its [DELETED] subcontractor and its proposal of inadequate staffing to support its proposed technical approach, the SSAC determined that AT& T's proposal was not well supported. Noting also that a "negative 25 % cost variance" -- cost overrun -- had occurred with respect to AT& T's performance on the underwater segment of another underseas surveillance system, the recent program most like the ADS program, the SSAC found AT& T's cost proposal to be characterized by "substantial cost risk."

In contrast, the SSAC concluded that while "it may have been optimistic, Loral's [cost] proposal is not considered unrealistic, and is more realistic than AT& T['s]." The panel based its conclusion on the fact that: (1) the range between Loral's proposed cost ($42.4 million) and the cost derived from the modified parametric cost model ($56.2 million) was considerably narrower than the range for the AT&T proposal, supporting the conclusion that Loral's proposed costs were more realistic; (2) a review of proposed labor hours by WBS element indicated that Loral's proposed staffing was adequate to support its technical approach; (3) its subcontractor had supported its proposed cost down to the seventh WBS level, attempted to account for known technical risks in terms of potential additional labor hours, and added [DELETED] percent to its own vendor cost inputs; (4) the proposed direct and indirect rates were reasonable; and (5) only an approximately 10 percent cost overrun had occurred with respect to Loral's performance on the contract for the shore processing segment of the previous underseas surveillance system contract. Notwithstanding its ultimate determination that Loral's cost proposal was "realistic and reasonable," and not characterized by the substantial cost risk associated with AT& T's proposal, the SSAC ultimately concluded that it "would not expect costs for AT&T and Loral to differ significantly at the end of DEM/VAL."

The evaluated technical superiority of Loral's approach was determinative in the source selection. The SSAC report noted that "[s]urvivability is the primary risk of the [DELETED] architectures. Deployment of the [DELETED] architectures is an aspect of the survivability risk . . . ." Likewise, according to the SSAC co-chairman (and program manager), "[d]eployment is everything in ADS."

[DELETED]

SPAWAR evaluated AT&T's [DELETED] approach to [DELETED] as an overly complex, high risk approach which conferred little or no added, real benefit to the deployment process and would be unable to deploy at the full range of speeds necessary to meet the ADS deployment time requirements. Hearing Transcript (Tr.) at 53-54, 108-109, 157-158.

[DELETED]

The lead evaluator (and agency expert) in this area reported to the SSAC that the AT&T deployment approach was "not a suitable method for installing ADS [DELETED] systems." The SSAC determined that "[t]he Loral approach is judged to be of lower technical risk than AT& T['s] to successfully deploy a survivable underwater segment. This is due to the simplicity of the Loral deployment approach compared to the method proposed by AT& T." Further, noting that cost "estimates at this stage of development are inherently uncertain due to technical risk," and concluding that discussions were unlikely to result in further useful information, the SSAC recommended against commencing discussions. The SSAC instead recommended award to Loral, explaining that:

"[g]iven the technical superiority of Loral, its reasonable and affordable [life-cycle cost] evaluation (which is virtually the same as AT&T), and its sufficiently realistic (and more realistic than AT&T) DEM/VAL offer, discussions would add little value to the selection.

"Despite the proposed cost difference, the SSAC would not expect costs for AT& T and Loral to differ significantly at the end of DEM/VAL. And even if AT& T were to cost $10 M[illion] less than Loral, as proposed, the latter's technical superiority, short term cost reasonableness and affordability, and long term cost effectiveness approximately equal to AT&T, make it the best value to the Government."

The source selection authority accepted the SSAC's recommendation and made award to Loral without discussions. AT& T thereupon filed this protest with our Office.

TECHNICAL EVALUATION

Although AT&T raises a number of arguments challenging the technical evaluation, the reasonableness of SPAWAR's overall conclusion that Loral submitted a technically superior proposal depends on the reasonableness of the agency's fundamental technical determination that Loral's less complex deployment approach was more likely to successfully and timely deploy a survivable ADS underwater segment than was AT& T's. We find the agency's determination in this regard reasonable.

Deployment Speed

AT&T challenges SPAWAR's determination that the deployment speed of its proposed [DELETED] would be inadequate. [DELETED]

As an initial matter, to the extent that the agency allegedly failed to fully appreciate AT& T's intent in this area, the responsibility must rest largely with AT& T for failing to submit an adequately, clearly written proposal. The solicitation required offerors to submit proposals which were:

"sufficiently specific, detailed, and complete so as to clearly and fully demonstrate to the Government that the prospective offeror has a thorough understanding of the requirements for and problems inherent in performing, as well as the capability to perform the Demonstration and Validation Phase and delivery of ADS."

As noted by the agency, AT& T's proposal did not describe in any detail its intended [DELETED]

Complexity

Even if AT& T's [DELETED] were to prove capable of deploying at speeds of interest for ADS, the record supports the agency's position that AT& T's approach would be more complex than Loral's [DELETED] approach, without adding any appreciable compensating benefit. Further, AT& T has furnished no basis for questioning SPAWAR's position that a less complex approach that satisfies the agency's needs is to be preferred. [DELETED]

It is clear from the record that the agency viewed the lesser complexity of Loral's deployment approach as significantly enhancing the likelihood of successful deployment of ADS.

We find that SPAWAR reasonably determined that Loral submitted a technically superior proposal which offered a fundamentally less complex deployment approach that was more likely to successfully and timely deploy a survivable ADS underwater segment than AT&T's.

COST EVALUATION

AT& T challenges SPAWAR's cost evaluation primarily on the basis that the agency failed to prepare independent estimates of the likely actual cost of DEM/VAL performance based on the approach proposed by each offeror.

As a general rule, agencies are required to include cost or price as a significant factor in the evaluation of proposals. Competition in Contracting Act (CICA), 10 U.S.C. Sec. 2305(a)(2)(A)(i) (1994); Federal Acquisition Regulation (FAR) Sec. 15.605(b). An evaluation and source selection which fails to give significant consideration to cost, or which varies from the RFP's cost evaluation provisions, is inconsistent with CICA and cannot serve as the basis for a reasonable source selection decision. See Lockheed, IMS, B-248686, Sept. 15, 1992, 92-2 CPD para. 180. Further, when a cost reimbursement contract is to be awarded, the offerors' estimated costs of contract performance should not be considered as controlling since the estimates may not provide valid indications of the final contract costs which the government is required to pay. See FAR Sec. 15.605(d). Consequently, the contracting agency must perform a cost realism analysis to determine the realism of an offeror's proposed costs and to determine what the costs are likely to be under the offeror's technical approach, assuming reasonable economy and efficiency. CACI, Inc.-Fed., 64 Comp.Gen. 71 (1984), 84-2 CPD para. 542; GTE Gov't Sys. Corp., B-260022; B-260022.2, May 16, 1995, 95-1 CPD para. 245.

SPAWAR's cost evaluation approach resulted in the calculation for AT& T's proposal of only a broad range -- from $30.5-$62.6 million -- of possible cost of DEM/VAL performance. However, SPAWAR's failure to more accurately measure the most probable cost of AT& T's proposal does not warrant sustaining AT& T's protest in this regard, since this failure was largely the result of AT& T's failure to submit an adequately supported cost proposal, in particular, [DELETED] "[a]dequate information . . . to allow Government evaluation of proposed labor hours, material costs, subcontractor costs, other direct costs and related overhead costs by the fourth level of the WBS," so that "an independent cost analysis verification can be performed."

Further, given the agency's inability to calculate a specific most probable DEM/VAL cost for AT& T's proposal (as a result of AT& T's failure to submit an adequately written cost proposal), there was no specific AT&T most probable cost with which to compare a specific Loral most probable cost. Moreover, the SSAC generally concluded that Loral's relatively detailed DEM/VAL cost proposal was not unrealistic. AT& T has not shown that Loral's cost proposal omitted any probable costs of DEM/VAL performance which were so significant as to affect the reasonableness of the source selection decision. As discussed above, it is clear from the record that any uncertainties with respect to cost were not significant in the source selection decision since the agency was primarily concerned with successful and timely deployment of a survivable ADS underwater segment.

CONCLUSION

Since SPAWAR reasonably determined that Loral submitted a technically superior proposal, the proposed approaches of AT&T and Loral also entailed approximately equal life-cycle costs, and the record provides no basis for concluding that Loral's DEM/VAL cost was likely to be so substantially higher than AT& T's as to offset Loral's superiority under the substantially more important technical factors, we find the selection of Loral unobjectionable. [3]

The protest is denied.

Comptroller General of the United States

1. Inasmuch as portions of the record are classified, the following unclassified discussion is necessarily somewhat general with respect to several aspects of the procurement.

2. The CET noted that the parametric cost model was based on a database that did not separate DEM/VAL and EMD costs and assumed the use of less commercial-off-the-shelf/nondevelopmental items (COTS/NDI) than proposed for ADS. According to SPAWAR, however, the limitation with respect to COTS/NDI did not prejudice AT& T, since AT& T and Loral reportedly proposed approximately the same level of COTS/NDI when common definitions are used.

3. AT& T contends that SPAWAR should have conducted discussions in order to calculate a most probable cost for each proposal and otherwise resolve uncertainties in the proposals. Again, however, the agency had substantial cost information, and given that this information reasonably indicated that any cost differences would not affect the award decision -- which was driven primarily by technical concerns -- discussions (in lieu of award based on revised initial proposals) were not required.

DOCUMENT FOR PUBLIC RELEASE A protected decision was issued on the date below and was subject to a GAO Protective Order. This version has been redacted or approved by the parties involved for public release.