Matter of: GTE Hawaiian Telephone Company, Inc. File: B-276487.2 Date: June 30, 1997 * REDACTED DECISION

Matter of: GTE Hawaiian Telephone Company, Inc. File: B-276487.2 Date: June 30, 1997 * REDACTED DECISION

DIGEST

Attorneys

DECISION

GTE Hawaiian Telephone Company, Inc. protests the award of a contract to AT&T Corporation under request for proposals (RFP) No. DCA300-96-R-0001, issued by the Defense Information Technology Contracting Organization, Defense Information Systems Agency (DISA) for the Hawaii Information Transfer System (HITS). GTE contends that AT&T's proposal is not compliant with mandatory solicitation requirements and that DISA performed an unreasonable best value analysis in selecting AT&T's lower-rated, lower-priced proposal for award.

We deny the protest.

The RFP provided for the award of a fixed-price, indefinite delivery services contract to provide end-to-end switched voice, switched data, integrated services digital network (ISDN) and dedicated transmission services to Department of Defense users throughout the State of Hawaii for a 10-year contract period. The HITS contract will replace the existing Oahu Telephone System, the Hawaii Area Wideband Systems, and other dedicated transmission service systems, [1] allowing the government to consolidate its telecommunication services under one contract. HITS will interface and interoperate with other communication networks, including the Defense Information System Network, the Government Emergency Telecommunication Service, the Defense Satellite Communication System, FTS2000, and public telephone networks.

A Functional Requirements Specification (FRS) was provided in the RFP, which specified the information transfer requirements for HITS, identified the services and features that HITS must provide, and stated performance specifications and network management system requirements. The contractor is required to provide whatever equipment, facilities, and network management are necessary to deliver services in compliance with the FRS requirements. The RFP stated with regard to the configuration of the HITS:

"The Contractor shall make maximum use of commercially available off-the-shelf technology and services. The Contractor's system engineering shall maximize configuring HITS from existing off-the-shelf equipment, services, and subsystems and minimize designing new equipment, subsystems, and services. The HITS configuration shall represent the most cost effective and technically efficient solution to meet Government requirements."

The RFP also identified the locations for which service was required and the types of service that would be provided at each location (e.g., switched voice service, dedicated transmission service, or all required services).

The contractor is required to provide Initial Operational Capability (IOC) within 8 months of contract award. IOC is defined as the point at which the first HITS location and services are operational and accepted by the government. The contractor is also required to provide Full Operational Capability (FOC) within 18 months of contract award. FOC is defined as the point at which all service requirements items and features for all locations have been "cutover" and accepted by the government. Offerors were to provide transition plans to describe their strategy for consolidating existing services into a single communication infrastructure and to satisfy the IOC and FOC schedule requirements. Detailed acceptance testing procedures were also set forth in the RFP; among other things, the contractor was required to plan for pre-cutover and post-cutover testing to demonstrate that all HITS requirements will be satisfied.

The RFP provided a best value basis for award and identified the following evaluation criteria: technical quality, management quality, and price. [2] Price was stated to be approximately equal in weight to the combination of the technical and management quality criteria; technical quality was stated to be of greater weight than management quality. The RFP cautioned that the government would "not pay a significantly higher price for slightly superior technical and management quality."

Offerors were informed that DISA would first evaluate proposals on a pass/fail basis to determine compliance with every requirement identified in the statement of work and FRS. Compliant proposals would then be qualitatively assessed under the technical and management quality criteria for understanding, compliance with requirements, and risk. In this regard, offerors were instructed to discuss their design and technical approach to satisfy each RFP requirement and to fully demonstrate their understanding of the requirements.

DISA received proposals from only GTE and AT&T. [3] Four rounds of discussions were conducted with the two offerors. During discussions and based upon the offerors' responses, DISA decided that the RFP requirements were overly restrictive in a number of regards and amended the RFP to relax 17 solicitation requirements. At the conclusion of technical discussions, DISA's source selection evaluation board (SSEB) found that both GTE's and AT&T's proposals were fully compliant with the RFP requirements, as amended; in so concluding, the SSEB compared the offerors' proposals against a "compliance matrix" that ensured that each proposal satisfied the 1,350 RFP performance requirements. Hearing Transcript (Tr.) at 52, 149, 211. [4] The final technical quality and management quality criteria scores were as follows:

Factor Score [5] Risk [6] Weight Result GTE Technical 6.13 Low to Moderate 60% 3.68 Management 5.73 Low 40% 2.29 TOTAL 5.97

AT&T Technical 4.20 Moderate to High 60% 2.52 Management 6.13 Low 40% 2.45 TOTAL 4.97

As indicated by the scores, the SSEB found GTE's technical proposal to be superior to AT&T's, while AT&T's management proposal was found to be superior to GTE's.

Regarding AT&T's technical proposal, the SSEB assessed as a strength AT&T's proposal to build its system around the 5ESS-2000 switch, [7] a telecommuncations switch considered established, reliable and flexible, which AT&T proposed to interconnect with multiple SONET rings. [8] The SSEB also noted a number of disadvantages and risks in AT&T's proposal arising primarily from AT&T's need to provide extensive software development to satisfy a number of critical HITS requirements, and AT&T's failure to adequately describe its approaches to accomplishing "grade of service" and timing and synchronization requirements.

Regarding GTE's technical proposal, the SSEB noted, as a strength, GTE's offer to provide the existing, "state-of-the-art" DMS-100 switch interconnected with a SONET transmission network. Disadvantages and risks were also noted for GTE's proposal based upon GTE's failure to completely document some elements of its proposed system, and GTE's need for considerable software customization and integration to support the network management system.

"Cost only" best and final offers (BAFO) were requested and evaluated as follows:

Actual Life Cycle Cost Discounted Life Cycle Cost [9] AT&T $291.29M [10] $197.19M GTE $365.33M $243.57M IGCE [11] $284.39M $187.55M

The SSEB prepared a report, detailing the advantages, disadvantages, and risks posed by each offeror's proposal, and briefed the source selection advisory council (SSAC) and SSA regarding the SSEB's evaluation findings. The SSEB's technical and management evaluation conclusions, including its determination that both proposals were compliant with all RFP requirements, were accepted by the SSAC, which then focused its assessment on the impact on the government of the SSEB's findings. Tr. at 292-294, 330-331.

A BVWG, consisting of some members of the SSAC, was convened to identify, analyze, and quantify significant differences between GTE's and AT&T's technical and management proposals. [12] In general, the BVWG concluded that the evaluated point difference between the two firms' proposals reflected a lack of detail in AT&T's proposal as well as greater risks of delay in the AT&T approach, primarily arising from AT&T's need to perform greater software development and customization than GTE.

Significant discriminators between the two firms' proposals were identified. These discriminators were grouped into the following categories--switched services, network management, network topology and survivability, and key personnel/experience. For the areas in which schedule or compliance risk was identified, the BVWG quantified the estimated costs to the government of those risks posed in additional government resources or to acquire the required services due to any possible delay in performance. The net calculation of this quantification was that AT&T's proposal posed an additional $1.1M of risk to the government. In addition, the BVWG found that AT&T offered slightly better, experienced key personnel than GTE, but that GTE offered slightly better critically assured service [13] than AT&T; these discriminators were not quantified. The SSAC prepared a report detailing the BVWG's findings and unanimously concluded that AT&T's substantially lower-priced proposal was a better value to the government than GTE's slightly superior, but much higher-priced, proposal.

On the basis of the SSEB's and SSAC's reports, as well as his attendance at the SSEB's briefing to the SSAC, the SSA selected AT&T's proposal for award. The SSA stated in his written source selection decision:

"In this instance, both offerors have proposed acceptable network architecture and network management plans. Both offerors propose proven, state of the art equipment. Both offerors have provided good management plans, with highly qualified personnel, and generally have good to exceptional records of past performance. Therefore, the award decision does not involve a serious question of whether an offeror would provide the required services--both offerors plainly would do so. The decision instead involves a determination of which proposal is most advantageous to the Government, consistent with the award criteria in the RFP.

"There are some significant differences in the technical and management proposals of the two offerors. The above point scores provide useful guidelines in distinguishing the differences between the two proposals. However, the point differentials do not necessarily accurately reflect the relative differences between the two proposals. In particular, while I agree that GTE generally has provided a better technical proposal than AT&T, I do not think the difference between those two proposals is as great as might be suggested by the difference in point score, i.e., a 6.13 compared to 4.20. I base this conclusion in large part on my reading of the SSAC report, as well as the BVWG/SSAC discussions which I attended."

The SSA then discussed in detail each of the technical discriminators and attendant potential costs, as identified by the BVWG. The SSA found that the $1.1M probable additional expense to the government associated with AT&T's proposal was "relatively trivial" in comparison to enormous price disparity between the proposals, i.e., $46M in discounted life cycle costs and $74M in actual life cycle costs. The SSA also found that the risk attributed to AT&T's proposal may be overstated because much of the risk was attributable to lack of detail in AT&T's proposal and not equipment or network deficiencies, and given AT&T's record of past performance, extensive past experience, and substantial financial and personnel resources. Accordingly, the SSA concluded:

"that the AT&T proposal is most advantageous to the Government. The AT&T proposal provides a enormous cost savings to the Government relative to the GTE proposal. Although the GTE proposal does offer some technical advantages, and a lower risk of delays in implementation, I do not believe that these technical advantages outweigh the substantial cost premium. When quantified, the technical advantages of the GTE proposal are relatively insignificant compared to the disparity in price, i.e., a net advantage to GTE of approximately $1.1 million versus a $46 million advantage in AT&T's [discounted life cycle cost]. Even if the quantification were vastly understated, such as by a factor of ten, the AT&T proposal nonetheless would be most advantageous to the Government by a wide margin."

Award was made to AT&T. This protest followed.

GTE complains that AT&T's proposal is not compliant with a number of mandatory solicitation requirements, such that AT&T's offer should not have been found eligible for award. Specifically, GTE asserts that AT&T's proposal is noncompliant with the ISDN, grade of service, and the timing and synchronization requirements specified in the FRS. [14] GTE also argues that AT&T's proposal is noncompliant because, contrary to the RFP requirement that offerors describe their design and technical approaches, AT&T did not sufficiently describe how it would satisfy the RFP requirements regarding specified MUFs, [15] the internal Stratum 3 clock, the Defense Switched Network Integrated Management Support System (DIMSS) interface, critical assured service, and the network management system. Finally, GTE argues that AT&T will not meet the mandatory FOC schedule date, given the likely delay that DISA evaluated in AT&T's proposal due to that firm's need to perform substantial software development and customization to satisfy RFP requirements to provide the MUFs, [DELETED], and the network management system.

DISA and AT&T dispute each of these allegations and contend that AT&T's proposal is fully compliant with all of the RFP's requirements. They state that AT&T's proposal, like GTE's, was found compliant based upon a commitment of the offeror to perform each RFP requirement and the SSEB's finding that the proposed solution to accomplishing the requirement was technically feasible. The agency and intervenor also argue that the implementation and schedule risks, which GTE points to in the evaluation record, do not demonstrate that AT&T's proposal is noncompliant, but rather represent the agency's best value analysis of AT&T's acceptable approach.

In considering protests of an agency's evaluation of proposals, we examine the record to determine whether the agency's judgment was rational and consistent with stated evaluation criteria and applicable statutes and regulations. Abt Assocs., Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD Para. 223 at 4. Such judgments are by their nature often subjective; nevertheless, the exercise of these judgments in the evaluation of proposals must be reasonable and bear a rational relationship to their announced criteria upon which competing offers are to be selected. Southwest Marine, Inc.; American Sys. Eng'g Corp., B-265865.3; B-265865.4, Jan. 23, 1996, 96-1 CPD Para. 56 at 10. From our review of the record, including the hearing testimony and the parties' protest arguments, we conclude that the agency's evaluation of proposals was reasonable.

First, the record does not support GTE's arguments that AT&T's proposal was not compliant with the ISDN, grade of service, and timing and synchronization requirements specified in the RFP.

ISDN is an international telecommunications standard for transmitting voice, video, and data over digital lines, allowing the transmission of digitized voice data and network management information simultaneously. See Tr. at 196-197. The RFP ISDN requirement specifies that the contractor provide basic rate and primary rate ISDN services in accordance with Bellcore National ISDN standards (NI-1 and NI-2), "except as otherwise specified by the HITS FRS." The RFP then identified the 28 features of the National ISDN standards which contractors must provide and required an additional feature, MLPP [16] --a MUF--which is not included in the National ISDN standards. Although software to comply with the National ISDN standards is readily available, DISA recognized that offerors would be required to offer customized software in order to provide the required MLPP feature.

AT&T offered [DELETED]. [17] The SSEB found that AT&T's [DELETED] ISDN [DELETED] satisfied the RFP requirements. Tr. at 198-199. Offerors were not required to provide all the National ISDN features, as GTE suggests; rather the RFP only required offerors to provide the 28 National ISDN features identified in the RFP. Although GTE complains that AT&T provided HITS users with [DELETED], GTE does not explain why this renders AT&T's proposal noncompliant, given that AT&T's [DELETED] satisfies the RFP requirements. Based on our review, we find reasonable the agency's determination that AT&T's proposal satisfied the ISDN requirement.

With regard to the next protested requirement, "grade of service" refers to the RFP requirements for a switched traffic surge capacity sufficient to handle up to 125 percent of the normal peak load network-wide [18] and a switching processor capacity sufficient to handle up to 200 percent of the switching system normal processor load. GTE contends that the SSEB found that AT&T's proposed [DELETED] switch and AT&T's switching system configuration cannot accommodate the required surge capacity.

The record shows that the SSEB determined that AT&T's [DELETED] switches and switching system configuration do both have sufficient capacity to satisfy the RFP surge requirements. Tr. at 189-196. While the SSEB was concerned that AT&T had not provided complete details as to how it would configure its system to meet these requirements, AT&T committed itself to providing services satisfying the surge requirements, and the SSEB found that sufficient information was provided by AT&T regarding its switches and architecture to allow the evaluators to determine that the grade of service requirements would be met. Tr. at 189. We find no basis in this record to question this determination. [19]

With regard to the timing and synchronization requirements--which are necessary to permit network switches to work as a synchronized team--the RFP required that all gateway switches be externally timed by reference to a Stratum 1 reference signal. AT&T proposed to [DELETED]. Although the SSEB determined that this approach was technically feasible by referring to technical literature for the [DELETED] that would support this requirement, AT&T had not cited this technical reference in its proposal to explain the feasibility of its approach, even though AT&T had provided the technical literature with its proposal. Tr. at 181. GTE, on the other hand, proposed a HITS synchronization plan based upon provisioning a Building Integrated Timing supply using the global positioning satellite system as its primary reference source at all HITS locations; this, the SSEB found, is a method commonly used by military communication sites in the Pacific region. The SSEB concluded, reasonably we find, that both AT&T's and GTE's proposals satisfied this requirement, but that GTE's proposed approach was superior to AT&T's. [20]

GTE also contends that AT&T provided insufficient information in its proposal to reasonably demonstrate compliance with the RFP requirements for the specified MUFs, the DIMSS interface, the critical assured service, the internal Stratum 3 clock, and the network management system. [21] Specifically, GTE contends that both the solicitation proposal preparation instructions and evaluation criteria required offerors to describe and document their approaches to satisfying the RFP requirements. GTE argues, citing the disadvantages and risks sections of the SSEB report, that AT&T did not satisfactorily demonstrate compliance with the above requirements.

Because the contracting agency is responsible for evaluating the data submitted by an offeror and ascertaining whether it is sufficient to determine the acceptability of the proposal, we will not disturb an agency's determination in this regard unless it is shown to be unreasonable. SAIC Computer Sys., B-258431.2, Mar. 13, 1995, 95-1 CPD Para. 156 at 8; Inframetrics, Inc., B-257400, Sept. 30, 1994, 94-2 CPD Para. 138 at 3.

Here, we find that DISA had a reasonable basis for its determination that AT&T had provided sufficient proposal information to demonstrate compliance with the RFP requirements. We do not agree with GTE that the RFP required that proposals be found noncompliant for any requirement where they failed to provide what the evaluators believed to be a complete discussion. In this regard, although there were more than 1300 technical requirements for which offerors had to show compliance, the RFP limited technical proposals to 800 pages. Given the complexity of HITS, the length and detail of the FRS, and this proposal page limitation, we do not find it unreasonable to expect that offerors would not fully discuss and describe their compliance with each and every requirement, and that the agency would accept statements of compliance with each offeror concerning some of the RFP requirements. [22] See SAIC Computer Sys., supra at 8-10. In this case, to determine compliance with the RFP's technical requirements, the SSEB sought a commitment to comply with the specified requirement and evidence of technical feasibility. Tr. at 53, 170, 215.

For example, in reviewing AT&T's proposal to determine whether AT&T had satisfied the requirement to provide the MUFs--e.g., MLPP, DSN7, [23] and PAT [24] --the SSEB found that AT&T had committed to providing these features in a future switch software revision. Specifically, the SSEB found AT&T was relying on the future release of Lucent Technologies software release [DELETED] [25] to provide a number of required HITS features, including DSN7 and MLPP, for its [DELETED] switches, and PAT for its [DELETED] switches. [26] Although the SSEB reported that AT&T had not documented the [DELETED] software in its proposal, the technical evaluation team chair explained in his hearing testimony that this meant that the unreleased software had yet to be tested and certified as to its ability to provide these features. Tr. at 47-48. The SSEB determined compliance in accordance with the above standard when it found that AT&T had committed to providing these features, which its proposal said would be provided through the future software release; that Lucent Technologies had committed, in a letter included in AT&T's management proposal, to providing these features in the [DELETED] software release; and that AT&T had successfully provided other 5ESS switches in other military networks with MLPP, PAT, and CCS7 (the commercial channel signaling protocol), which indicated to the evaluators that AT&T was capable of providing the MUFs here. See Tr. at 47-48, 55, 70, 234.

We note that GTE's proposal was found to be compliant, although it also was relying upon "undocumented," future switch software releases to provide HITS features. While, like AT&T's proposed future software releases, these future software releases were not documented or completely described to the evaluators' satisfaction, the SSEB similarly found GTE's proposal compliant (albeit representing a lower risk than AT&T's proposal), based upon GTE's commitment to provide the features and the evaluators' determination that GTE's solution was feasible. See Tr. at 56, 239-47.

GTE and AT&T were also treated similarly with regard to the agency's determination that both offerors proposed sufficient information to demonstrate compliance with the network management system requirements. [27] Both offerors proposed a commercial off-the-shelf (COTS) software solution to providing the HITS network management system features, and the SSEB found that both offerors would require extensive customization and integration of their software before implementation. In this regard, both offerors' network management system solutions were assessed to be high risk. Thus, the SSEB found, regarding AT&T's proposal, that:

"[AT&T's] proposed HITS network management solution is based on proved network management concepts and provides the comprehensive functionality needed to manage and operate the HITS network and services.

"The proposed [network management system] will employ high quality, highly capable, state-of-the-art, standards-based, COTS hardware and software.

"Based on the disparities, inconsistencies, and uncertainties in [AT&T's] proposal, proof that integration of the proposed [network management] tools and COTS application software to substantiate [AT&T's] statement of compliance has not been demonstrated or provided."

Regarding GTE's proposed network management system solution, the SSEB found:

"While the proposed [network management system] design appears to be technically sound and shows a good understanding of specified HITS network management requirements, the risk associated with implementing a fully compliant HITS [network management system] in time to meet specified milestones is high due to the apparent need to extensively customize and integrate the proposed COTS network management application software. Although stating compliance that network management of both switching and transmission network elements will be provided, this risk assessment is also based on the lack of detailed information in the proposal regarding the design, development, implementation and operation of network management functions for transmission network elements."

In sum, the SSEB had concerns with both offerors' proposed network management system solutions, see Tr. at 93-94, 254-257, but nonetheless found the offerors' proposals compliant with regard to these requirements. The SSEB found that AT&T's specific proposal commitment, coupled with the COTS hardware and software proposed and the evaluators' knowledge that the proposed hardware and software could meet the agency's needs, was sufficient to demonstrate AT&T's compliance with this requirement.

The SSEB also found that AT&T provided sufficient information in its proposal to demonstrate compliance with the RFP requirement that "[a]ll switches have an internal reference or system clock with a frequency and stability equal to a least a Stratum 3. . . ." Although AT&T committed itself to provide the required Stratum 3 internal clock for all switches, [DELETED]. Because AT&T did not specifically state that it was replacing the [DELETED] with a Stratum 3 clock, the SSEB assessed this aspect of AT&T's proposal as a risk. Nevertheless, the agency found that the switch could be equipped with a Stratum 3 clock as specifically promised by AT&T. In this regard, GTE does not assert that these switches cannot be provided with a Stratum 3 clock; [DELETED].

AT&T's proposal was also found to be compliant with the critical assured service requirement of the RFP, despite the SSEB's concerns with the lack of detail provided in the proposal, because AT&T described an approach that the SSEB found would meet this requirement. The agency also found that critical assured service is a routine feature of military communications networks and would not be difficult for experienced vendors, such as GTE and AT&T. We find no basis to question this aspect of DISA's evaluation of AT&T's proposal.

In sum, as noted by GTE, the SSEB found considerable risk with various aspects of AT&T's proposal because of AT&T's limited description of how it would accomplish the foregoing (and other) requirements and because AT&T required software development or customization to meet them. [28] However, this does not mean that AT&T was not compliant with these requirements; to the contrary, this evidences the reasonableness of the agency evaluation--that is, DISA recognized that AT&T had offered a compliant solution but one that entailed risk to the government. See TEAC Am. Corp., Inc., B-259831 et al., May 3, 1995, 95-1 CPD Para. 273 at 11.

GTE also complains that AT&T will not comply with the mandatory schedule requirements. Specifically, GTE argues that DISA's evaluation record establishes that it is probable, if not likely, that AT&T will not be able to satisfy the RFP's FOC date, inasmuch as the BVWG determined that AT&T would likely miss the IOC date. DISA and AT&T respond that AT&T took no exception to the schedule requirements in its proposal and therefore is compliant with the IOC and FOC requirements, and that GTE has misconstrued the purpose and conclusions of the BVWG's risk quantification analysis.

The record shows that the BVWG accepted the SSEB's assessment that AT&T's proposal complied with the RFP requirements as well as the SSEB's determination that AT&T's approach entailed a moderate to high risk to the government that the IOC and FOC dates would not be met; the BVWG quantified the potential for delay that arose from each of the areas of AT&T's proposal for which the SSEB assessed moderate or high risk. Tr. at 292-294, 330-331. The BVWG found that a significant potential for delay existed for AT&T meeting the IOC date because AT&T needed switch software revisions to satisfy various HITS requirements, such as providing MUFs on all switches and [DELETED]; customizing and integrating the COTS network management system software; and configuring AT&T's switches to satisfy grade of service requirements. The BVWG found that it was probable that AT&T's software revisions could result in a delay of [DELETED] after the IOC date and that AT&T's customization of the network management software could result in up to a [DELETED] delay in satisfying the IOC requirement.

The BVWG did not find that AT&T would not meet the FOC date or calculate any risks of AT&T's proposal not meeting the FOC date, see Tr. at 343-344, although the BVWG found it was likely that there may be grade of service problems during contract performance with AT&T's solution, and that, given the minimal impact on the user, it may take between [DELETED] after the FOC date to detect and correct any grade of service problem. Tr. at 359, 445-446. The BVWG chair testified, and the record otherwise evidences, that the BVWG's calculations of probable delay were prepared as a means of quantifying the risk assessed in AT&T's proposal and were not intended to represent "a forecast of [a] future event actually occurring or not." Tr. at 301-302; see Tr. at 324-325, 405-406, 409.

The BVWG's risk analysis, including estimates of probable delay, was fully presented to the SSA, who determined that AT&T would, in fact, comply with the RFP's required schedule and that, in this regard, the BVWG's estimates of risk and delay were too pessimistic. The SSA testified that, based upon his more than 20 years of software development experience, he believed that AT&T would satisfy the RFP's schedule requirements, notwithstanding the significant software development and customization that had to be performed by that firm. Tr. at 548. Specifically, the SSA testified that in his opinion the software development required would not be so difficult as to cause AT&T to miss the FOC date. Regarding the customization of the network management system software, he stated that "[t]his network stuff is done every day. We run millions of lines of this code around the country." Tr. at 548-49. Regarding the switch software, the SSA was aware that the features to be provided, although not under the DSN7 protocol, were already available and should not be difficult to develop for this contract. Tr. at 570, 573.

The SSA also concluded that he had two "very good companies"--AT&T and Lucent Technologies--doing the software revisions and software customization, and that this would mitigate any potential for delay, particularly given AT&T's financial incentive to satisfy the schedule requirements; that is, AT&T would not be paid until it provided service in accordance with the HITS contract. Tr. at 570; see Tr. at 548-549.

The record also shows that the BVWG's determination of the length of AT&T's likely delay was based upon that group's subjective judgment and not upon any calculation of the level of effort actually required to perform this software revision and customization. Tr. at 340, 361, 384-85. In addition, the BVWG's assessment of potential delay associated with AT&T's need to provide the [DELETED] software revision was based in part upon the SSEB's and BVWG's misunderstanding of when the relevant software release would occur. During the protest process, AT&T provided the affidavit of Lucent Technologies [DELETED] for the 5ESS-2000 switch, who states the MUFs required under DSN7 would be supplied by software release [DELETED]. In its proposal, AT&T had incorrectly identified the software to provide the MUF features as software release [DELETED], rather than [DELETED]. [DELETED], DISA incorrectly believed that this represented a delay in the anticipated release of the [DELETED] software, which was planned for late 1996. Tr. at 231-234, 330. In fact, software release [DELETED] (which does not include MUFs for this contract) was released on schedule in December 1996. The SSEB chair testified that this misunderstanding affected the evaluators' assessment of the risk associated with AT&T's need for switch software revisions. Tr. at 232. Moreover, [DELETED]. [29] Tr. at 351.

We find no basis to object to the SSA's judgment that AT&T would satisfy the RFP schedule requirements. Contrary to GTE's argument, DISA's lower-level evaluators did not determine that AT&T would not satisfy the contract schedule requirements, but found that AT&T's approach entailed risks that it would not meet the schedule. Indeed, the record establishes that the BVWG's estimates of delay were not intended to forecast AT&T's noncompliance with the schedule requirements, but to quantify possible risk to the government, should delays occur. In any event, source selection officials are not bound by recommendations or evaluation judgments of lower-level evaluators but may make their own judgments, which are subject to the tests of rationality and consistency with the stated evaluation criteria. See PRC, Inc., B-274698.2; B-274698.3, Jan. 23, 1997, 97-1 CPD Para. 115 at 7. Here, GTE has not shown that the SSA's judgment that AT&T would meet the RFP schedule requirements was unreasonable. [30]

GTE also complains that the agency's best value analysis was unreasonable because DISA did not assign any value to GTE's technical superiority and because DISA underestimated the cost to the government of the risk in AT&T's proposal. In this regard, GTE asserts that DISA effectively converted this procurement from a best value basis for award to one in which award was simply made to the offeror with the lowest-priced, technically acceptable proposal.

Source selection officials have broad discretion to determine the manner and extent to which they will make use of the technical and cost evaluation results in a negotiated procurement. Grey Advertising, Inc., 55 Comp.Gen. 1111, 1118-21 (1976), 76-1 CPD Para. 325 at 9-12. In deciding between competing proposals, cost/technical tradeoffs may be made, the propriety of which turns not on the difference in technical scores or ratings per se, but on whether the source selection officials judgment concerning the significance of the difference was reasonable and adequately justified in light of the RFP evaluation scheme. Southwest Marine, Inc.; American Sys. Eng'g Corp., supra at 17; DynCorp, B-245289.3, July 30, 1992, 93-1 CPD Para. 69 at 8.

The record contains detailed documentation that establishes that the BVWG and SSA performed a thorough and complete best value analysis of the two firms' proposals and gave GTE's proposal appropriate credit for its assessed technical superiority. As indicated above, the BVWG identified the significant discriminators between the two firms' proposals, where GTE was found to offer a superior solution or where AT&T's proposal was found to pose greater risk than GTE's. Each of the discriminators identified by DISA's evaluators was reviewed by the BVWG and SSA to assess the impact on the user. [31] Tr. at 296.

In making this assessment, the BVWG and SSA recognized that, given the nature of this contract as a fixed-price service contract with tight performance specifications, the level and quality of service received from a compliant vendor would not significantly vary; as stated by the agency, the services were basically "transparent to the user." Tr. at 260-262, 336-338, 488-489. In this regard, the SSA noted in his source selection decision [DELETED]--dedicated transmission and transmission-dependent switched voice services--that "[t]ransmission essentially is a commodity, which reputable vendors provide with little or no difference in quality." The RFP's extensive pre-acceptance testing procedure also provided confidence to the BVWG and SSA that any potential problems in AT&T's performance that would affect the level of service provided would be identified and corrected before implementation of the system. Tr. at 296-297, 520. We cannot say from our review of the record that the BVWG's and SSA's judgment in this regard was unreasonable.

Accordingly, DISA evaluated the value of what it considered to be the real benefits of GTE's higher-rated proposal to ascertain whether they justified the payment of the associated price premium. In performing this evaluation, the BVWG and SSA assessed the impact of the evaluated discriminators by focusing upon the cost to the government of acquiring services from other sources during any period of delay or of providing additional government oversight and monitoring, Tr. at 414, and the cost to the government of directing AT&T to replace a compliant solution with a more expensive solution. Tr. at 391.

For example, the BVWG and SSA, in reviewing the discriminator based upon AT&T's reliance upon the future [DELETED] software release, concluded that the risk of additional cost to the government for oversight/monitoring and acquisition of services elsewhere could be approximately $[DELETED]. Regarding the network management system software, AT&T was found to pose greater risk than GTE to perform necessary customization and integration of COTS software, because AT&T had failed to completely describe how its various software applications would interface; the BVWG and SSA assessed the potential cost of this to the government to be approximately $[DELETED] for additional oversight/monitoring. Regarding timing and synchronization, the BVWG and SSA found that GTE had proposed a more reliable and robust method of ensuring timing and synchronization than had AT&T, although AT&T's solution was compliant; to assess the impact of this discriminator, the BVWG and SSA determined that approximately $[DELETED] would be the additional cost to the government of directing AT&T to replace its compliant solution with GTE's superior solution. [32]

GTE's objection to DISA's quantification of the risk in AT&T's proposal is based upon its view that there will be a diminution in the level of service received by the government from AT&T. The protester calculates the value of this diminution of service by reference to the system of "outage credits" provided by the RFP where a contractor fails to meet specified grade of service requirements. [33] This argument, however, fails to account for the agency's assessment--which we have found reasonable--that there will be no diminution in the level of service received from AT&T, and for this reason provides us with no basis to object to DISA's quantification of the discriminators between the two firms' proposals.

The record simply does not support GTE's contention that award was based on the lowest-priced, technically acceptable proposal, but evidences that a best value award decision was reasonably made in accord with the RFP evaluation scheme. In this regard, the record shows that SSA accepted the findings of the lower level evaluators that GTE's technical proposal was superior to AT&T's; he also accepted that AT&T's management proposal was superior to GTE's. The record establishes that the SSA went beyond the evaluation ratings to assess the impact of the difference between the two firms' proposals. The SSA recognized that the essential technical difference between AT&T's and GTE's proposal was that AT&T's proposal posed a greater risk of delay and of a need for additional government oversight than did GTE's, Tr. at 563, but that the service the customer would ultimately receive from either vendor would be essentially the same. Tr. at 487. Thus, it was AT&T's greater risk, as quantified by the BVWG, that the SSA weighed against GTE's substantial price premium (even though the SSA believed that this risk had been exaggerated by the lower-level evaluators). The SSA also gave appropriate weight to AT&T's superior management proposal rating and good past performance record, which the SSA found mitigated the risks in AT&T's proposal. Tr. at 557.

Furthermore, the SSA was aware that telecommunications services were available under the FITS contract, at lower rates than offered by AT&T or GTE, for eleven months after the FOC date; this "safety net" also mitigated the risk of delay in AT&T's proposal. Tr. at 482, 499-501.

In sum, the SSA found that the evaluated risks in AT&T's fixed-price proposal were worth the substantial price advantage. Contrary to GTE's arguments, we find this is an appropriate best value analysis, which is both reasonable and consistent with the stated evaluation criteria.

The protest is denied.

Comptroller General of the United States

1. Although these contracts have expired, DISA has been acquiring telecommunication services under the Follow-on Interim Telephone System (FITS) contract, which was awarded to GTE on a sole source basis and can be extended through July 1999.

2. Subcriteria were also identified for the technical quality and management quality criteria.

3. GTE also submitted an alternate proposal that offered to continue use of the older switches that had supported service under the Oahu Telephone System contract. This proposal was rejected because it did not comply with several critical HITS requirements, posed significant scheduling risk, and would provide inferior service relative to that offered by the other proposals.

4. A hearing was conducted to elicit the testimony of the chair of the SSEB's technical team, the chair and other members of the best value working group (BVWG), and the source selection authority (SSA).

5. Under DISA's scoring methodology, a score of 4 reflected an acceptable proposal--"[t]he offeror's proposal meets requirements in an acceptable manner"; a 5 to 8 score reflected a good proposal--"[t]he offeror's proposal exceeds minimal requirements in some meaningful aspects and should result in successful completion of the requirement."

6. Low risk was defined as having little potential to cause disruption of schedule, increase in cost, or degradation of performance. Moderate risk was defined as having some potential to cause disruption of schedule, increase in cost, or degradation of performance, but where special contractor emphasis and close government monitoring could probably overcome difficulties. High risk was defined as likely to cause significant serious disruption of schedule, increase in cost, or degradation of performance, even with special contractor emphasis and close government monitoring.

7. AT&T proposed 12 5ESS-2000 switches in the following configurations: [DELETED].

8. SONET is an optical transmission network. Tr. at 90.

9. The discounted price was calculated by using a discount rate of 7.9 percent and mid-year discounting factors derived from Office of Management and Budget Circular A-94.

10. "M" equals a million.

11. "IGCE" refers to the independent government cost estimate.

12. The SSAC and BVWG were not aware of GTE's and AT&T's proposed prices until after the completion of the best value analysis. Tr. at 293.

13. "Critical assured service" is a sub-element of the topology and survivability discriminator category. It refers to the requirement that offerors provide certain designated critical switched voice users assured service "to guarantee these subscribers will be able to initiate and receive calls even when the subtending central office or access line is unable to process calls."

14. GTE originally argued that AT&T was not compliant with a number of other RFP requirements, such as the provision of specified military unique features (MUF) and an internal Stratum 3 clock. After review of the agency's report, GTE contended only that AT&T had provided insufficient information in its proposal to demonstrate compliance with these requirements.

15. The MUFs required by the RFP include multilevel precedence and preemption (MLPP), Defense Switched Network (DSN) 7 Common Channel Signaling (CCS), and Precedence Access Threshold (PAT).

16. MLPP provides specified users with the capability to get assured service. Tr. at 43.

17. Both AT&T and GTE proposed to satisfy some of the 28 required National ISDN features with functional equivalents. DISA treated GTE and AT&T similarly in finding that their proposals satisfied the ISDN requirement. GTE does not challenge this aspect of DISA's evaluation.

18. "Normal peak load" was defined as five call attempts per line per hour.

19. We note that AT&T's proposal's lack of detail concerning how its network design would satisfy this requirement was accounted for by the SSEB in its assessment of this aspect of AT&T's proposal as a moderate risk. GTE's proposal was treated similarly in this regard; although GTE's proposal was found compliant with this requirement, the SSEB was similarly concerned that GTE had not provided complete enough detail concerning its network to allow the evaluators to verify that GTE's network would satisfy surge requirements. Tr. at 251-253.

20. The difference in GTE's and AT&T's approaches was accounted for by DISA in its evaluation by assessing this to be a significant discriminator; in fact, AT&T's proposal was rated high risk for this requirement because of AT&T's vague explanation of its timing and synchronization plan, while GTE's proposal was assessed as low risk.

21. GTE does not contend that AT&T will be incapable of providing these features under the HITS contract.

22. In fact, during discussions DISA informed the offerors that their "failure to fully substantiate compliance may result in a lower evaluation score and/or higher risk assessment," which indicated to the offerors that any lack of detail could be addressed in the agency's comparative evaluation of proposals.

23. DSN7 is the military version of the commercial common channel signaling requirement. Tr. at 42-43.

24. PAT is the tool used by the switching element to allow implementation of MLPP. Tr. at 42-43.

25. The record shows that operating system and core applications--designated as 5E software--are the same for all the 5ESS-2000 switches regardless of configuration. Each software release is denominated as 5E with a revision number; thus, for example, [DELETED]. Although not explained in AT&T's proposal nor recognized by the evaluators, the actual software release that will provide the MUFs for the HITS is [DELETED].

26. AT&T is also relying upon future software releases--specifically, [DELETED]--to satisfy the [DELETED] requirements.

27. The HITS network management system will provide the government with access and control capability of all the HITS resources. Tr. at 89.

28. The risk to the government was fully identified and described in the SSEB and SSAC reports that were provided to and reviewed by the SSA.

29. [DELETED].

30. GTE also argues that DISA treated offerors unequally when it rejected GTE's alternate proposal allegedly because of the risk that GTE could not satisfy the schedule requirements, while accepting AT&T's proposal evidencing similar schedule risk. The record establishes, however, that DISA rejected GTE's alternate proposal because it was based upon outdated technology, would not meet numerous requirements even after GTE upgraded its proposed system, and was markedly inferior to AT&T's proposal as well as to GTE's other proposal.

31. We note that in objecting to the BVWG's analysis, GTE does not assert that the discriminators that were identified are not appropriate or that there were additional technical discriminators that should have been considered; rather, GTE's complaint is that it did not receive credit for its higher technical score and that the cost of AT&T's potential for failure was underestimated.

32. The agency also recognized that GTE had proposed a [DELETED] transmission backbone than did AT&T. Specifically, GTE proposed to provide an [DELETED] ("OC" refers to an optical carrier--i.e., a fiber optic transmission channel) to most HITS locations, while AT&T offered to provide [DELETED], if later required. DISA determined that the benefit of [DELETED] was minimal, however, because the agency [DELETED]. The BVWG quantified this difference between the proposals to be approximately $[DELETED], representing [DELETED]. GTE has not shown this determination to be unreasonable.

33. The RFP provides for payments by the contractor to the government during periods of severe service interruption.

* DOCUMENT FOR PUBLIC RELEASE

A protected decision was issued on the date below and was subject to a GAO Protective Order. This version has been redacted or approved by the parties involved for public release.