Matter of: Scientific-Atlanta, Incorporated File: B-255343.2; B-255343.4 Date: March 14, 1994 REDACTED VERSION[*]

Matter of: Scientific-Atlanta, Incorporated File: B-255343.2; B-255343.4 Date: March 14, 1994 REDACTED VERSION[*]

DIGEST

Attorneys

DECISION

We deny the protests in part and dismiss them in part.

BACKGROUND

On March 15, 1993, the agency issued the solicitation for a cost-plus-fixed-fee contract for design, development, and delivery of an engineering development model (EDM) of an onboard trainer for the AN/BQQ-5D and AN/BQQ-5E submarine combat systems. The AN/BQQ-5 submarine sonar system, the primary acoustic system employed on the SSN 637, SSN 688, and SSBN 726 submarine classes, uses passive and active acoustic signals to detect, classify, and localize potential threats; the purpose of the OBT is to provide sonar operators with realistic training at sea as well as in port by injecting acoustic signals into the AN-BQQ-5 system prior to the point where it processes data.

The solicitation included options for two additional EDMs, and delivery of production models on a fixed-price incentive, firm-target basis, as well as for additional engineering services. The RFP also contained the standard Federal Acquisition Regulation (FAR) clause advising offerors of the agency's intention to evaluate proposals and award a contract without discussions. FAR Sec. 52.215-16, alternate III. The solicitation provided for award to the technically acceptable offeror whose proposal represented the greatest value; estimated price would be divided by total technical score, and the proposal with the lowest price per technical point ratio would be considered to represent the greatest value.

The solicitation provided for evaluation of proposals based on technical capability and price; the two major technical evaluation factors were performance and management, with performance worth 70 percent of the technical evaluation versus 30 percent for management. The performance factor contained four subfactors; the subfactors of external interface and system design were of equal importance, but of greater importance than the subfactors of integrated logistics supportability (ILS) and design and construction. The three subfactors under the management factor--ability to meet schedule, resources, and participation of small and small disadvantaged businesses--were of equal importance. The source selection plan provided for numerical scores to be associated with adjectival ratings as follows: outstanding, 90 to 100 points; excellent, 80 to 89 points; acceptable, 70 to 79 points; and unacceptable, 0 to 69 points.

The solicitation advised offerors to submit proposals in three volumes--Performance, Management, and Price; the Price volume was to consist of seven sections, including a cost summary, supporting cost data, and completed sections A through K of the solicitation. Section 4 of the Price volume was for noting any comments or exceptions to the specifications, terms, and conditions of the RFP, but offerors were advised that any such comments or exceptions could "form the basis for an [o]fferor to be considered ineligible for award."

Seven offerors submitted proposals on June 11, and the agency referred the proposals to its technical evaluation review panel (TERP). Individual members of the TERP assigned adjectival ratings and assessed risk for each of the subfactors; the TERP used the individual ratings to develop a consensus technical and risk rating for each subfactor and factor. The TERP report was referred to the contract award review panel (CARP), which assigned raw point scores based on the technical ratings; these raw point scores were then adjusted for risk and weight. If a proposal was rated as low risk, its technical score received [deleted], medium risk proposals received [deleted] percent of their raw technical score, and high risk proposals received [deleted] percent of their raw technical score. In addition, performance scores were adjusted to reflect their [deleted] weight, and management scores were adjusted to reflect their [deleted] weight.

For example, IBM's proposal received ratings of [deleted] under all of the performance subfactors except ILS (one of the two subfactors of lesser importance), which was rated as [deleted]. In addition, the agency classified IBM's technical risk as [deleted] for all subfactors except external interface (one of the two subfactors of greater importance), which was rated as [deleted] risk. Overall, IBM was rated [deleted] ([deleted] to [deleted] points) with [deleted] risk; the CARP noted however that IBM had proposed a particularly innovative approach.[1] As a result, the CARP assigned IBM a score at [deleted] points, which was adjusted for [deleted] risk generating a weighted score of [deleted] points under the performance factor.[2] Under the management factors, IBM was rated [deleted] points, with [deleted] risk, for a weighted score of [deleted] points. Thus, IBM's total technical and management score was [deleted] points.

S-A received [deleted] ratings under all performance subfactors except design and construction (one of the two less important subfactors), but was rated as [deleted] risk under the external interface subfactor, [deleted] risk under the other two subfactors, and [deleted] risk under design and construction. S-A's raw score of [deleted] points ([deleted]), adjusted for [deleted] risk, resulted in a weighted score of [deleted] points under the performance factor. Under the management factor, S-A was rated as [deleted] with [deleted] risk. Thus, its score of [deleted] points (adjusted for risk to [deleted] points) resulted in a weighted score of [deleted] points for management. Since S-A's total score, [deleted] points for performance and [deleted] points for management, totaled only [deleted] points, the proposal was rated as [deleted].

The evaluators concluded that S-A's development schedule, which presumed that the agency would exercise the options for the additional EDMs, presented some risk; further, they found that the S-A proposal would require extensive software development. The evaluators also concluded that S-A's analysis of SNR degradation was based upon an incorrect noise shape, and that its proposal did not address methods to ensure training realism. Specifically, the evaluators concluded that S-A had not demonstrated how it would smooth frequency and amplitude changes in generating a contact signal.[3] They also found that S-A had not demonstrated the ability to generate uncorrelated (stripe free) noise as required by the system specification.[4] Further, S-A did not meet the specification requirement that high frequency active emissions be provided to the sonar array.

In addition to their technical concerns, the evaluators also concluded that S-A had taken exception to certain solicitation requirements. In this regard, section 4 of S-A's price proposal stated, in relevant part:

"EXCEPTIONS TO THE RFP/SOW, TERMS AND CONDITIONS "(RFP Sec. L, para 5.1.3, subsec 4)

"4.4 Subcontracts Approval

"S-A assumes that any requirement for advanced notification and/or [c]ontracting officer's written consent, as contemplated by FAR [Sec.] 52.244-1 `Subcontracts (Fixed Price Contracts),' shall be deemed granted upon contract award for all subcontracts proposed herein.

"4.5 Approval and Acceptance of Deliverable Documents

"Delivery of items specified as contract deliverables in the [s]tatement of [w]ork (SOW) shall satisfy S-A's obligations under the contract, and shall constitute full and final acceptance of the work performed by S-A.

. . . . .

"4.8 Other Terms

"All other terms will be subject to mutual agreement prior to contract award."

The agency interpreted these exceptions as (1) usurping the agency's right to approve subcontracts, (2) abrogating the contractor's warranty obligations, and (3) essentially leaving all contract terms open for further negotiation until the time of award.

After evaluations were completed, only [deleted] offerors received overall ratings of [deleted] and IBM with [deleted] points. S-A, with [deleted] points, received the [deleted] technical score. IBM's proposal represented the lowest price per technical point value of those received, [deleted] per point. S-A was [deleted] in price per technical point, at [deleted] per point.[5] On October 1, 1993, the agency awarded a contract to IBM and this protest followed.

DISCUSSION

S-A argues that the evaluation of its technical proposal was unreasonable based on the evaluators' conclusions regarding weaknesses in its approach to training realism--specifically, smoothing and uncorrelated noise.[6] In reviewing protests against an agency's technical evaluation and decision to eliminate a proposal from consideration for award, we review the record to determine whether the agency's judgments were reasonable and in accordance with the listed evaluation criteria and whether there were any violations of procurement statutes or regulations. CTA, Inc., B-244475.2, Oct. 23, 1991, 91-2 CPD Para. 360. Here, we see no basis to object to the agency's evaluation.

As a preliminary matter, we note that S-A initially received [deleted] rating under the performance factor, but ultimately received an overall rating of [deleted]. In the agency's view, S-A's raw score under the performance factor was of less importance overall than the agency's perception that the proposal presented [deleted] risk, and S-A's relatively [deleted] score for management. Apart from the risks to training realism of S-A's approach (and all offerors except [deleted] were rated [deleted] risk under the performance factor), S-A has not addressed the risks noted by the agency with regard to its software-intensive development schedule[7] or its presumption that the agency would purchase all three EDMs.

With regard to training realism, S-A has argued, alternatively, that page limitations precluded it from addressing smoothing, that the techniques used were so common that there was no need to describe them, and that its proposal adequately addressed the issue. The protester asserts that Figure 3.3.1.2.1-2 of its proposal describes smoothing methods, showing that amplitude smoothing will be accomplished by mix control between current and last amplitude. S-A also references other pages of its proposal in its claim that it provided the necessary information on smoothing.

Our review of S-A's proposal shows that mix control is addressed in a box of the flow diagram of Figure 3.3.1.2.1.2-2. Page 3-33 mentions that data is to be updated at a 8Hz rate, and page 3-80 states that S-A's processor calculates time domain "in such a way as to produce a continuous time domain output signal" using "an amplitude and frequency mix control." The agency found that this discussion lacked sufficient detail to properly evaluate this feature or to conclude that there was no risk inherent in the additional complication caused by S-A's use of two contact signals for the spherical array and the towed array. Based on these materials and S-A's pleadings, we are unable to conclude that the agency's concern was unreasonable.

With respect to the agency's conclusion that S-A had not established its ability to generate uncorrelated (stripe free) noise, the agency states that S-A's contact broadband generator can generate only 8 unique sounds per minute, but that the sound must be supplied to a minimum of 731 sensors per second to ensure that the trainer does not produce striping. S-A relies on figure 3.3.1.3.2-1 as evidence that the agency was unreasonably concerned about this issue.

Section 3.3.1.3.2 of S-A's proposal discusses the operation of S-A's decorrelation RAM. According to S-A, the system generates a random sequence of noise samples, which repeats every 2,863 minutes, and the decorrelation RAM allows the trainer to subdivide the sequence into 960 separate sequences, or a 3-minute repeat cycle. With a large enough decorrelation RAM, S-A hoped to insure that the delay between repetition of any sample in the sequence would be greater than the maximum delay rate of the tactical beamformer (which delays signals for the purpose of determining whether they are correlated). Our review shows that the information at issue is provided in a box diagram that mentions "decorrelation RAM" but provides little explanation about how it would ensure that the trainer does not produce striping. Based on the record before us, we have no basis to conclude that the agency was unreasonable in expressing concern about the risk of striping in S-A's approach.

Since S-A has not challenged the evaluation of its management proposal, and since the evaluation of risk has not been shown to be unreasonable, the record supports the agency's evaluation and its determination not to consider S-A's initial proposal, with a score of [deleted] points [deleted], for award. Since the solicitation advised offerors that the agency intended to make award without discussions, the protester could not presume that it would have a chance to correct deficiencies and weaknesses through discussions. The burden was on S-A to submit an initial proposal that adequately demonstrated its own merits, and the protester ran the risk of rejection by failing to do so. DRT Assocs., Inc., B-237070, Jan. 11, 1990, 90-1 CPD Para. 47. There is no basis in this record for concluding that the decision to award without discussions was unreasonable.

In addition to our conclusion that the evaluation was reasonable, we also note that the agency states that S-A's proposal was ineligible for award because of the exceptions taken to material terms of the RFP.

In our view, S-A's exceptions to the RFP, discussed above, created an ambiguity as to S-A's intentions. For example, S-A took exception to the agency's right of review and acceptance of data deliverables. Some 29 of the items in the contract data requirements list provide the government a right of review and approval before acceptance. S-A argues that this exception was intended to apply only where no acceptance terms were set forth in the solicitation, but S-A's proposal contains no such limitation, and we believe that the agency properly found that the proposal did not conform to this material requirement. By taking exception to the government's inspection and acceptance rights, the protester rendered its initial proposal ineligible for award. Any proposal in a negotiated procurement that fails to conform to material terms and conditions of the solicitation should be considered unacceptable and may not form the basis for award. Ralph Korte Constr. Co., Inc., B-225734, June 17, 1987, 87-1 CPD Para. 603.

Finally, we note that S-A has filed a second protest challenging the technical acceptability of IBM's proposal and alleging that the agency improperly allowed IBM to correct deficiencies in its proposal. Under our Bid Protest Regulations, a party is not interested to maintain a protest if it would not be in line for award if the protest were sustained. 4 C.F.R. Secs. 21.0(a) and 21.1(a). Since the agency properly found S-A's proposal unacceptable--and thus would not have included S-A in discussions even if it chose to hold them--S-A is not an interested party for purposes of challenging the award to IBM. Hughes Tech. Servs. Co., B-245546.3, Feb. 12, 1992, 92-1 CPD Para. 179.

The protests are denied in part and dismissed in part.

* The decision, issued on March 14, 1994, contained proprietary information and was subject to a General Accounting Office protective order. This version of the decision has been redacted. Deletions in text are indicated by "[deleted]."

1. The CARP noted that IBM's spherical array signal summation at the clipper output significantly reduced the concerns associated with a digital approach. While the CARP stated that this approach created some risk because of dynamic range limitations and IBM's failure to demonstrate conclusively that it could provide the required controlled summation over a full range of signal-to-noise ratios (SNR), the CARP ultimately agreed with the TERP's [deleted] conclusion.

2. The weighted score of [deleted] points was generated as follows: [deleted] point score X [deleted] (for [deleted] risk) X [deleted] (for performance weighing) = [deleted].

3. S-A's approach was based on the use of frequency domain processing, which the agency viewed as a good approach for reducing processing time. Frequency domain refers to an analysis of the frequencies associated with sound. For example, analysis of a foghorn produces a curve that peaks at the low frequencies since a foghorn is composed mostly of low frequency sounds. The sonar display used here, however, requires time domain information. Time domain refers to the change in sound over time and is graphed as a wave with alternating peaks and valleys. In time domain, the wave is either compressed or stretched if the source is moving toward or away from the receiver. In translating data from frequency domain to time domain, training realism requires that the trainer reproduce this effect of a signal changing in accordance with increasing or decreasing distance to the source--i.e., the Doppler effect. To produce this effect, which takes the form of a continuous curve, the trainer employs a process called smoothing. Improper smoothing can result in a discontinuous signal, as opposed to the smooth signal encountered in a live situation.

4. Paragraph 3.2.1.2.6 of the system specification states, "[s]ea state and ownship noise shall be random and not correlatable across elements of the tactical receive aperture." Thus, the RFP required that the recreation of background noise must be realistic, in the sense that noise samples for each element of the array aperture must appear independently generated rather than correlated. A contact signal, while it might appear slightly different to each element because of its slightly different position relative to each element, is correlated because it comes from one source. If background noise appears correlated (striped), operators might interpret it as a contact signal, thus ruining the training exercise.

5.[Deleted].

6. To the extent that S-A also challenges the Navy's determination that S-A used the wrong noise shape in its analysis of SNR degradation, its protest is untimely. S-A's initial protest, filed after an October 15 debriefing, did not raise the issue. S-A first raised the issue after receipt of the agency report--and chiefly in the context of a solicitation ambiguity with regard to the required noise shape--although S-A did offer to show that it had used a more stressful noise shape than was required. The agency filed a subsequent report in response to this argument, and the protester abandoned its contention that the solicitation was ambiguous, arguing for the first time that it had used the correct wave shape in its analysis. This argument is untimely whether we consider it raised in S-A's December 20 comments on the agency report, or in its February 12 comments on the supplemental report. See 4 C.F.R. Sec. 21.2(a)(2) (1993).

7. While S-A did assert that it was unreasonable to compare its total lines of software with IBM's new software, we note that S-A's proposal requires 60K lines of new code, nearly 12 times the amount of IBM's new code.