Matter of: Norden Systems, Incorporated File: B-255343.3 Date: April 14, 1994 94-1 CPD 257

Matter of: Norden Systems, Incorporated File: B-255343.3 Date: April 14, 1994 94-1 CPD 257

PROCUREMENT Competitive Negotiation Initial offers Rejection Propriety PROCUREMENT Competitive Negotiation Offers Evaluation errors Evaluation criteria Application Where evaluation was reasonable and consistent with the evaluation scheme listed in the solicitation, and agency properly found initial proposal unacceptable, agency properly rejected proposal. Since solicitation advised offerors of the agency's intention to make award without discussions, protester could not assume that it would have the opportunity to amend its proposal in discussions. PROCUREMENT Bid Protests GAO procedures Protest timeliness Apparent solicitation improprieties Argument that procurement was too complex for award without discussions is untimely, where the agency advised offerors prior to submission of proposals of its intention to make award without discussions.

Attorneys

DECISION

We deny the protest in part and dismiss it in part.

BACKGROUND

On March 15, 1993, the agency issued the solicitation for a cost-plus-fixed-fee contract for design, development, and delivery of an engineering development model (EDM) of an onboard trainer (OBT) for the AN/BQQ-5D and AN/BQQ-5E submarine combat systems. The AN/BQQ-5 submarine sonar system, the primary acoustic system employed on the SSN 637, SSN 688, and SSBN 726 submarine classes, uses passive and active acoustic signals to detect, classify, and localize potential threats; the purpose of the OBT is to provide sonar operators with realistic training at sea as well as in port by injecting acoustic signals into the AN-BQQ-5 system prior to the point where it processes data.

The solicitation included an option for delivery of production models on a fixed-price incentive, firm target basis, and contained the standard Federal Acquisition Regulation (FAR) Sec. 52.215-16 clause, alternate III, advising offerors of the agency's intention to evaluate proposals and award a contract without discussions. The solicitation provided for evaluation of proposals based on technical capability and price; the two major technical evaluation factors were performance and management, with performance worth 70 percent of the technical evaluation versus 30 percent for management. The solicitation advised offerors that the agency would divide estimated price by total technical score, and would make award to the acceptable proposal with the lowest price per technical point ratio as representing the greatest value to the government.

Seven offerors submitted proposals on June 11, and the agency referred the proposals to its technical evaluation review panel (TERP). The source selection plan provided for numerical scores to be associated with adjectival ratings as follows: outstanding, 90-100 points; excellent, 80-89 points; acceptable, 70-79 points; and unacceptable, 0-69 points. Individual members of the TERP assigned adjectival ratings and assessed risk for each of the subfactors; the TERP used the individual ratings to develop a consensus technical and risk rating for each subfactor and factor. The TERP report was referred to the contract award review panel (CARP), which assigned raw point scores based on the technical ratings; these raw point scores were then adjusted for risk and weight. High risk proposals received 78 percent of their raw technical score, and medium risk proposals received 87.5 percent of their raw technical score; the performance scores were adjusted to reflect the 70-percent weight given to performance and the management scores to reflect the 30-percent weight given to management.

For example, Norden received a rating of acceptable with high risk under the performance factor and was assigned a raw score of 72 points ("acceptable"). This raw score was adjusted for high risk and produced a weighted score of 39 points under the performance factor. Under the management factors, Norden was rated acceptable with moderate risk, a raw score of 74 points, which generated a weighted score of 19 points. Norden's weighted scores for performance and management totaled 58 points--unacceptable.

Evaluators found that Norden's use of a digital implementation minimized installation impact, and that its VME chassis architecture and use of commercial off-the-shelf (COTS) products enhanced supportability. The evaluators considered that the proposed design surpassed specification requirements for modeling in some areas, in particular, bearing update rate and simulated noise for in-port training. However, the evaluators perceived considerable risk in the proposal because they concluded that Norden's analysis of signal-to-noise ratio (SNR) was based upon an incorrect shape. Further, the evaluators were of the opinion that Norden did not understand the importance of ship safety and maintaining the integrity of the tactical system, because its design routed the tactical signal through a COTS processor unit, whose parts did not meet the shock requirements of the tactical system. The evaluators found that Norden's proposed hard drive storage capacity did not meet requirements for expansion capability and believed that Norden had underestimated the amount of software design necessary; they considered this risk increased by what they regarded as an inadequate discussion of the effort that was proposed. Validation and system repair information was vague and inconclusive and the proposal on the whole did not demonstrate understanding of logistics support requirements. The lack of committed personnel caused concern, in view of Norden's aggressive development schedule; further, the programs cited by Norden to support its experience were found to be behind schedule, with most of the delay attributed by customers to Norden.

Only two offerors received overall ratings of "acceptable," Raytheon Submarine Signal Division with 75 total points and IBM with 70 total points. Norden, with 58 total points, received the fourth high technical score. IBM's proposal represented the lowest price per technical point value of those received, $241,514 per point. Norden was third in price per technical point, at $289,730 per point.[1] On October 1, 1993, the agency awarded a contract to IBM as the offeror providing the best value, based on its "acceptable" rating and its low price per technical point score. This protest followed.

ISSUES CONSIDERED

With regard to the technical evaluation, Norden received an acceptable rating under the performance factor. Norden raises no specific objection to the evaluation of its management proposal. A proposal like Norden's, which received a weighted score of 19 points under the management factor, required a weighted score of 51 points under the performance factor to receive a rating of "acceptable" (70 points). To achieve a weighted score of 51 points, a proposal with a raw score in the "acceptable" range could not receive a risk rating other than "low," since even a raw score of 79 points (the highest available for an "acceptable" rating) does not produce a weighted score of 51 points if the risk is other than "low." Norden makes no showing that its proposal merited a rating in the "excellent" range under the performance factor, and our analysis therefore is concerned chiefly with the agency's finding that despite its overall acceptability, Norden's proposal presented substantial risks.

In reviewing protests against an agency's technical evaluation and decision to eliminate a proposal from consideration for award, we review the record to determine whether the agency's judgments were reasonable and in accordance with the listed evaluation criteria and whether there were any violations of procurement statutes or regulations. CTA, Inc., B-244475.2, Oct. 23, 1991, 91-2 CPD Para. 360. Here, we find that the evaluation was both reasonable and consistent with the solicitation and that the record supports the agency's assignment of a "high" risk rating to the performance aspects of Norden's proposal.

Initially, we note that Norden argues that the agency's decision to make award without discussions was unreasonable, given the complexity of the system being procured. We consider this argument untimely, in view of the express advice in the solicitation that the agency intended to make award without discussions. Our Office will examine the reasonableness of an agency's decision to make award without discussions, in light of the information before the source selection official at the time when he makes his decision. See The Jonathan Corp.; Metro Mach. Corp., B-251698.3; B-251698.4, May 17, 1993, 93-2 CPD Para. 174. Here, however, to the extent that Norden is arguing that the provisions for award without discussions were inappropriate for this type of solicitation, we consider the protest to concern an alleged impropriety that was apparent from the face of the solicitation and which should have been protested prior to the time set for receipt of initial proposals. See 4 C.F.R. Sec. 21.2(a)(1) (1993).

Our analysis will address those areas where Norden argues that the evaluation was unreasonable, or that the evaluation was flawed by the application of unstated criteria to the evaluation of proposals or latent ambiguities in the solicitation. In this regard, Norden argues that the agency unreasonably criticized its proposal for placing the bypass relay in a nonmilitarized processor unit, for its computation of tactical signal-to-noise (SNR) degradation, its limits on the OBT SNR, its lines of code analysis, the use of assembly code in its software development effort, and its level of repair analysis.

EVALUATION

Bypass Relay

Norden proposed that tactical data flow into a cabinet through input/output connectors, through the motherboard to the card bay interface connector to the bypass relay board, and back. Norden contends that in proposing to provide "ruggedized" equipment, it met the applicable specification, MIL-STD-2036, which defines "ruggedized" as referring to the modification of equipment to meet specification requirements. While Norden concedes that the tactical system specification for the AN/BQQ-5E sonar system requires all equipment in the tactical data path to withstand Grade A, class I shock as defined in MIL-S-901D, the protester contends that it could have affirmed during discussions its understanding that in some circumstances, the equipment would have to meet shock-testing standards of MIL-STD-901D. In any event, Norden asserts that its proposal specifically stated that the bypass relay would remain fully operational in accordance with that specification, and that the agency should have realized that Norden had committed itself to meeting the shock standards.

The Navy argues that by placing the bypass relay inside the cabinet, Norden essentially turned those parts of the processor unit that had contact with the tactical signal into external interfaces. The RFP system specification requires that the external interface between the OBT and tactical system meet the requirements of the tactical system, which in the case of the AN/BQQ-5E requires conformance to the shock standards of MIL-STD-901D. The specification further requires that the OBT interface not degrade the tactical system performance or the tactical data flow but be designed "such that a failure in the interface shall not degrade the tactical system performance or the tactical data flow."

If the system as described in the proposal is exposed to Grade A shock, the components may fail, degrading the tactical system performance and endangering the safety of the ship itself. In the view of the evaluators, by failing to propose a "militarized" (meeting grade A shock requirements) system--one in which all of the components were protected against shock--Norden's design presented a potential for failure of the tactical system, which stands in the data path. To the evaluators, Norden's failure properly to discuss the shock requirement presented a risk of system failure and danger to ship safety, as well as a failure to appreciate the critical nature of the spherical array external interface in insuring ship safety. With the tactical data interrupted, survival of the ship would be at issue.

We find the Navy's concern to be reasonable and consistent with the solicitation's emphasis on ship safety and its requirement that the interface meet the shock requirements of the tactical system. The proposal simply does not provide that the equipment in the tactical data flow--the connectors, the motherboard, and the card bay interface--will meet shock requirements. We do not find it unreasonable for the agency's concerns to continue despite Norden's general promise of compliance where the proposal as submitted presented the risk of failure.

Computations of Tactical SNR Degradation

SNR measures a sonar's ability to hear a particular signal, such as hostile ships, relative to background noise, such as the ambient sea noise. Norden essentially argues that any problems with its analysis of SNR degradation could and should have been cleared up in discussions. The RFP merely directed offerors to provide "any studies, computer analyses, prototypes, etc.," to demonstrate that a digital approach such as Norden used would meet performance specifications, but did not specify any particular type of analysis. Norden conducted a tonal analysis from 0 decibels (dB) to -20 dB that showed decreasing degradation as input SNR reached -20 dB; Norden contends that this analysis reasonably demonstrated SNR degradation at the maximum range of sonar detection and supported its claimed maximum SNR degradation loss of .15 dB. In connection with the protest, Norden has run a broadband analysis at -20 dB input, which is consistent with the claims in its proposal and which Norden could have provided if the Navy had discussed the issue with Norden. Norden also asserts that if the Navy wished a certain type of analysis or demanded a particular noise shape be used, it should have specified the analysis and noise shape in the solicitation.

The Navy argues that the solicitation requires that tactical system performance not be impacted at the maximum range of detection, that is, at the maximum distance at which the sonar is expected to recognize the presence of hostile ships. The evaluators found that Norden had not computed SNR at the maximum range of detection. Norden's design created additional broadband noise that degraded the SNR.[2] Norden's proposal, the evaluators found, did not consider the full range of OBT-generated noise, nor did it use a noise shape representative of the actual operating environment of the OBT (ambient sea noise). Sonar extracts tones from a contact out of the broadband noise generated by the sea as well as noise generated by the OBT, and the Navy considered that an analysis of SNR degradation should examine degradation to the contact broadband and tones relative to both the ambient noise and the OBT noise. Norden's failure to do so, in the opinion of the evaluators, indicated a lack of concern with the critical parameters of the OBT as well as a failure to comprehend submarine signal processing.

Norden indicated that tactical SNR degradation would be less than 0.15 dB, but the evaluators were concerned that Norden had failed to support this claim. The proposal contained two calculations, one showing a degradation of .20 dB and another showing a degradation of .25 dB. Norden not only failed to show how it derived the 0.15 dB figure, to the satisfaction of evaluators, but submitted computations in conflict with its claim. Responding to Norden's arguments that the Navy should have provided the correct noise shape, the agency asserts that the calculation should have been derived from available sources--adding the frequency response of the sonar system equalizer, +12 dB/octave (available to offerors from the technical information center) to the ambient sea noise -6 dB/octave (available in general reference works)--so that the noise shape representing the sea-state at the clipper input is +6 dB/octave. Norden used a figure of 0 dB/octave, which the evaluators viewed as unreliable in terms of addressing the in-band noise generated by the OBT and failing to recognize the frequency dependence of SNR degradation.

In view of the solicitation's advice that the agency intended to award a contract without discussions, the burden was on Norden to submit an proposal that adequately demonstrated its own merits. Vista Videocassette Servs., Inc., B-230699, July 15, 1988, 88-2 CPD Para. 55. The record supports the Navy's assertion that the correct noise shape, even if not specified in the solicitation, was readily available to offerors from other sources and that the use of an incorrect shape represented a fundamental flaw in the data. We are unable therefore to conclude that the Navy's concerns, that the proposal did not demonstrate a full understanding of the technical challenges of the requirement and presented a risk that the design could compromise the critical parameter of ship safety by degrading tactical system performance, were unreasonable.

Limits on the OBT SNR

Norden challenges the Navy's conclusion that limitations on the beamformer output SNR could affect the realism of strong contacts at close ranges. Norden asserts that the +18 dB limit on SNR is the inherent result of sparsing, which the Navy specifically endorsed, and that if the Navy had concerns over the reduction in training realism, it could have raised the issue in discussions. Further, Norden argues that the specification cited by the Navy refers to SNR at the bandpass filter output, which Norden believes to be located after the beamformer output, at which point the SNRs of the sonar signals would have been increased through filtering noise and integrating and averaging the signals. Norden therefore argues that the Navy is applying a higher and different requirement, since Norden's analysis was conducted at the beamformer output.

The TERP found that Norden's proposed spherical array approach limited the OBT SNR to +18 dB, which fails to meet system specifications and which will affect the realism of contacts at close range. The AN/BQQ-5E system specification requires a halfbeam SNR estimation that significantly exceeds Norden's proposed +18 dB; further, the information provided by Norden in its proposal does not even support this estimate. The Navy points out that Norden's own proposal recognized the problem, stating that introduction of a signal through a sparsed array "will result in a loss of fidelity which may impact on realism," and argues that Norden itself recognized that its design might not meet the specification requirements that broadband signal characteristics be acoustically realistic for each contact. Since the record shows that the evaluators' concerns merely reflected Norden's own statement that the design could compromise training realism, we have no basis to conclude that the agency's concerns over the risk to training realism was other than reasonable and consistent with the solicitation.

Software Development Effort

The evaluators expressed concern that Norden's proposal did not adequately describe the amount of new and reused software, that their software line of code (SLOC) analysis did not include software required for the display and control functions, and that Norden proposed to develop certain software in assembly language, in violation of the solicitation requirement for the use of high order language. Norden essentially argues that these concerns are de minimis because its proposal was to develop the displays with an interface that automatically generated the associated code; it therefore only provided the estimates in terms of numbers of displays rather than lines of code. Similarly, the use of assembly language represented a small portion of the overall effort.

The agency essentially agrees that its concerns were relatively minor and might not have rendered the proposal unacceptable had it had no other concerns with Norden's proposal. The evaluators considered the use of displays, overall, to be a strength; nevertheless, the solicitation required offerors to provide SLOC estimates and absent such estimates, the evaluators did not know how many lines of code were associated with the displays. Both criticisms, the agency argues, were reasonable and well supported, even if they might not provide an independent basis for rejection of the proposal. In view of the express solicitation requirements, we cannot conclude that the agency's criticisms were either unreasonable or inconsistent with the solicitation.

Level-of-Repair Analysis

The solicitation required that fault detection, fault localization and repair of modules, cards, and subassemblies be done at the organizational level, i.e., aboard the submarine at sea. In this respect, the lowest level of repair offered by Norden was the intermediate level; this conflicted with the solicitation. Further, Norden's calculation of operational availability appeared fundamentally flawed by this error. The 24-hour logistics delay contemplated by the solicitation was based on repair at the organizational level, with parts being immediately available; Norden used this 24-hour delay figure in its calculation of operational availability, although under its proposal, repair parts would not be available for a considerably longer period. In the agency's view, Norden's projection of 94.84 percent operational availability was patently unrealistic, given its proposal that repair be done at the intermediate rather than the organizational level.

Norden's only response is that its proposal indicated uncertainty whether certain portions of maintenance could be performed at the organizational level, and that the weakness principally involved an informational deficiency that could have been resolved during discussions. Since, as stated, Norden had no reasonable expectations that the agency would enter into discussions to clear up such uncertainties, we have no basis for finding the evaluators' conclusions either unreasonable or inconsistent with the solicitation.

Regardless of the above, Norden argues that it was unreasonable under the circumstances for the agency to make award without discussions because the awardee, IBM, was barely rated as acceptable under the performance factor and was considered to present high risk under that most important evaluation factor. The evaluation record shows that the agency carefully considered the rating of the IBM proposal, specifically, that IBM's spherical array signal summation at the clipper output significantly reduced concerns associated with the digital approach and appeared highly desirable but risky because of dynamic range limitations and IBM's failure to demonstrate conclusively that it could provide the required controlled summation over a full range of SNRs. Nevertheless, the agency designed the evaluation scheme with the acceptance of risky but desirable proposals in mind, and the selection of IBM, which also received the lowest price per point score ratio, was consistent with the evaluation scheme.

In any event, under our Bid Protest Regulations, a party is not an interested party to maintain a protest if it would not be in line for award if the protest were sustained. 4 C.F.R. Secs. 21.0(a) and 21.1(a). Since the agency properly found Norden's proposal unacceptable--and thus would not have included Norden in discussions even if it chose to hold them--Norden is not an interested party for purposes of challenging the award to IBM. Hughes Technical Servs. Co., B-245546.3, Feb. 12, 1992, 92-1 CPD Para. 179.

The protest is denied.

1. Raytheon, at $303,489 per point, was fourth, behind Norden and Scientific-Atlanta, Inc.

2. The analysis of OBT-generated noise concerns noise generated at the "clipper" output. Clipping, the process of taking linear signals and assigning them values of 1 or 0, uses less hardware and provides greater range to the sonar and is therefore desirable. Nevertheless, the process causes a loss in SNR sensitivity.