Matter of: Exploration Products File: B-279251.2; B-279251.3 Date: June 1, 1998 * Redacted Decision

Matter of: Exploration Products File: B-279251.2; B-279251.3 Date: June 1, 1998 * Redacted Decision

DIGEST

Attorneys

DECISION

Exploration Products protests the award of a contract to Alaska Industrial Resources, Inc. under request for proposals (RFP) No. F08626-97-R-0107, issued by the Department of Air Force for a quantity of small shelter/environmental control units (SS/ECU). Exploration argues that its SS/ECU was not properly evaluated and that the Air Force should have communicated with the firm concerning certain failures of its equipment during government testing.

We deny the protest because Exploration's SS/ECU was unacceptable and Exploration could not be awarded the contract due to its failure to meet certain mandatory specifications.

The RFP stated that the objective of the contract is to procure a commercially-available, nondevelopmental shelter and ECU to replace the agency's current shelters for housing and work areas and that the primary focus is to acquire a system that is less of a burden to transport than the current SS/ECU. The RFP included a "System Performance Specification" which described the performance requirements and physical characteristics of the new SS/ECU. That document stated, at section 1.1, that "[t]he primary objective of the specification is to outline the performance and physical characteristic requirements for a new system that reduces the transportability (weight and volume) and cost burdens of the existing . . . shelter system."

The acquisition was structured to permit the Air Force to make multiple awards for a limited number of units for testing and then to downselect to a single contractor to supply the units. The RFP stated that the government anticipated awarding up to three contracts, although it reserved the right to award fewer. The first four line items in the RFP were for four complete SS/ECUs (line item 0001), test samples of materials used in the shelters (0002), training for the SS/ECUs (0003), and maintenance and repair during testing (0004). All remaining line items were options to be exercised after the downselect decision.

The RFP contemplated award to the offeror whose proposal was most advantageous to the government based on an assessment of the following four areas, listed in descending order of importance:

1. technical capability 2. logistics and readiness 3. cost/price 4. general considerations

Under the technical capability area, the RFP listed the following three factors in descending order of importance: (1) key performance parameters, (2) manufacturing capability, and (3) system integration, technical tradeoffs, and other performance parameters. The RFP also indicated that proposals would be rated for proposal risk and performance risk under each of the factors and that past performance would be evaluated and assessed in each area at the factor level.

Concerning the downselect decision, the RFP stated:

Once the proposal review is completed, the Government intends to subject the contract award candidates to comprehensive tests and technical field evaluations within the parameters outlined in the System Performance Specification . . . . Knowledge gained from results of this evaluation/testing will be used by the source selection team to update and verify the initial evaluation of the 3 candidates/contractors. These updated evaluations will serve as the down select criteria for determining which of the three candidates/contractors proposed systems provides the "Best Value" to the Government.

The Air Force received nine technical proposals. After evaluation, the agency conducted discussions and then determined that four proposals were acceptable and requested price proposals from those firms. The proposals of Exploration and Alaska received ratings of acceptable or better on all of the technical capability and logistics and readiness evaluation factors, the only factors that were rated. /1/ At that point, the Air Force awarded contracts to Exploration and Alaska.

Pursuant to the first two line items, Exploration and Alaska provided shelter systems and material samples to be tested at the Army's Aberdeen Proving Test Center and Holloman Air Force Base. Pursuant to line item 0003, the firms also trained government personnel to assemble their SS/ECUs. Based on the tests, the original evaluation ratings were updated. The ratings assigned to Alaska and Exploration were all "acceptable" at this point with the exception of Exploration's ratings on the key performance parameters factor and the system integration, technical tradeoffs, and other performance parameters factor, both of which were in the technical capability area. Exploration's ratings on those factors were reduced to "unacceptable." /2/

A Downselect Proposal Analysis Report dated February 4, 1998 explains the two unacceptable ratings assigned to Exploration's proposal. Under the key performance parameters factor, the report lists a series of weaknesses. According to the report, although the RFP requirement was for an ECU with a volume of 48 cubic feet or less, the volume of Exploration's ECU was 62 cubic feet. The report also states that Exploration's ECU would not start at 125 degrees, as required by the specifications. Also under that factor, the report states that the fabric of Exploration's shelter ripped at attaching points of straps, numerous zippers broke, and the floor was easily punctured. Under the system integration, technical tradeoffs, and other performance parameters factor, the report states that Exploration's shelter failed a [deleted] test for [deleted], its anchoring system was only adequate for sidewalls, [deleted], and the shelter failed a [deleted] test.

In a "Source Selection Down Select Decision Document," the Air Force's source selection authority repeats the weaknesses noted in Exploration's SS/ECU in the Downselect Proposal Analysis Report and states that those weaknesses "made their proposal unacceptable." According to the report, there were no differences between the two proposals in the logistics and readiness area or in terms of past performance. The report notes that the prices submitted by the two firms ($ [deleted] for Exploration, and $25,147,464 for Alaska) had been evaluated for reasonableness and that Alaska's price was slightly above the government estimate while Exploration's price was below the government estimate.

In spite of the fact that Exploration's proposal was technically unacceptable, the Air Force's source selection authority performed a tradeoff analysis in which he determined that Alaska's proposed system offered the best value to satisfy the agency's needs. The source selection document states that the testing clearly demonstrated that the technical advantages of Alaska's SS/ECU far outweighed the differences in price between the two proposals. According to the source selection authority, given Alaska's combination of demonstrated technical capabilities, and low risk, it was in the best interests of the government to award the contract to Alaska.

PROTEST ALLEGATIONS

In its protest, Exploration first challenged the determination that its SS/ECU failed to meet the requirement for an ECU with a volume at least 25 percent less than the ECU it is to replace. According to Exploration, based on the RFP, it understood the requirement for a 25 percent reduction in volume and weight to apply to the complete system, not to each of the system components. Exploration also argues that, since its entire SS/ECU weighed 25 percent less and had 25 percent less volume than the current SS/ECU, it should have received at least an acceptable rating under the key performance parameters factor.

Exploration's initial protest also argued that its SS/ECU was not tested in a manner consistent with the RFP. Although Exploration's protest generally referred to "inconsistencies between the . . . testing and evaluation and the solicitation's stated criteria," the protest offered only one example of an alleged error in the testing. In that single example, Exploration argued that the method used to test the [deleted] capability of its shelter placed [deleted] on the shelter. According to Exploration, if tested properly, its shelter would have passed. Exploration also argued in its protest that the problems with the fabric, zippers and floor of its shelter were graded under the wrong factor.

Exploration also argued in its protest that agency personnel should have contacted the firm during the testing of its SS/ECU when they encountered "obvious discrepancies which logically should have been checked with Exploration . . . ." The only example of an "obvious discrepanc[y]" referenced by Exploration in its protest was the existence of stake loops at the base of the walls of the firm's shelter. The firm argues that the "obvious existence of those loops, and the problem that was experienced with the SS/ECU system during the wind test," should have demonstrated to the agency that it was necessary to contact Exploration to clarify whether those stake loops should be used. According to Exploration, had the agency done so, the problem could have been identified and the stakes could have been appropriately placed. In a related argument, Exploration also argued that the agency unreasonably failed to request technical support from Exploration pursuant to line item 0004, under which the government could call for maintenance and repair during the testing.

Finally, Exploration argues that the Air Force's best value analysis was unreasonable and flawed. According to Exploration, in light of the failure to evaluate Exploration's SS/ECU properly and particularly considering the cost saving that would result from an award to Exploration, the decision to award to Alaska was flawed.

ANALYSIS

Technical Evaluation

As explained, during the testing phase of the acquisition, Exploration's SS/ECU was found not to meet four mandatory requirements of the specifications and, as a result, Exploration's SS/ECU was unacceptable. First, the evaluators concluded that the specifications required a volume of 48 cubic feet or less for the ECU and, as explained above, Exploration's ECU was 62 cubic feet. Second, Exploration's ECU would not [deleted], as required by the specifications. Third, Exploration's shelter failed to pass the [deleted] test of [deleted]. Fourth, Exploration's shelter failed the [deleted] test.

We regard benchmarks, or, by analogy, demonstration tests of the type required here, as extensions of the technical evaluation of proposals, the principal purpose of which is to provide a demonstration of the capability of offered products to perform the required functions. Rand McNally-TDM, Inc., B-248927, Oct. 7, 1992, 92-2 CPD Para. 352 at 5. Consistent with this view, we have been critical of strict pass/fail benchmarks, which lead to the automatic exclusion of otherwise potentially acceptable offers, and have held instead that such tests provide "strong evidence" of system capabilities which must be considered in determining technical acceptability. See NBI, Inc., B-201853.3, Aug. 9, 1982, 82-2 CPD Para. 114 at 6. As far as the agency's actual determination of technical acceptability is concerned, we will not make an independent determination of the merits of an offeror's proposal, or in the case of a demonstration, the performance of the offeror's product; rather, we will review the evaluation record, including the results of any test demonstration, to ensure that the agency's technical judgment is based upon the requisite "strong evidence," has a rational basis, and is consistent with the stated evaluation criteria. Rand McNally-TDM, Inc., supra, at 5.

Here, we find that the record supports the agency's determination that Exploration's ECU exceeded the volume requirement set forth in the specifications and failed to operate at [deleted], also a requirement of the specifications. In addition, we conclude that Exploration has not timely challenged the [deleted] test of its shelter and the resulting determination that its shelter does not meet the [deleted] requirements of the RFP.

Addressing first the volume requirement, the RFP specification, at section 3.2.1.1, states that "[t]he new SS/ECU System components shall each reduce transport weight and packed volume by at least 25 percent over the current system components." Since the specifications state at section 3.1 that the currently used ECU has a packed volume of approximately 64 cubic feet, the maximum permitted volume for the new ECU is 48 cubic feet. Exploration's ECU, with a volume of 62 cubic feet, did not meet this requirement.

Exploration does not deny that the ECUs which it provided for testing exceeded 48 cubic feet. Rather, Exploration argues that it understood the requirement for a 25 percent reduction in volume and weight to apply to the complete SS/ECU, not to each of the system components. According to Exploration, this understanding was based on the RFP evaluation criteria--specifically, the first factor in the technical capability evaluation area, which reads in relevant part as follows:

The offeror's proposal must demonstrate a complete understanding of the [SS/ECU] System Performance Specification requirements. The offeror's proposed design will be evaluated to determine the suitability of the approach selected. The proposal will be evaluated to determine if the proposed system will weigh at least 25% less and has at least a 25% lower packed volume than the baseline system described in the Performance Specification.

According to Exploration, based on this provision, as well as the fact that the specifications repeatedly used the term "system," its understanding was that the requirement was for a 25 percent reduction in weight and volume of the entire SS/ECU, not each of the components of the SS/ECU. Exploration argues that its SS/ECU met this requirement.

Where a dispute exists as to the meaning of solicitation language, we will resolve the matter by reading the solicitation as a whole and in a manner that gives effect to all provisions of the solicitation. See Lithos Restoration, Ltd., B-247003.2, Apr. 22, 1992, 92-1 CPD Para. 379 at 4. To be reasonable, an interpretation must be consistent with the solicitation when read as a whole and in a reasonable manner. Id.

Here, the only reasonable reading of the RFP was that the weight and volume reduction requirements applied to both components of the SS/ECU, the shelter itself and the ECU, and not just to the SS/ECU as a whole. Although the evaluation factor relied upon by Exploration refers to the "system" weighing at least 25 percent less and having a 25 percent lower packed volume, that factor also required that "[t]he offeror's proposal must demonstrate a complete understanding of the [SS/ECU] System Performance Specification requirements"-- the specifications, which include the requirement that the "new SS/ECU System components shall each reduce transport weight and packed volume by at least 25 percent over the current system components." We think the use of the term "system" in the evaluation factor must be read in the context of its reference to the specifications which required that each system component be reduced by 25 percent in volume. Read in the context of the solicitation as a whole, and giving meaning to each provision, we conclude that offerors were reasonably on notice that the ECU was required to have a volume 25 percent less than that of the existing ECU. /3/ Under the circumstances, Exploration's proposed ECU was unacceptable due to its failure to meet the volume reduction requirement. /4/

Turning to the ECU [deleted] requirement, the System Performance Specification requires the ECU to operate at [deleted]. The Downselect Proposal Analysis Report indicates that Exploration's "ECU would not [deleted] as required-Cut off Design on ECU." In addition, the source selection memorandum states that "the ECU would not [deleted] requirement due to a built in cut off switch . . . ."

Exploration essentially does not challenge the determination that its ECU did not meet this requirement. In its discussion of the failure of its ECU to meet the requirement to operate [deleted], Exploration explains that when agency officials raised the matter with the firm during testing, the firm's representatives responded that the ECU would not start at that [deleted] because it has "a cut off safety design feature [that] kept the ECU from running over a certain [deleted]," and that they "informed the [agency] as to this feature and provided instruction as to how to adjust the feature." In a later reference to this issue, Exploration states that "the ECU would not start due to a cut off switch design feature, which was resolved after speaking to Exploration . . . ."

The record does not support Exploration's assertion that the ECU's failure to operate at [deleted] was "resolved" or that the cut off feature was "adjust[ed]." In fact, the contracting officer explains that Exploration's ECU would not start at [deleted] because the safety switch would not allow it to start and it could not be adjusted. The contracting officer reports that the ECU never started at [deleted], as required by the specifications; only after the [deleted] of the test chamber was lowered to [deleted] did the ECU start. Under the circumstances, Exploration's SS/ECU was unacceptable due to its failure to meet the requirement that its ECU operate at [deleted].

As explained above, Exploration's SS/ECU also was considered unacceptable because it failed the [deleted] test. The specifications required that shelters be able to withstand [deleted]. Concerning the [deleted] test, the test report states: "Requirement not met. Exploration shelter failed to withstand [deleted]."

In its comments on the agency report, for the first time, Exploration challenged the [deleted] test and the agency's conclusion that the Exploration shelter did not meet the [deleted] requirements. This contention is untimely. On March 17, the Air Force released to Exploration's counsel a video tape of the [deleted] test of its shelter, along with other relevant documents. However, Exploration did not challenge the conduct of the [deleted] test until it filed its comments on the contracting agency report on April 9. In that submission, for the first time, Exploration argued that the Air Force "did not conduct the [deleted] test consistent with stated criteria," and that the shelter was not properly assembled by the agency. Exploration's challenge of the methodology used in the [deleted] test and of the agency's conclusion that its shelter did not meet the [deleted] test requirements of the specifications is untimely. Under our Bid Protest Regulations, protests not based upon alleged solicitation improprieties must be filed not later than 10 days after the basis for protest is known. 4 C.F.R. Sec. 21.2(a)(2). Where a protester initially files a timely protest and supplements it with new and independent grounds of protest, the new allegations must independently satisfy these timeliness requirements; our Regulations do not contemplate the unwarranted piecemeal presentation of protest issues. Litton Sys., Inc., Amecom Div., B-275807.2, Apr. 16, 1997, 97-1 CPD Para. 170 at 4, n.1. Here, while Exploration's initial protest was filed in a timely manner, that protest did not challenge the [deleted] test or the conclusion that Exploration's shelter does not meet the [deleted] test requirements. /5/ Exploration's objection to the methodology used in the [deleted] test and to the agency's conclusion that its shelter does not meet the [deleted] test requirements were not raised until April 9 when its comments on the agency report were submitted to our Office--which was more than 10 days after the firm learned of the grounds for those objections when it came into possession of the video tape of the [deleted] test on March 17. See Watkins-Johnson Co., B-252790, July 7, 1993, 93-2 CPD Para. 8 at 3-4. Accordingly, we will not consider Exploration's challenge of the [deleted] test or the conclusion that its shelter did not meet the [deleted] test requirements.

Discussions

Exploration also argues that the Air Force failed to conduct meaningful discussions with the firm. According to Exploration, during the testing of its SS/ECU, government personnel called the firm three times concerning the failure of the firm's ECU to [deleted]. Under the circumstances, Exploration argues that the agency should have contacted the firm when it encountered "obvious discrepancies which logically should have been checked with Exploration . . . ." /6/ Protest at 10. The only example of an "obvious discrepanc[y]" which Exploration included in its protest was the existence of stake loops at the base of the walls of the firm's shelter. Although Exploration concedes that it omitted from its manual the direction to insert stakes through these loops, the firm argues that "the obvious existence" of the loops, "and the problem that was experienced with the SS/ECU system during the [deleted] test," Protest at 11, should have demonstrated to the agency that it was necessary to contact Exploration to clarify whether those loops should be used. According to Exploration, had the agency done so, the problem could have been identified and the stakes appropriately placed.

Although we have criticized the strict application of pass/fail test criteria that lead to the automatic and final exclusion of a potentially acceptable proposal, this generally involved situations where the offeror has been unable to demonstrate compliance with only one of a number of mandatory requirements and is eliminated from the competition solely for that reason. Checkpoint Sys., Inc., B-245834, Feb. 3, 1992, 92-1 CPD Para. 136 at 3. Here, as explained above, Exploration's SS/ECU was found not to meet at least three mandatory requirements during the testing: (1) the shelter, with a volume of 62 cubic feet, exceeded the required maximum volume of 48 cubic feet; (2) the ECU would not [deleted], as required; and (3) the shelter failed to pass the [deleted] test.

Moreover, with respect to the specific issue which Exploration argues should have been discussed with the firm--the stake loops--the record does not demonstrate that this problem was so obvious that the agency should have known to raise it with the firm. The contracting officer explains that the instructions provided with Exploration's shelter included no information concerning these stake loops and Exploration's personnel trained Army and Air Force personnel for 2 days on how to erect the Exploration SS/ECU. The contracting officer further explains:

This training consisted of Exploration's training team setting up and tearing down their shelter system and instructing U.S. Army and [Air Force] personnel on the proper procedures for doing the same. The Exploration shelter was erected and disassembled numerous times during these two days of training. Not once during this period did the experts . . . identify that these items were missing, therefore it stands to reason that the US Army personnel would not recognize this problem with the Exploration shelter when they (the US Army) were trained by [Exploration's] experts. Finally, there were no left over guy wires or stakes when the shelter system was erected, and so it is inconceivable that the [Air Force], having two days of training, should have recognized a problem with the [Exploration] shelter. Upon receipt of the protest, the Government checked [Exploration's] shelters at both Aberdeen and Holloman AFB for stake loops. Upon visual review there were no stake loops located on the endwalls. There are four small fabric grommets that are too small to place existing stakes through.

The Air Force also explains that there was no indication that the grommets served a purpose. Under the circumstances, since Exploration's own personnel failed to instruct agency personnel to use those stake loops in the 2 days of training and apparently failed to even notice that the stake loops were not being used during the numerous times the shelter was erected, we cannot agree with Exploration that this problem was an "obvious discrepanc[y]." On the contrary, since Exploration's own personnel did not recognize that parts and instructions were missing and the shelter was not properly erected, we do not think it is reasonable to hold agency personnel responsible for recognizing these problems. /7/

Other Issues

As explained above, Exploration also argues that the [deleted] test of its shelter was not performed in a manner consistent with the RFP, that problems with the fabric, zippers, and floor of its shelter were graded under the wrong evaluation factor, and that the Air Force's best value analysis was unreasonable. We will not consider these issues since, under our Bid Protest Regulations, a party is not interested to maintain a protest if it would not be in line for award if the protest were sustained. 4 C.F.R. Sec. 21.0(a), 21.1(a); System Resources Corp., B-270241, et al. Feb. 12, 1996, 96-1 CPD Para. 69 at 5. As indicated above, based on the testing results, the source selection authority concluded that Exploration's proposal was unacceptable and we consider that conclusion to be reasonable. Alaska is the sole acceptable offeror, and since Exploration's proposal was unacceptable, it could not form the basis for an award. Learjet, Inc., B-274385 et al., Dec. 6, 1996, 96-2 CPD Para. 215 at 6. In addition, Exploration has not challenged the acceptability of Alaska's proposal. Thus, even if we were to sustain Exploration's remaining protest grounds, Alaska's SS/ECU would remain technically acceptable, and Exploration's SS/ECU would be unacceptable. See also Intertec Aviation, B-239672.4, Apr. 4, 1991, 91-1 CPD Para. 348 at 7 (contracting agency is not required to consider lower cost proposals in its award decision where those proposals have been judged technically unacceptable). Accordingly, Exploration would not be in line for award, and we will not consider these additional issues.

The protest is denied.

Comptroller General of the United States

1. Ratings of exceptional, acceptable, marginal, and unacceptable were assigned.

2. An unacceptable rating was defined as: "Fails to meet a minimum requirement of the RFP and the deficiency is uncorrectable without a major revision to the proposal."

3. Exploration also argues that the RFP was ambiguous since the specifications called for a reduction in volume and weight of the components while the evaluation criteria referred to the "proposed system" meeting those requirements. Even if we were to agree with Exploration that there was an ambiguity in the RFP, since Exploration's contention of an ambiguity is based on an alleged conflict between two provisions of the RFP, such an ambiguity could not be latent, i.e., a defect which could not be detected prior to closing. See Inland Marine Indus., Inc., B-249914, B-249918, Dec. 24, 1992, 92-2 CPD Para. 442 at 3. In other words, any ambiguity concerning these requirements was apparent on the face of the solicitation and thus had to be protested prior to closing. Bid Protest Regulations, 4 C.F.R. Sec. 21.2(a)(1) (1998).

4. Exploration also argues that it [deleted]. According to Exploration, it was only due to this improvement, which it made at the Air Force's suggestion, that its ECU did not meet the volume requirement. Air Force officials, on the other hand, deny that they suggested to Exploration that the firm [deleted]. We need not resolve this factual disagreement. Even accepting Exploration's version of events, the Air Force never waived the volume reduction requirement and it was Exploration's responsibility to provide an ECU that met that requirement.

5. Prior to its protest, Exploration was informed that its "Shelter did not pass [deleted] requirements," and was given an explanation of how the shelter failed the test. Nonetheless, Exploration's protest did not challenge either that conclusion or the explanation of the test. The only discussion of the [deleted] test in Exploration's protest was under the heading "The USAF Failed to Conduct Meaningful Discussions." Under that heading, Exploration argued that during the testing, agency personnel should have noticed that there are stake loops at the base of the walls of its shelter. In addition, Exploration argued that due to the problems experienced during the [deleted] test, the agency should have known to raise this matter with Exploration in discussions. In other words, the only allegation in Exploration's protest concerning the [deleted] test was a contention that the matter should have been the subject of discussions; we do not read the protest as having challenged the conduct of the [deleted] test or the resulting conclusion that the shelter did not meet the [deleted] requirement.

6. To the extent that Exploration argues that the communications with it concerning the [deleted] range of its ECU constituted discussions--leading to a duty to conduct discussions on other issues--we do not agree. Those communications, which did not involve information essential for determining the acceptability of the firm's proposal and offered no opportunity to revise the proposal, did not constitute discussions. Federal Acquisition Regulation Sec. 15.601 (June 1997). Therefore, the downselect award decision was made without discussions, consistent with the RFP, which indicated that the agency intended to make award without discussions. Exploration refers to no authority, and we are aware of none, which would require discussions in these circumstances.

7. Exploration also argues that it was unreasonable for the agency to fail to utilize Exploration's technical support as permitted by line item 0004, under which the government could call for maintenance and repair during the testing. We do not agree. As the Air Force points out, the RFP stated that the purpose of line item 0004 was "to provide service and parts necessary to resolve major or catastrophic failures requiring other than minor repair of the system or components." RFP, attachment 3, at 2. According to the Air Force, the purpose of line item 0004 was to allow contractors to repair their SS/ECUs if a catastrophic failure occurred during testing--so that the test could continue--rather than to provide offerors with opportunities to repair their SS/ECU every time it failed. We think that is a reasonable reading of this provision.

* DOCUMENT FOR PUBLIC RELEASE

The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.