Miltope

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Miltope

File: B-416859.2; B-416859.3

Date: January 8, 2019

Howard J. Stanislawski, Esq., and Patrick K. O'Keefe, Esq., Sidley Austin LLP, for the protester.
W. Jay DeVecchio, Esq., Kevin P. Mullen, Esq., Rachael K. Plymale, Esq., and Caitlin A. Crujido, Esq., Morrison & Foerster LLP, for Leonardo DRS, the intervenor.
Jonathan A. Hardage, Esq., and Jered J. Leo, Esq., Department of the Army, for the agency.
Charmaine A. Stevenson, Esq., and Laura Eyester, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

1. An agency's assertion that the protester's challenge to its exclusion from the competitive range is untimely is unavailing where the protester's contention is that the agency failed to test the protester's sample in accordance with the solicitation, and the record shows that the protester could not have known its basis for protest, despite its receipt of a pre-award debriefing, prior to return of the sample to the protester, which did not occur until after contract award.

2. Protester's contention that the agency improperly changed the display resolution setting prior to testing the protester's sample is denied where the record does not establish that the agency changed the setting.

DECISION

Miltope, of Hope Hull, Alabama, protests its exclusion from the competitive range and subsequent award of a contract to Leonardo DRS, of Arlington, Virginia, under request for proposals (RFP) No. W15QKN-18-R-0037, issued by the Department of the Army, for multipurpose standard automatic test equipment. The protester contends that the agency altered the settings on its sample prior to testing, contrary to the solicitation's evaluation criteria, resulting in its lower test score and elimination from the competition. The protester also argues the agency's testing was inaccurate and unreliable.

We deny the protest.

BACKGROUND

The agency issued the RFP on March 23, 2018, using Federal Acquisition Regulation (FAR) subpart 15.3 procedures, for the award of a fixed-price, indefinite-delivery, indefinite-quantity contract with five 12-month ordering periods. Agency Report (AR), Tab 4, RFP, at 2.¹ The RFP stated that the contract would include a minimum guarantee for 40 first article test units and a contract ceiling of $111,277,000. Id. The test equipment devices will be used throughout all levels of maintenance as the Army's standard general purpose at-platform automatic test system to test and diagnose highly complex communications, other electronic commodity equipment, missiles, aircraft, and ground vehicles to identify failed line replaceable units. AR, Tab 5, Detail Specification,

at 2. The devices will also host interactive electronic technical manuals and/or specific application software and be used to upload/download mission data or software. Id.

Award was to be made to the offeror whose proposal offered the best value to the government utilizing a tradeoff source selection methodology. RFP at 2, 99. The RFP stated that proposals would be evaluated under the following factors, listed in descending order of importance: technical, price, past performance, and small business participation. Id. at 99. The technical factor included two subfactors, in descending order of importance: performance and display. Id. at 100. When combined, the non-price factors were significantly more important than price. Id. at 99. The technical factor and subfactors were to be assigned the following ratings: outstanding; good; acceptable; marginal; or unacceptable. Id. at 100-101.

Along with a written technical proposal, the RFP required offerors to submit a sample with standard accessories.² RFP at 94. The RFP further stated: "The bid sample shall be configured as specified in the [test equipment device] Detail Specification with Windows 10 Enterprise and all drivers necessary to operate, load, and execute PassMark[®] Software."³ Id. As relevant here, the required display resolution setting for the sample was "a native resolution of 1024 pixels by 768 pixels threshold (objective is 1920 pixels by 1080 pixels) or better." AR, Tab 5, Detail Specification, at 9. A native resolution of 1,024 pixels by 768 pixels is equivalent to a display resolution of 100 percent. Protest at 3.

The agency's evaluation under the performance subfactor was based on the offeror's sample meeting or exceeding the threshold benchmark performance score as specified in the detail specifications, utilizing the PassMark® performance test software. RFP at 101. Specifically, the detail specification stated: "Utilizing PassMark[®] Software Pty Ltd V8.0 Benchmark software, the [test equipment device] shall achieve or exceed an overall benchmark of 1400 threshold (objective is 2000)." AR, Tab 5, Detail Specification, at 27.

The agency received six proposals by the due date. Contracting Officer's Statement and Memorandum of Law (COS/MOL) at 4. The technical evaluation team (TET) performed all testing of the samples; the TET arrived at a benchmark score for each sample by running the PassMark® software to test the performance of each sample 10 times, and computing the average score. AR, Tab 14, Decl. of TET, at 1. When tested by the TET, Miltope's sample achieved a benchmark score of 1,953.7. Id. at 2. All other offerors' samples achieved a benchmark score that exceeded the objective requirement of 2,000. AR, Tab 7, Competitive Range Determination, at 3-6. In its final technical evaluation, Miltope was rated acceptable under the performance factor and good under the display factor, and received an overall rating of acceptable under the technical factor. AR, Tab 7, Competitive Range Determination, at 3. Miltope was the lowest-rated offeror under the technical factor, and Miltope and two other offerors were excluded from the competitive range. Id. at 11.

On August 2, the agency advised Miltope that its proposal was excluded from the competitive range and would not be considered for award. AR, Tab 11a, Miltope Unsuccessful Offeror Letter. On August 10, Miltope was provided a telephonic debriefing during which Miltope was advised that its proposal was not among the most highly rated proposals even though it was among the most competitively priced proposals. COS/MOL at 11-12; see AR, Tab 12, Debriefing Slides, at 6. After the debriefing and prior to award, Miltope several times requested that the agency return its sample. Response to Request for Dismissal, Oct. 16, 2018, at 4. On September 7, the Army awarded the contract to Leonardo DRS. COS/MOL at 13.

On September 13, Miltope received its sample back from the Army and performed a forensic investigation and testing to determine why its benchmark score had not exceeded 2,000. On September 21, Miltope concluded that agency personnel must have changed the display resolution setting on its sample from 100 to 150 percent prior to testing its sample. See AR, Tab 13, Letter from Miltope to Army, Sept. 24, 2018,

at 2-3. On October 1, Miltope filed this protest.

DISCUSSION

Miltope argues that the evaluation was unreasonable because the agency tested Miltope's sample using a display resolution setting of 150 percent, rather than 100 percent (the display resolution required by the solicitation). In Miltope's view, testing the sample at a resolution setting above 100 percent unfairly caused the performance of its sample to score below the objective of 2,000. Protest at 4. Miltope also argues that the agency's testing was inaccurate and unreliable because the test logs produced by the agency indicate an incorrect operating system may have been used. Comments and Supp. Protest at 11-15. As discussed below, we find no basis to sustain the protest.

Timeliness

As a threshold matter, the agency argues that Miltope's protest should be dismissed because Miltope knew the basis of its protest as a result of its pre-award debriefing on August 10, namely, that Miltope had been excluded from the competitive range because its sample had not achieved the objective score of 2,000 under the performance subfactor. COS/MOL at 16-19; Army Request for Dismissal, Oct. 12, 2018, at 7-8. Alternatively, the agency contends that because the RFP did not obligate the government to return the sample prior to award and did not warrant the condition of the samples upon return, it is unreasonable to allow the protest to proceed because the timeframe required by Miltope to discover its specific basis for protest was unknown and Miltope did not need to determine a specific basis of protest to know that it disagreed with the agency's test scores. COS/MOL at 20-22; Army Request for Dismissal,

Oct. 12, 2018, at 9-10.

The protester argues that its protest is timely because the display resolution setting must have been changed from 100 to 150 percent after Miltope sent the sample and prior to testing by the Army. Miltope explains that it only discovered on September 21 that its display resolution was set at 150 percent, rather than 100 percent, after performing a forensic investigation and tests on its returned sample. Comments and Supp. Protest at 2-6; Response to Request for Dismissal, Oct. 16, 2018.

Our Bid Protest Regulations contain strict rules for the timely submission of protests. Under these rules, a protest based on other than alleged improprieties in a solicitation must be filed no later than 10 days after the protester knew, or should have known, of the basis of protest, whichever is earlier. 4 C.F.R. § 21.2(a)(2). Our timeliness rules reflect the dual requirements of giving parties a fair opportunity to present their cases and resolving protests expeditiously without unduly disrupting or delaying the procurement process. United Valve Co., B-416277, B-416277.2, July 27, 2018, 2018 CPD ¶ 268 at 5-6.

This protest is not a broad challenge to the protester's elimination from the competitive range, but instead, is a specific allegation of either mistake or misconduct on the part of the agency when it tested Miltope's sample. Here, the RFP stated:

Each Offeror[']s bid sample unit will be retained by the Government, without charge, until contract award. Unless otherwise requested, the bid sample unit and [operations and maintenance] manual will be returned to the unsuccessful Offeror(s) at their own expense after contract award. The Government will not be held responsible for the condition of the bid sample returned to any Offeror.

RFP at 94. As noted, Miltope was advised in its pre-award debriefing held on August 10 that it was eliminated from the competition because its sample did not achieve the objective benchmark score of 2,000 when tested by the Army. However, Miltope apparently did not presume at that time that the agency may have tested its sample at the higher display resolution of 150 percent. Instead, Miltope simply requested that its sample be returned. Upon return of its sample, Miltope promptly performed a forensic investigation and testing, and concluded within eight days that the display resolution setting on its sample must have been set at 150 percent when tested, in violation of the RFP. Miltope explains that the record shows that it sought diligently to determine the cause for its sample's failure to meet the objective score, and did so within eight days of receiving its sample back. In circumstances such as here, we will resolve doubts over issues of timeliness in favor of protesters. See Sigmatech, Inc., B-296401, Aug. 10, 2005, 2005 CPD ¶ 156 at 5. On this record, we find the protest is timely.⁴

Sample Testing

Miltope argues that the agency altered the display resolution setting on Miltope's sample prior to testing its performance. In pertinent part, the protest states:

[T]he specification, referenced in the RFP, required a native screen resolution of 1024 pixels by 768 pixels, equivalent to 100%. At no time did the Agency advise Miltope or any of the other offerors that it would change the display resolution scale before conducting the benchmark tests. . . . In this case, the Solicitation required that bid samples be subject to a specified benchmark test, and that the screen resolution be set to 100%. In fact, however, the Agency tested the Miltope sample using a screen resolution of 150%, which directly led to a lower benchmark score and Miltope's elimination from the competitive range. As a result, the Agency failed to follow the evaluation criteria of the Solicitation, and in fact used an undisclosed evaluation factor. The Agency's actions were thus contrary to the FAR and unreasonable. As a direct result of the Agency's improper evaluation of Miltope's sample, Miltope received a rating of only Acceptable under the Performance subfactor, and was eliminated from the competition. If the Agency had evaluated Miltope's bid sample properly and in accordance with the evaluation criteria, without using any undisclosed criteria, Miltope would have achieved benchmark scores well above 2000, would have received a rating higher than Acceptable under the Performance subfactor, and would not have been eliminated from the competitive range.

Protest at 4. The agency categorically denies the allegation. COS/MOL at 22-25.

Here, as noted, the RFP required that samples be submitted in accordance with the configuration provided in the detail specification, with a Windows 10 operating system to enable the agency to operate, load, and execute the PassMark® Software. RFP at 94; see also RFP at 18 ("Answer 7: All bid samples need to be loaded with Windows 10 Enterprise."). Both the detail specification and the performance test plan, included as attachments to the RFP, specified that version 8.0 of the PassMark® software would be used to test the performance of the samples. AR, Tab 5, Detail Specification, at 27;

Tab 6, Performance Test Plan, at 2.

In response to the protest, the TET provided a detailed recitation of the procedures followed to test the performance of all offerors' samples. In pertinent part, the TET stated the following:

• Prior to testing the samples, the agency downloaded the PassMark® version 8.0 software from the PassMark website onto a compact disc, which was used to test the performance of all offerors' samples, in accordance with the RFP.

• The TET confirmed that all samples were running on a Windows 10 operating system prior to performing the testing.

• Any PassMark® software installed on the offerors' samples was uninstalled, and the same compact disc on which the agency downloaded the PassMark® version 8.0 software was used to install the agency's copy of the PassMark® software onto all offerors' samples to run the performance tests.

• No other settings on the samples were checked or changed, and specifically, the display resolution setting was never changed on any offeror's sample, including Miltope's sample.

AR, Tab 14, Decl. of TET, at 1-2; Tab 18, Supp. Decl. of TET and Contracting Officer. The contracting officer further stated that, aside from the TET, the only other individuals that interacted in any way with the samples were the contracting officer and two contract specialists. AR, Tab 18, Supp. Decl. of TET and Contracting Officer. The contracting officer further stated: "At no time did I, or any other individual, turn on the bid samples -- only the Government evaluators turned on/off the bid samples. This includes after bid sample testing was complete." Id.

Miltope provides a detailed explanation to support its contention that the display resolution setting on its sample was correctly set at 100 percent when it was shipped to the Army but was set at 150 percent when it was returned. The protester provided three declarations from Miltope's Director of Research and Development explaining how it concluded that the display resolution setting must have been changed by the Army. For example, Miltope states that it performed a "restore" function which restored the setting of the returned sample to the settings Miltope says were in place when the sample was shipped to the agency. Supp. Decl. of Miltope's Director of Research and Development, at 4. According to Miltope, performing the restore function confirmed that at the time the sample was shipped, the display resolution setting was set to 100 percent. Id.

In addition, the protester explains that it conducted its own tests. In this regard, Miltope explains that the agency's performance test results listed five groups of weighted component test results which were totaled to obtain the overall performance test score. Supp. Decl. of Miltope's Director of Research and Development, at 5. Miltope argues that two of these component tests would be affected by a change in the display resolution setting. Accordingly, Miltope conducted its own tests for these two components--graphics 2D fonts and texts, and graphics 2D windows interface--on the returned sample using a display resolution of 100 percent and 150 percent. Id. at 6. The component test result scores were significantly lower when the display resolution setting was set to 150 percent, which would result in a lower overall performance test score. Id.

In our view, Miltope has not established that the agency altered the display resolution setting on its sample. When Miltope submitted its sample to the agency, it also included a single test result showing that its sample achieved a score of 2,007.7, which exceeds the objective of 2,000. AR, Tab 15, Miltope Pre-Submission Test Result, Apr. 23, 2018; Supp. Decl. of Miltope's Research and Development Director, at 2 (¶ 7, stating that this single test result was consistent with subsequent tests performed on the sample prior to submission). In addition, the test result Miltope provided, using the sample it contends had a display resolution setting of 100 percent, also indicates that the scores for the graphics 2D fonts and texts, and graphics 2D windows interface were similar to the average test scores that the Army achieved when it allegedly used a 150 percent display resolution setting, as follows:

AR, Tab 15, Miltope Pre-Submission Test Result, Apr. 23, 2018; Supp. Decl. of Miltope's Research and Development Director, at 6 (¶ 31, providing the average scores reflected in AR, Tab 10, Miltope Sample Test Logs). Specifically, the Army's average test score results for the two component tests Miltope argues would be affected by a change in the sample's display resolution setting are strikingly similar to the results Miltope achieved in its pre-submission testing. Indeed, these test results undercut the protester's contention that its lower scores occurred because the display resolution setting was changed.

In addition, the protester does not dispute the evaluators' statement that they did not change the settings. The protester also does not allege that the agency engaged in sabotage or deliberate misconduct. Indeed, the protester expressly declines to assert that the Army evaluators acted in bad faith. Supp. Comments at 3. Rather, the protester has argued that someone else must have altered the display resolution setting on its sample. Supp. Comments at 3, 9; Comments and Supp. Protest at 8 ("[I]t may be that the evaluators did not change any of the settings on Bid Sample 1 or the Devices submitted by other offerors. But that does not mean that the settings were not changed by someone at the Army's facility other than an evaluator."). However, as noted, the contracting officer stated that no one other than the evaluators turned on the samples. AR, Tab 18, Supp. Decl. of TET and Contracting Officer. On this record, we cannot conclude, as Miltope urges, that the agency altered the display resolution setting on Miltope's sample prior to conducting the performance tests.⁵

The protester also argues that the agency's sample testing is inaccurate and unreliable because the test logs produced in the agency report indicate the use of a Windows 8.1 operating system, rather than the Windows 10 system required by the solicitation, and referenced in the declaration of the TET. Comments and Supp. Protest at 11-15; see AR, Tab 10, Miltope Sample Test Logs (stating "Operating System: Windows 8.1 Enterprise Edition build 9600 (64-bit)"). Miltope argues that the Army evaluators did nothing to determine the cause of the inaccuracy reflected in the test logs, and the Army's inability to explain the inaccuracy undermines the validity of the testing and demonstrates that the agency did not comply with the RFP. Comments and Supp. Protest at 14.

The agency concedes that it did not discover the "minor discrepancy" in the test logs identifying which Windows operating system was used until after the protest was filed, and asserts there cannot be any competitive prejudice to Miltope. Supp. COS/MOL

at 15; see also AR, Tab 14, Decl. of TET at 2 ("The evaluators can only speculate on why all of the bid sample test logs, for all offerors, noted a Windows 8.1 operating system[.]").⁶ The agency argues that any error in the software that caused the incorrect operating system to be indicated in the test logs likely did not affect the validity of the offerors' benchmark scores, and even if it did, all offerors were equally impacted. Supp. COS/MOL at 15-16.

We agree with the agency. As noted, the TET stated that it confirmed that all offerors' samples, including Miltope's, were using a Windows 10 operating system prior to testing their performance. AR, Tab 14, Decl. of TET, at 1-2; Tab 18, Supp. Decl. of TET and Contracting Officer. Indeed, the protester states that "the only [operating system] that was included in Miltope's Bid Sample 1 was a Windows 10 [operating system]; at no time did the Miltope Bid Sample 1 have a Windows 8.1 [operating system]." Comments and Supp. Protest at 12. Although the protester suggests four theories for how this error could have occurred, the protester does not explain how it could have been unequally or adversely impacted relative to the other offerors. C2G Ltd. Co.,

B-406093.3, Feb. 8, 2012, 2012 CPD ¶ 67 at 4 ("Competitive prejudice is an essential element of a viable protest; where the protester fails to demonstrate that, but for the agency's actions, it would have had a substantial chance of receiving the award, there is no basis for finding prejudice, and our Office will not sustain the protest, even if deficiencies in the procurement are found."). On this record, we find no basis to sustain the protest.

The protest is denied.

Thomas H. Armstrong
General Counsel