DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Apptis, Inc.

File: B-299457; B-299457.2; B-299457.3

Date: May 23, 2007

DIGEST

1. Post-closing time protest that evaluator who is not a government employee has an impermissible conflict of interest is untimely where the solicitation informed offerors of the agency’s intent to use the evaluator and the protester was aware of the factual basis of the evaluator’s alleged conflict of interest prior to closing time.

2. Agency’s evaluation of offerors’ “proof of concept” demonstrations cannot be determined to be reasonable where the record lacks adequate documentation supporting the evaluators’ findings.

3. Agency’s consideration of an offeror’s record of past performance as part of assessing technical approach risk was improper where past performance was not relevant and reasonably related to technical approach risk as defined in the solicitation.

4. Agency’s discussions with protester were not meaningful where the agency found significant weaknesses in the protester’s proposal but failed to identify them during discussions and give the firm the opportunity to comment on adverse past performance information to which it previously had not had an opportunity to respond.

5. Protest challenging agency’s price and past performance evaluation is denied where the record establishes that the evaluation was reasonable and consistent with the stated evaluation criteria

Apptis, Inc. protests the award of a contract to ViON Corporation under request for proposals (RFP) No. HC1013-06-R-2005, issued by the Defense Information Systems Agency (DISA), Department of Defense (DOD), for enterprise storage services (ESS). Apptis argues that the agency’s proposal evaluation and subsequent source selection decision were improper. Apptis also contends that a contractor that DISA employed to help evaluate offerors’ proposals had an impermissible organizational conflict of interest.

We sustain the protest in part and deny it in part.

BACKGROUND

DISA, through its Computing Services (CS) group, is a provider of information technology solutions for DOD customers that require data storage capacity services for a variety of applications and processing environments. The ESS initiative seeks to obtain state-of-the-art storage solutions to meet new and emerging customer requirements and provide storage solutions to replace existing DISA storage capacity that has exceeded its technical or economic life as determined by the agency. DISA’s goal for the ESS procurement here is to obtain a dynamically scalable storage capability utilizing an on-demand service approach that will readily adjust to changes in processing and throughput requirements, both increases and decreases, and is priced on a utility (“as used”) basis. In general terms, the statement of work required the contractor to acquire, install, de-install, transport, configure, and maintain all hardware, and provide software and software updates, for nine specified computer operating environments at any of the current or future DISA CS data centers located in the United States and Germany. Statement of Work (SOW) sect.C.1-4; Contracting Officer’s Statement at 2.

The RFP, issued on December 28, 2005, contemplated the award of a fixed-price, indefinite-delivery/indefinite-quantity (ID/IQ) contract for a base period of 5 years with three 1-year options. The solicitation identified four evaluation factors: technical solution; price/cost (hereinafter, price); service offerings; and past performance.[1] The solicitation notified offerors that proposal and past performance risk would also be evaluated. The solicitation stated that technical solution was significantly more important than price, and that price was more important than service offerings and past performance, which were approximately equal in weight to each other. Award was to be made to the responsible offeror whose proposal represented the best value to the government, all factors considered. RFP sect. M.3.2.

Three offerors, including Apptis and ViON, submitted written proposals by the February 23, 2006 closing date. A DISA source selection evaluation board (SSEB) evaluated offerors’ proposals as to the technical solution, service offerings, and past performance factors and subfactors using a color-coded rating system: blue, green, yellow, red, and white.[2] The RFP described the agency’s color ratings as follows:

	Technical Solution, Service Offerings Capability	Strengths	Weaknesses	Past Performance
Blue	The proposal exceeds requirements/objectives and clearly demonstrates the offeror’s capability to deliver exceptional performance.	There are numerous strengths that are of direct benefit to the Government.	Weaknesses are considered insignificant and have no apparent impact to the program.	Highly relevant/very recent past performance in all identified past performance efforts; excellent performance ratings.
Green	The proposal is satisfactory; the offeror is capable of meeting performance requirements/objectives.	Some strengths exist that are of benefit to the Government; the strengths clearly offset weaknesses.	A few weak-nesses exist; they are correctable with minimal Government oversight or direction.	Relevant/somewhat recent past performance in all identified past performance efforts; acceptable performance ratings.
Yellow	The proposal is minimally adequate; the offeror is most likely able to meet performance requirements/objectives.	Few strengths exist that are of benefit to the Government; the strengths do not offset the weaknesses.	Substantial weaknesses exist that may impact the program; they are correctable with some Government oversight and direction.	Somewhat relevant/not very recent past performance; most acceptable performance ratings.
Red	The proposal is highly inadequate; the offeror cannot meet performance requirements/objectives.	There are no beneficial strengths.	Numerous weaknesses exist that are so significant that a proposal rewrite is not feasible within a suitable timeframe.	Little relevant past performance identified; almost all unacceptable performance ratings.
White	Not used.	Not used.	Not used.	Completely lacks relevant performance history or past performance is unavailable, not due offeror’s failure to provide information.

RFP sect. M.4. The RFP also established that proposal and past performance risk would be evaluated as high, medium, or low, and that the price factor would not be color rated but would be evaluated for completeness, reasonableness, and if necessary, price realism. Id., sect. M.4-.5.

After the evaluation of offerors’ initial proposals, the contracting officer determined that all three proposals were in the competitive range. DISA subsequently conducted oral presentations, multiple rounds of discussions, and proof of concept (POC) demonstrations with the offerors. All three offerors submitted final proposal revisions (FPR) by the October 30 closing date.[3]

The SSEB then evaluated the offerors’ FPRs, as well as responses to discussion items, oral presentations, POC demonstrations, and past performance data obtained from other sources. The SSEB’s final evaluation ratings of ViON’s and Apptis’s proposals with regard to the evaluation factors and subfactors were as follows:

Factor	ViON	Apptis
Technical Solution	Green/Low	Green/Medium
Specified Operating Environments	Green/Low	Green/Medium
Performance/Availability Objectives	Green/Low	Green/High
Integration	Green/Low	Green/Medium
Storage Functionality Capabilities	Green/Low	Green/Low
Security	Green/Low	Green/Medium
Service Offerings	Blue/Low	Green/Low
Billing Methodology	Blue/Low	Green/Low
Flexibility of Utility Service	Blue/Low	Blue/Low
Technology Offerings	Blue/Low	Green/Low
Other Service Offerings	Blue/Low	Green/Low
Past Performance	Blue/Low	Green/Medium
Successful Implementation	Blue/Low[4]	Green/Medium
Quality Controls	Green/Low	Green/Low
Socioeconomic Goals	White/Low	Green/Low
Evaluated Price	$150,352,451	$122,015,340

Agency Report (AR), Tab 9B, SSEB Consensus Reports of Apptis; Tab 10B, SSEB Consensus Reports of ViON; Contracting Officer’s Statement at 16-17.

The SSEB subsequently briefed the agency source selection advisory council (SSAC) as to its evaluation of the offerors’ proposals. Id., Tab 13, SSEB Briefing to SSAC. On December 8, the SSAC (and the SSEB Chairperson) then briefed the SSA and recommended contract award to ViON. Id., Tab 14, SSAC Briefing to SSA. On January 12, 2007, after having reviewed the SSEB’s evaluation reports and findings, the SSA determined that ViON’s higher technically rated proposal represented the best value to the government, notwithstanding its higher price. Id., Tab 15, Source Selection Decision, at 1, 19-21. These protests followed.

DISCUSSION

Apptis’s protests raise numerous issues regarding the agency’s evaluation of offerors’ proposals and subsequent award determination. First, the protester alleges that DISA improperly used an employee of a contractor lacking impartiality to assist in the evaluation of proposals. Apptis also argues that DISA’s evaluation of proposals under the technical solution and past performance factors was improper and that the agency failed to engage in meaningful discussions with the firm. Apptis further argues that DISA’s price evaluation was improper and the best value tradeoff determination was unreasonable. As detailed below, we find that the agency’s evaluation of Apptis’s proposal under the technical solution factor was improper, and that DISA failed to conduct meaningful discussions with the firm.

Organizational Conflict of Interest

Apptis first protests that an organizational conflict of interest (OCI)[5] existed in connection with the evaluation due to DISA’s use of a contractor employee, C.F.,[6] as an SSEB evaluator. Specifically, Apptis alleges that at the time he evaluated offerors’ proposals, C.F. was employed by Shim Enterprises, Inc., a support services contractor for the DISA CS site in Ogden, Utah. Shim, the protester contends, was responsible for performing systems management for DISA at the time and location that a service outage occurred, and for which the equipment and/or support of Apptis’ primary subcontractor here, EMC, was alleged to have been at fault (the so-called “Fairchild chip issue”). Thus, Apptis asserts, Shim had a motivation to deflect blame to EMC and avoid any responsibility it may have had for the service outage problem that occurred.[7] Apptis argues that because the agency used as an evaluator an employee of a firm that had an impermissible OCI, the agency’s evaluation of proposals was unreasonable and the award to ViON improper.

The agency argues that Apptis’s protest regarding Shim’s alleged OCI is untimely. In support of its position, the agency contends that the RFP gave offerors notice of the fact that DISA planned to utilize Shim in the evaluation. Further, DISA asserts that the protester was aware of Shim’s role as DISA’s support services contractor for the Ogden site, and had interacted directly with Shim regarding the Fairchild chip failure incident. The agency contends that because Apptis was aware of the potential OCI involving Shim during the solicitation process, but did not protest this issue until after the closing time, the issue is untimely. We agree.

Our Bid Protest Regulations contain strict rules requiring timely submission of protests. Under these rules, protests based upon alleged improprieties in a solicitation which are apparent prior to bid opening or the time set for receipt of initial proposals must be filed prior to bid opening or the time set for receipt of initial proposals. 4 C.F.R. sect. 21.2(a)(1) (2007). Similarly, protests not based on solicitation improprieties must be filed within 10 days after the basis of protest is known or should have been known, whichever is earlier. 4 C.F.R. sect. 21.2(a)(2).

As a general rule, a protester is not required to protest that another firm has an impermissible OCI until after the agency has made an award determination. REEP, Inc., B-290688, Sept. 20, 2002, 2002 CPD para. 158 at 1-2. A different rule applies, however, where a solicitation informs offerors that the agency plans to utilize the services of a third-party contractor to assist in the evaluation of proposals, and the protester knew or should have known, by means of due diligence, that the nongovernmental entity may have, as alleged here, impaired objectivity. In such cases, the protester cannot wait until an award has been made to file its protest of an impermissible OCI, but instead must protest before the closing time for receipt of proposals. See Abt Assocs., Inc., B-294130, Aug. 11, 2004, 2004 CPD para. 174 at 2.

Here, the RFP expressly identified Shim as a nongovernmental evaluator of offerors’ proposals, RFP sect. L at 57, and the protester itself indicates that information regarding Shim’s role as DISA’s support services contractor for the Ogden location was readily available at Shim’s website. Protester’s Comments, Mar. 26, 2007, exh. A, Printout of Shim Enterprise webpage. Moreover, EMC employees had frequent and regular dealings with C.F. at the Ogden site, dealt directly with C.F. regarding the Fairchild chip failure incident, and were aware that C.F. was a Shim employee. Protester’s Comments, Mar. 26, 2007, Second Decl. of J.S.; AR, Apr. 17, 2007, exh. 1, Decl. of M.H.; exh. 2, Decl. of C.W. We think that, given EMC’s prior work for DISA at the Ogden site, and that EMC was Apptis’s primary subcontractor, Apptis knew or should have known of Shim’s role as the DISA support services contractor for the Ogden site where EMC had had the Fairchild chip failure issue. Apptis’s failure to protest the alleged OCI associated with Shim’s role in the evaluation of offerors’ proposals before the closing date for receipt of proposals makes this issue untimely.

Technical Solution Factor Evaluation

Apptis argues that the agency’s evaluation of proposals under the technical solution factor, the most important factor, was improper. Specifically, the protester argues that by considering various procedural aspects of its POC demonstration, DISA, in effect, applied an unstated evaluation criterion. Apptis also contends that DISA failed to properly document its adverse findings regarding Apptis’s POC demonstration. Apptis further asserts that the agency improperly considered the past performance of its primary subcontractor, EMC, in connection with its evaluation under the technical solution factor.[8] For the reasons set forth below, while we think that DISA’s consideration of the procedural aspects of Apptis’s POC demonstration did not constitute application of an unstated evaluation criterion and therefore was permissible, we nevertheless conclude that the agency failed to adequately document its conclusions in this area. Further, we conclude that it was improper for the agency to consider past performance as part of the evaluation of Apptis’s proposal under the technical solution factor.

With regard to the POC demonstration, the RFP stated as follows:

Offerors in the competitive range will be required to demonstrate their proposed solution to members of the SSEB at the offeror’s own demonstration location. The [POC] Demonstration is designed to validate the offeror’s solution in the following areas: Operating Environment Support, Performance, Integration and Manageability, Storage Capabilities, and Security. [POC] test instructions, together with any other Government provided data required to be used in the demonstration, will be provided to each offeror 14 calendar days prior to the demonstration. The results of the [POC] demonstration will not be separately rated, but may be used to adjust the ratings previously assigned.[9]

RFP sections L.1.13, M.2.1.4.1.

The agency subsequently provided offerors with additional, detailed instructions regarding how the POC demonstration would be conducted and the specific technical scenarios and solutions for which the offerors had to show proficiency. The instructions informed offerors that “[t]he purpose of the proof of concept is to demonstrate that a vendor’s proposed solution meets the Government’s stated objectives. The Government will determine if the demonstration meets all of the stated objectives and how efficiently those objectives are met.” AR, Tab 12A, POC Demonstration Instructions, May 15, 2006, at 1. The instructions also encouraged offerors to demonstrate any additional value-added capabilities they wanted DISA to be aware of in evaluating the merits of their proposals. Id.

Apptis’s POC demonstration took place at the facilities of its proposed subcontractor, EMC, on June 5-19, 2006, and was conducted primarily by EMC personnel. While the SSEB found that all of the solutions presented by Apptis worked and were validated, the agency asserts that the evaluators also observed various nonsubstantive deficiencies in Apptis’s demonstration. Specifically, the SSEB is said to have observed that the firm’s personnel had repeated difficulty in getting proposed solutions to work correctly the first time they were demonstrated. According to the agency, the problems evident with Apptis’s proposed solutions in a controlled testing environment raised concerns about the potential difficulties Apptis would have in the much-more complicated DISA production environment. The SSEB also purportedly observed that each of the various components of Apptis’s proposed solutions required a different technical expert, or experts, to present the solution. This raised concerns that successful implementation by Apptis of its proposed solution would require the coordination of a large number of individual technical disciplines, and was likely to result in an inability to get timely resolution of operational issues. AR, Tab 9B, SSEB Consensus Report of Apptis (Technical), at 2-4, 10-11. The SSEB’s findings and conclusions regarding these perceived deficiencies in Apptis’s nonsubstantive demonstration factored heavily into the evaluators’ ratings of the firm’s proposal under the technical solution factor and subfactors, as well as the SSAC’s recommendations and the agency’s eventual source selection decision. Id., Tab 14, SSAC Briefing to SSA, at 23; Tab 15, Source Selection Decision, at 6-7, 20.

Apptis first alleges that the agency’s decision to consider the nonsubstantive aspects of its POC demonstration constituted application of an unstated evaluation factor. Specifically, the protester alleges that the POC demonstration was intended to validate that an offeror’s proposed solution met the RFP’s stated objectives, and Apptis demonstrated its capabilities and received the agency’s agreement that it met all the stated test objectives. Apptis contends that the number of attempts required and the number of technicians it utilized were not properly part of the POC demonstration evaluation. We disagree that consideration of these matters was improper.

Although agencies are required to identify in a solicitation all major evaluation factors, they are not required to identify all areas of each factor which might be taken into account in an evaluation, provided that the unidentified areas are reasonably related to or encompassed by the stated factors. Chenega Technical Prods., LLC, B-295451.5, June 22, 2005, 2005 CPD para. 123 at 5.

Here, as set forth above, the RFP established that the purpose of the POC demonstration was to validate the offeror’s solution in specific technical areas. The agency also expressly informed offerors that the POC demonstration would be used to determine how efficiently each offeror met the stated performance objectives. We think that consideration of the number of attempts required, and the number of separate technicians required, as part of the validation of an offeror’s technical solutions is reasonably related to determining how efficiently an offeror meets the stated performance objectives. Thus, we conclude that the agency’s consideration of these aspects of Apptis’s POC demonstration as part of its evaluation of Apptis’s proposal was consistent with the stated evaluation criteria.

Apptis also argues that the agency’s determinations regarding its nonsubstantive POC deficiencies were not adequately documented. Apptis contends that the agency record contains almost no contemporaneous documents regarding its POC demonstration, and no contemporaneous documentation showing the POC deficiencies in core systems (as opposed to value-added solutions) which the SSEB allegedly observed.[10] The protester argues that in light of the fact that its proposal was significantly downgraded as a result of the SSEB’s finding that its solutions routinely required multiple attempts to implement, it was substantially prejudiced by the agency’s failure to adequately document the evaluation here.

In order for us to review an agency’s evaluation of proposals, an agency must have adequate documentation to support its judgment. Northeast MEP Servs., Inc., B-285963.5 et al., Jan. 5, 2001, 2001 CPD para. 28 at 7. While an agency is not required to retain every document or worksheet generated during its evaluation of proposals, the agency’s evaluation must be sufficiently documented to allow review of the merits of a protest. KMS Fusion, Inc., B-242529, May 8, 1991, 91-1 CPD para. 447 at 10. Where an agency fails to document or retain evaluation materials, it bears the risk that there may not be adequate supporting rationale in the record for us to conclude that the agency had a reasonable basis for its source selection decision. Southwest Marine, Inc.; American Sys. Eng’g Corp., B-265865.3, B-265865.4, Jan. 23, 1996, 96-1 CPD para. 56 at 10.

Here, the agency’s evaluation record contains the individual evaluator comment reports (over 600 pages), as well as various handwritten notes of SSEB members regarding Apptis’s POC demonstration. AR, Tab 9C, SSEB Evaluation Comments of Apptis; Tab 12, SSEB Notes from Apptis POC demonstration. However, none of these contemporaneous documents contains any reference to either of the nonsubstantive deficiencies that the evaluators found in Apptis’s demonstration. Likewise, there are no other documents in the evaluation record that support the SSEB’s conclusions that Apptis’s personnel had repeated difficulty in getting proposed solutions to work correctly the first time they were demonstrated, or that each of the various components of Apptis’s proposed solution required a different technical expert to implement.[11] Quite simply, while the agency described Apptis’s POC as a “problem plagued demonstration,” AR, Tab 14, SSAC Briefing to the SSA, at 23, it kept no records of which Apptis solutions required multiple attempts, how many attempts were required, or any reasons for the multiple attempts. In sum, we cannot tell if the evaluation of this aspect of Apptis’s proposal was reasonable because the agency record lacks adequate documentation to support its findings regarding Apptis’s POC demonstration.

In its report to our Office, the agency acknowledges that, “[t]here is no documentation from the POC that identifies how many times it took [Apptis] to successfully demonstrate any aspect of [its] proposed solutions.” AR, Apr. 5, 2007, at 9. DISA argues, however, that because the POC demonstration was not separately evaluated, there was no reason for the SSEB to document its concerns regarding Apptis’s nonsubstantive deficiencies. The agency’s argument is based on a fundamental misunderstanding of the requirement that agencies provide an adequate basis for their evaluation findings. While the POC demonstration was not itself a separate evaluation factor, it was a significant part of the agency’s overall evaluation of an offeror’s proposal and, as such, the agency was required to maintain records adequate to permit meaningful review. Tiger Enters., Inc., B-293951, July 26, 2004, 2004 CPD para. 141 at 2.

Apptis also argues that DISA’s evaluation of its proposal under the technical solution factor was improper because the evaluation included consideration of EMC’s past performance history with DISA. Specifically, the protester argues that as part of the evaluation of technical solution risk, the SSEB took into account EMC’s past performance with DISA. By considering EMC’s past performance history as part of technical solution risk, Apptis argues, the agency improperly elevated the relative importance of the past performance factor from the relative weightings established in the solicitation.

An agency is not precluded from considering an element of a proposal (such as past performance) under more than one evaluation criterion where the element is relevant and reasonably related to each RFP criterion under which it is considered. Infrared Techs. Corp., B-282912, Sept. 2, 1999, 99-2 CPD para. 41 at 5 n.2. Conversely, an agency may not consider an element of an offeror’s proposal under an evaluation criterion where the element is not relevant and reasonably related to that evaluation criterion.

With regard to the evaluation of proposal risk, the RFP stated as follows:

The Government will perform a risk assessment of each offeror’s proposal. The proposal risk assessment focuses on the risks and weaknesses associated with the offeror’s proposed approach. Assessment of risk identifies potential for disruption of schedule, degradation of performance, and need for increased government oversight. For any risk identified, the evaluation addresses the offeror’s proposal for mitigating those risks and why that approach is or is not feasible. A risk assessment of each offeror’s record of past performance will also be conducted and will form a part of the proposal evaluation.

RFP sect. M.4.1.2.

The solicitation also included distinct narrative descriptions for the risk assessment ratings for both the technical solution and past performance factors. For the assessment of technical solution risk, the agency’s stated focus was upon the severity of the disruption of schedule, increase in price/cost, or degradation of performance that the offeror’s proposed approach was likely to cause.[12] By contrast, for the assessment of past performance risk, the agency’s stated focus was upon the amount of doubt that existed, based upon an offeror’s past performance record, that the offeror could satisfactorily perform the proposed effort. Id.

It is clear, we think, that the solicitation intended for technical proposal and past performance risk to be analyzed differently. In general terms, the agency’s technical risk assessment was to be based on “what” the offeror’s proposed approach was, while the agency’s past performance risk assessment was to be based on “who” would be performing the various ESS services, and would include consideration of that firm’s past performance history. Given the distinction established here by the solicitation, we conclude that consideration of an offeror’s past performance was not relevant or reasonably related to the assessment of an offeror’s technical approach risk.

In its report to our Office, the agency does not argue that it was permissible for it to consider the past performance history of Apptis or EMC as part of the SSEB’s evaluation of the offeror’s technical approach risk. Rather, the agency contends that it did not do so. On the contrary, the record shows that the agency did in various instances improperly consider EMC’s past performance as part of its evaluation of Apptis’s technical approach. For example, in its consensus report regarding Apptis’s technical solution, the SSEB stated:

While there has not been an Apptis technical track record with DISA locations, EMC has performed in most all CS locations and has a complete service structure in place for their components, which make up the bulk of this offer. Service delivery is the greatest concern with this proposal. Prior experience with EMC support is that it has not always met government expectations.

AR, Tab 9B, SSEB Consensus Report of Apptis (Technical), at 2. The SSEB also reported that its technical solution medium risk rating for Apptis’s proposal was based on the “Medium level of DISA oversight anticipated to insure quality service delivery based on . . . DISA experience with service quality of EMC solutions.” Id., Tab 13, SSEB Briefing to SSAC, at 38. In turn, the SSAC stated that Apptis’s technical solution risk was in part the result of EMC’s past performance within DISA, id., Tab 14, SSAC Briefing to the SSA, at 23, and the SSA stated that the medium risk rating associated with Apptis’s technical solution was the result of EMC’s poor record of service delivery within DISA. Id., Tab 15, Source Selection Decision, at 20.

Viewing the evaluation record as a whole, we think that, as shown above, the agency’s evaluation of Apptis’s technical solution also took into consideration EMC’s record of past performance. As past performance was not relevant and reasonably related to the technical solution evaluation criterion as defined by the solicitation here, such consideration was improper. Moreover, it is clear that Apptis was prejudiced as a result; the ratings assigned to the firm’s proposal under the technical solution factor, as well as the reliance by the SSA upon those ratings, were the result of DISA’s improper consideration of who would be performing the ESS services as part of Apptis’s technical approach.

Past Performance Evaluation

Apptis contends that the agency improperly considered past performance information outside of the period specified by the terms of the RFP, and improperly regarded Apptis’s lack of past performance with the contracting agency as a weakness. The protester also raises other issues regarding the agency’s evaluation of offerors’ past performance, including the argument that DISA did not treat Apptis and ViON equally in its evaluation of their past performance. Although we do not here specifically address all of Apptis’s arguments about the evaluation of offerors’ past performance, we have fully considered all of them and find that they afford no basis upon which to sustain the protest.

Where a solicitation requires the evaluation of offerors’ past performance, we will examine an agency’s evaluation to ensure that it was reasonable and consistent with the solicitation’s evaluation criteria and procurement statutes and regulations. The MIL Corp., B-297508, B-297508.2, Jan. 26, 2006, 2006 CPD para. 34 at 10. Our review of the record leads us to conclude that the agency’s evaluation of Apptis’s past performance here was unobjectionable.

With regard to the evaluation of an offeror’s past performance, the RFP stated:

The Government will assess the offeror’s capability to perform the contract by evaluating their record of past performance as a prime contractor as well as the past performance record of their teaming/ subcontractors on comparable IT projects. Only past performance data regarding similar IT efforts completed within the last two years or currently under contract will be evaluated.

RFP sect. M.5.1.3.

When evaluating offerors’ proposals under the nonprice factors and subfactors, including past performance, the SSEB utilized a set of internal evaluation standards. Contracting Officer’s Statement at 8. For the first past performance subfactor, “successful implementation of a technical solution similar in size, scope and make-up” (hereinafter, successful implementation), the SSEB had five evaluation standards, including “Prior DISA CS experience with service and solutions offered by this vendor and its partners.” Id. at 12.

The SSEB rated Apptis’s proposal as Green/Medium Risk under the successful implementation subfactor. While finding that Apptis’s proposal met most of the evaluation criteria, the SSEB assessed as a weakness Apptis’s lack of prior DISA CS experience. Specifically, the agency evaluators stated:

EMC has a great deal of experience working in the DISA environment, however Apptis as an integrator does not have experience delivering services and solutions of this magnitude within [the] DISA computing services environment. There is considerable amount of risk for DISA in regard to integration when the integrator is not familiar with the full extent of our environment.

AR, Tab 9B, SSEB Consensus Report of Apptis (Past Performance), at 3. Also, the SSEB considered the past performance problems of EMC, Apptis’s proposed subcontractor, to be the second key reason (together with Apptis’s lack of direct DISA experience) for the firm’s medium risk ratings under both the successful implementation subfactor and the past performance factor. Id. at 2-3.

Apptis protests that DISA improperly considered past performance information regarding its subcontractor EMC that occurred outside of the 2-year period specified by the solicitation. The protester alleges that at the post-award debriefing, the agency mentioned EMC’s performance on the “EMS project” as one of the specific factual instances in support of DISA’s determination that EMC’s past performance had not always met expectations. According to Apptis, EMC’s portion of the EMS project was completed in 2003, and the contract itself ended in April 2004. Protest, Feb. 18, 2007, at 18. Apptis argues that because the EMS project contract was completed more than 2 years before the date when Apptis submitted its proposal, it could not be considered as part of DISA’s past performance evaluation here. Protester’s Comments, Mar. 26, 2007, at 49. The agency argues that the 2-year period for the consideration of performance information should be computed from the date the solicitation was issued and, as a result, consideration of the EMS project was appropriate. AR, Mar. 14, 2007, at 37.

As a preliminary matter, we note that it is unclear whether DISA actually relied on the EMS project in the evaluation of Apptis’s past performance or simply mentioned it at the debriefing as an additional factual instance of EMC’s adverse performance; for example, the SSEB report does not mention the EMS project as part of its past performance evaluation of Apptis. See id., Tab 9B, SSEB Consensus Report of Apptis (Past Performance). In any event, while it is not uncommon for a solicitation to specify exactly how the past performance period is to be computed, see, e.g., FR Countermeasures, Inc., B-295375, Feb. 10, 2005, 2005 CPD para. 52 at 2 (within 3 years of the solicitation’s initial closing date), the RFP here was silent as to how the past performance evaluation period was to be determined. Given that both the agency’s and the protester’s interpretations of the provision are reasonable ones,[13] the resulting ambiguity was readily apparent from the face of the RFP. Thus, to be timely, any protest on this ground had to be filed prior to the closing time for submission of proposals. Singleton Enters., B-298576, Oct. 30, 2006, 2006 CPD para. 157 at 5; see 4 C.F.R. sect. 21.2(a)(1). Since it was not, and since the agency’s interpretation of the time period is reasonable, we have no basis to question the agency’s consideration of the EMS project in the evaluation.

Apptis also protests that the agency’s past performance evaluation was improper because it employed an undisclosed evaluation criterion. Specifically, the protester argues that it was impermissible for the SSEB to consider its lack of past performance history with DISA to be a weakness.

We see nothing objectionable in the agency’s consideration of past performance history at DISA as part of its evaluation of offerors’ past performance. Past performance history with the agency for which the solicited work is to be performed clearly is a matter encompassed by the stated evaluation criteria. See Leach Mgmt. Consulting Corp., B-292493.2, Oct. 3, 2003, 2003 CPD para. 175 at 5. Accordingly, the agency reasonably considered Apptis’s lack of past performance history with DISA to be a weakness.[14]

Apptis argues that DISA’s evaluation of offerors’ past performance reflects disparate treatment. The protester alleges that, when considering past performance information from outside the offerors’ proposals, the SSEB generally considered only negative information about Apptis’s subcontractor EMC even though there was also positive information of which DISA had knowledge. Likewise, Apptis contends, the SSEB generally considered only positive information about ViON’s subcontractor Unisys even though there was also negative information of which DISA had knowledge. Moreover, Apptis asserts, the prior DISA contracts which formed the basis for the agency’s conclusions regarding the offerors’ past performance were ones in which EMC performed as a subcontractor to Unisys.

While agency evaluators may consider and rely upon information of which they are personally aware in the course of evaluating an offeror’s proposal, Del-Jen Int’l Corp., B-297960, May 5, 2006, 2006 CPD para. 81 at 7, it is a fundamental principle of government procurement that evaluators must treat all offerors equally. Infrared Techs. Corp.--Recon., B-255709.2, Sept. 14, 1995, 95-2 CPD para. 132 at 4-5. Our review of the record confirms that the agency evaluated offerors’ proposals fairly and without disparate treatment under the past performance factor.

In our view, Apptis’s argument of disparate treatment is mistakenly premised upon an improper “apples-to-oranges” comparison of the offerors’ proposals. When evaluating Apptis’s past performance, the SSEB was aware that its subcontractor EMC would be performing an extensive role in the delivery and servicing of Apptis’s ESS solutions. AR, Tab 9B, SSEB Consensus Report of Apptis (Technical), at 2. By contrast, when evaluating ViON’s past performance, the SSEB was aware that the role to be played by its proposed subcontractor Unisys was a more minor one, limited to discrete ESS solutions and services, and that ViON would be performing the majority of the services. Id., Tab 10B, SSET Consensus Report of ViON (Technical), at 1-14. Given the different degrees of reliance by ViON and Apptis on their respective proposed subcontractors, the SSEB properly found EMC’s past performance much more relevant to the evaluation than that of Unisys, and the SSEB’s overall past performance ratings for ViON and Apptis reflected this distinction. In sum, the difference in the past performance ratings of ViON and Apptis here was not the result of unequal treatment by the agency of identical similar facts, but instead resulted from the agency’s recognition of different underlying facts.

Lack of Meaningful Discussions

Apptis argues that neither of the issues on which its ratings under the technical solution and past performance factors were based--the POC demonstration deficiencies and EMC’s adverse past performance--was raised by the agency during discussions. Apptis argues that in order for the discussions conducted by the agency to be meaningful, the agency should have raised these perceived weaknesses, given that they significantly affected the evaluation of the firm’s proposal under multiple subfactors and factors. We agree.

Although discussions must address deficiencies and significant weaknesses identified in proposals, the precise content of discussions is largely a matter within the contracting officer’s judgment. See FAR sect. 15.306(d)(3); American States Utils. Servs., Inc., B-291307.3, June 30, 2004, 2004 CPD para. 150 at 6. We review the adequacy of discussions to ensure that agencies point out weaknesses that, unless corrected, would prevent an offeror from having a reasonable chance for award. Northrop Grumman Info. Tech., Inc., B-290080 et al., June 10, 2002, 2002 CPD para. 136 at 6. When an agency engages in discussions with an offeror, the discussions must be meaningful, that is, they must reasonably lead an offeror into the areas of its proposal requiring correction or amplification. See TRI-COR Indus., Inc., B-259034.2, Mar. 14, 1995, 95-1 CPD para. 143 at 5.

Here, DISA held discussions with all offerors after completing its evaluation of initial technical and price proposals. The agency conducted four rounds of discussions with Apptis and provided the firm with a total of 120 discussions items. One of the specific discussion items, which involved the service offerings evaluation factor, stated:

APP-DI-SRVO-105
Project Management Office (PMO) Support

Detail: During the POC APPTIS demonstrated a large variety of components that comprise their proposed storage solutions. The demonstration of these solutions involved a number of highly skilled expert technicians. What level of expertise for these components will be available within the PMO? If expertise required for a specific requirement is not available within the PMO, what will be the turn around time [for] assigning someone or dispatching a subject matter expert to provide on site assistance?

AR, Tab 9A, Discussions Items with Apptis, July 20, 2006, at 50.

None of the agency’s discussion items concerned either the SSEB’s observation that Apptis repeatedly required multiple attempts at the POC demonstration in order to get its proposed solutions to work correctly, or EMC’s adverse past performance.[15]

We think that DISA failed to conduct meaningful discussions with Apptis with respect to the nonsubstantive POC deficiencies. Regarding the technical solution factor, none of the discussion items provided to Apptis mentioned the SSEB’s observation that the firm’s personnel had repeated difficulty in getting proposed solutions to work correctly the first time they were demonstrated, or the agency’s associated concern (that Apptis would have greater difficulties in the much-more complicated DISA production environment). Further, only the one discussion item noted above makes any mention of the SSEB observation that Apptis’s demonstration involved a number of highly skilled expert technicians, and it does not reference the associated concern (that successful implementation would require the coordination of a large number of individual technical disciplines). Moreover, this discussion item was directed towards an aspect of Apptis’s proposal under the service offerings (not the technical solution) evaluation factor. Under the circumstances here, we cannot conclude that an offeror, reviewing the agency’s question in conjunction with the material that the offeror had submitted with its proposal, reasonably would have recognized the agency’s concern regarding the POC demonstration.

DISA does not dispute that its discussions with Apptis did not address the nonsubstantive POC deficiencies that its evaluators had observed. Rather, the agency argues that discussions with Apptis regarding the nonsubstantive POC deficiencies that the SSEB had observed would not have benefited Apptis. In its report to our Office, the agency states:

Given the circumstances under with the POC issues arose, it is not clear what discussions would have accomplished to assist Apptis in enhancing its demonstration. . . . Apptis knew the POC was only going to be conducted once and that there would not be a second opportunity to repeat any part of it. . . . It should have been apparent to the offerors that it would benefit them to put on a good demonstration. What is not apparent is what purpose after-the-fact discussions would have accomplished to improve on a demonstration that had already been completed. There is no information the Agency could have communicated at that point that would have assisted Apptis in improving on its demonstration.

AR, Mar. 14, 2007, at 13.

The agency essentially argues that Apptis was not prejudiced by its lack of meaningful discussions here, because even if the agency had mentioned the POC deficiencies, there was nothing that Apptis could do to fix them. We disagree. As a preliminary matter, while, as explained above, we think that DISA’s consideration of the nonsubstantive aspects of the POC demonstration did not involve application of an unstated evaluation criterion, neither did the RFP expressly inform offerors that the agency’s validation effort would include assessing the number of tries or the number of technical experts required. Further, even though Apptis could not change the events that transpired at its POC demonstration, the agency nevertheless was required to point out the weaknesses it observed and provide the firm with an opportunity to address them. Thus, for example, Apptis’s discussion responses and/or FPR could have refuted the agency’s purported observations, provided explanations as to why the events occurred, or proposed methods by which to address the agency’s associated concerns. DISA’s failure to conduct discussions with Apptis regarding the perceived POC deficiencies improperly foreclosed the offeror’s opportunity to address these proposal weaknesses.

We also conclude that the agency did not conduct meaningful discussions with Apptis regarding EMC’s past performance problems. Contracting agencies are required to provide an offeror with an opportunity to address adverse past performance information to which the offeror has not previously had an opportunity to respond. See FAR sect. 15.306(d)(3). As noted above, none of the agency’s discussion items inquired into EMC’s past performance, and, as a result, Apptis had no opportunity to respond to the agency’s adverse findings.[16] Moreover, while the agency was aware of and apparently relied upon specific instances of EMC past performance problems, none of these incidents was mentioned in discussions with Apptis.

In its report to our Office, DISA does not deny that its discussions failed to raise EMC’s past performance problems; rather, the agency argues did not have to do so. Specifically, the agency states:

When it considers past performance in a procurement, the Government has an obligation to provide offerors with an opportunity to address negative past performance information, provided they have not had a prior opportunity to do so. Whether or not specifically identified . . . in discussions, it is beyond question that [Apptis] was well aware of the [EMC past performance] incidents discussed at the debriefing, a point emphasized by the detailed rebuttals on pages 18-19 of [Apptis’s] Supplemental Protest (Tab 1). There was no obligation on the part of the Agency to explicitly remind them.

AR, Mar. 14, 2007, at 38.

The agency argument here indicates a misunderstanding of its obligations during discussions. The agency’s responsibility to conduct meaningful discussions is not conditioned, or qualified, by what it assumes the offeror already knows about its past performance history. Rather, the obligation is upon the agency in the course of discussions to point out adverse past performance information that would prevent the offeror from having a reasonable chance at award and that the offeror has not previously had the opportunity to rebut.

Price Evaluation

Apptis argues that DISA’s price evaluation was improper and not in accordance with the solicitation. Specifically, the protester states that the RFP established a price evaluation methodology that combined three different contract length possibilities into a single overall evaluated price, and asserts that it relied heavily upon the solicitation’s “blended” pricing methodology when developing its pricing structure. However, Apptis argues, DISA’s price evaluation disregarded the solicitation’s stated evaluation criteria and focused instead on offerors’ prices for only the single most likely contract length scenario, thereby failing to give sufficient consideration to the evaluated price difference between its proposal and that of ViON. We disagree.

Agencies must consider cost to the government in evaluating proposals. 10 U.S.C. sect. 2305(a)(3)(A)(ii) (2000). While it is up to the agency to decide upon the method for the evaluation of offerors’ prices, the method chosen must include some reasonable basis for evaluating or comparing the relative costs of proposals, so as to establish whether one offeror’s proposal would be more or less costly than another’s. See FAR sect. 15.405(b); Bristol-Myers Squibb Co., B-294944.2, Jan. 18, 2005, 2005 CPD para. 16 at 4. Where a protester challenges an agency’s evaluation of proposals, including the evaluation of an offeror’s proposed price, our Office will not reevaluate proposals, but instead will examine the record to determine whether the agency’s judgment was reasonable and consistent with the stated evaluation criteria and applicable procurement statutes and regulations. Liquidity Servs., Inc., B-294053, Aug. 18, 2004, 2005 CPD para. 130 at 5.

The RFP established that the agency’s evaluation of offerors’ proposed prices would be based on an analysis of proposals’ total discounted life cycle cost (DLCC) for the 5-year base period and three 1-year option periods. Additionally, the solicitation stated that the DLCC analysis would incorporate proposed prices into a pricing model containing three different contract-length scenarios--one based on contract termination at 48 months, one based on contract termination at 72 months, and a third based on contract termination at 96 months--but did not inform offerors of the predetermined weightings that DISA assigned to each scenario.[17] Notwithstanding the fact that the contract length scenarios were mutually exclusive in nature, and could not all occur, the RFP stated that all three scenarios would be evaluated and the relative weightings applied to determine the overall DLCC for each offeror’s proposal.[18] RFP sect. M.5.1.4.

The SSEB computed ViON’s and Apptis’s weighted DLCCs as follows:

	ViON	Apptis
Scenario 1	$11,453,986	$6,328,125
Scenario 2	$26,986,459	$19,531,631
Scenario 3	$111,912,006	$96,155,583
Total	$150,352,451	$122,015,340[19]

AR, Tab 9D, SSEB Price Summary for Apptis; Tab 10D, SSEB Price Summary for ViON.

In its briefing to the SSAC, the SSEB set forth the offerors’ total weighted DLCCs, demonstrating the $28 million difference between the total evaluated prices of Apptis and ViON.[20] Id., Tab 13, SSEB Briefing to SSAC, at 119. The SSAC’s subsequent briefing to the SSA described both offerors’ total evaluated prices and the weighted DLCCs for the single most likely contract length scenario (the 96-month scenario). Id., Tab 14, SSAC Briefing to SSA, at 26. In the source selection decision, the SSA considered only the offerors’ total weighted DLCCs, including specific reference to the $28 million difference between Apptis’s and ViON’s evaluated prices in her price/ technical tradeoff determination. Id., Tab 15, Source Selection Decision, at 19, 21.

Contrary to the protester’s allegations, the record here provides no basis to challenge the agency’s evaluation of offerors’ prices. As set forth above, the agency computed offerors’ evaluated prices in accordance with the RFP’s stated evaluation criteria, a conclusion which Apptis does not contest. Moreover, the record also reflects that the agency utilized offerors’ evaluated prices as the basis for its award determination. While presumably aware of offerors’ weighted DLCCs for each individual contract length scenario (including the most likely contract length scenario), the SSA, consistent with the RFP evaluation scheme, relied exclusively on offerors’ total weighted DLCCs, and the associated price difference between the proposals of Apptis and ViON here, in making her source selection determination.

We find no merit in Apptis’s argument that the agency could not consider offerors’ evaluated prices for individual contract length scenarios as part of the evaluation process. It was self-evident that only one, and not all three, of the contract length scenarios that comprised offerors’ total weighted DLCCs would actually occur. Further, nothing in the RFP precluded the agency from examining the DLCCs for the most likely contract length scenario in addition to offerors’ total weighted DLCCs. Quite simply, the fact that DISA also considered offerors’ evaluated prices for the most likely of the mutually-exclusive contract length scenarios here does not indicate, as Apptis contends, that the agency disregarded the RFP’s stated evaluation criteria in its price evaluation.

Source Selection Decision

Apptis argues that DISA’s pric