Nexant, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Nexant, Inc.

File: B-407708; B-407708.2

Date: January 30, 2013

DIGEST

1. Protest that the agency engaged in misleading discussions is sustained where the agency failed to convey its actual concerns to the protester in a manner that would have enabled the protester to meaningfully respond to the evaluated weaknesses identified in its proposal.

2. Protest challenging the agency’s technical evaluation is sustained where the agency’s evaluation was based on a flawed methodology and contained numerous unreasonable conclusions.

3. Protest that the agency’s award decision was flawed is sustained where the decision did not reasonably explain the basis for selecting the awardee’s higher-rated, higher-cost proposal.

Nexant, Inc., of Washington, D.C., protests the award of a contract to Deloitte Consulting, LLP, of Arlington, Virginia, under request for proposals (RFP) No. 486-12-018A, issued by the United States Agency for International Development (AID), for services in connection with the establishment of a private financing advisory network in Asia (PFAN-Asia). Nexant argues that the agency engaged in misleading discussions, misevaluated proposals, and failed to make a source selection decision in accordance with applicable procurement laws and regulations.

We sustain the protest.

BACKGROUND

The RFP was issued on April 12, 2012, and contemplated the award of a cost- reimbursement contract for a five-year period to perform various services in connection with the establishment of the PFAN-Asia program. The underlying purpose of the PFAN-Asia program is to assist Asian businesses, governments and other entities to mobilize and “scale up” investments in clean energy projects by providing increased financing from interested investors to project developers in Asia’s developing countries. See RFP at 9-11.

Offerors were advised that the agency intended to award a contract to the firm whose proposal was deemed to offer the “best value” to the government, considering cost and the following non-cost evaluation factors: (1) technical approach, draft work plan summary, and draft summary performance management plan (40 points); (2) management approach (20 points); (3) personnel (20 points); (4) corporate capability (10 points); and (5) past performance (10 points). RFP at 87-88. The RFP provided that cost proposals would be evaluated for reasonableness, realism, completeness and consistency with an offeror’s technical proposal. RFP at 86-87. The RFP advised that the combined non-cost evaluation considerations were significantly more important than cost. RFP at 86.

AID received a number of proposals in response to the solicitation. The agency evaluated the proposals, established a competitive range, engaged in discussions with the competitive range offerors, and solicited, received and evaluated final proposal revisions. The agency assigned the following evaluation ratings to, and determined the most probable cost for, the protester’s and awardee’s revised proposals as follows:

Agency Report (AR), exh. 15, Final Memorandum of Negotiations, at 24, 25. On the basis of these evaluation results, the agency made award to Deloitte, finding that its proposal represented the best value to the government. Id. at 29-30. After learning of the agency’s selection decision and receiving a debriefing, Nexant filed the instant protest.

Despite the agency’s award of this contract to Deloitte on October 1, there is no contemporaneous document that sets forth the rationale for the agency’s award decision. In fact, the only document in the record that embodies the agency’s source selection decision is the final memorandum of negotiations (FMN) cited above. This summary of the procurement, which includes the eventual best value decision, was executed on October 31--some 49 days after the source selection authority (SSA) states she made the decision to award the contract to Deloitte, 30 days after the actual award was made, and 9 days after Nexant filed its protest with our Office.

DISCUSSION

Nexant raises a large number of contentions, and we have considered all of them. We sustain Nexant’s protest based on the following arguments, which are discussed below: (1) the agency failed to provide Nexant meaningful discussions; (2) the agency’s evaluation of the offerors’ technical proposals was unreasonable; and (3) the agency’s selection decision was unreasonable. We deny Nexant’s remaining allegations.

Meaningful Discussions

Nexant argues that AID failed to provide it with an opportunity for meaningful discussions. It is a fundamental principle of negotiated procurements that discussions, when conducted, must be meaningful; that is, discussions must identify deficiencies and significant weaknesses in an offeror’s proposal that could reasonably be addressed so as to materially enhance the offeror’s potential for receiving award. PAI Corp., B-298349, Aug. 18, 2006, 2006 CPD ¶ 124 at 8; Spherix, Inc., B-294572, B-294572.2, Dec. 1, 2004, 2005 CPD ¶ 3 at 13. An agency may not mislead an offeror through the framing of a discussion question into responding in a manner that does not address the agency’s actual concerns, or otherwise misinform the offeror concerning a problem with its proposal. Metro Mach. Corp., B-281872 et al., Apr. 22, 1999, 99-1 CPD ¶ 101 at 6.

The solicitation here divides the requirements among five mandatory tasks (tasks 1 through 5) and one optional task (optional task 6). RFP at 12-13. Task 1 requires the contractor to provide analytical and consultative input in determining the scope and nature of activities to be implemented as part of the overall statement of work (SOW). RFP at 12. The geographic scope of these activities during the period of performance is described in section 4.2 of the RFP as follows:

The PFAN-Asia program will initially focus on the Association of Southeast Asian Nations (ASEAN) sub-region and support activities in Cambodia, Laos, Indonesia, Malaysia, Philippines, Thailand, and Vietnam. Based on the results of Task 1 . . . and direction from and approval of the Contracting Officer Representative (COR), starting in year 2 of the program the Contractor must either deepen support to the ASEAN sub-region or expand activities to also include the South Asia sub-region, including Bangladesh, India, Maldives, Nepal, and Sri Lanka. Considerations for expanding the program to South Asia include, but are not limited to: potential economies of scale as a result of replicating the approach in engaging the ASEAN sub-region; USAID bilateral Mission buy-ins in Option 6 (see below); and opportunities to leverage resources through public-private partnerships . . . .

RFP at 12 (emphasis in original).

Optional task 6, in turn, provides that the agency may, at its option, authorize country-specific support for PFAN activities through bilateral USAID Mission buy-ins. RFP at 12-13; 27. According to the agency, these task 6 bilateral buy-ins could involve USAID missions from any country, and were not limited to the South Asian countries enumerated in the geographic scope quoted above. See, AR, exh. 13, Final Technical Evaluation Committee (TEC) Report, at 24.

After evaluating initial proposals, AID assigned Nexant’s proposal a weakness for misunderstanding the geographic scope, as well as the nature of the tasks, outlined in the solicitation. Specifically, the evaluators identified the following weakness in the Nexant proposal:

Nexant failed to understand that Optional Task 6 is designed to support focused country-level PFAN activities in any country, not just countries in South Asia. Thus, Nexant did not provide any creative ideas on leveraging sufficient resources and partnerships to engage South Asia as part of the overall program rather than relying entirely on Optional Task 6 resources . . . .

AR, exh. 7, Initial TEC Report, at 33.

During discussions, AID asked Nexant to address the following question: “Please clarify Nexant’s overall understanding [of] Optional Task 6.” AR, exh. 9, Nexant Discussion Questions, at 2. In response to the question, Nexant explained that it would expand PFAN activities in up to 4 South Asian countries, should funding be made available, and explained that Nexant already had a presence in three of the South Asian countries identified in the solicitation: India, Maldives and Sri Lanka. AR, exh. 11, Nexant Discussion Responses, at 10-11.

In evaluating Nexant’s response, the agency evaluators continued to assign Nexant’s proposal a weakness in this area, finding that Nexant did not understand that task 6 was not limited to the South Asian countries identified in the solicitation. AR, exh. 13, Final TEC Report, at 24.

Nexant argues that AID’s discussion question was misleading because it did not afford it an opportunity to address the agency’s actual concerns regarding this weakness. As noted, the evaluators were concerned with two aspects of Nexant’s proposal in this area: (1) its apparent failure to understand that optional task 6 was intended to be available for exercise in any country, and not just in the countries identified by the RFP as South Asia; and (2) Nexant’s apparent failure to provide any creative ideas for leveraging resources and partnerships to engage South Asian countries as part of the overall program, rather than relying entirely on optional task 6 resources. AR, exh. 7, Initial TEC Report, at 33. We agree with Nexant that the agency’s discussion question in this area was misleading.

To the extent the agency’s discussion question focused on either of these two areas of concern, it was limited to directing Nexant’s attention--obliquely--to the first concern, namely, Nexant’s apparent misunderstanding of the geographical scope of optional task 6. We agree with the protester, however, that the discussion question did not adequately convey to Nexant the nature of the first concern because the protester was only asked, without elaboration, to clarify its understanding of optional task 6, and was never advised of its apparent misunderstanding of the geographic scope of optional task 6. More importantly, the question entirely failed to advise the protester of the agency’s second concern, which was Nexant’s apparent failure to provide creative ideas for leveraging resources and partnerships to engage South Asian countries as part of the overall PFAN program. Nexant therefore improperly was deprived of the opportunity to improve its proposal in this area.

Moreover, the record shows that these concerns were specifically identified in the source selection portion of the FMN as the basis for one of just three enumerated weaknesses in the protester’s proposal that factored into the SSA’s best value decision. In describing this weakness remaining in Nexant’s proposal after discussions, the SSA stated:

However, the TEC found several responses unsatisfactory and unclear. For example, Nexant continued to misinterpret the program’s Geographic Scope (RFP section 4.2), and presented a narrow approach for expanding program activities in South Asia. Along a similar vein, Nexant failed to describe an approach for deepening activities in any country where a USAID Mission might buy-in to the PFAN-Asia program through Optional Task 6 (an option which may or may not ever be exercised); rather, Nexant incorrectly considered Optional Task 6 the only means of working in South Asia.

AR, exh. 15, FNM, at 29. We conclude that Nexant was prejudiced by the agency’s misleading discussions in this area. We therefore sustain this aspect of Nexant’s protest.[1]

Technical Evaluation

Nexant argues that AID’s technical evaluation contained numerous errors, inconsistencies and unexplained scoring anomalies. Nexant maintains that, because the offerors’ numeric scores were relatively close (86 points for Nexant’s proposal versus 94.73 points for Deloitte’s), and because the agency’s source selection decision relied heavily on the numeric scores assigned to the proposals, these errors, if corrected, could affect the agency’s source selection decision.

We have reviewed Nexant’s allegations and agree that the agency’s scoring methodology and evaluation conclusions are not reasonable. In this connection, while our Office does not independently evaluate proposals, we nonetheless review the record to ensure that the agency’s evaluation is reasonable and consistent with the terms of the RFP, as well as applicable procurement statutes and regulations. Yang Enters., Inc.; Santa Barbara Applied Research, Inc., B-294605.4 et al., Apr. 1, 2005, 2005 CPD ¶ 65 at 5; Acepex Mgmt. Corp., B-283080 et al., Oct. 4, 1999, 99-2 CPD ¶ 77 at 3, 5. Additionally, while an agency’s use of point scores can serve to guide intelligent decision making, the assignment of the underlying point scores must be on an intelligible, reasonable, equal and consistent basis for all proposals. The Clay Group, LLC, B-406647, B-406647.2, July 30, 2012, 2012 CPD ¶ 214 at 9-10.

We note at the outset that the record provided by AID does not explain how the evaluators assigned their scores. The record does not, for example, include a source selection plan with definitions of the scoring scheme used by the evaluators, nor do the TEC reports explain how the evaluators translated their assessments of strengths and weaknesses into numeric point scores.[2] Although we recognize that there is no legal requirement for a source selection plan--or even, for that matter, pre-established definitions for use in scoring proposals--nonetheless, such evaluation tools can prove useful to an agency in achieving consistency in its evaluation. See e.g., Israeli Aircraft Indus., Ltd., MATA Helicopter Div., B-274389, et al., Dec. 8, 1996, 97-1 CPD ¶ 41 at 3 (source selection plan provided detailed basis for scoring proposals.)

The record shows that the AID evaluators first prepared individual evaluation worksheets, which listed the point scores assigned, and the strengths, weaknesses, significant weaknesses and deficiencies identified for the proposals under each evaluation factor. The evaluators then met to prepare a consensus TEC report. This procedure occurred during the initial evaluation, and again during the evaluation of the offerors’ revised proposals. In arriving at the consensus point scores for each offeror, the evaluators calculated a mathematic average of the point scores assigned by the individual evaluators, rather than developing a consensus score and rationale. AR, exh. 7, Initial TEC Report, at 3; AR, exh. 13, Final TEC Report, at 4, 21. The record shows that the agency’s reliance on the mathematic average of the scores assessed by the individual evaluators appears to have resulted in mechanically-calculated scores that did not reflect the consensus judgment of the evaluators. While the use of a mathematic average in scoring proposals is not per se improper, in this case it resulted in the scores being distorted in the ways discussed below.

First, not all of the strengths and weaknesses identified by the individual evaluators were carried forward to the consensus TEC evaluation report. For example, in the final evaluation, one of the individual evaluators assigned only a single weakness to the Nexant proposal under the personnel evaluation factor, and assigned the proposal a numeric score of only [deleted] out of 20 available points. AR, exh. 14, Final Evaluator A Scoresheet (Nexant), at 6.[3] The weakness assigned by evaluator A that resulted in the [deleted] point score (rather than a higher score), however, was not carried forward to the final TEC report. AR, exh. 13, Final TEC Report, at 28-29. Since the weakness identified by evaluator A appears not to have been accepted in the consensus evaluation, the evaluators’ averaging method for calculating the consensus scores resulted in Nexant’s score being artificially lower than it should have been.

Second, the record reflects instances where Nexant’s individual score was based on a weakness or significant weakness assessed by an individual evaluator, even though the consensus evaluation report shows that the evaluators expressly found that the weakness or significant weakness had been resolved during discussions. For example, evaluator B assigned the Nexant proposal a significant weakness under the management approach evaluation factor for not having adequate key personnel in the third year of the contract, and assigned the proposal a numeric score of only [deleted] out of 20 available points. AR, exh. 14, Final Evaluator B Scoresheet (Nexant), at 2. In the final TEC evaluation report, however, the evaluators specifically found that this significant weakness had been eliminated by Nexant during discussions. AR, exh. 13, Final TEC Report, at 25-25. The record therefore shows that inconsistencies such as these also had the effect of artificially lowering Nexant’s consensus scores.

Third, the record reflects similar errors regarding the individual evaluator scores for Deloitte’s proposal. For example, evaluator C originally identified a significant weakness in Deloitte’s proposal under the management approach factor for failing to present the awardee’s capacity in South Asian countries; the evaluator assigned the Deloitte proposal a score of [deleted] out of 20 available points under that evaluation factor. AR, exh. 8, Initial Evaluator C Scoresheet (Deloitte), at 2. This significant weakness was carried forward to the initial TEC report. AR, exh. 7, Initial TEC Report, at 9. After discussions, evaluator C continued to note this same significant weakness in the Deloitte proposal, but for reasons that are not explained in the record, evaluator C raised Deloitte’s score under the management factor to [deleted]. AID concedes that this scoring was in error. See Supp. AR at 10. Therefore, Deloitte’s score was artificially increased, even though the significant weakness remained.

Finally, the record shows that the TEC’s consensus evaluations contained internal inconsistencies that affected Nexant’s evaluation score, as well as the agency’s source selection decision. For example, under the management approach evaluation factor, the initial TEC report identified a weakness based on Nexant’s failure to adequately describe the roles and responsibilities of its program staff. AR, exh. 7, Initial TEC Report, at 34. After conducting discussions with Nexant, the evaluators expressly found that: “In its response to question 16, the Offeror fully addresses the TEC’s question regarding staff responsibilities and roles in implementing tasks.” AR, exh. 13, Final TEC Report, at 26. Notwithstanding this express finding, however, the final TEC report stated elsewhere, in its summary rationale for the protester’s final score for the management approach factor, that this remained an unresolved weakness. Id. at 28. This weakness, along with a second resolved weakness, is identified in the FMN as a basis for distinguishing the Nexant proposal from the Deloitte proposal. AR, exh. 15, FMN, at 29.

In sum, the record reflects a lack of rationality and consistency on the part of the agency evaluators in assigning point scores to the proposals, such that the scores relied on by the SSA cannot be said to be reliable or reflective of the comparative merits of the proposals. Moreover, as discussed in detail below, the record shows that the SSA placed heavy reliance on the scores in connection with her award decision. We therefore sustain this aspect of Nexant’s protest.

Source Selection Decision

Nexant argues that the FMN does not adequately document the agency’s cost/technical tradeoff. According to the protester, the SSA principally relied on the difference in the point scores assigned, and also the number--but not the quality--of the strengths and weaknesses assigned to the proposals in arriving at her selection decision.

We find that the agency’s selection decision is inadequate, in and of itself, to support the award to Deloitte. In this connection, where an agency makes award to a higher-rated, higher-cost proposal in a best value acquisition, the award decision must be supported by a rational and adequately-documented explanation of why the higher-rated proposal is, in fact, superior, and why its technical superiority warrants paying a price premium. FAR § 15.308; ACCESS Sys., Inc., B-400623.3, Mar. 4, 2009, 2009 CPD ¶ 56 at 7. An agency that fails to adequately document its source selection decision bears the risk that our Office may be unable to determine whether the decision was proper. ACCESS Sys., Inc., supra.

Here, the FMN does not identify any particular features of the Deloitte proposal for which the SSA thought it was worth paying a price premium. Instead, the record shows that she relied principally on the ranking of the proposals based on point scores or the generic number of strengths assigned to the proposals in concluding that award to Deloitte offered the best value to the government. She states:

As noted in the supplemental technical evaluation memorandum dated September 12, 2012, Deloitte had a significant lead in the original TEC review, and subsequently resolved most of the concerns the TEC raised during discussions with regard to its technical approach, draft work plan, draft PMP [performance management plan], management approach, and personnel. In all, Deloitte provided an outstanding and clear proposal for implementing the PFAN-Asia program.

Deloitte’s, [Offeror A’s] and Nexant’s proposals were all strengthened during discussions. However, while [Offeror A and] Nexant demonstrated general improvements, both again achieved a revised score considerably lower than Deloitte’s revised score. Deloitte continued to reflect significantly more strengths and fewer weaknesses than both [Offeror A] and Nexant’s proposals.

* * * * *

Given the significant overall quality of its response to the requirements of the RFP, the TEC unanimously concluded that Deloitte’s proposal offered a substantially better chance for success in achieving the program objectives of the PFAN-Asia program. Ultimately, Deloitte’s proposal had more exceptional strengths that would significantly benefit the Government than either Nexant’s or [Offeror A’s] proposals.

AR, exh. 15, FMN, at 28-29.

This generic, albeit often repeated, conclusion--that the Deloitte proposal offers the best value because it was ranked higher than the other two proposals and included more strengths than the other two proposals--does not meet the requirement for an explanation of why the higher-rated proposal is, in fact, superior, and why its technical superiority warrants paying a price premium.[4] As a result, we sustain Nexant’s protest that the agency failed to reasonably explain its basis for selecting Deloitte’s higher-rated, higher-cost proposal.

Finally, Nexant contends that AID’s source selection decision does not support the award to Deloitte because it was not a contemporaneously-prepared document. In this connection, as noted, the record shows that the agency’s FMN--which is the only document produced by the agency that embodies its source selection decision--was executed on October 31, some 49 days after the SSA states she made the decision to award the contract to Deloitte, 30 days after the actual award was made, and 9 days after Nexant filed its protest with our Office. The protester maintains that we should therefore accord little weight to the FMN because it was prepared in the heat of litigation.

Among the most basic requirements in a negotiated best value acquisition is the requirement that the agency adequately document its source selection decision. FAR § 15.308 expressly provides:

The source selection decision shall be documented, and the documentation shall include the rationale for any business judgments and tradeoffs made or relied on by the SSA, including benefits associated with additional costs.

While the FAR itself is silent on the question of when this documentation should be prepared, our Office has spoken on numerous occasions about the probity of information presented or prepared after the source selection decision has been made. Most notably, for example, in our decision in Boeing Sikorsky Aircraft Support, B-277263.2, B-277263.3, Sept. 29, 1997, 97-2 CPD ¶ 91 at 15, we stated:

While we consider the entire record, including statements and arguments made in response to a protest in determining whether an agency’s selection decision is supportable, we accord greater weight to contemporaneous source selection materials rather than judgments, such as the selection officials’ reevaluation here, made in response to protest contentions. [citations omitted]. As pointed out above, the agency does not acknowledge that it erred. Rather, we are faced with an agency’s efforts to defend, in the face of a bid protest, its prior source selection through submission of new analyses, which the agency itself views as merely hypothetical and which are based on information that the agency continues to argue is not accurate. The lesser weight that we accord these post-protest documents reflects the concern that, because they constitute reevaluations and redeterminations prepared in the heat of an adversarial process, they may not represent the fair and considered judgment of the agency, which is a prerequisite of a rational evaluation and source selection process.

We recognize that the situation we faced in Boeing Sikorsky is not completely analogous to the situation here. Of significance, the agency’s source selection decision in the Boeing Sikorsky case had been memorialized contemporaneously; the question we were considering was what weight to give to a new, revised source selection decision--prepared during the course of the protest--that the agency maintained demonstrated that the protester had not been prejudiced.

Here, we are faced with the unusual circumstance of there being no contemporaneous source selection document. Instead, the agency offers only a source selection document prepared well after the selection decision was made, and after Nexant filed its protest. While AID points out that the underlying technical and cost evaluations were completed and documented prior to the award and the protest, the rationale for the SSA’s selection of Deloitte’s higher-rated, higher-cost proposal for award was not documented until after Nexant’s protest was filed.

Because this document was prepared after the agency had the benefit of having reviewed Nexant’s protest allegations,[5] the protester expresses a reasonable concern that the document may not represent the agency’s fair and considered judgment about the selection, and instead was crafted in response to the protest. To answer this concern would require our Office to consider and decide what weight should be given to a selection decision prepared after the agency receives the protest.[6] However, since we otherwise conclude, as set forth above, that both the selection decision, and the underlying assessments on which it relies, are unreasonable, we need not answer at this time the question of the appropriate weight to be given the post-protest FMN.

RECOMMENDATION

We recommend that the agency reopen the acquisition and engage in meaningful discussions with the offerors, consistent with the discussion above. At the conclusion of those discussions, we recommend that the agency obtain and evaluate revised proposals, and make a new--contemporaneous--source selection decision that adequately documents the comparative merits of the proposals and the agency’s rationale for its award decision. Should the agency conclude that an offeror other than Deloitte has submitted the proposal that represents the best value to the government, we recommend that the agency terminate the contract awarded to Deloitte and make award to the successful offeror, if otherwise proper. Finally, we recommend that the agency reimburse Nexant the costs associated with filing and pursuing its protest, including reasonable attorneys’ fees. Bid Protest Regulations, 4 C.F.R. § 21.8(d) (2012). The protester’s certified claim for costs, detailing the time expanded and costs incurred, must be submitted to the agency within 60 days after receipt of this decision. Id. § 21.8(f).

The protest is sustained.

Susan A. Poling
General Counsel