Management Systems International, Inc.

B-409415,B-409415.2: Apr 2, 2014

Additional Materials:


Ralph O. White
(202) 512-8278

Kenneth E. Patton
(202) 512-8205


Office of Public Affairs
(202) 512-4800

Management Systems International (MSI), of Washington, D.C., protests the award of a contract to International Resources Group (IRG), of Washington, D.C., by the U.S. Agency for International Development (USAID) under request for proposals (RFP) No. SOL-OAA-12-000092, issued to obtain world-wide training services for agency employees. MSI challenges the agency's technical evaluation of proposals, price-technical tradeoff, and best value decision. The protester also alleges that the agency engaged in unequal discussions and disparately evaluated the final proposal revisions.

We deny the protest.

The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.


Matter of: Management Systems International, Inc.

File: B-409415; B-409415.2

Date: April 2, 2014

Robert Nichols, Esq., Anuj Vohra, Esq., and Kayleigh Scalzo, Esq., Covington & Burling, LLP, for the protester.
Kevin C. Dwyer, Esq., Daniel E. Chudd, Esq., James A. Tucker, Esq., and Charles L. Capito, Esq., Jenner & Block LLP, for the intervenor.
Scott Risner, Esq., United States Agency for International Development, for the agency.
Robert T. Wu, Esq., and Tania Calhoun, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.


1. Protest is denied where the agency reasonably justified paying a price premium for a technically superior proposal in a best value procurement where the solicitation authorized a price/technical tradeoff.

2. Use of an undisclosed point-weighting evaluation methodology is permissible where the solicitation notified offerors of percentage-weighting of the various technical factors, and the point-weighting corresponds with the percentage-weighting.


Management Systems International (MSI), of Washington, D.C., protests the award of a contract to International Resources Group (IRG), of Washington, D.C., by the U.S. Agency for International Development (USAID) under request for proposals (RFP) No. SOL-OAA-12-000092, issued to obtain world-wide training services for agency employees. MSI challenges the agency’s technical evaluation of proposals, price-technical tradeoff, and best value decision. The protester also alleges that the agency engaged in unequal discussions and disparately evaluated the final proposal revisions.

We deny the protest.


The RFP, issued on August 29, 2012, sought innovative proposals to provide services in support of agency-wide core and non-core training for its personnel. RFP, Cover Letter at 1. The RFP contemplated award of an indefinite-quantity contract comprised of one base year and up to four option years, with task orders issued on fixed-price, time-and-material, or labor-hour bases. RFP at 3. Award was to be made utilizing best value tradeoff procedures, considering both technical merit and price. Id. at 116-17.

Overall technical merit was to be assessed using the following weighted evaluation factors: technical plan (50 percent), management team approach and qualifications (20 percent), corporate staffing plan (20 percent), and past performance (10 percent). Id. at 117-18. Each technical factor included various elements that would be evaluated under the factor; the RFP did not disclose relative weights for each element. Id. Price was to be evaluated for both realism and reasonableness. Id. at 119. Total prices for the base period and all option years were to be added together in order to arrive at a total evaluated price. Id. The technical factors, when combined, were to be significantly more important than price.[1] Id. at 116. Finally, the RFP instructed offerors that the contracting officer might award to a higher-priced offeror if a determination was made that the higher technical evaluation of that offeror merited the additional cost/price. Id. at 120.

Six proposals were timely received and evaluated by the agency, including proposals from MSI and IRG. Agency Report (AR), Exh. 9, Technical Evaluation Memorandum, at 1. A Technical Evaluation Committee (TEC) was convened to conduct initial technical evaluations of the proposals, resulting in individual and consensus evaluation scores and narratives for each proposal.[2] The methodology used by the agency in assigning point scores was to give each technical factor a point total commensurate with the percentage-weighting stated in the RFP. Then, internal to each technical factor, the point scores assigned to each element equaled the total point score for that factor.[3] Id. at 5-7. The individual technical evaluators scored each technical factor and element contained in the factor, for a maximum score of 100 points for each proposal. A final consensus meeting was then held with the TEC and the contracting officer to discuss the findings, and to prepare final consensus scores and narratives. Id. at 4.

The agency established a competitive range consisting of three offerors, including MSI and IRG. AR, Exh. 13, Evaluation of Final Proposal Revisions, at 1. Discussion letters were sent to all offerors in the competitive range,[4] and final proposal revisions were received from each offeror. MSI’s final evaluated price submitted with its final proposal revisions was $29,399,542.00, and IRG’s was $33,937,736.00. Id. at 35.

The TEC was reconvened to evaluate revised technical proposals, resulting in a final technical consensus report. The original consensus score assigned to IRG’s technical proposal was 93.66 points, which was upwardly revised to 94.16 points after evaluation of final proposal revisions. Id. at 3. The upward revision resulted from IRG addressing an identified weakness, which resulted in removal of the weakness and the TEC’s decision to increase IRG’s point score by 0.5 points for the technical plan factor. Id. at 5. The record also shows that IRG addressed a separate weakness under the technical plan factor, resulting in the removal of that weakness. The TEC did not increase IRG’s score in this regard, explaining that addressing the weakness did not result in a “recognized strength, but rather is just meeting the contract requirement.”[5] Id. at 7.

The original consensus score assigned to MSI’s technical proposal was 88.83 points, which remained the same after evaluation of final proposal revisions. Id. at 3. The TEC informed MSI of two weaknesses during discussions, one in its technical plan, and the other in the firm’s management team approach and qualifications. Id. at 16-17. MSI addressed both weaknesses in its final proposal revision. However, the agency determined that the corrections did not result in strengths, and, therefore, did not increase MSI’s technical point score. Id.

The contracting officer adopted the TEC’s findings in rendering his award decision. AR, Exh. 14, Memorandum of Negotiation, at 14-15. He found that IRG’s proposal contained the most significant strengths and provided the best approach to ensure outstanding contract performance. Id. He determined that, despite offering a strong proposal, MSI’s proposal did not provide the best value, since the strengths noted in IRG’s proposal outweighed the potential cost savings to the agency found in MSI’s proposal. Id. at 36.

In this regard, the contracting officer noted that IRG’s technical proposal repeatedly offered multiple state-of-the-art training approaches, which were worth the additional cost of $4,538,194 over MSI’s proposal. Id. For example, he noted that the training methodologies, such as [DELETED] and [DELETED], described in IRG’s proposal were an excellent approach that would help the agency achieve greater learning and results from its training investment. Id. The contracting officer also found that IRG’s approach to ensure training improves the job performance of employees was a significant strength, and was an essential agency goal. He recognized that the [DELETED] proposed by IRG was of the highest caliber, and was complimentary of the core functions each [DELETED]. Finally, the contracting officer observed that the key personnel offered by IRG exceeded the RFP’s requirements, and that IRG had the corporate capacity and ability to provide trainers who could fully meet and exceed the requirements. Id.

The agency ultimately found that IRG’s proposal offered the best value to the government, and that award to IRG based on its superior technical proposal warranted payment of a price premium. Id. at 38. MSI was notified of the agency’s award decision on December 20, and this protest followed.


MSI raises various objections to the agency’s evaluation of proposals.[6] First, MSI challenges the agency’s evaluation methodology as being inconsistent with the solicitation’s terms. Comments at 10-18. Next, MSI argues that the TEC consensus scores upon which the contracting officer relied are irrational because they are inconsistent with the scores assigned by the individual TEC evaluators. Id. at 19-25. Third, MSI argues that the agency’s best value determination was flawed because each discriminator used to justify the price premium to be paid for IRG’s proposal were also present in MSI’s proposal. Id. at 25-30. For the reasons discussed below, we deny the protest.

Evaluation Methodology

MSI challenges the agency’s assignment of points to each technical evaluation factor as being contrary to the terms of the solicitation. Comments at 10-18. In this regard, the protester argues that the agency’s assignment of point totals to each of the technical factors was inconsistent with the percentage-weighting for these factors identified in the RFP. We disagree.

In reviewing a protest challenging an agency’s evaluation, our Office will not reevaluate proposals, nor substitute our judgment for that of the agency, as the evaluation of proposals is a matter within the agency’s discretion. Analytical Innovative Solutions, LLC, B- 408727, Nov. 6, 2013, 2013 CPD ¶ 263 at 2. Rather, we will review the record only to determine whether the agency’s evaluation was reasonable and consistent with the stated evaluation criteria and with applicable procurement statutes and regulations. Id. In resolving MSI’s allegation, here, we note that agencies are not required to inform offerors of the specific rating methodology utilized for evaluation of proposals. Arcus Properties, LLC, B- 406189, Mar. 7, 2012, 2012 CPD ¶ 107 at 6. However, the particular method of proposal evaluation utilized must provide a rational basis for source selection and be consistent with the evaluation criteria set forth in the solicitation. Id.

The RFP informed offerors that the technical factors would be weighted according to stated percentages, as discussed above. RFP at 117-18. The agency effectuated the RFP’s requirements by assigning points to each of the technical factors commensurate with the weighting percentage stated in the RFP. Thus, for example, the technical plan, which was to receive a 50 percent weight per the terms of the RFP, was assigned 50 total points during the agency’s evaluation. Likewise, the agency assigned 10 total points to the past performance factor, which was to receive a 10 percent weight according to the RFP. RFP at 117-18; AR, Exh. 9, Technical Evaluation Memorandum, at 5-6.

In challenging the agency’s evaluation, MSI argues that once the agency assigned a raw technical score to each factor, it was then required to apply the RFP’s weighting factor. Had it done so, MSI argues that the properly weighted differential between the two offerors’ technical proposals would have been significantly less,[7] and, according to the protester, “[a] proper comparison of the two proposals would have recognized their near identity, thus obliterating any reasonable justification for paying a more-than $4.5 million premium for IRG’s proposal.” Comments at 18.

We find that the agency’s evaluation methodology is consistent with the RFP’s weighting scheme, as its use of a total point methodology had the effect of recognizing the percentage weights stated in the RFP. For example, the effect of assigning the technical plan a total point allocation of 50 points out of 100 points for the entire technical factor has the same impact as weighting the technical plan as 50 percent of the entire technical factor. Under both the point and percentage methodologies, the technical plan comprised half of the entire technical evaluation factor. As a result, the agency’s evaluation in this regard is unobjectionable.

Technical Evaluation Scores

MSI next argues that the contracting officer’s best value decision was flawed because “the final TEC consensus scores upon which he relied are inconsistent with the final scores assigned by the individual TEC members.” Comments at 19. MSI also asserts that, because the record contains no explanation of why the scores changed between the individual evaluation worksheets and the final consensus report, the protest must be sustained for failure to adequately document the source selection decision. Id. Our review of the record affords us no basis to question the agency’s evaluation.

It is not unusual for individual evaluator ratings to differ significantly from one another, or from the consensus ratings eventually assigned; indeed, the reconciling of such differences among evaluators’ viewpoints is the ultimate purpose of a consensus evaluation. J5 Systems, Inc., B- 406800, Aug. 31, 2012, 2012 CPD ¶ 252 at 10. Our overriding concern is not whether an agency’s final ratings are consistent with preliminary ratings, but whether they reasonably reflect the relative merits of the proposals, consistent with the solicitation. Id. Further, our Office has consistently held that numerical point scores are useful only as guides for intelligent decision-making and are not generally controlling for award because they often reflect the disparate, subjective judgments of the evaluators. National Medical Seminars Tempharmacists, B-233452, Feb. 22, 1989, 89-1 CPD ¶ 191 at 2.

We are unaware of any requirement that every individual evaluator’s scoring sheet track the final evaluation report, or that the evaluation record document the various changes in evaluators’ viewpoints. J5 Systems, Inc., supra at 13 n.15. More importantly, our review of the record does not lead us to conclude that the agency’s evaluation was objectionable. The alleged inconsistencies upon which the protester asks us to sustain this protest amount to nothing more than quibbling with the minutia of the agency’s scoring of proposals.[8] The probative question here, however, is not whether the agency’s point scores were off by one or two points, but whether the agency properly justified paying a price premium for IRG’s technically superior solution. We next turn to MSI’s contention in this regard.

Best Value Decision

MSI contends that the agency’s best value decision was flawed because the specific aspects of IRG’s technical proposal that served as discriminators between the two proposals were also found in MSI’s proposal. Comments at 25. In making this argument, MSI provides a comparison between comments made in the technical evaluation worksheets about both offeror’s proposals. The agency responds that MSI’s reliance on “superficial similarities” between the proposals cannot overcome its reasoned determination that IRG’s proposal offered significant advantages relative to MSI’s. Agency Response to Comments at 5.

In support of its position, the agency relies on both the contemporaneous record and a supplemental declaration from the contracting officer expanding on this issue. See generally Contracting Officer’s Supplemental Declaration. MSI argues that this supplemental declaration amounts to nothing more than post-hoc rationalization, and, relying on our Office’s decision in Boeing Sikorsky Aircraft Support, B‑277263.2, B-277263.3, Sept. 29, 1997, 97-2 CPD ¶ 91, argues the statement should be given little weight. Protester’s Response at 8-9. MSI’s reliance on Boeing is misplaced.

Our decision in Boeing addresses the concern that post-protest documents that constitute reevaluations and redeterminations prepared in the heat of an adversarial process may not represent the fair and considered judgment of the agency. Boeing, supra at 11. Boeing is irrelevant in situations such as here, where the agency offers post-protest explanations that provide a detailed rationale for contemporaneous conclusions and simply fill in previously unrecorded details. Such explanations will generally be considered in our review of the rationality of selection decisions, so long as those explanations are credible and consistent with the contemporaneous record.[9] IBM Global Business Service-U.S. Federal, B- 409029, B- 409029.2, Jan. 27, 2014, 2014 CPD ¶ 43 at 9; Desbuild Inc., B- 409009, Jan. 6, 2014, 2014 CPD ¶ 23 at 7 n.11; Worldwide Information Network Systems, Inc., B- 408548, Nov. 1, 2013, 2013 CPD ¶ 254 at 4 n.1.

As to the substance of MSI’s protest ground, we have reviewed the record and conclude that the agency has provided support for its determination that IRG’s technical proposal was superior to the solution offered by MSI. While MSI challenges each of the several discriminators upon which the agency based its determination that IRG’s proposal was technically superior, we address here only one such discriminator as an example.

In the award decision, the contracting officer recognized IRG’s proposed training methodologies as a strength. AR, Exh. 14, Memorandum of Negotiation, at 35. In this regard, the contracting officer noted, “[IRG’s] training methodologies, such as [DELETED] and [DELETED], were an excellent approach that will help the agency to achieve greater learning and results from its training investment.” Id. In challenging the agency’s recognition of this discriminator, MSI points to its own evaluation where the agency stated that MSI proposed “continuing learning through [DELETED], and using [DELETED]. . . .” Comments at 26. MSI argues that this comment evidences the equivalence between the two proposals. Id. Thus, MSI contends that the agency improperly found IRG’s technical proposal to be superior in this regard. Id.

The contracting officer responds that, in his own review of proposals, he found IRG’s approach stronger than MSI’s. He states that, for example, “IRG’s proposal presented a stronger use of [DELETED], which is a technique for the [DELETED]. MSI’s proposal briefly addresses [DELETED], but only says that trainers will use [DELETED], without saying how they will [DELETED].” Contracting Officer’s Supplemental Declaration at 2. Thus, the contracting officer determined that IRG’s proposal was stronger not only because of the solution offered, but because of the level of detail and specificity in describing its solution. Id. We conclude that his determination was reasonable.

We are satisfied, in considering the parties’ arguments and the record before us, that the agency’s best value decision to award a contract to IRG, even at a price premium, is rationally-based and supported by the record. Further, it is apparent from the record that, while the agency utilized the challenged point scoring methodology as a tool in its evaluation of proposals, the best value decision was based on a comparison of the merits of each proposal. This comparison resulted in a tradeoff between IRG’s technically superior, higher-priced proposal, and MSI’s strong, but technically inferior, lower-priced proposal. Such a tradeoff is commended to the broad discretion afforded to the agency, to which we find no basis to object.[10]

The protest is denied.

Susan A. Poling
General Counsel

[1] The RFP states elsewhere that the technical factors, when combined, are more important than price. Id. at 117. The parties do not address this discrepancy and we do not view it as impacting this decision.

[2] Narratives were generated for the various elements under each technical factor. In these narratives, the TEC discussed significant strengths, strengths, weaknesses, significant weaknesses and deficiencies identified in each proposal.

[3] For example, the technical plan factor was assigned a weight of 50 percent in the RFP. RFP at 117. The TEC assigned this factor 50 points, and assigned points to each of the six elements to be considered under this factor, which then totaled 50 points. AR, Exh. 9, Technical Evaluation Memorandum, at 5-6. The sum total of the four technical factors was 100 possible points.

[4] The discussion letters for IRG and MSI identified different issues for each firm. AR, Exh. 10, Request for Final Proposal Revision.

[5] MSI argues that this reflects a disparate evaluation of the final proposal revisions. However, we find reasonable the agency’s explanation that it increased IRG’s technical score after final proposal revisions because its revision introduced a new strength to its proposal, but that MSI’s revisions only addressed weaknesses which, in the agency’s discretion, did not warrant a point increase. Contracting Officer’s Supplemental Declaration at 6.

[6] Our decision only discusses the more significant issues raised by MSI, but we have considered all of the firm’s allegations and find them to be without merit.

[7] For example, with respect to the evaluation of proposals under the technical plan factor, which carried a weight of 50 percent, MSI asserts that while IRG received a raw score of 47 and MSI received a raw score of 45, applying the weighting factor to each would have resulted in IRG having a score of 48.5 (50-(3*0.5)) and MSI would have a weighted score of 47.5 (50-(5*0.5)). Comments at 16. Thus, the weighted differential between proposals for this factor would have been 1.0 point, instead of the 2.0 point differential utilized by the agency in its evaluation.

[8] MSI argues that an average of the individual evaluator’s scores would have led to a point differential between IRG and MSI of 4.62 points, instead of the assigned differential of 5.33 points. Comments at 22. Using MSI’s formula for weighting the point differential between proposals, the overall weighted score differential would have been 1.13 points, instead of 1.62 points (on a 100 point scale).

[9] That is not to say that an agency may properly rectify an otherwise inadequately documented evaluation through submission of post-protest explanations; a situation the protester alleges has occurred here. An agency bears the risk that there may not be adequate supporting rationale in the record for us to conclude that it had a reasonable basis for its source selection decision. Apptis, Inc., B- 299457 et al., May 23, 2007, 2008 CPD ¶ 49 at 7. Here, however, we find that the source selection decision is adequately documented, and the agency’s post-protest submissions are credible and consistent with the contemporaneous record.

[10] MSI also challenges the agency’s reliance on technical point scores arguing that any perceived difference between the proposals was the “result of false precision.” Comments at 30-36. As we conclude that the agency’s award decision was reasonable, this protest ground is denied.

Mar 20, 2018

Mar 19, 2018

  • Ampcus, Inc.
    We deny the protest.
  • AMAR Health IT, LLC
    We dismiss the protest because our Office does not have jurisdiction to entertain protests of task orders issued under civilian agency multiple-award, indefinite-delivery, indefinite-quantity (IDIQ) contracts that are valued at less than $10 million.
  • Centurum, Inc.--Costs
    We grant the request.

Mar 15, 2018

  • ORBIS Sibro, Inc.
    We sustain the protest in part and deny it in part.

Mar 14, 2018

Mar 13, 2018

  • Interoperability Clearinghouse
    We dismiss the protest because the protester, a not-for-profit entity, is not an interested party to challenge this sole-source award to an Alaska Native Corporation under the Small Business Administration's (SBA) 8(a) program.
  • Yang Enterprises, Inc.
    We dismiss the protest.

Looking for more? Browse all our products here