TOTE Services, Inc.

Decision

Matter of:  TOTE Services, Inc.

File:  B-414295; B-414295.2

Date:  April 25, 2017

DIGEST

1.  Protest is denied in part where the agency’s evaluation of the past technical performance of the awardee reasonably considered two affiliates that would be meaningfully involved in the performance of the awarded contract.

2.  Protest challenging an agency’s evaluation of past technical performance is sustained in part where the agency failed to adequately document its evaluation and where the agency credited offerors for relevant performance without considering the quality of that performance experience, and credited offerors for positive performance without considering the relevance of that performance.

TOTE Services, Inc., of Jacksonville, Florida, protests the award of a contract to U.S. Marine Management, Inc. (USMMI), of Norfolk, Virginia, by the Department of the Navy, Military Sealift Command (MSC) under request for proposals (RFP) No. N32205-16-R-3101, for operation and maintenance of the agency’s Sea Band X-Based Radar (SBX-1) vessel.  TOTE challenges the agency’s evaluation of past technical performance and argues that the agency’s best-value tradeoff was flawed. 

We sustain the protest in part and deny it in part.

BACKGROUND

On April 13, 2016, RFP No.N32205-16-R-3101 was issued seeking qualified vessel officers and crew, operational and technical support, equipment, tools, provisions, and supplies to operate, maintain, and repair MSC’s SBX-1 vessel.  The RFP envisioned the issuance of a fixed-price contract with reimbursable elements for a 1-year base period with four 1-year option periods.

The solicitation called for the agency to conduct a best-value tradeoff analysis comparing two equally-weighted evaluation factors:  past technical performance and price.  RFP at 175-176.  Additionally, to be considered for award, an offeror needed to be rated acceptable under the RFP’s technical, small business participation, and solicitation compliance factors.  Id. at 175.  

Under the past technical performance factor, the agency would assess the offeror’s past performance to evaluate its probability of meeting the solicitation requirements.  Id. at 177.  To do this, the RFP stated that the agency would first evaluate the relevancy of performance references submitted in the offeror’s proposal as well as the relevancy of past performance obtained through other means (e.g., through the Past Performance Information Retrieval System (PPIRS) or through interviews).  Id. at 177.  After conducting the relevancy analysis, the agency would next evaluate the quality of this past performance through a review of questionnaire responses and information available in the Contractor Performance Assessment Reporting System (CPARS).  Id.  In addition, the agency reserved the right to contact references for verification or additional information and the right to consider “other relevant information.”  Id. at 178.  

Section L of the solicitation requested that offerors’ past technical performance narrative address four elements:  operations and manning, lifecycle management, planned maintenance, and management of repairs.  Id. at 172.  These four elements were divided further into a total of 10 subelements.  Id.  While the solicitation did not provide a definition for overall relevance, it did provide relevancy definitions specific to each subelement.  RFP, Attach. N.  The RFP also directed offerors to provide contract references to address each subelement.  RFP, Attach. K.   

The RFP stated that the agency would calculate an overall confidence rating “based on the amount of experience, relevancy of that experience to the current requirement and quality of [the] offeror’s past performance.”  Id. at 177.  As relevant here, a confidence rating of substantial confidence was defined as “[b]ased on the offeror’s recent/relevant performance record, the Government has a high expectation that the offeror will successfully perform the required effort.”  Id.

The Navy received seven offers in response to the solicitation, including offers from TOTE and USMMI.  Following final proposal revisions, the agency rated TOTE and USMMI as follows:

	TOTE	USMMI
Past Technical Performance	Substantial Confidence	Substantial Confidence
Technical	Acceptable	Acceptable
Small Business Participation	Acceptable	Acceptable
Solicitation Compliance	Acceptable	Acceptable
Total Evaluated Price	$39,303,590	$32,411,299

Agency Report (AR), Tab 13, Business Clearance Memorandum, at 6.

Following a review of the agency’s past technical performance evaluation and the written recommendation of the source selection advisory council, the source selection authority (SSA) concluded that an award to USMMI represented the best value to the agency.  AR, Tab 4, Source Selection Decision Document (SSDD), at AR0196.  In support of this conclusion, the SSA found that “[n]one of the other offerors presented any non-cost advantage over USMMI to justify paying a premium price.”  Id. at AR0195.  With regard to TOTE’s proposal, the SSA recognized that TOTE was the incumbent contractor for the SBX-1, and therefore had very relevant past technical performance.  Id. at AR0196.  The SSA noted, however, that TOTE’s proposal was 21.27 percent higher than the lowest-priced offeror and that USMMI had demonstrated significant relevant past technical performance under similar operations and maintenance contracts.  Id. at AR0196.

On January 12, 2017, the agency awarded the subject contract to USMMI.  This protest followed.

DISCUSSION

TOTE raises numerous challenges to the agency’s evaluation of USMMI’s past technical performance.  TOTE argues that the agency (1) improperly credited USMMI with the past performance of affiliated companies, (2) improperly gave USMMI credit for favorable quality ratings without evaluating the relevance of the underlying contract, (3) improperly gave USMMI credit for relevant past performance for which no quality of performance ratings were received, and (4) misevaluated the relevance of the only reference that USMMI provided pertaining to its own performance.[1]  As explained further below, we find that the agency’s evaluation of past technical performance was unreasonable and therefore sustain the protest in part.  

The solicitation’s evaluation scheme anticipated that the agency would determine the relevancy and quality of performance of each contract reference for each subelement, rather than determining an overall relevance and quality score for each contract.  RFP, Attach. N.  To accomplish this, offerors were provided with questionnaires that contained space allowing references to rate a contractor’s performance under categories that aligned with each of the RFP’s subelements.  RFP, Attach. O.  At the same time, offerors were instructed to identify in their proposals, for each subelement, contracts that were relevant to that subelement.  RFP at 171-172. 

In its proposal, USMMI identified 14 contracts to demonstrate its relevant past performance.  Of these 14, USMMI performed one contract as the prime contractor; [DELETED] performed 12 contracts; and an affiliated company [DELETED][2] performed one contract.  Comments at 4.  To assess USMMI’s past technical performance, the agency collected all of USMMI’s contract subelement relevance and rating scores to determine a range of scores, which the agency then used to determine the applicable subelement confidence assessment rating.  See AR, Tab 8, Past Technical Performance Report.  Based on these compiled ranges of scores, the Navy assigned USMMI a substantial confidence rating under each of the 10 subelements.  See id.  Because of these 10 substantial confidence subelement ratings, the agency assigned USMMI an overall confidence assessment of substantial confidence.  See id. at AR0237.

The protester asserts that the agency improperly credited USMMI with the past technical performance of [DELETED] and [DELETED].  Specifically, TOTE argues that the agency’s evaluation does not reflect that [DELETED] and [DELETED] would be meaningfully involved in USMMI’s proposed contract performance.  Additionally, the protester asserts that there is no evidence that the SSA considered this issue in reaching the agency’s source selection decision.

The evaluation of an offeror’s past performance is within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based evaluation ratings.  MFM Lamey Group, LLC, B-402377, Mar. 25, 2010, 2010 CPD ¶ 81 at 10.  Where a protester challenges an agency’s evaluation of past performance, we will review the evaluation to determine if it was reasonable and consistent with the solicitation’s evaluation criteria and procurement statutes and regulations, and to ensure that it is adequately documented.  Veteran Technologists Corp., B-413614.3, B-413614.5, Nov. 29, 2016, 2016 CPD ¶ 341 at 6.  In addition, an agency properly may attribute the past performance of a parent or affiliated company to an offeror where the firm’s proposal demonstrates that the resources of the parent or affiliate will affect the performance of the offeror.  Perini/Jones, Joint Venture, B-285906, Nov. 1, 2000, 2002 CPD ¶ 68 at 4.  The relevant consideration is whether the resources of the parent or affiliated company--its workforce, management, facilities or other resources--will be provided or relied upon for contract performance such that the parent or affiliate will have meaningful involvement in contract performance.  Ecompex, Inc., B-292865.4 et al., June 18, 2004, 2004 CPD ¶ 149 at 5.  

The crux of the protester’s argument is that the agency unreasonably considered the past technical performance of USMMI’s affiliates and that the contemporaneous evaluation record does not demonstrate that the agency considered the involvement of these affiliates before doing so.  Based on our review of the record, we conclude that the agency’s crediting of contracts performed by USMMI’s affiliates was reasonable in light of both affiliates’ meaningful involvement in USMMI’s proposed contract approach.  In this regard, USMMI’s proposal reflects that both [DELETED] and [DELETED] would provide significant resources to, and have extensive involvement in, any contract awarded to USMMI. 

For instance, with regard to [DELETED], USMMI proposed to leverage experience and expertise in [DELETED].  AR, Tab 11, USMMI Proposal Vol. 2, at AR0265.  With regard to [DELETED], USMMI’s proposal included a cover letter on [DELETED] letterhead, which was signed by [DELETED] corporate officers, and also expressly represented that an “award to USMMI provides access to the worldwide resources and capabilities of [DELETED] and The Maersk Group, the world’s largest shipping company.”  Id. at AR0278; AR, Tab 26, USMMI Cover Letter, at AR0499.  Additionally, USMMI’s proposal was replete with references to the resources [DELETED] would bring to the contract, including (1) a personnel database [DELETED], (2) [DELETED]’s budgeting, purchasing and accounting system, (3) [DELETED]’s emergency response, crisis management plan and business continuity plan, (4) [DELETED]’s small business subcontracting plan, and (5) a certificate of protection and indemnity insurance under which [DELETED] was designated as the group principal.  See AR, Tab 11, USMMI Proposal Vol. 2, at AR0267; AR, Tab 21, USMMI Insurance Certificate, at AR0475; AR, Tab 22, USMMI Proposal Vol. 1, at AR0479; AR, Tab 24, USMMI Proposal Vol. 1, at AR0489. 

In addition, the agency provided two declarations by members of the past technical performance evaluation team.  In their declarations, both evaluators represented that, following their consideration of USMMI’s proposal, they concluded that both [DELETED] and [DELETED] would have meaningful involvement in the performance of the SBX-1 contract.  See AR, Tabs 27 & 28, Evaluators’ Decls.  In determining the rationality of an agency’s evaluation and award decision, we do not limit our review to contemporaneous evidence, but consider all the information provided, including post-protest explanations that fill in previously unrecorded details, so long as the explanations are credible and consistent with the contemporaneous record.  See NCI Info. Sys., Inc., B-412680, B-412680.2, May 5, 2016, 2016 CPD ¶ 125 at 11 n.4.  In light of the significant contract involvement  proposed for both [DELETED] and [DELETED], we conclude that the agency’s decision to evaluate the past technical performance of both affiliates was reasonable. 

TOTE also challenges the agency’s consideration of questionnaire responses and relevancy determinations in its subelement confidence assessment.  In this regard, when the agency received completed questionnaire responses, it found that some of the references included ratings on subelements for which the offeror had not identified that contract.  For USMMI, the agency received ratings on 17 subelements for which USMMI had not identified the contract.  For these questionnaire responses, the agency did not perform an assessment of the relevancy of the particular contract to the applicable subelement.  Conversely, in 20 instances, the agency did not receive a questionnaire rating for subelements, despite the fact that USMMI had identified that particular contract for that subelement.  In both cases, the agency credited the orphan relevancy or rating scores in its confidence assessment, despite missing the other half of the score.  TOTE argues that this was unreasonable. 

The agency responded to this protest ground by arguing that the 17 orphan questionnaire ratings were only considered as further support for USMMI’s confidence ratings, and that this consideration did not impact USMMI’s substantial confidence rating because USMMI “received high ratings from its other contracts submitted, including Very Good ratings under the [DELETED] contract.[[3]]”  Supp. Memorandum of Law (MOL) at 10.  With respect to the 20 orphan relevancy ratings, the agency argued that, although it had not received questionnaire ratings on these contracts, it “properly relied on staff’s general and direct knowledge of the various contracts cited by USMMI for [past technical performance].”  Id. at 11.  In addition, the agency argued that, even removing consideration of these relevance ratings from the evaluation, USMMI’s overall confidence rating would have remained unchanged.  Id. at 11.

Our Office will question an agency’s past performance evaluation where the record indicates that the agency either failed to evaluate, or otherwise unreasonably considered, the relevance of past performance references in accordance with the solicitation’s stated evaluation criteria.  Arctic Slope Mission Servs., LLC,  B-410992.5, B-410992.6, Jan. 8, 2016, 2016 CPD ¶ 39 at 12.  Additionally, in reviewing an agency’s evaluation of proposals, we consider not only whether the agency deviated from the RFP evaluation criteria, but also whether the actual evaluation was unreasonable.  American Dev. Corp., B-251876.4, July 12, 1993, 93-2 CPD ¶ 49 at 10 (finding agency evaluation that “divorced the contract performance rating from the relevancy rating” to be unreasonable).

Based on our review of the record, we conclude that the agency’s past technical performance evaluation was unreasonable.  The RFP here provided that the agency would “first evaluate the relevancy of the past performance references submitted in the offerors past technical performance proposal as well as obtained through other means (PPIRS, interviews, etc.)” and would “next evaluate the quality of this past performance through a review of questionnaire responses and information available in CPARS.”  RFP at 177.  The agency would then calculate the overall confidence rating based on “the amount of experience, relevancy of that experience to the current requirement and quality of [the] offeror’s past performance.”  Id.  Of note, the solicitation defined a substantial confidence rating as “[b]ased on the offeror’s recent/relevant performance record, the Government has a high expectation that the offeror will successfully perform the required effort.”  Id.  Clearly, the RFP envisioned that the agency would make both relevancy and quality determinations for the past performances references submitted in USMMI’s proposal, and that the agency’s overall confidence assessment rating would be an integrated assessment based on these determinations. 

Here, however, the agency failed to make the dual determination envisioned by the RFP.  Instead, the agency credited the orphan questionnaire ratings without examining the relevance of such ratings, and conversely credited the orphan relevance determinations without receiving information about the quality of USMMI’s performance.  All of these relevancy and quality ratings were then included in a general range of scores, and the agency determined subelement confidence ratings based on whether the range included high relevance or quality scores.  The net effect of this was to distort the agency’s evaluation because it added quality ratings of unknown relevance and relevancy determinations of unknown quality.  This was inconsistent with the evaluation scheme, which anticipated that the agency would assess both the relevance and the quality of each past performance reference, and that the confidence assessment would examine the offeror’s recent/relevant performance to predict whether the offeror could successfully perform the required effort.

For instance, with regard to the subelement relating to lifecycle management experience providing maintenance and repair,[4] the agency’s evaluation was as follows:

2. Lifecycle Management
a. Experience, in the last five years, in executing lifecycle management programs to ensure:
i. Adequate and timely maintenance and repair.

USMMI provided eight contracts as evidence of its experience for element 2(a)(i). USMMI demonstrated [it] has Relevant and Very Relevant experience for element 2(a)(i). See attachment A for the relevancy evaluation ratings and summary.  Questionnaires were received from seven respondents that addressed element 2(a)(i). USMMI received questionnaire ratings ranging from Satisfactory to Very Good.

Based on an evaluation of the offeror’s recent/relevant performance record for this element, USMMI received a rating of SUBSTANTIAL CONFIDENCE for this element as the Government has a high expectation that the offeror will successfully perform the required effort.

AR, Tab 8, Past Technical Performance Report, at AR0235.  However, of the eight contracts noted above that were evaluated for relevancy, the agency only received three questionnaire scores.  See AR, Tab 10, Past Technical Performance Report, at AR0248.  Similarly, of the seven questionnaires received by the agency,[5] the agency only evaluated the relevance of three of the contracts.  Compare id. with AR, Tab 9, Past Technical Performance Report, at AR0243.  Removing these orphans, as well as the undocumented contract reference noted below, from the agency’s assessment, would result in only two contract references where the agency possessed both relevance and questionnaire ratings.  Both of these references were found to be relevant and satisfactory.  Accordingly, had the agency’s evaluation considered only these two contracts, rather than the numerous references quoted above, the agency may have reached a different conclusion with regard to its subelement confidence assessment.[6]   

TOTE next argues that the agency improperly determined that the only submitted contract on which USMMI was the prime contractor, the [DELETED] contract, was very relevant under five of the past technical performance subelements.[7]  In this regard, the solicitation defined “very relevant” under each subelement as having the same experience as that called for under the solicitation’s scope of work.  For instance, a determination of very relevant under subelement 2(a)(i) was defined as follows:

Element	Evaluation Criteria
2. Lifecycle Management a. Experience, in the last five years, in executing lifecycle management programs to ensure: i. Adequate and timely maintenance and repair -	Very Relevant - Experience in the last 5 years with a maintenance planning system to ensure adequate and timely maintenance and repair on systems the same to those on SBX-1 (Drop down azimuthing thrusters, diesel engines, dynamic positioning systems, and MODU/CSDU ballast control systems).

RFP, Attach. N.  For each of the five subelements in question, USMMI’s proposal narrative submitted the [DELETED] contract for each subelement but did not specifically explain how the contract met the specific relevancy criteria.  See AR, Tab 11, USMMI Vol. 2, at AR0264-276.  For instance, for subelement 2(a)(i), USMMI’s proposal narrative contained a general discussion of its maintenance experience, but did not contain any explanation of how the [DELETED] contract involved maintenance and repair on systems that were the same as those on the SBX-1.  See id. at AR0269. 

In response to this argument, the agency concedes that three of the subelement relevancy ratings were incorrect and should have been rated “relevant” rather than “very relevant,” though the agency contends these errors were harmless.[8]  The agency further asserts that the five subelement relevancy determinations were made based on the personal knowledge of the agency’s evaluators, who had general and direct knowledge of the [DELETED] contract and were therefore familiar with its systems. Supp. MOL at 9. 

While, as a general matter, the evaluation of an offeror’s past performance is within the discretion of the contracting agency, we will question an agency’s evaluation of past performance where it is unreasonable or undocumented.  Solers, Inc.,  B-404032.3, B-404032.4, Apr. 6, 2011, 2011 CPD ¶ 83 at 8.  Although an agency is not required to retain every document generated during its evaluation of proposals, the agency’s evaluation must be sufficiently documented to allow our Office to review the merits of a protest.  Apptis, Inc., B-299457 et al., May 23, 2007, 2008 CPD ¶ 49 at 10.  Where an agency fails to document or retain evaluation materials, it bears the risk that there may not be an adequate supporting rationale in the record for us to conclude that the agency had a reasonable basis for its source selection decision.  TriCenturion, Inc.; SafeGuard Services, LLC, B-406032 et al., Jan. 25, 2012, 2012 CPD ¶ 52 at 13. 

Here, we find the agency’s evaluation was not sufficiently documented to allow us to determine whether the agency’s conclusions were reasonable.  In this regard, while the agency asserted that its evaluators relied on their personal knowledge of the  [DELETED] contract, this assertion is not supported in the contemporaneous record and does not explain the agency’s admitted mistake with respect to three of the five subelement relevancy determinations.  Moreover, for the five subelements in question, the contemporaneous record does not contain any information regarding the systems in place on the [DELETED] vessels, or how such systems compare to the corresponding system on the SBX-1 vessel. 

In addition, the agency’s explanation that it relied on its evaluators’ knowledge is inconsistent with the past technical performance evaluation report.  The report points to USMMI’s proposal narrative as the source of the agency’s relevancy determinations; the report does not state that such relevancy determinations were made based on the evaluators’ personal knowledge of the contract.  See AR, Tab 9, Past Technical Performance Chart, at AR0242-245.  Accordingly, we find that the contemporaneous record does not contain sufficient documentation for us to determine the reasonableness of the agency’s relevancy determinations with respect to the five subelements in question.[9]

Competitive Prejudice

As discussed above, we find that the Navy’s past technical performance evaluation was inconsistent with the RFP’s evaluation criteria.  In addition, we find that the agency failed to adequately document its use and reliance on the personal knowledge of its evaluators.  As a result, the agency’s conclusions were not adequately supported in the record, and, in fact, the agency has conceded that they contained at least some errors.

In all, the agency evaluated 14 total USMMI/[DELETED]/[DELETED] contracts across 10 separate subelements.  For these 14 contracts, the Navy evaluated relevance and/or received questionnaire responses for a total of 76 subelements.  Of these 76 subelements, however, at least 42 contained one of the above flaws.  And, on the only contract on which USMMI was listed as a prime contractor, six of the 10 subelements contained flaws.  Removing these 42 subelement scores would remove either the highest-rated or most-relevant contract from four of the 10 subelements, and would remove the majority of scores from six of the 10 subelements.  In addition, seven of the 10 subelements would no longer have an identified contract on which USMMI received both a very relevant determination and a very good or excellent questionnaire score. 

While the agency argues that a majority of the subelements would still have a range of scores that contain both very relevant ratings and very good or excellent questionnaire scores (albeit for different contracts), such an analysis, based on the broad range of scores, does not provide a meaningful understanding of USMMI’s past performance because it ignores the fact that, in some instances, very relevant contracts received only satisfactory performance scores, and the fact that, in some instances, excellent-rated performance was relevant--as opposed to highly  relevant--to the instant scope of work.  In short, merely examining the range of scores across all of the contracts fails to fulfill the agency’s obligation under the RFP to qualitatively assess both the relevance and the quality of USMMI’s past performance for each contract, and to base its overall confidence assessment on those determinations. 

In light of the pervasiveness of these errors, and the fact that they substantially affect a majority of the subelements, we cannot conclude that TOTE has not suffered competitive prejudice.  In this regard, correction of the errors might result in USMMI receiving a different past technical performance rating; it might also affect the SSA’s conclusion that the strength that distinguishes USMMI’s proposal is its “steady history of positive performance on relevant contracts.”  AR, Tab 4, SSDD, at AR0196.  We also note with regard to the issue of prejudice that although the agency made similar errors in its evaluation of TOTE’s past technical performance, TOTE nonetheless received very good or excellent ratings for its experience on the incumbent contract, which the agency judged to be very relevant for each of the 10 subelements.  The record thus fails to demonstrate that correction of the errors identified above would have negatively affected TOTE’s past technical performance rating in the same way that it would USMMI’s rating.[10]

RECOMMENDATION

We recommend that the agency perform a past technical performance evaluation consistent with the requirements of the solicitation, and then rely on that evaluation as part of its source selection determination.  If, upon reevaluation of proposals, the proposal of an offeror other than USMMI is determined to offer the best value to the government, the Navy should terminate USMMI’s contract for the convenience of the government and award a contract to the selected offeror.  We also recommend that TOTE be reimbursed the costs of filing and pursuing this protest, including reasonable attorneys’ fees.  4 C.F.R. § 21.8(d)(1).  The protester’s certified claim for costs detailing the time expended and the costs incurred must be submitted to the agency within 60 days after receipt of this decision.  4 C.F.R. § 21.8(f).

The protest is sustained in part and denied in part.

Susan A. Poling
General Counsel

---