VariQ Corporation; Octo Consulting Group, Inc.

B-417135,B-417135.2,B-417135.3,B-417135.4,B-417135.5: Mar 18, 2019

Additional Materials:

Contact:

Ralph O. White
(202) 512-8278
WhiteRO@gao.gov

Kenneth E. Patton
(202) 512-8205
PattonK@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

VariQ Corporation, of Washington, D.C., and Octo Consulting Group, Inc. (Octo), of Reston, Virginia, protest the issuance of a task order to Slalom LLC, of Tysons Corner, Virginia, under request for quotations (RFQ) No. 70SBUR18Q00000248, issued by the Department of Homeland Security, U.S. Citizenship and Immigration Services (USCIS), for services to support USCIS Information Technology (IT) system delivery by performing cloud replacement of legacy systems involving heavy customer engagement.

We sustain in part and deny in part VariQ's protest, and dismiss Octo's protest.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: VariQ Corporation; Octo Consulting Group, Inc.

File: B-417135; B-417135.2; B-417135.3; B-417135.4; B-417135.5

Date: March 18, 2019

Scott M. McCaleb, Esq., Tracye W. Howard, Esq., Colin J. Cloherty, Esq., and Martha G. Vázquez, Esq., Wiley Rein LLP, for VariQ Corporation; and Daniel R. Forman, Esq., Christian N. Curran, Esq., Lauren H. Williams, Esq., and Puja Satiani, Esq., Crowell & Moring LLP, for Octo Consulting Group, Inc., the protesters.
Amy C. Hoang, Esq., Erica L. Bakies, Esq., Christopher A. Vandervoort, Esq., and Stuart B. Nibley, Esq., K&L Gates LLP, for Slalom, LLC, the intervenor.
John Cornell, Esq., Department of Homeland Security, for the agency.
Lois Hanshaw, Esq., and Christina Sklarew, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

1. Protest challenging the agency's evaluation of key personnel in Step 1 of a two-step evaluation is sustained, where the record shows that the agency unreasonably concluded that a proposed key person's resume met the solicitation's minimum qualifications based solely on the "TEC's [technical evaluation committee] experience."

2. Protest of the agency's evaluation and award decision in Step 2 of the evaluation is dismissed as academic based on the agency's improper evaluation of key personnel in the Step 1 evaluation.

DECISION

VariQ Corporation, of Washington, D.C., and Octo Consulting Group, Inc. (Octo), of Reston, Virginia, protest the issuance of a task order to Slalom LLC, of Tysons Corner, Virginia, under request for quotations (RFQ) No. 70SBUR18Q00000248, issued by the Department of Homeland Security, U.S. Citizenship and Immigration Services (USCIS), for services to support USCIS Information Technology (IT) system delivery by performing cloud replacement of legacy systems involving heavy customer engagement.

We sustain in part and deny in part VariQ's protest, and dismiss Octo's protest.

BACKGROUND

On June 27, 2018, the RFQ was issued to holders of General Services Administration Schedule 70 Special Item Number 132-51 contracts, in accordance with the ordering procedures of Federal Acquisition Regulation (FAR) § 8.405.1 Agency Report (AR), Tab 1, RFQ at 1. The RFQ's performance work statement (PWS) provides that the solicitation effort referred to as the verification future (VER Future) project, at issue here, will consist of teams to provide development, security, and operations (DevSecOps) services through a large-scale enterprise cloud solution. RFQ, PWS at 37. The RFQ contemplated the award of a fixed-price task order. RFQ at 23.2 Quotations were due by July 11. Id. at 25.

The RFQ identified four evaluation factors: relevant experience, staffing approach, technical demonstration, and price. Id. at 26-30. The non-price factors would be assigned adjectival ratings, which were, from highest to lowest, good, satisfactory, marginal, and unsatisfactory. Id. at 35. As relevant here, the RFQ established that an unsatisfactory rating reflected a quotation that failed to meet requirements and contained one or more deficiencies for which correction would require a major revision or redirection of the quotation, and would be considered ineligible for award. Id.

The RFQ contemplated a two-step approach to evaluating quotations. Id. at 34. In Step 1, the agency would consider relevant experience and staffing approach, and whether offered prices were fair and reasonable, to determine which firms would be invited to participate in the technical demonstration in Step 2. Id. In this evaluation, relevant experience would be more important than staffing. Id. The agency would select the three quotations with the highest technical ratings and fair and reasonable prices to proceed to the technical demonstration in Step 2. Id. The RFQ advised firms that a tradeoff analysis would not be conducted in Step 1. Id.

In Step 2, relevant experience and staffing approach would not be considered. Id. Rather, the agency would invite firms to perform a technical demonstration to enable the agency to assess the firm's ability to successfully accomplish a programming assignment that would be deployed and demonstrated in the cloud. Id. at 31, 34. The technical demonstration would be significantly more important than price. Id. at 34. The agency would conduct a best-value tradeoff determination between the technical demonstration and price. Id.

Firms not advancing to Step 2 would not be notified or debriefed after the Step 1 evaluation was completed. Id. Only the three firms found to be the highest technically rated after the Step 1 evaluation would be notified of their selection to proceed to Step 2. Id. All firms eliminated under either Step 1 or Step 2 would be notified of their elimination after award was made under Step 2. Id.

Thirteen vendors submitted quotations, including Slalom, VariQ, and Octo. Memorandum of Law (MOL) at 2. The final evaluation results, as relevant here, are as follows:

  Step 1 Step 2
  Relevant Experience Staffing approach Price Technical Demonstration Total Evaluated Price3
Slalom Good4 Satisfactory5 Fair and reasonable Good $99,278,904
MetroStar Good Satisfactory Fair and reasonable Marginal $97,640,274
Octo Good Satisfactory Fair and reasonable Satisfactory $93,231,554
VariQ Satisfactory Satisfactory Fair and reasonable N/A $92,606,544


VariQ AR, Tab 8, Down-Select Decision, at 5-6; Octo AR, Tab 9, Source Selection Decision (SSD), at 7.

Following the Step 1 evaluation, the three quotations that were rated unsatisfactory under the relevant experience or staffing approach factors were not invited to participate in Step 2. AR, Tab 8, Down Select Decision, at 9. The agency selected the quotations of MetroStar, Octo, and Slalom as technically superior with fair and reasonable prices and invited these firms to participate in Step 2. Id. at 12. After the technical demonstrations were performed and evaluated in Step 2, Slalom's quotation was selected as offering the best value. Octo AR, Tab 9, SSD, at 16. On November 13, the agency notified firms of award. These timely protests followed.

DISCUSSION

The protesters challenge the agency's evaluation and award in Step 1 and Step 2 of the procurement.6 VariQ contends that the agency improperly evaluated quotations under the relevant experience and staffing approach factors and engaged in disparate treatment in its evaluation of quotations. Octo challenges the agency's evaluation of quotations in Step 2 of the procurement under the technical demonstration factor and also asserts that the agency engaged in disparate treatment in its evaluation of quotations. We sustain VariQ's challenge to the agency's Step 1 evaluation of Octo's key personnel under the staffing approach factor. Based on our conclusion that the Step 1 evaluation was flawed, we dismiss Octo's challenges to the agency's Step 2 evaluation. We do not find any other independent basis for sustaining the protests.7

In reviewing protests of an agency's evaluation and source selection decision under FAR subpart 8.4, our Office will not reevaluate quotations; rather, we review the record to determine whether the evaluation and source selection decision are reasonable and consistent with the solicitation's evaluation criteria, and applicable procurement laws and regulations. DEI Consulting, B-401258, July 13, 2009, 2009 CPD ¶ 151 at 2. While we will not substitute our judgment for that of the agency, we will sustain a protest where the agency's conclusions are inconsistent with the solicitation's evaluation criteria, undocumented, or not reasonably based. See e.g., Solers, Inc., B-404032.3, B-24032.4, Apr. 6, 2011, 2011 CPD ¶ 83 at 17.

Evaluation of Octo's Key Personnel

In the Step 1 evaluation, for the staffing approach, the RFQ required firms to provide resumes for three key personnel, including, as relevant here, a DevSecOps Architect Senior Level (architect). RFQ, PWS at 37, 46. For each key personnel position, the resume was required to identify the name, the position the candidate was proposed to fill, and the proposed candidate's qualifications and experience in order to validate that the proposed candidate met or exceeded the minimum experience and qualifications for the associated position. RFQ at 28. In this regard, the architect was required to have a minimum of 10 years of experience in the IT field, focusing on development projects, DevSecOps, and technical architecture, of which 3 years was required to be in architecture and design, deploying enterprise applications in cloud platforms. RFQ, PWS at 46. Additionally, the PWS required the proposed architect to have either a bachelor's degree in computer science, IT management, or engineering; or, in lieu of a degree, to demonstrate two additional years, for a total of 12 years of experience in the IT field focusing on development projects, DevSecOps, and technical architecture. Id.

The staffing approach would be evaluated with regard to how well the vendor was expected to successfully perform the PWS requirements. RFQ at 33. As relevant here, the key personnel resumes would be evaluated on a pass/fail basis, based on whether the proposed key person met the required qualifications and experience identified in the PWS. Id. The RFQ advised that the government could, at its discretion, request additional details to support the candidate's experience claims. Id. The RFQ also stated that the failure to provide suitable key personnel would result in a deficiency and make the quotation ineligible for award. Id.

The record shows that the resume for Octo's proposed architect did not identify a bachelor's degree in computer science, IT management, or engineering. AR, Tab 13, Octo Quotation, at 10. The resume included a summary profile outlining the proposed architect's various qualifications. Id. Additionally, under a section entitled "Required Experience," the resume stated, "[DELETED] years of experience in the IT field focusing on development projects, DevSecOps, and technical architecture, of which [DELETED] years was in architecture and design, deploying enterprise applications in cloud platforms using Amazon Web Services (AWS)." Id. The resume then listed, under a section identified as "Professional Experience Highlights," eight positions dating from his current employment in 2018 back to 2005. Id. Three of the eight positions provided the name of the company, the name of the position, the dates of employment, and a brief description of work. Id. The remaining positions--identifying only the name of the company, the name of the position, and the dates of employment--included a position in which the proposed architect had previously been employed as a [DELETED] with [DELETED] from [DELETED] 2005 to [DELETED] 2007. Id.

In evaluating the proposed architect's experience, the technical evaluation committee (TEC) recognized that the resume did not show the required bachelor's degree and therefore required two additional years of experience. AR, Tab 6, TEC Report, at 21. In this regard, the report stated that the two additional years of IT experience were reflected on the resume as a [DELETED] for [DELETED]. Id. The report further stated that "in the TEC's experience," this role required the individual to attend meetings with customers to obtain requirements and determine suitability of the software for the customer, and "in some cases," to demonstrate the product, which is technical in nature. Id. The TEC also opined that a [DELETED] would be expected to be able to answer technical questions and present technical findings to the customer. Id. The TEC concluded, based on its "firsthand experience" with [DELETED] software and its sales cycle and knowledge of the tool and how it integrates in the operations of the DevSecOps team, that the experience as a [DELETED] met the requirement for two years of additional experience in the IT field, focusing on development projects, DevSecOps, and technical architecture. Id.

VariQ contends that the agency's conclusions, based on the TEC's experience rather than the information in Octo's quotation, were unreasonable. VariQ Comments and Supp. Protest at 5. In this regard, VariQ asserts that the TEC's conclusions--that the [DELETED] had supported technical architecture and understood how a specialized monitoring tool like [DELETED] integrates in the operations of a DevSecOps team--amount to unsupported assumptions and leaps of logic, because none of the alleged experience was included in Octo's quotation. Id.

In response, the agency contends that the TEC chair's conclusions, based on his personal experience with [DELETED], were reasonable and that this conclusion is consistent with the contracting officer's guidance to technical evaluators that if the resume contained language reflecting the RFQ requirement, then it was deemed to meet the requirement. VariQ Supp. MOL at 8.

We find the agency's evaluation unreasonable. The RFQ required the resume to identify qualifications and experience necessary to meet the minimum experience and qualifications for the associated position, and warned that the failure to provide a suitable key person would result in a deficiency. RFQ at 28, 33. In Octo's quotation, the resume provided no details for the [DELETED] position--and the [DELETED] other positions held prior to 2014--other than the name of the company, the position, and the dates of employment. AR, Tab 13, Octo Quotation, at 10. Nevertheless, the agency not only outlined the duties and expectations for the [DELETED] position, but also concluded, based on the same agency-identified experience, that the candidate met the specific requirement to show two years of experience in the IT field focusing on development projects, DevSecOps, and technical architecture, basing this conclusion on the TEC's personal experience. In this regard, the agency's conclusions were not based on information identified in or supported by the quotation.

To the extent the agency contends that it was reasonable for the evaluators to rely on their own experience to supply information not provided in the resume, we disagree. In this regard, the TET chair failed to identify any personal knowledge of the proposed candidate's experience or knowledge on which the chair's conclusion could be based. On these facts--where neither the protester's quotation, nor the agency's unsupported statements demonstrated personal knowledge of the experience that Octo's proposed candidate possessed--we conclude that the agency unreasonably found that Octo's key person met the minimum experience and qualifications requirements of the RFQ, and sustain this basis of VariQ's protest. See The OMO Grp., Inc., B-294328, Oct. 19, 2004, 2004 CPD ¶ 212 at 7 (agency's evaluation reasonably relied on an evaluator's personal knowledge in concluding a proposed key person's prior experience in the Navy met experience requirements where the candidate's experience was well known to the Navy and Department of Defense; the candidate was proposed by other offerors and found to meet the experience requirements; and the proposed candidate's declaration supported agency's conclusions); cf. VariQ Corp., B-414650.11, B-414650.15, May 30, 2018, 2018 CPD ¶ 199 at 6 (evaluation record did not contain sufficient information to establish that proposed candidate met specific experience requirements and evaluator's post-protest explanations not only failed to show personal knowledge of the candidate's experience with the solicitation requirements, but also contained other unsupported statements regarding the proposed candidate's experience and knowledge that amounted to "educated guesswork").

Evaluation of VariQ's Quotation

VariQ also raises various challenges to the evaluation of its own quotation and alleges that the agency treated vendors unequally, none of which provides a basis to sustain the protest.

For example, VariQ challenges a weakness assessed to its quotation under subfactor 3 of the relevant experience factor. VariQ Protest at 8. In this regard, the relevant experience factor required firms to submit examples of experience performing development and testing using agile and DevSecOps approaches. RFQ at 27. Offerors were instructed to do so by addressing seven separate subfactors, including, as relevant here, the firm's usage of innovative solutions to overcome challenges in modernizing a tightly coupled system to a microservice architecture using open source technologies. Id. at 27-28. When evaluating VariQ's experience in providing innovative solutions to overcome challenges, the agency essentially decided that the protester's example of developing a [DELETED] to determine features for the "[DELETED]" was not innovative, but common in the development of applications. AR, Tab 6, TEC Report, at 33. As a result, the agency assigned VariQ a weakness for this subfactor.

Although the protester contends that the agency ignored information in its proposal when assigning this weakness, in our view, the record is adequately documented and shows that the agency considered this information. The protester's disagreement with the agency's conclusions does not show that the evaluation was unreasonable.8

Additionally, the protester argues that the evaluation demonstrates unequal treatment between its quotation and those deemed technically superior. VariQ Comments and Supp. Protest at 7-11. VariQ argues that in at least five instances, the agency assigned strengths to the other firms' quotations where, in VariQ's view, the agency should have assigned similar strengths to VariQ's quotation. Id.

We find no merit to this aspect of VariQ's protest. Where a protester alleges unequal treatment in a technical evaluation, it must show that the differences in ratings did not stem from differences between the offerors' proposals. Red River Comput. Co., B-414183.4 et al. June 2, 2017, 2017 CPD ¶ 157 at 6. VariQ's protest fails to make this showing.

For example, VariQ argues that in evaluating its response to subfactor 6 of the relevant experience factor (which required vendors to show experience in integrating information security as part of the DevSecOps process, in particular to the AWS cloud environment), the evaluators assigned a strength only to Slalom's quotation even though VariQ's experience was "essentially the same." VariQ Comments and Supp. Protest at 10. Specifically, the protester contends that it should have received a strength because its approach to "[DELETED]" used nearly the same language as Slalom's approach in which "[DELETED]." Id. at 9 (citing AR, Tab 4, VariQ Quotation, at 8 and Tab 5, Slalom Quotation, at 6).

The record shows that Slalom's experience received a strength because "[DELETED]" was viewed as an approach that would "help[] to maintain security model around an agile governance process and [allow for a quick response] to user requirements." VariQ AR, Tab 6, TEC Report, at 28. Thus, the record demonstrates that it was not the similar "[DELETED]" language or [DELETED] that formed the basis for the assignment of the strengths, but rather the awardee's specific approach, which provides a reasonable basis for the difference in the assignment of a strength to Slalom's quotation, but not VariQ's. As a result, we deny this aspect of VariQ's protest.

Octo's Protest

Octo challenges the agency's evaluation and award in Step 2 of the procurement. We dismiss these allegations.

As relevant here, the RFQ advised firms that the failure to provide suitable key personnel would result in a deficiency and make a quotation ineligible for award. RFQ at 33. The RFQ also stated that at the conclusion of the Step 1 evaluation, the three most highly qualified quotations would advance to Step 2. Octo's quotation was found to be among the most highly qualified. However, as explained above, we concluded that the agency's evaluation of Octo's proposed architect was unreasonable. In our view, the agency improperly rated Octo's staffing approach as satisfactory; accordingly, it appears that Octo's quotation should not have been included as one of the most highly rated quotations that advanced to Step 2.

Thus, based on our above conclusions regarding the agency's Step 1 evaluation, Octo's challenges to the agency's Step 2 evaluation and source selection are effectively rendered moot. General Revenue Corp., et al., B-414220.2 et al., Mar. 27, 2017, 2017 CPD ¶ 106 at 11 (challenges to the source selection decision are rendered moot where GAO concludes that an agency improperly evaluated proposals and recommends that an agency conduct a new evaluation). Our Office will not consider a protest where the issue presented has no practical consequences with regard to an existing federal government procurement, and thus is of purely academic interest. We will not render to a protester what would be, in effect, an advisory decision. Ferris Optical, B-403012.2, B-403012.3 Oct. 21, 2010, 2010 CPD ¶ 265 at 2. Accordingly, Octo's protest is dismissed.

Prejudice

Competitive prejudice is an essential element of a viable protest and we will sustain a protest only where the protester demonstrates that, but for the agency's improper actions, it would have had a substantial chance of receiving the award. Deloitte Consulting, LLP et al., B-411884 et al., Nov. 16, 2015, 2016 CPD ¶ 2 at 18-19. Here, because we find that the agency's evaluation of Octo's key personnel under Step 1 was not reasonable, and because the record does not show how a proper evaluation would have affected the ranking of the vendors' quotations, we conclude that VariQ was prejudiced by the agency's evaluation.

Recommendation

We recommend that the agency reevaluate quotations in a manner consistent with the terms of the solicitation and our decision (or, alternatively, ask for additional details to support experience claims, as provided by the RFQ), conduct a new Step 1 evaluation, and prepare a new source selection decision based on that reevaluation. We also recommend that the agency reimburse VariQ its reasonable costs of filing and pursuing its protest, including reasonable attorneys' fees. 4 C.F.R. § 21.8(d)(1). VariQ's certified claim for costs, detailing the time spent and the cost incurred, must be submitted to the agency within 60 days after receipt of this decision. 4 C.F.R. § 21.8(f).

The protest is sustained in part, denied in part, and dismissed in part.

Thomas H. Armstrong
General Counsel

 


[1] The RFQ was initially issued on June 19. All references herein are to the conformed version of the RFQ.

[2] The agency produced separate reports for each protest. Citations in this decision to the central exhibits are to the AR common to both protests; citations to exhibits unique to a particular agency report identify the specific AR--for example, VariQ AR. Additionally, the ARs contain Bates numbering for individual tabs. Accordingly, our citations to pages in the record are based upon those numbers.

[3] The agency determined the total evaluated price in Step 1. VariQ AR, Tab 8, Down-Select Decision, at 6. The price here excludes pricing for FAR clause 52.217-8, Option to Extend Services.

[4] A good rating was assigned to a quotation that demonstrated a good understanding of the requirements and an approach that exceeded performance or capability standards. RFQ at 35. Additionally, the quotation was viewed as possessing one or more strengths that would benefit the government and demonstrating a low risk of unsuccessful performance. Id.

[5] A satisfactory rating was assigned to a quotation that demonstrated an understanding of the requirements and an approach that met performance or capability standards. Additionally, the quotation was viewed as presenting an acceptable solution, which may or may not include strength(s) and demonstrating a moderate risk of unsuccessful performance. Id.

[6] On December 17, prior to the filing of the agency report, we advised the parties that we anticipated dismissing two of VariQ's arguments; first, that the agency failed in its evaluation of the relevant experience factor to credit VariQ for its alleged significant relevant experience as compared to Slalom, and, second, that the agency, in its evaluation of the staffing factor, ignored whether Slalom's personnel would be able to perform within 60 days of contract award. The first challenge fails to state a valid basis of protest under our Bid Protest Regulations, 4 C.F.R. § 21.5(f), because the RFQ did not contemplate comparison of one vendor's responses against another's, but rather, an evaluation of how well a contractor would perform based on its record of relevant experience. See RFQ at 31. The second challenge is not for consideration by our Office because whether an awardee can perform after award is matter of contract administration. See Knight Point Sys. LLC, B-416602, B-416602.2, Oct. 26, 2018, 2018 CPD ¶ 371 at 6.

[7] VariQ acknowledges as untimely its contention that the agency violated FAR
§ 8.405-2(d), which requires an ordering activity to place the task order with the schedule contractor representing the best value, by allegedly failing to consider VariQ's lower price in its Step 1 evaluation; accordingly, VariQ requests that we invoke the significant issue exception to our timeliness rules under § 21.2(c) of our Bid Protest Regulations. See VariQ Protest at 7 n.3. We generally regard a significant issue as one of widespread interest to the procurement community that has not been previously decided. Cyberdata Techs., Inc., B-406692, Aug. 8, 2012, 2012 CPD ¶230 at 3-4. The protester has not made such a showing here. See Cyberdata Techs., supra.

[8] VariQ also contends that the RFQ contained a latent ambiguity because the term "innovative" is "amorphous, especially in the IT industry" and because the RFQ did not define what the agency considered to be innovative. VariQ Protest at 9 n.4. A patent ambiguity exists where the solicitation contains an obvious, gross, or glaring error, while a latent ambiguity is more subtle. Odyssey Sys. Consulting Grp., Ltd., B-412519,
B-412519.2, Mar. 11, 2016, 2016 CPD ¶ 86 at 5. A patent ambiguity must be protested prior to the next closing time for the submission of proposals in order to be considered timely. Id.; 4 C.F.R. § 21.2(a)(1). To the extent VariQ is arguing that the term "innovative" was ambiguous, by its own arguments, VariQ should have been aware that any alleged ambiguity was patent, and needed to be protested prior to closing. Because the protester failed to timely challenge the amended RFP, this allegation is untimely and will not be considered further. GCC Techs., LLC, B-416459.2, Nov. 19, 2018; 2018 CPD ¶ 394 at 4.