VivSoft Technologies, LLC

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: VivSoft Technologies, LLC

File: B-421561.15; B-421561.17

Date: April 11, 2024

DIGEST

1. Protest challenging portions of the agency’s evaluation of proposals is dismissed as untimely where the protester could have and should have raised the protest grounds during the prior protest before corrective action.

2. Protest challenging agency’s technical evaluation is denied where the agency evaluated the proposals in accordance with the terms of the solicitation.

VivSoft Technologies, LLC, a small business located in Brambleton, Virginia, protests the decision of the Department of Health and Human Services, Centers for Medicare and Medicaid Services (CMS), not to select its quotation for the establishment of a blanket purchase agreement (BPA) under request for quotations (RFQ) No. 75FCMC21Q0013, for agile collaboration and modernization endeavors (ACME). The RFQ sought to establish multiple BPAs with vendors holding contracts under General Services Administration Multiple Award Schedule special item number (SIN) 54151S, for information technology professional services. The protester challenges the agency’s evaluation and award decisions.

We dismiss in part, and deny in part, the protest.

BACKGROUND

On June 21, 2021, the agency issued the RFQ to vendors holding contracts under General Services Administration Multiple Award Schedule special item number (SIN) 54151S, for information technology professional services, using the procedures of Federal Acquisition Regulation (FAR) subpart 8.4. Agency Report (AR), Tab 3A, RFQ at 1, 3. The solicitation was set aside for small businesses and anticipated the establishment of multiple BPAs contemplating the issuance of fixed-priced, time-and-material, and labor-hour call orders for a 5-year ordering period.[1] Id. Award was to be made on a best-value tradeoff basis, considering price and the following four non‑price factors, of equal importance: design demonstration; development, security, and operations (DevSecOps) case study; technical challenge; and corporate capabilities.[2] Id. at 21. The solicitation provided that all evaluation factors other than price, when combined, are significantly more important than price. Id. at 20. As relevant to our discussion below, the RFQ provided that once the agency identified “the best-suited [vendors] (i.e., the apparent successful contractors),” the agency may “communicate with only those contractors to address any remaining issues,” including “technical and price.” Id. at 6.

Initial Evaluation and Award Decision

The procurement was conducted in three phases using an advisory “down select” methodology. Contracting Officer’s Statement (COS) at 2. The agency received 63 phase I quotations by the July 2021 due date; 36 phase II quotations by the March 16, 2022 due date; and 22 phase III quotations by the August 15, 2022 due date. Id. After evaluation by a technical evaluation panel (TEP), the contracting officer, who was also the source selection authority, conducted a comparative assessment of the quotations and performed a best-value tradeoff, finding that there were eight quotations that provided the best value to the government. AR, Tab 23, Post Negotiation Memorandum (PNM) at 10.

Consistent with the guidance above, the agency held exchanges and received quotation revisions from the eight vendors identified as the “best-suited.” Supp. COS at 1. The agency’s exchanges were limited to two issues: section 508 compliance, and concerns regarding labor category mapping that the agency identified during a labor category mapping exercise it performed on the quotations of the eight apparent successful vendors after completion of the best-value tradeoff. AR, Tab 21A, Revised Source Selection Decision (SSD) at 3; Supp. COS at 1. After considering the quotation revisions submitted by the eight apparent successful contractors, the contracting officer confirmed his selection decision that these eight vendors provided the best value to the government. AR, Tab 21A, Revised SSD at 3-4.

In March of 2023, the agency established BPAs for the solicited requirement with the eight vendors that had been identified as the apparent successful contractors.[3] Memorandum of Law (MOL) at 4. Five firms, including VivSoft, filed protests with our Office challenging the propriety of the agency’s actions. Id. Those protests raised a variety of challenges to the agency’s evaluation of quotations; the agency’s conduct during exchanges; and the reasonableness of the agency’s source selection decision. After record development in June 2023, CMS advised that it was taking corrective action in response to VivSoft’s protest to reevaluate quotations and make a new award decision.[4] Our Office dismissed the protests, including VivSoft’s, as academic.[5] VivSoft Techs., LLC, B‑421561.1, B-421561.6, June 27, 2023 (unpublished decision).

Corrective Action, Reevaluation, and New Award Decision

The agency’s corrective action involved a reevaluation under the technical evaluation factors.[6] MOL at 4; Supp. COS at 2-4, 8. The agency’s evaluation included the assignment of positives and negatives under each evaluation factor. As pertinent to this protest, the agency’s reevaluation resulted in the agency removing two negatives assigned to VivSoft under the design demonstration factor and assessing an additional positive to VivSoft’s quotation under the DevSecOps case study factor. Following the agency’s reevaluation, the agency assigned VivSoft and the eight vendors that had been identified as the apparent successful contractors the following ratings:[7]

Vendor	Design Demonstration	DevSecOps Case Study	Technical Challenge	Corporate Capabilities	Price
Bellese	High Confidence	High Confidence	High Confidence	High Confidence	$98,731,573
Flexion	Moderate Confidence	High Confidence	High Confidence	High Confidence	$97,943,150
Softrams	Moderate Confidence	High Confidence	High Confidence	High Confidence	$85,500,990
Coforma	High Confidence	High Confidence	High Confidence	Moderate Confidence	$115,257,393
Dynanet	High Confidence	High Confidence	Low Confidence	High Confidence	$94,897,286
Nava	High Confidence	High Confidence	Moderate Confidence	High Confidence	$109,154,358
Octo	Moderate Confidence	Moderate Confidence	High Confidence	High Confidence	$84,438,113
Oddball	High Confidence	Moderate Confidence	High Confidence	Moderate Confidence	$100,098,998
VivSoft	Moderate Confidence	Moderate Confidence	High Confidence	Moderate Confidence	$89,990,166

 

AR, Tab 21A, Revised SSD at 1-2; COS at 8.

After evaluating quotations, CMS again determined that the quotations submitted by the eight apparent successful contractors provided the best value to the government. AR, Tab 21A, Revised SSD at 3-4. Based on the reevaluation, the agency selected the same eight vendors for the establishment of BPAs. Id. On December 26, 2023, the agency notified VivSoft that its quotation had not been selected for establishment of a BPA. AR, Tab 25A, Unsuccessful Vendor Notification. After receiving a brief explanation on December 28, VivSoft filed a protest with our Office on January 5, 2024, and a supplemental protest on February 15, 2024. AR, Tab 25B, Brief Explanation.

DISCUSSION

VivSoft challenges numerous aspects of the agency’s evaluation of quotations. The protester first contends that its quotation is entitled to additional positive aspects under each of the four non-price evaluation factors. Next, VivSoft alleges the agency engaged in disparate treatment under all four factors because it failed to give VivSoft equal credit for aspects of its quotation, which, the protester asserts, were substantially identical to elements of the awardees’ quotations for which the awardees received positive evaluation findings. VivSoft further contends that two of the awardees are ineligible for award due to a lapse in the vendors’ System for Award Management (SAM) registrations. In addition, the protester claims that the agency unreasonably assessed its quotation with negative findings under three of the evaluation factors based on unstated evaluation criteria. Finally, VivSoft argues that the agency conducted unequal discussions and failed to conduct a reasonable best-value tradeoff. For the reasons discussed below, we dismiss most of the first two arguments--concerning the failure to assess additional positive aspects to VivSoft’s quotation and disparate treatment--as untimely. We dismiss the third argument, regarding the requirement for continuous SAM registration, for failing to state a valid basis of protest. We deny the remaining allegations.[8]

Untimely Evaluation Challenges

As a preliminary matter, VivSoft raises a number of issues in its current protest that were clearly apparent from the record developed in response to the firm’s earlier protest but were not raised in the earlier protest proceedings. As explained below, we dismiss these arguments as untimely.

Our Bid Protest Regulations contain strict rules for the timely submission of protests. These timeliness rules reflect the dual requirements of giving parties a fair opportunity to present their cases and resolving protests expeditiously without disrupting or delaying the procurement process. Verizon Bus. Network Servs., Inc., B-419271.5 et al., Apr. 26, 2021, 2021 CPD ¶ 191 at 14. Protest arguments raised after corrective action and re-award of a contract are untimely when the information underpinning such arguments was available to the protester as part of its earlier protest, and the protester failed to raise these arguments in a timely manner. Id. at 15; Loyal Source Gov’t Servs., LLC, B-407791.5, Apr. 9, 2014, 2014 CPD ¶ 127 at 5.

As noted above, in the current protest Vivsoft argues that it is entitled to additional positive aspects under each of the four non-price evaluation factors. See Protest at 16‑35. With the exception of arguments pertaining to two positive aspects under factor 2 (DevSecOps case study) and additional positives under factor 4 (corporate capabilities) which we discuss further below, however, the protester did not raise these allegations in connection with its earlier protest even though Vivsoft knew, or should have known, that its quotation was not credited with those positive aspects.[9]

Similarly, VivSoft raises an argument in the current supplemental protest that the agency engaged in disparate treatment under all four evaluation factors because it failed to give VivSoft equal credit for aspects of its quotation, which the protester asserts, were substantially identical to elements of the awardees’ quotations for which the awardees received positives. Comments & Supp. Protest at 11-33. Except for allegations regarding disparate treatment under the corporate capabilities factor, however, VivSoft did not raise any of these allegations in connection with its earlier protest, despite that VivSoft was aware, or should have been aware, based on the evaluation record provided during the prior protest that the awardees’ quotations received credit for these attributes, but that VivSoft’s quotation did not.[10]

In general terms, VivSoft argues that because the agency conducted a full reevaluation and made a new award decision, these issues are timely filed. Supp. Comments at 12‑14. We disagree. After the initial March 2023 award decision, CMS provided VivSoft with an explanation of its evaluation findings that was nearly identical to the explanation provided to VivSoft after the current award decision. Despite receiving the explanation of the evaluation findings, VivSoft did not raise its challenge to these conclusions until the instant protest, which was filed more than seven months later. The fact that an agency makes a new source selection decision or reevaluates vendors’ quotations does not provide a basis for reviving otherwise untimely protest allegations where, as here, the basis of the otherwise untimely protest allegations concern aspects of the agency’s evaluation that were not subsequently affected by the agency’s corrective action. Synergy Solutions, Inc., B-413974.3, June 15, 2017, 2017 CPD ¶ 332 at 7. Because VivSoft knew or should have known the basis of these arguments when it filed its prior GAO protest in March 2023 or supplemental protest in May 2023, but failed to raise these allegations at that time, we will not consider them now. Accordingly, these protest grounds are dismissed as untimely. 4 C.F.R. § 21.2(a)(2).

System for Award Management (SAM) Registration

VivSoft also raises an argument for the first time in its current protest that two of the awardees are ineligible for award due to a lapse in the vendors’ SAM registrations. We dismiss this aspect of VivSoft’s protest for failing to state a valid basis of protest.

Our Bid Protest Regulations require that a protest include a detailed statement of the legal and factual grounds for the protest, and that the grounds stated be legally sufficient. 4 C.F.R. §§ 21.1(c)(4), (f). These requirements contemplate that protesters will provide, at a minimum, either allegations or evidence sufficient, if uncontradicted, to establish the likelihood that the protester will prevail in its claim of improper agency action. Midwest Tube Fabricators, Inc., B-407166, B-407167, Nov. 20, 2012, 2012 CPD ¶ 324 at 3.

VivSoft bases its argument on the assumption that vendors were required to maintain a continuous registration in SAM during the course of the procurement in accordance with FAR provision 52.204-7. The protester acknowledges that the solicitation did not include or incorporate FAR provision 52.204-7, but argues that the clause is “made applicable under the terms of FAR 4.1105(a)(1), which provides that FAR 52.204-7 . . . be inserted ‘in all solicitations except when the conditions” not applicable here apply. Protest at 36. In this regard, the protester’s argument essentially is that our Office must read FAR provision 52.204-7 into the solicitation. The agency disagrees and asserts that its evaluation was reasonable because the RFQ did not contain FAR provision 52.204-7 and nothing in the RFQ otherwise required vendors to be registered in SAM continuously over the course of the procurement process.

Given that the agency did not include the provision in its solicitation, the agency did not violate the terms of the solicitation by failing to apply the continuous registration requirement.[11] To the extent VivSoft argues that our Office must read FAR provision 52.204-7 into the solicitation pursuant to FAR 4.1105(a)(1), the protester does not cite any legal authority for its position.[12] Protest at 36-37. Because the protester does not otherwise assert or explain why our Office should read this provision into the solicitation, we dismiss this allegation as failing to state a valid basis of protest. 4 C.F.R. §§ 21.1(c)(4), 21.5(f); cf. Excalibur Laundries, Inc., B-405814, B-405814.2, Jan. 3, 2012, 2012 CPD ¶ 1 at 6 (allegation that the awardee’s proposal did not comply with FAR clause is factually and legally insufficient where the solicitation did not incorporate the clause and the protest facially did not demonstrate unreasonable agency action). Accordingly, this protest ground is dismissed.

Technical Evaluation

Turning to VivSoft’s timely-raised issues, VivSoft argues that the agency unreasonably assigned negatives to VivSoft’s quotation under the design demonstration, DevSecOps case study, and technical challenge factors based on unstated evaluation criteria. The protester also maintains that its quotation should have been assessed additional positives under the corporate capabilities factor. In addition, the protester maintains that the agency engaged in unequal exchanges and failed to conduct a reasonable best-value tradeoff. As discussed below and based on our review, we find that none of the protester’s arguments provide a basis to sustain the protest.

Where an agency conducts a competition for the establishment of a BPA under FAR subpart 8.4, we will review the agency’s actions to ensure that the evaluation was conducted reasonably and in accordance with the solicitation and applicable procurement statutes and regulations. Citizant, Inc.; Steampunk, Inc., B-420660 et al., July 13, 2022, 2022 CPD ¶ 181 at 5. In reviewing an agency’s evaluation, we will not reevaluate quotations; a protester’s disagreement with the agency’s judgments does not establish that the evaluation was unreasonable. Id.; Digital Sols., Inc., B-402067, Jan. 12, 2010, 2010 CPD ¶ 26 at 3-4.

Design Demonstration

The protester challenges the agency’s evaluation under the design demonstration factor. The agency assigned the protester’s quotation a negative under this factor for difficult to read or incomplete personas.[13] The protester argues that the negative finding was unreasonable.

For this factor, vendors were instructed to submit a design case study, consisting of one or more projects, demonstrating their “design capabilities by showcasing design work[.]” RFQ at 12. Each vendor’s case study was to place specific “focus on the process and artifacts developed.” Id. A YouTube video submission was to accompany each vendor’s case study, and “demonstrate[] the products or services contained within the case study.” Id. The entire body of work was to demonstrate each vendor’s use of the following key design techniques: user research/generative research (research work done to understand the users and define the problem); interaction design/product design/UX design (the work done to solve the defined problem for the users); and usability testing/evaluative research (the work done iteratively to improve what was designed through testing with users as they use the product). RFQ at 13-14.

Under this factor, the solicitation provided that the agency would take into consideration a multitude of items including but not limited to: any risk identified in the vendor’s approach/quotation, potential benefits of the vendor’s approach, and innovations demonstrated in the vendor’s approach. Id.

Vivsoft submitted two separate design case studies and associated YouTube video submissions. AR, Tab 5B, VivSoft Design Quotation at 1. During the reevaluation, the TEP assessed the above-noted negative finding regarding VivSoft’s personas under user research/generative research. Specifically, the TEP explained that the “solicitation stated that ‘audio, images, and text contained in the video must be reasonably clear to the viewer.’” AR, Tab 6A, Consensus Eval. (Factor 1) at 4 (citing RFQ at 13). The TEP found that VivSoft “presents personas in the video . . . that are either difficult to read or appear to be incomplete.” Id. The TEP explained that VivSoft “does provide one persona that appears relatively complete . . . , but parts of that persona are not legible.” Id. The TEP noted that, “[o]therwise, the remaining personas [ ] seem to be incomplete as [the] personas were not completely filled out and looked more like a template.” Id. As such, the TEP concluded that “the personas do not provide sufficient information for CMS to assess their quality” and that “[t]his lack of information reduces CMS’ confidence in the Offeror’s ability to create artifacts which are effective and represent user needs.” Id. at 5.

The TEP also included in its consensus evaluation screen shots of the incomplete personas that VivSoft presented via video. See id. at 4-5. As the agency explains, these screen shots show that VivSoft left information frames empty or garbled. Id.; COS at 10-11.

The protester contends that CMS applied an unstated evaluation criterion when the TEP assigned this negative. Supp. Protest at 34-37. VivSoft argues that the “RFQ is devoid of any mention of ‘personas’” and “does not describe evaluation criteria related to the completeness or content of any personas.” Id. at 34.

The agency responds that its evaluation was not based on an unstated evaluation criterion because the solicitation contemplated evaluation of the artifacts[14] developed in the case study, and a persona is an artifact. Supp. TEP Statement at 23. As noted above, the RFQ instructed vendors to submit a design case study and specified that each vendor’s case study was to place specific “focus on the process and artifacts developed.” RFQ at 12. The TEP explains in response to the protest that a persona is “an artifact commonly used by [h]uman-[c]entered [d]esign (HCD) experts and is foundational to the HCD research practice.” Supp. TEP Statement at 23. The TEP further explains that a persona is a “fictional, yet realistic, description of a typical or target user of the product” and that it is “used to promote empathy, increase awareness and memorability of target users, prioritize features, and inform design decisions.” Id.

While procuring agencies are required to identify significant evaluation factors and subfactors in a solicitation, they are not required to identify every aspect of each factor that might be taken into account; rather, agencies reasonably may take into account considerations, even if unstated, that are reasonably related to or encompassed by the stated evaluation criteria. Guidehouse LLP, B-419848.3 et al., June 6, 2022, 2022 CPD ¶ 197 at 14; NCI Info. Sys., Inc., B-416926 et al., Jan. 9, 2019, 2019 CPD ¶ 18 at 7-8.

Based on our review, the agency’s evaluation conclusion that VivSoft’s personas were difficult to read and incomplete was reasonably encompassed by the design demonstration evaluation criteria, which placed vendors on notice that the agency would evaluate artifacts. As noted above, vendors were instructed to submit a design case study with a “focus on the process and artifacts developed.” RFQ at 12-13. Although the protester asserts that the RFQ did not mention specifically the term “persona,” the TEP explains in response to the protest, that a persona is a commonly used artifact. Supp. TEP at 23.

In evaluating ViVSoft’s quotation under this factor, the record reflects that the TEP identified that the protester’s personas were either difficult to read or appeared to be incomplete. Specifically, the TEP noted that one of the protester’s personas appeared to be relatively complete but found that “parts of that persona are not legible.” AR, Tab 6A, Consensus Eval. (Factor 1) at 4. With regard to VivSoft’s other personas, the TEP found that they seemed to be incomplete “as [the] personas were not completely filled out and looked more like a template.” Id. Because of this, the TEP could not evaluate the personas for their quality and found that this introduced a risk in the protestor’s ability to create artifacts which are effective and represent user needs. Id. at 4-5. The evaluation criteria clearly stated that this factor may take into consideration a multitude of items, including any risk identified in the vendor’s approach/quotation submission. RFQ at 21. As such, the TEP identified the issue with VivSoft’s personas as a risk and, in line with the evaluation criteria, documented the risk as a negative aspect.

As we have recognized, it is a vendor’s obligation to submit an adequately written quotation for the agency to evaluate, and a quotation that fails to address the solicitation requirements may reasonably be downgraded for lacking sufficient detail. See Undercover Training, LLC, B-418170, Jan. 9, 2020, 2020 CPD ¶ 25 at 4-5. Here, the protester failed to prove a sufficiently detailed quotation. Accordingly, this protest ground is denied.

DevSecOps Case Study

Under the DevSecOps case study factor, the agency assigned VivSoft’s quotation two negatives. The first was for VivSoft’s architectural diagram, and the second was for failing to demonstrate sufficient automated unit testing. VivSoft challenges both of the findings. As discussed below, we find no merit to the protester’s arguments.

Under this factor, vendors were required to submit a “DevSecOps Case Study.” AR, Tab 17A, RFQ amend. 0008 at 14. In addition, and as relevant here, as part of the 5‑page case study, the solicitation specified that vendors must include a “[h]igh-level infrastructure diagram outlining the built solution.” Id. The RFQ further explained that “[t]his page shall only include an image, with the exception of footers and headers consistent with the remainder of the [quotation]” and “may be landscape or portrait orientation.” Id. The solicitation also provided that the evaluation factor may take into consideration a multitude of items, including but not limited to “[a]ny risk identified in the [vendor’s] approach/[quotation] submission.” Id. at 29.

In evaluating VivSoft’s architectural diagram, the agency assessed a negative because the “architectural diagram does not clearly demonstrate the built system.” AR, Tab 13A, Consensus Eval. (Factor 2) at 5. The TEP explained that VivSoft’s diagram “lists services but does not provide sufficient context for the built system – for instance how the components are hosted, and what security measures are in place between components.” Id. The TEP found that “[g]iven this lack of meaningful detail, CMS is unable to fully evaluate the quality of the vendor’s built solution given the provided diagram.” Id. The TEP found that “[t]his lowers CMS’ confidence in the vendor’s ability to provide expertise in the key area of Cloud/Architecture, and presents a risk that the vendor may not build solutions in a manner which meets the business need.” Id. The TEP further concluded that “[t]his could lead to additional rework and frustration from Government staff as the proper information is not communicated in a thorough and complete manner, requiring additional Government oversight and guidance in order to understand the built solution during contract performance.” Id.

The protester contends that the agency’s evaluation is unreasonable and based on an unstated evaluation criterion because the RFQ requested that vendors provide a “high-level diagram.” Comments & Supp. Protest at 40. The protester argues that if the agency wanted a “detailed diagram--or wanted specific information regarding ‘how the components are hosted, and what security measures are in place between components,’” the agency should have set forth such requirements in the RFQ. Id.

The agency responds that its evaluation was not based on an unstated evaluation criterion; rather, the agency argues, its assessment of the weakness was reasonably encompassed within the RFQ evaluation criteria. Supp. TEP Statement at 26. In response to the protest, the agency explains that an “Infrastructure Diagram typically models an organization’s infrastructure consisting of” the following: hosts, virtual private clouds, virtual private networks, firewalls, networks, connection points, peripherals, locations, information systems, system components, databases, network connections, object dependencies, technology domains, technology capabilities and technology components, etc. Id. In this regard, the TEP states that “CMS expected that the vendor would follow industry best practices and provide an infrastructure diagram that outlined the built solution in a manner that made these types of components, and how they relate to one another, clear.” Id. The TEP maintains that, because “both security and hosting are part of an organization’s infrastructure,” it is reasonable the TEP would look for these to be part of an infrastructure diagram. Id.

We find nothing unreasonable regarding the agency’s assessment of the negative to VivSoft’s quotation. The solicitation required a “[h]igh-level infrastructure diagram outlining the built solution.” AR, Tab 17A, RFQ amend. 0008 at 154. The record reflects that the agency assessed a negative to VivSoft’s quotation under this factor because VivSoft’s diagram did not clearly demonstrate the built system; for example, the diagram did not provide sufficient context for the built system, such as how the components are hosted and what security measures are in place between components. AR, Tab 13A, Consensus Eval. (Factor 2) at 5.

The protester asserts that if the agency wanted a detailed diagram--or wanted specific information regarding “how the components are hosted, and what security measures are in place between components”--the agency should have set forth such requirements in the RFQ. Comments & Supp. Protest at 40. As the TEP explains in response to the protest, however, it was not “the Government’s role [ ] to define each element that need[ed] to be included in [a vendor’s] diagram.” Supp. TEP Statement at 26. In this regard, the TEP explains that the submissions under this factor “are based on real world case studies and DevSecOps solutions” and therefore “each [vendor’s] solution will be different, as will their infrastructure diagrams.” Id. The TEP notes that, while the agency expected that a vendor would follow industry best practices and provide a diagram that outlined the built solution in a manner that made the above-listed types of components, and how they relate to one another, clear, “it would be unreasonable for the Government to anticipate every possible solution and explicitly define in the solicitation each element the high level infrastructure diagrams will require.” Id. Furthermore, the TEP notes that VivSoft’s “submitted DevSecOps Case Study solution for [this factor] implements [DELETED]” and “as such, it [was] entirely reasonable for the TEP to look for security measures and hosting as part of the infrastructure diagram.”[15] Id. On this record, we think that expecting VivSoft to provide a diagram that included the aspects of its infrastructure--such as security measures and hosting--was reasonably encompassed within the solicitation requirement that vendors provide a high level infrastructure diagram outlining the built solution and that the agency’s assessment of the negative was reasonable. This protest ground is denied.

Similarly, we find no merit to the protester’s challenge to the second weakness assessed to VivSoft’s quotation under the DevSecOps case study factor for failing to demonstrate “sufficient automated unit testing for the vendor’s application ([DELETED]) in the provided repository.” AR, Tab 13A, Consensus Eval. (Factor 2) at 5. As noted above, under this factor, vendors were required to submit a “DevSecOps Case Study.” AR, Tab 17A, RFQ amend. 0008 at 14. The solicitation provided that the case study “shall demonstrate the [vendor’s] capabilities: [e]xecuting modern DevSecOps practices including, but not limited to, the following key areas: . . . automation.” Id. at 15. The RFQ provided that the evaluation of this factor “may take into consideration a multitude of items including but not limited to . . . [a]ny risk identified in the [vendor’s] approach/[quotation] submission” and the vendor’s “ability to demonstrate expertise in key areas identified in this factor, which may include but is not limited to . . . [a]utomation.” Id. at 29.

In evaluating VivSoft’s case study, the TEP assigned a negative to VivSoft’s quotation because “[t]he case study does not provide references to, and a code review by the Government does not validate the existence of[,] sufficient automated unit testing for the vendor’s application ([DELETED]) in the provided repository.” AR, Tab 13A, Consensus Eval. (Factor 2) at 5. The TEP found that “[t]his presents a risk that the vendor does not approach automated testing in a comprehensive or complete way.” Id. The TEP explained that “[a]utomation of unit tests is a measure of reliability and safety, as well as a core principle of DevSecOps” and that it is “necessary for increasing the speed of updates/deployments, which allows developers to make small changes quickly.” Id. The TEP further stated that “[t]his lack of evidence presents a risk to CMS, because if the vendor does not approach testing leveraging this key DevSecOps principle, then it increases the probability of launching deployments with errors,” which the TEP explained “creates unnecessary rework and frustration from end users as the application may have issues with function and usability.” Id.

VivSoft disagrees with the agency’s evaluation. Although the protester maintains that VivSoft’s quotation addresses automation and automated unit testing, the protester asserts that the RFQ “does not require any demonstration of automated unit testing,” and therefore the agency’s evaluation of the negative was unreasonably based on an unstated evaluation criterion. Comments & Supp. Protest at 41-42.

Based on our review, we find nothing unreasonable regarding the agency’s evaluation. The solicitation clearly stated that CMS would evaluate the vendor’s ability to demonstrate expertise in key areas identified in the factor, such as “[a]utomation.” AR, Tab 18A, RFQ amend. 0009, attach. 1 at 29. Additionally, the performance work statement (PWS), provided as an attachment to the RFQ, emphasizes the importance of automated testing, and includes requirements related to unit testing. See AR, Tab 1C, PWS attach. 3, ISG Working Environment at 8, 16-17. Furthermore, as stated in the TEP’s evaluation, “[a]utomation of unit tests is a measure of reliability and safety, as well as a core principle of DevSecOps.” AR, Tab 13A, Consensus Eval. (Factor 2) at 5; Supp. TEP at 29. Therefore, it was reasonable for the TEP to consider inclusion of automated unit testing as part of its evaluation of DevSecOps capabilities.

With regard to the protester’s assertion that VivSoft’s quotation does address automation and automated unit testing, the protester points to information in its quotation as support. Comments & Supp. Protest at 42 (citing AR, Tab 12B, VivSoft DevSecOp Quotation at 1). In response to the protest, the TEP explains it considered the information in VivSoft’s quotation cited by the protester during the evaluation, but that the information--“i.e., ‘[DELETED]’”--“do[es] not directly mention nor describe unit testing.” Supp. TEP at 31. In this regard, the TEP explains that “while descriptions can be helpful in understanding technical capabilities, the solicitation made it clear that the vendor needed to show ‘how the code demonstrates the particular practice.’” Supp. TEP at 31; RFQ at 15. Furthermore, the TEP notes that “while the vendor did include a file to demonstrate this functionality, the file was empty.” Supp. TEP at 31. As referenced above, it is a vendor’s responsibility to submit a well-written quotation. Here, VivSoft failed to do so. To the extent VivSoft disagrees with the agency’s evaluation, such disagreement, without more, fails to establish that the evaluation was unreasonable. This protest ground is denied.

Technical Challenge

Under the technical challenge factor, the agency assigned VivSoft’s quotation a negative relating to cloud expertise. VivSoft challenges the agency’s assessment of this negative.

Under this factor, the RFQ placed a burden on each vendor to ensure that the agency could sufficiently access and evaluate the vendor’s technical challenge solution, including by accessing the offeror’s GitHub repository[16] and deploying the offeror’s solution in an account with existing resources. RFQ at 16, 18.

In evaluating VivSoft’s quotation, the agency assessed a negative under this factor because the “vendor-provided resource file grants overly broad permissions to AWS [Amazon Web Services] services beyond the scope necessary for installation (e.g. identity [and] access management [IAM]).” AR, Tab 20A, Consensus Eval. (Factor 3) at 6. The TEP found that “[t]his indicates a lack of maturity in setting good IAM policies.” Id. The TEP explained that “[s]trong policies limit the scope of actions that an attacker could take on the system, such as gaining unauthorized access to services.” Id. The TEP found that “[l]ack of evidence that secure policies have been established is a risk to the Government and decreases the TEP’s confidence in the vendor’s ability to establish secure solutions.” Id. The TEP noted that “[t]his degrades the security posture, making the system vulnerable to cybersecurity attacks and data breaches.” Id.

The protester disagrees with the agency’s evaluation and contends that its quotation described in detail the manner in which VivSoft’s permissions are restricted for real‑world end-users to ensure that the permissions are not overly broad in practice. Comments & Supp. Protest at 43. As support, the protester points to its quotation. The protester asserts that, to assure adequate agency access for purposes of the evaluation under this factor, “VivSoft implemented a series of customized permissions at the infra layer to allow [a]gency evaluators to navigate the environment without impediment or back-end restrictions.” Id. The protester maintains that “[t]hese permissions were not VivSoft’s real-world end-user permissions, but rather were specifically implemented for use by the [a]gency evaluators only and to comply with VivSoft’s burden to ensure sufficient access for purposes of the [a]gency’s evaluation.” Id. (quoting AR, Tab 19F, VivSoft Quotation (Factor 3) at 1). The protester asserts that VivSoft’s quotation explained that “[DELETED]” and “[DELETED].” Id. The protester claims that its quotation further provided that “[DELETED][17] and [DELETED].” Id. (quoting AR, Tab 19F, VivSoft Quotation (Factor 3) at 1-4). The protester contends that, in assessing the negative to VivSoft’s quotation, the agency “either misunderstood or ignored the balance struck by VivSoft in its permissions--providing customized permissions for purposes of the [a]gency’s evaluation” and “describing in detail the manner in which permissions are restricted for real-world end-users[.]” Id.

In response to the protest, the TEP states that “CMS disagrees with the [protester’s] statement that the permissions had to be set up this way to permit proper evaluation.” Supp. TEP Statement at 33. The TEP explains that “[w]hile it is unclear what infra layer permissions Vivsoft is referring to, generally and from the TEP’s experience, this is referencing IAM permissions for Amazon Web Services (AWS).” Id. The TEP continues, stating that “[w]hile it is true the [g]overnment would need to use some AWS tools to deploy the solution, they would not need to use all AWS tools.” Id. The TEP explains that “[i]n this case, Vivsoft neglected to analyze the tools necessary to deploy, and to apply specific restrictions around those specific necessary tools.” Id. The TEP states that “[r]ather, Vivsoft provided whoever is deploying the application the ability to access all the tools and services whether they were required for deployment or not.” Id. The TEP explains that “such infra layer permissions were not and are not inevitable as stated by Vivsoft for the [g]overnment to evaluate the deployed solution” and that “Vivsoft did not take the extra step to apply specific permissions to the role or activity the [g]overnment had to take to deploy the solution.” Instead, the TEP states that VivSoft “gave a blanket permission to whoever is deploying the application to override/modify and/or add new permissions which essentially assigns complete administrator rights (i.e., no restrictions).” Id. The TEP explains that “[t]his allows for the ability to delete resources in the account that do not relate to the specific deployment steps necessary for the Government evaluators” and that “[u]nrelated AWS resources could include other vendors’ ACME submissions, or any other resources already contained in CMS’ ACME Challenge account.” Id.

The agency maintains that VivSoft’s approach conflicts with the RFQ instructions, which provided that “the solution may be deployed in an AWS account that has existing resources in it . . . and accounting for that possibility is the responsibility of the Quoter.” Supp. TEP Statement at 33; RFQ at 18. The agency explains that the vendor’s repository did not show evidence that they took any actions to account for existing resources in the AWS account, or to scope deployment permissions, such as placing limits on access to specific resources. Supp. TEP Statement at 33. The agency explains that “[t]his is a risk as it allows individuals to take actions that are not relevant to the tasks they need to do.” Id.

With regard to the examples cited by the protester in VivSoft’s quotation of how it met security requirements for “real-world end user access”, such as it being “[DELETED], and [DELETED],” the TEP responds that “these details are not relevant to or referenced in the documented negative finding[,] which strictly focused on broad AWS permissions within the vendor’s IAM policy for deploying the application (i.e. permissions for the developer deploying the solution), not the end users of the solution.” Supp. TEP Statement at 34. The agency maintains, therefore, that “these details do not mitigate or lessen the specific security concerns which the TEP documented in its negative finding.” Id.

Based on our review, we find nothing unreasonable regarding the agency’s evaluation. In evaluating VivSoft’s approach, the TEP found that “[l]ack of evidence that secure policies have been established is a risk to the [g]overnment and decreases the TEP’s confidence in the vendor’s ability to establish secure solutions,” which the TEP found “degrades the security posture, making the system vulnerable to cybersecurity attacks and data breaches.” AR, Tab 20A, Consensus Eval. (Factor 3) at 6. While, as noted above, the protester disagrees with the agency regarding the thoroughness of its response, it has not, in our view, demonstrated that the agency’s findings were unreasonable. As such, this protest ground is denied.

Corporate Capabilities

The protester contends that CMS should have assessed its quotation with additional positives under the corporate capabilities factor. VivSoft also argues that the agency engaged in disparate treatment in its evaluation of VivSoft’s quotation under the corporate capabilities factor because multiple awardees were assessed positives for attributes that VivSoft claims were also present in its quotation, but for which it was not credited with positives. The agency argues that the difference in evaluations was based on differences in the vendors’ quotations and that the aspects of protester’s quotation did not exceed the RFQ’s requirements. We find the protester’s arguments provide no basis to sustain the protest and discuss one representative example below.

In conducting procurements, agencies may not generally engage in conduct that amounts to unfair or disparate treatment of competing vendors. Arc Aspicio, LLC; et al., B-412612 et al., Apr. 11, 2016, 2016 CPD ¶ 117 at 13. Where a protester alleges unequal treatment in a technical evaluation, it must show that the differences in ratings did not stem from differences between the vendors’ quotations. See Camber Corp., B‑413505, Nov. 10, 2016, 2016 CPD ¶ 350 at 8.

VivSoft maintains that multiple awardees received positive findings under the corporate capabilities factor for having the ability to complete multiple overlapping projects, or to compete on orders simultaneously. Comments & Supp. Protest at 27 (citing AR, Tab 21B, SSD (Octo) at 20-21) (explaining that Octo received a “positive” under Scalability because it “demonstrate[d] an ability to successfully compete on Orders simultaneously”; Tab 21C, SSD (Nava) at 19-20 (explaining that Nava PBC was assigned a “positive” for its “ability to successfully compete on orders simultaneously”); Tab 21G, SSD (Softrams) at 21 (“[Softrams] had a positive under the Scalability capabilities item for ability to compete on orders simultaneously.”); Tab 21I, SSD (Oddball) at 20 (explaining that Oddball received a “positive” for “demonstrat[ing] an ability to successfully compete on orders simultaneously”).

As relevant here, under this factor, the RFQ required that offerors demonstrate that they “are capable of scaling to compete and potentially operate multiple overlapping orders.” RFQ at 16.

The protester, pointing to its quotation, asserts that “VivSoft demonstrated an essentially identical ‘ability to successfully compete on Orders simultaneously.’” Comments & Supp. Protest at 28 (citing AR Tab 19G, VivSoft Quotation (Factor 4) at 8‑9 (“Team VivSoft is a low-risk partner for CMS because we are experienced in staffing and operating multiple Task Orders and contracts simultaneously.”). In addition, VivSoft cites to a table included in its quotation describing “the number of [full-time equivalent] FTEs, number of concurrently staffed contracts, and number of agile teams supported by VivSoft or its team members,” which the protester asserts “demonstrate[s] the extent of VivSoft’s--and its team members’--ability to compete for and staff multiple orders simultaneously.” Id.

In response to the protest, the TEP explains that although VivSoft’s quotation stated that the vendor is “experienced in staffing and operating multiple Task Orders and contracts simultaneously” and provided a table showing the number of FTEs, number of concurrently staffed contracts, and number of agile teams supported, the solicitation articulated that vendors demonstrate that they are “capable of scaling to compete and potentially operate multiple overlapping orders.” Supp TEP Statement at 16-17; RFQ at 23. The TEP explains that “simply showing capability” to compete on orders simultaneously “meets expectations and is not exceptional and thus does not automatically merit a positive aspect[.]” Supp. TEP Statement at 16-17. The TEP states “[w]hile VivSoft met expectations in this area, the TEP did not find that the submission provided specific detail or examples of overlapping projects.” Id. at 17. In contrast, the TEP explains that the awardees that received a positive aspect for their quotations “identified specific examples of substantial FTE size, including details and the results of their approach.” Id. The TEP notes that “[t]he table provided by [VivSoft’s quotation] lacks this detail.” Id.

We find nothing unreasonable regarding the agency’s evaluation. The protester has failed to demonstrate how its quotation exceeded the solicitation requirement or that the positives received by the other vendors did not stem from differences between the quotations. Camber Corp., supra. We find no basis to sustain the protest. In sum, we find nothing unreasonable regarding the agency’s evaluation of VivSoft’s quotation under the corporate capabilities factor.

Exchanges

Finally, Vivsoft argues that the agency engaged in unequal discussions with the awardees. The protester asserts that the agency conducted “fundamentally unequal discussions ‘in such a way that allowed the bases of the [a]gency’s best-value determination to be altered, after the best-value tradeoffs had already been conducted.’” Supp. Comments at 3 (quoting Supp. Protest at 4-10). VivSoft claims that “[a]s a result of these unequal exchanges, certain [vendors]--such as Coforma, Nava PBC, and those others [that] received an award following the prior evaluation--received a substantial unfair advantage in the [a]gency’s corrective action reevaluation” because those vendors “had an opportunity, not afforded to VivSoft, to revise their quotations in advance of the corrective action reevaluation.” Id. As a result, VivSoft maintains that the agency’s best-value tradeoff conducted during the corrective action reevaluation was “materially flawed” because “while the prior awardees were credited with the improvements made in their revised proposals,” the agency “used Coforma’s and Nava PBC’s initial price quotations, which contained lower prices, rather than using Coforma’s and Nava PBC’s revised quotations with higher prices.” Id.

As noted above, the RFQ provides for the assessment of the best value based on consideration of price and the following four non-price factors: design demonstration; development, security, and DevSecOps case study; technical challenge; and corporate capabilities. RFQ at 21. The contracting officer based his initial best-value determination solely on these factors and found that there were eight quotations that provided the best value to the government. AR, Tab 21A, Revised SSD at 2‑3. Consistent with the terms of the solicitation, the contracting officer then conducted exchanges and received quotation revisions from the eight vendors identified as the “best-suited.” COS at 2; RFQ at 6.

Pertinent here, the agency’s exchanges were limited to two issues: section 508 compliance, which the RFQ provided was to be evaluated only for the proposed awardees; and concerns regarding labor category mapping that the agency identified during a labor category mapping exercise it performed on the quotations of the eight apparent successful vendors after completion of the best-value tradeoff. AR, Tab 21A, Revised SSD at 3; COS at 2. Nothing addressed during these exchanges concerned any aspect of the vendors’ quotations evaluated under the RFQ’s four non-price evaluation factors, although as a result of the exchanges, two of the best-suited vendors--Coforma and Nava PBC--raised their prices slightly in response to having changed labor category mappings. See AR Tab 23, Post Negotiation Memorandum (PNM) at 10; 21; 24; Supp. COS at 1-2. In this regard, the contracting officer explains in response to the protest that “Coforma and Nava PBC made minor mapping changes to their proposals which resulted in each awardee slightly revising its price,” as follows:

Supp. COS at 2. The contracting officer further explains that during the agency’s corrective action reevaluation, he used an older template and inadvertently forgot to update Coforma and Nava’s prices, meaning that he used the original prices, as opposed to their revised prices, during the best-value tradeoff. Id; see AR, Tab 21H, SSD (Coforma) at 1, 25; AR, Tab 21C, SSD (Nava) at 1, 28. The contracting officer states, however, that while this error in documentation is reflected in the source selection decision, the contemporaneous record also includes a post-negotiation memorandum which discusses and reflects the revised pricing from both Coforma and Nava. AR, Tab 23, PNM at 10. The contracting officer states that “[he] was aware of the higher pricing and considered it in making the tradeoff best value decision despite the inadvertent editing issue.” Supp. COS at 2.

The record reflects that the agency engaged in exchanges consistent with the RFQ’s terms. The solicitation specified that once the agency identified “the best-suited [vendors] (i.e., the apparent successful contractors),” the agency may “communicate with only those contractors to address any remaining issues,” including “technical and price.” RFQ at 6. The record reflects that the agency did not conduct exchanges with any of the vendors during the evaluation process, but rather, conducted exchanges with only the eight apparent successful contractors after determining that the eight vendors were best-suited. Supp. COS at 1. The contracting officer explains in response to the protest that because the exchanges were “limited to Section 508 assessments and labor category mapping exchanges” and because “this Section 508 assessment and the mapping exercise was not part of the general evaluation to determine the ‘best value’ [vendors],” the agency “did not perform this mapping exercise or the evaluation of Section 508 on the unsuccessful [vendors], including Vivsoft.” Id. The agency’s actions in this regard were consistent with the terms of the RFQ and reasonable.

To the extent VivSoft asserts that, during the corrective action reevaluation, it was treated unequally by the exchanges the agency conducted with only the apparent successful vendors, we find that the protester has failed to demonstrate prejudice.

Our Office will not sustain a protest unless the protester demonstrates a reasonable possibility that it was prejudiced by the agency’s actions, that is, unless the protester demonstrates that, but for the agency’s actions, it would have had a substantial chance of receiving the award. Raytheon Co., B-409651, B-409651.2, July 9, 2014, 2014 CPD ¶ 207 at 17.

Here, there is no indication in the record that the issues addressed during the agency’s exchanges with the apparent successful vendors concerned any aspect of the vendors’ quotations that were evaluated under the RFQ’s four non-price evaluation factors or had any impact on the vendors’ ratings (or the positives and negatives assessed) under the four non-price factors. In addition, although the record shows that two of the apparent successful vendors raised their proposed prices as a result of the exchanges, these price increases were slight, reflecting only a .6 percent and .1 percent increase, respectively. Supp. COS at 2. The record also shows that VivSoft’s quotation was deemed technically inferior to the quotations of these two vendors. AR, Tab 21H, Revised SSD (Coforma) at 15 (finding that compared to VivSoft, Coforma “offers more merit related to leveraging strong stakeholder relationships, demonstrating generative research, conducting research across a breadth and depth of users, and exhibiting an impressive focus on inclusivity” and that “[w]hile, Vivsoft had a positive for thorough demonstration of testing with users, Vivsoft’s quotation response is less robust as compared to the many positive aspects in Coforma’s submission.”); AR, Tab 21C, Revised SSD (Nava) at 17 (noting that “Nava offers more merit as their positives are more comprehensive than Vivsoft’s.”). VivSoft has not explained or demonstrated how, but for these exchanges, VivSoft would have been selected for award or at least determined best-suited. In sum, we find that even if the agency had conducted the same exchanges with VivSoft as it did with the apparent successful vendors, there is no indication that it would have resulted in VivSoft having a substantial change of receiving one of the awards.[18]

The protest is dismissed in part and denied in part.

Edda Emmanuelli Perez
General Counsel

 

---