QVine Corporation; Digital Age Experts, LLC
Highlights
QVine Corporation, of Herndon, Virginia, and Digital Age Experts, LLC (DAE), of Reston, Virginia, protest the Central Intelligence Agency's (CIA) award of contracts to other offerors, pursuant to request for proposals (RFP) No. 2019-18101100001S, to provide support for the CIA's Open Source Enterprise (OSE). QVine protests the agency's assessment of a significant weakness in its technical proposal, and both QVine and DAE assert that the agency's technical evaluation was contrary to the terms of the solicitation. Additionally, both protesters challenge the agency's cost/price evaluation and best-value tradeoff determinations.
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.
Decision
Matter of: QVine Corporation; Digital Age Experts, LLC
File: B-419017; B-419017.2; B-419017.3; B-419017.4
Date: November 23, 2020
G. Matthew Koehl, Esq., Kelley P. Doran, Esq., Gary J. Campbell, Esq., and Lidiya Kurin, Esq., Womble Bond Dickinson (US), LLP, for QVine Corporation; and C. Peter Dungan, Esq., and Holly Drumheller Butler, Esq., Miles & Stockbridge PC, for Digital Age Experts, LLC, the protesters.
Samantha S. Lee, Esq., Brian G. Walsh, Esq., Gary S. Ward, Esq., Moshe B. Broder, Esq., and Nicole E. Giles, Esq., Wiley LLP, for Novetta, Inc.; Jason A. Carey, Esq., J. Hunter Bennett, Esq., Kevin T. Barnett, Esq., Andrew R. Guy, Esq., and Darby Rourick, Esq., Covington & Burling, LLP, for Next Tier Concepts, Inc.; Laurel A. Hockey, Esq., David S. Cohen, Esq., Daniel J. Strouse, Esq., and John J. O’Brien, Esq., Cordatis LLP, for BTI360, Inc.; and, Craig A. Holman, Esq., Kara L. Daniels, Esq., Thomas A. Pettit, Esq., and James R. Mestichelli, Esq., Arnold & Porter Kaye Scholer LLP, for Asymmetrik, Ltd., the intervenors.
Lindsay Windsor, Esq., Dana E. Koffman, Esq., and Avi M. Baldinger, Esq., Central Intelligence Agency, for the agency.
Glenn G. Wolcott, Esq., and Christina Sklarew, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.
DIGEST
1. Agency reasonably assessed a significant weakness in protester’s technical proposal based on protester’s failure to meaningfully address solicitation requirements regarding information security.
2. Agency evaluated technical proposals in a manner that was consistent with the solicitation’s evaluation criteria, which provided for assessing the extent to which proposals balanced the solicitation’s potentially competing objectives of functionality, performance, and availability.
3. Agency reasonably determined that the higher cost/prices proposed by the awardees were reasonable based on consideration of the technical approach of each offeror, comparison of cost/prices to government cost estimates, and comparison of offerors’ cost/prices to each other.
4. Agency’s best-value tradeoff determinations were reasonable where the source selection authority (SSA) considered and documented the evaluated strengths, weaknesses, and cost/prices of the competing proposals, and the SSA’s judgments were rational and consistent with the evaluation criteria.
DECISION
QVine Corporation, of Herndon, Virginia, and Digital Age Experts, LLC (DAE), of Reston, Virginia, protest the Central Intelligence Agency’s (CIA) award of contracts to other offerors,[1] pursuant to request for proposals (RFP) No. 2019-18101100001S, to provide support for the CIA’s Open Source Enterprise (OSE). [2] QVine protests the agency’s assessment of a significant weakness in its technical proposal, and both QVine and DAE assert that the agency’s technical evaluation was contrary to the terms of the solicitation. Additionally, both protesters challenge the agency’s cost/price evaluation and best-value tradeoff determinations.
We deny the protests.
BACKGROUND
On August 30, 2019, the agency issued the solicitation, seeking proposals to provide “development and maintenance of applications, networks, and systems required to support Open Source Intelligence (OSINT)[3] and publically available information (PAI) tasking, collection, processing, exploitation, and dissemination.”[4] AR, Tab 4, SOW at 1. The agency states that this procurement (referred to as the “open source large scale architecture and applications support” or “OLA” contract) “will be the [redacted] of the CIA’s Open Source Enterprise (OSE) and will enable OSE’s core functional areas.”[5] AR, Tab 1, Contracting Officer’s Statement at 1. In summary, the agency states that, “[i]n a world driven by current events and the explosion of openly available information, [this contract] supports OSE’s mission to access large amounts of data, quickly determine intelligence value, integrate it with other sources of information, and provide it on a timely basis to senior U.S. policymakers.” Id. at 2.
The solicitation contemplated award of multiple indefinite-delivery indefinite-quantity (IDIQ) contracts, under which the agency will subsequently conduct task order competitions for specific requirements, and provided for a 5-year base performance period. The RFP stated that source selection decisions would be based on best-value tradeoffs, applying the following evaluation factors: technical,[6] past performance, management, security, and cost/price. AR, Tab 37, RFP § M at 2‑3. The solicitation provided that the technical factor was slightly more important than past performance, which was slightly more important than management, and that the non-cost/price factors combined were significantly more important than cost/price.[7] Id.
The solicitation’s statement of work identified five broad performance areas in which future task orders will require the contractor(s) to “design, develop and deliver” various products/services.[8] AR, Tab 4, SOW at 3-6. In discussing the various areas of performance, the solicitation’s SOW contained the following statement:
The use of commercially available technological solutions will be preferred over custom built solutions. Technological solutions will incorporate continuous integration and continuous delivery princip[le]s, and balance Sponsor [OSE] requirements for functionality, high availability, automation, performance and scalability.
Id.
With regard to the technical evaluation factor, the solicitation provided a sample task order with specific tasks related to each of the IDIQ SOW performance areas. See AR, Tab 34, STO SOW at 7-10. Offerors were required to submit technical proposals that addressed the STO requirements and demonstrated their understanding and capabilities regarding those requirements.[9] Of relevance to these protests, section M of the solicitation established the basis for the agency’s evaluation and, in describing the technical evaluation, stated that the agency would evaluate the extent to which each offeror’s proposal demonstrated the ability to: provide innovative solutions to integrate commercially available solutions; balance the need for functionality, high availability, performance, and scalability; apply continuous integration and delivery principles; and comply with the requirements of intelligence community directive No. 503[10] and IdAM (identity access management) principles.[11] AR, Tab 37, RFP § M at 3.
With regard to evaluation under the cost/price factor, the solicitation provided that each offeror’s cost/price for the STO would constitute its cost/price for source selection purposes.[12] The solicitation provided that the agency would apply the techniques in Federal Acquisition Regulation (FAR) 15.404-1 to evaluate STO CLIN 0001 for reasonableness and STO CLIN 0002 for realism and reasonableness, elaborating that the agency’s cost/price evaluation would “include a comparison of price differences among the offerors and their causes, including those due to differences in business methods and operating procedures.” AR, Tab 37, RFP § M at 6 (emphasis added).
On or before the closing date,[13] proposals were submitted by 18 offerors, including the two protesters and five awardees.[14] Thereafter, the proposals of the protesters and awardees were evaluated, as follows:[15]
|
Technical |
|
Management |
Evaluated |
|
|---|---|---|---|---|
|
Asymmetrik |
Exceptional |
Moderate Confidence |
Acceptable |
$24,731,259 |
|
Next Tier |
Very Good |
High Confidence |
Acceptable |
$17,464,170 |
|
BTI360 |
Exceptional |
High Confidence |
Acceptable |
$15,168,048 |
|
Novetta |
Very Good |
High Confidence |
Acceptable |
$13,309,211 |
|
KeyW Corp. |
Very Good |
Moderate Confidence |
Acceptable |
$12,652,607 |
|
QVine |
Very Good |
Moderate Confidence |
Acceptable |
$9,711,238 |
|
DAE |
Acceptable |
Significant Confidence |
Acceptable |
$6,929,489 |
AR, Tab 96, DAE Debriefing at 32; Tab 106 QVine Debriefing at 32.
In evaluating QVine’s proposal under the most important technical evaluation factor, the agency assigned an overall rating of very good; however, the agency identified a significant weakness with regard to the information security subfactor and rated QVine’s proposal marginal under that subfactor. In this regard, the solicitation required each offeror to describe its approach to complying with “ICD-503 requirements and Identity Access Management (IdAM) principles.”[18] AR, Tab 43, RFP § L at 9; Tab 37 RFP § M at 3. The agency’s technical evaluators determined that QVine’s proposal “did not provide convincing rationale to address how [QVine] intends to meet [IdAM] requirements,” adding that QVine failed to discuss: how it “will integrate identity and access management solutions outside the ICD-503 context”; how it will “control user access”; or how it will “use federated identity management.” AR, Tab 60, Tech. Evaluation Report (QVine), at 3-4. The agency further concluded that QVine’s failures in this regard “will lead to increased administrative burdens and/or information security risks resulting in schedule delays, which significantly decreases the Sponsor’s confidence in [QVine’s] ability to deliver technical solutions.” Id. at 4.
In evaluating the other offerors’ proposals under the technical factor, two of the five awardees (Asymmetrik and BTI360) received ratings of exceptional. In assigning the exceptional ratings, the agency identified three significant strengths and no weaknesses in each of their technical proposals. Next, the other three awardees, (KeyW, Next Tier Concepts, and Novetta) each received technical ratings of very good, with at least one significant strength and no weaknesses. Finally, DAE’s proposal received a technical rating of acceptable, and contained no weaknesses, but also received no significant strengths. AR, Tab 1, Contracting Officer’s Statement at 20, 22.
In evaluating cost/price, the agency’s cost evaluation team (CET) considered each offeror’s technical approach and level of innovation. In this regard, the agency established three different government cost estimates, based on whether the offeror’s proposed solution reflected: (1) [redacted]; (2) [redacted]; or (3) [redacted]. AR, Tab 1, Contracting Officer’s Statement at 10. In this regard, the CET, with assistance from the technical/management evaluation team (TMET), determined that Asymmetrik’s technical approach fell in the first category in that it reflected [redacted]; that KeyW’s, BTI360’s, and Next Tier Concepts’ technical approaches fell into the second category, [redacted]; and that Novetta’s, DAE’s, and QVine’s approaches fell into the third category, [redacted]. Id. at 11.
Thereafter, in determining cost/price reasonableness, the CET considered each proposal’s cost/price in the context of its proposed technical approach, comparing each offeror’s cost/price to: the government cost/price estimate that corresponded with its technical approach; the cost/price of other offerors within the same category; and the cost/price of all offerors. Id. at 11-15. In addition, the agency compared the offerors’ cost/prices to a separate government cost estimate that was based on historical costs. AR, Tab 1, Contracting Officer’s Statement at 7-9; AR, Tab 78, Historical Government Cost Estimate. Based on these comparisons, and consideration of the offerors’ various technical approaches, the cost/price of each protester and awardee was determined to be reasonable and realistic. AR, Tab 1, Contracting Officer’s Statement at 15.
Following completion of the evaluations described above, the source selection authority (SSA) reviewed the evaluation reports, determined that discussions were unnecessary,[19] and selected the proposals of Asymmetrik, KeyW, Next Tier Concepts, BTI360, Inc., and Novetta for award.
In making the award determinations, and documenting those decisions in a source selection decision memorandum (SSDM), the SSA addressed each of the protesters’ and awardees’ ratings under each evaluation factor, along with the strengths, weaknesses, risks and/or benefits associated with each rating, comparing each of the protesters’ proposals to each of the awardees’ proposals.
By way of example, in comparing QVine’s proposal to KeyW’s and Asymmetrik’s proposals,[20] the SSA stated:
[KeyW] proposed a technical solution based on [redacted] . . . [and] proposed a cost/price that was about $3M more than QVine. This premium is worth the reduced risk of unsuccessful contract performance. KeyW’s significant strength[[21]] and minor strength in its technical approach is more valuable than QVine’s significant strength, minor strength, and significant weakness under the most important factor particularly because QVine’s significant weakness . . . is critical to working within the Sponsor’s environment. While KeyW had a similar past performance rating to QVine . . . it also had four areas of minor strengths which are valuable compared to QVine’s [no past performance strengths].
* * * * *
Asymmetrik provided a total proposed cost/price that was about $15.1M more than QVine. The cost/price premium is based on its unique technical approach that significantly differed from QVine. The cost is driven by a significantly broader technical approach that is [redacted], and includes significantly [redacted] in both CLINs. The Asymmetrik technical solution enables [redacted] to be delivered for use by the Sponsor with minimal impact to other services or users; employs [redacted]; an exceptional approach to [redacted] that will ensure [the contract] requirements can be delivered on time, across multiple environments, and with reduced effort. This technical solution includes a significant investment in [redacted] that is a significant portion of its exceptional technical proposal and accounts for [redacted] of the total proposed Asymmetrik cost. The additional [approximately] $15.1M, mainly driven by a difference in [redacted], is an acceptable premium to pay for Asymmetrik’s exceptional Technical solution[.]
AR, Tab 83, SSDM at 60-62 (quoted in AR, Tab 1, Contracting Officer’s Statement at 28-29).[22]
Similarly, the SSA compared DAE’s proposal to those of the awardees, considering the relative merits under the non-cost/price factors, as well as comparing each offeror’s cost/price. In comparing DAE’s proposal to KeyW’s proposal, the SSA stated:
When comparing KeyW’s cost to DAE, the total proposed cost/price for KeyW is about $5.8M more. KeyW proposed a [redacted] solution that is driving the cost difference and includes significantly more [redacted] in both CLINs due to the technical complexity. KeyW’s technical solution includes an exceptional approach to integrate innovative open source or commercial solutions that is [redacted], which enables new services to be delivered for use by the Sponsor with minimal impact to other services or users. It also proposed [redacted] to further enable integration of technical solutions, and proposed [redacted]. While DAE had a higher confidence rating for past performance, including five more significant strengths, the Past Performance factor is weighted slightly less important than the Technical factor. In addition, the Government has Moderate confidence KeyW will successfully perform on [the contract]. The significant strength and strength of KeyW’s Very Good technical proposal, and the lower risk of unsuccessful contract performance, are worth at least $5.8M relative to DAE, even taking into account DAE’s minor technical strengths and the additional significant strengths it demonstrated in past performance.
AR, Tab 83, SSDM at 56-58 (quoted in AR, Tab 1, Contracting Officer’s Statement at 32).
Similarly, in comparing DAE’s proposal to Asymmetrik’s the agency: noted that Asymmetrik’s total cost/price was “about $17.8M more than DAE’s” due to Asymmetrik’s different technical approach; repeated the multiple technical strengths and benefits Asymmetrik’s proposal offered to the government, as quoted above; and acknowledged that DAE had a higher past performance rating than Asymmetrik, but concluded that “the quantity and quality of Asymmetrik’s significant strengths under the technical factor outweigh the advantages of DAE’s significant strengths under past performance” and “are worth at least $17.8M.” AR, Tab 1, Contracting Officer’s Statement at 31.
Thereafter, the unsuccessful offerors were notified of the agency’s source selection decisions. Following debriefings, QVine and DAE filed these protests.
DISCUSSION
QVine protests the agency’s assessment of a significant weakness in its technical proposal, and both QVine and DAE assert that the agency’s technical evaluation was inconsistent with the terms of the solicitation. Additionally, both protesters challenge the agency’s cost/price evaluations and best-value tradeoff determinations.[23]
Technical Evaluation
First, QVine protests that the agency improperly assessed a significant weakness in its technical proposal. As discussed above, the solicitation required each offeror to describe its approach to complying with “ICD-503 requirements and Identity Access Management (IdAM) principles,” and the agency concluded that QVine’s proposal failed to comply with this requirement in that its proposal did not meaningfully address how QVine “will integrate identity and access management [IdAM] solutions outside the ICD‑503 context.” AR, Tab 60, Tech. Evaluation Report (QVine) at 3-4.
QVine acknowledges that its proposal “mistakenly failed to address how it would address IdAM services outside of the ICD-503 context,” but maintains that this was a “minor proposal writing error” that could be “easily fixed.” Protest at 21; QVine Comments, Sept. 28, 2020, at 2. Accordingly, QVine maintains that the agency’s assessment of a significant weakness in its technical proposal was improper.
The agency responds that, due to the sensitive nature of the agency’s mission, an offeror’s demonstration of compliance with IdAM principles was “paramount in reducing the risk of unauthorized access.”[24] AR, Tab 1, Contracting Officer’s Statement at 19. More specifically, the agency states that [redacted]; accordingly, the agency expressed concern that QVine’s [redacted] solution could require significant additional costs to render its solution compliant with necessary security requirements. Id. at 18-19; see AR, Tab 60, Tech. Evaluation Report (QVine) at 3-4. On this basis, the agency maintains that QVine’s failure to meaningfully address the solicitation requirements regarding compliance with IdAM principles was not a minor flaw, nor could it be easily corrected. AR, Tab 1, Contracting Officer’s Statement at 19.
It is an offeror’s responsibility to submit a well‑written proposal, with adequately detailed information clearly demonstrating compliance with the solicitation requirements. See International Med. Corps, B‑403688, Dec. 6, 2010, 2010 CPD ¶ 292 at 8. An offeror is responsible for affirmatively demonstrating the merits of its proposal and risks the rejection of its proposal if it fails to do so. HDL Research Lab, Inc., B‑294959, Dec. 21, 2004, 2005 CPD ¶ 8 at 5. In reviewing protests challenging the rejection of a proposal based on the agency’s evaluation, it is not our role to reevaluate proposals; rather, our Office examines the record to determine whether the agency’s judgment was reasonable and in accordance with the solicitation criteria and applicable procurement statutes and regulations. Wolverine Servs. LLC, B‑409906.3, B‑409906.5, Oct. 14, 2014, 2014 CPD ¶ 325 at 3; Orion Tech., Inc., B‑405077, Aug. 12, 2011, 2011 CPD ¶ 159 at 4. A protester’s disagreement with the agency’s judgment, without more, is insufficient to establish that the agency acted unreasonably. STG, Inc., B-405101.3 et al., Jan. 12, 2012, 2012 CPD ¶ 48 at 7.
Here, we find no basis to question the reasonableness of the agency’s assessment of a significant weakness in QVine’s technical proposal. As noted above, there is no dispute that QVine’s proposal, in fact, failed to meaningfully address the solicitation requirements regarding compliance with IdAM principles; QVine states in its protest that it “mistakenly failed to address” these requirements. While QVine expresses its disagreement with the agency regarding the significance of QVine’s failure, it has not demonstrated that the agency’s assessment was unreasonable; accordingly, its protest challenging that assessment is denied.
Next, QVine and DAE both assert that the agency’s technical evaluation failed to properly apply the solicitation’s stated evaluation factors or, alternatively, applied an unstated factor. In this regard, one or both of the protesters assert that the solicitation required the agency to apply an evaluation preference for solutions based on COTS/GOTS platforms or, alternatively, to “downgrade” or “penalize” solutions based on customized platforms.[25] See Qvine Supp. Protest, Sept. 28, 2020, at 16, 21. In supporting this contention, the protesters refer to portions of the solicitation’s SOWs[26] that stated: “The use of commercially available technological solutions will be preferred over custom built solutions.” AR, Tab 4, SOW at 3-6; Tab 34, STO SOW at 2. Accordingly, the protesters complain that the agency improperly assigned higher technical ratings to proposals that relied on customized or partially customized platforms than the ratings assigned to the protesters’ COTS/GOTS-based platforms.[27] Additionally, each protester refers to various aspects of its own proposal, or the awardees’ proposals, and challenges various aspects of the agency’s technical evaluation assessments.
The agency responds that the protesters’ assertions regarding the stated evaluation criteria reflect a flawed interpretation of the solicitation requirements. In this regard, the agency maintains that, while the solicitation sought solutions that incorporated existing technologies, it did not establish a per se evaluation preference for solutions based on COTS/GOTS platforms over solutions based on customized platforms. With regard to the statement in the SOW on which the protesters rely to assert an evaluation preference, the agency notes that the protesters ignore the sentence directly following that statement--pointing out that the entire reference states:
The use of commercially available technological solutions will be preferred over custom built solutions. Technological solutions will incorporate continuous integration and continuous delivery princip[le]s, and balance Sponsor requirements for functionality, high availability, automation, performance and scalability.
AR, Tab 4, SOW at 3-6.
Accordingly, the agency maintains that the SOW directed offerors to balance the potentially competing objectives of functionality, performance, and availability, and that the specific evaluation factors contained in RFP section M advised the offerors that their proposals would be evaluated on the extent to which they incorporated and balanced these interests.[28] See AR, Tab 37, RFP § M at 3. The agency further notes that the section M evaluation factors do not provide for “downgrading” customized solutions, nor state that an offeror’s solution using a COTS/GOTS-based platform would be evaluated as superior to a solution with a customized platform. Id. For example, the agency states that it assigned the highest technical rating to Asymmetrik’s proposal on the basis that it presented innovative approaches to [redacted], as well as offering significant benefits to the agency with regard to the solicitation’s stated performance and functionality objectives. On this record, the agency maintains that it properly evaluated the offerors’ proposals against the section M evaluation criteria, and properly assigned the highest ratings to proposals that offered the greatest overall benefit to the agency.
Where a protester and agency disagree about the meaning of solicitation language, we will resolve the matter by reading the solicitation as a whole and in a manner that gives effect to all of its provisions; to be reasonable, and therefore valid, an interpretation must be consistent with the solicitation when read as a whole and in a reasonable manner. Crew Training Int’l, Inc., B‑414126, Feb. 7, 2017, 2017 CPD ¶ 53 at 4.
Here, we reject the protesters’ assertions that, in performing its evaluation under the most important evaluation factor, technical, the agency was required to assign higher or lower technical ratings based on whether the proposed solution was based on a COTS/GOTS platform or a customized platform. Rather, the solicitation established various performance, functionality, and availability objectives, and the offerors’ proposals were evaluated on the extent to which their proposed solutions would achieve those potentially competing objectives. We have also considered the protesters’ various complaints regarding particular aspects of the agency’s technical evaluation of the protesters’ and awardees’ proposals, and find no bases to sustain the protests. The protesters’ allegations regarding alleged flaws in the agency’s technical evaluation are denied.
Cost/Price Evaluation
Next, both protesters challenge the agency’s conclusion that the cost/prices of the higher-priced awardees were reasonable. More specifically, QVine asserts that, because the agency concluded that QVine’s cost/price was reasonable and realistic, and the awardees proposed “dramatically higher” prices “to perform the same work,” the agency “could not have performed a comparative analysis of the offerors’ proposed prices” or “otherwise reasonably determined” that the awardees’ cost/prices were reasonable. QVine Protest at 17, 19. Similarly, DAE complains that the agency performed a “facially unreasonable analysis,” complaining that it “defies logic” that the higher cost/prices of the awardees were “somehow reasonable” where the offerors submitted proposals “against the same requirements.” DAE Protest at 10.
The agency responds by noting that the solicitation specifically advised offerors that the agency’s cost/price evaluation would “include a comparison of price differences among the offerors and their causes, including those due to difference in business methods and operating procedures.” See AR, Tab 37, RFP § M at 6 (emphasis added). Accordingly, in performing its cost/price evaluation, the cost evaluation team (CET) considered each proposed cost/price in the context of its proposed technical approach, comparing each offeror’s cost/price to: a government cost estimate that corresponded to the offeror’s technical approach; the cost/price of other offerors that proposed a similar technical approach; and the cost/price of all offerors. Id. at 11-15. In addition, the agency compared the offerors’ cost/prices to a separate government cost estimate based on historical costs for the same or similar work. AR, Tab 1, Contracting Officer’s Statement at 7-9; AR, Tab 78, Historical Government Cost Estimate. Based on these comparisons, and in the context of each offeror’s technical approach, the agency maintains that it properly determined that the awardees’ cost/prices were reasonable and realistic.
Procuring agencies must condition the award of a contract upon a finding that the contract contains "fair and reasonable prices.” FAR 15.402(a), 15.404-1(a); see Crawford RealStreet Joint Venture, B-415193.2, B-415193.3, Apr. 2, 2018, 2018 CPD ¶ 121 at 9. An agency may use various price analysis techniques and procedures to ensure a fair and reasonable price, including the comparison of proposed prices to each other or to government cost estimates, FAR 15.404-1(b)(2); TransAtlantic Lines, LLC, B-411846.3, B-411846.4, May 18, 2016, 2016 CPD ¶ 148 at 7, and an agency may properly consider the price relative to the particular approach taken by an offeror. Marinette Marine Corp., B-400697, Jan. 12, 2009, 2009 CPD ¶ 16 at 25; Newport News Shipbuilding and Dry Dock Co. et al., B-261244.2 et al., Sept. 11, 1995, 95-2 CPD ¶ 192 at 8. The manner and depth of an agency’s price analysis is a matter committed to the discretion of the agency, and GAO will not disturb an agency’s judgment in this regard, provided it is reasonable and consistent with the solicitation’s evaluation criteria and applicable procurement statutes and regulations. Aegis Defence Servs., Ltd., B‑403226 et al., Oct. 1, 2010, 2010 CPD ¶ 238 at 5; Sig Sauer, Inc., B‑402339.3, July 23, 2010, 2010 CPD ¶ 184 at 5‑6.
Here, we find no basis to question the agency’s price reasonableness determinations. The solicitation placed offerors on notice that the agency’s cost/price evaluation would include a comparison of the offerors’ differing cost/prices, along with the causes for those differences. In this context, the record confirms that the agency divided the offerors’ technical approaches into three categories, reflecting significantly differing levels of effort; established government cost estimates for each category;[29] and compared each offeror’s cost/price to the government cost estimate associated with its proposed approach, as well as to the cost/price of other offerors with a similar technical approach. In addition, the agency compared the offerors’ cost/prices to a separate government cost estimate based on historical costs for similar work, and made note of where each offeror’s cost/price ranked in comparison to all of the other offerors. While the protesters, in effect, request that GAO conduct our own price reasonableness evaluation based on GAO’s judgment, we decline to do so. In sum, given the manner and depth of the agency’s cost/price evaluation, we do not question the agency’s determinations, and we reject the protesters’ challenges to the cost/price evaluation.
Best-Value Tradeoff
Finally, both protesters challenge the SSA’s best-value tradeoff determinations, asserting that the SSA failed to give sufficient consideration to the awardees’ cost/price premiums and/or failed to adequately document the selection decisions. More specifically, QVine asserts that the SSA “eliminated QVine’s technically acceptable proposal without giving meaningful consideration to its relative cost/price,” and characterizes the decision as “conclusory and not based upon specific, identifiable benefits to the government.”[30] QVine Protest at 2. Similarly, DAE complains about what it characterizes as the SSA’s “myopic focus on technical merit,” further asserting that the tradeoff determinations “ignored cost/price,” and maintaining that the SSA’s acceptance of “all of the awardees’ proposed price premiums” was “simply inconsistent” with the requirement to meaningfully consider cost/price. DAE Protest at 12; DAE Comments, Sept. 28, 2020, at 21-22.
The agency responds by first noting that the solicitation established the technical evaluation factor as the most important factor, and that cost/price was significantly less important than the combined non-cost/price factors. The agency further notes that each of the awardees’ proposals had at least one significant strength under the technical factor and no weaknesses; in contrast, QVine’s proposal reflected a significant weakness and DAE’s proposal contained no significant strengths. AR, Tab 2, Memorandum of Law at 21-22. Finally, the agency references the detailed analysis contained in the SSA’s SSDM, which identifies and compares the various strengths, weaknesses, benefits, and/or risks associated with each offeror’s proposal.
For example, in comparing KeyW’s proposal to QVine’s,[31] the SSA specifically referenced the significant strength in KeyW’s technical proposal (as discussed above); noted QVine’s significant weakness (as discussed above); concluded that KeyW’s technical proposal was superior to QVine’s “because QVine’s significant weakness . . . is critical to working within the Sponsor’s environment”; acknowledged the cost/price premium; and concluded that the premium “is worth the reduced risk of unsuccessful contract performance.” See AR, Tab 83, SSDM at 60-62 (quoted in AR Tab 1, Contracting Officer’s Statement at 28-29). Similarly in comparing QVine’s proposal to Asymmetrik’s,[32] the SSA referenced the multiple significant strengths in Asymmetrik’s proposal, noting its proposed use of [redacted]. Id. The SSA further acknowledged the cost/price premium associated with Asymmetrik’s proposal, concluding that it “is an acceptable premium to pay for Asymmetrik’s exceptional Technical solution” as compared to QVine’s lower-rated solution and the “risk of unsuccessful performance” related to the agency’s information security concerns. Id.
In comparing KeyW’s proposal to that of DAE, the agency similarly responds that the SSA’s best-value tradeoff noted the significant strength in KeyW’s technical proposal, and the fact that DAE’s technical proposal received no significant strengths; acknowledged that DAE had a higher rating under the less important past performance factor; and concluded that the technical superiority of KeyW’s proposal was worth the cost/price premium. Likewise, in comparing Asymmetrik’s proposal to DAE’s, the SSA considered the multiple significant strengths in Asymmetrik’s proposal; acknowledged that DAE had a higher rating under the less-important past performance factor; acknowledged the cost/price premium associated with Asymmetrik’s proposal; and concluded that the multiple significant strengths in Asymmetrik’s proposal under the most important technical evaluation factor were worth the cost/price premium.
In short, the agency maintains that the SSA properly considered the entirety of QVine’s and DAE’s proposals, compared their proposals to each of the higher-rated awardees’ proposals, and reasonably concluded that each awardee’s proposal reflected strengths and benefits for the government that, in the SSA’s judgment, were worth the associated cost/price premiums. AR, Tab 2, Memorandum of Law at 3.
In making best-value tradeoff determinations, source selection officials have broad discretion in exercising their judgment; specifically, in comparing technical merit, risk, and cost/price, the extent to which one may be sacrificed for another is subject only to rationality and consistency with the evaluation criteria. See, e.g., Diversified Tech. & Servs. of Virginia, Inc., B-412090.2, B-412090.3, Dec. 16, 2015, 2016 CPD ¶ 34 at 11; Mevatec Corp., B‑260419, May 26, 1995, 95-2 CPD ¶ 33 at 3. Accordingly, we will not question the award to higher-rated, higher-priced offerors where the agency’s documentation regarding its source selections establishes that the SSA was aware of, and considered, the relative strengths, weaknesses, and costs of the competing proposals, and made judgments that were rational and consistent with the evaluation critiera. OnPoint Consulting, B-417397.3 et al., Oct. 3, 2019, 2019 CPD ¶ 332 at 17; International Consultants, Inc.; Int’l Trade Bridge, Inc., B-278165, B‑278165.2, Jan. 5, 1998, 98-1 CPD ¶ 7 at 5-6.
Here, based on our review of the record, we find no basis to question the reasonableness of the SSA’s judgments or their consistency with the solicitation’s evaluation criteria. As noted, the solicitation specifically provided that technical was the most important evaluation factor, and that cost/price was significantly less important than the non-cost/price factors combined. Further, it is clear from the record that the SSA was aware of, considered, and documented in detail, the evaluated strengths, weaknesses, and associated cost/prices of the competing proposals. Although the protesters may disagree with the SSA’s judgments, they have not demonstrated them to be unreasonable; accordingly, we find no merit in their various assertions challenging the validity of the best-value tradeoff determinations.
The protests are denied.
Thomas H. Armstrong
General Counsel
[1] The agency awarded contracts to five offerors: Asymmetrik, Ltd., of Annapolis Junction, Maryland; BTI360, Inc., of Ashburn, Virginia; The KeyW Corporation, of Herndon, Virginia; Next Tier Concepts, Inc., of Vienna, Virginia; and Novetta, Inc., of McLean, Virginia.
[2] OSE is referred to as the “sponsor” of the procurement. The solicitation’s statement of work (SOW) explains that:
The Open Source Enterprise (OSE) is one of the oldest civilian intelligence organizations in the Intelligence Community (IC). Its predecessor organization was created in 1941 to monitor and process foreign broadcasts for the benefit of all government agencies. OSE continues this tradition today by collecting, exploiting, and disseminating PAI [publically available information] communications of interest to the IC.
Agency Report (AR), Tab 4, SOW at 1.
[3] The agency explains that OSINT “is the analysis of data and information collected from freely and publicly available sources” including “all types of media, government reports and other documents, scientific research and reports, commercial vendors of information, [and] the Internet,” and adds that “[o]pen-source information provides a base for understanding classified materials.” AR, Tab 4, SOW at 1-2.
[4] With regard to the contract requirements for “tasking, collection, processing, exploitation, and dissemination,” the agency states that these are the “five steps which constitute the Intelligence Cycle”--which is “the process of developing raw information into finished intelligence for policymakers.” AR, Tab 34, Sample Task Order (STO) SOW at 3.
[5] The agency describes OSE’s core functions as “early warning of potential crises/conflicts on the horizon; situational awareness of breaking events; bulk data feeding AI/ML [artificial intelligence/machine learning] development, exploitation, and optimization; monitoring of public sentiment/unrest worldwide; gauging international reaction to US activities; [and] tailored data for targeting, watchlisting, and sanctions enforcement.” AR, Tab 1, Contracting Officer’s Statement at 1.
[6] The solicitation established three equally weighted technical subfactors: technical automation; technical delivery; and information security. AR, Tab 37, RFP § M at 2‑3.
[7] The solicitation stated that the security factor would be evaluated on a pass/fail basis. Id. at 2.
[8] The five performance areas were: (1) Infrastructure services/applications (development, operation, and maintenance of systems on which software applications operate and the environments in which collection/dissemination of information will be performed); (2) advanced analytics (development, operation, and maintenance of software applications to shift from human-based operations to automated machine-based operations); (3) Identity/security services (development, operation, and maintenance of solutions to control user and machine access); (4) data layer/data management (development, operation, and maintenance of systems to manage information); (5) technical collection (development, operation, and maintenance of systems and services associated with the collection and dissemination of information). AR, Tab 4, SOW at 3.
[9] With reference to the 5 IDIQ performance areas, the STO’s SOW stated: “Within the five functional areas, the use of commercially available technological solutions will be preferred over custom built solutions.” AR, Tab 34, STO SOW at 2.
[10] Intelligence community directive No. 503 (or ICD 503) was issued by the Director of National Intelligence in 2008 and “establishes Intelligence Community policy for information technology systems security risk management, certification and accreditation.” AR, Tab 1, Contracting Officer’s Statement at 16.
[11] IdAM principles are defined as “the combination of technical systems, policies and processes that create, define, and govern the utilization and safeguarding of identity information, as well as managing the relationship between an entity, and the resources to which access is needed.” Id.
[12] The STO contained two contract line item numbers (CLINs): CLIN 0001 covered six work areas and was to be priced on a firm-fixed-price (FFP) basis; CLIN 0002 covered one work area and was to be priced on a cost‑plus-fixed-fee (CPFF) basis. AR, Tab 43, RFP § L at 13; AR Tab 25, Cost/Price Template.
[13] Offerors were required to submit the past performance portions of their proposals on September 26, 2019, and the remainder of their proposals on October 10. AR, Tab 43, RFP § L at 1-2.
[14] The proposals of the other offerors are not further discussed.
[15] A portion of the record in this procurement is classified. The agency made relevant classified documents available to this Office, and to the protesters’/intervenors’ representatives who hold appropriate security clearances, within the agency’s sensitive compartmentalized information facility (SCIF). Our decision here does not discuss any classified material and, accordingly, our discussion of certain issues is necessarily limited.
[16] In evaluating proposals under the technical factor, the agency assigned adjectival ratings of exceptional, very good, acceptable, marginal, or unacceptable.
[17] In evaluating proposals under the past performance factor, the agency assigned confidence ratings of high confidence, significant confidence, moderate confidence, no confidence, or neutral confidence.
[18] As noted above, the objective for both the ICD-503 requirements and IdAM principles is information security. The agency notes that “[w]hile ICD-503 concerns a comprehensive risk management framework . . . IdAM extends beyond this context as it relates to everything concerning managing digital identities, authenticating users, and authorizing access, including systems integrity and data management.” AR, Tab 1, Contracting Officer’s Statement at 17.
[19] The solicitation stated: “The Government intends to evaluate proposals and select, without discussions, an Offeror(s) for award unless the Contracting Officer determines them to be necessary.” AR, Tab 37, RFP § M at 1.
[20] Both protesters specifically challenge the awards to KeyW and Asymmetrik. See QVine Protest at 25-26; DAE Protest at 11-12.
[21] In assessing the significant strength in KeyW’s proposal, the TMET described the bases for the strength in detail, stating, in part:
[KeyW’s] proposal emphasized a mix of open source and commercial solutions that will be used for each capability in the STO SOW, and has identified how each will add value. The proposed commercial collection solutions included: [redacted], complimented by custom bot development when required. [KeyW] proposed to integrate with existing Sponsor collection capabilities [redacted]. The Offeror’s proposed solution for [redacted]. . . . All together, these are innovative commercial capabilities, and . . . represent[] an exceptional understanding of the technology landscape.
AR, Tab 63, Tech. Evaluation Report (KeyW) at 1-2.
[22] Although the SSDM is a classified document, the agency included quotations from that document in the unclassified contracting officer’s statement, explaining: “Block quotes from the . . . SSDM are being provided for the ease of all parties due to classification of documents that cannot be produced in EPDS [GAO’s electronic protest docketing system].” AR, Tab 1, Contracting Officer’s Statement at 20 n.5. Our decision here refers only to the quotations that have been included in the unclassified contracting officer’s statement.
[23] QVine’s and DAE’s protests include arguments that are in addition to, or variations of, those specifically discussed below. We have reviewed all of the protest allegations and find no basis to sustain the protests.
[24] As discussed above, IdAM principles involve “managing digital identities, authenticating users, and authorizing access, including systems integrity and data management.” AR, Tab 1, Contracting Officer’s Statement at 17.
[25] QVine states that “the core of each technical solution is the technology platform.” QVine Supp. Comments, Oct. 15, 2020, at 7. QVine and DAE both offered solutions based on COTS/GOTS platforms.
[26] The solicitation for the IDIQ contained an SOW covering the scope of the IDIQ requirements; additionally, the solicitation contained a separate STO SOW.
[27] The protesters acknowledge that all offerors’ proposed solutions, including their own, will require some amount of customization.
[28] Specifically, RFP section M states that an offeror’s technical approach will be evaluated with regard to the extent to which it: demonstrates the ability “to integrate innovative open source or commercial solutions”; “effectively balances needs for functionality, high availability, performance, and scalability”; and demonstrates the ability to “deliver innovative technical solutions in an iterative manner through the application of continuous integration and continuous delivery principles.” AR, Tab 37, RFP § M at 3.
[29] We have specifically found that an agency may establish different government cost estimates based on differing technical approaches. Marinette Marine Corp., supra.
[30] QVine also challenges the best-value tradeoff “because it rests upon the flawed assumption that QVine’s Technical Proposal contained a ‘significant weakness.’” QVine Protest at 2. As discussed above, the record provides no basis to question the agency’s determinations that: QVine’s proposal failed to meaningfully address the solicitation’s information security requirements; this flaw was neither minor nor easily corrected; and QVine’s failure to adequately address the requirement constituted a significant weakness in its proposal. Accordingly, we reject QVine’s assertion that it was improper for the SSA to consider the significant weakness in QVine’s proposal in making the best-value tradeoff determinations.
[31] As noted above, both protesters specifically challenge the award to KeyW, arguing that KeyW’s technical ratings were similar to those of the protesters, yet it offered a higher cost/price.
[32] As noted above, both protesters specifically challenge the award to Asymmetriks (the highest-priced awardee) on the basis of its cost/price premium.