Root9B, LLC

Decision

Matter of:  Root9B, LLC

File:  B-417801; B-417801.2

Date:  November 4, 2019

DIGEST

1.  Protest that agency misevaluated protester’s proposal is denied where evaluation was reasonable and consistent with solicitation’s evaluation criteria and reasonably assessed a deficiency in protester’s proposal, making it unacceptable.

2.  Protester is an interested party to challenge the agency’s price reasonableness determination, despite its lower-priced proposal being unacceptable, because the awardee was the only other offeror and, if the protest were sustained, the agency could find the awardee ineligible for award and would have to reopen the competition. 

3.  Protest that agency determination of price reasonableness was unreasonable is denied where the agency compared the awardee’s price to an independent government estimate that was based on agency’s historical costs for requirement, recent requirements by other agencies, and a reasonable estimate regarding increased future usage.

Root9B, LLC, of Colorado Springs, Colorado, a small business, protests the award of a contract to Chiron Technology Services, Inc., of Columbia, Maryland, under request for proposals (RFP) No. H98230-19-R-0081 (known as FLYREEL), issued by the National Security Agency (NSA), for training services at locations [DELETED].  Root9B argues that NSA misevaluated its proposal and awarded the contract at an unreasonable price. 

We deny the protest. 

BACKGROUND

The RFP, issued on November 26, 2018, was transmitted to seven selected firms,[1] inviting each to submit a proposal to deliver cybersecurity training courses[2] for the National Cryptologic School, under a single-award, indefinite-delivery, indefinite-quantity (IDIQ) contract with a $56 million ceiling.  The RFP provided for a 5-year ordering period, and anticipated that the agency would issue an order to the contractor for each course for its corresponding fixed price and for reimbursement of travel costs when applicable. 

The accompanying statement of work (SOW) indicated that the contractor would deliver training courses in 10 technical knowledge areas ([DELETED]), and would deliver each course at any of three levels of expertise.  RFP SOW at 2 (¶ 4.2); SOW Appendix F.  The RFP separately required the contractor to deliver intermediate cyber core (ICC) courses, which would provide the necessary academic background and proficiency to describe and navigate the digital environment.  RFP at 157-159.  The contractor would have to be able to present up to five courses concurrently in a single knowledge area, and up to 10 courses concurrently overall.  RFP SOW at 1 (¶ 4.0). 

Award would be made to the offeror whose proposal provided the best value to the government.  The RFP identified three non-price evaluation factors, listed in descending order of importance:  technical, management, and past performance.  RFP Proposal Evaluation Criteria (PEC) at 1, 6.  Those three factors, when combined, were more important than evaluated price.  Id. at 6.  The evaluation under the technical and management factors would result in adjectival ratings of outstanding, good, acceptable, marginal, or unacceptable.  Id. at 2-3.  An unacceptable rating would be assessed for a proposal evaluated as having one or more deficiencies or where the “risk of unsuccessful performance is unacceptable.”  Id. at 3. 

Within the technical and management factors, the RFP described subfactors, and within each subfactor, criteria.  Id. at 6.  The technical factor had two subfactors:  technical approach and performance risk assessment, with the former more important than the latter.  Id. at 6-7.  Under the performance risk assessment subfactor evaluation, at issue here, the agency would evaluate the offeror’s ability to assess the risk of the fixed price contract structure and account for the technical uncertainty of the course content to be delivered.  Id. at 7 (PEC ¶ 3.1.2).  In conducting the evaluation, the agency would utilize three criteria, one of which is at issue here: 

To what extent does the Offeror’s proposal account for the uncertainty in the specific nature of the classes to be delivered in each knowledge and skill level, and mitigate the performance risk created by this uncertainty? 

Id.  

NSA received proposals from only Chiron and Root9B.  After the evaluators reviewed the proposals, their individual views were synthesized into a source selection evaluation board (SSEB) consensus report.  AR, Tab 13, SSEB Consensus Report, at 1 8. 

NSA’s evaluators assessed multiple weaknesses and a deficiency in Root9B’s technical proposal.  The agency identified the deficiency under the performance risk assessment subfactor and, specifically, the above-identified criterion.  The evaluators found that Root9B did account for the uncertainties in the specific nature of the classes to be delivered and the performance risk created by this uncertainty, but failed to address strategies to mitigate the associated risks.  Id. at 7.  The SSEB found that Root9B’s proposal’s discussion of the mitigation of “content uncertainty” focused solely on the [DELETED] and its perceived ability to satisfy requirements with that [DELETED].  AR Tab 18, SSEB Recommendation, at 5.  The SSEB found that Root9B’s proposal did not:

offer the Government sufficient information related to who or how this ‘course uncertainty’ will be mitigated.  Root9B did not discuss any form of process, procedure or plan it would implement to handle multiple levels of complexity for dynamic content.

Id. 

As a result, the SSEB concluded that overall Root9B’s proposal had a material failure and posed an unacceptable risk of unsuccessful performance.  Id.  The contracting officer’s source selection memorandum summarized the same concern, and concluded that Root9B’s proposal was unacceptable and thus the contract should be awarded to Chiron.  AR Tab 19, Source Selection Decision, at 4. 

Each offeror calculated an evaluated price by multiplying the quantity specified in the RFP and the corresponding line item price, and by adding the results for each line item and each performance period.  Offerors were also required to propose a percentage burden to be added to the costs of “Base Material” and “Travel,” which the agency would reimburse; the reimbursable elements and the burden percentage were not included in the evaluated price.  AR Tab 16, Root9B Price Evaluation Memorandum, at 2; AR Tab 35, Chiron Price Evaluation Memorandum, at 2.  The agency’s price evaluation primarily confirmed the accuracy of the offeror’s price calculations, and assessed the reasonableness of the awardee’s price based on adequate competition and on comparison to an independent government cost estimate. 

The final evaluation produced the following adjectival ratings and evaluated prices:

AR, Tab 18, SSEB Recommendation Report, at 1. 

The contracting officer reviewed the evaluation and prepared a memorandum to document the conclusion that Chiron’s price was fair and reasonable, which also stated that the contract award would specify a total price of $56.0 million.[3]  On July 10, 2019, NSA awarded the contract to Chiron.  Root9B requested and received a debriefing and an extended debriefing, after which it filed this protest. 

DISCUSSION

Root9B argues that NSA misevaluated its proposal as unacceptable under the performance risk subfactor, which resulted in an unacceptable rating overall for the technical factor.  Root9B also argues that NSA improperly determined Chiron’s price was reasonable. 

Evaluation under the Technical Factor

Root9B challenges the evaluation of its technical proposal on multiple grounds, arguing that NSA unreasonably assessed weaknesses and a deficiency.  As relevant here, the firm challenges the assessment of the deficiency to its proposal under the performance risk assessment subfactor, for not adequately addressing mitigation strategies for uncertainty in the specific nature of the classes to be delivered.  Protest, attch. C, Debriefing to Root9B, at 18.[4]  As explained below, based on our review of the record, NSA’s evaluation was reasonable and there is no evidence of unequal treatment. 

Citing NSA’s assessment of a deficiency for Root9B’s failure to adequately address uncertainty in the “specific nature of the classes to be delivered,” the protester argues that the RFP directed it to focus on providing required courses despite not knowing which courses NSA might order.  Root9B argues it provided an approach to performing course currency task orders, but the RFP did not seek specific approaches to particular course content or specific topics.  Root9B argues that the RFP provided specific direction to address uncertainty in the specific nature of the classes to be delivered in relation to the different skill levels, and to address the uncertainty at a topical level.  Root9B contends that its proposal explained its ability to provide multiple courses on any particular topic area at varying skill levels at the same time in multiple locations, the depth of its team of instructors, and its ability to provide facilities in all potential class locations.  The proposal separately addressed the firm’s ability to implement course currency delivery orders[5] using instructors, instructional design engineers, technical writers and editors, and software to track the progress of those efforts.  See AR Tab 9, Root9B Technical Proposal, at 10.  Root9B argues that downgrading its proposal for failure to address specific topics was improper because it was inconsistent with the RFP evaluation criteria.  Protester’s Comments at 2. 

NSA argues that Root9B’s proposal did not provide a sufficient approach to addressing risk mitigation associated with dynamic course topics, which justified the assessment of a deficiency under the performance risk subfactor.  AR Legal Memorandum at 2.  NSA argues that the RFP instructed offerors to account for the specific nature of the classes to be delivered in each knowledge and skill level, and to mitigate performance risks created by these uncertainties, and that the agency properly assessed a deficiency because Root9B had not satisfactorily addressed how it would mitigate the risk associated with providing current course content for dynamic topic areas.  Contracting Officer’s Statement at 11. 

An agency’s method for evaluating the relative merits of competing proposals is a matter within the agency’s discretion, since the agency is responsible for defining its needs, and the best method for accommodating them.  Arctic Slope World Servs., Inc., B-284481, B-284481.2, Apr. 27, 2000, 2000 CPD ¶ 75 at 5.  In reviewing protests challenging an agency’s evaluation of proposals, our Office does not reevaluate proposals or substitute our judgment for that of agency evaluators; rather, we review the record to determine whether the agency’s evaluation was reasonable and consistent with the solicitation’s evaluation criteria, as well as applicable statutes and regulations.  Veterans Evaluation Servs., Inc. et al., B-412940 et al., July 13, 2016, 2016 CPD ¶ 185 at 5.  It is fundamental that the contracting agency must treat all offerors equally, and therefore it must evaluate offers evenhandedly against common requirements and evaluation criteria.  Walsh Constr. Co. II, LLC, B-410015 et al., Sept. 25, 2014, 2014 CPD ¶ 291 at 6. 

In our view, the contemporaneous record supports the reasonableness of NSA’s evaluation of Root9B’s proposal as unacceptable.  The criterion at issue required offerors to both account for the uncertainty in the specific nature of classes to be delivered and to mitigate the performance risks created by this uncertainty.  PEC at 7 (¶ 3.1.2).  The record shows that NSA expressly recognized that Root9B’s proposal “accounted for the uncertainties in the specific nature of the classes to be delivered in each knowledge and skill level,” and identified resulting performance risks, such as the contractor’s inability [DELETED].  AR, Tab 13, SSEB Consensus Report, at 7.  However, NSA found the proposal deficient because it did not describe a sufficient approach to mitigating the risks.  Id. at 1, 6, 7.  The proposal did not provide detail other than having a [DELETED] to address uncertainty, and did not describe who would mitigate issues and how those issues would be mitigated.  Id. at 7.  As an example of the need for mitigation measures, NSA cited a potential course that was dynamic and constantly evolving, versus a course that was mostly static, and stated that Root9B mentioned no [DELETED] on how to mitigate the risk of providing training of dynamic content.  Id.  In our view, NSA’s assessment of the deficiency was reasonable and consistent with the evaluation criteria. 

The record also fails to support Root9B’s challenge to the evaluation as resulting from unequal evaluation.  In contrast to Root9B’s failure to address course currency, Chiron proposed five triggers that would be used to identify and prioritize changes to course content outside of the normal maintenance cycle:  [DELETED].  AR Tab 28, Chiron Technical Proposal, at 27-28.  Chiron also proposed to extend greater communication [DELETED] that would then be used to recommend course updates.  Id. at 31.  Accordingly, we deny Root9B’s challenges to the technical evaluation of its proposal as unacceptable.
Price Reasonableness Determination

Root9B next argues that NSA made an unreasonable determination that Chiron’s price was reasonable.

As an initial matter, we regard Root9B as an interested party to pursue this protest despite our conclusion that its technical proposal was properly evaluated as unacceptable.  A protester whose proposal is found to be technically unacceptable is an interested party to challenge the eligibility of an awardee, where, as here, the exclusion of the awardee would result in no offerors being eligible for award.  TCG, Inc., B 417610, B-417610.2, Sept. 3, 2019, 2019 CPD ¶ 312 at 7.  Thus, where an agency properly finds the protester’s proposal unacceptable, that firm may nevertheless be an interested party to protest the agency’s determination that the awardee’s price was reasonable.  Id.  In the circumstances here, Chiron and Root9B were the only offerors, so in the absence of a reasoned determination that Chiron’s price was reasonable, NSA could not have awarded the contract to the firm. 

Root9B argues that NSA improperly based its price reasonableness determination of Chiron’s proposal on the mere fact that competing proposals--i.e., more than one proposal--had been submitted, and because the agency’s comparison of Chiron’s price to the independent government estimate was improper because the estimate is factually unsupported.  Comments & Supp. Protest at 16-17; Supp. Comments at 11-12.

NSA counters that its price reasonableness determination was reasonable and meets the applicable FAR requirements for this procurement.  NSA asserts that the price reasonableness determination properly recognized that Chiron’s price was obtained through competition in which the agency received multiple proposals, and that it reasonably assessed price reasonableness in recognizing that Chiron’s price was approximately four percent above the government estimate.  NSA also contends that its estimate was a valid means to assess the reasonableness of Chiron’s evaluated price.  Supp. AR Memorandum of Law at 4-5. 

Agencies are required to ensure that award of any contract is at a fair and reasonable price.  See FAR § 15.404-1 (“The objective of proposal analysis is to ensure that the final agreed-to price is fair and reasonable”).  The purpose of a price reasonableness evaluation in a fixed-price contract setting is to determine whether prices are too high.  The FAR recognizes the use of various price analysis techniques and procedures to ensure fair and reasonable pricing, including the comparison of proposed prices to those received in adequate competition, or to an independent government estimate.  FAR § 15.404-1(b)(2).  A price reasonableness determination is a matter of administrative discretion involving the exercise of business judgment by the contracting officer that we will question only where it is unreasonable.  Dewberry Crawford Group; Partner 4 Recovery, B-415940.10 et al., July 2, 2018, 2018 CPD ¶ 297 at 21.  An agency may consider a comparison to an independent government estimate to establish prices as fair and reasonable, so long as the estimate is itself reasonable and factually supported.  Black Hills Refuse Serv., B-228470, Feb. 16, 1988, 88-1 CPD ¶ 151 at 4-5. 

Although Root9B challenges the agency’s independent government estimate as being factually unsupported, our review shows that the agency calculated an estimate of its own training requirement using burdened costs of providing the training, which it then extended for the five years of the FLYREEL contract, resulting in a total of $37.1 million.  Supp. AR Tab 37, Independent Government Cost Estimate, at 1.  To that amount, NSA added $18.9 million, which, it explains, corresponds to funding the agency expects to receive from other agencies through military interdepartmental purchase requests.  Supp. AR Tab 39, Contracting Officer Supp. Statement, at 2.  The contracting officer states that this additional amount was calculated from the funding that NSA had received from other agencies for training courses in fiscal year 2016, plus an additional 17 percent to reflect the expectation that those agencies would have a higher demand for National Cryptologic School classes during the FLYREEL contract.  AR Tab 39, Supp. Contracting Officer’s Statement at 2.  The resulting combined estimate thus totaled $56.0 million. 

The record supports the reasonableness of NSA’s judgment that Chiron’s price was fair and reasonable.  In addition to considering that the price resulted from what the agency deemed to be adequate competition,[6] the agency compared Chiron’s price to an independent government cost estimate.  AR Tab 17, Price Reasonableness Memorandum, at 1.  The price reasonableness determination explained that Chiron’s total evaluated priced differed from the government estimate by about four percent, and that this fact “indicate[d] that the proposed price . . . is not unreasonable.”  Id.  The determination also noted that Chiron’s proposal “fully reflect[ed] the requirements . . . in every way.”  Id.  Although Root9B questions whether the agency’s estimate had sufficient factual support for the inclusion of funding NSA expects to receive from other agencies, the contracting officer explained that this element of the agency’s estimate was derived from the funding received in a previous fiscal year, and the agency’s best estimate that demand by those agencies would increase by 17 percent.  Even though the additional sum is not based on a detailed calculation of the costs of individual training classes, NSA used historical information and its best estimate to predict a reasonable increase in future requirements from other agencies.  We have no basis to question the agency’s estimate, or the conclusion that the comparison to Chiron’s price was sufficient for the contracting officer to conclude that the contract price was fair and reasonable. 

The protest is denied. 

Thomas H. Armstrong
General Counsel