Booz Allen Hamilton, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Booz Allen Hamilton, Inc.

File: B-420535; B-420535.2

Date: May 24, 2022

DIGEST

1. Agency reasonably identified multiple significant weaknesses in the protester’s response to the solicitation’s “challenge scenario” and rated the proposal unacceptable, thereby rendering it ineligible for award.

2. Based on the agency’s reasonable determination that protester’s proposal was ineligible for award, protester is not an interested party to further challenge the agency’s source selection process.

Booz Allen Hamilton, Inc. (BAH), of McLean, Virginia, protests the decision by the Department of Defense, Defense Health Agency, to award indefinite-delivery indefinite‑quantity (IDIQ) contracts and associated task orders to BCG Federal Corporation (BCG), Capgemini Government Solutions, LLC (Capgemini), and Deloitte Consulting, LLP (Deloitte), pursuant to request for proposals (RFP) No. HT003821R0001, to support the agency’s efforts to become a “world-class technology organization.” Agency Report (AR) Tab 24, RFP attach. 1, Scope and Ordering Guide at 4, 6. BAH challenges various aspects of the agency’s source selection process, including the agency’s determination that BAH’s proposal was unacceptable.

We deny the protest in part and dismiss it in part.

BACKGROUND

On April 22, 2021, the agency issued the solicitation,[1] seeking proposals for IDIQ contracts and associated task orders to provide “managed technology and organizational contract solutions intended to challenge PEO DHMS’ status quo and shape a more innovative [and] agile . . . workforce.”[2] AR, Tab 55, RFP at 76.[3] The solicitation contemplated awards on the basis of best-value tradeoffs and identified the following evaluation factors: (1) gate criteria; (2) transformation approach; (3) behavior model; (4) price; and (5) challenge scenario.[4] Finally, the solicitation provided that an offeror’s proposal “shall be clear, concise, and shall include sufficient detail for effective evaluation.” Id. at 42.

The agency’s evaluation with regard to the factor 5 challenge scenario is particularly relevant to this protest. In this regard, the solicitation provided that all competitive range offerors would be invited to prepare, record, and submit 60-minute “live session” responses to an agency-provided challenge scenario (defined as a “theoretical scenario requiring digital transformation”[5]) in order to, among other things, “demonstrate their ability to facilitate a productive and collaborative environment necessary for successful performance.” RFP at 54-55. Although the solicitation stated that offerors would be given advance notice as to the general area of the challenge scenario, it also provided that “[t]he details of the challenge will not be provided until the start of the Live Session.” Id. at 54.

The solicitation established ground rules for the challenge scenario, including the following: each offeror must provide a location for its live session,[6] along with the “appropriate facilitation technology, methods, and materials to ideate and present the response solution”;[7] each offeror must have at least one, but no more than three, representatives present at its designated location, with up to seven additional representatives attending virtually;[8] each offeror “will be expected to spend the first 50 minutes of [their response] collaborating on the solution and the final 10 minutes presenting the solution”; a timestamp must be visible throughout the video to demonstrate that it was a single, straight-through recording of the presentation; and within 72 hours after the live session began, each offeror must submit a “single video recording file” to the agency for evaluation. Id. at 54-55.

With regard to evaluation of offeror responses to the factor 5 challenge scenario, the solicitation provided that the agency would assess the quality of the proposed solution as well as how the offeror arrived at its solution. More specifically, offerors were advised that their responses would be evaluated against the following elements: understanding;[9] reachback/facilitation;[10] innovation;[11] and solution/pitch.[12] Id. at 66-67.

On or before the June 2 closing date, initial proposals were submitted by five offerors, including BAH. Thereafter, the agency established a competitive range consisting of BAH and the three offerors subsequently selected for award. On October 23, the agency opened discussions and provided a due date for submission of proposal revisions. The agency also invited the competitive range offerors to subsequently participate in live sessions responding to the factor 5 challenge scenario, and advised them that the challenge scenario would address the general area of quality control. AR, Tab 25, RFP amend. 3.

On November 1, each of the competitive range offerors participated in a live session responding to the factor 5 challenge scenario. At the beginning of each session, agency representatives at each offeror-provided location handed the offeror an envelope containing the following details of the challenge scenario:

Scenario: The technology portfolio for PEO DHMS continues to expand and move towards becoming a “world-class technology organization” rivaling leading commercial technology organizations. This presents scalability challenges as existing product quality control processes create bottlenecks and reduce development velocity. Current quality control processes and tools only target clearly-defined software test cases rather than a more comprehensive approach to enhancing product quality and customer delight.

Challenge: Propose an approach to designing, implementing, and administering a refreshed strategy that will increase development velocity, mitigate schedule bottlenecks, improve product quality, and enhance customer outcomes. Include details on technical tooling, process improvement, and how to integrate overarching governance frameworks to satisfy existing PEO reporting obligations. Discuss how this approach helps the PEO accomplish its goal of becoming a “world-class technology organization.”

AR, Tab 27, Challenge Scenario.

Proposal revisions were submitted on November 2 and each offeror’s challenge scenario video was submitted on November 4. Thereafter, the proposals were evaluated.

In evaluating the offerors’ videos under factor 5, the agency evaluation team met in a conference room where each video was projected onto a 100-inch screen with a high definition projector. AR, Tab 54, Declaration of Evaluation Team Chair at 1. The evaluation team sat approximately 8-10 feet from the screen and did not zoom in on, or otherwise enhance, any of the video recordings. Id. In the BAH video, a “mural board” was displayed behind the BAH in‑person participants;[13] however, most of the input on the mural board was illegible “given the size of the notes.”[14] AR, Tab 1, Contracting Officer’s Statement at 23; see AR Tab 54, Declaration of Evaluation Team Chair at 1.

Following the agency’s evaluations, the proposals were rated as follows:[15]

	Factor 3 Behavior Model	Factor 5 Challenge Scenario	Factor 4 Price
BAH	Good	Unacceptable[16]	$179,666,652
BCG	Good	Outstanding	$590,727,655
Capgemini	Good	Acceptable	$801,167,991
Deloitte	Good	Acceptable	$584,294,459

 

AR, Tab 45, Source Selection Decision Document (SSDD) at 5.

In evaluating BAH’s proposal as unacceptable under factor 5, the agency identified one strength, one weakness, and four significant weaknesses in BAH’s response to the challenge scenario. More specifically, the agency identified a significant weakness with regard to each of the solicitation’s four evaluation elements: understanding; reachback/facilitation; innovation; and solution/pitch. Further, the agency determined that the significant weaknesses in BAH’s proposal created an unacceptable risk of unsuccessful performance, rendering the proposal unacceptable and ineligible for award. AR, Tab 41, Source Selection Evaluation Board (SSEB) Report at 19.

In assessing a significant weakness in BAH’s proposal with regard to understanding, the agency stated that BAH’s response “identified problems with testing procedures,” but failed to recognize that testing “represents a symptom of underlying and fundamental issues at PEO DHMS rather than an issue itself.” Id. at 18. Noting that the challenge scenario specifically advised offerors that “current quality control processes and tools only target clearly-defined software test cases rather than a more comprehensive approach,” the agency concluded that BAH’s “focus on testing” to achieve “short-term results . . . rather than a more comprehensive approach” presented “appreciable risk to contract performance and will hinder PEO DHMS’s ability to become a world-class technology organization.” Id.

In assessing a significant weakness in BAH’s proposal with regard to reachback/ facilitation, the agency concluded that BAH’s response did not “maximize[] the capabilities of the full team.” Id. More specifically, the agency noted that BAH’s solution was “dominated by team members who were in the room, to the exclusion of virtual participants” and that two participants in particular “dominated ideation for all components considered in BAH’s facilitation process.”[17] Id. The agency concluded that “because [BAH] did not allow the resulting solution to benefit from the different and important expertise of the other team members,” its approach “appreciably increases the risk of unsuccessful contract performance.”[18] Id.

With regard to innovation, the agency assigned BAH’s response both a strength and a significant weakness. More specifically, the agency assessed a strength in BAH’s response for its identification of several “non-traditional technologies and methodologies.” Id. at 17. However, the agency also assessed a significant weakness on the basis that BAH “did not progress beyond the Ideation phase”; failed to discuss how the technologies or methodologies would be implemented; and did not discuss the “risks and mitigation strategies” associated with implementation. Id. at 18. As noted above, the solicitation advised offerors that the agency would evaluate an offeror’s ability to incorporate “disruptive, leading, and emergent techniques . . . including non-traditional technologies” in its proposed solution. RFP at 66. Accordingly, the agency concluded that BAH’s “incomplete process . . . appreciably increases the risk of unsuccessful contract performance.” AR, Tab 41, SSEB Report at 18.

Finally, in assessing a significant weakness with regard to solution/pitch, the agency found that, because BAH’s solution failed to include a plan to implement several of its ideas, it failed to “successfully persuade Government stakeholders on the viability of the plan.” Id. at 19. Accordingly, the agency concluded that this failure “appreciably increases the risk of unsuccessful contract performance.” Id.

Based on its evaluation of the proposals, the agency selected BCG, Capgemini, and Deloitte for award of IDIQ contracts, and eliminated BAH’s proposal from further consideration. On January 27, 2022, BAH was advised of the agency’s source selection decisions. AR, Tab 46, Notice to Unsuccessful Offeror. Subsequently, the agency provided comprehensive information to BAH regarding the basis for its elimination from the competition. AR, Tab 47, BAH Debrief; Tab 48, In-person Debriefing Slides; Tab 49, Enhanced Debrief Questions/Responses. This protest followed.

DISCUSSION

BAH protests various aspects of the agency’s source selection process, including assertions that: BAH’s response to the challenge scenario should have been evaluated as acceptable; the agency’s evaluation of BAH’s proposal under factor 3, behavior model was flawed; the agency’s price evaluation was flawed; and the agency’s best- value determinations were unreasonable. As discussed below, we find no basis to sustain BAH’s protest.

Evaluation of BAH’s Response to the Challenge Scenario

First, BAH challenges the agency’s assessment of each weakness and significant weakness identified by the agency regarding BAH’s response to the challenge scenario, asserting that all of them are unreasonable. We have reviewed the agency record regarding the evaluation, specifically including the video submitted by BAH for evaluation, and find no basis to question any of the agency’s assessments.

For example, with regard to understanding, BAH complains that the agency’s assessment of a significant weakness was improper because the agency inaccurately concluded that BAH’s response to the challenge scenario focused on testing--which the agency viewed as only a symptom of the underlying issues. BAH first asserts that, rather than focusing on testing, BAH “spent the large majority of [its] presentation . . . discussing culture.” Protest at 22-26. While BAH acknowledges that its presentation made multiple references to testing, it maintains that such references “were in the context of discussing associated benefits of improved tools and approaches,” and did not constitute the “focus” of BAH’s presentation. Id. at 24. Finally, BAH asserts that the agency’s assessment reflects an “Agency-created distinction between testing as a symptom of an issue and testing as an issue itself,” complaining that “[n]owhere does the Solicitation distinguish an issue from a symptom.” Id.

The agency first responds to BAH’s assertion that BAH spent the “majority of its presentation” discussing culture. In this regard, the agency maintains that BAH’s discussion of culture was limited to “generic” challenges faced by industry and, specifically, that the discussion “overlooked organizational culture as an underlying issue at PEO DHMS”. AR, Tab 1, Contracting Officer’s Statement at 17. Since BAH elected not to discuss cultural issues at PEO DHMS, the agency maintains it would have been improper to augment BAH’s response by inferring such information. Id.

Next, contrary to BAH’s assertions that the solicitation did not distinguish between underlying issues and symptoms, the agency maintains that the solicitation advised offerors that “organizational culture” is an “underlying issue at PEO DHMS.”[19] Id. at 16‑17. Further, the agency notes that the challenge scenario criticized the current quality control processes at PEO DHMS, stating that, “current quality control processes and tools only target clearly-defined software test cases rather than a more comprehensive approach,” and warning offerors that “approaches that focus on achieving short‑term results and . . . incident management rather than underlying issues may present a performance risk.” Id. at 15-16; see RFP at 66. Notwithstanding the clear statements in the solicitation, the agency notes that, in responding to the challenge scenario, BAH’s representatives effectively discounted organizational culture as an underlying issue, stating, “The culture is coming along”--and then repeatedly referred to software testing procedures as the problem. AR, Tab 1, Contracting Officer’s Statement at 16-20 (citing to AR, Tab 35, BAH Video at 8:10–9:35, 11:54‑13:23, 18:39‑19:00, 22:20-22:30, 27:16-28:48, 46:07-47:08).

Overall, the agency maintains that it properly determined that BAH’s approach was focused on “short-term results” and “incident management” rather than a more comprehensive approach to address underlying issues--as specifically sought by the solicitation. Accordingly, the agency maintains that it properly identified a significant weakness in BAH’s response due to BAH’s failure to meaningfully address the underlying issues at PEO DHMS, and that BAH’s approach would appreciably increase the risk of unsuccessful contract performance.

In reviewing protests challenging the evaluation of proposals, we do not conduct a new evaluation or substitute our judgment for that of the agency; rather, we examine the record to determine whether the agency’s judgment was reasonable and in accord with the terms of the RFP. Watts-Obayashi, JV; Black Constr. Corp., B-409391 et al., Apr. 4, 2014, 2014 CPD ¶ 122 at 9. A protester’s disagreement with the agency’s judgment, without more, is insufficient to establish that an agency acted unreasonably. 22nd Century Techs., Inc., B-413210, B-413210.2, Sept. 2, 2016, 2016 CPD ¶ 306 at 8.

Here, based on our review of the record, we find no basis to question the agency’s evaluation of BAH’s response to the challenge scenario with regard to understanding. First, as noted above, the solicitation clearly advised offerors that the agency was not seeking short-term results or incident management but, rather, sought solutions that recognized and targeted broader, underlying issues. In this context, offerors were expressly advised that the agency was struggling to recruit and retain a talented workforce; noted that existing policies create “significant barriers” and degrade morale; and concluded that “the level or organizational transformation required is substantial.” Further, the challenge scenario expressly criticized the agency’s current quality control processes for focusing on narrow solutions, such as software testing, as opposed to pursuing a more comprehensive approach. Finally, the solicitation warned offerors that a solution focused on narrow, short-term results, rather than underlying issues, could be viewed as presenting risk.

Our review of the record, including BAH’s video, provides no basis for us to question the agency’s determination that BAH focused on narrow, short‑term results, such as software testing, rather than meaningfully addressing broader, underlying issues such as the specific organizational culture at PEO DHMS. Accordingly, we find no basis to question the agency’s assessment that BAH’s response to the challenge scenario would appreciably increase the risk of unsuccessful contract performance and warranted assessment of a significant weakness. BAH’s protest arguments challenging those assessments are without merit.

By way of another example, BAH complains that the agency improperly assessed a significant weakness with regard to reachback/facilitation. As discussed above, the agency found that BAH’s response to the challenge scenario failed to maximize the capabilities of the full team and that “the vast majority of the discussion and development of the solution was dominated by [two] team members who were in the room, to the exclusion of virtual participants.”[20] AR, Tab 41, SSEB Report at 18. BAH disputes the agency’s assessments, maintaining that all of BAH’s team, including the virtual participants, “meaningfully contributed to the team discussion,” and asserting that all of the team’s contributions provided “a benefit to the Government.” Protest at 26-27. BAH acknowledges that BAH “decided as a team that [redacted] has a world class approach to quality control,” and opted to “reach back to their [redacted] partner for her expertise when appropriate,” further acknowledging that the [redacted] representative “led the parts of the discussion where [redacted]-specific process, innovations and methodologies” were introduced. Id. at 30-31. Nonetheless, BAH questions why “the team’s reliance on recommendations from [redacted]” were viewed by the agency as a negative, asserting that the [redacted] representative provided input that was “sound [and] responsive to the challenge.” Id. BAH also refers to its mural board, asserting that the virtual participants participated by inserting notes during the presentation, and complains that the agency improperly failed to consider that input. Id. at 28. Overall, BAH asserts that the agency’s assessment of a significant weakness was “unreasonable and inconsistent with the [s]olicitation.” Id at 30.

The agency first responds that the solicitation specifically advised offerors that the agency would evaluate “the extent to which the Offeror facilitates a productive yet collaborative team environment to maximize the capabilities of the full team in developing a responsive solution.” AR, Tab 1, Contracting Officer’s Statement at 23; see RFP at 66. (Emphasis added.) Further, the solicitation warned that “[a]pproaches where a single member or party dominates the facilitation, ideation, and presentation of the solution without inclusive participation amongst other participants . . . may present a performance risk.” Id. at 27; see RFP at 66. In this context, the agency notes that BAH’s in-room participants had more than twice as much speaking time as the virtual participants, and maintains that a review of BAH’s video confirms the agency’s determination that BAH’s solution consisted primarily of ideas from the [redacted] representative. AR, Tab 1, Contracting Officer’s Statement at 23-28 (citing to AR Tab 35, BAH Video at 10:37-10:41,33:23-35:48, 54:11, 55:30-56:50). Accordingly, the agency reiterates its conclusion that BAH’s presentation reflected heavy reliance on, and domination by, the representatives from [redacted] and, to a lesser extent, [redacted] to the exclusion of the virtual participants. Id.

With regard to its consideration of information on BAH’s mural board, the agency first notes that the solicitation required an offeror to ensure that its proposal was clear and legible. See RFP at 42. (“The proposal shall be clear, concise, and shall include sufficient detail for effective evaluation.”) Yet, despite reviewing BAH’s video on a high‑definition 100-inch screen, most of the information on the mural board was too small to be legible; accordingly, unless BAH’s participants specifically discussed information on the board, the agency evaluators were unable to consider it. AR, Tab 1, Contracting Officer’s Statement at 29. In this context, the agency maintains that if BAH intended for the agency to rely on the mural board content to demonstrate team member contributions, it was required to ensure that the information provided was clear and legible--but it did not. Finally, in light of the solicitation requirement that offerors submit a “single video recording file,” see RFP at 55, the agency maintains that it properly declined to augment BAH’s proposal by accessing the hyperlink BAH identified when it submitted its video. AR, Tab 1, Contracting Officer’s Statement at 29; see AR, Tab 34, BAH Cover Letter. Accordingly, the agency maintains it properly identified a significant weakness with regard to reachback/facilitation and that BAH’s approach appreciably increased the risk of unsuccessful contract performance.

As noted above, GAO will not conduct a new evaluation or substitute our judgment for that of the agency. Watts-Obayashi, JV; Black Constr. Corp., supra. Moreover, it is an offeror’s responsibility to submit an adequately prepared proposal that allows for a meaningful review by the procuring agency, and a firm runs the risk of having its proposal rejected as unacceptable if it fails to do so. See, e.g., ACC Constr.-McKnight JV, LLC, B-411073, Apr. 30, 2015, 2015 CPD ¶ 147 at 5; Hallmark Capital Group, LLC, B-408661.3 et al., Mar. 31, 2014, 2014 CPD ¶ 115 at 9.

Here, based on our review of the record, including the BAH video, we do not question the agency’s assessment of a significant weakness in BAH’s proposal due to its failure to maximize the capabilities of its full team. As noted above, the solicitation provided that the agency would evaluate the extent to which an offeror maximized the capabilities of its full team and warned that domination of the presentation by a single member could be viewed as creating risk. Here, the record reasonably supports the agency’s determination that BAH’s presentation was dominated by the participant representing [redacted] and, to a lesser extent, the participant representing [redacted].

With regard to the agency’s consideration of information on BAH’s mural board, the solicitation specifically advised offerors of their responsibility to submit proposals that clearly communicated information the offeror sought to have evaluated. Here, our review of the record provides no basis to question the agency’s determination that, despite being viewed on a high-definition 100-inch screen, most of the information presented on the mural board in BAH’s video was not legible. Further, the agency’s attempts to fully consider all of the information presented were clearly reasonable and consistent with terms of the solicitation.[21] In this regard, BAH’s attempt to augment its video submission by providing access to a hyperlink was precluded by the terms of the solicitation’s requirement that offerors submit a “single video recording file.”[22] On the record here, we find nothing unreasonable in the agency’s assessment of a significant weakness in BAH’s response with regard to reachback/facilitation and its determination that BAH’s approach appreciably increased the risk of unsuccessful contract performance.

In summary, we have considered all of BAH’s allegations regarding the agency’s assessment of a weakness and four significant weaknesses in BAH’s proposal under the factor 5 challenge scenario, and find no basis to question the basis for any of the agency’s judgments.[23] Accordingly, consistent with the terms of the solicitation we find no basis to question the agency’s determination that the significant weaknesses created an unacceptable level of risk for successful contract performance and, consistent with the terms of the solicitation, rendered the proposal ineligible for award. BAH’s protest challenging the agency’s evaluation under factor 5 is denied.

BAH’s Other Protest Allegations

Finally, as noted above, BAH’s protest challenges other aspects of the agency’s source selection process, including assertions that the agency failed to recognize strengths in BAH’s proposal under factor 3, behavior model; the agency’s price evaluation was flawed; and the agency’s best-value determinations were unreasonable. The agency maintains that, in light of its reasonable determination that BAH’s proposal was unacceptable under factor 5, BAH does not qualify as an interested party to challenge other aspects of the agency’s source selection process. We agree.

Our Bid Protest Regulations define an interested party as an actual or prospective bidder or offeror whose direct economic interest would be affected by the award of a contract or the failure to award a contract. 4 C.F.R. § 21.0(a)(1). As discussed above, the solicitation here provided that if a proposal received an unacceptable rating under evaluation factor 5, the proposal would be ineligible for award. Since we have found no basis to question the agency’s evaluation of BAH’s proposal as unacceptable under evaluation factor 5, BAH does not qualify as an interested party to further challenge the procurement. Accordingly, BAH’s additional allegations are dismissed.

The protest is denied in part and dismissed in part.

Edda Emmanuelli Perez
General Counsel

 

---