Computer Sciences Corporation

Decision

Matter of:  Computer Sciences Corporation

File:  B-412163.3; B-412163.4

Date:  January 4, 2016

DIGEST

Protest challenging agency’s findings regarding technical merit of protester’s proposal is denied where record reflects agency’s findings were reasonable and consistent with solicitation’s evaluation criteria.

Computer Sciences Corporation (CSC), of Falls Church, Virginia, protests the issuance of a task order to Vencore Services and Solutions, Inc., of Reston, Virginia, under task order request for proposals (RFP) No. HSSCCG-15-R-00003 issued by the Department of Homeland Security, United States Citizenship and Immigration Services (USCIS), for Agile development and maintenance services to support agency information technology (IT) systems.  CSC alleges that the agency’s evaluation of proposals was unreasonable in several respects and that its source selection decision was flawed.

We deny the protest.

BACKGROUND

On February 17, 2015, the agency issued the solicitation under the provisions of Federal Acquisition Regulation subpart 16.5 to firms holding contracts under functional category 1 (service delivery) of an existing multiple-award, indefinite-delivery, indefinite-quantity contract vehicle known as second-generation Enterprise Acquisition Gateway for Leading-Edge Solutions (EAGLE II).  See RFP at 1.[1]  The solicitation sought services to support the agency’s joint engineering teams--sustainment (JETS) program.  See id.  The solicitation included a performance work statement (PWS) specifying, at the highest level, that the JETS contractors would be responsible for providing “high-productivity Agile development services and Lean processes focused on IT sustainment of applications that support [the agency’s] core business areas.”  PWS § 3.

The solicitation anticipated the award of four fixed-price[2] task orders, each with a six‑month base period and four one-year option periods.  RFP at 2-9, 19.  The task orders were for work in four mission-area “portfolios.”  The portfolios were records, benefits, customer service, and biometrics.[3]  Id.  This protest concerns only the benefits portfolio.

Each award was to be made on a best-value tradeoff basis, considering price and the following four equally-weighted nonprice factors:  corporate experience of the prime contractor; technical approach of the contractor team; management approach of the contractor team; and oral presentation.  RFP at 19-20.  The solicitation stated that the nonprice factors, when combined, were more important than price.  Id. at 19.  Additionally, the solicitation included evaluation criteria for each nonprice factor, and stated that under these factors, the agency would assign adjectival ratings of high confidence, some confidence, or low confidence.  Id. at 19-20.  Finally, the solicitation stated that the agency would employ a multistep source selection process under which only those offerors whose proposals were evaluated as the “most likely to provide the best value solutions” under factors 1 (corporate experience), 2 (technical approach), 3 (management approach), and 5 (price) would be further considered for award and invited to give oral presentations under factor 4.  Id. at 20.

The agency received numerous proposals in response to the solicitation, including proposals from CSC and Vencore.  Agency Report (AR), Tab 32, Source Selection Decision Document (SSDD), at 1, 7.  A technical evaluation committee (TEC) evaluated the proposals under factors 1, 2, and 3 and assigned adjectival ratings.  See id.  Based on the TEC’s evaluations, as well as a price evaluation by a business evaluation committee, the source selection authority (SSA) selected 10 offerors to provide oral presentations under factor 4.  See AR, Tab 32, SSDD, at 1.  Of those 10 offerors, 3 submitted proposals for the benefits portfolio, including CSC and Vencore.  Id. at 4.

Following the oral presentations, the TEC completed its evaluation.  The adjectival ratings and total evaluated pricing for the proposals submitted for the benefits portfolio are shown in the table below.

	CSC	Vencore	Offeror 3
Factor 1-- Corporate Experience	High Confidence	High Confidence	[DELETED]
Factor 2-- Technical Approach	High Confidence	Some Confidence	[DELETED]
Factor 3-- Management Approach	Some Confidence	High Confidence	[DELETED]
Factor 4-- Oral Presentation	High Confidence	High Confidence	[DELETED]
Factor 5--Price	$89.6 million	$107.2 million	[DELETED]

AR, Tab 32, SSDD, at 4.

The SSA documented a source selection decision for each portfolio.  For the benefits portfolio, the SSA first documented his conclusion that offeror 3’s proposal did not represent the best value to the government.  AR, Tab 32, SSDD, at 4.  He then documented a lengthy tradeoff analysis explaining why, in his view, Vencore’s proposal represented the best value to the government, notwithstanding CSC’s lower evaluated price.  Id. at 4-6.  For example, the SSA cited the following concerns regarding CSC’s proposal:

[S]ome of CSC’s thinking around when to use Scrum versus Kanban and the possible use of CSC SMEs [subject matter experts] were troublesome; in particular the idea that the predictability of work is the primary factor in the Scrum/Kanban decision reduces my confidence in CSC’s understanding of the techniques and, correspondingly, its ability to achieve short lead times through continuous delivery.

Id. at 4.  With regard to Vencore, the SSA noted numerous specific strengths that the TEC identified for the firm’s proposal under factor 2, technical approach, and factor 3, management approach.  Id. at 5.  The SSA also documented the following:

Although the TEC had rated Vencore as Some Confidence for Factor 2, I found that their considerable strengths in technical approach could have justified a higher rating.  These strengths in Factors 2 and 3 give me a . . . significantly higher confidence than I have for CSC.  Vencore’s explanations of how it uses Kanban and metrics in its O&M [operations and maintenance] practices also give me high confidence in its management approach, again, a higher confidence than I have for CSC.

AR, Tab 32, SSDD, at 5.  Finally, the SSA documented his conclusion that the price premium associated with Vencore’s proposal was warranted, stating as follows:

Considering that the technical factors when combined are significantly more important than price and that the price difference between CSC’s offer and Vencore’s offer is substantial, I thought carefully about whether my confidence in Vencore’s ability to perform, and my concerns about CSC’s ability to perform as proposed, justified the large price difference.  The Benefits portfolio is especially critical to USCIS’s mission; in my judgment the holistic approach Vencore proposed, balancing technical approaches, management of cultural change, focus on retention, and their overall management approach is very likely to result in successful performance on this critical portfolio. Because these particular strengths will provide considerable value to USCIS . . . , I believe the price premium for Vencore is justified . . . .

Id. at 5-6.

Based on the SSA’s best-value determination, the agency awarded a task order to Vencore for the benefits portfolio.  Following a debriefing, CSC filed a protest with our Office.

DISCUSSION

CSC alleges that the agency’s evaluation of proposals was unreasonable in several respects and that its source selection decision was flawed.  We have considered all of CSC’s arguments, and we conclude, based on the record, that none have merit.  Below we discuss CSC’s two principal contentions.

We observe at the outset that in reviewing protests of an agency’s evaluation and source selection decision, even in a task or delivery order competition as here, we do not reevaluate proposals; rather, we review the record to determine whether the evaluation and source selection decision are reasonable and consistent with the solicitation’s evaluation criteria and applicable procurement laws and regulations.[4]  Ball Aerospace & Techs. Corp., B‑411359, B-411359.2, July 16, 2015, 2015 CPD ¶ 219 at 7.  While judgments in the evaluation of proposals are subjective by nature, they must be reasonable and bear a rational relationship to the solicitation’s evaluation criteria.  Population Health Servs., Inc., B-292858, Dec. 1, 2003, 2003 CPD ¶ 217 at 3.  A protester’s disagreement with the agency’s judgment, by itself, is not sufficient to establish that an agency acted unreasonably.  Ball Aerospace & Techs. Corp., supra.

CSC first alleges that the agency unreasonably evaluated the firm’s proposed use of SMEs.  Before discussing CSC’s claim, additional background is necessary.  The PWS described a “product owner” as an agency employee having the following role:

The USCIS product owner will specify high-level requirements to the Agile teams.  [T]he USCIS product owners will work together with the JETS teams (Government and Contractors) to develop and estimate user stories and establish acceptance criteria. . . .  The USCIS product owners, supported by business SMEs[,] will determine whether or not acceptance criteria have been satisfied.

PWS §§ 3, 4.  As with the product owners, the “business SMEs” referenced above also were described as agency employees.  Id. § 3.

As additional background, the management approach portion of CSC’s proposal included a table showing a number of “managerial risks and mitigation approaches.”  AR, Tab 21, CSC Proposal, pt. 1, at 27-28.  One of these risk/mitigation approaches was the following:

Risk	Mitigation Approach
Risk that product owners are not always available full time to teams due to other pressing priorities	Team CSC [will] staff domain experts in the role of SME who are able to elaborate user stories into specification by example, which is easier and faster for product owners to validate.  Supported by on‑going SME collaboration with Product Owners.

Id. at 28.  Regarding this aspect of CSC’s proposal, the TEC documented the following finding under factor 3, management approach:  “Use of CSC SMEs if a product owner is not available decreases confidence in responsiveness to changing government needs.”  AR, Tab 28, TEC Rep., Factors 2 & 3, at 55.  As stated above, the SSA in his tradeoff analysis cited CSC’s proposed use of SMEs as “troublesome.”  AR, Tab 32, SSDD, at 4.

CSC argues that the agency misinterpreted the firm’s proposal to mean that CSC was “proposing to use SMEs if a product owner is unavailable.”  Protest at 9-10.  According to CSC, the mitigation approach in its proposal actually conveyed that the firm “would utilize SMEs for the limited purpose of facilitating validation by product owners.”  Id. at 10 (emphasis in original).

In response to CSC’s protest, the TEC chair explains the agency’s concern as follows:

[T]he use of SMEs throughout the proposal made it appear CSC assumed from the start Product Owners would not be available, and that CSC would be the entity responsible for resolving the lack of availability of a Product Owner by providing a SME.  This . . . presupposed a particular resolution to that possibility.  What was lacking in the proposed approach was collaboration with the Government to find the best solution, which might not be using a contractor SME.

Further, the use of SMEs in place of Product Owners . . . actually has the potential to increase the length of the feedback cycles that are vital to high performing agile teams.  The Product Owner as an approver, with the SME from the contractor acting in the collaborator role, increases the likelihood of miscommunication and rework.  The Product Owner is the most important role in an agile process and any reduction in the Product Owner’s interaction with the team has significant negative impacts in delivery of the product.

Contracting Officer’s Statement, attach. 1, TEC Chair Decl., ¶¶ 3-4.  In other words, the TEC read CSC’s proposal to convey that the primary strategy to mitigate product owner unavailability would be the use of CSC SMEs, and the TEC viewed this strategy as problematic in the context of the agency’s requirement.

In its comments on the agency report, CSC continues to assert that the agency’s position is unreasonable, arguing that “[t]he fact that CSC may employ SMEs to mitigate the risk of product owner unavailability . . . does not mean that this is the only action CSC will take in the event of product owner unavailability.”  Comments at 2 (emphasis in original); see also Supp. Comments at 1-2.  CSC goes on to highlight various portions of its proposal that discuss the firm’s risk mitigation strategy generally, including instances where the proposal discusses collaboration with the agency.  Comments at 2-3 (referencing AR, Tab 21, CSC Proposal, pt. 1, at 24, 28); see also Supp. Comments at 1-2 (same).  We see no merit in CSC’s argument.

While CSC’s proposal discusses various approaches to risk mitigation (including collaboration with the agency), the proposal identifies one specific approach to mitigate the risk of product owner unavailability:  the use of CSC SMEs.  In particular, the proposal states that to mitigate this risk CSC would “staff domain experts in the role of SME[s]” who would “elaborate user stories into specification” for product owners to “validate.”  AR, Tab 21, CSC Proposal, pt. 1, at 28.  Hence, we view as reasonable the agency’s conclusion that CSC was proposing the use of its own SMEs as the primary mitigation strategy for product owner unavailability.  CSC does not substantively rebut the agency’s position that such a strategy could be problematic in the context of the requirement here.  Additionally, the PWS established that it was the role of the agency product owner--in collaboration with others--to “specify high-level requirements,” “develop and estimate user stories and establish acceptance criteria,” and ultimately “determine whether or not acceptance criteria have been satisfied.”  PWS § 4.  In sum, CSC’s claim amounts to disagreement with the agency’s evaluation judgment and does not furnish a basis on which to sustain the protest.

Next, CSC challenges the agency’s findings regarding CSC’s proposed use of two Agile development methodologies known as Kanban and Scrum.  Protest at 11-12; Comments at 5; Supp. Comments at 3.  As relevant to this claim, CSC’s proposal included the following explanation regarding the firm’s use of the Kanban and Scrum techniques:

When unplanned work exceeds 10 percent of the team’s velocity, we move toward a Kanban model to reduce the work in progress and increase the time to deliver.  When work becomes more planned, we leverage Scrum and time-boxed iterations to deliver work on a repeatable tempo.

AR, Tab 21, CSC Proposal, pt. 1, at 13.

Regarding this aspect of CSC’s proposal, the TEC documented the following finding under factor 2, technical approach:  “Deciding on kanban or scrum based on predictability of work decreases confidence in working toward continuous delivery and achieving short lead times.”  AR, Tab 28, TEC Rep., Factors 2 & 3, at 6.  Next to this finding, the TEC referenced the following requirement within PWS section 5.1:  “Deliver fully commented executable code and code revisions that follow Agile industry best practices, to include:  version control, automated builds, automated testing, and continuous integration.”[5]  As stated above, the SSA in his tradeoff analysis, cited the “idea that the predictability of work is the primary factor in the Scrum/Kanban decision” as a factor that reduced his confidence in CSC’s performance of the task order.  AR, Tab 32, SSDD, at 4.

CSC alleges that these findings were unreasonable, asserting that CSC’s approach “was entirely consistent with industry best practices for Agile program management.”  Protest at 11.  More specifically, CSC asserts that iterative, planned work is better suited for Scrum, whereas work prone to interruption is better suited to Kanban.  Protest at 11-12; Comments at 5.  Applying this rubric to the above-quoted requirement within PWS section 5.1, CSC posits that “[t]he requirement to deliver fully executable code as part of the contractor’s routine code maintenance activities is precisely the type of iterative work commonly better aligned with Scrum.”  Protest at 12.  Additionally, CSC argues that the proposal language that raised the TEC’s concern reflects merely a “general rule of thumb,” the importance of which both the TEC and the SSA improperly overemphasized.  Id.; Comments at 5.

The agency responds, in essence, that CSC’s proposed approach of deciding between Kanban and Scrum on the basis of the predictability of work caused concern because it suggested that CSC may not consider another factor that is important in the context of the requirement here:  the frequency of work.  See Supp. AR at 3.  More specifically, the TEC chair explains as follows:

The consideration of how often deployments are planned should be at least as important, if not more important, than the predictability of work.  Because Kanban deals with delivery of small pieces quickly (ideally at single piece flow to production) and Scrum is time-boxed (generally at 1-2 weeks) for delivery, the use of Kanban may be preferred for delivery on a daily or more frequent basis.

Supp. AR, attach. 1, TEC Chair Supp. Decl., ¶ 5.  In other words, the TEC chair identifies “how often deployments are planned” (i.e., the frequency that known work will occur) as a consideration of at least equal importance to “the predictability of work” (i.e., the degree to which there is certainty whether work will arise).  Id.  As a related matter, the TEC chair points out that under factor 2, technical approach, the solicitation provided that the agency would consider, among other things, the contractor’s approach to “continuous integration and continuous deployment.”  Id. referencing RFP at 19).  Thus, the TEC chair ties the agency’s concern that CSC’s approach did not give due consideration to the frequency of work to the solicitation evaluation criterion regarding the offeror’s approach to “continuous” integration and deployment.

In reviewing a protest challenging an agency’s technical assessments or determinations, our Office generally will defer to the technical expertise of the agency and will not substitute our judgment for that of the agency; rather, we will review the record to determine whether the agency’s judgments were reasonable. See CI Filing Sys., LLC, B-411012, Apr. 17, 2015, 2015 CPD ¶ 131 at 5.  Here, we see no basis to question the agency’s determination that generally basing the decision to use Kanban on the occurrence of “unplanned work” (as CSC proposed to do), without giving due consideration to the frequency that known work would occur, could, in the TEC’s words, impede “continuous delivery and achieving short lead times.”  AR, Tab 28, TEC Rep., Factors 2 & 3, at 6.

CSC in its comments on the agency’s response reiterates its stance that

[the] predictability of work is a widely recognized metric for choosing between Kanban and Scrum and, accordingly, there was no rational justification for the Agency’s criticism of CSC’s approach for utilizing predictability of work as a guide for deciding between Kanban and Scrum.

Supp. Comments at 3; see also Comments at 5.  CSC in its comments also responds to the TEC chair’s point regarding the consideration of the frequency of work by arguing that

[t]he ability to consider “how often deployments are planned” is a direct reflection of the extent to which work is predictable.  Accordingly, CSC’s proposed use of predictability of work as a guide is qualitatively the same thing as “consideration of how often deployments are planned.”

Supp. Comments at 3 (quoting Supp. AR, attach. 1, TEC Chair Supp. Decl., ¶ 5). 

The problem with CSC’s arguments is that CSC’s proposal did not clearly convey that the firm’s decision to use Kanban would take into account the frequency of known work.  Rather, the firm’s proposal conveyed that the decision to use Kanban generally would be based on something different:  the occurrence of “unplanned” work, i.e., work that was not known, planned, or predicted.  See AR, Tab 21, CSC Proposal, pt. 1, at 13 (“When unplanned work exceeds 10 percent of the team’s velocity, we move toward a Kanban model . . . .”).[6]  Again, the agency’s documented concern was that “the predictability of work is the primary factor in [CSC’s] Scrum/Kanban decision.”  AR, Tab 32, SSDD, at 4.

To conclude, given that the solicitation called for the consideration of an offeror’s approach to “continuous integration and continuous deployment,” RFP at 19, we find reasonable the agency’s concern that CSC’s proposal identified the occurrence of unplanned work as the general basis for deciding to use the Kanban technique, without reference to consideration of the frequency that work would occur.

The protest is denied.

Susan A. Poling
General Counsel

---