NCI Information Systems, Inc.

B-416926,B-416926.2,B-416926.3: Jan 9, 2019

Additional Materials:

Contact:

Ralph O. White
(202) 512-8278
WhiteRO@gao.gov

Kenneth E. Patton
(202) 512-8205
PattonK@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

NCI Information Systems, Inc., of Reston, Virginia, protests the issuance of task orders to ERPSI, of Gambrills, Maryland; Leidos Innovations Corp., of Rockville, Maryland; and REI Systems Inc., of Chantilly, Virginia, by the Department of Homeland Security (DHS), United States Citizenship and Immigration Services (USCIS), pursuant to request for proposals (RFP) No. 70SBUR18R00000008 for outcome based software development and operations services. The protester alleges that the agency's evaluation of NCI's proposal was unreasonable.

We deny the protest.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: NCI Information Systems, Inc.

File: B-416926; B-416926.2; B-416926.3

Date: January 9, 2019

Daniel P. Graham, Esq., Caroline E. Colpoys, Esq., John M. Satira, Esq., and Parker Hancock, Esq., Vinson & Elkins LLP, for the protester.

John Cornell, Esq., Department of Homeland Security, for the agency.

Nora K. Adkins, Esq., and Amy B. Pereira, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Protest challenging agency's evaluation of protester's proposal is denied where the evaluation conclusions were reasonable, consistent with the solicitation, and did not employ unstated evaluation criteria.

DECISION

NCI Information Systems, Inc., of Reston, Virginia, protests the issuance of task orders to ERPSI, of Gambrills, Maryland; Leidos Innovations Corp., of Rockville, Maryland; and REI Systems Inc., of Chantilly, Virginia, by the Department of Homeland Security (DHS), United States Citizenship and Immigration Services (USCIS), pursuant to request for proposals (RFP) No. 70SBUR18R00000008 for outcome based software development and operations services. The protester alleges that the agency's evaluation of NCI's proposal was unreasonable.

We deny the protest.

BACKGROUND

On April 4, 2018, USCIS issued the solicitation on an unrestricted basis to firms holding contracts under DHS's Enterprise Acquisition Gateway for Leading Edge Solutions (EAGLE) II indefinite-delivery, indefinite-quantity (IDIQ) multiple-award contract vehicle. RFP at 1. The RFP contemplated the issuance of up to five task orders, pursuant to the procedures of Federal Acquisition Regulation (FAR) subpart 16.5, for outcome based delivery and development operations services. Id. at 100. The RFP anticipated that the resulting orders would be hybrid fixed-price and time-and-materials task orders with a 3-month transition period, a 3-month base period, and four 6-month option periods. Id. at 61, 98.

The RFP provided for a two-step evaluation process addressing the following four factors, in descending order of importance: (1) technical demonstration, (2) technical approach (coding challenge), (3) past performance, and (4) price. Id. at 100. During step one of the evaluation, the offerors would complete a coding challenge. Id. The agency would evaluate the offerors' responses to the coding challenge under the technical approach factor, and conduct a down-select of the offerors on a best-value tradeoff basis considering the following factors: technical approach, past performance, and price. Id. Offerors submitting the most highly rated proposals would move to step two. Id. All other offerors would not be invited to proceed to step two of the evaluation (technical demonstration), and their proposals would be considered ineligible for award. Id. After the step two technical demonstration, the agency would select the awardee(s) on a best-value tradeoff basis considering the following factors: technical demonstration, past performance, and price. Id. The solicitation did not contemplate discussions with offerors. Id.

With respect to the technical approach factor, the solicitation required offerors to provide a technical proposal responding to the coding challenge. Id. at 94. The coding challenge, which was provided as an attachment to the solicitation five days prior to the RFP closing date, required the offerors to develop a conference room reservation system application. Id. Offerors would develop the application based on a specific business case and accompanying user stories provided in the attachment. Id. The coding challenge attachment also included certain criteria for each of three user roles: requestor, reservation manager, and administrator. Id., attach. 12, Coding Challenge, at 181-182. For example, the administrator role criteria required functionality to permit the administer role to "add, modify, and delete conference room users." Id. at 181. As another example, the reservation manger role criteria required functionality to "create detailed online reports including the ability to export data . . . ." Id. at 181-182. The solicitation required offerors to provide a URL1 for the working conference room reservation system application, credentials to access the application for each user role, and any other instructions as necessary. Id. at 94. The solicitation instructed offerors to store the conference room reservation system application's source code in a GitHub2 repository and deploy the application in Amazon Web Services. Id.

As relevant here, the agency's evaluation of the technical approach factor would consider how well the offeror creates and delivers the conference room reservation system application based on an offeror's technical proposal. Id. at 101. The solicitation specified that the agency's evaluation would be in accordance with an evaluation sheet attached to the solicitation, which provided 15 general architecture and design criteria. Id.; see Id., attach. 2, Evaluation Sheet, at 107-109. The following two criteria are relevant here: (criterion 3) the solution shall present a robust and modern security architecture and a high degree of understanding of security concerns; and (criterion 9) the solution shall satisfy the business needs and offer a good user experience. Id. attach. 2, Evaluation Sheet, at 107-108.

The agency received multiple proposals by the RFP closing date, including a proposal from NCI. AR, Exh. 6, Step One Evaluation, at 5. The agency's step one evaluation of the offerors' technical approach, past performance, and price factors resulted in the following ratings for NCI and the three awardees:

  NCI ERPSI Leidos REI
Technical Approach Unacceptable Superior Marginal Marginal
Past Performance Substantial Confidence Substantial Confidence Substantial Confidence Substantial Confidence
Price $87,671,300 $96,212,866 $93,505,738 $93,741,661

Id. at 10.

With respect to the protester, the evaluators concluded that NCI's technical approach was unacceptable because "the login failed causing the application to be untestable." Id. at 9. In reaching this conclusion, the evaluators found that NCI's coding challenge application failed to have "role-based access control" and failed to "present role-based pages offering the appropriate functionality to users." AR, Exh. 5, Technical Consensus, at 2. Based on the results of the technical, past performance and price evaluations, the source selection authority concluded that ERPSE, Leidos, and REI provided the most highly rated proposals and these offerors were selected to move on to step two of the evaluation. AR, Exh. 6, Step One Evaluation, at 11-12. NCI was not selected to move on to step two because it received an unacceptable rating under the technical approach factor. Id. at 10. Thereafter, the agency conducted the step two technical demonstrations with ERPSE, Leidos, and REI. Contracting Officer Statement (COS) at 3. Based on the results of the step two evaluations, the agency issued task orders to all three offerors on September 26.

On September 28, NCI was provided a debriefing. On October 3, NCI filed this protest.3

DISCUSSION

NCI challenges the agency's evaluation of its proposal. The protester argues that the evaluation relied on unstated evaluation criteria to find NCI's proposal unacceptable. Alternatively, NCI asserts that even if an evaluation based on these criteria was reasonable, the agency's evaluation was improper because NCI's proposal met the requirements. Based on our review of the record, we find no basis to sustain the protest. While we do not address each of the protester's allegations and variations thereof, we have reviewed them all and find the agency's evaluation reasonable and consistent with the solicitation's evaluation criteria.

The evaluation of proposals in a task order competition, including the determination of the relative merits of proposals, is primarily a matter within the contracting agency's discretion, because the agency is responsible for defining its needs and the best method of accommodating them. Engility Corp., B-413120.3 et al., Feb. 14, 2017, 2017 CPD ¶ 70 at 15. Moreover, as a general matter, when evaluating proposals in a task order competition, an agency may properly take into account specific, albeit not expressly identified, matters that are logically encompassed by, or related to, the stated evaluation criteria. M.A. Mortenson Co., B-413714, Dec. 9, 2016, 2016 CPD ¶ 361 at 5, 8. Our Office will review evaluation challenges to task order procurements to ensure that the competition was conducted in accordance with the solicitation and applicable procurement laws and regulations. URS Fed. Servs., Inc., B-413333, Oct. 11, 2016, 2016 CPD ¶ 286 at 6. A protester's disagreement with the agency's judgment, without more, is not sufficient to establish that an agency acted unreasonably. STG, Inc., B-405101.3 et al., Jan. 12, 2012, 2012 CPD ¶ 48 at 7.

NCI alleges that deficiencies identified in its proposal under the technical approach factor were based on unstated evaluation criteria: role-based access control and role-based pages offering the appropriate functionality to users. In response, the agency contends that the evaluation of NCI's conference room reservation system application was reasonable and based upon the stated evaluation criteria. We find the agency's evaluation unobjectionable.

NCI's technical proposal provided a written technical narrative; coding challenge photographs, screenshots, and notes; and its response to the coding challenge (the conference room reservation system application's URL, role-based login credentials, operating instructions, and source code). AR, Exh. 3, NCI Proposal, at 3-10. In response to the coding challenge, NCI explained that it created "Conference Peak (https://www.conferencepeak.com), a cloud-based, scalable, and secure conference room reservation system." Id. at 3. NCI's technical approach provided that its "solution architecture establishes token-based authentication with OAuth single sign-on via GitHub authentication." Id. at 4. NCI's proposal also provided the following information with respect to user credentials and Conference Peak operating instructions:

I.3.1.2 Login Credentials

Conference Peak has three user roles with varying levels of access. Figure I-6 provides login credentials for each of these user roles.

 

Figure I-6. Conference Peak Login Credentials.

Name Role User ID Password
Room Requestor Requestor requestor@conferencepeak.com RRpass123!
Reservation Manager Reservation Manager manager@conferencepeak.com RMpass123!
System Administrator Administrator admin@conferencepeak.com SApass123!

 

I.3.1.3 Instructions

Conference Peak is easy to use. Here are the instructions to login:

  • Go to https://www.conferencepeak.com

  • Click on the "GitHub Sign in" button on the right

  • Enter the User ID (from Figure I-6) in the Username or email address field

  • Enter the password (from Figure I-6) in the Password field

  • Click on the "Sign in" button

You are now in the Conference Peak web application and can reserve a conference room.

Id. at 10.

The agency's evaluation of the protester's proposal found NCI ineligible for a task order award due to deficiencies assigned to NCI's technical approach relating to the log in/log out functionality of the three role-based user accounts. AR, Exh. 6, Step One Evaluation, at 9. The individual evaluator notes demonstrate that the evaluators based these deficiencies on multiple issues that occurred during the testing of NCI's Conference Peak application. AR, Exh. 4, NCI Evaluation, at 1-12. For example, evaluator five assigned a deficiency under criterion 3 and explained,

User Ids have been provided for all 3 roles [requestor, reservation manager, system administrator]. However, once you login as a given user, you can never login as a different user even after logging out. Please note that even closing the browser and launching a new browser also doesn't work. The first user remains logged in.

Id. at 3. Multiple evaluators also assigned a deficiency under technical approach criterion 9 (solution shall satisfy the business needs and offer a good user experience). Evaluator one explained, "There is no 'logout' mechanism to switch to another user after making the first Github login. Therefore none of the application features can be tested and validated." Id. at 4. Evaluator two provided, "The conference site did not work. I could not log on." Id. Evaluator five also assigned a deficiency finding that he was "[u]nable to logout once logged in as a given user." Id. at 6. This evaluator however discovered that there was a work around to this issue and explained, "To log in as a different user, clear browser cache, close browser before getting back in." Id. Although the evaluator was able to work around NCI's login/logout issues, the evaluator assigned a deficiency. Id.

In addition to the deficiencies assigned under criteria 3 and 9, the evaluators also assigned deficiencies for NCI's failure to have role-based access control and failure to present role-based pages offering the appropriate functionality to users. Id. at 10. Evaluator one explained again that "There is no 'logout' mechanism to switch to another user after making the first Git[H]ub login. Therefore none of the application features can be tested and validated." Id. at 11. Evaluator two provided, "Login did not work." Id. Evaluator four stated that credit was given for the "admin login since [G]it[H]ub integrated login defaults to admin." Id. at 12. However, the evaluator noted that the other functionality was only testable and worked by "tweaking admin login" but the evaluator "could not directly login as res[ervation] M[anager] nor requestor." Id. Finally, evaluator five provided, "Unable to logout once logged in as a given user." Id.

The technical evaluation consensus document assessed two deficiencies based on the log in/log out issues: failure of role-based access control and a failure to present role-based pages offering the appropriate functionality to users. AR, Exh. 5, NCI Technical Consensus, at 2. The consensus document also provided that the remainder of the evaluation was discontinued due to the log in/log out issues. Id. The contracting officer, who was the source selection authority, concluded that NCI's technical approach was unacceptable because NCI's conference reservation room system application's login failed, which caused the application to be untestable. AR, Exh. 6, Step One Evaluation, at 9.

NCI argues that USCIS' consideration of role-based access controls and role-based pages offering the appropriate functionality to users were unstated evaluation criteria. NCI contends that the RFP's evaluation attachment did not contain these additional criteria and the terms do not appear in the RFP nor the coding challenge acceptance criteria. NCI also alleges that these two criteria are not related to or encompassed by the evaluation criteria provided in the RFP. Alternatively, NCI asserts that even if these criteria could be considered encompassed within the solicitation evaluation criteria, NCI met the requirements. In this regard, NCI argues that the evaluators were able to test its application by logging in to each of the three user roles, after concluding that the browser cache had to be cleared to logout of GitHub and remove the login authentication token.

To provide further understanding of the functionality of the Conference Peak application, NCI provided our Office with a white paper during the course of the protest to explain NCI's Conference Peak user authentication process. NCI White Paper at 1-59. NCI's white paper explained its log-in authentication process as follows: once a user logs in to the Conference Peak application and is authenticated (through GitHub), they are able to access all of the systems applications to which they have been authenticated to access. Id. at 3. Users will then remain authenticated until they are logged out of the source of the authentication--GitHub. Id. The white paper explains that simply logging out of the Conference Peak application, which receives the authentication from GitHub, does not suffice, as the user will still be authenticated by the GitHub and will be re-logged in once they return to the Conference Peak application. Id. Only when the user logs out of the GitHub, will they become unauthenticated; logging out of GitHub will then remove the session from that particular application and the user will be asked to re-authenticate when next logging in to Conference Peak. Id.

The white paper also provided the following instructions for testing NCI's Conference Peak application:

The following steps should be repeated for each test so that testing occurs in a controlled environment and so that testing begins from the same starting point for all tests.

1. Open a New Window of Google Chrome (Version 69.0.3497.100 (Official Build) (64-bit))

2. Ensure a clean test environment

a. Click "Customize and control Google Chrome" (three vertical dots)

b. Select "More Tools"

c. Select "Clear browsing data…"

d. Set the "Time Range" to "All Time"

e. Select the options:

i. Browsing history

ii. Cookies and other site data

iii. Cached images and files

f. Click "Clear data"

3. Verify that there is no active GitHub SAML single sign-on session in your web browser (https://help.github.com/articles/about-authentication-with-saml-single-sign-on/) by going to https://github.com.

Id. at 2.

Based on the record before us, we find the agency's evaluation reasonable. As an initial matter, while we agree with the protester that the two role-based requirements were not specified as one of the 15 evaluation criteria in the RFP attachment, we find the agency's evaluation reasonable because the ability to log in (and log out) of the application was reasonably related to and encompassed by, the stated evaluation factors. Although a solicitation must identify all major evaluation factors, it need not identify all areas within each factor that might be taken into account in an evaluation, provided such unidentified areas are reasonably related to, or encompassed by, the stated evaluation factors. URS Federal Servs, Inc., supra, at 7. Here, we see no basis to conclude that the agency applied unstated evaluation criteria when it considered whether NCI's technical approach to the coding challenge had role based access control and presented role-based pages offering the appropriate functionality to users.

The ability of the three user roles to log in/log out of the application is encompassed by and related to the consideration of how well the offeror creates and delivers the application. See RFP at 101. Specifically, whether the application demonstrates a high degree of understanding of security concerns, satisfies business needs, and offers a good user experience. See id. attach. 2, Evaluation Sheet, at 107-108. As stated above, multiple evaluators also assigned deficiencies under both evaluation criteria three and nine related to an inability to log in/log out of the application using the various user role credentials. Quite simply, the agency's unacceptable rating due to the application's login failure does not, as the protester argues, amount to reliance on unstated evaluation criteria. NCI's protest regarding this matter is without merit.

We also find that the agency reasonably concluded that NCI's application did not meet these requirements. NCI alleges that the Conference Peak application provided role-based access controls and role-based pages offering different functionality based on the user's role. NCI argues that the evaluators successfully used these controls and pages during their evaluation. The record does not support the protester's argument.

As stated above, the evaluation documents demonstrate that multiple evaluators encountered issues logging in/out of the Conference Peak application during testing. See AR, Exh. 4, NCI Evaluation, at 1-12. The record confirms that evaluators encountered problems logging in/out of each of the three user roles (requestor, reservation manager, administrator) using the credentials and instructions provided in NCI's proposal. Id. That is, if the evaluator logged in to test the reservation manager role, once the evaluator logged out of this role, using the log out button provided in the Conference Peak application, the evaluator was not able to log in using either the requestor or administrator roles. Id. This is because NCI's authentication process, which used GitHub, kept the user authenticated until that user logged out of GitHub. However, the instructions for logging out of GitHub (log out of Conference Peak and clear the browser cache prior to logging in as another user) were not provided in NCI's proposal.

Without these instructions, or knowledge of the need to clear the browser data of prior authentications, any subsequent login with a different role-based user's credential would not function. While the protester argues that the agency should have instructed the evaluators to clear all browser data prior to logging in as another user, this information should have been provided in NCI's proposal as required by the RFP. See RFP at 94 (provide a URL for the working conference room reservation system application, credentials to access the application for each user role, and any other instructions as necessary.) Instead, this information was provided for the first time in NCI's white paper, which was submitted in response to the protest. Since NCI did not provide the necessary information to enable the agency to log in/log out of its application using the role-based user credentials in its proposal, we find the agency's evaluation of NCI's proposal reasonable. That is, when following the instructions, as provided in NCI's proposal, the evaluators were unable to log in/log out of each of the three role-based user accounts. The evaluators found this to be a deficiency. We have no basis to question this assessment. It is an offeror's responsibility to submit a well-written proposal, with adequately detailed information which clearly demonstrates compliance with the solicitation requirements and allows a meaningful review by the procuring agency. Engility Corp., supra at 16. Agencies are not required to infer information from an inadequately detailed proposal, or to supply, information that the protester elected not to provide. Id. While the protester urges our Office to find the agency's unacceptable rating unreasonable because some evaluators were able to accommodate this log in failure by clearing the browser cache, this argument does not demonstrate that agency's assignment of the deficiencies was unreasonable.

In sum, we find no basis to question the agency's evaluation of NCI's technical proposal. The protester's allegations to the contrary only reflect its disagreement with the agency's evaluation, which provides no basis to question the reasonableness of the agency's judgments. STG, Inc., supra at 7. For these reasons, we conclude that the USCIS reasonably found NCI's proposal unacceptable and ineligible for award.

The protest is denied

Thomas H. Armstrong
General Counsel

 


[1] A uniform resource locator (URL) is the address of a resource on the Internet.

[2] According to the agency, GitHub is a web-based version control and collaboration platform for software developers that stores source code and tracks change history. Agency Report (AR), Exh. 8, Evaluator Declaration, at 1-2.

[3] The value of the task order at issue exceeds $10 million. Accordingly, this procurement is within our jurisdiction to hear protests related to the issuance of orders under multiple-award IDIQ contracts that were awarded under the authority of Title 41 of the U.S. Code. 41 U.S.C. § 4106(f).

Aug 16, 2019

Aug 15, 2019

Aug 14, 2019

Aug 13, 2019

Aug 9, 2019

Looking for more? Browse all our products here