FACE Recognition Technology:

FBI Should Better Ensure Privacy and Accuracy [Reissued on August 3, 2016]

GAO-16-267: Published: May 16, 2016. Publicly Released: Jun 15, 2016.

Multimedia:

Additional Materials:

Contact:

Diana Maurer
(202) 512-9627
maurerd@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Department of Justice's (DOJ) Federal Bureau of Investigation (FBI) operates the Next Generation Identification-Interstate Photo System (NGI-IPS)— a face recognition service that allows law enforcement agencies to search a database of over 30 million photos to support criminal investigations. NGI-IPS users include the FBI and selected state and local law enforcement agencies, which can submit search requests to help identify an unknown person using, for example, a photo from a surveillance camera. When a state or local agency submits such a photo, NGI-IPS uses an automated process to return a list of 2 to 50 possible candidate photos from the database, depending on the user's specification. As of December 2015, the FBI has agreements with 7 states to search NGI-IPS, and is working with more states to grant access. In addition to the NGI-IPS, the FBI has an internal unit called Facial Analysis, Comparison and Evaluation (FACE) Services that provides face recognition capabilities, among other things, to support active FBI investigations. FACE Services not only has access to NGI-IPS, but can search or request to search databases owned by the Departments of State and Defense and 16 states, which use their own face recognition systems. Biometric analysts manually review photos before returning at most the top 1 or 2 photos as investigative leads to FBI agents.

DOJ developed a privacy impact assessment (PIA) of NGI-IPS in 2008, as required under the E-Government Act whenever agencies develop technologies that collect personal information. However, the FBI did not update the NGI-IPS PIA in a timely manner when the system underwent significant changes or publish a PIA for FACE Services before that unit began supporting FBI agents. DOJ ultimately approved PIAs for NGI-IPS and FACE Services in September and May 2015, respectively. The timely publishing of PIAs would provide the public with greater assurance that the FBI is evaluating risks to privacy when implementing systems. Similarly, NGI-IPS has been in place since 2011, but DOJ did not publish a System of Records Notice (SORN) that addresses the FBI's use of face recognition capabilities, as required by law, until May 5, 2016, after completion of GAO's review. The timely publishing of a SORN would improve the public's understanding of how NGI uses and protects personal information.

Prior to deploying NGI-IPS, the FBI conducted limited testing to evaluate whether face recognition searches returned matches to persons in the database (the detection rate) within a candidate list of 50, but has not assessed how often errors occur. FBI officials stated that they do not know, and have not tested, the detection rate for candidate list sizes smaller than 50, which users sometimes request from the FBI. By conducting tests to verify that NGI-IPS is accurate for all allowable candidate list sizes, the FBI would have more reasonable assurance that NGI-IPS provides leads that help enhance, rather than hinder, criminal investigations. Additionally, the FBI has not taken steps to determine whether the face recognition systems used by external partners, such as states and federal agencies, are sufficiently accurate for use by FACE Services to support FBI investigations. By taking such steps, the FBI could better ensure the data received from external partners is sufficiently accurate and do not unnecessarily include photos of innocent people as investigative leads.

Technology advancements have increased the overall accuracy of automated face recognition over the past few decades. According to the FBI, this technology can help law enforcement agencies identify criminals in their investigations.

GAO was asked to review the FBI's use of face recognition technology. This report examines: 1) the FBI's face recognition capabilities; and the extents to which 2) the FBI's use of face recognition adhered to privacy laws and policies and 3) the FBI assessed the accuracy of these capabilities.

To address these questions, GAO reviewed federal privacy laws, FBI policies, operating manuals, and other documentation on its face recognition capability. GAO interviewed officials from the FBI and other federal and two state agencies that coordinate with the FBI on face recognition.

What GAO Recommends

GAO is making six recommendations, including, that the Attorney General determine why PIAs and a SORN were not published as required and implement corrective actions, and for the FBI director to conduct tests to verify that NGI-IPS is accurate and take steps to determine whether systems used by external partners are sufficiently accurate for FBI's use. DOJ agreed with one, partially agreed with two, and disagreed with three of the six recommendations. In response, GAO clarified one recommendation, updated another recommendation, and continues to believe that all six recommendations remain valid as discussed further in this report.

For more information, contact Diana Maurer at (202) 512-9627 or maurerd@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Priority recommendation

    Comments: In November 2018, DOJ officials told us that they had reviewed the PIA development process and determined that one reason that the FBI's face recognition PIAs were not completed more quickly was because the FBI and DOJ engaged in an extensive PIA revision process. In response, DOJ officials stated that they implemented a pilot to expedite the FBI PIA approval process, which included developing a PIA approval template and focusing the review on legal sufficiency instead of a more comprehensive review that included less significant editorial changes. According to DOJ and FBI officials, the pilot was a success and resulted in the ability to approve a greater number of PIAs in less time. DOJ reported that between April 2018, when DOJ began to implement the pilot, and April 2019, DOJ approved nineteen FBI PIAs, in contrast to the previous 12-month period, when DOJ approved only five FBI PIAs. From May 2019 to July 2019, DOJ approved 12 additional PIAs. In July 2019, DOJ formalized the expedited review process for FBI PIAs in a memorandum for the record issued by the DOJ Office of the Deputy Attorney General, Office of Privacy and Civil Liberties (OPCL). The memorandum states that DOJ is to provide comments or approve a PIA within 30 days of the FBI submitting a draft PIA to DOJ OPCL for review. Further, the memorandum provides that DOJ will collaborate with FBI to maximize transparency and make PIAs publicly available, if practicable, prior to operation of a system. DOJ officials also stated in June 2019 that based on the success of the FBI PIA pilot, the department will look to explore the feasibility of utilizing a similar expedited review process in the development and publication of PIAs issued by other DOJ components. The timely publishing of PIAs will provide the public with greater assurance that the FBI is evaluating risks to privacy when implementing systems.

    Recommendation: To improve transparency and better ensure that face recognition capabilities are being used in accordance with privacy protection laws and policy requirements, the Attorney General should assess the PIA development process to determine why PIAs were not published prior to using or updating face recognition capabilities, and implement corrective actions to ensure the timely development, updating, and publishing of PIAs before using or making changes to a system.

    Agency Affected: Department of Justice

  2. Status: Open

    Priority recommendation

    Comments: DOJ agreed, in part, with our recommendation and submitted the SORN for publication to the Federal Register on April 21, 2016, and it was published on May 5, 2016. In June 2019, DOJ officials told us that they had recently applied the pilot program for expediting the PIA review process to the way that DOJ develops and reviews SORNs. DOJ subsequently documented in its July 2019 memorandum for the record the requirement that the DOJ Office of Privacy and Civil Liberties (OPCL) is to provide comments or approval within 30 days of the FBI submitting a draft SORN to DOJ OPCL for review. The memorandum also outlines a more direct and expedient process for resolving feedback and finalizing drafts. Further, the memorandum provides that after DOJ approves an FBI SORN, it will begin the formal publication process, in accordance with the law and Office of Management and Budget (OMB) policy. DOJ officials told us that the intent of the July 2019 memorandum was to make changes to the current process that would permit DOJ to more efficiently and effectively facilitate all aspects of SORN development within its purview prior to submission to OMB. DOJ and FBI officials stated that it is their goal to consistently publish SORNs before a system's implementation. In follow-up with DOJ in December 2019, DOJ stated that it may have sufficient data to demonstrate the effect of these changes in the spring of 2020. We believe that DOJ's efforts to begin establishing a more streamlined review process for SORNs is a positive step and continue to monitor this recent pilot to determine whether these efforts help ensure SORNs are published before systems become operational.

    Recommendation: To improve transparency and better ensure that face recognition capabilities are being used in accordance with privacy protection laws and policy requirements, the Attorney General should assess the SORN development process to determine why a SORN was not published that addressed the collection and maintenance of photos accessed and used through NGI for the FBI's face recognition capabilities prior to using NGI-IPS, and implement corrective actions to ensure SORNs are published before systems become operational.

    Agency Affected: Department of Justice

  3. Status: Closed - Implemented

    Priority recommendation

    Comments: In March 2017, DOJ provided us with the audit plan the CJIS Audit Unit developed in June 2016 for NGI-IPS users. In February 2018, DOJ officials stated that they have conducted eight NGI-IPS audits, which have found no significant findings of noncompliance. DOJ also provided us with copies of the final audit results for one state and its audit NGI-IPS reference guide. Further, DOJ officials said CJIS developed an audit plan of the FACE Services and completed an initial audit in September 2018.The FBI reported that it finalized the audit report in April 2019, which concluded that Face Services is operating in accordance with privacy laws and policies. Further, the FBI reported in May 2019 that audits of FACE Services will continue on a tri-annual basis and that it conducts tri-annual audits of states that use NGI-IPS. As a result, DOJ has fully implemented our recommendation.

    Recommendation: To better ensure that face recognition capabilities are being used in accordance with privacy protection laws and policy requirements, the Director of the Federal Bureau of Investigation should conduct audits to determine the extent to which users of NGI-IPS and biometric images specialists in FACE Services are conducting face image searches in accordance with Criminal Justice Information Services Division policy requirements.

    Agency Affected: Department of Justice: Federal Bureau of Investigation

  4. Status: Closed - Implemented

    Priority recommendation

    Comments: In response, in June 2019, the FBI provided information on candidate list size testing. In 2017, the FBI tasked a federally funded research center to assess the NGI-IPS system against operational parameters, including the length of candidate lists. In December 2017, the center reported that the NGI-IPS system successfully met the detection rate for all candidate sizes between 2 and 50. The report did not specifically assess how often NGI-IPS face recognition searches erroneously matched persons to the database, but it characterized the impact that database size would have on false positives. Further, the test conducted is consistent with a law enforcement scenario in which a human reviewer is employed to review the candidates returned from an identification search, according to the National Institute of Standards and Technology. FBI officials told us that FBI image specialists review all images that are returned. These officials also stated that they provide training to state and local users of NGI-IPS. By conducting tests to verify that NGI-IPS is sufficiently accurate for all allowable candidate list sizes and adjudicating all candidate photos returned by NGI-IPS, the FBI has better assurance that NGI-IPS provides investigative leads that help enhance, rather than hinder or overly burden, criminal investigation work.

    Recommendation: To better ensure that face recognition systems are sufficiently accurate, the Director of the Federal Bureau of Investigation should conduct tests of NGI-IPS to verify that the system is sufficiently accurate for all allowable candidate list sizes, and ensure that the detection and false positive rate used in the tests are identified.

    Agency Affected: Department of Justice: Federal Bureau of Investigation

  5. Status: Open

    Priority recommendation

    Comments: In July 2019, the FBI reported that it has designed a Face Recognition Test Strategy for Operational Analysis with the objective of being able to perform an annual operational review of NGI-IPS. The FBI plans to develop and test the operational review in fiscal year 2020. As a first step, in January 2020, FBI officials told us that they had developed a tool to benchmark whether the FBI face recognition system returns the expected photos in the FBI's operational environment and were in the process of testing it. To fully implement this recommendation, the FBI should conduct operational reviews at least annually to assess the accuracy of face recognition searches on NGI-IPS.

    Recommendation: To better ensure that face recognition systems are sufficiently accurate, the Director of the Federal Bureau of Investigation should conduct an operational review of NGI-IPS at least annually that includes an assessment of the accuracy of face recognition searches to determine if it is meeting federal, state, and local law enforcement needs and take actions, as necessary, to improve the system.

    Agency Affected: Department of Justice: Federal Bureau of Investigation

  6. Status: Closed - Implemented

    Priority recommendation

    Comments: In June 2019, FBI officials stated that they plan to survey state partners about the face recognition technology they use and submitted the survey to the Office of Management and Budget for approval. According to FBI officials, they hoped to gain insights into the technology the states use, which could provide them with information on the accuracy of the state systems. We reviewed the survey questions and agreed that the survey would provide the FBI with information that can help the bureau determine if those systems are sufficiently accurate for its use. In August 2019, FBI officials reported that 13 of the 21 state agencies that partner with the FBI FACE Services Unit for face recognition searches responded to the survey, with none of the responses raising concerns with the accuracy of the states' face recognition systems, according to FBI officials. FBI officials stated that they also requested that federal agencies that partner with FACE Services complete the survey. The Department of Defense responded to the survey, according to FBI documentation. By taking such steps, the FBI has more assurance that the data received from external partners is sufficiently accurate for FBI's use and do not unnecessarily include photos of innocent people as investigative leads. Therefore, this recommendation is closed as implemented.

    Recommendation: To better ensure that face recognition systems are sufficiently accurate, the Director of the Federal Bureau of Investigation should take steps to determine whether each external face recognition system used by FACE Services is sufficiently accurate for the FBI's use and whether results from those systems should be used to support FBI investigations.

    Agency Affected: Department of Justice: Federal Bureau of Investigation

 

Explore the full database of GAO's Open Recommendations »

May 28, 2020

May 26, 2020

May 12, 2020

May 6, 2020

Mar 27, 2020

Feb 3, 2020

Jan 29, 2020

Jan 15, 2020

Dec 18, 2019

Looking for more? Browse all our products here