This is the accessible text file for GAO report number GAO-13-629 entitled 'Transportation Security: Action Needed to Strengthen TSA's Security Threat Assessment Process' which was released on August 2, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Ranking Member, Committee on Homeland Security, House of Representatives: July 2013: Transportation Security: Action Needed to Strengthen TSA's Security Threat Assessment Process: GAO-13-629: GAO Highlights: Highlights of GAO-13-629, a report to the Ranking Member, Committee on Homeland Security, House of Representatives. Why GAO Did This Study: TSA implements programs that, for example, ensure individuals with unescorted access to secure areas of the nation’s critical transportation infrastructure do not pose a security threat. Key to these programs are security threat assessments that screen individuals for links to terrorism, criminal history, and immigration status. TSA’ s Adjudication Center serves as the primary operational component in this process. GAO was asked to examine the performance and staffing strategy of the center. This report addresses the extent to which 1) TSA has measured performance for the center and what the data show; 2) TSA offices have coordinated to meet security threat assessment workload; and 3) TSA addressed potential risks posed by using a mix of government employees and contractors to adjudicate security threat assessments. GAO analyzed TSA data describing the center’s performance since October 2010; reviewed documentation, including staffing plans; and interviewed TSA officials about data measurement and staffing practices. What GAO Found: The Transportation Security Administration’s (TSA) Adjudication Center performance data show mixed results, and the center’s performance measurement practices have limitations. The Adjudication Center relies on contractors to adjudicate security threat assessments and uses three primary measures to evaluate their performance—timeliness for completing adjudication, adjudication accuracy, and caseload status. GAO found that the Adjudication Center contractor met its timeliness and accuracy measures, but faced challenges in meeting its caseload measure. The Adjudication Center’s timeliness and accuracy measures did not capture key data. According to TSA officials, the Adjudication Center’s accuracy rate is based on a review of all cases where adjudicators had disqualified an applicant. However, this calculation generally does not include the accuracy rate for those applicants adjudicators had approved—which account for roughly 90 percent of the Adjudication Center’s caseload. In this way, the accuracy rate provides a limited assessment of adjudicator performance. By developing an accuracy rate that includes data on both incorrectly disqualified and incorrectly approved applicants, TSA can better identify and addresses performance issues among its workforce. Two TSA offices that share responsibility for implementing security threat assessments-—the Program Management Division in the Office of Intelligence and Analysis and the Adjudication Center in the Office of Law Enforcement/Federal Air Marshal Service-—can improve coordination on workforce planning. While the offices share information on workload completion, they do not have a process in place to ensure that information in the Adjudication Center’s staffing plan-—which the Adjudication Center periodically updates to reflect caseload projections and associated staffing needs—-reflects the mutual understanding of both offices. For example, program managers in the Office of Intelligence and Analysis reported to GAO that they were unfamiliar with the staffing plan and they disagreed with workload projections in the plan. Establishing a mechanism for the offices to share and reconcile information in the plan can help better support the Adjudication Center’s workforce planning. TSA has been delayed in addressing risks posed by using contractors to adjudicate security threat assessments. In October 2011 TSA’s Balanced Workforce Strategy Working Group completed its assessment for the Adjudication Center and determined that an excessive risk exists by allowing contractors to make security threat assessment approvals without sufficient federal oversight. The Working Group recommended that TSA convert to an all government workforce. According to a May 2012 implementation plan, TSA planned to convert this workforce by the end of calendar year 2013. However, delays have rendered the timelines and cost information in its plan outdated and TSA has not updated the plan or determined a revised implementation schedule. Completing this review and updating the plan would help TSA and Department of Homeland Security (DHS) decision makers by providing a roadmap for moving forward. Finally, providing this plan to DHS for review will be important to help ensure TSA can begin its conversion and mitigate identified risks of using contract adjudicators to conduct security threat assessments. What GAO Recommends: GAO recommends that TSA, among other things: direct the Adjudication Center to calculate an accuracy rate that includes adjudicator performance for cases where applicants were both approved and disqualified; share adjudicator staffing plans among key program offices; and update its Adjudication Center workforce conversion plan and provide it to DHS for review and approval. DHS concurred with our recommendations. View [hyperlink, http://www.gao.gov/products/GAO-13-629]. For more information, contact Jennifer A. Grover at (202) 512-7141 or groverj@gao.gov. [End of section] Contents: Letter: Background: Adjudication Center Performance Data Show Mixed Results, Accuracy Measures Are Limited, and Performance Measurement Practices Are Not Fully Documented: TSA Offices Can Strengthen Coordination to Ensure Effective Adjudication Center Workload Planning: TSA Has Made Limited Progress in Addressing Risks Posed by Using Contract Employees to Adjudicate Security Threat Assessments: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: TSA Organizational Chart with Key Offices with Responsibility for Implementing Security Threat Assessments: Appendix II: Comments from the Department of Homeland Security: Appendix III: GAO Contact and Staff Acknowledgments: Figures: Figure 1: TSA TWIC, HME, and Aviation Worker Program Processes and Contractor and TSA Staff Functions in Security Threat Assessment Process at the Adjudication Center: Figure 2: Caseload Levels for TWIC, HME, and Aviation Worker Programs, October 2010 through January 2013: Figure 3 TSA Organization Chart Showing Key Offices Responsible for Managing and Implementing Transportation Security Threat Assessment Programs, as of May 2013. Abbreviations: BWS: Balanced Workforce Strategy: BW PMO: Balanced Workforce Strategy Program Management Office: DHS: Department of Homeland Security: FBI: Federal Bureau of Investigation: HME: Hazardous Materials Endorsement: MTSA: Maritime Transportation Security Act: OIA: Office of Intelligence and Analysis: OLE/FAMS: Office of Law Enforcement/Federal Air Marshal Service: TIM: Technology Infrastructure Modernization: TSA: Transportation Security Administration: TWIC: Transportation Worker Identification Credential: USA PATRIOT: Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism: [End of section] GAO: United States Government Accountability Office: 441 G St. N.W. Washington, DC 20548: July 19, 2013: The Honorable Bennie G. Thompson: Ranking Member: Committee on Homeland Security: House of Representatives: Dear Mr. Thompson: Securing the nation's transportation systems and facilities requires balancing security to address potential threats while facilitating the legitimate flow of people and goods that are critical to the U.S. economy and international commerce. Within the Department of Homeland Security (DHS), the Transportation Security Administration (TSA) is responsible for managing vetting and credentialing programs to ensure that individuals that transport hazardous materials or have unescorted access to secure or restricted areas of transportation facilities at maritime ports and TSA-regulated airports[Footnote 1] do not pose a security threat.[Footnote 2] Key to these programs are background checks--known as security threat assessments--which are TSA reviews of applicant information and searches of government databases to determine whether the applicant has known ties to terrorism and whether the applicant may be otherwise precluded from obtaining an endorsement, credential, access and/or privilege (hereafter called a credential) based on, among other factors, their immigration status and criminal history. Multiple TSA offices coordinate in implementing security threat assessments, with TSA's Adjudication Center serving as the primary operational component by reviewing derogatory applicant background information to determine if applicants are eligible to obtain credentials for 12 TSA programs, including the Transportation Worker Identification Credential (TWIC) program for maritime workers, Hazardous Materials Endorsement (HME) program for commercially licensed drivers, and the Aviation Worker program.[Footnote 3] These efforts are intended to reduce the probability of a successful terrorist or other criminal attack on the nation's transportation systems, which include approximately 360 seaports and 450 TSA- regulated airports. As we have previously reported, TSA's Adjudication Center has faced recurring challenges in meeting its security threat assessment performance requirements and has attributed these challenges in part to its reliance on a contractor workforce. For example, in December 2011, we reported that insufficient federal staffing had hampered the Adjudication Center's ability to meet its workload requirements and ensure necessary oversight of the credential decision-making process. [Footnote 4] In particular, TSA attributed Adjudication Center backlogs to turnover among its contractor workforce. At the time, TSA reported that it had initiated but not yet completed an assessment through the DHS Balanced Workforce Strategy (BWS)[Footnote 5] to determine if (1) the Adjudication Center's reliance on a largely contractor workforce to conduct security threat assessment adjudication was appropriate, or if the work is an inherently governmental function requiring a government workforce[Footnote 6] and (2) if there would be a cost savings resulting from conversion of the contract positions to government personnel positions. According to the Federal Acquisition Regulation, inherently governmental functions include activities that require either the exercise of discretion in applying government authority, or value judgments in making decisions for the government, and therefore should be performed by government employees, not contractors. Further, federal procurement policy explains that a function is not appropriately performed by a contractor where the contractor's involvement is or would be so extensive, or the contractor's work product so close to a final agency product, as to effectively preempt the federal officials' decision- making process, discretion or authority.[Footnote 7] Thus, determining whether TSA is using an appropriate Adjudication Center workforce, and addressing the potential risks associated with its use of contractors, is important for TSA to ensure it effectively implements security threat assessments. You requested that we review the performance and staffing capacity of the Adjudication Center. This report addresses the following questions: * To what extent has TSA measured performance for the Adjudication Center, and what do these data show? * To what extent have TSA offices coordinated to ensure that the Adjudication Center effectively meets an evolving security threat assessment workload? * To what extent has TSA addressed potential risks posed by using a mix of government employees and contractors to adjudicate security threat assessments? To determine the extent to which TSA has measured performance for the Adjudication Center, and what these data show, we focused our review on Adjudication Center performance in processing cases for the TWIC, HME, and Aviation Worker programs. We selected and obtained Adjudication Center performance in processing cases for these three programs because TSA reported these programs accounted for approximately 95 percent of its security threat assessment caseload. Therefore, we considered Adjudication Center performance in processing the caseload for these three programs to generally reflect the performance of the Adjudication Center as a whole. We analyzed pertinent laws and regulations related to TSA's security threat assessments and program requirements for the TWIC, HME, and Aviation Worker programs, including provisions enacted through the Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism (USA PATRIOT) Act of 2001,[Footnote 8] the Aviation and Transportation Security Act,[Footnote 9] and the Maritime Transportation Security Act (MTSA) of 2002.[Footnote 10] We also reviewed TSA program documentation that included performance standards and requirements for the Adjudication Center and its contractor, such as Standard Operating Procedures for Case Management and contract documentation which included performance standards and requirements for the Adjudication Center and its contractor, such as the Adjudication Center's Quality Assurance Surveillance Plan and Evaluation and Performance Reports. Through our review, we determined TSA used three key performance metrics to measure the performance of the Adjudication Center in processing security threat assessments: (1) timeliness for completing initial adjudication of a case, (2) caseload size, and (3) adjudication accuracy. We obtained and analyzed performance data for these programs and measures for the period of October 2010 through January 2013 because this time period covers the operations of the Adjudication Center's current contractor.[Footnote 11] We assessed the reliability of these data by reviewing TSA's data management practices and questioning knowledgeable officials about the data and the systems that produced the data. On the basis of our assessments, we determined that some of the data were sufficiently reliable for the purposes of this report, but that other data were not. In particular, we found that TSA's Aviation Worker caseload size data had been unreliable since April 2012--so as we describe later in this report, we did not include data after that time in our analysis. We evaluated Adjudication Center performance measurement practices against criteria outlined in Standards for Internal Control in the Federal Government.[Footnote 12] Finally, we interviewed officials from various TSA offices with responsibilities for implementing aspects of security threat assessments or measuring performance of the Adjudication Center. These included officials from the Office of Law Enforcement/Federal Air Marshal Service (OLE/FAMS) Adjudication Center and the Office of Intelligence and Analysis (OIA), including the (1) Technology Infrastructure Modernization (TIM) program, (2) Program Management Division which is responsible for managing TSA's maritime, surface, and aviation credentialing programs, which include the TWIC, HME, and Aviation Worker programs, and (3) Office of Technology, which is responsible for managing TSA's case management systems. In addition, we interviewed officials with the American Association of Airport Executives, to obtain information about their role as a vendor in TSA's Designated Aviation Channeler program, which TSA uses to transmit Aviation Worker case information with TSA-regulated airport authorities.[Footnote 13] To determine the extent to which TSA offices have coordinated to ensure that the Adjudication Center meets evolving security threat assessment workload demand, we reviewed and compared staffing related information in TSA documents, including the Adjudication Center's October 2012 staffing plan which details staffing levels and workload projections for current and future year needs, as well as Adjudication Center workload and performance reports and project management reports. We interviewed TSA officials, including OIA officials responsible for implementing maritime, surface, and aviation security programs and, OLE/FAMS Adjudication Center officials responsible for workforce planning and operations at the Adjudication Center. We evaluated TSA's reported collaboration practices in relation to those identified in Standards for Internal Controls in the Federal Government[Footnote 14] and the Government Performance and Results Act, as amended.[Footnote 15] To determine the extent to which TSA has addressed potential risks posed by using a mix of government employees and contractors to adjudicate security threat assessments, we reviewed DHS and TSA documents detailing the results of the Adjudication Center BWS assessment, including DHS and TSA plans and memorandums for addressing risks identified in the BWS Assessment. We analyzed DHS BWS guidance detailing the roles and responsibilities of the DHS BWS Program Management Office (BW PMO) and department components for implementing the BWS, and compared TSA actions against this guidance. We also interviewed DHS and TSA officials responsible for implementing the BWS, including officials from the DHS Office of the Chief Human Capital Officer, and the Executive Director and staff of the DHS BWPMO, which is responsible for developing, implementing, and overseeing department implementation of the BWS. We also interviewed TSA officials responsible for implementing the BWS for the Adjudication Center, including officials from the Office of Acquisition, which serves as the lead for the effort, the Office of Human Capital, the Office of the Chief Counsel, and from OLE/FAMS, the Business Management Office and the Adjudication Center. We conducted this performance audit from October 2012 to July 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: TSA Security Threat Assessments and Responsible Offices: TSA is responsible for administering background checks--known as security threat assessments--for maritime, surface, and aviation transportation security programs that have vetted approximately 15 million applicants since 2003, according to TSA officials.[Footnote 16] Security threat assessments are designed to ensure that only eligible individuals are granted TSA-related credentials, such as a TWIC. Specifically, security threat assessments focus on identifying threats posed by individuals seeking to obtain an endorsement, credential, access, and/or privilege for, among other purposes, unescorted access to secure or restricted areas of transportation facilities at maritime ports and TSA-regulated airports, and for commercial drivers transporting hazardous materials. Implementing these programs is a shared responsibility among multiple TSA offices, including the OIA Program Management Division which manages the programs, and the Adjudication Center within OLE/FAMS, which serves as the primary operational component for conducting security threat assessments for 12 of TSA's 17 aviation, maritime, and surface transportation credentialing programs--with the TWIC, HME and Aviation Worker programs accounting for a reported 95 percent of the Adjudication Center's workload.[Footnote 17] (See appendix I for a TSA organization chart showing TSA offices responsible for implementing transportation security threat assessment programs.) The security threat assessment process includes reviewing information to determine if applicants are disqualified to possess a credential based on criminal offenses, immigration status, or a link to terrorism. The security threat assessment involves two key components: * Automated watchlist and related vetting: The initial automated vetting process is conducted to determine whether any derogatory information is associated with the name and fingerprints submitted by an applicant during the enrollment process. Among the checks conducted by TSA, one is against criminal history records maintained by or available through the Federal Bureau of Investigation (FBI). These records contain information from federal, state and local sources in the FBI's National Crime Information Center database and the FBI's Integrated Automated Fingerprint Identification System/Interstate Identification Index, which maintain criminal records and related fingerprint submissions. A check is also conducted against the Terrorist Screening Database, which is the federal government's consolidated terrorist watchlist and from which the Selectee and No- Fly lists, among others, are compiled.[Footnote 18] To determine an applicant's immigration/citizenship status, applicant information is checked against the Systematic Alien Verification for Entitlements system. If the applicant is a U.S.-born citizen with no related derogatory information, the system can approve the individual's application for a credential with no further review of the applicant or human intervention. * Adjudication Center review: A manual, second level review is conducted as part of an individual's security threat assessment if (1) the automated vetting uncovers any derogatory information, such as a criminal offense[Footnote 19] or (2) the applicant has identified himself or herself to be a non-U.S.-born citizen or national. As such, not all applicants will be subjected to a second-level review.[Footnote 20] The Adjudication Center plays an integral role in the security threat assessment process by adjudicating cases for which an initial automated check finds potential links to criminal history or immigration eligibility issues. Adjudication Center staff review the program applicant's enrollment file to determine if derogatory or other information may be potentially disqualifying. The applicant's files are processed from credentialing program enrollment centers through two-web enabled case management systems, called the Screening Gateway and Consolidated Screening Gateway. Adjudication Center staff use the Screening Gateways as their tool for gathering, viewing, and synthesizing the information needed to conduct security threat assessments. Since its establishment in 2005, the Adjudication Center has relied primarily upon contractor staff to complete its security threat assessment workload, and a smaller number of federal government staff to conduct oversight and other functions.[Footnote 21] Contractor staff performs initial adjudication of cases, and may either approve applications if they determine an applicant is eligible to obtain a credential or refer the application to a federal (that is, TSA) adjudicator for further review if they determine the applicant to be ineligible. Federal staff review cases of potential ineligibility, issue Preliminary Determination of Ineligibility letters to applicants, and conduct redress actions, among other things.[Footnote 22] As of May 2013, TSA reported that about two-thirds (37 of 55) of Adjudication Center staff were contractors. Figure 1 shows the TSA credentialing process for the TWIC, HME, and Aviation Worker programs from enrollment through credential issuance, and the functions of the Adjudication Center's TSA and contract staff in the security threat assessment process. Figure 1: TSA TWIC, HME, and Aviation Worker Program Processes and Contractor and TSA Staff Functions in Security Threat Assessment Process at the Adjudication Center: [Refer to PDF for image: process illustration] Enrollment: Applicants pay an enrollment fee and submit biographic (name, address, etc.) and biometric information, as applicable by program (photo, if applicable, and fingerprints) to TSA. Transportation Worker Identification Credential (TWIC): Applicant provides biographic and biometric information at one of 136 TWIC enrollment centers. Information routed and screened on TSA Consolidated Screening Gateway system. Hazardous Materials Endorsement (HME): Depending on the state from which an application is being made, applicant provides biographic and biometric information to either TSA enrollment sites or state-designated sites, which transmit applications to TSA. Information routed and screened on TSA Screening Gateway system. Aviation worker: Applicant provides biographic and biometric information to airport operator. Airport operator submits information to TSA. Information routed through TSA Designated Aviation Channeler program and screened on TSA Consolidated Screening Gateway system.[A] Adjudication Center security threat assessment process: 1) Electronic background check: No potentially derogatory results: go to step 2; Potentially derogatory results: go to step 4. 2) Quality assurance process to identify adjudicator errors. 3) “No threat” determination, applicant eligible for credential: Credential issuance: TWIC: TSA issues TWIC to applicant; HME: State licensing agency issues HME to applicant; Aviation worker: Airport issues access badge to employee. 4) Initial review of background check results by contractor adjudicators to determine credential eligibility: Favorable: go to step 3 and continue; Unfavorable: go to step 5. 5) All denials reviewed by mid- and senior-level adjudicators: Favorable: go to step 3 and continue; Unfavorable: go to step 6. 6) Preliminary determination of ineligibility letter sent to applicant: Applicant may not be eligible for credential; May seek redress[B]. Source: GAO analysis of TSA information. [A] TSA uses the Designated Aviation Channeler program to transmit Aviation Worker case information between TSA and TSA-regulated airport authorities. [B] If TSA determines an applicant meets one or more disqualifying criteria, the agency issues a Preliminary Determination of Ineligibility letter to the applicant. An applicant who receives such a letter may take steps to appeal the determination as instructed by TSA and consistent with applicable regulations--known as the redress process. [End of figure] Use of Contractor Personnel and the DHS Balanced Workforce Strategy: Federal agencies face a complicated set of decisions in finding the right mix of government and contractor personnel to conduct their missions. While contractors, when properly used, can play an important role in helping agencies accomplish their missions, our prior work has shown that agencies face challenges with increased reliance on contractors to perform core agency missions.[Footnote 23] Consistent with Office of Management and Budget procurement policy, agencies should provide a greater degree of scrutiny when contracting for professional and management support, program evaluation, and other services that can affect the government's decision-making authority-- functions that may be considered as being closely associated with inherently governmental functions.[Footnote 24] Contractors can provide services that closely support inherently governmental functions, but agencies must provide greater scrutiny and enhanced management oversight to ensure that the contractors' work does not limit the authority, accountability, and responsibilities of government employees.[Footnote 25] The DHS BWS refers to the department's effort to identify the appropriate balance of federal and contractor employees required to support critical agency functions. Consistent with our recommendations[Footnote 26] and in accordance with the Omnibus Appropriations Act, 2009[Footnote 27] DHS adopted the BWS in August 2010 to undertake risk analyses that are to enable the department to achieve the appropriate mix of Federal employees and contractors to accomplish its mission while minimizing mission risk that may result from an over-reliance on contractors.[Footnote 28] DHS uses an automated tool to help components--such as TSA--perform the necessary analysis to categorize work as appropriate for use of a contractor, inherently governmental, or closely associated with an inherently governmental function. The assessment tool is intended to facilitate an assessment of mission risk, level of contractor oversight needed, risk mitigation strategies, and cost analysis. Based on component responses, the tool is to provide a recommended sourcing decision on whether the work is appropriate for federal or contractor performance, or both. For example, should the BWS assessment find that a function is inherently governmental, the component would recommend the function be insourced to government employees, whereas a determination that the function was closely associated with an inherently governmental function would require the agency to either insource the function (also known as federalizing), or strengthen oversight of the contractor workforce.[Footnote 29] Adjudication Center Contractor Related Performance Challenges: In December 2011, we reported that the Adjudication Center had faced recurring challenges in meeting its security threat assessment workload requirements and largely attributed these challenges to its reliance on a contractor workforce.[Footnote 30] Specifically, the Adjudication Center had experienced recurring backlogs in completing its caseload, and Adjudication Center officials attributed these backlogs to staffing limitations caused by contractor turnover. Officials at the time reported that the challenge was that the Adjudication Center had used three different contractors since establishing the Adjudication Center in 2005, and on each occasion the contract adjudicator turnover had led to backlogs as adjudicators were hired and trained. TSA reported that it did not consider the risks of acquiring contractor support services to provide adjudication services before awarding its first contract in 2005. Rather, TSA reported that it chose to use contract adjudicators when the Adjudication Center was created because, at the time, it considered them to be the most readily available workforce and effective way to augment federal staff with skilled resources. TSA reported that the agency had initiated an assessment in March 2011 through the DHS BWS process to determine whether the adjudication functions were appropriate to be performed by a contractor workforce, whether the work was inherently governmental and whether there would be cost savings resulting from conversion of the contract positions to government positions. We recommended TSA develop a workforce staffing plan with timelines articulating how the Adjudication Center will effectively and efficiently meet its current and emerging workload requirements, and incorporate the results of TSA's study examining the appropriateness and costs and benefits of using contractors. TSA concurred with our recommendation and reported that it had begun taking steps to implement it. Adjudication Center Performance Data Show Mixed Results, Accuracy Measures Are Limited, and Performance Measurement Practices Are Not Fully Documented: TSA has evaluated the Adjudication Center largely based on contractor performance in meeting established metrics and data shows mixed performance since 2011; however the Adjudication Center's performance measures and practices are limited. We found that the Adjudication Center contractor met two of its three performance measures--for timeliness and accuracy--but did not do so for its caseload size measure.[Footnote 31] Further, these measures and practices were limited. For example, the Adjudication center's methodology for calculating contractor adjudicator accuracy was limited because it did not include key information. Moreover, the Adjudication Center has not documented key elements of its performance measurement practices. Two Performance Measures Showed Mixed Results: TSA has used performance data for three primary metrics to measure the performance of the Adjudication Center in conducting security threat assessments for the TWIC, HME, and Aviation Worker programs. The three metrics are timeliness for completing initial adjudication, caseload size, and adjudication accuracy. According to TSA Adjudication Center officials, these performance measures were established to evaluate the performance-based contract for adjudication services at the Adjudication Center.[Footnote 32] Timeliness. The Adjudication Center contractor met timeliness standards for completing initial adjudication of its TWIC, HME, and Aviation Worker caseloads (see figure 1 for a description of this process). TSA requires that its contract adjudicator workforce complete initial adjudication of 95 percent of cases within 7 calendar days of the case entering TSA's Screening Gateway case management systems for TWIC, HME, and Aviation Worker cases.[Footnote 33] According to TSA data, from August 2011 to January 2013, the adjudicator workforce met this standard for TWIC, HME, and Aviation Worker cases.[Footnote 34] While the Adjudication Center's timeliness measure shows the Adjudication Center's contractor met TSA's standard for completing initial adjudication, the measure does not show the extent to which the agency has communicated its adjudication decision to the applicant in a timely manner--key statutory and TSA policy requirements for its credentialing programs--and TSA officials reported they did not maintain such documentation. For example, as specified in statute, TSA shall review an initial TWIC application and provide a response to the applicant, as appropriate, within 30 days of receiving the initial application.[Footnote 35] Moreover, officials with the OIA Program Management Division and Adjudication Center reported that TSA had established internal requirements for the agency to meet 30 day and 14 day applicant response times for HME and Aviation Worker applicants. Officials from the OIA Program Management Division reported tracking this measure through weekly Adjudication Center performance reports and identifying and addressing those cases that do not meet applicant response time standards. However, officials reported that they did not maintain documentation showing the extent to which TSA had responded to applicants within their applicant response timeframe requirements. Officials reported that maintaining such performance data would be of use, but noted it was rare that they did not meet their initial adjudication standards and respond to applicants on or within established applicant response time requirements. Officials noted that functional limitations in TSA's Screening Gateway reporting system limits their ability to efficiently run reports showing the extent to which TSA responds to applicants within required timeframes. A senior OIA Program Management Division official reported it was her understanding that the division would have to obtain the capability to automatically run applicant response time reports from TSA's Technology Infrastructure Modernization program, known as TIM. [Footnote 36] However, we reviewed TIM program documentation and did not find this data management capability requirement in TIM planning documents. We raised this issue with TSA TIM program officials, and in response to our inquiry, in May 2013, the TIM program added documentation of this requirement to its plans and reported that the capability would be available to the Adjudication Center beginning in March 2014 for TWIC program cases, and by 2016 for surface and aviation program cases.[Footnote 37] Caseload size. The Adjudication Center generally did not meet its contract caseload performance standards and experienced backlogs for its TWIC and HME program caseloads the majority of the time between October 2010 and January 2013. According to TSA contractor evaluation and performance reports, the Adjudication Center requires its contract workforce to maintain a total number of new TWIC, HME, and Aviation Workers cases at or below 1,500 cases--and Adjudication Center officials told us that a caseload above this threshold was considered a backlog. Adjudication Center data we reviewed for the period of October 2010 through January 2013 showed that the Adjudication Center had a backlog of HME cases approximately 60 percent of the time and TWIC cases approximately 61 percent of the time. In addition, the Adjudication Center had a backlog of Aviation Worker cases approximately 15 percent of the time from October 2010 through March 2012.[Footnote 38] Moreover, many of these backlogs were far higher than the Adjudication Center's 1,500 caseload standard. For example, the Adjudication Center had a backlog of more than 4,000 HME cases roughly 16 percent of the time (20 of 122 weeks) during this period. Figure 2 shows Adjudication Center caseload levels for TWIC, HME, and Aviation Worker cases from October 2010 through January 2013. Figure 2: Caseload Levels for TWIC, HME, and Aviation Worker Programs, October 2010 through January 2013: [Refer to PDF for image: multiple line graph] Backlog standard: 1,500. October 2010; Hazardous Materials Endorsement (HME): 1,931; Aviation workers[A]: 1,547; Transportation Worker Identification Credential (TWIC): 2,986. January 2011; Hazardous Materials Endorsement (HME): 1,899; Aviation workers[A]: 550; Transportation Worker Identification Credential (TWIC): 2,453. April 2011; Hazardous Materials Endorsement (HME): 10,379; Aviation workers[A]: 1,798; Transportation Worker Identification Credential (TWIC): 3,908. July 2011; Hazardous Materials Endorsement (HME): 1,300; Aviation workers[A]: 310; Transportation Worker Identification Credential (TWIC): 575. October 2011; Hazardous Materials Endorsement (HME): 1,184; Aviation workers[A]: 538; Transportation Worker Identification Credential (TWIC): 1,245. January 2012; Hazardous Materials Endorsement (HME): 1,542; Aviation workers[A]: 209; Transportation Worker Identification Credential (TWIC): 1,530. April 2012; Hazardous Materials Endorsement (HME): 4,080; Transportation Worker Identification Credential (TWIC): 2,484. July 2012; Hazardous Materials Endorsement (HME): 1,178; Transportation Worker Identification Credential (TWIC): 1,793. October 2012; Hazardous Materials Endorsement (HME): 2,277; Hazardous Materials Endorsement (HME): 1,819; Transportation Worker Identification Credential (TWIC): 1,724. January 2013; Hazardous Materials Endorsement (HME): 1,533; Transportation Worker Identification Credential (TWIC): 2,107. Source: GAO analysis of TSA Adjudication Center data. [A] TSA identified the Aviation Worker caseload size data as being unreliable since April 2012. TSA reported that these data were unreliable because of technical problems TSA has experienced with its Designated Aviation Channeler program TSA used to process Aviation Worker security threat assessment applications. TSA reported it had yet to resolve the causes for its data reliability limitations and continues to seek a solution from the vendor experiencing technical problems. [End of figure] According to Adjudication Center officials and TSA documentation we reviewed, technical issues and a lack of sufficiently trained contract adjudicators contributed to the workload backlogs at the Adjudication Center.[Footnote 39] First, the Adjudication Center operations manager reported that technical problems with its case reporting systems had contributed to both challenges in assessing workload backlogs and, in some cases, growth in the backlog itself. For example, the Screening Gateway systems, which the Adjudication Center relies on for processing applicant cases and communicating results to TSA enrollment centers, has experienced periodic technical errors that have delayed the Adjudication Center's ability to process new cases. According to TSA evaluation and performance reports we reviewed, between February 2012 and August 2012, TSA was unable to evaluate contractor performance in meeting its workload on several occasions, including approximately 3 months, because of technical problems with its case management systems. Adjudication Center officials reported that TSA's Office of Technology was pursuing a solution to the technical errors with a solution expected by May 2012; however, as of May 2013 this had not been corrected. They also reported that TSA plans to replace this system with a more functional system through its TIM program, but as noted earlier, according to TSA's schedule for the program and TSA officials, this system is not scheduled to be fully operational until 2016. In addition, since April 2012 TSA has experienced technical problems related to the Designated Aviation Channeler program that TSA uses to process Aviation Worker program cases into the Screening Gateway systems. According to TSA officials, technical problems with one of its vendors were delaying processing of cases and returning previously adjudicated cases into the Adjudication Center's new caseload queue and not distinguishing between the two sets of cases. This was delaying processing time and Adjudication Center management was unable to determine the true extent of its new caseload. TSA officials responsible for managing the Designated Aviation Channeler program reported that they had been in discussions with the vendor since April 2012 to address the technical processing issues, and as of May 2013, the vendor was in the process of implementing corrective actions. Another factor contributing to growth in the workload backlog according to TSA Adjudication Center management officials and a contractor performance report we reviewed has been the lack of trained adjudicators provided by the contractor.[Footnote 40] According to a senior Adjudication Center official, the contractor lacked a sufficient number of staff who had been certified as self approvers, and this had required the Center's limited federal staff to assume additional responsibilities and reduced the Center's progress in meeting its caseload.[Footnote 41] Adjudication Center officials reported that they were working with the contractor to address this issue. We discuss the Adjudication Center's contractor-related staffing issues, and actions to address them, in more detail later in this report. The Adjudication Center's Contractor Met TSA's Accuracy Rate, but the Rate Does not Include Key Information: For its third key performance measure, TSA requires its contract adjudicators to maintain an average accuracy rate of at least 95 percent.[Footnote 42] According to Adjudication Center data, from August 2011 to December 2012, the Adjudication Center's contract workforce met TSA's accuracy standard for the TWIC, HME, and Aviation Worker programs.[Footnote 43] However, the accuracy rate is not a complete representation of Adjudicator contract accuracy because it does not include evaluation of a key population of cases. According to Adjudication Center officials, the Adjudication Center's average accuracy rate is generally based on error rates identified from a daily review of all cases where adjudicators found an applicant was disqualified, but reviewers found an applicant should not have been (i.e., incorrectly disqualified). However, according to officials, this calculation generally does not include those cases where adjudicators had approved applicants, but reviewers found they should have been disqualified (i.e., incorrectly approved). For example, according to our analysis of TSA data, approvals comprised over 90 percent of the Adjudication Center's TWIC and HME caseload from August 2011 to January 2013--and TSA reviewed roughly 7 percent of these approvals. In this way, the average accuracy rate TSA uses to evaluate the performance of its contractor is incomplete and limited because it does not include the extent contract adjudicators incorrectly approved applicants.[Footnote 44] The Adjudication Center official responsible for reporting the accuracy rate told us that the accuracy rate of the contract workforce includes only those cases that were incorrectly found to have disqualifying factors because that is how the contract evaluation standards were established. The official noted that the Adjudication Center processes included a review of all trainee adjudicators approved cases and a separate quality assurance review process to spot check approved cases[Footnote 45] to identify errors among all adjudicators who are certified to approve cases without further review.[Footnote 46] However, the official reported that these performance measurement practices were not documented and that a lack of staffing capacity had limited the extent to which the Adjudication Center conducted the quality assurance spot checks--with the Center meeting only about two-thirds of its 10 percent goal for the number of cases selected for spot checking. Nonetheless, the results of this quality assurance review are not factored into the rate TSA uses to measure contractor accuracy performance and award funds to its contractor. Standards for Internal Control in the Federal Government specifies the need to comprehensively identify risks and consider all significant interactions. Once risks have been identified, they should be analyzed for possible effect. Moreover, internal control and all transactions and other significant events need to be clearly documented, and the documentation should be readily available for review.[Footnote 47] The overall accuracy rate calculated by the Adjudication Center is generally limited to incorrectly disqualified cases and does not include incorrectly approved cases. In this way, TSA does not have a representative assessment of the Adjudication Center's average accuracy rate. If error rates for approved cases were included in its evaluation, the Adjudication Center's reported average accuracy rate may ultimately be higher or lower than it has reported--but it will remain unclear until the Adjudication Center captures this information in its accuracy rate. Determining the performance of the workforce in adjudicating security threat assessments for this population is important for overseeing adjudicator performance and identifying cases where the Adjudication Center is incorrectly approving applicants. By developing and documenting an accuracy rate measure that includes data on both types of incorrectly adjudicated cases (approved and disqualified), the Adjudication Center can determine an accuracy rate that comprehensively captures accuracy performance and enables Adjudication Center management to more effectively identify and address performance issues among its workforce. The Adjudication Center's Manual Performance Management Process is not Documented: Since beginning operations in 2005, Adjudication Center management officials told us that they have used a complex, manual process to track the performance data of its contract adjudicator workforce. In particular, because of functional limitations of TSA's Screening Gateway systems, officials reported that the Adjudication Center lacks an automated process for tracking adjudicator performance of the estimated 7,500 to 10,000 security threat assessment cases that adjudicators process each week.[Footnote 48] As a result, Adjudication Center management has used a cumbersome, manual process to track case production and performance of its contract adjudicator workforce.[Footnote 49] For example, each week, one adjudication center official is responsible for reviewing contractor reported caseload information, compiling spreadsheets summarizing contractor performance, verifying and reconciling the information with the contractor, and preparing weekly summary reports for distribution to TSA credentialing program stakeholders. Adjudication Center management told us that it has used these reports to measure Adjudication Center performance and support oversight of its contract adjudicator workforce. The manual process exists because, according to Adjudication Center officials, TSA's Screening Gateway case management systems were not designed to meet the functional requirements of the Adjudication Center for tracking contractor operational performance, and TSA has been unsuccessful to date in developing a technical solution to do so. [Footnote 50] TSA officials recognized that the system did not meet the needs of the Adjudication Center and reported that the agency's TIM program would replace the Screening Gateway systems and enable the adjudication center to automate its case tracking and performance requirements. However, as noted earlier, TSA officials reported that this new system is not scheduled to be fully operational until 2016. In the meantime, however, Adjudication Center management officials reported that they had not documented the manual process currently in use. Adjudication Center management officials told us that they had placed some information on an internal web sharing system in the past, but that this information was neither thorough nor updated to reflect the case management reporting system that the Adjudication Center has used since 2010--when TSA began its most recent contract for Adjudication Center staff. According to Adjudication Center officials, time constraints in meeting the Adjudication Center's workload of security threat assessments had been a factor that had prevented the Operations Manager from updating or developing new documentation of the procedures in recent years. Further, given the complexity of the process and that two officials were familiar it, a senior Adjudication Center management official said that documenting this process would be of value should the two officials be unavailable. Standards for Internal Control in the Federal Government specifies the need for appropriate documentation of transactions and internal control. Internal control and all transactions and other significant events need to be clearly documented, and the documentation should be readily available for review.[Footnote 51] The documentation should be included in directives, policies, or manuals to help ensure operations are carried out as intended. Documenting the Adjudication Center's case reporting performance measurement practices is important to allow someone unfamiliar with this process to assume responsibilities in the event of attrition by the Adjudication Center managers. This is particularly important considering the complexity of the Adjudication Center's case performance reporting process and TSA's need to ensure effective performance and operational continuity in its security threat assessments. TSA Offices Can Strengthen Coordination to Ensure Effective Adjudication Center Workload Planning: Implementing credentialing-related programs is a shared responsibility between the Program Management Division in TSA's OIA and the Adjudication Center in OLE/FAMS. Officials from these offices reported taking various actions to ensure its offices coordinate information related to security threat assessment adjudication workload planning and performance. These include: Sharing weekly Adjudication Center performance reports: These reports include information for the TWIC, Aviation Worker, and HME programs such as the number of cases the Adjudication Center receives for each of these programs during the prior week, the number of cases ready for adjudication, and the number of applicants who have sought redress based on initial determinations of ineligibility. The three program managers for TSA's maritime, aviation, and surface credentialing programs reported that they rely on these reports to ensure the program offices are meeting workload demands for the various credentialing programs TSA supported, and to identify and develop strategies to address performance challenges. Convening monthly program management review meetings: These meetings are used to share information relating to changes that may impact the Adjudication Center's workload.[Footnote 52] As part of these meetings, the Adjudication Center contractor provides a monthly report which provides details pertaining to contractor staffing levels and changes, training status, contractor accuracy rates, and challenges in need of resolution. Developing spend plans: Adjudication Center and the OIA Program Management Division officials meet to develop a spend plan to support the credentialing programs' annual budgets and discuss population projections for the programs that would affect Adjudication Center workload. For example, the Aviation Worker program manager reported that the workload had increased by about 5 percent annually, and that this information was used to inform the Adjudication Center's spend plan. Notwithstanding these actions, opportunities exist for the Adjudication Center and OIA Program Division to strengthen their coordination. While officials with the two offices coordinate on a routine basis to share information on workload completion, they do not have a process in place to ensure that information in the Adjudication Center's staffing plan--such as caseload projections and associated staffing needs--reflects the mutual understanding of both Adjudication Center and credentialing program management officials.[Footnote 53] For instance, Adjudication Center management officials have periodically updated a staffing plan that they use to guide Adjudication Center workforce planning. However, an Adjudication Center program management official reported that while the staffing plan had been shared with credentialing program managers in the past, it had not been shared in recent years. He reported that a prior plan had been shared with the OIA Program Management Division to communicate staffing needs, and said that sharing the updated versions of the staffing plan with the Program Management Division may be valuable for guiding decisions on workforce planning. OIA Program Management Division officials reported that they were unfamiliar with the Adjudication Center's staffing plan and questioned workload projections in the Adjudication center's current staffing plan. For example, the current Adjudication Center staffing plan cites an anticipated regulation that will address the security threat assessment process and that according to the plan would double the Adjudication Center's security threat assessment workload from 500,000 to 1 million per year by the end of fiscal year 2014, and triple the workload by the end of fiscal year 2015.[Footnote 54] In October 2012, we shared this staffing plan with the OIA Program Management Division Manager responsible for Aviation programs and that official questioned the accuracy of the aviation worker workload increase projections in the staffing plan. The official said that TSA had yet to issue this regulation and that the timeline for doing so would take longer than officials had initially planned. Thus, the projected workload increases in the Adjudication Center's staffing plan would not be realized, and the plan would need to be revised. However, as of March 2013, the Adjudication Center's staffing plan had not been updated. According to key collaboration practices that we have identified, federal agencies engaged in collaborative efforts need to create the means to monitor and evaluate their efforts to enable them to identify areas for improvement.[Footnote 55] Reporting on these activities can help key decision makers within the agencies, as well as clients and stakeholders, obtain feedback for improving both policy and operational effectiveness. Such reporting mechanisms can then be used to modify plans. Moreover, a focus on results, as envisioned by the Government Performance and Results Act, as amended, implies that federal programs contributing to the same or similar results should collaborate to ensure that goals are consistent and, as appropriate, program efforts are mutually reinforcing.[Footnote 56] In this way, ensuring these components have access to respective workforce planning documents by establishing a mechanism for OIA Program Management Division and Adjudication Center officials to share and reconcile information included in the Adjudication Center's staffing plan updates, such as timelines for anticipated workload growth, will help ensure TSA is using accurate workload projections to guide the Adjudication Center's workforce planning. TSA Has Made Limited Progress in Addressing Risks Posed by Using Contract Employees to Adjudicate Security Threat Assessments: TSA Assessment Identified Risks in the Adjudication Center's Use of Contractors: Between January 2011 and September 2011, TSA conducted and completed its DHS required BWS assessment for the Adjudication Center contract and determined that the adjudicator position represented work that is "closely associated with inherently governmental functions" and that an excessive risk exists by allowing contractors to make security credential approvals without sufficient federal oversight. According to the assessment, the adjudicator functions performed by the contractor are critical to TSA's accomplishment of the security threat assessment process to ensure terrorist and other security threats are identified and prevented from gaining credentialed access to critical U.S. transportation system infrastructure.[Footnote 57] The assessment found that TSA was reliant upon contractors for making decisions regarding criminal history and immigration status for a majority of applicants and if contractors were to continue performing the adjudicator function, the government would need to provide continuous and substantive oversight of them to ensure successful performance. [Footnote 58] However, the assessment found that the Adjudication Center did not have an effective oversight process in place to do so-- noting that the federal government staffing at the Adjudication Center is not sufficient to adequately oversee contractor case processing for quality control, as contract staff have independent decision making ability on the majority of cases.[Footnote 59] Further, the assessment noted that the Adjudication Center's use of a mixed contractor and government workforce was inefficient. For example, according to the assessment, for every contractor work hour, a federal government employee must check that work and this had led federal government staff to work more than 2,500 overtime and compensation hours over the preceding year--an inefficient and duplicative process that would not be necessary if the workforce were all federal government officers. [Footnote 60] In light of these factors, in October 2011, TSA's BWS Departmental Working Group determined that the adjudicator function was closely associated with inherently governmental functions and recommended TSA end the Adjudication Center's reliance on a contract workforce and convert to an all federal employee workforce.[Footnote 61] The working group reported that doing so was designed to improve the Adjudication Center's security threat assessment processing by having a better oversight process, streamlining overall operations, reducing training requirements, and better managing resources. TSA Has Been Delayed in Implementing Workforce Conversion: TSA has been delayed in implementing the proposed workforce conversion at the Adjudication Center. According to TSA's May 2012 Adjudication Center conversion plan, TSA offices were to take several actions before the Adjudication Center may begin implementing its conversion plan and hiring a new federal employee workforce; however, these steps have generally not been implemented. For instance, as of May 2013, responsible stakeholders in TSA's BWS effort, including the Office of Human Capital and OLE/FAMS, reportedly had not approved the plan-- necessary steps before the plan can proceed to TSA leadership, and ultimately, DHS for review. According to the conversion plan, TSA proposed to convert to a government adjudicator workforce by hiring TSA employees during fiscal years 2013 and 2014--with the hiring to be completed by the end of calendar year 2013. However, as of May 2013, TSA had not begun hiring its new federal workforce and TSA officials reported that the agency had not determined new timelines to do so. TSA officials attributed the agency's delays in implementing the Adjudication Center conversion plan to its prioritization on implementing agency reorganization efforts. According to a senior TSA official, implementing the BWS assessment was delayed because TSA was undergoing a large reorganization and agency resources were prioritized to that effort.[Footnote 62] With this reorganization completed in January 2013, the official reported that implementing the conversion plan would become a greater priority. TSA Has Not Updated and Approved the Conversion Plan and Key Plan Elements are Unclear: TSA's delay in acting on the recommendations of Adjudication Center conversion has rendered the implementation timelines and key hiring level and cost information in its May 2012 conversion plan outdated or unclear and TSA has not updated the plan to reflect these changes. In particular, TSA's plan to convert to an all-federal Adjudication Center workforce has not been updated although information in this plan, such as the timeline for hiring federal employees and cost information, is no longer valid or is unclear. For example: * The implementation schedule in TSA's plan is no longer valid. TSA officials responsible for managing the conversion effort acknowledged that the timelines for implementing its plan had been delayed, and that TSA would not complete its workforce conversion by the end of calendar year 2013 as proposed in its plan. TSA officials reported that determining a revised schedule for the Adjudication Center conversion was dependent on various factors, such as when responsible TSA offices completed their respective reviews, and when OLE/FAMS approved the plan and sent it to TSA leadership for review. TSA officials reported that TSA did not have timelines for when this would occur. * Implementing the in-sourcing plan may present a cost saving opportunity to TSA, but TSA is unclear on extent of those savings. According to an August 2012 OLE/FAMS memorandum, converting from contractor to federal employees at the Adjudication Center would save the federal government over $5.4 million in fiscal years 2013 and 2014.[Footnote 63] However, TSA's May 2012 conversion plan reports that the conversion plan would result in approximately $1 million in savings, rather than the $5.4 million cited in an August 2012 OLE/FAMS memorandum. According to the May 2012 plan, TSA used the DHS Modular Cost Table to determine the potential cost savings from converting to a federal employee workforce at the Adjudication Center.[Footnote 64] TSA budget officials reported that they could not determine why the cost savings estimates varied between the May 2012 conversion plan and the August 2012 memorandum. The officials reported that the cost savings estimate was still speculative and that TSA would need to revisit its calculations. As of May 2013, TSA had provided no further information. TSA officials involved in the BWS Adjudication Center conversion noted that the delays in implementing the plan may pose challenges for TSA. For example, TSA's contract for the Adjudication Center is a performance-based contract, with 1 option year remaining that would begin in February 2014 and run to January 2015.[Footnote 65] The official reported that continuing the contract would delay TSA from potential cost savings, while the cost to TSA continuing the contract increases 3 percent per option year.[Footnote 66] Officials reported that if they did not begin hiring new federal employees by August 2013, they would need to begin the process to recompete the Adjudication Center staffing contract to ensure continuity of operations in the case that TSA does not implement its conversion plan. According to DHS BWS guidance, the more important the function, the more important it is to have internal capability to maintain control of the department's mission and operations. TSA's BWS assessment found that (1) TSA lacked sufficient internal capacity to control its use of contractors in Adjudication Center mission and operations, (2) TSA's reliance on a contractor workforce carried excessive risk, (3) the adjudicator functions were closely associated with inherently governmental functions, and (4) that the positions should be insourced. This assessment was made almost 2 years ago. While senior TSA Adjudication Center management officials support implementation of the plan, collectively, TSA has not mitigated the risks and operational inefficiencies identified in the DHS BWS assessment. Moreover, TSA has not completed its internal review of the conversion plan, including determining a revised implementation schedule as well hiring target levels and cost information. Completing this review, determining this information, and updating the conversion plan to ensure the plan reflects current conditions and an estimation of cost savings will help TSA and DHS decision makers by providing a roadmap for moving forward. Finally, implementing TSA's Adjudication Center workforce conversion will be important to ensure TSA has sufficient and appropriate adjudication personnel to make the decisions that may deny or allow individuals unescorted access to the nation's critical transportation infrastructure. Conclusions: TSA's Adjudication Center plays a critical role by conducting security threat assessments to ensure individuals posing security threats are identified and are not granted TSA-related credentials for, among other things, unescorted access to secure areas of the nation's transportation systems. However, the Adjudication Center has faced challenges in fulfilling this role. First, while the Adjudication Center uses three key measures to evaluate the performance of the Adjudication Center's contract workforce, it has not documented its methods and two of its measures are limited. For example, TSA's timeliness measure does not capture the extent to which the agency has communicated its adjudication decision to the applicant in a timely manner--a key TSA requirement for its credentialing programs--and TSA officials reported they did not maintain such documentation. Ensuring that the TIM program provides the capability for Adjudication Center officials to efficiently prepare and document applicant response time reports would help ensure TSA meets standards and decisionmakers identify and address performance challenges. In addition, the Adjudication Center's accuracy rate generally does not include cases in which contract adjudicators incorrectly approved an applicant--and these constitute roughly 90 percent of the Adjudication center's caseload. Developing, documenting and implementing an accuracy rate that includes this information will provide TSA with a more complete assessment of the performance of its workforce--regardless of whether the members of that workforce are contractors or TSA employees. Second, because of functional limitations in its case reporting systems, Adjudication Center management uses an undocumented, manual, process to track adjudicator performance. Documenting the Adjudication Center's case reporting performance measurement practices is important to ensure continuity of operations in the event of attrition by the two Adjudication Center officials familiar with this process. Third, the Adjudication Center relies on its staffing plan to guide its workload planning decisions but has not shared updated versions of this plan with the credentialing program offices that it serves. Establishing a mechanism for the Adjudication Center to share and reconcile information included in the staffing plan updates, such as timelines for anticipated workload growth, would help improve internal coordination and support the Adjudication Center's workload planning efforts. Fourth, TSA's 2011 BWS assessment for the Adjudication Center found that the adjudicator function is closely associated with inherently governmental functions and recommended that TSA insource its Adjudication Center workforce to mitigate the risks that contractors were making security credential approvals without sufficient federal oversight. Taking additional steps to end its use of contract adjudicators and convert to an all-federal employee adjudicator workforce would help TSA mitigate such risks, but it has been delayed in doing so. Completing its internal review and updating and documenting the conversion plan to ensure the plan reflects current condition conditions, including timelines for hiring, planned hiring numbers, and cost information would help TSA and DHS decision makers by providing a roadmap for moving forward. Finally, providing this plan to TSA and DHS leadership for review are important steps to help ensure TSA addresses the risks identified in the 2011 BWS assessment and has an appropriate workforce to make the decisions that may ultimately deny or allow individuals credentials for unescorted access to the nation's critical transportation infrastructure. Recommendations for Executive Action: We recommend that the Secretary of Homeland Security direct the TSA Administrator to take the following 5 actions: * To ensure that the Adjudication Center accuracy rate effectively captures the center's accuracy in completing security threat assessments, the Adjudication Center should develop an accuracy rate measure that includes accuracy data for cases where adjudicators both approved and disqualified applicants, document this methodology, and implement the process. * To ensure continuity of case reporting, the Adjudication Center should document its case reporting performance management processes. * To ensure workforce planning is based on accurate workload projections, establish a mechanism for TSA's OIA Program Management Division and OLE/FAMS Adjudication Center to share and reconcile information included in the Adjudication Center's staffing plan updates, such as timelines for anticipated workload growth. * To advance efforts to address risks identified in the Adjudication Center BWS assessment, - update and document its Adjudication Center insourcing conversion plan to reflect revised schedule timeframes, cost and hiring level information, and: - review the updated Adjudication Center insourcing conversion plan, and provide it to TSA and DHS leadership for review and implementation approval. Agency Comments and Our Evaluation: We provided a draft of this report to DHS for review and comment. DHS, in written comments received July 2, 2013, concurred with all five of the recommendations in the report and identified actions taken, planned, or under way to implement the recommendations. Written comments are summarized below, and official DHS comments are reproduced in appendix II. In addition, DHS provided written technical comments, which we incorporated into the report, as appropriate. In commenting on the draft report, DHS described efforts underway or planned to address our recommendations. DHS also noted that the Adjudication Center's caseload performance measure of keeping backlogs below 1,500 cases is a self-imposed standard that TSA established to provide the best possible customer service to applicants. We agree that the Adjudication Center's caseload performance measurement was developed by TSA. Regardless of source, however, TSA's caseload standard is a contractual requirement and our analysis of TSA data found that the Adjudication Center contractor generally did not meet this requirement between October 2010 and January 2013. In addressing our recommendations, DHS concurred with our first recommendation that TSA should develop an accuracy rate that includes accuracy data for both cases where an applicant is approved and cases where an applicant is disqualified, document this methodology, and implement the process. DHS stated that TSA OIA will modify its current quality control process to include both approved and disqualified cases that will more accurately reflect the adjudications performed. Furthermore, DHS reported that it will develop, document, and formalize an accuracy rate measure that includes review of approved and disqualified cases. Such actions will ensure that the Adjudication Center's accuracy rate measure provides a more comprehensive assessment of adjudicator performance. DHS also concurred with our second recommendation that TSA should document the Adjudication Center's case reporting performance management processes. DHS stated that while TSA anticipates that the current manual process will be phased out and replaced by an automated process as the TIM program is implemented, TSA OIA will document the current manual performance management process. DHS stated that documenting the process will confirm the Adjudication Center's performance is accurately tracked and will also ensure continuity in the event of personnel turnover. These actions, if implemented effectively, should address the intent of our recommendation. Regarding our third recommendation that OIA's Program Management Division and the OLE/FAMS Adjudication Center should establish a mechanism to share and reconcile information included in the Adjudication Center's staffing plan updates, such as timelines for anticipated workload growth, DHS concurred. DHS reported that the OIA Program Management Division and the OLE/FAMS Adjudication Center were already working to resolve the issues and had begun coordination to ensure security threat assessment workload estimates and the staffing plan are updated. DHS stated that TSA will formalize a quarterly review process between the Program Management Division and the Adjudication Center to meet and discuss these issues. DHS concurred with our fourth recommendation to update and document the Adjudication Center's insourcing conversion plan to reflect revised schedule timeframes and cost and hiring level information. In its comments, DHS stated that OIA is working with the DHS Office of the Chief Human Capital Officer to address any potential issues posed by using a mix of government employees and contractors. Furthermore, DHS reported that TSA will update its insourcing conversion plan to reflect current timelines, costs, and hiring levels. Such actions should improve TSA's ongoing insourcing efforts. Lastly, DHS concurred with our fifth recommendation that TSA review the updated Adjudication Center insourcing conversion plan and provide it to TSA and DHS leadership for review and implementation approval. DHS stated that OIA has already begun updating the insourcing conversion plan and intends to provide it for review and approval. We will continue to monitor DHS's efforts. As arranged with your office, unless you publicly announce its contents earlier, we plan on no further distribution of this report until 30 days after its issue date. At that time, we will send copies of this report to the Secretary of Homeland Security, the Assistant Secretary for the Transportation Security Administration, and appropriate congressional committees. In addition, this report is available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-7141 or groverj@gao.gov. Contact points for our Offices: of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are acknowledged in appendix III. Sincerely yours, Signed by: Jennifer A. Grover: Acting Director, Homeland Security and Justice Issues: [End of section] Appendix I: TSA Organizational Chart with Key Offices with Responsibility for Implementing Security Threat Assessments: Figure 3 shows that responsibility for carrying out programs related to issuing credentials for transportation workers is divided among multiple Transportation Security Administration (TSA) offices. In particular, the TSA Office of Intelligence and Analysis manages transportation security credentialing programs--including the three largest programs: the Transportation Worker Identification Credential (TWIC) program for maritime workers; the Hazardous Materials Endorsement (HME) program for truckers seeking a commercial driver's license endorsement to carry hazardous materials; and the Aviation Worker program. Within the Office of Law Enforcement/Federal Air Marshal Service, the Adjudication Center is responsible for providing security threat assessment adjudication services to meet the workload needs of TSA programs. Figure 3: TSA Organization Chart Showing Key Offices Responsible for Managing and Implementing Transportation Security Threat Assessment Programs, as of May 2013. [Refer to PDF for image: organization chart] TSA: Chief of Staff; Policy Coordination and Intergovernmental Affairs. Second level, reporting to Chief of Staff: Finance and Administration; Acquisition; Information Technology; Human Capital; Civil Rights and Liberties, Ombudsman and Traveler Engagement; Professional Responsibility; Inspection; Legislative Affairs; Public Affairs; Chief Counsel. Third level, reporting to Chief of Staff: Intelligence and Analysis: Program Management Division: * Transportation Worker Identification Credential (TWIC); * Hazardous Materials Endorsement (HME); * Aviation Workers Program; Security Capabilities; Risked-based Security; Security Policy and Industry Engagement; Training and Workforce Engagement; Security Operations; Law Enforcement/Federal Air Marshal Service: * Adjudication Center; Global Strategies. Source: GAO Analysis of TSA Information. [End of figure] [End of section] Appendix II: Comments from the Department of Homeland Security: U.S. Department of Homeland Security: Washington, DC 20528: July 2, 2013: Jennifer A. Grover: Acting Director, Homeland Security and Justice Issues: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Re: Draft Report GAO-13-629, "Transportation Security: Action Needed to Strengthen TSA's Security Threat Assessment Process" Dear Ms. Grover: Thank you for the opportunity to review and comment on this draft report. The U.S. Department of Homeland Security (DHS) appreciates the U.S. Government Accountability Office's (GAO's) work in planning and conducting its review and issuing this report. As highlighted in the report, the Transportation Security Administration (TSA) has a challenging role balancing the legitimate flow of people and goods that are critical to the U.S. economy and international commerce, while simultaneously providing effective security that addresses potential threats. TSA implements programs to ensure individuals with unescorted access to secure areas of the Nation's critical transportation infrastructure do not pose a security threat. Key to these programs are Security Threat Assessments (STAs) that screen individuals for links to terrorism, criminal history, and immigration status. TSA's Adjudication Center serves as the primary operational component in this process and continues to provide effective and efficient adjudication support to applicants and employees involved in vital transportation-related industries. In accordance with the USA PATRIOT Act,[Footnote 1] the Aviation and Transportation Security Act,[Footnote 2] and the Maritime Transportation Security Act,[Footnote 3] the Adjudication Center conducts STAs to determine employment applicant eligibility for: Transportation Worker Identification Credentials, Hazardous Material Endorsements to a Commercial Drivers License, Aviation Workers credentials, and Federal Aviation Administration Airman Certificates. TSA works closely with national intelligence entities, as well as other federal, state and local law enforcement agencies. This collaboration ensures terrorist-related investigations are conducted using all available information and that relevant information is shared with the appropriate agencies for action. Adjudicators identify and prevent individuals who pose a threat to transportation and national security from receiving or maintaining transportation credentials and privileges. TSA conducts STAs by reviewing applicant background information to determine eligibility for credentials, accesses, and privileges based on the criteria established in legislation, regulation, and applicable security directives. Applicant cases are screened and adjudicated for eligibility based upon established thresholds that include criminal offenses, immigration or citizenship issues, links to terrorism, and/or mental health history. Adjudication screening includes reviews of Federal Bureau of Investigation criminal history records checks, state or national immigration or criminal wants and warrants, counterterrorism intelligence records, and immigration eligibility and citizenship status. Currently, TSA successfully adjudicates between 1,500 and 2,000 STAs daily. To date, TSA has adjudicated approximately 4.7 million STAs. Although the draft report mentions that TSA is not meeting its standard of keeping program backlogs below 1,500 cases on a consistent basis, it is important to note that the 1,500 case-standard is a self- imposed TSA performance target established to provide the best possible customer service to applicants. TSA continues to work to reduce any backlog; however, TSA believes what is more important than the current number of open cases is instead how quickly it is responding to applicants. In this area, TSA is consistently delivering adjudication response times far better than the congressional requirement, under 46 United States Code 70105(p), to respond to applicants " ... within 30 days after receipt of the initial application." In fact, TSA responds to over 96 percent of initial applications within 3 to 7 days. Additionally, when cases do exceed this timeframe, it is almost exclusively due to potential security issues and/or derogatory findings requiring additional investigation prior to a decision to deny. The draft report contained five recommendations, with which the Department concurs. Specifically, GAO recommended that the Secretary of Homeland Security direct the Administrator of TSA to: Recommendation 1: Develop an accuracy rate measure that includes accuracy data for cases where adjudicators both approved and disqualified applicants, document this methodology, and implement the process. Response: Concur. TSA's Office of Intelligence & Analysis (orA) will modify its current quality control (QC) process to include both approved and disqualified cases and will put in place a QC program that accurately reflects the quality of adjudications being performed. TSA will develop, document, and formalize an accuracy rate measure that includes reviews of both approved and disqualified cases. Estimated Completion Date (ECD): August 30,2013. Recommendation 2: Document the Adjudication Center's case reporting performance management processes. Response: Concur. TSA's OIA will document the manual process for reporting on the Adjudication Center's performance measures to confirm the Center's performance is accurately tracked and to ensure continuity in the event of personnel turnover. TSA anticipates the manual process will be phased out as the Transportation Infrastructure Modernization program comes online during CY 2014-CY 2016, enabling the Adjudication Center to use an automated case management system. ECD: August 30, 2013. Recommendation 3: Establish a mechanism for TSA's Office of Intelligence & Analysis Program Management Division and Office of Law Enforcement/Federal Air Marshal Service Adjudication Center to share and reconcile information included in the Adjudication Center's staffing plan updates, such as timelines for anticipated workload growth. Response: Concur. TSA's OIA is already actively working to resolve the issues identified. The Adjudication Center has begun coordination with the Program Management Division to ensure security threat assessment workload estimates and the Adjudication Center's Staffing Plan are updated. TSA will formalize a quarterly process between the Program Management Division and the Adjudication Center to meet and review staffing plans, program workload projections, spending plans, and other areas. ECD: August 30,2013. Recommendation 4: Update and document the Adjudication Center's insourcing conversion plan to reflect revised schedule timeframes, and cost and hiring level information. Response: Concur. TSA's OIA is actively working with the DHS Office of the Chief Human Capital Officer to address any potential issues posed by using a mix of government employees and contractors to adjudicate security threat assessments. TSA will update its insourcing conversion plan to reflect current timelines, costs, and hiring levels. ECD: September 30, 2013. Recommendation 5: Review the updated Adjudication Center insourcing conversion plan, and provide it to TSA and DHS leadership for review and implementation approval. Response: Concur. TSA's OIA has already begun updating the Adjudication Center's insourcing conversion plan and will provide it to TSA and DHS leadership for review and approval. ECD: October 30,2013. Again, thank you for the opportunity to review and comment on this draft report. Technical comments were previously provided under separate cover. Please feel free to contact me if you have any questions. We look forward to working with you in the future. Sincerely, Signed by: Jim H. Crumpacker: Director: Departmental GAO-OIG Liaison Office: Appendix II Footnotes: [1] The Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism Act, Pub. L. 107-56. [2] Pub. L. 107-71. [3] Pub. L. 107-295. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Jennifer A. Grover, (202) 512-7141 or groverj@gao.gov: Staff Acknowledgments: In addition to the contact names above, David Bruno (Assistant Director), Jason Berman (Analyst-in-Charge), Carl Barden, Melissa Bogar, Jennifer Dougherty, Eric Hauswirth, Richard Hung, Thomas Lombardi, Stephen M. Lord, Steve Morris, Jessica Orr, Minette Richardson, Katherine Trimble, and Jonathan Tumin made key contributions to this report. [End of section] Footnotes: [1] Under Coast Guard maritime security regulations, a secure area, in general, is an area on board a vessel or at a facility over which the owner/operator has implemented security measures for access control in accordance with a Coast Guard-approved security plan. See 33 C.F.R. § 101.105. For most maritime facilities, the secure area is generally any place inside the outer-most access control point. For a vessel or outer continental shelf facility, such as off-shore petroleum or gas production facilities, the secure area is generally the whole vessel or facility. Under TSA aviation security regulations, the secured area of an airport includes, generally, those portions of an airport, specified in the airport security program, in which TSA-mandated security measures are conducted and within which air carriers operating under TSA-approved security programs enplane and deplane passengers and sort and load baggage. See 49 C.F.R §1540.5. [2] TSA regulations define the term security threat to mean an individual whom TSA determines or suspects of posing a threat to national security; to transportation security; or of terrorism. See 49 C.F.R. § 1570.3. Depending on the program, TSA may not be responsible for issuing the credentials to applicants for whom it has completed security threat assessments. [3] TSA officials reported that the TWIC, HME, and Aviation Worker programs accounted for approximately 95 percent of security threat assessment cases processed by the Adjudication Center. The remaining 5 percent of cases are from a number of smaller programs, including general and commercial aviation and air cargo. According to TSA data, in calendar year 2012, the Adjudication Center processed about 404,000 new TWIC, HME, and Aviation Worker security threat assessment applications. [4] We discuss the recommendation we made to TSA later in this report. See: GAO, Transportation Security: Actions Needed to Address Limitations in TSA's Transportation Worker Security Threat Assessments and Growing Workload, [hyperlink, http://www.gao.gov/products/GAO-12-60] (Washington, DC: Dec. 8, 2011). [5] The BWS is the process used by DHS to manage its workforce and achieve the proper mix of federal and contractor skills to meet mission needs. DHS developed its BWS pursuant to, among other authorities, the Omnibus Appropriations Act, 2009, which required federal agencies to "devise and implement guidelines and procedures to ensure that consideration is given to using, on a federal basis, federal employees to perform new functions and functions that are performed by contractors and could be performed by federal employees." See Pub. L. No. 111-8, Div. D, § 736, 123 Stat. 524, 689-91 (2009). See also Pub. L. No. 111-117, Div. C, § 743, 123 Stat. 3034, 3216-29 (2009) (requiring executive agencies to prepare an annual inventory of their service contracts and to analyze the inventory to, among other things, identify contracts that should be considered for conversion to performance by federal employees of the agency). [6] An inherently governmental function is, as a matter of policy, a function that is so intimately related to the public interest as to require performance by government employees and includes activities that require either the exercise of discretion in applying government authority, or value judgments in making decisions for the government. See 48 C.F.R. § 2.101. The Federal Acquisition Regulation lists examples of functions considered to be inherently governmental or that should be treated as such. See 48 C.F.R. § 7.503(c). In addition, closely associated with inherently governmental functions are those that while not inherently governmental, may approach the category because of the nature of the function, the manner in which the contractor performs the contract, or the manner in which the government administers contractor performance. See 48 C.F.R. § 7.503(d). [7] Office of Management and Budget, Office of Federal Procurement Policy, Policy Letter 11-01, Performance of Inherently Governmental and Critical Functions, 76 Fed. Reg. 56,227 (Sept. 12, 2011). [8] See Pub. L. No. 107-56, § 1012, 115 Stat. 272, 396-98 (2001) (codified as amended at 49 U.S.C. § 5103a). [9] See Pub. L. No. 107-71, § 138, 115 Stat. 597, 639-40 (2001) (amending 49 U.S.C. § 44936). [10] See Pub. L. No 107-295, § 102(a), 116 Stat. 2064, 2073-74 (2002) (codified as amended at 46 U.S.C. § 70105). [11] TSA implemented its most recent adjudication center workforce contract in 2010; however, officials reported that technical challenges in establishing its performance measurement system had limited their ability to track data until February 2011. [12] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [13] TSA Designated Aviation Channelers transfer airport worker biographical and fingerprint information to TSA. [14] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [15] See, e.g., 31 U.S.C. § 1115(b)(7) (requiring that agency performance plans provide a basis for comparing actual program results with the established performance goals). [16] According to TSA officials, this number refers to the total number of names TSA reported vetting through its security threat assessment programs since 2003. While these programs include varying purposes, standards, or agency responsibilities, they include some or all of the following components: enrollment, security threat assessments, credential issuance, verification, renewal, and revocation. [17] For example, the TSA OIA manages the three largest programs: the TWIC program for maritime workers; the HME program for truckers seeking a commercial driver's license endorsement to carry hazardous materials; and the Aviation Workers program for airport workers seeking unescorted access into airport secured areas. The Adjudication Center within OLE/FAMS is responsible for security threat assessment adjudication services to meet the workload needs of TSA credentialing programs. [18] TSA's Colorado Springs Operations Center within OIA is responsible for conducting the terrorism related checks in the security threat assessment process. The Selectee List contains information on individuals who must undergo additional security screening before being permitted to board an aircraft. The No Fly List contains information on individuals who are prohibited from boarding an aircraft. [19] Criminal history record checks, which are fingerprint-based, require an adjudicator to review the applicant's criminal history for a designated time frame and compare this information against a set of disqualifying criminal offenses identified in statute and corresponding regulations in order to make a determination of eligibility. See, e.g., 49 U.S.C. § 44936 (listing criminal offenses that, if committed at least within the most recent 10-year period, would disqualify an applicant from obtaining, for example, unescorted access to the secure area of an airport); see also 49 C.F.R. §§ 1542.209, 1544.229. [20] According to TSA data about 42 percent of applicants are identified through automated vetting as having some derogatory information therefore requiring Adjudication Center review. [21] The Adjudication Center is located in Herndon, Virginia at a contractor-leased facility. TSA's most recent contract for operating the Adjudication Center began in February 2010. TSA estimates the Adjudication Center operating costs for the first 3 years of the current contract (February 2010 to February 2013) at $20.2 million and the costs for the contract's fourth year (February 2013 to February 2014) at $7.2 million. [22] If TSA determines an applicant meets one or more disqualifying criteria, the agency issues a Preliminary Determination of Ineligibility letter (formerly referred to as an Initial Determination of Threat Assessment) to the applicant. An applicant who receives such a letter may take steps to appeal the determination as instructed by TSA and consistent with applicable regulations--known as the redress process. See, e.g., 49 C.F.R. §§1572.15(d), 1572.21(d) (addressing final dispositions and opportunity to appeal for the HME and TWIC programs, respectively); see also §§ 1515.5, 1515.9 (addressing appeals of determinations based on criminal convictions, immigration status, mental capacity, or other analyses). In some instances, an applicant disqualified based on criminal offenses, immigration status, or mental capacity standards may seek a waiver of this determination provided the applicant is not determined to oppose a security threat. See 49 C.F.R. § 1515.7. [23] See, for example, GAO, Defense Acquisitions: Further Actions Needed to Address Weaknesses in DOD's Management of Professional and Management Support Contracts, [hyperlink, http://www.gao.gov/products/GAO-10-39] (Washington, D.C. Nov. 20, 2009) and Department of Homeland Security: Improved Assessment and Oversight Needed to Manage Risk of Contracting for Selected Services, [hyperlink, http://www.gao.gov/products/GAO-07-990] (Washington, DC.: Sept. 17, 2007). [24] Office of Management and Budget, Office of Federal Procurement Policy, Policy Letter 11-01, Performance of Inherently Governmental and Critical Functions, 76 Fed. Reg. 56,227 (Sept. 12, 2011). [25] For example, the Federal Acquisition Regulation identifies services that can approach being inherently governmental, based on the nature of the function or the manner in which the work is performed or administered. See 48 C.F.R. § 7.503(d). Without proper management and oversight of contractors' work, such services may unduly influence a federal agency's control over and accountability for decisions. [26] [hyperlink, http://www.gao.gov/products/GAO-07-990]. [27] See Pub. L. No. 111-8, Div. D, § 736, 123 Stat. 524, 689-91 (2009); see also Pub. L. No. 111-117, Div. C, § 743, 123 Stat. 3034, 3216-29 (2009) (requiring executive agencies to prepare an annual inventory of their service contracts and to analyze the inventory to, among other things, identify contracts that should be considered for conversion to performance by federal employees of the agency. According to the DHS FY 2011 Service Contract Inventory Analysis report, DHS components have used the BWS process to review a total of 248 contracts through the end of fiscal year 2011. According to this report, the total amount obligated to these 248 contracts is approximately $800,115,993. DHS reported that it had not yet completed its report of the FY 2012 Service Contract Inventory, as of March 2012. [28] According to DHS Balanced Workforce Strategy guidance, the purpose of the BWS is to (1) ensure compliance with applicable statutes, regulations, and policies, through a repeatable documented decision-making process; (2) determine the proper workforce balance for each component activity; and (3) reduce mission risk while, as practicable, reducing or controlling costs. [29] A sourcing decision is the determination of the appropriate workforce balance to perform a function. The sourcing decision may include a recommendation to either insource or outsource a government function. Insourcing is a decision and related functions to use federal employees to perform activities that are presently performed by contractors. Outsourcing is the use of contractors to perform new, unrestricted activities or to perform unrestricted activities currently performed by federal employees. [30] [hyperlink, http://www.gao.gov/products/GAO-12-60]. [31] Based in part on these data, TSA evaluated and awarded performance based fees. The award fees are based on the contractor's performance in meeting its caseload, timeliness, and accuracy rates. As of August 2012, TSA had awarded performance based fees to the Adjudication Center contractor totaling roughly $312,000 for work performed since October 2010. According to Adjudication Center officials, contractor employees were not formally evaluated on performance standards for the first six months of the contract, allowing them time to become familiar with Adjudication Center operations. When the Adjudication Center started to formally evaluate contractor performance after the 6 month period, technical limitations delayed analysis of some of the metrics. [32] The Adjudication Center contract is performance-based. Based on the evaluations, the Adjudication Center provides its contractor with a varying award fee based on the extent to which the contractor meets performance measures, known as Acceptable Quality Levels. These Acceptable Quality Levels are documents in TSA's contract for the Adjudication Center. [33] According to TSA documentation, the Screening Gateway system provides an on-demand turnaround time report that details the number and timing of cases that are initially adjudicated. The turnaround time report counts the number of days it takes for cases to make their first move from when the case was received for adjudication to the time when the initial adjudication was made for either approving or disqualifying an applicant. [34] Adjudication Center management officials told us that they track adjudicator performance for Aviation Worker cases differently than for the TWIC and HME programs. According to these officials, Aviation Worker cases are simpler and to adjudicate because the Adjudication Center is responsible for adjudicating the immigration status of an applicant, but is not responsible for adjudicating criminal history records. This is because, unlike with the TWIC and HME programs, local authorities (airport operators and aircraft operators) are responsible for adjudicating the results of TSA-provided FBI criminal history records, to determine whether applicants have potentially disqualifying records. [35] See 46 U.S.C. § 70105(p) (providing further that, to the greatest extent practicable, a written decision or request for additional information shall be provided to an applicant in response to an applicant's written request for an appeal or waiver within 30 days of receiving the request, and that for applicants required to submit additional information, a written decision shall, to the greatest extent practicable, be sent to the applicant within 30 days after receipt of all requested information). See also, e.g., 72 Fed. Reg. 3,492, 3,556 (Jan 25, 2007) (indicating that, as a general rule, security threat assessments and issuance of a TWIC should take no longer than 30 days). [36] Among other things, the TIM program seeks to consolidate and standardize the processes and systems for TSA screening and credentialing programs and populations, including for the TWIC, HME, and Aviation Workers programs. According to TSA, the TIM design and development contract was awarded in August 2012 and is budgeted for $48.7 million in fiscal year 2014. TSA estimates the program to have a lifecycle cost of $631.2 million through 2025 to be funded through fees and appropriations. GAO has previously reported on the TIM program. For more information on the TIM program, see GAO, Transportation Security Infrastructure Modernization May Enhance DHS Screening Capabilities, but It Is Too Early to Assess Results, [hyperlink, http://www.gao.gov/products/GAO-12-192R] (Washington, D.C. Dec. 8, 2011). [37] TSA officials reported that the TIM program is in the development process with initial operational capability planned for March 2014 and full operational capability in 2016. According to officials, after March 2014, the TWIC program will be migrated to the TIM platform and Adjudication Center officials will review cases via the TIM system. Officials reported that the surface and aviation programs are scheduled for migration to TIM in 2015 and 2016, respectively. The Screening Gateway systems are scheduled to be replaced in 2016. [38] TSA data for the period of October 2010 through January 2013 includes 122 weeks of evaluation data. However, for the Aviation Worker program, our analysis covered the time period of October 2010 through March 2012--78 of 122 weeks. TSA officials reported TSA has not had have reliable Aviation Worker caseload data since April 2012. TSA reported that these data were unreliable because of technical problems TSA has experienced with its Designated Aviation Channeler program for processing Aviation Worker program security threat assessment applications submitted by airport authorities. TSA has reported experiencing technical problems with one of its three approved Designated Aviation Channeler vendors since April 2012. TSA Office of Intelligence and Analysis officials reported they were working with the vendor to resolve the causes for its data reliability limitations and were in the process of updating policies and procedures to address the issues. [39] The Adjudication Center program manager also reported on other factors that had contributed to workload backlogs. The official told us that enrollment for the TWIC, HME, and Aviation Worker programs experienced peaks and lulls at different times of the year, depending on the program, and that this had accounted for some of the larger spikes in its backlogs over the past few years. For example, TWICs, in general, are to expire 5 years after the date of issuance. See 49 C.F.R. § 1572.23. With respect to HMEs, each state must require that hazardous material endorsements be renewed every 5 years or less so that individuals are subject to a TSA security threat assessment at least every 5 years. See, e.g., 72 Fed. Reg. 3,492, 3,495 (Jan. 25, 2007) (providing that a security threat assessment is valid for 5 years). [40] For example, according to a TSA Adjudication Center contractor evaluation and performance report we reviewed, contractor performance in meeting this measure was unsatisfactory because the contractor did not have a sufficient number of trained initial adjudicators to meet case load and overall production requirements. [41] According to Adjudication Center Standard Operating Procedures, initial adjudicators are required to meet acceptable case production and accuracy rates before they can be nominated by the contract program manager for self-approval rights for those cases that are approved (no determination of threat). [42] According to TSA, adjudication accuracy is measured weekly by compiling daily contractor and government mid-level adjudicator production reports to calculate the total number of (1) cases reviewed; (2) cases with incorrect errors; and (3) cases with incomplete errors. Incorrect errors are either those cases approved when the applicant should have been disqualified or cases where an adjudicator found an applicant was disqualified when the applicant had no disqualifying factors. Incomplete errors refer to leaving any element of the case not adjudicated that should have been part of the initial adjudication, such as administrative errors. [43] According to our analysis of TSA data, the Adjudication Center had accuracy rates of approximately 99 percent for HME and TWIC cases. TSA reported a 99 percent accuracy rate for Aviation Worker cases. For Aviation Worker cases, the Adjudication Center reviews only immigration history, and transmits the results of this review to airports which are responsible for adjudicating criminal history for aviation workers. [44] Adjudication Center program managers reported that the accuracy rate includes errors found occasionally in cases when reviewers are reconciling administrative errors in the Screening Gateway systems. If errors are found, officials said these errors are calculated into the average accuracy rate. [45] According to Adjudication Center officials, all cases in which the contract adjudicator workforce determines an applicant had committed disqualifying offenses within applicable time frames (i.e. applicant would not pass a security threat assessment) are verified by TSA employees. However, contract adjudicators that determine an applicant had not committed a disqualifying offense within applicable time frames may approve that case without this second level of review if the contract adjudicator has been certified by Adjudication Center management as a "self-approver" and is no longer considered a trainee. Those cases approved by adjudicators certified as self approvers are subject to the Adjudication Center's quality assurance check. According to Adjudication Center officials, cases with errors found in these two processes are corrected before the Adjudication Center informs the applicant of the results of the security threat assessment. [46] According to Adjudication Center officials, the quality assurance checks are a way to ensure that cases are being adjudicated properly and that the Adjudication Center does not incorrectly approve applicants with disqualifying offenses, while also identifying and addressing performance issues among its contract adjudicator workforce. [47] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [48] According to Adjudication Center officials, the cases requiring Adjudication Center review are forwarded, through the automated Screening Gateway process, to the Adjudication Center. The Operations Manager then assigns cases to individual contractor adjudicators. [49] For example, according to officials, a single case can change hands 50 times among Adjudication Center staff during the adjudication process and therefore the Gateways record different start and stop dates for each time and record these as separate cases. Therefore, the Gateways cannot track a single case as it progresses through the adjudication process, and that is why data such as initial adjudication duration have to be tracked manually. [50] According to Adjudication Center officials, efforts to incorporate these functional requirements into the system were unsuccessful. TSA officials reported that while they had explored several options, including the use of off the shelf technologies, the system TSA currently had in place was not compatible, and that these technical limitations and the cost considerations for using other technical solutions were contributing factors. The key obstacle, according to Office of Technology officials, was that the Screening Gateway case management system was designed without the capability for absorbing the Adjudication Center's case management requirements. [51] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [52] According to a senior Adjudication Center official, those invited to attend include TSA offices from the Office of Acquisition, OLE/FAMS, the Office of Human Capital, and credentialing program offices. [53] Adjudication Center management officials develop and periodically update a staffing plan that identifies current and future year caseload projections and estimated workforce needs, as well as performance challenges. [54] TSA officials reported that the agency is engaged in a rulemaking effort under which, among other things, TSA would assume responsibility for the adjudication of criminal background checks for airport and aircraft operator workers, for which airport and air carrier operators are currently responsible. TSA officials reported that this additional responsibility would account for a considerable increase in the workload of the Adjudication Center. [55] GAO, Results-Oriented Government: Practices That Can Help Enhance and Sustain Collaboration among Federal Agencies, [hyperlink, http://www.gao.gov/products/GAO-06-15] (Washington, DC: Oct. 21, 2005). [56] See, e.g., 31 U.S.C. § 1115(b)(7). [57] According to DHS BWS guidance, a critical function is a function that is necessary to the department being able to effectively perform and maintain control of its mission and operations. [58] Out of a scale of 1 to 10, the assessment rated the need for continuous and substantive oversight of the contractors as a 10, the highest level. [59] The assessment found that the department did not have the resources or capability to give heightened management attention to the contractor performance to ensure such performance does not expand to include inherently governmental functions, limit or guide the contractor's exercise of discretion, and ensure reasonable identification of contractor products. [60] The assessment also noted that there was a duplication of government and contract management staff, and that some of the redundancies could be eliminated with a single, federal government workforce. [61] Officials from several TSA offices are responsible for conducting, reviewing and approving the BWS implementation plan. These components include: the Office of Acquisition, which serves as the lead for the effort, as well as the Office of Human Capital, Office of Finance and Administration, and from OLE/FAMS, the Business Management Office and Adjudication Center. [62] Since December 2011, TSA has implemented two reorganizations affecting security threat assessment program offices. First, in December 2011, TSA transferred the Adjudication Center from the Transportation Threat Assessment and Credentialing Office to OLE/FAMS. Second, in January 2013, TSA realigned its transportation industry vetting program management functions, responsibilities, positions, and funding from the Office of Security Policy and Industry Engagement to OIA. [63] Adjudication operations are funded through numerous sources, including TWIC and HME fee collections and amounts appropriated for aviation security. [64] The DHS Modular Cost Standards were developed to enable program managers to calculate the true cost of a new Federal position. These standards can be used to identify salaries and expenses as well as one- time and recurring costs associated with establishing new positions. [65] The contract includes a 1-year base period with four, 1-year options. According to a TSA official, at the conclusion of this contract TSA would have to recomplete the contract in accordance with federal acquisition guidelines if it were to continue with a contract workforce. [66] According to TSA officials, once the plan is approved by TSA and DHS, implementing the workforce conversion--from posting the federal positions to completing hiring of new staff--would take approximately 18 months. Training the new workforce may take up to 12 months according to the Adjudication Center staffing plan. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]