This is the accessible text file for GAO report number GAO-07-445 entitled 'Aviation Security: Cost Estimates Related to TSA Funding of Checked Baggage Screening Systems at Los Angeles and Ontario Airports' which was released on April 2, 2007. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: United States Government Accountability Office: GAO: March 2007: Aviation Security: Cost Estimates Related to TSA Funding of Checked Baggage Screening Systems at Los Angeles and Ontario Airports: GAO-07-445: Contents: Letter: Summary of Findings: Scope and Methodology: Agency Comments and Our Evaluation: Appendix I: GAO Contact and Staff Acknowledgments: Appendix II: Briefing Slides: Appendix III: Comments from Los Angeles World Airports: Abbreviations: DHS: Department of Homeland Security: EDS: explosive detection system: GAO: Government Accountability Office: LAWA: Los Angeles World Airports: LAX: Los Angeles International Airport: LOI: letter of intent: LOI/MOA: letter of intent/memorandum of agreement: MOA: memorandum of agreement: ONT: Ontario International Airport: TSA: Transportation Security Administration: United States Government Accountability Office: Washington, DC 20548: March 30, 2007: The Honorable Robert C. Byrd: Chairman: The Honorable Thad Cochran: Ranking Minority Member: Subcommittee on Homeland Security: Committee on Appropriations: United States Senate: The Honorable David E. Price: Chairman: The Honorable Harold Rogers: Ranking Minority Member: Subcommittee on Homeland Security: Committee on Appropriations: House of Representatives: To meet the mandate to screen all checked baggage for explosives by December 31, 2003, the Transportation Security Administration (TSA) placed minivan-sized explosive detection systems (EDS) and other screening equipment in airport lobbies.[Footnote 1] However, these interim lobby solutions have caused operational inefficiencies, in part because they require a large number of screeners. According to TSA, in- line baggage screening--where EDS machines are integrated with an airport's baggage conveyor system--can be a more cost-effective and efficient alternative to lobby-based, stand-alone equipment. For example, in-line systems can increase the efficiency of airport, airline, and TSA operations, and lower costs by reducing the number of screeners. Moreover, in-line explosive detection systems can enhance security because they reduce congestion in airport lobbies, thus removing a potential target for terrorists. However, installing in-line systems can have large up-front costs, related to the need for airport modifications. To help defray these costs, in 2003, Congress authorized TSA to reimburse airports up to 75 percent of the cost to install these systems by entering "letter of intent" (LOI) agreements.[Footnote 2] An LOI, though not a binding commitment of federal funding, represents TSA's intent to provide the agreed-upon funds in future years if the agency receives sufficient appropriations to cover the agreement. TSA has issued eight letters of intent to help defray the costs of installing in-line systems at nine airports as of February 2007, but none since February 2004. In September 2003, TSA and the City of Los Angeles signed an LOI and an attached memorandum of agreement (LOI/MOA) in which TSA agreed to pay an amount not to exceed 75 percent of the agreed upon estimated total project cost of $341 million (about $256 million) to install in-line checked baggage screening systems at both Los Angeles (LAX) and Ontario (ONT) International Airports. However, in December 2003, officials from the City of Los Angeles' airport authority--Los Angeles World Airports (LAWA)--informed TSA that aspects of the design concept were infeasible and that additional construction modifications would be needed. LAWA subsequently submitted a revised cost estimate to TSA in April 2005 and requested that TSA amend the LOI/MOA to increase the federal reimbursement by about $122 million. TSA has not amended the LOI to provide for additional reimbursements; however, as of February 2007, TSA had obligated the $256 million for the City of Los Angeles LOI/MOA in accordance with the schedule agreed to in the LOI and had reimbursed LAWA for about $26 million in expenses. Senate Report 109-273 directs us to review the reasons for the differences between the original 2003 cost estimate and the revised 2005 cost estimate submitted by LAWA.[Footnote 3] In response and as agreed with committee offices,[Footnote 4] we identified the key factors that contributed to the differences between the two cost estimates. On January 23, 2007, we briefed staff of the Senate Subcommittee on Homeland Security, Committee on Appropriations, on the results of our work (see app. II). Summary of Findings: A key reason for the difference between the 2003 total project cost estimate and the revised 2005 estimate to install in-line baggage screening systems at LAX and ONT was that the 2003 estimate was developed at an early stage in the design process and was therefore based on preliminary data and assumptions that were subject to change. Consequently, the estimate did not adequately foresee some of the costs of retrofitting new systems into existing buildings or allow for sufficient space for the EDS machines, baggage inspection rooms, and conveyor belts. LAWA officials stated that they were under a tight timeframe to apply for the LOI because TSA had told them that federal funding was limited and that 17 other airports were competing for the funding. The 2003 total project cost estimate used concepts and construction estimates developed in about 12 weeks by Boeing, TSA's contractor. LAWA relied on designs and estimates developed by Boeing and its subcontractors to determine the total project cost estimate because the company had expertise in integrating EDS equipment into airports. According to TSA and LAWA officials, both TSA and Los Angeles signed the LOI/MOA knowing the preliminary nature of the cost estimate. According to construction industry guidance, an estimate's accuracy depends on the quality of information known about the project at the time the estimate is prepared.[Footnote 5] The 2003 estimate was made at the "concept development" stage where the final project cost can be expected to range from 50 percent under to 100 percent over the estimated cost, according to this guidance. The 2005 revised estimate was made at the "design development" stage where the range of the final project cost estimate can be expected to be more accurate--from 20 percent under to 30 percent over the estimated cost. In December 2003, LAWA presented TSA with a summary of inadequacies it had found in the original Boeing concept and the associated potential cost and scheduling impacts. LAWA then began an engineering study to update the in-line system concepts at LAX and ONT, the results of which it presented to TSA in September 2004. TSA reviewed these updated concepts and determined that they would meet its performance requirements; however, TSA's review did not address cost issues. LAWA used these updated concepts to develop its 2005 estimate, which was based on more definitive information about terminal design requirements than the 2003 estimate. According to LAWA, new construction and excavation included in the 2005 designs increased the estimated costs. Among the design changes, LAWA determined that the placement of EDS machines in the 2003 concepts was infeasible in five of nine of the LAX terminals and both ONT terminals. In addition, the 2005 estimate included 20 additional baggage inspection rooms, 9 rooms for on-screen resolution of EDS alarms, and 10 computer rooms at LAX and ONT terminals. The 2005 estimate also included over $11 million in computer networking costs and costs associated with on-screen resolution of EDS alarms, which the 2003 estimate did not foresee. TSA also highlighted two additional factors that caused differences between the two estimates--cost increases due to the delay in beginning construction of the project and the escalation of construction costs between 2003 and 2005. LAWA also determined that TSA's contractor and subcontractor made a mathematical error in the 2003 concept development estimate: construction costs were only included for one of the two baggage screening facilities and neither of the connected tunnels at ONT. TSA officials told us in January 2007 they were not able to substantiate this error. Further, according to LAWA, system redesigns were required because TSA's guidance on in-line baggage screening systems changed between the 2003 and 2005 estimates, leading to higher estimates. Because few in- line systems were in use at the time of the September 2003 LOI/MOA, only limited information on the capabilities of the in-line EDS machines, including actual bags screened per hour and false alarm rates, was available for modeling the systems. In June 2006, TSA produced the Recommended Security Guidelines for Airport Planning, Design and Construction to guide future construction of in-line checked baggage screening systems based on its past experiences. TSA also expects to release more detailed guidelines for in-line system planning and design in a few months. The LOI/MOA affords TSA flexibility to amend the agreement to account for changed circumstances. However, under the terms agreed to in the LOI/MOA, TSA has no obligation to amend the LOI/MOA or to reimburse the City of Los Angeles for any additional costs beyond those agreed to in the LOI/MOA, and TSA officials have stated that the agency does not have plans for such reimbursement. Scope and Methodology: To review key factors that contributed to the differences between the 2003 and 2005 cost estimates, we reviewed TSA and LAWA documents used in developing the cost estimates, including design plans, reports, briefings, and emails. We interviewed officials from TSA and LAWA, as well as TSA contractors and other relevant officials who participated in the cost-estimation process to learn about the factors that contributed to the increased estimate of the cost of in-line checked baggage screening systems at LAX and ONT. We visited LAX and ONT to obtain a first-hand perspective of the modifications needed to install the in-line EDS systems at both airports. Additionally, we examined industry guidance on estimating costs for construction projects. We did not independently verify the 2003 or 2005 cost estimates. We conducted our work in accordance with generally accepted government auditing standards from October 2006 through March 2007. Agency Comments and Our Evaluation: We provided a draft of this report to LAWA and the Department of Homeland Security (DHS) for review and comment. LAWA provided written comments which we have included in their entirety in appendix III. DHS provided no written comments. TSA provided e-mail comments. In addition, LAWA and TSA provided technical comments concerning facts in the report which we incorporated as appropriate. In its March 8, 2007, comments, LAWA wrote that it believes the draft did not paint an accurate or complete picture of the facts. In general, LAWA raised three points: (1) TSA has the authority to revise the LOI to reflect accurate cost figures and explicitly anticipated doing so during the LOI/MOA development process, (2) the report fails to assign specific responsibility for initial designs and any errors, as directed by the Report of the Senate Appropriations Committee, and (3) the report does not recognize that LAWA responded to TSA urgency in completing the agreements and, as a result, used preliminary design and cost estimates as the basis for entering into the LOI/MOA. We do not agree with LAWA's comments. With respect to its first point, it is true that the LOI/MOA agreements afford TSA flexibility to amend the agreements to account for changed circumstances. As stated in our report, however, under the terms of the LOI/MOA and in accordance with the law, TSA is under no obligation to amend the LOI/MOA or to reimburse LAWA for any costs beyond those agreed upon in the LOI/MOA. To date, as noted in our report, TSA has not indicated any intent to amend the LOI/MOA agreements to provide LAWA with additional funding for this project. LAWA states that it did not believe it would be held financially responsible for increases in eligible and allowable costs due to reasons beyond its control. When subsequent estimates revealed that the project costs would exceed the LOI/MOA-estimated amount, LAWA requested an amendment to the LOI to receive a 75 percent federal reimbursement of the $485 million revised estimate. LAWA also commented that a senior TSA official provided written assurances that the agency "would have the opportunity to cover an increase in costs due to design changes" and referenced an April 2004 e- mail from a TSA official to LAWA in support of this assertion. LAWA noted that it relied on this and other assurances from TSA, "reinforced in various discussions," at the time it concluded the LOI/MOA process. While the April 2004 e-mail cited above did note that LAWA would not "be held to estimates that do not prove to be right on the mark," this particular statement was made at least seven months after TSA and LAWA had concluded the LOI/MOA process and entered into the agreements. Furthermore, the MOA clearly provides that the agreement signed by both parties constitutes the "complete integration of all understandings between the parties." More generally, it provides that any prior, contemporaneous, or subsequent changes, whether written or oral, have no force or effect, and that any changes or modifications to the MOA must be in writing, signed by the TSA Contracting Officer, and duly executed by the City of Los Angeles to have such force or effect. Neither LAWA nor TSA presented any documentation suggesting that steps prescribed in the MOA had been (or were anticipated to be) taken to amend the LOI/MOA with respect to the reimbursable amount. LAWA also stated that the report does not reflect the extensive and protracted discussions LAWA had with TSA, leaving the impression that LAWA simply presented a new set of design concepts to TSA in September 2004. Our objective in this report, as agreed with congressional offices, was to identify the key factors that contributed to the differences between the 2003 and the 2005 cost estimates. As such, we reported that LAWA reported the findings of its engineering study in September 2004 and that TSA approved the concepts. We believe this statement sufficiently demonstrates the agreement between TSA and LAWA on the revised designs. In its second point, LAWA suggests that the GAO report avoids assigning responsibility to TSA or its contractors and, as a result, "failed to answer the Senate Committee's direction to provide a detailed explanation of the reasons for any differences the original estimate, including identification of and the party responsible for any material mistakes, omissions, and infeasible design concepts in the original estimate." The objective of the report, as agreed with the appropriate congressional offices in accordance with our Congressional Protocols, was to identify factors contributing to differences between the estimates. To the extent appropriate, we identified the roles and responsibilities of the various parties. Specifically, we noted that Boeing produced the conceptual designs that served as the basis for the 2003 estimate agreed to in the LOI/MOA. We noted that these designs had been developed at an early stage of the design process, which assumes costs that can differ greatly from final project costs. The report also states that LAWA had determined that Boeing had made a mathematical error in the Ontario estimate. Further, we determined that both TSA and LAWA had signed the LOI/MOA knowing of the preliminary nature of the cost estimate. Finally, as referenced in our scope and methodology and as agreed with the congressional offices, in identifying factors associated with the estimates, we did not independently verify the 2003 or 2005 cost estimates. In its third point, LAWA suggests the report does not recognize that LAWA responded to TSA urgency in completing the agreements and, as a result, used preliminary design and cost estimates as the basis for entering into the LOI/MOA. The report notes that "LAWA officials stated that they were under a tight timeframe because TSA had told them that federal funding was limited and 17 other airports were competing for the funding." We identified this as a factor associated with the preliminary nature of the 2003 estimate. LAWA also commented that it was essentially required to accept the Boeing design and cost estimates. The report states that TSA and LAWA used the Boeing estimate to provide the basis for the estimate agreed to in the September 2003 LOI/MOA. Neither TSA nor LAWA provided evidence suggesting that TSA had required LAWA to accept the Boeing design and cost estimates. In its e-mail comments, TSA stated that concept development for in-line solutions at all of the airports with LOIs was a collaborative effort between TSA and the respective airport entity. TSA further stated that most of the LOI estimates were developed early in the concept development phase. We incorporated a comment into the report to acknowledge that TSA viewed the development of the concepts as a partnership; however, we did not review concept development at other airports. In its comments, TSA also states that there were only two changes in its guidance between 2003 and 2005. The first was the addition of the use of On-Screen Alarm Resolution Protocol during the alarm resolution process, which TSA acknowledged required redesign and associated cost increases. The second was the increase in the baggage throughput number per EDS which led to the deletion of 13 EDS machines from the quantity estimated in the 2003 concepts. According to TSA, this would have a significant impact on lowering the overall project cost, which would be supported by decreasing required space, baggage handling system infrastructure (generally up to $4 million per machine on average) and associated electrical, mechanical, data and other infrastructure. In its comments, TSA described two additional factors that caused differences between the two estimates--cost increases due to the delay in beginning construction of the project and the escalation of construction costs between 2003 and 2005. We incorporated this comment into the report. Finally, TSA stated in its comments that it had reviewed the mathematical error LAWA determined Boeing made in the 2003 concept for ONT, and had not been able to validate that the error had been made. In our report we acknowledge that TSA was not able to substantiate the error. We will send copies of this report to the Secretary of Homeland Security and the Assistant Secretary, Transportation Security Administration, and interested congressional committees. We will send a copy of the report to LAWA and will also make copies available to others upon request. In addition, the report will be available at no charge on GAO's Web site at http://www.gao.gov. If you have any questions or need additional information, please contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are acknowledged in appendix I. Signed by: Robert Goldenkoff: Acting Director: Homeland Security and Justice Issues: [End of section] Appendix I: GAO Contact and Staff Acknowledgments: GAO Contact: Robert Goldenkoff (202) 512-2757: Acknowledgments: In addition to the contact named above, Charles Bausell, Kevin Copping, Kimberly Cutright, Glenn Davis, Terrell Dorn, Maria Edelstein, Richard Hung, Julian King, Brian Lipman, Thomas Lombardi, Amanda Miller, and Linda Miller made key contributions to this report. [End of section] Appendix II: Briefing Slides: Review of Cost Estimates for Installing Baggage Screening Systems at Los Angeles and Ontario Airports: Briefing for Congressional Committees: Introduction: Pursuant to 49 U.S.C. § 44901(d), all checked baggage at U.S. airports was to be screened using explosive detection systems by December 31, 2003. We have reported that, according to the Transportation Security Administration's (TSA) analysis, in-line baggage screening-where explosive detection systems (EDS) are integrated with an airport's baggage conveyor equipment-can be more cost-effective compared to placing EDS machines in airport lobbies.[Footnote 6] However, installing in-line EDS systems can involve large up-front costs. To help defray the cost of installing in-line baggage screening systems, in 2003, Congress authorized TSA to reimburse airports up to 75 percent of the systems' cost by issuing "letters of intent" (LOI).[Footnote 7] An LOI, though not a binding commitment of federal funding, represents TSA's intent to provide funds in future years if they are appropriated by Congress. TSA has signed 8 letter of intent agreements for the installation of in-line checked baggage screening systems at 9 airports (one LOI covered both LAX and ONT). In September 2003, TSA and the City of Los Angeles signed a LOI and an attached memorandum of agreement (MOA) to help fund the installation of in-line checked baggage screening systems at Los Angeles (LAX) and Ontario (ONT) International Airports. Under the LOI/MOA, TSA agreed to pay for 75 percent of the estimated total project cost of $341 million (about $256 million). In April 2005, the City of Los Angeles' airport authority-Los Angeles World Airports (LAWA)-submitted to TSA a revised cost estimate of $485 million and requested that the LOI/MOA be amended to increase the federal reimbursement to about $378 million to reflect increased estimated total project costs. Objectives: Senate Report 109-273 directs GAO to review the differences between the original 2003 cost estimate and LAWA's revised 2005 cost estimates.[Footnote 8] As agreed with committee offices, GAO identified the key factors that contributed to the differences in the two estimates. Scope and Methodology: To address the objective, we: * reviewed key TSA and LAWA documents used in developing the cost estimates, including design plans, reports, briefings, and emails; * interviewed TSA, LAWA, and other officials who participated in the cost-estimation process; * visited LAX and ONT to obtain a first hand perspective of the modifications needed to install the in-line EDS Systems at both airports; and: * reviewed industry guidance on estimating costs for construction projects. We did not independently verify the 2003 or 2005 cost estimates for installing the in-line EDS systems or the reasonableness/adequacy of the designs. We conducted our work in accordance with generally accepted government auditing standards from October 2006 through March 2007. Results in Brief: The rise in the estimate between 2003 and 2005 was primarily related to the fact that the 2003 estimate was developed at an early stage in the design process and was therefore based on preliminary data and assumptions that were subject to change. LAWA officials stated that they were under a tight timeframe to apply for the LOI because TSA had told them that federal funding was limited and that 17 other airports were competing for the funding. The 2003 estimate did not adequately foresee some of the costs of retrofitting new systems into existing buildings. Further, TSA's guidance on in-line baggage screening systems changed between the 2003 and 2005 estimates, which contributed to changes in terminal designs and related cost estimates. According to TSA and LAWA officials, both TSA and Los Angeles signed the LOI/MOA knowing the preliminary nature of the cost estimate. The LOI/MOA affords TSA the flexibility to amend the agreements to account for changed circumstances. However, under the terms agreed to in the LOI/MOA, TSA has no obligation to amend the LOI/MOA or to reimburse Los Angeles for any costs beyond those agreed to in the LOI/ MOA and does not have plans to do so. Background: In-line EDS systems are advantageous because they can: Increase the efficiency airport, airline, and TSA operations. Lower costs by reducing the number of transportation security officers required to screen checked baggage. Move screening equipment out of airport lobbies where it causes overcrowding, creating a potential target for terrorists. [See PDF for image] Source: GAO: In-line EDS systems are integrated with airports' baggage-handling conveyor systems. [End of figure] TSA and LAWA initiated discussions about in-line solutions for LAX and ONT in January 2003 in order to replace stand-alone EDS machines in the airports' lobbies. LAWA used concepts and construction estimates developed by TSA's contractor, Boeing, for the LAX and ONT in-line baggage screening systems because of Boeing's and its subcontractors' expertise in integrating explosive detection systems into airports. TSA reported that concepts for LAX and ONT were developed through a partnership between LAWA and the agency. Boeing's 2003 cost estimate consisted of $260 million for LAX and $24 million for ONT. LAWA and TSA agreed to add another $57 million (or 20 percent) for estimated administrative expenses for a total project cost of $341 million. In December 2003, LAWA presented TSA with a summary of inadequacies it had found in the original concepts, and the associated potential cost and scheduling impacts. In April 2005, LAWA submitted a revised cost estimate to TSA based on TSA approved concepts from a September 2004 LAWA engineering study. This revised cost estimate included a request that TSA amend the LOI/ MOA to increase the federal reimbursement by about $122 million, a sum that would raise the total federal reimbursement to about $378 million.[Footnote 9] As of February 2007, TSA has obligated $256 million in accordance with the schedule set forth in the LOI/MOA for Los Angeles. As of October 2006, TSA had reimbursed LAWA for about $26 million in expenses from the total amount obligated. Development of the 2003 Estimate: LAWA officials stated that they were under a tight timeframe to apply for the LOI because TSA told them that federal funding was limited and that 17 other airports were competing for the funding. TSA's Boeing contracting team spent about 12 weeks from January to April 2003 to develop initial drawings (known as concept development drawings) and construction cost estimates. Between April and September 2003, TSA and LAWA negotiated the addition of administrative and contingency costs to the estimate. TSA's in-line system design guidance was evolving while the estimate was being developed, requiring frequent changes to the conceptual drawings. Accuracy of Cost Estimates Depends on Completeness and Maturity of Information: The accuracy of an estimate depends on the quality of information known about the project at the time the estimate is being prepared. The 2003 estimate was made at the "concept development" stage where the final project cost can be expected to range from 50 percent under to 100 percent over the estimated cost based on construction industry guidance.[Footnote 10] The 2005 estimate was made at the "design development" stage where the final project cost can be expected to range from 20 percent under to 30 percent over the estimated cost. (See fig. 1.) Figure 1: 2003 Estimate Was Conducted at an Early Stage of the Design Process: [See PDF for image] Source: GAO analysis based on Construction Industry Institute, Improving Early Estimates Research Team, Improving Early Estimates: Best Practices Guide (September 1998). [End of figure] The Use of Preliminary Information Limited the Precision of the 2003 Estimate: Due to the preliminary information on which it was based, the 2003 estimate did not foresee some of the challenges of retrofitting new systems into existing buildings and did not include space for various screening operations. LAWA determined that the 2003 concepts for each terminal did not allow for sufficient space for the EDS machines, conveyor belts, and the construction of baggage inspection rooms, rooms for computer monitors for on-screen resolution of EDS alarms and rooms for computer routers and servers. LAWA determined that the placement of EDS machines in the 2003 concepts needed to be changed in five of the nine terminals at LAX and both ONT terminals. The 2005 Estimate Was Based on More Definitive Information: The revised 2005 estimate was based on concepts approved by TSA in 2004 which included more definitive information about terminal design requirements than the 2003 estimate. The 2005 estimate designs included new construction and excavation, leading to cost estimate increases. The 2005 estimate included the incorporation of 20 additional baggage inspection rooms, 9 on-screen resolution rooms and 10 computer rooms at LAX and ONT terminals. The 2005 estimate included over $11 million in networking costs and costs associated with on-screen resolution of EDS alarms. TSA also highlighted two additional factors that impacted the estimates- cost increases due to the delay in beginning construction of the project and the escalation of construction costs between 2003 and 2005. In 2004, LAWA determined that Boeing had made a mathematical error in the 2003 concept development estimate: construction costs were only included for one of the two baggage screening facilities and neither connecting tunnel at ONT. In January 2007, TSA officials told us that they were not able to substantiate the mathematical error in the 2003 concept development estimate. Table 1: Estimated Costs for LAX and ONT Terminals-2003 and 2005: Terminal 1 (LAX); 2003 LOI/MOA estimate (dollars in millions): $42.9; 2005 estimate (dollars in millions): $108.8[1]; Rise is estimate cost (dollars in millions): $35.1 (48 percent); Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * 2003 concept envisioned separate screening areas for each terminal; * 2004 LAWA review determined the concept for terminal 1 was infeasible; * 2005 LAWA design consolidated the screening area in one new building between the terminals. Terminal 2(LAX); 2003 LOI/MOA estimate (dollars in millions): $30.1; 2005 estimate (dollars in millions): [Empty]; Rise is estimate cost (dollars in millions): [Empty]; Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * 2003 concept envisioned separate screening areas for each terminal; * 2004 LAWA review determined the concept for terminal 1 was infeasible; * 2005 LAWA design consolidated the screening area in one new building between the terminals. Terminal 3(LAX); 2003 LOI/MOA estimate (dollars in millions): $45.0; 2005 estimate (dollars in millions): $50.8; Rise is estimate cost (dollars in millions): $5.8 (13 percent); Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * 2005 LAWA review determined that terminal would need to be modified to accommodate the EDS machines and conveyor belts; * 2005 LAWA review also determined electrical upgrades were needed to accommodate EDS power requirements; * 2005 LAWA design modifications included the construction of a structure for new baggage inspection and screening rooms. Terminal 4 (LAX); 2003 LOI/MOA estimate (dollars in millions): $21.0; 2005 estimate (dollars in millions): $52.6; Rise is estimate cost (dollars in millions): $31.6 (150 percent); Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * 2003 concept envisioned above ground and elevated EDS machine location; * 2004 LAWA review determined that space constraints and construction disruptions made the 2003 concept infeasible; * 2005 design envisioned constructing a building below ground level to accommodate the required EDS machines. This excavation required constructing tunnels and reinforcing the roof of the building to support the tarmac. Terminal 5 (LAX): 2003 LOI/MOA estimate (dollars in millions): $18.0; 2005 estimate (dollars in millions): $40.2; Rise is estimate cost (dollars in millions): $22.2 (123 percent); Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * 2005 design included additional control software needed to integrate baggage handling system; * 2005 design extended roofing to cover baggage handling system; * 2005 design included construction of new baggage inspection and screening rooms; * LAWA believes that the 2003 estimate for the conveyor system was substantially low and inconsistent with estimates for similar LAX terminals. Terminal 6 (LAX); 2003 LOI/MOA estimate (dollars in millions): $37.8; 2005 estimate (dollars in millions): $46.0; Rise is estimate cost (dollars in millions): $8.2 (22 percent); Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * 2005 designs include additional space to accommodate the baggage screening system; * Baggage system control room added in 2005 designs. Terminal 7 (LAX); 2003 LOI/MOA estimate (dollars in millions): $17.8; 2005 estimate (dollars in millions): $32.5; Rise is estimate cost (dollars in millions): $14.7 (82 percent); Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * For the 2005 design, LAWA determined that the terminal building would need to be expanded to accommodate required EDS machines; * 2005 design included construction of new baggage inspection and screening rooms; * LAWA believes that the 2003 estimate for the conveyor system was substantially low and inconsistent with estimates for similar LAX terminals. Terminal 8 (LAX); 2003 LOI/MOA estimate (dollars in millions): $6.3; 2005 estimate (dollars in millions): $17.2; Rise is estimate cost (dollars in millions): $11.0 (174 percent); Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * 2004 LAWA review determined that the 2003 concept was infeasible because of space constraints; * 2005 design included modification and demolition work on the terminal building; * 2005 design incorporated the construction of a baggage inspection room and an addition to the terminal building to accommodate the EDS machines; * LAWA believes that the 2003 estimate for the conveyor system was substantially low and inconsistent with estimates for similar LAX terminals. Tom Bradley International; 2003 LOI/MOA estimate (dollars in millions): $91.2; 2005 estimate (dollars in millions): $91.8; Rise is estimate cost (dollars in millions): $0.6 (1 percent); Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * 2005 design included new buildings to accommodate a second baggage inspection room and EDS machines for re-screening checked baggage from international flights. ONT; 2003 LOI/MOA estimate (dollars in millions): $28.5; 2005 estimate (dollars in millions): $53.3; Rise is estimate cost (dollars in millions): $24.8 (87 percent); Design changes/key reasons for estimate increase based on GAO review of LAWA documentation: * 2003 concept development estimate drawings envisioned constructing a below ground checked baggage screening facility, including conveyor tunnels for both terminals; * In 2004, LAWA determined that Boeing had made a mathematical error in the 2003 concept development estimate: construction costs were only included for one of the two baggage screening facilities and neither connecting tunnel. In January 2007, TSA officials told us that they were not able to substantiate this error; * The 2005 design envisioned construction an above ground EDS facility at each terminal. Source: GAO analysis of LAWA data. [1] Number reflects the total cost for terminals 1 and 2. [End of table] Changes in TSA Guidance and Evolving Technologies Contributed to Rise in Estimate: According to LAWA, changes in TSA guidance on in-line system designs after the LOI/MOA was signed necessitated system redesigns. Because few in-line systems were in use at the time of the LOI/MOA, limited information on the capabilities of the in-line EDS machines, including actual bags screened per hour and false alarm rates, was available for modeling the systems. In June 2006, TSA produced Recommended Security Guidelines for Airport Planning, Design and Construction to guide future construction of in- line checked baggage screening systems. TSA also expects to release more detailed guidelines for in-line system planning and design in a few months. Concluding Observations: The rise in the estimate between 2003 and 2005 was primarily related to the fact that the 2003 estimate was developed at an early stage in the design process and was therefore based on preliminary data and assumptions that were subject to change. According to TSA and LAWA officials, both TSA and Los Angeles signed the LOI/MOA knowing the preliminary nature of the cost estimate. The LOI/ MOA affords TSA flexibility to amend the agreements to account for changed circumstances. However, under the terms agreed to in the LOI/MOA, TSA has no obligation to amend the LOI/MOA or to reimburse Los Angeles for any additional costs beyond those agreed to in the LOI/MOA and does not have plans to do so. Time Line of Key Events from the 2003 and 2005 Cost Estimates: [See PDF for image] Source: GAO analysis of TSA and LAWA data. [End of figure] [End of section] Appendix III: Comments from Los Angeles World Airports: Los Angeles World Airports: March 8, 2007: By electronic mail: goldenkoffr@gao.gov Mr. Robert Goldenkoff: Acting Director: Homeland Security and Justice: United States Government Accountability Office: 441 G Street, N.W. Washington, D.C. 20548: Dear Mr. Goldenkoff, This letter provides the comments of Los Angeles World Airports 9LAWA) on the February 2007 Draft Report entitled "Aviation Security: Cost Estimates Related to TSA Funding of Checked Baggage Screening Systems at Los Angeles and Ontario Airports" (Report). We thank you for the opportunity to review the draft and to discuss our views on this matter. In three respects, we believe the Report does not paint an accurate or complete picture of the facts. 1. TSA has the authority to revise the LOI to reflect accurate cost figures and explicitly anticipated doing so during the LOI/MOA development process, but has failed to meet that expectation to correct reimbursement levels for LAWA. Despite contemporaneous assurances made by TSA to adjust the LOI/MOA and provide additional reimbursements based on the anticipated need for future design modifications, the Report only states, at page 4, that "TSA has no obligation to reimburse the City of Los Angeles for any additional costs beyond those agreed to in the LOI/MOA, and TSA officials stated that the agency does not have plans for such reimbursements." TSA explicitly incorporated in the LOI the ability to make an adjustment to reflect additional costs, anticipating the need for a correction. The LOI states, at pages 1-2 (emphasis added): The TSA may, from time to time, following consultation with the City, amend this LOI and the MOA to adjust the payment schedule, and such adjustments may be made by the TSA when occasioned by changes in the actual allowable costs of the Project, the the actual time to complete the Project, in actual or estimated future obligating authority, or otherwise, when determined at the Administrator's discretion to be in the best interests of the United States. LAWA considered this provision important, because TSA and LAWA agreed that the LOI cost estimates were developed from preliminary designs-- prepared by TSA's own contractor and subcontractor--that were likely to change, including as a result of subsequent TSA changes to guidelines in baggage system design requirements. Indeed, TSA has since exercised this authority by amending the LOI/MOA to reflect changes in timing and scope of the project. Moreover, in 2004 a senior TSA official provided written assurances to LAWA that TSA would have an opportunity to cover an increase in costs due to design changes. In an April 22, 2004 e-mail provided to GAO, the TSA official stated: We really do understand that as designs get fleshed-out and actual bids come in, there will have to be some adjustments made to the MOA. Please do not feel as if you'll be left holding the bag or be held to estimates that do not prove to be right on the mark. There will be many reasons to "adjust." As we get into construction, we'll again be dealing with "puts and takes." Since we're talking reimbursement of allowable costs (not firm-fixed-price), we'll have the opportunity on the back-end to reconcile (FY06-07). LAWA relied on this and other assurances from TSA, reinforced in various discussions, at the time it concluded the LOI/MOA process. The Report also does not reflect the subsequent extensive and protracted discussions LAWA has with TSA, leaving the impression that LAWA simply presented a new set of design concepts to TSA in September 2004. In fact, TSA and LAWA worked extensively over several months, culminating in its submission of an engineering study in September 2004, which TSA approved later that month. LAWA used these updates, TSA- approved concepts for reimbursement in 2006[Footnote 11] LAWA appropriately requested an amendment to the LOI to receive the statutory required 75% federal reimbursement of the $485 million (without inflation) that TSA has deemed actual eligible and allowable costs of the project, an increase of $107.25 million in revised coss estimates were agreed to by TSA and based on TSA-approved final designs and guidelines revised by TSA with substantial input from LAWA. The Report recognizes that both TSA and LAWA assumed design cost estimates were preliminary, which gives meaning to the highlighted text of the LOI quoted above. The agreement LAWA and other airport sponsors entered with TSA was based on receiving reimbursement of 75% of eligible and allowable project costs, as mandated by Congress. LAWA did not believe it would be held financially responsible for an increase in eligible and allowable costs due to reasons beyond its control. Having received assurances from TSA, it moved forward and expects ultimately to be reimbursed for those costs. 2. The Report fails to assign specific responsibility for initial designs and many errors, as directed by Senate Committee language. The Report avoids assigning responsibility to the TSA, Boeing, its contractor, or Turner, its subcontractor, even where doing so is highly relevant to the equities in LAWA's request for additional reimbursement. GAO thus has failed to answer the Senate Committee's direction to "provide a detailed explanation of the reasons for any differences from the original estimate, including identification of and the party responsible for any material mistakes, omissions, and infeasible design concepts in the original estimate." Most obvious, is the Report's treatment of the "mathematical error at ONT." in failing to account for two terminals at that airport and neither of the connecting tunnels. Although the Report states that the TSA "was not able to substantiate this error," significant issues contributing to the error should have been evident upon close inspection of the documents. GAO performed no independent verification of TSA or LAWA numbers, the omission of these facilities is readily apparent from the documents that Boeing submitted, and the former Boeing project manager responsible for development of the estimates confirmed the error. In addition, TSA's statement, in a footnote on slide 8, that "a LAWA contractor provided Boeing with baggage handling concept designed for LAW and ONT," is simply untrue. Turner, a subcontractor, to TSA's contractor Boeing, provided this information to Boeing in 2003. LAWA later hired Turner , in 2004, to assist on this project, but only after Turner was no longer working for TSA and Boeing. Turner was not an LAWA contractor, but a TSA subcontractor to Boeing, and thus did not act of behalf of LAWA in any capacity to determine the initial design and cad cost estimates. LAWA relied on these estimates because there was not enough time to make an independent assessment that would have identified the errors, omissions, and fact critical to the question of accountability. In determining whether, as a matter of public policy, the TSA or LAWA should shoulder the financial burden of the increase in allowable costs, it is important to determine the proportionate amount of costs due to TSA, its contractor, Boeing, its subcontractor Turner LAWA, or the inherent and inevitable inadequacies of preliminary design concepts. Although table 1 breaks down the revision in cost estimates per airport and terminal, the Report refrains from assigning responsibility to any party, and does not attempt to determine what portion of the increase in costs was due to any particular error or design change. 3. The Report does not recognize that LAWA responded to TSA urgency in completing the agreements, resulting in the use of preliminary design and cost estimates open to future revisions. The Report does not convey the critical fact that LAWA operated under TSA direction that time was of the essence to complete the LOI/MOA. LAWA's statement that it was under a tight time frame to apply for the LOI is fact. TSA's push to conclude the LOI/MOA as son as practicable is supported by e-mail correspondence between TSA and LAWA before the LOI/MOA was signed. Given this time pressure, LAWA essentially was required to accept the Boeing design and cost estimates and did not have sufficient opportunity to conduct a thorough and detailed review. That the design of such a complicated and complex project may well take years and cannot be expected to be completed within the short time frame provided is confirmed but the publication, in 2006, of the TSA Design and Construction Guidelines referred to in the Report. These Guidelines were based in substantial part on input received from LAWA and LAWA consultants during the design development period at LAX and ONT from 2003 to 2005, some of whom are listed as contributors to the Guidelines. Conclusion: In summary, this Report fails to reflect, fully and clearly, the context in which LAWA has sought the LOI/MOA cost adjustment. Specifically, in determining the original cost figures for inclusion in the LOI/MOA, responding to TSA's urgency, LAWA was forced to rely on the designs and cost estimates submitted to it by TSA's contractor and subcontractor, but with the mutual understanding that those costs would be adjusted later. Shortly after the LOI/MOA was signed, LAWA determined that cost estimates were based on errors, omissions, and infeasible design concepts, and promptly notified TSA. LAWA then worked with TSA over many months to revise these designs, not only to address identified problems, but also to follow changes in TSA design requirements. The increased cost estimates are based on TSA-approved final designs and TSA determinations of eligible and allowable costs. Throughout this entire period, LAWA acted responsibly and in good faith to comply with TSA guidelines. As provided in the LOI/MOA and anticipated during discussions with TSA, LAWA seeks Federal reimbursement only of its Congressionally-mandated share of these eligible and allowable costs. Again, we appreciate the opportunity to provide our views on the report. Sincerely, Signed by: Alan I. Rothenberg, President: Board of Airport Commissioners: Signed by: Samson Mengistu: Acting Executive Director: [End of section] FOOTNOTES [1] See 49 U.S.C. § 44901(d). [2] See Consolidated Appropriations Resolution, 2003, Pub. L. No. 108- 7, § 367, 117 Stat. 11, 423-24. See also 49 U.S.C. § 44923. [3] See S. Rep. No. 109-273, at 44 (2006) (accompanying H.R. 5441, as passed by the Senate; subsequently enacted into law as the Department of Homeland Security Appropriations Act, 2007, Pub. L. No. 109-295, 120 Stat. 1355 (2006)). [4] Our Congressional Protocols state that GAO will work with the majority and minority of the designated committee to clarify the scope of work, reporting objectives, and time frames. [5] Improving Early Estimates Research Team, Improving Early Estimates: Best Practices Guide, Construction Industry Institute, September 1998. [6] GAO, Aviation Security: Systematic Planning Needed to Optimize the Deployment of Checked Baggage Screening Systems, GAO-05-365 (Washington, D.C.: Mar.15, 2005). [7] See Consolidated Appropriations Resolution, 2003, Pub. L. No. 108- 7, § 367, 117 Stat. 11, 423-24. See also 49 U.S.C. § 44923. [8] See S. Rep. No. 109-273, at 44 (2006) (accompanying H.R. 5441, as passed by the Senate; subsequently enacted into law as the Department of Homeland Security Appropriations Act, 2007, Pub. L. No. 109-295, 120 Stat. 1355 (2006)). [9] In comments on a draft of this report, LAWA wrote that it requested 75 percent federal reimbursement of the $485 million, an increase of around $107 million. This amount differs from the almost $122 million it requested in its April 19, 2005, letter to TSA. [10] Construction Industry Institute, Improving Early Estimates Research Team, Improving Early Estimates: Best Practices Guide (September 1998). [11] While the corrected designs reflected a significant increase in costs, they also reduced the number of EDS machines needed at LAX and ONT by 15, providing a substantial cost savings for TSA of approximately $45 million. GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to www.gao.gov and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, D.C. 20548: Public Affairs: Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: