This is the accessible text file for GAO report number GAO-05-726 
entitled 'Aviation Safety: System Safety Approach Needs Further 
Integration into FAA's Oversight of Airlines' which was released on 
October 26, 2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

September 2005: 

Aviation Safety: 

System Safety Approach Needs Further Integration into FAA's Oversight 
of Airlines: 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-726]: 

GAO Highlights: 

Highlights of GAO-05-726, a report to congressional requesters: 

Why GAO Did This Study: 

The Federal Aviation Administration (FAA) uses the Air Transportation 
Oversight System (ATOS), which was developed around the principles of 
system safety, to oversee seven “legacy” airlines” and nine other 
airlines. In this report, we refer to airlines that are not in ATOS as 
non-legacy airlines. Two other processes are used to oversee 99 non-
legacy passenger airlines, which represent a fast-growing segment of 
the commercial aviation passenger industry and carried about 200 
million passengers in 2004. The National Work Program Guidelines (NPG) 
establishes a set of inspection activities for non-legacy airlines. The 
Surveillance and Evaluation Program (SEP) uses principles of system 
safety to identify additional risk-based inspections for those 
airlines. 

GAO’s objective was to assess the processes used by FAA to ensure the 
safety of non-legacy passenger airlines. GAO reviewed the strengths of 
FAA’s inspection oversight for non-legacy passenger airlines and the 
issues that hinder its effectiveness. 

What GAO Found: 

A key strength of FAA’s inspection oversight of non-legacy airlines is 
the introduction of system safety concepts to some inspections, which 
FAA accomplished by adding SEP to its traditional inspection process, 
NPG. Although NPG has risk-based elements, it lacks the structured 
approach to risk identification found in SEP. Under SEP, data are used 
to help determine trends or problems. The SEP process uses a team of 
inspectors to identify inspection activities, which we have previously 
reported is generally more effective than the use of individuals due to 
their collective ability to identify risks. Under SEP, inspectors also 
ascertain risks internal to FAA, such as staffing shortages. FAA’s 
oversight of non-legacy airlines further incorporates processes to 
ensure that inspectors follow up on airline actions taken in response 
to inspection findings. These efforts address several past GAO 
concerns, including that NPG did not allow FAA to identify risks and 
allocate inspection resources accordingly. 

The full potential of FAA’s inspection program for non-legacy airlines, 
however, is not being realized due to incomplete implementation of its 
system safety approach and other challenges. The inspection workload is 
still heavily oriented to nonrisk-based activities, with 77 percent of 
inspection activities being identified through the NPG and the 
remaining relatively small percentage identified through SEP. The 
emphasis on NPG, including FAA’s guidance that inspectors must complete 
NPG-required inspection activities, acts as a disincentive to 
identifying further inspection activities through SEP. Inspectors face 
workload challenges as staff lost through attrition may not be replaced 
due to a hiring freeze. FAA estimates that over 1,100 inspectors of non-
legacy airlines will leave the agency in fiscal years 2005 to 2010. In 
addition, some FAA inspectors indicated that a lack of technical 
training on airline systems and equipment posed potential risks to the 
agency’s oversight process. Finally, FAA lacks a process to communicate 
information to inspectors on how certain internal risks identified 
through SEP are being resolved. Moreover, FAA has not established a 
process to evaluate the effectiveness of SEP. 

Aircraft Safety Inspection: 

[See PDF for image] 

[End of figure] 

What GAO Recommends: 

GAO recommends that FAA develop an evaluative process for SEP and 
improve communications and training for inspectors in system safety and 
risk management. FAA agreed with the recommendations on training and 
communications and will consider developing an evaluative process. 

www.gao.gov/cgi-bin/getrpt?GAO-05-726. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Gerald L. Dillingham at 
(202) 512-2834 or dillinghamg@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

FAA's Oversight Process Uses System Safety, Which Is Designed to 
Identify and Control Risks and Improve Resource Utilization: 

Most Inspection Activities Are Not Prioritized Based on a Structured 
Risk Assessment Process: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendixes: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: FAA's Surveillance and Evaluation Process: 

SEP Incorporates System Safety into FAA's Inspection Oversight of Non- 
legacy Airlines: 

SEP Incorporates Some Elements of ATOS, While NPG Relies on a Set 
Number of Inspections: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Number of Times Risks Internal to FAA Were Identified by 
Inspectors through SEP, Fiscal Years 2002-2004: 

Table 2: Severity of Internal Risks Identified by FAA Inspectors, 
Fiscal Years 2002-2004: 

Table 3: SEP-and NPG-Initiated Required Inspections for the Top 25 Non- 
legacy Airlines, Fiscal Years 2002-2004: 

Table 4: Allocation of FAA Inspection Staff under SEP and ATOS for Four 
Airlines: 

Table 5: Number of Enplanements for the Top 25 Non-legacy Airlines, 
2004 

Table 6: FAA Offices and Airlines Interviewed by GAO: 

Table 7: Organizations Interviewed by GAO: 

Table 8: Safety Systems and Examples of Risk Indicators in SEAT: 

Table 9: Various Elements of ATOS, NPG, and SEP: 

Figures: 

Figure 1: Percentage of Enplaned Passengers on Non-legacy Airlines and 
Legacy Airlines, 1995-2004: 

Figure 2: Number of FAA Aviation Safety Inspectors for Commercial 
Airlines, Fiscal Years 2002-2004, and Estimates for Fiscal Years 2005- 
2007: 

Figure 3: Non-legacy Airline Inspectors' Views on the Extent Technical 
Training Received in the Last 2 Years Has Helped Them Do Their Jobs: 

Figure 4: Views of Inspectors of Non-legacy Airlines on the Extent 
Inspectors Have Received Technical Training in a Timely Manner during 
Their FAA Careers: 

Figure 5: View of Inspectors of Non-legacy Airlines on the Extent to 
Which They Received Training in Automated Systems Used in the 
Inspection Process in a Timely Manner: 

Abbreviations: 

ATOS: Air Transportation Oversight System: 

DOT: Department of Transportation: 

FAA: Federal Aviation Administration: 

NPG: National Work Program Guidelines: 

PTRS: Program Tracking and Reporting Subsystem: 

SEAT: Surveillance and Evaluation Assessment Tool: 

SEP: Surveillance and Evaluation Program: 

Letter September 28, 2005: 

The Honorable James L. Oberstar: 
Ranking Democratic Member: 
Committee on Transportation and Infrastructure: 
House of Representatives: 

The Honorable Jerry F. Costello: 
Ranking Democratic Member: 
Subcommittee on Aviation: 
Committee on Transportation and Infrastructure: 
House of Representatives: 

The Honorable Peter A. DeFazio: 
House of Representatives: 

The U.S. commercial aviation industry, with less than one fatal 
accident per 5 million flights from 2002 through 2004, has an 
extraordinary safety record. However, when passenger airlines have 
accidents or serious incidents, regardless of their rarity, the 
consequences can be tragic. In order to maintain a high level of 
aviation safety, it is critical to have well-established, efficient, 
and effective systems in place to provide an early warning of hazards 
that can lead to accidents. The Federal Aviation Administration (FAA) 
has established a number of systems and processes to inspect and 
oversee various aspects of passenger airline safety, such as aircraft 
maintenance and flight operations. About 585 of FAA's approximately 
3,200 inspectors are dedicated to overseeing the largest commercial 
passenger airlines, including the seven "legacy airlines," and nine 
other airlines through FAA's Air Transportation Oversight System 
(ATOS), which was developed around the principles of system 
safety.[Footnote 1] System safety involves the continual evaluation of 
all of an airline's operations for the purpose of identifying and 
mitigating risks. Approximately 1,100 inspectors[Footnote 2] oversee 
other entities and individuals, including 99 smaller commercial 
passenger airlines--which we refer to as non-legacy passenger airlines-
-about 5,200 aircraft repair stations, and approximately 625,000 
pilots. Non-legacy passenger airlines, a fast-growing segment of the 
commercial aviation passenger industry, carried about 200 million 
passengers in 2004. 

FAA's inspection process for airlines not covered by ATOS has two 
components. The National Work Program Guidelines (NPG) is the baseline 
component of the oversight program for these airlines. In 2002, FAA 
added another component, the Surveillance and Evaluation Program (SEP), 
to the inspection process to incorporate principles of ATOS into its 
oversight of non-legacy passenger airlines. The two components are used 
together to establish the number of annual inspections for non-legacy 
airlines. Inspections can encompass many different activities, such as 
visually spot-checking an airplane at a gate, monitoring procedures on 
a scheduled flight, or observing maintenance being performed on an 
aircraft. Each year, FAA headquarters establishes baseline inspections 
for each airline through NPG, while through SEP, teams of FAA 
inspectors analyze the results of an airline's prior inspections at 
periodic meetings and, based on their assessment of specific risks, 
establish other inspections that may be needed. 

In response to your request, we assessed FAA's processes for ensuring 
the safety of non-legacy passenger airlines. Specifically, we addressed 
the following questions: (1) What are the strengths of FAA's inspection 
approach for non-legacy passenger airlines? and (2) What issues hinder 
the effectiveness of FAA's inspection approach? 

To address these questions, we obtained and analyzed information from a 
variety of sources. We examined FAA documents about SEP, NPG, and ATOS. 
We also reviewed prior reports prepared by us and others on SEP and NPG 
to determine significant issues involving those programs and how they 
were resolved. In addition, we surveyed a statistical sample of FAA 
safety inspectors to obtain their views about the training they 
receive. We had no practical way to assess information on the amount of 
training necessary for inspector proficiency or the timeliness of the 
training provided. We conducted semistructured interviews and analyzed 
relevant documents from FAA headquarters officials, and from field 
managers and inspectors in 7 regional and 13 field offices,[Footnote 3] 
which were selected because they oversee the top 25 non-legacy 
airlines[Footnote 4] ranked by the number of enplanements in 2004. At 
these locations, we collected information on the inspection process and 
inspector staffing levels, workload, and training. In addition, we 
conducted semistructured interviews with safety officials at 16 of 
those top 25 non-legacy airlines, and with officials at seven industry 
organizations that represent airlines, inspectors, pilots, mechanics, 
and maintenance facilities. We analyzed data on required NPG and SEP 
inspections for fiscal years 2002 through 2004 from FAA's nationwide 
inspection database--the Program Tracking and Reporting Subsystem 
(PTRS).[Footnote 5] We assessed the reliability of the database and 
found the data sufficiently reliable for the types of analyses that we 
conducted for this report. We also tested for the presence of several 
management controls, including the processes for verifying inspection 
results, establishing a process to evaluate FAA's inspection oversight 
process, managing the PTRS database, and communicating among managers 
and inspectors relating to NPG and SEP. In addition, we reviewed 
literature on system safety and compared FAA's system safety framework 
with that presented in the literature. We conducted our work from 
August 2004 through September 2005 in accordance with generally 
accepted government auditing standards. Additional information on our 
methodology is found in appendix I. 

Results in Brief: 

A key strength of FAA's inspection oversight of non-legacy airlines is 
the introduction of system safety concepts to some inspections, which 
FAA accomplished by adding SEP to its traditional inspection process, 
NPG. SEP presents a shift in concept from FAA's customary method that 
relied on conducting a set number of inspections of an airline's 
operations to an approach that allows for the efficient use of 
inspection staff and resources by prioritizing workload based on areas 
of highest risk. To facilitate the implementation of a system safety 
approach, FAA has made an effort to train its staff in system safety. 
In addition, FAA utilizes teams of inspectors in SEP, which we have 
previously reported is generally more effective than the use of 
individuals because of the team's collective ability to identify risks. 
SEP also allows inspectors to identify risks internal to FAA, such as 
staffing shortages or training deficiencies, and FAA has established a 
chain of command to address these risks. FAA's oversight of non-legacy 
airlines further incorporates processes to ensure that inspectors 
follow up on airline actions taken in response to inspection findings. 
These efforts address several past concerns noted by us--that FAA 
needed to better identify risks and allocate inspection resources 
accordingly, and verify and monitor inspection findings to ensure that 
priorities were achieved. 

The full potential of FAA's inspection program for non-legacy airlines, 
however, is not being realized due to incomplete implementation of the 
agency's system safety approach and other challenges. The inspection 
workload is still heavily oriented to the NPG's nonrisk-based 
activities. For fiscal years 2002 through 2004, 77 percent of 
inspection activities required for the top 25 non-legacy airlines were 
identified through NPG, so that only the remaining relatively small 
percentage of inspection activities were identified based on risk 
through SEP. We found that the large percentage of NPG-identified 
activities occurred because FAA's guidance places greater emphasis on 
the NPG-identified activities, even though the guidance establishes a 
process--referred to as retargeting--whereby inspectors can replace NPG-
identified activities with SEP-identified activities that they deem 
constitute a greater safety risk. For example, the guidance emphasizes 
the importance of the NPG-identified activities by requiring inspectors 
to complete all of the NPG-identified activities by the end of each 
fiscal year but permitting SEP-identified activities to be rescheduled 
to the following fiscal year. In addition, very few activities are 
being retargeted, partly because field offices have interpreted FAA's 
emphasis on NPG activities as discouraging retargeting. Inspector 
workload also presents a challenge to FAA's oversight, as the number of 
inspector staff available to oversee non-legacy airlines has declined 
due to attrition and workload shifts related to transferring staff to 
ATOS, and most losses were not replaced due to a hiring freeze. In 11 
of the 13 FAA field offices that we contacted, officials indicated a 
shortage of different types of inspectors needed to oversee the non- 
legacy airlines, which has sometimes resulted in inspections being 
delayed or eliminated. We also identified limitations concerning 
technical and SEP-specific training for inspectors of non-legacy 
airlines. For example, inspectors identified a lack of technical 
training on airline systems and equipment as an internal risk to the 
agency. In addition, FAA does not provide SEP-specific procedural 
training to certain types of inspectors, which some inspectors told us 
was needed to maximize their usefulness.[Footnote 6] Further, there is 
inadequate communication from headquarters to inspectors on the 
resolution of internal risks identified under SEP, according to some 
inspectors. Headquarters officials acknowledged that there is no formal 
feedback process to inform the inspectors about issues they raised 
concerning internal risks. FAA has not established a way to evaluate 
its inspection oversight process for non-legacy airlines. Moreover, the 
agency's ability to evaluate this process is hindered by the lack of 
important inspection-related information--such as whether the risks 
identified through SEP have been mitigated--in its nationwide 
inspection database. 

To improve the effectiveness of the agency's oversight of non-legacy 
airlines, we recommend that the Secretary of the Department of 
Transportation (DOT) direct the FAA Administrator to develop a 
continuous evaluative process for FAA's activities under SEP and link 
SEP to the performance-related goals and measures developed by the 
agency, track performance toward these goals, and determine appropriate 
program changes. We also recommend that the agency improve 
communication and training to ensure inspectors understand FAA's 
policies and procedures in areas such as system safety and risk 
management. DOT generally agreed with our recommendations to improve 
communication and training. DOT said that it would consider our 
recommendation to develop a continuous evaluative process for SEP and 
link SEP to agency goals, but that its plan to put the remaining non- 
legacy airlines in the ATOS program by the end of fiscal year 2007 may 
make this recommendation unnecessary. In the past, FAA's efforts to 
move airlines to ATOS have experienced delays, therefore, we retained 
this recommendation. The department also provided clarifying comments 
and technical corrections, which we incorporated as appropriate. 

Background: 

Non-legacy airlines represent a fast-growing segment of the passenger 
airline industry. From 2002 through 2004, the annual enplanements for 
these airlines grew from 122 million to about 200 million passengers, 
or from about 20 percent to 28 percent of all passenger air travel. 
During this same period, the percentage of people flying on legacy 
airlines declined from 80 percent to about 72 percent of all passengers 
flown (see fig. 1). 

Figure 1: Percentage of Enplaned Passengers on Non-legacy Airlines and 
Legacy Airlines, 1995-2004: 

[See PDF for image] 

[End of figure] 

FAA's Inspector Workforce Carries Out Oversight: 

FAA's safety oversight of non-legacy airlines is carried out by 
inspectors located at 109 field offices throughout the world that are 
part of 9 regional offices. For each airline, FAA puts together a team 
led by principal inspectors who maintain primary responsibility for 
managing the airline's certificate requirements[Footnote 7] and focus 
on one of three disciplines: avionics,[Footnote 8] maintenance, or 
operations. Additional team members include those based at the FAA 
office that holds the airline's operating certificate--typically an 
aircraft dispatch inspector, a cabin safety inspector, and assistants. 
FAA locates its principal inspectors close to their respective 
airlines' primary operational base. For example, the principal 
inspectors for Independence Air (formerly Atlantic Coast Airlines) are 
located at FAA's field office at Dulles International Airport, where 
the airline has its headquarters. In addition, FAA has geographic 
inspectors based at the 109 field offices to conduct additional 
inspections. Rather than being designated to particular airlines, 
geographic inspectors may conduct inspections of aircraft of any non- 
legacy airlines that land in their area. 

FAA's safety inspector workforce for all commercial airlines has 
remained steady, averaging about 1,780 inspectors over fiscal years 
2002 through 2004. FAA cannot determine how many of these inspectors 
were assigned to non-legacy passenger airlines during those 3 years, 
because the agency did not collect that information, according to an 
FAA headquarters' official. FAA expects the number of inspectors 
assigned to non-legacy airlines to remain steady for fiscal years 2005 
and 2006 and decline slightly in fiscal year 2007, as shown in figure 
2. Inspectors record information about these inspections in PTRS--a 
nationwide computerized database that maintains such information as 
inspector findings and airline activities in response to the findings. 

Figure 2: Number of FAA Aviation Safety Inspectors for Commercial 
Airlines, Fiscal Years 2002-2004, and Estimates for Fiscal Years 2005- 
2007: 

[See PDF for image] 

Note: FAA does not have information to separate out the number of ATOS 
inspectors and inspectors of non-legacy airlines prior to fiscal year 
2005. 

[End of figure] 

FAA Uses NPG and SEP to Oversee Non-legacy Airlines: 

NPG and SEP are the main inspection processes that FAA uses to oversee 
the safety of non-legacy airlines. Since 1985, FAA has used NPG, which 
includes both required and planned inspections, as its primary means of 
ensuring that airlines comply with safety regulations. In NPG, an FAA 
committee of program managers identifies an annual minimum set of 
required inspections that are to be undertaken to ensure that airlines 
are in compliance with their operating certificates. In addition, 
inspectors determine annual sets of planned inspections based on their 
knowledge and experience with the particular airlines they oversee. 
Typically, inspections would include ramp inspections, in which 
inspectors examine an aircraft while it is parked at the airport, and 
maintenance inspections. However, we found problems with NPG throughout 
the 1990s, including: (1) FAA's routine inspections were ineffective in 
identifying serious safety problems, (2) critical airline inspections 
had not been conducted, (3) FAA's follow-up actions often did not 
ensure that problems were corrected once identified, and (4) FAA did 
not have a methodology for estimating airline safety risks so that it 
could target limited inspection resources to high-risk inspections. We 
also found that FAA's inspection database was of limited use in 
providing early warning of potential risks or targeting inspection 
resources. 

In response to these findings, and in the aftermath of the 1996 ValuJet 
crash, an FAA task force reviewed the agency's safety inspection 
process and recommended in part that the agency initiate a project to 
make surveillance of airlines more systematic and targeted to deal with 
identified risks. This recommendation resulted in the agency's 
development and implementation of ATOS in 1998 at the nation's 10 
largest commercial passenger airlines, with the goal of eventually 
including all commercial passenger and cargo airlines.[Footnote 9] ATOS 
emphasizes a system safety approach that extends beyond periodically 
checking airlines for compliance with regulations to the use of 
technical and managerial skills to identify, analyze, and control 
hazards and risks.[Footnote 10] The goal of ATOS is to identify safety 
trends in order to spot and correct problems at their root cause before 
an accident occurs. This program allows FAA inspectors to look at an 
airline as a whole, to see how the many elements of its operations, 
including aircraft, pilots, maintenance facilities, flight operations, 
and cabin safety, interact to meet federal standards. Collectively, the 
airlines under ATOS had a dedicated inspector staff of 585 inspectors 
as of July 2005. The number assigned to each airline depends on the 
size of the airline's operations. ATOS uses special checklists and 
databases that are intended to cover all areas of airline operations. 
Part of FAA's oversight is expected to include an in-depth look at an 
airline's policies and procedures and whether the airline is following 
them. There are 16 airlines in the ATOS program, including two cargo 
carriers as of September 2005. Due to resource constraints, FAA 
determined it would not be able to immediately place the remaining 
passenger airlines in the ATOS program. FAA developed SEP as a bridge 
to introduce safety risk concepts used in ATOS into the oversight 
process for non-legacy airlines in order to facilitate the ultimate 
transition of these airlines to ATOS.As of September 2005, FAA 
estimates that it will move the remaining non-legacy airlines to ATOS 
by the end of fiscal year 2007. 

SEP Implemented as a Transition to ATOS: 

Since the introduction of ATOS, FAA has been moving toward integrating 
system safety into its oversight activities. However, the agency has 
been delayed in placing a significant number of non-legacy airlines in 
the ATOS program, resulting in those airlines continuing to be overseen 
through NPG, a process that is not system safety oriented. FAA agrees 
that NPG is not a system safety process, but explained that the planned 
NPG inspections, which inspectors identify based on their expertise, 
provide a risk-based element to the process. To address the delay in 
moving airlines to ATOS, FAA, in 2002, added SEP to NPG as a way to 
introduce airlines to a system safety oversight process until those 
airlines were transitioned to ATOS.[Footnote 11] Although originally 
envisioned by FAA as a transitional program for airlines that were 
awaiting placement in ATOS, SEP has become a more permanent oversight 
process as budget and staff constraints have prevented the agency from 
moving all airlines to ATOS. 

SEP is complementary to NPG and permits the risk-based, data-driven 
alteration of required NPG inspections. SEP provides a formal structure 
to risk identification that is absent from the NPG process. Under SEP, 
the principal FAA inspectors for each airline meet periodically during 
the year to discuss the results of their inspections and identify risks 
using the same risk assessment principles and checklists used in 
ATOS.[Footnote 12] An outcome of the meetings is the identification of 
inspections to augment the NPG baseline of required inspections, which 
FAA views as the minimum number of inspections that need to occur to 
ensure that certain areas of all airlines are reviewed.[Footnote 13] 
During the SEP process, FAA headquarters allows principal inspectors to 
replace NPG- required inspections with inspections targeting higher-
risk areas identified through the SEP meetings. This process is known 
as retargeting. At the end of the meeting, entries are created in the 
PTRS database that indicate all inspections to be performed for that 
airline. SEP-initiated inspections that are designated as priorities 
are required to be completed.[Footnote 14] See appendix II for 
additional information on SEP, ATOS, and NPG. 

FAA Also Uses Industry Partnership Programs to Provide Safety Oversight 
of Airlines: 

FAA also oversees the safety of both legacy and non-legacy passenger 
airlines through participation in industry partnership programs. Two of 
these programs--the Aviation Safety Action Program and Voluntary 
Disclosure Reporting Program--encourage certain airline employees, such 
as pilots and mechanics, or airlines to voluntarily report safety 
information that might be critical to identifying potential precursors 
to accidents without fear that FAA or their companies will use reports 
accepted under the programs to take legal enforcement or disciplinary 
actions against them. The Aviation Safety Action Program provides for 
the voluntary self-reporting of safety incidents under procedures set 
out in memorandums of understanding between FAA, airlines, and 
participating employee groups such as aircraft mechanics. As of June 
2005, 20 of the top 25 non-legacy airlines (in terms of the number of 
enplanements) were participating in the Aviation Safety Action Program. 
The Voluntary Disclosure Reporting Program allows airlines to 
voluntarily report safety incidents to FAA. However, our prior work 
found that FAA has not analyzed violation data derived from these two 
programs to monitor national trends in airline operations, so that it 
can target resources to address operational risks.[Footnote 15] A third 
program, the Internal Evaluation Program, requires airlines to 
continuously monitor and evaluate their practices and procedures. 

FAA's Oversight Process Uses System Safety, Which Is Designed to 
Identify and Control Risks and Improve Resource Utilization: 

The incorporation of system safety into FAA's oversight presents a 
shift in concept from the agency's traditional oversight method, which 
relied upon periodic inspections, to an approach that allows for the 
prioritization of inspections based on areas of highest risks. The 
development of SEP, a key element in FAA's system safety oversight 
process, allows for the efficient use of inspection staff and resources 
by incorporating this risk-based approach. FAA's incorporation of SEP 
into its oversight addresses a past concern by us that the agency was 
not prioritizing inspectors' workload. Our review of literature by 
government and private organizations supported FAA's concept of system 
safety because it makes apparent the risks that are the basis for 
changes. In addition, FAA's oversight process includes a requirement 
that inspectors verify that corrective actions have occurred, providing 
a level of assurance that the safety problem has been mitigated. 

FAA Has Provided System Safety Training for Inspectors: 

To facilitate the implementation of a system safety approach, FAA has 
made an effort to train its staff in system safety, which is a strength 
of its oversight approach. FAA has recognized that inspectors need 
training in the system safety concept in order to effectively 
incorporate this approach into airline oversight. Specifically, FAA 
believed that its inspectors have gaps in their needed competencies and 
skills that could affect the agency's system safety approach to 
inspections. Among the largest training gaps for inspectors, according 
to FAA, were training in risk analyses and systems thinking. As a 
result of these findings, FAA has undertaken training activities to 
improve inspector competencies in system safety.[Footnote 16] According 
to FAA, between fiscal years 2002 and 2004, it provided basic system 
safety training to almost 3,675 staff, including non-legacy and ATOS 
inspectors. 

SEP Incorporates System Safety by Focusing on the Identification and 
Prioritization of Inspections Based on Risks: 

By incorporating risk assessment into the inspection process, SEP 
addresses a long-standing problem. Since the mid-1980s, our reports 
have shown that FAA did not have a methodology for assessing airline 
safety risks so that it could target limited inspection resources to 
high-risk conditions.[Footnote 17] SEP's system safety concept of 
reducing risk through the identification, analysis, and control of 
hazards is also consistent with the views presented in literature of 
other government and private organizations, such as the Department of 
Defense, Canada's civil aviation authority (Transport Canada), and the 
Flight Safety Foundation. For example, DOD looks at system safety as a 
means of reducing risk through early identification, analysis, 
elimination, and control of hazards. For the airlines it oversees, 
Transport Canada requests that each airline incorporate system safety 
into its operations by integrating safety into its policies, management 
and employee practices, and operating procedures. FAA incorporates such 
concepts in its system safety oversight approach. 

SEP's system safety process has a number of strengths. At periodic 
meetings, the principal inspectors of an airline identify potential 
risks that they believe should be addressed. These inspectors consider 
a variety of information that they have obtained through personal 
observation and from reports prepared by geographic inspectors and 
located in the PTRS database. A strength of this approach, consistent 
with findings in our past reports,[Footnote 18] is that teams of 
inspectors are generally more effective than individual inspectors in 
their ability to collectively identify concerns. Another strength is 
that FAA has developed risk assessment worksheets aligned with key 
airline systems that guide inspectors through identifying and 
prioritizing risks. The worksheets guide inspectors to organize the 
results of their previous inspections and surveillance into a number of 
areas, such as flight operations, personnel training, and cabin safety, 
in order to identify specific risks in each area and target the 
office's resources to mitigating those risks. (See app. II for more 
details on the risk assessment process.) 

SEP Also Allows for the Identification of Internal Risks That Might 
Hinder Oversight: 

During their periodic meetings, principal inspectors also have the 
opportunity to identify risks in FAA's internal operations that could 
adversely affect the inspectors' ability to conduct safety oversight of 
non-legacy airlines, which is another strength of FAA's oversight 
approach. FAA inspectors and managers agreed that identifying internal 
risks is helpful, as they considered it an efficient and effective way 
of identifying needed program improvements. Inspectors can identify 
deficiencies in a number of categories, including inspector staffing, 
training, availability of geographic inspectors, and resources such as 
travel funds. In addition, the inspectors can quantify the degree of 
the risk that each of these categories poses. Being able to identify 
and make officials aware of situations that can hinder an agency's 
ability to perform its mission is a key management control. 

This process has resulted in FAA headquarters receiving significant 
information on areas that inspectors believe are deficient. For fiscal 
years 2002 through 2004, inspectors identified about 560 risks that, in 
their opinion, could have an impact on how FAA managed the inspection 
process of airlines, as shown in table 1. Inadequate training (aircraft-
specific training and other training) was the most frequently 
identified concern of the inspectors, accounting for about 40 percent 
of the total risks identified during the 3-year period--27 percent of 
the risks were related to inadequate or nonexistent aircraft-specific 
training and 15 percent were related to the lack of other training. We 
discuss this issue in detail later in this report. The total number of 
risks identified by FAA inspectors declined over the 3 years. 
Inspectors from two FAA field offices opined that the decline may be 
due to better staff understanding of risk identification and to the 
risks being addressed expeditiously so that inspectors did not need to 
restate them the following year. 

Table 1: Number of Times Risks Internal to FAA Were Identified by 
Inspectors through SEP, Fiscal Years 2002-2004: 

Risk: Inadequate staffing on the certificate management team[A]; 
Number of risks identified: 2002: 67; 
Number of risks identified: 2003: 53; 
Number of risks identified: 2004: 55; 
Number of risks identified: 175 (31%). 

Risk: Inadequate or nonexistent aircraft-specific training; 
Number of risks identified: 2002: 58; 
Number of risks identified: 2003: 57; 
Number of risks identified: 2004: 33; 
Number of risks identified: 148 (27%). 

Risk: Lack of other training; 
Number of risks identified: 2002: 35; 
Number of risks identified: 2003: 21; 
Number of risks identified: 2004: 29; 
Number of risks identified: 85 (15%). 

Risk: Inadequate geographic staffing ; 
Number of risks identified: 2002: 30; 
Number of risks identified: 2003: 9; 
Number of risks identified: 2004: 5; 
Number of risks identified: 44 (8%). 

Risk: Inadequate resources (such as travel funds) for the certificate 
management team; 
Number of risks identified: 2002: 24; 
Number of risks identified: 2003: 19; 
Number of risks identified: 2004: 10; 
Number of risks identified: 53 (9%). 

Risk: Other; 
Number of risks identified: 2002: 28; 
Number of risks identified: 2003: 11; 
Number of risks identified: 2004: 13; 
Number of risks identified: 52 (9%). 

Total; 
Number of risks identified: 2002: 242; 
Number of risks identified: 2003: 170; 
Number of risks identified: 2004: 145; 
Number of risks identified: 557 (99%)[B]. 

Source: GAO analysis of FAA data. 

[A] The certificate management team may include the principal avionics, 
maintenance, and operations inspectors; aircraft dispatch inspector; 
and cabin safety inspector. 

[B] Percentages do not add to 100 due to rounding. 

[End of table] 

The process not only allows inspectors to identify the risk, but also 
to describe the severity of the risk in order to assist FAA 
headquarters in prioritizing its consideration of internal risks. 
Inspectors rate each risk they identify in terms of severity 
(negligible to catastrophic) and likelihood of occurrence (improbable 
to frequent). Risks are then categorized using a scale of high, medium, 
or low depending on the severity and likelihood of occurrences. For 
example, risks rated high are those that have high levels of likelihood 
and severity. As shown in table 2, over a 3-year period, 125 out of 557 
risks were rated as high severity, which could help FAA determine which 
issues most need targeting. Among the risks rated as high, training and 
inadequate staffing were identified most frequently, accounting for 34 
percent and 31 percent, respectively. As of May 2005, about 87 percent 
of identified risks had been closed, including about 90 percent of the 
risks identified as high. Risks are closed either because action has 
been taken to mitigate the risk or FAA has determined that the level of 
risk is acceptable and no action is warranted. 

Table 2: Severity of Internal Risks Identified by FAA Inspectors, 
Fiscal Years 2002-2004: 

Severity of risk: High; 
Number of risks: 2002: 37; 
Number of risks: 2003: 58; 
Number of risks: 2004: 30; 
Number of risks: Total: 125 (22%). 

Severity of risk: Medium; 
Number of risks: 2002: 183; 
Number of risks: 2003: 88; 
Number of risks: 2004: 107; 
Number of risks: Total: 378 (68%). 

Severity of risk: Low; 
Number of risks: 2002: 17; 
Number of risks: 2003: 22; 
Number of risks: 2004: 8; 
Number of risks: Total: 47 (8%). 

Severity of risk: Unknown; 
Number of risks: 2002: 5; 
Number of risks: 2003: 2; 
Number of risks: 2004: 0; 
Number of risks: Total: 7 (1%). 

Total; 
Number of risks: 2002: 242; 
Number of risks: 2003: 170; 
Number of risks: 2004: 145; 
Number of risks: Total: 557 (99%)[A]. 

Source: GAO analysis of FAA data. 

[A] Percentages do not add to 100 due to rounding. 

[End of table] 

An additional strength of the internal risk identification process is 
that FAA has established a chain of command for responding to these 
internal risks at the appropriate level. For example, local training 
issues are likely to be handled by field offices, geographic support 
issues would normally be handled by regional offices, and issues that 
cannot be resolved by these field and regional offices would be 
referred to headquarters. 

FAA's Oversight Process Includes Monitoring to Follow Up on Airline 
Actions Taken in Response to Findings: 

Another strength of FAA's oversight of non-legacy airlines is that 
inspectors monitor the actions that those airlines have taken in 
response to inspection findings through subsequent inspections and 
participation in safety partnership programs with the airlines. 
However, the inspectors' monitoring process could be improved by better 
data, as we discuss later in this report. SEP requires inspectors to 
monitor the actions taken by non-legacy airlines that will address the 
risks identified during an inspection and to verify that the actions 
taken have resulted or will result in resolution of the problems. This 
requirement addresses a concern of ours that the NPG inspection program 
did not often ensure that the agency follow-up actions corrected 
problems once they were identified.[Footnote 19] In addition, FAA's 
requirement for monitoring and verifying corrective action conforms to 
a management control standard on the need for federal agencies to have 
monitoring and verification policies or procedures for ensuring that 
the findings of audits and other reviews are promptly resolved. FAA 
inspectors monitor and confirm that corrective action has been taken by 
either conducting a special inspection of the area, following up during 
a subsequent inspection, or reviewing airline documentation that 
addresses the deficiency. According to inspectors and field office 
managers, the verification method used is based in many instances on 
the severity of the deficiency, with special inspections conducted for 
areas that could affect air safety. 

According to some FAA inspectors and airline officials, the regular 
meetings held to discuss concerns about airlines have opened up 
communications both within FAA and between FAA and the airlines. Lack 
of communication among the airlines and FAA was, according to the 
director of FAA's Flight Standards Service, historically a significant 
cause of airline safety problems. FAA inspectors and managers noted 
that they maintain an open line of communication with airline 
management by meeting regularly with key airline officials (such as 
directors of operations and directors of maintenance) to discuss the 
results of the SEP risk analyses and the airline's proposed 
resolutions. 

FAA's Oversight Process Is Hindered by Certain Program Weaknesses and 
Human Capital Management Challenges: 

FAA's oversight of non-legacy airlines is hindered by the incomplete 
implementation of its system safety approach and other challenges. 
FAA's incorporation of system safety into its oversight process is 
incomplete, as the agency continues to emphasize a nonsafety system- 
based process, NPG, to identify most of its inspection activities. 
Human capital management challenges also impede FAA's oversight. These 
challenges include a hiring freeze that began in January 2005; 
attrition; in certain cases, the reallocation of inspectors' duties as 
airlines are moved to the ATOS program; insufficient training for 
inspectors on risk management, a key element of SEP; and insufficient 
communication between FAA headquarters and field offices. In addition, 
FAA lacks a process to continuously evaluate SEP. Finally, the agency's 
PTRS database lacks some important inspection-related information, such 
as whether risks have been mitigated, that would aid in targeting 
further oversight activities as well as a nationwide analysis of 
inspection results. 

Most Inspection Activities Are Not Prioritized Based on a Structured 
Risk Assessment Process: 

We found that NPG remains the primary basis for FAA's inspection of non-
legacy airlines. For fiscal years 2002 through 2004, about 77 percent 
of the required inspection activities for the top 25 non-legacy 
airlines were initiated through the NPG annual planning session, 
compared with 23 percent that were SEP-initiated. As a result, most of 
the required inspection activities are not prioritized based on risk. 
(See table 3.) In addition, the total number of required inspections 
for non-legacy airlines declined during the 3-year period, while the 
number of passengers on those airlines grew from 122 million to 200 
million annual enplanements. The decline in inspections during this 
period may be due, in part, to the movement of three airlines from SEP 
to ATOS. FAA headquarters officials did not know why this decline 
occurred, but told us that it may be due to agency efforts to eliminate 
unnecessary activities and shift from a "quality control" oversight 
approach, in which individual products or outputs are inspected to 
determine if they meet specifications, to a "quality assurance" 
approach, in which airlines' processes are inspected. While NPG also 
includes planned inspections, which have a risk-based element, these 
inspections lack the structured approach to risk identification that 
SEP has.[Footnote 20] FAA officials acknowledged that the NPG planned 
inspections would benefit from the use of risk management tools. 

Table 3: SEP-and NPG-Initiated Required Inspections for the Top 25 Non- 
legacy Airlines, Fiscal Years 2002-2004: 

SEP-initiated; 
2002: 1,261; 
2003: 1,567; 
2004: 927; 
Total: 3,755 (23%). 

NPG-initiated; 
2002: 5,470; 
2003: 3,623; 
2004: 3,338; 
Total: 12,431 (77%). 

Total; 
2002: 6,731; 
2003: 5,190; 
2004: 4,265; 
Total: 16,186 (100%). 

Source: GAO analysis of information from FAA's PTRS database. 

[End of table] 

According to FAA officials, there are no minimum numbers or percentages 
of required SEP inspections. Therefore, a large percentage of NPG- 
initiated activities is not unexpected or negative in their view. FAA 
officials said NPG-initiated activities are an important part of the 
inspection process because they provide information to the agency on an 
annual basis for certain types of inspections and enable periodic 
assessments of key programs in an airline's operations. According to 
FAA, NPG is organized to systematically validate on an annual basis the 
performance of all of an airline's safety-critical programs. FAA 
considers this type of recurring assessment to be an important system 
safety principle and believes that it provides an opportunity to 
discover and correct latent failures before they cause safety problems. 
In addition, according to FAA officials, NPG identifies a minimum 
number of inspections that they believe need to occur to ensure that 
certain areas of an airline are reviewed, unlike SEP, which does not 
provide the built-in assurance that these areas will be inspected 
eventually. 

We agree with FAA officials that the agency should not establish 
minimum numbers or expected percentages of SEP inspections. However, we 
found that the large percentage of required NPG activities compared 
with SEP activities occurs, in part, because FAA's guidance places a 
greater emphasis on completing required NPG activities. This situation, 
along with other factors we discuss below, may deter inspectors from 
identifying additional inspections through SEP. FAA's guidance includes 
the following: 

* Field offices must complete 100 percent of the required NPG- 
identified activities by the end of each fiscal year[Footnote 21] 
(unless they are retargeted).[Footnote 22] 

* Field offices are not required to complete SEP-initiated activities 
by the end of the fiscal year and may reschedule them to the following 
fiscal year. 

In addition, the budget given to field offices for inspections is based 
on required NPG-initiated activities, and funding to cover SEP- 
initiated activities must come out of that budget. Inspectors have 
indicated concern regarding resources for inspections; for example, a 
lack of resources, including a lack of travel funds to conduct 
inspection activities, made up about 10 percent of the internal risks 
identified by inspectors through SEP during fiscal years 2002 through 
2004. According to headquarters officials, requests for funding high- 
risk SEP-identified inspections are always approved; however, some 
noncritical activities may be delayed due to budget constraints. For 
example, they said that some certification work--such as certifying 
that new aircraft or aircraft parts meet FAA standards--is being 
delayed until an inspector travels to the airline to perform another 
activity and can do the certification at little or no extra cost. 

Under FAA's guidance, if inspectors identify a risk under SEP that they 
wish to mitigate through an additional inspection activity, they must 
either add the SEP-initiated activity to the list of required NPG- 
initiated activities or retarget a required NPG activity and replace it 
with the SEP-initiated activity. Adding SEP-initiated activities to the 
required NPG-identified activities may increase the overall workload of 
inspectors. This situation may affect the number of additional 
activities that inspectors identify during the SEP meetings. In fact, 
we found that in some cases, no SEP-initiated activities were included 
with NPG-initiated activities as part of an airline's overall required 
inspections. For example, for one airline overseen by FAA's Central 
Region, PTRS data show that, in fiscal year 2002, inspectors conducted 
about 400 required NPG-initiated activities and no SEP-initiated 
activities. According to FAA headquarters officials, many inspectors 
view SEP-initiated activities as additional work. This view was also 
held by some inspectors we spoke with. 

Moreover, our analysis of PTRS data indicates that retargeting occurs 
infrequently. In fiscal year 2002, about 5 percent of all NPG-required 
inspections were retargeted by inspectors and 3 percent were retargeted 
in fiscal years 2003 and 2004. This low rate occurs, in part, because 
field offices have interpreted FAA's emphasis on NPG inspections as 
discouraging retargeting. For example, three of the principal 
inspection teams for the 16 airlines told us they do not retarget any 
NPG activities. Further, in 5 of the 13 field offices we visited, we 
were told by both managers and inspectors that they believe NPG 
activities take priority over all but the most risky situations 
identified through SEP. Similarly, an official with the Professional 
Airways Systems Specialists, a union that represents many FAA safety 
inspectors, told us that its members believe that FAA's management does 
not give as much attention and priority to SEP as it does to ATOS. 

FAA's emphasis on required NPG-initiated inspections is also shown 
through its data-tracking efforts. According to FAA officials, for 
fiscal years 2002 through 2004, all required NPG activities had been 
completed by the end of the year. FAA did not know how many completed 
SEP-initiated activities were mitigated or required further inspection 
activities that were then carried over to the next fiscal year because 
it does not collect such information nationwide in PTRS. Moreover, 
field office managers also rely on PTRS, which lacks information on the 
number of activities that are carried over. For example, one field 
office supervisor told us that one inspector had carried over 19 
activities from fiscal year 2004 to fiscal year 2005. However, the 
field office supervisor said it was difficult to determine how many 
additional activities were carried over because PTRS does not track 
this information. 

Shifts in Inspectors' Workload Present a Challenge to FAA's Inspection 
Oversight Process: 

The number of FAA inspectors available to oversee non-legacy airlines 
will be affected by recent and anticipated trends in attrition and a 
hiring freeze. First, for fiscal year 2004, the number of inspectors 
(those assigned to non-legacy airlines, ATOS, and general aviation) who 
left FAA was greater than the number of inspectors hired, resulting in 
a net loss of 67 inspectors. Second, the number of inspectors of 
commercial airlines (both ATOS and non-legacy) is expected to continue 
to decline from 1,810 at the end of fiscal year 2004 to 1,704 at the 
end of fiscal year 2005. Third, from fiscal years 2006 to 2007, the 
agency anticipates the number of inspectors of non-legacy airlines will 
decline from 1,119 to 1,088. Fourth, for fiscal years 2005 through 
2010, FAA estimated that over 1,100 inspectors of non-legacy airlines 
and general aviation will leave the agency, with an average loss due to 
attrition of about 195 inspectors per year. 

FAA will have to hire inspectors to offset the anticipated losses. 
However, the agency put a hiring freeze in place in January 2005 for 
budgetary reasons, which prevents the agency from hiring inspectors to 
replace those lost due to attrition unless they are deemed critical. 
Furthermore, in order to absorb a budget reduction in fiscal year 
2005,[Footnote 23] FAA expects to eliminate about 200 inspector 
positions in the Office of Aviation Safety. These reductions will be 
accomplished through attrition. In fiscal year 2006, the office expects 
the staffing level to increase by about 80, which would still be below 
the fiscal year 2004 staffing level.[Footnote 24] FAA is filling safety-
critical positions that become vacant through internal appointments. 
According to FAA officials, principal inspectors and managers are 
considered safety critical. As other safety inspectors, including 
geographic inspectors, leave the agency or are appointed to safety-
critical positions, they are not being replaced at this time, according 
to those officials. Moreover, their workload is being divided among the 
remaining inspectors, thereby increasing the inspectors' workload, 
which has resulted in other, less critical, work being delayed or 
deferred. In 11 of the 13 FAA field offices we contacted, officials 
indicated a shortage of different types of inspectors needed to oversee 
the non-legacy airlines. For example, the Washington, D.C., field 
office did not have a principal avionics inspector. Officials with FAA 
field and regional offices said they have developed ways to deal with 
the inadequate staffing of inspectors in their offices. For example, 
the Southern Regional Office told us that it uses geographic inspectors 
more frequently than in the past to help conduct inspections rather 
than having the inspectors assigned to the non-legacy airline travel to 
conduct them. However, two field offices---Chicago and Minneapolis---
indicated the need for more geographic inspectors. Another way that FAA 
offices are dealing with inspector shortages is by having an inspector 
perform inspections for several offices. For example, the Atlanta field 
office has been without a cabin safety inspector for a year and uses 
the inspector assigned to the Southern Regional Office. According to an 
FAA inspector, some inspections are delayed due to lack of staff 
availability. In addition, according to our analysis of FAA data 
(discussed earlier in this report), for fiscal years 2002 through 2004, 
principal inspectors identified lack of staff for the airlines' 
certificate management teams as among the top two risks for each year, 
and overall, more than one-third of the risks identified were related 
to lack of inspectors. 

The move of additional airlines into ATOS, which requires more 
inspectors per airline than SEP, also affects the workload of 
inspectors for non-legacy airlines. In addition, unlike SEP inspectors, 
ATOS inspectors are dedicated to an airline and generally cannot be 
used to conduct inspections of other entities. When four airlines were 
recently transitioned into ATOS, the total size of the four inspection 
teams increased 30 percent, from 73 to 95 inspectors, as shown in table 
4. As a result, the number of inspectors available to oversee non- 
legacy airlines was reduced not only by those 73 who had covered the 
four airlines while they were under the NPG and SEP processes (who 
would have also had responsibilities for other entities) but by an 
additional 22 inspectors as well. 

Table 4: Allocation of FAA Inspection Staff under SEP and ATOS for Four 
Airlines: 

Airline: Champion; 
Number of FAA inspectors under SEP: 4; 
Number of FAA inspectors under ATOS: 6; 
Change in number of inspectors: +2. 

Airline: American Eagle; 
Number of FAA inspectors under SEP: 31; 
Number of FAA inspectors under ATOS: 35; 
Change in number of inspectors: +4. 

Airline: ExpressJet; 
Number of FAA inspectors under SEP: 18; 
Number of FAA inspectors under ATOS: 24; 
Change in number of inspectors: +6. 

Airline: SkyWest; 
Number of FAA inspectors under SEP: 20; 
Number of FAA inspectors under ATOS: 30; 
Change in number of inspectors: +10. 

Total; 
Number of FAA inspectors under SEP: 73; 
Number of FAA inspectors under ATOS: 95; 
Change in number of inspectors: +22. 

Source: FAA. 

[End of table] 

FAA is aware that transitioning additional inspectors to airlines in 
ATOS is putting a burden on the workload of those inspectors who 
continue to inspect non-legacy airlines, and the agency has, in one 
case, revised its procedures for dedicating staff to individual ATOS 
airlines. When Champion Airlines became part of ATOS in January 2005, 
FAA changed ATOS policy to allow the Northwest Airlines inspection team 
to share its data analyst and manager with the Champion inspection 
team. FAA officials believe that sharing staff resources will address 
the issue of staff shortages. 

All of the inspectors at the offices we contacted had other duties in 
addition to overseeing non-legacy airlines. The inspectors performed 
activities such as certifying and approving new aircraft types; 
overseeing repair stations and aviation schools; reviewing FAA 
directives and aircraft updates; responding to complaints; 
investigating accidents, incidents, and complaints; overseeing 
designees;[Footnote 25] and attending training. Although primarily 
responsible for a particular airline, some principal inspectors 
conducted inspections of other airlines and entities. For example, a 
principal inspector in the Great Lakes Region was responsible for 
overseeing three repair stations in addition to a non-legacy airline. 
In addition, a principal operations inspector in the Southern Region 
was also responsible for overseeing designees such as pilot examiners. 
Our analysis of the inspectors' workload contained in the PTRS database 
showed that for fiscal years 2002 through 2004, about 75 percent of FAA 
inspectors had responsibility for more than 3 entities, and about half 
had responsibility for more than 15. While the size and complexity of 
the various entities will dictate the inspector workload, having 
responsibility for a large number of entities can increase an 
inspector's workload because of the need to be familiar with the 
operating procedures of each entity as well as to spend time physically 
inspecting each entity. 

Inspectors Are Concerned That Training Limitations Present a Challenge 
to Inspection Oversight Process: 

While FAA provided system safety training to its inspector workforce, 
as mentioned earlier in this report, the lack of sufficient training in 
other areas was identified by principal inspectors during their risk 
analysis meetings and by inspectors we spoke with.[Footnote 26] Our 
analysis of FAA's database of internal risks identified by principal 
inspectors of non-legacy airlines indicated that 42 percent (233 out of 
558) of the internal risks identified for fiscal years 2002 through 
2004 related to training. Although FAA policy requires that principal 
inspectors assigned to non-legacy airlines complete technical training 
on the airline's specific policies, procedures, and equipment before 
they can conduct airline inspection activities, principal inspectors 
identified risks associated with the lack of such training. For 
example, our analysis of the internal risk database found that 
principal inspectors identified a lack of technical training for 
themselves or other inspectors on their team on specific aircraft, 
engine types, navigation equipment, and avionics that pertained to the 
airlines they were overseeing.[Footnote 27] Others cited a lack of 
training in cargo loading, hazardous material identification, or weight 
and balance. Still others indicated the need for training in FAA's 
systems or processes. For example, several indicated that they had not 
received the basic courses for principal inspectors (such as 
"foundation for principal inspectors"). As another example, several 
principal inspectors reported that certain inspectors on their team 
lacked training in using the computer system that analyzes information 
collected in PTRS, noting that without this training they were unable 
to directly access the system. Even though inspectors expressed these 
concerns, approximately 87 percent of identified internal risks have 
been closed by FAA, according to our analysis of FAA data. However, we 
could not determine how many of those risks were actually mitigated, as 
FAA closes risks either when action has been taken to mitigate the risk 
or because the agency has determined that the level of risk is 
acceptable and no action is warranted. 

During our field office visits, inspectors also spoke about the lack of 
technical training in certain areas. For example, inspectors told us 
they had not received training on some newer aircraft they were 
responsible for inspecting and that they were not always able to attend 
free training provided by the manufacturers, which they felt would help 
them obtain needed technical knowledge. We found instances where 
management acknowledged that such training was needed. In the past, 
concerns had been raised that FAA's acceptance of free training from a 
regulated entity might have the appearance of a conflict of interest. 
In September 2005, we reported that FAA accepts such training under 
limited circumstances.[Footnote 28] FAA has established safeguards to 
help preclude the appearance of a conflict of interest when FAA accepts 
training from the entities it regulates at no cost to the agency or in 
exchange for an in-kind service.[Footnote 29] 

In addition, the principal inspectors we spoke with indicated that some 
geographic inspectors who assist them in providing oversight of non- 
legacy airlines lack the technical and SEP-specific procedural training 
to maximize their usefulness. Several field office managers and 
principal inspectors told us that geographic inspectors needed to be 
knowledgeable about the operations of airlines they inspect in order to 
provide effective assistance. Several principal inspectors indicated 
that misperceptions about an airline's operating requirements are not 
uncommon because geographic inspectors often lack training on the 
aircraft they inspect.[Footnote 30] As a result, the inspectors 
indicated that geographic inspectors had at times incorrectly coded 
inspection information in PTRS and erroneously identified risks, which 
resulted in principal inspectors having to spend time determining that 
a problem did not actually exist. In addition, several airline 
officials told us that geographic inspectors had incorrectly identified 
problems and appeared to not understand airlines' operational 
procedures. For example, we were told that a geographic inspector 
improperly cited an airline for not complying with a deicing program 
that the airline had previously informed FAA inspectors it was 
changing. FAA headquarters officials told us that it is not feasible to 
train all geographic inspectors on all airlines they may encounter. 
Further, they noted it would be good for geographic inspectors to 
contact principal inspectors when they are unsure of whether an action 
or condition they observe is a problem. However, FAA headquarters has 
not provided that instruction to the geographic inspectors. Industry 
representatives also told us this was a problem that could be 
alleviated by inspectors having access to airlines' operating manuals 
online. 

Another issue identified as a problem by principal inspectors we spoke 
with is that not all geographic inspectors are provided training in 
SEP, including how to use the uniform coding scheme that was developed 
for SEP to facilitate the identification and analysis of risks. 
Geographic inspectors may therefore note the results of their 
inspections through narrative in the comments section of PTRS rather 
than through SEP codes, so that the principal inspectors must read 
through all the comments in order to identify geographic inspectors' 
concerns. As PTRS may contain thousands of entries on one airline each 
year, reading every narrative comment can be time consuming and 
difficult to interpret. The lack of coding of geographic inspectors' 
findings could therefore result in principal inspectors missing 
problems or risks identified by geographic inspectors. We do not know 
how often this problem occurs, as FAA has not assessed the reliability 
of this data in PTRS. However, several FAA staff brought this issue to 
our attention. One field office manager and SEP coordinator told us 
that PTRS entries by geographic inspectors are not helpful because they 
do not provide meaningful comments for FAA's risk assessments, and 
several principal inspectors told us that training for geographic 
inspectors on SEP coding would help them more easily identify risks 
during their periodic meetings. 

These training issues may arise in part because FAA has not 
systematically identified technical training needs for principal or 
other inspectors. Our previous report found that FAA has not 
systematically identified overall training needs of its inspectors to 
ensure that the curriculum addresses the unique training needs of each 
type of inspector.[Footnote 31] Instead, course development focuses on 
individual courses. FAA said it recognizes that it manages courses as 
individual components of an overall curriculum that is only loosely 
defined and that it needs to develop courses and address training needs 
as part of an overall curriculum. It has established a curriculum 
transformation plan that it estimates it will fully implement in 2008. 

In responding to our survey, inspectors had mixed views on the 
usefulness of technical training.[Footnote 32] For about three- 
quarters[Footnote 33] of the responses, roughly equal percentages of 
inspectors of non-legacy airlines responded that the technical training 
they received in the last 2 years[Footnote 34] helped them do their 
current jobs to some extent, to a moderate extent, or to a great 
extent. On the other hand, 6 percent indicated that the technical 
training had not helped them do their jobs and another 26 percent 
indicated it had only helped to some extent. (See fig. 3.) 

Figure 3: Non-legacy Airline Inspectors' Views on the Extent Technical 
Training Received in the Last 2 Years Has Helped Them Do Their Jobs: 

[See PDF for image] 

Note: Approximately 10 percent of inspectors responded that they had no 
basis to judge or did not know. 

[End of figure] 

Further, the timeliness of certain training was identified as a concern 
by both principal inspectors we spoke with and inspectors responding to 
our survey. Specifically, some principal inspectors we spoke with 
indicated a lack of timeliness for SEP training. Since FAA's 
introduction of the SEP training for non-legacy inspectors in 2001, a 
total of 700 inspectors have received the training---just over 300 
inspectors between 2001 and 2002 and almost 400 inspectors between 2003 
and 2004. This means that less than 40 percent of the approximately 
1,800 inspectors of both ATOS[Footnote 35] and non-legacy airlines (as 
of fiscal year 2004) could have received SEP training. FAA requires all 
principal inspectors and other inspectors who are part of certificate 
management teams to receive SEP training. Geographic inspectors are not 
part of those teams and may not receive that training. While all of the 
principal inspectors who were on staff when SEP was implemented in 2002 
received initial SEP training, according to FAA, since that time, newly 
assigned inspectors may not have received the training before beginning 
their responsibilities. Four principal inspectors who were assigned 
relatively recently to non-legacy airlines and were not provided the 
initial SEP training told us they had received subsequent training 
given to all inspectors to explain revisions to the SEP program. They 
said that the lack of initial SEP training hindered their ability to 
understand SEP terms and processes and participate fully in the 
periodic meetings to analyze information from inspections to identify 
risks. Those inspectors also believed that the subsequent SEP training 
was not as helpful without the initial SEP training. 

The views of the inspectors we spoke with are consistent with our 
survey results, which indicated that many inspectors of non-legacy 
airlines are not highly satisfied with the timeliness of technical 
training they say they need to do their jobs. Our survey found that 
only about 16 percent of non-legacy inspectors believe to a great or 
very great extent that they have received technical training in a 
timely manner to do their jobs. Moreover, about the same percentage (15 
percent) of the inspectors reported that the timing of their training 
had not been useful to their jobs at all. (See fig. 4.) 

Figure 4: Views of Inspectors of Non-legacy Airlines on the Extent 
Inspectors Have Received Technical Training in a Timely Manner during 
Their FAA Careers: 

[See PDF for image] 

Note: Approximately 1 percent of inspectors responded that they did not 
know. 

[End of figure] 

About one-third of the inspectors[Footnote 36] in our survey indicated 
that the training they received in the automated systems used in the 
inspection process--including PTRS--was provided in a timely manner to 
a moderate extent. Another 30 percent thought the training was only 
somewhat timely.(See fig. 5.) Timely training on automated systems is 
important in order to have accurate data. Inaccurate PTRS data hinder 
the identification of risks by principal inspectors, as we discussed 
previously in this report. 

Figure 5: View of Inspectors of Non-legacy Airlines on the Extent to 
Which They Received Training in Automated Systems Used in the 
Inspection Process in a Timely Manner: 

[See PDF for image] 

Note: Approximately 1 percent of inspectors responded that they did not 
know. 

[End of figure] 

FAA Lacks Effective Communication in How Internal Risks Are Resolved: 

Some inspectors told us that when they identified internal risks to 
FAA, they were able to see the results of actions taken by their field 
and regional offices to address the issues, but that they did not know 
what actions, if any, were taken by FAA headquarters. This situation 
indicates the lack of a key management control that calls for 
information to be recorded and communicated to individuals within an 
entity who need the information in a form and within a time frame that 
enable them to carry out their responsibilities. However, unless the 
inspectors have access to a secure intra-agency Web site and know how 
to locate the issue they submitted, they do not know how FAA 
headquarters has responded to the risks because this information is not 
directly provided to them by headquarters. Headquarters officials 
acknowledged that there is no formal feedback loop by which to inform 
the inspectors about the issues that they raised. In prior work, we 
found that a lack of communication with inspector staff on the 
resolution of enforcement actions that resulted from inspections is 
frustrating to inspectors and might be a disincentive to reporting 
violations they find during inspections.[Footnote 37] This suggests 
that a similar lack of communication could reduce inspectors' interest 
in identifying internal risks to FAA. We found that the overall number 
of times internal risks were identified by inspectors declined for 
fiscal years 2002 through 2004. 

PTRS Data Are Missing Elements That Would Make the Data More Useful: 

The PTRS database has limitations that reduce the usefulness of the 
data in helping inspectors ensure that they are effectively conducting 
inspections. PTRS does not facilitate FAA's ability to monitor whether 
risks identified through SEP have been mitigated. While the SEP process 
generates required inspection activities based on risks that have been 
identified through analysis of prior inspections, inspectors record 
these inspection activities as closed in the PTRS database once they 
have completed the inspection, whether or not the risks identified in 
SEP have been mitigated, according to some principal inspectors we 
spoke with. While inspectors are expected to use the comments section 
in PTRS to record additional information, such as whether identified 
risks have been mitigated, they do so infrequently. Our analysis of 
about 1.8 million PTRS records showed that fewer than 400 records 
included any form of the words "mitigate," "fix," or "resolve." 
Moreover, such analysis is time consuming to conduct because a uniform 
coding scheme is not employed for entering the information. As a 
result, managers are not easily able to use PTRS data to monitor 
whether risks identified through SEP have been mitigated and lack data 
that would be useful in evaluating SEP. It is left to inspectors to 
determine subsequently if the risks have been mitigated, usually 
through follow-up on-site inspections. 

In addition, while a specific risk identified through SEP might result 
in multiple inspections, there is no clear linkage between these 
inspections when they are recorded in PTRS, according to a regional SEP 
manager. For example, if a risk was identified through SEP-generated 
inspection activities that included looking at manuals, checking 
aircraft records, and performing a ramp check, inspectors would have to 
review all comment records to identify the resulting activities in PTRS 
that are related to that particular risk. This lack of linkage in PTRS 
may reduce inspectors' overall understanding of an airline's situation. 
Inspectors who do not know why activities have been generated may not 
target or fit their work to assess these identified risks, while a 
greater understanding of how inspection activities are connected to 
identified risks could help ensure that inspections are conducted in a 
way that maximizes the benefits of a risk-based approach. 

FAA Has Not Evaluated Its Inspection Oversight Process or Linked It to 
Agency Goals: 

FAA has not evaluated its inspection oversight process for non-legacy 
airlines to determine how the process contributes to the agency's 
mission and overall safety goals. In addition, FAA has not explicitly 
linked SEP to the overall safety goals. According to management control 
standards for federal agencies and our prior reports on results- 
oriented management,[Footnote 38] federal agencies should establish 
measurable performance goals for their programs and operations. 
Agencies should also have an evaluation process for their programs so 
that agency officials, Members of Congress, and others will be able to 
determine whether goals are being achieved. 

While FAA has not established a specific goal for SEP, it has an 
overall goal to achieve the lowest possible accident rate and 
constantly improve safety. To address this goal, FAA's strategy 
includes expanding cost-effective safety oversight and surveillance by 
targeting its inspection resources better. However, FAA's strategy does 
not explicitly show how SEP contributes to the safety goal. In 
addition, FAA has not yet evaluated SEP to determine if it is achieving 
the agency's goal. As a consequence of not having an evaluative process 
for SEP, FAA does not have the information it needs to determine what 
changes should be made to improve its system safety process. Also, FAA 
does not have a process to examine the nationwide implications of or 
trends in the risks that inspectors have identified through their risk 
assessments. Consequently, FAA does not have the information it needs 
to proactively determine on a continuous basis risk trends at the 
national level. While FAA has an evaluation office, this office is not 
doing this type of analysis; rather, the office conducts analyses of 
the types of inspections generated under SEP by airline and FAA region, 
according to the manager of that office. One FAA field office we 
contacted had taken the initiative to analyze risks identified during 
the SEP meetings for trends. In FAA's Eastern Region, a manager 
routinely analyzes trends in types of findings within an airline and 
across airlines and provides this information to the inspectors. This 
is an informal process, and it is not certain that it will continue 
since the manager has been reassigned to another office. Moreover, such 
analyses at the local or regional level do not fulfill the management 
control to have a process in place to determine if program goals are 
being achieved. 

In addition, we found that while FAA has created specific national 
goals for NPG that call for completing all required inspections each 
year, in some cases these goals impede the agency's ability to 
effectively implement its system safety approach. For example, in order 
to meet the annual goal, the regional offices have established interim 
goals to complete 25 percent of the required inspections each quarter. 
Inspectors in one field office told us that this situation created a 
disincentive for identifying additional activities under SEP due to 
concerns about completing the quarterly requirements. Those inspectors 
also noted that these goals encouraged them to prioritize their work 
based on what they can most easily accomplish, rather than on what 
represents the most significant risks--the antithesis of a system 
safety approach. 

Conclusions: 

Our review of FAA's oversight of non-legacy airlines suggests that the 
full benefits of a system safety approach can be realized only if the 
approach is more fully implemented and utilized. FAA's articulation of 
its system safety approach, its application of system safety principles 
to its oversight process through SEP, and its widely distributed 
training on system safety are positive steps toward improving oversight 
by using the advantages of system safety--particularly the ability to 
identify and prioritize inspections based on risk. Fully developing SEP 
is important since FAA has been unable to move significant numbers of 
airlines to ATOS and has only recently established the goal to do so by 
the end of fiscal year 2007. A process such as SEP is needed to 
identify risks in and among non-legacy airlines and system-wide 
problems. However, the usefulness of FAA's system safety approach is 
reduced by limitations in the implementation of SEP--such as FAA 
headquarters' predominant focus on NPG, which has led to only a small 
percentage of inspection activities being SEP-initiated; the lack of 
training of geographic inspectors on SEP codes that could make their 
inspections more useful to identifying risks during the SEP process; 
and the lack of linkage to national goals and evaluations of SEP. Until 
SEP is more fully implemented, it is clear that FAA's approach to 
overseeing non-legacy airlines is not largely risk based. Unless SEP is 
more thoroughly integrated into its oversight, FAA may not be fully 
maximizing the efficiency of its inspections in identifying and 
mitigating risks in order to ensure the safety of non-legacy airlines. 
With FAA operating under a hiring freeze and the number of inspectors 
available for non-legacy airlines possibly further reduced by attrition 
and the move of inspection staff to the ATOS program, the need to 
maximize the effectiveness of inspection activities in ensuring the 
safety of non-legacy airlines is even more critical. 

Recommendations for Executive Action: 

To improve the effectiveness of the agency's oversight of non-legacy 
airlines, we recommend that the Secretary of Transportation direct the 
FAA Administrator to implement the following four recommendations: 

* To improve its safety oversight of airlines, FAA should (1) develop a 
continuous evaluative process for its activities under SEP and link SEP 
to the performance-related goals and measures developed by the agency, 
track performance towards agency goals, and determine appropriate 
program changes. The evaluation should include an analysis of 
inspection findings to identify trends and risks at the national level. 

* In order to ensure that all regional and field offices have a 
complete and timely understanding of FAA's policies relating to the 
inspection process, FAA needs to (2) improve communication in areas 
such as whether and how internal risks identified by inspectors have 
been resolved and (3) improve training in areas such as risk 
management, coding items in the PTRS database, and how and under what 
circumstances SEP-identified activities can replace NPG-identified 
activities through retargeting. 

* To better utilize geographic inspectors' support, FAA needs to (4) 
improve the geographic inspectors' understanding of the system safety 
approach and operations of the airlines they inspect. FAA should 
consider actions such as additional training, additional oversight in 
particular areas, having airlines' operating manuals available online 
for review by inspectors, and improving communication between 
geographic inspectors and principal inspectors on issues related to 
identifying safety violations. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to DOT for review and comment. We 
obtained oral comments from FAA and DOT officials, including FAA's 
Deputy Associate Administrator for Aviation Safety. In particular, FAA 
officials made the point that planned NPG activities have an element of 
risk identification. We agree and revised the report. The FAA officials 
generally agreed with our recommendations to improve communication, 
training, and geographic inspectors' understanding of the system safety 
approach. The officials said that FAA would consider our recommendation 
to develop a continuous evaluative process for SEP and link it to the 
agency's goals, but that its plan to eventually place the remaining non-
legacy airlines in the ATOS program might make this recommendation 
unnecessary. In the past, FAA's efforts to move airlines to ATOS have 
experienced delays, therefore, we retained this recommendation. FAA 
officials also provided clarifying comments and technical corrections, 
which we incorporated as appropriate. 

As agreed with your office, unless you announce the contents of this 
report earlier, we plan no further distribution until 14 days from the 
report date. At that time, we will send copies of this report to other 
congressional committees, the Secretary of Transportation, and the 
Administrator, FAA. We will also make copies available to others upon 
request. In addition, the report will be available at no cost on GAO's 
Web site at [Hyperlink, http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-2834 or [Hyperlink, dillinghamg@gao.gov]. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who 
made major contributions to this report are listed in appendix III. 

Signed by: 

Gerald L. Dillingham, Ph.D. 
Director, Physical Infrastructure Issues: 

[End of section] 

Appendixes: 

Appendix I: Objectives, Scope, and Methodology: 

The objective of this report is to assess the Federal Aviation 
Administration's (FAA) processes for ensuring the safety of a fast- 
growing portion of the commercial airline industry--the non-legacy 
passenger airlines.[Footnote 39] Specifically, this report responds to 
the following questions: (1) What are the strengths of FAA's inspection 
oversight of non-legacy passenger airlines? (2) What issues hinder the 
effectiveness of FAA's inspection approach? 

The scope of our review included FAA's oversight activities from fiscal 
year 2002--when FAA implemented a new process (the Surveillance and 
Evaluation Program or SEP) to its oversight of non-legacy airlines-- 
through fiscal year 2004. To address the two questions, we obtained and 
analyzed information about FAA's oversight of the top 25 non-legacy 
airlines ranked according to the number of enplanements in 2004. Those 
25 airlines are identified in table 5, and each was covered by SEP 
during the entire 3-year period or a portion of the period.[Footnote 
40] Together, these 25 airlines accounted for about 90 percent of all 
non-legacy passenger enplanements in 2004. 

Table 5: Number of Enplanements for the Top 25 Non-legacy Airlines, 
2004: 

Rank: 1; 
Airline: American Eagle[A]; 
Enplanements: 14,869,258. 

Rank: 2; 
Airline: ExpressJet[A]; 
Enplanements: 13,664,642. 

Rank: 3; 
Airline: SkyWest[A]; 
Enplanements: 13,417,720. 

Rank: 4; 
Airline: Airtran; 
Enplanements: 13,178,118. 

Rank: 5; 
Airline: Comair; 
Enplanements: 12,637,210. 

Rank: 6; 
Airline: JetBlue; 
Enplanements: 11,731,733. 

Rank: 7; 
Airline: Atlantic Southeast; 
Enplanements: 10,427,885. 

Rank: 8; 
Airline: American Trans Air; 
Enplanements: 10,340,914. 

Rank: 9; 
Airline: Mesa; 
Enplanements: 9,122,237. 

Rank: 10; 
Airline: Atlantic Coast; 
Enplanements: 7,046,971. 

Rank: 11; 
Airline: Air Wisconsin; 
Enplanements: 6,954,187. 

Rank: 12; 
Airline: Frontier; 
Enplanements: 6,437,921. 

Rank: 13; 
Airline: Pinnacle; 
Enplanements: 6,362,805. 

Rank: 14; 
Airline: Horizon; 
Enplanements: 5,930,448. 

Rank: 15; 
Airline: Chautauqua; 
Enplanements: 5,608,947. 

Rank: 16; 
Airline: Mesaba; 
Enplanements: 5,427,694. 

Rank: 17; 
Airline: Hawaiian; 
Enplanements: 5,234,766. 

Rank: 18; 
Airline: Spirit; 
Enplanements: 4,592,640. 

Rank: 19; 
Airline: Aloha; 
Enplanements: 4,187,019. 

Rank: 20; 
Airline: Trans States[A]; 
Enplanements: 3,462,869. 

Rank: 21; 
Airline: Executive; 
Enplanements: 2,796,163. 

Rank: 22; 
Airline: Midwest Express; 
Enplanements: 2,376,304. 

Rank: 23; 
Airline: PSA; 
Enplanements: 2,030,870. 

Rank: 24; 
Airline: Piedmont; 
Enplanements: 1,948,292. 

Rank: 25; 
Airline: Ryan International; 
Enplanements: 1,626,437. 

Source: GAO analysis of U.S. Department of Transportation information. 

[A] These airlines were transitioned into the ATOS program from 2003 
through 2005. 

[End of table] 

To determine FAA's legal oversight responsibility for commercial 
passenger airlines, we obtained and analyzed regulations that govern 
FAA oversight of these airlines. We also obtained and reviewed FAA 
handbooks, procedures, and orders that describe the role and 
responsibilities of FAA managers and inspectors[Footnote 41] in 
implementing the two inspection oversight processes for non-legacy 
airlines--SEP and the National Work Program Guidelines (NPG). We 
conducted literature searches and reviewed prior reports and articles 
on the oversight processes, including those prepared by the Department 
of Transportation's Inspector General and us. We reviewed documentation 
provided by FAA, and contacted officials there and at GAO, to determine 
whether findings identified in those reports have been addressed. We 
also reviewed literature on system safety and compared it with FAA's 
system safety framework, which is incorporated in SEP. 

We interviewed FAA headquarters officials from the Office of Aviation 
Safety and its Flight Standards Service to obtain descriptions of NPG 
and SEP. To understand how these processes were implemented, we 
collected information from 7 regional offices and 13 field and 
certificate management offices. We selected these offices because they 
have oversight responsibility for most of the top 25 non-legacy 
airlines ranked by the number of enplanements in 2004. We also 
interviewed officials from 16 of the 25 non-legacy airlines. The 
regional offices, field and certificate management offices, and 
airlines we interviewed are shown in table 6. For each FAA office, we 
conducted semistructured interviews with managers, SEP coordinators, 
and inspectors to obtain information on how they implemented NPG and 
SEP. We also collected information on inspector staffing levels, 
workload, and training. In addition, we reviewed whether management 
controls have been established for NPG and SEP,[Footnote 42] and 
determined if they were linked to FAA's overall safety performance 
goals. These controls included the (1) establishment of performance 
goals, (2) verification of inspection results, (3) management of the 
inspection database (Program Tracking and Reporting Subsystem or PTRS), 
and (4) communication among managers and inspectors relating to NPG and 
SEP. We also interviewed airline safety officials to obtain their views 
on FAA's inspection and surveillance activities and their participation 
in FAA-industry partnership programs. 

Table 6: FAA Offices and Airlines Interviewed by GAO: 

Regional office: Central; 
FAA field office: None; 
Airline: None. 

Regional office: Eastern; 
FAA field office: Baltimore Flight Standards District Office (FSDO); 
Airline: Piedmont. 

FAA field office: Garden City FSDO; 
Airline: JetBlue. 

FAA field office: Washington FSDO; 
Airline: Independence Air[A]. 

Regional office: Great Lakes; 
FAA field office: Detroit FSDO; 
Airline: Spirit. 

FAA field office: Indianapolis FSDO; 
Airline: American Trans Air. 
Airline: Chautauqua. 

FAA field office: Minneapolis FSDO; 
Airline: Mesaba. 

FAA field office: Chicago FSDO; 
Airline: Air Wisconsin. 
Airline: Midwest Express. 

Regional office: Northwest Mountain; 
FAA field office: None; 
Airline: None. 

Regional office: Southern; 
FAA field office: Atlantic Southeast (Atlanta) Certificate Management 
Unit; 
Airline: Atlantic Southeast. 

FAA field office: Comair (Louisville) Certificate Management Office 
(CMO); 
Airline: Comair. 

FAA field office: Orlando FSDO; 
Airline: Executive. 

FAA field office: Airtran (Orlando) CMO; 
Airline: Airtran. 

FAA field office: Memphis FSDO; 
Airline: Pinnacle. 

Regional office: Southwest; 
FAA field office: None; 
Airline: None. 

Regional office: Western Pacific; 
FAA field office: Honolulu FSDO; 
Airline: Aloha. 
Airline: Hawaiian. 

Source: GAO. 

[A] Formerly Atlantic Coast Airlines. 

[End of table] 

We also conducted interviews with seven industry organizations to 
obtain their views on FAA's oversight processes. (See table 7.) We 
selected these organizations because they represented diverse segments 
of the aviation industry. In addition, we selected organizations that 
met one or more of the following criteria: membership inspected by FAA, 
familiarity with FAA's safety oversight of airlines, familiarity with 
FAA's SEP or Air Transportation Oversight System processes, and 
familiarity with inspector training. 

Table 7: Organizations Interviewed by GAO: 

Type of organization: Trade associations and unions; 
Organization contacted: Aircraft Mechanics Fraternal Association; 
Professional Aviation Maintenance Association; Professional Airways 
Systems Specialists. 

Type of organization: Transportation safety organization; 
Organization contacted: National Transportation Safety Board. 

Type of organization: Organizations whose members are certificated by 
FAA and subject to FAA oversight; 
Organization contacted: Regional Airline Association; Aeronautical 
Repair Station Association; Airline Pilots Association International. 

Source: GAO. 

[End of table] 

We analyzed information about FAA's inspection activities for the top 
25 non-legacy airlines using the agency's PTRS database for fiscal 
years 2002 through 2004. Specifically, we analyzed information on the 
number of required NPG and SEP inspections, inspector workload (i.e., 
number and type of entities inspected), and number of NPG activities 
that were retargeted (i.e., NPG-identified activities that were 
replaced with SEP-identified activities). This analysis excluded 
enroute activities, which take place when an inspector monitors an 
aircraft as it travels from one destination to another. These 
inspections were excluded because inspectors often use these as 
secondary inspections as a means to travel to a location where their 
primary inspection is such as a repair station. In addition, we 
analyzed data that FAA maintains on the internal risks identified by 
inspectors through regular SEP meetings. For fiscal years 2002 through 
2004, we examined the type, severity, and closure status of the 
identified risks. We also analyzed comment fields in that data to 
determine if they indicated that the risks had been mitigated. 

To assess the reliability of the data used in this report, we 
interviewed knowledgeable agency officials about the data, performed 
electronic testing of relevant data fields for obvious errors in 
accuracy and completeness, and collected and reviewed documentation 
from data system managers about the data and the systems that produced 
them. We determined that the data were sufficiently reliable for the 
purposes of this report. 

To gather information about inspectors' perspectives on the technical 
training available to them, we conducted a Web-based survey of a 
representative sample of FAA safety inspectors. The survey asked a 
combination of questions concerning the amount and timeliness of 
training that allowed for open-ended and close-ended responses. We had 
no practical way to assess information on the amount of training 
necessary for inspector proficiency or the timeliness of the training 
provided. We drew a stratified random probability sample of 496 
inspectors from the population of 2,989 FAA aviation safety 
inspectors.[Footnote 43] We stratified the population into 12 groups on 
the basis of the type of work the inspector performed. Each sample 
element was subsequently weighted in the analysis to account 
statistically for all members of the population. For this report, we 
used a subgroup sample of 205 non-legacy safety inspectors. 

Because we followed a probability procedure based on random selections, 
our sample is only one of a large number of samples that we might have 
drawn. Since each sample could have provided different estimates, we 
express our confidence in the precision of our overall sample's results 
as a 95 percent confidence interval (e.g., plus or minus 7.1 percent). 
This is the interval that would contain the actual population value for 
95 percent of the samples we could have drawn. As a result, we are 95 
percent confident that each of the confidence intervals in this report 
will include the true values in the study population. The percentage 
estimates for the non-legacy subgroup has a margin of error of plus or 
minus 7.1 percent or less. Survey estimates presented as comparisons 
between groups are statistically significant when the 95 percent 
confidence intervals do not overlap. 

The survey was conducted using self-administered electronic 
questionnaires accessible on the Internet through a secure Web browser. 
We sent e-mail notifications to 496 inspectors (205 of which were in 
our subgroup of inspectors of non-legacy airlines) beginning on 
December 6, 2004. We then sent each potential respondent a unique 
password and user name to ensure that only members of the target 
population could participate in the survey. To encourage respondents to 
complete the questionnaire, we sent a subsequent e-mail message to 
further prompt each nonrespondent approximately 2 weeks after the 
initial e-mail message. We sent nonrespondents two more notices and 
closed the survey on February 4, 2005. Of the 496 inspectors we 
surveyed, we received 392 usable responses (79 percent). Among our 
subgroup of 205 non-legacy inspectors, we received 161 usable responses 
(79 percent). 

In addition to sampling errors, the practical difficulties in 
conducting surveys of this type may introduce other types of errors, 
commonly referred to as nonsampling errors. For example, questions may 
be misinterpreted, or the respondents' answers may differ from those of 
the inspectors who did not respond. We took steps to reduce these 
errors. 

Finally, we pretested the content and format of the questionnaire with 
safety inspectors at local FAA offices in Baltimore, Los Angeles, and 
Seattle. During the pretests we asked the inspectors questions to 
determine whether (1) the survey questions were clear, (2) the terms 
used were precise, (3) the questionnaire placed an undue burden on the 
respondents, and (4) the questions were unbiased. We made changes to 
the content and format of the final questionnaire based on the pretest 
results. 

We conducted our work from August 2004 through September 2005 in 
accordance with generally accepted government auditing standards. 

[End of section] 

Appendix II: FAA's Surveillance and Evaluation Process: 

SEP Incorporates System Safety into FAA's Inspection Oversight of Non- 
legacy Airlines: 

The Surveillance and Evaluation Program (SEP) is a process designed to 
introduce a data driven risk analysis system for non-legacy airlines 
and is guided by a Surveillance and Evaluation Assessment Tool (SEAT). 
SEP models itself on a system safety approach that is incorporated in 
the oversight of legacy airlines through the Air Transportation 
Oversight System (ATOS). SEP allows teams of inspectors to identify 
suspected trends through a data based system rather than relying 
totally on past experiences. A certificate management team oversees 
each of the 99 airlines under SEP. Each team is led by three principal 
inspectors, one for each major area of inspections (operations, 
maintenance, and avionics). Additional team members include those based 
at the FAA office that holds the airline's operating certificate-- 
typically an aircraft dispatch inspector, a cabin safety inspector, and 
assistants. 

The principal inspectors are required to meet as a team at least twice 
a year to assess the safety risks associated with the airlines they 
oversee and develop a surveillance plan for the airline, including 
designating inspection duties to other FAA offices at locations to 
which the airline flies or conducts business. In some field offices, 
teams meet each quarter. The teams use a planning tool--SEAT--to assess 
the systems in place at an airline and to identify any potential 
internal and external risks. Using SEAT, the team analyzes 10 systems, 
shown in table 8. 

Table 8: Safety Systems and Examples of Risk Indicators in SEAT: 

System and purpose: Aircraft configuration and control: maintains the 
physical condition of the aircraft and associated components; 
Selected examples of risk indicators in SEAT: 
1. Does the airline have the proper maintenance and inspection programs 
in place? 
2. Do the airline and vendor follow their fueling procedures, policies, 
and controls? 
3. Is the airline following its approved deicing procedures and 
policies? 

System and purpose: Manuals: controls the information and instructions 
that define and govern an airline's activities; 
Selected examples of risk indicators in SEAT: 
1. Is content consistent and complete across manuals? 
2. Are manuals up-to-date and available? 

System and purpose: Flight operations: governs aircraft movement; 
Selected examples of risk indicators in SEAT: 
1. Are passengers boarding in a safe environment? 
2. Are airlines ensuring that cargo is handled and carried safely 
according to their policies? 
3. Does the airline adhere to its approved weight and balance program? 

System and purpose: Personnel training and qualifications: ensures that 
an airline's personnel are trained and qualified; 
Selected examples of risk indicators in SEAT: 
1. Does the airline adhere to its training programs for crew members 
(attendants, dispatchers, station personnel, check airman and 
instructors, and maintenance personnel)? 
2. Are current and appropriate certifications of personnel available 
upon request, and does the airline have the programs, policies, and 
procedures in place to ensure certificates are valid? 

System and purpose: Route structures: maintains an airline's facilities 
on approved routes; 
Selected examples of risk indicators in SEAT: 
1. Does the airline adhere to its policies on weather-reporting 
facilities? 
2. Do the airline's maintenance and service facilities comply with its 
policies, procedures, and controls? 

System and purpose: Airman/crew member flight, rest, and duty time: 
prescribes time limitations for airline employees; 
Selected examples of risk indicators in SEAT: 
1. Does the airline adhere to its policies on flight crew (attendant or 
dispatcher) flight/duty/rest time? 
2. Does the airline adhere to its procedures and controls for its 
scheduling and reporting system? 

System and purpose: Technical administration: addresses other aspects 
of an airline's certification and operations, including key safety 
personnel and programs; 
Selected examples of risk indicators in SEAT: 
1. Does the director of maintenance (chief inspector, director of 
operations, and chief pilot) accomplish assigned duties and 
responsibilities? 
2. Does the director of safety effectively administer the safety 
program? 

System and purpose: Risk indicators: reflects the impact external and 
internal events have on an airline's system safety and stability; 
Selected examples of risk indicators in SEAT: 
1. Consider the impact of changes in required management personnel 
(airline management, turnover in personnel, reduction in workforce, 
layoffs, buyout, etc.); 2. Consider the complaints that affect 
surveillance planning and how that airline responded to them; 3. 
Consider the age of the airline's fleet, its process to survey and 
inspect aging aircraft, and the effectiveness of its aging aircraft 
program. 

System and purpose: Other: records the presence and implementation of 
airline developed security and substance abuse programs; 
Selected examples of risk indicators in SEAT: 
1. Does the airline understand FAA's authority to conduct drug tests 
and demonstrations? 
2. Does the airline have a security (drug and alcohol) program and is 
the airline in compliance with the program? 

System and purpose: Certificate management risks: reflects the impact 
of FAA's resources on providing oversight of an airline; 
Selected examples of risk indicators in SEAT: 
1. Consider if there is an adequate number and type of inspectors 
assigned to the certificate; 2. Consider if there is an adequate amount 
of geographic surveillance; 3. Consider the resources (travel budget, 
country clearance, etc.) of the certificate holding district office. 

Source: FAA. 

[End of table] 

To complete SEAT, the principal inspectors rely on their knowledge of 
the airline and on the data available through FAA's Safety Performance 
Analysis System database, which contains the Program Tracking and 
Reporting Subsystem and 12 additional databases with safety and 
performance information. SEAT includes a set of risk indicators for 
each of the elements (often in the form of a question) to be discussed 
by the team to indicate concerns about any real or potential problem 
that could contribute to the failure of one of the airline's elements, 
subsystems, or systems. Inspectors rate each risk they identify in 
terms of severity (negligible to catastrophic) and likelihood of 
occurrence (improbable to frequent), and SEAT calculates an overall 
risk rating. Risks rated high have high levels of likelihood and 
severity. 

Using this information, the team identifies a set of required 
inspections that must be completed during that fiscal year and planned 
inspections. The required inspections are added to the NPG-identified 
inspections, and the combined list becomes the annual surveillance plan 
for the airline. During the SEAT process, principal inspectors are 
allowed to substitute NPG-required inspections with SEP-identified 
inspections targeting higher-risk areas. 

SEP Incorporates Some Elements of ATOS, While NPG Relies on a Set 
Number of Inspections: 

Table 9 describes FAA's three inspection processes for overseeing 
legacy and non-legacy airlines: ATOS, NPG, and SEP. Many of the 
elements of ATOS, such as the use of data to identify risks and the 
development of surveillance plans by inspectors, are incorporated in 
the SEP process. The NPG process, in contrast, is not focused on the 
use of data and relies on an established set of inspections that are 
not risk based. 

Table 9: Various Elements of ATOS, NPG, and SEP: 

Description of program; 
ATOS: 
* Focuses on safety vulnerabilities rather than regulatory compliance; 
* Analysts and inspectors review airline data to identify areas of 
safety risk; 
* Inspectors develop surveillance plans for each airline, based on data 
analysis and assessment of risks, and adjust the plans periodically 
based on inspection results; 
NPG: 
* Focuses on inspectors completing a prescribed number of inspection 
activities; 
* Primarily based on checking airline compliance with regulations; 
* Relies on inspectors' expertise to identify trends and risks; 
SEP: 
* Focuses on inspectors conducting a risk assessment of various areas; 
* Inspectors review data to identify areas of safety risk and use 
Flight Standards Safety Analysis Information Center and the Safety 
Performance Analysis System as analytical tools; 
* Inspectors develop surveillance plans for each airline, based on data 
analysis and assessment of risks, and adjust plans periodically based 
on inspection results; 
* Inspectors can also verify that planned NPG activities meet the 
surveillance needs for a particular year. 

Type of commercial passenger airline inspected; 
ATOS: Legacy commercial airlines; 
NPG: Non-legacy commercial airlines; 
SEP: Non-legacy commercial airlines. 

Frequency of inspections; 
ATOS: Continuous safety oversight; 
NPG: Periodic; regular inspections are established annually by an FAA 
headquarters committee; 
SEP: Periodic; inspections are established during meetings held at 
least twice a year using risk-based criteria. 

Approximate number of aviation safety inspectors conducting 
inspections[A]; 
ATOS: 585; 
NPG: 1,100[B]; 
SEP: 1,100[B]. 

Number of commercial passenger airlines under the program[A]; 
ATOS: 14[C]; 
NPG: 99; 
SEP: 99. 

Sources: GAO and FAA. 

[A] As of July 2005. 

[B] There are a total of about 1,100 inspectors for both NPG and SEP 
inspections. 

[C] FedEx and United Parcel Service, two cargo air carriers, are also 
in the ATOS program. 

[End of table] 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Gerald L. Dillingham, (202) 512-2834: 

Staff Acknowledgments: 

In addition to the above, Teresa Spisak, Assistant Director; Carl 
Barden; Bari Bendell; Curtis Groves; Bert Japikse; Catherine Kim; 
Minette Richardson; Phillis Riley; Larry Thomas; Alwynne Wilbur; and 
Matt Zisman made key contributions to this report. 

(542039): 

FOOTNOTES 

[1] The legacy airlines are Alaska, American, Continental, Delta, 
Northwest, United, and US Airways. As of July 2005, the following non- 
legacy passenger airlines and cargo airlines were also part of ATOS: 
America West, American Eagle, Champion, ExpressJet, SkyWest, Southwest, 
Trans States, FedEx, and United Parcel Service. In this report, we 
refer to all passenger airlines that are not in the ATOS program as non-
legacy airlines. 

[2] The remaining approximately 1,500 inspectors oversee general 
aviation. 

[3] For this report, we use "field office" to refer to FAA field, 
regional, and certificate management offices, unless otherwise noted. 

[4] Four of the 25 non-legacy airlines in our review are now in the 
ATOS program---American Eagle, ExpressJet, SkyWest, and Trans States. 

[5] PTRS also includes information on "planned" inspection activities 
and other surveillance activities, which are not inspections, such as 
conducting telephone conversations with airline officials and reading 
documents related to an airline. However, the database does not 
distinguish between planned inspections and other activities. As a 
result, we excluded these activities from our analyses. 

[6] Geographic inspectors, who are based around the country and not 
assigned to any particular airline, do not normally receive training in 
the SEP process. 

[7] Before commencing operations, an airline must obtain an operating 
certificate from FAA. FAA issues the certificate after determining that 
an airline's manuals, aircraft, facilities, and personnel meet federal 
safety standards. FAA subsequently monitors the airline's operations, 
primarily through safety inspections to ensure that it continues to 
meet the terms of its certificate. 

[8] Avionics focuses on the electronic components of an aircraft. 

[9] Our review of ATOS's early implementation found problems, which FAA 
addressed by improving guidance to inspectors and increasing data 
usefulness. Since then, DOT's Inspector General has found additional 
problems with FAA's implementation of ATOS. GAO, Aviation Safety: FAA's 
New Inspection System Offers Promise, but Problems Need to be 
Addressed, GAO/RCED-99-183 (Washington, D.C.: June 28, 1999); and DOT 
Office of Inspector General, Safety Oversight of an Air Carrier 
Industry in Transition, AV-2005-062 (Washington, D.C.: June 3, 2005). 

[10] System safety is a process by which inspectors perform continual 
evaluations of an airline's operations for the purpose of identifying 
safety hazards and assessing the severity, frequency, and probability 
of the hazards and monitoring their resolution. 

[11] SEP was also developed to increase surveillance of new entrant 
airlines that are less than 5 years old. 

[12] The checklists are required under ATOS and are suggested but not 
mandatory for use under SEP. 

[13] Required NPG inspections consist of activities to help validate 
that critical air carrier subsystems do not have potential latent 
failures. According to FAA, periodic validation of these critical 
subsystems is important due to the inherent risks associated with their 
failures. 

[14] Both NPG and SEP consist of required and planned activities. 
According to FAA, the required NPG activities account for 12 percent of 
the total inspections performed in a given year and the planned NPG 
activities account for about 45 percent. The remaining 43 percent of 
NPG activities are created by unplanned requirements, according to the 
agency. SEP inspections are also designated as required or planned. Any 
planned SEP inspections that are not completed by the end of the fiscal 
year become required inspections for the following year. 

[15] GAO, Aviation Safety: Better Management Controls Are Needed to 
Improve FAA's Safety Enforcement and Compliance Efforts, GAO-04-646 
(Washington, D.C.: July 6, 2004). 

[16] GAO, Aviation Safety: FAA Management Practices for Technical 
Training Generally Effective; Further Actions Could Enhance Results, 
GAO-05-728 (Washington, D.C.: Sept. 7, 2005). 

[17] GAO/RCED-98-6; GAO/T-RCED-92-25. 

[18] GAO/RCED-99-183 and GAO, Aviation Safety: FAA Oversight of Repair 
Stations Needs Improvement, GAO/RCED-98-21 (Washington, D.C.: Oct. 24, 
1997). 

[19] GAO/RCED-92-14. 

[20] In addition to required NPG and SEP inspections, FAA inspectors 
conduct planned inspections and unplanned activities, such as 
enforcement investigations. While we recognize that unplanned 
activities account for a portion of an inspector's workload, we did not 
include them in our analysis because they are generated on an "as 
discovered" basis and, therefore, lack comparability with other 
inspections that are generated through the NPG or SEP processes. For 
our analysis, we looked only at required inspections because they have 
higher priority than the planned inspections and because PTRS does not 
distinguish between planned inspections and other planned activities. 
According to FAA, the required NPG inspections account for about 12 
percent of the total inspections performed in a given year, planned NPG 
activities account for about 45 percent, and unplanned activities 
account for the remaining 43 percent of NPG activities. According to 
data provided by FAA, for the planned inspections in fiscal years 2002- 
2004, 6 percent were SEP-initiated and 94 percent were NPG-initiated, 
indicating that an even larger percentage of these inspections compared 
with required inspections are not prioritized based on a structured 
risk identification process. 

[21] FAA has no similar requirement for planned NPG activities, the 
completion of which depends upon available resources. 

[22] Inspectors can substitute or retarget required NPG-initiated 
activities with activities identified during the periodic SEP meetings 
that they deem constitute a greater safety risk. 

[23] The Office of Aviation Safety was expected to absorb about a $17.4 
million reduction in its fiscal year 2005 budget. The reductions were 
required by Congress and FAA. 

[24] In September 2005, FAA headquarters officials told us they 
expected Congress to appropriate an additional $8 million in fiscal 
year 2006, which would allow them to hire more inspectors than the 
previously planned 80. 

[25] Designees are private persons and organizations that handle the 
vast majority of FAA's safety certification activities, such as testing 
pilots and mechanics for FAA-issued certificates. 

[26] We have reported that FAA has followed or is taking many effective 
management practices in planning, developing, and delivering technical 
training, yet inspectors express widespread dissatisfaction with this 
training. See GAO-05-728. 

[27] We did not verify these reports that inspectors made in the 
internal risk database. However, FAA procedures call for all risks to 
be reviewed by managers, who can also enter comments indicating 
concurrence or disagreement with the risk, among other things. The 
examples we cite in this report do not contain information indicating 
that management questioned the accuracy of the inspectors' statements. 

[28] GAO-05-728. 

[29] The safeguards include (1) executing an agreement with the 
aviation industry training provider outlining the conditions under 
which the training will be accepted, (2) conducting a legal review of 
the agreement to ensure that there are no conflict of interest issues, 
and (3) obtaining the approval of the FAA Administrator for the 
acceptance of the training. 

[30] We were not provided sufficient information to verify these 
statements. 

[31] GAO-05-728. 

[32] We defined "technical training" as training that develops 
knowledge of the production, maintenance and operation of aircraft 
(including currency training), aircraft parts, and systems. For the 
purposes of our survey, technical training did not include automation 
training. Our definition differs somewhat from FAA's use of the term. 
FAA defines "technical training" to include aviation technologies as 
well as topics such as inspector job skills, risk analysis, data 
analysis, and training in software packages, such as spreadsheets. 

[33] The actual estimate is 77 percent. All percentage estimates from 
the survey have a margin of error of plus or minus 7.1 percentage 
points or less. Survey estimates presented are statistically 
significant when the 95 percent confidence intervals do not overlap. 

[34] FAA officials told us that training occurs over the span of a 
career and cautioned that asking inspectors' views about 2 years' 
experience would present a distorted view. We recognize FAA's point and 
the fact that FAA requires candidates for safety inspector positions to 
have extensive technical qualifications and experience. However, it is 
not reasonable to expect inspectors' to recall their views on training 
received over a large time span, as doing so could lead to unreliable 
results. 

[35] FAA does not have information to separate out the number of ATOS 
inspectors and inspectors of non-legacy airlines prior to fiscal year 
2005. 

[36] The actual estimate is 33 percent. 

[37] GAO/RCED-98-6. 

[38] GAO, Managing for Results: Strengthening Regulatory Agencies' 
Performance Management Practices, GAO/GGD-00-10 (Washington, D.C.: Oct. 
28, 1999); and Executive Guide: Effectively Implementing the Government 
Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June 
1996). 

[39] In this report, we refer to all passenger airlines overseen by 
FAA's Air Transportation Oversight System (ATOS) as legacy airlines and 
those that are not as non-legacy airlines. As of July 2005, the 
following passenger airlines and cargo airlines were part of ATOS: 
Alaska, America West, American, American Eagle, Champion, Continental, 
Delta, ExpressJet, FedEx, Northwest, SkyWest, Southwest, Trans States, 
United, United Parcel Service, and US Airways. 

[40] Four of the 25 airlines--American Eagle, ExpressJet, SkyWest, and 
Trans States--were transitioned into the ATOS program from 2003 through 
2005. Prior to then, they were covered under SEP. 

[41] In this report, we refer to FAA staff who perform safety audits, 
inspections, and surveillance as inspectors. 

[42] Management controls are the continuous processes that federal 
agencies are required by law--i.e., the Federal Managers' Financial 
Integrity Act of 1982, the Government Performance and Results Act of 
1993, the Chief Financial Officers Act of 1990, and the Federal 
Financial Management Improvement Act of 1996--and by the Office of 
Management and Budget to use to provide reasonable assurance that their 
goals, objectives, and missions are being met. We identified, with the 
assistance of GAO specialists in that area, those control standards 
established by the Office of Management and Budget and us that are 
appropriate to FAA's inspection process. See GAO, Standards for 
Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 
(Washington, D.C.: November 1999); and Internal Control Management and 
Evaluation Tool, GAO-01-1008G (Washington, D.C.: August 2001). 

[43] Our population included only those inspectors that actively 
participate in inspection activities as part of their regular job 
duties. It did not include managers, supervisors, or inspectors 
detailed to headquarters or regional offices. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, 

NelliganJ@gao.gov 

(202) 512-4800 

U.S. Government Accountability Office, 

441 G Street NW, Room 7149 

Washington, D.C. 20548: