Aviation Safety:

FAA's New Inspection System Offers Promise, but Problems Need to Be Addressed

RCED-99-183: Published: Jun 28, 1999. Publicly Released: Jul 6, 1999.

Additional Materials:

Contact:

Gerald Dillingham, Ph.D.
(202) 512-4803
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Pursuant to a congressional request, GAO provided information on the Federal Aviation Administration's (FAA) Air Transportation Oversight System (ATOS), focusing on: (1) to what extent ATOS addresses past concerns about FAA's aviation safety inspections; (2) what factors, if any, surfaced during the implementation of ATOS that could impede its success; and (3) what FAA is doing to address any factors that could impede the success of ATOS.

GAO noted that: (1) ATOS is largely responsive to past concerns raised about key aspects of FAA's aviation safety inspections and the usefulness of inspection data; (2) these concerns centered on FAA's unstructured inspection process, the adequacy of technical training for inspectors, the quality and consistency of inspection data, and the usefulness of those data for identifying safety problems and targeting the agency's resources to the greatest risks; (3) addressing these concerns involved a fundamental redesign of the way FAA inspects the nation's airlines; (4) to improve inspection quality, the new program emphasizes a system safety approach that goes beyond spot-checking airlines for compliance with Federal Aviation Regulations; (5) using safety principles originally created for the nuclear industry, it calls for a systematic review of airlines' policies and procedures to ensure that they incorporate basic safety principles, such as clear lines of responsibility and written documentation; (6) it fosters more consistent, structured inspections by standardizing inspection tasks, linking inspectors' training more closely to their assigned responsibilities, and using teams rather than individual inspectors to perform many inspections; (7) the program also calls for a number of enhancements to improve the usefulness of inspection data for analysis and targeting; (8) they include a standardized database for reporting inspection results and the addition of data quality assurance managers and analysts; (9) the goal of this redesign is to target inspection resources to those areas that present the greatest safety risks; (10) ATOS offers promise for significantly strengthening FAA's inspection process, but FAA must also address the problems identified in this report to ensure that the new system fulfills its promise; (11) FAA's ability to conduct effective inspections remains limited by a lack of clear guidance, staff turnover, and continued difficulties with the adequacy of inspectors' technical training and experience; (12) the anticipated enhancements to make inspection data more useful have not been achieved because of problems with reporting requirements and the incompatibility of the program's database with FAA's primary inspection analysis system; (13) these problems resulted largely from FAA's decision to implement the new inspection system on an overly ambitious schedule; (14) meeting FAA's target date for implementation meant that complex, critical steps had to be compressed into a very short time; and (15) FAA management acknowledged that ATOS faces significant challenges.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In January 2002, FAA integrated its ATOS database with SPAS, its computer analysis tool designed to aid inspectors in determining areas to inspect based on safety risk. The need to analyze inspection data to decide when and where to direct FAA's limited inspection resources has been a long-standing concern reported to FAA since 1987. FAA responded to this concern by developing SPAS.

    Recommendation: To strengthen the efforts to improve FAA's aviation safety inspections and the usefulness of the data that result from these inspections for analysis and for targeting the agency's resources to the greatest potential safety threats, the Secretary of Transportation should direct the Administrator, FAA, to determine what revisions will be needed to the ATOS database and FAA's existing Safety Performance Analysis System database to maximize the potential of these two systems by coordinating their trend analyses to identify potential safety risks.

    Agency Affected: Department of Transportation

  2. Status: Closed - Implemented

    Comments: FAA concurred with this recommendation. Air Transportation Oversight System (ATOS) Data Quality Standardization Guidelines have been developed and provided to all ATOS-assigned inspectors. Automation controls have been developed to prevent inspectors from saving data without location and other mandatory fields. All essential fields have been identified, and required automation changes were completed in September 2000. As of February 9, 2001, automation links every regulatory-based job aid question to the applicable specific regulatory requirement to the Code of Federal Regulations (CFR). Inspectors have the ability to query data repository by CFR to determine regulatory compliance status of the air carrier at the time inspections were completed. Reports can be constructed to highlight "no" responses to regulatory-based questions and the inspector action taken as a result. Existing actions meet the intent of the recommendation.

    Recommendation: To strengthen the efforts to improve FAA's aviation safety inspections and the usefulness of the data that result from these inspections for analysis and for targeting the agency's resources to the greatest potential safety threats, the Secretary of Transportation should direct the Administrator, FAA, to restructure the inspection database to: (1) require that essential data fields be completed before inspection reports can be closed out; and (2) clearly indicate the proportion of inspection observations in which an airline complies with regulations.

    Agency Affected: Department of Transportation

  3. Status: Closed - Implemented

    Comments: FAA updated its guidance on data gathering and analysis to provide a more accurate and comprehensive evaluation of air carrier safety and compliance with the Federal Aviation Regulations to improve it decision-making process.

    Recommendation: To strengthen the efforts to improve FAA's aviation safety inspections and the usefulness of the data that result from these inspections for analysis and for targeting the agency's resources to the greatest potential safety threats, the Secretary of Transportation should direct the Administrator, FAA, to develop a plan that involves both inspectors and experts in risk assessment and database development in revising and refining the analysis of the data needs of users of the new inspection program. The requirements analysis should describe in detail the questions that need to be asked to improve safety, determine precisely what data are needed to answer those questions, and plan the appropriate analyses to be conducted on those data to answer the questions.

    Agency Affected: Department of Transportation

  4. Status: Closed - Implemented

    Comments: FAA revised its Air Transportation Oversight System guidance on the safety attribute inspection and element performance inspection data collection tools to identify questions required by Federal Aviation Regulations. These revisions clarify the guidance on the number of inspections required to achieve compliance and safety. The relevant FAA Order was completed in September 2001, and signed on October 19, 2001.

    Recommendation: To strengthen the efforts to improve FAA's aviation safety inspections and the usefulness of the data that result from these inspections for analysis and for targeting the agency's resources to the greatest potential safety threats, the Secretary of Transportation should direct the Administrator, FAA, to revise the inspection guidance to: (1) include guidelines on the minimum number of times to perform various inspection tasks; and (2) distinguish between tasks based on regulatory requirements and those based on handbook or other guidance.

    Agency Affected: Department of Transportation

  5. Status: Closed - Implemented

    Comments: FAA revised its Air Transportation Oversight System policies and procedures on its inspection guidance in October 2001. This guidance defines and describes required tasks to comply with Federal Aviation Regulations.

    Recommendation: To strengthen the efforts to improve FAA's aviation safety inspections and the usefulness of the data that result from these inspections for analysis and for targeting the agency's resources to the greatest potential safety threats, the Secretary of Transportation should direct the Administrator, FAA, to develop a structured process and timeline for working with inspectors to revise the Air Transportation Oversight System's planning and inspection guidance. The process should involve the inspectors now using this guidance to: (1) identify problems with the clarity of the guidance; (2) revise the inspection guidance to include tasks related to all applicable Federal Aviation Regulations; and (3) define the tasks to be completed during inspections.

    Agency Affected: Department of Transportation

  6. Status: Closed - Implemented

    Comments: FAA concurred with this recommendation. After the job aid revisions were completed in September 1999, a draft of each revised element performance inspection received a quality assurance review by an alpha test group, and was forwarded to field inspectors at each CMT for beta testing. The work group reviewed feedback from the alpha and beta tests and incorporated additional changes to the job aids, which were issued in January. Since that time, FAA has been using an electronic system to gather feedback on their use. A continuous improvement job aid work group meets quarterly to review the feedback and make additional revisions as necessary. Existing actions meet the intent of the recommendation.

    Recommendation: To strengthen the efforts to improve FAA's aviation safety inspections and the usefulness of the data that result from these inspections for analysis and for targeting the agency's resources to the greatest potential safety threats, the Secretary of Transportation should direct the Administrator, FAA, to test and validate the revised guidance and database for the new inspection program.

    Agency Affected: Department of Transportation

 

Explore the full database of GAO's Open Recommendations »

Sep 14, 2016

Sep 2, 2016

Aug 8, 2016

Jul 28, 2016

Jul 13, 2016

Jul 7, 2016

Jun 24, 2016

Jun 21, 2016

May 26, 2016

Looking for more? Browse all our products here