Aviation Safety:

Data Problems Threaten FAA Strides on Safety Analysis System

AIMD-95-27: Published: Feb 8, 1995. Publicly Released: Feb 8, 1995.

Additional Materials:


Joel C. Willemssen
(202) 512-6253


Office of Public Affairs
(202) 512-4800

Pursuant to a congressional request, GAO reviewed the Federal Aviation Administration's (FAA) Safety Performance Analysis System (SPAS), focusing on: (1) whether FAA is effectively managing the SPAS acquisition; (2) the extent to which SPAS will rely on Aviation Safety Analysis System (ASAS) databases; and (3) whether FAA is effectively addressing known data quality problems with the ASAS databases.

GAO found that: (1) FAA has generally implemented good development and acquisition procedures for SPAS; (2) FAA has maximized user involvement and system prototyping in developing and evaluating SPAS; (3) FAA has reduced SPAS development risks by using an independent verification and validation agent; (4) FAA is exploring the potential of its proposed corporate-wide area network to accommodate SPAS in order to avoid the acquisition of duplicate communication networks; (5) FAA cost estimates for SPAS software may not be reliable, since they are subjective; (6) FAA lacks a strategy for improving SPAS data sources, particularly ASAS, which jeopardizes the system's utility; (7) ASAS databases contain incomplete, inaccurate, and inconsistent data on airline inspections; (8) FAA has not yet defined its long-term data quality goals; and (9) if FAA fails to improve ASAS data, it could improperly target its limited inspection and certification resources on less important problems.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: Department of Transportation (DOT) concurred with the recommendation. In January 1996, FAA used a cost estimating tool called CHECKPOINT to develop a SPAS I cost estimate of $6.4 million. This tool is also being used to develop a SPAS II cost estimate. The SPAS II cost estimate is to be finalized by the end of August 1996.

    Recommendation: The Administrator, FAA, should direct the Associate Administrator for Regulation and Certification to ensure that SPAS software costs are estimated using systematic and rigorous estimating techniques and methods.

    Agency Affected: Department of Transportation: Federal Aviation Administration

  2. Status: Closed - Implemented

    Comments: FAA concurred with the recommendation and issued the Comprehensive Data Quality Plan for the Flight Standards Service in October 1996. The plan satisfies the last three of the five specifications provided in GAO's recommendation. FAA is now implementing the plan. To address specifications 1 and 2 of GAO's recommendation, FAA is developing a process for defining and developing SPAS performance measures that will include a review of specific data elements.

    Recommendation: The Administrator, FAA, should direct the Associate Administrator for Regulation and Certification to require the Office of Flight Standards to develop and implement a comprehensive and coordinated strategy, specifying how the quality of all data residing on SPAS source data systems will be brought up to the minimum level needed for SPAS to meet operational requirements. At a minimum, this strategy must include: (1) clear and measurable data quality objectives for each SPAS source data system that recognize the sensitivity of various SPAS analyses to the respective source data inputs; (2) accurate assessments of the current quality of the data on each SPAS source data system; (3) clear statements of organizational responsibility and authority for improving the source systems' data quality; (4) both interim and long-term milestones for attaining stated quality objectives that tie closely to SPAS development schedules; and (5) estimates of resource requirements to meet stated objectives and agency commitments to providing these resources.

    Agency Affected: Department of Transportation: Federal Aviation Administration


Explore the full database of GAO's Open Recommendations »

Nov 8, 2018

Oct 3, 2018

Sep 13, 2018

Sep 5, 2018

Jul 30, 2018

Jul 26, 2018

Jul 19, 2018

Jul 12, 2018

May 31, 2018

May 29, 2018

Looking for more? Browse all our products here