Certification and Approval Processes Are Generally Viewed as Working Well, but Better Evaluative Information Needed to Improve Efficiency
GAO-11-14: Published: Oct 7, 2010. Publicly Released: Oct 29, 2010.
Among its responsibilities for aviation safety, the Federal Aviation Administration (FAA) issues thousands of certificates and approvals annually. These certificates and approvals, which FAA bases on its interpretation of federal standards, indicate that such things as new aircraft, the design and production of aircraft parts and equipment, and new air operators are safe for use in the national airspace system. Past studies and industry spokespersons assert that FAA's interpretations produce variation in its decisions and inefficiencies that adversely affect the industry. GAO was asked to examine the (1) extent of variation in FAA's interpretation of standards for certification and approval decisions and (2) views of key stakeholders and experts on how well these processes work. To perform the study, GAO reviewed industry studies and reports and FAA documents and processes; convened a panel of aviation experts; and interviewed officials from various industry sectors, senior FAA officials, and unions representing FAA staff.
Studies, stakeholders, and experts indicated that variation in FAA's interpretation of standards for certification and approval decisions is a long-standing issue, but GAO found no evidence that quantified the extent of the problem in the industry as a whole. Ten of the 13 industry group and company officials GAO interviewed said that they or members of their organization had experienced variation in FAA certification and approval decisions on similar submissions. In addition, experts on GAO's panel, who discussed and then ranked problems with FAA's certification and approval processes, ranked inconsistent interpretation of regulations, which can lead to variation in decisions, as the first and second most significant problem, respectively, with these processes for FAA's Flight Standards Service (which issues certificates and approvals for individuals and entities to operate in the national airspace system) and Aircraft Certification Service (which issues approvals to the designers and manufacturers of aircraft and aircraft parts and equipment). According to industry stakeholders, variation in FAA's interpretation of standards for certification and approval decisions is a result of factors related to performance-based regulations, which allow for multiple avenues of compliance, and the use of professional judgment by FAA staff and can result in delays and higher costs. Industry stakeholders and experts generally agreed that FAA's certification and approval processes contribute to aviation safety and work well most of the time, but negative experiences have led to costly delays for the industry. Industry stakeholders have also raised concerns about the effects of process inefficiencies on the implementation of the Next Generation Air Transportation System (NextGen)--the transformation of the U.S. national airspace system from a ground-based system of air traffic control to a satellite-based system of air traffic management. They said that the processes take too long and impose costs that discourage aircraft operators from investing in NextGen equipment. FAA has taken actions to improve the certification and approval processes, including hiring additional inspectors and engineers and increasing the use of designees and delegated organizations--private persons and entities authorized to carry out many certification activities. Additionally, FAA is working to ensure that its processes are being followed and improved through a quality management system, which provides a mechanism for stakeholders to appeal FAA decisions. However, FAA does not know whether its actions under the quality management system are achieving the intended goals of reducing inconsistencies and increasing consistency and fairness in the agency's application of regulations and policies because FAA does not have outcome-based performance measures and a continuous evaluative process that would allow it to determine progress toward these goals. Without ongoing information on results, FAA managers do not know if their actions are having the intended effects. GAO recommends that FAA develop a continuous evaluative process with measurable performance goals to determine the effectiveness of the agency's actions to improve its certification and approval processes. DOT did not comment on the recommendations but provided technical comments, which were included as appropriate.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: In October 2010, GAO reported that FAA had taken action to address variation in decisions and inefficiency in its certification and approval processes but without outcome-based performance measures and a continuous evaluative process to determine if these actions are having their intended effects. GAO recommended that the Secretary of Transportation direct the FAA Administrator to determine the effectiveness of actions taken to improve the certification and approval processes by developing a continuous evaluative process and use it to create measurable performance goals for the actions, track performance toward those goals, and determine appropriate process changes. To the extent that this evaluation of agency actions identifies effective practices, consider instituting them agency wide. On October 1, 2011 FAA implemented new metrics that provide the ability to track process performance and product conformity. These metrics will allow FAA to set measurable performance goals necessary to determine the effectiveness of the certification and approval processes and assist it in deciding the appropriate actions necessary to address systemic issues that may negatively impact certain processes and their outcomes.
Recommendation: To ensure that FAA actions contribute to more consistent decisions and more efficient certification and approval processes, the Secretary of Transportation should direct the Administrator of FAA to determine the effectiveness of actions to improve the certification and approval processes by developing a continuous evaluative process and use it to create measurable performance goals for the actions, track performance toward those goals, and determine appropriate process changes. To the extent that this evaluation of agency actions identifies effective practices, consider instituting those practices agency wide.
Agency Affected: Department of Transportation
Status: Closed - Implemented
Comments: GAO reported in October 2010 that FAA's Flight Standards and Aircraft Certification Services notify applicants for certification services as to whether resources are available to begin work on their projects. However, while Aircraft Certification tracks in a national database how long individual submissions are wait-listed, Flight Standards does not track wait-listed submissions and, therefore, is unaware how long projects are wait-listed and unable to reallocate resources to better meet demand for certification services. Further, industry stakeholders face uncertainty with respect to any plans or investments that depend on obtaining a certification in a timely manner. GAO recommended that the Secretary of Transportation direct the FAA Administrator to develop and implement in Flight Standards a process to track how long certification and approval submissions are wait-listed, the reasons for wait-listing them, and the factors that eventually allowed initiation of the certification process. In October 2011, FAA indicated that it had altered the software in Flight Standards' Certification Service Oversight Process (CSOP) database to designate when certification submissions are wait-listed. The database now tracks how long certification submissions are wait-listed. As a result, FAA now has the capability to track how long certification submissions are waitlisted and reallocate resources, as appropriate, to better meet demand.
Recommendation: To ensure that FAA actions contribute to more consistent decisions and more efficient certification and approval processes, the Secretary of Transportation should direct the Administrator of FAA to develop and implement a process in Flight Standards to track how long certification and approval submissions are wait-listed, the reasons for wait-listing them, and the factors that eventually allowed initiation of the certification process. Use the data generated from this process to assess the extent of wait-listing delays and to reallocate resources, as appropriate, to better meet demand.
Agency Affected: Department of Transportation