Air Traffic Control Modernization:

Management Challenges Associated with Program Costs and Schedules Could Hinder NextGen Implementation

GAO-12-223: Published: Feb 16, 2012. Publicly Released: Feb 16, 2012.

Additional Materials:

Contact:

Gerald Dillingham, Ph.D.
(202) 512-2834
dillinghamg@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

In a review of 30 major ATC acquisition programs, all of which will contribute to the transition to NextGen, GAO found that costs for 11 of the 30 programs have increased from their initial estimates by a total of $4.2 billion and 15 programs experienced delays. The 11 acquisitions that experienced cost increases account for over 60 percent of FAA’s total acquisition costs ($11 billion of $17.7 billion) for the 30 programs. The 15 acquisitions that experienced schedule delays, of which 10 also had cost increases, ranged from 2 months to more than 14 years and averaged 48 months.

Cost increases and schedule delays occurred due to several factors, many of which have been longstanding challenges for FAA. Specifically, these have involved (1) additional or unanticipated system requirements; (2) insufficient stakeholder involvement (such as controllers’ input) throughout system development; (3) underestimating the complexity of software development; and (4) unanticipated events including funding shortfalls or work toppages. These challenges, if they persist, will impede the implementation of NextGen, especially in light of the interdependencies among many acquisition programs, where cost increases or delays in one program can affect the costs and schedules of other programs.

For the four programs GAO selected to analyze in depth, FAA is not consistently following the characteristics of high-quality cost estimates and scheduling best practices that GAO previously identified. Regarding cost estimates, GAO found that although all four of the programs generally provided well documented and comprehensive estimates, which are two of the four characteristics, no program fully met the two other characteristics. Specifically, each program estimate was not credible because each lacked an independent cost estimate, which provides a check against FAA’s estimate and three programs lacked risk or uncertainty analysis. The estimates also lacked accuracy because they were not updated regularly or based on comparable programs. Regarding scheduling practices, most programs did not substantially or fully meet the majority of the 9 best practices GAO previously identified including developing a fully integrated master schedule of all program activities and performing a schedule risk analysis. For example, without a schedule risk analysis, FAA is unable to predict, with any degree of confidence, if the estimated completion dates are realistic. FAA is implementing new processes and organizational changes to better manage acquisitions. However, by not consistently following the characteristics of highquality cost estimate and scheduling best practices, FAA cannot provide reasonable assurance to Congress and other stakeholders that NextGen and other ATC programs will avoid additional cost increases or schedule delays.

Why GAO Did This Study

The Federal Aviation Administration (FAA), partnering with other federal agencies and the aviation industry, is implementing the Next Generation Air Transportation System (NextGen), a new satellite-based air traffic management system that will replace the current radar-based system and is expected to enhance the safety and capacity of the air transport system by 2025. Concurrently, FAA continues to maintain and upgrade existing air traffic control (ATC) systems that will also be needed for NextGen. This involves acquiring and implementing new software and hardware.

GAO was asked to determine (1) how, if at all, costs and schedules of FAA ATC acquisitions programs, including those related to NextGen, have changed since they were first submitted to Congress, (2) the reasons for any such changes, and (3) the extent that selected ATC programs adhere to cost and schedule best practices. To do its work, GAO reviewed 30 programs and conducted cost and schedule analysis on four programs that had an approved baseline and were NextGen related. GAO reviewed acquisition documents and interviewed FAA officials.

What GAO Recommends

To better estimate the cost and completion dates for major acquisitions, FAA should, among other things, require cost and schedule risk analysis, independent cost estimates and integrated master schedules. FAA did not comment on whether or not it agreed with the recommendations.

For more information, contact Gerald L. Dillingham, Ph.D., at (202) 512-2834 or dillinghamg@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: When we confirm the action(s) that FAA has taken in response to this recommendation, we will provide updated information.

    Recommendation: To improve cost estimates and schedules for NextGen and other major air traffic control acquisition programs, the Secretary of Transportation should direct FAA, when appropriate for major acquisition programs based on a program's cost, schedule, complexity, and risk, to conduct independent cost estimates and schedule risk analysis for major acquisition programs.

    Agency Affected: Department of Transportation

  2. Status: Closed - Implemented

    Comments: The Federal Aviation Administration (FAA), partnering with other federal agencies and the aviation industry, is implementing the Next Generation Air Transportation System (NextGen), a new satellite-based air traffic management system that will replace the current radar-based system and is expected to enhance the safety and capacity of the air transport system by 2025. Concurrently, FAA continues to maintain and upgrade existing air traffic control (ATC) systems that will also be needed for NextGen. This involves acquiring and implementing new software and hardware. In 2012, GAO reviewed four major ATC acquisition programs in depth and found that FAA was not consistently following the characteristics of high-quality cost estimates and scheduling best practices that GAO previously identified. Regarding scheduling practices, most programs did not substantially or fully meet the majority of the 9 best practices GAO previously identified including developing a fully integrated master schedule of all program activities. A fully integrated master schedule is a best practice and tool in management of business-system development that is crucial to estimating the overall schedule and cost of a program. The schedule should be horizontally and vertically link all program activities and milestones, including government and contractor schedules and program segments. FAA was implementing new processes and organizational changes to better manage acquisitions. However, FAA's lack of a fully integrated master schedule for these programs hampers its ability to provide accurate information on the schedule for these programs. This information would be needed as FAA simultaneously works to develop an integrated master schedule for the overall NextGen initiative. Moreover, without this information FAA could not provide reasonable assurance to Congress and other stakeholders that NextGen and other ATC programs will avoid additional cost increases or schedule delays. Therefore, GAO recommended FAA require a fully integrated master schedule for each major acquisition program, including those that are components of NextGen. In 2016, GAO confirmed that FAA assessed its scheduling practices against GAO's Schedule assessment guide and found many of its program schedules were not compliant with the guide's leading practices including the need to have a fully integrated master schedule. To incorporate scheduling best practices into its acquisition practices, FAA developed a new training class in its Acquisition curriculum which incorporated key elements of GAO Schedule guide. Also, the Acquisition Executive Board established a workgroup to develop policy and guidance for best practices in the acquisition lifecycle, mechanisms that can be used to monitor compliance and identify when to perform updates on a program in the acquisition lifecycle. In addition, the Acquisition Management System guidance was updated to reflect acquisitions should follow GAO's Scheduling best practices, which directs acquisitions, including NextGen acquisitions to have a fully integrated master schedule. According to FAA, these actions will assist the agency in having a solid foundation for the creation and integration of scheduling concepts for its acquisition professionals. Consequently, FAA's use of an integrated master schedule will assist the agency in monitoring a program, identifying problems that could affect later stages of the program's implementation, improving the accuracy of cost estimates and schedules for individual programs, and improving the accuracy of information FAA is compiling to monitor the costs and schedules for the NextGen initiative.

    Recommendation: To improve cost estimates and schedules for NextGen and other major air traffic control acquisition programs, the Secretary of Transportation should direct FAA, when appropriate for major acquisition programs based on a program's cost, schedule, complexity, and risk, to require a fully integrated master schedule for each major acquisition program, including those that are components of NextGen. An integrated master schedule should horizontally and vertically link all program activities and milestones, including government and contractor schedules and program segments.

    Agency Affected: Department of Transportation

  3. Status: Closed - Implemented

    Comments: The Federal Aviation Administration (FAA), partnering with other federal agencies and the aviation industry, is implementing the Next Generation Air Transportation System (NextGen), a new satellite-based air traffic management system that will replace the current radar-based system and is expected to enhance the safety and capacity of the air transport system by 2025. Concurrently, FAA continues to maintain and upgrade existing air traffic control (ATC) systems that will also be needed for NextGen. This involves acquiring and implementing new software and hardware. In 2012, GAO reviewed four major ATC programs in depth and found that FAA had not consistently followed the characteristics of high-quality cost estimates and scheduling best practices that GAO previously identified in its Cost Guide. Regarding cost estimates, GAO found that although all four of the programs generally provided well documented and comprehensive estimates, which are two of the four characteristics, no program fully met the two other characteristics. Regarding scheduling practices, most programs did not substantially or fully meet the majority of the 9 best practices identified in GAO's Cost Guide including developing a fully integrated master schedule of all program activities and performing a schedule risk analysis. For example, without a schedule risk analysis, FAA was unable to predict, with any degree of confidence, if the estimated completion dates are realistic. FAA was implementing new processes and organizational changes to better manage acquisitions. However, by not consistently following the characteristics of high-quality cost estimate and scheduling best practices, FAA could not provide reasonable assurance to Congress and other stakeholders that NextGen and other air traffic control (ATC) programs will avoid additional cost increases or schedule delays. Therefore, GAO recommended that FAA conduct an assessment of major acquisition programs to ensure they meet all of the established best practices for cost estimates and schedules contained in GAO guidance. In 2016, GAO confirmed that FAA formed a task force that assessed the agency's scheduling practices for 6 acquisition programs that represented a cross section of size, scope and complexity within the agency against the GAO schedule assessment guide. The task force found, among other things, that the schedules for many of the programs were not in compliance with the GAO schedule assessment guide. In response to the task force's findings, FAA implemented schedule best practices for major acquisition programs as warranted. To incorporate scheduling best practices into its acquisition practices, FAA developed a new training class in its acquisition curriculum which covers key elements of GAO Schedule assessment guide. Also, the Acquisition Executive Board established a workgroup to develop policy and guidance for cost and schedule best practices in the acquisition lifecycle, mechanisms that can be used to monitor compliance and when to perform updates on a program in the acquisition lifecycle. As result, these actions will help ensure that FAA"s conducts assessments of its major acquisition programs that adhere to GAO's cost and schedule guidance produces reliable program cost estimates and schedules for Congress.

    Recommendation: To improve cost estimates and schedules for NextGen and other major air traffic control acquisition programs, the Secretary of Transportation should direct FAA, when appropriate for major acquisition programs based on a program's cost, schedule, complexity, and risk, to conduct an assessment of major acquisition programs to ensure theymeet all of the established best practices for cost estimates andschedules contained in GAO guidance.

    Agency Affected: Department of Transportation

  4. Status: Closed - Implemented

    Comments: The Federal Aviation Administration (FAA), partnering with other federal agencies and the aviation industry, is implementing the Next Generation Air Transportation System (NextGen), a new satellite-based air traffic management system that will replace the current radar-based system and is expected to enhance the safety and capacity of the air transport system by 2025. Concurrently, FAA continues to maintain and upgrade existing air traffic control (ATC) systems that will also be needed for NextGen. This involves acquiring and implementing new software and hardware. In 2012, GAO reviewed four major ATC programs in depth and found that FAA was not consistently following the characteristics of high-quality cost estimates and scheduling best practices that GAO previously identified. In response to this finding, FAA provided information on steps it was taking to improve its processes for both cost estimates and schedules and noted that some of the cost estimates and schedules GAO reviewed were developed before the improvements were in place. FAA stated that strengthening its cost estimation process was part of the seven key acquisition processes it had developed. FAA updated its guidelines for FAA cost estimation practices to be consistent with the GAO cost guide, filling in gaps that it had identified during a comparison of its practices to those contained in the GAO cost guide. As of November 2011, 11 of the 12 best practices were addressed in the guidelines. According to FAA, the remaining best practice--involving the creation of independent cost estimates--were unlikely to be implemented at FAA in the foreseeable future because FAA believed the resources required to create independent estimates were prohibitive in current budget environments. In its 2012 report, GAO recommended that FAA should, among other things, conduct cost and schedule risk analysis, independent cost estimates and require integrated master schedules. However, GAO recognized that conducting independent cost estimates and schedule risk analysis took both financial resources and some time and that it may be appropriate to limit one or both of these analyses to instances where a program is particularly costly, complex, or on a compressed schedule. Given these constraints, GAO recommended that FAA determine which programs should be subject to the recommendations regarding independent cost estimates and schedule risk analysis. In 2016, GAO confirmed that FAA implemented a new acquisition category determination process--which was established to ensure that the right amount of oversight and rigor is applied to each FAA investment--to recommend to the Joint Resources Council whether an independent cost estimate and schedule risk analysis is needed based on a program's risk versus available resources. In addition, FAA is documenting its decisions regarding acquisition investments to include whether a investment will require cost and schedule risk analysis, which supports the decisions. As a result, FAA is now taking into account the cost and schedule risk when making investment decisions regarding major acquisitions which puts the agency in a better position to minimize the risk of cost overruns and schedule delays.

    Recommendation: Given constrained budgets, FAA should determine which programs should be subject to these recommendations, such as those that are particularly costly, complex, or on a compressed schedule.

    Agency Affected: Department of Transportation: Federal Aviation Administration

 

Explore the full database of GAO's Open Recommendations »

Dec 7, 2016

Dec 2, 2016

Nov 25, 2016

Nov 22, 2016

Nov 17, 2016

Nov 15, 2016

Nov 9, 2016

Oct 31, 2016

Looking for more? Browse all our products here