NASA:

Earned Value Management Implementation across Major Spaceflight Projects Is Uneven

GAO-13-22: Published: Nov 19, 2012. Publicly Released: Nov 19, 2012.

Additional Materials:

Contact:

Cristina T. Chaplain
(202) 512-4859
chaplainc@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The National Aeronautics and Space Administration's (NASA) 10 major spaceflight projects discussed in this report have not yet fully implemented earned value management (EVM). As a result, NASA is not taking full advantage of opportunities to use an important tool that could help reduce acquisition risk. GAO assessed the 10 projects against three fundamental EVM practices that, according to GAO's best practices cost guide, are necessary for maintaining a reliable EVM system. GAO found shortfalls in two of three fundamental practices. Specifically, we found that

  • More than half of the projects did not use an EVM system that was fully certified as compliant with the industry EVM standard.
  • Only 4 of the 10 projects established formal surveillance reviews, which ensure that key data produced by the system was reliable. The remaining 6 projects provided evidence of monthly EVM data reviews; however, the rigor of both the formal and informal surveillance reviews is questionable given the numerous data anomalies GAO found.

GAO also found that 3 projects had reliable EVM data while 7 had only partially reliable data. For the EVM data to be considered reliable per best practices it must be complete and accurate with all data anomalies explained.

NASA EVM focal points, headquarters officials, project representatives, and program executives cited cultural and other challenges as impediments to the effective use of EVM at the agency. Traditionally, NASA's culture has focused on managing science and engineering challenges and not on monitoring cost and schedule data, like an effective EVM system produces. As a result, several representatives said this information traditionally has not been valued across the agency. This sentiment was also echoed in a NASA study of EVM implementation. Also cited as a challenge to the effective use of EVM was NASA's insufficient number of staff with the skills to analyze EVM data. Without a sufficient number of staff with such skills, NASA's ability to conduct a sound analysis of the EVM data is limited. However, NASA has not conducted an EVM skills gap analysis to determine the extent of its workforce needs.

NASA has undertaken several initiatives aimed at improving the agency's use of EVM. For example, NASA strengthened its spaceflight management policy to reflect the industry EVM standard and has developed the processes and tools for projects to meet these standards through its new EVM system. While these are positive steps, the revised policy contains only the minimum requirements for earned value management. For example, it lacks a requirement for rigorous surveillance of how projects are implementing EVM and also does not require use of the agency's newly developed EVM system to help meet the new requirements. NASA has attempted to address EVM shortcomings through policy changes over the years, but these efforts have failed to adequately address the cultural resistance to implementing EVM.

Why GAO Did This Study

NASA historically has experienced cost growth and schedule slippage in its portfolio of major projects and has taken actions to improve in this area, including adopting the use of EVM. EVM is a tool developed to help project managers monitor risks. GAO was asked to examine (1) the extent to which NASA is using EVM to manage its major space flight acquisitions, (2) the challenges that NASA has faced in implementing an effective EVM system, and (3) NASA's efforts to improve its use of EVM. To address these questions, GAO obtained contractor and project EVM data and used established formulas and tools to analyze the data and assess NASA's implementation of EVM on 10 major spaceflight projects; interviewed relevant NASA headquarters, center and mission directorate officials on their views on EVM; and reviewed prior reports on EVM and organizational transformations. GAO compared NASA policies and guidance on EVM to best practices contained in GAO's cost estimating best practices guide.

What GAO Recommends

GAO recommends that NASA establish a time frame for requiring new spaceflight projects to implement its new EVM system; conduct an EVM skills gap assessment; develop a change management plan for EVM; and strengthen its EVM requirements by requiring projects to implement formal EVM surveillance. NASA concurred with two recommendations and partially concurred with two others citing resource constraints. Despite NASA's plans to address some issues GAO identified, not addressing all key issues lessens the usefulness of EVM at NASA.

For more information, contact Cristina Chaplain at (202) 512-4841 or chaplainc@gao.gov.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To improve NASA management and oversight of its spaceflight projects, the NASA Administrator should direct the appropriate offices to establish a time frame by which all new spaceflight projects will be required to implement NASA's newly developed EVM system, unless the project is proposing to use a certified system, to ensure that in-house efforts are compliant with ANSI/EIA-748. The time frame selected should take in to account the need to increase NASA's institutional capability for conducting EVM and analyzing and reporting the data.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: In fiscal year 2013, NASA started its phased rollout by implementing the EVM Capability process on the Space Launch System project at the Marshall Space Flight Center and the Ice, Clouds and Land Elevation Satellite project at Goddard Space Flight Center. The projects receive implementation support from the agency as part of the rollout. During implementation, the respective Center is to develop an institutional capability to support future projects. Currently, NASA is in the process of identifying other projects at different centers for rollout in fiscal year 2014. The plan is to support implementation of the common agency EVM process at each NASA center. However, NASA has not established a time frame by which all new spaceflight projects will be required to implement NASA's newly developed EVM process.

    Recommendation: To improve NASA management and oversight of its spaceflight projects, the NASA Administrator should direct the appropriate offices to conduct an EVM skills gap analysis to identify areas requiring augmented capability across the agency. Based on the results of the assessment, develop a workforce training plan to address any deficiencies.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: NASA completed a skills gap survey in May 2013. Currently, NASA is analyzing the responses and expects to complete it by the end of July 2013. The results will be used to develop a workforce training plan in September 2013.

    Recommendation: To improve NASA management and oversight of its spaceflight projects, the NASA Administrator should direct the appropriate offices to develop an EVM change management plan to assist managers and employees throughout the agency with accepting and embracing earned value techniques while reducing the operational impact on the agency. The plan should include a strategy for having the agency's senior leadership communicate their commitment to implementation of EVM.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: NASA has developed an outline for a change management plan. The agency is waiting for the final analysis of the EVM skills gap survey, expected in September 2013, which will be used to finalize the change management plan. In the interim, the agency has been trying to influence culture change by frequently briefing the Program and Project Management Board, which is comprised of senior level personnel from each Center and Mission Directorate. The Program and Project Management Board is kept up to date on the current status of NASA's implementation of the EVM Capability, EVM survey participation, change management, and other activities. NASA has also setup an EVM Steering Committee that consists of senior level NASA officials and provides guidance to the agency on the implementation of EVM. The EVM steering committee is briefed as needed on issues such as budgets, challenges, and perspective projects for rollout of the EVM Capability. The Steering Committee and the Program and Project Management Board are in the process of identifying perspective project(s) for rollout of the EVM Capability in fiscal year 2014. NASA explained that the EVM Steering Committee has provided support for increasing the NASA EVM budget as scope of the implementation effort increases. However, any decision on the final agency EVM budget will be based on NASA's final budget for fiscal year 2014 and outyears. According to agency officials, they have seen improvement in senior management sponsorship as evidenced by a strong participation rate for the recently conducted EVM skills gap survey. The Office of the Chief Engineer has been coordinating with the Independent Program Assessment Office to refine processes associated with EVM surveillance as part of the review. A communication strategy will be implemented to ensure all parties have a common expectation for surveillance.

    Recommendation: To improve NASA management and oversight of its spaceflight projects, and to improve the reliability of project EVM data, the NASA Administrator should direct the appropriate offices to modify the NASA Procedural Requirements (NPR) 7120.5 to require projects to implement a formal surveillance program that: (1) Ensures anomalies in contractor-delivered and in-house monthly earned value management reports are identified and explained, and report periodically to the center and mission directorate's leadership on relevant trends in the number of unexplained anomalies. (2) Ensures consistent use of work breakdown structures (WBS) for both the EVM report and the schedule. (3) Ensures that lower level EVM data reconcile to project level EVM data using the same WBS structure. (4) Improves underlying schedules so that they are properly sequenced using predecessor and successor dependencies and are free of constraints to the extent practicable so that the EVM baseline is reliable.

    Agency Affected: National Aeronautics and Space Administration

    Status: Open

    Comments: NASA has not modified its NASA Procedural Requirements (NPR) 7120.5 to require projects to implement a formal surveillance program. NASA has, however, taken steps to provide information on implementing EVM surveillance and assessment. For example, NASA finalized its EVM Handbook which contains an acceptance and surveillance process. The EVM handbook is now available on the NASA EVM website for all NASA projects, though use of the handbook is not required. In addition, NASA has a plan to conduct EVM Assessments to test the maturity of its largest and most complex projects (Category I projects) during major milestone reviews. The agency has several projects scheduled to go through this assessment in fiscal year 2014. The agency also has scheduled an EVM Validation review at the Applied Physics Laboratory facility during fiscal years 2013 and 2014. The agency's EVM acceptance and surveillance process will be used during this review. In addition, NASA has developed automated tools to assess schedules and EVM data reliability. The plan is to rollout the tools to projects during fiscal years 2013-2014. The tools are expected to assist the projects to identify data anomalies so that the project can take corrective actions.

    Jul 16, 2014

    Jun 20, 2014

    May 15, 2014

    May 8, 2014

    Apr 15, 2014

    Mar 12, 2014

    Feb 4, 2014

    Jan 13, 2014

    Jan 8, 2014

    Sep 19, 2013

    Looking for more? Browse all our products here