NASA Acquisition Management
The National Aeronautics and Space Administration (NASA) plans to invest billions of dollars in the coming years to explore space, understand Earths environment, and conduct aeronautics research. GAO has designated NASAs acquisition management as high risk in view of NASAs history of persistent cost growth and schedule slippage in the majority of its major projects. GAOs work has identified a number of causal factors, including antiquated financial management systems, poor cost estimating, and underestimating risks associated with the development of its major systems. This area was added to GAOs High Risk List in 1990.
NASA has taken steps to address issues with its acquisition management function, which, as GAO has reported, have helped the agency to make progress in improving overall acquisition outcomes. For example, NASA has revised and implemented new policies, such as enhanced cost estimating, that have increased oversight of its projects both internally and externally. NASA leadership has also been focused on continuous monitoring and reporting of progress for its major projects. As a result of these efforts and others, NASA has been able to demonstrate progress in meeting its cost and schedule goals for some of its more recent projects. For example, in 2011, two of NASAs spacecraft projectsJuno and the Gravity Recovery and Interior Laboratorylaunched within their cost and schedule baselines. In addition, GAO reported in 2012 that many of the newer projects in the portfolio have not reported significant cost and schedule growth from established baselines.
Continued schedule and cost growth on other projects, however, indicates that it may take several years before it is apparent whether initiatives NASA has undertaken to improve its acquisition performance will be sustained and ultimately effective. For example, GAO reported that cost and schedule growth on one of NASAs most expensive and complex science projects, the James Webb Space Telescope (JWST), has had a significant impact on NASAs overall performance. JWST was rebaselined in 2011 with a $3.7 billion increase in lifecycle costs and a 52 month launch delay. Such a significant increase impacted NASAs ability to fund other important missions going forward. Even after acknowledging this large cost and schedule growth through rebaselining the project, GAO reported in December 2012 that the reliability of the JWST projects rebaselined cost estimate could have been strengthened. Also, GAO found that the JWSTs schedule lacked reserve flexibility in the latter, complex phases of the project which could challenge NASAs ability to complete the JWST project on schedule. Significant effort to ensure that other large, complex and expensive projectssuch as the Space Launch System and Orion Multi-Purpose Crew Vehicle, which are in early stages of developmentare planned and executed appropriately will be key to ensuring continued agency progress in meeting cost and schedule goals as issues with large projects often can have reverberating effects across the portfolio of projects.
NASA has taken steps to improve its acquisition management and continues to work to address systemic weaknesses by adopting practices that focus on closing gaps in knowledge about requirements, technology, design, funding, time, and other resources before commitments are made to a new project. Based upon findings of several GAO reports over the last 2 years, however, NASA needs to take additional steps to continue to refine its practices to better ensure that improvements are implemented effectively as an important part of managing and overseeing major project development. For example, since 2009, NASA has enhanced its cost-estimating methodologies to ensure that independent analyses are used to provide decision makers with an objective representation of likely project cost and schedule results. Specifically, NASA began using an estimation tool to calculate a particular Joint Cost and Schedule Confidence Level (JCL) for development cost and schedule estimates. NASA has adopted a policy to budget its projects at a 70 percent JCL, unless approved otherwise. NASAs budget documentation shows that for the most part, projects are reporting JCLs in line with this policy. However, in March 2012, GAO reported that there was a lack of uniformity in the methodology used to create the JCL. For example, some projects excluded or did not consider relevant risks, such as launch vehicle costs. In addition, in December 2012, GAO reported that the schedule used by the JWST project to conduct its JCL could impact its overall reliability. GAO reported in March 2012 that NASA has not yet launched a project that used a JCL to inform its budget and schedule baselines; therefore, NASA officials stated it will take several years to evaluate the impact of the JCL in improving cost and schedule estimating for its major projects.
NASA has implemented additional changes to its policies governing acquisition to enable managers to more effectively monitor a projects performance related to cost, schedule, and cross-cutting technical and nontechnical issues. Cultural, resource, and other issues, however, have led to less than optimal implementation of these efforts. For example, NASA has undertaken several initiatives aimed at improving the agencys use of Earned Value Management (EVM)a tool designed to help project managers monitor risks. Specifically, NASA recently strengthened its spaceflight management policy to reflect the industry EVM standard and has developed the processes and tools for projects to meet these standards through its new EVM system. While these are positive steps, GAO reported in November 2012 that EVM has not been fully and consistently implemented by NASAs major projects and as a result many projects lack reliable data for monitoring contractor performance. GAO reported that cultural influences resulted in a devaluing of cost and schedule data as a way to help manage projects, because traditionally project officials have focused more on addressing science and engineering challenges. Another challenge cited by NASA EVM experts, headquarters officials, and project and program representatives include a lack of staff with the skill set needed to analyze EVM data. GAO reported that these factors are impediments to the effective use of EVM at the agency. Reliable EVM data is one mechanism that NASA can use as a means to support identification of measurable project progress toward meeting cost and schedule goals.
GAO reported in 2011 that NASA lacked a common set of measurable and proven criteria, such as the percentage of engineering drawings employed at a key point in the development lifecycle, to provide evidence and insight to decision-makers that the requisite knowledge has been attained to allow the project to proceed. GAO recommended that NASA develop such criteria to provide NASA management with the information necessary to assess the performance of individual projects against the overall portfolio of projects. NASAs newer projects have come closer to meeting GAOs best practices metric for assessing design stability and increasing their design knowledge at critical design reviews. NASA, however, continues to lack its own proven, consistent metric for assessing design stability. In 2012, NASA took steps to address this issue and modified its systems engineering policy to improve its ability to monitor project progress throughout a projects development. Specifically, the new policy requires projects to monitor three technical indicators during the design process. While GAO agrees that this is a positive step to bring more focus to a projects design progress, there is a lack of data from NASAs projects to support the agencys metrics as indicators of design stability. Additional time will be required to monitor their effectiveness.
 The JCL is a probabilistic analysis that includes among other things, all cost and schedule elements, incorporates and quantifies potential risks, assesses the impacts of cost and schedule to date, and addresses available annual resources to arrive at development cost and schedule estimates associated with various confidence levels.
GAO has previously reported that NASA implemented a plan for improvement for how it manages its acquisitions, which included points of accountability and metrics to assess progress. The ultimate test of whether the plan is successful and improvements NASA has made over the past several years are effective is whether the agency can demonstrate sustained positive outcomes in controlling cost and schedule growth across its portfolio of major projects. Key to doing so is ensuring that improvements to its acquisition policies and practices are effectively implemented and refined as needed. This includes, for example, the JCL approach, strengthened EVM requirements, and metrics to consistently and effectively assess design stability. Successful implementation of such improvements will gain even more importance in an increasingly constrained fiscal environment when there may be limited, if any, additional resources available for projects that overrun on cost and schedule without major impacts to the mission. GAO will monitor NASAs efforts in this regard to determine whether it can sustain and expand the progress made thus far.