This is the accessible text file for GAO report number GAO-13-276SP 
entitled 'NASA: Assessments of Selected Large-Scale Projects' which 
was released on April 17, 2013. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Committees: 

April 2013: 

NASA: 

Assessments of Selected Large-Scale Projects: 

GAO-13-276SP: 

GAO Highlights: 

Highlights of GAO-13-276SP, a report to congressional committees. 

Why GAO Did This Study: 

This is GAO’s fifth annual assessment of NASA’s major projects. This 
report provides a snapshot of how well NASA is planning and executing 
its major acquisitions. Due to persistent cost and schedule growth 
associated with its major projects, this area is on GAO's high risk 
list. GAO previously reported that NASA has taken steps to address its 
acquisition management issues and was making progress toward improving 
the cost and schedule performance of its major projects. 

This report provides observations about the cost and schedule 
performance of NASA's major projects, identifies factors that have 
contributed to this condition, and highlights challenges to NASA’s 
management of the portfolio. To conduct this review, GAO assessed data 
on 18 current projects with an estimated life-cycle cost of over $250 
million, including data on projects’ cost, schedule, technology 
maturity, design stability, and contracts; analyzed monthly project 
status reports; and interviewed NASA and contractor officials. 

What GAO Found: 

The performance of the National Aeronautics and Space Administration’s 
(NASA) portfolio of major projects has improved in the areas of cost 
and schedule growth since GAO’s first assessment in 2009. Average 
development cost growth and schedule delay for the current portfolio 
have decreased to about a third of their 2009 levels. 

Figure: Average Development Cost and Schedule Growth of Selected Major 
NASA Projects in the Implementation Phase, Excluding JWST: 

[Refer to PDF for image: vertical bar graph] 

Year: 2009; 
Average development cost growth: 12%; 
Average launch delay: 11 months. 

Year: 2010; 
Average development cost growth: 13.6%; 
Average launch delay: 12 months. 

Year: 2011; 
Average development cost growth: 17.8%; 
Average launch delay: 9 months. 

Year: 2012; 
Average development cost growth: 14.6%; 
Average launch delay: 8 months. 

Year: 2013; 
Average development cost growth: 3.9%; 
Average launch delay: 4 months. 

Source: GAO analysis of NASA data. 

[End of figure] 

These figures exclude the cost and schedule growth of the James Webb 
Space Telescope (JWST), NASA’s most expensive science project, in part 
because of its disproportionate effect on the portfolio average. 
Including the JWST in the calculation would increase the 2013 portfolio’
s average development cost growth from 3.9 percent to 46.4 percent and 
would double the average launch delay, from 4 to 8 months and obscure 
the progress the rest of the portfolio has made toward maintaining 
cost and schedule baselines. Of the 12 projects in implementation, 9 
reported no development cost growth and or launch schedule delay in 
the past year, but 2 of these are currently facing cost and/or 
schedule pressures. Three projects reported development cost growth or 
a launch delay, but for two projects the impetus was outside of the 
project's direct control. 

A number of factors appear to contribute to NASA's improved 
performance. For example, in prior reviews, a majority of projects 
exceeded their cost and schedule baselines. Most of these projects, 
however, have launched and are no longer affecting the portfolio. 
Consistent with prior recommendations, projects have also demonstrated 
some gains toward meeting best practices criteria for technology 
maturity and design stability. GAO has reported that conformity with 
these practices decreases cost and schedule risk. For example, 62 
percent of the projects met technology maturity criteria this year as 
compared to 29 percent in 2010. Current projects also appear to be 
incorporating less technology risk, as the number of critical 
technologies per project has decreased from 4.7 in 2009 to 2.3 in 
2013. NASA has also implemented new management practices that have 
likely contributed to improved performance, in part by increasing 
oversight. 

Continued leadership attention will be needed to ensure that good 
practices are maintained in the face of several challenges including: 
(1) managing competing priorities within the context of constrained 
budgets, (2) estimating costs associated with several large-scale 
projects, (3) improving overall cost and schedule estimation, and (4) 
using consistent and proven design stability metrics. 

What GAO Recommends: 

GAO is not making recommendations in this report, but is highlighting 
several challenges for NASA's attention, including managing competing 
priorities and improving cost and schedule estimating practices. GAO 
has made prior recommendations aimed at improving oversight, including 
improving the use of earned value management, implementing design 
stability best practices, and providing transparency into costs. NASA 
agreed with GAO's assessment of its progress and remaining challenges 
and stressed its commitment to sustaining progress. 

View [hyperlink, http://www.gao.gov/products/GAO-13-276SP]. For more 
information, contact Cristina Chaplain at (202) 512-4841 or 
chaplainc@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

Observations on NASA's Portfolio of Major Projects: 

Contributing Factors to the Portfolio's Current Performance: 

Challenges Remain That Could Affect Continued Progress: 

Project Assessments: 

ExoMars Trace Gas Orbiter: 

Global Precipitation Measurement Mission: 

Ice, Cloud, and Land Elevation Satellite-2: 

James Webb Space Telescope: 

Landsat Data Continuity Mission: 

Lunar Atmosphere and Dust Environment Explorer: 

Magnetospheric Multiscale: 

Mars Atmosphere and Volatile EvolutioN: 

Orbiting Carbon Observatory 2: 

Origins-Spectral Interpretation-Resource Identification-Security-
Regolith Explore: 

Orion Multi-Purpose Crew Vehicle: 

Radiation Belt Storm Probes: 

Soil Moisture Active and Passive: 

Solar Probe Plus: 

Space Launch System: 

Space Network Ground Segment Sustainment: 

Stratospheric Observatory for Infrared Astronomy: 

Tracking and Data Relay Satellite Replenishment: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Selected Major NASA Projects Reviewed in GAO's Annual 
Assessments: 

Appendix III: Technology Readiness Levels: 

Appendix IV: Elements of a Sound Business Case: 

Appendix V: Best-Practices Analysis of Products from Three NASA 
Projects: 

Appendix VI: Comments from the National Aeronautics and Space 
Administration: 

Appendix VII: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: 18 Selected Major NASA Projects Reviewed in GAO's 2013 Annual 
Assessment: 

Table 2: Development Cost and Schedule Growth of Selected Major NASA 
Projects Currently in the Implementation Phase: 

Table 3: Percent of New Mission Critical Software and Critical 
Technologies Reported by Selected NASA Major Projects: 

Table 4: Selected Major NASA Projects and Programs Reviewed in GAO's 
Annual Assessments. 

Table 5: Summary Assessment of GPM Schedule Estimate Compared to Best 
Practices: 

Table 6: Summary Assessment of MMS Schedule Estimate Compared to Best 
Practices: 

Figures: 

Figure 1: NASA's Life Cycle for Flight Systems: 

Figure 2: Average Development Cost and Schedule Growth of Selected 
Major NASA Projects in the Implementation Phase, Excluding JWST: 

Figure 3: Selected Major NASA Projects with Development Cost Growth 
and the Portfolio Average Development Cost Growth, 2009-2013: 

Figure 4: Percentage of Selected Major NASA Projects Meeting and Not 
Meeting Technology Maturity Criteria at Preliminary Design Review: 

Figure 5: Average Number of Critical Technologies Reported for 
Selected Major NASA Projects Reviewed from 2009 to 2013: 

Figure 6: Percentage of Engineering Drawings Releasable at CDR for 
Selected Major NASA Projects: 

Figure 7: Percentage of Releasable Drawings for Selected Projects 
Reviewed That Held a Critical Design Review: 

Figure 8: Average Percentage of Drawing Growth after CDR For Selected 
Projects from Fiscal Years 2010 through 2013: 

Figure 9: Fiscal Year 2013 Budget Request for JWST, MPCV, SLS, and All 
Other Major NASA Projects, 2013 through 2017: 

Figure 10: Illustration of a Project's Two-Page Summary: 

Abbreviations: 

AFB: Air Force Base: 

AFS: Air Force Station: 

ATLAS: Advanced Topographic Laser Altimeter System: 

CDR: critical design review: 

DCI: data collection instrument: 

EFT: exploration flight test: 

EMTGO: ExoMars Trace Gas Orbiter: 

ESA: European Space Agency: 

EVM: earned value management: 

GLAST: Gamma-ray Large Area Space Telescope: 

GPM: Global Precipitation Measurement (mission): 

GRAIL: Gravity Recovery and Interior Laboratory: 

HEPS: High Efficiency Power Supply: 

ICESat-2: Ice, Cloud, and Land Elevation Satellite-2: 

ISIM: Integrated Science Instrument Module: 

JAXA: Japan Aerospace Exploration Agency: 

JCL: Joint Cost and Schedule Confidence Level: 

JWST: James Webb Space Telescope: 

KDP: key decision point: 

LADEE: Lunar Atmosphere and Dust Environment Explorer: 

LDCM: Landsat Data Continuity Mission: 

LRO: Lunar Reconnaissance Orbiter: 

MagEIS: Magnetic Electron Ion Spectrometer: 

MAVEN: Mars Atmosphere and Volatile EvolutioN: 

MDR: mission definition review: 

MMS: Magnetospheric Multiscale: 

MPCV: Multi-Purpose Crew Vehicle: 

MSL: Mars Science Laboratory: 

NASA: National Aeronautics and Space Administration: 

NGIMS: Neutral Gas and Ion Mass Spectrometer: 

NPR: NASA Procedural Requirements: 

NPOESS: National Polar-orbiting Operational Environmental Satellite 
System: 

NPP: NPOESS Preparatory Project: 

OCFO: Office of the Chief Financial Officer (NASA): 

OCO: Orbiting Carbon Observatory: 

OSIRIS-REx: Origins-Spectral Interpretation-Resource Identification-
Security-Regolith Explorer: 

PDR: preliminary design review: 

QMS: Quadrupole Mass Spectrometer: 

RBSP: Radiation Belt Storm Probes: 

SDO: Solar Dynamics Observatory: 

SDR: system definition review: 

SGSS: Space Network Ground Segment Sustainment: 

SIR: system integration review: 

SLS: Space Launch System: 

SMAP: Soil Moisture Active and Passive: 

SOFIA: Stratospheric Observatory for Infrared Astronomy: 

SPP: Solar Probe Plus: 

TDRS: Tracking and Data Relay Satellite: 

TIRS: Thermal Infrared Sensor: 

TPS: Thermal Protection System: 

TRL: technology readiness level: 

USGS: U.S. Geological Survey: 

WISE: Wide-field Infrared Survey Explorer: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

April 17, 2013: 

Congressional Committees: 

This is GAO’s fifth annual assessment of the National Aeronautics and 
Space Administration’s (NASA) major projects. This report provides a 
snapshot of how well NASA is planning and executing its major 
acquisitions—-an area that has been on GAO’s high risk list since 
1990. Over the past 5 years, our review has covered a range of 
projects, from highly complex and sophisticated space transportation 
vehicles, to robotic probes, to satellites equipped with advanced 
sensors to study the earth. During the last 2 years, NASA has launched 
eight projects, of which six have begun returning science results. For 
example, in June 2011, NASA launched the Aquarius project. The data 
returned from Aquarius made possible the first global map of the 
salinity of the ocean surface, which allows scientists to study how 
the ocean’s salinity influences Earth’s climate. The Gravity Recovery 
and Interior Laboratory (GRAIL), launched in September 2011, has 
created the most accurate gravity map of the Moon’s surface, allowing 
scientists to learn about the Moon’s internal structure and 
composition in unprecedented detail. NASA’s latest Mars rover, 
Curiosity, successfully landed and has begun beaming back images and 
data to scientists, such as the first indication that a stream once 
flowed vigorously across Mars. Notably, NASA has launched two missions 
in the last 2 years—-GRAIL and Juno-—within their established cost and 
schedule commitments. 

Despite these successes, the effects of the additional resources 
needed to support NASA’s largest science project—-the James Webb Space 
Telescope (JWST)—-began to resonate as the project identified the need 
for an additional $1.4 billion from fiscal years 2012 through 2017. At 
the same time, NASA’s portfolio of major projects has decreased in 
size. For example, while eight science projects launched in the past 2 
years and transitioned into operations, only two have entered the 
portfolio in that same period. One of those two projects, the ExoMars 
Trace Gas Orbiter, was subsequently proposed for termination by NASA 
in 2012. 

The explanatory statement of the House Committee on Appropriations 
accompanying the Omnibus Appropriations Act of 2009 directed GAO to 
prepare project status reports on selected large-scale NASA programs, 
projects, or activities. This report responds to that mandate. 
Specifically, we (1) assess the performance of NASA’s portfolio of 
major projects in terms of cost and schedule; (2) identify factors 
that have contributed to the portfolio’s current performance; and (3) 
highlight remaining challenges to continued improvement. The report 
expands on the importance of providing decision makers with an 
independent, knowledge-based assessment of individual systems that 
identifies potential risks and allows them to take actions to put 
projects that are early in the development cycle in a better position 
to succeed. We are not making recommendations in this report, but we 
are highlighting several key areas for NASA management’s attention, 
including continued focus on implementing positive management 
practices. We have, however, made prior recommendations aimed at 
improving oversight of NASA’s projects, including improving the use of 
earned value management, implementing best practices for design 
stability and technology maturity, and providing more transparency 
into project costs. 

Our approach included an examination of 18 major projects each with an 
estimated life-cycle cost of over $250 million. These projects were in 
either the formulation phase or the implementation phase of the 
project life cycle. In the formulation phase, the project defines 
requirements—-(i.e., what the project is being designed to do)-—
matures technology, establishes a schedule, estimates costs, and 
produces a plan for implementation. In the implementation phase, the 
project carries out these plans, performing final design and 
fabrication as well as testing components and system assembly, 
integrating these components and testing how they work together, and 
launching the project. This phase also includes the period from a 
project’s launch through mission closeout. 

In order to assess the performance of NASA projects in terms of cost 
and schedule, we compared the projects’ baseline cost and schedule 
data with cost and schedule data that was current as of February 2013 
for projects in implementation during our review, or 12 of the 18 
major projects.[Footnote 1] The remaining 6 projects were in 
formulation and NASA provided preliminary cost and schedule estimates 
for 5 of the 6. We reviewed and compared the 12 projects’ current cost 
and schedule data to previously established cost and schedule 
baselines as applicable. We assessed the projects’ cost and schedule 
and characterized growth as significant if it exceeded the thresholds 
that trigger cost or schedule growth reporting to certain 
congressional committees under the law.[Footnote 2] To identify 
factors that contributed to the portfolios’ current performance, we 
identified the number of technologies and lines of software code each 
project was developing, reviewed historical data on past projects and 
compared it to current project performance, and compared projects’ 
technology maturity and design stability against established criteria 
for knowledge-based acquisitions and other GAO work on system 
acquisitions.[Footnote 3] To identify remaining challenges to 
continued improvement, we principally relied on outstanding issues 
identified in our prior work on NASA, such as earned value management 
implementation issues and cost and schedule growth on NASA’s most 
technologically advanced and costly projects. We examined how NASA is 
managing its large and complex missions within the current budget 
environment, and assessed the extent to which NASA has implemented GAO’
s prior recommendation that the agency develop a consistent set of 
proven metrics to assess design stability. In addition, we assessed 
the schedules for three major projects against best practices. 
[Footnote 4] As a result of our interviews with project officials and 
analysis of information provided by the projects, we identified other 
challenges—launch, contractor management, parts, development partner, 
test and integration, workforce, and funding—that can affect project 
outcomes and are reported on the two-page project summaries. This list 
of challenges is not exhaustive, and we believe these challenges will 
evolve, as they have in previous years, as we continue this work in 
the future. We took appropriate steps to address data reliability, 
such as clarifying data discrepancies and corroborating NASA-generated 
data with other sources where applicable. We determined that the data 
were reliable enough for our purposes. The individual project offices 
were given an opportunity to provide comments and technical 
clarifications on our assessments prior to their inclusion in the 
final product, which were incorporated as appropriate. Appendix I 
contains detailed information on our scope and methodology. 

We conducted this performance audit from March 2012 to April 2013 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

NASA's Life Cycle for Flight Systems: 

NASA's life cycle for flight systems is defined by two phases--
formulation and implementation--and several key decision points. 
[Footnote 5] These phases are then further divided into incremental 
pieces: phase A through phase F. See figure 1 for a depiction of 
NASA's life cycle for flight systems. 

Figure 1: NASA's Life Cycle for Flight Systems: 

[Refer to PDF for image: life cycle diagram] 

Formulation: 

Pre-phase A: Concept Studies; 
KDP A. 

Phase A: Concept and Technology Development; 
KDP B; 
SDR/MDR. 

Phase B: Preliminary Design and Technology Completion; 
KDP C (confirmation review); 
PDR. 

Program Start: 

Phase C: Final Design and Fabrication; 
CDR; 
KDP D; 
SIR. 

Phase D: System Assembly, Integration and Test, Launch; 
KDP E. 

Phase E: Operations and Sustainment; 
KDP F. 

Phase F: Closeout. 

Implementation. 

Management decision reviews: 
KDP = key decision point. 

Technical reviews: 
SDR/MDR = system definition review/mission definition review; 
PDR = preliminary design review; 
CDR = critical design review; 
SIR = system integration review. 

Source: NASA data and GAO analysis. 

[End of figure] 

Project formulation consists of phases A and B, during which time the 
projects develop and define requirements and the cost/schedule basis 
and design for implementation, including developing an acquisition 
strategy. During the end of the formulation phase, leading up to the 
preliminary design review (PDR), the project team completes its 
preliminary design and technology development.[Footnote 6] NASA 
Procedural Requirements 7120.5E, NASA Space Flight Program and Project 
Management Requirements, specifies that during formulation, the 
project must complete a formulation agreement to establish the 
technical and acquisition work that needs to be conducted during this 
phase and define the schedule and funding requirements for that work. 
The formulation agreement identifies new technologies and their 
planned development, the use of heritage technologies, risk mitigation 
plans, and testing plans to ensure that technologies will work as 
intended in a relevant environment. The project also develops and 
maintains the status of a set of programmatic and technical leading 
indicators to ensure proper progress and management of the project or 
single-project program is achieved during the formulation phase. The 
formulation phase culminates in a review at key decision point C, 
known as project confirmation, where cost and schedule baselines are 
confirmed and documented in the agency baseline commitment.[Footnote 
7] Project progress can subsequently be measured against these 
baselines. 

After a project is confirmed, it begins implementation, consisting of 
phases C, D, E, and F. Senior NASA officials must approve the project 
before it can proceed from one phase of implementation to another. A 
second design review, the critical design review (CDR), is held during 
the latter half of phase C in order to determine if the design is 
stable enough to support proceeding with the final design and 
fabrication.[Footnote 8] After CDR and just prior to beginning phase 
D, the project completes a system integration review (SIR) to evaluate 
the readiness of the project and associated supporting infrastructure 
to begin system assembly, integration and test.[Footnote 9] In phase 
D, the project performs system assembly, integration, test, and launch 
activities. Phases E and F consist of operations and sustainment and 
project closeout. 

NASA Projects Reviewed in GAO's Annual Assessment: 

NASA's mission is to drive advances in science, technology, and 
exploration to enhance knowledge, education, innovation, economic 
vitality, and stewardship of the Earth. To accomplish its mission, 
NASA establishes many programs and projects that rely on complex 
instruments and spacecraft. NASA's portfolio of major projects ranges 
from space satellites equipped with advanced sensors to study the 
earth, to a spacecraft which plans to return a sample from an 
asteroid, to telescopes intended to explore the universe, to 
spacecraft to transport humans and cargo beyond low-Earth orbit. Some 
of NASA's projects are expected to incorporate new and sophisticated 
technologies that must operate in harsh, distant environments. This 
year, we assessed 18 major projects--6 projects in formulation and 12 
projects in implementation. Three of the 12 projects in 
implementation--Radiation Belt Storm Probes (RBSP), Tracking and Data 
Relay Satellite Replenishment (TDRS K), and Landsat Data Continuity 
Mission (LDCM)--successfully launched during 2012 and 2013. The year 
after a project launches, we no longer include a two-page summary in 
our annual report. When NASA determines that a project will have a 
life-cycle cost estimate of more than $250 million, we include that 
project in the next review. See table 1 for a list of the projects we 
reviewed in this year's assessment, and appendix II for a list of 
projects that we have reviewed from 2009 to 2013. 

Table 1: 18 Selected Major NASA Projects Reviewed in GAO's 2013 Annual 
Assessment: 

Projects in formulation: 
ExoMars Trace Gas Orbiter (EMTGO)[A]; 
Orion Multi-Purpose Crew Vehicle (MPCV); 
Origins-Spectral Interpretation-Resource Identification-Security-
Regolith Explorer (OSIRIS-REx); 
Solar Probe Plus (SPP); 
Space Launch System (SLS); 
Space Network Ground Segment Sustainment (SGSS)[B]. 

Projects in implementation: 
Global Precipitation Measurement (GPM) Mission; 
Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2); 
James Webb Space Telescope (JWST); 
Landsat Data Continuity Mission (LDCM)[C]; 
Lunar Atmosphere and Dust Environment Explorer (LADEE); 
Magnetospheric Multiscale (MMS); 
Mars Atmosphere and Volatile EvolutioN (MAVEN); 
Orbiting Carbon Observatory 2 (OCO-2); 
Radiation Belt Storm Probes (RBSP)[C]; 
Soil Moisture Active and Passive (SMAP); 
Stratospheric Observatory for Infrared Astronomy (SOFIA); 
Tracking and Data Relay Satellite (TDRS) Replenishment K[C] and L[D]. 

Source: GAO analysis of NASA data. 

[A] In February 2012, NASA proposed canceling the EMTGO project as 
part of its fiscal year 2013 budget request. 

[B] SGSS entered the implementation phase in April 2013, after our 
review of projects had concluded. This project is not included in our 
analyses of projects in implementation, however, the project's 
baseline cost and schedule is reported in the two page assessment. 

[C] NASA projects that launched during our review. 

[D] TDRS Replenishment includes TDRS-K and TDRS-L. These satellites 
will launch separately, but are counted as only one mission. 

[End of table] 

Observations on NASA's Portfolio of Major Projects: 

When Excluding JWST, Portfolio Development Cost and Schedule Growth 
Have Significantly Improved Since Our Initial Review in 2009: 

In 2009, the major NASA projects in our review that were in the 
implementation phase had an average development cost growth of 12 
percent and an average launch delay of 11 months. In the current 
portfolio of major projects, excluding JWST, the average development 
cost growth and launch delay of projects in implementation has 
decreased to a third of 2009 levels. As seen in figure 2 below, the 
portfolio of projects currently in implementation--excluding JWST--has 
an average development cost growth of 3.9 percent and an average 
launch delay of 4 months. 

Figure 2: Average Development Cost and Schedule Growth of Selected 
Major NASA Projects in the Implementation Phase, Excluding JWST: 

[Refer to PDF for image: vertical bar graph] 

Year: 2009; 
Average development cost growth: 12%; 
Average launch delay: 11 months. 

Year: 2010; 
Average development cost growth: 13.6%; 
Average launch delay: 12 months. 

Year: 2011; 
Average development cost growth: 17.8%; 
Average launch delay: 9 months. 

Year: 2012; 
Average development cost growth: 14.6%; 
Average launch delay: 8 months. 

Year: 2013; 
Average development cost growth: 3.9%; 
Average launch delay: 4 months. 

Source: GAO analysis of NASA data. 

Note: The calculations do not include the cost or schedule growth 
experienced by the JWST project. 

[End of figure] 

For the purposes of this analysis we excluded JWST cost and schedule 
growth from the calculations for the portfolios because including them 
would mask the progress that the rest of the portfolio has made toward 
maintaining cost and schedule. Of those projects in implementation, 
JWST makes up 51 percent of the total development costs and including 
it in the portfolio calculation would increase the 2013 average 
development cost growth of projects in implementation by nearly a 
factor of 12, from 3.9 percent to 46.4 percent. It would also double 
the average launch delay from 4 months to 8 months. Nonetheless, while 
identifying progress within the portfolio is important, as a result of 
JWST's cost growth NASA will likely have less flexibility to address 
issues on other projects, should they arise. In addition, cost growth 
associated with projects over the past 5 years of our review has also 
limited the agency's flexibility to manage its entire portfolio and 
implement additional projects to achieve its mission. For example, 
while NASA has only had to find $119 million over the past year due to 
project cost growth, the agency has had to find $5.2 billion over the 
last 5 years--a 24 percent cost increase for projects in 
implementation--due to project cost growth. 

Most Projects in the Portfolio Have Not Reported Cost and Schedule 
Growth in the Last Year, but a Few Projects Are Experiencing Cost and/ 
or Schedule Pressure: 

NASA has had success in the last 2 years in launching missions on cost 
or on schedule. For example, the RBSP project is the third mission in 
the last 2 years that has launched on cost, building on the prior 
year's launches of the GRAIL and Juno missions that both launched 
within their cost and schedule baseline estimates. 

Of the 12 projects in implementation, 9 reported no development cost 
growth and no launch schedule delay in the last year. Of those 
projects that have not reported cost and schedule growth in the last 
year, some have been replanned, baselined, or rebaselined within the 
last 2 years and in one instance cut capabilities.[Footnote 10] 
Specifically, the Global Precipitation Measurement (GPM) project 
reduced development costs in its fiscal year 2012 replan by 
eliminating an instrument from the development plan that project 
officials stated will impact the science data returned by the mission. 
See table 2 below for more details on projects' development cost and 
schedule growth reported this year and against their baselines. 

Table 2: Development Cost and Schedule Growth of Selected Major NASA 
Projects Currently in the Implementation Phase: 

Dollars in millions. 

Project: GPM[A]; 
Cumulative development cost growth: -$45.3; 
Percentage cost growth: -8.2%; 
Development growth reported in last year: -$9.4; 
Cumulative launch delay (months): 11; 
Launch delay reported in last year (months): 0. 

Project: ICESat-2[B]; 
Cumulative development cost growth: 0.0; 
Percentage cost growth: 0.0; 
Development growth reported in last year: 0.0; 
Cumulative launch delay (months): 0; 
Launch delay reported in last year (months): 0. 

Project: LADEE; 
Cumulative development cost growth: $7.9; 
Percentage cost growth: 4.7%; 
Development growth reported in last year: $0.3; 
Cumulative launch delay (months): 0; 
Launch delay reported in last year (months): 0. 

Project: LDCM[C]; 
Cumulative development cost growth: -$6.2; 
Percentage cost growth: -1.1%; 
Development growth reported in last year: 0.0; 
Cumulative launch delay (months): -4; 
Launch delay reported in last year (months): -4. 

Project: MAVEN; 
Cumulative development cost growth: -$15.5; 
Percentage cost growth: -2.7%; 
Development growth reported in last year: -$15.5; 
Cumulative launch delay (months): 0; 
Launch delay reported in last year (months): 0. 

Project: MMS; 
Cumulative development cost growth: -$0.1; 
Percentage cost growth: 0.0; 
Development growth reported in last year: 0.0; 
Cumulative launch delay (months): 0; 
Launch delay reported in last year (months): 0. 

Project: OCO-2[D]; 
Cumulative development cost growth: $122.8; 
Percentage cost growth: 49.3%; 
Development growth reported in last year: $122.8; 
Cumulative launch delay (months): 24; 
Launch delay reported in last year (months): 24. 

Project: RBSP[E]; 
Cumulative development cost growth: -$23.6; 
Percentage cost growth: -4.4%; 
Development growth reported in last year: -$20.6; 
Cumulative launch delay (months): 4; 
Launch delay reported in last year (months): 0. 

Project: SMAP[F]; 
Cumulative development cost growth: 0.0; 
Percentage cost growth: 0.0; 
Development growth reported in last year: 0.0; 
Cumulative launch delay (months): 0; 
Launch delay reported in last year (months): 0. 

Project: SOFIA; 
Cumulative development cost growth: $208.9; 
Percentage cost growth: 22.7%; 
Development growth reported in last year: 0.0; 
Cumulative launch delay (months): 12; 
Launch delay reported in last year (months): 0. 

Project: TDRS Replenishment[G]; 
Cumulative development cost growth: -$24.8; 
Percentage cost growth: -11.8%; 
Development growth reported in last year: $1.0; 
Cumulative launch delay (months): 2; 
Launch delay reported in last year (months): 2. 

Portfolio excluding JWST: 

Project: Average; 
Cumulative development cost growth: $20.4; 
Percentage cost growth: 3.9%; 
Development growth reported in last year: $7.1; 
Cumulative launch delay (months): 4; 
Launch delay reported in last year (months): 2. 

Portfolio including JWST: 

Project: JWST[H]; 
Cumulative development cost growth: $3,616.8; 
Percentage cost growth: 140.1%; 
Development growth reported in last year: $0.0; 
Cumulative launch delay (months): 52; 
Launch delay reported in last year (months): 0. 

Project: Average; 
Cumulative development cost growth: $320.1; 
Percentage cost growth: 46.4%; 
Development growth reported in last year: $6.6; 
Cumulative launch delay (months): 8; 
Launch delay reported in last year (months): 2. 

Source: GAO analysis of NASA data. 

Note: Shading indicates projects that exceeded the cost or schedule 
thresholds that trigger reporting to certain House and Senate 
committees under the law. 

[A] GPM's development cost decreased largely because one instrument 
was removed from the project. 

[B] ICESat-2 entered Implementation and was baselined in December 2012. 

[C] NASA reported that LDCM's development costs were reduced because 
of progress in delivering an instrument and integrating it with the 
spacecraft and in completing environmental testing on another 
instrument. 

[D] OCO-2's development costs have increased largely because the 
project set a new baseline due to a change in launch vehicle. 

[E] The life-cycle cost for RBSP has not changed. The reduction in 
development costs were offset by an increase in operation costs. 

[F] SMAP established its initial cost and schedule baseline in 2012. 

[G] NASA's development costs for TDRS Replenishment decreased for 
several reasons, including greater than expected contributions from 
one of NASA's partners and the inadvertent inclusion of costs for 
another TDRS satellite. Although TDRS Replenishment experienced a 
launch delay for both TDRS-K and TDRS-L, we only report the longer 
delay for TDRS-L. 

[H] JWST was rebaselined in 2011. 

[End of table] 

Only three projects in the portfolio reported development cost growth 
or launch schedule delay in the last year, but impetus for the growth 
or delay, or both, was outside of two projects' direct control. For 
example, the Orbiting Carbon Observatory (OCO)-2 project reported the 
bulk of its development cost growth and schedule delay for the last 
year--$122.8 million and 24 months--was due to a change in launch 
vehicle late in the development cycle after two successive NASA 
science mission launch failures on OCO-2's planned launch vehicle. In 
addition, NASA has delayed the launch of the Tracking and Data Relay 
Satellite (TDRS) Replenishment project's TDRS-L satellite by 2 months 
due to other government mission launch schedules. 

Two projects--although reporting little or no cost and schedule growth 
in the last year--are currently facing cost or schedule pressures. For 
example, the Lunar Atmosphere and Dust Environment Explorer (LADEE) 
project had spent more than three-quarters of its reserve funding 
prior to entering its test and integration phase and, according to 
officials, faces a potential launch schedule conflict with an Air 
Force mission that could delay its planned launch date. The test and 
integration phase is when issues generally arise that require 
additional money and time to address. The Magnetospheric Multiscale 
(MMS) project entered its test and integration phase in October 2012 
and had already used more than 85 percent of its reserve funding as of 
December 2012. A significant portion of the reserve funding was used 
to pay for the costs associated with moving the project's thermal 
vacuum testing to the Naval Research Laboratory because of conflicts 
with other NASA projects. 

Contributing Factors to the Portfolio's Current Performance: 

Most of the Projects That Exceeded Their Cost and Schedule Baselines 
Are No Longer in the Portfolio: 

In the first 2 years of our review, 2009 and 2010, a majority of the 
projects in the portfolio exceeded their cost and schedule baselines; 
however, most of these projects have launched and therefore are no 
longer being accounted for in our cost growth and schedule delay 
calculations. For example, three projects that were removed from the 
portfolio in the last year--Aquarius, Mars Science Laboratory (MSL), 
and the National Polar-orbiting Operational Environmental Satellite 
System Preparatory Project[Footnote 11]--accounted for a combined 
development cost growth of more than $1 billion and 91 months of 
launch delays. Even including the cost increase of the OCO-2 project's 
launch vehicle change and subsequent launch delay, the removal of 
three costly projects from the portfolio lowered the average 
development cost growth from 15 percent in 2012 to 4 percent in 2013. 
As shown in figure 3 below, when projects with development cost growth 
and launch delays exit the portfolio, and when newer projects are 
maintaining their cost and schedule baselines, the portfolio's average 
development cost growth drops significantly. 

Figure 3: Selected Major NASA Projects with Development Cost Growth 
and the Portfolio Average Development Cost Growth, 2009-2013: 

[Refer to PDF for image: vertical bar and line graph] 

Year: 2009; 
Percentage of projects with development cost growth: 77%; 
Average development cost growth percentage: 12%. 

Year: 2010; 
Percentage of projects with development cost growth: 77%; 
Average development cost growth percentage: 14%. 

Year: 2011; 
Percentage of projects with development cost growth: 40%; 
Average development cost growth percentage: 18%. 

Year: 2012; 
Percentage of projects with development cost growth: 36%; 
Average development cost growth percentage: 15%. 

Year: 2013; 
Percentage of projects with development cost growth: 27%; 
Average development cost growth percentage: 4%. 

Source: GAO analysis of NASA data. 

Note: The calculations do not include the cost growth experienced by 
the JWST project. 

[End of figure] 

Major Projects Show Overall Improvement Meeting Technology Maturity 
Criteria, but Generally Plan to Develop Fewer New Technologies and 
Less New Software: 

In addition to observed improvements in cost and schedule performance 
this year, projects also demonstrated some gains in meeting best 
practices criteria for technology maturity. Nearly two-thirds of the 
projects in our current review meet best practice standards for 
technology maturity, showing a marked improvement from recent years. 
Our best practices work has shown that a technology readiness level 
(TRL) of 6--demonstrating a technology as a fully integrated prototype 
in a relevant environment that simulates the harsh conditions of 
space--is the level of technology maturity that can minimize risks for 
space systems entering product development.[Footnote 12] NASA's 
systems engineering policy also states that by the preliminary design 
review, a TRL of 6 is desirable prior to integrating a new technology 
in a project.[Footnote 13] Demonstrating that technologies will work 
as intended in a relevant environment serves as a fundamental element 
of a sound business case, and projects falling short of this standard 
by preliminary design review often experience subsequent technical 
problems. Specifically, 13 of the 18 projects in the current portfolio 
have completed their preliminary design review and of the 13, 8--
approximately 62 percent--have met best-practice standards for 
technology maturity. These results mark a significant improvement from 
prior years. For example, only 29 percent of projects assessed in 2010 
met the criterion. See figure 4 for an analysis of the projects that 
we reviewed in the past 4 years that held their preliminary design 
review and the percentage of those projects that proceeded into 
implementation with immature technologies. 

Figure 4: Percentage of Selected Major NASA Projects Meeting and Not 
Meeting Technology Maturity Criteria at Preliminary Design Review: 

[Refer to PDF for image: stacked vertical bar graph] 

Year: 2010; 
Projects meeting technology maturity criteria: 29%; 
Projects not meeting technology maturity criteria: 71%. 

Year: 2011; 
Projects meeting technology maturity criteria: 38%; 
Projects not meeting technology maturity criteria: 63%. 

Year: 2012; 
Projects meeting technology maturity criteria: 38%; 
Projects not meeting technology maturity criteria: 63%. 

Year: 2013. 
Projects meeting technology maturity criteria: 62%; 
Projects not meeting technology maturity criteria: 38%. 

Source: GAO analysis of NASA data. 

Note: Totals may not add to 100 percent due to rounding. 

[End of figure] 

While NASA has made improvements in its technology development 
efforts, it appears that recent projects are taking on less technology 
risk than their predecessors by incorporating fewer new critical 
technologies into their design. As a result, we have also seen a 
decline over time in the number of critical technologies that projects 
we reviewed plan to develop. Specifically, we found that the average 
number of critical technologies reported per project has declined from 
4.7 in our first review in 2009 to 2.3 in 2013. Significantly, the 
reduced averages in 2012 and 2013 include the JWST, which reported 10 
critical technologies, substantially more than most other projects 
assessed. Without JWST included the average number of critical 
technologies declines to 1.67. See figure 5. 

Figure 5: Average Number of Critical Technologies Reported for 
Selected Major NASA Projects Reviewed from 2009 to 2013: 

[Refer to PDF for image: vertical bar graph] 

Year: 2009; 
Average number of critical technologies: 4.7. 

Year: 2010; 
Average number of critical technologies: 3.8. 

Year: 2011; 
Average number of critical technologies: 2.6. 

Year: 2012; 
Average number of critical technologies: 2.6. 

Year: 2013; 
Average number of critical technologies: 2.3. 

Source: GAO analysis of NASA data. 

[End of figure] 

In addition to the hardware being developed by NASA projects, projects 
also require the development of computer software to perform critical 
mission functions, thus the amount of new mission-critical software 
developed by a project provides another indication of that project's 
technical complexity.[Footnote 14] We received data from NASA projects 
concerning their development of new mission-critical software, and in 
most cases this information was consistent with the low number of 
critical technologies that projects planned to develop. Most projects 
in implementation have undertaken low amounts of new software 
development and have incorporated a lower number of critical 
technologies, which indicates a less-complex, lower-risk development 
effort. Specifically, six of nine projects in this year's portfolio 
that have completed a preliminary design review reported that less 
than 10 percent of the project's mission-critical software resulted 
from new software development, and all but one of these six projects 
reported one or no critical technologies. In contrast to this trend 
toward fewer critical technologies per project, the JWST project--one 
of NASA's most advanced science projects--reported 10 critical 
technologies and 60 percent of mission-critical software resulting 
from new development efforts. See table 3. 

Table 3: Percent of New Mission Critical Software and Critical 
Technologies Reported by Selected NASA Major Projects: 

NASA Project: James Webb Space Telescope (JWST); 
Reported percent of new source lines of code for mission-critical 
software: 60%; 
Reported number of critical technologies: 10. 

NASA Project: Magnetospheric Multiscale (MMS); 
Reported percent of new source lines of code for mission-critical 
software: 46%; 
Reported number of critical technologies: 2. 

NASA Project: Lunar Atmosphere and Dust Environment Explorer (LADEE); 
Reported percent of new source lines of code for mission-critical 
software: 29%; 
Reported number of critical technologies: 1. 

NASA Project: Landsat Data Continuity Mission (LDCM); 
Reported percent of new source lines of code for mission-critical 
software: 8%; 
Reported number of critical technologies: 3. 

NASA Project: Soil Moisture Active and Passive (SMAP); 
Reported percent of new source lines of code for mission-critical 
software: 5%; 
Reported number of critical technologies: 1. 

NASA Project: Global Precipitation Measurement (GPM); 
Reported percent of new source lines of code for mission-critical 
software: 5%; 
Reported number of critical technologies: 1. 

NASA Project: Mars Atmosphere and Volatile EvolutioN (MAVEN); 
Reported percent of new source lines of code for mission-critical 
software: 4%; 
Reported number of critical technologies: 1. 

NASA Project: Tracking and Data Relay Satellite Replenishment (TDRS); 
Reported percent of new source lines of code for mission-critical 
software: 4%; 
Reported number of critical technologies: 1. 

NASA Project: Orbiting Carbon Observatory 2 (OCO-2); 
Reported percent of new source lines of code for mission-critical 
software: 0; 
Reported number of critical technologies: 0. 

Source: GAO analysis of NASA data. 

[End of table] 

Both GAO and NASA have reported that reusing software can benefit 
projects by reducing development time, costs, and testing. In one 
instance, we observed that one project reused software developed by 
another NASA project in order to recover from a delay in software 
development. While this decision demonstrated managers' 
resourcefulness in addressing a project challenge by making efficient 
use of resources and by reducing risk, the feasibility of adapting 
code from another project in this way does tend to illustrate similar 
degrees of lower technical complexity among recent projects, which 
could contribute to improved cost and schedule performance. Since the 
LDCM, OCO-2, and TDRS Replenishment projects each are deploying 
spacecraft with similar missions to their predecessors, we would 
expect relatively high levels of software reuse and relatively low 
amounts of new software development in comparison to other major 
projects. 

Most Projects Did Not Meet the Design Stability Best Practices, but 
Progress Has Been Made: 

Nine of the 10 projects in this year's portfolio that held a critical 
design review did not meet the best practice criteria of having at 
least 90 percent of engineering drawings in a releasable state at the 
time of the review. However, six of these nine projects reported 
between 67 and 90 percent of drawings releasable at the time of the 
review.[Footnote 15] See figure 6. Our work on product development 
best practices shows that at least 90 percent of engineering drawings 
should be releasable by the critical design review to lower the risk 
of subsequent cost growth and schedule delays.[Footnote 16] Guidance 
in NASA's Systems Engineering Handbook mirrors this metric. Because 
the critical design review is the time in a project's life cycle when 
the integrity of the project design and its ability to meet mission 
requirements are assessed, it is important that a project's design is 
stable enough to warrant continuation with design and fabrication. A 
stable design allows projects to "freeze" the design and minimize 
changes prior to beginning the fabrication of hardware. It also helps 
to avoid re-engineering and rework efforts due to design changes that 
can be costly to the project in terms of time and funding. 

Figure 6: Percentage of Engineering Drawings Releasable at CDR for 
Selected Major NASA Projects: 

[Refer to PDF for image: vertical bar graph] 

Best practice criteria for design stability: 90%. 

Project: GPM; 
Percentage of engineering drawings releasable at Critical Design 
Review: 46%. 

Project: RBSP; 
Percentage of engineering drawings releasable at Critical Design 
Review: 68%. 

Project: TDRS Replenishment; 
Percentage of engineering drawings releasable at Critical Design 
Review: 81%. 

Project: JWST; 
Percentage of engineering drawings releasable at Critical Design 
Review: 84%. 

Project: LDCM; 
Percentage of engineering drawings releasable at Critical Design 
Review: 78%. 

Project: MMS; 
Percentage of engineering drawings releasable at Critical Design 
Review: 74%. 

Project: OCXO-2[A]; 
Percentage of engineering drawings releasable at Critical Design 
Review: 95%. 

Project: LADEE; 
Percentage of engineering drawings releasable at Critical Design 
Review: 44%. 

Project: MAVEN; 
Percentage of engineering drawings releasable at Critical Design 
Review: 48%. 

Project: SMAP; 
Percentage of engineering drawings releasable at Critical Design 
Review: 77%. 

Source: GAO analysis of NASA data. 

[A] The OCO-2 project is a rebuild of a prior design that launched and 
therefore a majority of its drawings were releasable. 

[End of figure] 

While most of the projects in our review did not attain the 90 percent 
criteria, many of the recent projects have considerably higher 
percentages of drawings releasable at the critical design review than 
we reported in prior reviews. Specifically, the 10 projects in this 
year's portfolio that completed their critical design reviews averaged 
73 percent of engineering drawings releasable at the time of that 
review, while projects at the same stage of development averaged 31 
percent in 2010. See figure 7. 

Figure 7: Percentage of Releasable Drawings for Selected Projects 
Reviewed That Held a Critical Design Review: 

[Refer to PDF for image: vertical bar graph] 

Best practice criteria for design stability: 90%. 

Year: 2010; 
Percentage of engineering drawings releasable at Critical Design 
Review: 31%. 

Year: 2011; 
Percentage of engineering drawings releasable at Critical Design 
Review: 62%. 

Year: 2012; 
Percentage of engineering drawings releasable at Critical Design 
Review: 62%. 

Year: 2013; 
Percentage of engineering drawings releasable at Critical Design 
Review: 73%. 

Source: GAO analysis of NASA data. 

[End of figure] 

An additional indicator of an unstable design is the degree to which 
the expected number of design drawings for a project increases after 
the critical design review. We continue to see a positive trend in 
this regard. As projects we reviewed improved their efforts to reach 
design stability by their CDR, they have also reported significantly 
improved performance in controlling increases in engineering drawings 
following critical design review. Specifically, nine projects assessed 
in our fiscal year 2010 study reported an average of a 181.6 percent 
increase in engineering drawings after CDR. The analogous figure has 
fallen to 11.7 percent for the 10 projects in this year's report that 
have completed CDR. See figure 8. 

Figure 8: Average Percentage of Drawing Growth after CDR For Selected 
Projects from Fiscal Years 2010 through 2013: 

[Refer to PDF for image: vertical bar graph] 

Fiscal year: 2010: 181.6%; 
Fiscal year: 2011: 36.4%; 
Fiscal year: 2012: 35.9%; 
Fiscal year: 2013: 1.7%. 

Source: GAO analysis of NASA data. 

[End of figure] 

Six of these 10 projects report post-CDR growth less than or equal to 
4 percent. Most of these projects held their critical design reviews 1 
to 3 years ago, providing some time over which growth in the number of 
drawings might take place. The remaining 4 projects have had post-CDR 
growth in drawings exceeding 14 percent. Two of these projects--GPM 
and LADEE--had less than 50 percent of their engineering drawings 
releasable at CDR, and have reported post-CDR drawing growth of 14.4 
and 65.2 percent, respectively. 

Progress Made on the Management and Oversight of Major Projects: 

Over the past several years, NASA has made positive changes that have 
helped contribute to the improved performance of its projects. Among 
other things, we previously reported that NASA adopted a new policy to 
help project officials with management, cost and schedule estimating, 
and maintenance of adequate levels of reserves; established a 
management review process to enable NASA's senior management to more 
effectively monitor a project's performance, including cost, schedule, 
and cross-cutting technical and nontechnical issues; and have improved 
external oversight by increasing transparency into project costs. For 
example, NASA instituted the joint cost and schedule confidence level 
(JCL) process, which is expected to quantify potential risks and 
calculates cost, schedule, and reserve estimates based on all 
available data.[Footnote 17] NASA also addressed one of our 2011 
recommendations by beginning to provide more transparency into project 
costs in the early phases of development, such as life cycle cost 
estimate ranges for projects in formulation and information on prior 
year costs. This information should allow the Congress sufficient 
information to conduct oversight and ensure earlier accountability and 
should bring more attention to and focus on conducting early, reliable 
estimates of project costs. 

Challenges Remain That Could Affect Continued Progress: 

The portfolio's improved cost and schedule performance is beginning to 
reflect changes that have increased oversight and brought more 
attention to disciplined project management. Continued leadership in 
this regard, however, will be necessary to ensure that such changes 
are sustained, especially as the agency deals with flat or decreasing 
budgets and the implementation of several large-scale, complex 
projects. In addition, by continuing to implement tools and metrics to 
monitor performance of projects, as we have recommended, NASA 
officials will better be able to ensure that progress will continue 
and that managers are equipped to oversee NASA's major projects and 
deliver positive results. Several challenges need to be addressed to 
ensure continued progress: 

* Managing competing priorities within the context of constrained 
budgets: NASA's primary challenge in the next few years will be to 
complete a series of complex and expensive projects in the context of 
constrained budgets and sometimes competing priorities set by the 
Congress and the administration. This challenge will not be easily met 
if any of NASA's three flagship efforts--JWST, the Space Launch System 
(SLS), and the Orion Multi Purpose Crew Vehicle (MPCV)--experience 
large cost overruns as there is little flexibility in NASA's portfolio 
to address significant cost growth beyond canceling projects. 

Although cost and schedule growth can occur on any project, increases 
associated with its most costly and complex missions can have 
cascading effects on the rest of the portfolio. For example, recent 
JWST cost growth will have reverberating effects on the portfolio for 
years to come and required the agency to identify $1.4 billion in 
additional resources over fiscal years 2012 through 2017, according to 
Science Mission Directorate officials. NASA identified approximately 
half of this required funding from the four science divisions within 
the Science Mission Directorate account. The majority of the cuts were 
related to future high priority missions, missions in the operations 
and sustainment phase, and research and analysis. In essence, NASA has 
had to mortgage future high priority missions and research to address 
JWST's additional resource needs. Cost growth on SLS and MPCV would 
likely have similar effects. NASA initiated work on SLS and MPCV in 
November 2011 to replace the launch vehicle and crew capsule for the 
Constellation program, which was canceled in February 2010. Together 
these three projects--JWST, SLS, and MPCV--are estimated to require 
approximately $3 billion a year through fiscal year 2017. Based on 
NASA's budget estimates for fiscal years 2012 and 2013, NASA expects 
to provide approximately $4.6 billion for its major projects. However, 
they are the types of costly and complex projects that NASA has 
struggled to complete within time and funding constraints over the 
past two decades. NASA cannot realistically expect to have significant 
additional funding available to cover any potential cost overruns on 
its major projects without negatively affecting its ability to begin 
new projects or its performance on existing projects. With a 
significant portion of this funding already committed to existing 
projects, NASA will have limited flexibility to address potential cost 
growth or begin new projects over the next 5 years. See figure 9. 

Figure 9: Fiscal Year 2013 Budget Request for JWST, MPCV, SLS, and All 
Other Major NASA Projects, 2013 through 2017: 

[Refer to PDF for image: stacked line graph] 

Dollars in millions: 

Year: 2013; 
MPCV: $1,024.9; 
SLS: $1,340; 
JWST: $627.6; 
Other major projects: $1,583; 
Total major project funding: $4.6 billion. 

Year: 2014; 
MPCV: $1,028.2; 
SLS: $1,429.3; 
JWST: $659.1; 
Other major projects: $1,151.7; 
Total major project funding: $4.6 billion. 

Year: 2015; 
MPCV: $1,028.2; 
SLS: $1,429.3; 
JWST: $646.6; 
Other major projects: $934.1; 
Total major project funding: $4.6 billion. 

Year: 2016; 
MPCV: $1,028.2; 
SLS: $1,429; 
JWST: $621.6; 
Other major projects: $816.1; 
Total major project funding: $4.6 billion. 

Year: 2017; 
MPCV: $1,028.2; 
SLS: $1,429.3; 
JWST: $571.1; 
Other major projects: $502.6; 
Total major project funding: $4.6 billion. 

Source: GAO analysis of NASA data. 

Note: Budget data is from the fiscal year 2013 President's budget 
request. Total line indicates the 2013 request for major projects 
included in the current portfolio and assumes a relatively flat 
investment in future years for other major projects based on flat or 
declining budget estimates for all included projects. 

[End of figure] 

Should NASA experience significant cost growth on JWST, SLS, or MPCV, 
competing priorities set by the Congress and the administration will 
limit the agency's flexibility to make tradeoffs. On the one hand, the 
Congress has placed a priority on funding JWST, SLS and MPCV and 
Members of Congress have voiced concerns about the possibility of cuts 
or cancellations being made to other parts of NASA's portfolio such as 
projects focused on exploring Mars. On the other hand, other projects 
may be difficult to delay or cancel because they are providing 
critical capabilities to NASA and/or other government agencies. For 
instance, NASA's Space Network Ground Segment Sustainment project 
plans to develop and deliver a new ground system for NASA's Space 
Network--which provides essential communications and tracking services 
to NASA and non-NASA missions--because the existing ground system is 
based on 1980s technology and software and the systems are becoming 
obsolete and unsustainable. 

Another project that would be difficult to cancel or delay is TDRS, as 
it is focused on replenishing a space-based network that provides 
continuous high bandwidth digital voice, video, and mission payload 
data, as well as health and safety data relay services to Earth-
orbiting spacecraft such as the International Space Station and the 
Hubble Space Telescope. At the same time, the administration has 
placed a high priority on funding the Commercial Crew Program, which 
is fostering the development of spacecraft that can service the 
International Space Station, as there is no domestic means of doing so 
with the retirement of the space shuttle. This program has already 
seen delays in development activities and a major revision in its 
acquisition approach after NASA received less than anticipated funding 
from the Congress for fiscal years 2011 and 2012. Further cuts may 
prevent NASA from meeting its goals for servicing the space station 
with services provided by commercial companies. 

An alternative to canceling specific programs in the face of 
significant cost growth within a large program is to cut or delay 
funding across many programs. But our work has consistently shown that 
this practice creates instabilities in programs that ultimately lead 
to more cost growth and delays and that it strengthens incentives to 
produce optimistic estimates. Both outcomes would likely erode NASA's 
progress in reducing acquisition risk. 

Estimating costs associated with SLS and MPCV to ensure oversight and 
transparency: Estimating the full life-cycle costs of SLS and MPCV 
projects will be critical to ensuring transparency into their cost and 
schedule and will enable effective oversight. At this point, however, 
NASA currently plans to estimate only the costs of portions of the SLS 
and MPCV projects, which is not a true life-cycle cost estimate. For 
example, the preliminary cost estimates of SLS only extend through the 
first non-crewed flight in 2017, plus 3 months of data analysis. This 
estimate does not include costs for the first crewed flight of the 
same vehicle type of the SLS in 2021, nor does it include costs 
associated with substantial development for future flights of other 
variants of the launch vehicle.[Footnote 18] According to project 
officials, the cost of the first crewed flight will be tracked by the 
project. Similarly, the MPCV cost estimate extends only through the 
first crewed flight in 2021. MPCV officials told us that separate cost 
estimates will be completed for future flights of MPCV. Developing 
cost estimates in this manner will not allow NASA to provide a total 
life cycle cost for the SLS and MPCV projects. As a result, a true 
life cycle cost may never be established and oversight of the projects 
would be hampered because no one would be able to track the baseline 
cost and associated cost growth. 

According to NASA officials, the agency is developing a tailored 
definition for SLS and MPCV life-cycle cost estimating since it is one 
of evolving capabilities on a continuum, not a traditional life cycle. 
Additionally, NASA officials stated that the full life-cycle costs of 
SLS and MPCV cannot be calculated because SLS and MPCV are really 
programs and not projects that have discrete start and endpoints. 
Thus, NASA is reporting the cost for attaining a certain level of 
capability and what it costs to fly each version of SLS and MPCV. In 
an era of declining budgets, a true life cycle cost estimate will 
inform decision makers about the long term affordability of these 
programs before making any long term investment decisions. Credible 
cost estimates also help assess the reasonableness of a contractor's 
proposals and program budgets, and can be used to determine how budget 
cuts may hinder a program's effectiveness. 

While NASA indicated that they will provide transparency into the cost 
and schedule for various research and development segments, given the 
significant costs that will likely be associated with these projects 
and the impact that any overruns could have on the rest of the agency, 
we plan to review this issue further to ensure better cost oversight 
of the projects. 

* Implementing effective cost and schedule estimation practices across 
major NASA projects: Our work as well as that of the National Research 
Council, NASA's Inspector General and NASA-commission studies have 
consistently pointed to long-term weaknesses in cost and schedule 
estimation at NASA, including a tendency to produce overly optimistic 
estimates, the lack of reliance on independent estimates, the lack of 
adherence to best practices, as well as insufficient resources and 
qualified staff dedicated to cost estimating. Recently, for example, 
NASA's Inspector General found that "a culture of optimism and a can-
do spirit permeate all levels of NASA, from senior management to front-
line engineers. Although this optimistic organizational culture is 
essential for realizing groundbreaking scientific achievement, it can 
also lead to unrealistic projections about what can be achieved within 
approved budgets and timeframes. In addition, this culture has 
manifested itself in a tendency to view the success of projects 
primarily in technical rather than cost and schedule terms. More 
specifically, NASA's optimistic culture contributes to development of 
unrealistic plans and performance baselines that fail to account for 
all relevant risks." NASA's adoption of the JCL process is a positive 
step toward addressing these weaknesses and ensuring estimates are 
more realistic. But it still remains to be seen whether this process 
will be sustained long enough to overcome a cultural tendency to be 
optimistic and whether enough resources will be dedicated to producing 
estimates. 

Further, one important part of estimating and monitoring costs for 
projects is the effective implementation of Earned Value Management. 
We reported in November 2012 that NASA's implementation of earned 
value management (EVM) across the projects in the current portfolio 
was inconsistent.[Footnote 19] For example, we reported that: 

* NASA has not yet fully implemented EVM for 10 of the projects we 
reviewed in the current portfolio, 

* half of the 10 projects did not use an EVM system that had been 
certified as compliant with industry standards,[Footnote 20] and: 

* only 4 of the 10 projects had reviews to ensure that key data 
produced by the system was reliable, and only 3 projects were found to 
have reliable EVM data. 

We found that cultural and other challenges impeded the effective use 
of EVM at NASA as the agency's culture has traditionally focused on 
managing science and engineering challenges and not on monitoring cost 
and schedule data, like an effective EVM system produces. We also 
found that NASA does not have the policies in place, or the workforce 
with the skills and experience, to analyze EVM data to ensure correct 
implementation of EVM so that NASA managers could rely on EVM as a 
tool to measure progress on projects. In addition, we reported that 
NASA's current policy does not require rigorous oversight of how 
projects are implementing EVM. We made several recommendations 
including that NASA establish a timeframe for when projects will be 
required to use the new EVM system, conduct an EVM skills gap 
assessment of its staff, and require projects to implement formal EVM 
oversight. In response, the agency plans to conduct a skills gap 
assessment, to augment its EVM training to address any identified 
gaps, and to develop an EVM change management plan to better embrace 
the implementation of EVM techniques throughout the agency. We plan to 
monitor the agency's progress in addressing these recommendations. 

Along with earned value data, development of realistic, dynamic, and 
logic-driven schedules provide another critical input to estimating 
and monitoring project costs. We have previously reported that a 
reliable project schedule can enhance cost estimation by contributing 
to an understanding of the cost impact if a particular project does 
not finish on time. In an effort to continually assess NASA's progress 
in improving NASA's cost-estimating policies and external oversight of 
these policies, we selected three projects--GPM, MMS, and JWST--that 
planned significant amounts of implementation-phase work during 2012, 
and assessed these projects' schedules against best practices for 
project scheduling developed by GAO. We could not validate any of the 
three projects' schedules as reliable based on our methodology. While 
we cannot generalize the results of these analyses to NASA's 
scheduling practices across the entire portfolio of projects, the 
results do provide us with insights into areas that could require 
further investigation into all of NASA's major projects, given the 
critical importance of a reliable schedule to NASA's overall ability 
to estimate the time and funds necessary to complete its projects. 
Appendix V contains detailed information about the criteria for 
schedule analyses, and the performance of the GPM and MMS project 
schedules against the criteria. The appendix also includes findings 
with respect to the JWST project schedule, developed through work for 
another GAO report.[Footnote 21] 

* Using consistent and proven design stability metrics: Along with 
continued NASA leadership to manage projects through constrained 
budgets and continuous improvement of project offices' estimating 
capacities, further progress on delivering projects more consistently 
within cost and schedule baselines would benefit from NASA's full 
implementation of a consistent and proven set of design stability 
metrics. We have previously reported that NASA's acquisition policy 
does not specify a metric to measure a project's design stability at 
the critical design review. In 2010, the National Research Council 
found that the critical design review milestone for many NASA missions 
may be held prematurely--driven by schedule rather than driven by 
design maturity.[Footnote 22] They added that most of the cost growth 
on NASA missions takes place after the critical design review even 
though the design of the mission's instruments and spacecraft should 
be frozen at the critical design review. In 2011, we recommended that 
NASA develop a common set of measurable and proven criteria to assess 
design stability and amend its systems engineering policy to that 
effect.[Footnote 23] In response to this recommendation, NASA 
established three technical indicators to assess design maturity. The 
indicators are (1) the percentage of actual mass margin versus planned 
mass margin, (2) the percentage of actual power margin versus planned 
power margin, and (3) the percentage of overdue project requests for 
action.[Footnote 24] In 2012, NASA updated its program management and 
systems engineering policies to reflect these changes, and each 
project must now report on these metrics. Since that time, the 
projects in our portfolio have reported these metrics in their monthly 
updates to NASA leadership. In the future, officials told us that NASA 
also plans for these projects to incorporate design indicators 
applicable to their unique project, such as requirement and schedule 
trends, to ensure proper progress and management of the project is 
achieved during the formulation phase. These indicators will also be 
provided in the project's plan and will be shared with other projects. 

GAO and NASA have different opinions about the importance of assessing 
the design stability of a project at the critical design review. Our 
metric seeks design stability at the critical design review as a means 
to determine the readiness of the project to proceed with fabrication. 
On the other hand, NASA prefers to use technical indicators that are 
focused on whether the design is maturing over time, not necessarily 
whether the design is stable or not at any given point. In addition, 
NASA officials have stated that given the unique nature of each of its 
projects, there are no one-size-fits-all indicators that can be 
applied across the board to assess design stability. GAO is working 
with experts from various sectors of the government and industry to 
assess additional criteria that can be applied to NASA and other 
complex, unique projects for assessing design stability. 

Project Assessments: 

The two-page assessments of the projects we reviewed provide a profile 
of each project and describe the challenges we identified this year as 
well as challenges that we have identified in the past. On the first 
page, the project profile presents a general description of the 
mission objectives for each project; a picture of the spacecraft, 
aircraft, or ground system; a schedule timeline identifying key dates 
for the project, including a manifested launch date if known and 
different than the project committed launch readiness date;[Footnote 
25] a table identifying programmatic and contract information; a table 
showing the current baseline year cost and schedule estimates and the 
February 2013 cost and schedule data; a table showing the challenges 
relevant to the project; and a project summary narrative. To maintain 
information on challenges the projects experience over their 
lifetimes, we continued to identify project challenges that were 
previously reported. On the second page of the assessment, we provide 
an analysis of the project challenges, and outline the extent to which 
each project faces cost, schedule, or performance risk because of 
these challenges, if applicable. NASA project offices were provided an 
opportunity to review drafts of the assessments prior to their 
inclusion in the final product, and the projects provided both 
technical corrections and more general comments. We integrated the 
technical corrections as appropriate and summarized the general 
comments below the project update. 

See figure 10 for an illustration of the layout of each two-page 
assessment. 

Figure 10: Illustration of a Project's Two-Page Summary: 

[Refer to PDF for image: illustration] 

Depicted in each summary: 

* Project Challenges: Summary listing of the challenges facing the 
project; 

* General description of the mission's objectives; 

* Project Essentials: Programmatic information including the 
responsible NASA center, international or domestic partners, launch 
information, mission duration, and the driver for the mission's 
requirements; 

* Contract Information: Current highest value contractor, contractor 
activity, the type of contract and when it was awarded, and initial 
and current value of the contract; 

* Illustration of the spacecraft, aircraft or ground system; 

* Project Performance Cost and schedule baseline estimates and the 
latest estimate updates; 

* Project Summary: Brief narrative describing the current status of 
the project; 

* Schedule timeline identifying key dates for the project including 
when the project began formulation, major design reviews, confirmation 
to begin the implementation phase, and launch or operating capability 
information; 

* Project Update: Analysis of the project challenges and the extent to 
which each project faces cost, schedule, or performance risk because 
of these challenges; 

* Project Office Comments: General comments provided by the cognizant 
project office. 

Source: GAO analysis. 

[End of figure] 

ExoMars Trace Gas Orbiter: 

Common name: EMTGO. 

Project Essentials: 

NASA Center Lead: 
Jet Propulsion Laboratory. 

International Partner: 
European Space Agency. 

Requirement derived from: 
2011 Planetary Decadal Survey. 

The ExoMars Trace Gas Orbiter (EMTGO) was planned to be the first of 
two joint missions that were to be developed by NASA and the European 
Space Agency (ESA) for launch in 2016 and 2018. EMTGO was envisioned 
to investigate trace gases on Mars that may be signatures of active 
biological and/or geographical processes. EMTGO instruments were 
designed to work across different bands of the electromagnetic 
spectrum, including infrared, visible, and ultraviolet light. EMTGO 
was to provide data relay services for future missions. 

[Figure: Refer to PDF for image: illustration] 

Source: NASA (artist depiction). 

[End of figure] 

Project Summary: 

NASA’s EMTGO project and planning for the NASA/ESA Mars 2018 mission 
concept was proposed for termination in NASA’s fiscal year 2013 
President’s Budget Request. With NASA’s departure from the project, 
ESA and the Russian Federal Space Agency-—Roscosmos—-are planning to 
continue development of the mission. NASA, however, is researching the 
feasibility of future missions to Mars in 2018 and beyond. According 
to project officials, NASA is continuing development of EMTGO’s 
instrument designs through the preliminary design phase to reduce risk 
should future missions decide to use them. NASA recently announced 
plans to build a new Mars rover that is planned to launch in 2020. 

Formulation: 
03/11: Formulation start; 
02/12: Proposed for termination; 
12/12: GAO review. 

Project Update: 

NASA’s portion of the EMTGO project was proposed for termination in 
the fiscal year 2013 President’s Budget Request. ESA decided to go 
forward with the project and has subsequently partnered with the 
Russian Federal Space Agency, Roscosmos, to do so. NASA’s Mars Program 
Planning Group has identified options for future missions to Mars 
through the 2030s, including missions that could launch in 2018 and 
2020. In December 2012, NASA announced plans to build a new Mars 
robotic science rover with an expected launch in 2020. The agency 
stated that the new rover will be based on the Mars Science 
Laboratory’s rover, Curiosity, which is currently on Mars. 

In March 2012, NASA’s Administrator stated that the fiscal environment 
had forced NASA’s decision not to continue development of EMTGO with 
ESA. Specifically, the Mars Exploration budget-—of which EMTGO is a 
part—-was reduced in the fiscal year 2013 President’s Budget Request 
by about $353 million for fiscal years 2013 through 2016, or about 
25 percent. NASA officials stated that the agency has redirected most 
of the fiscal year 2012 funding originally allocated to EMTGO to other 
priorities. At this point, however, Congress has not taken formal 
action to terminate the program. 

According to project officials, the project developed a plan to mature 
the designs of the project’s instruments as much as possible for use 
on future missions once the project was proposed for cancellation, but 
these instruments will not be used on the 2020 robotic science rover. 
Three of the project’s four instruments have completed their 
preliminary design reviews. According to project officials, all 
technologies are mature and the project is working to ensure that the 
design drawings are complete and available so that other projects 
would be able to use them if needed. Project officials noted that the 
instruments designed for EMTGO may be used on future Earth, Mars, or 
other planetary missions and the risk in using them on future projects 
will be reduced because three of the four instruments have been 
developed through the preliminary design phase. For example, project 
officials stated that the Mars Atmosphere Trace Molecule Occultation 
Spectrometer-—which is designed to detect molecules in the atmosphere 
and map their location-—can be used to study almost any atmosphere. 
They also added that the ExoMars Climate Sounder-—which is designed to 
track Martian wind and atmospheric dust—-may be useful in planning 
landings for human exploration of Mars. In addition, the project 
intends to continue development of the Electra Relay Radio-—which 
relays radio data between the orbiter and assets on the surface of 
Mars—-for the ESA’s ongoing ExoMars mission development. The Electra 
is also expected to support future assets on Mars. 

Project Office Comments: 

The EMTGO project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. 

[End of summary] 

Global Precipitation Measurement Mission: 

Common Name: GPM. 

Project Challenges: 
* Test and integration issues (new). 

Previously Reported Challenges: 
* Funding Issues; 
* Development Partner Issues; 
* Contractor Issues; 
* Technology Issues; 
* Design Issues. 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

International Partner: Japan Aerospace Exploration Agency (JAXA). 

Launch Location: 
Tanegashima Space Center (JAXA), Japan; 
Launch Vehicle: H-IIA (supplied by JAXA). 

Mission Duration: 3 years. 

Requirement derived from: 
Revalidated in the 2007 Earth Science Decadal Survey. 

Contract Information: 

Current highest value contract. 

Contractor: 
Ball Aerospace and Technologies Corp. 

Contractor Activity: 
GPM Microwave Imager development. 

Type of Contract: Cost Plus Award Fee; 
Date of Award: 2005; 
Initial Value of Contract: $97.6 million; 
Current Value: $232.2 million. 

Project Performance: 

Then year dollars in millions: 

Total Project Cost: 
Baseline estimate, FY 2010: $975.9;
Latest, February 2013: $932.8;
Change: -4.4%. 

Formulation Cost: 
Baseline estimate, FY 2010: $349.2; 
Latest, February 2013: $349.2;
Change: 0.0%. 

Development Cost[A]: 
Baseline estimate, FY 2010: $555.2;
Latest, February 2013: $509.9;
Change: -8.2%. 

Operations Cost: 
Baseline estimate, FY 2010: $71.6; 
Latest, February 2013: $73.7; 
Change: 2.9%. 

Launch schedule: 
Baseline estimate, FY 2010: July 2013; 
Latest, February 2013: June 2014; 
Change: 11 months. 

[A] GPM reduced development costs in its fiscal year 2012 replan by 
eliminating an instrument from the original development plan. 

The Global Precipitation Measurement (GPM) mission, a joint NASA and 
Japan Aerospace Exploration Agency (JAXA) project, seeks to improve 
the scientific understanding of the global water cycle and the accuracy 
of precipitation forecasts. GPM is composed of a core spacecraft 
carrying two main instruments: a Dual-frequency Precipitation Radar 
and a GPM Microwave Imager. GPM builds on the work of the Tropical 
Rainfall Measuring Mission, and will provide an opportunity to 
calibrate measurements of global precipitation. 

[Figure: Refer to PDF: illustration] 

Source: NASA GSFC. 

[End of figure] 

Project Summary: 

GPM officials are making progress toward their plan of launching in 
February 2014. The project integrated both of GPM’s instruments onto 
the spacecraft and all components have been delivered. In October 
2012, GPM officials completed several performance tests in order to 
begin thermal vacuum testing. However, in order to avoid conducting 
thermal vacuum testing at an external facility, project officials 
modified GPM’s environmental testing schedule so that it occurred 
after thermal vacuum testing, which is not typical and could require 
re-testing of components later on. GPM completed thermal vacuum 
testing a month later than planned, which impacted the James Webb 
Space Telescope (JWST) mission’s schedule, because it was scheduled to 
begin testing in the same testing facility as GPM beginning in January 
2013. 

Formulation: 

07/02: Formulation start; 
12/05: Mission/System design review; 
11/08: Preliminary design review. 

Implementation: 
12/09: Project confirmation; 
12/09: Critical design review; 
02/12: System Integration review; 
12/12: GAO review; 
02/14: Manifested launch date; 
06/14: Committed launch readiness date. 

Project Update: 

GPM officials are making progress toward their plan of launching in 
February 2014. In October 2011, GPM conducted an internal re-plan of 
its schedule that established a committed launch readiness date in 
June 2014, with the potential to launch by February 2014 if the 
project resolved a schedule conflict for use of the thermal vacuum 
testing facility at NASA’s Goddard Space Flight Center. GPM 
successfully completed its system integration review in February 2012. 
The project integrated both of GPM’s instruments-—the GPM Microwave 
Imager and the Dual-frequency Precipitation Radar—-onto the spacecraft 
in May 2012. All components, such as the integration and test 
batteries, have been delivered. In October 2012, GPM officials 
completed several performance tests in order to begin the project’s 
first environmental test—thermal vacuum testing. 

Test and Integration Issues: 

Project officials are monitoring a risk that the spacecraft may 
experience a failure that is not discovered prior to launch because 
they changed GPM’s environmental testing schedule to conduct thermal 
vacuum testing first in order to avoid testing at an external 
facility. A NASA official stated that conducting thermal vacuum 
testing first is not typical. While project officials reported that 
NASA has successfully used this approach on two other Earth Science 
missions, they stated that components which failed during thermal 
vacuum testing of the spacecraft would likely be re-tested at the 
component level, if necessary. In that case, project officials stated 
that there would be some residual risk because the spacecraft was not 
re-tested in its entirety. 

Other Issues to be Monitored: 

The project completed its thermal vacuum testing one month behind 
schedule. The James Webb Space Telescope (JWST) mission planned to use 
the same testing facility beginning in January 2013 but that also has 
been delayed by one month. As a result, the JWST mission’s test 
schedule has been impacted. 

Project Office Comments: 

The GPM project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. 

[End of summary] 

Ice, Cloud, and Land Elevation Satellite-2 

Common Name: ICESat-2. 

Project Challenges: 
* Launch Issues; 
* Funding Issues; 
* Workforce Issues (new). 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

International Partner: None. 

Launch Location: Vandenberg AFB, CA; 
Launch Vehicle: Delta II; 
Mission Duration: 3 years. 

Requirement derived from: 
2007 Earth Science Decadal Survey. 

Contract Information: 

Current highest value contract. 

Contractor: Orbital Sciences Corp. 

Contractor Activity: Spacecraft development. 

Type of Contract: Fixed Price; 
Date of Award: September 2011; 
Initial Value of Contract: $135.1 million; 
Current Value: $135.1 million. 

Project Performance: 

Then year dollars in millions: 

Total Project Cost: 
Baseline Established FY 2013: $860.3; 
Latest: Feb 2013: $860.3; 
Change: 0.0%. 

Formulation Cost: 
Baseline Established FY 2013: $248.8; 
Latest: Feb 2013: $248.8; 
Change: 0.0%. 

Development Cost: 
Baseline Established FY 2013: $558.9; 
Latest: Feb 2013: $558.9; 
Change: 0.0%. 

Operations Cost[A]: 
Baseline Established FY 2013: $52.9;
Latest: Feb 2013: $52.9; 
Change: 0.0%. 

Launch Schedule: 
Baseline Established FY 2013: May 2017; 
Latest: Feb 2013: May 2017; 
Change: 0 months. 

NASA’s Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) is a 
follow-on mission to ICESat, tasked with measuring changes in polar 
ice-sheet mass with space-borne altimetry measurements to understand 
mechanisms that drive change and the impact of change on future global 
sea level. ICESat-2 will utilize a micro-pulse multi-beam laser 
instrument with a photon counting approach to measurement. This 
process will allow for dense cross-track sampling with a high 
repetition rate, allowing ICESat-2 to provide better elevation 
estimates than ICESat over high slope and rough areas. 

[Figure: Refer to PDF for image: project logo] 

Source: Trax International. 

[End of figure] 

Project Summary: 

In February 2013, NASA selected the Delta II as the launch vehicle for 
ICESat-2. This is a change from project officials’ original plan to 
launch the satellite on the same vehicle, or dual-manifest, with an 
Air Force mission. As a result of this change, planned costs for the 
project are $84 million more than the preliminary cost estimate range. 
Several ICESat-2 milestones, such as the project’s preliminary design 
review, were delayed because of funding challenges. In addition, the 
cost of the Advanced Topographic Laser Altimeter System (ATLAS)-—
which, among other things, measures the topography of ice sheets-—has 
increased due to workforce and funding challenges. 

Formulation: 
12/09: Formulation start; 
05/11: Mission/System definition review. 

Implementation: 
12/12: Project confirmation; 
12/12: GAO review; 
09/13: Critical design review; 
06/15: System integration review; 
07/16: Manifested launch date; 
05/17: Committed launch readiness date. 

Project Update: 

Launch Issues: 

In February 2013, NASA selected the Delta II as the launch vehicle for 
ICESat-2. This is a change from project officials’ original plan to 
launch the satellite on the same vehicle, or dual-manifest, with an 
Air Force mission, which would have lessened launch costs for the 
project. However, because the project must now launch separately, 
planned costs for the project are $84 million more than the 
preliminary cost estimate range we reported last year. In April 2012, 
the Air Force informed NASA that the planned dual-manifest launch was 
no longer a viable option because the Air Force mission’s launch date 
slipped to 2020, beyond ICESat-2’s preliminary launch schedule in 2016. 

Funding/Workforce Issues: 

Several ICESat-2 milestones were delayed because of funding 
challenges. Specifically, project officials stated that the milestones 
were delayed because of a proposal to defer some of ICESat-2’s funding 
for fiscal year 2013. For example, project officials delayed the 
mission’s preliminary design review by about four months to October 
2012, and allocated a significant portion of the project’s fiscal year 
2012 funding reserves to mitigate potential cost impacts caused by 
holding the review later than planned. Project officials stated that 
they delayed their manifested launch date by three months to July 2016 
due to the proposed funding deferment. 

The cost of the Advanced Topographic Laser Altimeter System (ATLAS)—-
which, among other things, measures the topography of ice sheets-—has 
increased by about $30.2 million, or 13.4 percent, since December 2011 
due to workforce and funding challenges. For example, slightly more 
than half of this increase occurred because the project incorporated 
recommendations from the instrument’s preliminary design review that, 
according to project officials, caused the project to hire additional 
staff. Project officials stated that ICESat-2 hired staff to provide 
additional expertise in certain areas, such as integrated systems 
engineering. 

The remaining amount of the ATLAS cost increase occurred because 
project officials delayed the instrument’s schedule by three months 
due to the proposed deferment of some of ICESat-2’s funding. Project 
officials stated that the schedule delay increased workforce costs 
because the project has to pay for personnel costs for three months 
longer than planned. 

Project Office Comments: 

The ICESAT-2 project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. 

[End of summary] 

James Webb Space Telescope: 

Common Name: JWST. 

Project Challenges: 

* Design Issues; 
* Technology Issues; 
* Test and Integration Issues (new). 

Previously Reported Challenges: 
* Funding Issues; 
* Contractor Issues. 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

International Partners: 
European Space Agency, Canadian Space Agency. 

Launch Location: Kourou, French Guiana;
Launch Vehicle: Ariane 5 (ESA Supplied). 

Mission Duration: 5 years (10 year goal). 

Requirement derived from: 
2001 Astrophysics Decadal Survey. 

Contract Information: 

Current highest value contract. 

Contractor: Northrop Grumman Aerospace Company. 

Type of Contract: Cost Plus Award Fee; 
Date of Award: 2002; 
Initial Value of Contract: $824.8 million; 
Current Value: $2.12 billion[A]. 

[A] NASA is currently in negotiation with the contractor and the 
expected value will likely exceed $3 billion. 

Project Performance: 

Then year dollars in millions. 

Total Project Cost: 
Baseline Established FY 2009: $4,963.6; 
Latest, February 2013: $8,835.0; 
Change: 78%. 

Formulation Cost: 
Baseline Established FY 2009: $1,800.1; 
Latest, February 2013: $1,800.1; 
Change: 0.0%. 

Development Cost: 
Baseline Established FY 2009: $2,581.1; 
Latest, February 2013: $6,197.9; 
Change: 140.1%. 

Operations Cost: 
Baseline Established FY 2009: $582.4; 
Latest, February 2013: $837.0; 
Change: 43.7%. 

Launch Schedule: 
Baseline Established FY 2009: June 2014; 
Latest, February 2013: October 2018; 
Change: 52 months. 

The James Webb Space Telescope (JWST) is a large, infrared-optimized 
space telescope that is designed to help understand the origin and 
destiny of the universe, the creation and evolution of the first stars 
and galaxies, the formation of stars and planetary systems, and 
characteristics of planetary systems. JWST’s instruments will be 
designed to work primarily in the infrared range of the 
electromagnetic spectrum, with some capability in the visible range. 
JWST will have a large primary mirror composed of 18 smaller mirrors 
and a sunshield that is the size of a tennis court. Both the mirror 
and sunshield will unfold and open once JWST is in outer space. JWST 
will reside in an orbit about 1 million miles from the Earth. 

Figure: Refer to PDF for image: illustration] 

Source: Northrop Grumman Aerospace Systems (artist depiction). 

[End of figure] 

JWST is currently experiencing technical issues on the spacecraft and 
integrated science instrument module (ISIM) that have impacted the 
test schedule. For example, project and contractor officials are 
working to reduce spacecraft mass because it is currently over its 
allocation. In addition, only two instruments have been delivered for 
integration with ISIM and the other two instruments will be delivered 
at least 11 months late. NASA established a 66 percent confidence level 
for its revised baselines; however, we found that the process used to 
develop the revised cost estimate was not fully consistent with best 
practices. For example, the credibility of the estimate was lessened 
because a sensitivity analysis was not performed that would have 
identified key drivers of costs, such as workforce size. 

Formulation: 
3/99: Formulation start; 
01/06: Mission/System definition review; 
03/08: Preliminary design review. 

Implementation: 
07/08: Project confirmation; 
03/10: Critical design review; 
12/12: GAO review; 
09/17: System integration review; 
10/18: Committed launch readiness date. 

Project Update: 

In December 2012, GAO issued an in-depth report on JWST and made 
several recommendations regarding the project’s cost estimate and 
oversight approach. Below is a summary of key issues identified in that 
report.[B] 

Design/Technology Issues: 

The project is currently experiencing issues with the expected mass of 
the spacecraft and technical issues on several instruments that are 
requiring a significant amount of time to address and have impacted 
instrument deliveries. For example, the spacecraft is currently over 
its mass allocation due to greater than expected increases in the 
estimated weight of the electrical wiring harnesses and other 
structures that make up the spacecraft. Project and contractor 
officials have focused on reducing spacecraft mass because it is the 
least mature subsystem and can more easily accommodate design changes 
that will be necessary to address the issue. 

ISIM is experiencing technology and engineering challenges that 
resulted in the use of 18 of ISIM’s 26 months of schedule reserve and 
the addition of a third test cycle to test replacement detectors in 
three instruments. Only two instruments were delivered on plan by 
September 2012. The other two instruments are estimated to be 
delivered at least 11 months late due to additional issues found 
during test and integration and workmanship issues. As a result, the 
test schedule for the ISIM has been delayed by 17 months. Another 
technical challenge associated with ISIM is the development of the 
cryo-cooler system that removes heat and cools one of the instrument 
detectors and heat shield. 

Test and Integration Issues: 

Project officials reported that the JWST schedule has 14 months of 
funded reserve, which meets Goddard guidance for schedule reserve; 
however, only 7 of the 14 months will likely be available for the 
project’s last three of five complex integration and test efforts. 
GAO’s prior work shows that it is during integration and test where 
problems are commonly found and schedules tend to slip. Given that 
JWST has a challenging integration and test schedule, this could 
particularly be the case. Currently, NASA’s plan for project oversight 
calls for one independent system integration review about 13 months 
before launch. While this is consistent with what NASA requires for 
its projects, this approach may not be sufficient for a project as 
complex as JWST and may be inadequate to ensure key technical and 
management issues are identified early enough to be addressed within 
the project’s current schedule. 

Other Issues to be Monitored: 

Despite NASA investing considerable time and resources replanning and 
establishing a 66 percent confidence level for its revised baselines, 
we found that the process used to develop the revised cost estimate 
was not fully consistent with cost estimating best practices. For 
example, the accuracy of the cost estimate, and therefore the 
confidence level assigned to the estimate, was lessened by the 
schedule used in the joint cost and schedule confidence level analysis 
because the schedule prevented us from, among other things, 
identifying the activities that were on the critical path-—defined as 
time associated with activities that drive the overall schedule. The 
credibility of the estimate was lessened because project officials did 
not perform a sensitivity analysis that would have identified key 
drivers of costs, such as workforce size. 

[B] GAO, James Webb Space Telescope: Actions Needed to Improve Cost 
Estimate and Oversight of Test and Integration, [hyperlink, 
http://www.gao.gov/products/GAO-13-4] (Washington, D.C.: Dec. 3, 2012) 

Project Office Comments: 

The JWST project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. Project officials 
also commented that, other than the delayed delivery of two 
instrument, the project completed all external milestones for fiscal 
year 2012. They added that meeting these milestones reflects 
performance in line with the rebaseline plan. 

[End of summary] 

Landsat Data Continuity Mission: 

Common Name: LDCM. 

Recent/Continuing Project Challenges: 
* Design Issues; 
* Test and Integration Issues (new). 

Previously Reported Challenges: 
* Funding Issues; 
* Development Partner Issues; 
* Technology Maturity. 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

Partner: 
U.S. Geological Survey. 

Launch Location: Vandenberg AFB, CA; 
Launch Vehicle: Atlas V. 

Mission Duration: 
5 years (10 years propellant). 

Requirement derived from: 
Continuation of Landsat data series, 1972. 

Contract Information: 

Current highest value contract. 

Contractor: Orbital Sciences. 

Contractor Activity: Spacecraft development. 

Type of Contract: Firmed Fixed Price. 

Date of Award: April 2008. 
Initial Value of Contract: $119.0 million; 
Current Value: $193.7 million. 

Project Performance: 

Then year dollars in millions. 

Total Project Cost: 
Baseline Established FY 2010: $941.7;
Latest, February 2013: $931.2;
Change: -1.1%. 

Formulation Cost: 
Baseline Established FY 2010: $341.5; 
Latest, February 2013: $341.4;
Change: 0.0%. 

Development Cost: 
Baseline Established FY 2010: $583.4; 
Latest, February 2013: $577.2; 
Change: -1.1%. 

Operations Cost[A]: 
Baseline Established FY 2010: $16.8;
Latest, February 2013: $12.5; 
Change: -25.6%. 

Launch Schedule: 
Baseline Established FY 2010: June 2013;
Latest, February 2013: February 2013; 
Change: -4 months. 

[A] In FY 2012, LDCM reported a decrease in operations because USGS 
agreed to bear additional cost for LDCM operations. 

The Landsat Data Continuity Mission (LDCM), a partnership between NASA 
and the U.S. Geological Survey (USGS), seeks to extend the ability to 
detect and quantify changes on the Earth’s surface at a scale where 
natural and man-made causes of change can be differentiated. It is the 
successor mission to Landsat 7. The Landsat data series, begun in 
1972, is the longest continuous record of changes in the Earth’s 
surface as seen from space. Landsat data is a resource for people who 
work in agriculture, geology, forestry, regional planning, education, 
mapping, and global change research. 

[Figure: refer to PDF for image: photograph] 

Source: NASA-0011 0, Delivery Order 17 with Orbital Sciences 
Corporation. 

[End of figure] 

Project Summary: 

LDCM successfully launched on February 11, 2013. Prior to its launch, 
LDCM was facing schedule pressure, in part, because of a parts issue 
with the Thermal Infrared Sensor (TIRS) instrument and an anomaly with 
the spacecraft’s electronics. These issues, as well as issues with the 
broader launch manifest, caused a delay to the LDCM manifested launch 
date by about two months to the end of the launch window in February 
2013. Although LDCM’s committed launch date was in June 2013, NASA was 
able to launch the project earlier. 

Formulation: 
10/03: Formulation start; 
05/08: Mission/System design review; 
07/09: Preliminary design review. 

Implementation: 
12/09: Project confirmation; 
05/10: Critical design review; 
09/11: System integration review; 
12/12: GAO review; 
02/11/13: Launch date; 
06/13: Committed launch date. 

Project Update: 

LDCM successfully launched on February 11, 2013. Prior to its launch, 
LDCM was facing schedule pressure, in part, because of a parts issue 
with the Thermal Infrared Sensor (TIRS) instrument and an anomaly with 
the spacecraft’s electronics. These issues, as well as issues with the 
broader launch manifest, caused project officials to delay their 
manifested launch date to the end of the launch window in February 
2013. Although LDCM committed to a June 2013 launch date, NASA was 
able to launch the project earlier. 

Design Issues: 

Project officials reported that a parts issue with the TIRS 
instrument-—which measures the Earth’s temperature—-contributed to 
their decision to delay the earlier launch date they were working 
toward. Specifically, LDCM and contractor officials changed the design 
of a component on the TIRS instrument in order to eliminate a helium 
leak. Project officials repaired the component, which exhausted the 
project’s remaining schedule reserve at that time. As a result, the 
project reported that LDCM would not be ready to launch on the earlier 
date-—December 2012—-project officials were working toward. 

Test and Integration Issues: 

The spacecraft experienced an anomaly during installation of its 
flight battery in April 2012 that delayed environmental testing and 
damaged electronics components. Project and contractor officials 
determined that the anomaly may have been caused by an electrical 
current from a battery cable that damaged the spacecraft. Project 
officials delayed LDCM’s environmental testing schedule by about two 
months in order to evaluate which components were damaged and replace 
them. Project officials recovered from the anomaly with the 
spacecraft’s electronics by, among other things, pushing back the 
launch date they were working toward and implementing a 7-day work 
week. 

Project Office Comments: 

The LDCM project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. 

[End of summary] 

Lunar Atmosphere and Dust Environment Explorer: 

Common Name: LADEE. 

Recent/Continuing Project Challenges: 
* Test and Integration Issues (new); 
* Launch Issues. 

Previously Reported Challenges: 
* Technology Issues; 
* Design Issues. 

Project Essentials: 

NASA Center Lead: 
Ames Research Center. 

Partners: None. 

Launch Location: Wallops Flight Facility, VA; 
Launch Vehicle: Minotaur V. 

Mission Duration: 180 days. 

Requirement derived from: 
The Scientific Context for the Exploration of the Moon (National 
Research Council, 2007) and Visions and Voyages for Planetary Science 
(2013-2022). 

Contract Information: 

Current highest value contract. 

Contractor: Space Systems Loral. 

Major Contractor: Spacecraft Propulsion. 

Type of Contract: Firm Fixed Price; 
Date of Award: January 2010; 
Initial Value of Contract: $9.3 million; 
Current Value: $12 million. 

Project Performance: 

Then year dollars in millions. 

Total Project Cost: 
Baseline Established: $262.9; 
Latest, February 2013: $262.9; 
Change: 0.0%. 

Formulation Cost: 
Baseline Established: $79.5; 
Latest, February 2013: $79.5; 
Change: 0.0%. 

Development Cost[A]: 
Baseline Established: $168.2; 
Latest, February 2013: $176.1; 
Change: 4.7%. 

Operations Cost[A]: 
Baseline Established: $15.2; 
Latest, February 2013: $7.6; 
Change: -52.0%. 

Launch Schedule: 
Baseline Established: November 2013; 
Latest, February 2013: November 2013; 
Change: 0 months. 

[A] In FY 2012, NASA shifted $7.9 million from the operations phase to 
the development phase. 

The Lunar Atmosphere and Dust Environment Explorer (LADEE) mission is 
planned to assess the global density, composition, and time 
variability of the lunar atmosphere. LADEE’s measurements should 
determine the size, charge, and spatial distribution of 
electrostatically transported dust grains. Additionally, LADEE is 
designed to carry an optical laser communications demonstrator that 
will test high-bandwidth communication from lunar orbit. 

[Figure: refer to PDF for image: illustration] 

Source: LADEE Project Office (artist depiction). 

[End of figure] 

Project Summary: 

The LADEE project is under both cost and schedule pressure to meet its 
baseline commitments. The project expended three-quarters of its 
reserve funds prior to entering the test and integration phase, 
leaving little margin for any potential issues that may arise in the 
future. The project has experienced schedule pressure because two of 
the project’s instruments have had failures or workmanship issues 
during vibration testing. As a result, the project is left with little 
schedule reserve and has slipped its manifested launch date by 3 
months from May to August 2013. LADEE must launch before mid-October 
2013 to meet science requirements or the mission will be delayed until 
at least April 2014. 

Formulation: 
02/09: Formulation start; 
07/09: Mission/System design review; 
07/10: Preliminary design review; 
08/10: Project confirmation. 

Implementation: 
05/11: Critical design review; 
08/12: System integration review; 
12/12: GAO review; 
08/13: Manifested launch date; 
11/13: Committed launch readiness date 

Project Update: 

LADEE is under both cost and schedule pressure to meet its baseline 
commitments. The project has experienced delays developing its 
instruments and has expended three-quarters of its reserve funds prior 
to entering the test and integration phase. The reduced amount of 
reserve funds leaves the project with little margin for any potential 
issues that may arise during its test and integration phase. The 
project originally did not pass its system integration review—-which 
evaluates the readiness of the system to begin integration activities—-
but it passed its follow-up review held in August 2012. 

Test and Integration Issues: 

In May 2012, the project experienced vibration test failures in its 
Neutral Mass Spectrometer—-which is designed to detect various 
elements and molecules in the lunar atmosphere and dust. As a result, 
the instrument had little schedule margin remaining. Project officials 
stated that some of the instrument’s internal wiring connections were 
causing electrical shorts and officials had to reopen the instrument 
to fix the issue and then reseal and requalify it. Additional funds 
were necessary to pay for the repairs and the instrument was delivered 
in October 2012-—5 months later than planned—-and installed on the 
spacecraft. 

The development of the Lunar Laser Communications Demonstration-—a 
communications demonstrator being developed by NASA’s Human Exploration 
and Operations Mission Directorate—-is behind schedule due to damage 
and workmanship issues and has little schedule margin. Project 
officials stated that a fiber optic line was damaged in transport to 
the testing facility. Once testing began, the project also discovered 
workmanship issues with the demonstrator’s modem. In total, officials 
expected delivery of the instrument to be 4 to 6 weeks behind 
schedule. They stated that LADEE will launch without it should the 
instrument be unable to meet its scheduled delivery date. In November 
2012, the project reported that one of the three flight components of 
the Lunar Laser Communications Demonstration was delivered and is 
integrated onto the spacecraft. 

Launch Issues: 

The project has experienced schedule pressure due to development 
delays and slipped the launch date it was working toward by three 
months from May to August 2013. According to NASA officials, this delay 
has created a potential launch schedule conflict because the U.S. Air 
Force has requested the same launch pad for one of its missions, also 
in August 2013. However, the Air Force mission does not have a 
confirmed launch date and the launch services provider is working on 
several possible courses of action to de-conflict the Air Force and 
the LADEE launch dates. In addition, while LADEE’s committed launch 
readiness date is November 2013, project officials have stated that it 
cannot launch later than mid-October 2013 due to the start of the 
lunar eclipse season. If delayed beyond mid-October, the project would 
not be able to launch until at least April 2014 after the lunar 
eclipse season. 

Project Office Comments: 

The LADEE project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. NASA officials 
also commented that in October 2012, LADEE developed a plan to 
potentially accommodate a Lunar Laser Communications Demonstration 
late delivery. A risk assessment and determination will be made in 
January 2013. 

[End of summary] 

Magnetospheric Multiscale: 

Common Name: MMS. 

Project Challenges: 
* Test and Integration Issues (new); 
* Parts Issues; 
* Contractor Issues. 

Previously Reported Challenges: 
* Design Issues; 
* Development Partner Issues; 
* Funding Issues; 
* Technology Issues. 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

International Partners: 
Austria, France, Japan, Sweden. 

Launch Location: Kennedy Space Center, FL; 
Launch Vehicle: Atlas V. 

Mission Duration: 2 years. 

Requirement derived from: 2003 Solar and Space Physics Decadal Survey. 

Contract Information: 
Current highest value contract. 

Contractor: Southwest Research Institute. 

Major Contractor: Instrument development. 

Type of Contract: Cost Plus Fixed Fee; 
Date of Award: April 2004; 
Initial Value of Contract: $229.4 million; 
Current Value: $225 million. 

Project Performance: 

Then year dollars in millions. 

Total Project Cost: 
Baseline Estimate FY 2009: $1082.7; 
Latest FY 2013: $1082.6; 
Change: 0.0%, 

Formulation Cost 
Baseline Estimate FY 2009: $173.0; 
Latest FY 2013: $172.9; 
Change: 0.0%. 

Development Cost 
Baseline Estimate FY 2009: $857.3; 
Latest FY 2013: $857.3; 
Change: 0.0%. 

Operations Cost: 
Baseline Estimate FY 2009: $52.3; 
Latest FY 2013: $52.4; 
Change: 0.0%. 

Launch Schedule: 
Baseline Estimate FY 2009: March 2015; 
Latest FY 2013: March 2015; 
Change: 0 months. 

The Magnetospheric Multiscale (MMS) is comprised of four identical 
spacecraft—each containing 27 instrument components. The mission is 
planned to use the Earth’s magnetosphere as a laboratory to study the 
microphysics of magnetic reconnection. Magnetic reconnection is the 
primary process by which energy is transferred from solar wind to 
Earth’s magnetosphere and is the physical process determining the size 
of a space weather storm. The four spacecraft will fly in a pyramid 
formation, adjustable over a range of approximately 6 to 250 miles. 
The data from MMS is intended to be used to help predict space weather 
in support of terrestrial and space exploration activities. 

[Figure: refer to PDF for image: illustration] 

Source: NASA GSFC Project Office. 

[End of figure] 

Project Summary: 

The MMS project is working toward a manifested launch date of October 
2014 and should soon complete final integration of the first 
spacecraft. The project had to shift locations for spacecraft thermal 
vacuum testing because of test schedule conflicts with other projects, 
which required it to use about $33 million in reserve funds. Two 
instruments have experienced technical issues or parts failures that 
consumed cost reserves and resulted in slips to their integration and 
test delivery dates. After schedule slips driven by parts failures, 
the contractor overseeing development of all MMS instruments overran 
its cost plan and project officials asked the contractor to propose a 
new budget with cost reductions. An independent review board found 
that the project’s reserve funding was adequate, but could be 
threatened by possible future problems. 

Formulation: 
05/02: Formulation start; 
09/07: Mission/System definition review; 
05/09: Preliminary design review. 

Implementation: 
06/09: Project confirmation; 
08/10: Critical design review; 
08/12: System integration review; 
12/12: GAO review; 
10/14: Manifested launch date; 
03/15: Committed launch readiness date. 

Project Update: 

The MMS project is working toward a manifested launch date of October 
2014, which is 5 months earlier than the project’s March 2015 committed 
launch readiness date. In August 2012, the project held its System 
Integration Review, which evaluates the project’s readiness to begin 
assembly, integration, and test activities. Integration of each of the 
four spacecraft is at various stages of completion. 

Test & Integration/Parts Issues: 

The MMS project was directed to use the Naval Research Laboratory’s 
facility for thermal vacuum testing of the fully integrated spacecraft 
rather than the one at Goddard Space Flight Center. This move was 
necessary because of test schedule conflicts with other NASA projects. 
The move delayed the project’s planned launch date from August to 
October 2014 and required the project to use about $33 million of its 
reserve funding. Project officials told us that they will incur costs 
to build transport and testing equipment and to have project staff at 
both locations. 

Two instruments have experienced parts failures that required the 
project to use cost reserves and resulted in slips to the two 
instruments’ respective integration and test delivery dates. The 
Electron Drift Instrument-—which measures magnetic and electric 
fields-—had a part failure that delayed its delivery for integration. 
The project has replanned its integration schedule to allow this 
instrument to be one of the last items integrated on the spacecraft. 
Two types of sensors in the Fast Plasma Investigation instrument—-
which measure ions and electrons in plasma—-have had parts issues that 
required rework and additional testing. These issues delayed the 
sensors’ delivery for instrument integration and testing by 4 months. 

Contractor Issues: 

The contractor responsible for managing instrument development has 
experienced cost overruns following cost and schedule issues with 
several instruments, and project officials required the contractor to 
prepare a new budget proposing cost reductions in several areas. The 
project has set aside reserve funding to account for the cost overruns 
and the possibility that the planned cost savings might not be 
realized. 

Other Issues to be Monitored: 

An independent review board found that the project’s reserve funding 
was adequate, but could be threatened by possible future problems. The 
project has used or set aside at least 80 percent of this funding and 
has just recently begun the test and integration phase where cost and 
schedule growth is typically realized. 

Project Office Comments: 

The MMS project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. 

[End of summary] 

Mars Atmosphere and Volatile EvolutioN 

Common Name: MAVEN. 

Project Challenges: 
* Parts Issues; 
* Workforce Issues (new). 

Previously Reported Challenges: 
* Launch Issues; 
* Design Issues. 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

International Partner: Institute of Research for Astrophysics and 
Planetology, Toulouse, France. 

Launch Location: Cape Canaveral AFS, FL; 
Launch Vehicle: Atlas V. 

Mission Duration: 21 months of operations, including a 10 month cruise. 

Requirement derived from: 
2003 Planetary Exploration Decadal Survey. 

Contract Information: 
Current highest value contract. 

Contractor: Lockheed Martin. 

Contractor Activity: Spacecraft development. 

Type of Contract: Cost Plus Award Fee; 
Date of Award: April 2009; 
Initial Value of Contract: $237 million; 
Current Value: $250 million. 

Project Performance: 
Then year dollars in millions. 

Total Project Cost: 
Baseline Estimate: $671.2; 
Latest, February 2013: $671.2; 
Change: 0.0%. 

Formulation Cost: 
Baseline Estimate: $63.9; 
Latest, February 2013: $63.9; 
Change: 0.0%. 

Development Cost[A]: 
Baseline Estimate: $567.2; 
Latest, February 2013: $551.7;
Change: -2.7%. 

Operations Cost: 
Baseline Estimate: $40.1;
Latest, February 2013: $55.7;
Change: 38.9%. 

Launch Schedule: 
Baseline Estimate: November 2013; 
Latest, February 2013: November 2013; 
Change: 0 months. 

[A] Development costs were reduced because the negotiated price of the 
launch vehicle decreased. The majority of these funds were rephased to 
operations. 

The Mars Atmosphere and Volatile EvolutioN (MAVEN) mission, a robotic 
orbiter mission, is planned to provide a comprehensive picture of the 
Mars upper atmosphere, ionosphere, solar energetic drivers, and 
atmospheric losses. MAVEN is planned to deliver comprehensive answers 
to long-standing questions regarding the loss of Mars’ atmosphere, 
climate history, liquid water, and habitability. MAVEN is planned to 
provide the first direct measurements ever taken to address key 
scientific questions about Mars’ evolution. 

[Figure: refer to PDF for image: illustration] 

Source: NASA GSFC MAVEN Project Office (artist depiction). 

[End of figure] 

Project Summary: 

MAVEN remains on target to meet its cost and schedule commitments with 
adequate cost and schedule reserves. The project, however, has 
experienced several issues with the development of the Neutral Gas and 
Ion Mass Spectrometer (NGIMS). In particular, the instrument’s main 
component has lagged against NGIMS milestones. These issues have 
nearly exhausted the instrument’s schedule reserve. Despite these 
issues, the project office has taken steps to regain schedule reserve 
on the NGIMS instrument. NASA officials have reported adequate cost 
reserves to compensate for these delays. 

Formulation: 
09/08: Formulation start; 
07/10: Preliminary design review. 

Implementation: 
10/10: Project confirmation; 
07/11: Critical design review; 
06/12: System integration review; 
12/12: GAO review; 
11/13: Committed launch readiness date. 

Project Update: 

The MAVEN project formally began Phase D in September 2012 following a 
successful KDP D review. The project remains on target to meet its 
baseline commitments with adequate cost and schedule reserves. At the 
confirmation review, NASA managers set the project’s cost and schedule 
baselines at a higher confidence level than required by NASA policy, 
which is intended to increase the likelihood of launching within cost 
and on schedule. NASA officials told us that meeting the November 2013 
committed launch readiness date allows MAVEN to reach Mars during a 
period in the 11-year cycle of solar-radiation activity essential to 
obtaining science returns critical for mission success. 

Parts/Workforce Issues: 

The project has experienced several issues with the development of the 
Neutral Gas and Ion Mass Spectrometer (NGIMS). In particular, the 
instrument’s main component, the Quadrupole Mass Spectrometer (QMS) 
has lagged against NGIMS milestones which could impact the NGIMS test 
and integration schedule. As of June 2012, the project had nearly 
consumed the instrument’s schedule reserve to address delays on QMS. 
These delays resulted from several factors. For example, the project 
discovered a gas leak into the QMS and also received parts for the 
QMS that were manufactured below specifications. Some delays stemmed 
from the project’s redirection of assembly staff. The project also had 
to use NGIMS schedule reserves because of resource sharing on other 
projects’ instruments, such as the Lunar Atmosphere and Dust 
Environment Explorer’s Neutral Mass Spectrometer, and when the project 
also augmented NGIMS’s test approach based on lessons learned from 
failures late in testing on a similar instrument. 

The project office has taken steps to regain schedule reserve on the 
NGIMS instrument, including working longer days and weekends. The 
project’s current plans include delivering the NGIMS instrument in 
late February 2013. Officials also reported that the project office 
has coordinated with the integration and test contractor to develop 
plans to accommodate late delivery of NGIMS into integration and 
testing activities so they are not disrupted. In June 2012, NASA 
officials reported that MAVEN holds adequate cost reserves to 
compensate for NGIMS-related delays. 

Issue Update: 

We reported last year that project officials were concerned with the 
high probability of failure of the High-Efficiency Power Supply (HEPS) 
card. Since our last report, the project has resolved the HEPS card’s 
previously-identified workmanship issues. However, a power-supply 
error during post-rework testing required the project to remove the 
HEPS card and replace its fuses. The flight model HEPS card has been 
integrated into the power and drive unit in which it will fly, and is 
undergoing final testing. 

Project Office Comments: 

The MAVEN project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. MAVEN officials 
reported that MAVEN continues to meet cost and schedule commitments 
with adequate reserves. All spacecraft critical path elements have 
been delivered for assembly, test, launch and operations to start, and 
testing has begun. 

[End of summary] 

Orbiting Carbon Observatory 2: 

Common Name: OCO-2. 

Project Challenges: 
* Launch Issues; 
* Funding Issues; 
* Parts Issues. 

Previously Reported Challenges: 
* Design Issues. 

Project Essentials: 

NASA Center Lead: 
Jet Propulsion Laboratory. 

International Partner: None. 

Launch Location: Vandenberg AFB, CA; 
Launch Vehicle: Delta II. 

Mission Duration: 2 years. 

Requirement derived from: 
2000-2010 NASA Earth Science Research Strategy. 

Contract Information: 
Current highest value contract. 

Contractor: Orbital Science Corporation. 

Contractor Activity: Spacecraft development. 

Type of Contract: Cost Plus Fixed Fee/Incentive Fee; 
Date of Award: May 2010; 
Initial Value of Contract: $48 Million; 
Current Value: $53.6 Million. 

Project Performance: 
Then year dollars in millions 

Total Project Cost: 
Baseline Estimate FY 2010: #349.9; 
Latest, February 2013: $467.7; 
Change: 33.7%. 

Formulation Cost: 
Baseline Estimate FY 2010: $60.9; 
Latest, February 2013: $60.9; 
Change: 0.0%. 

Development Cost: 
Baseline Estimate FY 2010: $249.0; 
Latest, February 2013: $371.8; 
Change: 49.3%. 

Operations Cost: 
Baseline Estimate FY 2010: $40.0; 
Latest, February 2013: $35.0; 
Change: -12.5%. 

Launch Schedule: 
Baseline Estimate FY 2010: February 2013; 
Latest, February 2013: July 2014; 
Change: 17 months. 

NASA’s Orbiting Carbon Observatory 2 (OCO-2) is being designed to 
enable more reliable predictions of climate change and is based on the 
original OCO mission that failed to reach orbit in 2009. It is planned 
to make precise, time-dependent global measurements of atmospheric 
carbon dioxide. These measurements will be combined with data from a 
ground-based network to provide scientists with the information needed 
to better understand the processes that regulate atmospheric carbon 
dioxide and its role in the carbon cycle. NASA expects enhanced 
understanding of the carbon cycle will improve predictions of future 
atmospheric carbon dioxide increases and the potential impact on the 
climate. 

[Refer to PDF for image: illustration] 

Source: Jet Propulsion Laboratory (artist depiction). 

[End of figure] 

Project Summary: 

In 2013, NASA rebaselined OCO-2’s cost and schedule because of a 
change in its launch vehicle from Orbital’s Taurus XL to the United 
Launch Alliance’s Delta II following the failure of the Taurus XL 
launch vehicle on two prior NASA missions. According to project 
officials, the change of launch vehicle is the primary reason for an 
estimated $117 million cost increase and a projected 17-month launch 
delay. Due to this delay, the project plans to store the observatory 
for about 3 months after completing approximately half of planned 
integration and test activities. The project is concerned about losing 
key personnel during that time. The project has decided to 
replace the existing reaction wheel assemblies due to unexplained in-
flight anomalies on similar units flown on commercial satellites. 

Formulation: 
03/10: Formulation start; 
08/10: Critical design review. 

Implementation: 
09/10: Project confirmation; 
05/12: System integration review; 
12/12: GAO review; 
07/14: Committed launch readiness date. 

Project Update: 

Launch Issues: 

In July 2012, a new launch vehicle, the Delta II, was selected for OCO-
2 given two failures of its original launch vehicle, the Taurus XL. As 
a result, NASA officials stated that the original analysis of the 
thermal, loads, vibration, and integration characteristics—-which are 
different between a Taurus XL and a Delta II—-must be reworked in 
order to ensure that the launch vehicle meets observatory constraints. 
NASA officials stated that the Delta II launch vehicle will be 
configured to meet the requirements of the OCO-2 observatory, rather 
than OCO-2 being configured for the launch vehicle. 

Funding Issues: 

Due to the launch vehicle change and related delay, the project 
rebaselined its cost and schedule. NASA reported that OCO-2 had 
experienced a $117 million life cycle cost estimate increase and a 17-
month launch delay. As required by law, NASA reported to Congress that 
the OCO-2 project had exceeded both its development cost and schedule 
baselines. According to officials, the project is also planning to 
store the observatory for about 3 months and reduce project staff 
during this period to control costs. This has led the project to carry 
a risk that some key engineering personnel at the Jet Propulsion 
Laboratory will be required to stop working on the project due to lack 
of funds and may not be available when the project restarts at a later 
date. The project may incur schedule delays if it loses key personnel 
and has to train new staff or, according to officials, it may incur 
cost increases if it has to retain staff during the observatory 
storage period. NASA is working to temporarily place key personnel on 
other projects with the understanding that they will return following 
observatory storage. 

Parts Issues: 

NASA officials stated that a constellation of commercial 
telecommunication satellites utilizing reaction wheel assemblies—-
rotating wheels used to point and stabilize the spacecraft-—with a 
similar design to those on OCO-2 have experienced a number of in-
flight anomalies that are, as of yet, unexplained. After review, the 
project decided to replace the existing assemblies with units of a 
different design, however procurement and replacement of the 
assemblies may impact the project’s revised launch readiness date. 

Project Office Comments: 

The OCO-2 project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. 

[End of summary] 

Origins-Spectral Interpretation-Resource Identification-Security-
Regolith Explorer: 

Common Name: OSIRIS-REx. 

Project Challenges: 
* Development Partner Issues (new). 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

International Partner: Canadian Space Agency. 

Launch Location: Cape Canaveral AFS, FL; 
Launch Vehicle: TBD; 
Mission Duration: 7 years. 

Requirement derived from: Mid-decade update to the 2003 Planetary 
Science Decadal Survey. 

Contract Information: 
Current highest value contract. 

Contractor: Lockheed Martin Space Systems Company. 

Contractor Activity: Spacecraft Development; 
Type of Contract: Cost Plus Award Fee; 
Date of Award: January 2012; 
Initial Value of Contract: $26 million; 
Current Value: $44 million. 

Project Performance: 
Then year dollars in millions. 

Preliminary estimate of Project Life Cycle Cost:* 

Latest: Feb 2013: $1,085 - $1,210 

* This estimate is preliminary, as the project is in formulation and 
there is uncertainty regarding the costs associated with the design 
options being explored. NASA uses these estimates for planning 
purposes. 

Launch Schedule: 09/2016. 

The Origins-Spectral Interpretation-Resource Identification-Security-
Regolith Explorer (OSIRIS-REx) spacecraft will travel to a near-Earth 
asteroid and use a robotic arm to retrieve samples that could better 
explain our solar system’s formation and how life began. The OSIRIS-
REx mission has five planned science objectives: (1) return and 
analyze a sample, (2) document the sample site, (3) create maps of the 
asteroid, (4) measure forces on the asteroid’s orbit that makes it an 
impact threat to the Earth, and (5) compare the asteroid’s 
characteristics with ground-based telescopic data of the entire 
asteroid population. If successful, OSIRIS-REx will be the first U.S. 
mission to return samples from an asteroid to Earth. 

[Figure: refer to PDF for image: illustration] 

Source: OSIRIS-Rex Project Office, NASA/GSFC. 

[End of figure] 

Project Summary: 

OSIRIS-REx officials are working toward the project’s scheduled 
preliminary design review in March 2013 and developing instruments 
that fulfill the mission’s science requirements. However, project 
officials stated that they are closely monitoring issues that could 
impact the project’s schedule because missing the launch window could 
delay the launch by five years due to the orbit of the project’s 
destination. For example, project officials are monitoring a potential 
delay in development of the partner-provided laser altimeter 
instrument, which could cause the instrument to be removed from the 
mission. Project officials are also monitoring a risk that potential 
undiscovered defects could affect the reliability of a critical 
technology that is used during sample collection. 

Formulation: 
05/11: Formulation start; 
05/12: Mission/System design review; 
12/12: GAO review; 
03/13: Preliminary design review. 

Implementation: 
05/13: Project confirmation; 
04/14: Critical design review; 
02/15: System integration review; 
09/16: Projected launch date. 

Project Update: 

The OSIRIS-REx mission entered Phase B—-the preliminary design and 
technology completion phase of formulation—-in December 2011 with a 
preliminary life cycle cost estimate of $1.085 to $1.210 billion. 
OSIRIS-REx successfully completed its mission definition review in May 
2012, and project officials scheduled its preliminary design review 
for March 2013. 

Officials are developing five instruments that fulfill the project’s 
planned science requirements. Project officials stated that the mission’
s primary objective is to return samples from the asteroid to Earth. 
Therefore, they explained that four of the five instruments could be 
removed from the mission if necessary because they are not needed for 
sample return. According to NASA officials, the mission’s science 
requirements will not be finalized until the project is confirmed. 

Development Partner Issues: 

Project officials are monitoring a potential nine-and-a-half month 
delay in development of the project’s mapping and ranging instrument-—
the OSIRIS-REx Laser Altimeter—-provided by the Canadian Space Agency. 
Specifically, the Canadian Space Agency received authority to spend 
development funds on the altimeter later than planned, which 
contributed to delays in the instrument’s schedule. Project officials 
reported that the altimeter’s schedule remains a significant 
challenge, and the Canadian Space Agency is evaluating de-scope 
options for reducing the instrument’s development schedule. 

Other Issues to be Monitored: 

Project officials stated that missing the launch window or backup 
opportunity in 2017 could result in a five year launch delay due to 
the orbit of the project’s destination. According to project 
officials, the mission must launch by September 2016 in order to use 
the Earth’s gravity to assist the spacecraft with intercepting the 
asteroid. They added that the mission could launch in 2017 if 
necessary, but the spacecraft would have to carry more fuel for 
maneuvering in space. After 2017, the mission would not be able to 
launch again until 2022, when the Earth and the asteroid orbit phasing 
returns to a favorable alignment. 

Project officials are also monitoring a risk with the maturity of the 
mission’s Light Detection Ranging technology, which is critical for 
collecting the sample. They stated that this technology was not 
designed for a long mission and has only flown to the International 
Space Station; whereas OSIRIS-REx requires the sensor to operate 
during asteroid operations, which complete approximately 4.5 years 
after launch. Project officials are studying navigation techniques 
that would provide redundancy and reduce the criticality of the light 
detection and ranging technology. 

Project Office Comments: 

The OSIRIS-REx project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. In general, the 
OSIRIS-Rex project agrees with the GAO findings. NASA officials added 
that the only issue currently being tracked is the ability of the 
international partner to deliver the laser altimeter on schedule. 
Project officials also said that there are a number of risks that are 
being closely watched to ensure that critical hardware is delivered on 
schedule. 

[End of summary] 

Orion Multi-Purpose Crew Vehicle: 

Common Name: MPCV. 

Project Challenges: 
* Funding Issues; 
* Design Issues. 

Project Essentials: 

NASA Center Lead: Johnson Space Center. 

International Partner: European Space Agency. 

Launch Location: Kennedy Space Center, FL; 
Launch Vehicle: Space Launch System. 

Mission Duration: 
Varied based on destination. 

Requirement derived from: 
NASA Authorization Act of 2010. 

Contract Information: 
Current highest value contract. 

Contractor: Lockheed Martin. 

Contractor Activity: Spacecraft Development; 
Type of Contract: TBD; 
Date of Award: TBD; 
Initial Value of Contract: TBD; 
Current Value: TBD. 

Project Performance: 
Then year dollars in millions. 

Preliminary estimate of Project Life Cycle Cost:* 

Latest: Feb 2013: $8.5 - $10.3 billion. 

* This estimate is preliminary, as the project is in formulation and 
there is uncertainty regarding the costs associated with the design 
options being explored. NASA uses these estimates for planning 
purposes. 

Launch Schedule:
First Non-Crewed Launch Date: December 2017; 
First Crewed Launch Date: August 2021. 

The Orion Multi-Purpose Crew Vehicle (MPCV) is being developed to 
conduct in-space operations beyond low Earth orbit and to service the 
International Space Station if necessary. Under the MPCV project, NASA 
is continuing to advance development of the human safety features, 
designs, and systems of the former Orion project. Orion was under the 
Constellation program which was canceled in February 2010. MPCV is 
planned to launch atop NASA’s Space Launch System, which has different 
objectives than the Ares launch vehicle designed under the 
Constellation program. The current design of MPCV consists of a crew 
module, service module, and launch abort system. 

[Figure: refer to PDF for image: photograph] 

Source: Lockheed Martin Space Systems Company. 

[End of figure] 

The MPCV program is developing and building hardware for its first 
exploration flight test (EFT-1), but has encountered development 
challenges which could threaten its September 2014 test date. In order 
to manage vehicle development within the constraints of the current 
budget, the project has deferred development and testing of critical 
components. For example, the project shifted the ascent abort test two 
years later than originally proposed, because of budget constraints 
through the first non-crewed launch. MPCV’s current design exceeds the 
allowed mass for the first crewed flight tests on the Space Launch 
System. Project officials are studying a risk that the spacecraft’s 
heatshield may crack during reentry into the Earth’s atmosphere, and 
these analyses have impacted production of the heatshield for EFT-1. 

07/06: Orion Project formulation start; 
08/09: Orion preliminary design review; 
02/10: Orion canceled under Constellation Program; 

Formulation: 
02/12: Orion MPCV formulation start; 
12/12: GAO review; 
04/14: Preliminary design review. 

Implementation: 
09/14: Project confirmation; 
01/15-04/15: Critical design review; 
12/2017: First non-crewed launch date; 
08/2021: First crewed launch date. 

Project Update: 

The project has experienced delays in building and delivering hardware 
for its first exploration flight test (EFT-1) which could threaten the 
launch currently scheduled for September 2014. EFT-1 will test 
numerous separation events, the thermal protection system, and the 
parachutes, all of which protect the crew during flight, reentry, and 
landing events. Development challenges coupled with an aggressive 
schedule have led the project to miss more internal milestones than 
anticipated for this test event. The project is currently working to 
develop a mitigation plan in order to meet their planned September 2014 
test date. In addition, NASA is in discussion with the European Space 
Agency regarding an agreement for development of the service module. 

Funding issues: 

According to NASA officials, vehicle development that cannot be 
managed within the constraints of the current budget will either cause 
the project to accept additional risk, defer capabilities to later 
flights, or delay the first crewed launch date. For example, in order 
to stay within current budget constraints, the project has deferred 
development and testing of the launch abort system, which is needed to 
carry the crew away from the launch vehicle in case of a failed 
launch. By shifting the ascent abort test to two years later than 
originally planned, the project has shortened the length of time 
between the ascent abort test and the first crewed flight which 
decreases the amount of time the project has to address any issues 
that may be discovered during the ascent abort test. 

Design Issues: 

The current projected mass of the spacecraft for the first crewed 
flight test exceeds the recommended mass by over 5,000 pounds. Project 
officials have deferred mass reduction activities for the first non-
crewed flight to the first crewed flight. The project plans to use EFT-
1 component testing and flight test results, among other analyses, in 
order to further refine the spacecraft’s design to meet mass 
requirements. 

Project officials are tracking a risk that the thermal protection 
system could crack due to the thermal expansion stress loads of the 
heatshield prior to reentering the Earth’s atmosphere, which could 
threaten the safety of the crew and success of the mission. This 
cracking property was known prior to selection of the heatshield 
material, but project officials have been conducting stress analyses on 
the heatshield, among other studies, to understand the magnitude of 
the cracking. These analyses have delayed production of the heatshield 
for EFT-1. The heatshield was deemed mature during its critical design 
in May 2012, and project officials expect its capability to be fully 
demonstrated in EFT-1. 

Other Issues to be Monitored: 

Development of MPCV continues under a contract awarded in August 2006 
for development of the Orion vehicle under the Constellation Program. 
This contract is currently valued at $6.2 billion. Project officials 
told us they are working to modify the contract to reflect budgetary, 
technical and schedule changes of the reformulated program consistent 
with Congressional direction. 

Project Office Comments: 

The MPCV project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. 

NASA officials also commented that while MPCV continues to make 
progress on the first production spacecraft to fly on EFT-1 in 2014, 
the project is reformulating to the objectives and constraints of the 
revised human space exploration policy. 

[End of summary] 

Radiation Belt Storm Probes: 

Common Name: RBSP. 

Project Challenges: 
* Parts Issues; 
* Test and Integration Issues (new). 

Previously Reported Challenges: 
* Design Issues; 
* Contractor Issues. 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

Partner: National Reconnaissance Office. 

Launch Location: Cape Canaveral AFS, FL; 
Launch Vehicle: Atlas V. 

Mission Duration: 2 years. 

Requirement derived from: 
2003 Solar and Space Physics Decadal Survey. 

Contract Information: 
Current highest value contract. 

Contractor: Johns Hopkins University/Applied Physics Laboratory. 

Contractor Activity: Spacecraft Development & Operations. 

Type of Contract: Cost Plus Fixed Fee; 
Date of Award: 2006; 
Initial Value of Contract: $435.5 million; 
Current Value: $502.8 million. 

Project Performance: 
Then year dollars in millions. 

Total Project Cost: 
Baseline Estimate FY 2009: $685.8; 
Latest, February 2013: $686.0; 
Change: 0.0%. 

Formulation Cost: 
Baseline Estimate FY 2009: $88.2; 
Latest, February 2013: $88.2; 
Change: 0.0%. 

Development Cost[A]:
Baseline Estimate FY 2009: $533.9 
Latest, February 2013: $510.3; 
Change: 

Operations Cost[A]: 
Baseline Estimate FY 2009: $63.7; 
Latest, February 2013: $87.5; 
Change: 37.4%. 

Launch Schedule: 
Baseline Estimate FY 2009: May 2012; 
Latest, February 2013: August 2012; 
Change: 3 months. 

[A] Funds remaining in development after launch were rephased to 
operations. 

The Radiation Belt Storm Probes (RBSP) mission is planned to explore 
the Sun’s influence on the Earth and near-Earth space by studying the 
planet’s radiation belts at various scales of space and time. The two 
spacecraft are expected to measure the particles, magnetic and 
electric fields, and waves that fill geospace and provide new 
knowledge on the dynamics and extremes of the radiation belts. 
Understanding the radiation belt environment has practical applications 
in the areas of spacecraft system design, mission planning, spacecraft 
operations, and astronaut safety. 

[Figure: refer to PDF for image: illustration] 

Source: © 2010 The Johns Hopkins University/Applied Physics Laboratory 
(artist depiction). 

[End of figure] 

Project Summary: 

RBSP successfully launched on August 30, 2012 within its planned cost 
and schedule baselines. However, in July 2011 NASA rebaselined RBSP’s 
launch date to a date 3 months later because of changes to the launch 
manifest caused by other missions. Several instruments had to be de-
integrated from the spacecraft or had their integration delayed, then 
underwent rework in order to address challenges identified during 
integration and test. According to project officials, after launch 
RBSP will be completing 60 days of on-orbit acceptance activities, 
such as commissioning instruments. The project began returning science 
data in October 2012. 

Formulation: 
09/06: Formulation start; 
10/07: Mission/System design review; 
10/08: Preliminary design review. 

Implementation: 
12/08: Project confirmation; 
12/09: Critical design review; 
10/10: System integration review; 
08/30/2012: Launch date; 
12/12: GAO review. 

Project Update: 

RBSP successfully launched on August 30, 2012 within its planned cost 
and schedule baselines. However, in July 2011 NASA rebaselined RBSP’s 
committed launch date to a date three months later because of changes 
to the launch manifest caused by other missions, including some from 
the Department of Defense. According to project officials, after 
completing 60 days of on-orbit acceptance activities, such as 
commissioning instruments, the project began returning science data on 
October 30, 2012. 

Parts/Test and Integration Issues: 

The project had to rework several instruments in order to address 
challenges identified during integration and test. For example, one of 
the mission’s six Magnetic Electron Ion Spectrometers (MagEIS) failed 
during thermal vacuum testing. Project officials removed the failed 
instrument and one other MagEIS instrument from one spacecraft, and 
delayed delivery of two other MagEIS instruments for associated rework 
on all MagEIS instruments. After spacecraft-level vibration testing, 
the outstanding MagEIS units were delivered to the spacecraft. In 
addition, after completing spacecraft environmental testing, the 
project removed the Helium-Oxygen Proton-Electron instruments and 
power and communications components from both spacecraft for rework. 
Project officials reported retesting both instruments prior to re-
integrating these instruments on both spacecraft prior to launch. 

Project Office Comments: 

The RBSP project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. NASA officials 
added that the slip in the launch date from August 15, 2012 to August 
30, 2012 was due to launch vehicle availability and unfavorable 
weather conditions that were outside of the project’s control. The 
project completed the commissioning phase and transitioned into 
operations on October 30, 2012 within cost. 

[End of summary] 

Soil Moisture Active and Passive: 

Common Name: SMAP. 

Project Challenges: 
* Launch Issues; 
* Design Issues (new). 

Previously Reported Challenges: 
* Funding Issues; 
* Technology Issues. 

Project Essentials: 

NASA Center Lead: 
Jet Propulsion Laboratory. 

Partner: None. 

Launch Location: Vandenberg AFB, CA; 
Launch Vehicle: Delta II. 

Mission Duration: 3 years. 

Requirement derived from: 
2007 Earth Science Decadal Survey. 

Contract Information: 
Current highest value contract. 

Contractor: Northrop Grumman Aerospace Systems. 

Contractor Activity: Reflector Boom Assembly development. 

Type of Contract: Cost Plus Fixed Fee; 
Date of Award: June 2009; 
Initial Value of Contract: $18.56 million; 
Current Value: $35.0 million. 

Project Performance: 
Then year dollars in millions. 

Total Project Cost: 
Baseline Estimate FY 2012: $916.5; 
Latest, February 2013: $916.5; 
Change: 0.0%. 

Formulation Cost: 
Baseline Estimate FY 2012: $388.2;
Latest, February 2013: $388.2; 
Change: 0.0%. 

Development Cost: 
Baseline Estimate FY 2012: $485.7; 
Latest, February 2013: $485.7; 
Change: 0.0%. 

Operations Cost: 
Baseline Estimate FY 2012: $42.6; 
Latest, February 2013: $42.6; 
Change: 0.0%. 

Launch Schedule: 
Baseline Estimate FY 2012: March 2015; 
Latest, February 2013: March 2015; 
Change: 0 months. 

NASA’s Soil Moisture Active and Passive (SMAP) mission leverages 
previous Earth Science missions and is based on the soil moisture and 
freeze/thaw mission concept developed by an earlier mission known as 
Hydros. SMAP is designed to provide new information on global soil 
moisture and its freeze/thaw state enabling new advances in 
hydrospheric science and applications. These measurements will improve 
understanding of regional and global water cycles, improve weather, 
flood, and drought forecasts, and climate changes. 

[Figure: refer to PDF for image: illustration] 

Source: 2011 California Institute of Technology/Jet Propulsion 
Laboratory. 

[End of figure] 

Project Summary: 

In July 2012, NASA selected a launch vehicle for SMAP. This decision 
took place 16 months later than the project originally requested. The 
project postponed key reviews but continued development work to remain 
on schedule. SMAP was able to continue development by keeping members 
of its review board aware of progress with spacecraft design and by 
modifying its launch vehicle adapter. Project officials are working to 
complete the analysis necessary to understand how the spacecraft and 
launch vehicle will interact. The project has implemented plans to 
address concerns with instrument components that were identified 
during developmental shock tests near the end of the design phase. 

Formulation: 
09/08: Formulation start; 
02/09: Mission/System design review; 
10/11: Preliminary design review. 

Implementation: 
06/12: Project confirmation; 
07/12: Critical design review; 
12/12: GAO review; 
04/13: System integration review; 
10/14: Manifested launch date; 
03/15: Committed launch readiness date. 

Project Update: 

The SMAP project successfully held its confirmation review in May 2012 
and its critical design review in July 2012. At the confirmation 
review, NASA approved a March 2015 committed launch date, though the 
project continues to work toward a launch in October 2014. In July 
2012, NASA selected the Delta II as the SMAP project’s launch vehicle. 

Launch Issues: 

NASA selected SMAP’s launch vehicle 16 months later than the project 
originally requested to accommodate the selection process for a new 
vehicle. The preliminary design review is the time in the development 
lifecycle when project officials prefer to have a launch vehicle 
identified due to its importance to the design of the spacecraft. In 
order to accommodate the delayed launch vehicle selection, the project 
postponed its preliminary design review, project confirmation, and 
critical design review by several months, but the preliminary design 
review and confirmation were still held prior to the selection of a 
launch vehicle. NASA officials have previously stated that changing a 
launch vehicle after a project’s preliminary design review can delay 
design or lead to a fundamental change to the mission design, resulting 
in potential cost and schedule growth. As a result, NASA authorized 
the project to take steps to continue development as planned and 
maintain its schedule to the extent possible. For example, the project 
mitigated some of the risk associated with late selection of the 
launch vehicle by modifying its launch vehicle adapter design so that 
the spacecraft could be mated to any of the launch vehicles that were 
under consideration. 

Project officials stated that while no significant issues or concerns 
remain following the project’s critical design review, important 
analyses need to be completed. Specifically, the project is working 
with the launch vehicle provider to ensure that the analysis of how 
the spacecraft and launch vehicle interact will be completed on 
schedule so that there is time to address any issues that might arise. 

Design Issues: 

The project is working to resolve an issue with the deployment 
mechanism for the reflector boom assembly-—which is part of the 
instrument antenna and is used by the radar and the radiometer. 
Specifically, the structural loads analysis indicated that mechanical 
loads on specific parts may exceed acceptable limits for those parts. 
The project is making design changes to address this issue and 
currently estimates a one month delivery delay for the boom assembly. 

The project is also working to address a risk to the radiometer-—
designed to provide data to measure soil moisture by detecting radiant 
energy emitted from the Earth—-and to the control electronics as 
initial development tests on the engineering model of the instrument 
showed that shock levels could be above their heritage levels. To 
mitigate this risk, the project reduced the shock levels transmitted to 
these components by modifying the design and using shock-absorbing 
materials to reduce shock levels on the electronics and radiometer 
assemblies. 

Project Office Comments: 

The SMAP project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. NASA officials 
also commented that the design issues identified are typical of those 
encountered near the end of a design phase where developmental 
hardware testing and maturing analyses reveal a small number of 
isolated design issues with a few components. They added that finding 
such issues at the end of the design phase helps to avoid the 
downstream risk of discovering a design issue while testing the actual 
flight hardware. These issues were discovered before the project’s 
critical design review, they were reported to the Standing Review 
Board at the critical design review, which determined that the 
project’s approach for addressing the issues was sound. 

[End of summary] 

Solar Probe Plus: 

Common Name: SPP. 

Recent/Continuing Project Challenges: 
* Launch Issues; 
* Technology Issues (new); 
* Design Issues (new). 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

Partner: None. 

Launch Location: Cape Canaveral AFS, FL; 
Launch Vehicle: TBD. 

Mission Duration: 7 years. 

Requirement derived from: 
2003 Solar and Space Physics Decadal Survey. 

Contract Information: 
Current highest value contract. 

Contractor: Johns Hopkins University Applied Physics Laboratory. 

Contractor Activity: Aerospace Research Development and Engineering 
Support. 

Type of Contract: Cost Plus Award Fee; 
Date of Award: May 2010; 
Initial Value of Contract: $218.6 million; 
Current Value: $232.9 million. 

Project Performance: 
Then year dollars in millions. 

Preliminary estimate of Project Life Cycle Cost:* 

Latest: Feb 2013 $1,233 - $1,439. 

* This estimate is preliminary, as the project is in formulation and 
there is uncertainty regarding the costs associated with the design 
options being explored. NASA uses these estimates for planning 
purposes. 

Launch Schedule: 07/2018. 

Solar Probe Plus (SPP) is designed to explore the Sun’s outer 
atmosphere, or corona, as it extends into space. The spacecraft will 
orbit the Sun 24 times and its instruments will observe the generation 
and flow of solar winds from very close range. By observing the 
corona, where solar energetic particles are energized, there is 
potential to further the science of heliophysics by shedding light on 
the origin and evolution of solar wind and why the Sun’s outer 
atmosphere is so much hotter than the visible surface. In order to 
achieve its mission, parts of the spacecraft must be able to withstand 
temperatures exceeding 2,500 degrees Fahrenheit, as well as endure 
blasts of extreme radiation. 

[Figure: refer to PDF for image: illustration] 

Source: 2012 Johns Hopkins University/Applied Physics Lab (artist 
depiction). 

[End of figure] 

Project Summary: 

The SPP project has begun designing the project to launch on an Atlas 
V, but a launch vehicle has not been selected by NASA. If the Atlas V 
is selected, it will need to be modified and a modified solid rocket 
motor upper stage will need to be added in order to obtain enough 
energy to reach SPP’s orbit. The SPP project will not rate the solid 
rocket motor as mature until after the project’s critical design 
review. SPP also faces challenges with the development of a heat 
shield and high intensity solar arrays. 

Formulation: 
12/09: Formulation start; 
11/11: Mission/System design review; 
12/12: GAO review; 
01/14: Preliminary design review; 
03/14: Project confirmation. 

Implementation: 
03/15: Critical design review; 
06/16: System integration review; 
07/18: Projected launch date. 

Project Update: 

Launch Issues: 

The SPP project has begun designing the project to launch on an Atlas 
V, but a launch vehicle has not been selected. If the Atlas V is 
selected, it will need to be modified in order to reach SPP’s orbit. 
As a result, the project is developing a new solid rocket upper stage 
in order to achieve enough energy to reach the mission’s trajectory 
around the Sun. 

Technology Issues: 

The SPP project will not rate the upper stage’s solid rocket motor as 
mature-—meaning that it has been demonstrated as a fully integrated 
prototype in a relevant environment—-until after the project’s 
critical design review. The solid rocket motor relies on existing 
technologies, but the technologies have not been integrated together 
before and therefore need to be tested together to assess performance. 

Design Issues: 

SPP faces challenges with two other technologies—-the spacecraft’s sun 
shield and the system used to cool its solar arrays. For example, the 
project has begun development of a Thermal Protection System (TPS)-—a 
carbon-foam filled sun shield and connecting structure-—which will 
protect SPP’s instruments from the Sun’s heat. The project has 
identified a risk that the physical loads exerted during launch on the 
sun shield could exceed the TPS design margins and require a redesign. 
The project is considering an alternate TPS design to mitigate this 
risk and is conducting an analysis to further understand the TPS 
performance under the expected loads. 

Project officials are also focused on the development of two sets of 
solar arrays—-essentially solar power generators—-and have concerns 
that the cells within the arrays may provide insufficient power to the 
spacecraft or place unsustainable thermal demands on SPP’s solar array 
cooling system. In order to mitigate the risk, the project has 
fabricated SPP’s solar cell assemblies using alternative techniques 
and is currently testing their performance. 

The project will not be able to test a full scale probe under 
conditions like those in the near-Sun environment. To address this 
risk to the extent possible, the project demonstrated the 
functionality of a 1/6 scale cooling system and is currently validating 
subsystem design performance in conditions like those in the near-Sun 
environment using engineering models. 

Issue to be Monitored: 

The project has reported that NASA did not structure its current 
budget to align with its development and execution needs. As a result, 
the project may have had to cut short future development and test 
activities, which raised the technical risk level of the project. To 
address this risk, in July 2012, NASA’s Science Mission Directorate 
realigned its FY2014 budget allocation to mitigate the risk, and plans 
on addressing the other budget needs in the future. 

Project Office Comments: 

The SPP project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. NASA officials 
stated that the project remains on track for its preliminary design 
review in January 2014 and on track for a July 2018 launch. 

[End of summary] 

Space Launch System: 

Common Name: SLS. 

Project Challenges: 
* Design Issues; 
* Funding Issues. 

Project Essentials: 

NASA Center Lead: 
Marshall Space Flight Center. 

Partner: None. 

Launch Location: Kennedy Space Center, FL. 

Mission Duration: 
Varied based on destination. 

Requirement derived from: 
NASA Authorization Act of 2010 and the NASA 2011 Strategic Plan. 

Contract Information: 
Current highest value contract. 

Contractor: TBD. 

Contractor Activity: TBD. 

Type of Contract: TBD; 
Date of Award: TBD; 
Initial Value of Contract: TBD; 
Current Value: TBD. 

Project Performance: 
Then year dollars in millions. 

Preliminary estimate of Project Cost through first non-crewed launch:* 

Latest: February 2013: $7.65 – $8.59 billion. 

* This estimate is preliminary through the first non-crewed launch in 
December 2017, as the project is in formulation and there is 
uncertainty regarding the costs associated with the design options 
being explored. NASA uses these estimates for planning purposes. 

Launch schedule: 
First non-crewed launch: 2017; 
First crewed launch: 2021. 

The Space Launch System (SLS) is intended to be the nation’s first 
human heavy-lift launch vehicle since the Saturn V was developed for 
the Apollo program. SLS is planned to launch NASA’s Multi-Purpose Crew 
Vehicle and service the International Space Station if necessary. The 
vehicle is being designed with an initial lift capacity of 70 metric 
tons to low-Earth orbit and evolvable to 130 metric tons. The initial 
70 metric ton capability will include a core stage and two five-
segment boosters. The 130 metric ton capability will include a core 
stage, an upper stage powered by a J-2X engine, and advanced boosters. 

[Figure: refer to PDF for image: illustration] 

Source: SLS Project Office (artist depiction). 

[End of figure] 

Project Summary: 

In November 2012, the SLS project produced a preliminary cost estimate 
range as part of the project’s system requirements and definition 
review, but this estimate does not include costs of launch vehicles 
beyond the first non-crewed launch in 2017. While the project is 
utilizing heritage hardware to control cost, modifications to the 
hardware will be required in order to meet SLS objectives. As a result 
of budget constraints, the project has focused practically all of its 
current resources on development of the 70 metric ton launch vehicle 
needed for the first non-crewed and crewed launch dates, and has 
deferred some work on future versions of SLS. 

Formulation: 
11/11: Formulation start; 
07/12: Mission/System design review; 
12/12: GAO review; 
06/17-09/13: Preliminary design review 

Implementation: 
10/13-12/13: Project confirmation; 
01/15-03/15: Critical design review; 
2017: First non-crewed launch date; 
2021: First crewed launch date. 

Project Update: 

In November 2012, NASA produced a preliminary estimate of $7.65 to 
$8.59 billion for the 70 metric ton version of SLS. This is not a life 
cycle cost estimate, however, because it only covers the first non-
crewed launch date in December 2017, plus three months of data 
analysis. This estimate does not include costs for the first crewed 
flight of the same vehicle type of the SLS in 2021, nor does it 
include costs associated with substantial development for future 
flights of other variants of the launch vehicle. NASA officials stated 
that the full life-cycle cost of SLS cannot be calculated because SLS 
is really a program and not a project that has a discrete start and 
endpoint. Thus, NASA is thinking in terms of what the cost is for 
attaining a certain level of capability, and what it costs to fly each 
version of SLS. 

Design Issues: 

Project officials are still assessing whether existing, or heritage 
hardware, can meet performance requirements without modifications. For 
example, project officials will not know if the shuttle-era RS25 
engines as currently designed can meet SLS’s performance requirements 
without significant modifications until the engine preliminary design 
review. In addition, the controllers for the RS-25 engines—designed 
over 30 years ago to monitor and control the engines and perform 
diagnostics on other components—will need to be redesigned to address 
parts obsolescence issues. 

The project is working with NASA officials to determine what human 
safety requirements will be required for SLS and whether the existing 
SLS design will meet those requirements. For example, the project is 
currently evaluating options for a propulsion stage that is designed 
to provide additional power to push the Multi-Purpose Crew Vehicle 
spacecraft into deep space. The project is also concluding a study 
on whether an existing propulsion stage could meet human rating and 
performance requirements, or would require additional design 
modifications. 

Funding Issues: 

In order to stay within its short-term funding projections, the 
project has deferred work on the 130 metric ton vehicle. For example, 
the project is has deferred work on the upper stage--which will store 
liquid hydrogen and liquid oxygen needed to feed the 
rocket’s engines—-for the 130 metric ton vehicle. The project also 
reported that developing the core stage for the 70 metric ton vehicle 
in parallel with the core stage of the 130 metric ton vehicle would be 
preferable in order to ensure optimum core stage design and full 
evaluation of integration issues between the two stages, and would be 
more cost effective. However, due to budget constraints this is not 
possible. As a result, once development of the 130 metric ton vehicle 
begins, modifications may be needed. 

Other Issues to be Monitored: 

Project officials told us that development of SLS components continue 
under modified undefinitized contracts awarded under the Constellation 
program’s Ares project. Project officials reported that the SLS work 
conducted under these contracts remains within the scope of the Ares 
prime contracts, however the contracts need to be modified to be in 
line with the design requirements and flight objectives of the SLS 
project. As of November 2012, the project plans to definitize all SLS 
contracts by the fourth quarter of fiscal year 2013. 

Project Office Comments: 

The SLS project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. 

[End of summary] 

Space Network Ground Segment Sustainment: 

Common Name: SGSS. 

Project Challenges: 

* Funding Issues (new); 
* Contractor Issues (new); 
* Technology Issues (new). 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

Mission Duration: 9 years. 

Requirement derived from: 
March 2008 Space Network modernization concept study. 

Contract Information: 
Current highest value contract. 

Contractor: General Dynamics C4 Systems, Inc. 

Contractor Activity: Modernizing the Ground System and Network. 

Type of Contract: Cost Plus Award Fee; 
Date of Award: June 2010; 
Initial Value of Contract: $626.2 million; 
Current Value: $644.0 million. 

Project Performance: 
Then year dollars in millions: 

Preliminary estimate of Project Life Cycle Cost:* 

Latest: February 2013 $516.5 - $627.5.** 

* This estimate is preliminary, as the project is in formulation and 
there is uncertainty regarding the costs associated with the design 
options being explored. NASA uses these estimates for planning 
purposes. 

** This estimate reflects that the SGSS project does not have an 
operations phase. The NASA Space Network program will assume delivery 
of the SGSS products and continue the operations phase. This estimate 
only includes NASA direct funding, and does not include funds received 
from commercial users of the NASA Space Network. 

Estimated Final Acceptance Review: 12/2015–09/2017. 

The Space Network Ground Segment Sustainment (SGSS) project plans to 
develop and deliver a new ground system that will enable the Space 
Network—-which provides essential communications and tracking services 
to NASA and non-NASA missions-—to continue safe, reliable, and cost 
efficient operations for the next several decades. Existing ground 
systems are based on 1980s technology and software, and the systems 
are becoming obsolete and unsustainable. Updated systems and equipment 
will allow the Space Network to maintain critical communications 
services to customer missions while reducing operations and 
maintenance costs. 

[Figure: refer to PDF for image: photograph] 

Source: NASA. 

[End of figure] 

Project Summary: 

NASA has delayed implementation of the project. According to NASA 
officials, the agency has reviewed the impact of current and future 
funding constraints and the SGSS project has been approved for 
implementation, but has not completed the administrative steps to 
finalize the process. Although SGSS officials reported that all of the 
project’s technologies are mature, a management review board 
determined that two of the project’s heritage technologies that need 
to be adapted for use in the SGSS system were not at an appropriate 
level of maturity at the project’s preliminary design review. Project 
officials are also monitoring several risks related to software 
development and the deployment and transition of the new system that 
could significantly affect the project’s cost and schedule. 

Formulation: 
11/11: Formulation start; 
07/12: Preliminary design review; 
12/12: GAO review. 

Implementation: 
06/13: Critical design review; 
TBD: Project confirmation; 
12/2015–09/2017: Estimated final acceptance review. 

project update: 

Funding Issues: 

Although SGSS successfully completed its system preliminary design 
review in July 2012, NASA has delayed implementation of the project. 
According to NASA officials, the agency has reviewed the impact of 
current and future funding constraints and the SGSS project has been 
approved for implementation, but has not completed the administrative 
steps to finalize the process. 

Contractor Issues: 

SGSS officials are monitoring a risk that staffing levels and costs 
could increase, and the project’s schedule could be delayed because a 
sub-contractor estimated that it could develop software at an 
unrealistic rate. SGSS and contractor officials estimate that this risk 
could delay the project’s schedule by 3-to-9 months, and about 22 
percent of the project’s reserve funding could be needed to mitigate 
this risk. SGSS officials requested progress metrics so that they 
could monitor the sub-contractor’s performance, but project officials 
reported that they have not yet received the metrics. 

Technology Issues: 

Although SGSS officials reported that all of the project’s 
technologies are mature, a management review board determined that two 
of the project’s heritage technologies that need to be adapted for use 
in the SGSS system were not at an appropriate level of maturity at the 
project’s preliminary design review. According to SGSS officials, the 
management review board determined that the technologies were not 
mature because they have not been demonstrated in an operational 
environment, which is consistent with our best practices work on 
technology maturity. According to SGSS officials, both technologies 
will be tested at one or more of the Space Network’s ground terminals 
beginning in 2014. Until then, they stated that SGSS and contractor 
officials will continue developing and testing prototypes of both 
technologies. Project officials said they are not concerned about the 
maturity of the technologies and do not anticipate needing to use 
backup technologies because prototypes of both technologies have 
performed well in laboratory environments. 

Other Issues to be Monitored: 

Project officials estimate that potential risks during the scheduled 
deployment of and transition to the new system in 2014 could delay the 
schedule and require utilization of a quarter of the project’s reserve 
funding. For example, the SGSS project is tracking a risk that there 
is inadequate time for verification and validation testing at the 
ground terminals, such as demonstrating control of a tracking and data 
relay satellite, because, among other things, the schedule and planned 
staffing levels are aggressive. Project officials stated that this 
risk contributed to a projection that SGSS’s schedule could increase 
by 9 months. 

Project Office Comments: 

The SGSS project office provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. 

[End of summary] 

Stratospheric Observatory for Infrared Astronomy: 

Common Name: SOFIA. 

Recent/Continuing Project Challenges: 
* Design Issues. 

Previously Reported Challenges: 
* Funding Issues. 

Project Essentials: 

NASA Center Lead: 
Dryden Flight Research Center. 

International Partner: DLR, the German Aerospace Center 
Aircraft: Modified 747SP. 

Sortie Location: 
Dryden Aircraft Operations Center, CA. 

Mission Duration: 20 years of science flights. 

Requirement derived from: 
Astronomy and Astrophysics Committee, National Research Council, 1991. 

Contract Information: 
Current highest value contract. 

Contractor: Universities Space Research Association. 

Contractor Activity: Provide the SOFIA Science Center and the science 
missions operations. 

Type of Contract: Cost Plus Fixed Fee; 
Date of Award: December 1996; 
Initial Value of Contract: $484 Million; 
Current Value: $581 Million. 

Project Performance: 
Then year dollars in millions. 

Total Project Cost: 
Baseline Estimate FY 2007: $2,954.5; 
Latest, February 2013: $3.002.9; 
Change: 1.6%. 

Formulation Cost: 
Baseline Estimate FY 2007: $35.0; 
Latest, February 2013: $35.0; 
Change: 0.0%. 

Development Cost: 
Baseline Estimate FY 2007: $919.5; 
Latest, February 2013: $1,128.4; 
Change: 22.7%. 

Operations Cost: 
Baseline Estimate FY 2007: $2,000.0; 
Latest, February 2013: $1,839.5; 
Change: -8.0%; 

Launch Schedule: 
Baseline Estimate FY 2007: December 2013; 
Latest, February 2013: December 2014; 
Change: 12 months. 

SOFIA is a joint project between NASA and the German Aerospace Center 
(DLR) to install a 2.5 meter telescope in a specially modified Boeing 
747SP aircraft. This airborne observatory is designed to provide 
routine access to the visual, infrared, far-infrared, and sub-
millimeter parts of the electromagnetic spectrum. Its mission 
objectives include studying many different kinds of astronomical 
objects and phenomena, including star birth and death; the formation 
of new solar systems; planets, comets, and asteroids in our solar 
system; and black holes at the center of galaxies. Interchangeable 
instruments for the observatory are being developed to allow a range 
of scientific measurement to be taken by SOFIA. 

[Figure: refer to PDF for image: photograph] 

Source: NASA. 

[End of figure] 

Project Summary: 

SOFIA has spent most of calendar year 2012 performing maintenance, 
upgrades, and repairs on the aircraft and preparing for the start of a 
new round of science flights. The changes include software updates, 
avionics upgrades, and electrical rewiring as well as repairs to the 
back side of the observatory’s primary mirror. However, the project 
has experienced delays while performing these changes, which has 
delayed the resumption of science flights. Despite these delays, 
officials indicate that the SOFIA is on track to meet full 
operational capability in late 2013. 

Formulation: 
10/91: Formulation start. 

Implementation: 
11/95: Project confirmation; 
08/00: Critical design review; 
12/10: Initial operational capability; 
12/12: GAO review; 
12/14: Full operational capability. 

Project Update: 

Although the project’s life cycle cost has increased since it was 
baselined, the project has experienced limited development cost growth 
since the project was replanned in 2010. In the replan, the project 
shifted—from 8 to 4—the number of instruments required to demonstrate 
full operational capability and the project remains ahead of schedule 
to achieve full operational capability. Additionally, the project has 
reduced—from 8 to7—the number of first generation science instruments 
that will eventually fly on SOFIA. In December 2011, the project 
entered a planned “downtime” period to perform maintenance, upgrades, 
and repairs on the aircraft. During this downtime period, the project 
experienced delays related to software development, avionics and wiring 
upgrades, and primary mirror repairs. The start of the downtime period 
was also delayed, in part, due to an extension of the previous science 
flight segment. The project plans to resume science flights in April 
2013, 5 months later than previously planned. 

Design Issues: 

The critical design review for the project’s Mission Control and 
Communication System—-a hardware and software system used, in part, to 
control the observatory and the telescope assembly, and ensure correct 
positioning of the telescope and cavity door-—was delayed and its 
software delivery occurred later than planned. NASA officials told us 
that anomalies were discovered and addressed during software testing 
and the current version contains all of the required functionality for 
observatory verification and validation in October 2012. 

The project has also had to upgrade the aircraft’s avionics and 
electrical wiring, which has taken longer than originally planned. 
According to officials, the project is updating the aircraft’s 
avionics from the 1970s to current technology. Technical issues have 
persisted during the upgrade process and are causing delays. The 
project has addressed issues with an element of the avionics, 
necessary for both the validation and verification and science flights, 
and successfully completed a series of test flights in January 2013. 
In addition, the aircraft required some electrical rewiring due to 
undersized wiring that had to be replaced with a larger gauge wire in 
order to avoid unacceptable voltage drops. The new electrical wiring 
was designed and fabricated in-house and is complex—-it has over 
10,000 connection points—-and as a result has contributed to the 
project’s schedule delays. 

Issue Update: 

We previously reported that damage occurred on the project’s primary 
mirror because of unequal thermal expansion of aluminum wire mount 
tabs that led to chipping of the back side of the primary mirror’s 
glass. To address this issue, the project successfully removed many of 
these tabs and repaired the chips. The mirror’s manufacturer inspected 
the repairs and determined that the issue was addressed. 

Project Office Comments: 

The SOFIA project office provided technical comments to a draft of 
this assessment, which were incorporated as appropriate. Project 
officials reported that the project has completed a significant amount 
of aircraft system upgrades and an avionics modernization effort that 
will allow it to meet airspace requirements. Further, they stated that 
the project continues to hold healthy margin against its Full 
Operational Capability milestone and plans to complete commissioning 
of 4 instruments by the end of calendar year 2013 to meet an early 
full operational capability. 

[End of summary] 

Tracking and Data Relay Satellite Replenishment: 

Common Name: TDRS. 

Recent/Continuing Project Challenges 
* Test and Integration Issues (new); 
* Contractor Issues;
* Launch Issues. 

Previously Reported Challenges 
* Technology Issues; 
* Parts Issues. 

Project Essentials: 

NASA Center Lead: 
Goddard Space Flight Center. 

Partner: Non-NASA Agencies. 

Launch Location: Cape Canaveral AFS, FL; 
Launch Vehicle: Atlas V. 

Mission Duration: 15 years. 

Requirement derived from: 
Support and expand existing TDRS System fleet. 

Contract Information: 
Current highest value contract. 

Contractor: Boeing Satellite Systems. 

Contractor Activity: Spacecraft development. 

Type of Contract: Fixed Price Incentive Fee; 
Date of Award: December 2007; 
Initial Value of Contract: $1.38 billion[A]; 
Current Value: $1.41 billion[A]. 

[A] This represents the full cost of the Boeing contract that NASA is 
managing; however, the cost is shared with NASA’s partners and includes 
options for future versions of the spacecraft. 

Project Performance: 
Then year dollars in millions. 

Total Project Cost: 
Baseline estimate FY 2010: $451.3; 
Latest, February 2013: $426.5; 
Change: -5.5%. 

Formulation Cost: 
Baseline estimate FY 2010: $241.9; 
Latest, February 2013: $241.9; 
Change: 0.0%. 

Development Cost[A]: 
Baseline estimate FY 2010: $209.4;
Latest, February 2013: $184.6; 
Change: -11.8%. 

Operations Cost: 
Baseline estimate FY 2010: $0.0; 
Latest, February 2013: $0.0; 
Change: 0.0%. 

Launch Schedule: K: 
Baseline estimate FY 2010: December 2012; 
Latest, February 2013: January 2013; 
Change: 1 month. 

Launch Schedule: L: 
Baseline estimate FY 2010: December 2012; 
Latest, February 2013: February 2013; 
Change: 2 months. 

[A] In fiscal year 2012, NASA removed funding for future versions of 
TDRS. 

The Tracking and Data Relay Satellite (TDRS) System consists of in-
orbit communication satellites stationed at geosynchronous altitude 
coupled with two ground stations located in New Mexico and Guam. The 
satellite network and ground stations provide mission services for 
near-Earth user satellites and orbiting vehicles. TDRS K and L are the 
11th and 12th satellites, respectively, to be built for the TDRS 
system. They are planned to contribute to the existing network by 
providing continuous high bandwidth digital voice, video, and mission 
payload data, as well as health and safety data relay services to 
Earth-orbiting spacecraft such as the International Space Station and 
the Hubble Space Telescope. 

[Figure: refer to PDF for image: illustration] 

Source: © Boeing (artist depiction). 

[End of figure] 

Project Summary: 

TDRS K successfully launched on January 30, 2013, one month after its 
committed launch date, due to an engine issue on the satellite’s own 
launch vehicle and problems a similar propulsion system experienced 
during a recent Air Force launch. Project officials rescheduled TDRS 
L’s launch readiness date to February 2014, 2 months later than its 
committed launch date, because of other missions’ launch schedule 
needs. 

Formulation: 
02/07: Formulation start; 
07/08: Mission/System design review; 
03/09: Preliminary design review. 

Implementation: 
07/09: Project confirmation; 
02/10: Critical design review; 
08/11: System integration review (TDRS K); 
06/12: System integration review (TDRS L); 
12/12: GAO review; 
01/30/13: Launch date (TDRS K); 
02/14: Committed launch date (TDRS L). 

Project Update: 

Launch Issues: 

TDRS K: TDRS K successfully launched on January 30, 2013, one month 
after its committed launch date. This launch was delayed, however, due 
to launch vehicle concerns. In September 2012, the project reported 
the failure of a steering component in the first-stage engine aboard 
the Atlas V launch vehicle provided for TDRS K. The project also noted 
that such a failure could result in loss of control during launch and 
subsequent loss of the mission. The launch-vehicle provider 
implemented additional shifts to remove and replace the first-stage 
engine assembly and is investigating the root cause of the failure. 
However, NASA did not have the results of the launch vehicle 
provider’s investigation of this failure by TDRS-K’s launch date. 

In addition, in November 2012, the project reported the possibility of 
a slip in TDRS K’s December 2012 launch date following a power 
shortfall during an October 2012 Department of the Air Force launch. 
The Air Force launch employed the same upper-stage engine used in the 
Atlas V launch vehicle that launched TDRS K. Early in December 2012, 
the Atlas V contractor, United Launch Alliance, cleared the propulsion 
systems for the next manifested launch. However, NASA had already 
slipped TDRS K’s launch date to January 2013. 

TDRS L: Project officials have rescheduled TDRS L’s launch date to 
February 2014, 2 months later than its original committed launch 
readiness date. NASA officials told us that they made this change 
because of other missions’ launch schedule needs. Officials also 
indicated they hoped to make adjustments to TDRS L based on TDRS K’s 
on-orbit commissioning experience, and preferred to separate the K and 
L launches by at least 9 months. Since the project schedule currently 
indicates that Boeing will complete TDRS L in March 2013, NASA will 
incur some cost to store the satellite between its completion and 
launch. 

Contractor/Test and Integration Issues: 

The project’s prime contractor—-Boeing-—experienced problems with 
spacecraft test and integration that impacted its schedule. For 
example, in December 2011, an electrical anomaly during thermal vacuum 
testing damaged electronic components in one of TDRS K’s power-
distribution units. In order to maintain schedule, the project began 
working 7 days a week to replace damaged parts with rebuilt 
components, and conducted an additional round of environmental testing 
on these rebuilt components. TDRS L also experienced schedule delays 
due to this issue. The project also reported losing 9 days of schedule 
reserve in February 2012 as a result of delays encountered during 
integration of the satellite’s flight antenna. 

Project Office Comments: 

The TDRS project provided technical comments to a draft of this 
assessment, which were incorporated as appropriate. The TDRS-L 
spacecraft has progressed through integration and test, and has 
successfully completed the spacecraft thermal vacuum test program. The 
spacecraft has several months of margin to the early 2014 launch date. 

[End of summary] 

[End of section] 

Agency Comments and Our Evaluation: 

We provided a draft of this report to NASA for its review and comment. 
In its written response, NASA generally agreed with our findings and 
stated that it remains dedicated to continuous improvement of its 
acquisition management processes and performance, and will continue to 
identify and address the challenges that lead to cost and schedule 
growth on its major projects. 

NASA agreed with us that its initiatives to mitigate acquisition 
management risk have helped contribute to the improved performance of 
its projects. NASA also agreed with our observations that any large 
cost overruns that occur on large complex projects, such as the James 
Webb Space Telescope, the Space Launch System, or the Orion Multi-
Purpose Crew Vehicle, would complicate the management of NASA's entire 
major-project portfolio. NASA further observed that an uncertain and 
unstable funding environment could drive cost increases on such 
projects through less-than-optimal phasing of current and future work. 
Given the current fiscal environment, our findings underscore the 
importance of NASA remaining committed to its initiatives to reduce 
acquisition risk, especially with regard to management of its larger 
and more complex missions. Doing so will help NASA continue the 
improvements it has made to reduce cost and schedule growth in its 
portfolio and improve its ability to successfully manage the fiscal 
uncertainty that is likely to continue for many years. 

In its response, NASA also noted its commitment to advance 
implementation and improve the use of the joint cost and schedule 
confidence level (JCL) process and its intention to implement 
probabilistic analyses of project cost and schedule for those projects 
transitioning into the last phase of formulation. The agency has 
previously noted that a key to improving the use of the JCL process is 
increasing the consistency of practices used by NASA projects in 
developing their JCLs. Completion of the JCL implementation handbook, 
which was expected to be completed in 2012, could help projects 
institute more consistent practices. We encourage NASA to prioritize 
completion of the JCL handbook as part of efforts to advance 
implementation of the JCL process. Extending the preparation of these 
analyses, as NASA indicated in its response that it plans to do, into 
the last phase (Phase B) of formulation cycle could allow for earlier 
identification and mitigation of risks for the projects. 

In its response, NASA noted that it has identified many technical 
indicators to assess a project's design stability and maturity over 
time, and projects have begun to consistently track their progress 
against these indicators. NASA's approach includes developing and 
implementing a set of indicators based on the project's unique 
complexity and mission objectives. We are encouraged by the attention 
and focus that NASA has placed on instituting metrics to assess design 
stability. We plan to review how the tailored use of the newly-
developed indicators by NASA's projects yields a common set of 
measurable and proven metrics to assess design stability and how this 
information will be used by agency decision makers. To assist in our 
ability to assess NASA's progress in this regard, we recently convened 
a panel of experts to discuss measuring design stability in NASA 
projects. We expect that the results of these discussions will enhance 
our ability to assess projects in the future. We remain committed to 
further discussions with NASA in this area. 

NASA's written comments are reprinted in appendix VI. NASA also 
provided technical comments. We carefully considered and incorporated 
those changes that were supported by evidence consistent with GAO 
standards and our role as an independent auditor of executive agencies. 

We are sending copies of the report to NASA's Administrator and 
interested congressional committees. In addition, the report will be 
available at no charge on GAO's website at [hyperlink, 
http://www.gao.gov]. 

Should you or your staff have any questions on matters discussed in 
this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who 
made major contributions to this report are listed in appendix VII. 

Signed by: 

Cristina T. Chaplain: 
Director: 
Acquisition and Sourcing Management: 

List of Committees: 

The Honorable Barbara A. Mikulski: 
Chairwoman: 
The Honorable Richard Shelby: 
Ranking Member: 
Subcommittee on Commerce, Justice, Science, and Related Agencies: 
Committee on Appropriations: 
United States Senate: 

The Honorable Bill Nelson: 
Chairman: 
The Honorable Ted Cruz: 
Ranking Member: 
Subcommittee on Science and Space: 
Committee on Commerce, Science, and Transportation: 
United States Senate: 

The Honorable Frank R. Wolf: 
Chairman: 
The Honorable Chaka Fattah: 
Ranking Member: 
Subcommittee on Commerce, Justice, Science, and Related Agencies: 
Committee on Appropriations: 
House of Representatives: 

The Honorable Steven Palazzo: 
Chairman: 
The Honorable Donna Edwards: 
Ranking Member: 
Subcommittee on Space: 
Committee on Science, Space, and Technology: 
House of Representatives: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Our objectives were to discuss broader trends and challenges faced by 
the agency in its management of acquisitions and to report on the 
status and challenges faced by 18 National Aeronautics and Space 
Administration (NASA) major projects with life-cycle costs of $250 
million or more. Specifically, we (1) assessed the performance of 
NASA's portfolio of major projects in terms of cost and schedule, (2) 
identified factors that have contributed to the portfolio's current 
performance, and (3) highlighted remaining challenges to continued 
improvement. 

To respond to these objectives, we collected and analyzed data from a 
variety of sources. We developed a standardized data collection 
instrument (DCI) that was completed by each project office. Through 
the DCI, we gathered and assessed data on each project's technology 
and design maturity, parts issues, and development partners. We 
developed other DCIs that were completed by NASA's Office of the Chief 
Financial Officer (OCFO) and Office of Procurement that gathered data 
on each project's cost performance, current and projected development 
activities (including the project's schedule and manifested/committed 
launch readiness dates), and contracts information.[Footnote 26] 

NASA provided several updates to these data collection instruments. 
NASA's Office of the Chief Engineer provided data on software-related 
metrics for selected projects. We also analyzed data and documentation 
from three NASA major projects on the extent to which these projects 
were meeting GAO scheduling best practices.[Footnote 27] We evaluated 
projects' monthly or quarterly status reports and other project 
documentation and conducted interviews with project, NASA 
headquarters, and contractor officials to identify projects' progress 
to date and any risks. We did not validate the data provided by the 
project or other NASA offices, but reviewed the data and performed 
various checks to determine that the data were reliable enough for our 
purposes. Where we discovered discrepancies, we clarified the data 
accordingly. We determined that the data were sufficiently reliable 
for our purposes. 

The information collected from each project office, Mission 
Directorate, and OCFO was summarized in a two-page report format 
providing a project overview; key cost, contract, and schedule data; 
and a discussion of the challenges associated with the deviation from 
relevant indicators from best practice standards. The aggregate 
measures and averages calculated were analyzed for meaningful 
relationships, for example, relationship between cost growth and 
schedule slippage and knowledge maturity attained both at critical 
milestones and through the various stages of the project life cycle. 

To assess factors contributing to the cost and schedule performance of 
NASA's portfolio of major projects, we reviewed current cost and 
schedule data, technology maturity, design stability, and other 
challenges affecting each of the projects. To determine the extent to 
which each project exceeded its cost and schedule baselines, we 
compared the current cost and schedule data reported by NASA in 
February 2013 to previously established project cost and schedule 
baselines to determine the extent to which each project exceeded its 
baselines. We identified cost and/or schedule growth as significant 
where, in either case, a project's cost and/or its schedule exceeded 
the thresholds that trigger reporting to certain Senate and House 
committees. We also compared the average development cost growth and 
average schedule delay since our initial assessment to this year's 
report to determine whether NASA major projects had improved in 
adhering to cost and schedule baselines. All cost information is 
presented in nominal then-year dollars for consistency with budget 
data.[Footnote 28] Baseline costs are adjusted to reflect the cost 
accounting structure in NASA's fiscal year 2009 budget estimates. For 
the fiscal year 2009 budget request, NASA changed its accounting 
practices from full-cost accounting to reporting only direct costs at 
the project level. 

To identify factors that contributed to the portfolio's current 
performance, we identified the number of technologies and lines of 
software code each project was developing, reviewed historical data on 
past projects and compared it to current project performance, and 
compared projects' technology maturity and design stability against 
established criteria for knowledge-based acquisitions and other GAO 
work on system acquisitions.[Footnote 29] We also analyzed data and 
documentation from three NASA major projects to determine the extent 
to which these projects were meeting GAO scheduling best practices. 
[Footnote 30] See appendix V for more information on GAO's assessment 
of schedule for the three major projects. 

To assess technology maturity, we asked project officials to provide 
the technology readiness levels of each of the project's critical and 
heritage technologies at various stages of project development--
including the preliminary design review--and compared those levels 
against our technology maturity best practice to determine the extent 
to which the portfolio was meeting the criteria. Our work has shown 
that a technology readiness level of 6--demonstrating a technology as 
a fully integrated prototype in a relevant environment--by the 
preliminary design review is the level of maturity needed to minimize 
risks for space systems entering product development. Originally 
developed by NASA, technology readiness levels are measured on a scale 
of one to nine, beginning with paper studies of a technology's 
feasibility and culminating with a technology fully integrated into a 
completed product. See appendix III for the definitions of technology 
readiness levels. We also compared this year's results against those 
in prior years to assess whether NASA was improving in this area. We 
did not assess technology maturity for those projects that had not yet 
reached the preliminary design review at the time of this assessment. 
We also collected information on the use of heritage technologies in 
the projects, including what heritage technologies were being used; 
what effort was needed to modify the form, fit, and function of the 
technology for use in the new system; whether the project encountered 
any problems in modifying the technology; and whether the project 
considered the heritage technology as a risk to the project. We also 
worked with NASA's Office of the Chief Engineer to develop a data 
collection instrument to collect three metrics on software-related 
data for selected projects, which include data on the software 
requirements, software size, and current Computer Software 
Configuration Items/major systems currently under development. 

To assess design stability, we asked project officials to provide the 
percentage of engineering drawings completed or projected for 
completion by the preliminary and critical design reviews and as of 
our current assessment.[Footnote 31] In most cases, we did not verify 
or validate the percentage of engineering drawings provided by the 
project office. However, we collected the project offices' rationale 
for cases where it appeared that only a small number of drawings were 
completed by the time of the design reviews or where the project 
office reported significant growth in the number of drawings released 
after the critical design review. In accordance with best practices, 
projects were assessed as having achieved design stability if they had 
at least 90 percent of projected drawings releasable by the critical 
design review. We compared this year's results against those in prior 
years to assess whether NASA was improving in this area. We did not 
assess design stability for those projects that had not yet reached 
the critical design review at the time of this assessment. In 
addition, although some projects used other methods to assess design 
stability, such as computer and engineering models and analyses, we 
did not assess the effectiveness of these other methods. 

To identify remaining challenges to continued improvement, we 
principally relied on outstanding issues identified in our prior work 
on NASA, such as cost and schedule growth on one of NASA's most 
technologically advanced and costly projects and earned value 
management implementation issues. We examined how NASA is managing 
large and complex missions within the current budget environment, and 
the extent to which NASA has implemented GAO's prior recommendation 
that the agency develop a consistent set of proven metrics to assess 
design stability.[Footnote 32] 

Our work was performed primarily at NASA headquarters in Washington, 
D.C. In addition, we visited NASA's Ames Research Center at Moffett 
Field in California; Dryden Flight Research Center at Edwards Air 
Force Base in California; Goddard Space Flight Center in Greenbelt, 
Maryland; the Jet Propulsion Laboratory in Pasadena, California; 
Johnson Space Center in Houston, Texas; Kennedy Space Center, Florida; 
and Marshall Space Flight Center in Huntsville, Alabama, to discuss 
individual projects. 

NASA's Integrated Budget and Performance Documents: 

To supplement our analysis, we relied on GAO's work over past years 
examining acquisition issues across multiple agencies. These reports 
cover such issues as contracting, program management, acquisition 
policy, and cost estimating. GAO also has an extensive body of work 
related to challenges NASA has faced with specific system 
acquisitions, financial management, and cost estimating. This work 
provided the context and basis for large parts of the general 
observations we made about the projects we reviewed. Additionally, the 
discussions with the individual NASA projects helped us identify 
further challenges faced by the projects. Together, the past work and 
additional discussions contributed to our development of a short list 
of challenges discussed for each two-page project assessment. The 
challenges we identified and discussed do not represent an exhaustive 
or exclusive list. They are subject to change and evolution as GAO 
continues this annual assessment in future years. The challenges, 
indicated as "issues" in each two-page assessment, are based on our 
definitions and assessments, not those of NASA. 

Project Profile Information on Each Individual Two-Page Assessment: 

We summarized our assessments of each individual project or program in 
two components--a project profile and a detailed discussion of project 
challenges. This section of the two-page assessment includes a 
description of each project's objectives; information concerning the 
NASA center, major contractor, or other partner involved in the 
project; the project's cost and schedule performance; a schedule 
timeline identifying key project dates; and a brief narrative 
describing the current status of the project. 

Project performance is depicted according to cost and schedule changes 
in the various stages of the project life cycle. To assess the cost 
and schedule changes of each project, we obtained data directly from 
NASA/OCFO through our data collection instrument and from NASA's 
Integrated Budget and Performance documents. 

The project's timeline is based on acquisition cycle time, which is 
defined as the number of months between the project's start, or 
formulation start, and projected or actual launch date.[Footnote 33] 
Formulation start generally refers to the initiation of a project; 
NASA refers to a project's start as key decision point A, or the 
beginning of the formulation phase. The preliminary design review 
typically occurs toward the end of the formulation phase, followed by 
a review at key decision point C, known as project confirmation, which 
allows the project to move into the implementation phase. The critical 
design review is generally held during the latter half of the final 
design and fabrication phase of implementation and demonstrates that 
the maturity of the design is appropriate to support continuing with 
the final design and fabrication phase. The manifested launch date is 
the launch date which the project is working toward, and when a launch 
vehicle is available to launch the project. This date is only a goal 
launch date for the project, not a commitment that they will launch on 
this date. The committed launch readiness date is determined through a 
launch readiness review that verifies that the launch system and 
spacecraft/payloads are ready for launch. The implementation phase 
includes the operations of the mission and concludes with project 
disposal. 

Project Challenges Discussion on Each Individual Two-Page Assessment: 

To assess the project challenges for each project, we submitted a DCI 
to each project office. In the DCI, we requested information on the 
maturity of critical and heritage technologies, number of releasable 
design drawings at project milestones, software development 
information, project contractors with related contract values and 
award fees, and project partnerships. We also held interviews with 
representatives from 17 of the projects to discuss the information on 
the DCI. One project, RSBP, provided written responses. These 
discussions led to identification of further challenges faced by NASA 
projects. The challenges we identified were largely apparent in the 
projects that had entered the implementation phase; however, there 
were instances where these challenges were identified in projects in 
the formulation phase. We then reviewed pertinent project 
documentation--such as the project plans, schedules, risk assessments, 
and major project review documentation--to corroborate any testimonial 
evidence we received in the interviews. A challenge was identified for 
a project if project performance had been or would be affected by the 
issue. For this year's report, we identified the following challenges 
across the projects we reviewed: launch, contractor management, parts, 
development partners, funding, workforce, and test and integration. 

The individual project offices were given an opportunity to comment on 
and provide technical clarifications to the two-page assessments prior 
to their inclusion in the final product. We incorporated these 
comments as appropriate and where sufficient supporting documentation 
was provided. 

Data Limitations: 

NASA provided updated cost and schedule data in February 2013 for 
projects in implementation, or 12 of the 18 projects in our review. 
NASA provided preliminary estimated life-cycle cost ranges and 
associated schedules for four of the projects that had not yet entered 
implementation, which are generally established at key decision point 
B (KDP-B).[Footnote 34] We did not receive cost estimates or ranges 
for one project--ExoMars Trace Gas Orbiter--since this project had not 
yet reached KDP-B, the point in the acquisition life cycle where a 
preliminary life-cycle cost estimate would normally be developed. One 
project, the Space Network Ground Segment Sustainment system entered 
the implementation phase in April 2013, after our review of projects 
had concluded. This project is not included in our analyses of 
projects in implementation; however, the project's baseline cost and 
schedule is reported in the two page assessment. NASA formally 
establishes cost and schedule baselines, committing itself to cost and 
schedule targets for a project with a specific and aligned set of 
planned mission objectives, at key decision point C (KDP-C), which 
follows a preliminary design review. KDP-C reflects the life-cycle 
point where NASA approves a project to leave the formulation phase and 
enter into the implementation phase. NASA explained that preliminary 
estimates are generated for internal planning and fiscal year 
budgeting purposes at KDP-B, which occurs midstream in the formulation 
phase, and hence, are not considered a formal commitment by the agency 
on cost and schedule for the mission deliverables. NASA officials 
stated that because of changes that occur to a project's scope and 
technologies between KDP-B and KDP-C, estimates of project cost and 
schedule can change significantly heading toward KDP-C. 

We conducted this performance audit from March 2012 to April 2013 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Selected Major NASA Projects Reviewed in GAO's Annual 
Assessments: 

We have reviewed 34 major National Aeronautics and Space 
Administration (NASA) projects since our initial review in 2009. See 
table 4 below for a list of projects included in our assessments from 
2009 to 2013 and whether each project was in formulation or 
implementation at the time of our review. 

Table 4: Selected Major NASA Projects and Programs Reviewed in GAO's 
Annual Assessments. 

Projects in formulation: 

2009: 
Ares I; 
GPM; 
JWST; 
LDCM; 
Orion. 

2010: 
Ares I; 
GPM; 
LDCM; 
Orion. 

2011: 
Ares I[C]; 
ICESat-2; 
Orion[C]; 
SMAP; 
SPP. 

2012: 
EMTGO[D]; 
ICESat-2; 
Orion MPCV; 
SLS; 
SMAP; 
SPP. 

2013: 
EMTGO[D]; 
Orion MPCV; 
OSIRIS-REx; 
SPP; 
SLS; 
SGSS[E]. 

Projects in implementation: 

2009: 
Aquarius; 
Dawn[A]; 
GLAST[A]; 
Glory; 
Herschel; 
Kepler; 
LRO; 
MSL; 
NPP; 
OCO[B]; 
SDO; 
SOFIA; 
WISE. 

2010: 
Aquarius; 
Glory; 
GRAIL; 
Herschel[A]; 
Juno; 
JWST; 
Kepler[A]; 
LRO[A]; 
MMS; 
MSL; 
NPP; 
RBSP; 
SDO[A]; 
SOFIA; 
WISE[A]. 

2011: 
Aquarius; 
Glory[B]; 
GPM; 
GRAIL; 
Juno; 
JWST; 
LADEE; 
LDCM; 
MAVEN; 
MMS; 
MSL; 
NPP; 
OCO-2; 
RBSP; 
SOFIA; 
TDRS Replenishment. 

2012: 
Aquarius[A]; 
GPM; 
GRAIL[A]; 
Juno[A]; 
JWST; 
LADEE; 
LDCM; 
MAVEN; 
MMS; 
MSL[A]; 
NPP[A]; 
OCO-2; 
RBSP; 
SOFIA; 
TDRS Replenishment. 

2013: 
GPM; 
ICESat-2; 
JWST; 
LADEE; 
LDCM[A]; 
MAVEN; 
MMS; 
OCO-2; 
RBSP[A]; 
SMAP; 
SOFIA; 
TDRS Replenishment[A]. 

Source: GAO analysis of NASA data. 

[A] NASA projects that launched during the review year. 

[B] NASA projects that have launched but failed to reach orbit. 

[C] NASA projects that were canceled before entering implementation. 

[D] In February 2012, NASA proposed canceling the EMTGO project as 
part of its fiscal year 2013 budget request. 

[E] SGSS entered the implementation phase in April 2013, after our 
review of projects had concluded. 

[End of table] 

Appendix III: Technology Readiness Levels: 

Technology readiness level: 1. Basic principles observed and reported; 
Description: Lowest level of technology readiness. Scientific research 
begins to be translated into applied research and development. 
Examples might include paper studies of a technology's basic 
properties; 
Hardware: None (paper studies and analysis); 
Demonstration environment: None. 

Technology readiness level: 2. Technology concept and/or application 
formulated; 
Description: Invention begins. Once basic principles are observed, 
practical applications can be invented. The application is speculative 
and there is no proof or detailed analysis to support the assumption. 
Examples are still limited to paper studies; 
Hardware: None (paper studies and analysis); 
Demonstration environment: None. 

Technology readiness level: 3. Analytical and experimental critical 
function and/or characteristic proof of concept; 
Description: Active research and development is initiated. This 
includes analytical studies and laboratory studies to physically 
validate analytical predictions of separate elements of the 
technology. Examples include components that are not yet integrated or 
representative; 
Hardware: Analytical studies and demonstration of nonscale individual 
components (pieces of subsystem); 
Demonstration environment: Lab. 

Technology readiness level: 4. Component and/or breadboard; 
Validation in laboratory environment; 
Description: Basic technological components are integrated to 
establish that the pieces will work together. This is relatively "low 
fidelity" compared to the eventual system. Examples include 
integration of ad-hoc hardware in a laboratory; 
Hardware: Low fidelity breadboard; 
Integration of nonscale components to show pieces will work together. 
Not fully functional or form or fit but representative of technically 
feasible approach suitable for flight articles; 
Demonstration environment: Lab. 

Technology readiness level: 5. Component and/or breadboard validation 
in relevant environment; 
Description: Fidelity of breadboard technology increases 
significantly. The basic technological components are integrated with 
reasonably realistic supporting elements so that the technology can be 
tested in a simulated environment. Examples include high-fidelity 
laboratory integration of components; 
Hardware: High-fidelity breadboard; Functionally equivalent but not 
necessarily form and/or fit (size, weight, materials, etc). Should be 
approaching appropriate scale. May include integration of several 
components with reasonably realistic support elements/subsystems to 
demonstrate functionality; 
Demonstration environment: Lab demonstrating functionality but not 
form and fit. May include flight demonstrating breadboard in surrogate 
aircraft. Technology ready for detailed design studies. 

Technology readiness level: 6. System/subsystem model or prototype 
demonstration in a relevant environment; 
Description: Representative model or prototype system, which is well 
beyond the breadboard tested for TRL 5, is tested in a relevant 
environment. Represents a major step up in a technology's demonstrated 
readiness. Examples include testing a prototype in a high fidelity 
laboratory environment or in simulated realistic environment; 
Hardware: Prototype. Should be very close to form, fit, and function. 
Probably includes the integration of many new components and realistic 
supporting elements/subsystems if needed to demonstrate full 
functionality of the subsystem; 
Demonstration environment: High-fidelity lab demonstration or 
limited/restricted flight demonstration for a relevant environment. 
Integration of technology is well defined. 

Technology readiness level: 7. System prototype demonstration in a 
realistic environment; 
Description: Prototype near or at planned operational system. 
Represents a major step up from TRL 6, requiring the demonstration of 
an actual system prototype in a realistic environment, such as in an 
aircraft, vehicle, or space. Examples include testing the prototype in 
a test bed aircraft; 
Hardware: Prototype. Should be form, fit, and function integrated with 
other key supporting elements/subsystems to demonstrate full 
functionality of subsystem; 
Demonstration environment: Flight demonstration in representative 
realistic environment such as flying test bed or demonstrator 
aircraft. Technology is well substantiated with test data. 

Technology readiness level: 8. Actual system completed and "flight 
qualified" through test and demonstration; 
Description: Technology has been proven to work in its final form and 
under expected conditions. In almost all cases, this TRL represents 
the end of true system development. Examples include developmental 
test and evaluation of the system in its intended weapon system to 
determine if it meets design specifications; 
Hardware: Flight qualified hardware; 
Demonstration environment: Developmental Test and Evaluation (DT&E) in 
the actual system application. 

Technology readiness level: 9. Actual system "flight-proven" through 
successful mission operations; 
Description: Actual application of the technology in its final form 
and under mission conditions, such as those encountered in operational 
test and evaluation. In almost all cases, this is the end of the last 
"bug-fixing" aspects of true system development. Examples include 
using the system under operational mission conditions; 
Hardware: Actual system in final form; 
Demonstration environment: Operational Test and Evaluation (OT&E) in 
operational mission conditions. 

Source: GAO analysis of NASA data. 

[End of table] 

Appendix IV: Elements of a Sound Business Case: 

The development and execution of a knowledge-based business case for 
the National Aeronautics and Space Administration's (NASA) projects 
can provide early recognition of challenges, allow managers to take 
corrective action, and place needed and justifiable projects in a 
better position to succeed. Our studies of best practice organizations 
show the risks inherent in NASA's work can be mitigated by developing 
a solid, executable business case before committing resources to a new 
product's development.[Footnote 35] In its simplest form, a knowledge-
based business case is evidence that (1) the customer's needs are 
valid and can best be met with the chosen concept and that (2) the 
chosen concept can be developed and produced within existing 
resources--that is, proven technologies, design knowledge, adequate 
funding, adequate time, and adequate workforce to deliver the product 
when needed. A program should not be approved to go forward into 
product development unless a sound business case can be made. If the 
business case measures up, the organization commits to the development 
of the product, including making the financial investment. The 
building of knowledge consists of information that should be gathered 
at these three critical points over the course of a program: 

* When a project begins development, the customer's needs should match 
the developer's available resources--mature technologies, time, and 
funding. An indication of this match is the demonstrated maturity of 
the technologies required to meet customer needs--referred to as 
critical technologies. If the project is relying on heritage--or pre-
existing--technology, that technology must be in the appropriate form, 
fit, and function to address the customer's needs within available 
resources. The project will generally enter development after 
completing the preliminary design review, at which time a business 
case should be in hand. 

* Then, about midway through the project's development, its design 
should be stable and demonstrate it is capable of meeting performance 
requirements. The critical design review takes place at that point in 
time because it generally signifies when the program is ready to start 
building production-representative prototypes. If project development 
continues without design stability, costly re-designs to address 
changes to project requirements and unforeseen challenges can occur. 

* Finally, by the time of the production decision, the product must be 
shown to be producible within cost, schedule, and quality targets and 
have demonstrated its reliability, and the design must demonstrate 
that it performs as needed through realistic system-level testing. 
Lack of testing increases the possibility that project managers will 
not have information that could help avoid costly system failures in 
late stages of development or during system operations. 

[End of section] 

Appendix V: Best-Practices Analysis of Schedules from Three NASA 
Projects: 

We have previously reported that a reliable project schedule can 
enhance cost estimation by contributing to an understanding of the 
cost impact if a particular project does not finish on time. In an 
effort to continually assess the National Aeronautics and Space 
Administration's (NASA) progress in improving its cost-estimating 
policies and external oversight of these policies, we selected three 
projects--Global Precipitation Measurement (GPM), Magnetospheric 
Multiscale (MMS), and James Webb Space Telescope (JWST)--that planned 
significant amounts of implementation-phase work during the audit 
year, and assessed these projects' schedules against best practices 
for project scheduling developed by GAO. We cannot generalize the 
results of these analyses to NASA's scheduling practices across the 
entire portfolio of projects, but the results do provide us with 
insights into areas that could require further investigation in all of 
NASA's major projects, given the critical importance of a reliable 
schedule to NASA's overall ability to estimate the time and funds 
necessary to complete its projects. 

The methodology for analysis of project scheduling was developed by 
GAO in consultation with experts from government and industry and 
assesses a given schedule's reliability based on a combination of four 
characteristics--whether the estimate is comprehensive, well-
constructed, credible, and controlled. A schedule's achievement of 
each of these four characteristics results from the degree to which 
the schedule embodies two or more best practices.[Footnote 36] To 
qualify as reliable, a schedule should substantially or fully meet all 
four of these characteristics. Neither the GPM nor MMS schedules fully 
met any of these four characteristics. See tables 5 and 6. 

Table 5: Summary Assessment of GPM Schedule Estimate Compared to Best 
Practices: 

Characteristic: Comprehensive; 
Overall assessment[A]: Substantially met; 

Best practice: 1. Capturing all activities; 
Individual assessment: Substantially met. 

Best practice: 3. Assigning resources to all activities; 
Individual assessment: Partially met. 

Best practice: 4. Establishing the durations of all activities; 
Individual assessment: Substantially met. 

Characteristic: Well constructed; 
Overall assessment[A]: Partially met; 

Best practice: 2. Sequencing all activities; 
Individual assessment: Partially met. 

Best practice: 6. Confirming that the critical path is valid; 
Individual assessment: Partially met. 

Best practice: 7. Ensuring reasonable total float; 
Individual assessment: Partially met. 

Characteristic: Credible; 
Overall assessment[A]: Partially met; 

Best practice: 5. Verifying that the schedule is traceable 
horizontally and vertically; 
Individual assessment: Partially met. 

Best practice: 8. Conducting a schedule risk analysis; 
Individual assessment: Partially met. 

Characteristic: Controlled; 

Overall assessment[A]: Substantially met; 
Best practice: 9. Updating the schedule with actual progress and logic; 
Individual assessment: Substantially met. 

Best practice: 10. Maintaining a baseline schedule; 
Individual assessment: Substantially met. 

Source: GAO Analysis of NASA Data: 

[A] Not Met - project provided no evidence that satisfies any of the 
criterion, Minimally Met - project provided evidence that satisfies a 
small portion of the criterion, Partially Met - project provided 
evidence that satisfies about half of the criterion, Substantially Met 
- project provided evidence that satisfies a large portion of the 
criterion, and Fully Met - project provided complete evidence that 
satisfies the entire criterion. 

[End of table] 

The GPM schedule included a sufficient amount of detail and data that 
enabled the program to ensure that all activities were included. 
Resources were not identified in the schedule because they were 
managed in the earned value system instead. Durations were mostly 
identified at a manageable level, but we did find some durations that 
were too long to effectively monitor status. While almost all 
activities reflected schedule logic, we found an abundance of 
constraints that were causing breaks in the critical path and kept the 
schedule from responding dynamically to changes. We also found 
activities with high amounts of total float, some of which could be 
delayed anywhere from 4 to 6 years without affecting the program end 
date, which was not realistic.[Footnote 37] In addition, the program 
provided evidence that a rigorous process was followed to conduct a 
schedule risk analysis that identified schedule reserve, however, we 
had no insight into the details of this analysis and we could not 
verify the results. Finally, the schedule had been recently updated by 
a full time scheduler and had been baselined so that variances could 
be tracked. However, due to the abundance of constraints in the 
schedule, we questioned the ability of the schedule to forecast 
realistic start and finish dates and the program did not have a 
schedule baseline document for tracking changes to the schedule. 
[Footnote 38] According to officials in the GPM project office, best 
practices collected for the GAO Schedule Assessment Guide are not 
requirements for NASA schedules. Project officials provided GAO with 
explanations for a number of instances when GPM's schedule did not 
fully meet best practices. In addition, the GPM project office noted 
that an independent Standing Review Board produced the schedule risk 
analysis provided to GAO for assessment against best practices. 
Officials also indicated that the project has successfully used 
current scheduling practices through GPM's development and integration 
and test period, and that scheduling practices are consistent with the 
current phase of the project. 

Table 6: Summary Assessment of MMS Schedule Estimate Compared to Best 
Practices: 

Characteristic: Comprehensive; 
Overall assessment[A]: Substantially met; 

Best practice: 1. Capturing all activities; 
Individual assessment: Fully met. 

Best practice: 3. Assigning resources to all activities; 
Individual assessment: Partially met. 

Best practice: 4. Establishing the durations of all activities; 
Individual assessment: Substantially met. 

Characteristic: Well constructed; 
Overall assessment[A]: Partially met; 

Best practice: 2. Sequencing all activities; 
Individual assessment: Partially met. 

Best practice: 6. Confirming that the critical path is valid; 
Individual assessment: Minimally met. 

Best practice: 7. Ensuring reasonable total float; 
Individual assessment: Partially met. 

Characteristic: Credible; 
Overall assessment[A]: Partially met; 

Best practice: 5. Verifying that the schedule is traceable 
horizontally and vertically; 
Individual assessment: Partially met. 

Best practice: 8. Conducting a schedule risk analysis; 
Individual assessment: Partially met. 

Characteristic: Controlled; 
Overall assessment[A]: Partially met; 

Best practice: 9. Updating the schedule with actual progress and logic; 
Individual assessment: Partially met. 

Best practice: 10. Maintaining a baseline schedule; 
Individual assessment: Partially met. 

Source: GAO analysis of NASA data: 

[A] Not Met - project provided no evidence that satisfies any of the 
criterion, Minimally Met - project provided evidence that satisfies a 
small portion of the criterion, Partially Met - project provided 
evidence that satisfies about half of the criterion, Substantially Met 
- project provided evidence that satisfies a large portion of the 
criterion, and Fully Met - project provided complete evidence that 
satisfies the entire criterion. 

[End of table] 

The MMS schedule covered the entire scope of work including both 
government and contractor efforts, but half of the remaining 
activities did not have resources assigned. For the activities that 
did reflect resources, we found that many of the resources were over 
allocated. Project officials indicated that total resources were not 
identified in the schedule but they were managed in the financial 
system. In keeping with best practices, activity durations were mostly 
short allowing for effective management. In addition, while logic 
links were in place for the majority of the remaining activities, 
about 20 percent were missing predecessor and successor logic and 28 
percent had constraints that caused us to question the validity of the 
critical path. Furthermore, several activities had high total float 
values with some greater than 1,000 days meaning that an activity 
could slip up to 4 years without affecting the program end date which 
may not be reasonable. There were also several activities with 
negative float implying that this work was behind schedule. The 
problems with missing logic and constraints affected the credibility 
of the schedule. For example, when we extended two activity durations 
by hundreds of days, there was no corresponding effect on the schedule 
end date due to constraints which prohibited the finish date from 
moving. The project had conducted a joint confidence level assessment, 
but the review was done more than 4 years ago so the results were 
outdated. While project officials indicated that they received 
subsequent schedule risk analysis reviews conducted by independent 
groups, due to the problems with the schedule network, we questioned 
the results of these analyses. Finally, although the schedule had been 
baselined and the project office provided evidence that they 
continually monitored the schedule as part of their project management 
efforts, the overall integrated master schedule showed a status date 
from 4 years ago. Moreover, out of the 23 underlying sub-schedules 
only 3 reflected valid status dates. There were also some activities 
with start and finish dates recorded in the future which suggested a 
lack of control. 

The JWST project has an integrated master schedule,[Footnote 39] but 
it is not finalized because major contracts have yet to be negotiated 
and definitized. Because we did not assess the project's integrated 
master schedule, we conducted a preliminary assessment of two JWST 
major subsystems--the Integrated Science Instrument Module and the 
Optical Telescope Element and Integrated Science Instrument Module 
(OTIS)--that represent key in-house efforts for the project. Most of 
the issues we identified lay in the OTIS schedule, including the lack 
of a valid critical path--defined as time associated with activities 
that drive the overall schedule.[Footnote 40] Until the schedule can 
produce a true critical path, the project will not be able to provide 
reliable timeline estimates or identify when problems or changes may 
occur and their effect on work occurring later in the development life 
cycle. If left unaddressed, these issues could affect the reliability 
of the overall project's integrated master schedule once finalized. We 
plan to conduct a best-practices assessment of JWST's integrated 
master schedule as part of our next annual review of JWST. 

[End of section] 

Appendix VI: Comments from the National Aeronautics and Space 
Administration: 

National Aeronautics and Space Administration   
Office of the Administrator     
Washington, DC 20546-0001: 
        
February 26, 2013: 

Ms. Cristina Chaplain: 
Director: 
Acquisition and Sourcing Management: 
United States Government Accountability Office: 
Washington, DC 20548: 

Dear Ms. Chaplain: 

The National Aeronautics and Space Administration (NASA) appreciates 
the opportunity to comment on the Government Accountability Office 
(GAO) report entitled "Assessments of Selected Large-Scale Projects" 
(GAO-13-276SP). NASA remains dedicated to continuous improvement of 
its acquisition management processes and performance and will continue 
to work with GAO to identify and address any challenges that may lead 
to cost and schedule growth of our projects. 

The GAO's Congressionally mandated annual assessment is a good 
opportunity for NASA to receive an independent perspective on its 
performance in acquisition of major programs and projects. While we 
are not always fully in agreement, we appreciate the GAO's insights 
and the opportunity for open dialogue. NASA values the continued open 
and constructive communications between NASA and the GAO on this 
effort and appreciates the ongoing work by the GAO assessment team.
NASA is pleased that the GAO has recognized NASA's achievements over 
the past years, including our successful launch of three projects in 
2012 and early 2013. NASA launched the Radiation Belt Storm Probes 
(RBSP), the Tracking and Data Relay Satellite (TDRS-K), and Landsat 
Data Continuity Mission (LDCM). Since its launch on August 30, 2012, 
RBSP has completed its commission phase of operations, all of the 
spacecraft's systems and instruments are now active, and the satellite 
is now in its prime science mission RBSP was renamed the Van Allen 
Probes to honor Dr. James Van Allen who led the first efforts to study 
Earth's magnetic environment in situ via the first Explorer mission; 
the first of NASA's three next-generation Tracking and Data Relay 
Satellites (MRS), TDRS-K launched on January 30, 2013. TORS-K bolsters 
our network of satellites that provides essential communications to 
support space exploration. It will improve the overall health and 
longevity of our system. LDCM launched on February 11, 2013, and is 
the eighth satellite in the joint NASA-U.S. Geological Survey (USGS) 
Landsat program. Following its commissioning later this year, LDCM 
will continue the over 40-year record of the Landsat land imaging data 
set. 

NASA also recognizes that continued refinement, improvement, 
implementation, and monitoring of sound acquisition practices, 
policies, and processes are essential to reducing project cost and 
schedule growth on future missions. As highlighted in the GAO's draft
report, NASA continues to implement several initiatives to mitigate 
acquisition management risks. We are extremely pleased to see progress 
toward improving cost and schedule performance, although we will look 
to see if this trend continues over the next several years before 
claiming success that our processes have reached the level of maturity 
and stability to endure the continuing challenges NASA faces in future 
years. 

As part of NASA's continuous improvement in cost estimating policies 
and processes, NASA instituted a Joint Cost and Schedule Confidence 
Level (JCL) Policy in 2009 to improve the assessment of the likelihood 
of project success at the specified funding level. The JCL is one of 
many inputs into the baseline decision process when a project is 
confirmed at Key Decision Point (KDP)-C, and has the desired effect of 
increasing insight by both the project manager and NASA's leadership 
by appropriately taking into account risk-based uncertainties 
associated with the integrated cost and schedule plan. NASA continues 
to implement and improve the use of the JCL process and will continue 
to assess the impact of this process on cost and schedule performance 
as more projects baselined under this policy complete their 
development phase. Another aspect to the continuous improvement in 
cost estimating policies and processes policy is the implementation of 
probabilistic analyses for project cost and schedule for those 
projects that are transitioning into the last phase of formulation.
The draft report identifies four challenges that could affect NASA's 
continued progress including: (1) managing competing priorities within 
the context of constrained budgets, (2) estimating costs associated 
with several large-scale projects, (3) improving overall cost and 
schedule estimation, and (4) using consistent and proven design 
stability metrics. The first two are closely related, and the GAO's 
report rightly observes that any large cost overruns that might occur 
in NASA's three flagship missions--James Webb Space Telescope (JWST), 
Space Launch System (SLS), and Orion Multi-Purpose Crew Vehicle (MPCV) 
would make it challenging for NASA to manage its portfolio of 
projects. In addition, programs are experiencing an uncertain and 
unstable funding environment, which can drive less than optimal 
phasing of current and future program work and can result in program 
cost increases. 

Since the rebaseline of JWST in 2011, the project has consistently 
remained within its new cost and schedule baseline, The project 
continues to make excellent progress meeting its annually established 
milestones and is actively working toward spacecraft critical design 
review (CDR) later this year. While challenges will remain throughout 
the implementation phase, the project has implemented several new 
mechanisms to improve communication between NASA and its contractors, 
at all levels of the project, in order to proactively respond to any 
issues as they arise. NASA credits the rigor placed into the 
continuous improvement plan, as stated previously. Orion and SLS are 
making excellent progress, though the current lack of flexibility to 
internally rebalance funding across the major elements of the Human 
Exploration and Operations (EEO) Mission Directorate is a challenge. 

Improving cost and schedule estimating practices is important to NASA 
as mentioned previously. In the draft report, GAO mentions a long-term 
weakness in cost and schedule estimating practices leading to 
optimistic and unrealistic plans based on lack of adherence to GAO's 
best practices, insufficient resources, and qualified staff. NASA does 
take pride in having an optimistic view, which is why NASA has 
accomplished great achievements throughout the years. We also take 
pride in being good stewards to our stakeholders and thus work hard to 
define, and then manage, project cost and schedule commitments. 
Although our practices and policies may not completely align with 
GAO's best practices, NASA is confident in our project management 
practices and takes the necessary steps to address any gaps as they 
relate to an identified weakness. The report specifically mentions 
earned value management (EVM) as important to estimating practices. 
NASA is willing to focus on these concerns, as demonstrated by the 
steps we are taking to address GAO's recommendations in its EVM audit. 
Another area noted in your draft report is the development of 
realistic, dynamic, and logic driven schedules. While GAO could not 
validate NASA projects against GAO's bust practices for project 
scheduling, we hope that GAO would agree that the NASA projects 
demonstrated an understanding of the critical path and scheduling 
practices, based on NASA policy and practices. 

With regard to use of consistent and proven design stability metrics, 
NASA has continued to work with the GAO to adapt the recommendations 
related to design metrics, as indicated by the implementation of 
changes to NASA policy to identify technical indicators to assess 
project maturity. As noted in the draft report, GAO mentions the 
differing approaches to design stability between NASA and GAO. NASA 
has identified many technical indicators to assess a project's design 
stability and maturity over time, and projects have begun to 
consistently track their progress against these metrics. NASA's 
opinion is that there are many indicators to measure a project's 
readiness as they move forward from CDR and a one-size-fits-all 
approach may not be appropriate. NASA's plan is to continue to enhance 
the ability to monitor and assess the stability of programs and 
projects as well as the maturity of their products through the 
development process using a more comprehensive approach, including a 
set of technical indicators developed and implemented, based on the 
project's unique complexity and mission objectives. This approach is 
consistent with our policies and handbooks for program and project 
management. 

Finally, NASA would like to thank the GAO for considering and 
incorporating many technical corrections provided by projects' subject 
matter experts as part of the audit. Inclusion of these comments is 
important to present an accurate and balanced view of the projects' 
technical status, and as such, NASA looks forward to working with GAO 
to make sure the number of approved technical corrections continues to 
increase. 

NASA is committed to working with GAO to address any questions. We 
appreciate the ongoing opportunity to comment on the assessments and 
will continue to work with the GAO to ensure that comments are 
submitted and appropriately reflected in final reports. If you have 
any questions or require additional information, please contact Tracy 
Osborne at (202) 358-3795. 

Sincerely, 

Signed by: 

Lori B. Garver: 
Deputy Administrator: 

[End of section] 

Appendix VII: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Cristina Chaplain, (202) 512-4841 or chaplainc@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Shelby S. Oakley, Assistant 
Director; Jessica M. Berkholtz; Richard A. Cederholm; Tisha D. 
Derricotte; Laura Greifner; Jeffrey L. Hartnett; Don Kiggins; Ramzi N. 
Nemo; Kenneth E. Patton; Sylvia Schatz; Ryan Stott; and Roxanna T. Sun 
made key contributions to this report. 

[End of section] 

Footnotes: 

[1] The Space Network Ground Segment Sustainment (SGSS) project 
entered the implementation phase in April 2013, after our review of 
projects had concluded. This project is not included in our analyses 
of projects in implementation, however, the project's baseline cost 
and schedule is reported in the two page assessment. 

[2] NASA is required to report to certain committees in the House and 
Senate if the development cost of a program is likely to exceed the 
baseline estimate by 15 percent or more, or if a milestone is likely 
to be delayed by 6 months or more. 51 U.S.C. § 30104(e). 

[3] GAO, Best Practices: Using a Knowledge-Based Approach to Improve 
Weapon Acquisition, [hyperlink, 
http://www.gao.gov/products/GAO-04-386SP] (Washington, D.C.: Jan. 
2004). 

[4] GAO, Schedule Assessment Guide: Best Practices for Project 
Schedules, [hyperlink, http://www.gao.gov/products/GAO-12-120G] 
(Washington, D.C.: May 30, 2012). 

[5] NASA defines the formulation phase as the identification of how 
the program or project supports the agency's strategic goals; the 
assessment of feasibility, technology, concepts, and performance of 
trade studies; risk assessment and possible risk mitigations and 
continuous risk management processes; team building, development of 
operations concepts and acquisition strategies; establishment of high-
level requirements, requirements flow down, and success criteria; 
assessing the relevant industrial base/supply chain to ensure program 
or project success, the preparation of plans, budgets, and schedules 
essential to the success of a program or project; and the 
establishment of control systems to ensure performance of those plans 
and alignment with current agency strategies. NASA Procedural 
Requirements (NPR) 7120.5E, paragraph 1.3.1.a (Aug. 14, 2012). The 
implementation phase is defined as the execution of approved plans for 
the development and operation of the program or project, and the use 
of control systems to ensure performance to approved plans and 
requirements and continued alignment with the agency's strategic 
goals. NPR 7120.5E, paragraph 1.3.1.c (Aug. 14, 2012). 

[6] According to NPR 7120.5E, Table 2-5 (Aug. 14, 2012), the PDR 
evaluates the completeness/consistency of the planning, technical, and 
cost/schedule baselines developed during formulation. It assesses 
compliance of the preliminary design with applicable requirements, and 
determines if the project is sufficiently mature to begin the final 
design and fabrication phase. 

[7] The agency baseline commitment establishes and documents an 
integrated set of requirements, cost, schedule, technical content, and 
an agreed-to joint cost and schedule confidence level that forms the 
basis for NASA's commitment with OMB and Congress. NPR 7120.5E, 
Appendix A (Aug.14, 2012). 

[8] According to NPR 7120.5E, Table 2-5 (Aug. 14, 2012), the CDR 
evaluates the integrity of the project design and its ability to meet 
mission requirements, with appropriate margins and acceptable risk, 
within defined project constraints, including available resources. It 
determines if the design is appropriately mature to continue with the 
final design and fabrication phase. 

[9] The SIR evaluates the readiness of the project and associated 
supporting infrastructure to begin system assembly, integration, and 
test. SIR evaluates whether the remaining project development can be 
completed within available resources and determines if the project is 
sufficiently mature to begin phase D, where test and integration 
activities occur. NPR 7120.5E, Table 2-5 (Aug.14, 2012). 

[10] A rebaseline is when cost, resources, and/or schedule commitments 
are revised. NASA projects are rebaselined when their estimated 
development cost exceeds NASA's baseline commitment development cost 
by 30 percent or more and Congress has reauthorized the project; 
events external to NASA make a rebaseline appropriate; or a NASA 
Associate Administrator determines that the project's scope changed 
from the approved project baseline. 

[11] Following this mission's launch, NASA renamed it the Suomi-
National Polar-orbiting Partnership. 

[12] Appendix III provides a description of the metrics used to assess 
technology maturity and appendix IV contains detailed information 
about the project attributes highlighted by knowledge-based metrics at 
each stage of systems development. 

[13] NASA Procedural Requirements 7123.1A, NASA Systems Engineering 
Processes and Requirements with Change 1, Appendix G, paragraph G.19b 
(Mar. 26, 2007). 

[14] For purposes of this analysis, we use the term "mission-critical 
software" to refer to software included under NASA software classes A 
or B. NASA defines Class A software as, among other things, ground and 
flight software systems needed to perform a primary mission objective 
of human space flight, and which directly interacts with human space 
flight systems. NASA defines Class B software as, among other things, 
software that must perform reliably to accomplish primary mission 
objectives or major function(s) in Non-Human Space Rated Systems. 

[15] Engineering drawings are considered to be a good measure of the 
demonstrated stability of a product's design because the drawings 
represent the language used by engineers to communicate to the 
manufacturers the details of a new product design--what it looks like, 
how its components interface, how it functions, how to build it, and 
what critical materials and processes are required to fabricate and 
test it. Once the design of a product is finalized, the drawing is 
"releasable." 

[16] Appendix IV contains detailed information about the project 
attributes highlighted by knowledge-based metrics at each stage of 
systems development. 

[17] The JCL is a probabilistic analysis that includes, among other 
things, all cost and schedule elements, incorporates and quantifies 
potential risks, assesses the impacts of cost and schedule to date, 
and addresses available annual resources to arrive at development cost 
and schedule estimates associated with various confidence levels. 

[18] NASA plans to design a 70 metric ton SLS vehicle which will be 
capable of launching a crew capsule into low-Earth orbit. NASA plans 
to evolve this design and develop a vehicle capable of carrying 130 
metric ton payloads to orbit. 

[19] GAO, Earned Value Management Implementation across Major (NASA) 
Spaceflight Projects is Uneven, [hyperlink, 
http://www.gao.gov/products/GAO-13-22] (Washington, D.C.: Nov. 19, 
2012). EVM is a project management tool that, when properly used, can 
provide accurate assessments of project progress, produce early 
warning signs of impending schedule delays and cost overruns, and 
provide unbiased estimates of anticipated costs at completion. 

[20] When an EVM system is certified the agency has assurance that the 
implemented system can be considered to provide reliable and valid 
data from which to manage a project. Certification of an EVM system 
ensures that the implemented system was validated for compliance with 
American National Standards Institute/Electronic Industries Alliance 
(ANSI/EIA)-748 standard by independent and qualified staff. ANSI/EIA-
748 is regarded as the national standard and an industry best practice 
for EVM systems. It describes 32 guidelines that a certified EVM 
system must meet in the areas of organization; planning, scheduling, 
and budgeting; accounting; analysis and management reports; and 
revisions and data maintenance. 

[21] GAO, James Webb Space Telescope: Actions Needed to Improve Cost 
Estimate and Oversight of Test and Integration, [hyperlink, 
http://www.gao.gov/products/GAO-13-4] (Washington, D.C.: Dec. 3, 2012). 

[22] National Research Council, the National Academies. Controlling 
Cost Growth of NASA Earth and Space Science Missions, (Washington D.C. 
2010). 

[23] GAO, Additional Cost Transparency and Design Criteria Needed for 
National Aeronautics and Space Administration (NASA) Projects, 
[hyperlink, http://www.gao.gov/products/GAO-11-364R] (Washington, 
D.C.: Mar. 3, 2011); GAO, NASA: Issues Implementing the NASA 
Authorization Act of 2010, [hyperlink, 
http://www.gao.gov/products/GAO-11-216T] (Washington, D.C.: Dec.1, 
2010); GAO, NASA: Implementing a Knowledge-Based Acquisition Framework 
Could Lead to Better Investment Decisions and Project Outcomes, 
[hyperlink, http://www.gao.gov/products/GAO-06-218] (Washington, D.C.: 
Dec. 21, 2005). 

[24] Mass is a measurement of how much matter is in an object. It is 
related to an object's weight and is mathematically equal to mass 
multiplied by acceleration due to gravity. Margin is the spare amount 
of mass or power allowed or given for contingencies or special 
situations. A request for action is a formal written request sponsored 
by the review panel asking for additional information or action by the 
project team. It is generally developed as a result of insufficient 
safety, technical, or programmatic information being available at the 
time of the review. 

[25] The launch dates on the NASA launch manifest (or manifested 
launch date) are the desired launch dates as determined by the payload 
mission and approved by the NASA Flight Planning Board, and are not 
typically the same as the Agency Baseline Commitment schedule dates. 
According to NASA, a launch manifest is a dynamic schedule that is 
affected by real world operational activities conducted by NASA and 
multiple other entities. It reflects the results of a complex process 
that requires the coordination and cooperation by multiple users for 
the use of launch range and launch contractor assets. The launch dates 
shown on NASA's launch manifest are a mixture of "confirmed" range 
dates for missions launching within approximately 6 months, and 
contractual/planning dates for the missions beyond 6 months from 
launch. The launch manifest date is typically earlier than the Agency 
Baseline Commitment schedule date, which according to NASA allows for 
the operationally driven fluctuations to the launch schedule. The 
launch manifest is updated on a periodic basis throughout the year. 

[26] For the fixed-price contracts discussed in this report, the 
initial contract values plus contract modifications issued to 
equitably adjust the contract costs equal the current contract values. 
For the cost-reimbursement contracts, the current contract value can 
be greater than the initial contract value when the government is 
required to reimburse the contractor for increased costs associated 
with performance. 

[27] [hyperlink, http://www.gao.gov/products/GAO-12-120G]. 

[28] Because of changes in NASA's accounting structure, its historical 
cost data are relatively inconsistent. As such, we used then-year 
dollars to report data consistent with the data NASA reported to us. 

[29] [hyperlink, http://www.gao.gov/products/GAO-04-386SP]. 

[30] [hyperlink, http://www.gao.gov/products/GAO-12-120G]. 

[31] In our calculation for the percentage of total number of drawings 
projected for release, we used the number of drawings released at the 
critical design review as a fraction of the total number of drawings 
projected, including where a growth in drawings occurred. So, the 
denominator in the calculation may have been larger than what was 
projected at the critical design review. We believe that this more 
accurately reflected the design stability of the project. 

[32] [hyperlink, http://www.gao.gov/products/GAO-11-364R]. 

[33] Some projects reported that their spacecraft would be ready for 
launch sooner than the date that the launch authority could provide 
actual launch services. In these cases, we used the actual launch date 
for our analysis rather than the date that the project reported 
readiness. 

[34] These missions include Ice, Cloud, and Land Elevation Satellite-
2, Origins-Spectral Interpretation-Resource Identification-Security-
Regolith Explorer, Solar Probe Plus, and Space Launch System. 

[35] GAO, Defense Acquisitions: Key Decisions to Be Made on Future 
Combat System, [hyperlink, http://www.gao.gov/products/GAO-07-376] 
(Washington, D.C.: Mar. 15, 2007); Defense Acquisitions: Improved 
Business Case Key for Future Combat System's Success, [hyperlink, 
http://www.gao.gov/products/GAO-06-564T] (Washington, D.C.: Apr. 4, 
2006); NASA: Implementing a Knowledge-Based Acquisition Framework 
Could Lead to Better Investment Decisions and Project Outcomes, 
[hyperlink, http://www.gao.gov/products/GAO-06-218] (Washington, D.C.: 
Dec. 21, 2005); and NASA's Space Vision: Business Case for Prometheus 
1 Needed to Ensure Requirements Match Available Resources, [hyperlink, 
http://www.gao.gov/products/GAO-05-242] (Washington, D.C.: Feb. 28, 
2005). 

[36] [hyperlink, http://www.gao.gov/products/GAO-12-120G]. 

[37] Total float, or slack, is the amount of time an activity can be 
delayed before the dates of its successor activities or the program's 
finish milestone are affected. Float is calculated from an activity's 
early start date and is the amount of time the activity can be delayed 
before delaying the early start of its successor or the project finish 
date. 

[38] Constraints can be placed on an activity's start or finish date 
and can limit the movement of an activity into the past or future or 
both. Because constraints override network logic and restrict how 
planned dates respond to actual accomplished effort or resource 
availability, they should be used only when necessary and only if they 
are justified in the schedule documentation. Generally, constraints 
are used to demonstrate an external event's effect on the schedule. 

[39] As a document that integrates the planned work, the resources 
necessary to accomplish that work, and the associated budget, the 
integrated master schedule should be the focal point of project 
management. In this guide, an integrated master schedule constitutes a 
project schedule that includes the entire required scope of effort, 
including the effort necessary from all government, contractor, and 
other key parties for a project's successful execution from start to 
finish. 

[40] A project's critical path indicates the path of longest duration 
through a sequence of scheduled activities. Establishing a valid 
critical path is necessary for examining the effects of any activity's 
slipping along this path. A project's critical path determines the 
project's earliest completion date and focuses the team's energy and 
management's attention on the activities that will lead to the 
project's success. 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]