This is the accessible text file for GAO report number GAO-05-570R entitled 'Defense Acquisitions: Incentives and Pressures That Drive Problems Affecting Satellite and Related Acquisitions' which was released on June 24, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Washington, DC 20548: June 23, 2005: The Honorable C.W. Bill Young: Chairman, Subcommittee on Defense: Committee on Appropriations: House of Representatives: Subject: Defense Acquisitions: Incentives and Pressures That Drive Problems Affecting Satellite and Related Acquisitions: Dear Mr. Chairman: In fiscal year 2006, the Department of Defense (DOD) expects to spend more than $23 billion to develop, acquire, and operate satellites and other space-related systems. These systems are becoming increasingly critical to every facet of military operations as well as the U.S. economy and homeland security. Satellite systems collect information on the capabilities and intentions of potential adversaries. They enable U.S. military forces to be warned of missile attacks and to communicate and navigate while avoiding hostile actions. They provide information that allows forces to precisely attack targets in ways that minimize collateral damage and loss of life. DOD's satellites also enable global communications; television broadcasts; weather forecasting; disaster planning; navigation of ships, planes, trucks, and cars; and synchronization of computers, communications, and electric power grids. DOD's introduction of these desirable capabilities over time has not come without difficulties. Space system acquisitions have experienced problems over the past several decades that have driven up costs by hundreds of millions, even billions of dollars, stretched schedules by years, and increased performance risks. In some cases, capabilities have not been delivered to the warfighter after decades of development. As a result of these problems, DOD is now contending with important trade-off decisions, such as the following. * Whether to keep striving to build its Space-Based Infrared System (SBIRS) High as intended or cut back on capabilities. This system is intended to replace and upgrade an older generation of missile-warning satellites, but its cost has already more than doubled and continues to increase, and its schedule has stretched for years. * Whether and how much to employ lower orbiting satellites equipped with similar capabilities to facilitate missile defense activities. DOD had spent two decades on this effort without launching a single satellite. Cost and schedule problems forced DOD to rebaseline the program several times. Overall affordability of missile defense has driven DOD to assess whether to continue with this particular effort as well as pursue development of a newer generation of missile-tracking satellites. * Whether to limit the acquisition of new communication satellites, known as the Advanced Extremely High Frequency (AEHF) satellites, in favor of developing a newer generation of laser-linked satellites, known as the Transformational Satellite Communications System (TSAT). The AEHF program is running over cost and schedule, but it incorporates more mature technologies. TSAT promises dramatically greater bandwidth and processing capabilities and is considered integral to DOD's efforts to network all of its weapon systems, but there is much less certainty as to how much the system will cost or when it can be delivered because critical technologies are not mature. * Whether to pursue incremental increases in capability for the Global Positioning System or embark on a more expensive program that would offer more dramatic capability advances. Two years ago, we issued a report to your subcommittee that analyzed reports we had previously issued on satellite and other space-related programs over the past two decades as well as other studies. Our 2003 report identified common problems affecting those acquisitions.[Footnote 1] Generally, the problems we identified were common to DOD weapons acquisitions and were recognized within DOD and the space community. In February 2005, you requested that we identify underlying incentives and pressures that drive the problems we had identified earlier. You also asked that we complete our fieldwork by April 2005 to support the subcommittee's decisions on DOD's appropriations. To respond to your request, we analyzed a wide body of GAO, DOD, and industry studies (see enc. III) that discuss acquisition problems and underlying incentives and pressures, including our work on best practices in weapon system development that we have conducted over the past decade, our individual reviews of space system acquisitions and crosscutting problems, DOD's independent study of problems affecting SBIRS High, past DOD studies of crosscutting problems with space system acquisitions, and a more recent DOD joint task force study on the acquisition of national security space programs. We also conducted interviews with more than 40 individuals (see enc. IV)--including experienced space acquisition program managers and program executive officials within Air Force Space Command and its Space and Missile Systems Center, officials responsible for science and technology (S&T) activities that support space, former and current officials within the Office of the Secretary of Defense who have specific responsibility for space oversight or more general weapon system acquisition policy and oversight, and individuals representing various aspects of industry. We conducted our review from February 2005 to April 2005 in accordance with generally accepted government auditing standards. Results in Brief: The officials we spoke with for this review cited a set of incentives and pressures underlying the space acquisition problems that are largely reflective of a lack of an overall investment strategy and a corresponding tendency to set start dates for programs before a sound business case for them has been established. Specifically, they told us that DOD starts more programs than it can afford and rarely prioritizes them for funding purposes. Such an approach has cascading effects--from creating negative behaviors associated with competing for funds, to increasing technology challenges, to creating unanticipated and disruptive funding shifts, to stretching out schedules in order to accommodate the whole portfolio of space programs. Our previous reports have found these pressures are long-standing and common to weapon acquisitions, not just space systems. In addition, officials we spoke with also cited pressures resulting from having a diverse array of officials and organizations involved with the acquisition process, tensions between the S&T and acquisition communities as to who is better suited to translate technology concepts into reality, pressures resulting from short tenures among staff critical to achieving acquisition success, and difficulties in overseeing contractors. We are not making recommendations in this report because it was not within the scope of our work to determine the actions needed to redirect the complex set of incentives and pressures affecting space programs. However, as we point out, our previous reports have already made recommendations--some of which have been implemented--that we believe would enable DOD to put space acquisition and other weapons programs on a sounder footing. In commenting on our report, DOD pointed out that it has recently taken steps such as improving requirements setting for all weapons systems and ensuring that decisions to start space acquisition programs are based on adequate knowledge. Where appropriate in this report, we also identify and present our views on solutions being discussed and implemented within DOD. Background: The majority of satellite acquisition programs that DOD has pursued over the past several decades cost more than expected and took longer to develop and launch than planned. In our 2003 report, we tied these results to four problems. 1. Requirements for what the satellite needed to do and how well it must perform were not adequately defined at the beginning of a program or were changed significantly once the program had begun. 2. Investment practices were weak. For example, potentially more cost- effective approaches were not examined and cost estimates were optimistic. 3. Acquisition strategies were poorly executed. For example, competition was reduced in order to get a program started quickly, or DOD did not adequately oversee contractors. 4. Technologies were not mature enough to be included in product development. We also reported that several factors contributed to these problems. First, DOD often set dates for delivering capabilities on the basis of optimism rather than the knowledge that critical technologies would work as intended by those dates. As a result, activities essential to understanding and containing costs, maximizing competition among contractors, and testing technologies were compressed or not done. Second, a diverse array of organizations with competing interests have been involved in overall satellite development--from the individual military services to testing organizations, contractors, civilian agencies, and, in some cases, international partners. This created challenges in making tough trade-off decisions. Third, space acquisition programs have historically attempted to satisfy all requirements in a single step, regardless of the design challenge or the immaturity of technologies to achieve the full capability. This approach made it difficult to match requirements to available resources (in terms of time, money, and technology). We also reported that other factors created challenges for the satellite acquisition programs we reviewed. These include a shrinking industrial base, a declining space workforce, difficulties associated with testing satellites in a realistic environment, as well as challenges associated with launching satellites. DOD's own reviews have identified similar problems as our review and expanded on factors that helped drive those problems. Most recently, DOD conducted a Defense Science Board/Air Force Scientific Advisory Board Joint Task Force study in 2003 (known as the Young Panel report) to assess the acquisition of national security space programs and develop a road map for reform. The Young Panel found that over time, "cost has replaced mission success" as the primary driver in managing acquisitions, resulting in excessive technical and schedule risk. Specifically, the Young Panel reported that program managers face far less scrutiny on program technical performance than they do on executing against the cost baseline. The Young Panel said there are a number of reasons this is so detrimental--the primary ones being that space is unforgiving, thousands of good decisions can be undone by a single engineering flaw or workmanship error, and these flaws and errors can result in catastrophe. The best way to avoid such problems is an unrelenting emphasis on quality. The Young Panel noted that in the past, space programs had embraced this approach. Our own reports have shown that space programs have not done a good job at executing against their cost baselines. For example, costs for one of DOD's most important programs, SBIRS High, have more than doubled, and they continue to grow. Our studies have also found that cost increases within DOD's space programs are often attributable to the fact that programs were started without sufficient knowledge as to what resources would be needed to achieve success. The Young Panel similarly recognized that the best cost performance is achieved when there is an emphasis on mission success, which means taking steps to reduce technical and schedule risk and making investments that enhance quality. The Young Panel also found that unrealistic cost estimates had led to unrealistic budgets and unexecutable programs. Specifically, the panel found that the space acquisition system is strongly biased to produce unrealistically low cost estimates throughout the process. During program formulation, advocacy tends to dominate and a strong motivation exists to minimize program cost estimates. Moreover, proposals from competing contractors typically reflected the minimum program content and a "price to win." Our own studies as well as other DOD studies have found that unrealistic estimates are common among all weapon systems, not just space systems, and that low estimates help ensure that the program will win support over competing programs and be funded. Like our study, the Young Panel also found that undisciplined definition of and uncontrolled growth in requirements contributed to cost growth and schedule delays and that flawed acquisition strategies did so as well. In particular, the Young Panel, as well as a preceding review of SBIRS High, found that the adoption of a Total System Performance Responsibility policy in the 1990s--which lessened the government program management role in favor of a stronger industry role--essentially eroded the government's ability to effectively manage and oversee space programs and placed too much responsibility on industry to define requirements and make tradeoff decisions. Over time, this shift as well as other well-intended reforms resulted in declines in critical capabilities within the government space workforce, particularly for systems engineering. DOD has recognized that problems with its space acquisitions need to be addressed, and it has taken a range of actions, including shifting away from Total System Performance Responsibility to stronger government management and oversight of space programs, strengthening cost estimating capabilities, adding independent oversight reviews to the decision-making process, and adding discipline to requirements setting. However, our recent reports and testimonies have recommended that DOD also focus on ensuring that acquisition programs not begin until adequate knowledge has been accumulated on critical technologies and suggested that DOD still needs to guide its overall space portfolio with an investment strategy that makes high-level trade-offs before beginning programs. Moreover, our reports on all weapon system acquisitions have continually pointed out a need to recognize and find ways to address the underlying incentives and pressures that drive acquisition problems. Without doing so, the impact of changes in policies or processes will continue to be limited. Incentives and Pressures that Drive Space System Acquisition Problems: The officials we spoke with for this review cited a set of incentives and pressures underlying the space acquisition problems that are largely reflective of a lack of an overall investment strategy and priority setting and a corresponding tendency to set start dates for programs before a sound business case for them has been established. In addition, officials we spoke with also cited pressures resulting from having a diverse set of individuals and organizations involved with the acquisition process, tensions between the S&T and acquisition communities as to who is better suited to translate technology concepts into reality, pressures resulting from short tenures among staff key to achieving acquisition success, and difficulties in overseeing contractors. Our own reviews have identified similar incentives and pressures and found them common among weapon system acquisitions. Too Many Programs Competing for Funding: Many of the officials we spoke with identified pressures and incentives that are rooted in the widely held belief that DOD starts more space programs than it can afford and rarely prioritizes programs for funding purposes before or after starting them. Such an approach has cascading effects--from creating negative behaviors associated with competing for funds, to increasing technology challenges, to creating unanticipated and disruptive funding shifts, to stretching out schedules in order to accommodate the whole portfolio of space programs. Figure 1 highlights the cycle of pressures when DOD initiates too many programs with too little funding. Figure 1: Overview of Pressures Resulting from Beginning More Programs than DOD Can Afford in the Long Run: [See PDF for image] [End of figure] Specifically, officials told us the following. * DOD starts more programs than it can afford over the long run, forcing programs to underestimate costs and overpromise capability. This was attributed to both the Office of the Secretary of Defense and the Air Force. The September 11, 2001, terror attacks on the United States spurred DOD to attempt to pursue even more satellite programs, believing that there was now a greater need for persistent surveillance and more robust communication and networking capabilities. * When faced with a lower budget, senior executives within Office of the Secretary of Defense and the Air Force would rather make across- the-board cuts to all space programs than hard decisions as to which ones to keep and which ones to cancel or cut back. * Because programs are funded annually and priorities have not been established, competition for funding continues over time, forcing programs to view success as the ability to secure the next installment rather than the end goal of delivering capabilities when and as promised. * More often than not, DOD seeks substantial leaps in capability versus incremental leaps. While this approach helps a program to gain support, it substantially increases the technical challenge and the level of unknowns about a program at the time it is started. * Having to continually "sell" a program also creates incentives to suppress bad news about the program's status and avoid activities that uncover bad news. * Launching demonstrators in space is a good way to reduce risks and learn about technologies before starting a new acquisition program. But because of the high cost of testing technologies in space and the overall competition for funding, programs are incentivized not to pursue this approach. At the same time, resources outside acquisition programs devoted to testing in an operational environment are declining. * DOD faces resource shortages beyond funding because it starts more programs than it can afford. Principally, it does not have a sufficient workforce to support space acquisitions or experienced program managers to guide them. Our previous reports have found that these pressures are long-standing and common to weapon acquisitions, not just space acquisitions. The competition within DOD to win funding and get approval to start a new program is intense, creating strong incentives to make a weapon system stand out from existing or alternative systems. If the system does not stand out or prevail over alternatives, the program could be terminated. Moreover, overall DOD funding constraints put a high priority on affordability, making it important for program sponsors to provide cost estimates that will fit within the funding constraints. Instead of forcing trade-offs, challenging performance requirements-- when coupled with other constraints, such as cost or the weight of the satellite--can drive product developers to pursue exotic solutions and technologies that, in theory, can do it all. Moreover, in weapon acquisitions, optimistic cost estimates are encouraged because they help gain program approval and attract budgetary resources. The consequences of cost growth are not directly felt by an individual program because they are "accommodated" through delivery delays and quantity changes and by spreading the cost impact across many programs. We have also reported that the practice of breaching cost and schedule objectives to meet difficult requirements would not persist without a customer's cooperation. Unlike commercial customers, DOD customers tend to be tolerant of cost overruns and delays in order to get a high- performance weapon system. Traditionally, customers have been willing to wait long periods of time for a capability. They would rather wait for the most desirable system to be developed than accept a less capable system, thinking that they may not get the opportunity to acquire a new or modified system in the future. Our recent reports on space and other weapon systems have suggested that having a departmentwide investment strategy for weapon systems or even space systems would help reduce these pressures. Critical components of such a strategy would include identifying overall capabilities and how to achieve them, that is, what role space will play versus other air-, sea-, and land-based assets; identifying priorities for funding; and implementing mechanisms that would enforce the strategy. DOD has made revisions to its requirements-setting and budgeting processes to strengthen investment planning. However, it is unclear as to how these changes will be implemented over time and whether they can serve as a foundation for directing S&T and acquisition investments. To help close knowledge gaps at the onset of programs and shorten development time, DOD has adopted an evolutionary development approach- -that is, pursuing incremental increases in capability versus significant leaps. Our examinations of best practices have found that this approach can decrease time and cost for development because it closes gaps in unknowns. Many of the officials we spoke with believe that evolutionary development could be achieved in space by developing constellations of larger numbers of smaller, more affordable satellites instead of constellations of a few, very large and heavy, complex satellites (commonly referred to as "Battlestar Galacticas" in the space community). In addition to reducing cost and time associated with longer, more challenging programs, this approach could help keep the space industrial base more productive. Complementary alternatives include developing common rather than unique satellite components, cheaper and more responsive launch systems, as well as systems that enable DOD to modify and fix satellites in orbit. DOD is pursuing a range of S&T efforts along these lines. However, DOD's executive agent for space recently testified that these approaches are not technically suitable for some of the capabilities DOD is now pursuing, such as Space Radar (formerly the Space-Based Radar program) and TSAT. We will be undertaking a review to further assess the potential that these approaches offer for producing better outcomes as well as potential barriers to integrating them into the acquisition process. Another solution that has been advocated by the Young Panel and many of the officials we spoke with as a way of addressing gaps between resources and requirements is management reserves. The Young Panel recommended using reserves only to execute the approved program baseline and not for new requirements. The officials we spoke with also said that management reserves may not be needed as much as they currently are if programs do a better job of matching resources to requirements before they begin. In addition, several officials noted that broader investment strategies should be in place so that DOD can afford management reserves. Diverse Array of Officials and Organizations Involved with Space Systems Add Pressure to Requirements Setting: The officials we spoke with widely agreed that the diverse array of officials and organizations involved with a space program make it even more difficult to pare back and control requirements. As officials we spoke with pointed out, space systems may suffer from more requirements pressures because there is usually a very broad constituency behind each satellite program. The Global Positioning System, for example, not only serves military users; it also serves civilians, supports various key economic sectors such as transportation and communications, and is used by allies. The National Polar-Orbiting Operational Environmental Satellite System currently under development will serve military weather forecasters as well as civilian forecasters and a broad community of scientists studying environmental issues. The Space Radar system is expected to play a major role in transforming military as well as intelligence-collecting operations and other critical governmental functions, such as homeland security. As a result, when starting these new systems, space program managers can expect to be inundated with competing demands--not just among military users--but also among civilian and industry users. Our prior reports have identified related pressures with all weapon systems. More than 30 organizations within the requirements community may have a hand in determining a weapon system's performance requirements before a contractor with systems engineering expertise can identify the gaps between the requirements and available resources. This process means the "doability" of the requirements is often not known with certainty until well into product development or until a significant percentage of funds planned to develop the system has been invested. By this point in time, customers' expectations have been set, making it difficult to change requirements if gaps between requirements and available resources are found. In the past, DOD has not implemented effective mechanisms to help mitigate these pressures. In fact, as DOD's own studies have shown that these pressures were exacerbated when DOD pursued its Total System Performance Responsibility approach because it turned over responsibilities related to requirements definition to contractors who had less understanding and ability to negotiate requirements, leaving program managers in the position of having to continually address requirements growth without additional resources. Moreover, the Young Panel observed that space program managers have not had the authority needed to make trade-offs between requirements and control requirements growth. The panel recommended giving program managers this authority, accompanied by greater accountability for requirements. Developing Technologies within the S&T Environment versus the Acquisition Environment: An important problem cited in our reports about space system acquisition programs is the tendency to take on technology development that should occur within the S&T environment. Our reports have stressed that the S&T environment is more forgiving and less costly than the acquisition environment, which is focused on delivery. This is because events such as test failures, new discoveries, and time spent in attaining knowledge are considered normal in the S&T environment rather than negative. Further, when acquisition programs take on technology development, estimates for cost, schedule, and performance are formally approved without the benefit of knowing that technologies will work as intended. Officials we spoke with for this review and previous reviews cited a number of reasons that program managers and senior leaders choose to have acquisition programs take on technology development activities that should occur within the S&T environment. * The lengthy development period required for space systems puts pressure on program managers to continually develop technologies. There is a fear that if these technologies do not reach maturity during this time frame, they will be outdated by the time the satellites are ready to be launched. * Once a program has formally begun, it is easier to secure current and future years' funding. * Satellites tend to last longer than expected, and they cannot be retrieved for upgrades, putting more pressure on programs to push for attaining as much technological capability as possible within the acquisition program. * The acquisition community does not believe that labs in charge of developing space technologies adequately understand its needs--in terms of capabilities and time frames--and would rather pursue its own goals. * Program managers also believe that they would have more control over technology development if it was conducted by contractors who answered to them rather than to DOD labs. * DOD has not had an effective strategy for steering activities within the S&T community to ensure that they will eventually fit in with acquisition needs. (Note: DOD has recently developed a space S&T strategy. We reported on this effort in January 2005.) Our previous reports have found that many of these views tend to work against, rather than for, DOD's ability to achieve timely technology advances. When acquisition programs seek to translate advanced concepts into reality, they invariably run into problems that require time and money to fix. The effects of these problems are often revealed in a later stage of development, where they have reverberating effects on other aspects of the acquisition program and often require reworking design. For example, early technological problems as well as more recent system integration issues have severely affected the SBIRS High program, among others. We also previously found that DOD's new space acquisition policy increased acquisition risks by allowing programs to begin without having technologies demonstrated in an operational or simulated environment or even begun with technologies in even lower stages of maturity. Many officials believed this policy was necessary because of the unique aspects of space acquisition programs, that is, their long length, their complexity, and the high cost of operational testing associated with space systems. As we have reported in the past, however, DOD has found ways to test sensors and other critical technologies on experimental satellites in the past, and it has built and launched technology demonstrator satellites before starting acquisition programs. Moreover, as noted earlier, the length of space and other weapon system development can be reduced by pursuing evolutionary development. This approach does not prevent DOD from concurrently seeking technological advances, but such activity should occur outside an acquisition program, rather than inside, to minimize disruptions. Last, officials within the Office of the Secretary of Defense did not believe that space programs warranted a separate approach than other weapon systems. They noted that ships, for example, have unique aspects but still fall under the same acquisition policy as other weapon systems, which encourages programs to test technologies in an operational environment before starting. They also noted that having allowed space programs to follow a separate acquisition process has effectively reduced direct oversight from the Office of the Secretary of Defense. DOD has recently revised its space acquisition policy, in part to encourage programs to attain more knowledge about technologies before starting. It has also taken steps to strengthen its commitment to fully fund space programs. However, the revised policy still allows space acquisition programs to begin before demonstrating technologies in an operational or simulated environment. Short Tenures and Workforce Deficiencies May Disrupt Programs as Well as DOD's Overall Ability to Implement Reform: Short tenures for top leadership and program managers within the Air Force and the Office of the Secretary of Defense have lessened the sense of accountability for acquisition problems and further encouraged a short-term view of success, according to officials we interviewed. Turnover makes it difficult for upper-level managers to establish effective working relationships with program managers, resulting in less trust when divulging problems. These concerns have been echoed in prior GAO and DOD reports. DOD has taken action over the past decade to lengthen the tenure of program managers. However, the Young Panel reported that the average tenure of a space program manager is just 2 years and recommended that tenures be extended to a minimum of 4 years to minimize disruption to programs. Even with this extension, it is likely that programs, which typically last longer than 4 years, will continue to experience turnover in program management. Many current programs, including SBIRS High, the Space Tracking and Surveillance System, Global Positioning System II modernization effort, and AEHF, were started in the mid-to late 1990s. We have also reported in the past that the short tenures typical of program managers make it difficult for them to change the system of incentives because other participants can wait out reforms they oppose. Moreover, DOD acquisition executives do not necessarily stay in their positions long enough to develop the needed long-term perspective or to effectively change traditional incentives. Officials we spoke with frequently cited other workforce-related deficiencies that put pressure on program managers and acquisition executives. For example, there are not enough experienced program managers to run space programs and or enough experts in software engineering--a consequence of starting more programs than DOD can afford and effectively manage. Earlier policies of having industry assume more responsibility also contributed to this dearth of expertise within DOD. As a result, DOD has increasingly relied on outside experts to help manage programs. At the same time, a limited number of these outside experts are available to provide technical support to DOD's various space programs. Industry-Related Pressures: Officials we spoke with pointed out a number of pressures associated with contractors who develop space systems for the government--mostly having to do with the level of oversight and insight program managers have with their contractors as well as pressures among contractors to produce low-cost estimates while bidding on contracts. Specific concerns mentioned include the following. * Nonincumbent contractors are often able to submit a lower price than the incumbent because they can be optimistic without being challenged by DOD. These optimistic estimates enable them to win new contracts. At the same time, however, nonincumbents are not necessarily the best organizations to carry out the development program, particularly because they do not have the technical and management experience associated with the legacy system being replaced. * Industry has been consolidated to a point where there may be only one company that can develop a needed component for a satellite system. This has enabled contractors to hold some programs hostage. * Program managers are often not equipped to understand what is behind a contractor's proposal, particularly because contractors are not likely to disclose technical risks and highlight other negative aspects. * Industry puts pressure on programs to have contractors develop critical technologies within an acquisition environment versus having the labs do it. When labs build technologies, the government allows the contractors that work on the system that would ultimately use the technologies to scrap them in favor of employing their own methods and expertise. * Program managers are not always experienced enough to stand up to contractors when development is being mismanaged. Program managers also may not understand the best ways to incentivize contractors and gain insight into their performance. * Contractors are facing workforce pressures similar to those experienced by the government, that is, not enough technical expertise to develop highly complex space systems. (Our recent report on space S&T echoed this concern as well, pointing out that several studies have found that both industry and the U.S. government face substantial shortages of scientists and engineers and that recruitment of new personnel is difficult because the space industry is one of many sectors competing for the limited number of trained scientists and engineers.) * Some space programs are facing pressures related to funding and technology development because of an expectation widely held in the 1990s that the commercial space market would experience a boom. At the time, DOD decreased funding for some capabilities, principally space launch, assuming the market could pay for a portion of research and development and that economies of scale would result. It also relied on the commercial sector to develop knowledge about production of satellites that eventually were purchased as part of the Wideband Gapfiller Satellite program. However, when anticipated commercial orders using the same technologies did not pan out, the government experienced unanticipated schedule delays. Conclusion: By delving into the underlying incentives and pressures that cause space system acquisitions to go awry, DOD will be better equipped to take the steps needed to attain successful outcomes. The comments of the 40-plus experts interviewed for this report reinforce our past findings that those steps should include the development of an overall investment strategy that prioritizes funding and the establishment of a sound business case before starting an acquisition program. Agency Comments and Our Evaluation: In written comments on our draft report (see enc. I), DOD presented its views in two primary areas. First, in responding to our conclusion that problems in space acquisitions are largely reflective of a lack of an investment strategy for space programs, DOD commented that it is implementing a new requirements process--known as the Joint Capabilities Integration and Development System (JCIDS)--designed to ensure that each new military program is aligned with current and future joint needs. We have acknowledged in our past work that the implementation of JCIDS is a positive step toward realizing a DOD-wide investment strategy for weapon systems. In using this new process to achieve better outcomes, DOD will need to systematically prioritize its weapon system programs against funding plans and consistently perform rigorous analysis of alternatives that weigh the costs and benefits of achieving each desired capability via a space platform versus an air, land, or sea platform. Until DOD's strategic weapon system plans for the future are better linked to DOD's budget, these programs will continue to experience funding shortfalls, the shifting of funding from program to program, and accompanying schedule delays. Second, DOD commented on our conclusion that problems in space acquisitions are also a result of DOD's tendency to begin these programs before establishing a sound business case. DOD pointed out that criteria in its new space acquisition policy, which was updated on December 27, 2004, are designed to ensure a program's readiness to proceed into the development phase (or "program start"). We recognize that the new policy should increase knowledge about space programs before investment decisions are made but remain concerned that DOD will start acquisition programs and commit to cost, schedule, and performance baselines before it has established a sound business case- -which we have found to be a match between requirements and resources (time, money, and mature technologies). DOD also provided a set of comments that it termed "corrections to errors in fact." In enclosure II, we respond to this set of comments; changes made to the draft report and areas of disagreement between us and DOD are highlighted. We are sending copies of this report to the Secretaries of Defense and the Air Force and interested congressional committees. We will make copies available to others on request. In addition, the report will be available on the GAO Web site at http://www.gao.gov. If you or your staff have any questions concerning this report, please contact me at (202) 512-4841. Other staff making key contributions to this report include Cristina Chaplain, Maricela Cherveny, Lily Chin, Art Gallegos, Jean Harker, John Krump, and Nancy Rothlisberger. Sincerely yours, Signed by: Robert E. Levin, Director: Acquisition and Sourcing Management: [End of section] Enclosure I: Comments from the Department of Defense: OFFICE OF THE ASSISTANT SECRETARY OF DEFENSE: 6000 DEFENSE PENTAGON: NETWORKS AND INFORMATION INTEGRATION: WASHINGTON, DC 20301-5000: JUN 6 2005: Mr. Robert E. Levin: Director, Acquisition and Sourcing Management: U.S. General Accounting Office: Washington, D.C. 20548: Dear Mr. Levin: This is the Department of Defense (DoD) response to the GAO draft report, "DEFENSE ACQUISITIONS: Incentives and Pressures That Drive Problems Affecting Satellite and Related Acquisitions," dated May 2, 2005 (GAO Code 120402/GAO-05-570R). The Department of Defense acknowledges receipt of the draft report and does not concur with all the GAO's issues. Our formal comments are attached. We also attempted to correct several factual errors and provided some routine administrative corrections. Please note that the Under Secretary of the Air Force updated the National Security Space Acquisition Policy 03-01 on December 27, 2004. This revised version contains additional guidance in some of the areas which the GAO expressed concern. Until such time as programs have the opportunity to progress under the revised policy, we will not begin to see evidence of change. We welcome the opportunity to work with the GAO to ensure the final report reflects a clear understanding of those aspects of the space acquisition process over which the Department has control. I recommend that the GAO reassess the impact of the revised National Security Space Acquisition Policy 03-01 following a year of implementation. Signed for: Linton Wells II: Principal Deputy: Enclosure: As stated: GAO DRAFT REPORT DATED MAY 2, 2005 GAO-05-570R (GAO CODE 120402): "DEFENSE ACQUISITIONS: INCENTIVES AND PRESSURES THAT DRIVE PROBLEMS AFFECTING SATELLITE AND RELATED ACQUISITIONS": DEPARTMENT OF DEFENSE COMMENTS TO THE GAO CONCERNS: CONCERN 1: The GAO found that space acquisition problems are largely reflective of a lack of an overall investment strategy (p. 3/GAO Draft Report). DOD COMMENT: The basis behind a sound investment strategy includes establishing a sound requirements identification and prioritization process, determining the architectural context of the investment, and deciding on the amount of risk that will be accepted within resource constraints. The Department's recently developed Joint Capabilities Integrations and Development System (JCIDS) represents a major step toward outlining the necessary discipline in the requirements process. In addition, the Milestone Decision Authority (MDA) has required architectural context determinations be made prior to all Defense Space Acquisition Board Reviews held since inception of the NSS Acquisition Policy 03-01. These are completed as part of the Information Support Plan and integrated architectural products development, and they outline the relevance of the space system being proposed in light of crucial interfaces with or impacts upon supporting/supported systems, external organizations, and missions. After the requirements are validated through the JCIDS process and the relevance of the new system in the overall architecture is established, the MDA determines if a new system is feasible and affordable - with an appropriate focus on mission success. Prior to starting a new space program, the MDA determines the amount of acceptable program risk given the urgency of need and potentially available resources. When followed with rigor and discipline, this end-to-end process provides a sizable improvement in the refinement of our existing space investment strategy. Finally, the acquisition process requires constant coordination with the Planning, Programming, Budgeting and Execution process and consideration of other competing portfolios, in addition to the space portfolio, on a program- by-program basis. We continue to look for areas of improvement in all aspects of our current space investment strategy while allowing the process to mature. CONCERN 2: The GAO found that space acquisition problems are largely reflective of a tendency to set start dates for programs before a sound business case for them has been established (p. 3/GAO Draft Report). DOD COMMENT: National Security Space (NSS) Acquisition Policy 03-01 identifies Key Decision Point (KDP)-B as the official "Program Initiation" point for an NSS program. In terns of establishing a solid business case, KDP-B is the point by which a funding baseline must be established. The purpose of KDP-B is to determine the program's readiness to begin the preliminary design development activities of Phase B. It is designed to increase confidence in the selected NSS system altemative(s) by assessing the estimated risk levels and projected performance envelope at a detailed engineering level. Where feasible, critical technology should complete testing in a relevant environment during Phase B. Technology that has not been tested in a relevant environment should be deferred to the next increment. Key acquisition documentation required for review during the KDP-B Independent Program Assessment include: initial integrated architecture from the system program office; updated system level CONOPS from concept sponsor; Milestone Decision Authority (MDA)-approved Acquisition Strategy; JROC- approved updated Capability Development Document; Director, OT&E approved Test and Evaluation Master Plan; Integrated Program Summary; draft Acquisition Decision Memorandum; and draft Acquisition Program Baseline. Based on a thorough assessment of a program's maturity as outlined above, the DoD Space MDA decides on readiness to proceed to the next acquisition phase and consequently, "official" program start. CORRECTIONS TO ERRORS IN FACT: 1. Page 2, first bullet. Report states, "'This system [SBIRS] is intended to replace an older generation of missile-warning satellites...." This phrasing fails to acknowledge that in addition to serving as a replacement for DSP, SBIRS will also provide critical, new capabilities for current and future threats. SBIRS will significantly improve upon DSP . with JROC-validated capabilities that are essential for countering today's and tomorrow's threats. It is extremely important to understand this distinction between DSP and SBIRS, especially given the drive to "cut back on [SBIRS] capabilities" as mentioned earlier in the document. 2. Page 2, same bullet. Report goes on to state that delays in the Space-Based Infrared System (SBIRS) High Program are "forcing DoD to operate the older generation of satellites much longer than expected." The DoD is operating the Defense Support Program satellites longer than expected because they are fortuitously living longer than expected. 3. Page 10, second full paragraph. Report implies that unlike the rest of the DoD, the space community has not adopted the evolutionary development approach--that is, pursuing incremental increases in capability versus significant leaps. NSS 03-01 states that Evolutionary Acquisition (EA) is the preferred strategy for rapid acquisition of mature technology for the user. The System Program Director/ Program Manager should describe the program's EA strategy in the program's Acquisition Strategy. The two main processes to perform EA are: a) Spiral Development. In this process, a desired capability is identified, but the end-state requirements are not known at program initiation. Those requirements are refined through demonstration and risk management, there is continuous user feedback, and each increment provides the user the best possible capability. The requirements for future increments depend on feedback from users and technology maturation. b) Incremental Development. In this process, a desired capability is identified, an end-state requirement is known, and that requirement is met over time by development of several increments, each dependent on available mature technology. 4. Page 14, first paragraph, last sentence states: "In fact, problems related to technology discovery have severely impacted the SBIRS High program, among others." Technology discovery has not impacted SBIRS development. The 2002 Independent Review Team assessed the SBIRS space segment as technologically mature. The issues identified by the IRT involving immaturity to enter development were not technology issues. To quote the IRT report: "In general, faulty and overly optimistic assumptions laid the ground work for SBIRS High program activation in 1996. These included: extensive software reuse; high software productivity levels; commercial practice benefits; economic order efficiencies with satellite lot buys; availability of technical models; mature understanding of requirements, Concept of Operations (CONOPS), and interface specifications; and management stability." The same IRT report also noted that SBIRS was following a phased implementation approach to address operational requirements. 5. Page 14, second paragraph states: "We also previously found that DoD's new space acquisition policy increased acquisition risks by allowing programs to begin without having technologies proven in an operational or simulated environment or even begun with technologies in even lower stages of maturity." KDP-B is the official "Program Initiation" point for an NSS program as stated in the response to Finding 2. Design work is best tested when a technical baseline has been established. We are committed to testing technology in a relevant environment no later than KDP-C. Because the design baseline is not finalized until after Phase B, there is no technical baseline to be "proven in an operational or simulated environment" before "allowing programs to begin," that is, by KDP-B. Regarding maturity, the program office is required to conduct Technology Readiness Assessments during Phases A and B. In addition, for KDP-B and KDP-C, the Component Science and Technology Executive will conduct an independent review of the program office generated Technology Readiness Assessment. Finally, the Integrated Program Summary's detailed Risk Management section addresses technical maturity. At each KDP and Build Approval, the program office is expected to identify the key technology components of the system and provide an assessment of the maturity of each key component using the Technology Readiness Level (TRL) method identified in the DoD Acquisition Guidebook. The Independent Program Assessment Team (IPAT) reviews the program office assessment and determines if, in their view, all key technology components of the program have been identified. The ]PAT also provides its own independent assessment of the maturity of the key components using the TRL method. The intent is to make the DoD Space MDA knowledgeable of the state of key component maturity so appropriate direction can be given in the Acquisition Decision Memorandum for additional technology maturation/risk reduction activities. 6. Page 14, last paragraph cites several unique aspects of space acquisition programs: long length, complexity, and high cost of operational testing. A more commonly quoted set of criteria is: low quantities produced, continual development during the acquisition cycle, long operational life, infeasibility of conducting operational testing, and inability to repair once on orbit. 7. Page 14, last paragraph states that officials within the Office of the Secretary of Defense "noted that allowing space programs to follow a separate acquisition process has effectively blocked-out oversight from the Office of the Secretary o^ Defense." NSS 03-01 is a policy of inclusiveness. In addition to the Vice Chairman of the Joint Chiefs of Staff co-chairing the Defense Space Acquisition Board with the MDA, multiple OSD-level advisors and representatives serve as DSAB principals and, as such, advise the MDA. The OSD Cost Analysis Improvement Group is also responsible for developing independent cost analyses of DoD space MDAPs in support of the DoD space MDA's DSAB process. The role of OSD in oversight of defense space programs with respect to the Planning, Programming, Budgeting and Execution System remains unchanged. Each Service regularly submits Selective Acquisition Reports, Unit Cost Reports, and Defense Acquisition Executive Summary reports to the OSD staff. OSD is also required to coordinate on key acquisition documentation, such as the Acquisition Strategy, Acquisition Decision Memorandum, and Acquisition Program Baseline, Test and Evaluation Master Plan, and Integrated Program Summary. The March 1, 2003 OSD Report to Congress on Defense Space Acquisition Programs provides further detail on this topic. 8. Page 22, Enclosure III, Officials Interviewed for This Review, lists the Secretary of the Air Force's Space Plans and Policy Division as being interviewed. The correct organization is the Office of the Under Secretary of the Air Force's Directorate of Space Acquisition. [End of section] Enclosure II: Department of Defense Comments and GAO's Responses: The Department of Defense (DOD) stated that we failed to acknowledge that the SBIRS High system, as currently planned, would serve to replace the Defense Support Program and improve on the capabilities it provides. GAO's Response: We have changed the text in this report to reflect that SBIRS High is intended to also upgrade existing capabilities. 5. DOD stated that it is operating its Defense Support Program satellites longer than expected because they are "fortuitously living longer than expected." GAO's Response: Although DOD's original plan was to begin launching SBIRS High satellites in 2002 regardless of the health or longevity of the Defense Support Program satellites, we revised this point to concentrate on the acquisition of SBIRS High. DOD stated that our draft report implies that unlike the rest of DOD, the space community has not adopted the evolutionary development approach--that is, pursuing incremental increases in capability versus significant leaps. GAO's Response: While the space acquisition policy has a section on Evolutionary Acquisition, the Air Force continues to pursue significant leaps in technology within its acquisition programs. For example, DOD plans to migrate from the Advance Extremely High Frequency (AEHF) satellites with radio frequency crosslinks capable of transmitting data at 60 megabits per second to the Transformational Satellite Communication System (TSAT) with laser crosslinks capable of supporting 20,000 megabits per second. DOD commented that technology discovery has not affected the development of SBIRS High. GAO's Response: Although we agree that currently the major problems on SBIRS High are related to system integration, earlier in the program, there were technical development problems related to the sensors and satellites. We revised the text to show that system integration issues have also affected the development of SBIRS High. DOD stated that it is committed to testing technology in a relevant environment no later than key decision point C. DOD added that regarding maturity, the program office is required to conduct technology readiness assessments. Finally, DOD stated that the integrated program summary's detailed risk management section addresses technical maturity. GAO's Response: We note that the recently revised space acquisition policy added processes to assess the maturity of critical technologies and provided that "where feasible, critical technology should complete testing in a relevant environment during Phase B." The policy also states that technology that has not been tested in a relevant environment should be moved to the next increment. However, the policy still allows programs to continue to mature technology while they are designing the system and undertaking other product development activities during Phase B. Our work on best practices shows that successful acquisition programs do not start product development unless a match between requirements and the resources (time, technology, and money) can be made, and technologies should be matured in an environment that is focused on technology development. Under the DOD space acquisition policy, programs are allowed to enter Phase B with technologies that are immature. Regarding our citing some unique aspects of space acquisition programs, DOD mentioned low quantities produced, continual development during the acquisition cycle, long operational life, infeasibility of conducting operational testing, and inability to repair once on orbit. GAO's Response: The three unique aspects of space acquisition programs cited in our report were those given by interviewed officials. Concerning the DOD-mentioned criteria, our work has found that technology development is best conducted before product development rather than continually throughout the acquisition cycle. Also, low quantities produced and long operational life are aspects shared by many non-space weapon systems. In addition, we have reported that although operational testing is expensive, it is feasible. DOD took issue with a comment that the separate space acquisition process has effectively blocked out oversight from the Office of the Secretary of Defense (OSD). DOD clarified that its space policy is one of inclusiveness and identified the various organizational participants in the space acquisition process. GAO's Response: We note that DOD is taking issue with comments we obtained from officials within OSD. Because this information is opinion and is qualified as such in this report, there are no errors to correct. The space acquisition policy does provide for OSD stakeholders on the Defense Space Acquisition Board. However, it is the Under Secretary of the Air Force who has milestone decision authority over space programs, and not the Under Secretary of Defense for Acquisition, Technology, and Logistics, who has milestone decision authority for other weapon systems. We revised our report to state that the separate space acquisition process has effectively reduced rather than blocked direct OSD oversight. DOD commented that we had incorrectly identified the Secretary of the Air Force's Space Plans and Policy Division as being interviewed. GAO's Response: We changed that item in enclosure IV to read the "Directorate of Space Acquisition." [End of section] Enclosure III: Related Reports: DOD: Report of the Defense Science Board/Air Force Scientific Advisory Board Joint Task Force on Acquisition of National Security Space Programs, May 2003 (also referred to as the "Young Panel report") and the July 2004 update to this report. Space-Based Infrared System Independent Review Team, Final Report, February 2002. GAO: Defense Acquisitions: Assessments of Selected Major Weapon Programs, GAO-05-301 (Washington, D.C.: March 31, 2005). Technology Development: New DOD Space Science and Technology Strategy Provides Basis for Optimizing Investments but Future Versions Need to Be More Robust, GAO-05-155 (Washington, D.C.: Jan. 28, 2005). Defense Acquisitions: Space-Based Radar Effort Needs Additional Knowledge before Starting Development, GAO-04-759 (Washington, D.C.: July 23, 2004). Defense Acquisitions: Risks Posed by DOD's New Space Systems Acquisition Policy, GAO-04-379R (Washington, D.C.: Jan. 29, 2004). Space Acquisitions: Committing Prematurely to the Transformational Satellite Program Elevates Risks for Poor Cost, Schedule, and Performance Outcomes, GAO-04-71R (Washington, D.C.: Dec. 4, 2003). Defense Acquisitions: Improvements Needed in Space Systems Acquisition Policy to Optimize Growing Investment in Space, GAO-04-253T (Washington, D.C.: Nov. 18, 2003): Defense Acquisitions: Despite Restructuring, SBIRS High Program Remains at Risk of Cost and Schedule Overruns, GAO-04-48 (Washington, D.C.: Oct. 31, 2003). Defense Acquisitions: Improvements Needed in Space Systems Acquisition Management Policy, GAO-03-1073 (Washington, D.C.: Sept. 15, 2003). Military Space Operations: Common Problems and Their Effects on Satellite and Related Acquisitions, GAO-03-825R (Washington, D.C.: June 2, 2003). Military Space Operations: Planning, Funding, and Acquisition Challenges Facing Efforts to Strengthen Space Control, GAO-02-738 (Washington, D.C.: Sept. 23, 2002). Best Practices: Capturing Design and Manufacturing Knowledge Early Improves Acquisition Outcomes, GAO-02-701 (Washington, D.C.: July 15, 2002). Defense Acquisitions: DOD Faces Challenges in Implementing Best Practices, GAO-02-469T (Washington, D.C.: Feb. 27, 2002). Best Practices: Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes, GAO-01-288 (Washington, D.C.: March 8, 2001). Defense Acquisitions: Employing Best Practices Can Shape Better Weapon System Decisions, GAO/T-NSIAD-00-137 (Washington, D.C.: April 26, 2000). Best Practices: Better Management of Technology Development Can Improve Weapon System Outcomes, GAO/NSIAD-99-162 (Washington, D.C.: July 30, 1999). Best Practices: Successful Application to Weapon Acquisitions Requires Changes in DOD's Environment, GAO/NSIAD-98-56 (Washington, D.C.: Feb. 24, 1998). Other: Booz, Allen, Hamilton, Space Systems Development Growth Analysis, McLean, Va., October 2002. [End of section] Enclosure IV: Officials Interviewed for This Review: Office of the Secretary of Defense: * Office of the Director, Program Analysis and Evaluation and Cost Analysis Improvement Group: * Office of Force Transformation: * Under Secretary of Defense for Acquisition, Technology, and Logistics: * Assistant Secretary of Defense for Networks and Information Integration: Secretary of the Air Force: * National Security Space Office: * Office of Science, Technology, and Engineering: * Directorate of Space Acquisition: Air Force Space Command: * Directorate of Requirements: * Systems Engineering and Integration Office: * Space and Missile Systems Center: * Program Executive Office: * Advanced Extremely High Frequency Program: * Development and Transformation Directorate: * Evolved Expendable Launch Vehicle Program: * NAVSTAR GPS Joint Program: * Space-Based Infrared Systems Program: * Space Radar Program: Missile Defense Agency: * Space Tracking and Surveillance System Program: Naval Research Laboratory: Various industry and former high-ranking Department of Defense officials: [End of section] FOOTNOTES  GAO, Military Space Operations: Common Problems and Their Effects on Satellite and Related Acquisitions, GAO-03-825R (Washington, D.C.: June 2, 2003).