This is the accessible text file for GAO report number GAO-11-640 entitled 'Defense Acquisitions: DOD Can Improve Its Management of Configuration Steering Boards' which was released on July 7, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Committee on Armed Services, U.S. Senate: July 2011: Defense Acquisitions: DOD Can Improve Its Management of Configuration Steering Boards: GAO-11-640: GAO Highlights: Highlights of GAO-11-640, a report to the Committee on Armed Services, U.S. Senate. Why GAO Did This Study: GAO has previously reported that requirements changes are factors in poor cost and schedule outcomes on Department of Defense (DOD) weapon programs. In 2007, DOD introduced Configuration Steering Boards (CSBs) to review requirement and configuration changes that could adversely affect programs. In 2008, Congress made annual CSB meetings a requirement for all of the military departments’ major defense acquisition programs. In response to the Senate report accompanying the bill for the Ike Skelton National Defense Authorization Act for Fiscal Year 2011, GAO assessed (1) the extent to which DOD has complied with the statutory requirements for CSBs, and (2) the extent to which CSBs have been effective in controlling requirements and mitigating cost and schedule risks. To conduct this work, GAO surveyed DOD’s major defense acquisition programs, reviewed CSB documentation, and interviewed relevant military service and program officials. What GAO Found: The military departments varied in their compliance with the CSB requirements in statute. The Air Force and Navy did not fully comply with the requirement to hold annual CSB meetings for all major defense acquisition programs in 2010, while the Army did. In total, the military departments held an annual CSB meeting for 74 of 96 major defense acquisition programs they managed in 2010. According to GAO’s survey results, when the military departments held CSB meetings, 19 programs endorsed requirements or configuration changes. In most of these cases, strategies were developed to mitigate the effects of these changes-—a key provision in the statute and DOD policy. However, key acquisition and requirements personnel were often absent from Air Force and Navy CSB meetings when these issues were discussed. Two major defense acquisition programs-—the Ballistic Missile Defense System and the Chemical Demilitarization-Assembled Chemical Weapons Alternatives programs—-are not subject to the CSB provisions in statute because the statute only applies to programs overseen by military departments; the programs are managed by other DOD components. These programs are subject to DOD’s CSB policy, which differs from the statute in that it only requires major defense acquisition programs that are in development to hold annual CSB reviews. Individual programs varied in the extent to which they utilized CSBs to control requirements and mitigate cost and schedule risks. According to GAO’s survey results, the majority of CSB meetings neither reviewed requirement changes nor discussed options to moderate requirements or reduce the scope of programs. There were a number of specific instances where CSB meetings were effective in mitigating the effect of necessary changes, rejecting other changes, facilitating discussion of requirements, and endorsing “descoping” options with the potential to improve or preserve cost or schedule. However, in response to a survey, program officials cast some doubts about the effectiveness of CSBs, and in interviews, acquisition officials indicated that program managers may be reluctant to recommend descoping options due to cultural biases that encourage meeting warfighters’ stated needs rather than achieving cost savings, a preference not to elevate decisions to higher levels of review, and concerns that future funding may be cut if potential savings are identified. In response, the Army and Air Force have issued additional descoping guidance and set savings or budget targets. The types of discussions for which CSBs were useful changed based on whether programs were in development or production. Development programs found them more useful to consider requirements changes and descoping options, and production programs found CSBs more useful to prevent changes. In an effort to further increase effectiveness and efficiency of CSBs, some of the military departments have taken steps to coordinate CSB meetings among programs that provide similar capabilities and align CSB meetings with other significant reviews. What GAO Recommends: Among GAO’s recommendations for DOD components are that they amend their CSB policies to be consistent with statute and align CSBs with other reviews when possible. In comments on a draft of this report, DOD concurred or partially concurred with all seven of GAO’s recommendations and agreed to take action to address six of them. View [hyperlink, http://www.gao.gov/products/GAO-11-640] or key components. For more information, contact Michael J. Sullivan at (202) 512-4841 or sullivanm@gao.gov. [End of section] Contents: Letter: Background: Compliance with the CSB Provisions in Statute Varied by Military Department: CSB Meetings Had Some Positive Effects on Programs' Efforts to Control Requirements and Costs: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: Comments from the Department of Defense: Appendix III: GAO Contact and Acknowledgments: Tables: Table 1: Configuration Steering Board Meetings Held for Major Defense Acquisition Programs by Military Department in 2010: Table 2: Explanations Provided by Military Departments for Not Holding 2010 Configuration Steering Boards: Table 3: Descoping Options Endorsed at CSB Meetings: Table 4: Program Officials' Opinions on the Utility of CSB Meetings by Acquisition Phase: Abbreviations: ASD (NCB): Assistant Secretary of Defense for Nuclear, Chemical, and Biological Programs: BMDS: Ballistic Missile Defense System: CSB: Configuration Steering Board: DOD: Department of Defense: MDA: Missile Defense Agency: OSD: Office of the Secretary of Defense: SAR: Selected Acquisition Report: USD (AT&L): Undersecretary of Defense for Acquisition, Technology and Logistics: [End of section] United States Government Accountability Office: Washington, DC 20548: July 7, 2011: The Honorable Carl Levin: Chairman: The Honorable John McCain: Ranking Member: Committee on Armed Services: United States Senate: The Department of Defense's (DOD) major defense acquisition programs have historically cost more and taken longer to field capabilities to the warfighter than initially planned.[Footnote 1] The total acquisition cost of DOD's portfolio of major programs has increased by $135 billion since 2008, and the average delay in delivering initial capability is now 22 months.[Footnote 2] We have previously reported that requirements changes and the inability of program managers to defer requirements that could not be completed under existing cost and schedule targets are factors in poor acquisition program outcomes. [Footnote 3] To address this issue, the Under Secretary of Defense (USD) for Acquisition, Technology and Logistics (AT&L) introduced Configuration Steering Boards (CSBs) in 2007 to review requirement and configuration changes that could adversely affect cost and schedule for major programs in development. Congress has also identified the CSB as a way to enable this process. In the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009, Congress made annual CSB meetings a requirement for all major defense acquisition programs. [Footnote 4] In our 2010 assessment of selected weapons programs, we found that few programs reported holding CSB meetings in 2009[Footnote 5]. In response, the Senate report that accompanied the bill for the Ike Skelton National Defense Authorization Act for Fiscal Year 2011 asked us to review DOD's use of CSBs in fiscal year 201[Footnote 6]0. This report assesses: (1) the extent to which DOD has complied with the statutory requirements for CSBs and (2) the extent to which CSBs have been effective in controlling requirements and mitigating cost and schedule risks. To determine the extent to which DOD complied with the statutory requirement to hold annual CSB meetings, we identified 98 active major defense acquisition programs using the DOD's Defense Acquisition Management Information Retrieval System.[Footnote 7] We defined an active program as one that issued a selected acquisition report in December 2009.[Footnote 8] For each program, we asked the acquisition organization overseeing it--the Army, Navy, Air Force, Missile Defense Agency, and the Assistant Secretary of Defense for Nuclear, Chemical, and Biological Defense Programs--to provide the minutes and lists of attendees from the CSB meetings held in calendar year 2010. To determine the extent to which CSBs have been effective in controlling requirements and mitigating cost and schedule risks, we surveyed all 98 program offices to gather information on their programs, on the CSB meetings their programs held in fiscal year 2010, and on the utility of these meetings. We collected fiscal-year data in our survey because the Senate report language that contained our mandate focused on fiscal-year 2010. All 98 programs completed the survey. We also conducted interviews with 17 programs to collect more information about how requirements changes were reviewed, how they mitigated cost and schedule changes, and what made CSB meetings effective or ineffective. We selected these programs based on the types of activities that took place during their CSB meetings as reported in their survey responses and minutes. Appendix I contains more information regarding our scope and methodology. We conducted this performance audit from September 2010 to July 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: In July 2007, the USD (AT&L) established CSBs for every current and future major defense acquisition program in development as a measure to limit requirements change and avoid cost increases. The CSBs were to have a broad membership, including senior representatives from the offices of USD (AT&L) and Joint Staff. CSBs were intended to review all requirements and significant technical configuration changes with the potential to adversely affect the program.[Footnote 9] The USD (AT&L) directed that these changes should generally be rejected or deferred unless funds and schedule adjustments could be identified to mitigate their effects. In addition, program managers were asked to identify options to reduce program cost or moderate requirements, referred to as "descoping" options, on a roughly annual basis. USD (AT&L) also instructed that, while policy would be to keep within planned costs as much as possible even at the expense of scope and content, all expected increases in program costs must be budgeted at the absolute earliest opportunity. USD (AT&L) incorporated CSBs into DOD's primary acquisition policy--DOD Instruction 5000.02--in December 2008. In October 2008, Congress enacted the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009, which required the establishment of CSBs for the major defense acquisition programs of the military departments.[Footnote 10] According to the statute, a CSB must meet at least once each year for each of these programs.[Footnote 11] The statute also provided direction on CSB membership and responsibilities. It requires CSBs to: * include the appropriate service acquisition executive as chair and include representatives from USD (AT&L), the Chief of Staff for the armed forces, representatives from other armed forces as appropriate, the Joint Staff, the comptroller of the military department, the military deputy to the service acquisition executive, the program executive officer for the program concerned, and others as appropriate; * prevent unnecessary changes to programs that could have an adverse impact on program cost or schedule, mitigate adverse cost and schedule effects of changes that may be required, and ensure that each program delivers as much planned capability as possible at or below the planned cost and schedule; * review and approve or disapprove any proposed changes to program requirements or system configuration with the potential to adversely affect cost and schedule; and: * review and recommend proposals that could reduce requirements and improve cost and schedule. In addition, the statute provided program managers the authority to: * object to adding new requirements that would be inconsistent with previously established parameters unless approved by the CSB and: * propose opportunities to reduce program requirements to improve cost and schedule consistent with program objectives. In our March 2010 assessment of selected weapon programs, we reported that only 7 of the 42 programs we assessed held CSB meetings in 2009. [Footnote 12] As a result, in the Senate report accompanying the bill for the Ike Skelton National Defense Authorization Act for Fiscal Year 2011, the Senate Armed Services Committee directed USD (AT&L) to take appropriate steps to ensure that CSBs meet at least once a year to consider the full range of proposed changes to program requirements or system configuration for each major defense acquisition program. [Footnote 13] Compliance with the CSB Provisions in Statute Varied by Military Department: The military departments' compliance with statutory CSB requirements varied. The Air Force and Navy did not fully comply with the requirement to hold annual CSB meetings for all major defense acquisition programs in 2010; the Army did comply. In total, the military departments held an annual CSB meeting for 74 of 96 major defense acquisition programs they managed in 2010. According to our survey results, when the military departments held CSB meetings, 19 programs endorsed requirements or configuration changes. In most of these cases, strategies were developed to mitigate any effect on a program's cost and schedule--a key provision in the statute and DOD policy. However, key acquisition and requirements personnel were often absent from Air Force and Navy CSB meetings when these issues were discussed. Two major defense acquisition programs--the Ballistic Missile Defense System (BMDS) and the Chemical Demilitarization- Assembled Chemical Weapons Alternatives programs, which are managed by DOD components rather than military departments--are not subject to the CSB provisions in statute, but rather to DOD policy, because the statute only applies to programs overseen by military departments. This policy differs from the statute in that it only requires major defense acquisition programs in development to hold annual CSB reviews and does not require the same members, including the comptroller of the military department. The Air Force and Navy Did Not Hold CSB Meetings for All Programs, While the Army Did: The Air Force and Navy did not hold CSB meetings for all of their major defense acquisition programs in 2010. The Air Force did not hold CSB meetings for 13 of 31 programs, and the Navy did not hold CSB meetings for 9 of 37 programs. The Army held a CSB meeting for each of its 28 major defense acquisition programs. Of the 96 major defense acquisition programs managed by the military departments, 74 held CSB meetings in 2010 and 22 failed to do so. Table 1 shows how many programs had CSB meetings by military department. Table 1: Configuration Steering Board Meetings Held for Major Defense Acquisition Programs by Military Department in 2010: Military department: Air Force; Programs with CSB meetings: 18; Programs without CSB meetings: 13; Total programs: 31. Military department: Army; Programs with CSB meetings: 28; Programs without CSB meetings: 0; Total programs: 28. Military department: Navy; Programs with CSB meetings: 28; Programs without CSB meetings: 9; Total programs: 37. Military department: Total; Programs with CSB meetings: 74; Programs without CSB meetings: 22; Total programs: 96. Source: GAO analysis of information from DOD components. [End of table] Of the 22 programs that did not have CSB meetings in 2010, 9 programs had meetings in early 2011. In addition, according to the Air Force and Navy, 8 other programs were in the process of being completed or canceled. Table 2 includes explanations from the Air Force and Navy about why CSB meetings were not held for individual programs. Table 2: Explanations Provided by Military Departments for Not Holding 2010 Configuration Steering Boards: Air Force: Advanced Medium Range Air-to-Air Missile (AIM-120); CSB held in December 2009 and January 2011. C-130J Hercules; CSB held in January 2011. C-17A Globemaster III; Exceeded 90 percent of quantities delivered[A]. C-5 Avionics Modernization Program; Exceeded 90 percent of quantities delivered[A]. C-5 Reliability Enhancement and Reengining Program; CSB held in January 2011. Joint Air-to-Surface Standoff Missile/Joint Air-to-Surface Standoff Missile Extended Range; CSB held in December 2009 and January 2011. Joint Cargo Aircraft; Program in process of transferring from the Army. Joint Direct Attack Munition; Exceeded 90 percent of quantities delivered[A]. Joint Primary Aircraft Training System; CSB held in January 2011. Large Aircraft Infrared Countermeasures; CSB held in January 2011. Minuteman III Propulsion Replacement Program; Exceeded 90 percent of quantities delivered[A]. National Airspace System; CSB held in February 2011. National Polar-orbiting Operational Environmental Satellite System; Program in process of cancellation. Navy: Advanced Anti-Radiation Guided Missile (AGM-88E); CSB held in January 2011. EA-6B Improved Capability III; Exceeded 90 percent of quantities delivered[A]. Expeditionary Fighting Vehicle; Program in process of cancellation. Joint High Speed Vessel; CSBs held in October 2009 and March 2011. Littoral Combat Ship; No need for CSB as the configuration is locked. Mobile User Objective System; No need for CSB as requirements are stable. Nimitz Class Carrier (CVN 68); Exceeded 90 percent of quantities delivered[A]. Remote Minehunting System; Other review held. Zumwalt Class Destroyer (DDG 1000); Other reviews held. Source: GAO presentation of information from DOD components. [A] According to DOD, programs with 90 percent of items delivered are no longer covered by the statute as changes to requirements or configuration could no longer occur after a program reaches its inventory objective. [End of table] Most Programs That Made Changes Developed Ways to Mitigate the Cost and Schedule Effects: For each of the military departments, when a CSB meeting reviewed requirements or configuration changes, most were endorsed and strategies to mitigate the effects on a program's cost and schedule were developed and discussed. However, most of the programs we surveyed did not present requirements or configuration changes to be approved or rejected at their fiscal-year-2010 CSB meetings. Specifically, our survey showed the following results: * Air Force: 6 CSB meetings reviewed requirements or configuration changes, 5 of these meetings endorsed changes, and 4 discussed the cost and schedule effects and ways to mitigate them. * Army: 6 CSB meetings reviewed and endorsed requirements or configuration changes, and 4 of these discussed the cost and schedule effects and ways to mitigate them. * Navy: 10 CSB meetings reviewed requirements or configuration changes; 8 meetings endorsed changes, and 7 of these discussed the cost and schedule effects and ways to mitigate them. The Navy did not hold CSB reviews for all programs that experienced requirements changes in fiscal-year 2010. According to our survey results, three Navy programs changed system requirements or specifications yet did not hold a CSB meeting. Two of these programs, the Advanced Anti-Radiation Guided Missile and the Remote Minehunting System, held other high-level reviews during this period--two program management reviews and a critical Nunn-McCurdy breach review,[Footnote 14] respectively--and officials reported that a third program, the Expeditionary Fighting Vehicle, did not conduct its CSB meeting because DOD proposed canceling the program. Key Acquisition and Requirements Personnel Were Absent from Many CSB Meetings: Key acquisition and requirements personnel were absent from many of the CSB meetings held by the Air Force and Navy in 2010. The CSB provision in the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009 lists seven officials or offices that should be part of a CSB, including the service acquisition executive who should serve as the chairperson of the CSB; representatives from the acquisition, requirements, and funding communities; and others as appropriate. Army CSB meetings held in 2010 included the full array of board members in all but one case. Although USD (AT&L) was invited to the meeting in this case, Army officials reported that the office did not send a representative. The medium of CSB board members' participation also varied among the military departments. The Army conducted all its CSB meetings in person, whereas both the Air Force and the Navy conducted virtual, otherwise known as paper,[Footnote 15] CSB meetings for certain programs in 2010 and early 2011. The Air Force held all of its 2010 CSB meetings without key acquisition participants listed in the CSB statute. According to Air Force officials, their CSB meetings may be chaired by either the service acquisition executive or the principal military deputy to provide for flexibility in scheduling meetings.[Footnote 16] Generally, the principal military deputy acts as chair in the place of the service acquisition executive and does not attend those meetings that the service acquisition executive chairs. According to the attendee lists provided by the Air Force, only 2 of the 18 CSB meetings held were attended and chaired by the service acquisition executive. At one of those meetings neither the principal military deputy nor a representative of the comptroller was in attendance although officials report that both had been invited. The CSB meetings the service acquisition executive did not attend included numerous discussions of changes that could affect programs' costs and schedules, including requirements and configuration changes or descoping opportunities. For example, one meeting discussed changes to the Space Based Infrared System's architecture that could accelerate the program's delivery of initial capability by 2 years but would cost an additional $45 million. The Air Force also allows paper CSBs to fulfill the requirement for an annual CSB for programs it believes are stable. A program is eligible to conduct paper CSB meetings if (1) it has a Probability of Program Success score of greater than 80;[Footnote 17] (2) it has made no requirements and/or significant technical configuration changes since the last CSB that have the potential to affect the cost and schedule of the program; (3) when in production, it is in steady state production but has not reached 90 percent of planned expenditures completed or 90 percent of quantities delivered; and (4) descoping options will not yield any real cost savings. The Air Force did not conduct any paper CSBs in 2010; however, 6 of the 13 Air Force programs that did not hold a CSB meeting in 2010 conducted paper reviews in January 2011. According to Air Force officials, the process for these paper reviews began in December 2010. The Navy held most of its 2010 CSB meetings without key acquisition and requirements personnel. The Navy has incorporated CSB meetings into the Navy's gate review process and uses the gate 6 review, with the service acquisition executive or his designee acting as chair, to fulfill the requirement for an annual CSB.[Footnote 18] However, the Navy's policy on gate reviews does not include the Joint Staff--a key player in the requirements process and a participant required by statute and DOD policy--as a participant, and at least 22 of the 28 CSB meetings held in 2010 lacked a representative of the Joint Staff. As a result of our review, Navy officials reported that they are revising their policy and procedures for CSBs to ensure the Joint Staff is invited to future CSB meetings. Navy policy allows the service acquisition executive to delegate the chair to another official within the Navy's acquisition office, which officials stated provides flexibility in scheduling CSBs. In practice, this resulted in meetings where required members of the CSB did not participate in discussions of requirements, configuration, or descoping. In 2010, the Navy service acquisition executive chaired and attended 12 of the 28 CSB meetings and participated in at least 2 others, both CSBs conducted via paper. According to our review of CSB documentation, six CSB meetings clearly discussed descoping options, and the service acquisition executive did not attend any of the five held in person. The sixth meeting was a paper CSB and it is unclear whether the service acquisition executive participated. When the Navy service acquisition executive or others chair the CSB meeting, the principal military deputy typically does not attend. In addition, at least three CSB meetings in 2010 did not include a representative from USD (AT&L). The Navy also allows paper CSBs to fulfill the requirement for an annual CSB. In four cases, the Navy used paper CSBs to review requirement and configuration changes sometimes requiring millions of dollars or tens of millions of dollars in additional funding. According to Navy officials, Navy policy allows CSB members to reach decisions on issues of requirements and configuration by circulating briefing slides and memoranda rather than holding an actual meeting; however, there are not clear criteria specifying the circumstances under which a program may hold a paper CSB. Multiple Navy program managers stated that they do not understand which programs are eligible or when and how to request a paper CSB. In one case, a program manager stated that although the program was planning for and preferred a CSB meeting in person, Navy officials changed the format to a paper CSB a few days before the scheduled meeting time. Two Major Defense Acquisition Programs Are Not Covered By The CSB Statute: Two major defense acquisition programs--the Ballistic Missile Defense System (BMDS) and the Chemical Demilitarization-Assembled Chemical Weapons Alternatives programs, which are managed by DOD components rather than military departments--are not subject to the CSB provisions in statute because the statute only applies to major defense acquisition programs overseen by the military departments. However, DOD acquisition policy, which requires CSBs for all major defense acquisition programs in development, applies to these programs. The Missile Defense Agency (MDA), which is responsible for the management of BMDS, did not hold a CSB for the system in 2010; however, it did conduct reviews that discussed many of the same issues and included some of the same participants as those required for CSBs. The Program Change Board manages the development, fielding, and integration of BMDS through separate program elements and ensures the integrity of the system as a whole. This board, which is the primary forum for discussing and mitigating changes to program elements' requirements and configuration, met 42 times in 2010. The Program Change Board is chaired by the equivalent of a service acquisition executive--the director of MDA--and, according to an MDA official, includes the equivalent of the comptroller, the program executive officer, and the program manager. MDA policy also requires USD (AT&L) to be invited to Program Change Boards, and allows for the military services' participation when deemed appropriate, but does not include the Joint Staff. The Missile Defense Executive Board oversees implementation of strategic plans and reviews the priorities and budget for BMDS as a whole. The Missile Defense Executive Board includes the Joint Staff as well as the MDA director and an array of Office of the Secretary of Defense (OSD) and military service representatives, but according to DOD it does not generally discuss requirements and configuration at the element level. The executive board met seven times in 2010. The Assistant Secretary of Defense for Nuclear, Chemical, and Biological Defense Programs, who is responsible for the management of the Chemical Demilitarization-Assembled Chemical Weapons Alternatives program, also did not hold a CSB in 2010. However, a similar board-- the Chemical Demilitarization Program Strategic Governance Board--met three times in 2010 to discuss program progress, including how it is performing against its requirements and funding issues, including those related to significant cost and schedule growth. In 2010, the Assistant Secretary acted as the chair for this board which also includes representatives from the OSD comptroller, the Joint Staff, and the Army. DOD Policy Is Not Consistent with the CSB Provisions in Statute: The CSB requirements in DOD's primary acquisition instruction are not fully consistent with the provisions in statute. Most significantly, the instruction only requires CSB meetings for major defense acquisition programs in development, rather than major defense acquisition programs in development and production.[Footnote 19] Additionally, the instruction does not include the comptroller as a CSB member. According to USD (AT&L) officials, the CSB provisions in statute may not have been fully incorporated into USD (AT&L)'s December 2008 revision of DOD's acquisition instruction because the statute was enacted in October 2008 and there was not enough time to reconcile them. USD (AT&L) is in the process of updating the instruction and is considering changes to the CSB requirements. USD (AT&L), according to officials, has also not consistently tracked whether programs are fulfilling the current requirements in DOD policy because the statute makes CSBs a military department responsibility. CSB Meetings Had Some Positive Effects on Programs' Efforts to Control Requirements and Costs: Individual programs varied in the extent to which they utilized CSBs to control requirements and mitigate cost and schedule risks. According to our survey results, the majority of CSB meetings neither reviewed requirement changes nor discussed options to reduce requirements or the scope of programs. We found a number of instances in which CSB meetings were effective in mitigating the effect of necessary changes, rejecting other changes, facilitating discussion of requirements, and endorsing descoping options with the potential to improve or preserve cost or schedule. Program managers, however, may be reluctant to recommend descoping options because of cultural biases about the role of a program manager, a preference not to elevate decisions to higher levels of review, and concerns that future funding will be cut. In an effort to increase descoping proposals, the Army and Air Force have issued additional descoping guidance and set savings or budget targets. The perceived effectiveness of the CSB meetings also varied based on the acquisition phase of a program and which CSB members participated. To further increase effectiveness and efficiency of CSBs, some of the military departments have taken steps to coordinate CSB meetings among programs that provide similar capabilities and align CSB meetings with other significant reviews. Programs Have Had Some Success in Using CSBs to Control and Reduce Requirements: We identified individual examples from each military department in which CSB meetings were used to prevent or reject requirements or configuration changes, mitigate the cost and schedule effects of endorsed changes, facilitate the prioritization of requirements, and provide program managers with opportunities to reduce requirements or suggest other programmatic changes to lower costs and field systems faster. However, most of the program officials who held CSB meetings and responded to our survey reported that CSB meetings were not useful for preventing changes to requirements or configuration, mitigating the potential effects on cost and schedule when changes were endorsed, or recommending ways to improve a program's cost and schedule by moderating requirements. In interviews with program officials, some explained that they did not utilize the CSB meetings to control requirements because they addressed requirement issues as they arose within the program rather than waiting for their program's scheduled CSB meeting to occur. Others stated that their program was stable and that there were no requirement changes or descoping options to discuss. According to our survey results, reviews of CSB documentation, and interviews: * 26 percent of the programs in our survey with CSB meetings reported that these meetings were useful forums to prevent changes to requirements. Moreover, 35 percent reported that the meetings were useful to make necessary changes to requirements. In an interview, several program officials stated that the mere suggestion of convening a CSB meeting to discuss a new requirement was enough to deter changes. * 25 percent of the programs in our survey with CSB meetings reported that these meetings were useful forums to prevent changes to technical configuration. Conversely, 23 percent reported that the meetings were useful to make necessary changes to technical configurations. Our review of minutes and presentations also show at least one CSB meeting that rejected a change that had the potential to adversely affect program cost; the August 2010 CSB review for the LPD 17 amphibious ship program rejected a proposed configuration change that would have added new equipment to the ship at an estimated cost of $26 million. * Some CSB meetings also included discussions of how to prioritize requirements. For example, according to officials, the Air Force used a June 2010 CSB meeting for the Global Hawk--an unmanned surveillance aircraft--to prioritize joint urgent operational needs. According to program officials, the Global Hawk program has received numerous requests to add new capabilities to the platform due to its use in current operations. The program manager stated that the CSB meeting provided the opportunity to present the costs and benefits of those requests to decision makers and receive guidance from them on which ones to pursue or defer. * 28 percent of the programs in our survey with CSB meetings reported that these meetings were useful forums to mitigate the potential cost and schedule effects of changes brought to the CSB for consideration. Moreover, 18 percent of programs reported CSB meetings were useful forums to mitigate the potential cost and schedule effects of changes made as a result of the CSB. The Vertical Take Off and Landing Tactical Unmanned Aerial Vehicle program used a CSB meeting to discuss ways to restructure the program in response to cost growth. At the meeting, the members of the CSB encouraged the program manager to go beyond his proposals and investigate changes to program quantities, contract strategy, and operational plans when restructuring the program, in order to reduce cost. * CSB meetings seem to have been effective in mitigating the cost and schedule effects of changes or only endorsing changes that would not affect costs and schedules. Of the 19 programs in our survey in which a CSB meeting endorsed changes to requirements or technical configuration, 1 reported an increase in program cost and 2 reported a delay in the delivery of an initial operational capability. * 30 percent of programs in our survey with CSB meetings reported that these meetings were useful forums to offer options to lower costs and field systems faster. Survey results show that descoping options were presented for 19 programs and those options were endorsed for 8 of them. For example, at the December 2009 CSB meeting for the Air Force's Joint Air-to-Surface Standoff Missile, the program office recommended adopting the extended range version's lower reliability requirement for the baseline missile. The program office stated the existing baseline requirement, which was 5 percent higher, had the potential to become a cost driver in testing for the program. The CSB endorsed the program office's recommendation. * Program officials also reported that the exercise of formulating descoping options, regardless of whether or not they were endorsed, helped their office identify and develop mitigation strategies in the event costs increased. Table 3 provides examples of programs across the military departments that used CSB meetings to endorse requirement, configuration, or other programmatic changes to improve or preserve cost or schedule. Table 3: Descoping Options Endorsed at CSB Meetings: Air Force: Program: Joint Air-to-Surface Standoff Missile; Action endorsed: Relaxed reliability requirement; Result: May avoid test costs. Program: Joint Strike Fighter; Action endorsed: Deleted requirement to jettison stores at supersonic speeds; Result: Avoided test and development cost. Program: Predator; Action endorsed: Transferred two test units to the Army; Result: May reduce costs. Army: Program: Excalibur; Action endorsed: Reduced quantity of projectiles procured; Result: Reduced total program cost by $893.5 million. Program: Family of Medium Tactical Vehicles; Action endorsed: Eliminated the self-recovery winch on most variants; Result: Reduced unit cost by $9,535 per vehicle. Program: Increment 1 Early Infantry Brigade Combat Team; Action endorsed: Ceased development of three portions of the program; Result: Reduced total program cost by $112.8 million. Program: Joint Land Attack Cruise Missile Defense Elevated Netted Sensor System; Action endorsed: Reduced quantities and relaxed requirement for emplacement time; Result: May result in preservation or improvement of program cost or schedule. Program: Joint Tactical Radio System Airborne & Maritime/Fixed Station; Action endorsed: Relaxed requirement for startup time; Result: May result in preservation or improvement of program cost or schedule. Program: Joint Tactical Radio System Handheld, Manpack, and Small Form Fit Radio; Action endorsed: Eliminated requirements for two radios as well as a requirement for radios to operate one waveform; Result: May result in preservation or improvement of program cost or schedule. Program: Joint Tactical Radio System Network Enterprise Domain; Action endorsed: Eliminated an information assurance requirement for one waveform; Result: Avoided costs of $75 million. Program: Stryker Family of Vehicles; Action endorsed: Canceled production of flat bottom variants and rearranged decision points for others; Result: Avoided $1.7 billion and $24.1 million, respectively, in costs. Navy: Program: AIM-9X Air-to-Air Missile; Action endorsed: Limited use of missiles in training; Result: Possible maintenance cost avoidance of 60%. Program: CH-53K; Action endorsed: Deferred a communications requirement to future increments; Result: Avoided adverse impact on program cost and schedule. Program: Joint High Speed Vessel; Action endorsed: Reduction of the ship's transit speed; Result: Avoided significant redesign and the possibility of increased cost and schedule. Source: GAO analysis of CSB minutes. [End of table] Program Managers May Be Reluctant to Offer Options to Moderate Requirements: Program managers may be reluctant to recommend descoping options to moderate requirements during a CSB meeting because of cultural biases about the role of a program manager, a preference not to elevate decisions to higher levels of review, and concerns that future funding will be cut. According to several acquisition officials, there is a cultural bias throughout DOD that the role of the program manager is to meet the requirements handed to them, not to seek to reduce them to achieve cost savings. In this context, if a program manager recommends reducing requirements, it may suggest the person is not managing the program or serving the warfighter well. Still others preferred to reduce requirements that were within their span of control through their program's internal change-management process rather than waiting for a CSB meeting to ask permission. For example, the DDG-51 program office proposed changes to the ships' configuration to reduce cost by removing or relocating equipment and the CH-53K program avoided cost by relaxing a requirement for self-sealing fuel tanks. Our interviews with program officials also suggest that there may be a reluctance to present descoping options at a CSB meeting because it could be interpreted as an opportunity to reduce the program's budget. Army and Air Force Are Encouraging More Descoping Options to Reduce Costs: The Army and Air Force have both taken steps to encourage or require program managers to seek options to lower costs by reducing scope. Acquisition officials noted that the presentation of descoping options and the focus on reducing costs has increased in importance since CSBs were first established, as the budget environment has become more constrained. In a November 2010 memorandum, the Army emphasized the need for program officials to aggressively seek descoping opportunities with the goal of reducing per-unit or total program costs by 5 percent. Army officials stated that the memorandum was signed by senior leaders from the requirements, acquisition, and budgeting communities specifically to address the bias that reducing requirements is unacceptable. According to officials, the Air Force amended its guidance for CSB meetings to require programs to present three to four descoping options along with the effect of those options on performance and program execution, the dollar amount already invested, and the estimated savings likely to result. Program managers are instructed to treat the descoping options as a budgeting exercise and to present the decisions that would need to be made if the program's current budget were reduced by 10, 20, and 30 percent. Several program offices told us that forcing programs to present options to reduce requirements or scope led them to spend time preparing options that were not viable or that they would have to recommend against implementing. CSBs' Effects Differ for Programs in Development and Production: The types of discussions for which CSBs were useful changed based on whether programs were in development or production. According to our survey results, programs in development found CSB meetings to be more useful than programs in production for making necessary changes to requirements or technical configuration, mitigating the potential cost and schedule effects of changes, and recommending proposals to improve program costs and schedule. Table 4 presents our survey results of program officials' opinions on the usefulness of CSB meetings. Table 4: Program Officials' Opinions on the Utility of CSB Meetings by Acquisition Phase: CSB was useful for: Preventing changes to requirements; Percentage of programs in development that responded "Yes": 18; Percentage of programs in production that responded "Yes": 30; Percentage of all programs that responded "Yes": 26. CSB was useful for: Preventing changes to technical configuration; Percentage of programs in development that responded "Yes": 12; Percentage of programs in production that responded "Yes": 30; Percentage of all programs that responded "Yes": 25. CSB was useful for: Making necessary changes to requirements; Percentage of programs in development that responded "Yes": 53; Percentage of programs in production that responded "Yes": 28; Percentage of all programs that responded "Yes": 35. CSB was useful for: Making necessary changes to technical configuration; Percentage of programs in development that responded "Yes": 24; Percentage of programs in production that responded "Yes": 23; Percentage of all programs that responded "Yes": 23. CSB was useful for: Mitigating the potential cost and schedule impacts of changes for which CSB convened; Percentage of programs in development that responded "Yes": 35; Percentage of programs in production that responded "Yes": 25; Percentage of all programs that responded "Yes": 28. CSB was useful for: Mitigating the potential cost and schedule impacts of changes made as a result of the CSB; Percentage of programs in development that responded "Yes": 31; Percentage of programs in production that responded "Yes": 13; Percentage of all programs that responded "Yes": 18. CSB was useful for: Recommending proposals to improve the program costs and schedule; Percentage of programs in development that responded "Yes": 35; Percentage of programs in production that responded "Yes": 28; Percentage of all programs that responded "Yes": 30. Source: GAO analysis of survey data. [End of table] Programs in development also proposed changes to requirements or configuration, presented options for reducing scope, and had those options endorsed at a higher rate than those in production. Even so, an official for one program in development stated that its CSB meeting was not effective because the program was meeting cost and schedule targets and its requirements were narrowly defined, which decreased opportunities for reducing scope. According to our survey results, a higher percentage of programs in production reported that CSBs were useful in preventing changes compared to programs in development. We have previously reported that stabilizing a program's requirements and design well before production is important because changes have increasingly negative effects on cost and schedule the further a program progresses.[Footnote 20] Program officials were wary about using CSB meetings to try to reduce costs for programs in production either through requirements changes or reductions in scope because the configuration should be locked, the available trade space is probably limited, and potential changes could be disruptive. For instance, the E-2D program reported in its April 2010 CSB meeting that its configuration was extremely stable and, with development and demonstration almost complete, reducing the scope of the programs could prove detrimental because it could lead to redesigns or decreases in capability. Changes at this stage of a program can still have a positive effect on cost if they do not require extensive design changes. For example, the program manager for the Family of Medium Tactical Vehicles--which is well into production with over 40,000 vehicles fielded--recommended removing the self- recovery winch from some vehicles, resulting in savings of $9,535 per vehicle. Broad Senior Level Participation in CSB Meetings Facilitates Decision Making: CSBs provide a unique opportunity for program managers to address programmatic issues in front of a broad group of high-level decision makers that includes the acquisition, requirements, and funding communities. In some cases, the makeup of the CSB helped to accelerate the resolution of issues and facilitate decision making. For example, the Grey Eagle program utilized its CSB meeting to endorse an increase in the number of active units from 13 to 17. The program office reported that this decision, which otherwise may have taken years to approve and fund, was made and implemented quickly by the CSB because of the senior leadership present. Other program offices stated that the broad membership on CSBs, which includes key stakeholders and other interested parties, helps to create institutional buy-in for programmatic changes. CSB meetings also raised stakeholders' awareness of cost increases. Specifically, CSB meetings provided the Joint Staff with its first knowledge of cost growth on at least four programs and triggered separate reviews by the Joint Requirements Oversight Council. When critical stakeholders are absent, the decision-making ability of the CSB may be limited. In particular, some programs with users from across the military services and organizations external to DOD reported that the utility of CSBs was limited when those users were not represented. For example, the primary users of the Air Force's Global Positioning System IIIA program include the Army, Navy, and other organizations external to DOD. The September 2010 CSB meeting for the system did not include these stakeholders, and program officials stated that as a result, the CSB was not empowered to make significant changes to the program. Aligning CSB Meetings with Other Reviews May Increase Effectiveness and Efficiency: The decisions made at CSB meetings can affect complementary programs, as well as the funding required for programs. As a result, acquisition and program officials told us there is value in aligning CSB meetings so they are held together with reviews of similar programs or sequencing them to occur before key funding decisions are made. For example, in 2010, the Army grouped programs into capability portfolios, such as aviation or precision fires capabilities, and held one CSB meeting to discuss requirement changes and descoping options for all the programs. These CSB meetings generally occurred after the Army's capability portfolio reviews--which revalidate, modify, or terminate requirements and ensure the proper allocation of funds across programs--and reviewed, endorsed, and implemented the recommendations coming from them. Holding CSB meetings for capability portfolios can facilitate discussions about interoperability and interdependency and promote an examination of requirements and capabilities across programs, including potential redundancies. Officials also stated that if two well-executed, high-performing programs within the same portfolio were reviewed independently, those discussions might not take place. For example, the Army's Excalibur--a precision-guided munition--and Guided Multiple Launch Rocket System were both relatively stable programs in production. However, according to officials, during a capability portfolio review, the Army identified an overlap in the two programs' capabilities and missions and recommended reducing the number of Excalibur munitions to be procured. At the subsequent April 2010 CSB meeting, the Army reviewed and implemented the proposal, which reduced the cost of the Excalibur program by $893.5 million. According to acquisition officials, grouping programs in this manner can also ease the difficulty of scheduling a large number of meetings that require senior leadership participation. According to program officials, when CSB meetings were aligned with budget deliberations, it enabled an informed discussion of funding issues and rapid changes to program budgets. USD (AT&L)'s 2007 memorandum establishing CSBs stressed the importance of making necessary budget adjustments, especially those involving expected increases in program costs, at the earliest opportunity. In one example, the Army's November 2009 CSB for the Patriot and Medium Extended Air Defense System programs corresponded with the service's fiscal-year-2011 budget-formulation process. Program officials stated that this helped facilitate the transfer of funds and efforts among the two programs, which had been endorsed by senior leaders from the acquisition and funding communities at the CSB. However, it may be functionally challenging to align CSB meetings with the budget formulation process in all cases, as CSB meetings in some cases must be event driven while the budget process is calendar driven. Conclusions: With the prospect of slowly growing or flat defense budgets for years to come, DOD must get better returns on its weapon system investments than it has in the past. CSBs, which are intended to ensure that a program delivers as much planned capability as possible at or below the expected cost, can be a key tool in furthering this goal. They represent a unique forum that brings together a broad range of high- level decision makers from the acquisition, requirements, and funding communities, who can make and implement decisions quickly. DOD's experience with CSBs to date has already demonstrated their potential value--costly new requirements have been rejected, and options to moderate requirements and reduce program costs by millions of dollars have been endorsed. However, the efficiency and effectiveness of CSBs can still be improved. Ensuring key CSB members from the acquisition and requirements community are present at meetings could help build consensus more quickly and make decisions more efficiently. Similarly, while the law is silent on whether paper CSB meetings may be used to meet the annual requirement, holding in- person meetings may be more effective because a paper meeting may not provide the opportunity for in-depth discussion or proper oversight. Holding CSBs in conjunction with capability portfolio reviews and other similar meetings has the potential to expand opportunities to review and rationalize requirements across programs. Improving the connection between CSBs and the budget process and other reviews can help further efforts to match weapon system requirements with funding resources. Reviewing programs at CSBs on a case-by-case basis well into production would help decision makers identify cost savings and shift funding as warfighter needs and funding priorities change. Taken together, these steps have the potential to improve not only the efficiency and effectiveness of CSBs but also the affordability and execution of DOD's major defense acquisition programs. Recommendations for Executive Action: We recommend that the Secretary of Defense take the following seven actions directing: * the Navy to amend its policy on CSBs to ensure that all statutorily required participants, particularly the Joint Staff, are included; * the MDA to amend its policy to ensure that all statutorily required participants for military department CSBs are included in MDA's Program Change Board, particularly the Joint Staff, if it is to serve as an equivalent review; * USD (AT&L) to amend its acquisition instruction to: - ensure that all statutorily required participants, in particular the comptroller, are included on CSBs; - require CSB meetings for major defense acquisition programs in production as well as development but also coordinate with the military departments and the Congress to evaluate the effectiveness of CSB meetings for programs well into production; and: - develop the means to better track CSBs and ensure compliance with the requirement that CSBs hold a meeting at least once each year; * USD (AT&L) to work with DOD components to determine whether paper CSBs are as effective as in-person meetings and, if not, amend the acquisition instruction accordingly; and: * DOD components to amend their policies to encourage alignment between CSB meetings and other complementary reviews whenever possible. Agency Comments and Our Evaluation: DOD provided us with written comments on a draft of this report. In its comments, DOD concurred or partially concurred with all seven of our recommendations and agreed to take action to address six of them. The comments are reprinted in appendix II. DOD also provided technical comments, which we addressed in the report, as appropriate. In concurring with our recommendation that the Navy amend its policy on CSBs to include all statutorily required participants, DOD stated that the Navy has already issued two policy memorandums that do so. DOD also stated that the Navy will continue to issue policy guidance consistent with our recommendation. This will be particularly important as the Navy is currently in the process of revising its primary acquisition instruction. DOD also concurred with our recommendations to amend its acquisition instruction to ensure that all statutorily required participants are included in CSBs and that meetings occur for programs in development as well as those in production. DOD did not address the portion of our recommendation to coordinate with the military departments and the Congress to evaluate the effectiveness of CSB meetings for programs well into production. Given our mixed findings on the utility of CSB meetings late in production, we continue to believe it would be in the interest of the department to study this issue. DOD partially concurred with our recommendation that MDA amend its policy to ensure that all statutorily required participants for military department CSBs, in particular the Joint Staff, are included in MDA's Program Change Board, if it is to serve as an equivalent review. In its comments, DOD stated that Joint Staff participation would provide little value because of the role of the Joint Staff in the acquisition of BMDS. In addition, DOD pointed out that the Joint Staff participates in the Missile Defense Executive Board, a forum in which strategic direction and funding priorities are established. However, we continue to believe that if the Program Change Board is to act as the forum for discussing configuration and requirements changes, it is important that the user communities, as represented by the Joint Staff, participate in these discussions. DOD partially concurred with our recommendations on improving the tracking of CSB meetings, determining the effectiveness of paper CSBs, and aligning complimentary reviews with CSB meetings, when possible. In its comments, DOD stated that it would address these issues in "best practices" guidance to the military departments. With regard to developing the means to better track CSB meetings and compliance with the requirement to hold a meeting at least once each year, DOD stated the best practices guidance will direct the military departments to ensure adequate tracking vehicles are in place. We continue to believe that USD (AT&L) should play a role in tracking compliance and holding the military departments accountable, given our findings that the military departments did not hold CSBs for all the required programs. We are sending copies of this report to the Secretary of Defense; the Secretaries of the Army, Navy, and Air Force; USD (AT&L); and the Director of the Office of Management and Budget. In addition, the report will be made available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions concerning this report, please contact me at (202) 512-4841. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. Staff members making key contributions to this report are listed in appendix III. Signed by: Michael J. Sullivan: Director, Acquisition and Sourcing Management: [End of section] Appendix I: Scope and Methodology: Selection and Classification of Major Defense Acquisition Programs: This report presents information on the Department of Defense's (DOD) use of Configuration Steering Boards (CSB) for the major defense acquisition program portfolio in 2010. We used the Defense Acquisition Management Information Retrieval system to identify 98 active major defense acquisition programs. We defined an active program as one that issued a selected acquisition report in December 2009. This report presents information on all of these programs. One program, the Ballistic Missile Defense System, is managed by the Missile Defense Agency (MDA), which reports acquisition information on the system by functional elements. We reviewed nine elements and analyzed them separately from the rest of the major programs. We categorized programs by the five acquisition organizations designated as having oversight--Army, Navy, Air Force, MDA, and the Assistant Secretary of Defense for Nuclear, Chemical, and Biological Defense programs--to assess trends in the use of CSBs. The selected acquisition report for each program designates the program's acquisition organization. As the lead authority for joint programs rotates among the acquisition organizations as determined by the Office of the Secretary of Defense, we categorized all joint programs according to the service that was designated as the lead authority in the December 2009 selected acquisition report. All of the programs in our audit fall into one of two phases: engineering manufacturing and development (referred to as development) or production and sustainment (referred to as production). Development generally begins with the initiation of an acquisition program as well as the start of engineering and manufacturing development and generally ends with entry into production. Production generally begins with the decision to enter low-rate initial production. For most programs in our assessment, the placement of programs in one of these two phases was determined by the dates of their Milestone B/II and Milestone C/III decisions. For instance, we categorized programs that have held a Milestone B/II decision but not a Milestone C/III as in the development phase and those that have held a Milestone C/III decision as in the production phase. The dates of milestone decisions for the programs used in the audit were determined through use of the Defense Acquisition Management Information Retrieval system. Due to the nature of individual programs, select programs were not classified by milestone decision because they either have multiple increments that may begin production in advance of the notional Milestone C/III date,[Footnote 21] or the programs do not report milestone dates.[Footnote 22] In these cases, we used the program's selected acquisition reports to determine the appropriate phase. The Navy often authorizes shipbuilding programs to begin production of the lead ship at Milestone B/II. We classified these programs as in the production phase.[Footnote 23] As the MDA programs develop systems' capabilities incrementally instead of following the standard DOD acquisition model, we did not identify acquisition phases for Ballistic Missile Defense System elements. Compliance with Statutory Requirements: To assess the extent that DOD has complied with the statutory requirements for CSB meetings in 2010, we compared CSB execution to provisions in the statute that call for annual CSB meetings and discussion of specific content. To determine the extent to which DOD complied with the requirement to hold an annual CSB for each program, we analyzed CSB records provided by the acquisition organization we reviewed and, using these records, calculated the number of CSBs held for each program in calendar-year 2010. To determine whether the components established boards that included the statutorily required participants, we analyzed policy and procedure documentation from each of the components as well as attendance lists of CSBs held in calendar- year 2010, provided by the acquisition organizations we reviewed. To identify issues discussed at CSBs and actions resulting from these CSBs, we reviewed CSB documents and questionnaire data and interviewed acquisition officials. We also reviewed and analyzed current and draft documentation related to department and service-level CSB policies, directives, guidance, and instructions to determine if they establish a structure that would facilitate compliance with the statute; examples of these documents include Department of Defense Instruction 5000.02, Department of the Army Pamphlet 70-3 regarding Army Acquisition Procedures, SECNAV Instruction 5000.2D, Air Force Instruction 63-101, and Missile Defense Agency Directive 5010.18 regarding Acquisition Management. We also interviewed officials representing organizations that participate in CSBs or their equivalents including the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics, Joint Staff, military service and MDA offices, program offices, and capabilities and requirements offices to address department, military service, and MDA policies and execution. Effectiveness of Configuration Steering Boards: To assess how effective CSBs have been controlling requirements and mitigating cost and schedule risks on programs, we analyzed CSB documentation to identify actions proposed and actions taken as a result of the CSB and their effect on cost, schedule, performance, and system configuration. We also asked program officials in our questionnaire to identify requirement changes or descoping options discussed at the CSB, the impact of decisions made, perceived effectiveness of the CSB, and explanations for not conducting a CSB, if applicable. To further analyze the effectiveness, challenges, and benefits of holding CSBs, we selected 17 programs for interviews. We based our selection on answers to our questionnaire, discussions with officials, and programmatic factors such as acquisition organization and phase. Specifically, we met with program officials at Wright Patterson Air Force Base, Ohio; Redstone Arsenal, Alabama; Washington Navy Yard in Washington DC; the Naval Air Station Patuxent River in Patuxent River, Maryland; and conducted video teleconferences with program officials at Picatinny Arsenal in New Jersey and at Los Angeles Air Force Base in El Segundo, California. We also interviewed acquisition officials, reviewed selected acquisition reports, and examined documentation related to service-level CSB policies, directives, guidance, and instructions to determine whether other reviews or acquisition processes influenced the effectiveness of CSBs. DOD Major Defense Acquisition Programs Questionnaire: To collect information about DOD's use of CSBs in fiscal year 2010, we developed and administered a Web-based questionnaire to the program offices of all 98 programs. Fiscal-year data was collected in our survey to be consistent with the Senate report language that contained our mandate. We administered separate questionnaires to nine Ballistic Missile Defense System elements and analyzed the results separately from the rest of the programs in our review. We fielded the survey from October 2010 to December 2010, and after extensive follow-up, we received responses from all 98 programs. Our questionnaire of the 98 program offices, was not a sample questionnaire, so it has no sampling errors. However, the practical difficulties of conducting any questionnaire may introduce errors, commonly referred to as nonsampling errors. For example, difficulties in interpreting a particular question or limitations in the sources of information available to respondents can introduce unwanted variability into the questionnaire results. We took steps in developing the questionnaire, collecting the data, and analyzing the responses to minimize such nonsampling errors. For example, social science survey specialists designed the questionnaire in collaboration with GAO's subject-matter experts. We conducted pretests with program managers to check that (1) the questions were clear and unambiguous, (2) terminology was used correctly, (3) the questionnaire did not place an undue burden on agency officials, (4) the information could feasibly be obtained, and (5) the questionnaire was comprehensive and unbiased. For the pretests, we selected programs from each military department and from various phases of the acquisition life cycle. We conducted four pretests. We made changes to the content and format of the questionnaire after each pretest, based on the feedback received. When we analyzed the data, an independent analyst checked all computer programs to reduce risk of error. Since this was a Web-based questionnaire, respondents entered their answers directly into the electronic questionnaire, eliminating the need to key data into a database, minimizing error. We did not validate the data provided by the program offices, but reviewed the data and performed various checks to determine that the data were reliable enough for our purposes. Where we discovered discrepancies from reviewing responses and interviewing program offices, we clarified the data with the program office and made changes to the questionnaire data accordingly. [End of section] Appendix II: Comments from the Department of Defense: Office Of The Under Secretary Of Defense: Acquisition, Technology And Logistics: 3000 Defense Pentagon: Washington, DC 20301-3000: June 24, 2011: Mr. Michael J. Sullivan: Director, Acquisition and Sourcing Management: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Sullivan: This is the Department of Defense (DoD) response to the GAO Draft Report, GA0-11-640, "Defense Acquisitions: DoD Can Improve Management of Programs Using Configuration Steering Boards," dated June 1, 2011 (GAO Code 120935). The DoD concurs with three of the draft report's recommendations and partially concurs with four. The rationale for our position is enclosed. I submitted separately a list of technical and factual errors for your consideration. We appreciate the opportunity to comment on the draft report. My point of contact for this effort is Mr. Ronald Woods, Ronald.Woods@osd.mil, 703-697-8183. Sincerely, Signed by: David G. Ahern: Deputy Assistant Secretary of Defense: Portfolio Systems Acquisition: Enclosure: As stated: [End of letter] GAO Draft Report Dated June 1, 2011: GAO-11-640 (GAO Code 120935): "Defense Acquisitions: DOD Can Improve Management Of Programs Using Configuration Steering Boards" Department Of Defense Comments To The GAO Recommendations: Recommendation 1: The GAO recommends that the Secretary of Defense direct the Navy to amend its policy on Configuration Steering Boards (CSB) to ensure that all statutorily required participants, particularly the Joint Staff and USD (AT&L), are included. (See page 24/GAO Draft Report.) DOD Response: Concur. The Navy has already issued two policy memos that provide guidance that invitations to Configuration Steering Boards must include the Joint Staff and OSD. The Department of the Navy will continue to issue policy guidance ensuring all statutorily required participants are included in the conduct of Configuration Steering Boards. Recommendation 2: The GAO recommends that the Secretary of Defense direct the Missile Defense Agency (MDA) to amend its policy to ensure that all statutorily required participants for military department CSBs are included in MDA's program change board, particularly the Joint Staff, if it is to serve as an equivalent review. (See page 24/GAO Draft Report.) DOD Response: Partially Concur. Joint Staff participation in Missile Defense Agency (MDA) Program Change Boards would provide little value because the Joint Staff is not a stakeholder in the detailed execution of the MDA Ballistic Missile Defense System (BMDS) acquisition process. Requirements for the MDA BMDS are not derived from the Joint Capabilities Integration Development System, MDA BMDS acquisition decision reviews do not include Joint Staff participation, and the Joint Staff does not oversee the BMDS Accountability Report. Joint Staff participation in the Missile Defense Executive Board, a forum where strategic direction and funding priorities are established, provides the desired guidance and oversight. Recommendation 3: The GAO recommends that the Secretary of Defense direct the USD (AT&L) to amend its acquisition instruction to ensure that all statutorily required participants, in particular the comptroller, are included on CSBs. (See page 24/GAO Draft Report.) DOD Response: Concur. The Under Secretary of Defense for Acquisition, Technology and Logistics will amend acquisition policy to require that all statutorily required participants, including the comptroller of the relevant military department, are included on Configuration Steering Boards. Recommendation 4: The GAO recommends that the Secretary of Defense direct the USD (AT&L) to amend its acquisition instruction to require CSB meetings for major defense acquisition programs in production as well as development but work with the military departments and the Congress to evaluate the effectiveness of CSB meetings for programs well into production. (See page 24/GAO Draft Report.) DOD Response: Concur. The Under Secretary of Defense for Acquisition, Technology and Logistics will amend acquisition policy to ensure that the Configuration Steering Board meet at least annually for Acquisition Categories I and IA programs in either development or production. Recommendation 5: The GAO recommends that the Secretary of Defense direct the USD (AT&L) to amend its acquisition instruction to develop the means to better track CSBs and ensure compliance with the requirement that CSBs hold a meeting at least once each year. (See page 24/GAO Draft Report.) DOD Response: Partially concur. The Department will issue Configuration Steering Board (CSB) "best practices" to the Military Departments. We will address the need for the Military Departments to ensure adequate tracking vehicles are in place to ensure compliance with CSB statute and policy. Recommendation 6: The GAO recommends that the Secretary of Defense direct the USD (AT&L) to work with DOD components to determine whether paper CSBs are as effective as in person meetings and, if not, amend the acquisition instruction accordingly. (See page 24/GAO Draft Report.) DOD Response: Partially concur. The Department will issue Configuration Steering Board (CSB) "best practices" to the Military Departments. We will address the appropriateness of paper CSBs in those "best practices." Recommendation 7: The GAO recommends that the Secretary of Defense direct DoD components to amend their policies to encourage alignment between CSB meetings and other complementary reviews whenever possible. (See page 24/GAO Draft Report.) DOD Response: Partially concur. The Department will issue Configuration Steering Board (CSB) "best practices" to the Military Departments. We will address the advantages of aligning CSBs with complementary reviews. [End of section] Appendix III: GAO Contact and Acknowledgments: GAO Contact: Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov: Acknowledgments: In addition to the contact named above, Ronald E. Schwenn, Assistant Director; Noah B. Bleicher; MacKenzie Cooper; Morgan Delaney Ramaker; J. Kristopher Keener; Jean McSween; Kenneth E. Patton; and Brian Schwartz made key contributions to this report. [End of section] Footnotes: [1] Major defense acquisition programs are those identified by the Undersecretary of Defense for Acquisition, Technology, and Logistics that will eventually require a total expenditure for research development, test, and evaluation of more than $365 million or procurement funding, including all increments, of more than $2.19 billion (in fiscal-year-2000 constant dollars) or those designated by the milestone decision authority as a major defense acquisition program. [2] About $65 billion of this growth can be attributed to quantity changes. [3] GAO, Best Practices: Better Support of Weapons Systems Program Managers Needed to Improve Outcomes, [hyperlink, http://www.gao.gov/products/GAO-06-110] (Washington, D.C.: Nov. 30, 2005) and Defense Acquisitions: Assessments of Selected Weapons Programs, [hyperlink, http://www.gao.gov/products/GAO-10-388SP] (Washington, D.C.: Mar. 30, 2010). [4] Pub L. No. 110-417, § 814 (2008). [5] [hyperlink, http://www.gao.gov/products/GAO-10-388SP]. [6] S. Rep. No. 111-201, at 170 (2010). [7] We excluded two of these programs, the Ballistic Missile Defense System and Chemical Demilitarization-Assembled Chemical Weapons Alternatives, from some of our analysis because the CSB provisions in statute only apply to military department major defense acquisition programs. These programs are managed by the Missile Defense Agency (MDA) and the Assistant Secretary of Defense (ASD) for Nuclear, Chemical, and Biological Defense programs (NCB) respectively. [8] DOD is required to submit selected acquisition reports (SAR) to Congress at the end of each fiscal-year quarter on current major defense acquisition programs, though certain exemptions apply. SARs for the first quarter of a fiscal year are known as comprehensive annual SARs. Each comprehensive annual SAR is required to be submitted within 60 days after the date on which the President transmits the Budget to Congress for the following fiscal year. 10 U.S.C. § 2432(b)(1), (c)(4), (f). [9] A requirement is an established need justifying the timely allocation of resources to achieve a capability to accomplish approved military objectives, missions, or tasks. These are often communicated in requirements and other documentation as key performance parameters, key systems attributes, or contract specifications. Configuration refers to the functional and physical characteristics of a product. [10] Pub L. No. 110-417, § 814 (2008). [11] The statute does not require CSBs to be established for programs outside the military departments, such as the Ballistic Missile Defense System and Chemical Demilitarization-Assembled Chemical Weapons Alternatives, which are managed by the Missile Defense Agency (MDA) and the Assistant Secretary of Defense (ASD) for Nuclear, Chemical, and Biological Defense programs (NCB) respectively. These programs are covered by the CSB provision in DODI 5000.02. [12] [hyperlink, http://www.gao.gov/products/GAO-10-388SP]. [13] S. Rep. No. 111-201, at 170 (2010). [14] A breach of the critical cost growth threshold occurs when the program's acquisition unit cost or the procurement unit cost increases by at least 25 percent over the current baseline estimate or at least 50 percent over the original baseline estimate. 10 U.S.C. § 2433. [15] Although the law requires a CSB to "meet ... at least once each year," it is silent with regard to in-person or virtual "paper" meetings. Pub. L. No. 110-417, § 814(c)(4) (2008). [16] While the law states that the service acquisition executive should chair the CSB itself, it does not address whether the chair for a particular CSB meeting can be delegated. [17] The Probability of Program Success model, as developed and implemented within DOD, reviews the factors and metrics that contribute to the success of a program with the goal of projecting a program's future performance. [18] The Navy has six gate reviews, which recur over time. While program officials can present and discuss issues related to requirements or configuration at any of the gate reviews, only the gate 6 review is generally used to fulfill the requirement for an annual CSB. [19] The statute does not specify a point at which meetings are no longer required, only that the CSBs for military departments must be held for each major defense acquisition program at least once a year. According to DOD, programs with 90 percent of items delivered are no longer covered by the statute as changes to requirements or configuration could no longer occur after a program reaches its inventory objective; in addition, at this point, official reporting through the SAR is no longer required. [20] GAO, Best Practices: Capturing Design and Manufacturing Knowledge Early Improves Acquisition Outcomes, [hyperlink, http://www.gao.gov/products/GAO-02-701] (Washington, D.C.: July 15, 2002). [21] Global Hawk (RQ-4A/B); Guided Multiple Launch Rocket System/Guided Multiple Launch Rocket System Alternative Warhead; Patriot/Medium Extended Air Defense System Combined Aggregate Program. [22] Airborne Signals Intelligence Payload; Ballistic Missile Defense System; Chemical Demilitarization–Chemical Materials Agency; Chemical Demilitarization–Assembled Chemical Weapons Alternatives; CVN-68; Family of Advanced Beyond Line-of-Sight Terminals; Family of Medium Tactical Vehicles; Global Broadcast Service; Joint Tactical Radio System, Network Enterprise Domain; Space-Based Space Surveillance Block 10; T-AKE Lewis and Clark Class Dry Cargo/Ammunition Ship; Wideband Global SATCOM. [23] Cobra Judy Replacement; DDG 1000 Zumwalt Class Destroyer; CVN 78; LHA 6 America Class Amphibious Assault Ship; Littoral Combat Ship; LPD 17; SSN 774 Virginia Class Submarine. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: