This is the accessible text file for GAO report number GAO-02-856 entitled 'Military Transformation: Actions Needed to Better Manage DOD's Joint Experimentation Program' which was released on August 29, 2002. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products’ accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. Report to Congressional Committees: United States General Accounting Office: GAO: August 2002: Military Transformation: Actions Needed to Better Manage DOD’s Joint Experimentation Program: Military Transformation: GAO-02-856: Contents: Letter: Results in Brief: Background: Participation in Joint Experimentation Continues to Evolve: No Recommendations for Change Have Been Approved or Implemented: Some Key Management Elements Are Missing: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Scope and Methodology: Appendix I: Chronology of Key Events Important to Joint Experimentation: Appendix II: Additional Information on Joint Concept Development and Experimentation Activities: Appendix III: Comments from the Department of Defense: Appendix IV: Organizations and Offices Contacted: Appendix V: GAO Contacts and Staff Acknowledgments: Tables: Table 1: Organizations Involved in Joint Experimentation: Table 2: First Three Recommendations Submitted by Joint Forces Command: Table 3: Types of Activities and Experiments Associated with Joint Experimentation: Figures: Figure 1: Joint Concept Development and Experimentation Process: Figure 2: Involvement of Organizations in Preparing Joint Experimentation Guidance and Campaign Plan: Figure 3: Joint Concept Development and Experimentation Activities: Letter: August 29, 2002: The Honorable Carl Levin Chairman The Honorable John W. Warner Ranking Minority Member Committee on Armed Services United States Senate: The Honorable Bob Stump Chairman The Honorable Ike Skelton Ranking Minority Member Committee on Armed Services House of Representatives: The Department of Defense (DOD) considers the transformation[Footnote 1] of the U. S. military a strategic imperative to meet the security challenges of the new century. In October 1998, DOD established a joint concept development and experimentation program to provide the engine of change for this transformation. The U.S. Joint Forces Command, the executive agent of the program, carries out experiments--with support from the military services, the combatant commands, and other DOD organizations- -to explore and analyze new joint operational and organizational concepts. The experiments can include studies and analyses, conferences, workshops, seminars, war games, and simulated or live field experiments. On the basis of these experiments, the Joint Forces Command recommends changes aimed at achieving significant improvements in joint capabilities. The Joint Requirements Oversight Council[Footnote 2] reviews the Command’s recommendations, and the Chairman of the Joint Chiefs of Staff is responsible for approving them. The Council also oversees the implementation of approved recommendations. The cumulative changes are expected to contribute to the way U.S. military forces are transformed. In November 2001, DOD also created an Office of Force Transformation to foster innovation and experimentation and assist the Secretary of Defense to develop guidance and identify strategies and policies for transformation. Because of the central role DOD’s joint experimentation program plays in military transformation, we reviewed related activities to determine whether changes are needed to improve the program’s ability to focus on future joint operations, military transformation, and the rapid integration of results. Specifically, we assessed the extent to which (1) the Joint Forces Command obtains participation from all relevant sources in developing and conducting its experimentation activities; (2) the recommendations flowing from the joint experimentation process have been approved and implemented; and (3) the necessary management elements are in place to foster program success. We are providing this report to you because of your oversight responsibilities and interest in military transformation issues. Results in Brief: In the nearly 4 years since becoming the executive agent for joint concept development and experimentation, the Joint Forces Command has increased the participation of key DOD stakeholders--the military services, the combatant commands, and other organizations and agencies- -in its experimentation activities. The Command has also expanded the participation of federal agencies and departments, academia, the private sector, and some foreign allies. The Command uses many different ways to obtain input--including meetings, seminars, workshops, video conferencing, E-mail, and the Internet--and integrates the results of military operations, technology efforts, and other DOD organizations’ experiments into its activities. Although stakeholders determine the extent of their involvement in joint experimentation activities, many more organizations are in fact involved in the planning and execution of this year’s major field experiment than there were in a similar experiment 2 years ago. Most DOD officials whom we spoke with were generally satisfied with the way the Command reached out to obtain their input, but many also believe more frequent contacts with the Joint Forces Command would improve communications. Some believe that the Command could do more to expand the participation of non-DOD agencies and departments and allies and coalition partners, and the Command is planning initiatives to do so, though this outreach would involve broader agency-resource and national-security considerations. No recommendations flowing from joint experimentation have been approved or implemented. Although the Joint Forces Command issued three recommendations nearly a year ago, they were not approved by the Joint Requirements Oversight Council because of confusion among the Joint Staff and the Joint Forces Command about a proposed change in guidance that required additional data (on costs and timelines) be included when submitting these recommendations. As a result, it is not clear when these recommendations will contribute to military transformation. The Command plans to re-submit the recommendations with the required data later this year. However, at the time we concluded our review, the official guidance on what information should accompany joint experimentation recommendations had not been approved and issued. In addition, although it is too early to assess how DOD integrates the results of its experimentation into its plans, programs, and budgets, several DOD officials expressed concern that the resource allocation process may be too slow to provide rapid and timely funding for the implementation of new concepts emerging from joint experimentation and that other, more rapid resource mechanisms may be needed. While DOD has been providing more specific and clearer guidance for joint experimentation, DOD and the Joint Forces Command are missing some key management elements that are generally considered necessary for successful program management. DOD and the Joint Forces Command thus lack the following very important management tools needed to carry out their joint experimentation responsibilities: * It is not yet clear what role the Office of Force Transformation will have in joint experimentation and how it will coordinate its activities with other DOD stakeholders. As a result, the Secretary of Defense may not be effectively using this new office to interact with stakeholders to foster and monitor joint experimentation as a means to advance DOD’s transformation agenda. * DOD has not developed meaningful performance measures to assess joint experimentation’s contribution to the advancement of military transformation. Therefore, DOD cannot gauge the progress of joint experimentation efforts and of the program’s overall contribution to transformation. * The Joint Forces Command does not have the strategic-planning tools (a strategic plan with associated performance plans and reports) that would allow it to clearly define what it is seeking to accomplish, identify what strategy it will use to achieve desired results, and finally measure how well it is succeeding in meeting results-oriented goals and objectives. We are making recommendations designed to improve the processes and the management oversight and accountability for the joint concept development and experimentation program and to clarify the role of the Office of Force Transformation. In its comments on a draft of this report, DOD agreed with our recommendations and stated that our findings provide a factual summary of the history and progress of joint experimentation and identify the critical weaknesses currently being addressed by DOD. It expects that a forthcoming Transformation Planning Guidance or subsequent guidance documents will address the issues and problems with the joint experimentation program identified in our report. Background: The Joint Forces Command, in coordination with the Joint Staff,[Footnote 3] the services, and other combatant commands and DOD agencies, is responsible for creating and exploring new joint war- fighting concepts, as well as for planning, designing, conducting, and assessing a program of joint experimentation. The Command executed its second large-scale field experiment, Millennium Challenge 2002,[Footnote 4] this year, and it plans another one in 2004 and others every third year thereafter. These experiments are intended to examine how well the concepts previously explored by the Command in smaller venues will work when applied with the emerging concepts being developed by the services and other combatant commands. For example, Millennium Challenge 2002 tested how well U.S. forces fared against a regional power with a sizable conventional military force and so called “anti-access” capabilities--which can include advanced surface-to-air missiles, antiship missiles and mines, and chemical and biological weapons--and validated the results of earlier experiments to develop the Command’s “rapid decisive” operations concept. The aim of the experiment was to come up with changes that can be made during the current decade. (App. I provides a chronology of major events important to joint experimentation.): Over the next several years, the Command’s experimentation will focus primarily on two concepts: one to develop a standing joint force headquarters to improve joint command and control; another to conduct more effective joint operations through “rapid decisive” operations. In November 2001, the Chairman of the Joint Chiefs of Staff[Footnote 5] directed that the Command make development of the prototype headquarters its highest near-term priority.[Footnote 6] Additionally, the Command will develop a number of other concepts aimed at specialized issues or operational problems to support the two primary concepts. Joint experimentation is a continuous process that begins with the development of new operational and organizational concepts that have the potential to improve significantly joint operations (see fig. 1). The Joint Forces Command identifies new joint concepts including those developed by other DOD organizations (such as the Joint Staff, services, and combatant commands) and the private sector and tests them in experiments that range from simple (workshops, seminars, war games, and simulations) to complex (large-scale virtual simulations and “live” field experiments). Appendix II provides additional information on joint experimentation program activities. Figure 1: Joint Concept Development and Experimentation Process: [See PDF for image] Source: Joint Forces Command. [End of figure] After analyzing experimentation data, the Command prepares and submits recommendations to the Joint Requirements Oversight Council for review and, ultimately, to the Chairman of the Joint Chiefs of Staff for approval.[Footnote 7] Before submitting them to the Council, however, the Command submits its recommendations to the Joint Staff for preliminary review and coordination. The recommendations are distributed for review and comment to the Joint Staff directorates, the military services, the combatant commands, and other DOD and federal government organizations. The Council then reviews the recommendations and advises the Chairman of the Joint Chiefs of Staff on whether they should be approved. The changes, if approved, provide the basis for pursuing the capabilities needed to implement a specific operational concept. The Council is also responsible for overseeing the implementation of the recommendations, but it can designate an executive agent, such as the Joint Forces Command, to do so. The Council (or its designated executive agent) is responsible for obtaining the resources needed to implement the recommendations through DOD’s Planning, Programming, and Budgeting System.[Footnote 8] The Council also assists the Chairman, in coordination with the combatant commands, the services, and other DOD organizations, to identify and assess joint requirements and priorities for current and future military capabilities. The Council considers requirements (and any proposed changes) for joint capabilities across doctrine, organizations, training, materiel, leadership and education, personnel, and facilities. The Department of the Navy’s budget[Footnote 9] provides funding to the Joint Forces Command for joint experimentation and other Command missions. In fiscal year 2002, the Command received from the Navy about $103 million for its joint concept development and experimentation program, and it planned to spend about half of this amount for Millennium Challenge 2002. The Command has requested that the Navy provide about $98 million for the program in fiscal year 2003. The Command also provides some funds to the services, the combatant commands, and other DOD organizations for efforts that support its program activities. However, the services fund the operations and support costs of forces participating in joint experimentation. Also, the individual experimentation efforts of the services and the combatant commands are funded from within their own budgets. Participation in Joint Experimentation Continues to Evolve: Since it first began joint experimentation, the Joint Forces Command has broadened and deepened the inclusion of other DOD organizations, federal agencies and departments, the private sector, and allies and coalition partners in its process for capturing and identifying new joint ideas and innovations. Organizations participating in joint experimentation are generally satisfied with current opportunities for their ideas to be considered, and many have increased their participation in the program. However, the participation of different stakeholders--the extent of which is determined by the stakeholder-- varies considerably and some would like more visits and contacts with the Command. The Command is planning initiatives to increase stakeholder participation in the future, particularly for federal agencies and departments and key allies, but this increase will depend on agency-resource and national-security considerations. Participation Has Grown and Continues to Expand: As the program gradually evolved, the Joint Forces Command solidified a process to involve the military services, the combatant commands, and other DOD organizations in the planning and execution of its joint experimentation activities. Because future joint operations will involve diplomatic, information, and economic actions, as well as military operations, many DOD, federal, and private organizations and governments participate and provide input into the joint experimentation program (see table 1). The Joint Forces Command functions as a facilitator to solicit and coordinate the involvement of these organizations and incorporate their input, as appropriate, into concept development and experimentation activities. Table 1: Organizations Involved in Joint Experimentation: Organizational area: DOD headquarters and support organizations; Examples of organizations and government entities: * Office of the Secretary of Defense; * Chairman of the Joint Chiefs of Staff and Joint Staff; * Office of Force Transformation; * Joint Advanced Warfighting Program; * Defense Intelligence Agency; * National Defense University. Organizational area: Military services; Examples of organizations and government entities: * Army; * Navy; * Air Force; * Marine Corps. Organizational area: Combatant commands; Examples of organizations and government entities: * U.S. Central Command; * U.S. European Command; * U.S. Pacific Command; * U.S. Southern Command; * U.S. Special Operations Command; * U.S. Transportation Command; * U.S. Space Command; * U.S. Strategic Command. Organizational area: Federal agencies and departments; Examples of organizations and government entities: * Department of State; * Department of Justice; * Central Intelligence Agency. Organizational area: Private sector; Examples of organizations and government entities: * Academia; * Think tank groups; * Industry. Organizational area: Allies and potential coalition partners; Examples of organizations and government entities: * Some North Atlantic Treaty Organization countries; * Australia. Source: GAO interviews and DOD data. [End of table] Because the stakeholders determine the extent of their participation in the program, it can vary considerably. However, Joint Forces Command officials stated that participation by the services, the combatant commands, and other DOD organizations has grown steadily since the program was created and continues to grow, as participants become increasingly aware of the strong emphasis that DOD leaders are placing on experimentation. For example, in contrast to the first field experiment in 2000, which had limited involvement by the services, this year’s Millennium Challenge has seen the services more actively involved in early planning, and their individual experiments better coordinated and integrated into the field experiment. Our comparison of participation in the Command’s major field experiment in 2000 with plans for this year’s experiment found a significant increase in the diversity and number of participating organizations and in the number of concepts and initiatives proposed by these organizations. For example, the total number of organizations participating in Millennium Challenge 2002 more than doubled from the prior experiment in 2000 (from 12 to 29 organizations), and the total number of service initiatives increased from 4 to 29. The Command provides several ways for organizations to participate and provide inputs: they can review program plans and strategies; attend meetings, seminars, and workshops; take part in experimentation activities; and use various communication tools such as E-mail, Internet, and video conferencing. Additionally, the Command obtains input from the various experimentation and research and development organizations of the military services and of some combatant commands and DOD organizations.[Footnote 10] The Command also considers the results of Advanced Concept Technology Demonstrations efforts,[Footnote 11] innovations, and recent military operations in developing its program. For example, as a result of its operational experiences in Kosovo, the U.S. European Command identified various joint capability shortfalls in its recent list of Command priorities as a means of guiding the Joint Forces Command in selecting focal areas and activities for experimentation. Further, the Command is taking steps to (1) align its experimentation activities with the schedules of major service and combatant command exercises and (2) adjust its program to allow for earlier consideration of new concepts proposed by the services and the combatant commands in the input process. These adjustments would improve synchronization of experiments with the availability of forces and the training schedules of the services and the combatant commands, allow for greater involvement of these entities in the process, and increase the likelihood that joint requirements are sufficiently considered early in the development of concepts. Participating organizations also provide input during the annual preparation of two key joint experimentation-program documents: the Chairman of the Joint Chiefs of Staff’s guidance on joint experimentation and the Joint Forces Command’s Joint Concept Development and Experimentation Campaign Plan[Footnote 12] (see fig. 2). Each year the Chairman provides guidance to the Joint Forces Command to use in developing its Campaign Plan for joint concept development and experimentation. The basis for the Chairman’s guidance is derived from several sources, including strategy and planning documents, studies, and other assessments.[Footnote 13] Additionally, key DOD stakeholders, including the Chairman’s Joint Warfighting Capability Assessment teams[Footnote 14] and the Joint Requirements Oversight Council, provide input to the Joint Staff to use in developing the Chairman’s guidance. The Joint Forces Command uses this guidance, with additional input from DOD stakeholders, in preparing its Campaign Plan, which is the primary vehicle for synchronizing its joint experimentation activities and coordinating resources. The Command also solicits and considers input for the Campaign Plan from some other federal agencies and departments, academia, private sector, and allies. After review and endorsement by the combatant commands, the services, and the Joint Requirements Oversight Council, the Chairman approves the Campaign Plan. Figure 2: Involvement of Organizations in Preparing Joint Experimentation Guidance and Campaign Plan: [See PDF for image] Source: Joint Staff and Joint Forces Command. [End of figure] Stakeholders Generally Satisfied but Point to Some Possible Improvements: Officials at the military services, the combatant commands, and other DOD organizations we talked with said they were generally satisfied with the opportunities for input provided by the Joint Forces Command. At the same time, DOD stakeholders have taken various actions to increase their participation. Some, however, would like more contacts and communication with the Command. The Command is responding with some initiatives. Each service, the Joint Staff, the U.S. Special Operations Command, the U.S. Space Command, as well as some DOD and federal agencies (such as the National Imagery and Mapping Agency and the National Security Agency) have assigned liaison officers at the Joint Forces Command.[Footnote 15] However, officials at the Central, Pacific, and Southern Commands stated that their staffing levels currently do not allow them to devote personnel in this role. Combatant command officials indicated that the frequency and number of meetings, conferences, and other events held at the Joint Forces Command often make it difficult for their organizations to attend. The officials believe that as a result, the views and positions of their organizations are not always fully captured in some discussions and deliberations. Some of the combatant commands have or are planning to establish their own joint experimentation offices. Officials from the Pacific and Special Operations Commands stated that although their respective joint experimentation offices are largely focused on supporting their own experimentation efforts, the offices provide a cadre of staff who can better coordinate and participate more consistently in the Joint Forces Command’s joint experimentation program. For example, Pacific Command officials said that their own experimentation efforts to improve the command of joint operations over the past few years have contributed to joint experimentation by providing significant insights for the Joint Forces Command’s development of the standing joint-force headquarters concept. Central Command and Southern Command officials said their Commands have plans to establish similar offices soon. While satisfied with their participation and their ability to provide input into the program, officials at some combatant commands believe that a number of things could be done to improve the program, assuming resources are available. They believe that the Joint Forces Command could increase its visits to and participation in combatant-command activities. Some of the officials also believe that if the Joint Forces Command assigned liaison officers to their commands, the Command would obtain first-hand knowledge and a better appreciation of the various commands’ individual requirements. These officials believe that such a presence at their commands would demonstrate the Joint Forces Command’s commitment to joint experimentation and would allow for interaction with staff throughout their commands. The Joint Forces Command does not favor doing this because of the cost and the difficulty in providing the staff necessary to fulfill this role. Officials at the Pacific, Central, and Southern Commands also believe that some level of funding should be provided to the combatant commands for their use in supporting individual command and the Joint Forces Command experimentation efforts. Combatant command officials stated that currently, funds from other command activities must be diverted to support these efforts. Out of concern about the need to improve communications and participation in joint experimentation planning, the Joint Forces Command is planning some initiatives such as the following: * It plans to create a virtual planning-center site for joint experimentation on its Intranet to provide DOD stakeholders with easily accessible weekly updates to information on planned experiments; participants; goals and objectives; and ongoing experimentation by the Joint Forces Command, the services, the combatant commands, and DOD agencies. It plans to develop the requirements for the site during fall 2002 and to initiate the project soon after. * It established Project Alpha--a “think-tank” group--in early 2002 to provide another source of input and outputs. The project will interface with researchers throughout DOD, Department of Energy national laboratories, private industry, and academia to find cutting-edge technologies for inclusion in service and joint experimentation. This relationship will provide an opportunity for the Joint Forces Command to leverage the work of these organizations and similarly, for these organizations to gain a better understanding of and include their work in the joint experimentation program. Expanding Interagency and Foreign Participation in Experimentation Events Is Being Considered: As the joint experimentation program matured, participation by non-DOD federal agencies and departments gradually increased. Participation, however, depends upon the agencies’ desire to be involved and their available resources. Lack of involvement could lead to missed opportunities. And participation by allies and coalition partners has been limited by security concerns. The Joint Forces Command’s input process allows individual federal agencies and departments, such as the Departments of State and Justice, to participate in joint experimentation events as they choose. Interagency participation is improving, according to Command officials. For example, federal agencies and departments are participating in Millennium Challenge 2002 to assist the Command in developing its standing joint-force headquarters concept. However, resource and staffing constraints prevent some agencies and departments from taking part in experiments. For example, according to a Joint Forces Command official, the Department of Transportation and the Central Intelligence Agency decided not to send representatives to Millennium Challenge 2002 because of staffing constraints. Not only could non-DOD agencies provide important insights and contributions to joint operations, but also some important opportunities could be missed if these agencies do not consistently participate in joint experimentation events. While federal agencies and departments are beginning to increase their role in joint experimentation, several service and combatant command officials we spoke with believe that greater involvement is needed because of the role these organizations are likely to have in future joint operations. For example, these non-DOD federal agencies and departments would provide support (economic, diplomatic, and information actions) to U.S. military forces in their conduct of operations aimed at defeating an adversary’s war-making capabilities--support that is critical to implementation of the Joint Forces Command’s rapid decisive operations concept. Several DOD (service, combatant command, Office of the Secretary of Defense, and other DOD organizations) officials we spoke with believe that the Joint Forces Command should explore ways to boost the participation and involvement of allies and coalition partners in joint experimentation. Joint Forces Command officials agree and believe that such cooperation would foster a better understanding of allied perspectives, allow the Command to leverage concept development work, expand available capabilities, and facilitate the development of multinational capabilities. The Command recently created a multinational concept-development and experimentation site on its Intranet to facilitate the involvement of allies and coalition partners in joint experimentation. However, some DOD officials believe that the Joint Forces Command should do more because future U.S. military operations will likely be conducted with other countries. The officials stress that other nations’ military personnel should be included in experiments to develop new operational concepts, if these concepts are to be successful. Joint Forces Command officials pointed out, however, that the participation and involvement of other countries are often constrained by restrictions on access to sensitive security information. For example, North Atlantic Treaty Organization countries only participated as observers in Millennium Challenge 2002 because of security information restrictions. The Command, however, plans to develop ways to better handle these restrictions to allow greater participation by other nations in its next major field experiment in 2004. No Recommendations for Change Have Been Approved or Implemented: Nearly 4 years after the program was established, only three recommendations have flowed from the joint experimentation program, and none of them have been approved. Confusion about proposed changes in guidance regarding the information required for submitting these recommendations has partly delayed their approval. At the time we concluded our review, official guidance on what information should accompany joint experimentation recommendations had not been approved. In addition, several DOD officials expressed concern that the process used to review and approve recommendations, the same as that used for major acquisition programs, may not be the most appropriate for a program whose aim is to integrate changes quickly. However, the officials could not pinpoint any specific impasses in the approval process. The DOD officials are also concerned about potential delays in the integration of new concepts because of the lengthy DOD resource allocation process. Concerns Raised about the Delays in Approving Recommendations: The Joint Forces Command submitted one recommendation to the Chairman of the Joint Chiefs of Staff in August 2001 and two more in November 2001 (see table 2). At the time we ended our review, none of the recommendations had been approved. The recommendations to improve the planning and decision-making capabilities of joint forces and provide better training for personnel conducting theater missile defense operations were based on analyses of results of experiments carried out in the first 3 years of joint experimentation. Inputs included two major experiments: Millennium Challenge 2000 (live field experiment in August-September 2000) and the Unified Vision 2001 (virtual simulation experiment in May 2001). The first recommendation was submitted for review just 3 months after the end of the last experiment. Table 2: First Three Recommendations Submitted by Joint Forces Command: Recommendation: Collaborative environment and collaborative tools; Description: Develop deployable Internet-like tools to improve the collaborative planning and decision-making capabilities of joint forces.; Changes required[A]: * Joint doctrine; * Training; * Leadership and education; * Personnel; Date submitted for review: August 18, 2001. Recommendation: Joint intelligence preparation of the battle space; Description: Develop objectives to train and exercise joint forces on intelligence requirements for preparing the battle space, with a focus on theater missile defense.; Changes required[A]: * Joint doctrine; * Training; * Leadership and education; Date submitted for review: November 1, 2001. Recommendation: Theater missile defense and time-critical targeting team training; Description: Develop a training course to improve the effectiveness of teams involved in theater missile defense and time- critical targeting activities.; Changes required[A]: * Training; * Leadership and education; * Personnel; * Facilities; Date submitted for review: November 1, 2001. [A] Recommendations can propose changes be made to doctrine, organizations, training, materiel, leadership and education, personnel, and/or facilities. Source: Joint Forces Command. [End of table] According to a Joint Staff official, however, approval of the recommendations has been delayed because Joint Forces Command and Joint Staff officials were confused about proposed changes in guidance. In May 2001, the Joint Requirements Oversight Council proposed new guidance, which would require that information on costs and timelines be included in joint experimentation recommendations. Prior guidance did not require such information. Although the recommendations went through preliminary review by the Joint Staff, the omission was not caught until the recommendations were to be scheduled for review by the Joint Requirements Oversight Council. Joint Forces Command officials told us that they were not aware of the change in guidance until that time. When we ended our review, Joint Forces Command officials were working with the Joint Staff to assess how much data could be prepared and when. Command officials said that the recommendations will be resubmitted in fall 2002 together with other recommendations emerging from Millennium Challenge 2002. As a result, no recommendations have yet been reviewed or approved. Also, at the time we ended our review, the draft guidance on joint experimentation recommendations had not been approved and issued. This guidance will become especially important because joint experimentation is expected to produce new recommendations more rapidly as the program matures. The requirement for costs and timeline data is consistent with that of recommendations for major weapon-system-acquisition programs. However, joint experimentation officials at the Joint Forces Command believe that requiring this type of information on joint-experimentation recommendations may not be appropriate because (1) these recommendations are generally intended to convince decision makers to develop particular joint capabilities, not specific weapon systems; (2) the new requirement may slow the preparation of future recommendations; and (3) it will be difficult to provide accurate estimates of costs and timelines for recommendations that span further into the future. It is too early to determine whether these concerns are valid. Concerns Raised about the Process for Funding Implementation of Recommendations: Some DOD officials were also concerned that the system currently used to allocate resources to implement joint-experimentation recommendations--DOD’s Planning, Programming, and Budgeting System-- may not be the most efficient because it usually takes a long time to review, approve, and provide funding in future budgets. A recommendation approved in 2002, for example, would not be incorporated into DOD’s budget until 2004 or even later. This delay could result in missed opportunities for more rapid implementation. A Joint Staff official told us that the Joint Staff and the Joint Forces Command recently adjusted the timing of events to better align the joint experimentation process with the Planning, Programming, and Budgeting System. Additionally, DOD established a special fund for the Joint Forces Command to use as a temporary funding source to speed up the implementation of certain critical or time-sensitive recommendations. This source will provide early funding for implementation until funding is provided through DOD’s Planning, Programming, and Budgeting System. However, Joint Forces Command and other DOD officials believe other ways to implement new joint capabilities within the framework of existing budget and oversight practices may need to be considered. Some Key Management Elements Are Missing: DOD has been providing more specific and clearer guidance on its goals, expectations, and priorities for the joint experimentation program. Nevertheless, the management of joint experimentation is missing a number of key elements that are necessary for program success: some roles and responsibilities have not yet been defined; current performance measures are not adequate to assess progress; and the Joint Forces Command lacks strategic planning tools for the program. Guidance More Clearly Defined: DOD officials stated that the joint experimentation program had difficulty in its first years because guidance was evolving and was not specific: DOD’s transformation goals were not adequately linked to transformation efforts, and roles and responsibilities were not clearly defined.[Footnote 16] Over time, the Secretary of Defense and the Chairman of the Joint Chiefs of Staff have provided more specific guidance on the goals and expectations for joint experimentation and its contribution to DOD’s transformation efforts. Guidance for joint experimentation has evolved gradually over the program’s nearly 4-year life span, partly because of shifting defense priorities and lack of clarity about the roles of various DOD stakeholders. Roles and responsibilities have also matured with the program.[Footnote 17] The Secretary of Defense’s 2001 Quadrennial Defense Review Report[Footnote 18] established six transformation goals, which include improving U.S. capabilities to defend the homeland and other bases of operations, denying enemies sanctuary, and conducting effective information operations. According to DOD officials, the Secretary of Defense’s most recent planning guidance tasked the Joint Forces Command to focus its experimentation on developing new joint operational concepts for these goals. To begin meeting these goals, the Chairman has also provided the Joint Forces Command with clarifying guidance that identified specific areas for the Command to include in its experimentation, such as the development of a standing joint-force headquarters concept and of a prototype to strengthen the conduct of joint operations. The Command has reflected this new guidance in its latest Joint Concept Development and Experimentation Campaign Plan. Additionally, the Secretary of Defense reassigned the Command’s geographic responsibilities to focus it more clearly on its remaining missions, particularly transformation and joint experimentation. DOD officials at both headquarters and the field believe that the recent guidance begins to provide a better framework for the Joint Forces Command to establish and focus its joint experimentation efforts. Some officials, however, believe that future guidance should further clarify the link between joint experimentation and DOD priorities and the required resources necessary to support joint experimentation. DOD, in its comments to a draft of this report, stated that it expects the Transformation Planning Guidance--currently being prepared by the Office of the Secretary of Defense--will establish the requirements necessary to link experimentation to changes in the force. Role of Office of Force Transformation Is Not Clear: While roles and responsibilities for DOD organizations are now broadly defined, the new DOD Office of Force Transformation’s role in joint experimentation and its relationship to other stakeholders have not yet been clearly established. The Office’s charter or terms of reference have not been released. DOD plans to issue a directive later this year that will include a charter and description of the Office’s authorities and responsibilities. However, there is still uncertainty about the extent of authority and involvement the Office will have in the joint experimentation program and the Office’s ability to link the program with DOD’s overall transformation efforts. Joint Forces Command and other DOD officials consider having a transformation advocate in the Office of the Secretary of Defense as a beneficial link between the Joint Forces Command’s, the services’, and the combatant commands’ joint experimentation programs and DOD’s overall transformation agenda. According to DOD’s 2001 Quadrennial Defense Review Report, the Office of Force Transformation, created in November 2001, is to play a role in fostering innovation and experimentation and should have an important responsibility for monitoring joint experimentation and for providing the Secretary of Defense with policy recommendations. An Office of Force Transformation official told us that the Office will be an advocate for transformation and will help develop guidance and make recommendations on transformation issues to the Secretary of Defense (the Office provided comments on the Secretary’s annual planning guidance and developed instructions for the services on preparing their first transformation road maps). The Office has also decided to take a cautious approach in carrying out its mission because of possible resistance from other DOD organizations, the same official said. The Office plans to offer its assistance to DOD organizations in their transformation efforts and attempt to influence their thinking on key issues, rather than asserting itself directly into their efforts, for example by funding military use of existing private-sector technology to act as a surrogate for evaluating possible concepts, uses, and designs. Joint Forces Command officials stated that as of May 2002, they had had only limited discussions with the Office and had not established any working agreements on how the Office would participate in the joint experimentation program. The Office of Force Transformation has only recently assembled its staff and is beginning to plan its work and establish contacts within DOD and with other organizations. The Office’s budget for fiscal years 2002 and 2003 is about $18 million and $35 million, respectively. Current Performance Measures Are Not a Useful Assessment Tool: DOD’s performance measures (or metrics) for assessing joint experimentation--by measuring only the number of experiments carried out--do not provide a meaningful assessment of the program’s contribution toward meeting its performance goal for military transformation because they are only quantitative. Consistent with good management practices and in order to effectuate the purposes of the Government Performance and Results Act of 1993,[Footnote 19] federal agencies devise results-oriented metrics that provide an assessment of outcomes or the results of programs as measured by the difference they make. In its fiscal year 2000 performance report,[Footnote 20] the most recent it has issued, DOD described the performance indicators for the joint experimentation program in terms of the number of experiments conducted against a target goal for the prior, current, and following fiscal years. In fiscal year 2000, DOD exceeded its target number of experiments and did not project any shortfalls in meeting its target in the next fiscal year. Although this measure does provide a quantitative assessment of experimental activity, it does not provide a meaningful method for assessing how joint experimentation is helping to advance military transformation. An Office of the Secretary of Defense official stated that DOD recognizes that better performance measures are needed for assessing how joint experimentation advances transformation and for two other metrics currently used to assess its military transformation goal.[Footnote 21] The official stated that developing such measures is a challenge because joint experimentation does not easily lend itself to traditional measurement methods. For example, most programs consider a failure as a negative event, but in joint experimentation, a failure can be considered as a success if it provides insights or information that is helpful in evaluating new concepts or the use of new technologies. An Office of the Secretary of Defense official told us that the RAND Corporation and the Institute for Defense Analyses recently completed studies to identify possible performance measures for assessing the progress of transformation. DOD is evaluating them and is preparing the Transformation Planning Guidance to provide more specific information on the priorities, roles, and responsibilities for executing its transformation strategy. The same official stated that the new guidance will include a discussion of the types of performance measures needed for assessing transformation progress or will assign an organization to determine them. In either case, measures will still need to be developed and implemented. DOD plans to issue the new guidance later in 2002 but has not determined how new performance measures would be incorporated into its annual performance report. Joint Forces Command’s Program Lacks Strategic Planning Tools: The Joint Forces Command has not developed the strategic planning tools--a strategic plan, an associated performance plan, and performance-reporting tools--for assessing the performance of the joint experimentation program. Strategic planning is essential for this type of program, especially considering its magnitude and complexity and its potential implications for military transformation. Such planning provides an essential foundation for defining what an organization seeks to accomplish, identifies the strategy it will use to achieve desired results, and then determines--through measurement--how well it is succeeding in reaching results-oriented goals and achieving objectives. Developing strategic-planning tools for the joint experimentation program would also be consistent with the principles set forth in the Government Performance and Results Act of 1993, which is the primary legislative framework for strategic planning in the federal government. The Joint Forces Command prepares an annual Joint Concept Development and Experimentation Campaign Plan that broadly describes the key goals of its program, the strategy for achieving these goals, and the planned activities. However, a February 2002 progress report, prepared by the Joint Forces Command’s Joint Experimentation Directorate, on the development of the Directorate’s performance management system indicated that one-fourth of those organizations providing feedback[Footnote 22] on the Campaign Plan believed that the Plan lacks specificity in terms of the program’s goals and objectives and an associated action plan that outlines the activities to be carried out in order to achieve those goals. Officials we spoke with at the military services, the combatant commands, and the Joint Forces Command all cited the need for more specific and clearer goals, objectives, and performance measures for the program. In the progress report, the Command acknowledged the benefits of strategic planning and the use of this management tool to align its organizational structure, processes, and budget to support the achievement of missions and goals. The report proposed that the Command develop a strategic plan, possibly by modifying its annual Campaign Plan, and subsequently prepare a performance plan and a performance report. Command officials indicated that the basic requirements of a strategic plan could be incorporated into the Campaign Plan, but they were unsure, if such an approach were taken, whether the changes could be made before the annual Campaign Plan is issued later this year. Similarly, the Joint Forces Command has had difficulty in developing specific performance measures for joint experimentation. A Command official stated that the Command has tried to leverage the performance measures developed by other organizations like itself, but found that there is widespread awareness throughout the research and development community, both within and outside DOD, that such measures are needed but do not exist. Additionally, a Joint Forces Command official stated that whatever metrics the Command develops must be linked to its mission-essential tasks for joint experimentation and that the Command is currently developing these tasks. At the time we ended our review, the Command had identified six broad areas for which specific metrics need to be developed. These included quality of life, customer relationships, and experimentation process management. Conclusions: After nearly 4 years, the Joint Forces Command’s process for obtaining inputs for the development and execution of DOD’s joint experimentation program has become more inclusive. However, questions continue about whether the program is the successful engine for change envisioned when it was established. Since the program’s inception, only three recommendations have flowed from experimentation activities and their review, approval, and implementation have been delayed from confusion over a change in guidance that required additional information be included in the recommendations. As a result, no recommendations for change have been approved or implemented to date. To the extent that the draft guidance on what should be submitted with joint experimentation recommendations can be officially approved and issued, future recommendations could be submitted for approval and implementation more quickly. Underscoring the need to finalize the guidance is the anticipated recommendations to be made after this year’s major field experiment, Millennium Challenge 2002. The lack of strategic planning for joint experimentation deprives the Joint Forces Command of necessary tools to effectively manage its program. Implementation of strategic planning at the Joint Forces Command would create a recurring and continuous cycle of planning, program execution, and reporting and establish a process by which the Command could measure the effectiveness of its activities as well as a means to assess the contributions of those activities to the operational goals and mission of the program. Such planning could also provide a tool--one that is currently missing--to identify strengths and weaknesses in the development and execution of the program and a reference document for the effective oversight and management of the program. Performance measures developed under the Command’s strategic planning could provide the standard for assessing other experimentation efforts throughout DOD, which are also lacking such metrics. The lack of a meaningful performance measure for assessing the contribution of the joint experimentation program to advance DOD’s transformation agenda limits the usefulness and benefit of this management tool to assist congressional and DOD leaders in their decision-making responsibilities. Establishing a “meaningful” joint experimentation performance measure for its annual performance report would provide congressional and DOD leadership a better assessment of the program’s contribution and progress toward advancing transformation. Such a metric would also be consistent with the intent of the Results Act to improve the accountability of federal programs for achieving program results. Because the role and relationships of the Secretary of Defense’s new Office of Force Transformation have not yet been clarified, the Secretary may not be effectively using this office in DOD’s transformation efforts. This office, if given sufficient authority, could provide the Secretary with a civilian oversight function to foster and monitor the joint experimentation program to ensure that it is properly supported and provided resources to advance the DOD’s overall transformation agenda. Rectifying these shortcomings is critical in view of the importance that DOD has placed on joint experimentation to identify the future concepts and capabilities for maintaining U.S. military superiority. Recommendations for Executive Action: To improve the management of DOD’s joint experimentation program, we recommend that the Secretary of Defense direct the Chairman of the Joint Chiefs of Staff to: * approve and issue guidance that clearly defines the information required to accompany joint experimentation recommendations for the Joint Requirements Oversight Council’s review and approval and: * require the Commander in Chief of the U.S. Joint Forces Command to develop strategic planning tools to use in managing and periodically assessing the progress of its joint experimentation program. We further recommend that the Secretary of Defense: * develop both quantitative and qualitative performance measures for joint experimentation in DOD’s annual performance report to provide a better assessment of the program’s contribution to advancing military transformation and: * clarify the role of the Office of Force Transformation and its relationship to the Chairman of the Joint Chiefs of Staff, the Joint Forces Command, and other key DOD stakeholders in DOD’s joint experimentation program. Agency Comments and Our Evaluation: We received written comments from DOD on a draft of this report, which are included in their entirety as appendix III. DOD agreed with our recommendations and indicated that it expects that a forthcoming Transformation Planning Guidance and subsequent guidance will be responsive to them by clarifying roles and missions across DOD, implementing recommendations for changes, and establishing clear objectives. We believe such strategic guidance from the Secretary of Defense could provide a significant mechanism for better linking and clarifying the importance of the joint experimentation program with DOD’s transformation agenda. DOD also provided technical comments to the draft that were incorporated in the report where appropriate. Scope and Methodology: To determine the extent to which the Joint Forces Command obtains input from stakeholders and other relevant sources in developing and conducting its joint experimentation activities, we reviewed an array of documents providing information about participants in joint experimentation, including guidance and other policy documents, position papers, fact sheets, reports, and studies of the military services, the combatant commands, the Joint Staff, and other DOD organizations. We also reviewed Joint Forces Command plans and reports. Additionally, we made extensive use of information available on public and DOD Internet web sites. To assess the change in participation by various stakeholders over time, we compared the differences in the numbers of participating organizations and initiatives provided by these organizations between the Joint Forces Command’s first two major field experiments in 2000 and 2002 (Millennium Challenge 2000 and Millennium Challenge 2002). We conducted discussions with officials at five combatant commands, the Joint Staff, the military services, and other DOD organizations, such as the Joint Advanced Warfighting Program and the Defense Advanced Research Projects Agency. Appendix IV lists the principal organizations and offices where we performed work. At the Joint Forces Command, we discussed with joint experimentation officials the process for soliciting and incorporating inputs for joint experimentation from the military services and the combatant commands. We also attended conferences and other sessions hosted by the Joint Forces Command to observe and learn about joint experimentation participants and their contributions and coordination. For example, we attended sessions for the Command’s preparation of its annual Joint Concept Development and Experimentation Campaign Plan and planning for this year’s Millennium Challenge experiment. With officials from each of the services and the combatant commands, we discussed perceptions of the effectiveness of coordination and participation in joint experimentation. We also obtained observations about participants’ involvement from several defense experts who track joint experimentation and military transformation. Although we did not include a specific assessment of the individual experimentation efforts of the services and combatant commands, we did discuss with service and command officials how their efforts were coordinated and integrated into joint experimentation. We also did not determine the extent that individual inputs obtained from various participating organizations were considered and incorporated into the joint experimentation program. To determine the extent to which recommendations flowing from the joint experimentation process have been approved and implemented, we reviewed and analyzed data that tracked the progress of the first three joint experimentation recommendations submitted by the Joint Forces Command. We also obtained and analyzed relevant guidance and held discussions with Joint Staff, Joint Forces Command, and Office of the Secretary of Defense officials on the Joint Requirements Oversight Council process for reviewing and approving joint experimentation recommendations. We also discussed issues relating to implementation of joint experimentation recommendations through DOD’s Planning, Programming, and Budgeting System. To assess whether key management elements, such as policy, organization, and resources, were in place for the program, we conducted a comprehensive review of current legislative, policy, planning, and guidance documents and reports and studies. We used the principles laid out in the Government Performance and Results Act of 1993 as an additional benchmark for assessing the adequacy of performance measures established for the program and of tools used to manage the program. We also discussed the status and evolution of joint experimentation oversight and management, including office roles and responsibilities and joint experimentation metrics, with officials at the Joint Forces Command, the Joint Staff, the services, the combatant commands, the Office of the Secretary of Defense, the Office of Force Transformation, and other DOD organizations. Several defense experts who follow joint experimentation and military transformation discussed with us joint experimentation oversight and management and gave us their impressions regarding current joint experimentation management practices. Our review was conducted from October 2001 through May 2002 in accordance with generally accepted government auditing standards. We are sending copies of this report to interested congressional committees, the Secretary of Defense, the Chairman of the Joint Chiefs of Staff, and the Commander in Chief, U.S. Joint Forces Command. We will also make copies available to others upon request. In addition, this report will be available at no charge on the GAO Web site at http://www.gao.gov. Please contact Richard G. Payne at (757) 552-8119 if you or your staff have any questions concerning this report. Key contacts and contributors to this report are listed in appendix V. Henry L. Hinton, Jr. Managing Director Defense Capabilities and Management: Signed by Henry L. Hinton, Jr.: [End of section] Appendix I: Chronology of Key Events Important to Joint Experimentation: Date: May 1996; Event: Chairman of the Joint Chiefs of Staff issued Joint Vision 2010.; Relevance to joint experimentation: This vision of future war fighting provides a conceptual template for the Department of Defense’s (DOD) transformation efforts across all elements of the armed forces. Date: May 1997; Event: DOD’s Report of the Quadrennial Defense Review issued.; Relevance to joint experimentation: Report discussed the importance of preparing for future national security challenges. It concluded that DOD needed to institutionalize innovative investigations, such as war-fighting experiments, to ensure future concepts and capabilities are successfully integrated into the forces in a timely manner. Date: May 1998; Event: Secretary of Defense designated Commander in Chief, U.S. Joint Forces Command,[A] as executive agent for joint experimentation.; Relevance to joint experimentation: The Secretary of Defense tasked the Joint Forces Command to design and conduct joint war-fighting experimentation to explore, demonstrate, and evaluate joint war-fighting concepts and capabilities. Date: April 1998; Event: Joint Advanced Warfighting Program established.; Relevance to joint experimentation: DOD established the program at the Institute for Defense Analyses to serve as a catalyst for achieving the objectives of Joint Vision 2010 (and later Joint Vision 2020). To that end, the program is to develop and explore breakthrough operational concepts and capabilities that support DOD’s transformation goals. Date: October 1998; Event: Joint Forces Command assumed responsibility as the executive agent for joint experimentation.; Relevance to joint experimentation: Joint concept development and experimentation program initiated. Date: June-August 1999; Event: Joint Advanced Warfighting Program conducted the first joint experiment for Joint Forces Command.; Relevance to joint experimentation: An experiment--J9901--that investigated approaches for attacking critical mobile targets. Experiment allowed the Joint Forces Command to begin its learning process on how to conduct joint experimentation. Date: September 1999; Event: Report of the Defense Science Board Task Force on DOD Warfighting Transformation issued.; Relevance to joint experimentation: Report proposed several recommendations to promote military transformation. Date: June 2000; Event: Chairman of the Joint Chiefs of Staff issued Joint Vision 2020.; Relevance to joint experimentation: Updated vision statement described the joint war-fighting capabilities required through 2020. Date: August-September 2000; Event: Millennium Challenge 2000 major field experiment conducted.; Relevance to joint experimentation: The first major field experiment coordinated by the Joint Forces Command among the services and other stakeholders. Date: April 2001; Event: Chairman of the Joint Chiefs of Staff issued updated Joint Vision Implementation Master Plan.; Relevance to joint experimentation: Guidance described the process for generation, coordination, approval, and implementation of recommendations emerging from joint experimentation and defined the roles and responsibilities of DOD stakeholders. Date: April 2001; Event: Transformation Study Report: Transforming Military Operational Capabilities issued.; Relevance to joint experimentation: Study conducted for the Secretary of Defense to identify capabilities needed by U.S. forces to meet the twenty-first century security environment. Made several recommendations directed at improving joint experimentation. Date: May 2001; Event: Joint Forces Command conducted Unified Vision 2001 experiment.; Relevance to joint experimentation: A major joint experiment--largely modeling and simulation--conducted to refine and explore several war-fighting concepts, such as “rapid decisive” operations. Date: August 2001; Event: Secretary of Defense’s planning guidance issued.; Relevance to joint experimentation: Required studies by defense agencies and the Joint Staff to develop transformation road maps and a standing-joint-force headquarters prototype. Date: September 2001; Event: DOD’s Quadrennial Defense Review Report issued.; Relevance to joint experimentation: The report established priorities and identified major goals for transforming the Armed Forces to meet future challenges. It called for new operational concepts, advanced technological capabilities, and an increased emphasis on joint organizations, experimentation, and training. Date: November 2001; Event: Chairman of the Joint Chiefs of Staff issued joint experimentation guidance.; Relevance to joint experimentation: The guidance directed the Joint Forces Command to focus its near-term experimentation on developing a standing joint force headquarters prototype. Date: November 2001; Event: Office of Force Transformation established.; Relevance to joint experimentation: Office assists the Secretary of Defense in identifying strategy and policy, and developing guidance for transformation. Date: April 2002; Event: Unified Command Plan 2002 issued.; Relevance to joint experimentation: Plan reduced the number of missions assigned to the Joint Forces Command to allow the Command to devote more attention to its remaining missions such as joint experimentation. Date: May 2002; Event: Secretary of Defense’s planning guidance issued.; Relevance to joint experimentation: The guidance directed the Joint Forces Command to develop new joint concepts that focus on the six transformation goals set forth in the 2001 Quadrennial Defense Review Report. Date: July-August 2002; Event: Joint Forces Command conducted Millennium Challenge 2002.; Relevance to joint experimentation: Second major field experiment conducted to culminate a series of experiments to assess “how” to do rapid decisive operations in this decade. [A] In October 1999, the U.S. Atlantic Command was redesignated as the U.S. Joint Forces Command. [End of table] [End of section] Appendix II: Additional Information on Joint Concept Development and Experimentation Activities: The Joint Forces Command uses various types of assessment activities to develop, refine, and validate joint concepts and associated capabilities. As shown in figure 3, the Command begins to move through the five joint concept development phases by conducting workshops, seminars, and war games to develop information and identify possible areas to explore in developing new concepts and associated capabilities and then uses simulated or live experiment events to confirm, refute, or modify them. These activities vary in scale and frequency, but each activity becomes larger and more complex. They can involve a small group of retired flag officers and academics, up to 100 planners, operators, and technology experts, or several thousand in the field. Near the end of the process, the Command will conduct a large-scale simulation experiment (such as Unified Vision 2001), followed by a major field experiment (such as Millennium Challenge 2002). The process continuously repeats itself to identify additional new concepts and capabilities. Figure 3: Joint Concept Development and Experimentation Activities: [See PDF for image] Source: Joint Forces Command. [End of figure] Table 3 provides additional information about the characteristics, scale, and frequency of these and other associated activities and experiments. Table 3: Types of Activities and Experiments Associated with Joint Experimentation: Type of activities and experiments: Workshop; Characteristics: Brief, intense experimentation program used to discover and generate concepts. Participants identify focus areas and research topics. Workshops are conducted live and may have virtual components. Subject matter experts participate.; Scale and frequency: Small scale Many each year. Type of activities and experiments: Seminar; Characteristics: Discussion forums used to discover and define problem boundaries, pose solutions, and exchange information. Seminars are conducted live, and may have virtual components. Subject matter experts participate.; Scale and frequency: Small scale Many each year. Type of activities and experiments: War game; Characteristics: An event, using simulated and emulated systems, centering on the interaction of two or more opposing forces depicting a possible or actual scenario. Identifies key variables in an experiment; refines concepts and assesses alternatives.; Scale and frequency: Scale varies One or more each year. Type of activities and experiments: Limited objective experiment; Characteristics: An event, limited in scope, that permits participants to see how specific, focused ideas or concepts will work in a given context and works with supporting elements of the integrating concept. It confirms or refines experiments and is primarily virtual or simulated. May include workshops, seminars, and war games.; Scale and frequency: Small scale Varies (6 in 2001). Type of activities and experiments: Leveraged event; Characteristics: Service or other organization’s event in which the Joint Forces Command and its partners share lessons learned. The event may involve the use of live forces in a theater scenario.; Scale and frequency: Scale varies Frequency varies. Type of activities and experiments: Spiral; Characteristics: A group of smaller events and meetings requiring live participants; concepts are refined and plans are made for larger live Joint Forces Command experiments.; Scale and frequency: Small to medium scale A few each year. Type of activities and experiments: Major simulation experiment; Characteristics: A constructive, virtual, or live experiment that makes extensive use of models and simulation to provide detailed information on the expected outcomes of an event or sequence of events. It may use live and/or simulated participants.; Scale and frequency: Large scale Not more than one every 2 years. Type of activities and experiments: Major field experiment; Characteristics: Integrates several smaller, previous events. Determines how effectively the concepts previously explored in smaller experiments will operate when integrated with the concepts of the other services and combatant commands. It validates concepts and requires live participants as well as simulated opponents.; Scale and frequency: Large scale Not more than one each year. [End of table] [End of section] Appendix III: Comments from the Department of Defense: OFFICE OF THE UNDER SECRETARY OF DEFENSE: 2000 DEFENSE PENTAGON WASHINGTON, DC 20301-2000: POLICY: August 8, 2002: Mr. Henry L. Hinton, Jr. Managing Director, Defense Capabilities and Management: U.S. General Accounting Office Washington, DC 20548: Dear Mr. Hinton: This is the Department of Defense (DoD) response to the GAO draft report, GAO-02-853, “Military Transformation: DOD Actions Needed to Better Manage Its Joint Experimentation Program,” (GAO code 350120). DoD concurs with the report and the GAO’s overall assessment. The enclosure contains the Department’s response to GAO recommendations. The Department provided the GAO separately the substantive and general comments with supporting rationale. In general, the Department believes that the GAO findings are thoroughly researched and accurately reflect both the history and the progress that the Department has made towards improving the joint experimentation process. The report is fair and balanced in that it acknowledges the scope of the task of transformation and the critical importance of the supporting experimentation process. The Department is aggressively pursuing actions that will transform our military forces. Roles and responsibilities for implementing the Department’s transformation strategy, including joint experimentation, will be clarified in the Department’s Transformation Planning Guidance, currently in staffing. The Department appreciates the opportunity to review and comment on the draft GAO report. Sincerely, Christopher J. Lamb: Deputy Assistant Secretary For Plans and Resources (Acting): Signed by Christopher J. Lamb: w/enclosure: GAO DRAFT REPORT DATED JULY 11, 2002 (GAO CODE 350120): “MILITARY TRANSFORMATION: DOD ACTIONS NEEDED TO BETTER MANAGE ITS JOINT EXPERIMENTATION PROGRAM”: DEPARTMENT OF DEFENSE COMMENTS TO THE GAO RECOMMENDATIONS: RECOMMENDATION l: To improve the management of DoD’s joint experimentation program, the GAO recommended that the Secretary of Defense direct the Chairman of the Joint Chiefs of Staff to: *approve and issue guidance that clearly defines the information required to accompany joint experimentation recommendations for the Joint Requirements Oversight Council’s review and approval; and: *require the Commander in Chief of the U.S. Joint Forces Command to develop strategic planning tools to use in managing and periodically assessing the progress of its joint experimentation program. (p. 27/GAO Draft Report): DOD RESPONSE: Concur with the problems identified. The Transformation Planning Guidance (TPG), currently in the staffing process, or subsidiary guidance documents identified in the TPG, will address and clarify both the JROC process issue and the requirement for the implementation of strategic planning tools at JFCOM. RECOMMENDATION 2: The GAO recommended that the Secretary of Defense develop both quantitative and qualitative performance measures for joint experimentation in DoD’s annual performance report to provide a better assessment of the program’s contribution to advancing military transformation. (p. 27/GAO Draft Report): DOD RESPONSE: Concur with the recommendation. The Transformation Planning Guidance, currently in the staffing process, or subsidiary guidance documents identified in the TPG, will address this issue by directing JFCOM to revise its Concept Development and Experimentation Campaign Plan. RECOMMENDATION 3: The GAO recommended that the Secretary of Defense clarify the role of the Force Transformation Office and its relationship to the Chairman of the Joint Chiefs of Staff, the Joint Forces Command, and other key DoD stakeholders in DoD’s joint experimentation program. (p. 27/GAO Draft Report): DOD RESPONSE: Concur with the recommendation. The report discusses several problems and issues with the joint experimentation program. In general these relate to the need to clarity of roles and missions across DOD, the implementation of change recommendations, the necessity for strategic management tools, and the requirement for meaningful performance measures. The Transformation Planning Guidance will address these issues by clarifying roles and missions across the Department, implementing change recommendations, and establishing clear objectives. When coordinated and implemented, the TPG will ensure that all stakeholders are prepared to respond positively to the observations and recommendations listed in the GAO report. [End of section] Appendix IV: Organizations and Offices Contacted: Washington, D.C., Area: Department of Defense: * Office of the Secretary of Defense, Program Analysis and Evaluation: * Office of the Under Secretary of Defense for Policy: * Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics: * Joint Advanced Warfighting Program: * Defense Advanced Research Project Agency: * Office of Force Transformation: The Joint Staff: * Operational Plans and Interoperability Directorate: * Joint Vision and Transformation Division: * Command, Control, Communications, and Computers Directorate: * Force Structure, Resources, and Assessment Directorate: Department of the Army: * Directorate of Training: * Directorate of Integration: * Directorate for Strategy, Concepts, and Doctrine: Department of the Navy: * Office of the Deputy Chief of Naval Operations for Warfare Requirements and Programs: * Marine Corps Combat Development Command: Department of the Air Force: * Wargaming and Experimentation Division: Other Organizations: * Booz Allen Hamilton: * The Carlyle Group: * Center for Strategic and Budgetary Assessments: * Hicks & Associates, Inc. Honolulu, Hawaii, Area: * U.S. Pacific Command: * Joint Experimentation Division: * Intelligence Architecture and Systems Division: * Exercise Division: * Regional Strategy and Policy Division: U.S. Pacific Fleet: * Joint Experimentation Office: U.S. Army Pacific: * Internal Review Office: U.S. Pacific Air Forces: * Operational Requirements: Marine Forces Pacific: Miami, Fla., Area: U.S. Southern Command: * Future Operations: * Programs and Resources Directorate: * Office of Analysis and Simulations: Newport, R.I., Area: Navy Warfare Development Command: Norfolk, Va., Area: U.S. Joint Forces Command: * Joint Experimentation Directorate: * Joint Integration and Interoperability Directorate: Department of the Army: * Training and Doctrine Command: * Army Joint Venture Directorate: * Army Audit Agency: Department of the Air Force: * Air Combat Command: * Air Force Experimentation Office: Joint Forces Staff College: Tampa, Fla., Area: U.S. Central Command: * Futures Division: * Exercise Division: U.S. Special Operations Command: Special Operations Requirements and Resources: * Joint Experimentation Division: * Special Operations Acquisition and Logistics: * Intelligence and Information Operations: [End of section] Appendix V: GAO Contacts and Staff Acknowledgments: GAO Contacts: Sharon L. Pickup (202) 512-9619 Richard G. Payne (757) 552-8119: Acknowledgments: In addition to the individuals named above, Carol R. Schuster, Mark J. Wielgoszynski, John R. Beauchamp, Kimberley A. Ebner, Lauren S. Johnson, and Stefano Petrucci made key contributions to this report. [End of section] FOOTNOTES [1] DOD states that transformation of the military “results from the exploitation of new approaches to operational concepts and capabilities, the use of old and new technologies, and new forms of organization that more effectively anticipate new or still emerging strategic and operational challenges and opportunities and that render previous methods of conducting war obsolete or subordinate.” [2] The Joint Requirements Oversight Council is an advisory council that assists the Chairman of the Joint Chiefs of Staff in identifying and assessing the priorities for joint military requirements to achieve current and future military capabilities. Chaired by the Vice Chairman of the Joint Chiefs of Staff, the Council is comprised of a senior officer from each of the military services. Representatives from other DOD entities, such as the combatant commands and the Joint Staff, serve in an advisory role to the Council. [3] The staff that assists the Chairman of the Joint Chiefs of Staff in executing the Chairman’s responsibilities. [4] Millennium Challenge 2002 focuses on how the U.S. military can use its current and near-term capabilities differently in this decade. It will involve the U.S. Special Operations Command and forces from all the services. Another major field experiment--Olympic Challenge--is planned for 2004. [5] The Unified Command Plan, April 30, 2002, assigns the Chairman of the Joint Chiefs of Staff responsibility for providing overarching guidance on joint experimentation. [6] The Command is directed to develop this prototype headquarters by 2004 and make it available for implementation by the regional combatant commands during fiscal year 2005. The headquarters will provide uniform, standard operating procedures, tactics, techniques, and technical system requirements, with the ability to move expertise among commands. [7] We recently reported on the status of actions taken by the Chairman of the Joint Chiefs of Staff to improve the Joint Requirements Oversight Council’s oversight and the joint requirements process, including establishing procedures for submitting recommendations stemming from joint experimentation to the Council. See U.S. General Accounting Office, Defense Plans: Status of DOD’s Efforts to Improve Its Joint Warfighting Requirements Process, GAO-02-100R (Washington, D.C.: Nov. 9, 2001). [8] The Planning, Programming, and Budgeting System is designed to assist the Secretary of Defense in making choices about the allocation of resources among a number of competing or possible programs and alternatives to accomplish specific national security objectives. The objective of the system is to provide the optimal mix of forces, equipment, and support that can be achieved within fiscal constraints. [9] DOD Directive 5100.3, Support of the Headquarters of Combatant and Subordinate Joint Commands, Nov. 15, 1999, designates the Navy as the executive agent responsible for arranging for the program-funding of the Joint Forces Command’s activities, including the joint experimentation program. [10] Participating organizations would include each of the services’ experimentation programs, service battle labs and supporting research and development organizations, DOD organizations such as Defense Advanced Research Projects Agency and Joint Advanced Warfighting Program, and federally funded research and development organizations such as the Institute for Defense Analyses, the RAND Corporation, and the Center for Naval Analysis. [11] DOD initiated the Advanced Concept Technology Demonstration program to determine the extent to which a given mature technology will improve military capabilities before entering the normal acquisition process and, by using mature technology, reduce the length of time to develop and acquire weapon systems. [12] The Campaign Plan provides the structure for executing experimentation and outlines the Joint Forces Command’s objectives, methods, and events to achieve those objectives. The most recent Campaign Plan is: U.S. Joint Forces Command, The Joint Concept Development and Experimentation Campaign Plan Fiscal Years 2002-2007 (Norfolk, Va.: 2002). [13] Examples of these sources include the President’s National Security Strategy, the Secretary of Defense’s annual defense planning guidance, and the Chairman of the Joint Chief of Staff’s Joint Vision and National Military Strategy. [14] Teams of DOD war-fighting and functional-area experts appraise various aspects of joint war fighting within their respective areas. The Chairman of the Joint Chiefs of Staff uses the teams’ assessments to develop guidance and to support decision making. [15] Additionally, Germany, the United Kingdom, Canada, and Australia have each assigned a liaison officer at the Joint Forces Command. [16] We have pointed this difficulty out in prior transformation- related reports. See U.S. General Accounting Office, Military Transformation: Navy Efforts Should Be More Integrated and Focused, GAO-01-853 (Washington, D.C.: Aug. 2, 2001) and U.S. General Accounting Office, Military Transformation: Army Has A Comprehensive Plan for Managing Its Transformation but Faces Major Challenges, GAO-02-96 (Washington, D.C.: Nov. 16, 2001). [17] The roles and responsibilities for most DOD organizations are broadly defined in the Chairman of the Joint Chiefs of Staff Instruction 3010.02A, Joint Vision Implementation Master Plan, Apr. 15, 2001. [18] Department of Defense, Quadrennial Defense Review Report (Washington, D.C.: 2001). [19] Pub.L. 103-62, sec. 2(b), Aug. 3, 1993, sometimes referred to as the Results Act. [20] Department of Defense, Government Performance and Results Act: Department of Defense Fiscal Year 2000 Performance Report (Washington, D.C.: 2001). As of August 2002, DOD did not submit a performance report to the Congress for fiscal year 2001. [21] DOD’s fiscal year 2000 performance report included three metrics for assessing its transformation of U.S. military forces performance goal: joint experimentation, procurement spending, and defense technology objectives. [22] DOD organizations providing comments to the draft Campaign Plan included various offices and activities of the Army, the Navy, the Air Force, the Marine Corps, the U.S. Special Operations Command, the U.S. Strategic Command, the U.S. Southern Command, the U.S. Pacific Command, and the U.S. European Command. GAO’s Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: