This is the accessible text file for GAO report number GAO-03-861 entitled 'Defense Pilot Programs: DOD Needs to Improve Implementation Process for Pilot Programs' which was released on July 28, 2003. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Committee on Armed Services, U.S. Senate: United States General Accounting Office: GAO: July 2003: Defense Pilot Programs: DOD Needs to Improve Implementation Process for Pilot Programs: GAO-03-861: GAO Highlights: Highlights of GAO-03-861, a report to the Committee on Armed Services, U.S. Senate Why GAO Did This Study: In fiscal years 1999, 2000, and 2003, the Congress authorized pilot programs to help the Department of Defense (DOD) laboratories and test centers explore innovative business partnerships and human capital strategies. Congressional concerns about DOD’s implementation of the pilot programs have been growing. The Congress mandated that GAO review pilot program implementation. GAO (1) identified the pilot initiatives proposed and their current status, (2) examined factors that affected implementation, and (3) assessed implementation challenges the 2003 pilot program faces. What GAO Found: The 1999 and 2000 pilot programs have not worked as intended. Since their inception, 178 initiatives have been proposed by the participating laboratories and test centers but only 4—or 2 percent— were implemented under the pilot programs, as shown below. Participants proposed initiatives covering a variety of areas, including business-like practices, partnerships, and human capital innovations. The pilot programs were not effective because DOD lacked an effective implementation process and proposed human capital initiatives were not consistent with statutory provisions. First, DOD did not provide standardized guidance on proposal requirements, coordinate proposals, or clarify decision-making authority for proposal review and approval. Furthermore, DOD did not designate a strong focal point to provide assistance and advice to participants and advocate process improvements. The lack of a strong focal point exacerbated other process gaps. Second, DOD attorneys advised that the pilot programs did not provide authority to make most of the proposed human capital changes. Implementation of the new 2003 pilot program faces several challenges. First, DOD has not addressed implementation problems. For example, clear guidance is still lacking and decision-making authority is still unclear. Second, the 2003 pilot program provides no change in authority concerning human capital initiatives. Finally, laboratories and test centers may be reluctant to participate. Many participants in the earlier pilots told us they were discouraged by their experience and consequently unwilling to repeat it. [End of section] what GAO Recommends: GAO recommends that by March 31, 2004, the Secretary of Defense (1) inform the Congress of DOD’s objectives regarding human capital and business operations in the laboratories and test centers; (2) develop a process for proposing, evaluating, and implementing human capital and business operations initiatives, regardless whether by the pilot authority or by some other vehicle; and (3) designate a strong focal point to coordinate and facilitate this process. DOD did not concur with GAO’s recommendations. www.gao.gov/cgi-bin/getrpt?GAO-03-861. To view the full product, including the scope and methodology, click on the link above. For more information, contact Catherine Baltzell at (202) 512-8001 or baltzelld@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: Many Initiatives Were Proposed but Few Were Implemented under Pilot Programs: The Pilot Programs Were Not Effective for Two Primary Reasons: The 2003 Pilot Program Faces Implementation Challenges: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Scope and Methodology: Appendix I: Fiscal Year 1999 and 2000 Pilot Program Participants: Appendix II: Comments from the Department of Defense: Table: Table 1: Laboratory and Test Center Pilot Program Proposals: Figure: Figure 1: Status of Proposed Initiatives: Abbreviations: DOD: Department of Defense: DDR&E: Directorate of Defense Research and Engineering: NSPS: National Security Personnel System: OSD: Office of the Secretary of Defense: P&R: Personnel and Readiness: United States General Accounting Office: Washington, DC 20548: July 28, 2003: The Honorable John Warner Chairman The Honorable Carl Levin Ranking Minority Member Committee on Armed Services United States Senate: Over the last decade a number of studies have raised concerns that dwindling budgets and an aging workforce have contributed to serious shortfalls in the infrastructure and capabilities of Department of Defense (DOD) laboratories and test centers. In fiscal years 1999 and 2000, the Congress enacted legislation aimed at helping DOD laboratories and test centers address problems by undertaking pilot programs to explore innovative partnerships and human capital strategies.[Footnote 1] In fiscal year 2003, the Congress extended the 1999 and 2000 pilot programs until 2005 and enacted a new pilot program that runs until 2006. However, congressional concerns about pilot program implementation have been growing. Consequently, the Senate Committee on Armed Services directed us to review the implementation of the 1999 and 2000 pilot programs.[Footnote 2] In response, this report (1) identifies initiatives proposed to date and determines their current status, (2) examines factors that affected implementation of proposed initiatives, and (3) assesses implementation challenges the new 2003 pilot program faces. Results in Brief: The 1999 and 2000 pilot programs have not worked as intended. Since their inception, 178 initiatives have been proposed by the participating laboratories and test centers but only 4--or 2 percent-- have been implemented using the pilot program authorities. Twelve times as many--24 percent--were implemented using other authorities than those provided by the pilot programs. Participating laboratories and test centers proposed initiatives covering a variety of areas, including business-like practices, partnerships with industry and academia, and human capital innovations. In general, laboratories tended to propose initiatives dealing with human capital innovations and test centers focused on business-like practices and partnerships. The pilot programs were not effective because DOD lacked an effective implementation process and proposed human capital initiatives were not consistent with statutory provisions. First, DOD did not provide standardized guidance on proposal requirements, coordinate proposals, or clarify decision-making authority for proposal review and approval. Furthermore, DOD did not designate a strong focal point to provide assistance and advice to participants and advocate process improvements. The lack of a strong focal point exacerbated other process gaps. Second, DOD attorneys advised participants that the 1999 and 2000 pilot programs did not provide authority to make most of the proposed human capital changes. Implementation of the new 2003 pilot program faces several challenges. First, DOD has not addressed implementation problems. For example, clear guidance is still lacking and decision-making authority has not been clarified. Second, the 2003 pilot program provides no change in authority concerning human capital initiatives. DOD officials believe that the human capital management legislation the department recently proposed to the Congress will provide flexibility throughout DOD to make necessary human capital changes, thereby eliminating the need for the pilot programs in this area.[Footnote 3] However, this legislation, if enacted, would still require an implementation process. Finally, laboratories and test centers may be reluctant to participate in the new pilot program. Many participants in the earlier pilots told us they were discouraged by their experience and consequently unwilling to repeat it. We are making recommendations aimed at clarifying how DOD plans to address concerns about the laboratories and test centers and improving the implementation of initiatives proposed for that purpose. In written comments on a draft of this report, DOD stated that it did not concur with our recommendations. Background: The United States has a long history of military research and development. To help conduct and manage this research, DOD has a diverse network of 80 in-house laboratories and 26 test centers. Their missions range from basic scientific research to direct technical support to operational commands. The management, operations, and funding for these disparate laboratories and test centers also vary among the services. Over the past decade, several organizations, panels, and commissions have identified significant personnel and resource problems facing the laboratories and test centers. For example, several studies found that the laboratories needed more flexibility in personnel rules governing the scientific workforce in order to attract and retain staff.[Footnote 4] Similarly, several recent studies identified problems with declines in investment and infrastructure, resulting in outdated facilities and technical equipment.[Footnote 5] To help the laboratories and test centers with these problems, the Congress enacted legislation in fiscal years 1999 and 2000 establishing pilot programs for laboratories and test centers to propose innovative partnerships, business-like practices, and human capital initiatives.[Footnote 6] The 1999 pilot program focused on partnerships and business-like practices, while the 2000 program focused more on human capital initiatives. Together, the two pilot programs authorized the Secretary of Defense to provide one laboratory and one test center in each service the authority to: * explore innovative methods for partnering with universities and private sector entities to conduct defense research and development; * attract a workforce balance between permanent and temporary personnel and with an appropriate skill and experience level; * develop or expand innovative methods of operation that provide more defense research for the dollar; and: * waive any restrictions on these methods that are not required by law. A total of 10 laboratories and test centers from all 3 services participated in the pilot programs. They are listed in appendix I. Both programs were authorized for 3 years. The 1999 pilot expired in March 2002; the 2000 pilot, in March 2003. For both programs, DOD was required to submit preliminary and final reports to the Congress on program activities. The preliminary report for the 1999 program was submitted in July 1999. However, as of the date of this report, the three other reports have not been submitted. In fiscal year 2003, the Congress authorized another 3-year pilot program and extended the 1999 and 2000 pilot programs until 2005.[Footnote 7] Under the new 2003 pilot program, the Secretary of Defense is to provide one laboratory and one test center in each service the authority to: * use innovative personnel management methods to ensure that the participants can employ and retain an appropriately balanced workforce, and effectively shape the workforce to fulfill the organization mission; * develop or expand innovative methods of using cooperative agreements with private sector and educational organizations to promote the technological industrial base for critical defense technologies and facilitate the training of a future scientific and technical workforce; and: * waive any restrictions not required by law. As of May 2003, DOD had not identified any participants for the 2003 pilot program. The 2003 legislation also requires DOD to issue three reports, including a January 2003 report on its experience with the 1999 and 2000 pilot programs, barriers to implementation of these programs, and proposed solutions to overcome these barriers.[Footnote 8] According to DOD officials, this report has been drafted, but as of May 2003, it had not been submitted to the Congress. Many Initiatives Were Proposed but Few Were Implemented under Pilot Programs: Since the inception of the pilot programs in 1999, 178 initiatives have been proposed, but only 4--or 2 percent--have been implemented under the pilot programs. Participating laboratories and test centers proposed initiatives covering a variety of areas, including business- like practices, partnerships with industry and academia, and human capital innovations. We found that laboratories focused many of their proposals on human capital innovations, while test centers tended to concentrate on business-like practices and partnerships. Range and Volume of Proposed Initiatives: Over the course of the 1999 and 2000 pilot programs, the laboratories and test centers proposed 178 human capital, business, and partnership initiatives. As shown in table 1, slightly over half of the initiatives dealt with human capital and the remainder dealt with business-like practices and partnerships. Table 1: Laboratory and Test Center Pilot Program Proposals: Laboratories; Business/partnerships: Number: 49; Business/ partnerships: Percent: 33; Human capital: Number: 98; Human capital: Percent: 67; Total: Number: 147; Total: Percent: 100. Test centers; Business/partnerships: Number: 27; Business/ partnerships: Percent: 87; Human capital: Number: 4; Human capital: Percent: 13; Total: Number: 31; Total: Percent: 100. Total; Business/partnerships: Number: 76; Business/partnerships: Percent: 43; Human capital: Number: 102; Human capital: Percent: 57; Total: Number: 178; Total: Percent: 100. Source: GAO. [End of table] Overall, the laboratories proposed substantially more initiatives than did the test centers. Furthermore, the laboratories and test centers focused on different types of initiatives. The laboratories more often proposed human capital initiatives, while the test centers overwhelmingly focused on business and partnership initiatives. Laboratory officials told us that they are especially concerned about attracting top-quality scientists to replace a retiring workforce. Test center officials told us that they are focused on modernizing their infrastructure and developing new methods of sharing the cost of operations. Proposals for business-like practices included many initiatives to streamline or improve local operations. Some initiatives focused on expanding the use of innovative techniques such as other transactions or cooperative agreements.[Footnote 9] Several other proposals sought the authority to reinvest fees or revenues into facilities revitalization. For example, one Navy laboratory proposed imposing a surcharge for its services and using that revenue to fund capital investments, and an Air Force laboratory proposed using facility construction as a valid in-kind contribution under cooperative agreements. Partnership proposals included initiatives such as collaborative research agreements with Arnold Engineering Development Center and the University of Tennessee Space Institute to create a formal business bond to pursue research in laser-induced surface improvement technology and university flight research. The Army's Aberdeen Test Center proposed a limited liability company. Under this concept, industry, academia, and government would form a profit-making company to conduct research and testing at the installation. The test center proposed using its share of the profits to reinvest in the infrastructure at Aberdeen. Several human capital initiatives focused on recruiting and retention flexibilities as well as additional voluntary separation incentives. These proposals included initiatives to streamline hiring of experts and consultants; accelerate promotions for scientists and engineers; provide retention bonuses for key scientists; and hire students directly after graduation. Several participants submitted proposals for direct hire authority to allow faster hiring of scientists, and several submitted proposals for voluntary retirement incentives as a mechanism for reshaping the workforce. Few Proposals Were Implemented under Pilot Program Authorities: Almost none of the 178 proposed initiatives were approved and implemented using the pilot programs' authorities. As figure 1 shows, only 2 percent--or 4 proposals--were implemented under the pilot programs. In contrast, 74 percent were blocked or dropped during the review process or remain on hold awaiting resolution. Figure 1: Status of Proposed Initiatives: [See PDF for image] [End of figure] The four implemented initiatives were: * donating laboratory equipment directly to local schools, * waiving top-level certification of certain service agreements with private industry, * streamlining cooperative agreements to facilitate collaborative work agreements with outside activities,[Footnote 10] and: * granting temporary relief from some mandatory personnel placement reviews.[Footnote 11] Officials at the laboratories that proposed these initiatives told us that they were considered minor changes with little impact on the larger problems facing the laboratories. Twelve times as many initiatives--24 percent--were implemented using different authorities than the pilot programs. For example, several laboratories requested the authority to appoint retired military members to civilian positions without having to wait the required 180 days. This requirement was waived using a different authority than the pilot programs. Another human capital initiative--to appoint senior scientists from private industry--was authorized by subsequent legislation.[Footnote 12] In the business/partnership category, the 46th Test Group at Holloman Air Force Base used other authorities to negotiate a complex leasing arrangement with industry to install a radar test facility at White Sands Missile Range. This effort took several years and overcame many contractual and regulatory barriers. In addition, a Navy laboratory streamlined foreign license applications using another authority.[Footnote 13] The Pilot Programs Were Not Effective for Two Primary Reasons: The low level of implementation of the proposed initiatives occurred for two primary reasons. First, DOD did not develop an effective process for implementing the pilot programs. Second, DOD determined that proposed human capital initiatives--for example, requests for the authority to hire directly or offer voluntary retirement incentives-- were in conflict with statutory provisions. Lack of an Effective Implementation Process: DOD did not provide standardized guidance on proposal requirements or feedback for improving proposals; coordinate or prioritize proposals; or clarify decision-making authority for proposal review and approval. DOD also did not designate a strong focal point to coordinate the pilot programs, advocate process improvements, and provide assistance and advice to participants. The lack of a strong focal point exacerbated other process gaps. Lack of Guidance and Coordination: According to officials at DOD laboratories, test centers, and headquarters, DOD did not provide standardized guidance on proposal requirements or feedback for improving proposals (or, in many cases, information on the status of proposals submitted for approval). Proposals often lacked specificity and detail. Many were broadly conceptual or generic in nature and lacked a detailed business case that linked their contribution to overall objectives for the pilot programs. For example, a proposal to permit scientists to serve in a leadership role in professional societies failed to include details of the problems encountered, and the potential to improve operations. Similarly, several proposals for direct hire authority failed to include a business case to explain what specific needs this authority would address or how it would address them. Lack of specificity and business case detail led to the failure of many initiatives to win approval. DOD attorneys told us that many proposals were so vague that it was impossible to determine whether or not the proposed initiatives could meet legal requirements. At a department level, DOD also did not coordinate or prioritize proposals, thereby precluding decisions on how best to pursue common interests and issues such as direct hiring authority or forming partnerships with universities. Instead, each participant submitted proposals individually, and thus multiple independent proposals were often submitted for the same or similar issues. DOD attorneys pointed out that it would have been more effective to group proposals by common theme and prioritize them. They believed a unified approach and prioritized proposals with clearly written, specific plans for solving well-defined problems would have enabled them to more effectively assist participants with resolving legal issues. Unclear Decision-Making Authority: DOD did not clarify decision-making authority for proposal review and approval. Many organizations and individuals were stakeholders in proposal review and approval, and they often had differing management structures, concerns, and interests. Stakeholders included military and civilian leaders, attorneys, and human capital and personnel staff at several levels: the local installation where participating laboratories and test centers were housed; the individual service; and OSD. The roles and decision-making authority of the various stakeholders were never negotiated and clarified. As a result, many players at multiple organizational levels had--and took--an opportunity to say "no" to a particular proposal, but it remained unclear who had the authority to say "yes.": For example, some participants believed that the pilot program legislation gave the director of a participating laboratory or test center the authority to approve a proposed initiative. OSD officials, however, believed that the proposed initiatives had to be approved at higher levels. The role of the services was also unclear. Some laboratory and test center directors initially sent proposals directly to OSD's Directorate of Defense Research and Engineering (DDR&E), bypassing their service headquarters. Others sent proposals to their service headquarters for approval before submitting the proposals to DDR&E. Eventually, however, each of the service headquarters decided to become more heavily involved in the approval process and provide service-level responses to proposals. These service-level responses often came into play after proposals had been sent directly to DDR&E for approval, further complicating the approval process. Within OSD, both DDR&E and Personnel and Readiness (P&R) had substantial stakes in the human capital proposals--DDR&E because it is charged with oversight and management of defense laboratories and P&R because it has the authority within DOD for human capital issues.[Footnote 14] However, DDR&E and P&R never agreed on a process for approving proposals. In addition, for the past year P&R's attention has been focused primarily on developing DOD's proposed new civilian human capital management system, the National Security Personnel System (NSPS), which the Secretary of Defense recently submitted to the Congress. DOD officials believe that, if enacted, NSPS will provide flexibility to make necessary human capital changes.[Footnote 15] The Undersecretary of Defense P&R directed that implementation of new personnel initiatives be placed on hold during the development of NSPS so that the existing system could be studied to identify needs and best practices. Consequently, P&R officials believed it would be premature for DOD to implement new personnel initiatives during this time. No Strong Focal Point: DOD did not designate a strong focal point to coordinate the pilot programs, advocate process improvements, and provide assistance and advice to participants. This exacerbated the other process gaps. Without such a focal point, participants found their own individual ways to develop proposals and get them reviewed. Several officials agreed that a strong focal point would be helpful. For example, DOD attorneys stated that the laboratories or someone acting as their focal point needed to define the issues they wanted to resolve. The attorneys noted that a focal point could have more successfully drawn upon their expertise and experience with addressing legal challenges in other innovative programs (e.g., demonstration projects). Some pilot program participants also agreed a strong focal point was needed, but they had some concerns regarding the amount of influence and authority he or she should have. Human Capital Proposals Were in Conflict with Existing Statutory Provisions: According to officials at DOD laboratories, test centers, and headquarters, human capital initiatives were generally in conflict with title 5 of the United States Code. Title 5 provides the framework for standard and equitable personnel practices across the federal government and is the current foundation for management of the DOD civilian workforce. Over time, the Office of Personnel Management has added implementing rules and regulations to the framework. Proposed human capital initiatives often sought relief from these provisions, for example, requests for the authority to hire directly or offer voluntary retirement incentives. However, after reviewing the legislation, the DOD Office of General Counsel advised that the 1999 and 2000 legislation did not provide the authority to waive personnel rules based on title 5 provisions. Rather, the office advised that the pilot programs' authorities allow only for changes that could already be accomplished under existing DOD regulations. In other words, the pilot programs did not provide any new or additional authority to waive existing personnel rules and regulations grounded in title 5. Consequently, absent statutory authority beyond that provided by the pilot programs, human capital proposals in conflict with title 5 and its implementing rules and regulations could not be implemented.[Footnote 16] Many initiatives fell into this category. The 2003 Pilot Program Faces Implementation Challenges: The 2003 pilot program faces several implementation challenges. First, as of May 2003, DOD had not addressed implementation problems. Thus, proposals made via the 2003 pilot program will face the same obstacles as previous proposals. Second, human capital initiatives will continue to face title 5 challenges. Like the earlier legislation, the 2003 legislation does not provide DOD any new authority. Hence, initiatives proposed under the 2003 pilot program will encounter the same statutory restrictions as previous initiatives. P&R officials believe that, if implemented, NSPS will provide the flexibility to make necessary human capital changes, thereby eliminating the need for the pilot programs in this area. However, NSPS has not yet been enacted, and if enacted, it will still require an implementation process. Finally, laboratories and test centers may be reluctant to participate in the new pilot program. Many participants in the earlier pilots told us they were discouraged by their experience and consequently unwilling to repeat it. Some expressed frustration with the lack of guidance and feedback on their proposals; others questioned whether management was really committed to the pilot program. Even those few participants that had proposals approved were wary of expending additional resources on another pilot program. Conclusions: While DOD appears to recognize a need to address human capital and business operations issues specific to laboratories and test centers, it has not effectively managed the pilot programs. If DOD intends to use the pilot programs to address laboratory and test center issues, it will have to address the factors--both process and statutory--that blunted previous proposals made through the pilot programs. The small volume of approved proposals, coupled with DOD's not providing status reports required by the Congress, has left the Congress uninformed about what objectives DOD would like to achieve with the laboratories and test centers, how it plans to achieve those objectives, and what vehicles it plans to use. This information will be important to the success of any future actions. Recommendations for Executive Action: We recommend that by March 31, 2004, the Secretary of Defense inform the Congress of DOD's objectives regarding human capital and business operations in the laboratories and test centers, how it plans to meet these objectives, and what vehicles it will use to meet them. We also recommend that by March 31, 2004, the Secretary of Defense develop a process for proposing, evaluating, and implementing human capital, business, and partnership initiatives for the laboratories and test centers, regardless whether by the pilot authority or by some other vehicle. Such a process should include: * clear decision-making authority, * instructions for proposal requirements such as linking to overall goals and measurable objectives and the need for a business case, and: * specification of procedures for proposal submission and review and providing feedback on proposal quality and scope. Finally, we recommend that the Secretary of Defense designate a strong focal point to: * receive, evaluate, and prioritize all proposals and: * work with laboratory and test center directors, legal counsel, personnel and other specialists to develop sound and well-developed business cases and strategies to obtain needed changes. Agency Comments and Our Evaluation: In written comments on a draft of this report, DOD states that it does not concur with our recommendations because it has already taken actions that in effect implement them.[Footnote 17] While the actions DOD cites that it has taken are important to implementing our recommendations, they are not sufficiently specific to address the problems identified in our report. DOD's written comments are contained in appendix II. Regarding our first recommendation--that DOD inform the Congress of its human capital and business objectives for the laboratories and test centers and the strategies it will employ to meet them--DOD did not concur. DOD discusses various high-level, agencywide initiatives it has taken to address human capital and business issues in general and stated that the Congress has been made aware of these initiatives, obviating the need for additional reporting. We continue to believe that additional reporting is necessary. We recognize that the general initiatives DOD discusses may provide ways of helping the laboratories and test centers; however, to be effective, they must be made specific, that is, developed into targeted strategies and plans that address the particular problems the laboratories and test centers face. DOD has not provided the Congress sufficient details on how the general initiatives will be used to address laboratories' and test centers' objectives and problems. Regarding our second recommendation--that DOD develop a process for proposing, evaluating, and implementing human capital and business-like practices initiatives for the laboratories and test centers--DOD did not concur. DOD states that it has already introduced new agencywide management processes--the Business Initiative Council and the submission of the NSPS proposal to the Congress--to address human capital and business issues in general. However, DOD has not detailed how these general initiatives will apply to the laboratories and test centers or address our process concerns. For example, while the Business Initiative Council may have an effective process for proposing, evaluating, and implementing laboratory and test center business-like practices initiatives, DOD has not provided sufficient information for us to make such a determination. We also recognize that NSPS may address some of the human capital problems faced by the laboratories and test centers, but this system is still under consideration by the Congress. Until it becomes law, we believe it is premature to cite it as an effective management tool. With regard to our third recommendation--that DOD designate a strong focal point to work with the laboratories and test centers to develop, evaluate, prioritize, and coordinate proposed initiatives--DOD did not concur. DOD states that the recently created position of Undersecretary for Laboratories and Basic Sciences has oversight responsibility for all laboratory initiatives and that it is establishing a new Defense Test Resources Management Center that will oversee the test centers.[Footnote 18] DOD asserts that these two organizations will perform as focal points. However, DOD has not detailed how these organizations will fulfill this role and work with the laboratories and test centers to overcome the many barriers noted in our report. Scope and Methodology: During our review, we met with officials from the following organizations in the Office of the Secretary of Defense: the Director, Defense Research and Engineering; the Director, Operational Test and Evaluation; the General Counsel, and the Deputy Undersecretary of Defense for Personnel and Readiness. We also met with officials from the Army Research Laboratory, Aberdeen Test Center, Army Medical Research and Materiel Command, Naval Research Laboratory, Naval Undersea Warfare Center, Air Force Research Laboratory, Air Force Research Laboratory's Space Vehicles Directorate, and 46th Test Wing. We also discussed pilot program issues with each participating laboratory or center. To determine the initiatives proposed to date and their status, we obtained records from OSD and service officials. From these records and from discussions with each participant, we compiled a listing of initiatives proposed by each participating laboratory and test center. We verified the listing and the current status of each initiative with cognizant service officials. To determine what obstacles inhibited DOD's implementation of the pilot programs, we obtained documentation and data from pilot program participants as well as from OSD officials. We also discussed statutory obstacles with the officials from DOD's Office of General Counsel and Undersecretary of Defense for Personnel and Readiness. We discussed management and procedural obstacles with officials from the Director, Operational Test and Evaluation and Defense Research and Engineering. In addition, we discussed all obstacles with the participating laboratories and test centers. The problems facing the laboratories and test centers have been documented by many organizations, panels, and commissions. We did not independently verify these problems or the findings and conclusions of these entities. We conducted our review from July 2002 to April 2003 in accordance with generally accepted government auditing standards. We are sending copies of this report to the Secretary of Defense; the Secretaries of the Army, Navy, and Air Force; and interested congressional committees. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. Major contributors to this report were Catherine Baltzell, Arthur Cobb, Christopher Durbin, Rae Ann Sapp, Sylvia Schatz, and Katrina Taylor. If you have any questions regarding this report, please call me at (202) 512-4841. Paul L. Francis Director Acquisition and Sourcing Management: Signed by Paul L. Francis: [End of section] Appendix I: Fiscal Year 1999 and 2000 Pilot Program Participants: Laboratory/test center: Army Research Laboratory; 1999 pilot: No; 2000 pilot: Yes. Laboratory/test center: Army Medical Research and Materiel Command; 1999 pilot: Yes; 2000 pilot: No. Laboratory/test center: Aberdeen Test Center; 1999 pilot: Yes; 2000 pilot: Yes. Laboratory/test center: Naval Research Laboratory; 1999 pilot: Yes; 2000 pilot: No. Laboratory/test center: Naval Undersea Warfare Center; 1999 pilot: No; 2000 pilot: Yes. Laboratory/test center: Naval Air Warfare Center-Aircraft and Weapons Divisions; 1999 pilot: Yes; 2000 pilot: Yes. Laboratory/test center: Air Force Research Laboratory-Information Directorate; 1999 pilot: No; 2000 pilot: Yes. Laboratory/test center: Air Force Research Laboratory-Space Vehicles Directorate; 1999 pilot: Yes; 2000 pilot: No. Laboratory/test center: Air Armament Center, 46th Test Wing; 1999 pilot: No; 2000 pilot: Yes. Laboratory/test center: Arnold Engineering Development Center; 1999 pilot: Yes; 2000 pilot: No. Source: DOD. [End of table] [End of section] Appendix II: Comments from the Department of Defense: OFFICE OF THE DIRECTOR OF DEFENSE RESEARCH AND ENGINEERING: 3040 DEFENSE PENTAGON WASHINGTON, D.C. 20301-3040: 2003: Mr. Paul Francis: Director, Acquisition and Sourcing Management U.S. General Accounting Office: 441 G Street, N.W. Washington, D.C. 20548: Dear Mr. Francis: This is the Department of Defense (DoD) response to the GAO draft report, "DEFENSE PILOT PROGRAMS: DoD Needs to Improve Procedures for Implementation," dated June 9, 2003 (GAO Code 120159/GAO-03-861). The Department non-concurs with the recommendations of the report because the Department has already taken strong management actions that essentially implement the recommendations or, in many cases, go beyond the recommendations in the report. Please see detailed comments in the enclosure. We appreciate the opportunity to comment on the draft report. My action officer for this effort is Dr. William Berry at (703) 696-0363. Sincerely, John H. Hopps Jr. Deputy Director and Deputy Under Secretary of Defense (Laboratories and Basic Sciences): Signed by John H. Hopps Jr.: Enclosure: As Stated: GAO DRAFT REPORT - DATED JUNE 9, 2003 GAO CODE 120159/GAO-03-861: "DEFENSE PILOT PROGRAMS: DoD Needs to Improve Procedures for Implementation": DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS: RECOMMENDATION 1: The GAO recommended that by March 31, 2004, the Secretary of Defense inform the Congress of DoD's objectives regarding human capital and business operations in the laboratories and test centers, how it plans to meet these objectives, and what vehicles it will use to meet them. (p. 12/GAO Draft Report): DOD RESPONSE: Non-Concur with recommendation 1 as written. The Department agrees with the objective of striving for improved management of human capital and business operations in the laboratories and test centers. Beginning in 2001, the Department took significant steps in implementation of the 1999 and 2000 demonstration projects. The resulting Departmental management processes provide a fully adequate vehicle to pursue initiatives and innovative processes. In July 2001 a DoD Business Initiative Council was established with the mission to improve efficiency of DoD business operations by identifying and implementing business reform actions which allow savings to be reallocated to higher priority efforts (i.e., people, readiness, modernization, and transformation). In November 2001 the Department created and filled the position of Deputy Under Secretary for Laboratories and Basic Sciences. This position serves as the advocate for laboratory quality, and is concerned with issues of personnel as well as business practices. The Congress has been made aware of each of these new management processes. An additional report to Congress about either is unnecessary. From the standpoint of human capital, DoD currently has a Human Resources (HR) Strategic Plan that addresses the workforce's capability of responding rapidly, efficiently, and effectively to mission requirements. A report to Congress is unnecessary to explore options already offered by new authorities and flexibilities in the proposed National Security Personnel System (NSPS). The Department has been testing personnel flexibilities for over two decades in our personnel demonstration projects and went through a year-long study of the best practices of those projects. We are implementing the best practices in the laboratories now and will extend the flexibilities to the rest of the workforce under NSPS. The draft report states that, "this legislation has not yet been enacted, and if enacted would still require an implementation process" (page 2). However, this description does not fully depict the current situation. Congress is now actively considering NSPS. Additionally, even before NSPS implementation, the Department is currently taking steps to ensure that top scientific and engineering talent is recruited through such innovative hiring flexibilities as on-the-spot hiring and scholastic achievement appointment authority. The importance of human capital strategic planning was clearly recognized in the Quadrennial Defense Review. It is the first item on the President's Management Agenda and is a top priority for the Department. In early 2003, the Department published its FY 2003 Year of Execution Plan as an Annex to the integrated DoD Civilian Human Resources (HR) Strategic Plan. The Department Component strategic plans will link to the DoD corporate goals and objectives and include actions needed to help ensure viability of DoD's white-collar civilian workforce. In addition to the DoD Human Resources Strategic Plan, the Department has spent the last year developing demonstration project best practices which allows for the sharing of information and development of a single set of personnel flexibilities for the entire laboratory community. Learning from past practices and improving on the process is what best practices is all about. RECOMMENDATION 2: The GAO recommended that by March 31, 2004, the Secretary of Defense develop a process for proposing, evaluating, and implementing human capital and business-like practices initiatives for the laboratories and test centers, regardless whether by the pilot authority or by some other vehicle. Such a process should include: * Clear decision-making authority; * Instructions for proposal requirements such as linking to overall goals and measurable objectives and the need for a business case; and: * Specification of procedures for proposal submission and review and providing feedback on proposal quality and scope. (p. 12/GAO Draft Report): DOD RESPONSE: : Non-concur with recommendation 2 as described. New management processes have been introduced within the Department since the implementation of the 1999 and 2000 demonstration programs. The new processes have focused considerable effort on the management of the DoD workforce. Legislative proposals have been prepared and submitted with regularity. The recently submitted National Security Personnel System is an example. In July 2001 a DoD Business Initiative Council was established with the mission to improve efficiency of DoD business operations by identifying and implementing business reform actions which allow savings to be reallocated to higher priority efforts (i.e., people, readiness, modernization, and transformation). The Service Secretaries and the Under Secretary of Defense (Acquisition, Technology & Logistics) sit on the council. It is doubtful that an additional process will have sufficient value to justify their implementation. RECOMMENDATION 3: The GAO recommended that the Secretary of Defense designate a strong focal point to: * Receive, evaluate, and prioritize all proposals; and: * Work with laboratory and test center directors, legal counsel, personnel and other specialists to develop sound and well-developed business cases and strategies to obtain needed changes. DOD RESPONSE: Non-concur with this recommendation as written as it is already accomplished. Since implementing the 1999 and 2000 demonstration programs, the Department has designated strong focal points for the laboratories and for the test centers. In November 2001 the Department created a new position Deputy Undersecretary for Laboratories and Basic Sciences. The incumbent of this position also serves as the, Deputy Director, Defense Research and Engineering. Among other responsibilities, the position is responsible for all laboratory initiatives including those dealing with personnel and business operations. The Department is establishing a Defense Test Resources Management Center to oversee the T&E ranges and centers. These two organizational elements provide the functions sought by the GAO. DoD laboratories and centers can pursue initiatives through this new and existing management chain. [End of section] FOOTNOTES [1] Public Law 105-261, Strom Thurmond National Defense Authorization Act for Fiscal Year 1999, section 246. Public Law 106-65, National Defense Authorization Act for Fiscal Year 2000, section 245. [2] Senate Report 107-151. [3] Defense Transformation for the 21st Century Act of 2003, as transmitted by letter, dated April 10, 2003, from the DOD General Counsel to the Speaker of the House and the President of the Senate. [4] Blue Ribbon Panel on Management Options for Air Force Laboratories, January 1994; Improving Federal Laboratories to Meet the Challenges of the 21st Century, National Science and Technology Council, July 1999; Science and Technology Community in Crisis, Naval Research Advisory Committee, May 2002. [5] Defense Science and Technology Base for the 21st Century, Defense Science Board, June 1998; House Report 105-532, House National Security Committee, May 1998; Science and Technology Community in Crisis, Naval Research Advisory Committee, May 2002. [6] Public Law 105-261, Strom Thurmond National Defense Authorization Act for Fiscal Year 1999, section 246; Public Law 106-65, National Defense Authorization Act for Fiscal Year 2000, section 245. [7] Public Law 107-314, Bob Stump National Defense Authorization Act for Fiscal Year 2003, section 241. [8] The other two reports are a September 2003 report on all three pilot programs and a final report on the 2003 pilot at its conclusion. [9] "Other transactions" is a term commonly used to refer to 10 U.S.C. 2371 authority to enter into agreements that are not generally covered by federal laws and regulations applicable to standard procurement contracts. Consequently, the arrangements include broader latitude to negotiate terms and conditions than standard procurement contracts under the Federal Acquisition Regulations. [10] This initiative included several closely related but separate waivers that were grouped together by the service of the laboratories that proposed them. [11] This waiver expired in August 2002. It was extended until February 2003 as part of a separate pilot program sponsored by the DOD Business Initiative Council. [12] Public Law 106-398, National Defense Authorization Act for Fiscal Year 2001, section 1113. [13] The Office of the Secretary of Defense (OSD) and the Navy disagree on what authority was used to implement this proposal. OSD believes that it was implemented using pilot program authority. [14] Unlike the laboratories, the test centers are not overseen centrally but by the individual service to which they belong. [15] The Comptroller General recently testified on NSPS. U. S. General Accounting Office. Human Capital: DOD's Civilian Personnel Strategic Management and the Proposed National Security Personnel System. GAO-03- 493T (Washington, D.C.; May 12, 2003). Defense Transformation: DOD's Proposed Civilian Personnel System and Governmentwide Human Capital Reform. GAO-03-741T (Washington, D.C.; May 1, 2003). [16] Our attorneys reviewed the pilot program legislation and concurred with the DOD General Counsel's view. [17] In its letter, DOD refers to the "1999 and 2000 demonstration programs." We confirmed with DDR&E that these demonstration programs were indeed the 1999 and 2000 pilot programs as described in this report. [18] This new center is not yet operational. GAO's Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: