This is the accessible text file for GAO report number GAO-13-286 entitled 'Defense Technology Development: Technology Transition Programs Support Military Users, but Opportunities Exist to Improve Measurement of Outcomes' which was released on March 7, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Committees: March 2013: Defense Technology Development: Technology Transition Programs Support Military Users, but Opportunities Exist to Improve Measurement of Outcomes: GAO-13-286: GAO Highlights: Highlights of GAO-13-286, a report to congressional committees. Why GAO Did This Study: DOD and Congress recognize that technology innovation sometimes moves too slowly from the lab to the field. Programs have been created in DOD to help facilitate the transition of new technologies. The conference report accompanying the fiscal year 2012 National Defense Authorization Act directed GAO to undertake a body of work that will provide a holistic assessment of DOD’s S&T enterprise. This report reflects the results from GAO’s first review, which focuses on technology transition. Generally, when technologies have been sufficiently matured in the S&T environment, the technologies are available to transition to a military user. GAO’s specific objectives were to (1) determine what DOD programs are dedicated to facilitating technology transition, (2) assess the outcomes of these transition programs, and (3) identify practices among the programs that may facilitate technology transition. GAO conducted interviews with and collected information from each technology transition program to identify their selection, management, and assessment practices, as well as project outcomes. What GAO Found: GAO identified 20 technology transition programs-—managed by the Office of the Secretary of Defense (OSD) and the military departments—- that provide structured mechanisms and funding to facilitate technology transition. All of the programs GAO reviewed are consistent in providing opportunities to transition technologies from the science and technology (S&T) environment to a user, such as a weapon system acquisition program or the warfighter in the field. To help speed the delivery of technologies to users, most transition programs target fairly mature technologies, which are suitable for final stages of development and demonstration. Collectively, the programs GAO reviewed obligated about $7.9 billion in Department of Defense (DOD) research, development, test, and evaluation funding for fiscal years 2010 through 2012 to support technology transition. Most programs that GAO assessed track whether their projects were completed and successfully transitioned to intended users. On average, programs reported a historical transition rate of over 70 percent for projects. The vast majority of these projects resulted in technologies transitioning to acquisition programs or directly to the warfighter. However, about one-quarter of the projects transitioned to other organizations, such as test and evaluation centers, for further development. Prior GAO work found that tracking technology transitions and the impact of those transitions, such as cost savings or deployment of the technology in a product, provides key feedback that can inform the management of programs. For the most part, transition programs that GAO reviewed do not track projects beyond transition, which limits their ability to know and report final outcomes for transitioned technologies and the associated benefits realized from those technologies. As GAO has reported in the past, effective selection and management processes as well as tools are needed to ensure that new technologies can be successfully transitioned to military users. GAO found that OSD’ s and the Military Departments’ technology transition programs make use of these practices to varying degrees. Most programs have formal review processes to determine whether candidate projects have sufficiently mature technologies, are in demand by users, and have schedules and costs that fit within the programs’ criteria. Once selected, projects require effective management to ensure risks are minimized and transition commitments are confirmed. Many program officials indicated that regular stakeholder communication during project execution is important to ensure projects stay on track and transition commitments are sustained. Moreover, many program officials identified the use of formal management tools, such as technology transition agreements, as key mechanisms to help hold stakeholders accountable and facilitate technology transition. What GAO Recommends: GAO recommends that DOD require programs to track and measure project outcomes to document transition results and benefits from transition, as well as assess programs to identify opportunities for more widespread use of existing transition management tools. DOD generally concurred with these recommendations and stated that it will initiate actions to address potential opportunities for improvement identified in the report. View [hyperlink, http://www.gao.gov/products/GAO-13-286]. For more information, contact Michael J. Sullivan at (202) 512-4841 or sullivanm@gao.gov. [End of section] Contents: Letter: Background: DOD Has a Variety of Technology Transition Programs That Support Military Users: Technology Transition Programs Provide Technologies to Military Users, But Tracking of Project Outcomes and Benefits after Transition Is Limited: Programs Facilitate Transition Through Sound Project Selection and Management Practices: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: DOD Research, Development, Test, and Evaluation Budget Activities: Appendix III: DOD Technology Readiness Levels: Appendix IV: Comments from the Department of Defense: Appendix V: GAO Contact and Staff Acknowledgments: Tables: Table 1: OSD Technology Transition Programs Funding and Project Characteristics: Table 2: Military Departments' Transition Program Funding and Project Characteristics: Table 3: Historical Transition Rates Reported by OSD and Military Departments Technology Transition Programs: Figures: Figure 1: DOD Fiscal Year 2013 President's Budget for Research, Development, Test, and Evaluation: Figure 2: DOD Technology Management: Investment, Development, and Transition: Figure 3: Technology Maturity and Typical Project Duration for Transition Programs: Figure 4: Technology Transition Results Reported by Programs for Completed Projects, Fiscal Years 2010-2012: Figure 5: Example of Transition Commitment Level Project Evaluation Tool: Abbreviations: ATD: Advanced Technology Demonstration: ATO-D: Army Technology Objectives-Demonstration: CP3: Core Process 3: DAC: Defense Acquisition Challenge: DOD: Department of Defense: FCC: Flagship Capability Concepts: FCT: Foreign Comparative Testing: FNC: Future Naval Capabilities: FY: fiscal year: JCTD: Joint Capability Technology Demonstration: ManTech: Manufacturing Technology: ONR: Office of Naval Research: OSD: Office of the Secretary of Defense: QRF: Quick Reaction Fund: RDT&E: research, development, test, and evaluation: REF: Rapid Equipping Force: RIF: Rapid Innovation Fund: RRF: Rapid Reaction Fund: RTT: Rapid Technology Transition: S&T: science and technology: SBIR: Small Business Innovation Research: SW/Exp: SwampWorks and Experimentation: TCL: transition commitment level: TECD: Technology Enabled Capability Demonstration: TIPS: Technology Insertion Program for Savings: TMI: Technology Maturation Initiative: TRL: technology readiness level: TS: TechSolutions: VTUAV: Vertical Takeoff and Landing Tactical Unmanned Aerial Vehicle: [End of section] United States Government Accountability Office: Washington, DC 20548: March 7, 2013: Congressional Committees: The Department of Defense (DOD) plans to spend about $70 billion in fiscal year 2013 to develop and enhance weapon systems. Despite the high caliber of systems delivered to the warfighter, weapon system programs too often experience cost, schedule, and performance problems, in part because they attempt to incorporate advanced technologies that have not been proven. DOD's science and technology (S&T) community, which receives about $12 billion annually, is tasked with identifying, developing, and ensuring that high value technologies are mature and available for use by the department's acquisition and military user communities. Generally speaking, technology transition means identifying technologies that have been sufficiently matured in the S&T environment and are ready to transition to a user such as a weapon acquisition program or the warfighter in the field. Transitioning technologies from defense S&T labs and research organizations to military users, however, has been a long-standing challenge for DOD. Sometimes technologies are not ready to transition when needed because they may still be too risky or too costly to adopt. At other times promising technologies are not taken advantage of because of insufficient processes and mechanisms to expedite their transition to users. Congress and DOD recognize the difficulties involved in transitioning technologies and have established many programs over time aimed at removing barriers to technology transition and accelerating the flow of technologies to military users. GAO was directed by the Congress, in the conference report accompanying the fiscal year 2012 National Defense Authorization Act, to undertake a body of work that will provide a holistic review of DOD's S&T enterprise. This report reflects the findings from our first review, which focuses on technology transition. Specifically, as discussed with your offices, the objectives of this review were to (1) determine what DOD programs exist that are dedicated to facilitating technology transition from the S&T base to military users, (2) assess the outcomes for these transition programs, and (3) identify practices among the programs that may facilitate technology transition. To conduct this work, we evaluated documentation from, as well as interviewed, Office of the Secretary of Defense (OSD) and military department officials as appropriate. We reviewed 20 technology transition programs and assessed information on program structure, processes, transition tools, and metrics kept to track transition outcomes. This included data on each transition program and its technology transition projects for fiscal years 2010 through 2012. We conducted this performance audit from April 2012 to March 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix I provides further details on the scope and methodology used for this work. Background: DOD relies on its S&T community--DOD research laboratories, test facilities, industry, and academia--to identify, pursue, and develop new technologies that improve and enhance military operations and ensure technological superiority over adversaries. These activities by DOD's S&T community support the development of technologies for new or existing weapon systems, as well as development of capabilities that are transitioned directly to warfighters in the field. The S&T community engages in activities ranging from basic research through advanced technology development that are conducted by the government or externally by universities and commercial industry. Once the S&T community has completed its technology development, additional product development activities, such as technology demonstration and testing, are often needed before incorporating the technologies into military weapon systems. Under the management of the acquisition community, product development further advances technology received from S&T developers and integrates it into systems that are ultimately delivered to support the warfighter. Supporting these activities is DOD's research, development, test, and evaluation budget. As shown in figure 1 below, this budget is divided into seven categories that follow a generally sequential path for developing technologies from basic research to operational system development. The first three categories represent DOD's S&T activities to advance research and technology development, while the remaining categories support product development for DOD acquisition programs. Appendix II provides further details on research and development activities in these budget categories. Figure 1: DOD Fiscal Year 2013 President's Budget for Research, Development, Test, and Evaluation: [Refer to PDF for image: vertical bar graph] Technology development: Basic research: $2.1 billion; Applied research: $4.5 billion; Advanced technology development: $5.3 billion; Total DOD S&T: $11.9 billion. Product development: Advanced component development & prototypes: $12.4 billion; System development & demonstration: $14.7 billion; RDT&E management support: $4.3 billion; Operational system development: $26.4 billion. Source: GAO analysis of DOD data. Note: RDT&E = research, development, test, and evaluation. [End of figure] The large size and scope of DOD's national security mission imposes broad and competing demands on the S&T community, and determining the right mix of technology investments to pursue is challenging. One key expectation is that S&T funding will result in relevant and feasible technologies that can transition into weapon system programs or go directly to the warfighter in the field. DOD has also tasked its S&T community with anticipating its technological needs for an uncertain future. Further, the S&T community serves as a technology innovation mechanism to expand scientific knowledge and investigate technologies that may provide breakthrough warfighting capabilities. As a result, some investments focus on conducting research to generate scientific knowledge, exploring new technologies, demonstrating the feasibility of a technology concept, and other science and technology endeavors. The challenge is finding the right balance between taking risk to develop breakthrough technologies versus investing in moderate technology enhancements that are likely to transition to a military user. Although not precisely defined, technology transition generally occurs when advanced technology development ends and product development begins. Figure 2 below provides a notional picture of how DOD's S&T community manages technology investment, development, and transition to a user. Figure 2: DOD Technology Management: Investment, Development, and Transition: [Refer to PDF for image: illustration] Technology investment: Market research; Capability needs; S&T resources; Prioritized S&T projects. Technology development: Technology feasibility analyzed and validated; Feasible and relevant technology developed and demonstrated. Technology transition: Implementation- or user-ready technology transitions: Technology shelved, nut available for use; or: Technology integrated into acquisition program or fielded. Source: GAO analysis of DOD S&T management process. [End of figure] DOD's research laboratories and test facilities, as well as industry and academia, provide many examples of developing new technologies that improve warfighter capabilities and enhance military operations. However, as we have reported in the past, for a variety of reasons DOD historically has experienced problems in transitioning technologies out of its S&T environment and into military systems.[Footnote 1] Technologies may not leave the lab because their potential has not been adequately demonstrated or recognized, acquisition programs may be unwilling to fund final stages of development, or private industry chooses to develop the technologies itself. Despite these challenges, technology transition can be facilitated in many ways, and is affected by choices made throughout the S&T management cycle. Before technology development is begun, critical S&T investment decisions must be made that balance needs, resources, and technical feasibility in a way that ensures the S&T community is responsive to warfighter priorities. Once S&T projects are underway, technology development provides opportunities to demonstrate feasibility and management decisions must be made along the way by S&T developers--with input from the acquisition community--on which technologies should be developed to a final state that would facilitate technology transition. DOD Has a Variety of Technology Transition Programs That Support Military Users: We identified 20 technology transition programs, managed by OSD and the military departments, that provide structured mechanisms and funding to facilitate technology transition. These programs vary in size, mission, approach, funding, and technology maturity expectations. However, all are consistent in providing opportunities to transition technologies from the S&T community to a military user, such as an acquisition program or the warfighter in the field. Many of the transition programs are focused on rapid response to warfighter needs, which has received greater emphasis in the recent past because of the United States' operations in Iraq and Afghanistan. In addition, most programs target fairly mature technologies, which are suitable for final stages of development and demonstration. Collectively, the programs we reviewed use a mix of S&T and other research, development, test, and evaluation funding--about $7.9 billion obligated from fiscal years 2010 through 2012--to facilitate technology transition. OSD Has Seven Technology Transition Programs That Vary in Their Broader Missions, Approaches, and Funding: OSD manages seven transition programs that are intended to accelerate development, testing, and delivery of mature technologies that provide new solutions for military needs. Six programs are managed by the Deputy Assistant Secretary of Defense, Rapid Fielding, who reports to the Under Secretary of Defense for Acquisition, Technology, and Logistics. They include three programs authorized by Congress--the Defense Acquisition Challenge (DAC),[Footnote 2] Foreign Comparative Testing (FCT), and Rapid Innovation Fund (RIF)[Footnote 3]--and three programs established by DOD--Joint Capability Technology Demonstration (JCTD), Quick Reaction Fund (QRF), and Rapid Reaction Fund (RRF). In addition, OSD manages a Small Business Innovation Research (SBIR) program,[Footnote 4] which was established by Congress and has technology transition--described by the program as "commercialization"--as a tenet. The OSD programs share the general purpose to transition technologies, but vary in what type of technology developers and operational needs are targeted. For example, the Joint Capability Technology Demonstration program addresses joint warfighting needs of the combatant commands, in partnership with the military services, by demonstrating mature technology prototypes that may transition to acquisition programs or directly to the warfighter in the field. In contrast, the Foreign Comparative Testing program mission is to identify and test technologies that have already been developed by other countries and may have utility in addressing U.S. military needs. Other programs like the Rapid Innovation Fund and Small Business Innovation Research seek to leverage technology solutions from small businesses. And still others, like the Quick Reaction Fund and Rapid Reaction Fund programs, seek to target specific urgent conventional and irregular warfighting problems, respectively, and rapidly deliver technologies to operational users. Collectively, research, development, test, and evaluation funding for the OSD technology transition programs we reviewed included about $1.75 billion for fiscal years 2010 through 2012.[Footnote 5] The Joint Capability Technology Demonstration and Rapid Innovation Fund programs were the most substantial part of that funding--over $1.2 billion for fiscal years 2010 through 2012. The majority of OSD technology transition programs constrain project time frames to 2 years or less, since their focus is rooted predominately on providing funds to accelerate transition for technologies that rapidly respond to current or near-term military needs. Individual projects also represent relatively small funding commitments for the department, with $3 million on the high end of costs, although Joint Capability Technology Demonstration projects can be higher. It also is important to note that several of the programs utilize co-funding provided by prospective technology users to further advance projects. Table 1 provides further information on funding as well as project-related details for each OSD program. Table 1: OSD Technology Transition Programs Funding and Project Characteristics: Program: FCT; Year established: 1980; FY2010-2012 funding: $72.4 million; Typical funding per project: $2 million or less; Typical completion timeframe: 18-24 months; Examples of projects: * Complete arrestment gear breaking system that supports F-22 through full range and operational loads; * Enhanced fuse for 70mm warhead that will enable pilots to change settings in-flight to engage a wide range of targets. Program: SBIR[A]; Year established: 1982; FY2010-2012 funding: $179.8 million; Typical funding per project: $1.65 million or less; Typical completion timeframe: 42 months or less; Examples of projects: * Cloud based critical knowledge retriever will allow soldiers with portable devices to consult remote information services for critical information; * Active software defenses that will recognize a broad spectrum of threats and respond intelligently. Program: JCTD; Year established: 1994[B]; FY2010-2012 funding: $538.6 million; Typical funding per project: Varies; Typical completion timeframe: 12-36 months; Examples of projects: * Airborne weapons surveillance system will detect, classify, and relay locations of enemy artillery, rocket, and mortar fires; * Rapid reaction tunnel detection will provide detection and mapping technologies capable of detecting, characterizing, and interdicting tunnels. Program: DAC; Year established: 2002; FY2010-2012 funding: $80.9 million; Typical funding per project: $2 million or less; Typical completion timeframe: 18-24 months; Examples of projects: * Wearable battery power source that will provide increased flexibility and performance for the warfighter; * New high strength tempered glass that will provide blast and impact protection that is far superior to traditional options. Program: QRF; Year established: 2002; FY2010-2012 funding: $63.4 million; Typical funding per project: $2.5 million-$3 million; Typical completion timeframe: 12 months or less; Examples of projects: * Robust, lightweight fuel cell charging system capable of charging military batteries from a liquid fuel source; * Adaptive clutter map algorithm will mitigate false target detections resulting from wind farms within specific radar coverage area. Program: RRF; Year established: 2004[C]; FY2010-2012 funding: $115.2 million; Typical funding per project: $0.5 million; Typical completion timeframe: 6-18 months; Examples of projects: * Field-deployable, rapid DNA profiling and matching using automated prototype systems; * Small, lightweight, expendable, hand or canister launched airborne surveillance system that will provide real-time electro-optical/ infrared video. Program: RIF; Year established: 2011; FY2010-2012 funding: $700.0 million[D]; Typical funding per project: $3 million or less; Typical completion timeframe: 24 months or less; Examples of projects: * System designed to monitor areas of interest that automatically detects and warns of intruders; * C-130 aircraft corrosion monitoring system for inaccessible areas that reduces labor and maintenance cycles. Source: GAO analysis; DOD data. Notes: FY = fiscal year. [A] There are distinct phases in which each SBIR project may participate. SBIR Phase I provides for up to $150,000 for a 6-month period. If a project is selected to continue past Phase I, Phase II provides up to $1 million over 24 months. Some projects may also participate in Phase II Enhancement, which provides up to $500,000 in additional funding and an additional 12 months for projects where matching funding of equal value is provided by DOD non-SBIR programs or from an outside investor. [B] The JCTD program was originally known as the Advanced Concept Technology Demonstrations program. [C] RRF was funded through a reprogramming action in fiscal year 2004 for a single year to allow a rapid response to operations in Iraq and initiate high-priority S&T projects in the execution years. It received reprogrammed funding again in fiscal year 2005 and its own funding line in the fiscal year 2006 President's budget. [D] RIF funding is for fiscal years 2011-2012 because this pilot program was not established until fiscal year 2011. [End of table] The Military Departments' 14 Technology Transition Programs Differ in Size, Structure, and Expectations: The Departments of the Navy, Army, and Air Force have created distinct programs that rapidly respond to pressing warfighter needs, while also initiating programs that work to address needs that are more mid-term in nature. In addition, as directed by Congress, DOD has established a Manufacturing Technology (ManTech) program and Small Business Innovation Research program that provide technology transition opportunities.[Footnote 6] All three of DOD's military departments manage ManTech and Small Business Innovation Research programs. In the Navy, the Office of Naval Research (ONR) has a well-established technology transition focus. ONR's Office of Transition manages the Future Naval Capabilities (FNC) portfolio, which is the Navy's largest transition program--for which nearly $450 million was budgeted in fiscal year 2013. The program, which was initiated in 1999, seeks to provide the best technology solutions to address operational requirements, delivering technology products to acquisition programs that enhance capabilities within a 5-year time frame. ONR's Offices of Transition and Innovation also support rapid technology transition to the fleet, force, and acquisition communities via the Rapid Technology Transition (RTT), Technology Insertion Program for Savings (TIPS), TechSolutions (TS), and SwampWorks and Experimentation (SW/Exp) programs. The Army's efforts to facilitate transition are evolving. In 2011, the Army began a new Technology Enabled Capability Demonstration (TECD) program, which is replacing Army Technology Objectives-Demonstrations (ATO-D). According to Army officials, Technology Enabled Capability Demonstration characteristics were influenced by the Navy's Future Naval Capabilities program, with the Army identifying a need for a more integrated S&T approach. Key characteristics of Technology Enabled Capability Demonstrations include senior leadership input at the outset for S&T decisions, greater focus on providing capabilities, and better planning on how a technology will transition to provide an operational capability. When fully implemented, the Army expects Technology Enabled Capability Demonstrations to garner about 50 percent of its advanced technology development budget. The Army also established another technology transition effort in 2012, the Technology Maturation Initiative (TMI), which is intended to encourage a stronger partnership between the S&T and acquisition communities. The Army's Rapid Equipping Force (REF) program straddles the boundary between being a technology transition program and rapid acquisition program, scanning the S&T base for technologies that require limited development, testing, or both, as well as readily available technologies that can be delivered immediately to the warfighter to fill capability gaps. Similar to the Army, the Air Force is modifying its approach to technology transition. The Advanced Technology Demonstration (ATD) program is an established Air Force Research Laboratory program that provides technology transition opportunities, though the size of the program has declined in recent years. According to Air Force officials, a challenge for the Advanced Technology Demonstration program is that priorities of the major commands, such as Air Combat Command, can change due to environmental factors, internal analysis, and funding constraints. This can result in ongoing Advanced Technology Demonstration projects not being completed because of new, higher priorities overtaking existing projects. A new program-- Flagship Capability Concepts (FCC)--is expected to provide a mechanism through which the Air Force can ensure commitments from senior leadership as well as from the major commands to transition technologies to the acquisition community. Specifically, Flagship Capability Concepts are vetted through Air Force senior leadership to ensure they align with strategic priorities and have clear intent for transition. Finally, the Core Process 3 (CP3) program provides the Air Force with its own program focused on rapidly responding to urgent needs. In total, the military departments' technology transition programs received over $6 billion for fiscal years 2010 through 2012. Over 40 percent of this funding--about $2.6 billion--supported DOD's Small Business Innovation Research program, which in addition to pursuing technology development that leads to technology transition, also supports early-stage research and development to explore technology feasibility. In addition to the Small Business Innovation Research funding, the military departments made substantial investments in their other technology transition programs during the fiscal year 2010- 2012 timeframe--about $1.7 billion by the Navy, over $1.5 billion by the Army, and nearly $370 million by the Air Force. Table 2 provides a distribution of funding and project-related details for each of the military department's transition programs. Table 2: Military Departments' Transition Program Funding and Project Characteristics: All military departments: Program: ManTech[A]; Year established: 1956; Funding, FY2010-2012: $465.9 million; Typical funding per project: Varies; Typical completion timeframe: Varies; Examples of projects: * Automated application of rotorcraft blade erosion coating; * Submarine material management system to reduce material delivery lead time and increase material availability at build site. Program: SBIR[B]; Year established: 1982; Funding, FY2010-2012: $2.56 billion; Typical funding per project: $1.65 million or less; Typical completion timeframe: 42 months or less; Examples of projects: * Aerosol mass spectrometer improvement for measuring the size and chemical composition of submicron particles in real-time from an aircraft; * Air-activated flameless heating without water to heat food for warfighters in the field. Navy: Program: FNC; Year established: 1999; Funding, FY2010-2012: $1.3112 billion; Typical funding per project: Varies; Typical completion timeframe: 3-5 years; Examples of projects: * Improved means of rapidly testing donor's and recipient's blood types to decrease risk of transfusion reactions/transmitted disease; * Submarine track and trail provides covert forward area surveillance using unmanned underwater vehicle technologies. Program: RTT; Year established: 2000; Funding, FY2010-2012: $59.1 million; Typical funding per project: $2 million or less; Typical completion timeframe: 24 months or less; Examples of projects: * Variable exhaust nozzle seals with chevrons to provide ~50 percent reduction in engine exhaust noise of the F-18 engines; * Continuous active sonar displays, trackers, and system integration for anti-submarine warfare. Program: SW/Exp; Year established: 2000; Funding, FY2010-2012: $77.0 million; Typical funding per project: $1-$2 million/$1 million or less; Typical completion timeframe: 12-24 months/12 months; Examples of projects: * Submarine advanced control effectors that improve low speed maneuvering and use of submerged operating envelope; * Mobile fuel cell technology for unmanned underwater vehicles. Program: TS; Year established: 2001; Funding, FY2010-2012: $26.5 million; Typical funding per project: $1 million or less; Typical completion timeframe: 12 months or less; Examples of projects: * New mortar fire control unit loaded that can be used day or night and also incorporates an integrated heat shield; * Catapult capacity selector valve calculator provides electronic energy levels information to launch aircraft from aircraft carriers. Program: TIPS; Year established: 2004; Funding, FY2010-2012: $24.9 million; Typical funding per project: $2 million or less; Typical completion timeframe: 24 months or less; Examples of projects: * High durability, environmentally friendly, spray applied coatings to reduce Navy platforms' maintenance and sustainment costs; * High bay lighting systems that improve performance and reduce maintenance via high output fluorescent and light emitting diodes. Army: Program: REF; Year established: 2003; Funding, FY2010-2012: $53.4 million; Typical funding per project: $1 million or less; Typical completion timeframe: 3-6 months; Examples of projects: * Integrated-blast effect sensor suite that monitors blasts from explosions for head trauma assessment; * SandFlea, a robot with the capability of jumping 30 meters into the air that detects improvised explosive devices. Program: ATO-D; Year established: 2005; Funding, FY2010-2012: $1.1697 billion; Typical funding per project: Varies; Typical completion timeframe: 2-4 years; Examples of projects: * Rotorcraft drive system technologies that increases horsepower and reduces weight, enhancing payload/range and affordability; * Improvised explosive device/mine detection and neutralization capabilities for route clearance vehicles. Program: TECD; Year established: 2011; Funding, FY2010-2012: $128.6 million; Typical funding per project: Varies; Typical completion timeframe: 36-60 months; Examples of projects: * Develop and prototype seats for occupant centric Army vehicles; * Modular, scalable, tailorable soldier ensemble and small unit surveillance equipment to improve protection and effectiveness. Program: TMI; Year established: 2012; Funding, FY2010-2012: $8.3 million; Typical funding per project: Varies; Typical completion timeframe: Varies; Examples of projects: * Next generation wireless communications for logistics; * Advanced weapon sight technology to support a family of sights. Air Force: Program: ATD; Year established: 1999; Funding, FY2010-2012: $126.3 million; Typical funding per project: Varies; Typical completion timeframe: 6 years or less; Examples of projects: * Light-weight, portable and handheld evaluation tool for low observable signature assessment systems on F-35 and F-22; * Learning management system for distributed mission operations and live, virtual, and constructive operations for training. Program: CP3; Year established: 2005; Funding, FY2010-2012: $21.0 million; Typical funding per project: $2.2 million; Typical completion timeframe: 12-13 months; Examples of projects: * Modular pod for tactical unmanned air vehicles to pre-deploy wind measuring to support cargo aircraft single pass airdrops; * Lightweight, highly maneuverable, high speed small air vehicle with automatic image-based tracking to engage fleeting targets. Program: FCC; Year established: 2011; Funding, FY2010-2012: $88.3 million; Typical funding per project: Varies; Typical completion timeframe: 6 years or less; Examples of projects: * Series of high velocity penetrating weapon technologies; * Selective cyber operations technology integration that provides a framework from which to launch graphic user interface programs. Source: GAO analysis; DOD data. Note: FY = fiscal year. [A] The distribution of ManTech funding across the military department components includes - Air Force, $131.9 million; Army, $177.1 million; and Navy, $156.9 million, respectively. [B] The distribution of SBIR funding across the military department components includes - Air Force, $1,050 million; Army, $633.0 million; and Navy, $877 million, respectively. [End of table] Programs Differ in the Maturity of Technologies They Accept and Transition: Beyond funding availability and technology turnaround times, technology maturation plays a key role in technology transition. DOD uses Technology Readiness Levels (TRL) to measure technology maturity using a scale of one to nine. Level one starts at the lowest level of technology readiness, where scientific research begins to be translated into applied research and development. TRL 1, for example, may begin with a paper study of a technology's basic properties. It proceeds through laboratory demonstrations, and ends with an application of the technology in its final form and under mission conditions. DOD generally expects a TRL 6 or higher for technologies to be included in a new acquisition program.[Footnote 7] Figure 3 portrays each transition program's TRLs expectations for a technology to be selected for inclusion in the program and for a technology at project completion. For example, OSD's Defense Acquisition Challenge program typically accepts projects that are already a TRL 7 or higher, though on occasion will accept projects at a TRL 6, and will advance the technologies to as high as a TRL 9 before project completion and transition. Further details on the demonstrated maturity associated with each TRL are provided in appendix III. In addition to technology maturity, the figure distinguishes programs that target conventional transition with longer project timelines and with acquisition programs often the intended recipients, and programs focused on rapid response to warfighter needs. Figure 3: Technology Maturity and Typical Project Duration for Transition Programs: [Refer to PDF for image: grid] OSD and military departments: Small Business Innovation Research (SBIR): TRL 1 to TRL 7; Mixed program (conventional and rapid - varies by project). Military departments: Manufacturing Technology (ManTech): TRL 3 to TRL 9; Mixed program (conventional and rapid - varies by project). OSD: Rapid Reaction Fund (RRF): TRL 4 to TRL 8; Rapid reaction program (less than or equal to 2 years). Joint Capability Technology Demonstration (JCTD): TRL 5 to TRL 9; Mixed program (conventional and rapid - varies by project). Quick Reaction Fund (QRF): TRL 6 to TRL 8; Rapid reaction program (less than or equal to 2 years). Rapid Innovation Fund (RIF): TRL 6 to TRL 9; Rapid reaction program (less than or equal to 2 years). Defense Acquisition Challenge (DAC): TRL 6 to TRL 9; Rapid reaction program (less than or equal to 2 years). Foreign Comparative Testing (FCT): TRL 8 to TRL 9; Rapid reaction program (less than or equal to 2 years). Navy: Future Naval Capabilities: TRL 3 to TRL 7; Conventional program (greater than 2 years). TechSolutions (TS): TRL 4 to TRL 6; Rapid reaction program (less than or equal to 2 years). SW/Exp (SwampWorks and Experimentation: TRL 5 to TRL 6; Rapid reaction program (less than or equal to 2 years). Rapid Technology Transition RTT): TRL 6 to TRL 9; Rapid reaction program (less than or equal to 2 years). Technology Insertion Program for Savings (TIPS): TRL 6 to TRL 9; Rapid reaction program (less than or equal to 2 years). Army: Army Technology Objectives - Demonstration (ATO-O): TRL 4 to TRL 7; Conventional program (greater than 2 years). Technology Enabled Capability Demonstration (TECD): TRL 4 to TRL 7; Conventional program (greater than 2 years). Technology Maturation Initiative (TMI): TRL 4 to TRL 7; Conventional program (greater than 2 years). Rapid Equipping Force (REF): TRL 6 to TRL 9; Rapid reaction program (less than or equal to 2 years). Air Force: Advanced Technology Demonstration (ATD): TRL 3 to TRL 6; Conventional program (greater than 2 years). Flagship Capability Concept (FCC): TRL 3 to TRL 7; Conventional program (greater than 2 years). Core Process 3 (CP3): TRL 5 to TRL 9; Rapid reaction program (less than or equal to 2 years). Source: GAO analysis of DOD data. [End of figure] While the figure shows the full range of TRLs that may be accepted by programs, programs generally prefer to limit the amount of less mature technologies they will accept, particularly because immature technologies pose greater risk to successfully completing projects in the short time frames that many programs demand. Higher TRLs tend to be desired by most OSD transition programs as well as several military department programs, such as the Navy's Technology Insertion Program for Savings, Army's Rapid Equipping Force, and the Air Force's Core Process 3 program. Because these programs are focused on rapidly delivering technology solutions for the warfighter, many require technologies of TRL 6 or higher for each project from the outset. Figure 3 also provides context on the outlook for technology transition programs that are focused more on addressing anticipated technology needs for future warfighters. Military department programs like the Navy's Future Naval Capabilities, Army's Technology Enabled Capability Demonstrations, and Air Force's Flagship Capability Concepts, encompass a broader focus that primarily targets technology development opportunities that are expected to meet capability needs further into the future. Typical transitions for technologies in these programs are expected to occur in 3 to 6 years, and may be to acquisition programs that can take several more years to actually deploy the technologies for warfighter use. Because there is more time afforded by these programs to develop technologies, they are able to accept less mature technologies (which is demonstrated in figure 3), as these types of programs are shown to select technologies that are TRLs 3 and 4 and advance them to TRL 6 or beyond. Technology Transition Programs Provide Technologies to Military Users, But Tracking of Project Outcomes and Benefits after Transition Is Limited: Most programs that we assessed track whether their projects were completed and successfully transitioned to intended users. On average, programs reported a historical transition rate of over 70 percent for their technology transition projects. The vast majority of these projects resulted in technologies transitioning to acquisition programs or directly to the warfighter. However, about one-quarter of the projects transitioned to other organizations, such as test and evaluation centers and industry, for further development. For the most part, the programs do not track their projects beyond transition, which limits their ability to know and report final outcomes for transitioned technologies and any associated benefits DOD achieved from those technologies. Most Programs Reported the Majority of Their Projects Successfully Transitioned: Technology transition programs reported rates of technology transition ranging from 56 to 86 percent. Newer programs--such as Army's Technology Enabled Capability Demonstrations and Technology Maturation Initiative, Air Force's Flagship Capability Concepts, and OSD's Rapid Innovation Fund--are still in the process of determining what steps they will take to measure transition outcomes. The DOD Small Business Innovation Research program relies heavily on self-reporting by small businesses for transition data, and because there is no requirement that all small businesses participating in the program report transitions, comprehensive program data on transitions is not available. The Air Force Advanced Technology Demonstration program does not track transition outcomes for completed projects, which precludes the program from readily identifying a transition rate. Table 3 provides details on reported transition rates by program. Table 3: Historical Transition Rates Reported by OSD and Military Departments Technology Transition Programs: All: Transition program: SBIR--OSD and military departments; Transition rate: Not available. OSD: Transition program: DAC; Transition rate: 80%. Transition program: FCT; Transition rate: 73%. Transition program: JCTD; Transition rate: 80%. Transition program: QRF; Transition rate: 65%-70%. Transition program: RIF; Transition rate: Not available. Transition program: RRF; Transition rate: Navy: 60%-65%. Navy: Transition program: FNC; Transition rate: 86%. Transition program: ManTech; Transition rate: 57%[A]. Transition program: RTT; Transition rate: 62%. Transition program: SwampWorks; Transition rate: 75%. Transition program: TechSolutions; Transition rate: 75%. Transition program: TIPS; Transition rate: 70%. Army: Transition program: ATO-D; Transition rate: 83%. Transition program: ManTech; Transition rate: 79%. Transition program: REF; Transition rate: 56%. Transition program: TECD; Transition rate: Not available. Transition program: TMI; Transition rate: Not available. Air Force: Transition program: ATD; Transition rate: Not available. Transition program: CP3; Transition rate: 75%-80%. Transition program: FCC; Transition rate: Not available. Transition program: ManTech; Transition rate: 70%-80%. Source: GAO analysis; DOD data. Note: Programs provided self-reported historical rates of transition either as ranges or specific estimates and the timeframes for which the rates applied varied. [A] Navy ManTech tracks implementation by the manufacturer, which is only achieved by a subset of their projects that transition to a program office or user. [End of table] While most programs have demonstrated success in transitioning technologies to military users, differences in expectations across programs are key to understanding what programs provide to the warfighter. For example, programs have different definitions for what project outcomes are considered as "transitions," particularly as they relate to transition recipients. To gain a better understanding of transition outcomes for each program, we evaluated who the recipients of transitioned technologies were in recent years. Figure 4 shows the distribution of transitions reported by the transition programs we assessed for fiscal years 2010-2012 that fall into one of three categories--transition to an acquisition program, transition directly to the field for use by the warfighter, and transitions to "other" users such as S&T organizations, test and evaluation centers, or industry. Figure 4: Technology Transition Results Reported by Programs for Completed Projects, Fiscal Years 2010-2012: [Refer to PDF for image: stacked horizontal bar graph] Number of projects: All: Small Business Innovation Research; Number of projects: NA. OSD: Defense Acquisition Challenge; Acquisition program: 44; Direct to field: 1; Other: 1; Total: 46. Foreign Comparative Testing; Acquisition program: 43; Direct to field: 3; Other: 1; Total: 47. Joint Capability Technology Demonstration; Acquisition program: 15; Direct to field: 23; Other: 5; Total: 43. Quick Reaction Fund; Acquisition program: 5; Direct to field: 10; Other: 0; Total: 15. Rapid Reaction Fund; Acquisition program: 4; Direct to field: 12; Other: 18; Total: 34. Navy: Future Naval Capabilities; Acquisition program: 53; Direct to field: 0; Other: 4; Total: 57. Manufacturing Technology; Acquisition program: 34; Direct to field: 0; Other: 6; Total: 40. Rapid Technology Transition; Acquisition program: 10; Direct to field: 1; Other: 1; Total: 12. Swampworks/Experimentation; Acquisition program: 13; Direct to field: 0; Other: 43; Total: 56. TechSolutions; Acquisition program: 15; Direct to field: 10; Other: 3; Total: 28. Technology Insertion Program for Savings; Acquisition program: 4; Direct to field: 0; Other: 1; Total: 5. Army: Army Technology Objectives - Demonstration; Acquisition program: 52; Direct to field: 3; Other: 7; Total: 62. Manufacturing Technology; Acquisition program: 12; Direct to field: 0; Other: 7; Total: 19. Rapid Equipping Force; Acquisition program: 2; Direct to field: 7; Other: 0; Total: 9. Air Force: Advanced Technology Demonstration; Number of projects: NA. Core Process 3; Acquisition program: 3; Direct to field: 6; Other: 0; Total: 9. Manufacturing Technology; Acquisition program: 39; Direct to field: 0; Other: 24; Total: 63. Source: GAO analysis of DOD data. Note: Figure does not include the recently initiated Flagship Capability Concept, Rapid Innovation Fund, Technology Enabled Capability Demonstration, and Technology Maturation Initiative programs because no data are available. Additionally, program totals may omit some fiscal year 2012 results based on the date of program reporting. [End of figure] Collectively, these programs reported that 545 projects transitioned in fiscal years 2010 through 2012. The vast majority of the projects resulted in technologies transitioning to acquisition programs or directly to the warfighter in the field. However, some programs take a broader view and consider the transfer of technologies to other organizations for further development and demonstration, such as test and evaluation centers, or to industry as technology transition. In total, nearly one-fourth of reported transitions constituted this type of transition. For example, Navy's SwampWorks and Experimentation, which has a mission to focus on high-risk technologies and concepts, stated that successful transition can be showing the feasibility of a high risk technology to allow further development of it at another level in the Navy's S&T community. The following are a few examples of transition program projects that provide context for the array of technologies and the users supported by these transitions. Persistent Ground Surveillance System: A Joint Capability Technology Demonstration project that transitioned a family of wide area surveillance systems consisting of intelligence, surveillance, and reconnaissance technologies, as well as communications technologies to forward operating bases in Afghanistan. The systems were integrated into tailored tower and aerostat platforms that are in use and sustained by the Army. Mosquito prototype: An Air Force Core Process 3 project that provided a field-tested prototype of a backpack-carried system for rapidly determining ground bearing strength for landing aircraft in remote sites. Following evaluations by users, the Air Force funded efforts to ruggedize and deliver additional units to the field for use. Tactical Control System Digital Video Enhancements: A Navy Rapid Technology Transition project to upgrade the computer interface and video processing capability for the Vertical Takeoff and Landing Tactical Unmanned Aerial Vehicle (VTUAV) system's ground control station. The VTUAV acquisition program has already purchased and fielded the complete system upgrade, and plans to acquire more of the systems. Few Programs Track Project Outcomes and Benefits after Transition: As GAO has reported in the past, tracking technology transitions and the impact of those transitions, such as cost savings or deployment of the technology in a product, provides key feedback that can inform the management of programs.[Footnote 8] In particular, GAO found that leading commercial companies tracked technology transition not only to enable them to measure success, but also to assess their processes and determine what changes are necessary to improve transition rates. In addition, tracking outcomes provides the laboratories and management with feedback on the impact technology investments have on their companies. The transition programs we reviewed predominately stop tracking a project once the program is no longer funding it, or shortly thereafter, and generally have limited insight into benefits realized by users for their transition project portfolios. For example, most program officials we spoke to indicated they lack the ability to obtain feedback that would indicate a technology that transitioned to a weapon system acquisition program was ultimately integrated into the weapon system and deployed for use by the warfighter, or was not integrated into the system, but did support competition that led to enhanced capabilities or reduced cost for the weapon system. A few programs did identify processes for tracking transitioned projects through the acquisition process and to deployment within a fielded weapon system. The Navy uses a Transition Review Board to monitor completed projects from the Future Naval Capabilities, Rapid Technology Transition, and Technology Insertion Program for Savings programs. The board determines and reports on whether transitioned projects are utilized in systems that support Navy warfighters. The Navy determined, for example, that of the 155 technology products the Future Naval Capabilities program delivered to acquisition programs between fiscal years 2006-2011, 21 percent were subsequently deployed to fleet forces, 35 percent were still with the acquisition programs, and 44 percent failed to deploy. For projects that do not successfully deploy, the board assesses whether there are other benefits achieved, such as technologies leveraged for follow-on S&T work. The board also identifies obstacles to transition, such as loss of interest by the user or inadequacy of funding. These findings, along with a detailed one-page summary for each project, are then used to inform the Navy's annual review process. We found that by maintaining this level of tracking, the Navy is better aware of the benefits and obstacles associated with a substantial portion of their S&T portfolio, which may better inform decisions made by Navy leadership. In addition to tracking from transition to deployment, another notable project outcome measure we observed among the transition programs-- specifically within the ManTech program--is the amount of cost savings or cost avoidance for projects. ManTech is somewhat unique when compared to other transition programs because its mission is focused on reducing acquisition and supportability costs of DOD weapon systems and improving manufacturing and repair efficiency for weapon systems. The ability of the Navy, Army, and Air Force components of ManTech to provide actual as well as projected cost-reduction benefits coming from transition and implementation of their projects presents an opportunity to clearly demonstrate value to the warfighter and value of the program. A couple of examples for ManTech cost savings and cost avoidance measures are illustrated by the projects below. Energy Storage Manufacturing Very High Power Batteries: Army project that reportedly has produced $20 million in cost avoidance benefit by developing an automated lithium-ion battery manufacturing line to include improvements that yield affordable, high-power battery packs. Virginia Class Submarine Affordability Initiative: The Navy estimates that this effort has already yielded over $21 million in cost savings per hull through a series of projects transitioned to the acquisition program that provides manufacturing efficiencies. Programs Facilitate Transition Through Sound Project Selection and Management Practices: As GAO has reported in the past, technology transition programs need to establish disciplined selection and management processes as well as tools to ensure that new technologies can be effectively transitioned to a military user.[Footnote 9] Programs must have processes in place that support the selection of projects that have realistic schedule and cost expectations, as well as sufficiently mature technologies. In addition, early project endorsement from intended users and other key stakeholders, including senior leadership and S&T developers, is critical. Once selected, projects require effective management to ensure technology risks are minimized, costs and schedules are maintained, and transition commitments are confirmed. To achieve this, stakeholders must engage in regular communication during project execution. Formal agreements and other project assessment measures also are important to ensure projects stay on track and stakeholders sustain their commitments to transition. In conducting this review, we found that OSD and military department technology transition programs make use of these practices to varying degrees. All of the programs we reviewed have established project selection processes that strive to select relevant and feasible technologies that have strong stakeholder support. However, management practices varied across programs, with some programs using more robust, formal tools, such as technology transition agreements, to guide project management and communicate project progress and expectations. Clear Expectations for Cost, Schedule, and Technology Maturity Objectives Are Central to Project Selection and Management: Establishing a path to facilitate technology transition begins with each transition program's ability to select relevant and feasible technologies to meet military needs. Programs must identify technologies that have the potential to provide new or enhanced capabilities, and can generally be matured or demonstrated in a timely manner with modest program investments. Identifying viable candidate technologies requires that programs clearly communicate what capabilities are needed and solicit proposals from the right audience. Most of the transition programs we reviewed identify and set forth high-priority capability needs or topics for project solicitations based on input from some combination of the S&T and acquisition communities, operational users, and senior military leadership. Having individuals with different focuses involved in generating topics helps ensure that the right technology projects will be proposed and that projects will have interested users. A limited number of programs we reviewed do not set topics, but do require submitted project proposals to demonstrate a capability need. For example, Navy TechSolutions, which has a relatively small budget--about $9 million per year--offers an open invitation for warfighters in the field to propose projects based on identified capability needs. Regardless of solicitation approach, all transition programs we reviewed have established criteria for evaluating whether to pursue proposed projects. As we previously identified, each program has its own project cost and timeline constraints, as well as technology maturity criteria in place to support projection selection. Clear expectations for these factors can reduce the risk of a technology failing to transition because of cost, schedule, or technology maturity concerns. Early Project Endorsements by Stakeholders Can Influence Transition: Securing comprehensive stakeholder endorsements for projects from their outset is another important influence on whether projects transition. Most of the transition programs we reviewed have processes or mechanisms they use to foster some type of commitment from prospective users to transition projects once completed. However, officials in some programs also reported that garnering senior leadership support for proposed projects is critical, as it ensures that projects are aligned with the priorities of department leadership as well as user needs. For example, the Air Force's Flagship Capability Concept program emphasizes senior leadership involvement by requiring all projects to be approved by the Air Force Requirements Oversight Council. Approval is supported by recommendations from the Air Force S&T governance structure, which is composed of senior-level leadership from Air Force headquarters and major commands. According to an Air Force official, this governance structure serves to ensure S&T efforts are properly aligned with Air Force needs, thereby facilitating transition and deployment to the operational community. The commitment of sponsor funding, or co-funding is another means used to solidify stakeholder endorsement for transition projects. The investment by prospective users serves as an endorsement of sorts, as it can create greater buy-in for transition since the intended users have a monetary stake in the project. Specifically, we found that several of the transition programs we reviewed require or encourage prospective users to help fund projects. For example, programs like OSD's Foreign Comparative Testing and Rapid Reaction Fund and the Navy's Rapid Technology Transition program do not require co-funding but do use it in some cases. The Joint Capability Technology Demonstration program requires that at least 50 percent of a project is co-funded, and program officials noted that their projects normally have multiple sponsors that provide funding. Stakeholder Commitment and Communication Are Key to Effective Management of Transition Programs and their Projects: Establishing clear and consistent commitments and communication channels among stakeholders is fundamental to managing transition projects and achieving transition. Senior leadership plays a key role in initiating and reinforcing effective program management practices. At the organizational level, we found several examples, such as with the Army, where officials indicated that their top S&T leadership championed institutional changes to emphasize transition, including the introduction of the Technology Enabled Capability Demonstration and Technology Maturation Initiative programs. These programs are intended to produce an integrated Army S&T strategy that uses dedicated transition programs to more effectively respond to warfighter needs. At the program level, many program officials indicated that senior leadership engagement, particularly in providing oversight for projects through to transition, is essential to having an effective program. We found the Future Naval Capabilities program provides a good example of senior leadership positively affecting project management activities. Specifically, due to funding constraints in its fiscal year 2013 S&T budget, Navy senior leadership supported the termination of ongoing Future Naval Capabilities projects that were determined to be lower priorities so that new, higher priority projects could be pursued. Navy officials stated that this type of awareness and understanding at senior levels enables the Future Naval Capabilities program to make efficient decisions that are less likely to meet resistance and that support the highest priority projects being developed for transition opportunities. Several transition programs also emphasized the relationship between "working-level" stakeholders--S&T developers and acquisition programs or warfighters in the field--when discussing the keys to technology transition. These stakeholders manage expectations throughout a project and ensure it will meet user needs. This reduces the risk of completed projects languishing because funding is not available or because user requirements have changed, or both. Some programs that we reviewed use integrated product teams, which may be composed of individuals representing the requirements, acquisition, operational, and S&T communities, among others, to facilitate continuous communication with stakeholders and ensure that transition planning is on track. In the case of the Navy, integrated product teams identify capability gaps, provide input on which S&T projects may address those gaps, assess project progress, make sure transition strategies remain valid, and confirm funding is aligned to support transition. According to Navy officials, the results of integrated product team efforts also support information sharing across senior-and working-level stakeholders to validate development status and transition planning activities. Transition Agreements and Other Project Assessment Tools Are Used to Formalize Project Expectations and Help Sustain Commitments to Transition Technologies: We found a range of management tools being used by some transition programs that underlie support and communication among stakeholders. Technology transition agreements are used by 9 of the 20 transition programs we reviewed to manage projects.[Footnote 10] These are "good- faith" agreements between stakeholders that document the expectations for developing, demonstrating, delivering, and integrating technologies into systems, or using them as standalone products. Agreement specifics vary by program and can be tailored for each project, but typically outline technology and readiness metrics, such as cost, schedule, and performance parameters that labs must meet for transition to occur. As GAO has reported in prior work, these metrics help sustain a strong transition path by providing a formal way to track progress against requirements.[Footnote 11] Also as GAO has reported, technology transition agreements are used by stakeholders to make informed decisions on projects that are not meeting expectations. For example, if the labs cannot develop a technology within cost and schedule expectations, or cannot meet specified performance characteristics, the prospective user can terminate the agreement and pursue alternative technology options. The Joint Capability Technology Demonstration program within OSD and the Navy's Future Naval Capabilities program both require technology transition agreements for all projects and provide insights into the content and value these agreements can offer to stakeholders. The Joint Capability Technology Demonstration program agreements, at a minimum, must outline: * Operational need: * Proposed technical solution: * Transition target information: * Transition requirements: * Integration strategy: * Business case: * Risks: * Costs and schedule: * Project points of contact: We found the Future Naval Capabilities program uses technology transition agreements as management tools to increase the level of documented commitment as a project progresses over time. To accomplish this, the program has three levels for agreements that reflect the requisite knowledge available at different phases of a project. Key elements of an initial agreement include a basic project description, identification of initial exit criteria, a high-level integration strategy, and a likely transition funding source. As a project progresses, the other two levels of agreement require increasing commitment and specificity of requirements from stakeholders to develop, deliver, and integrate a Future Naval Capabilities project into an acquisition program or other form of deployment. Key elements of the second and third tier agreements involve refining and finalizing project descriptions, detailing exit criteria, providing greater specificity about the integration strategy, and providing estimates for transition costs and eventually executing transition funding. Stakeholders review the agreements annually to revalidate the commitments laid out within the document. We also found Transition Commitment Level (TCL) assessment tools--used by the Joint Capability Technology Demonstration and Future Naval Capabilities programs--offer another means of validating that transition programs are investing in projects that have a firm transition commitment from prospective users. These tools provide scorecards that chart how well-defined the fundamental characteristics that support a strong commitment to transition projects are at a given point in time. The Future Naval Capabilities program uses a single TCL tool that documents level of transition commitment from project start to completion. The Joint Capability Technology Demonstration program uses two TCL assessment tools--one to identify transition commitment to support project selection and one to track commitment through project execution and to completion. Figure 5 shows the TCL assessment tool used for Joint Capability Technology Demonstration project execution and completion. Figure 5: Example of Transition Commitment Level Project Evaluation Tool: [Refer to PDF for image: table] Transition Commitment - JCTC in execution/completion: Green: successful transition: 1. A capability is funded or scheduled to be funded as part of an existing acquisition program of record or activity has begun to initiate a new acquisition program of record. 2. Some or all of the JCTC fieldable prototypes are fielded and sustained. 3. Some or all of the JCTC demonstrated components are placed on General Services Administration Schedule. Yellow: limited transition: 1. Negotiations are underway for incorporation into an acquisition program of record, use of fieldable prototypes, or placement on General Services Administration Schedule with no or partial funding. Red: No transition: 1. No transition occurred. 2. Technology returned to technology base (S&T). 3. Unable to determine if technology will transition to acquisition program of record, fieldable prototype, or General Services Administration Schedule. Source: Joint Capability Technology Demonstration program. [End of figure] As shown by the figure, a TCL tool provides a formal way for determining which proposed and active projects have a high, medium, and low probability to transition. For example, in order for a Joint Capability Technology Demonstration project to be designated as "green," project managers must show the project has a clear path to one of the three transition recipients as defined by the program. The TCL tools used by both the Joint Capability Technology Demonstration and Future Naval Capabilities programs provide insights into the progress of projects based on established benchmarks. In addition, Navy officials stated that the TCL tool used by Future Naval Capabilities provides information that can be used by program managers to inform investment decisions. In particular, the TCL tool can help identify active projects that may require greater attention to solidify transition commitments from stakeholders, and can also be used to support decisions to end projects that no longer have stakeholder support. Eleven of the programs we reviewed do not use formal technology transition agreements, but incorporate some of the elements of agreements in managing projects. For example, OSD's Foreign Comparative Testing program requires documentation of a valid need for a technology when a project starts and a letter of endorsement from a prospective military user that supports procuring the technology at completion. The Rapid Reaction Fund program does not require commitments from users, but does require them to identify a potential transition path at project start and then the program will follow-up with the users to assess their interest in the technology if it is successfully developed. Although prior GAO work found that technology transition agreements do not guarantee transition success because requirements and funding changes can occur, the agreements are useful tools to solidify expectations and secure user commitment.[Footnote 12] At the time, GAO recommended that DOD expand the use of technology transition agreements. DOD concurred with the recommendation, but did not identify specific actions it planned to take to do so. Conclusions: For the most part, technology transition programs we reviewed have been reasonably successful in delivering technologies to military users. However, a limited number of programs do not have metrics established to track completed projects. In addition, few programs measure whether transitioned technologies actually result in a benefit to users. For example, many programs track whether a technology is delivered to an acquisition program of record, but often have limited or no insight into whether the technology resulted in enhanced performance or new capabilities, cost savings, or reduced times for testing and evaluation and system integration. Without some measure of technology implementation or other benefit after transition, questions remain as to whether programs are providing the right technologies at the right time to users, are using effective approaches to select, develop, and transition technologies, and are providing tangible benefits. The transition programs we reviewed generally have disciplined project selection and management processes in place to facilitate the transition of technologies to intended users. To varying degrees, programs emphasize the need for early and sustained commitments from senior military leadership, S&T developers, and military users in order to ensure projects are needed, have a sound basis, and risks are reduced. A key mechanism some programs use for obtaining these commitments are formal tools, such as technology transition agreements and technology commitment level assessments. These tools help to clarify expectations, hold stakeholders accountable for what they must deliver, and gauge progress towards achieving project objectives. There may be opportunities for more widespread use of these tools among the programs we reviewed, which could help strengthen technology transition success. Recommendations for Executive Action: To improve visibility and management of the department's efforts to transition technologies to support the needs of the warfighter, we recommend that the Secretary of Defense take the following two actions: * Require that all technology transition programs track and measure project outcomes, to include not only whether technologies transitioned to an intended user but also the longer-term impact of whether the technologies benefited acquisition programs or military users in the field. * Assess transition programs to identify opportunities for more widespread use of existing transition management tools, such as technology transition agreements and technology commitment level evaluation mechanisms. Agency Comments and Our Evaluation: We provided a copy of a draft of this report to DOD for review and comment. In written comments on the draft, DOD partially concurred with our first recommendation and fully concurred with the second recommendation. DOD's comments appear in appendix IV. In responding to the first recommendation, DOD agreed that all transition programs should track and measure project outcomes, including how the technology or capability is being employed by the intended user. However, the department raised concerns that tracking and measuring technology project outcomes would be a labor-intensive and time-consuming process requiring significant investment. DOD also suggested that the recommendation be revised to have the department assess technology transition programs for opportunities to implement processes to track and measure project outcomes. We continue to believe that it is important that DOD have a formal means to track and document technology transition in order to demonstrate not only whether technology investments under these programs are reaching their intended users, but also that they are providing the desired benefits. We believe that DOD should require all programs to track and measure technology transition, and that the department's suggested revision to our recommendation could potentially remove any responsibility for programs to do so. Moreover, while we acknowledge that tracking and measuring project benefits following transition would require additional time and effort, we believe our recommendation provides DOD with significant flexibility to measure outcomes in different ways that would be less resource-intensive than DOD envisions. For example, rather than tracking all transitioned projects, the department could assess a subset of projects or establish mechanisms for military users to report back on the results. In addition, as we reported, the Navy has recently used an approach that could be adopted by other programs in DOD to measure potential benefits from transitioned projects. Specifically, the Office of Naval Research has employed a Technology Review Board to complete independent reviews of Navy transition programs and report whether transitioned projects were being used by the warfighter. Consequently, we have not revised our recommendation as suggested by DOD. In concurring with our second recommendation, the department agreed that all technology transition programs should identify opportunities to use existing transition management tools. DOD stated that it will ensure guidelines are published and widely disseminated to improve the understanding as to when, how, and under what circumstances transition management tools can be employed. We are sending copies of this report to appropriate congressional committees, the Secretary of Defense, and the Secretaries of the Air Force, Army, and Navy. In addition, this report will be available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions concerning this report, please contact me at (202) 512-4841 or by email at sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix V. Signed by: Michael J. Sullivan: Director, Acquisition and Sourcing Management: List of Committees: The Honorable Carl Levin: Chairman: The Honorable James Inhofe: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Richard Durbin: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Howard P. "Buck" McKeon: Chairman: The Honorable Adam Smith: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable C.W. Bill Young: Chairman: The Honorable Pete Visclosky: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Appendix I: Scope and Methodology: To identify what Department of Defense (DOD) programs exist that are dedicated to facilitating technology transition from the science and technology (S&T) community to a user, we reviewed DOD reports and documents. As a starting point, we reviewed a March 2012 report to Congress by the Under Secretary of Defense for Acquisition, Technology and Logistics that assessed technology transition programs within DOD. In conducting this assessment, DOD reviewed budget program elements to identify potential technology transition programs and then surveyed these programs to validate those whose primary function is technology transition. We discussed the programs identified in the DOD report with officials from the Office of the Secretary of Defense (OSD) and the military departments and made adjustments to our scope based on their inputs and our own analysis. The resulting list of programs, which is the focus of our review, includes 20 transition programs across OSD and the military departments--Air Force, Army, and Navy. While there are other technology transition activities that occur within the military departments and other DOD organizations, such as those within the Joint Improvised Explosive Device Defeat Organization and Defense Advanced Research Projects Agency, we limited our scope to programs within DOD dedicated to technology transition. We then interviewed officials from the programs, as well as other officials within OSD and the military departments affiliated with technology transition activities, to become familiar with their respective technology transition programs. We collected and analyzed information from the programs on their structures, processes, tools, and technology transition metrics used. To assess what technology transition programs are doing to meet their objectives, we reviewed program policies and procedures for selecting technology transition projects, monitoring and managing their progress, and determining transition outcomes. For the purpose of assessing the programs' activities--including projects started, completed, and transitioned to users--we gathered data from the programs for fiscal years 2010 through 2012. For the purpose of reviewing and reporting historical transition rates for the programs, we collected self-reported transition rates from program officials for those programs that could provide such information. We did not validate the data provided by the technology transition program officials, but reviewed and discussed the data with them and determined it was sufficiently reliable for the purposes of this report. To identify what practices may facilitate technology transition, we reviewed prior GAO studies on DOD technology transition and best practices for transition, interviewed officials from OSD and the military departments, and reviewed programmatic information and other related DOD documentation. Using this information, we were able to isolate a number of practices that influence a program's ability to transition technology from the S&T community to a prospective user. Additionally, we reviewed what practices were present in each program. In performing our work, we obtained information and interviewed OSD and military department officials from: OSD: Assistant Secretary of Defense, Research and Engineering, Arlington, Virginia; Defense Advanced Research Projects Agency, Arlington, Virginia. Army: Office of Science and Technology, Arlington, Virginia; Rapid Equipping Force, Ft. Belvoir, Virginia; Small Business Innovation Research and Manufacturing Technology programs, Aberdeen Proving Ground, Maryland. Air Force: Air Force Research Laboratory, Wright-Patterson Air Force Base, Ohio; Air Force Acquisition and Research, Arlington, Virginia. Navy: Office of Naval Research, Arlington, Virginia; Navy Research and Development, U.S. Naval Yard, Washington, D.C.; Deputy Assistant Secretary of the Navy for Research, Development, Test, and Evaluation, Arlington, Virginia. We conducted this performance audit from April 2012 to March 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: DOD Research, Development, Test, and Evaluation Budget Activities: Science and technology funding: DOD RDT&E Budget Activity: Basic Research (6.1); Description: Includes all scientific study and experimentation directed toward increasing fundamental knowledge and understanding in physical, engineering, environmental, and life sciences related to long-term national security needs. It is farsighted high payoff research that provides the basis for technological progress. Program elements in this category involve pre-Milestone A efforts, which is the point where entry into the Technology Development phase is approved. DOD RDT&E Budget Activity: Applied Research (6.2); Description: Varies from systematic mission-directed research beyond that done through basic research, to sophisticated breadboard hardware, study, programming and planning efforts that establish the initial feasibility and practicality of proposed solutions to technological challenges. The dominant characteristic is it is directed toward general military needs with a view toward developing and evaluating the feasibility and practicality of proposed solutions and determining their parameters. Program elements in this category involve pre-Milestone B efforts, also known as Technology Development phase tasks. DOD RDT&E Budget Activity: Advanced Technology Development (6.3); Description: Includes concept and technology demonstrations of components and subsystems or system models. The results of this type of effort are proof of technological feasibility and assessment of subsystem and component operability and producibility rather than the development of hardware for service use. Advanced Technology Development demonstrates the general military utility or cost reduction potential of technology when applied to different types of military equipment or techniques. Program elements in this category involve pre-Milestone B efforts, such as system concept demonstration, and generally have technology readiness levels of 4, 5, or 6. Projects in this category should have the goal of moving out of S&T and into the acquisition process within the future years defense program. Upon successful completion of projects that have military utility, the technology should be available for transition. Acquisition-based Funding: DOD RDT&E Budget Activity: Advanced Component Development & Prototypes (6.4); Description: Efforts necessary to evaluate integrated technologies, representative modes or prototype systems in a high fidelity and realistic operating environment. Includes system specific efforts that help expedite technology transition from the laboratory to operational use. Emphasis is on proving component and subsystem maturity prior to integration in major and complex systems and may involve risk reduction initiatives. Program elements in this category involve efforts prior to Milestone B acquisition program start. Completion of technology readiness levels 6 and 7 should be achieved for major programs. Program control is exercised at the program and project level. DOD RDT&E Budget Activity: System Development & Demonstration (6.5); Description: Post-Milestone B, conducting engineering and manufacturing development tasks aimed at meeting validated requirements prior to full-rate production. Prototype performance is near or at planned operational system levels. Characteristics involve mature system development, integration and demonstration to support Milestone C production decisions, and conducting live fire test and evaluation and initial operational test and evaluation of production representative articles. DOD RDT&E Budget Activity: RDT&E Management Support (6.6); Description: Efforts to sustain and/or modernize the installations or operations required for general RDT&E. Includes test ranges, military construction, maintenance support of laboratories, operation and maintenance of test aircraft and ships, and studies and analyses in support of the RDT&E program. Military construction costs directly related to major development programs are included. DOD RDT&E Budget Activity: Operational System Development (6.7); Description: Efforts to upgrade systems that have been fielded or have received approval for full-rate production and anticipate production funding in the current or subsequent fiscal year. Programs in this category involve systems that have received Milestone C production decision approval. Source: DOD. Note: RDT&E = research, development, test, and evaluation. [A] Milestones and associated activities are provided by DOD Instruction 5000.02, Operation of the Defense Acquisition System, Dec. 8, 2008. [End of table] [End of section] Appendix III: DOD Technology Readiness Levels: Technology readiness level (TRL): 1. Basic principles observed and reported; Description: Lowest level of technology readiness. Scientific research begins to be translated into applied research and development. Examples might include paper studies of a technology's basic properties. Technology readiness level (TRL): 2. Technology concept and/or application formulated; Description: Invention begins. Once basic principles are observed, practical applications can be invented. Applications are speculative, and there may be no proof or detailed analysis to support the assumptions. Examples are limited to analytic studies. Technology readiness level (TRL): 3. Analytical and experimental critical function and/or characteristic proof of concept; Description: Active research and development is initiated. This includes analytical studies and laboratory studies to physically validate the analytical predictions of separate elements of the technology. Examples include components that are not yet integrated or representative. Technology readiness level (TRL): 4. Component and/or breadboard validation in laboratory environment; Description: Basic technological components are integrated to establish that they will work together. This is relatively low fidelity compared with the eventual system. Examples include integration of ad hoc hardware in the laboratory. Technology readiness level (TRL): 5. Component and/or breadboard validation in relevant environment; Description: Fidelity of breadboard technology increases significantly. The basic technological components are integrated with reasonably realistic supporting elements so they can be tested in a simulated environment. Examples include high-fidelity laboratory integration of components. Technology readiness level (TRL): 6. System/subsystem model or prototype demonstration in a relevant environment; Description: Representative model or prototype system, which is well beyond that of TRL 5, is tested in a relevant environment. Represents a major step up in a technology's demonstrated readiness. Examples include testing a prototype in a high-fidelity laboratory environment or in a simulated operational environment. Technology readiness level (TRL): 7. System prototype demonstration in an operational environment; Description: Prototype near or at planned operational system. Represents a major step up from TRL 6 by requiring demonstration of an actual system prototype in an operational environment (e.g., in an aircraft, in a vehicle, or in space). Technology readiness level (TRL): 8. Actual system completed and qualified through test and demonstration; Description: Technology has been proven to work in its final form and under expected conditions. In almost all cases, this TRL represents the end of true system development. Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications. Technology readiness level (TRL): 9. Actual system proven through successful mission operations; Description: Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation. Examples include using the system under operational mission conditions. Source: DOD. [End of table] [End of section] Appendix IV: Comments from the Department of Defense: Assistant Secretary of Defense: Research And Engineering: 3040 Defense Pentagon: Washington, DC 20301-3040: March 1, 2013: Mr. Michael J. Sullivan: Director, Acquisition and Sourcing: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Sullivan: This is the Department of Defense (DoD) response to the GAO Draft Report, GAO-13-286, "Defense Technology Development: Technology Transition Programs Supply Military Users, but Opportunities Exist to Improve Measurement of Outcomes," dated January 31, 2013 (GAO Code 121058). Detailed responses to each of the recommendations are enclosed. The Department conducted a security review of the report, and finds that the document is properly classified as Unclassified and cleared as amended for open publication. Sincerely, Signed by: Alan Shaffer: Acting: Enclosure: As stated. GAO Draft Report Dated January 31, 2013: GAO-13-286 (GAO Code 121058): "Defense Technology Development: Technology Transition Programs Support Military Users, But Opportunities Exist To Improve Measurement Of Outcomes" Department Of Defense Comments To The GAO Recommendations: Recommendation 1: To improve visibility and manageMent of the department's efforts to transition technologies to support the needs of the warfighter, GAO recommends that the Secretary of Defense take the following action. Require that all technology transition programs track and measure project outcomes, to include not only whether technologies transitioned to an intended user but also the longer-term impact of whether the technologies benefited acquisition programs or military users in the field. DoD Response: Partially concur. The Department agrees that all transition programs should track and measure project outcomes, including how the technology or capability is being employed by the intended user. The Department is concerned: however, that the GAO recommendation to track and measure project outcomes would be a labor- intensive and very time-consuming process. As GAO notes in the report, there are generally hundred; of ongoing transition projects in place annually. Tracking these projects is generally accomplished within the period of performance for each research and development project, e.g., six months to three years. Monitoring the lone-term impact as to where, when, and how individual projects are used in acquisition programs and with military users reclaims a sustained presence - at a minimum of two to five years beyond the end of the research investment to monitor performance. This requires a significant investment in manpower along with a complex tracking process and/or robust database. Our experience, based on long-term tracking of our legacy technology transition programs, suggests that each Service or Agency would require a full man-year level of effort to institutionalize a process for long-term tracking and more man-years of effort to track programs and projects at the unit level. Another impediment to tracking would he the lack of an agreed upon definition of technology transition. Other reasons for the difficulty of long-term tracking are: 1) the lack of a contract vehicle to monitor contractor performance associated with technology transition and the view of the acquisition community that doing so would be an excess, or undue cost; 2) program manager and contracting officer turnover, or reassignment, resulting in loss of continuity of access to the information needed to measure longer-term success; and 3) a commitment of manpower, either in-house government of contractor support, being unsupportable in the current budget environment. The Department will continue to anecdotally measure the results of technology investments for three, five, or even ten years after investment and highlight the long-term benefits, as needed, to validate the investment levels associated with the research and development programs. We recommend the following revision to Recommendation 1: To improve visibility and management of the department's efforts to transition technologies to support the needs of the warfighter, GAO recommends that the Secretary of Defense take the following action. Assess technology transition programs for opportunities to implement processes to track and measure project outcomes, to include not only whether technologies transitioned to an intended user but also the longer-term impact of whether the technologies benefited acquisition programs or military users in the field. Recommendation 2: To improve visibility and management of the departments efforts to transition technologies to support the needs of the warfighter, GAO recommends that the Secretary of Defense take the following action: Assess transition programs to identify opportunities for more widespread use of existing transition management tools, such as technology transition agreements and technology commitment level evaluation mechanisms. DoD Response: Concur. The Department agrees that all transition programs should identify opportunities to use existing transition management tools. The Navy has implemented the use or transition tools as a best practice and the Army expressed their support of using technology transition agreements and technology commitment level evaluations to assess technology development and maturation programs. The Department will ensure guidelines are published and widely disseminated to improve the understanding as to when, how, and under what circumstances transition management tools can be employed. [End of section] Appendix V: GAO Contact and Staff Acknowledgments: GAO Contact: Michael J. Sullivan, (202) 512-4841, sullivanm@gao.gov: Staff Acknowledgments: In addition to the contact named above, John Oppenheim, Assistant Director; Danielle Greene; John Krump; Sean Merrill; Robert Miller; Carol Petersen; and Scott Purdy also made key contributions to the report. [End of section] Footnotes: [1] GAO, Best Practices: Stronger Practices Needed to Improve DOD Technology Transition Processes. [hyperlink, http://www.gao.gov/products/GAO-06-883] (Washington, D.C.: September 14, 2006). [2] The Defense Acquisition Challenge program, which was still active in fiscal year 2012, is expected to be terminated as part of budget efficiency efforts beginning in fiscal year 2013. [3] The Rapid Innovation Fund pilot program was funded by Congress in fiscal years 2011 and 2012 to provide DOD with another mechanism for rapid technology transition efforts. Funding for fiscal year 2013 will depend on whether Congress reauthorizes the program. [4] Every federal agency with a budget of $100 million or more for extramural research or research and development--which is conducted by nonfederal employees outside federal facilities--is required to establish and operate a Small Business Innovation Research program funded by a legislatively specified percent of that budget--2.5 percent in recent years. The DOD Small Business Innovation Research program is made up of 13 participating components, including the Army, Navy, Air Force, Missile Defense Agency, Defense Advanced Research Projects Agency, Chemical Biological Defense, Special Operations Command, Defense Threat Reduction Agency, National Geospatial- Intelligence Agency, Defense Logistics Agency, Defense Microelectronics Activity, Defense Health Program, and the Office of Secretary of Defense. [5] The OSD component of DOD's Small Business Innovation Research program represents $179.8 million of the $1.75 billion associated with OSD technology transition programs in the fiscal year 2010-2012 time frame. Because Small Business Innovation Research includes early-stage research and development to explore technology feasibility, not all program funding is associated with technology transition efforts. [6] ManTech is the oldest program with technology transition ties, and each Military Department manages its own efforts. The program focuses on technologies to enhance manufacturing capabilities, with deliverables that often result in measurable returns on investment from addressing process deficiencies and improvements to sustainment practices for military systems. [7] TRL 6 constitutes a representative model or prototype system, which is well beyond that of TRL 5, that is tested in a relevant environment. It represents a major step up in a technology's demonstrated readiness. Examples include testing a prototype in a high- fidelity laboratory environment or in a simulated operational environment. [8] [hyperlink, http://www.gao.gov/products/GAO-06-883]. [9] GAO, Defense Technology Development: Management Process Can be Strengthened for New Technology Transition Programs. [hyperlink, http://www.gao.gov/products/GAO-05-480] (Washington, D.C.: June 17, 2005). [10] Within the ManTech program, the Army and the Navy use technology transition agreements for their projects. The Air Force does not use technology transition agreements for its ManTech projects. [11] [hyperlink, http://www.gao.gov/products/GAO-05-480] and [hyperlink, http://www.gao.gov/products/GAO-06-883]. [12] [hyperlink, http://www.gao.gov/products/GAO-06-883]. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]