This is the accessible text file for GAO report number GAO-15-127 entitled 'Technology Transfer: Federal Laboratory Consortium Should Increase Communication with Potential Customers to Improve Initiatives' which was released on October 3, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Requesters: October 2014: Technology Transfer: Federal Laboratory Consortium Should Increase Communication with Potential Customers to Improve Initiatives: GAO-15-127: GAO Highlights: Highlights of GAO-15-127, a report to congressional requesters. Why GAO Did This Study: The federal government spends about one-third of its annual $145 billion research and development budget at hundreds of federal agency labs. Technology generated by this research may have application beyond agencies' immediate goals if commercialized by the private sector. For example, federal research has contributed to innovative products, including antibiotics and the Internet. FLC-—a nationwide consortium of federal labs-—helps labs transfer technology to the private sector. In recent years, FLC created new initiatives to provide a clearinghouse-—a central point for collecting and disseminating information-—for technology transfer opportunities. GAO was asked to review FLC's efforts to provide information on technology transfer opportunities. This report assesses (1) the extent to which FLC has communicated with potential customers when designing and implementing its clearinghouse initiatives, and (2) how FLC measured the results of those initiatives. GAO reviewed relevant laws and FLC guidance, and interviewed a nonprobability sample of officials from four federal agencies with the highest research budgets, and a spectrum of eight customer groups, among others. What GAO Found: The Federal Laboratory Consortium for Technology Transfer (FLC) has taken steps to communicate with potential customers, including small businesses and entrepreneurs, but has not obtained feedback from them to assess their needs when designing and implementing technology transfer clearinghouse initiatives. This resulted in missed opportunities to better meet potential customer needs. For example, in 2012, when developing a web-based search tool to help potential customers identify relevant federal technology transfer opportunities across federal laboratories (labs), FLC discussed how to implement the tool with its federal member labs and agencies. However, FLC did not assess the information needs of potential customers to ensure the tool would provide relevant information in a format that customers consider useful, as called for by leading practices and federal internal control standards on communicating with and obtaining information from stakeholders. FLC officials said they conducted testing to ensure the new website functioned as intended before launching it, but did not involve potential customers in these tests. Moreover, after developing the tool, FLC did not communicate with potential customers to collect feedback from them consistent with leading practices regarding the extent to which the tool met their needs or how it might be improved before implementing it. Potential customers of FLC's initiatives expressed concerns about the extent to which FLC's recent web-based search tool would meet their needs, specifically noting that the tool: * provides limited information to facilitate personal interaction between federal researchers and customers, despite the importance of spontaneous idea sharing to facilitate technology transfer; * provides limited information on the full range of technology transfer opportunities, focusing instead on federally patented technologies; * affords customers limited ability to compare technologies across labs; and; * provides limited information on the market relevance of a given technology. FLC faces challenges in communicating with potential customers without also engaging its agency and lab members, given the relatively small size of FLC's annual budget and available staff. By working collaboratively with agency and lab members to collect feedback, FLC can enlist their help in enhancing the information provided through its initiatives. FLC collects data on the use of its clearinghouse initiatives but has not developed and used performance goals and measures consistent with federal agency leading practices. For example, FLC collects data on the general use of its clearinghouse initiatives, such as the number of technology transfer inquiries it receives, the number of unique views of its web pages, and the average time spent on a web page. However, FLC has not developed performance goals or measures related to the key strategic goals to which its clearinghouse initiatives contribute. Without performance measures, FLC is unable to determine whether its initiatives are having their desired effect or how their performance might be improved. FLC also cannot fully demonstrate in its annual report to Congress its progress toward the achievement of its relevant strategic goals, limiting the information that the administration and Congress receive on the effectiveness of FLC's initiatives. What GAO Recommends: GAO recommends, among other things, that FLC work collaboratively with agency and lab members to increase communication with potential customers to obtain feedback and improve its clearinghouse initiatives, and develop performance measures. FLC generally agreed with the report's findings and recommendations. View [hyperlink, http://www.gao.gov/products/GAO-15-127]. For more information, contact John Neumann at (202) 512-3841 or neumannj@gao.gov. [End of section] Contents: Letter: Background: FLC Has Not Fully Communicated with and Obtained Feedback from Potential Customers when Designing and Implementing Clearinghouse Initiatives: FLC Has Not Measured the Performance of Its Clearinghouse Initiatives: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Selected Technology Transfer Organizations with Technology Search Tools: Appendix II: Comments from the Federal Laboratory Consortium for Technology Transfer: Appendix III: GAO Contact and Staff Acknowledgments: Figures: Figure 1: Example of Technology Transfer from a Federal Lab to the Private Sector: Figure 2: Examples of Search Results from the Available Technologies Tool from the Federal Laboratory Consortium for Technology Transfer: Figure 3: Ways Potential Customers Can Identify Technology Transfer Opportunities Using Tools from the Federal Laboratory Consortium for Technology Transfer: Abbreviations: CRADA: cooperative research and development agreement: DOD: Department of Defense: DOE: Department of Energy: FLC: Federal Laboratory Consortium for Technology Transfer: R&D: research and development: GSA: General Services Administration: GPRA: Government Performance and Results Act: GPRAMA: GPRA Modernization Act of 2010: NASA: National Aeronautics & Space Administration: NIH: National Institutes of Health: NIST: National Institute of Standards and Technology: OSTP: Office of Science and Technology Policy: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: October 3, 2014: The Honorable John D. Rockefeller IV: Chairman: Committee on Commerce, Science, and Transportation: United States Senate: The Honorable Michael F. Bennet: United States Senate: The federal government spends approximately one-third of its annual $145 billion research and development (R&D) budget on research at hundreds of federal laboratories (labs) to help meet agencies' missions.[Footnote 1] The knowledge and technological innovations generated by this research regularly have application beyond agencies' immediate, mission-related goals if these innovations can be commercialized by the private sector. Technology transfer is the term used to refer to the process by which technology or expertise developed for one purpose is used for another purpose. In some instances, it involves the transfer of legal rights, such as licensing a government-owned patent to a private-sector entity. Technology transfer also includes collaboration between private companies and federal labs, for example, in the testing of advanced batteries. In other instances, technology transfer involves the informal transmission of information, knowledge, and skills through person-to- person or organization-to-organization interaction.[Footnote 2] Over the past decades, through technology transfer, federal R&D has contributed to the development of innovative products and processes in the commercial marketplace, including advances in airplanes, antibiotics, bioengineered drugs, computers, plastics, and the Internet, which have created new industries and significant economic growth. The Federal Laboratory Consortium for Technology Transfer (FLC) is a member-based federal entity established in law in 1986 by the Federal Technology Transfer Act.[Footnote 3] FLC's membership is comprised of officials from federal agencies and approximately 350 federal labs. FLC began largely as a forum for the education, training, and networking of federal technology transfer officials to promote the integration of technical knowledge developed by federal departments and agencies into the U.S. economy. Over time, FLC's role has included serving as a clearinghouse--a central point for collecting and disseminating information--for federal technologies and assisting outside entities in identifying technology transfer opportunities at federal labs. Its budget, which is provided out of its members' R&D budgets, was more than $2.9 million in fiscal year 2013. With the lingering effects of the recent economic recession, both the Obama Administration and congressional committees have emphasized the importance of transferring available technological innovations out of federal labs and into the hands of those who can commercialize them and thereby create jobs and economic growth. For example, among other actions, in October 2011, the President issued a memorandum to executive agencies calling for strategies to increase the accessibility and usefulness of information about federal technology transfer opportunities and the development of a database of publicly available, federally owned inventions.[Footnote 4] FLC took the lead in developing this database. In FLC's 2012 annual report,[Footnote 5] FLC identified two primary efforts to serve as a clearinghouse for federal technologies: (1) a Technology Locator service through which FLC provides referrals to relevant labs based on technology transfer inquiries it receives, and (2) the Available Technologies tool, which allows potential customers looking to license federal technologies (e.g., patented inventions) to search federal labs' websites for available technologies. As of September 2014, FLC officials said they are nearing completion of a new web-based search tool called FLCBusiness that is intended to allow users to search for technology development programs, available funding, and to learn about the capabilities of federal lab facilities. Federal technology transfer remains a priority for the President and Congress. Most recently, in his fiscal year 2015 budget, the President announced 15 cross-agency priority goals.[Footnote 6] The focus of one of these goals, the Lab-To-Market goal, is to increase the economic impact of federally funded R&D by improving and accelerating technology transfer from federal labs to the commercial marketplace. Recent congressional committee hearings have also addressed making opportunities for technology transfer more readily available to private sector businesses.[Footnote 7] You asked us to review FLC's efforts to make information about technology transfer opportunities available through its technology clearinghouse initiatives. This report assesses (1) the extent to which FLC has communicated with and obtained feedback from potential customers when designing and implementing its clearinghouse initiatives and (2) how FLC measured the results of those initiatives. To assess the extent to which FLC has engaged with potential customers when designing and implementing its clearinghouse initiatives, and how FLC measured the results of those initiatives, we reviewed FLC's authorizing legislation, bylaws, guidance, and other FLC documents regarding its goals, organization, initiatives, and metrics. We also interviewed FLC officials, including members of FLC's national and regional leadership, such as the FLC chair and several of FLC's six regional coordinators. We used this information to identify how FLC develops initiatives and measures results, including how FLC engages with potential customers that may want to participate in technology transfer opportunities, such as small and large businesses, entrepreneurs, and organizations focused on state and local economic development. We compared FLC's efforts to design, implement, and measure the results of its initiatives to leading federal practices outlined in Executive Order 12862,[Footnote 8] HowTo.gov,[Footnote 9] the GPRA [Government Performance and Results Act] Modernization Act of 2010 (GPRAMA),[Footnote 10] and GAO standards for internal control in the federal government.[Footnote 11] In addition, we conducted interviews using a standard set of questions with nonprobability samples of: * officials from the technology transfer offices at four federal agencies: the National Aeronautics & Space Administration (NASA), Department of Energy (DOE), National Institutes of Health (NIH), and Department of Defense (DOD) to obtain their views on FLC and its clearinghouse initiatives;[Footnote 12] * representatives from six technology transfer organizations to learn about their technology transfer clearinghouse initiatives and key features their customers found useful;[Footnote 13] and: * representatives from eight customer groups who could potentially use the information provided through FLC's clearinghouse initiatives.[Footnote 14] Our selection of agencies, technology transfer organizations, and potential customer group representatives was based on nonprobability samples and their views are not generalizable to those we did not interview but provide illustrative examples. We also attended FLC's annual meeting in April 2014. Additionally, we interviewed officials with the Office of Science and Technology Policy (OSTP) to obtain their views on FLC's role. We conducted this performance audit from September 2013 to October 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: Technology transfer (i.e., the process by which technology or expertise developed in federal labs for one purpose is used for another purpose) is a collaborative effort between the federal labs and outside organizations and traditionally includes: * Technology licensing--businesses can license federal technologies, such as patented inventions, in order to integrate them into their products;[Footnote 15] although not every new or improved technology developed through federal lab research can be commercialized.[Footnote 16] * Cooperative research and development agreements (CRADA)--under a CRADA, federal labs collaborate with nonfederal partners to carry out research projects that will directly benefit lab missions and the partners' R&D goals, and a lab may contribute personnel, equipment, or other resources to a project, while its CRADA partners may contribute funds or resources, or both. * Work for others--under a nonfederal work-for-others agreement, a federal lab is paid to conduct research on behalf of a nonfederal sponsor, such as a university or corporation. * User facility agreements--under a user-facility agreement, scientists or researchers from outside organizations can use federal lab equipment for their own research, sometimes in collaboration with federal lab staff. According to a report prepared by the Interagency Workgroup for Technology Transfer in response to the Presidential Memorandum of 2011, other important means of technology transfer include publishing scientific articles and other technical publications, as well as making software developed by federal researchers available for download.[Footnote 17] Figure 1 provides an example of how technology developed in a federal lab for one purpose may be transferred to the private sector for another purpose. Figure 1: Example of Technology Transfer from a Federal Lab to the Private Sector: [Refer to PDF for image: illustration] Federal lab – NASA Ames Research Center: NASA scientists in California developed a non-toxic and bio-degradable compound to prevent ice buildup on airplane wings. Technology transfer: An employee of a private company in Ohio happened to read about the invention being used on military aircraft and realized that it might improve the performance of one of the company's existing products designed to prevent ice from building up on railroad track switches. New commercial products: Economic benefits: The company contacted the Ames Research Center and licensed the technology for commercial use. It now offers two rail de-icing products, which resulted in additional revenue of up to $600,000 per year for the company and helped prevent seasonal layoffs. Source: GAO analysis of NASA information. GAO-15-127. [End of figure] Since the early 1980s, the federal government has enacted several laws related to technology transfer from federal labs. One of the first technology transfer laws, the Stevenson-Wydler Technology Innovation Act of 1980, articulated the need for a strong national policy supporting domestic technology transfer and required federal labs to establish an office of research and technology applications and devote budget and personnel resources to promoting technology cooperation and the transfer of federal technologies.[Footnote 18] These offices (which we refer to as technology transfer offices) are generally responsible for coordinating each lab's efforts to identify technologies available for transfer, obtaining patents or other legal protections for lab technologies, and negotiating technology transfer agreements with outside parties. In addition, the administration has emphasized the importance of technology transfer from federal labs. For example, in his October 2011 memorandum, President Obama directed federal agencies to take actions to accelerate technology transfer and support private sector commercialization in an effort to promote innovation and increase the economic and societal impact of federal R&D investments. Moreover, President Obama's fiscal year 2015 budget identified a new Lab-To-Market cross-agency priority goal to support innovation, economic growth, and job creation. Specifically, the goal outlines several actions to, over the next four years, accelerate and improve the transfer of technologies from federal labs to the commercial marketplace, including optimizing the management, discoverability, and ease-of-license of federally funded patents and increasing the utilization of federally funded research facilities by entrepreneurs and innovators. Even with this emphasis on technology transfer, we and others have identified a number of challenges associated with technology transfer from federal labs.[Footnote 19] These challenges include the following: * Technology transfer is often not a priority for lab managers and funding tends not to allow for widespread technology transfer outreach. * Scientists may not understand the potential commercial applicability of their innovations. * Rules and requirements that labs must follow in transferring technology, as well as a lack of standardization thereof, increase the complexity and length of time of the negotiation process with potential partners, and create a disincentive to working with the labs. * Lab technologies are often not well-developed enough for use in market-ready products and may require investment of additional time and money to develop. * Companies are often not aware of the potentially useful technologies being developed in federal labs. * Companies and entrepreneurs often prefer access to and personal contact with researchers to turn an early stage innovation into a market-ready product, and this may be difficult because conflict-of- interest rules that ensure that public employees do not unfairly benefit from federally funded inventions prevent scientists from being involved in business ventures. One of the federal efforts to facilitate technology transfer from federal labs to the private sector and others was the creation of FLC. FLC was established by law in the Federal Technology Transfer Act of 1986 to,[Footnote 20] among other things, (1) develop training for federal lab employees engaged in technology transfer and (2) facilitate communication and cooperation between federal lab technology transfer offices and regional, state, and local technology transfer organizations.[Footnote 21] The Federal Technology Transfer Act also established requirements for FLC's membership and funding. [Footnote 22] Since its establishment in 1986, FLC's mission has evolved. Earlier in its history, FLC's efforts, in large part, focused on providing training and networking for federal technology transfer staff. However, FLC has, in recent years, taken on more responsibility for helping federal agencies provide potential customers with information on available technology transfer opportunities and for functioning as a clearinghouse.[Footnote 23] These efforts were bolstered by the President's 2011 memorandum calling for an online clearinghouse of federal technologies, and FLC took the lead role in establishing such a tool. Changes to how FLC defines its mission from being primarily focused on the federal labs to expanding its focus beyond the federal labs reflect this transition. For example, in FLC's 2009 annual report to the President and Congress, FLC defined its mission as to help federal labs transfer technologies developed through the federal government's R&D efforts. In FLC's 2012 annual report to the President and Congress, FLC defined its mission as to promote and facilitate the rapid movement of federal lab research results and technologies into the mainstream of the U.S. economy. FLC is governed by an Executive Board with committees and a regional structure to carry out its activities. The Executive Board makes FLC policy on the basis of issues brought before it and establishes FLC's annual budget. The Executive Board consists of the chair, vice-chair, finance officer, recording secretary, a host agency representative from National Institute of Standards and Technology (NIST) within the Department of Commerce, 6 regional coordinators, 6 members-at-large, and the 6 chairs of the standing committees, for a total of 23 members.[Footnote 24] The Executive Board also has a National Advisory Council that provides user community views and suggestions about FLC's operations. The advisory council members are selected from FLC's user communities, including but not limited to, academia, federal laboratories, industry, and state and local governments. The Council meets and reports its findings to the Executive Board at FLC's annual national meeting. The Executive Committee, consisting of the members above minus the members-at-large and committee chairs, conducts FLC business between meetings of the Executive Board.[Footnote 25] FLC is organized into six regions, and each region has a regional coordinator, its own website, social media outlets, regional newsletter, and can implement its own initiatives including training and information meetings for its members.[Footnote 26] FLC Has Not Fully Communicated with and Obtained Feedback from Potential Customers when Designing and Implementing Clearinghouse Initiatives: FLC has taken steps to communicate with potential customers, including small business and entrepreneurs, but has not fully communicated with and obtained feedback from them to assess potential customers' needs and incorporate leading practices when designing and implementing technology transfer clearinghouse initiatives, resulting in missed opportunities to better meet potential customer needs. As stated in FLC's 2012 annual report, FLC carries out its mission to promote and facilitate the rapid movement of federal lab research results and technologies into the mainstream of the U.S. economy by implementing two primary clearinghouse initiatives--the Technology Locator service and Available Technologies tool--that provide information on available technology transfer opportunities. Currently, FLC is developing a third initiative--FLCBusiness--to provide additional information on available technology transfer opportunities. While FLC officials said they view federal agencies and labs as FLC's main clients for its training and other activities, the principal customers of the information provided through FLC's clearinghouse initiatives include the private sector, and state and local government organizations, among others, specifically: * Technology Locator service. Since 1987, FLC has provided a Technology Locator service whereby an FLC staff member will try to connect customers with relevant labs based on technology needs submitted by customers through a phone call, e-mail, or online form. FLC's online form provides a space for potential customers to explain their technology request or problem, desired results, and expectations of a federal lab, among other information. Using this information, FLC will identify laboratory resources and contacts that can respond to specific requests and will facilitate communication between the potential customer and the lab contacts. Once contact is made, further discussion or negotiation of a technology transfer agreement occurs between the potential customer and the lab representative. FLC officials said customers who submit requests usually receive a response within 48 hours. * Available Technologies tool. In 2012, FLC released the Available Technologies tool to provide a web-based search tool for collecting and disseminating information on publicly available federally owned inventions and, when available, licensing agreements.[Footnote 27] The tool can help potential customers identify more than 20,000 available lab technologies across approximately 225 labs by searching the websites of 13 participating agencies. FLC reported spending approximately $30,000 for a contractor to develop the tool, which relies on a customized Google search tool to link potential customers to the existing information on agencies' technology transfer websites based on key words the customer enters. The key word search can be refined to include or exclude certain words and may also be focused on technologies published within a certain time period. In addition, the Available Technologies tool webpage includes a listing of the latest technologies available for transfer from federal labs. According to FLC, it plans to continue to develop this tool as it gains access to additional data from federal labs through various government-wide initiatives.[Footnote 28] * FLCBusiness. FLC is developing a new web-based search tool called FLCBusiness that is scheduled to become publicly available in fall 2014, according to FLC officials, and is intended to allow potential customers to search for technology development programs, available funding, and to learn about the capabilities of federal lab facilities. The tool is designed to allow potential customers to perform key word searches for labs, or funding or technology development programs, and to refine searches to a specific agency, geographic location, or technology area. The tool also is to provide links to additional information on licenses, CRADAs, and other technology transfer methods. As of August 2014, FLC reported spending approximately $150,000 to develop the tool. In addition, FLC has utilized various communication tools, such as social media, outreach by regional FLC members to local technology transfer organizations, its annual meeting, and an online newsletter and news forum, to communicate with potential customers and promote its clearinghouse initiatives and technology transfer success stories. For example, in 2013, FLC began providing an opportunity for industry representatives from large corporations to present their companies' technology needs to participating federal lab officials as part of its annual meeting, according to an industry representative we interviewed. During the 2014 FLC annual meeting, we observed that industry representatives described technology needs ranging from advanced materials to robotics. FLC used the opportunity to do a presentation to publicize its clearinghouse initiatives and, after the presentation, industry representatives and agency members had an opportunity to interact and exchange contact information. FLC's process to design and implement its primary clearinghouse initiatives involves FLC committees and regional coordinators annually proposing new ideas for consideration by FLC's Executive Board. According to FLC officials, the Executive Board considers proposals in relation to available resources and conducts outreach to agency and lab members. For example, in 2012 when developing its Available Technologies tool, FLC reached out to its agency and lab members when considering possible options for designing a clearinghouse of available technology transfer opportunities. After receiving input from agency and lab members, including concerns about the burden, ability, or willingness to compile data on available technology transfer opportunities for FLC, FLC officials said they designed the Available Technologies tool to minimize the burden on agencies to collect and share their data. Specifically, rather than ask agencies to gather relevant data on federally owned inventions, which were not already centralized and did not exist in a standard format, FLC developed a web-based search tool that would direct potential customers to existing information on federal labs' websites. FLC officials determined this tool (1) would not require FLC resources to manipulate agency data and (2) would not require agency members to update information separately from what they already planned to include on their websites. FLC officials said they considered FLC to be best-positioned to take the lead in developing the kind of clearinghouse tool described in the President's 2011 memorandum because it was an interagency body, had already developed a clearinghouse tool when the memorandum called for the creation of a similar database, and because FLC's funding for the clearinghouse tool was potentially more stable than if the tool were funded by a specific agency appropriation that needed to be approved each fiscal year. However, FLC did not assess the information needs of potential customers of federal lab technologies to ensure the Available Technologies tool would provide relevant information in a format that customers consider useful, as called for by leading practices. Federal leading practices outlined in Executive Order 12862 and HowTo.gov direct agencies to assess customer needs when designing and implementing initiatives.[Footnote 29] For example, Executive Order 12862 directs agencies to identify and survey customers to determine the kind and quality of services they want in an effort to develop and meet customer service standards. HowTo.gov also outlines the need to engage with customers to collect and address customer feedback through, for example, online surveys or focus groups, and indicates a need to conduct regular user testing of government websites with actual customers.[Footnote 30] In addition, according to federal standards for internal control, federal management should ensure that there are adequate means of communicating with and obtaining information from external stakeholders that may have a significant impact on an agency's achieving its goals.[Footnote 31] FLC has not taken such steps as part of its approach to design and implement its clearinghouse initiatives. For example, FLC officials said that when designing the Available Technologies tool, they conducted testing to make sure searches on the site functioned as intended, but that potential customers were not involved in this testing. Moreover, after developing the tool, FLC did not communicate with potential customers to collect feedback from them consistent with leading practices regarding the extent to which the tool met customers' needs or how it might be improved before implementing it. Similarly, we found that, while FLC is in the final stages of developing its new web-based search tool, FLCBusiness, FLC officials said potential customers had not seen early versions of the tool and had an opportunity to provide feedback. Instead, FLC officials said the tool was extensively tested by its web-design contractor and reviewed by members of its Executive Board. FLC officials told us that they demonstrated the tool for agency members and sent the early, or beta, version to laboratory representatives for user testing and data input. The tool was introduced at FLC's 2014 annual meeting, although it was not clear whether any feedback was collected from potential customers based on our observation of the meeting. Without communicating with potential customers on FLC's tools, allowing for customers to test them, and collecting feedback on their usefulness, FLC is not following internal control standards or leading practices and does not have assurance that its tools will meet customer needs. Representatives of several technology transfer organizations and potential customer groups we interviewed were unsure of the extent to which they would rely on FLC to help facilitate technology transfer to help meet their technology needs. Specifically, to examine these representatives' views on the likelihood that they would turn to FLC to help meet their technology needs, we asked them to familiarize themselves with the content and format of the information provided through FLC's Available Technologies tool.[Footnote 32] While some representatives of technology transfer organizations and potential customer groups we interviewed agreed that FLC's effort to provide a clearinghouse of information on available technology transfer opportunities was important, they also provided the following comments regarding the Available Technologies tool: * Limited information to facilitate personal interaction with federal researchers: Some customer representatives said FLC's Available Technologies tool is not a platform to facilitate personal interaction between federal researchers and potential customers. Many representatives of potential customer groups we interviewed, as well as FLC officials, indicated that technology transfer is effectively facilitated when there are opportunities for personal interaction and spontaneous idea sharing. For example, one representative said that entrepreneurs' technology transfer efforts often are focused on identifying researchers on the cutting edge of their field to help understand the state of the technology and generate ideas about how to solve a particular problem. We observed that search tools from some other organizations that provide information on technology transfer opportunities allow for such interactions in a virtual environment. [Footnote 33] For example, the tool developed by one organization we spoke with allows users to search for groups of researchers working in particular technical areas, which representatives of the organization said can help customers access ideas from multiple sources. * Limited information on the full range of technology transfer opportunities: Federally patented technologies available for licensing are the primary type of technology transfer opportunity available through FLC's Available Technologies tool, although some representatives of potential customer groups we interviewed said they are often more interested in other technology transfer opportunities, such as access to unique lab facilities or researcher expertise.[Footnote 34] For example, they noted that lab testing facilities can allow businesses to test and improve new products before bringing them to market. * Limited ability to compare technologies across multiple labs: Customers often search for complementary technologies that exist across multiple labs, but these may be difficult to identify through FLC's Available Technologies tool because labs and agencies describe technologies differently. For example, some technologies from NASA are presented as scientific research results, while technologies with similar search terms from DOD are presented as potential product components. A representative from one technology transfer organization we interviewed said that a small company he worked with would not have been able to identify relevant patents on its own because the company needed patents from 11 different federal labs to cover different components of a product it was developing to filter water used in oil and natural gas drilling. Other representatives of potential customer groups and technology transfer organizations said that while large companies have the time and money to search for technologies across numerous labs, smaller companies generally do not. * Limited information on technologies' market relevance: Customers may benefit from more information on the potential market relevance of technologies, which is not always provided in the agency and lab information available through FLC's Available Technologies tool. For example, one representative from a technology transfer organization said that clearinghouse search results that are highly technical--as with those in FLC's Available Technologies tool that are primarily based on patents--may make sense to research scientists who know relevant technical terms, but not necessarily to entrepreneurs who may not be aware of the technical key words relevant to a variety of potential solutions to their technology needs. In addition, some customer representatives said information on market relevance is useful because federally patented technologies are generally not close to being market-ready, and additional R&D is needed to bring products to market. We observed a few of these limitations during a keyword search we conducted using FLC's Available Technologies tool. In particular, when we attempted to identify similar technologies across agencies and labs based on a keyword search, the results were not standardized, and some results from some agencies offered different information than others. For example, a result from one agency provided a link to a 15-page research publication related to the technology, while a result from another provided a one-sentence overview of the technology's potential market relevance. Moreover, not every result included contact information for researchers affiliated with the technology. For instance, one result provided a list of links to agency-wide opportunities for partnerships between the private sector and the agency, but this list did not include contact information for any researchers, nor was it clear which links on the page related to the original search term. Figure 2 shows the different types of information that may be included in FLC search results. Figure 2: Examples of Search Results from the Available Technologies Tool from the Federal Laboratory Consortium for Technology Transfer: [Refer to PDF for image: illustration] Available Technologies tool: Federal Laboratory Consortium for Technology Transfer: Search: Search results: Scientific research paper; Patent abstract; Brief summary of technology. Sources: GAO and Art Explosion (computer). GAO-15-127. [End of figure] FLC has taken steps that may help address some of the issues representatives of technology transfer organizations and other potential customer groups identified and that we observed. For example, in 2012, FLC began a systematic effort to identify potential customers of federal technology transfer opportunities, including entrepreneurs and small businesses, in different regions of the country. This effort started with an initial study conducted in FLC's Midwest region that found a number of unmet customer needs related to technology transfer, including a need for technical and financial help for entrepreneurs and improved customer service and dissemination of information on available technologies. Although this effort represented an important step in engaging with potential customers, it did not focus on obtaining their input on what information they might want from an FLC clearinghouse tool and in what format. In addition, FLC's clearinghouse tools, taken together when FLCBusiness becomes publicly available, could provide potential customers with more information on available technology transfer opportunities than FLC currently provides through the Available Technologies tool. For example, FLC anticipates using FLCBusiness to provide information on lab facilities, which representatives of potential customer groups and other technology transfer organizations noted is generally missing from the Available Technologies tool. Moreover, if a potential customer is unable to find a facility or technology in FLCBusiness or the Available Technologies tool, the potential customer could contact the Technology Locator service to get in touch with a laboratory representative. Figure 3 depicts how FLC's clearinghouse tools could work together to help potential customers identify technology transfer opportunities to meet their needs. However, since FLC has not fully communicated with and obtained feedback from potential customers in developing FLCBusiness, it lacks assurance that FLCBusiness or its suite of clearinghouse tools will meet customers' needs. Figure 3: Ways Potential Customers Can Identify Technology Transfer Opportunities Using Tools from the Federal Laboratory Consortium for Technology Transfer: [Refer to PDF for image: illustration] Search the Available Technologies tool: Did you find a technology? Yes: Contact the laboratory representative to set up a meeting; No: Contact the Technology Locator service; Contact the laboratory representative to set up a meeting. Search FLCBusiness for available facilities and equipment: Did you find a facility or equipment? Yes: Contact the laboratory representative to set up a meeting; No: Contact the Technology Locator service; Contact the laboratory representative to set up a meeting. Search FLCBusiness for available funding opportunities: I found a funding option; Apply for the funding. Search FLCBusiness for available federal programs: I found a program; Apply for the program. Source: GAO analysis of FLC documents. GAO-15-127. [End of figure] FLC faces challenges in its ability to more fully communicate with potential customers to facilitate technology transfer from federal labs without also engaging FLC's agency and lab members. FLC's web- based clearinghouse tools depend on information provided by agency and lab members to help build or update the databases on which the tools are based. For instance, FLC officials said the Available Technologies tool depends on information about agency technologies available to FLC through existing agency websites. As noted above, this information often varies across agencies and can present similar technologies differently or use different technical terminology, and representatives of potential customers said these differences could make it difficult for outside parties to find the most appropriate information to meet their needs. Agency officials from NIH, DOE, DOD, and NASA also told us that it could be challenging for them to justify gathering new or different information related to technology transfer opportunities until FLC demonstrated that customer needs would be better met with additional information presented in a different format. They also noted that FLC may not be well-positioned to conduct additional customer outreach to collect feedback without working with agency and lab members, making it difficult for the consortium to improve its existing search tools on its own. Some agency officials said that any improvements based on customer feedback would require a significant collaborative effort on the part of FLC and its agency and lab members, given the relatively small size of FLC's annual budget and available staff. By working collaboratively with agency and lab members to collect feedback on its initiatives' usefulness, FLC can take advantage of its members' knowledge and expertise to better communicate with potential customers consistent with federal leading practices. Even with the challenges FLC faces, several FLC and agency officials as well as representatives of other technology transfer organizations supported FLC's clearinghouse initiatives. Representatives from some other organizations that provide information on federal technology transfer opportunities said they only have information about technology at a small number of federal labs, and one organization said it charges users or labs a fee for use of its tools. In contrast, some FLC and lab officials said FLC's clearinghouse initiatives serve an important function because they have the potential to provide more comprehensive information about technology transfer opportunities at federal labs at no cost to potential customers. One FLC official also noted that, while the Available Technologies tool has limited capability since the data it reports are unstructured and not standardized in any way, the relatively low cost of the tool and the quick time frame over which it was developed made it an important interim step to providing a global search capability for available federal technologies. FLC Has Not Measured the Performance of Its Clearinghouse Initiatives: Although FLC collects some data related to the use of its clearinghouse initiatives, it has not used performance goals and measures consistent with federal agency leading practices to measure results and does not have the information necessary to determine the extent to which its efforts help achieve FLC's overall strategic goals. FLC tracks the number of requests for assistance it receives through its Technology Locator service by (1) the technology area the customer is interested in (e.g., energy, pharmaceutical, or manufacturing); (2) the nature of the customer's interest (e.g., informational or technology need); and (3) type of customer that submitted the request (e.g., academic, investor, or small business). According to FLC officials, in 2013, the Technology Locator service received 172 requests for assistance. This number represented a decrease from the 203 requests for assistance FLC received in 2012. In addition, FLC collects success stories, which it posts on its website. For example, one of the success stories describes a business owner who placed a Technology Locator request to ask for help with software development-- the Locator successfully put her in touch with relevant staff at federal labs who were able to offer assistance. FLC also collects data on the general use of the Available Technologies tool, such as the number of unique views of web pages and the average time spent on a web page.[Footnote 35] One report showed that, from January 1 through July 14, 2014, FLC's Available Technologies tool had 2,010 page views from unique customers and 3,240 page views overall. FLC has developed a number of strategic goals over the years. The last strategic plan FLC developed was for 2009 and it contained three goals: (1) develop FLC members to be leaders in technology transfer; (2) foster an environment for technology transfer; and (3) enhance the professional organizational structure of FLC. More recently, in its 2012 annual report to the President and Congress, FLC identified four strategic goals for all FLC activities: (1) outreach/communication, (2) training/education, (3) networking, and (4) recognition. An FLC workgroup is currently working on a new strategic plan, which FLC reported will not be ready until the end of 2014. FLC has not, however, developed performance goals or measures to align its initiatives with its strategic goals, as recommended by leading practices to improve government performance and results. In particular, GPRAMA requires that agencies develop strategic plans that include general, long-term goals and objectives, along with an agency performance plan that describes how specific performance goals for the current and subsequent fiscal year contribute to an agency's strategic goals and objectives. The performance plan should contain, among other things, a description of how the performance goals are to be achieved, including clearly defined milestones, and establish a balanced set of performance indicators, or measures, that will help demonstrate progress toward achieving the performance goals. We have found that these requirements also can serve as leading practices for strategic planning at lower levels within federal agencies, such as planning for individual divisions, programs or initiatives.[Footnote 36] Among these leading practices is developing and using performance measures, which allow an agency to track the progress it is making toward its mission and goals, provide managers information on which to base their organizational and management decisions, and create powerful incentives to influence organizational and individual behavior. Further, federal leading practices outlined by HowTo.gov state that agencies should use performance metrics to influence the design of and drive improvements for government websites. However, FLC did not translate the strategic goals identified in its 2012 annual report into performance goals and measures. In particular, FLC did not develop any performance goals or measures related to its strategic goals of outreach or networking, which are, as FLC officials stated, the current goals to which its clearinghouse initiatives would contribute.[Footnote 37] In addition, while FLC tracks usage of its clearinghouse initiatives, FLC has not established performance measures for these initiatives that would help it assess the extent to which the initiatives are achieving FLC's goals of outreach and networking or how performance might be improved. For example, FLC does not collect information on the number of connections between lab staff and potential customers that resulted from use of the Available Technologies tool. As a result, some agency officials told us it was unclear how well the tool helps to facilitate technology transfer from federal labs. Working collaboratively with agency and lab members to develop and collect data on performance measures for FLC's clearinghouse initiatives could help FLC demonstrate the usefulness of its tools to its members while also helping FLC measure progress toward achieving its own goals. Some representatives of other technology transfer organizations we interviewed said they measure the results of their efforts using performance measures, such as the number of companies formed, the number of joint investment agreements between labs and private companies, and the number of successful technology transfer partnerships they help to facilitate. Without some type of performance measures, FLC will be unable to assess whether its initiatives are meeting FLC's goals of outreach and networking. Further, given that FLC has not developed such measures, its annual report to Congress does not contain information on the progress toward the achievement of FLC's goals of outreach and networking, limiting the information that the administration and Congress receive on the effectiveness of FLC's initiatives. Conclusions: Recognized as one of 15 cross-agency priority goals in the President's 2015 budget, technology transfer has become an increasingly important strategy to support economic growth, create jobs, and increase the global competitiveness of U.S. industries. As FLC's role in promoting technology transfer has expanded, particularly following the President's 2011 memorandum, the consortium has launched important initiatives to make information about federal technology transfer opportunities more available to potential customers. FLC is uniquely positioned to contribute to the President's goal of supporting innovation, economic growth, and job creation as the only interagency body federally mandated to facilitate technology transfer. However, FLC's efforts to communicate with potential customers often are not consistent with federal standards for internal control on communicating with and obtaining information from stakeholders and do not incorporate leading practices that call for agencies to consider customer needs when designing and implementing government services. Without better communicating with potential customers during the design and implementation of clearinghouse initiatives, FLC may not be able to effectively assess customer needs, conduct user tests of its web-based initiatives, and collect customer feedback. As a result, FLC does not have assurance that its initiatives will meet customer needs, and representatives of potential customer groups indicated that there was a significant mismatch between the information that would allow them to identify opportunities and what FLC's initiatives currently provide. While commercialization of technological innovations is a private sector role rather than a responsibility of the federal government, representatives of potential customer groups stressed that federal labs have extensive value and expertise that they find difficult to access. Officials from federal agencies told us that any improvements to FLC's initiatives based on customer feedback would require a significant collaborative effort on the part of FLC and its agency and lab members, given the relatively small size of FLC's annual budget and available staff. Also, because the information FLC provides through its initiatives depends on what is available from its agency and lab members, agency officials said that FLC would need to collaborate with them to help determine what additional or different information they could provide to help meet customers' needs based on any feedback received. By working collaboratively with agency and lab members to collect feedback on its web-based initiatives' usefulness, FLC can take full advantage of its members' collective knowledge and expertise to better communicate with potential customers consistent with federal internal control standards and leading practices to determine those customers' needs, enhancing the clearinghouse initiatives in achieving their technology transfer goals. In addition, FLC has not established performance goals and measures that would help it assess the extent to which the initiatives are meeting FLC's goals of outreach and networking or how performance might be improved. Working collaboratively with agency and lab members to develop and collect data on performance measures for FLC's clearinghouse initiatives could help FLC demonstrate the usefulness of its tools to its members while also helping FLC measure progress towards achieving its own goals. Representatives of other technology transfer organizations we interviewed said they measure the results of their clearinghouse efforts using performance measures. Without performance measures, FLC cannot fully demonstrate in its annual report to Congress its progress toward the achievement of its goals of outreach and networking, limiting the information that the administration and Congress receive on the effectiveness of FLC's initiatives. Recommendations for Executive Action: To more effectively fulfill its expanded role in providing a clearinghouse of information on available federal technology transfer opportunities, we recommend that the Chair of FLC, in coordination with the other members of the Executive Board, take the following four actions: * Work collaboratively with agency and lab members to take steps to better communicate with potential customers during the design and implementation of FLC's clearinghouse initiatives, including conducting customer needs assessments, conducting customer testing of current and future web-based initiatives, and collecting customer feedback on all FLC initiatives to make the initiatives more useful. * Once feedback is collected from potential customers, work collaboratively with agency and lab members to use this feedback to improve FLC's initiatives to make them more useful to potential customers, including asking FLC members for additional or different information, as appropriate. * Work collaboratively with agency and lab members to develop performance goals and measures for FLC's clearinghouse initiatives and use the results to evaluate progress toward meeting FLC's goals on outreach and networking. * Report on FLC's progress in these efforts in its annual report to the President and Congress. Agency Comments and Our Evaluation: We provided a draft of this report to FLC, DOD, DOE, NIH, NASA, and OSTP for review and comment. In written comments, reproduced in appendix II, FLC concurred with our findings and recommendations, and noted that it will take steps to incorporate our recommendations as it plans, implements, and monitors its efforts to help improve how it performs outreach beyond its members to the potential end users of its products. FLC also stated that it is examining how it is organized and is considering making changes to better leverage government and industry best practices for the use of digital systems. In its written comments, FLC also stated that although it generally agrees with the report's findings, there are some areas that it believes require some additional clarification. First, FLC stated that the report implies that FLC recently assumed a clearinghouse function when it has been performing this function since its establishment and that it more recently began to provide digital-based services. We agree that the recent change in FLC's clearinghouse efforts has been to provide web-based tools, and we acknowledged that FLC has offered one of its primary clearinghouse initiatives--the Technology Locator service--since 1987. In response to FLC's comment, we clarified portions of the report to indicate that FLC has, over time, served as a clearinghouse, but has increased its efforts in this regard in recent years. Second, FLC noted that it developed the Available Technologies tool as an initial Phase I version, not as a final product and that it plans to continue to develop it as additional structured data from laboratories become available. We revised the report to acknowledge that FLC plans to further develop this tool. Moreover, our recommendation that FLC increase communication with potential customers to obtain feedback on how the tool might be improved will help ensure the effectiveness of FLC's efforts. Third, FLC stated that we should include additional organizations in appendix I. We did not incorporate FLC's comments in appendix I because the suggested additions would have been beyond the scope of the appendix, which was to provide information on other technology transfer organizations with technology search tools that we examined during the course of our review. DOD, DOE, NASA, and OSTP indicated by e-mail that they had no technical or written comments. NIH provided technical comments by e- mail but did not provide formal written comments. We incorporated these technical comments, as appropriate. We are sending copies of this report to the appropriate congressional committees, the Chair of the Federal Laboratory Consortium, the Secretary of Defense, the Secretary of Energy, the Secretary of Health and Human Services, the Administrator of NASA, the Director of OSTP, and other interested parties. In addition, the report is available at no charge on the GAO website at [hyperlink, http://www.gao.gov/]. If you or your staffs have any questions about this report, please contact me at (202) 512-3841 or neumannj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. Signed by: John Neumann: Acting Director, Natural Resources and Environment: [End of section] Appendix I: Selected Technology Transfer Organizations with Technology Search Tools: We identified a number of other organizations with technology search tools that had features consistent with some of the needs identified by the representatives of potential customer groups we interviewed. These are some features that representatives from other technology transfer organizations said they included in their technology transfer search tools based on their assessment of customer needs. Common themes among these tools include a collaborative interface and an ability to contact researchers affiliated with the technologies. We did not evaluate the content of any of these search tools, nor are we endorsing any of these tools or organizations. For more information on these organizations, see the links below. AutoHarvest Foundation - [hyperlink, http://www.autoharvest.org/] * According to its website, AutoHarvest is an online meeting place that allows users of all types to showcase capabilities, technologies, and needs, then privately connect with other users to explore technology and business development opportunities of mutual interest. * Entrepreneurs, industry representatives, and researchers from universities, federal labs, and private industry can interact with one another through virtual "collaboration rooms." These virtual rooms allow two-way communication between registered users of the tool. * In addition to posting technologies available for licensing, federal laboratories and universities can also post facility capabilities to encourage cooperative research and development agreements and collaborative partnerships. Collective IP - [hyperlink, https://www.collectiveip.com/] * According to its website, the Collective IP tool provides technology transfer staff with a technology marketing platform, while simultaneously providing a search tool for other users who focus on identifying opportunities from companies and technology transfer organizations. * Users can create and receive customized updates about organizations and researchers from universities, federal labs, or private industry working on specific technologies or in particular research areas. * Users can review data on technologies and technology transfer opportunities in a standardized format from across a variety of sources, including the U.S. Patent and Trademark Office, federal grant agencies, and academic literature. iBridge - [hyperlink, http://www.ibridgenetwork.org/] (part of the Innovation Accelerator Foundation): * According to its website, the iBridge Network serves as an online community for sharing ideas, research, and knowledge about early stage technologies and inventions. * Users can search for groups of related patents across multiple sources, mostly university labs. * Users can also post technologies they wish to license or receive personalized e-mail updates about technologies that interest them. Technology Ventures Corporation - [hyperlink, http://www.techwhiteboard.com/] * According to the Technology Ventures Corporation's website, its Technology Whiteboard is a communication hub centered on Department of Energy (DOE) technology and intellectual property commercialization. * Users can collaborate and share ideas with entrepreneurs, university and federal researchers, and investors through social networking. * Users can search for technologies from a number of DOE laboratories by category, laboratory, or keyword. [End of section] Appendix II: Comments from the Federal Laboratory Consortium for Technology Transfer: FLC: Federal Laboratory Consortium For Technology Transfer: 950 N. Kings Highway, Suite 105: Cherry Hill, NJ 08034: P: 856-617-0502: F: 856-617-0494: [hyperlink, http://www.federallabs.org] 17 September 2014: Mr. John Neumann: Acting Director, Natural Resources and Environment: U.S. Government Accountability Office: 441 G Street, NW Washington, DC 20548: Subject: Report GAO-14-843: Dear Mr. Neumann: Thank you for the opportunity to review the GAO Report 14-483 [Now GAO- 15-127] regarding initiatives by the Federal Laboratory Consortium for Technology Transfer (FLC) to increase communication with our potential customers for technology transfer. As you are aware, the FLC was created by the Congress to assist federal laboratories with their technology transfer mission. This mission of supporting innovation in the nation's economy is as important today as when the FLC was formally created under the Federal Technology Transfer Act of 1986 (P.L. 99-502). As noted by the report, we have been very active in developing new tools to better support our federal laboratories outreach to potential partners to help build a foundation for economic growth through technology and innovation. We are thrilled that these new tools are well used by the public and we continue to receive extremely positive feedback about the enhanced capabilities the tools offer in accessing information across government agencies and laboratories. The FLC has reviewed this report and fundamentally concurs with the recommendations presented. As we continue to develop new tools for our laboratories and their customers, we will incorporate the recommendations of this report into our planning, implementation, and monitoring of these tools and systems. These efforts will improve how we perform out-reach beyond our membership to the potential end users of our products. We are examining how we are organized, and are considering amendments to our by-laws to better leverage government and industry best practices for the use of digital systems. Although we generally agree with the report's findings, there are some areas that we believe require some additional clarification: * The report implies that FLC has recently assumed a "clearinghouse function". Although we have recently provided more digital data-based services, the FLC has been performing the clearinghouse function since its establishment as required by the Federal Technology Transfer Act (see 15 U.S.C. §371O(e)(1)(C)). * We note that Available Technologies Tool was developed as an initial Phase I version, not as a final product. This tool serves as an important first step as a one-stop-shop methodology for government inventions as noted by the President's memorandum on commercialization. We plan to continue to develop this product as additional structured data from laboratories becomes available through the Open Data Initiative and as part of the continuing Lab-to-Market Cross Agency Priority GoaL Although user feedback as noted would be very helpful, currently the data searched is mostly unstructured due to the nature of the information available. * The Other Technology Transfer Organizations noted in Appendix 1 should include some additional entries. Most notably, the Association of University Technology Manager (AUTM) serves a somewhat analogous role to the FLC for university technology transfer offices (see [hyperlink, http://www.autm.net]). The Licensing Executive Society (LES) also provides a professional society focused on the licensing of intellectual property including private, academic, and government organizations (see [hyperlink, http://www.lesllsacanada.org]). Thank you for the thorough consideration given to these important topics. The FLC will continue to carefully review the recommendations provided as we strive to connect our investments in science and technology to the continued growth of the American economy. Respectfully, Signed by: Paul Zielinski: Chair: Federal Laboratory Consortium for Technology Transfer: CC: FLC Executive Board. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: John Neumann, (202) 512-3841 or neumannj@gao.gov: Acknowledgments: In addition to the individual named above, Chris Murray (Assistant Director), Antoinette Capaccio, John Delicath, Cindy Gilbert, Richard Hung, Rich Johnson, Mae Liles, Les Mahagan, Rob Marek, David Powner, Kiki Theodoropoulos, and Michelle Wong made important contributions to this report. [End of section] Footnotes: [1] The term "lab" is used in this report in a broad sense to include federally funded laboratories and R&D centers. Annual federal R&D budgets averaged $145 billion between fiscal years 2009 and 2013. [2] For the purposes of this review, we focused on formalized technology transfer agreements between labs and outside entities and did not examine informal outreach and information sharing between lab scientists and outside entities, such as academic conference participation or publishing. [3] Federal Technology Transfer Act of 1986, Pub. L. No. 99-502, § 3, 100 Stat. 1785, 1787, codified as amended at 15 U.S.C. § 3710(e). [4] Presidential Memorandum, "Accelerating Technology Transfer and Commercialization of Federal Research in Support of High-Growth Businesses," October 28, 2011. [5] 15 U.S.C. § 3710(e)(6) requires FLC to report on its expenditures and activities annually to the President, the appropriate authorization and appropriation committees of Congress, and each agency that provides funding for FLC. [6] Cross-agency priority goals were called for by the GPRA [Government Performance and Results Act] Modernization Act of 2010 and are a tool used by leadership to accelerate progress on a limited number of presidential priority areas where implementation requires active collaboration between multiple agencies. To establish these goals, the Office of Management and Budget solicited nominations from federal agencies and several congressional committees. The 15 cross- agency priority goals identified in the President's fiscal year 2015 budget include 7 mission-oriented and 8 management-focused goals with a 4-year time horizon. [7] For example, see Improving Technology Transfer at Universities, Research Institutes and National Laboratories, hearing before the S. Comm. on Subcommittee on Research and Technology, 112th Cong., 1ST sess., July 24, 2013. [8] Executive Order 12862 was issued in 1993 by President Clinton to establish and implement customer service standards to guide the operations of the executive branch. [9] HowTo.gov is a key source of leading practices for federal website development and management. The website is managed by the General Services Administration (GSA) and is designed as a resource to improve how agencies communicate and interact with customers and use innovative tools and technologies to provide services and information. HowTo.gov offers best practices, guidance, and training on strategic planning; federal web requirements and policies; applications, data, and web infrastructure tools; web content management, usability, and design; and performance metrics. In 2014, GSA moved much of the leading practices content from HowTo.gov to a new platform, Digitalgov.gov. However, we refer to the leading practices on HowTo.gov because these changes did not occur before FLC designed the clearinghouse initiatives discussed in this report. See [hyperlink, http://www.HowTo.gov]. [10] GPRAMA replaced certain provisions of the Government Performance and Results Act (GPRA) of 1993, including requirements that agencies develop strategic plans and performance goals to help measure results. Both the Executive Order we reviewed as well as the provisions of GPRAMA relevant to this report apply to departments and agencies, not to their subcomponents or to interagency consortiums like FLC. However, we have previously reported that such practices can also serve as leading practices for lower levels of federal agencies or other federally funded organizations, such as FLC. [11] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [12] We selected these agencies because they had the highest annual federal R&D spending between fiscal years 2011 and 2013. [13] We selected organizations based on a variety of factors, such as involvement with federal technology transfer and whether an organization had technology transfer clearinghouse initiatives. Five of these were non-profit organizations, and one provided a technology search to clients on a subscription basis. Four of these organizations had their own collaborative technology search tools, and we examined features of these tools through demonstrations and our own observations. We identified organizations through a literature search and from referrals provided by knowledgeable parties. [14] We identified these representatives through a literature search and from referrals provided by knowledgeable parties and selected these representatives because they represented a broad spectrum of potential FLC customers and said they could speak generally about federal technology transfer. We interviewed representatives from each of the following: a large industrial business group, two venture capital firms knowledgeable of small business needs, one technology scouting firm, one firm that focuses on building companies based on federal technologies, one patent broker knowledgeable of business technology needs, and two organizations focused on state and local development. We obtained their views on the information needs related to federal technology transfer opportunities and the extent to which FLC engaged with potential customers in developing its initiatives. [15] A patent is an exclusive right granted for a fixed period of time to someone who invents or discovers (1) a new and useful process, machine, manufacture, or composition of matter or (2) any new and useful improvement of such items. A patent owner can grant permission to use a patented invention by licensing others to use, make, sell, or import the patented invention. [16] According to a RAND study, a large percentage of federally funded technologies are not likely to enter the marketplace due to factors, including that some defense R&D is not suited for consumer applications. See Mark Wang, et al., Science and Technology Policy Institute, RAND, Technology Transfer of Federally Funded R&D: Perspectives from a Forum, (Arlington, VA: 2003). [17] According to a Department of Commerce report on technology transfer that was prepared in response to the Presidential Memorandum of 2011, the Interagency Workgroup for Technology Transfer was initially established under Executive Order 12591, Section 7, to identify and disseminate creative approaches to technology transfer from federal labs. The Workgroup is currently hosted by the National Institute of Standards and Technology for the department. [18] Pub. L. No. 96-480, 94 Stat. 2211 (codified as amended at 15 U.S.C. §§ 3701-3714). [19] See GAO, Technology Transfer: Clearer Priorities and Greater Use of Innovative Approaches Could Increase the Effectiveness of Technology Transfer at Department of Energy Laboratories, [hyperlink, http://www.gao.gov/products/GAO-09-548] (Washington D.C.: June 16, 2009), and Stephanie Shipp et al., Science and Technology Policy Institute, Institute for Defense Analyses, Technology Transfer and Commercialization Landscape of the Federal Laboratories (Washington, D.C.: June 2011). [20] Prior to establishing FLC, a consortium of defense labs was originally established in 1971, under the auspices of DOD, to improve communication between defense labs. In 1974, it was expanded to include other federal departments in a voluntary organization that grew to approximately 350 federal labs and changed its name to the Federal Laboratory Consortium for Technology Transfer. [21] Pub. L. No. 99-502, § 3, 100 Stat. 1785, 1787, codified as amended at 15 U.S.C. § 3710(e)(1)(C) and (D). [22] Labs with 200 or more full-time equivalent scientific, engineering, and related technical positions must participate in the consortium. Representatives must include a senior staff member from each member lab and a senior representative from each agency with one or more member labs. Other labs and individuals may also participate. Agencies must contribute 0.008 percent of any funds spent by or on behalf of its member labs. [23] Prior to FLC's establishment by law, the Stevenson-Wydler Technology Innovation Act of 1980 established a federal role to, among other things, serve as a central clearinghouse for the collection, dissemination, and transfer of information on federally owned or originated technologies having potential application to state and local governments and to private industry. [24] FLC's standing committees include a state and local government committee and a communications committee, among others. NIST is the legislatively designated FLC host and handles FLC's finances. [25] According to its bylaws, FLC's nationally elected officers may serve no more than two consecutive 2-year terms. [26] Far West Region: Alaska, Arizona, California, Hawaii, Idaho, Nevada, Oregon, and Washington; Midwest Region: Illinois, Indiana, Michigan, Minnesota, Ohio, and Wisconsin; Mid-Atlantic Region: Delaware, Maryland, Pennsylvania, Virginia, West Virginia, and Washington, DC; Mid-Continent Region: Arkansas, Colorado, Iowa, Kansas, Missouri, Montana, New Mexico, Nebraska, North Dakota, Oklahoma, South Dakota, Texas, Utah, and Wyoming; Northeast Region: Connecticut, Maine, Massachusetts, New Hampshire, New Jersey, New York, Puerto Rico, Rhode Island, and Vermont; Southeast Region: Alabama, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, and Tennessee. [27] An earlier version of the tool was originally launched in 2011 using a different format that relied on pulling data on lab technologies into a searchable database. [28] For example, efforts under the Lab-To-Market cross-agency priority goal include, among other things, making all relevant data about federally funded technologies and federal R&D facilities more accessible. [29] As noted previously, Executive Order 12862 established customer service standards for the executive branch and HowTo.gov is a website that is a key source of leading practices, including guidance training on strategic planning; federal web requirements and policies; applications, data, and web infrastructure tools; web content management, usability, and design; and performance metrics. [30] Such user testing may include asking a customer to perform website tasks while the web team and stakeholders watch, listen, and take notes to find out what is or is not working. For more information, visit [hyperlink, http://www.usability.gov/]. [31] See [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [32] We focused on the Available Technologies tool because (1) the President's 2011 memorandum called for the development of such a tool, and (2) FLCBusiness was not publicly available for representatives of technology transfer organizations or potential customer groups to comment on during the period of our review. We asked representatives of other technology transfer organizations and potential customer groups to familiarize themselves with the Available Technologies tool prior to our discussion if they had not already used it. In instances where the representatives had not familiarized themselves with the tool prior to our discussion, we provided them with a brief summary of how the tool works and the data sources upon which it relies. [33] See appendix I for more information on features of technology search tools of other technology transfer organizations. [34] According to some representatives from customer groups and one technology transfer organization, federal lab patents may not afford adequate intellectual property protection to support commercialization because they sometimes protect only the specific version of the invention relevant to the lab's mission from infringement, which would potentially allow a competitor to offer a similar product. [35] FLC uses Google Analytics to collect these data for various webpages on FLC's website. A Google Analytics report for FLC's Technology Locator and Available Technologies tool included data for the following: page views, unique page views, average time on page, entrances, bounce rate, percent exit, and page value. More information on Google Analytics can be found at [hyperlink, http://www.google.com/analytics/]. [36] GAO, Environmental Justice: EPA Needs to Take Additional Actions to Help Ensure Effective Implementation, [hyperlink, http://www.gao.gov/products/GAO-12-77] (Washington, D.C.: Oct. 6, 2011). [37] FLC's next strategic plan may contain additional goals relevant to these initiatives. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]