Fast Facts

Public access to the results of federally funded research can accelerate scientific breakthroughs. In 2013, certain federal agencies were directed to create plans for increasing access to publications and data they funded.

The 19 agencies we reviewed made progress, but some have not fully implemented their plans. For example:

7 agencies have not taken steps to make data findable, such as creating a single web access point

4 don’t require all researchers to submit a plan to provide access to data

11 don’t fully ensure that researchers comply with access requirements

We made 37 recommendations to 16 agencies to address these and other issues.

Hands on a keyboard

Hands on a keyboard

Skip to Highlights
Highlights

What GAO Found

The 19 agencies that GAO reviewed have made progress implementing their plans to increase public access to federally funded research results (publications and data), as called for in a 2013 Office of Science and Technology Policy (OSTP) memorandum. However, some agencies have not fully implemented some aspects of their plans, in particular those related to data access and mechanisms to ensure researchers comply with public access requirements.

Examples of Agencies' Progress Implementing Plans to Increase Public Access to Federally Funded Research Results

Public access plan topic

Extent of agency progress

Repositories

All 19 agencies have identified federally owned or managed locations, known as repositories, for preservation and public access to publications. For data, agencies rely on an array of federal and nonfederal repositories. However, seven agencies have not taken steps, such as establishing a single web-based point of access, or have not fully implemented plans to help the public find data stored across repositories. Taking such steps could better support public access to federally funded data.

Data management plans (DMPs)

Sixteen of 19 agencies reported requiring researchers to submit a DMP, which is supposed to describe how researchers will provide for long-term preservation and access to data they generate, or a justification for why that cannot be done. However, four agencies reported they have not established such requirements or have done so on a limited basis. Without requiring DMPs from agency-funded researchers, agencies may not be able to ensure that agency-funded data are being made publicly available.

Compliance

Eleven agencies reported that they have not fully developed or implemented mechanisms to ensure researchers comply with applicable public access requirements. Officials cited several reasons for this, including resource constraints and difficulty with tracking and measuring compliance. Without fully implementing compliance mechanisms—as called for in the OSTP memorandum—agencies may not have assurance that all appropriate federally funded research results are being made publicly available.

Source: GAO analysis of agency public access plan implementation efforts. | GAO-20-81

Agencies are coordinating with each other and with nonfederal stakeholders to implement public access plans, including through an interagency group led by OSTP and five other agencies. However, the group has not fully implemented selected leading practices identified by GAO that can enhance and sustain interagency collaboration, such as defining and articulating common outcomes. For example, according to OSTP staff, key outcomes have not yet been decided upon. Agency officials and stakeholders identified several challenges to implementing public access plans that interagency coordination might help them address, such as

Absence of common standards in several areas;

Measuring effectiveness of public access plan implementation; and

Balancing providing public access with safeguarding sensitive information.

By taking steps to fully implement relevant leading collaboration practices, the interagency group could help agencies better marshal their collective efforts to address common challenges to public access plan implementation.

Why GAO Did This Study

Research and development helps catalyze breakthroughs that improve the overall health and wellbeing of our society. Federal research and development expenditures averaged about $135 billion annually for fiscal years 2015 to 2017. According to OSTP, providing free public access to federally funded research results can improve both the impact and accountability of this important federal investment. In February 2013, OSTP directed federal agencies with more than $100 million in annual research and development expenditures to develop a plan to support increased public access to the results of federally funded research. GAO was asked to examine public access to federally funded research results. This report examines the extent of agencies' (1) progress implementing plans to increase public access to federally funded research results and (2) coordination on public access plan implementation. GAO administered a questionnaire to 19 federal agencies selected based on annual research and development expenditure amounts, among other criteria; reviewed agency documents; and interviewed officials from 11 agencies, OSTP, and 21 stakeholder organizations.

Skip to Recommendations

Recommendations

GAO is making 37 recommendations to 16 agencies to promote full and effective implementation of agency public access plans. For example, GAO recommends that OSTP and 5 agencies leading a public access interagency group take steps to fully implement selected leading collaboration practices. Of the 16 agencies, 15 agreed with GAO's recommendations while 1 (OSTP) disagreed. GAO continues to believe the recommendation to OSTP is warranted.

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Defense 1. The Secretary of Defense should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 1)
Open
The Department of Defense partially concurred with this recommendation noting the challenge with balancing ensuring public access to research data with considerations of national security and personally identifiable information. As discussed in our report, our recommendation to DOD regarding findability and accessibility of agency-funded research data was qualified to pertain to appropriate agency-funded research data--recognizing that it might not be appropriate to make certain datasets publically available because of national security or other concerns. In June 2020, DOD stated research data associated with public journal articles became publicly available on PubDefense, but that DOD is still researching methodologies that would identify situations where aggregation of multiple sets of unclassified data could reveal national security vulnerabilities. DOD officials said when an acceptable methodology is developed, they would issue guidance on its use. When we confirm what methodology DOD has developed, we will provide updated information.
Department of Education 2. The Secretary of Education should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 2)
Open
The Department of Education concurred with this recommendation. According to its response to our report, the Department awarded a contract to support enhancements to its Education Resources Information Center (ERIC) to link scholarly research publications supported by the Department to its publicly accessible datasets. The Department indicated it expects to complete this work by September 30, 2020. When we confirm what actions the Department has taken to implement this recommendation we will provide additional information.
Agency for Healthcare Research and Quality 3. The Director of the Agency for Healthcare Research and Quality should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 3)
Open
The Agency for Healthcare Research and Quality concurred with this recommendation. According to AHRQ officials, a draft data management policy articulating expectations for researchers to make agency-funded data readily findable and accessible to the public has been developed and is currently in the clearance process with AHRQ's Office of the Director and General Counsel. When we confirm these steps have been taken, we will provide updated information.
Food and Drug Administration 4. The Commissioner of the Food and Drug Administration should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 4)
Open
The Food and Drug Administration concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Department of Homeland Security 5. The Secretary of Homeland Security should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 5)
Open
The Department of Homeland Security concurred with this recommendation and noted that it was in the process of establishing a portal on its website to increase public access to agency-funded research. The department estimated these efforts would be completed by June 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Department of Veterans Affairs 6. The Secretary of Veterans Affairs should take steps to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 6)
Open
The Department of Veterans Affairs concurred with this recommendation but indicated in its comments on the report that had already taken steps to implement it. As discussed in the report, the department's efforts to ensure research data availability pertained to a portion of the agency's federally-funded research data. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
National Science Foundation 7. The Director of the National Science Foundation should fully implement plans to ensure appropriate agency-funded research data are readily findable and accessible to the public. (Recommendation 7)
Open
NSF concurred with this recommendation. According to its response to our report, NSF is expanding its public access repository to include metadata records about data that support publications resulting from NSF-funded research. NSF's response stated that, by storing metadata records for supporting datasets alongside metadata records for publications, the public will be able to more easily find and access appropriate agency funded research data. We will provide an update when we obtain additional information about the status of implementing this recommendation.
Department of Defense 8. The Secretary of Defense should complete development of data management plan requirements for extramural researchers. (Recommendation 8)
Open
The Department of Defense concurred with this recommendation stating data management plan requirements would be in place by September 2023. DOD officials said requirements for award recipients have been proposed for the DoD Grant and Agreement Regulations, while similar requirements for contract awardees have been proposed for the Defense Federal Acquisition Regulation Supplement. Once these requirements are complete, we will provide updated information.
Agency for Healthcare Research and Quality 9. The Director of the Agency for Healthcare Research and Quality should complete development of data management plan requirements. (Recommendation 9)
Open
The Agency for Healthcare Research and Quality concurred with this recommendation. When we confirm what actions the agency has taken to implement the recommendation we will provide updated information.
Department of Homeland Security 10. The Secretary of Homeland Security should complete development of data management plan requirements. (Recommendation 10)
Open
The Department of Homeland Security concurred with this recommendation, stating that it would develop a departmentwide management directive for research and development data as well as data management plan guidance and a template to document requirements. The department estimated that these efforts would be completed by June 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
United States Agency for International Development 11. The U.S. Agency for International Development Administrator should complete development of data management plan requirements for extramural researchers. (Recommendation 11)
Open
The U.S. Agency for International Development agreed with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Department of Agriculture 12. The Secretary of Agriculture should complete development of guidance and provide training to agency officials or others involved in reviewing the merits of researchers' data management plans. (Recommendation 12)
Open
The U.S. Department of Agriculture agreed with the findings of our report. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
United States Agency for International Development 13. The U.S. Agency for International Development Administrator should complete development of and provide training for agency officials or others involved in reviewing the merits of researchers' data management plans. (Recommendation 13)
Open
The U.S. Agency for International Development agreed with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Food and Drug Administration 14. The Commissioner of the Food and Drug Administration should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 14)
Open
The Food and Drug Administration concurred with this recommendation. When we confirm what actions the agency has taken to implement the recommendation we will provide updated information.
Department of Homeland Security 15. The Secretary of Homeland Security should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 15)
Open
The Department of Homeland Security concurred with this recommendation, and indicated it would evaluate training needs for data management plan reviews and develop plans to fulfill any additional training needs identified. The department estimated these efforts would be completed by September 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Department of Veterans Affairs 16. The Secretary of Veterans Affairs should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 16)
Open
The Department of Veterans Affairs concurred with this recommendation and identified several planned steps to identify and meet training needs for those involved in reviewing researchers' data management plans. The department estimated these efforts would be completed by September 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Agency for Healthcare Research and Quality 17. The Director of the Agency for Healthcare Research and Quality should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 17)
Open
The Agency for Healthcare Research and Quality concurred with this recommendation stating that it would assess training needs for agency officials and others involved in reviewing the merits of researchers' data management plans and would develop and provide additional training if warranted. When we confirm what actions the agency has taken to implement the recommendation we will provide updated information.
Department of Defense 18. The Secretary of Defense should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 18)
Open
The Department of Defense concurred with this recommendation. DOD officials stated they are developing an automated data management plan tool as a template for researchers to use at the beginning of a research project. According to DOD officials, after development of this tool, a training program for researchers and program managers will be developed, if warranted, including an outline of the preferred structure for the DMP, a description of the essential elements of the plan and a methodology to evaluate its content. DOD officials expect to complete this work by September 30, 2021. Once these steps have been completed, we will provide updated information.
Department of Energy 19. The Secretary of Energy should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 19)
Open
The Department of Energy concurred with this recommendation and stated it would assess and develop a plan to meet the training needs of internal DOE staff and external peer reviewers of data management plans. DOE estimated these efforts would be completed by December 31, 2020. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Environmental Protection Agency 20. The Environmental Protection Agency Administrator should evaluate training needs for agency officials or others involved in reviewing the merits of researchers' data management plans and, if additional training is found to be warranted, develop and provide such training. (Recommendation 20)
Open
The Environmental Protection Agency agreed with this recommendation stating that it planned to evaluate training needs for agency officials who review researchers' data management plans. According to information EPA provided in December 2020, the agency had developed a checklist for EPA managers and grant managers to use when reviewing data management plans. EPA indicated it plans to conduct data management plan training in 2021. When we confirm what additional actions the agency has taken in response to this recommendation, we will provide updated information.
Department of Agriculture 21. The Secretary of Agriculture should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 21)
Open
The U.S. Department of Agriculture agreed with the findings of our report. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Department of Defense 22. The Secretary of Defense should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 22)
Open
The Department of Defense concurred with this recommendation. According to DOD officials, DOD research organizations will perform a self-assessment of their compliance with public access requirements and also investigate methods to analyze DOD research projects to more easily identify instances of research organization non-compliance with their public access plan. DOD expects this work to be completed by September 2022. Once this work is completed, we will provide updated information.
Agency for Healthcare Research and Quality 23. The Director of the Agency for Healthcare Research and Quality should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements.(Recommendation 23)
Open
The Agency for Healthcare Research and Quality concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Food and Drug Administration 24. The Commissioner of the Food and Drug Administration should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 24)
Open
The Food and Drug Administration concurred with this recommendation. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
National Institutes of Health 25. The Director of the National Institutes of Health should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 25)
Open
The National Institutes of Health concurred with this recommendation but stated that the agency already had mechanisms in place to ensure compliance with public access plan and associated requirements for publications and data. As discussed in the report, we believe our recommendation, as worded, appropriately reflected the extent to which the agency had implemented researcher compliance mechanisms at the time of issuance of our report. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Department of Homeland Security 26. The Secretary of Homeland Security should develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 26)
Open
The Department of Homeland Security concurred with this recommendation and stated it would develop a mechanism to ensure researcher compliance with the department's public access plan and data management plan requirements. The department estimated these efforts would be completed by September 30, 2020. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
National Oceanic and Atmospheric Administration 27. The National Oceanic and Atmospheric Administration Administrator should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 27)
Open
The Department of Commerce agreed with this recommendation and the National Oceanic and Atmospheric Administration indicated it was pursuing multiple mechanisms to implement it. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Department of Energy 28. The Secretary of Energy should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 28)
Open
The Department of Energy concurred with this recommendation stating that it would develop a compliance mechanism to identify researchers receiving funding from DOE financial assistance awards who are not compliant with DOE's public access plan for publications. DOE estimated these efforts would be completed by December 31, 2020. When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Department of Veterans Affairs 29. The Secretary of Veterans Affairs should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 29)
Open
The Department of Veterans Affairs concurred with this recommendation but indicated in its comments on the report that had already taken steps to implement it. As discussed in the report, the department's public access compliance mechanism covered a portion of the agency's federally-funded research data.. When we confirm what actions the agency has taken in response to the recommendation, we will provide updated information.
Department of Transportation 30. The Secretary of Transportation should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 30)
Open
The Department of Transportation concurred with this recommendation. According to its response to our report, the Department will build upon existing compliance mechanisms to ensure researcher compliance with its public access plan and associated requirements. As part of this process, the Department reported that it plans to update its public access plan, and expects to complete these actions by December 31, 2020. When we confirm what actions the Department has taken to implement this recommendation we will provide additional information.
National Institute of Standards and Technology 31. The National Institute of Standards and Technology Director should fully develop and implement a mechanism to ensure researcher compliance with the public access plan and associated requirements. (Recommendation 31)
Open
The Department of Commerce agreed with this recommendation and the National Institute of Standards and Technology (NIST) identified several planned steps to implement it. Specifically, in January 2021, NIST deployed a system developed to identify NIST-authored papers that have been published, and ensure that all NIST authors are in compliance with the requirement that papers are made freely available within 12 months of publication. Previously, NIST also completed a review of awardees' compliance with data management plan requirements, and took steps (e.g., developing a template and checklist) to address issues NIST identified. Additionally, NIST took steps to evaluate an option to ensure compliance with public access requirements by NIST awardees (i.e., extramural researchers). NIST determined that the option it evaluated would not be practical as of April 2021. We will further update the recommendation status when NIST provides additional information on steps taken to ensure compliance with public access requirements by NIST's extramural researchers.
Office of Science and Technology Policy 32. As the Subcommittee on Open Science moves forward, the Office of Science and Technology Policy co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 32)
Open
OSTP disagreed with GAO's November 2019 recommendation, stating that the subcommittee had already taken steps to implement the leading practices GAO identified. However, OSTP officials did not provide documentation of these efforts and GAO continued to believe the recommendation was warranted. In March 2020, OSTP provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.
Department of Defense 33. As the Subcommittee on Open Science moves forward, the Department of Defense co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 33)
Open
DOD concurred with GAO's November 2019 recommendation, noting that DOD participates in subcommittee initiatives, including a working group on disclosure risk management, which is a topic of great importance to DOD. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.
Department of Energy 34. As the Subcommittee on Open Science moves forward, the Department of Energy co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 34)
Open
DOE concurred with GAO's November 2019 recommendation. DOE noted that, as a co-chair of the subcommittee, it is actively identifying areas of collaboration across agencies in implementing open science practices. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.
National Institutes of Health 35. As the Subcommittee on Open Science moves forward, the National Institutes of Health co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 35)
Open
NIH concurred with GAO's November 2019 recommendation. NIH stated that the subcommittee and its working groups are actively coordinating and building consensus on issues and processes to implement leading practices that enhance and sustain collaboration across federal agencies. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.
National Oceanic and Atmospheric Administration 36. As the Subcommittee on Open Science moves forward, the National Oceanic and Atmospheric Administration co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 36)
Open
NOAA concurred with GAO's November 2019 recommendation, noting that NOAA would work with the subcommittee to identify more opportunities for collaboration to promote access to research results. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.
National Science Foundation 37. As the Subcommittee on Open Science moves forward, the National Science Foundation co-chair, in coordination with other co-chairs and participating agencies, should take steps to fully implement leading practices that enhance and sustain collaboration. (Recommendation 37)
Open
The National Science Foundation concurred with GAO's November 2019 recommendation. In March 2020, the Office of Science and Technology Policy provided information on steps the subcommittee has taken to address issues associated with public access to federally funded research results, including certain areas GAO identified as presenting challenges to public access plan implementation in the November 2019 report. GAO will collect and evaluate additional information to determine the extent to which these steps incorporate leading practices for interagency collaboration GAO has identified. By taking steps to fully implement the relevant leading practices we have identified, the subcommittee and its member agencies could better marshal their collective efforts to address common public access plan implementation challenges that agency officials and stakeholders identified.

Full Report

GAO Contacts