Federally Funded Research and Development Centers: Improved Oversight and Evaluation Needed for DOD's Data Access Pilot Program
Fast Facts
DOD sponsors research and development activities at 10 Federally Funded Research and Development Centers that provide innovative solutions to national security threats.
These centers sometimes need to access sensitive DOD data (such as proprietary information from DOD’s contractors), which can be a long, time-consuming process. DOD began a pilot in 2017 to streamline these centers’ access to sensitive data.
We found that some centers reported easier access to sensitive data during the pilot. However, DOD has not consistently collected information on the pilot or developed a plan to evaluate it.
We recommended that DOD do so.
Aerial view of the Pentagon
Highlights
What GAO Found
The Department of Defense (DOD) launched a 3-year pilot program in December 2017 to enable a streamlined process to share certain sensitive data, such as data collected from its contractors, with its Federally Funded Research and Development Centers (FFRDC). At times, FFRDCs need to access such data to support DOD. The pilot was intended to reduce the burden on FFRDCs to seek permission from hundreds of contractors to access information needed for their research. Six of DOD's 10 FFRDCs have taken part in the pilot, enrolling a combined total of 33 projects, as shown in the table.
Department of Defense (DOD) FFRDC Pilot Projects Enrolled as of September 2019
|
FFRDC |
Project status |
Projects (number) |
|
National Defense Research Institute |
5 complete; 6 ongoing; 7 removed |
18 |
|
Systems and Analyses Center |
1 complete; 3 ongoing; 3 removed; 1 on hiatus |
8 |
|
Project AIR FORCE |
3 complete; 1 removed |
4 |
|
Arroyo Center |
1 ongoing |
1 |
|
National Security Engineering Center |
1 complete |
1 |
|
Software Engineering Institute |
1 complete |
1 |
|
Total |
|
33 |
Source: GAO analysis of DOD and Federally Funded Research and Development Centers (FFRDC) information. | GAO-20-272
Note: Projects were removed when researchers discovered that they did not need access to the data requested. The project on hiatus is pending a decision about the continuation of this study.
DOD officials and FFRDC representatives reported that the streamlined process made the use of sensitive data feasible. As a result, FFRDCs with completed projects in GAO's sample indicated they were able to provide more robust analyses or insights to DOD.
DOD guidance for the pilot program established procedures to protect sensitive data. But GAO found that DOD did not incorporate all of the details of the required protections into its agreements with FFRDCs. Further, GAO found that not all FFRDCs were performing annual certification of financial disclosure forms, as required by its agreements with DOD. DOD does not have a process to ensure that all the protections pertaining to FFRDCs' streamlined access to sensitive data are being followed. Without a process that defines roles and responsibilities, DOD cannot ensure that FFRDCs adhere to the protections.
DOD developed goals for the pilot program and outlined what information was to be obtained for each participating project, actions that are consistent with GAO's leading practices for pilot design. However, DOD has not developed a plan for evaluating the program nor has it consistently collected information on about a third of the pilot projects. Leading practices for pilot design call for an evaluation plan, which should include an assessment methodology and identify responsibilities as to how the evaluation will be conducted. Without an evaluation plan and a mechanism to collect information on pilot projects, DOD will not be positioned to identify the effectiveness of the pilot program and benefit from lessons learned. Such information will be useful as Congress considers the path forward after the pilot ends in December 2020.
Why GAO Did This Study
FFRDCs provide federal agencies with research and development functions, technical systems engineering capabilities, and policy development and decision-making studies, among other services. The Federal Acquisition Regulation states that FFRDCs have a special relationship with DOD, which can give FFRDCs access to sensitive data beyond what would commonly be shared with contractors. The National Defense Authorization Act for Fiscal Year 2017 directed DOD to establish a 3-year pilot program that allows FFRDCs streamlined access to sensitive data maintained by DOD. It also included a provision for GAO to report on the pilot program within 2 years of implementation. This report addresses the extent to which (1) FFRDCs are using the pilot program, (2) DOD put procedures in place to protect data accessed, and (3) DOD is evaluating the pilot program.
GAO reviewed DOD guidance and FFRDC processes, pilot reports for January 2018 through September 2019, and DOD's plans and efforts for evaluating the pilot program. GAO also selected a nongeneralizable sample of six projects—at least one from each FFRDC with an enrolled project as of December 2018—for further review. In addition, GAO assessed the pilot program against leading practices for pilot design.
Recommendations
GAO is making six recommendations, including that DOD take steps to ensure data protections are in the agreements and followed, collect information on projects, and evaluate the pilot. DOD agreed with the recommendations.
Recommendations for Executive Action
| Agency Affected Sort descending | Recommendation | Status |
|---|---|---|
| Office of the Under Secretary of Defense for Research and Engineering | The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to take steps to ensure that the FFRDCs and sponsors are implementing the pilot program's protections for sensitive data. (Recommendation 2) |
Closed – Not Implemented
The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. L&PO officials confirmed that as part of their assessment and pilot closeout activities they have requested that each participating FFRDC primary sponsor work with the contracting officers for FFRDC projects that participated in the pilot program to confirm the steps taken to ensure that the FFRDCs and sponsors have implemented the pilot program's protections for sensitive data, such as ensuring that all financial disclosure forms were on file. In our follow up, L&PO could not provide evidence that they had received this confirmation. However, L&PO did provide evidence that they followed up with FFRDCs and sponsors to determine if there were any violations of the trade secrets act or if other data violations had been appropriately reported, and that the results were negative, no violations had been reported. Ensuring the data protections were in place and followed was a key control to prevent improper disclosure. Because we could not verify that all data protections were in place, we are closing this recommendation as not implemented. However we recognize that no violations were reported in the time that L&PO officials were monitoring this effort.
|
| Office of the Under Secretary of Defense for Research and Engineering | The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to establish a monitoring and oversight mechanism to ensure that primary sponsors submit complete information on pilot projects, as required by DOD's guidance for the pilot program. (Recommendation 3) |
Closed – Implemented
The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. , OUSD(R&E)/L&PO officials have identified a mechanism for obtaining complete information on pilot projects through quarterly and post-action reporting. For example, L&PO followed up to obtain missing information at various times, including November 2020 and March 2021.
|
| Office of the Under Secretary of Defense for Research and Engineering | The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan that outlines the methodology by which DOD will assess the pilot and how and when information collected will be analyzed to evaluate the pilot program. (Recommendation 4) |
Closed – Implemented
The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In March 2021, OUSD(R&E)/L&PO provided GAO with the plan, which included information they planned to use to evaluate the pilot and efforts to collect input from stakeholders.
|
| Office of the Under Secretary of Defense for Research and Engineering | The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan to identify and evaluate lessons learned from the pilot program. (Recommendation 5) |
Closed – Implemented
The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In March 2021, OUSD(R&E)/L&PO provided GAO with examples of documented lessons learned from its implementation, and projects' use, of the pilot program. While OUSD(R&E)/L&PO did not develop a plan for evaluating the lessons learned, we believe that by soliciting input and compiling the information, OUSD(R&E)/L&PO has met the intent of the recommendation.
|
| Office of the Under Secretary of Defense for Research and Engineering | The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan for obtaining input from stakeholders on the pilot program. (Recommendation 6) |
Closed – Implemented
The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. OUSD(R&E)/L&PO has obtained input from users of the pilot program. For example, OUSD(R&E)/L&PO officials obtained participating FFRDCs' sponsors' input in November 2020 about challenges implementing and using the pilot program. While OUSD(R&E)/L&PO did not develop a plan for obtaining input, we believe that by soliciting input, OUSD(R&E)/L&PO has met the intent of the recommendation.
|
| Office of the Under Secretary of Defense for Research and Engineering | The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to take steps to ensure that the details of the pilot program's data protections are incorporated into the existing agreements. (Recommendation 1) |
Closed – Not Implemented
The Under Secretary of Defense for Research and Engineering (OUSD(R&E)), Laboratories and Personnel Office (L&PO) concurred with this recommendation. In May 2020, OUSD(R&E)/L&PO stated it would require the Primary Sponsor for each applicable federally funded research and development center (FFRDC) to provide documentation that shows that the details of the pilot program's data protections have been incorporated into existing sponsoring agreements and contracts. However, according to officials, OUSD(R&E)/L&PO had not done so prior to the end of the pilot program. Instead, officials said they are planning to have each Primary Sponsor confirm whether there have been any violations and, if so, if the violations were appropriately reported. This recommendation was time-sensitive and because the pilot has ended there is no benefit in pursuing updated addendums.
|