Federal Autism Activities:

Better Data and More Coordination Needed to Help Avoid the Potential for Unnecessary Duplication

GAO-14-16: Published: Nov 20, 2013. Publicly Released: Nov 20, 2013.

Additional Materials:

Contact:

Marcia G. Crosse
(202) 512-7114
crossem@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Eighty-four percent of the autism research projects funded by federal agencies had the potential to be duplicative. Of the 1,206 autism research projects funded by federal agencies from fiscal years 2008 through 2012, 1,018 projects were potentially duplicative because the projects were categorized to the same objectives in the Interagency Autism Coordinating Committee's (IACC) strategic plan. Funding similar research on the same topic is sometimes appropriate--for example, for purposes of replicating or corroborating results--but in some instances, funding similar research may lead to unnecessary duplication. The potentially duplicative research projects included those funded by the Department of Defense (DOD), Department of Education (Education), National Science Foundation (NSF), and agencies within the Department of Health and Human Services (HHS)--Administration for Children and Families, Agency for Healthcare Research and Quality, Centers for Disease Control and Prevention (CDC), Centers for Medicare & Medicaid Services, Health Resources and Services Administration, National Institutes of Health (NIH), and the Substance Abuse and Mental Health Services Administration. Each agency funded at least 1 autism research project in the same strategic plan objective as another agency. For example, 5 agencies awarded approximately $15.2 million for 20 autism research projects related to 1 objective to test methods to improve dissemination, implementation, and sustainability of evidence-based interventions, services, and supports in diverse community settings.

The IACC's and federal agencies' efforts to coordinate and monitor federal autism activities were limited. The IACC--composed of federal and nonfederal members--met regularly and issued several reports, such as a strategic plan and portfolio analysis--a report that provides information on autism research projects, organized by the strategic plan objectives. The IACC has also released a companion database to its portfolio analysis. However, IACC members provided mixed views on the usefulness of the IACC's meetings, strategic plan, and portfolio analysis in aiding coordination and monitoring. While three agencies--CDC, DOD, and NIH--regularly used the committee's strategic plan and portfolio analysis, others did not. Shortcomings in the data the IACC used for its portfolio analysis limited its ability to coordinate HHS autism activities and monitor federal autism activities--as required by the Combating Autism Act of 2006 (CAA). For example, GAO found that the data used by the IACC was outdated, not tracked over time, inconsistent, and incomplete. These weaknesses limited the IACC's ability to monitor its progress on its coordination and monitoring efforts--which, in prior work, GAO established as a best practice for inter-agency collaboration, as well as a federal internal control standard. In addition, these weaknesses limited agencies' ability to use these data to identify coordination opportunities and avoid the potential for unnecessary duplication. Such information is important because of the involvement of multiple agencies. Lastly, apart from their participation on the IACC, there were limited instances of agencies coordinating, and agencies did not have robust or routine procedures for monitoring federal autism activities. Per federal internal control standards, agencies should establish a means of communicating with other agencies; this is important to maximize the efficiency of the federal autism investment and minimize the potential for unnecessary duplication.

Why GAO Did This Study

Autism—a developmental disorder involving communication and social impairment—is an important public health concern. From fiscal years 2008 through 2012, 12 federal agencies awarded at least $1.4 billion to support autism research and other autism-related activities. The CAA directed the IACC to coordinate HHS autism activities and monitor all federal autism activities. It also required the IACC to develop and annually update a strategic plan for autism research. This plan is organized into 7 research areas that contain specific objectives.

GAO was asked to examine federal autism efforts. In this report, GAO (1) analyzes the extent to which federal agencies fund potentially duplicative autism research, and (2) assesses the extent to which IACC and agencies coordinate and monitor federal autism activities. GAO analyzed agencies’ data and documents, and interviewed federal agency officials and select nonfederal IACC members.

What GAO Recommends

GAO is recommending that (1) HHS improve the usefulness of IACC data to enhance coordination and monitoring of federal autism activities, and (2) DOD, Education, HHS, and NSF improve their coordination of autism research. HHS disagreed with the first recommendation stating that it was already making adequate efforts. The agencies supported the need for improved coordination but, except for DOD, disputed that any duplication occurs. GAO continues to believe the recommendations are warranted as discussed in the report.

For more information, contact Marcia Crosse at (202) 512-7114 or crossem@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: HHS disagreed with this recommendation and reiterated its disagreement in August 2014. While it stated it recognizes that consistency is important when conducting analysis, it said it was also important to adapt to changes in the field, incorporate new knowledge, and be responsive to other needs. We recognize that these factors deserve consideration. However, the annual changes of the type we observed are not productive. The exclusion of training projects in one year of data collection, their inclusion in the subsequent year, and their partial exclusion in the next year is a change in methodology and guidance that makes it difficult to analyze and interpret data over time. Additionally, consistent guidance was not provided to organizations--both federal and nonfederal--that submit information for inclusion in HHS's portfolio analysis. For example, one year HHS requested the Health Resources and Services Administration to include information on its training projects in the portfolio analysis, but did not make the same request of other organizations. GAO believes that guidance should be developed so that accurate, consistent, and meaningful comparisons of changes in federal funding of autism research can be made over time and used to inform future funding decisions.

    Recommendation: To improve the usefulness of IACC data and enhance its efforts to coordinate HHS autism activities and monitor all federally funded autism activities, the Secretary of Health and Human Services should direct the IACC and NIH, in support of the IACC, to provide consistent guidance to federal agencies when collecting data for the portfolio analysis and web tool so that information can be more easily and accurately compared over multiple years.

    Agency Affected: Department of Health and Human Services

  2. Status: Open

    Comments: HHS disagreed with this recommendation and reiterated its disagreement in August 2014. HHS said it does not think that producing a document or database to catalogue non-research activities would be useful as this would duplicate ongoing efforts. HHS stated that other agencies have public accessible reports that describe these activities such as the 2010 HHS report, Report to Congress on Activities Related to Autism Spectrum Disorder and Other Developmental Disabilities under the Combating Autism Act of 2006, a 2011 report from the Centers for Medicare & Medicaid Services entitled, Report on State Services to Individuals with Autism Spectrum Disorders (ASD), and a 2013 report by the Health Resources and Services Administration entitled, Efforts to Improve Autism Spectrum Disorders Service Delivery under the Combating Autism Act Initiative: Research, Training, and State Implementation Grants for Fiscal Year(s) 2008-2011. GAO agrees these documents are important and helpful. However, they are static reports that are not useful for monitoring projects over time. GAO believes that having a document or database that contains current information on these non-research activities is an important aspect of fulfilling the IACC's responsibility to monitor all federal autism activities, not just research.

    Recommendation: To improve the usefulness of IACC data and enhance its efforts to coordinate HHS autism activities and monitor all federally funded autism activities, the Secretary of Health and Human Services should direct the IACC and NIH, in support of the IACC, to create a document or database that provides information on non-research autism-related activities funded by the federal government and make this document or database publicly available.

    Agency Affected: Department of Health and Human Services

  3. Status: Open

    Comments: HHS disagreed with this recommendation and reiterated its disagreement in August 2014. HHS stated that an analysis by the IACC to identify duplication would not likely provide the type of information on actual duplication needed by agency officials when making funding decisions. HHS said that even if the IACC further refined its analysis to identify several grants on the same general topic within each research objective as potentially duplicative, this type of information would lack adequate specificity to be useful to agency officials who are involved in the grant award process. HHS expressed concern that the IACC did not have the expertise to take on this challenge and also that it would be a time-consuming endeavor. HHS also stated that a recent review of the autism research portfolio by the IACC found that in almost all areas of the portfolio there is a need for more-not less-research activity. Given that funding agencies conduct their own reviews of grant applications prior to funding, HHS views any efforts made by the IACC to identify duplicative grants would actually duplicate the efforts of other agencies and be of limited utility. GAO understands that the IACC's identification of instances of duplication would need further review to determine actual duplication. However, we believe that the IACC?s identification of such projects would be worthwhile as it could effectively lead to their further review by the funding agencies. Certainly, if unnecessary duplication or opportunities to enhance coordination between HHS agencies were identified, the IACC could make related recommendations to the agencies that would carry important weight. We acknowledge that while the IACC does not have the authority to eliminate duplicative projects, it does have the responsibility to make recommendations to the Secretary of Health and Human Services. GAO never took the position that the federal government is funding too much autism research. Instead, GAO believes that autism is a significant public health concern making it all the more important that scarce federal resources be used wisely. Many agencies currently devote considerable time to identifying and providing HHS with spending information and HHS invests in this data collection. GAO continues to question the purpose of devoting federal resources to collecting these data, if they are not then used to ensure federal funds are used appropriately.

    Recommendation: To improve the usefulness of IACC data and enhance its efforts to coordinate HHS autism activities and monitor all federally funded autism activities, the Secretary of Health and Human Services should direct the IACC and NIH, in support of the IACC, to identify projects through its monitoring of federal autism activities--including Office of Autism Research Coordination's annual collection of data for the portfolio analysis and the IACC's annual process to update the strategic plan--that may result in unnecessary duplication and thus may be candidates for consolidation or elimination, and identify potential coordination opportunities among agencies.

    Agency Affected: Department of Health and Human Services

  4. Status: Open

    Comments: In August 2014 Education reported that it has worked with the Office of Autism Research Coordination (OARC) within HHS's National Institutes of Health on issues concerning the scope of autism research. These coordination activities were ongoing but will be contingent upon HHS decisions about re-configuration of OARC based on the recently enacted Autism CARES Act, which directs the Secretary of HHS to designate an official within that Department to oversee national autism research activities. The law also requires that this official must, in consultation with the Secretaries of Defense and Education, ensure that autism research funded by HHS and other federal agencies is not unnecessarily duplicative. Education reported that it welcomes the opportunity to coordinate, under the reconfigured OARC on issues concerning the scope of autism research, including identification of awards and examining protocols for coordinating with the Department of Defense and the National Science Foundation.

    Recommendation: To promote better coordination among federal agencies that fund autism research and avoid the potential for unnecessary duplication before research projects are funded, the Secretary of Health and Human Services, the Secretary of Defense, the Secretary of Education, and the Director of NSF should each determine methods for identifying and monitoring the autism research conducted by other agencies, including by taking full advantage of monitoring data the IACC develops and makes available.

    Agency Affected: Department of Education

  5. Status: Open

    Comments: In August 2014 DOD reported that its Congressionally Directed Medical Research Programs (CDMRP) has engaged in a major project to network and share data with federal agencies including the National Institutes of Health (NIH) through participation in the Federal RePORTer system (http://federalreporter.nih.gov/). Additionally, the NIH, DoD, and the Department of Veterans Affairs (VA) have conducted meetings to configure a methodology to share data for the purposes of reducing unnecessary duplication of research and provide visibility to the public in a central location. This Federal RePORTer initiative allows for multiple agencies to house award data in a central database. Although separate from NIH RePORTer, it utilizes some of its basic functions on a core set of data required from all agencies allowing for analysis and comparison. The site is still in development, but is live to the public. Additional features are in the planning stages. Participating agencies include the Department of Health and Human Services, Department of Agriculture, Department of Defense, Department of Education, VA, Environmental Protection Agency, National Aeronautics and Space Administration, and National Science Foundation. With common data elements the research from multiple federal agencies including NIH and the CDMRP, projects can be analyzed and evaluated for funding overlaps and duplication. In addition, DOD, along with the NIH, plans to conduct a pilot data transfer of CDMRP data into an NIH system for evaluation of research funded by the NIH, VA, and DoD. This pilot will use the tools available in Query View Report (QVR) module will use the QVR tools to compare and analyze data from CDMRP proposed research applications that have not yet been funded against funded projects and proposed applications from other agencies. The comparability of the data elements will allow for the creation of a fingerprint for each project and allow for assessment against other projects both funded and unfunded. This will permit program staff to review and eliminate possible overlap and/or duplication prior to any possible award. Also, the CDMRP Autism Research Program (ARP) continues to monitor for potential overlap and/or duplication using traditional methods and by taking full advantage of monitoring data available through the IACC. The CDMRP ARP Program Manager is member of the Interagency Autism Coordinating Committee (IACC) and provides the CDMRP ARP funding investment data for inclusion in the IACC portfolio. The CDMRP ARP then utilizes all of the materials the IACC publishes on their strategic plan and portfolio analysis report to inform the ARP advisory committee, the Integration Panel. Additionally, the standard methods by the Program Office to monitor overlap and/or duplication using traditional methods are continued to be used and are detailed on our website at http://cdmrp.army.mil/funding/researchDup.shtml.

    Recommendation: To promote better coordination among federal agencies that fund autism research and avoid the potential for unnecessary duplication before research projects are funded, the Secretary of Health and Human Services, the Secretary of Defense, the Secretary of Education, and the Director of NSF should each determine methods for identifying and monitoring the autism research conducted by other agencies, including by taking full advantage of monitoring data the IACC develops and makes available.

    Agency Affected: Department of Defense

  6. Status: Open

    Comments: HHS disagreed with this recommendation and reiterated its disagreement in August 2014. While HHS stated that it is important to avoid unnecessary duplication, it has not encountered any actual examples of this nor did GAO identify any such instances. HHS stated that the National Institute of Health (NIH), which funded 81 percent of autism research during fiscal years 2008 through 2012, has a comprehensive internal data base that is used extensively to prevent unnecessary duplication and overlap. HHS also stated that the IACC provides opportunities for cross-agency information sharing. GAO maintains that NIH's database, while important, is not sufficiently comprehensive as it is used to identify duplicative projects led by the same principle investigator. It does not identify project applications led by another principle investigator that may be unnecessarily duplicative of a project that has already been federally funded?a project with the same purpose, strategies, and target population that is not necessary to, for example, corroborate or replicate prior research results. Searches would not identify similar projects led by different principal investigators. Therefore, GAO stands by its conclusion that these monitoring effort and coordination of federal autism activities are limited. GAO agrees the IACC provides opportunities for cross-agency information sharing, but such opportunities are not alone sufficient to monitor all federal autism activities.

    Recommendation: To promote better coordination among federal agencies that fund autism research and avoid the potential for unnecessary duplication before research projects are funded, the Secretary of Health and Human Services, the Secretary of Defense, the Secretary of Education, and the Director of NSF should each determine methods for identifying and monitoring the autism research conducted by other agencies, including by taking full advantage of monitoring data the IACC develops and makes available.

    Agency Affected: Department of Health and Human Services

  7. Status: Open

    Comments: In August 2014 NSF reported that it is remaining an active participant in the Interagency Autism Coordinating Committee (IACC). NSF's program officer for Research in Disabilities Education is currently doing a monthly check of the Autism Spectrum Disorder Research Portfolio Analysis Web Tool (PAWT). He also represented NSF at the January 2014 meeting with the IACC Full Committee and a February 2014 meeting with the Interagency Committee on Disability Research (ICDR). At the ICDR meeting NSF discussed the IACC's efforts, NSF's contributions to the PAWT, and how the PAWT is a useful resource for all agencies to use for it uses the PAWT for the purposes of searching for expert reviewers and learning about currently, and previously, funded educational research about students with Autism Spectrum Disorder (ASD). NSF plans to continue attending the IACC meetings in addition to checking and providing information on NSF grants for the PAWT.

    Recommendation: To promote better coordination among federal agencies that fund autism research and avoid the potential for unnecessary duplication before research projects are funded, the Secretary of Health and Human Services, the Secretary of Defense, the Secretary of Education, and the Director of NSF should each determine methods for identifying and monitoring the autism research conducted by other agencies, including by taking full advantage of monitoring data the IACC develops and makes available.

    Agency Affected: National Science Foundation

 

Explore the full database of GAO's Open Recommendations »

Sep 8, 2014

Sep 2, 2014

Aug 22, 2014

Aug 13, 2014

Aug 11, 2014

Jul 30, 2014

Jul 29, 2014

Jul 23, 2014

Looking for more? Browse all our products here