Women in STEM Research:
Better Data and Information Sharing Could Improve Oversight of Federal Grant-making and Title IX Compliance
GAO-16-14: Published: Dec 3, 2015. Publicly Released: Dec 14, 2015.
- Highlights Page:
- Full Report:
- Accessible Version:
What GAO Found
GAO's analysis of Science, Technology, Engineering, and Mathematics (STEM) research grant awards made between fiscal years 2009 and 2013 identified no disparities in success rates between women and men at three agencies selected for review, but data limitations provided limited insight into success rates at three other agencies selected for review. At two of the agencies with data limitations—the Departments of Defense (DOD) and Energy (DOE)—GAO found evidence of disparities in success rates for women and men within certain agency components. Data limitations at the National Aeronautics and Space Administration (NASA) prevented GAO's analysis of success rates altogether. This lack of complete, linked electronic proposal and award data at NASA and some components at DOD and DOE impacts their ability to fully evaluate their programs' performance against their stated goals of funding the most qualified scientists, irrespective of gender. Adhering to federal internal control standards regarding data collection of an entire process can ensure that these agencies have the data needed for effective program management and monitoring.
Success Rates between Women and Men at Federal STEM Grant-making Agencies
Two of six agencies GAO reviewed that fund STEM research at universities—DOD and the Department of Health and Human Services (HHS)—are not conducting required Title IX compliance reviews. Since HHS oversees Title IX compliance of National Institutes of Health (NIH) funding recipients, which account for the bulk of STEM research grantees, billions of federal research dollars may not be subject to potential Title IX oversight. The Department of Justice (DOJ) is designated by Executive Order to coordinate Title IX compliance across federal agencies, including information sharing, but it has no formal information sharing process among STEM agencies. Officials at five of the six agencies GAO interviewed reported a desire for DOJ to facilitate interagency information sharing on Title IX best practices for compliance activities. Without such information sharing, these STEM agencies may miss opportunities to improve their compliance programs and coordinate with each other.
GAO identified through a literature review and expert interviews 13 potential actions federal agencies could take to address the underrepresentation of women in STEM research. These actions fell into four areas: (1) enhancing agency leadership and collaboration, (2) establishing family-friendly policies for grantees, (3) overseeing the research proposal review process, and (4) funding and assisting academic institutions. While not all of the actions GAO identified are relevant to or feasible for each agency, all six agencies in GAO's review indicated that they are either taking actions in some of these areas, or would be willing to explore their applicability. Most of the agency officials GAO spoke with acknowledged the potential benefits of these actions.
Why GAO Did This Study
In fiscal year 2014, U.S. universities received nearly $25 billion in federal grant funding for STEM research. Studies show women are largely underrepresented in STEM fields. Federal agencies are required to enforce Title IX—a law prohibiting discrimination on the basis of sex in education programs receiving any federal financial assistance—including at universities they fund. GAO was asked to provide information on federal grant-making to women in STEM.
This report examines: (1) the extent to which differences exist in federal grant awards between women and men in STEM fields, (2) the extent to which federal agencies enforce Title IX at universities they fund for STEM research, and (3) possible actions federal agencies could take to address the representation of women in STEM research. GAO analyzed data on all STEM research grants made in fiscal years 2009 through 2013—the most recent data available—by the six federal agencies that provided 90 percent of STEM research funding in fiscal year 2012 through 2014; reviewed literature, federal laws and regulations, and agency documents; interviewed federal officials; and consulted 19 STEM diversity experts.
What GAO Recommends
GAO recommends that DOD, DOE, and NASA collect additional data; DOD and HHS conduct Title IX compliance reviews; and DOJ facilitate information sharing among STEM agencies. Agencies agreed in principle, but some cited potential implementation challenges. GAO maintains action is feasible and warranted as discussed in the report.
For more information, contact Melissa Emrey-Arras at (617) 788-0534 or email@example.com.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: DOD agreed with our recommendation. In August 2017, the Basic Research Office (BRO) drafted an implementation plan and schedule for the collection of demographic data on grant applicants and lifecycle grant data. In October 2017, the Acting Secretary of Defense for Research and Engineering (ASD-R&E) advised DOD components that funding opportunity announcements issued on or after January 1, 2018 must include the Standard Form-424 Research and Related Personal Data form and a Senior/Key Persons form to collect demographic information, including gender, from applicants and key project personnel for grants and cooperative agreements. The demographic data collection form states that this information will be separated from the funding application and not used in the application review process. In April 2018, another memo was issued advising components to submit their linked applicant and award data to the office of ASD-R&E by November 15, 2018 for use in calculating success rates for female grant applicants, among other statistics. Both of these data collection instructions apply to all components that issue funding announcements for basic research, including the Army, Navy, Air Force, and other research components, and they apply to all funding opportunities where a grant or cooperative agreement is anticipated and an institution of higher learning is eligible to apply for the award. DOD plans to analyze the fiscal year 2018 data in December 2018. The data collection and analysis requirement took effect on January 1, 2018, applying to all funding opportunities from that point forward.
Recommendation: In order to ensure complete, analyzable records regarding research grant award decisions are available for management and analysis, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology and Logistics to lead the implementation of additional data collection efforts in coordination with DOD's grant-making components. These should include: (1) Retaining complete records of pre-proposal, proposal, and award data, including a record of proposal disposition, in linked electronic files to facilitate aggregate, statistical analysis of the grant-making process, including the calculation of success rates. (2) Collecting demographic, education, and career information from applicants, on a voluntary basis, that is not available to proposal reviewers but is used for analysis of success rates.
Agency Affected: Department of Defense
Comments: DOE generally agreed with this recommendation. According to DOE officials, as of September 2017, of the four components audited at DOE, all four have taken actions toward implementing the recommendation and one component has completed its implementation. Specifically, the Office of Science began collecting investigator demographics during the second quarter of fiscal year 2015 and already retained complete records that enabled the calculation of success rates. Three additional DOE components conducted a joint feasibility study and all concur that it is feasible to collect data on demographic, education and career information of applicants. The Office of Nuclear Energy (NE) revised its approach to data collection and now retains complete grant life cycle information for each individual award, including complete records of pre-proposal, proposal, and award data in linked electronic files. NE is also changing existing data systems to input/track voluntarily submitted demographic information on Principle Investigators on applications to facilitate aggregate, statistical analysis of the grant-making process, including the calculation of success rates. In August 2018, NE reported that they are collecting post award demographic information for significant contributors, but their update does not make clear whether this is also being collected for all grant applicants such that success rates for men and women applicants can be calculated. Both the Office of Energy Efficiency and Renewable Energy (EERE) Advanced Research Projects Agency-Energy (ARPA-E) indicated that they intend to comply with this recommendation. In May 2019, EERE's Deputy Assistant Secretary for operations issued a memorandum requesting that the Assistant Secretary of EERE review and approve a Federal Register Notice (FRN) for "Comments on the Voluntary Collection of Demographic Information from Financial Assistance Applicants in Response to GAO Women in STEM Research Report." EERE reported that they have coordinated this FRN with NE and ARPA-E and that both offices concurred. The Deputy Assistant secretary indicated approving comments on this FRN is the first step in being able to collect these data and recommended that the Assistant Secretary approve this FRN. As of October 2, 2019, the FRN was not published. EERE provided an update on October 30, 2019 that stated that EERE has decided to look to existing OMB approved information collections and Standard Forms to gather the desired demographic data to comply with this recommendation.
Recommendation: In order to ensure complete, analyzable records regarding research grant award decisions are available for management and analysis, the Secretary of Energy should direct DOE's grant-making agencies to implement additional data collection efforts, which should include: (1) Retaining complete records of pre-proposal, proposal, and award data, including a record of proposal disposition, in linked electronic files to facilitate aggregate, statistical analysis of the grant-making process, including the calculation of success rates. (2) Collecting demographic, education, and career information from applicants, on a voluntary basis, that is not available to proposal reviewers but is used for analysis of success rates.
Agency Affected: Department of Energy
Status: Closed - Implemented
Comments: NASA agreed with our recommendation and indicated it will begin collecting basic demographic, education, and career data from its research grant applicants on a voluntary basis by the end of fiscal year 2016. NASA officials reported that the notice of grant award document (form 1687) was modified to require entry of the proposal number on the form in order to capture the linkage between proposal and award. When the transition to the new contract/grant writing system (Procurement for Public Sector) occurred in June 2017, NASA began using the amended award notice. As of June 2018, NASA officials reported that they have implemented the recommendation by collecting basic demographic data, including gender, from research grant applicants on a voluntary basis, utilizing NASA Form 1839. The data collected are not used to determine awards and are not given to reviewers during the proposal review process. This information is being collected via the NASA Solicitation and Proposal Integrated Review and Evaluation System (NSPIRES) and NASA staff are analyzing the collected data.
Recommendation: As NASA begins to collect demographic data on its grant proposals and awards, the NASA Administrator should include the following key components: (1) Retain complete records of pre-proposal, proposal, and award data, including a record of proposal disposition, in linked electronic files to facilitate aggregate, statistical analysis of the grant-making process, including the calculation of success rates. (2) Collect demographic, education, and career information from applicants, on a voluntary basis, that is not available to proposal reviewers but is used for analysis of success rates.
Agency Affected: National Aeronautics and Space Administration
Status: Closed - Implemented
Comments: In February 2016, the department hosted a meeting of the broad interagency STEM Working Group at which it informed agencies of its availability to assist them with Title IX enforcement and presented on Title IX enforcement in the STEM context. In a September 2017 letter, DOJ officials stated that DOJ has taken several actions to increase information sharing about Title IX enforcement between agencies. Since the February 2016 meeting of the STEM Working Group, DOJ has held quarterly meetings with the six federal STEM agencies that were the focus of the GAO report. The meetings have included discussions of the Title IX compliance review process, current compliance activities, recent case law and other developments related to Title IX and STEM. DOJ reports positive feedback from agencies regarding these meetings and they plan to continue holding them on a quarterly basis.
Recommendation: To improve Title IX enforcement by federal STEM grant-making agencies, the Principal Deputy Assistant Attorney General for the DOJ Civil Rights Division should establish a process to facilitate information sharing across federal STEM grant-making agencies regarding current Title IX compliance efforts to promote equitable access to STEM research funds.
Agency Affected: Department of Justice: Civil Rights Division
Comments: DOD agreed with this recommendation and noted it is in the process of revising current DOD guidance which will address its Title IX enforcement requirements. In September 2017, a DOD official stated that the agency is in the process of formulating instructions related to both Title IX and Title VI that they believe will address the recommendation regarding Title IX enforcement. In a memorandum issued in December 2017, a DOD official described the agency's corrective action plan (CAP), including drafting an updated rule for the Code of Federal Regulations and the development and issuance of internal DOD policy documents regarding Title IX enforcement requirements. Both of these activities were expected to be completed by June 2019. DOD reported in March 2020 that it is continuing to revise current DOD guidance to address its Title IX enforcement requirements. The Director of DOD's Office of Diversity, Equity, and Inclusion (ODEI) stated in a letter to GAO that the revised Titile IX policy is in the early stages of DOD's policy process and is on track to be published in fall 2020. ODEI expressed its commitment to developing the revised policy and ensuring Title IX compliance reviews are conducted as per the revised policy and GAO's recommendation.
Recommendation: To comply with Title IX enforcement requirements, the Secretary of the Department of Defense, which funds STEM research at universities, should direct the Director of the Office of Diversity Management and Equal Opportunity to ensure that Title IX compliance reviews of DOD's grantees are periodically conducted.
Agency Affected: Department of Defense
Status: Closed - Implemented
Comments: HHS agreed with this recommendation and indicated it would consult with NIH and initiate a sex discrimination compliance review program that includes grantee institutions with STEM programs. The HHS Office of Civil Rights launched the National Title IX Compliance Review Initiative and, in 2018, conducted Title IX reviews of three top funded NIH research colleges and universities. The reviews evaluated each entity's compliance with Title IX which included examinations of nondiscrimination policies and procedures, anti-harassment and complaint grievance procedures, notification and dissemination requirements and outreach and recruitment efforts. HHS OCR provided technical assistance, as necessary, but did not find evidence of non-compliance with Title IX at the three institutions it reviewed. Officials indicated that, as of March 2019, two reviews are in process and the agency is considering the number and focus of reviews going forward.
Recommendation: To comply with Title IX enforcement requirements, the Secretary of the Department of Health and Human Services, which funds STEM research at universities, should ensure that Title IX compliance reviews of NIH's grantees are periodically conducted.
Agency Affected: Department of Health and Human Services