This is the accessible text file for GAO report number GAO-14-800 entitled 'Health Resources and Services Administration: Action Taken to Train and Oversee Grantee Monitoring Staff, but Certain Guidance Could Be Improved' which was released on September 23, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Requesters: September 2014: Health Resources and Services Administration: Action Taken to Train and Oversee Grantee Monitoring Staff, but Certain Guidance Could Be Improved: GAO-14-800: GAO Highlights: Highlights of GAO-14-800, a report to congressional requesters. Why GAO Did This Study: In fiscal year 2013, HRSA awarded over $6 billion in grants, which accounted for over 80 percent of the agency's appropriations. HRSA's workforce—-its staff and contractors--is responsible for monitoring the agency's grantees to ensure they are financially sound, using federal funds appropriately, and in compliance with program requirements. In recent years, GAO has reported on weaknesses in HRSA's monitoring of certain grant programs and made recommendations for the agency to improve grantee monitoring. GAO was asked to review HRSA's oversight mechanisms for its grantee monitoring workforce. This report examines (1) the extent to which HRSA has developed guidance for staff who monitor grantees; (2) the extent to which it implemented training for this staff; (3) HRSA's oversight of its staff; and (4) its practices to ensure contractors who conduct grantee monitoring activities are qualified and carry out their work as appropriate. GAO reviewed HRSA training documentation and guidance related to grantee monitoring and interviewed HRSA officials. GAO also reviewed the contracts that HRSA identified as including grantee monitoring activities in fiscal year 2013. What GAO Found: In 2012, the Department of Health and Human Services' (HHS) Health Resources and Services Administration (HRSA) began systematically developing guidance for the key staff involved in grantee monitoring—- project officers in its programmatic bureaus and grants management specialists and financial integrity staff in its Office of Federal Assistance Management (OFAM). Specifically, HRSA issued the first agency-wide guidance written primarily for project officers in November 2012 and is in the process of developing new guidance and revising existing guidance for staff in OFAM. Additionally, HRSA required each of the bureaus to develop more detailed guidance for project officers by the end of 2012. To assist the bureaus in systematically developing this guidance, HRSA provided a template outlining the minimum components to be included in the documents. While all of HRSA's bureaus developed guidance for their project officers, GAO found that the bureaus' guidance did not address all of the key components of grantee monitoring identified by HRSA, such as components related to conducting site visits. Although HRSA conducted reviews of the bureaus' guidance, these reviews were not sufficient to ensure that the guidance addressed all required grantee monitoring components. Consequently, there is an increased possibility that project officers will not conduct, or document, all the monitoring activities required by HRSA, which puts the agency at risk for incomplete or insufficient grantee monitoring. Since 2013, HRSA has implemented agency-wide training programs for key staff responsible for conducting monitoring of grantees—-a project officer certification program, which the majority of project officers have completed, and a career development program for grants management specialists. Both programs include courses that cover material related to grantee monitoring. In addition to these training programs, the bureaus and OFAM offer formal and informal training related to grantee monitoring, such as training on specific grant program requirements. HRSA has four main methods for overseeing its staff's monitoring of grantees. According to HRSA officials, supervisors * review information in, and reports from, HRSA's online system for documenting grantee monitoring activities, such as staff's assessments of grantee compliance; * participate in staff interactions with grantees, such as monitoring calls; * regularly communicate with staff in one-on-one and other meetings; and; * conduct annual performance appraisals to hold staff accountable for their monitoring responsibilities. Additionally, HRSA has mechanisms in place to ensure that the contractors that conduct grantee monitoring activities are qualified, including reviewing staffing proposals prior to awarding the contract, requiring contractor staff to have specific qualifications, and requiring training for contractor staff. To oversee contractors' work, HRSA staff regularly communicate with the contractors, review their deliverables, and obtain feedback from project officers and grantees on contractor performance. What GAO Recommends: GAO recommends that HRSA develop a process to ensure that the guidance developed by its bureaus addresses all of the key components of grantee monitoring as established by the agency. HHS agreed with GAO's recommendation and said HRSA has begun to take action to implement it. View [hyperlink, http://www.gao.gov/products/GAO-14-800]. For more information, contact Debra A. Draper at (202) 512-7114 or draperd@gao.gov. [End of section] Contents: Letter: Background: HRSA Has Developed Guidance for Staff on Grantee Monitoring, but Bureau-Level Guidance Did Not Address All of the Key Components of Monitoring: HRSA Implemented Agency-Wide Training Programs for Grantee Monitoring Staff Beginning in 2013; Bureau-Level Training Varied: HRSA Has Four Main Methods for Overseeing Its Staff's Monitoring of Grantees: HRSA Has Several Mechanisms in Place Designed to Ensure Contractors That Monitor Grantees Are Qualified and to Oversee Their Work: Conclusions: Recommendation for Executive Action: Agency Comments: Appendix I: Health Resources and Services Administration Contracts with Grantee Monitoring Activities, Fiscal Year 2013: Appendix II: Comments from the Department of Health and Human Services: Appendix III: GAO Contact and Staff Acknowledgments: Table: Table 1: Examples of Performance Elements for Key Health Resources and Services Administration (HRSA) Staff with Grantee Monitoring Responsibilities: Figure: Figure 1: Extent to Which Bureaus' Standard Operating Procedures (SOP) Addressed Components of Grantee Monitoring Specified by the Health Resources and Services Administration (HRSA): Abbreviations: COR: contracting officer's representative: EHB: Electronic Handbooks: HHS: Department of Health and Human Services: HIV/AIDS: human immunodeficiency virus and acquired immunodeficiency syndrome: HRSA: Health Resources and Services Administration: OFAM: Office of Federal Assistance Management: SOP: standard operating procedures: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: September 23, 2014: Congressional Requesters: The Health Resources and Services Administration (HRSA), an agency within the Department of Health and Human Services (HHS), is charged with improving access to health care services for people who are uninsured, isolated, or medically vulnerable and enhancing the capacity of the health care workforce. To carry out its mission, HRSA provides funding and support for a wide variety of programs, most commonly through grants. In fiscal year 2013, HRSA awarded over $6 billion in grants, which accounted for over 80 percent of the agency's appropriations. Through these grants, HRSA provided financial support to more than 3,000 recipients, including organizations in every state. These grantees serve millions of people each year through programs that provide health care to uninsured individuals; people with human immunodeficiency virus and acquired immunodeficiency syndrome (HIV/ AIDS); pregnant women, mothers, and children; people living in rural communities; and others. To manage its grant programs, HRSA employs project officers in its programmatic bureaus and staff in its Office of Federal Assistance Management (OFAM).[Footnote 1] These staff are supplemented by contractors that perform a variety of tasks to support HRSA's programs and operations.[Footnote 2] HRSA's workforce--its staff and contractors--is responsible for monitoring grantees. Specifically, they are responsible for collecting and assessing information from reports, audits, site visits, and other sources to ensure that grantees are in compliance with the terms of their grants. The effective monitoring of grantees is critical to ensure that they are financially sound, in compliance with programmatic and administrative requirements, and appropriately utilizing federal funds. In addition, monitoring is essential to ensure that grantees receive the support and assistance they need so that funding is used as effectively as possible for the populations HRSA serves. However, in 2012 we reported on weaknesses in HRSA's monitoring of certain grant programs. Specifically, we reported that HRSA did not consistently follow HHS regulations and guidance in its monitoring of grantees under the Ryan White Comprehensive AIDS Resources Emergency Act of 1990, as amended. [Footnote 3] We also reported that HRSA's monitoring of its Health Center Program grantees was insufficient to ensure that the agency consistently identified all instances of grantee noncompliance with program requirements.[Footnote 4] We recommended actions to improve HRSA's monitoring of these grantees. HHS concurred with our recommendations, some of which have already been implemented by HRSA. You asked us to review HRSA's management and oversight mechanisms, including the systems HRSA has in place to train and oversee its workforce responsible for monitoring grantees. In this report, we examine (1) the extent to which HRSA has developed guidance for its staff responsible for monitoring grantees; (2) the extent to which HRSA has implemented training for its staff responsible for monitoring grantees; (3) HRSA's oversight of its staff responsible for monitoring grantees; and (4) what practices, if any, HRSA has in place to ensure that the contractors conducting grantee monitoring activities are qualified and carry out their work as appropriate. To examine the extent to which HRSA has developed guidance for its staff responsible for monitoring grantees, we reviewed guidance that HRSA developed related to monitoring grantees, including standard operating procedures (SOP) developed by HRSA's programmatic bureaus and OFAM.[Footnote 5] We assessed HRSA's guidance on grantee monitoring against federal internal control standards related to monitoring,[Footnote 6] as well as standards that HRSA established for the guidance developed by its bureaus. In addition, we conducted interviews with HRSA officials, including representatives from HRSA's SOP workgroup--assembled by HRSA to facilitate the bureaus' development of guidance on monitoring grantees--and leadership of OFAM and each of the bureaus about guidance for staff on grantee monitoring at both the agency and bureau level. We also conducted interviews with groups of four to six project officers and their supervisors from each of four selected bureaus--the Bureau of Health Professions,[Footnote 7] the Bureau of Primary Health Care, the HIV/AIDS Bureau, and the Maternal and Child Health Bureau--in order to gain their perspectives and experiences with HRSA's guidance on grantee monitoring.[Footnote 8] The four selected bureaus had the largest number of project officers and active grants and the greatest amount of grant awards in fiscal year 2013.[Footnote 9] We selected project officers that varied in their level of HRSA experience (including new project officers hired after June 1, 2013) and the number of grants that they monitored. To examine the extent to which HRSA has implemented training for its staff responsible for monitoring grantees, we reviewed documentation of HRSA's training, including training materials and announcements, and HRSA's mechanisms for tracking and evaluating training. We assessed HRSA's training practices against leading federal practices for workforce training.[Footnote 10] In addition, we interviewed HRSA officials, including representatives from the HRSA Learning Institute- -the training center HRSA established in 2010--and leadership from OFAM and each of the bureaus. We also used our small group interviews with project officers and their supervisors to gain their perspectives and experience with the training HRSA has implemented related to grantee monitoring. To examine HRSA's oversight of its staff responsible for monitoring grantees, we reviewed HRSA's written policies and documentation related to oversight of staff, including sample performance appraisals and performance elements for staff with grantee monitoring responsibilities. We discussed methods used to oversee staff responsible for grantee monitoring during our interviews with HRSA, OFAM, and bureau leadership, as well as during our project officer and supervisor small group interviews. To examine what practices, if any, HRSA has in place to ensure that the contractors conducting grantee monitoring activities are qualified and carry out their work as appropriate, we reviewed and analyzed the 13 contracts HRSA identified as including grantee monitoring activities, such as grantee site visits and financial reviews, in fiscal year 2013.[Footnote 11] See appendix I for more information on each of the contracts we reviewed. We also discussed oversight of contractors during our interviews with HRSA and OFAM leadership, leaders from the Bureau of Primary Health Care and the HIV/AIDS Bureau--the two bureaus that used contractors to conduct grantee monitoring activities in fiscal year 2013--as well as our small group interviews with project officers and supervisors from these two bureaus. In addition, we interviewed representatives from the contractors with the two largest contracts (based on dollar amount of obligations), as well as the HRSA officials responsible for oversight of these contractors.[Footnote 12] We conducted this performance audit from July 2013 to September 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: HRSA provides grants to a variety of entities using the HHS grant management process, which consists of seven phases. The first five phases of the grant management process cover the planning of the grant funding opportunity through the awarding of the grant funds.[Footnote 13] The sixth phase of the process, which occurs postaward, is the monitoring phase during which HRSA is to review grantees to identify potential problems and areas of noncompliance with programmatic and administrative requirements that must be addressed by the grantee. This monitoring continues for as long as HRSA retains a financial interest in the grantee's activities and is followed by the seventh and final phase--closeout.[Footnote 14] Within HRSA, the grants management process, including grantee monitoring, is a coordinated effort of project officers located in the programmatic bureaus, and grants management specialists and financial integrity staff in OFAM. * Project officers. The nearly 400 project officers within the bureaus have primary responsibility for the day-to-day monitoring of an assigned portfolio of grantees.[Footnote 15] Project officer monitoring activities include reviewing grantee reports on various topics to ensure performance requirements are met and to identify performance or financial problems; communicating with grantees regularly and documenting that communication; consulting with grantees on programmatic issues; and conducting and arranging for site visits. For example, project officers conduct the initial review of requests from grantees for prior approval of certain actions, such as carrying over unobligated funds to subsequent funding periods; review grantee progress and financial reports; and are generally expected to have at least quarterly contact with their grantees. They are expected to keep their supervisors and relevant OFAM staff informed regarding grantee performance and the need for any intervention. * Grants management specialists. Located in OFAM's Division of Grants Management Operations, grants management specialists are responsible for managing business and financial issues for a portfolio of grantees. For example, they review the Federal Financial Report, a required report to document the financial progress of a grant award, which includes information on the funds a grantee has received and how they were spent. Grants management specialists and project officers are expected to coordinate in monitoring grantees; for example, grants management specialists are responsible for approving or disapproving prior approval requests after the project officer's initial review. As of June 2014, there were nearly 70 grants management specialists within OFAM. * Financial integrity staff. Located in OFAM's Division of Financial Integrity, financial integrity staff are responsible for the financial oversight and monitoring of grantees, which includes conducting or coordinating financial audits and reviews of grantees. For example, some financial integrity staff conduct program integrity analyses to ensure that financially at-risk grantees are identified for targeted intervention; others work with grantees to ensure the resolution of financial audit findings. As of June 2014, there were 34 financial integrity staff with grantee monitoring responsibilities. HRSA supplements its staff with contractors that perform grantee monitoring activities such as conducting site visits to monitor compliance with grant requirements, or financial audits and reviews. Each contractor is overseen by a HRSA contracting officer's representative (COR) who is generally located within the bureau or office that requested the grantee monitoring support, but works on behalf of the contracting officer. The COR is responsible for overseeing certain aspects of the contractor's performance, including reviewing required reports and assessing timeliness of deliverables. HRSA Has Developed Guidance for Staff on Grantee Monitoring, but Bureau-Level Guidance Did Not Address All of the Key Components of Monitoring: HRSA began systematically developing guidance for the key staff responsible for grantee monitoring in 2012. However, we found that guidance developed by the bureaus for project officers did not address all of the key components of grantee monitoring established by HRSA. HRSA Began Systematically Developing Guidance in 2012 for the Key Staff Responsible for Grantee Monitoring: In 2012, HRSA began systematically developing guidance for project officers and staff in OFAM responsible for monitoring grantees. Specifically, HRSA issued the first agency-wide guidance written primarily for project officers in November 2012, and it required each of the bureaus to develop more detailed guidance for project officers by the end of 2012. In addition, HRSA is in the process of developing new guidance and revising existing guidance for grants management specialists and financial integrity staff in OFAM. Collectively, this guidance addresses the 2007 HHS Awarding Agency Grants Administration Manual requirement for each agency to develop monitoring plans. [Footnote 16] Agency-wide guidance for project officers. HRSA issued its Grantee Oversight Manual in November 2012--the first agency-wide guidance developed primarily for project officers. This manual, updated in 2014, is intended to outline a common set of minimum requirements for monitoring grantees. It generally describes the roles and responsibilities of project officers and other staff responsible for conducting monitoring of grantees and builds on guidance that HRSA and HHS had previously developed. Prior to the issuance of the HRSA Grantee Oversight Manual, HRSA's main source of guidance on monitoring of grantees was OFAM's 2009 Division of Grants Management Operations Operational and Procedures Manual; however, this manual was primarily written for grants management specialists in OFAM. Additionally, HRSA has periodically issued Grants Policy Operations Memoranda to provide guidance to staff on various topics, some of which are relevant to project officers' monitoring of grantees, such as carrying forward unobligated funds to subsequent funding periods. Project officers also relied on the overarching HHS policies for guidance on monitoring of grantees, including the Awarding Agency Grants Administration Manual. Bureau-specific guidance for project officers. In addition to developing the agency-wide guidance, HRSA required each of its bureaus to develop monitoring plans and document them in the form of SOPs. HRSA specified that these SOPs should supplement the Grantee Oversight Manual by including more detailed, program-specific monitoring requirements and be developed by the end of 2012. To assist the bureaus in systematically developing their SOPs, HRSA provided the bureaus with a template outlining the minimum components to be included in their documents and established a workgroup to facilitate implementation of the SOPs across the agency. At the time of our review, all of the bureaus had developed SOPs. Officials from four of the bureaus told us that, prior to the agency's 2012 requirement to develop SOPs, they did not have any in place. In addition to SOPs, HRSA officials described other sources of guidance developed by the bureaus, such as site visit guides, policy notices, frequently asked questions, checklists, and templates. [Footnote 17] Project officers and supervisors we interviewed generally found the guidance on grantee monitoring to be helpful, and one project officer who had been with HRSA for over 10 years stated that the level of guidance provided to project officers has significantly improved in recent years. However, a few project officers and supervisors we interviewed identified areas where they could use additional guidance, such as guidance on specific requirements for certain programs and additional guidance on the information that should be entered into HRSA's Electronic Handbooks (EHB)--the agency's online system for documenting grantee monitoring activities. Guidance for OFAM staff. OFAM has also developed new guidance for grants management specialists and financial integrity staff. The agency is in the process of replacing the 2009 Division of Grants Management Operations Operational and Procedures Manual for grants management specialists with a series of SOPs. In addition to making necessary updates, HRSA officials said that this approach would allow them greater flexibility in revising and updating the written guidance in the future. As of July 2014, OFAM had developed 4 SOPs for grants management specialists related to grantee monitoring--such as "Reviewing Financial Reports"--and indicated plans to develop approximately 10 more. HRSA officials said that staff can still rely on the guidance provided by the Operational and Procedures Manual during the transition to SOPs. In addition to the guidance for grants management specialists, OFAM's Division of Financial Integrity has 3 SOPs related to grantee monitoring for financial integrity staff-- "Audit Resolution," "Special Reviews," and "EHB Financial Assessments"--all of which were developed or revised since 2013. Bureau-Level Guidance Did Not Address All of the Key Components of Grantee Monitoring: While HRSA's bureaus have developed bureau-level guidance for project officers on monitoring of grantees, we found that the bureaus' SOPs did not address all of the key components of grantee monitoring identified by HRSA. Federal internal control standards state that management is responsible for developing detailed policies to fit their agency's operations.[Footnote 18] HRSA specified the detail that bureaus should include in their SOPs by providing a template outlining the key components of monitoring that should, at a minimum, be included. The key components of monitoring identified in the template fall into three main categories--communication, reviewing grantee reports, and conducting site visits. Across the 13 SOPs that were developed by the bureaus as of April 2014, we found that the SOPs most thoroughly addressed the components related to communication and least thoroughly addressed those related to conducting site visits.[Footnote 19] Specifically, 7 of the 13 SOPs fully addressed the six key components related to communication; each of the remaining 6 SOPs did not fully address one communication component.[Footnote 20] The communication components that were not fully addressed by the SOPs were specifying the frequency and purpose of communication with grantees (3 SOPs), the process for communicating concerns or noncompliance (1 SOP), and coordinating with OFAM (2 SOPs). For reviewing grantee reports, 5 of the 13 SOPs fully addressed the five key components.[Footnote 21] Of the 8 remaining SOPs, the majority did not fully address at least two components and only one of these SOPs addressed the component on documentation of grantee compliance.[Footnote 22] Of the 10 SOPs for which the site visit components were applicable, none fully addressed all nine key site visit components.[Footnote 23] More than half of the SOPs did not fully address at least four of the nine site visit components, and 1 SOP did not address any of them. Additionally, one component in particular--requirements for tracking site visits--was not addressed by any of the SOPs.[Footnote 24] (See fig. 1 for information about the extent to which the SOPs addressed each of the key components of grantee monitoring by category.): Figure 1: Extent to Which Bureaus' Standard Operating Procedures (SOP) Addressed Components of Grantee Monitoring Specified by the Health Resources and Services Administration (HRSA): [Refer to PDF for image: stacked horizontal bar graph] Number of SOPs: Communication (n=13): Frequency and purpose of communication with grantees: Fully addressed: 10; Partially addressed: 2; Not Addressed: 1. Mode or type of communication with grantees: Fully addressed: 13; Partially addressed: 0; Not Addressed: 0. Process for communicating concerns or noncompliance: Fully addressed: 12; Partially addressed: 0; Not Addressed: 1. Documentation of grantee communication: Fully addressed: 13; Partially addressed: 0; Not Addressed: 0. Coordinating with the Office of Federal Assistance Management: Fully addressed: 11; Partially addressed: 0; Not Addressed: 2. Communication about grantee status and issues with supervisors: Fully addressed: 13; Partially addressed: 0; Not Addressed: 0. Reports From Grantees (n=13): Criteria for reviewing grantee requests requiring prior approval[A]: Fully addressed: 9; Partially addressed: 0; Not Addressed: 3. Requirements for reviewing grantee reports: Fully addressed: 13; Partially addressed: 0; Not Addressed: 0. Documentation of grantee compliance: Fully addressed: 6; Partially addressed: 0; Not Addressed: 7. Process for addressing concerns about grantee noncompliance: Fully addressed: 8; Partially addressed: 2; Not Addressed: 3. Documentation of monitoring activities: Fully addressed: 10; Partially addressed: 2; Not Addressed: 1. Site Visits (n=10)[B]: Defining site visit types: Fully addressed: 6; Partially addressed: 0; Not Addressed: 4. Frequency of site visits: Fully addressed: 6; Partially addressed: 2; Not Addressed: 2. Process for approving site visits: Fully addressed: 5; Partially addressed: 2; Not Addressed: 3. Process for notifying grantees about site visits: Fully addressed: 6; Partially addressed: 1; Not Addressed: 3. Required activities before, during, and after the visit: Fully addressed: 5; Partially addressed: 3; Not Addressed: 2. Process and time frames for developing site visit report: Fully addressed: 5; Partially addressed: 3; Not Addressed: 2. Requirements for tracking site visits: Fully addressed: 0; Partially addressed: 0; Not Addressed: 10. Maintenance of site visit reports and follow-up: Fully addressed: 9; Partially addressed: 0; Not Addressed: 1. Documentation of a site visit protocol: Fully addressed: 6; Partially addressed: 0; Not Addressed: 4. Source: GAO analysis of HRSA bureaus' SOPs developed as of April 2014. GAO-14-800. Notes: We considered a component fully addressed if a given SOP included all of the information specified by HRSA for that component in its template for developing SOPs; partially addressed if a given SOP included some, but not all, of the information; and not addressed if the SOP did not include the information specified in the template. [A] The grantee requests for prior approvals component was only applicable for 12 of the 13 SOPs because 1 SOP was specific to a program for which HRSA has not provided funding to grantees since 1997. Therefore, there are no grantee requests that would require prior approval. [B] The site visit components were only applicable for 10 of the 13 SOPs because 3 SOPs were specific to programs that do not include site visits in their monitoring approach. [End of figure] Members of HRSA's SOP workgroup told us that they conducted an initial, high-level review of the bureaus' SOPs to assess their consistency with the template, but the review was not formal or in- depth. Additionally, leaders from each of the bureaus reported having processes for internally developing and reviewing the SOPs, such as establishing internal workgroups, soliciting participation and feedback from project officers, reviewing the HRSA template, and conducting reviews at the bureau-leadership level. However, neither the SOP workgroup's review nor those done by the bureaus ensured that the SOPs addressed all required grantee monitoring components. Because the SOPs do not include the level of detail specified by HRSA in the template, there is an increased possibility that project officers will not conduct, or appropriately document, all of the required monitoring activities, which puts HRSA at greater risk for having incomplete or ineffective grantee monitoring. HRSA Implemented Agency-Wide Training Programs for Grantee Monitoring Staff Beginning in 2013; Bureau-Level Training Varied: Since 2013, HRSA has implemented agency-wide training programs for key staff responsible for monitoring grantees, including a project officer certification program and a training program for grants management specialists.[Footnote 25] HRSA's project officer certification program, launched in May 2013, includes a standard set of seven courses that were to be completed by all experienced project officers by April 30, 2014.[Footnote 26] Under the program, new project officers hired after June 1, 2013, are to complete 40 hours of self- study and in-person training within 6 months of being hired. HRSA reported that 86 percent of experienced project officers completed the certification program by the April 2014 deadline and that 90 percent of new project officers completed the program by June 2014.[Footnote 27] After the initial certification, all project officers are required to meet annual continuing education requirements. Both the new and experienced project officer training includes courses that cover material related to grantee monitoring. For example, these courses cover the goals of grantee monitoring, project officer responsibilities, sources of information, and helpful tips for effective monitoring of grantees. Project officers, supervisors, and bureau-leadership officials we interviewed said they were generally satisfied with the certification program, and that the level of training it provides is sufficient and helpful. HRSA also initiated a new training program for grants management specialists in 2014. HRSA officials described this as a career development program with tracks at the entry, mid-career, or advanced- career level; staff work with their supervisors to determine which track is most appropriate. Officials said that each level offers grants management courses with a focus on monitoring. For example, there are courses that cover grants management specialist responsibilities, coordinating with project officers, and various types of reports that grants management specialists are responsible for reviewing.[Footnote 28] According to HRSA officials, these training programs were developed based on the competencies for project officers and grants management specialists, respectively.[Footnote 29] Additionally, HRSA officials told us that they track individuals' completion of the training and plan to evaluate its effectiveness, which is consistent with federal leading practices for workforce training.[Footnote 30] For example, HRSA officials told us that they run monthly reports for tracking project officer completion of the certification program and they provide these reports to the bureaus. Officials said that they also are tracking grants management specialists' completion of training and working closely with OFAM to encourage staff participation and provide them with resources and information related to the training. HRSA plans to use a variety of mechanisms to evaluate the effectiveness of the training programs and assess outstanding training needs. For example, staff have the opportunity to complete course evaluations and respond to periodic surveys on their training needs. HRSA also conducts annual reviews of all of the courses they offer, including the courses in the project officer and grants management specialist training programs, to evaluate their effectiveness and make any necessary changes. Officials explained that these reviews consist of a team of experts reviewing multiple sources of information, including staff evaluations of the courses and staff surveys on the extent to which they have applied what they learned. HRSA officials said that they had completed course evaluations for the project officer and grants management specialist training programs and would be using the results to plan training for 2015. In addition for the project officer certification program, HRSA plans to hold sessions in September 2014 for project officers to provide feedback and suggestions for improving the program. Officials noted that moving forward, the focus for project officer training will be on continuing education as all existing project officers should have a foundation of knowledge from completing the initial certification. In addition to agency-wide training programs, the bureaus provided their own training--such as training on specific grant programs--to varying degrees. Three bureaus--the Bureau of Health Professions, the Bureau of Primary Health Care, and the HIV/AIDs Bureau--provided some formal training on grantee monitoring in 2013. These bureaus, which represented three of the four largest bureaus at HRSA in 2013, offered trainings ranging from a single course on grantee performance to a series of courses on specific grant program requirements and monitoring activities, such as financial monitoring and site visits. Officials from the HIV/AIDs Bureau said they had mechanisms in place for formally tracking and evaluating the training they offered. Leadership from three of the four bureaus that did not provide formal training related to grantee monitoring explained that their bureaus are relatively small. Thus, the officials indicated that their project officers benefit from frequent communication with their supervisors in lieu of formal training.[Footnote 31] Leadership from the remaining bureau that did not provide formal training said that because the bureau has many different programs with different requirements, it would not make sense to offer formal bureau-wide training. Officials from all of the bureaus described various informal training mechanisms in place that help provide project officers with the knowledge and skills to conduct their work, including one-on-one meetings between project officers and their supervisors, mentoring programs, and having new project officers shadow more experienced project officers on site visits prior to conducting them. For example, the Bureau of Health Professions developed a tool for new project officers that specifies timelines for completing certain tasks that will prepare them for their role as project officers within the bureau, such as reviewing key bureau and HRSA guidance, shadowing a senior project officer and identifying a mentor, and reviewing bureau resources. In addition, supervisors we spoke with generally agreed that it is their responsibility to provide on-the-job training to project officers, particularly related to program-specific grant requirements. While project officers and supervisors we interviewed generally felt that the training provided by HRSA and the bureaus related to grantee monitoring was sufficient, a few identified areas where project officers could use more training; for example, training on financial monitoring, effectively conducting monitoring calls with a large number of grantees, and documenting monitoring activities in EHB. A few more experienced project officers told us that training on grantee monitoring has improved significantly in recent years. Similar to the bureaus, OFAM officials described additional training for their staff related to monitoring grantees. For example, OFAM officials told us they have offered training related to the new SOPs for grants management specialists, such as sessions on federal financial reviews, and they plan to provide training on each of the additional SOPs as they are developed and rolled out to staff. Officials said that the expectation is that all staff participate in these trainings and that they are tracking staff completion. OFAM officials said they also provide training to financial integrity staff on changing requirements and relevant topics, such as consistent treatment of audit findings, on an as-needed basis. HRSA Has Four Main Methods for Overseeing Its Staff's Monitoring of Grantees: HRSA has four main methods for overseeing project officers, grants management specialists, and financial integrity staff that are responsible for monitoring its grantees--supervisors reviewing information in, and reports from, EHB; participating in staff interactions with grantees; communicating regularly with staff; and conducting performance appraisals. Reviewing information in, and reports from, EHB. HRSA officials and supervisors we interviewed identified reviewing information in, and reports from, EHB as one of the main methods for overseeing staff responsible for monitoring grantees; EHB is the agency's online system for documenting grantee monitoring activities, as well as corresponding and exchanging documents with grantees. There are three main ways that supervisors use EHB for oversight purposes. * Supervisors review certain documents or decisions made by staff about grantees for quality, accuracy, and completeness; the EHB system requires supervisory review and approval to occur before certain grant- related actions can be taken. Examples of items requiring supervisory review and approval in EHB include project officers' assessments of grantee compliance, noncompliant grantees' plans and time frames to address their noncompliance, and grantee prior approval requests, as well as certain financial assessments that are completed by financial integrity staff. Additionally, officials told us that there are multiple levels of required supervisory review and approval in EHB for certain monetary actions made by grants management specialists. In reviewing staff's work, supervisors told us that they consider the documentation used to support staff's recommendations. For example, when reviewing a project officer's assessments of grantee compliance, a supervisor may review the reports submitted by the grantee, which also are to be documented in EHB. In doing so, the supervisor is able to determine if the EHB record includes the necessary documentation and if that documentation supports the project officer's assessment, and then provide feedback to the project officer. * Supervisors also spot-check staff's day-to-day activities in EHB to oversee their work. For example, at any time a project officer's supervisor can access a grantee's EHB record to assess whether a project officer has communicated with the grantee at the required intervals and documented that communication. By reviewing the documentation of the communications, the supervisor can determine the information the project officer and grantee discussed. Supervisors can also see whether site visit reports have been entered or uploaded into EHB within the established time frames, and can review the audit reports completed by financial integrity staff, including reviewing the supporting materials from EHB. * Supervisors also use EHB-generated reports to oversee staff. According to the HRSA officials we spoke with, supervisors generate EHB reports to review information, such as the number and types of tasks in a staff member's queue, the timeliness of task completion, and whether anything is overdue.[Footnote 32] Participating in interactions with grantees. Supervisors provide oversight of staff by participating in some of their interactions with grantees. According to HRSA officials, supervisors may participate in monitoring calls between project officers and grantees. Supervisors may also review project officers' email or other written correspondence in advance of it being sent to grantees, or may be copied on email exchanges between project officers and grantees, so that they are cognizant of grantee interactions. In addition, supervisors may participate in grantee site visits with project officers. According to officials, participating in these interactions allows supervisors to observe how the staff interact with the grantees, how they handle any problems that come up, and the customer service staff provide. If they identify any problems or concerns with these interactions, supervisors told us they discuss it with their staff. Communicating regularly with staff. Supervisors use regular communications with staff as another oversight method. For example, according to HRSA officials, supervisors hold individual, one-on-one, meetings with members of their staff on a regular basis, ranging from weekly to quarterly. Two officials said that the frequency of one-on- one meetings varies based on the staff's experience. Topics discussed during these meetings include updates on the grantees that the staff is responsible for monitoring, any grantee compliance issues, upcoming or recent site visits, and submissions from grantees. In preparation for these meetings, officials we interviewed reported that supervisors often review the staff's EHB documentation, and then discuss it during their meetings. Officials from one bureau stated that supervisors use EHB information to guide conversations with their staff during their one-on-one meetings to discuss grantee progress, concerns, and best practices. In addition to one-on-one meetings, some supervisors hold regular group meetings with all of their staff responsible for grantee monitoring, where, among other things, staff can report on any issues they are encountering with their grantees or ask questions of their peers or supervisors. Conducting performance appraisals. HRSA also uses its annual performance appraisal process to hold its staff accountable for their grantee monitoring responsibilities. The performance appraisals for all staff with responsibility for monitoring grantees include performance elements related to monitoring. The specific elements upon which staff are assessed are based on the roles and responsibilities for their job function, and can vary by individual. Table 1 provides examples of performance elements related to monitoring for the three key positions with responsibility for grantee monitoring. In addition, supervisors and bureau and office leadership also are held accountable through their performance appraisals for their oversight of the staff who perform grantee monitoring activities. For example, HRSA officials told us bureau and office leadership are held accountable for overseeing the monitoring of grantees through the business acumen element of their appraisal, under which they are expected to ensure program integrity and responsible stewardship of resources. Table 1: Examples of Performance Elements for Key Health Resources and Services Administration (HRSA) Staff with Grantee Monitoring Responsibilities: Job function: Project officer; Examples of performance elements related to monitoring grantees: * Monitors, reviews, and tracks the performance of grantees against baseline and established targets to achieve the overall bureau goals; * Contacts grantees on a quarterly basis to support program compliance; * Conducts and properly documents site visits, including taking follow- up actions as needed; * Anticipates, plans for, and takes appropriate, timely actions to correct, enforce, and/or mitigate adverse grantee outcomes; * Ensures and properly documents that programmatic objectives have been met and that any required programmatic report is received and accepted in accordance with applicable policies and procedures. Job function: Grants management specialist; Examples of performance elements related to monitoring grantees: * Monitors grantee submissions, such as required reports, for timeliness and compliance with grants policy and follows up on late actions; * Processes grantee requests requiring prior approvals in a timely manner, no later than 30 days after receipt; * Records all correspondence and pertinent documents in official files and communicates with relevant HRSA staff as necessary. Job function: Financial integrity staff; Examples of performance elements related to monitoring grantees: * Performs and completes financial assessments [of grantees] using established review procedures and within established timeframes. Ensures financial assessments conform to standards. These standards include making sure financial assessments are accurate, complete, self- explanatory, and grammatically correct; * Closes 90 percent of the assigned [grantee] audit reports [of normal complexity] within 6 months of issuance; * Prepares draft and final reports that completely fulfill the objective of the review [of grantee issues] and are thoroughly researched and accurate, including any findings and recommendations if appropriate. Source: HRSA. GAO-14-800. Note: The specific performance elements an individual is rated on can vary based on the individual's assignments and the requirements of the grant programs for which they conduct monitoring. For example, the specific elements for financial integrity staff vary depending on whether they are responsible for financial assessments of grantees, special grantee reviews, or working with grantees to resolve audit findings. [End of table] HRSA Has Several Mechanisms in Place Designed to Ensure Contractors That Monitor Grantees Are Qualified and to Oversee Their Work: HRSA has several mechanisms in place designed to ensure that HRSA contractors that conduct grantee monitoring activities are qualified and to oversee the work performed by these contractors. HRSA officials told us that the agency's process for awarding contracts is designed to ensure the contractors that conduct grantee monitoring activities are qualified. As part of the process for awarding such contracts, HRSA considers staffing proposals and the expectations outlined in the statement of work--a section of the contract solicitation that describes the specific tasks required of the contractor. For all 13 contracts that HRSA identified as including grantee monitoring activities in fiscal year 2013, the personnel proposed by contractors were a factor in HRSA's evaluation and selection of the contractors. For each of these contracts, HRSA required prospective contractors to submit staff résumés and, for some contracts, also required letters of commitment or other information regarding the expertise and qualifications for certain staff designated as "key personnel"-- contractor staff who are proposed to lead the work, such as the project director. Officials told us they considered the experience of these individuals when evaluating the potential contractors. For some contracts, HRSA identified specific qualifications required for contractor staff. Nine of the 13 contracts--4 used by OFAM for financial reviews and 5 used by programmatic bureaus for grantee monitoring--included specific requirements for the qualifications or characteristics of contractor staff. For example, the OFAM contracts included information on educational requirements, years of experience, and skill sets needed for certain positions including senior financial analysts and senior auditors. One of the bureau contracts required contractor site visit staff to have specific clinical experience, as well as experience with administrative and management information systems, fiscal management, and familiarity with the support services available for the population served by the grant. Additionally, almost all of the contracts (12 of 13) included a clause that specifies procedures for the contractor to follow if it makes a change to key personnel. For example, some contracts stated that if the contractor needed to change any key personnel, it would need to notify HRSA in advance in writing and provide justification for the change. The contractor would then need to receive written consent from HRSA to make the change in personnel. After a contractor is selected, HRSA has other mechanisms designed to ensure that the contractor staff conducting grantee site visits are qualified to conduct this monitoring activity. For example, some of the contracts used by the HIV/AIDS Bureau and the Bureau of Primary Health Care stated that the contractor is responsible for providing training to its staff. One contract specified that contractor staff must achieve a particular score on a post-training assessment before they are considered eligible to perform site visits. HRSA officials and the contractors we spoke to explained that development of this training is a joint effort; that is, HRSA and the contractor work together to develop the training and ensure it covers the necessary information. Contractor staff may also attend HRSA webinars or other trainings offered to grantees that discuss grant program requirements. In addition to training, HRSA has resources available for contractor staff that conduct site visits. For example, contractor staff have access to site visit guides, which help to ensure that the staff are knowledgeable about the requirements of the grant programs for which they are conducting site visits. Additionally, HRSA officials and contractors we interviewed told us that new contractor staff may shadow more experienced contractor staff during site visits, allowing them to observe an actual site visit prior to being responsible for conducting one on their own. HRSA officials also stated that when possible, HRSA project officers attend the site visits to help new contractor staff understand what to look for when assessing a grantee's compliance with program requirements. HRSA officials and one of the contractors we interviewed told us that HRSA staff also provide input to contractors about the staff available or selected for grantee monitoring activities, such as site visits; however, the contractor is ultimately responsible for selecting its staff and assigning them to particular assignments. Specifically, officials said that when there is a specific need for a particular type of expertise among the contractor staff conducting grantee monitoring activities, HRSA officials may recommend to the contractor that certain individuals be added to the roster of contractor staff available to provide services or assigned to conduct a particular site visit. HRSA officials reported several mechanisms they use to oversee contractors' work, including regular communications with contractors, the review of contractors' deliverables, and the collection of feedback from project officers and grantees on contractor performance. * The contracts we reviewed all required regular communications between HRSA and contractors for the purpose of, among other things, discussing the status of the contractor's work or any problems the contractor encountered with the grantees. For example, several contracts required contractors to provide monthly progress reports to HRSA staff on the status of their accomplishments for the month and any problems encountered. CORs and contractors we spoke with stated they have standing meetings ranging in frequency from weekly to monthly and have more frequent informal communication as needed to carry out the day-to-day activities of the contract. Additionally, the contractor we spoke with that conducts site visits of Bureau of Primary Health Care grantees told us that it has quarterly meetings with representatives from the bureau, which include discussion of the site visit process and any grantee trends staff are seeing during the site visits. * CORs and other HRSA officials review contractor deliverables, such as routine progress reports and site visit reports, to assess timeliness and quality of the products. The CORs we interviewed said they regularly rely on project officers, who have the programmatic expertise, to review and provide feedback on site visit reports submitted by contractor staff. Contractor staff are then responsible for making any necessary revisions to the reports. Officials also said that CORs work with financial integrity staff when reviewing certain contractor deliverables, such as financial ratio analyses to assess grantees' financial health and viability, as they have expertise related to these financial reviews. * For at least some contracts, contractors collect feedback from grantees and project officers that interact with the contractor staff. For these contracts, grantees have the opportunity to provide feedback on contractor staff site visit performance by completing a post-visit survey. Contractors for the HIV/AIDS Bureau and the Bureau of Primary Heath Care told us that they administer such surveys to grantees. Additionally, the Bureau of Primary Heath Care contractor we spoke with sends project officers a similar post-visit survey about their experiences with contractor staff, while the HIV/AIDS Bureau contractor we spoke with solicits contractor staff feedback from debriefing calls with the project officer after the site visit is completed. Conclusions: HRSA uses a large portion of its appropriations--over 80 percent in fiscal year 2013--to award grants to external entities. As a grant- making agency, HRSA is responsible for monitoring its grantees to ensure that they are meeting the requirements of their grants. Effectively monitoring grantees is essential to ensure that government funds are being used appropriately and efficiently. In the past few years, HRSA has taken steps to increase and improve its guidance and training related to monitoring grantees, including implementing new training programs for key staff responsible for grantee monitoring and developing SOPs for these staff at both the agency and bureau level. These efforts reflect HRSA's recent emphasis on systematically providing guidance and training on grantee monitoring. To assist with the development of the SOPs, HRSA provided its bureaus with a template outlining the key components of grantee monitoring that should be addressed in the guidance. However, we found that the bureau-level SOPs did not address all of the key components of grantee monitoring specified in that template; the lack of detailed policies is inconsistent with federal standards for internal control. Because HRSA did not ensure that the bureaus' SOPs included the required information and detail, and thus provided comprehensive guidance to staff responsible for monitoring grantees, the agency has put itself at risk for incomplete or ineffective grantee monitoring, which could jeopardize its stewardship of federal dollars. Recommendation for Executive Action: To help ensure complete and effective monitoring of grantees, we recommend that the Administrator of HRSA develop a process, either at the agency or bureau level, to ensure that bureaus' SOPs address all of the key components of grantee monitoring as established by the agency. Agency Comments: We provided a draft of this report to HHS for comment. In its written comments, reproduced in appendix II, HHS concurred with our recommendation. In addition, HHS provided information on its plans for implementing the recommendation, stating that HRSA officials had already begun to take action. We are sending copies of this report to the Secretary of Health and Human Services and the Administrator of HRSA. In addition, the report will be available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staffs have any questions about this report, please contact me at (202) 512-7114 or at draperd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Signed by: Debra A. Draper: Director, Health Care: List of Requesters: The Honorable Lamar Alexander: Ranking Member: Committee on Health, Education, Labor, and Pensions: United States Senate: The Honorable Tom Coburn: Ranking Member: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable Michael B. Enzi: Ranking Member: Subcommittee on Children and Families: Committee on Health, Education, Labor, and Pensions: United States Senate: The Honorable Richard Burr: Ranking Member: Subcommittee on Primary Health and Aging: Committee on Health, Education, Labor, and Pensions: United States Senate: [End of section] Appendix I: Health Resources and Services Administration Contracts with Grantee Monitoring Activities, Fiscal Year 2013: Contractor: Brown & Company, CPAs; Contract purpose: To assist HRSA's Division of Financial Integrity in conducting financial ratio analyses to assess grantees' ability to manage federal funds in accordance with federal grants management rules and regulations; Health Resources and Services Administration (HRSA) bureau or office: Office of Federal Assistance Management; Amount of obligations: FY 2012: $30,060; FY 2013: $0. Contractor: Brown & Company, CPAs; Contract purpose: To assist HRSA's Division of Financial Integrity in conducting financial assessments to assess grantees' ability to manage federal funds in accordance with federal grants management rules and regulations; Health Resources and Services Administration (HRSA) bureau or office: Office of Federal Assistance Management; Amount of obligations: FY 2012: $12,780; FY 2013: $0. Contractor: Brown & Company, CPAs; Contract purpose: To assist HRSA's Division of Financial Integrity with conducting follow-up on grantees that are delinquent in the completion of their financial audit; Health Resources and Services Administration (HRSA) bureau or office: Office of Federal Assistance Management; Amount of obligations: FY 2012: $11,160; FY 2013: $0. Contractor: Claxton and Company, P.C. Contract purpose: To conduct a limited scope review of a Bureau of Primary Health Care grantee to determine if the grantee has financial systems adequate to support the grant awards received, has the financial capability to perform grant activities, and has adequate policies and procedures in place to meet the program guidelines and terms and conditions of the grant; Health Resources and Services Administration (HRSA) bureau or office: Office of Federal Assistance Management; Amount of obligations: FY 2012: $58,561; FY 2013: $0. Contractor: Claxton and Company, P.C. Contract purpose: To conduct a limited scope review to determine if a Bureau of Primary Health Care and Office of Rural Health Policy grantee used grant funds in accordance with grant terms and conditions and grant laws and regulations; is properly allocating and monitoring grant funds; has adequate internal controls in place, including, but not limited to, for disbursements of funds, procurement procedures, and allowable costs; and if there are conflicts of interest involving the grantee's Chief Executive Officer; Health Resources and Services Administration (HRSA) bureau or office: Office of Federal Assistance Management; Amount of obligations: FY 2012: $52,586; FY 2013: $0. Contractor: Claxton and Company, P.C. Contract purpose: To conduct a limited scope review to determine if a Bureau of Primary Health Care grantee has adequate financial systems in place, has spent grant funds properly, has adequate policies and procedures in place to meet program guidelines, and is addressing previously identified issues; Health Resources and Services Administration (HRSA) bureau or office: Office of Federal Assistance Management; Amount of obligations: FY 2012: $49,167; FY 2013: $0. Contractor: Dixon Group, Inc. Contract purpose: To provide logistical support and conduct diagnostic compliance site visits to grantees; Health Resources and Services Administration (HRSA) bureau or office: HIV/AIDS Bureau; Amount of obligations: FY 2012: $428,335; FY 2013: $0. Contractor: Global Evaluation & Applied Research Solutions; Contract purpose: To provide technical assistance and conduct diagnostic compliance site visits to grantees; Health Resources and Services Administration (HRSA) bureau or office: HIV/AIDS Bureau; Amount of obligations: FY 2012: $2,480,090; FY 2013: $3,304,972. Contractor: Jaybro Group, LLC; Contract purpose: To conduct a limited scope review of an HIV/AIDS Bureau grantee in response to concerns about the grantee's use of grant funds; Health Resources and Services Administration (HRSA) bureau or office: Office of Federal Assistance Management; Amount of obligations: FY 2012: $75,774; FY 2013: $0. Contractor: Management Solutions Consulting Group, Inc. Contract purpose: To provide assistance and support to grantees and HRSA staff by providing on-site visits, including visits to assess compliance with program requirements; document reviews; and consultation to new and existing grantees; Health Resources and Services Administration (HRSA) bureau or office: Bureau of Primary Health Care; Amount of obligations: FY 2012: $13,356,664; FY 2013: $9,798,176. Contractor: Olshesky Design Group, LLC; Contract purpose: To provide School-Based Health Center Capital Program application review support, ensure grantees are in compliance with federal requirements, and analyze grantees' progress in implementing funded projects; Health Resources and Services Administration (HRSA) bureau or office: Bureau of Primary Health Care; Amount of obligations: FY 2012: $289,531; FY 2013: $0. Contractor: Seamon Corporation; Contract purpose: To provide logistical support and conduct site visits to assess grantee compliance; Health Resources and Services Administration (HRSA) bureau or office: HIV/AIDS Bureau; Amount of obligations: FY 2012: $0; FY 2013: $796,346. Contractor: Stantec Consulting Services, Inc. Contract purpose: To provide support for application reviews, assess compliance with federal requirements, and provide postaward oversight; Health Resources and Services Administration (HRSA) bureau or office: Bureau of Primary Health Care; Amount of obligations: FY 2012: $584,347; FY 2013: $0. Source: HRSA and the Federal Procurement Data System-Next Generation. GAO-14-800: Legend: FY = fiscal year. Notes: The term "contracts" refers to the collection of contracts, task orders, and blanket purchase agreements that HRSA used to obtain services from contractors who monitored grantees in fiscal year 2013. We included contract obligations for both fiscal years 2012 and 2013 because work carried out in fiscal year 2013 may have been a result of funds obligated in fiscal year 2012. The amount of obligations represents the total amount of the contract and is not limited to obligations related to grantee monitoring activities. [End of table] [End of section] Appendix II: Comments from the Department of Health and Human Services: Department of Health & Human Services: Office of The Secretary Assistant Secretary for Legislation: Washington, DC 20201: September 8, 2014: Debra A. Draper: Director, Health Care: U.S. Government Accountability Office: 441 G Street NW: Washington, DC 20548: Dear Ms. Draper: Attached are comments on the U.S. Government Accountability Office's (GAO) report entitled, "Health Resources and Services Administration: Action Taken to Train and Oversee Staff Who Monitor Grantees, but Certain Guidance Could Be Improved" (GAO-14-800). The Department appreciates the opportunity to review this report prior to publication. Sincerely, Signed by: Jim R. Esquea: Assistant Secretary for Legislation: Attachment: General Comments Of The Department Of Health And Human Services (HHS) On The Government Accountability Office's (GAO) Draft Report Entitled "Health Resources And Services Administration: Action Taken To Train And Oversee Grantee Monitoring Staff, But Certain Guidance Could Be Improved" (GAO-14-800): The Department of Health and Human Services (HHS) appreciates the opportunity to review and comment on this draft report. GAO Recommendation: To help ensure complete and effective monitoring of grantees, we recommend that the Administrator of Health Resources and Services Administration (HRSA) develop a process, either at the agency or bureau level, to ensure that bureaus' standard operating procedures (SOPS) address all of the key components of grantee monitoring as established by the agency. HHS Response: HHS concurs with GAO's recommendation and has already begun a process to ensure that Bureau and Office SOPs address the key components of grantee monitoring established in the agency wide template. In addition, HHS policy will direct bureaus and offices to annually review and update their SOPs to reflect any changes in legislation, policy, or program operations. To assist in this process, HRSA is establishing an annual process to discuss with bureaus and offices post-award monitoring activities, best practices, and implementation of agency and government-wide changes. HHS is pleased to note GAO's recognition of HRSA's efforts to systematically develop guidance for key staff involved in grantee monitoring, implement training programs for these key staff, and institute mechanisms to ensure contractors involved in grantee monitoring are qualified. As noted in this report, HRSA fulfills an extremely important role in improving access to health care by strengthening the health care workforce, building healthy communities, and achieving health equity. HHS will continue to learn from the evolving public health environment and adapt with improved operational efficiencies. We remain committed to our core management philosophy to ensure maximum impact of every dollar the public has entrusted to us. We will also continue to value feedback from outside sources, like GAO, to assist in our continuous cycle of evaluation and improvement. Thank you again for the opportunity to review this report. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Debra A. Draper, (202) 512-7114 or draperd@gao.gov: Acknowledgments: In addition to the contact named above, Michelle B. Rosenberg, Assistant Director; Jill K. Center; Kathleen Diamond; Cathleen J. Hamann; Emily Loriso; Kaitlin McConnell; Julie T. Stewart; and Jennifer M. Whitworth made key contributions to this report. [End of section] Footnotes: [1] At the time of our review, HRSA had seven programmatic bureaus-- the Bureau of Clinician Recruitment and Service, Bureau of Health Professions, Bureau of Primary Health Care, Healthcare Systems Bureau, HIV/AIDS Bureau, Maternal and Child Health Bureau, and the Office of Rural Health Policy--that each manage a portfolio of activities, including awarding grants, dealing with a specific area of health care services, systems, or workforce. In May 2014, HRSA combined the Bureau of Health Professions and the Bureau of Clinician Recruitment and Service into a single bureau called the Bureau of Health Workforce. Throughout this report, we use the term "bureaus" to refer to the seven organizational components that existed at the time of our review; HRSA considers the Office of Rural Health Policy to be similar in function and organizational structure to the units with "bureau" in their titles. OFAM serves as a central office for administration and management of the agency's grants and works in partnership with the programmatic bureaus. [2] For more information on HRSA's staffing and its use of contractors, see GAO, Health Resources and Services Administration: Review of Internal Communication Mechanisms, Staffing, and Use of Contracts, [hyperlink, http://www.gao.gov/products/GAO-14-52] (Washington, D.C.: Dec. 3, 2013). [3] See GAO, Ryan White CARE Act: Improvements Needed in Oversight of Grantees, [hyperlink, http://www.gao.gov/products/GAO-12-610] (Washington, D.C.: June 11, 2012). [4] See GAO, Health Center Program: Improved Oversight Needed to Ensure Grantee Compliance with Requirements, [hyperlink, http://www.gao.gov/products/GAO-12-546] (Washington, D.C.: May 29, 2012). [5] We reviewed SOPs that were issued by the bureaus as of April 2014 and by OFAM as of July 2014. [6] See GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [7] In May 2014, HRSA combined the Bureau of Health Professions with the Bureau of Clinician Recruitment and Service to create the Bureau of Health Workforce. [8] We conducted a total of nine group interviews--four with project officers based in HRSA's headquarters, one with project officers who worked in locations other than HRSA's headquarters, and four with supervisors. [9] In selecting these bureaus, we excluded project officers that exclusively monitored block grants, and also excluded information related to block grants from the numbers of active grants and grant awards. A block grant is a type of grant where funding recipients have substantial discretion over the type of activities to support, with minimal federal administrative requirements, restrictions, and monitoring. After these exclusions, the four selected bureaus, together, employed approximately 86 percent of HRSA's project officers and oversaw 87 percent of HRSA grants, which accounted for nearly 98 percent of non-block grant funding. [10] See GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO-04-546G] (Washington, D.C.: March 2004). [11] In this report, we use the term "contracts" to refer to the collection of contracts, task orders, and blanket purchase agreements that HRSA used to obtain services from contractors who monitored grantees in fiscal year 2013. [12] An obligation is a definite commitment that creates a legal liability for the payment of goods and services ordered or received. We considered both fiscal year 2012 and 2013 obligations because work carried out in fiscal year 2013 may have been a result of funds obligated in fiscal year 2012. To determine the amount of obligations for each of the 13 contracts we reviewed, we obtained data from the Federal Procurement Data System-Next Generation, the primary government-wide contracting database, which provides information on government contracting actions and procurement trends. [13] HRSA awards a grant through the issuance of a Notice of Award-- which, among other things, sets forth the amount of funds granted, the terms and conditions of the award, and the effective date of the award. [14] The closeout phase involves reviewing expiring grants to ensure that all requirements have been met; resolving any known business management issues, such as the proper disposition of property purchased with grant funds; and ensuring the grant file is complete. [15] The number of project officers is as of June 2014. [16] At the time of our review, HHS was in the process of revising and replacing this manual and other guidance related to grants with a new Grants Policy Administration Manual. [17] Several project officers we interviewed said that they also rely on legislation that establishes requirements for the programs they work on as a key source of guidance for monitoring grantees. [18] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [19] HRSA gave the bureaus flexibility in determining how to organize their SOPs. To the extent that the bureaus opted to divide their SOPs into multiple sections and document them separately, we looked at all of the sections as a single SOP. For example, if a bureau had multiple sections for the same grant program, we looked at them collectively as a whole. Likewise, if a bureau had one overarching section, as well as separate sections for four different grant programs, we considered the bureau to have four SOPs, each made up of the overarching section and the specific section for that program. [20] We considered a component fully addressed if a given SOP included all of the information specified in the template for that component, and partially addressed if a given SOP included some, but not all, of the specified information. For example, we considered the communication component on specifying the frequency and purpose of communication with grantees to be fully addressed if the SOP covered both the frequency and purpose of grantee communication, partially addressed if only one of these was covered, and not addressed if neither were covered. [21] One of the 5 SOPs fully addressed four of the key components related to reviewing grantee reports; the fifth component was not applicable. Specifically, the grantee requests for prior approvals component was not applicable because this SOP was specific to a program for which HRSA has not provided funding to grantees since 1997. Therefore, there are no grantee requests that would require prior approval. [22] The template specifies that the SOPs need to address the processes for documenting grantee compliance, including a clear designation of compliance or noncompliance. The 7 SOPs that did not address this component were not explicit that project officers need to do this. [23] The site visit components were only applicable for 10 of the 13 SOPs because 3 SOPs were specific to programs that do not include site visits as part of their monitoring approach. [24] The template specifies that the SOPs need to address the processes for inputting data for identification of, and tracking, site visits and site visit reports in a centralized location accessible to all relevant staff. While HRSA's EHB may have the functionality that allows for this sort of tracking, none of the SOPs explicitly addressed it. [25] For information on practices used by other federal agencies to train the workforce responsible for overseeing grants, see GAO, Grant Workforce: Agency Training Practices Should Inform Future Government- wide Efforts, [hyperlink, http://www.gao.gov/products/GAO-13-591] (Washington, D.C.: June 28, 2013). [26] HRSA defined experienced project officers as those that were on board as of June 1, 2013, and new project officers as those that were hired after that date. [27] Many experienced project officers located outside of HRSA's headquarters had difficulty completing the certification program by the deadline due to unanticipated issues in scheduling two vendor- offered courses for these staff. Officials reported that 95 percent of the experienced project officers in headquarters had completed the certification as of the April 2014 deadline. [28] HRSA does not have a formal certification or career development program for its financial integrity staff. However, HRSA officials explained that there are specific credentials required of certain financial integrity staff. Specifically, to be hired for an accounting or auditor position, individuals must have at least 24 college accounting credits. [29] Competencies describe measurable patterns of knowledge, skills, abilities, behaviors, and other characteristics that an individual needs to perform work roles or occupational functions successfully. Understood another way, competencies specify what the person needs to do the job successfully. See [hyperlink, http://www.gao.gov/products/GAO-13-591]. [30] Federal leading practices for workforce training identify four components of the training and development process and elements to look for in assessing agencies' training efforts. These components include planning, design and development, implementation, and evaluation. See [hyperlink, http://www.gao.gov/products/GAO-04-546G]. [31] Leadership from one of these bureaus (the Office of Rural Health Policy) told us that they plan to begin offering formal training in the future. [32] Additionally, OFAM and two bureaus have created snapshot reports in EHB called "dashboards," which can be used for oversight purposes. For example, a dashboard can provide information on the status of certain types of assignments within an office or bureau, providing information on the total number of assignments that need review; how many have not been started or are in progress; and how many are on target to be completed on time. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]