National Preparedness:

Improvements Needed for Measuring Awardee Performance in Meeting Medical and Public Health Preparedness Goals

GAO-13-278, Mar 22, 2013

Additional Materials:

Contact:

Marcia G. Crosse
(202) 512-7114
crossem@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Available measures and awardee data provide some evidence that Hospital Preparedness Program (HPP) awardees have generally made progress in carrying out activities to achieve medical preparedness goals; however, the Department of Health and Human Services (HHS) Office of the Assistant Secretary for Preparedness and Response (ASPR) lacked a comprehensive performance management system to fully assess awardee progress. According to prior GAO work and the GPRA Modernization Act of 2010 (GPRAMA), successful performance measurement systems should include a select set of performance measures tied to realistically achievable targets with clearly defined milestones. GAO's analysis of ASPR data showed general progress. For example, the percentage of all 62 awardees' participating hospitals with medical evacuation and shelter-in-place plans increased from 79.9 percent to 88.3 percent from fiscal year 2007 to fiscal year 2011. However, while ASPR collected data on a range of activities, it did not have consistent performance measures and targets in place across this entire period. Beginning with fiscal year 2012, ASPR developed new provisional performance measures for the eight new capabilities that awardees are to use for HPP planning for the next 5 years and set 5-year targets for these measures. However, it did not develop annual milestones, which may make it difficult for ASPR and awardees to assess incremental progress toward meeting HPP goals.

Although Public Health Emergency Preparedness (PHEP) program awardees are improving in their ability to carry out preparedness activities, HHS's Centers for Disease Control and Prevention (CDC) lacked a consistent set of performance measures and targets to adequately assess the degree of awardee progress toward meeting the PHEP goal. For example, from fiscal years 2007 through 2011, the number of measures ranged from 5 to 30 in any one year, and CDC had only four targets for any of them. GAO's analysis of CDC data showed general progress on the measures. For example, for one measure, the average time it took the 62 awardees to assemble appropriate response staff decreased from 35 minutes in 2007 to 31 minutes in 2011, although the 50 state awardees did not always meet the 60-minute target that CDC set for them starting in 2009. Beginning with fiscal year 2012, CDC released 47 provisional performance measures for 14 of the 15 new PHEP capabilities but developed only four associated targets. Without consistent performance measures and associated targets, in accordance with prior GAO work and GPRAMA, CDC may not be able to assess how awardees are making progress toward meeting the PHEP goal.

HHS uses internal databases, site visits, and audit reports to help awardees meet HPP and PHEP application, financial, and reporting requirements. ASPR and CDC use internal databases to generate reports on awardee progress in meeting application renewal and reporting deadlines, to assess application completeness, and to periodically query databases in order to review financial information. They also conduct regular site visits and review state and federal audit reports to help awardees meet program requirements and assess awardees' use of funds. ASPR and CDC require awardees that have problems managing their HPP or PHEP funds to complete corrective action plans, and they restrict awardees' access to funds in more serious cases.

Why GAO Did This Study

HHS provides funding to the 50 states and 12 municipalities, territories, and freely associated states, primarily through ASPR's HPP and CDC's PHEP cooperative agreements, to help them build their capability to respond to emergencies such as hurricanes, pandemics, or terrorist events. The 62 awardees are to use this funding to help achieve the HPP goals of strengthening hospital preparedness and medical surge capacity and the PHEP goal of strengthening public health preparedness, and they must meet certain application, financial, and reporting requirements. GAO was asked about the effects of federal support on state and local response capabilities. GAO (1) assessed awardee progress in meeting HPP goals and how ASPR measures that progress, (2) assessed awardee progress in meeting the PHEP goal and how CDC measures that progress, and (3) identified the mechanisms HHS uses to ensure that awardees are meeting application, financial, and reporting requirements. GAO reviewed HPP and PHEP guidance, performance measures, and other documents; interviewed HHS officials; and analyzed HPP and PHEP data for fiscal years 2007 through 2011.

What GAO Recommends

GAO recommends that (1) HHS develop objective and quantifiable performance targets and incremental milestones tied to HPP and PHEP performance measures and (2) ensure that measures remain consistent and comparable to sufficiently measure progress. HHS generally agreed with GAO's recommendations but indicated that it would not be able to fully implement them for several years.

For more information, contact Marcia Crosse at (202) 512-7114 or crossem@gao.gov.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To help ensure that HHS is adequately and comprehensively assessing HPP and PHEP awardees' performance and progress in meeting the medical and public health preparedness goals of the cooperative agreements, the Secretary of Health and Human Services should direct ASPR and CDC to develop objective and quantifiable performance targets and incremental milestones that correspond to the new HPP and PHEP performance measures, against which HHS can gauge progress toward the medical and public health preparedness goals of the cooperative agreements and direct technical assistance, as needed.

    Agency Affected: Department of Health and Human Services

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help ensure that HHS is adequately and comprehensively assessing HPP and PHEP awardees' performance and progress in meeting the medical and public health preparedness goals of the cooperative agreements, the Secretary of Health and Human Services should ensure that performance measures and targets remain consistent across the 5-year project cycle and that any future measures be comparable to determine whether awardees are making progress toward meeting short- and long-term medical and public health preparedness goals of the cooperative agreements.

    Agency Affected: Department of Health and Human Services

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.