This is the accessible text file for GAO report number GAO-13-61 entitled 'Workplace Safety and Health: Further Steps by OSHA Would Enhance Monitoring of Enforcement and Effectiveness' which was released on February 25, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Requesters: January 2013: Workplace Safety and Health: Further Steps by OSHA Would Enhance Monitoring of Enforcement and Effectiveness: GAO-13-61: GAO Highlights: Highlights of GAO-13-61, a report to congressional requesters. Why GAO Did This Study: OSHA is responsible for overseeing occupational safety and health for more than 130 million workers. In about half the states, OSHA sets and enforces compliance with safety and health standards. The remaining states set and enforce their own standards under OSHA-approved plans. In fiscal year 2010, OSHA strengthened its monitoring of state-run programs following a dozen worker deaths in one of those states. Questions have since been raised about how closely OSHA monitors its own enforcement efforts. GAO examined 1) how OSHA’s monitoring of its own and state enforcement efforts compares, and 2) recent steps OSHA has taken to evaluate the effectiveness of federal and state enforcement efforts. GAO reviewed OSHA’s monitoring policies and procedures and relevant federal laws and regulations; analyzed federal and state audits; visited three OSHA regional offices; and interviewed OSHA officials and other experts. What GAO Found: The Department of Labor’s Occupational Safety and Health Administration (OSHA) provides more frequent monitoring and more consistent guidance for its federal enforcement activities than for state enforcement activities. More specifically, OSHA regional officials review performance reports on federal activities at least every other week but review reports on state-run program activities quarterly. In addition, OSHA’s guidance for audits of its regional and area offices is more consistent than the guidance for its audits of state-run programs. Guidance for audits of its offices requires that regional offices conduct a comprehensive audit with on-site review of inspection case files at least once every 4 years and other audits focused on more specific activities in all other years. In contrast, guidance for regional office audits of state-run programs changes from year to year and does not include a regular schedule for comprehensive audits with on-site case file reviews. While the frequency of OSHA’s monitoring of state-run programs is necessarily different because of the independent enforcement authority of participating states, OSHA’s lack of consistent guidance for audits of these state-run programs may allow enforcement deficiencies to go undetected, increasing the risk of worker injuries, illnesses, or death. In addition, there is little participation by OSHA’s national office in comprehensive audits of its regional offices despite a 2010 directive to do so. As a result, OSHA cannot ensure that the results of regional audits are impartial. OSHA is taking steps to better assess the effectiveness of both its federal enforcement efforts and of state enforcement efforts, but it is often not clear how these steps will help OSHA demonstrate what efforts result in better outcomes for workers, such as reduced worker injuries, illnesses, and fatalities. For example, OSHA recently revised some of the measures it uses to assess state-run programs by adding acceptable ranges of performance. However, the revised measures still largely focus on outputs-—such as the average number of violations per inspection-—rather than outcomes. OSHA is also conducting studies to examine the results of specific enforcement activities, including one designed, in part, to evaluate the effect of OSHA providing additional educational support to employers. While tracking progress in meeting performance goals-—such as the annual number of conducted inspections—-is useful, the lack of focus on outcomes makes it difficult for OSHA to determine which specific enforcement activities are most effective or to convince states to implement changes designed to improve outcomes. For example, only two state-run programs raised their penalty amounts in fiscal year 2011 as recommended by OSHA; state-run program representatives stated that OSHA lacked evidence to show that higher penalties are more effective in deterring future employer violations. OSHA also does not use data already in hand to assess the effectiveness of federal and state enforcement efforts. For example, OSHA does not use data from its annual audits of its regional and area offices or of state-run programs to inform its planning or share information across regions. Additionally, OSHA annually collects data on activities conducted under emphasis programs that focus on national safety and health issues, but it does not evaluate these data to determine whether these programs are responsible for desired outcomes. What GAO Recommends: GAO recommends that OSHA standardize guidance for its audit practices, include outcomes in its assessments of its enforcement initiatives, better use data from its audits, and ensure national office participation in audits. OSHA generally agreed with the recommendations but expressed concern about overuse of outcomes to assess effectiveness. GAO continues to believe the recommendations are valid as discussed later in the report. View [hyperlink, http://www.gao.gov/products/GAO-13-61]. For more information, contact Revae Moran at (202) 512-7215 or moranr@gao.gov. [End of section] Contents: Letter: Background: OSHA's Monitoring Is More Frequent and Its Guidance Is More Consistent for Federal than State Enforcement: OSHA Is Taking Steps to Better Assess Effectiveness but Does Not Focus on Outcomes: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: Comments from the Department of Labor: Appendix III: GAO Contact and Staff Acknowledgments: Related GAO Products: Figures: Figure 1: Federal and State Responsibilities for Enforcement in OSHA's 10 Regions, FY 2012: Figure 2: OSHA's Enforcement Monitoring Structure: Figure 3: OSHA Inspections by Region, FY 2012: Figure 4: Enforcement Activities for State-Run Programs and OSHA, FY 2012: Figure 5: Average Current Penalty Amounts for Serious Violations for State-run Programs Compared to OSHA's Average Penalty (in dollars), FY 2012: Abbreviations: EFAME: Enhanced Federal Annual Monitoring and Evaluation: FAME: Federal Annual Monitoring and Evaluation: GPRA: Government Performance and Results Act of 1993: Labor: Department of Labor: MAP: Management Accountability Program: NACOSH: National Advisory Committee on Occupational Safety and Health: NEP: National Emphasis Program: NIOSH: National Institute for Occupational Safety and Health: OMB: Office of Management and Budget: OSH Act: Occupational Safety and Health Act of 1970: OSHA: Occupational Safety and Health Administration: OSHSPA: Occupational Safety and Health State Plan Association: [End of section] United States Government Accountability Office: Washington, DC 20548: January 24, 2013: The Honorable John Kline: Chairman: Committee on Education and the Workforce: House of Representatives: The Honorable Tim Walberg: Chairman: Subcommittee on Workforce Protections: Committee on Education and the Workforce: House of Representatives: In an 18-month period from 2006 to 2008, a dozen workers died on construction sites in Las Vegas, Nevada, prompting a congressional investigation into the Department of Labor's (Labor) Occupational Safety and Health Administration's (OSHA) oversight of worker safety in that state. OSHA has responsibility under the Occupational Safety and Health Act of 1970 (OSH Act) to assure safe and healthful working conditions for the nation's workers.[Footnote 1] Under the act, OSHA sets and directly enforces occupational safety and health standards in about half the states. The remaining states, including Nevada, have chosen to set and enforce their own standards, as permitted by the OSH Act, under an OSHA-approved state plan.[Footnote 2] OSHA, in turn, monitors enforcement by these state-run programs. Nevada had previously received favorable annual reviews from OSHA of its occupational safety and health program. However, a special review of Nevada by OSHA in 2009--after the construction deaths--revealed significant problems in the state's enforcement program. In fiscal year 2010, as a result of the problems identified in Nevada, OSHA strengthened its annual reviews of all state-run programs. Unlike prior reviews, these reviews included on-site examination of state inspection case files by OSHA officials. In this context, we examined 1) how OSHA's monitoring of its own and state enforcement efforts compares, and 2) recent steps OSHA has taken to evaluate the effectiveness of federal and state enforcement efforts. To answer our research questions, we reviewed relevant federal laws and regulations as well as Labor and OSHA policies and procedures for monitoring federal and state enforcement efforts. We analyzed OSHA audits of its regional and area offices and of state-run programs. [Footnote 3] We conducted site visits to three OSHA regional offices: Region 5 in Chicago, Illinois, Region 3 in Philadelphia, Pennsylvania, and Region 8 in Denver, Colorado. In selecting regional offices for site visits, we took into account region size based on the total number of worksite inspections conducted annually, geographic location, and the mix of states with state-run programs and those for which OSHA enforces compliance in each region. At each office, we interviewed regional officials about their oversight of federal enforcement efforts and those of states, and visited two area offices. Within each of the three regions, we visited two area offices: one responsible for overseeing federal enforcement and one responsible for overseeing state enforcement. For the area offices that oversee state- run program(s), we selected states with a range of total number of annual worksite inspections: Virginia (a large number), Indiana (a medium number), and Wyoming (a small number). We interviewed regional officials in the remaining seven regional offices by phone to obtain similar information. We also collected key documents from each region, including recent audits of OSHA federal and state enforcement efforts and inspection activity tracking reports. In addition, we interviewed officials from Labor and OSHA's national office responsible for enforcement, evaluation, strategic planning, and monitoring of state-run programs about their roles in monitoring enforcement and measuring effectiveness. To obtain information from outside parties about OSHA's oversight of federal and state enforcement, we interviewed officers of the Occupational Safety and Health State Plan Association (OSHSPA), which represents the interests of states with state-run programs, and officials at the Department of Health and Human Services' National Institute for Occupational Safety and Health (NIOSH).[Footnote 4] We also discussed with academic experts and other stakeholders Labor and OSHA's performance in conducting and monitoring enforcement, challenges faced by states with state-run programs, and current research on these issues.[Footnote 5] Appendix I provides more information on our scope and methodology. We conducted this performance audit from November 2011 through January 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: Federal and State Responsibilities: OSHA and state-run programs are responsible for protecting the safety and health of an estimated 130 million private-and public-sector workers at approximately 8 million worksites. One of the primary ways OSHA carries out its responsibilities is by setting and enforcing occupational safety and health standards.[Footnote 6] States with state-run programs typically adopt standards identical to federal standards, according to OSHA, but they may set standards that are more stringent or that address hazards not covered by federal standards. For example, according to OSHA, California, Michigan, Oregon, and Washington have standards that are different from federal standards. To approve a state-run program, OSHA must determine that the state's plan meets certain criteria specified in the OSH Act, including the development and enforcement of standards that are "at least as effective in providing safe and healthful employment" as the federal standards.[Footnote 7] As required by the act, OSHA also conducts continuing evaluations of states' implementation of their plans. [Footnote 8] When evaluating state-run programs, OSHA considers a program to be "at least as effective as" the federal program if it is making reasonable progress toward meeting established performance goals and fulfilling mandated responsibilities. OSHA enforces standards for private sector workplaces not covered by a state-run program, and conducts inspections at federal workplaces in all states.[Footnote 9] Most states with a state plan enforce employer compliance with safety and health standards for the private sector as well as state and local public-sector workplaces in their state; however, some have chosen to provide such enforcement only for public- sector workplaces (see figure 1).[Footnote 10] There are currently 27 state-run programs responsible for providing enforcement to protect about 40 percent of U.S. workers. Figure 1: Federal and State Responsibilities for Enforcement in OSHA's 10 Regions, FY 2012: [Refer to PDF for image: illustrated U.S. map] Region 1: Headquarters: Boston; Connecticut; Maine; Massachusetts; New Hampshire; Rhode Island; Vermont. Region 2: Headquarters: New York City; New Jersey; New York; Puerto Rico; Virgin Islands. Region 3: Headquarters: Philadelphia; Delaware; District of Columbia; Maryland; Pennsylvania; Virginia; West Virginia. Region 4: Headquarters: Atlanta; Alabama; Florida; Georgia; Kentucky; Mississippi; North Carolina; South Carolina; Tennessee. Region 5: Headquarters: Chicago; Illinois; Indiana; Michigan; Minnesota; Ohio; Wisconsin. Region 6: Headquarters: Dallas; Arkansas; Louisiana; New Mexico; Oklahoma; Texas. Region 7: Headquarters: Kansas City, Missouri; Iowa; Kansas; Missouri; Nebraska. Region 8: Headquarters: Denver; Colorado; Montana; North Dakota; South Dakota; Utah; Wyoming. Region 9: Headquarters: San Francisco; American Samoa; Arizona; California; Guam; Hawaii; Nevada; Northern Mariana Islands. Region 10: Headquarters: Seattle; Alaska; Idaho; Oregon; Washington; OSHA provides enforcement for the private sector: Alabama; American Samoa; Arkansas; Colorado; Connecticut; Delaware; District of Columbia; Florida; Georgia; Guam; Idaho; Kansas; Louisiana; Maine; Massachusetts; Mississippi; Missouri; Montana; Nebraska; New Hampshire; North Dakota; Northern Mariana Islands; Ohio; Oklahoma; Pennsylvania; Rhode Island; South Dakota; Texas; West Virginia. State provides enforcement under an OSHA-approved state plan for the private sector and the state and public sector: Alaska; Arizona; California; Hawaii; Indiana; Iowa; Kentucky; Maryland; Michigan; Minnesota; Nevada; New Mexico; North Carolina; Oregon; Puerto Rico; South Carolina; Tennessee; Utah; Vermont; Virginia; Washington; Wyoming. State provides enforcement under an OSHA-approved state plan for the state and local public sector. OHSA provides enforcement for the private sector: Connecticut; Illinois; New Jersey; New York; Virgin Islands. Source: GAO analysis of OSHA information. Note: In all states, OSHA generally has authority to conduct inspections of federal agencies. [End of figure] OSHA provides grants to states with state-run programs of up to 50 percent of state-run program costs to assist with each state's operating costs.[Footnote 11] In fiscal year 2012, Congress appropriated just over $104 million for the purposes of providing these grants to states. OSHA requires each state to submit in its grant application an annual performance plan for approval. The annual performance plan includes a detailed enforcement plan describing how the state intends to meet its inspection and other goals and must be approved by OSHA before a state receives its federal grant. OSHA assesses the performance of a state-run program against the goals in the state's performance plan, as well as on mandated activities, such as program administration and investigation of complaints.[Footnote 12] In a recent congressional hearing, an OSHA official said state-run programs provide a number of advantages in helping OSHA carry out its enforcement efforts. These advantages include adding resources to the federal program that would not otherwise be available, providing enforcement for state and local public-sector workers in those states, and addressing workplace hazards that are sometimes not addressed by OSHA. OSHA Monitoring and Enforcement: OSHA uses directorates in its national office, as well as 10 regional offices and 87 area offices, to implement and oversee enforcement. [Footnote 13] The national office includes the Directorate of Enforcement Programs, which provides guidance to OSHA inspectors on how to enforce safety and health standards and how employers are to comply with them; the Directorate of Evaluation and Analysis, which analyzes safety and health data; and the Directorate of Cooperative and State Programs, which develops policy and oversees the grants provided to state-run programs. The directorates provide guidance to the regions and to states. At the same time, OSHA affords significant autonomy to its 10 regions to both conduct enforcement activities through their area offices and to monitor enforcement by state-run programs within their regions. While OSHA's regional offices work with OSHA's national directorates, they report to the Assistant Secretary of Labor for Occupational Safety and Health (see figure 2). Figure 2: OSHA's Enforcement Monitoring Structure: [Refer to PDF for image: organizational chart] OHSA: Top level: Assistant Secretary; Deputy Assistant Secretary. Second level, reporting to Assistant Secretary: * Directorate of Cooperative and State Programs; * Directorate of Enforcement Programs; * Directorate of Evaluation and Analysis. Third level, reporting to Assistant Secretary: Regional Offices. Fourth level, reporting to Regional Offices: * Federal enforcement Area offices; * State enforcement State-run programs. Source: GAO analysis of OSHA information. [End of figure] As part of enforcement efforts, OSHA and states conduct programmed and unprogrammed inspections of employer worksites to ensure compliance with safety and health standards. OSHA's programmed inspections are scheduled inspections that target specific worksite hazards or high- hazard industries and worksites. OSHA's unprogrammed inspections are conducted in response to imminent danger, fatalities or catastrophes (death of one or more workers or in-patient hospitalization of three or more workers), complaints, referrals, and as follow-up and monitoring of prior inspections. In fiscal year 2012, OSHA's federal inspectors conducted about 41,000 inspections, and state-run programs conducted about 52,000 inspections. Fifty-six percent of the federal and state inspections were programmed. OSHA also conducts outreach to educate employers and workers about safe worksite practices; supports employers in reducing safety and health risks; and works with other agencies, unions, businesses, and stakeholders to address occupational safety and health issues. OSHA also requires its regions and state-run programs to participate in or respond to periodic National Emphasis Programs (NEP) that target outreach and enforcement to specific hazards or industries, such as preventing amputations or reducing hazardous exposures in manufacturing. According to OSHA, a state-run program may adopt the federal program, or it may adopt an equivalent state program if it can document that it is "at least as effective." In addition to these NEPs, regional offices and state-run programs may develop regional or local emphasis programs to address safety and health issues prevalent in their mix of businesses and industries. OSHA's Monitoring Is More Frequent and Its Guidance Is More Consistent for Federal than State Enforcement: OSHA Monitors Federal Efforts More Frequently than State Efforts: OSHA uses a variety of methods to monitor the progress of federal and state enforcement efforts, and monitors federal enforcement efforts more frequently than those of states. For federal enforcement, OSHA national officials monitor regional office activities based on the goals it sets in collaboration with regional officials each year-- including the number of worksite inspections each region is expected to complete--and report progress quarterly to the Department of Labor. OSHA national officials monitor regional offices' progress toward meeting these goals by reviewing their performance measures and holding weekly conference calls with regional administrators to discuss performance and other issues. Regional officials generally monitor their area offices' progress toward meeting goals by also reviewing inspection activity reports submitted weekly by each area office, and through meetings, typically by phone, with area office managers held at least every other week to review the data. Inspection activity reports provide information on, among other things, the number and type of inspections completed, violations found, and penalties assessed. Some regions also hold in-person meetings a few times a year with all area office managers to review their progress. These reports and meetings are the primary ways officials keep track of each region's progress toward meeting its goals. For example, in fiscal year 2012, 7 out of 10 OSHA regional offices met or exceeded their annual goal for conducting inspections (see figure 3). Figure 3: OSHA Inspections by Region, FY 2012: [Refer to PDF for image: stacked horizontal bar graph] Number of inspections: Region: 1; Conducted: 3,693; Goal: 4,133. Region: 2; Conducted: 5,801; Goal: 5,829. Region: 3; Conducted: 4,443; Goal: 4,448. Region: 4; Conducted: 5,940; Goal: 6,288. Region: 5; Conducted: 7,207; Goal: 7,343. Region: 6; Conducted: 5,907; Goal: 6,319. Region: 7; Conducted: 2,772; Goal: 2,814. Region: 8; Conducted: 2,214; Goal: 2,315. Region: 9; Conducted: 1,045; Goal: 1,073. Region: 10; Conducted: 1,185; Goal: 1,192. Source: GAO analysis of OSHA data. [End of figure] To monitor state enforcement efforts, OSHA regional and area officials also review performance reports and meet with state officials to monitor their progress toward meeting their inspection and other goals, but they do so less frequently for state-run programs than for federal enforcement efforts. OSHA regional and area offices work directly with states to set inspection goals, taking into account staffing and funding levels in each state. Regional officials told us that some state-run programs face funding challenges. For example, Hawaii recently reported problems in carrying out its program due to lack of state staffing resources and funds. States generally have primary responsibility for monitoring the performance of their state- run programs, subject to oversight by OSHA. States are afforded flexibility in setting their own goals as long as they report their activities across a range of performance measures, such as the percentage of programmed inspections that find certain types of violations, and the average number of violations per inspection. Regional and area officials also meet quarterly with state managers to review states' performance, whereas regional officials meet with area officials at least once every other week to review their performance. For state-run programs, regional officials look for outliers or significant variations from the national averages or a state's own performance goals.[Footnote 14] When variations are found, OSHA's regional officials and state-run program officials discuss how the state will address them. For example, state-run program officials could work with OSHA regional officials to develop a corrective action plan that specifies steps the state will take to improve its program. During these quarterly meetings with the states, OSHA regional and area officials review several types of reports that compare a state's progress with federal and national data on multiple measures for the same period. The State Indicator Report tracks the percentage of state- run program inspections completed by type and compares them to the percentage of federal inspections, typically by quarter. The State Activity Mandated Measures report tracks a state's performance across various measures, such as the amount of time it takes a state to begin an inspection after receiving a complaint about a safety or health hazard, and the percentage of programmed inspections conducted by the state for which serious, willful, or repeat violations are found. In addition, OSHA officials review aggregate data on inspections, violations, and penalties as a part of their annual audits of states. OSHA's regional offices have transitioned to a new data system, the OSHA Information System, which officials said should allow for more sophisticated data runs and evaluation of outputs. State-run programs have not yet adopted the new system and are still using the Integrated Management Information System, or compatible systems, to report results to OSHA. OSHA offices now manually merge data from the two systems to produce totals for their enforcement activities. For fiscal year 2012, inspectors in state-run programs conducted more inspections and cited more violations than federal OSHA inspectors, but cited fewer serious violations and assessed lower penalties overall (see figure 4).[Footnote 15] Figure 4: Enforcement Activities for State-Run Programs and OSHA, FY 2012: [Refer to PDF for image: horizontal bar graph] Total inspections: State-run programs: 51,133; OSHA: 40,961. Total violations: State-run programs: 109,784; OSHA: 78,726. Total serious violations: State-run programs: 50,499; OSHA: 57,118. Total penalties: State-run programs: $73,023,000; OSHA: $168,740,000. Source: GAO analysis of OSHA data. [End of figure] OSHA also monitors the significant cases it identifies--those that involve a fatality, inspections in which the total proposed penalty is $100,000 or more, and those in which citations are issued for violating the general duty clause of the OSH Act[Footnote 16]--more closely than significant cases in state-run programs. Before citations are issued by federal inspectors for these types of cases, OSHA's national office requires a higher level of review and approval than for other cases. For example, regional office officials must carefully and critically review high penalty cases and notify national OSHA in advance of the citations. Because states with state-run programs have independent enforcement authority, OSHA's national office does not require this type of review of significant state cases before citations are issued. National officials said they encourage regional officials to review cases involving fatalities during their annual audits of state-run programs. Guidance Is More Consistent for Audits of Federal Enforcement than State Enforcement: While the guidance OSHA provides for auditing its regional and area offices is fairly consistent, its guidance for auditing state-run programs has varied significantly each year since 2009. Under OSHA's Management Accountability Program (MAP), OSHA regional offices conduct annual audits of their own offices and their area offices. These audits are either comprehensive audits or focused audits of the office's inspections and administrative functions. Comprehensive audits, which are conducted at least once every 4 years, cover a full range of enforcement and administrative topics. Focused audits, which are conducted in the years between comprehensive audits, evaluate one or more selected enforcement or administrative topics. The purpose of MAP audits is to assess the efficiency and effectiveness of both OSHA's regional and area offices' enforcement field activities in relation to established policies and procedures and to identify best practices and deficiencies in order to improve performance. During comprehensive audits, OSHA conducts on-site case file reviews and addresses 41 different topics on enforcement activities and administrative issues, such as timekeeping and the appropriate use of government vehicles.[Footnote 17] Topics for focused audits are determined by the regional officials based on areas of concern for them or the national office. Although focused audits of area offices are normally conducted by OSHA's regional offices, area office managers may conduct focused audits of their own offices and report back to the regional office. Area office managers may not, however, conduct comprehensive audits of their own offices--these audits can only be conducted by regional offices and must be done on site. MAP audits of OSHA's regional offices are conducted with little participation on the part of OSHA's national office, despite a 2010 directive issued by OSHA for national staff to take a more direct role in overseeing and participating in the comprehensive audits of its regional offices. Regional officials generally assign staff to audit their own office, so these audits do not provide an independent review of regional offices' performance. Providing for such an independent review is an important aspect of an effective audit program and a key aspect of generally accepted government auditing standards. Government Auditing Standards describe the criteria for independence as follows: "The audit organization and the individual auditor, whether government or public, must be independent…Auditors and audit organizations maintain independence so that their opinions, findings, conclusions, judgments, and recommendations will be impartial and viewed as impartial by reasonable and informed third parties. Auditors should avoid situations that could lead reasonable and informed third parties to conclude that the auditors are not independent and thus are not capable of exercising objective and impartial judgment on all issues associated with conducting the audit and reporting on the work." [Footnote 18] Instead of adhering to these criteria, regional officials organized, managed, and conducted their own audits, as well as audits of their area offices. For the most recent comprehensive audits of OSHA's 10 regional offices, none of the audit summaries or our interviews with regional officials indicated participation by national office staff.[Footnote 19] During our review, a national office official verified that no national office staff has participated in any of the comprehensive audits of regional offices to date, and the official was unsure whether there were any plans to involve national office staff in the audits next year. OSHA's regional offices also conduct annual audits of state-run programs called Federal Annual Monitoring and Evaluation (FAME) audits. The purpose of these audits is to assess the performance of states in meeting their established goals and complying with statutory and regulatory requirements, as well as identify areas of concern and excellence, among other things. During these audits, regions evaluate state-run programs using certain performance measures, such as the average number of days it takes to begin an investigation after receiving a complaint. OSHA officials may also examine case files or accompany inspectors to worksites to verify that proper procedures are being followed. In November 2009, after significant problems were identified in Nevada's occupational safety and health program, OSHA strengthened its audits of all state-run programs and called them Enhanced Federal Annual Monitoring and Evaluation (EFAME) audits. OSHA's guidance for its fiscal year 2009 EFAME audits required that OSHA conduct site visits to states and inspect their case files. According to OSHA, prior to beginning the EFAME audits, the agency generally did not include these elements in the annual FAME audits, which may have limited its ability to identify and address performance problems in some states. The EFAME audits increased the requirements for OSHA's audits of states, aligning them with some of its requirements for the comprehensive MAP audits it performs, which require regular on-site case file reviews. However, beginning in 2009, OSHA guidance on what must be included in audits of state-run programs has changed each year. For fiscal year 2009, OSHA established the EFAME audits. For fiscal year 2010, OSHA issued guidance that required only a follow-up to the fiscal year 2009 reviews. In 2011, OSHA issued guidance that required EFAME audits with on-site case file reviews, and which included a special emphasis on the whistleblower protection program.[Footnote 20] Yet, OSHA's guidance does not include a timetable for conducting future EFAME audits of state-run programs. Not having consistent guidance on audits of state-run programs and no set schedule for when OSHA will conduct these more comprehensive audits can make it difficult for officials in state-run programs to prepare for these audits, and limits OSHA's ability to identify problem areas and help states address them. Regional officials told us EFAME audits have value in detecting problems in the administration of state-run programs that might not otherwise be identified. They have also resulted in multiple corrective actions for each state. Indeed, officials in one regional office said the region has seen improvements in states as these corrective actions have been implemented, such as improved case file documentation. Officials from OSHSPA, the organization that represents states with state-run programs, stated that the 2009 EFAME audits contained a number of recommendations that were useful, although they said the summaries in some reports did not clearly represent the audit findings and did not include the positive aspects of the state-run programs. OSHA national officials said they plan to conduct EFAME audits every other year, with follow-up audits in intervening years; however, they have not developed a formal written plan to implement this new schedule. OSHA Is Taking Steps to Better Assess Effectiveness but Does Not Focus on Outcomes: OSHA Has Several Initiatives to Assess Effectiveness, but Few Focus on Outcomes: OSHA recently developed several initiatives to better assess the effectiveness of state and federal enforcement efforts. It is unclear, however, how some of these initiatives will help OSHA demonstrate which activities result in desired outcomes--a reduction of worker fatalities, injuries, and illnesses. With respect to state enforcement efforts, OSHA created a task force comprised of federal and state representatives to revise state-run program performance measures, and in the summer of 2012 held a stakeholder meeting and solicited public comment on 15 revised measures--which took effect October 1, 2012.[Footnote 21] The task force was created, in large part, in response to a March 2011 report by Labor's Office of Inspector General that shows OSHA had not developed a method of determining whether state programs are at least as effective as federal programs, as required by the OSH Act.[Footnote 22] The board chairman of OSHSPA said OSHA has inadequate guidance for what is needed to meet this requirement and added that, in OSHSPA's view, OSHA often wrongly interprets it to mean that states' measures, trends, and policies must be identical to those of federal OSHA. Some of the revised performance measures developed by the task force are modifications of measures used previously and several add a range of acceptable state performance. For example, the measure for the average number of violations found per inspection formerly assessed a state's performance against a national average calculated across 3 years. The revised measure compares the state's average to a range of 20 percent plus or minus from the national average. OSHA officials said the range provides a better guide for regional officials reviewing state-run programs on what is acceptable performance, and affords states the flexibility they have requested in setting their own performance and strategic goals. For example, one national OSHA official who oversees state-run programs said that if a state's performance is outside the range it does not necessarily mean its performance is unacceptable, but prompts the reviewer to do additional work to find out why that particular mark was missed. However, the revised measures remain largely refinements of indicators previously used to track activities conducted (outputs), not effectiveness (outcomes). They mostly focus on outputs such as average penalty, average number of violations per inspection, and average number of days to issue citations. It is unclear how adding a 20 to 25 percent plus-or-minus range to targets for each of these enforcement activities will help OSHA officials determine whether a state's program is at least as effective as the federal program because neither the state's range nor the federal number is linked to effectiveness--improving employer compliance and reducing worker fatalities, injuries, and illnesses. In commenting on the proposed measures, the Assistant Secretary said they are not pure effectiveness measures but rather measures of enforcement activities. OSHA's difficulty in demonstrating the effectiveness of its enforcement efforts has affected its ability to convince states to implement enforcement changes OSHA believes will result in better outcomes. For example, in 2010, the average penalty for serious violations levied by OSHA was approximately $1,000, and the average penalty for these violations in state-run programs was frequently lower--in three states, it was less than $500. According to the Assistant Secretary, these amounts were far too low to exert a credible effect in deterring employer violations. There were also wide discrepancies from state to state in penalty amounts, which one state- run program manager said raises substantial concerns about equal expectations of employers and protection for workers across states. [Footnote 23] To address these concerns, OSHA increased its penalties in October 2010, and recommended that states match these increases. [Footnote 24] This increase more than doubled the average OSHA penalty by fiscal year 2012, to $2,154. Officers at OSHSPA opposed the higher penalties, saying OSHA had failed to provide empirical studies to show increased penalties were effective in deterring employer violations. They also said that state-run programs could achieve greater compliance with safety and health standards by increasing the likelihood of an inspection. Only two states--Nevada and Wyoming-- increased their penalty amounts in response to OSHA's recommendation, according to a national OSHA official. Average penalty amounts for all but three states (California, Kentucky, and Nevada) remained below the average federal penalty (see figure 5).[Footnote 25] More recently, in a memorandum dated March 27, 2012, OSHA's national office said states were not immediately required to adopt the increased penalty policy, so it is unclear how OSHA plans to take penalty amounts into account when evaluating state-run programs. Figure 5: Average Current Penalty Amounts for Serious Violations for State-run Programs Compared to OSHA's Average Penalty (in dollars), FY 2012: [Refer to PDF for image: vertical bar graph] State: California; Average penalty: $6,301. State: Kentucky; Average penalty: $3,456. State: Nevada; Average penalty: $2,380. Average federal penalty: $2,154. State: Wyoming; Average penalty: $1,705. State: Vermont; Average penalty: $1,065. State: New Mexico; Average penalty: $1,060. State: Arizona; Average penalty: $1,035. State: Hawaii; Average penalty: $1,004. State: Utah; Average penalty: $990. State: North Carolina; Average penalty: $971. State: Alaska; Average penalty: $970. State: Indiana; Average penalty: $920. State: Puerto Rico; Average penalty: $892. State: Iowa; Average penalty: $862. State: Minnesota; Average penalty: $854. State: Maryland; Average penalty: $815. State: Washington; Average penalty: $772. State: Virginia; Average penalty: $731. State: Tennessee; Average penalty: $704. State: South Carolina; Average penalty: $526. State: Michigan; Average penalty: $524. State: Oregon; Average penalty: $375. Source: GAO analysis of OSHA data. Note: Five states--Connecticut, Illinois, New Jersey, New York, and the Virgin Islands--are not shown because their state-run programs cover only state and local public employees and their employers (state and local governments) are generally not assessed penalties, according to OSHA. In these states, private sector employers are assessed penalties by OSHA. California has the highest penalty amounts in the country and some of the most stringent standards, but the state classifies fewer violations as serious, according to OSHA officials. [End of figure] With respect to federal enforcement efforts, OSHA has taken additional steps to assess the effectiveness of enforcement efforts overall. In summer 2012, OSHA began discussions with the National Advisory Committee on Occupational Safety and Health (NACOSH) on ways to determine the outcomes of specific enforcement activities.[Footnote 26] Labor, meanwhile, has called on all its agencies to intensify their focus on assessing outcomes. As part of its strategic planning, Labor in 2010 announced a new approach for measuring outcomes for worker protection. The document states that a single numerical target, or even a range, fails to recognize the sustained effort required to truly change behavior.[Footnote 27] It calls on OSHA to use statistical techniques such as random sampling and stratified random sampling, and social science research methods, to collect and analyze data for assessing performance. OSHA acknowledged in its revised fiscal year 2011 Operating Plan that it has not devoted substantial resources to evaluation activities in the past and, as a result, knows relatively little about the effectiveness of its policies, programs, and strategies. OSHA is currently involved in several studies in an effort to address this knowledge gap. In fiscal year 2011, OSHA began working with researchers at the Rand Corporation to evaluate the ripple, or deterrent, effects of enforcement and of providing assistance and education to employers on future compliance with safety and health standards. Another recently issued report by Labor's Office of Inspector General shows that OSHA knows little about the results of its Site Specific Targeting program and so cannot be sure it is effective.[Footnote 28] OSHA is currently working with Labor's new chief evaluation officer and a consulting firm to design and conduct an evaluation of this program, in which OSHA selects a sample of non- construction worksites for inspections based on their high rates of injuries and illnesses. OSHA plans to compare the impact of these inspections to the impact of an OSHA letter notifying employers of their high injury and illness rates and urging action, or a combination of both, on employer compliance and injury and illness rates. As part of this study, OSHA selected a sample of 840 worksites for inspection in 2011 and 2012.[Footnote 29] OSHA Does Not Use Existing Data to Assess Enforcement Effectiveness: OSHA is missing opportunities to use data already in hand to assess the effectiveness of federal and state enforcement efforts. For example, OSHA does not analyze data from its audits of its offices and of state-run programs, and information on its emphasis programs. According to the Office of Management and Budget (OMB), the GPRA Modernization Act of 2010[Footnote 30] serves as a foundation for creating a culture in which data and empirical evidence play a greater role in management decisions and policy. OMB also encourages agencies to adopt increasingly effective practices.[Footnote 31] OSHA does not analyze data from its MAP audits of regional and area offices or its FAME audits of state-run programs to inform its planning or to share information across regions. While OSHA summarized the most common findings from its fiscal year 2011 MAP audits, it provided no analysis of the data, and no such analysis was performed in 2010 to look for shared deficiencies or national trends. Similarly, according to a national OSHA official who oversees state-run programs, the agency has not analyzed the results of its annual FAME reviews of state-run programs to identify systemic issues. In accordance with government standards for internal controls, findings from audits should be used to identify risks, analyze their effects, and correct identified deficiencies.[Footnote 32] In our interviews with regional officials, those in two regions said OSHA's national office does not analyze the results of their MAP audits, and officials in two other regions said they were unaware of any such analysis and have not received feedback from OSHA's national office on the results of their MAP audits. In September 2012, Labor's Office of Inspector General issued a report that shows national OSHA's Directorate of Evaluation and Analysis did not analyze and disseminate the results of MAP audits to inform management decision making and improve operations.[Footnote 33] The report stated that OSHA failed to emphasize the critical importance of these audits in providing OSHA information on its performance. OSHA collects information on activities conducted for NEPs each year, but officials said they do not evaluate these data to determine whether these programs were responsible for desired outcomes in regions and across states.[Footnote 34] In our previous work, we have emphasized the need for evaluative information to help decision makers determine whether, and why, a program is working well.[Footnote 35] Unlike measuring performance, which focuses on whether a program achieves its objectives, program evaluations can be used to isolate a program's impact from external factors that can influence outcomes and to identify adjustments that may improve results. At least one regional administrator said data from NEPs can be used to flag safety and health problems and to identify best practices to share with other employers. However, while national OSHA compiled information submitted by each regional office for NEP activities in fiscal years 2010 and 2011, it did not analyze the data or compare activities across regions. Officials said that variations from state to state make such evaluations difficult. OSHA instructions for NEPs state that OSHA's national office is to disseminate information to its regional offices on the results of these programs. But national officials responsible for overseeing NEPs told us that while they collect data--primarily metrics or numbers--on each NEP, these are not true evaluations. In its newly implemented NEP for nursing homes, for example, OSHA includes a section on evaluation which instructs its area and regional offices to collect data on the NEP's effectiveness, including outreach conducted and resulting new safety and health programs implemented. However, there is no discussion about how to evaluate whether the NEP was directly responsible for desired outcomes, such as a reduction in worker injuries caused by lifting patients or falls. This lack of evaluation hinders OSHA in determining whether the NEP effort was effective and in identifying adjustments to improve results. An academic expert and Labor consultant said there is a recurring lack of planning at OSHA for post-initiative evaluation, which hinders the agency's ability to determine whether its enforcement efforts, including NEPs, have an impact on outcomes, such as reducing occupational injuries. NIOSH, an agency that conducts research and advises OSHA on occupational safety and health issues, has faced challenges similar to OSHA's in assessing its effectiveness. To address this challenge, NIOSH contracted with the National Academies to evaluate eight NIOSH research programs and to help NIOSH hold those programs accountable for real outcomes that affect life and health.[Footnote 36] For these evaluations, NIOSH devised questions to be posed by participating researchers, such as whether external factors affected outcomes, and the degree to which a NIOSH program was responsible for any improvement. The evaluation acknowledged the difficulty in assessing the causal relationship between NIOSH's functions as a research agency and specific occupational safety and health outcomes, but NIOSH officials said it would be easier for OSHA to do this because OSHA has more concrete data, such as reports on its worksite inspections. Conclusions: OSHA and state-run programs conduct a number of enforcement activities to protect the nation's workers, but OSHA knows little about their effectiveness. While data on the number of inspections conducted and violations identified at worksites are useful ways to monitor federal and state enforcement activities, they do not provide the type of outcome-oriented information needed to determine which enforcement activities are most effective in ensuring employer compliance and reducing risks to workers. OSHA recently developed initiatives to better assess the effectiveness of federal and state enforcement activities, but few include the evaluative components needed to assist the agency in examining effectiveness. Also, national OSHA collects some relevant information through audits of its own offices and state- run programs, but it does not annually analyze that information to identify systemic problems and improvements. In addition, despite a 2010 directive calling for greater participation of the OSHA national office in regional office comprehensive audits, there is still minimal participation by the national office in these audits. Without the independent perspective of the national office, OSHA cannot assure that results of regional office audits of themselves are impartial and reasonable. Further, OSHA has not provided consistent guidance for conducting state EFAME audits that include on-site case file reviews similar to comprehensive MAP audits. Lack of such guidance may allow enforcement deficiencies to go undetected, increasing the risk of worker injuries, illnesses, and death. Enhancing monitoring of enforcement activities and determining which efforts work best must be a sustained priority if OSHA is to achieve the best results and succeed in its mission to protect U.S. workers. Recommendations for Executive Action: In order to improve OSHA's assessments of its effectiveness and its monitoring of federal and state enforcement efforts, we recommend that the Secretary of Labor direct the Assistant Secretary of Labor for Occupational Safety and Health to: 1. consistently incorporate outcomes in its assessments of enforcement activities; 2. annually analyze the results of its MAP and FAME audits to identify and address systemic problems and leading practices; 3. ensure that OSHA national staff participate in regional office comprehensive audits to enhance independence in accordance with OSHA's MAP 2010 directive; and: 4. provide consistent guidance for scheduling state EFAME audits to ensure that they are conducted on a regular basis and include mandatory on-site case file reviews. Agency Comments and Our Evaluation: We provided a draft of this report to the Secretary of Labor for review and comment. We received written comments from OSHA, which are reproduced in their entirety in appendix II. OSHA agreed with the essence of our recommendations and stated that it would carefully consider them. However, OSHA raised some concerns about them. For recommendation 1, OSHA expressed concern that it may overemphasize the significance of outcome measurements in determining the effectiveness of state safety and health programs. OSHA pointed out that it currently uses other measures in conjunction with outcome measures to evaluate effectiveness. We agree that using other measures to evaluate state-run programs is useful but note that OSHA has not linked them to its assessments of effectiveness. We continue to believe that OSHA's assessments would benefit from further analysis of the results, or outcomes, of both state and federal enforcement activities. We revised our report to clarify that we do not think OSHA should stop measuring activities. For recommendations 2 and 3, OSHA stated that it has taken several steps to improve its analysis of audits and enhance resources assigned to the program. For recommendation 4, OSHA stated that it updates guidance for FAME reviews each year and is updating the State Plan Policies and Procedures Manual to include these revisions and guidance for on-site case file reviews. While we agree with OSHA that providing annual guidance to focus audits on areas of concern is valid, OSHA has not yet provided formal guidance for how often enhanced audits of state- run programs (EFAMEs) with on-site case file reviews will be conducted. We revised the wording of our recommendation to refer specifically to the scheduling of these EFAMEs. Finally, OSHA noted that our draft report did not adequately acknowledge the state-plan states' distinct role in directly and closely overseeing their own performance. We added further acknowledgment of the states' authority in this area to our report. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Secretary of Labor, and other interested parties. In addition, the report will be available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff members have any questions about this report, please contact Revae Moran at (202) 512-7215 or moranr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in Appendix III. Signed by: Revae Moran, Director: Education, Workforce, and Income Security Issues: [End of section] Appendix I: Scope and Methodology: The objectives of our review were to examine 1) how the Department of Labor's (Labor) Occupational Safety and Health Administration's (OSHA) monitoring of its own and state enforcement efforts compares, and 2) recent steps OSHA has taken to evaluate the effectiveness of federal and state enforcement efforts. To address our first objective, we reviewed Labor's and OSHA's policies and procedures for monitoring federal and state enforcement activities, including OSHA manuals, directives, and memoranda for its 10 regional offices, 87 area offices, and the 27 state-run programs. We also reviewed relevant federal laws and regulations. We analyzed audits of OSHA regional and area offices, which assess their activities in relation to OSHA policies and procedures and identify best practices and deficiencies in performance, such as in targeting and scheduling inspections; conducting fatality investigations; case file documentation; and outreach to employers, workers, and other stakeholders. We also analyzed OSHA's audits of state-run programs, which assess each state's progress toward meeting its individual performance goals and compliance with statutory and regulatory requirements. To further evaluate OSHA's audit processes, we compared these processes to criteria for independent review in GAO's Government Auditing Standards.[Footnote 37] We conducted site visits to three OSHA regional offices: Region 5 in Chicago, Illinois; Region 3 in Philadelphia, Pennsylvania; and Region 8 in Denver, Colorado. We selected these regional offices for site visits using the following criteria: the size of the region based on total number of worksite inspections conducted annually (one region with a small number of inspections, one with a medium number, and one with a large number); geography (offices located in different parts of the country); and the mix of states with federal-and state-provided enforcement within the region (each region selected has at least two states of each type). At each regional office we visited, we interviewed the regional or assistant regional administrator, program managers for OSHA enforcement and for state-run programs, and administrators in two of the region's area offices: one area office with oversight responsibility for state-run program(s) and one with oversight responsibility for state(s) for which OSHA provides enforcement. We selected area offices that oversee state-run programs for site visits to include a range of states based on total number of annual worksite inspections: Virginia (a large number of inspections), Indiana (a medium number), and Wyoming (a small number).[Footnote 38] In Region 8, we interviewed Denver regional office officials with direct oversight responsibility for Wyoming because there is no area office in that state. To obtain a national perspective, we conducted semi- structured phone interviews with the remaining seven regional office administrators or their managers about how they monitor and oversee enforcement efforts and each office's processes for overseeing the OSHA offices and state-run programs in its region. From each region, we collected key documents, including recent audit reports on regional and area offices, interim and annual reviews of states, inspection tracking reports and interim tracking of other key measures, and logs of complaints about state-run programs. To address our second objective, we interviewed the heads of directorates in OSHA's national office about their roles in monitoring enforcement and in planning for greater effectiveness, including the directorates responsible for state-run programs, overall enforcement, strategic planning, and evaluation. To obtain information from outside parties about OSHA's oversight of federal and state enforcement, we interviewed Labor officials who work with OSHA on its performance planning and research agenda; officers of the Occupational Safety and Health State Plan Association (OSHSPA), which represents the interests of states with state-run programs; and officials at the Department of Health and Human Services' National Institute for Occupational Safety and Health (NIOSH), which conducts research, experiments, and demonstrations relating to occupational safety and health. In addition, we interviewed selected academic experts and stakeholders about Labor and OSHA performance. We identified these academic experts and stakeholders through a literature search, as well as from recommendations of other academic experts versed in OSHA and worker protection issues and the recommendations of OSHA and NIOSH officials. Finally, we reviewed relevant federal laws and regulations, including the Occupational Safety and Health Act of 1970; the Government Performance and Results Act of 1993 (GPRA); and the GPRA Modernization Act of 2010.[Footnote 39] We reviewed OSHA's use of performance goals and measures in accordance with criteria in Office of Management and Budget Circular A-11, Labor's current 5-year strategic plan, and other agency plans for performance.[Footnote 40] We also reviewed previous GAO reports and NIOSH's 2004-2008 evaluation of its own effectiveness. [Footnote 41] We assessed the reliability of data we used from the OSHA Integrated Management Information System and OSHA Information System by (1) reviewing existing information about the data and the systems that produced them, (2) interviewing agency officials knowledgeable about the data, and (3) reviewing these data for obvious inconsistency errors. We determined that the data were sufficiently reliable for the purpose of this report. We conducted this performance audit from November 2011 through January 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Comments from the Department of Labor: U.S. Department of Labor: Assistant Secretary for Occupational Safety and Health: Washington, D.C. 20210 January 4, 2013: Ms. Revae E. Moran, Director: Education, Workforce, and Income Security Issues: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Ms. Moran: Thank you for the opportunity to comment on the Government Accountability Office's (GAO) proposed report, Workplace Safety and health: Further Steps by OSHA Would Enhance Monitoring of Enforcement and Effectiveness. The following comments are submitted on behalf of the Department of Labor's (DOL) Occupational Safety and Health Administration (OSHA). OSHA appreciates your detailed review of OSHA's monitoring process of both our federal and state enforcement activities. The Agency is committed to assessing and enhancing the effectiveness of its enforcement efforts, and we welcome your suggestions. OSHA agrees with the essence of the GAO's recommendations, including your analysis and suggestions on how to better use our data in addressing the many challenges we face when fulfilling our obligations to assure, as far as possible through the enforcement of our standards, safe and healthful working conditions for working men and women. OSHA also understands and appreciates the different strengths that diverse states bring to the national effort to reduce workplace injuries and illnesses, and in no way expects the states' policies, measures or trends to be identical to those of federal OSHA. Regarding Recommendation #1, OSHA is concerned that the draft report may overemphasize the significance of outcome measurements, such as injury and fatality rates, in determining the effectiveness of state safety and health programs. OSHA currently uses outcome measures in conjunction with other measures to evaluate effectiveness. However, assessing the quality and effectiveness of state plans by relying primarily on a system of outcome measures would be problematic for several reasons. Congress required OSHA to approve state plans and ensure that they remain "at least as effective" as the federal program. Although section 18 of the Occupational Safety and Health Act of 1970 ("the Act") does not define the term "at least as effective," the Act requires federal OSHA to conduct a "continuing evaluation of the manner in which each State ... is carrying out such plan." It has been OSHA's experience that injury rates, taken alone or over a relatively short period of time, may not be a reliable indicator of program quality. Unrelated factors such as changes in economic activity, changes in industry mix, and the composition of the working population, often affect these rates, or conversely, rates may remain low simply because a particular hazard or a particular program deficiency has not yet resulted in major accidents. Thus, for example, a number of the serious issues that OSHA identified in the Nevada plan in 2009, and in other plans as a result of subsequent Federal Annual Monitoring and Evaluations (FAMEs), would not have been identified had OSHA relied solely on injury, illness, and fatality rates as exclusive measures of effectiveness. Overreliance on injury rates, therefore, would not adequately address the requirements of the Act. Injury and illness rates reflect general trends and help measure the effects of OSHA intervention over long periods; the recent RAND study, for instance, measured injury rates over a period of four years. However, any audit or evaluation system intended to produce annual recommendations for program adjustment and improvement must largely be based on appropriate activity or performance measures. OSHA therefore believes that appropriate activity or performance measures are indispensable in determining states' effectiveness in carrying out their plans. OSHA would point out that its activity measures are not solely counts of numbers of inspections or other activities; they focus on the timeliness of responses to complaints, fatalities, and other events; on the preservation of employee and employer rights, including the protection of whistleblowers; on the ability of states to target their inspections to those workplaces where hazards are likely to occur; and on the actions taken when hazards are discovered. A state's effectiveness in ensuring worker rights and promoting similar objectives of the OSHA program cannot easily be measured by injury rates. OSHA believes that these and other factors, as set out in the Act, must also be considered in evaluating overall effectiveness. The DOL Strategic Plan notes that if agencies are doing their jobs properly, producing outputs in a sufficient quantity should produce the desired outcomes. Despite these difficulties, OSHA is currently engaged in an unprecedented effort to evaluate the impact of its activities and develop outcome measures that could help measure the effectiveness of OSHA's enforcement program and other related efforts. It should also be noted that OSHA does review trends in state plans in several ways. For the FY 2011 FAMEs, a summary was compiled that highlighted the handful of state plans with the most significant issues and identified those issues. We also routinely look at various trends in the state plans including staffing, budget expenditures, other measures including penalties, total violations per inspection, and measures related to their whistleblower programs. As GAO notes in the report, OSHA and DOL are working on several initiatives that will employ outcome measures to better measure effectiveness, not only in OSHA's overall program, but also as part of our National Emphasis Programs. In addition to those evaluation studies conducted at DOL, OSHA utilizes evidence and findings from effectiveness evaluations conducted by outside experts. For example, there have been three recent peer-reviewed academic studies that provide evidence that OSHA's enforcement activities are associated with lower injury rates, number of injury claims, and/or lost workdays. The first study was done by faculty at the University of California, Berkeley and Harvard Business Schools, and published in Science Magazine in May, 2012. Those researchers found that Cal/OSHA programmed inspections reduced injury rates by 9.4% and workers' compensation costs by 26% in the four years following the inspection, with no impact on employment, sales, credit rating or firm survival. The second study, conducted by the Safety and Health Assessment and Research for Prevention (SHARP) Program of the Washington State Department of Labor and Industries and presented at an OSHSPA meeting, examined the effectiveness of inspections and free consultations among fixed site and non-fixed site (often construction) employers. For those inspections resulting in a citation by Washington State OSHA, lost workday claims fell 22% during the following year; if an employer had an inspection but no citation, the claims fell about 7%, compared with the baseline (no OSHA inspection) of 2%. The third study, conducted by researchers affiliated with the Rand Corporation, examined the impact of federal OSHA inspections on workers' compensation claims among employees of Pennsylvania manufacturing firms. This study found that following inspections in which OSHA levies a penalty, injury claims fall by 19-24% per year during the next two years. With respect to Recommendations #2 and # 3, regarding the MAP and FAME, OSHA has already taken several steps to improve and strengthen its audit process, as well as enhance the resources assigned to the program. OSHA's Directorate of Evaluation and Analysis (DEA) will disseminate MAP analyses to OSHA management, and a report of findings for the audits initiated during each fiscal year is now being annually prepared and transmitted to OSHA senior managers. In addition, DEA management is now required to continually identify best practices and deficiencies in performance and share these with OSHA managers with the goal of improving program results. The Agency has designated a field audit team leader to make improvements to this program. Consistent with Recommendation #3, DEA has implemented and prioritized the development and enforcement of procedures for regional audits, including the participation of national office staff as independent auditors. OSHA has also developed a two-day training session for regional audit coordinators which is planned for the second quarter of FY2013. With regard to Recommendation #4, OSHA implemented revised guidance for the FY 2009 FAME in response to the problems that arose in Nevada. OSHA issues FAME guidance each year that further hones the ideas incorporated in the FY 2009 guidance, including lessons learned. OSHA is updating the State Plan Policies and Procedures Manual to include the revised oversight process and provide guidance regarding on-site case file reviews. However, in order to focus on critical programs, such as the Whistleblower Protection Program, OSHA will also build into the process the ability to add an area of special focus to the FAME. It is difficult to understand why this would be called inconsistent, or how such enhancements "allow enforcement deficiencies to go undetected, increasing the risk of worker injuries, illnesses and death." Finally, OSHA would also like to note that this report, in several instances, draws direct comparisons between the frequency of OSHA's oversight of federal jurisdiction performance and the frequency of monitoring of state plan performance without acknowledging the state plans' distinct role in directly and closely overseeing their own performance. For example, while OSHA may only formally monitor state plan performance metrics on a quarterly basis, the states themselves manage and oversee their own performance on a more frequent basis, often weekly to assess certain performance measures. OSHA will carefully consider all four GAO recommendations regarding oversight and monitoring of enforcement efforts. The Agency strives continually to strengthen our operations and performance, while focusing our limited resources on OSHA's mission to keep America's workers safe and healthy. OSHA appreciates the opportunity to review and respond to GAO's draft report. Sincerely, Signed by: David Michaels, PhD, MPH: [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Revae E. Moran, (202) 512-7215 or moranr@gao.gov: Staff Acknowledgments: In addition to the contact named above, Assistant Director Mary Crenshaw, Matthew Alemu, Susan Aschoff, and Paul Schearf made significant contributions to this report. Assistance, expertise, and guidance were provided by James Bennett, Ted A. Burik, Sarah Cornetto, Meeta Engle, Gretta Goodwin, Isabella P. Johnson, Kathy Leslie, Luann Moy, Carol Patey, Bryan Rogowski, Linda Siegel, and Kathleen van Gelder. [End of section] Related GAO Products: Workplace Safety and Health: Multiple Challenges Lengthen OSHA's Standard Setting. [hyperlink, http://www.gao.gov/products/GAO-12-330]. Washington, D.C.: April 2, 2012. Workplace Safety and Health: Enhancing OSHA's Records Audit Process Could Improve the Accuracy of Worker Injury and Illness Data. [hyperlink, http://www.gao.gov/products/GAO-10-10]. Washington, D.C.: October 15, 2009. OSHA's Voluntary Protection Programs: Improved Oversight and Controls Would Better Ensure Program Quality. [hyperlink, http://www.gao.gov/products/GAO-09-395]. Washington, D.C.: May 20, 2009. Whistleblower Protection Program: Better Data and Improved Oversight Would Help Ensure Program Quality and Consistency. [hyperlink, http://www.gao.gov/products/GAO-09-106]. Washington, D.C.: January 27, 2009. Workplace Safety and Health: Safety in the Meat and Poultry Industry, while Improving, Could Be Further Strengthened. [hyperlink, http://www.gao.gov/products/GAO-05-96]. Washington, D.C.: January 12, 2005. Workplace Safety and Health: OSHA's Oversight of Its Civil Penalty Determination and Violation Abatement Processes Has Limitations. [hyperlink, http://www.gao.gov/products/GAO-04-920]. Washington, D.C.: August 13, 2004. Workplace Safety and Health: OSHA Can Strengthen Enforcement through Improved Program Management. [hyperlink, http://www.gao.gov/products/GAO-03-45]. Washington, D.C.: November 22, 2002. Occupational Safety and Health: Efforts to Obtain Establishment- Specific Data on Injuries and Illnesses. [hyperlink, http://www.gao.gov/products/GAO/HEHS-98-122]. Washington, D.C.: May 22, 1998. Occupational Safety and Health: Changes Needed in the Combined Federal and State Approach. [hyperlink, http://www.gao.gov/products/GAO/HEHS-94-10]. Washington, D.C.: February 28, 1994. [End of section] Footnotes: [1] Pub. L. No. 91-596, 84 Stat. 1590, codified as amended at 29 U.S.C. §§ 553, 651-78. [2] Although OSHA refers to states with approved state plans as "state- plan states," in this report, we refer to them as "state-run programs." [3] In this report, we refer to federal audits and state audits, although state audits are more commonly called evaluations in OSHA documents. [4] NIOSH was created by the OSH Act to conduct research, experiments, and demonstrations relating to occupational safety and health, among other responsibilities. [5] We selected academic experts and stakeholders for interviews based on the results of a literature search, the recommendations of other academicians and experts versed in OSHA and labor issues, and the recommendations of OSHA and NIOSH officials. [6] See 29 U.S.C. §§ 655, 657-59, 666. The OSH Act also authorizes states to assume responsibility for setting and enforcing their own standards by submitting a state plan to OSHA for approval. 29 U.S.C. §§ 667(b). Under the OSH Act, "state" is defined to include the District of Columbia, Puerto Rico, the Virgin Islands, American Samoa, Guam, and the Trust Territory of the Pacific Islands. See 29 U.S.C. § 652(7). [7] 29 U.S.C. § 667(c).OSHA's regulations further specify various "indices of effectiveness" that OSHA will use in making this determination. 29 C.F.R. § 1902.4. [8] 29 U.S.C. § 667(f), 29 C.F.R. part 1954. If OSHA determines that a state-run program fails to comply substantially with any provision of the state plan, including the criteria required by the OSH Act, it may withdraw approval of the plan. 29 U.S.C. § 667(f); 29 C.F.R. § 1955.3(a)(3). [9] OSHA does not enforce standards for state and local public-sector workplaces in any state because the OSH Act does not apply to state and local government employers. 29 U.S.C. § 652(5). Federal agencies, with some exceptions, are generally required to establish and maintain their own comprehensive occupational safety and health programs consistent with OSHA's regulations. OSHA is generally authorized to conduct inspections of federal workplaces, although it does not impose fines for violations of occupational safety and health standards. 29 U.S.C. § 668, 29 C.F.R. part 1960, Executive Order 12196, 45 Fed. Reg. 12,769 (Feb. 26, 1980). [10] States that choose to operate their own state-run programs are required to cover state and local government workers. 29 U.S.C. § 667(c)(6). Five states (Connecticut, Illinois, New Jersey, New York, and the Virgin Islands) have state plans that only include state and local government workers, not those in the private sector. [11] Section 23(g) of the OSH Act authorizes OSHA to provide these grants to states to assist with the administration and enforcement of state-run programs. See 29 U.S.C. § 672(g). [12] These activities, among others, are required by the OSH Act and OSHA's regulations. See generally 29 U.S.C. § 667(c) and 29 C.F.R. parts 1902, 1953, 1954, and 1956. [13] OSHA also has four district offices that are subordinate to an area office. [14] The national average is an average of state-run program and federal data combined. [15] For purposes of assessing penalties, the OSH Act provides that a serious violation exists if there is a substantial probability that death or serious physical harm could result, unless the employer did not, and could not with the exercise of reasonable diligence, know of the presence of the violation. 29 U.S.C. § 666(k). [16] The general duty clause requires employers to provide a workplace free from recognized hazards that are causing, or are likely to cause, death or serious physical harm to their employees. 29 U.S.C. § 654(a)(1). In certain circumstances, OSHA may issue citations to employers for violating this requirement, even if no specific health or safety standard applies. 29 U.S.C. § 658(a). States operating state- run programs may also enact and enforce analogous provisions under state law. [17] During a case file review, OSHA officials select a sample of inspection case files and review the documents in each to ensure case file policy was followed and to identify any issues with the inspection process and follow-up. [18] GAO, Government Auditing Standards, 2011 Revision, [hyperlink, http://www.gao.gov/products/GAO-12-331G], Sections 3.02 and 3.04 (Washington, D.C.: December 2011). [19] In fall 2010, OSHA's national office began requiring its regional offices to submit the full audit report to the national office rather than a summary. Previously, OSHA required regional offices to submit a summary of their audit findings to the national office. [20] OSHA's Whistleblower Protection Program enforces the "whistleblower protection" provisions of a variety of federal laws, which generally prohibit employers from discriminating against employees for taking certain actions protected by these laws. According to OSHA, the program enforces the whistleblower provisions of 21 federal workplace safety, airline, commercial motor carrier, consumer product, environmental, financial reform, food safety, health care reform, nuclear, pipeline, public transportation agency, railroad, maritime, and securities laws. Employee actions protected by these laws include, but are not limited to, reporting a work-related injury or illness or reporting a violation of the laws. [21] After receiving stakeholder feedback, OSHA added three additional measures for a total of 18. [22] U.S. Department of Labor Office of Inspector General-Office of Audit, OSHA Has Not Determined If State OSH Programs Are at Least as Effective in Improving Workplace Safety and Health as Federal OSHA's Programs, 02-11-201-10-105 (Washington, D.C.: March 31, 2011). [23] California's state-run program is an exception: The state has the highest penalty amounts in the country. According to its fiscal year 2011 State Activity Mandated Measures report, its average initial penalty for a serious violation was $6,390 compared to $1,680 for OSHA. However, state program staff have previously raised concerns that penalties which are appealed by employers to the California Occupational Safety and Health Appeals Board are significantly reduced, prompting a study by OSHA's Region 9 and the California Senate Office of Research. The research office reviewed appeals of a certain type of citation and found that in contested cases where the citation was upheld, the appeals board judges reduced penalties more than 87 percent of the time and, when a penalty was reduced, it was reduced by at least 75 percent of the initial amount in over half the cases. [24] Federal penalties must fall within limits specified in the OSH Act, 29 U.S.C. § 666. To determine the amount of an individual penalty, an inspector applies a series of factors that can result in a reduction or an increase in the penalty. For example, a penalty for a small employer may be reduced based on the size of the employer or a penalty may be increased based on the gravity of the safety hazard identified. Effective Oct. 1, 2010, OSHA revised these factors to increase penalties. On April 1, 2012, OSHA readjusted one of the factors to reduce penalties for small (1 to 25 employees) businesses. [25] Enforcement decisions can be appealed by employers, which in some cases may result in reduced penalties. Citations issued by OSHA can be appealed to the Occupational Safety and Health Review Commission, whose rulings may then be appealed to the appropriate U.S. Court of Appeals. State-run programs have a similar review process, in which employers can appeal to a state review board or equivalent authority. [26] The 12-member NACOSH was established by the OSH Act to advise the Secretaries of Labor and Health and Human Services on matters relating to the administration of the OSH Act. 29 U.S.C. § 656. [27] Department of Labor, A New Approach to Measuring the Performance of U.S. Department of Labor Worker Protection Agencies (June 28, 2010). [28] The report, released in September 2012, shows some of the industries and worksites at which the most serious injuries and illnesses occurred were outside the Site Specific Targeting program and not always inspected. The report noted the ongoing evaluation study discussed here. U.S. Department of Labor Office of Inspector General-Office of Audit, OSHA's Site Specific Targeting Program Has Limitations on Targeting and Inspecting High-Risk Worksites, 02-12-202- 10-105 (Washington, D.C.: September 28, 2012). [29] According to OSHA Directive 11-03 (CPL 02), states with state-run programs are expected to conduct similar targeting efforts of their own. [30] The GPRA Modernization Act of 2010 amended the Government Performance and Results Act of 1993 (GPRA). Pub. L. No. 103-62, 107 Stat. 285, as amended by Pub. L. No. 111-352, 124 Stat. 3866 (2011). [31] OMB Circular No. A-11, Sec.200, p. 200-4. [32] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999) [33] The report also shows that there was a lack of consistency in how audits were conducted region to region and a lack of training for OSHA staff on how to conduct them. U.S. Department of Labor Office of Inspector General-Office of Audit, OSHA Needs to Improve Oversight Over the Management Accountability Program, 02-12-204-10-105 (Washington, D.C.: September 27, 2012). [34] For fiscal year 2012, OSHA had approximately one dozen NEPs, including some that had been in effect for several years. An OSHA official said OSHA has a preliminary report on its NEP on amputations. [35] We defined a program as any activity, project, function, or policy that has an identifiable purpose or set of objectives. GAO, Performance Measurement and Evaluation: Definitions and Relationships, [hyperlink, http://www.gao.gov/products/GAO-11-646SP] (Washington, D.C.: May 2011). [36] Institute of Medicine and National Research Council of the National Academies, Evaluating Occupational Health and Safety Research Programs: Framework and Next Steps (Washington, D.C.: The National Academies Press, 2009). The study's recommendations to NIOSH were to 1) develop a system for regular ongoing external evaluations to complement NIOSH's self-assessments; 2) continue to improve its process for listening to workers, employers, health care providers, and others to identify needed intervention and get early feedback; 3) increase and improve its surveillance of work-related injuries, illnesses, and other working conditions to inform future evaluations; and 4) integrate internal and external research in its strategic planning. [37] GAO, Government Auditing Standards, 2011 Revision, [hyperlink, http://www.gao.gov/products/GAO-12-331G], Sections 3.02 and 3.04 (Washington, D.C.: December 2011). [38] Two of our six area office visits were conducted by phone: Indianapolis, Indiana in Region 5 and Norfolk, Virginia in Region 3. [39] Occupational Safety and Health Act of 1970, Pub. L. No. 91-596, 84 Stat. 1590; Government Performance and Results Act of 1993, Pub. L. No. 103-62, 107 Stat. 285; and GPRA Modernization Act of 2010, Pub. L. No. 111-352, 124 Stat. 3866 (2011). [40] OMB Circular N. A-11, Sect. 200. [41] Institute of Medicine and National Research Council of the National Academies, Evaluating Occupational Health and Safety Research Programs: Framework and Next Steps (Washington, D.C.: The National Academies Press, 2009). [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]