This is the accessible text file for GAO report number GAO-02-804 entitled 'Performance Management Systems: IRS’s Systems for Frontline Employees and Managers Align with Strategic Goals but Improvements Can Be Made' which was released on July 12, 2002. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States General Accounting Office: GAO: Report to Congressional Requesters: July 2002: Performance Management Systems: IRS’s Systems for Frontline Employees and Managers Align with Strategic Goals but Improvements Can Be Made: GAO-02-804: Contents: Letter: Results in Brief: Background: Critical Job Responsibilities Align with Strategic Goals but Do Not Always Align with Other Elements of the Frontline Employee Performance Management System: Enforcement Group Managers’ Commitments Align with IRS’s Strategic Goals but Are Not Always Written So that Raters Can Hold Managers Accountable: Raters Generally Provided Feedback to Enforcement Group Managers on Meeting Critical Job Responsibilities and Commitments but Provided More Extensive Feedback on Some Critical Job Responsibilities Than Others: IRS Has Made Progress but Could Further Enhance Its New Managers’ and Frontline Employees’ Performance Management Systems: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Objectives: Scope and Methodology: Appendix II: GAO Sampling Methodology: Sample Designs: Sample Disposition: Sampling Error: Nonsampling Error: Appendix III: Samples of Manager and Frontline Employee Evaluation Forms: Appendix IV: Examples of Evaluations Addressing Group Managers’ Responsibilities: Appendix V: Distribution of Raters’ Comments for Each Supporting Behavior: Appendix VI: Examples of Group Managers’ Commitments: Commitments that Were Specific, Outcome or Output Oriented, or Easily Monitored: Commitments that Were Not Specific, Outcome or Output Oriented, or Easily Monitored: Appendix VII: Comments from the Internal Revenue Service: Appendix VIII: GAO Contacts and Staff Acknowledgments: GAO Contacts: Acknowledgments: Tables: Table 1: Examples of Enforcement Group Managers’ Commitments that Align with IRS’s Strategic Goals: Table 2: Percentage of Commitments Reflecting Critical Job Responsibilities for Fiscal Years 2000 and 2001: Table 3: Percentage of Commitments Not Specific, Outcome/Output Oriented, or Easily Monitored for Fiscal Years 2000 and 2001: Table 4: Estimated Percentage of Fiscal Years 2000 and 2001 Written Evaluations in Which Raters Assessed Critical Job Responsibilities by Number of Responsibilities: Table 5: Estimated Percentage of Enforcement Group Managers Reporting Verbal Feedback by Number of Critical Job Responsibilities for Fiscal Year 2001: Table 6: Estimated Percentage of Fiscal Year 2000 and 2001 Evaluations Assessing Enforcement Group Managers’ Performance in Each Critical Job Responsibility: Table 7: Estimated Percentage of Enforcement Group Managers Reporting Feedback to a Great or Very Great Extent for the Critical Job Responsibilities for Fiscal Year 2001: Table 8: Estimated Percentage of Enforcement Group Managers Reporting that the Respective Critical Job Responsibilities Are Very Important or Important to Their Raters for Fiscal Year 2001: Table 9: Disposition of Sample Cases for Our Review of Group Manager Evaluations: Table 10: Disposition of Sample Cases for Our Survey of Group Managers’ Perceptions of Raters’ Feedback: Table 11: Examples of Critical Job Responsibilities, Supporting Behaviors, and Evaluation Narratives for Group Managers: Table 12: Distribution of Raters’ Comments: Figures: Figure 1: Alignment of Strategic Goals, Critical Job Responsibilities, and Supporting Behaviors for Enforcement Group Managers: Figure 2: Misalignment of Critical Job Responsibility, Narrative Description, and Supporting Behaviors for Revenue Agents: Abbreviations: CQMS: Collections Quality Management System: IDPs: Individual Development Plans: IRM: Internal Revenue Manual: IRS: Internal Revenue Service: NTEU: National Treasury Employee Union: RRA 98: Restructuring and Reform Act of 1998: TFRP: Trust Fund Recovery Penalty: [End of section] United States General Accounting Office: Washington, DC 20548: July 12, 2002: The Honorable Bill Thomas: Chairman, Committee on Ways and Means: House of Representatives: The Honorable Amo Houghton: Chairman, Subcommittee on Oversight: Committee on Ways and Means: House of Representatives: Performance management systems can be powerful tools in helping an agency achieve its mission and ensuring employees at every level of the organization are working toward common ends. Performance management systems should help employees understand their responsibilities and how their day-to-day work contributes toward meeting their agency’s strategic goals as well as providing a mechanism for giving employees candid, specific feedback on how well they are meeting their rater’s expectations. For agencies like the Internal Revenue Service (IRS) that are undergoing a cultural change, performance management systems help reinforce behaviors and actions that support the agency’s mission. [Footnote 1] In February 2000, IRS implemented a new performance management system for its executives and managers and in October 2001 implemented a new performance management system for frontline employees. These systems were built upon IRS’s three strategic organizational goals—”top- quality service to each taxpayer in every interaction,” “top-quality service to all taxpayers through the fair and uniform application of the law,” and “productivity through a quality work environment”—and the corresponding balanced performance measures of customer satisfaction, business results (quality and quantity), and employee satisfaction that are applied to all organizational units, from IRS-wide down to the group level. IRS executives and managers are evaluated on five critical job responsibilities (customer satisfaction, business results, employee satisfaction, leadership, and equal employment opportunity) and on written commitments they develop on the specific actions they will take to support IRS’s strategic goals over the course of the evaluation period. [Footnote 2] Frontline employees are evaluated against five critical job responsibilities, (customer satisfaction—knowledge, customer satisfaction—application, business results—quality, and business results—efficiency, and employee satisfaction—employee contribution). Because of the challenging nature of implementing a new performance management system, you asked us to assess how well IRS is aligning its performance management systems with its strategic goals and is using these systems to hold managers and employees accountable for meeting those goals. Specifically, as agreed with your office, our objectives were to determine (1) the extent to which group managers’ and frontline employees’ critical job responsibilities and other elements of the employee performance management systems are structurally aligned with IRS’s strategic goals and organizational unit performance measures, (2) the extent to which group managers’ commitments align with IRS’s strategic goals and are written so that raters can hold managers accountable for meeting their commitments, (3) the extent to which raters provide feedback to enforcement group managers on their performance in meeting critical job responsibilities and commitments, and (4) whether IRS has adequate plans to monitor and assess the implementation of its performance management systems. To address these objectives, we reviewed documents related to IRS’s performance management systems and discussed the systems with officials responsible for developing and implementing them. We performed qualitative analyses of the extent to which the critical job responsibilities and supporting behaviors for enforcement group managers and frontline employees [Footnote 3] align with IRS’s strategic goals and organizational unit performance measures. We performed a content analysis of a representative sample of the written evaluations for 126 enforcement group managers for fiscal years 2000 and 2001. We also surveyed the enforcement group managers about their perceptions of the verbal and written feedback they received on their performance. [Footnote 4] We used information contained in IRS documents and guidance given to group managers and frontline employees as criteria for assessing those systems. We did our work between April 2001 and July 2002 in accordance with generally accepted government auditing standards. A more detailed description of our methodology is provided in appendix I and information on our sampling methodology is provided in appendix II. Results in Brief: IRS has established critical job responsibilities for group managers and frontline employees that align with each of the agency’s three strategic goals. In the group managers’ performance management system, all the supporting behaviors—those actions and competencies that the enforcement group manager is expected to demonstrate during the performance cycle—clearly align with the critical job responsibilities. However, for frontline employees, the supporting behaviors do not always align with IRS’s description of the critical job responsibilities. Misalignments occur when supporting behaviors reflect concerns not expressed in the description, when supporting behaviors that relate to a responsibility are located under other responsibilities, or when no supporting behavior exists to support the description. For example, the description of the customer satisfaction–application responsibility for revenue agents states that employees should communicate with taxpayers in a clear, user-friendly manner. However, the supporting behaviors focus more on IRS’s procedural requirements, such as the quality of internal work products, than on whether taxpayers received understandable audit reports. In addition to the misalignments in evaluating supporting behaviors, IRS has not fully aligned its frontline employee performance management system with its organizational unit performance measures. These misalignments can reduce the new system’s ability to promote frontline employees behaviors that would help IRS achieve its mission. During both fiscal years 2000 and 2001, almost all of the enforcement group managers’ commitments that we reviewed were aligned with IRS’s strategic goals. However, commitments were not always well written. About 85 percent of the commitments did not meet IRS’s criteria of being specific, outcome or output oriented, or easily monitored, although almost all met IRS’s criteria of being clear and achievable. Thus, even though these commitments may encourage desired behaviors, raters may have difficulty holding managers accountable for meeting the majority of their commitments because of the way they were written. Our review of enforcement group managers’ evaluations showed that raters generally provided feedback to managers on their performance in meeting critical job responsibilities and commitments. For example, during fiscal year 2000 and 2001, we estimate that more than 80 percent of the written performance evaluations assessed managers’ performance in meeting four or more of the five critical job responsibilities. In addition, about 76 percent of managers’ commitments were addressed in the written evaluations. In our survey of enforcement group managers, more than an estimated 60 percent of managers reported receiving verbal feedback in all five of the critical job responsibilities, and about half of managers said they received verbal feedback on how well they met most or all of their commitments. According to our analysis of written evaluations, raters provided feedback on the three critical job responsibilities related to IRS’s organizational goals in 90 percent or more of evaluations and provided feedback on leadership and equal employment opportunity in about 70 percent of the evaluations. In our survey, enforcement group managers reported that raters addressed business results more often than employee satisfaction or customer satisfaction during both verbal and written feedback. IRS senior executives said the relative emphasis raters were placing on the critical job responsibilities was in line with their expectations, and noted that a heavy emphasis on a subset of critical job responsibilities could lead managers away from the balanced behaviors IRS seeks to achieve. IRS senior executives have no firm plans to monitor how well the group managers’ and frontline employees’ systems are being implemented or to assess whether changes need to be made, even though IRS’s management processes call for obtaining data on how well programs are achieving their goals. As IRS continues to implement its performance management systems, IRS can ensure the systems continue to meet its needs by monitoring how well the systems provide useful feedback to managers and employees about their performance and assessing whether the systems encourage activities and behaviors consistent with its goals. Although IRS senior executives are currently satisfied with the implementation of the new systems, if the difference in raters’ emphasis on critical job responsibilities grows over time, IRS could find that enforcement group managers are receiving a skewed perception of the behaviors that IRS values. In addition, IRS can begin planning for a more thorough assessment of how well the systems are achieving their overall objectives, such as improving communications, encouraging performance that actually leads to organizational goals, and ensuring fair and consistent performance evaluations. We are making recommendations to the Commissioner of Internal Revenue that IRS improve the alignment between strategic goals and elements of the frontline employee performance management system and ensure the new performance management systems are working as intended. In a July 5, 2002 letter, the Commissioner agreed with the recommendations contained in this report. (See agency comments and our evaluation section and app. VII.) Background: The enactment of the IRS Restructuring and Reform Act of 1998 (RRA 98) signaled congressional concern that the IRS had been emphasizing revenue production at the expense of fairness and consideration of taxpayers. In response, the Commissioner of Internal Revenue sought to transform the agency’s culture to one that more equally embraced responding to legitimate taxpayer needs, ensuring compliance with tax laws, and considering employee needs as core organizational values. IRS has undertaken a number of long-range initiatives to make this transformation a reality, including: * establishing three strategic goals of “top-quality service to each taxpayer in every interaction,” “top-quality service to all taxpayers through the fair and uniform application of the law,” and “productivity through a quality work environment;” * developing a strategic planning and budgeting process that provides a framework for operating units to develop goals and action-oriented business plans that support the strategic goals; * using balanced performance measures corresponding to each of the three strategic goals—customer satisfaction, business results in terms of quality and quantity, and employee satisfaction—to measure the performance of every organizational unit in achieving those strategic goals; and; * establishing new performance management systems [Footnote 5] designed to promote employee behaviors and actions that support IRS’s strategic goals and the operational business plans that lay out each operating division’s strategy for achieving IRS’s strategic goals. IRS’s goal in redesigning, or modernizing, its performance management systems was to make the process more meaningful for all participants. The new managers’ and frontline employee performance management systems were required to include goals and objectives at the organizational, group, or individual level, and a retention standard [Footnote 6] to ensure that managers and frontline employees administer the tax laws fairly and equitably, protect taxpayers’ rights, and treat taxpayers with honesty, integrity, and respect. The systems were intended to provide a framework for managers and employees to improve communications, coordinate planning activities, align individual performance to organizational performance, and ensure fair and consistent performance evaluations. Further, the performance management systems were intended to accurately reflect employees’ performance, facilitate their development, and improve and enhance their work. As part of the new process, IRS establishes performance expectations at the beginning of the performance cycle that serve as the basis for the group managers’ and frontline employees’ annual performance evaluations. Beginning in fiscal year 2000, IRS began implementing its new performance management system for executives and managers. The key components of the new system for group managers are the critical job responsibilities (behaviors) and commitments (actions). The critical job responsibilities, which represent the new core values of IRS, replaced the general standards for group managers that had been in effect since 1984. They apply to all IRS executives and managers and are further defined by supporting behaviors–broad actions and competencies that IRS expects its executives and managers to demonstrate during the performance cycle. The commitments, which are specific to and developed by each manager, are distinct actions or desired results to be achieved during the performance period. Each commitment should relate to, and support, one or more of the critical job responsibilities. For group managers, the performance management process includes an annual performance evaluation—an assessment of performance supported by written narrative commentary. While there is no current IRS requirement that raters address managers’ performance in each critical job responsibility or each commitment individually in the written annual evaluation, raters are expected to provide a summary of managers’ performance that accurately describes how well managers performed in meeting both their critical job responsibilities and commitments. IRS began implementing the performance management system for frontline employees in fiscal year 2001. The frontline employees’ system is conceptually similar to that for executives and managers but there are some differences. As with managers, frontline employees are evaluated on how well they met their critical job responsibilities. However, the responsibilities are tailored to reflect the activities and behaviors that employees with direct contact with taxpayers are expected to display. Each responsibility consists of a brief description of the key behaviors related to the responsibility and supporting behaviors, which IRS calls performance aspects, that further define the responsibility and demonstrate the expected level of performance. In addition, frontline employees are not required to develop written commitments about the specific actions they will take to support IRS’s strategic goals. IRS has not completed the redesign of its performance management systems. For example, IRS envisions that its performance management system for frontline employees will eventually include a requirement to develop commitments, mirroring the system for executives and managers. In addition, IRS expects to integrate the new performance management systems with its overall human resource systems. This integration would, for example, link evaluations to decisions about developmental needs, rewards and recognition, and compensation. IRS expects the complete redesign and implementation of both managers’ and frontline employees’ performance management systems to take about five years. Critical Job Responsibilities Align with Strategic Goals but Do Not Always Align with Other Elements of the Frontline Employee Performance Management System: IRS has established critical job responsibilities for managers and frontline employees that align with each of the agency’s three strategic goals. However, supporting behaviors described in the frontline employees’ performance management system do not always align with critical job responsibilities or with their organizational unit’s performance measures. Enforcement Group Managers’ Critical Job Responsibilities and Supporting Behaviors Align with Strategic Goals: IRS has established critical job responsibilities and supporting behaviors for group managers that align with the agency’s strategic goals. These responsibilities and supporting behaviors, described on the evaluation form as shown in appendix III, provide managers with a consistent message about how their daily activities are to reflect the organization’s core values. IRS’s performance management system for group managers assesses performance in five critical job responsibilities: customer satisfaction, business results, employee satisfaction, leadership, and equal employment opportunity. Of these five responsibilities, three align directly to IRS’s strategic goals, as shown in figure 1. For example, by establishing a critical job responsibility and supporting behaviors in customer satisfaction, IRS aligns managers’ performance to its strategic goal of “top-quality service to each taxpayer in every interaction.” Figure 1: Alignment of Strategic Goals, Critical Job Responsibilities, and Supporting Behaviors for Enforcement Group Managers: [See PDF for image] This figure illustrates the alignment of Strategic Goals, Critical Job Responsibilities, and Supporting Behaviors for Enforcement Group Managers, as follows: Strategic goal: Top-quality service to each taxpayer in every interaction; Critical job responsibility: Customer satisfaction; Example of supporting behavior: Listen to customers, constantly gathering their feedback, actively seeking to identify their needs and expectations. Strategic goal: Top-quality service to all taxpayers through the fair and uniform application of the law; Critical job responsibility: Business results (quality and quantity); Example of supporting behavior: Develops and executes plans to achieve organizational goals, leveraging resources (human, financial) to maximize efficiency). Strategic goal: Productivity through a quality work environment; Critical job responsibility: Employee satisfaction; Example of supporting behavior: Promotes cooperation and teamwork among employees. Coaches and develops employees so that they realize their full potential. Source: GAO analysis of IRS’s group manager performance management system. [End of figure] The other two critical job responsibilities, leadership and equal employment opportunity, support the agency’s strategic goals not because they directly address IRS’s strategic goals but because they reinforce behaviors that IRS considers necessary for organizational change and an open and fair work environment. Frontline Employees’ Critical Job Responsibilities Align with Strategic Goals but Not Always with Supporting Behaviors and Organizational Unit Performance Measures: As with enforcement group managers, the critical job responsibilities for frontline employees, shown on the evaluation form in appendix III, align with the agency’s strategic goals. IRS evaluates frontline employees against five critical job responsibilities: customer satisfaction—knowledge, customer satisfaction—application, business results—quality, business results—efficiency, and employee satisfaction—employee contribution. All five responsibilities directly align with IRS’s strategic goals. For example, the two critical job responsibilities addressing customer satisfaction (customer satisfaction—knowledge and customer satisfaction—application) align with IRS’s strategic goal of “top-quality service to each taxpayer in every interaction.” Similarly, the two critical job responsibilities addressing business results (business results—quality and business results—efficiency) align with IRS’s strategic goal of “top-quality service to all taxpayers through the fair and uniform application of the law.” Finally, the critical job responsibility addressing employee satisfaction—employee contribution aligns with IRS’s strategic goal of “productivity through a quality work environment.” Although the framework for IRS’s frontline employee performance management system supports the agency’s strategic goals, the description of the critical job responsibility given to IRS employees and the accompanying supporting behaviors do not always align with one another. This misalignment occurs when supporting behaviors reflect concerns not expressed in the description, when supporting behaviors that relate to a responsibility are located under other responsibilities, or when no supporting behavior exists to support the description. For example, under the customer satisfaction—application responsibility for revenue agents, IRS’s description states that employees should communicate with taxpayers in a clear, user-friendly manner. However, the related supporting behaviors focus more on IRS’s own procedural requirements and internal work products, such as the quality of audit workpapers, than on whether taxpayers received understandable reports of their audit, as shown in figure 2. These supporting behaviors would better align with the business results—quality critical job responsibility, which covers how well employees accomplish tasks within IRS’s procedures and established guidelines. Figure 2: Misalignment of Critical Job Responsibility, Narrative Description, and Supporting Behaviors for Revenue Agents: [See PDF for image] This figure is an illustration of the misalignment of critical job responsibility, narrative description, and supporting behaviors for revenue agents, as follows: Strategic goal: Top-quality service to each taxpayer in every interaction; Critical job responsibility: Customer satisfaction - Application; Description of critical job responsibility in the area of written communication: Communication to the customer is appropriate for the issue and encourages voluntary compliance; Example of supporting behaviors: Aligned: Written Communication: Produces written products that are complete, accurate, understandable and presented in a logical manner according to procedures and guidelines; Example of supporting behaviors: misaligned: Prepares workpapers that are sufficiently detailed, easy to follow and clearly reflect tasks completed and conclusions reached. Source: GAO analysis of IRS’s frontline employee performance management system. [End of figure] Also, for the customer satisfaction—knowledge responsibility for both revenue agents and revenue officers, the description states that “correct interpretation of laws, rules, and regulations” is a key component of the critical job responsibility. However, for revenue agents, the supporting behaviors that address “correct interpretation of laws” can also be found under two other responsibilities, customer satisfaction—application and business results—quality, even though correctly interpreting laws is not explicitly described as a key component of those responsibilities. For revenue officers, correctly interpreting laws is not directly addressed under any of the critical job responsibilities. In addition to assessing frontline employees individually using its performance management system, IRS uses organizational unit performance measures to assess how well frontline employees are collectively achieving the agency’s strategic goals. Unit performance measures provide aggregated feedback at different organizational levels on how well employees are achieving IRS’s strategic goals. For example, customer satisfaction surveys measure the quality of service employees provided to taxpayers and quality measurement systems assess whether employees followed IRS standards and procedures. In many cases, the activities addressed on these organizational unit performance measures align with the behaviors assessed in the frontline employee performance management system. However, in some cases, the unit performance measures and the frontline performance management system’s supporting behaviors do not align. We found seven activities relating to informing taxpayers of their rights, explaining the audit process, and being courteous that were addressed in organizational unit performance measures but not in the performance management system. For example, IRS does not explicitly evaluate revenue agents on whether they explained the audit process to taxpayers, an activity addressed on its examination customer satisfaction survey. During discussions with IRS senior executives, they agreed that the frontline employee performance management system should better align with organizational unit performance measures. Enforcement Group Managers’ Commitments Align with IRS’s Strategic Goals but Are Not Always Written So that Raters Can Hold Managers Accountable: According to our analysis of a sample of enforcement group managers’ evaluations during both fiscal years 2000 and 2001, almost all of the commitments managers developed aligned with IRS’s strategic goals but these commitments were not always written so that managers could be held accountable for meeting them because they were not specific, outcome or output oriented, or easily monitored. Commitments Align with Strategic Goals: Almost all of the commitments written by our sample of enforcement group managers aligned with their critical job responsibilities and thus ultimately with IRS’s strategic goals. Table 1 provides examples of commitments that align with strategic goals. Table 1: Examples of Enforcement Group Managers’ Commitments that Align with IRS’s Strategic Goals: Strategic goals: Top-quality service to each taxpayer in every interaction; Commitments: “I will ensure employees are prompt and professional in their dealings with taxpayers by being directly involved in examinations through observations and visits.” (improves customer satisfaction) Strategic goals: Top-quality service to all taxpayers through the fair and uniform application of the law; Commitments: “I will review quality measurement system results to identify deficient areas, address deficiencies at group meetings and take corrective actions as needed.” (improves business results) Strategic goals: Productivity through a quality work environment; Commitments: “I will perform evaluative reviews, such as case reviews, on-the-job visits and workload reviews, on each employee and provide immediate feedback on their performance.” (improves employee satisfaction) Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ commitments in the written performance evaluation. [End of table] Although commitments align with IRS’s strategic goals, our analysis of fiscal year 2000 and 2001 written performance evaluations shows that enforcement group managers’ commitments were not evenly distributed among the five critical job responsibilities. [Footnote 7] Employee satisfaction and business results were each addressed in more than a third of commitments each year, while a quarter addressed customer satisfaction. Leadership and equal employment opportunity were each addressed by fewer than 10 percent of the commitments in each year. Although these critical job responsibilities do not directly align with IRS’s strategic goals, IRS senior executives believe that effective leadership is an important component of changing organizational culture and supporting equal employment opportunity principles is important to establishing a fair work environment. Table 2 shows the percentage of commitments that focused on each responsibility. Table 2: Percentage of Commitments Reflecting Critical Job Responsibilities for Fiscal Years 2000 and 2001: Critical job responsibility: Customer satisfaction; Percentage of commitments, FY 2000: 24; Percentage of commitments, FY 2001: 25. Critical job responsibility: Business results; Percentage of commitments, FY 2000: 34; Percentage of commitments, FY 2001: 42. Critical job responsibility: Employee satisfaction; Percentage of commitments, FY 2000: 37; Percentage of commitments, FY 2001: 40. Critical job responsibility: Leadership; Percentage of commitments, FY 2000: 7; Percentage of commitments, FY 2001: 6. Critical job responsibility: Equal employment opportunity; Percentage of commitments, FY 2000: 7; Percentage of commitments, FY 2001: 4. Note: Because managers’ commitments might apply to more than critical job responsibility, percentages do not add to 100. Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ commitments in the written performance evaluation. [End of table] IRS senior executives were generally pleased with the distribution of enforcement group managers’ commitments among the critical job responsibilities. They said that the distribution of commitments among the three responsibilities aligned most directly with IRS’s strategic goals was relatively equal and that it was more difficult for managers to develop commitments for the other two responsibilities. Raters May Have Difficulty Holding Managers Accountable: In an attempt to assist managers in developing their commitments and hold them accountable for meeting commitments, IRS issued guidance in November, 2000 regarding criteria for well-constructed commitments. According to the guidance, well-constructed commitments should be clear, specific, achievable, outcome or output oriented, and easily monitored. Appendix I provides an explanation of how we used these criteria to assess enforcement group managers’ individual commitments. In keeping with the guidance for well-written commitments, we found that almost all of the enforcement group managers’ commitments in our sample of evaluations met two of IRS’s five criteria—being clear and achievable. For example, we concluded that the following commitments would be understood by both the manager and the rater, did not include jargon unfamiliar to IRS employees, and described actions that could realistically be accomplished within the normal work environment. “I will provide resources where necessary to ensure an effective filing season.” “I will encourage and support the development of employees through the use of individual development plans, as submitted, and the assignment of appropriate details whenever possible.” “I will work with my group members to ensure that our jointly developed FY 2000 action plan objectives are met or exceeded.” In contrast, our analysis showed that about 85 percent of the enforcement group managers’ commitments, for both rating years, did not meet the remaining three IRS criteria for well-written commitments of being specific, outcome or output oriented, or easily monitored. We found that the majority of commitments were broad, vague statements that did not give a relatively clear indication of the action that would be taken, and did not include relevant information such as the frequency of the action or what would be accomplished. Moreover, there was little focus on expected outcomes, such as the potential impact of actions on employees or taxpayers, or outputs, such as the number of activities to be accomplished, for which the manager’s performance in meeting the commitment could be assessed or measured. In IRS’s performance management system, raters and managers share responsibility for ensuring that commitments meet the criteria for well-written commitments—managers initially develop the commitments and raters review and, if necessary, revise them. The majority of group managers reported in our survey that their raters were actively involved in revising their commitments. Table 3: Percentage of Commitments Not Specific, Outcome/Output Oriented, or Easily Monitored for Fiscal Years 2000 and 2001: Criteria: Specific; Percentage of commitments not meeting criteria, FY 2000: 84; Percentage of commitments not meeting criteria, FY 2001: 85. Criteria: Outcome/output oriented; Percentage of commitments not meeting criteria, FY 2000: 84; Percentage of commitments not meeting criteria, FY 2001: 85. Criteria: Easily monitored; Percentage of commitments not meeting criteria, FY 2000: 85; Percentage of commitments not meeting criteria, FY 2001: 86. Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ commitments in the written performance evaluation. [End of table] The commitments that were not specific, outcome or output oriented, or easily monitored could, nevertheless, be encouraging desired behaviors. However, because so many commitments did not meet these criteria, raters may have difficulty holding enforcement group managers accountable for meeting the majority of their commitments because of the way they were written. The following two commitments illustrate a vague commitment and a specific, outcome/output oriented and easily monitored commitment: * “I will provide leadership by increased managerial involvement in my employees’ casework.” * “I will conduct quarterly reviews of casework focusing on IRM (Internal Revenue Manual) requirements and CQMS (Collections Quality Management System) to ensure timely closing of cases.” It would be difficult to definitively judge whether a manager met the first commitment because the statement is vague about what the manager intends to do. In contrast, the second commitment is explicit about the type of managerial involvement, the frequency of the reviews, the standards to be applied during the reviews, and the potential impact of the manager’s involvement. A rater could monitor and assess whether the manager conducted these types of reviews as often as promised, and use timeliness statistics to assess the manager’s performance in actually closing cases in less time. Although performance management systems are inherently subjective to some degree because raters have to assess how well employees carry out their duties, raters also must make subjective judgments about whether commitments are met when those commitments are vague. When commitments are more explicit, there is a greater likelihood that both managers and raters will know and agree whether managers have met their commitments, and that commitments are specific, outcome or output oriented, and easily monitored. Appendix VI provides additional examples of commitments. Raters Generally Provided Feedback to Enforcement Group Managers on Meeting Critical Job Responsibilities and Commitments but Provided More Extensive Feedback on Some Critical Job Responsibilities Than Others: According to our review of fiscal year 2000 and 2001 written evaluations and our survey of enforcement group managers, raters generally provided feedback to managers on how well they met their critical job responsibilities and commitments by addressing them in written or verbal feedback. However, raters provided more extensive written and verbal feedback for some critical job responsibilities than for others. In addition, for each critical job responsibility, raters provided more extensive feedback on certain supporting behaviors than on others. In our survey, enforcement group managers reported receiving more extensive written and verbal feedback on business results than on the other critical job responsibilities and because of that they believed that business results was more important to their raters than the other critical job responsibilities. Additionally, half of the enforcement group managers we surveyed reported receiving verbal feedback on how well they met most or all of their commitments. Raters Provided Feedback to Enforcement Group Managers on Meeting Most Critical Job Responsibilities and Commitments in Evaluations and Verbal Feedback: Our review of enforcement group managers’ performance evaluations showed that raters provided written and verbal feedback to managers on their performance in meeting most critical job responsibilities and commitments. However, raters provided more extensive written and verbal feedback for some critical job responsibilities and for certain supporting behaviors than for others. As shown in table 4, our sample of performance evaluations for IRS employees who were managers in both fiscal years 2000 and 2001 showed that more than 80 percent of the written evaluations assessed managers’ performance in meeting four or more of the five critical job responsibilities. Table 4: Estimated Percentage of Fiscal Years 2000 and 2001 Written Evaluations in Which Raters Assessed Critical Job Responsibilities by Number of Responsibilities: Number of critical job responsibilities assessed: 5; Percentage of written evaluations, FY 2000: 54; Percentage of written evaluations, FY 2001: 47. Number of critical job responsibilities assessed: 4; Percentage of written evaluations, FY 2000: 29; Percentage of written evaluations, FY 2001: 41. Number of critical job responsibilities assessed: 3 or fewer; Percentage of written evaluations, FY 2000: 17; Percentage of written evaluations, FY 2001: 12. Number of critical job responsibilities assessed: Total; Percentage of written evaluations, FY 2000: 100; Percentage of written evaluations, FY 2001: 100. Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ written performance evaluations. [End of table] As noted in the previous section, commitments were not always written in a way that was conducive for raters to hold enforcement group managers accountable. This fact notwithstanding, raters provided feedback to managers on how well they met their commitments. Specifically, raters provided written feedback on managers’ performance in meeting most of their commitments. Our sample of fiscal year 2000 and 2001 written performance evaluations indicates that raters addressed 76 percent of commitments in employee evaluations, discussing each commitment separately. Raters addressed 6 percent of commitments by providing summary feedback such as “you met or exceeded your commitments” and provided no feedback on 18 percent of commitments. According to senior executives, IRS is considering new guidance that would require raters to assess managers’ performance in meeting each commitment, with the intent of holding managers more accountable for achieving the commitments they make. In our survey, most enforcement group managers reported receiving verbal feedback about their performance in meeting critical job responsibilities and commitments. We estimate that for more than 60 percent of the managers, raters discussed all five responsibilities in the verbal feedback that occurs with the delivery of the written performance evaluation, and in ongoing verbal feedback throughout the year. Table 5 below shows the percentage of enforcement group managers who reported receiving verbal feedback throughout the year in each of the critical job responsibilities. Table 5: Estimated Percentage of Enforcement Group Managers Reporting Verbal Feedback by Number of Critical Job Responsibilities for Fiscal Year 2001: Number of critical job responsibilities covered during feedback: 5; Verbal feedback with written performance evaluation: 63; Ongoing verbal feedback: 73. Number of critical job responsibilities covered during feedback: 4; Verbal feedback with written performance evaluation: 6; Ongoing verbal feedback: 7. Number of critical job responsibilities covered during feedback: 3 or fewer; Verbal feedback with written performance evaluation: 31; Ongoing verbal feedback: 20. Number of critical job responsibilities covered during feedback: Total; Verbal feedback with written performance evaluation: 100; Ongoing verbal feedback: 100. Source: GAO survey of IRS enforcement group managers. [End of table] In addition, an estimated half of the enforcement group managers in our survey reported receiving verbal feedback on how well they met most or all of their commitments. Raters Provide More Extensive Feedback on Some Critical Job Responsibilities and Supporting Behaviors Than on Others: In our review of enforcement group managers’ written evaluations, we found that raters were more likely to assess a manager’s performance in terms of customer satisfaction, business results, and employee satisfaction, and less likely to assess performance in leadership and equal employment opportunity. As shown in table 6, about 72 percent of written performance evaluations in fiscal years 2000 and 2001 assessed leadership or equal employment opportunity responsibilities. The employee satisfaction responsibility was the only critical job responsibility that was assessed in all evaluations and in both fiscal years. Table 6: Estimated Percentage of Fiscal Year 2000 and 2001 Evaluations Assessing Enforcement Group Managers’ Performance in Each Critical Job Responsibility: Critical job responsibility: Customer satisfaction; Percentage of written evaluations, FY 2000: 92; Percentage of written evaluations, FY 2001: 96. Critical job responsibility: Business results; Percentage of written evaluations, FY 2000: 96; Percentage of written evaluations, FY 2001: 99. Critical job responsibility: Employee satisfaction; Percentage of written evaluations, FY 2000: 100; Percentage of written evaluations, FY 2001: 100. Critical job responsibility: Leadership; Percentage of written evaluations, FY 2000: 72; Percentage of written evaluations, FY 2001: 71. Critical job responsibility: Equal employment opportunity; Percentage of written evaluations, FY 2000: 72; Percentage of written evaluations, FY 2001: 71. Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ written performance evaluations. [End of table] Furthermore, for each critical job responsibility, certain key supporting behaviors were emphasized more than others in the written performance evaluations, as shown in appendix V. For example, IRS has identified certain key behaviors that support the critical job responsibility of employee satisfaction. Of these behaviors, we found that “effectively uses ongoing feedback, coaching, and timely evaluations of performance to promote cooperation, teamwork, knowledge/skill sharing and goal accomplishment” was a key behavior that raters discussed in about 80 percent of written performance evaluations. In contrast, our sample indicated that a key supporting behavior that was discussed in less than 40 percent of evaluations was “creates an environment for continuous learning, pursuing development opportunities for self and others, with the intent to increase individual and organizational effectiveness.” During our discussions, IRS’s senior executives said they were comfortable with the relative emphasis placed on each of the supporting behaviors. Group Managers Report Raters’ Feedback Focused More on Business Results: As shown in Table 7, the evidence from our study is that enforcement group managers believed that their raters emphasized business results the most and emphasized equal employment opportunity the least in the written performance evaluation and ongoing verbal discussions. In addition, we estimate that equal employment opportunity was emphasized the least in the verbal feedback occurring when the written evaluation was delivered. Table 7: Estimated Percentage of Enforcement Group Managers Reporting Feedback to a Great or Very Great Extent for the Critical Job Responsibilities for Fiscal Year 2001: Critical job responsibility: Customer satisfaction; Type of feedback: Written evaluation: 51; Type of feedback: Verbal feedback with written performance evaluation: 31; Type of feedback: Ongoing verbal feedback: 31. Critical job responsibility: Business results; Type of feedback: Written evaluation: 60; Type of feedback: Verbal feedback with written performance evaluation: 36; Type of feedback: Ongoing verbal feedback: 43. Critical job responsibility: Employee satisfaction; Type of feedback: Written evaluation: 51; Type of feedback: Verbal feedback with written performance evaluation: 33; Type of feedback: Ongoing verbal feedback: 33. Critical job responsibility: Leadership; Type of feedback: Written evaluation: 49; Type of feedback: Verbal feedback with written performance evaluation: 34; Type of feedback: Ongoing verbal feedback: 30. Critical job responsibility: Equal employment opportunity; Type of feedback: Written evaluation: 31; Type of feedback: Verbal feedback with written performance evaluation: 23; Type of feedback: Ongoing verbal feedback: 13. Source: GAO survey of IRS enforcement group managers. [End of table] As shown in table 8, we found that managers believed raters conveyed that business results was the most important to them and equal employment opportunity the least, when they took all forms of feedback into consideration. Table 8: Estimated Percentage of Enforcement Group Managers Reporting that the Respective Critical Job Responsibilities Are Very Important or Important to Their Raters for Fiscal Year 2001: Critical job responsibility: Customer satisfaction; Perception of which respective critical job responsibilities are very important or important to their raters: 69. Critical job responsibility: Business results; Perception of which respective critical job responsibilities are very important or important to their raters: 87. Critical job responsibility: Employee satisfaction; Perception of which respective critical job responsibilities are very important or important to their raters: 68. Critical job responsibility: Leadership; Perception of which respective critical job responsibilities are very important or important to their raters: 74. Critical job responsibility: Equal employment opportunity; Perception of which respective critical job responsibilities are very important or important to their raters: 58. Source: GAO survey of IRS enforcement group managers. [End of table] IRS officials explained that while they do not intend that equal emphasis be always given to all five critical job responsibilities, or to all commitments, they believe that an over emphasis on one or just a few of the job responsibilities would be inappropriate. Overall, they were satisfied with the emphasis given to each of the five critical job responsibilities in both the evaluations and the commitments. However, they noted that a heavy emphasis on some critical job responsibilities could lead managers away from the balanced behaviors that IRS seeks to achieve through its new performance management system. IRS Has Made Progress but Could Further Enhance Its New Managers’ and Frontline Employees’ Performance Management Systems: IRS has made progress in the challenging task of redesigning its performance management systems to help reinforce behaviors and actions that support the agency’s mission. IRS senior executives were generally satisfied with the implementation of the new performance management systems but expect that the systems will continue to evolve as the agency gains more experience in implementing the new culture and processes. However, IRS has not established mechanisms to monitor how well the systems provide useful feedback to group managers and frontline employees about their performance and whether feedback aligns with IRS’s strategic goals. For example, IRS does not have plans to determine if raters are over-emphasizing one critical job responsibility during written and verbal feedback and when managers develop commitments. Further, IRS does not have plans to assess at an appropriate time whether the new performance management systems are achieving their objectives. Such an assessment would be consistent with IRS’s guidance for implementing strategic initiatives. IRS Has Made Progress in Making Its Managers’ and Frontline Employees’ Performance Management Systems an Effective Management Tool: IRS has implemented a number of initiatives to support its performance management systems, and set the stage for the reform of its entire performance management system over the coming years. According to officials, in response to RRA 98 and as part of its overall modernization efforts, IRS realized it needed to redesign its performance management process to better communicate the behaviors constituting customer satisfaction, business results, and employee satisfaction, and to ensure managers and employees adopt the newly desired behaviors in their day-to-day activities. In an attempt to make managers more familiar with the new system, IRS has undertaken several initiatives, with similar actions also planned for its frontline employees, such as: * providing interim guidance and templates of sample commitments, self- assessments, and summary evaluations; * distributing computer discs with user-friendly access to information on performance management and added performance management information to the IRS intranet website, where it is constantly being updated; * conducting an interactive videoconference for all managers on the new performance management system. Further, IRS has started its implementation of a pay-for-performance system, which emphasizes performance instead of longevity in determining compensation. IRS is also experimenting to see whether the manager evaluation forms might also be suitable for making decisions about developmental needs, rewards and recognitions, and compensation. IRS has already placed senior managers in a senior pay-band, and expects that other managers, including those at the group manager level, will be placed in pay-bands by October 2002. Ultimately, IRS plans to rollout the pay-for-performance system to include all managers as well as frontline employees. IRS Officials Generally Satisfied with Implementation of Managers’ and Frontline Employees’ Performance Management Systems but Realize Implementation Challenges Remain: IRS recognizes that revamping its employee performance management systems is a major effort that presents significant implementation challenges. IRS senior executives were generally satisfied with the implementation of the managers’ performance management system given that it has only been in place for two years. They expect that implementation of both systems will improve as raters gain more experience in implementing the new culture and processes. For example, they believe it will take some time for raters of group managers and frontline employees to fully interpret the new critical job responsibilities and become comfortable with how to apply them. However, senior executives believe that IRS’s plans to expand its pay- for-performance system underscore the need to ensure verbal and written feedback are consistent with the message IRS wants to deliver to its managers and employees about the activities and behaviors it values. They believe the validity of pay and bonus decisions will depend heavily on how well the performance management systems clearly communicate what managers and employees are expected to do as well as provide comprehensive feedback on how well they performed in carrying out their responsibilities and meeting their commitments. IRS senior executives are concerned that if misunderstandings about performance expectations creep into the system, for example because commitments are vaguely worded, managers and employees will begin to discredit the performance management system and the pay-for-performance decisions. IRS senior executives believe that raters have to be clear about their expectations so that managers know in advance what they need to do to be successful and receive bonuses or salary increases. Monitoring and Assessment Mechanisms Would Help Ensure IRS’s Employee Performance Management Systems Are Implemented as Intended: IRS has not established a monitoring mechanism to ascertain whether raters are implementing the new employee performance management systems as intended and has no plans to assess whether the new systems are achieving their objectives. IRS senior executives said they have done a limited assessment of the evaluations for executives and senior managers because they are the only group currently under the pay-for-performance system. IRS senior executives said they have not established a monitoring mechanism or a plan to assess the managers’ or frontline employees’ systems due to resource constraints. A monitoring mechanism could provide useful information on whether IRS’s processes for evaluating employees are being followed. A monitoring system could include, for example, an employee survey to obtain information on whether raters are involved in the development of their subordinates’ commitments, giving appropriate emphasis to each of the critical job responsibilities and supporting behaviors, and are providing useful feedback. Also, content analyses of commitments and written performance evaluations could help senior executives monitor whether the systems are encouraging the kinds of behaviors and activities that support IRS’s strategic goals. Active monitoring could give IRS senior executives a sense of how the systems are working in practice and whether any modifications are needed to provide more useful feedback to managers and employees about their performance and better align the systems with IRS’s strategic goals. Although we have not estimated the costs to perform such monitoring, costs likely could be minimal. Relatively small statistical samples of performance evaluations and of employees to survey likely would be sufficient, and once a monitoring methodology is developed it could be used for several years. Further, to the extent such monitoring could prevent potential problems, IRS would avoid the costs associated with employee performance that is not well-balanced and aligned with IRS’s goals. In addition to active monitoring of the employee performance management systems, a more thorough future assessment of whether the systems are achieving their various goals and objectives could provide IRS assurance that the systems are properly designed and implemented, or identify whether changes are needed. For example, IRS could assess whether the systems are providing a framework for improving communications between raters and their subordinates, coordinating planning activities, aligning individual performance to organizational performance, and ensuring fair and consistent performance evaluations. Although it would be premature to perform an assessment now—since the systems are not fully implemented—planning now for such an assessment could better ensure that IRS will have the necessary data to assess the performance management systems’ performance in the future. Such upfront planning is, for instance, part of IRS’s guidance for planning and implementing reorganizations. That guidance requires that a plan for assessing the results of a reorganization be developed at the same time that the reorganization is proposed. Monitoring and assessing IRS’s new performance management systems also would be in line with the expectations IRS has established for the strategic initiatives being carried out by its operating divisions. In March 2000, IRS implemented a new strategic planning, budgeting, and performance management process intended to provide a more structured format for establishing strategic direction, identifying the resources needed to accomplish the operational activities supporting the strategy, and assessing performance results. [Footnote 8] IRS recognizes that conducting assessments of operational activities at all levels of the organization is essential to ensuring its programs are achieving their goals. As IRS’s strategic plan points out, regular, structured feedback on program performance is essential if managers are to recognize and react to successes or failures and ensure programs are adhering to the agency’s strategic intent. Although IRS would incur costs to assess whether its employee performance management systems achieve their objectives, IRS’s strategic planning, budgeting and performance management guidance implicitly recognizes that such costs are properly part of a sound management system. Further, planning ahead for such assessments can help ensure that costs are minimized. Finally, as with monitoring, to the extent that an assessment results in further improvements to IRS’s performance management systems, costs due to unbalanced or misaligned employee behaviors would be avoided. Conclusions: IRS is in the midst of a cultural change that requires group managers and frontline employees to better balance taxpayer needs, business results, and employee needs in carrying out their day-to-day responsibilities. By aligning its new performance management systems with its strategic and operational goals, IRS has made progress in supporting this cultural change and providing guidance to managers and employees on how to prioritize their activities and carry out their responsibilities. In addition, most enforcement group managers received written or verbal feedback on how well they met their critical job responsibilities and commitments. However, we identified several opportunities to enhance IRS’s performance management systems for managers and employees. IRS could better align employee performance with strategic goals by establishing greater consistency within the frontline employee performance management system. When misalignment exists between the unit performance measures and the frontline employee performance management system, IRS is emphasizing different behaviors at the unit level than at the employee level. As a result, IRS is not consistently communicating its strategic goals to employees. Commitments could be developed to be more specific, outcome or output oriented, and easily monitored so that group managers can be held accountable for actions they intend to take to support IRS’ goals. IRS is considering new guidance that would require raters to assess managers’ performance in meeting each commitment, with the intent of holding managers more accountable. Such guidance on providing more comprehensive feedback would further reinforce use of IRS’s new performance management system for managers to encourage behaviors and actions that support IRS’s strategic goals. Further, if unequal emphases in written and verbal feedback grow over time, the performance management systems might not adequately align with strategic goals and provide the intended message about how managers and employees should conduct their daily activities. If so, managers could receive a skewed perception of the behaviors that IRS values. Misusing even a well-designed performance management system can drive dysfunctional behaviors, such as an overemphasis on certain activities to the detriment of the balanced approach IRS is trying to achieve and maintain. Accordingly, it is important for IRS to monitor and assess its progress in fully implementing the systems and position itself to take corrective action if and when needed. Recommendations for Executive Action: To better hold managers accountable for meeting strategic goals and ensure the new performance management systems are working as intended, we recommend that the Commissioner of Internal Revenue take steps to: * improve the linkage between frontline employees’ critical job responsibilities, the supporting behaviors, and organizational unit performance measures and; * develop plans for monitoring and assessing whether the new employee performance management systems are operating as intended and take the necessary actions to resolve any identified problems. Agency Comments and Our Evaluation: On July 5, 2002, we received written comments on a draft of this report from the Commissioner of Internal Revenue (see app. VII). The commissioner agreed with the recommendations contained in this report and said they would be included as IRS works to continuously improve its performance management system. In his comments, the Commissioner noted that IRS currently uses performance review boards to ensure executives’ and senior managers’ performance agreements and evaluations align with and reflect IRS’s strategic goals but this approach is less practical at lower levels of the organization due to the larger numbers involved. Therefore, IRS is going to explore other means of ensuring alignment. In addition, the Commission said IRS will continue to monitor the overall efficacy of its performance management system. As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from the date of this letter. At that time, we will send copies to the Ranking Minority Member of the Committee on Ways and Means; the Ranking Minority Member of the Subcommittee on Oversight; the Secretary of the Treasury; and the Commissioner of Internal Revenue. We will also make copies available to others on request. In addition, the report will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. Please contact me at (202) 512-9039, Ralph Block at (415) 904-2150, or Jonda Van Pelt at (415) 904-2186 if you or your staff have any questions. Key contributors to this report are acknowledged in appendix VIII. Signed by: Michael Brostek: Director, Tax Issues: [End of section] Appendix I: Objectives, Scope, and Methodology: Objectives: Our objectives in this report were to determine (1) the extent to which group managers’ and frontline employees’ critical job responsibilities and other elements of the employee performance management systems are structurally aligned with IRS’s strategic goals and organizational unit performance measures, (2) the extent to which group managers’ commitments align with IRS’s strategic goals and are written so that raters can hold managers accountable for meeting their commitments, (3) the extent to which raters provide feedback to enforcement group managers on their performance in meeting critical job responsibilities and commitments, and (4) whether IRS has adequate plans to monitor and assess the implementation of its performance management systems. Scope and Methodology: To determine the extent to which the critical job responsibilities for group managers and frontline employees align with IRS’s strategic goals, we performed separate qualitative analyses of both performance management systems to assess whether the critical job responsibilities align with IRS’s goals and whether supporting behaviors were consistent with the critical job responsibilities. Our analysis of the frontline employee system also included reviewing organizational unit performance measures to determine if these measures were being reflected in the performance management system’s supporting behaviors. We only included those components of the unit measures that deal with employee behaviors and actions, not those measures that address how well IRS’s procedures and systems function or that set organizational performance goals. To determine the extent to which group managers’ commitments align with IRS’s strategic goals and are written so that managers can be held accountable for meeting those commitments, we analyzed the commitments of a statistically valid random sample of group managers in IRS’s Small Business/Self-Employed Division. [Footnote 9] We selected this IRS division because it included managers in key enforcement positions who supervise employees dealing directly with taxpayers when auditing tax returns or collecting unpaid taxes. To determine whether commitments aligned with IRS’s strategic goals, we assessed whether the objective of the commitment was to improve customer satisfaction, business results, employee satisfaction, leadership or equal employment opportunity. [Footnote 10] To determine the extent to which managers could be held accountable for meeting their commitments, we applied IRS’s criteria for formulating well-constructed commitments, which included that they are clear, specific, achievable, outcome or output oriented, and are easily monitored. As we applied the criteria to individual commitments, we found interrelationships between certain criteria. For example, commitments that are specific also tend to be outcome or output oriented and easily monitored. Similarly, commitments that include explicit outcomes or outputs tend to be easier to monitor. As agreed with IRS officials, we addressed the following questions in determining whether commitments met the criteria: * Clear: Is the commitment easy to understand? Does it contain jargon that might make it unfamiliar to managers or their supervisors? * Specific: Does the commitment include details, such as the specific actions to be taken, when the actions are to be taken, whether other IRS employees are involved, the expected result, or a numeric target? Is the commitment specific enough so that a third party can understand the actions and timeframes to which the manager is committing? * Achievable: Can the manager accomplish the task in the normal work environment? To what extent does the manager control the resources necessary to accomplish the commitment? * Outcome or output oriented: Does the commitment include an expected result, the type and level of activity to be accomplished, or a program impact? * Easily monitored: Does the commitment include a deadline, an expected result, a numeric target, or some other means of measurement? To determine the extent to which raters provided feedback to group managers on how well they met their critical job responsibilities and commitments, we performed a content analysis of the narrative portion of the evaluation that discussed managers’ performance. To assess the extent to which managers were given feedback on meeting their critical job responsibilities, we assessed whether the evaluation narrative described behaviors that were consistent with the key supporting behaviors for each critical job responsibility as laid out on the managers evaluation form (see app. III for a sample of the evaluation.) We also reviewed the narrative to determine whether the rater assessed how well the managers met their commitments. In addition, we surveyed the managers for information they received on their performance during verbal feedback sessions held with their raters. [Footnote 11] To determine whether IRS has adequate plans to monitor and assess the implementation of its employee performance management systems, we talked to IRS officials about initiatives taken to date and reviewed IRS guidance for strategic planning. Our review was subject to some limitations. Our assessment of group managers’ commitments and our content analysis required us to make judgments that were, in part, subjective. To maximize the objectivity of our analyses, we had our reviewers conduct separate and independent assessments of (1) whether commitments met IRS’s criteria and (2) the narrative discussions of each sampled evaluation. When differences arose between the two reviewers, a collaborative approach was used to resolve them. [End of section] Appendix II: GAO Sampling Methodology: This appendix discusses the sampling methodology we used to determine (1) to what extent group managers’ commitments aligned with IRS’s strategic goals and were written so that raters can hold group managers accountable for meeting their commitments and (2) to what extent raters provided feedback to group managers on how well they met their commitments and critical job responsibilities. Sample Designs: We randomly selected a probability sample of 150 from the study population of 1,374 group managers with enforcement responsibilities working in IRSs Small Business/Self-Employed Division. These managers were identified from a database of IRS personnel information current as of January 2001. With this statistically valid probability sample each member of the population had a nonzero probability of being included, and that probability could be computed for any member. Each manager selected in the sample was subsequently weighted in the analysis to account statistically for all the members of the population, including those that were not selected. For the objective of determining to what extent group managers’ commitments aligned with IRS’s strategic goals and were written so that raters can hold managers accountable for meeting their commitments, we reviewed fiscal year 2000 and fiscal year 2001 written annual employee evaluations. For this objective, we found that 126 group managers in our sample had enforcement responsibilities in fiscal years 2000 and 2001. We therefore estimate that about 1,154 group managers in the study population had enforcement responsibilities in fiscal years 2000 and 2001. For the objective of determining to what extent raters provided feedback to group managers on how well they met their commitments and critical job responsibilities, we reviewed fiscal year 2000 and fiscal year 2001 employee evaluations. For this objective, we found that 120 group managers in our sample had enforcement responsibilities in fiscal year 2002. We therefore estimate that about 1,099 group managers in the study population had enforcement responsibilities in fiscal year 2002. We also administered a web-based survey to obtain information from current, fiscal year 2002, managers on the verbal feedback that raters provided to them. As we received evaluations from IRS we found that some of the managers in our sample were temporary, acting group managers who were not assessed using the same evaluation form as permanent group managers. We were unable to use these evaluations in our analysis and deleted temporary, acting group managers from our sample. Sample Disposition: For our review of written annual performance evaluations, we received 112 usable evaluations (those that indicated the employee was an enforcement group manager for both, complete fiscal years and included both commitments and summary narrative) out of 126 eligible respondents for a response rate of approximately 89 percent. We eliminated 24 ineligible group managers, including those for whom we did not have two complete evaluations as a group manager or who functioned in a special capacity such as training specialist. For our survey of enforcement group managers’ perceptions of the amount and type of feedback they received from raters, we received 84 usable responses (those who completed the survey and indicated that they were currently a group manager with enforcement responsibilities) out of 120 eligible respondents for a response rate of 70 percent. We eliminated 30 ineligible employees, including those who were no longer a group manager due to job reassignment or retirement. The disposition for the sampled cases for our review of group manager evaluations and for our group manager survey is shown in table 9 and table 10, respectively. Table 9: Disposition of Sample Cases for Our Review of Group Manager Evaluations: Total number of initially sampled elements: 150; Sampled elements outside the study population (i.e., “ineligibles”): Not a group manager with enforcement responsibilities for FY 2000 and 2001: 24; Subtotal of ineligible elements: 24. Sampled elements in the study population (i.e., “eligibles”): Nonrespondents: Provided unusable evaluation(s): 6. Sampled elements in the study population (i.e., “eligibles”): Nonrespondents: No evaluation(s) available: 8. Respondents (provided usable evaluations): 112. Subtotal of eligible elements: 126. Evaluation response rate (respondents/subtotal of eligible elements): 89%. Source: IRS data and GAO sample. [End of table] Table 10: Disposition of Sample Cases for Our Survey of Group Managers’ Perceptions of Raters’ Feedback: Total number of initially sampled elements: 150. Sampled elements outside the study population (i.e., “ineligibles”): Not a current group manager with enforcement responsibilities for FY 2002: 30. Subtotal of ineligible elements: 30. Sampled elements in the study population (i.e., “eligibles”): Nonrespondents: Refused: 2; Sampled elements in the study population (i.e., “eligibles”): Nonrespondents: Unable to respond: 1; Sampled elements in the study population (i.e., “eligibles”): Nonrespondents: No response after contact: 26; Sampled elements in the study population (i.e., “eligibles”): Nonrespondents: Unable to contact due to missing or incorrect contact information: 2; Sampled elements in the study population (i.e., “eligibles”): Nonrespondents: Returned unusable survey: 5; Respondents (returned usable surveys): 84. Subtotal of eligible elements: 120. Evaluation response rate (respondents/subtotal of eligible elements): 70%. Source: IRS data and GAO sample. [End of table] Sampling Error: Because we followed a probability sampling procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s estimates by providing a sampling error (for example, +/- 10 percentage points) with a specified confidence level, for example, the 95-percent confidence level. The confidence interval (i.e., the estimate plus or minus its sampling error) would contain the actual population value for 95 percent of the samples we could have drawn. As a result, we are 95-percent confident that a confidence interval includes the true value in the study population. In this report, all percentage estimates dealing with our review of evaluations have sampling errors of +/- 11 percent or less of the value of those numerical estimates, unless otherwise stated. All percentage estimates dealing with the group manager survey have sampling errors of +/- 12 percent or less, unless otherwise stated. Nonsampling error: In addition to the reported sampling errors, the practical difficulties of conducting any survey may introduce other types or “nonsampling” errors. For example, differences in how a particular question is interpreted or the types of individuals that do not respond can introduce unwanted variability into the survey results. To minimize such nonsampling errors, we pre-tested the survey with four enforcement group managers and incorporated their feedback into the survey design. We did not analyze the survey non-respondents or employee evaluations non-respondents to determine whether the non-response was random. We assumed that the survey and employee evaluations non-respondents had the same characteristics as group managers who responded to the survey and provided usable evaluations, respectively. [End of section] Appendix III: Samples of Manager and Frontline Employee Evaluation Forms: The following six-page form is used to evaluate performance for managers. IRS Performance Management System: Manager Performance Agreement: Form 12450-A (Rev. 10-2000): Catalog Number 28621D: Department of the Treasury-Internal Revenue Service: Managers in the Internal Revenue Service are accountable for supporting the Service's mission to "provide America's taxpayers top quality service by helping them understand and meet their tax responsibilities, and by applying the tax laws with integrity and fairness to all." This Agreement is intended to establish annual performance expectations in this regard. As described below, those critical expectations consist of three parts: Responsibilities that are common to all IRS managers, Commitments that are specific to each, and a Retention Standard for fair and equitable treatment of taxpayers. These performance expectations (set at the beginning of each fiscal year) serve as the basis for a manager's annual performance evaluation. Starting date for the Performance Agreement: Ending date for the Performance Agreement: Name: Social Security Number: Position title: Series and grade: Organization segment (Office code symbols): Part I: Critical Performance Expectations: A. Responsibilities: All IRS managers share certain, critical responsibilities for achieving performance excellence. Set forth below, these responsibilities reflect the core values of the Service - what is important to us as an organization - and they are shared by all IRS executives and managers. These responsibilities guide achievement of the incumbent's commitments in Part I-B. Equal weight should be given to both the responsibilities and commitments in the summary narrative and evaluation. The incumbent and immediate supervisor jointly review these responsibilities to ensure mutual understanding. Leadership: Demonstrates integrity and the highest ethical standards of public service. Develops, prioritizes and aligns strategies, objectives and goals, taking into account key influences on organizational performance. Successfully leads organizational change, effectively communicating the Service's mission, core values, and strategic goals to employees and other stakeholders and responding creatively to changing circumstances. Creates and sustains a positive workplace that inspires others to support the IRS mission and goals. Shows a commitment to public service and citizenship. Uses sound judgment to make effective and timely decisions. Employee Satisfaction: Ensures that a safe, healthy work environment is maintained. Motivates employees to achieve high performance through empathetic, open and honest communication, by involving them in decision making, and ensuring that they have the tools and training to perform their jobs. Creates an environment for continuous learning, pursuing development opportunities for self and others, with the intent to increase individual and organizational effectiveness. Develops and recognizes employees so that they realize their full potential as members of the Service. Effectively uses ongoing feedback, coaching, and timely evaluations of performance to promote cooperation, teamwork, knowledge/skill sharing and goal accomplishment. Acts on employee concerns and values ideas and perspectives of people from diverse backgrounds. Ensures all employees are treated in a fair and equitable manner. Customer Satisfaction: Communicates to employees the importance of customer focus as a critical component of the Service's mission. Constantly listens to customers, analyzes their feedback to identify their needs and expectations, and acts to continuously improve products and services. Identifies and utilizes policies, and economic, political, and social trends in an effort to improve organizational performance. Builds strong alliances, involving stakeholders (for example NTEU, internal customers, suppliers, etc.) in making decisions, and gaining cooperation to achieve mutually satisfying solutions. Initiates actions and manages risks to develop new products and services within or outside the organization. Shares innovations with others. Business Results: Pursues business excellence through effective process management and the application of balanced measures. Develops and executes plans to achieve organizational goals, leveraging resources (human, financial, etc.) to maximize efficiency and produce high quality results. Takes steps to prevent waste. fraud and abuse and instill public trust. Identifies and analyzes problems to resolve individual and organizational issues in accordance with law, regulations and Service policy. Learns about current and emerging issues/developments in own field of expertise and applies knowledge to make technically sound operational decisions. Understands and uses organizational realities, networks, and accepted practices to achieve desired results. Equal Employment Opportunity: Takes steps to implement the EEO and affirmative employment goals established by the bureau. Supports staff participation in special emphasis programs. Promptly responds to allegations of discrimination and/or harassment and initiates appropriate action to address the situation. Cooperates with EEO counselors, EEO investigators, and other officials who are responsible for conducting inquiries into EEO complaints. Assigns work and makes employment decisions in areas such as hiring, promotion, training and developmental assignments without regard to sex, race, color, national origin, religion, age, disability, sexual orientation or prior participation in the EEO process. Monitors work environment to prevent instances of prohibited discrimination and/or harassment. B. Commitments: In the space below, the incumbent and his or her immediate supervisor should describe a limited number of critical actions, objectives, and/or results that the incumbent will be expected to accomplish during the upcoming evaluation year. These Commitments should be derived from, and directly contribute to, the program priorities and objectives established by the organization's annual business or operations plan; they should also be balanced, based on the Service-wide responsibilities described above, and may include personal developmental objectives relating to those responsibilities. Generally, Commitments should be qualitative in nature, but they can (and should) be guided and informed by the organization's quantitative performance measures. Commitments may be modified during the evaluation period if circumstances warrant. Changes should be made at least 60 days before the end of the evaluation period to prevent having to extend the evaluation period. C. Retention Standard - Fair and Equitable Treatment of Taxpayers: Consistent with the incumbent's official responsibilities, administers the tax laws fairly and equitably, protects taxpayers' rights, and treats them ethically with honesty, integrity, and respect. The incumbent and supervisor jointly review to ensure mutual understanding. D. Acknowledgement: By signature, the incumbent and his or her rating and reviewing officials acknowledge that they have discussed the critical performance expectations set forth above, and that the discussion included examples of behavior that would/would not meet those standards, as well as the consequences of each. The discussion occurs at the beginning of the evaluation period. The incumbent is given a copy of the agreement, and the original is placed in the incumbent's Employee Performance File. Employee's signature: Date: Rating official's signature: Date: Reviewing official's signature: Date: Part II: Midyear Progress Review: This mandatory review generally takes place in April of the evaluation year. The rating official and the incumbent are required to discuss the incumbent's progress toward expectations (Responsibilities, commitments, Retention Standard) set forth in Part I. That discussion should be summarized below. Date review conducted: Rating official's signature: Employee's signature: Part III: Summary Evaluation: A. Summary Narrative (The incumbent is to provide the supervisor a self- assessment of no more than two pages): Limited to the space provided below the supervisor must describe the incumbent's achievements during the past evaluation year, as compared against the Critical Performance Expectations (Responsibilities, Commitments, Retention Standard) established in Part I. Emphasize areas of significant accomplishment that may meet or exceed expectations, and where appropriate, indicate opportunities for personal and professional development. Any performance problem resulting in a rating of "Not Met" must also be addressed in the Summary Narrative. Performance on the Retention Standard need only be addressed if the incumbent did not meet that standard. B. Expectations - Ratings: Reason for Rating: - Annual rating: - Departure rating: - Other: Retention Standard: - Not met: - Met: - Not applicable: Responsibilities: - Not met: Placed insufficient emphasis on one or more sets of Responsibilities. Actions taken were inappropriate, ineffective, or undermined strategic goals or annual business plan accomplishment. - Met: Placed appropriate emphasis on each of the five sets of Responsibilities. Appropriate actions were taken to support accomplishment of the annual business plan and strategic goals. - Exceeded: In addition to placing appropriate emphasis on the five sets of Responsibilities, served as a role model in one or more of the five sets. Actions taken were exemplary in promoting accomplishment of the annual business plan and strategic goals. Commitments: - Not met: Did not achieve or make substantial progress toward achievement of desired results. - Met: Achieved or made substantial progress toward achievement of desired results. - Exceeded: Overcame significant obstacles, such as insufficient resources, conflicting demands, or unusually short timeframes, in achieving or exceeding desired results. C. Summary Evaluation: Not met[A]: The incumbent failed to meet the retention standard, responsibilities and/or commitments. Repeated observations of performance indicated negative consequences in key outcomes (e.g., quality, timeliness, business results, morale, etc.). Immediate improvement is essential. Met: The incumbent met the retention standard and the responsibilities and commitments in his or her Agreement with solid, dependable performance. Incumbent consistently demonstrates the ability to meet the requirements of the job. Challenges encountered and resolved are part of the day to day operation and are generally routine in nature. Exceeded: The incumbent met the retention standard and generally exceeded both the responsibilities and commitments in his or her Agreement. However, he or she may have met the retention standard and demonstrated exceptional performance in either responsibilities or commitments and met the expectations of the other. He or she may have overcome significant organizational challenges such as coordination with external stakeholders (NTEU, Congress, etc.) or insufficient resources. His or her effectiveness and contributions may have had impact beyond his or her purview. Outstanding: The incumbent met the retention standard and performed as a model of excellence by exceeding the responsibilities and commitments in his or her Agreement, despite constantly changing priorities, insufficient or unanticipated resource shortages, and externally driven deadlines. He or she consistently demonstrated the highest level of integrity and performance in promoting the annual business plan and the Service's strategic goals and objectives. His or her contributions had impact beyond his or her purview. [A] The IRS is requesting demonstration authority to eliminate the statutory requirement for a minimally satisfactory rating level for SES positions. To maintain rating consistency between executives, managers, and management officials, an amended rating form, containing the minimally satisfactory rating level, will be issued prior to the end of the evaluation period if the request is not approved. Rating official's signature: (I certify that records of tax enforcement results were not used in preparing this evaluation): Date: Approving official's signature: (I certify that records of tax enforcement results were not used in preparing this evaluation): Date: This evaluation has been discussed with me and I have been given a copy. I am aware that if I decide to submit a narrative response and/or request a review by a higher official, one or both must he submitted in writing within 15 workdays of receipt of my evaluation. Employee's signature: Date: Privacy Act Notice: The Privacy Act of 1974 requires that when we ask you to provide information about yourself, we must tell you: our legal right to ask for the information; the principal purpose(s) for which the information is intended to be used; what could happen if we do not receive any or all of the information; and whether your response is voluntary or mandatory. This statement is being provided pursuant to the Privacy Act of 1974, as amended, for individuals who have been requested to submit a statement of accomplishment/self-assessment. The authority to solicit this information is derived from 5 USC 4301, et seq., and 5 CFR Part 430, Performance Management. In order to allow you the opportunity to provide input into the evaluation process, management will request this information from you. Your supervisory officials will consider the information you furnish in preparing an evaluation of your performance or conducting periodic progress reviews. The information contained in your performance evaluation may be disclosed to IRS employees who have a need for the record in their official duties. Disclosures may also be made when appropriate, under routine uses published in the Federal Register for Privacy Act system of records, Treasury/IRS 36.003, General Personnel and Payroll Records. Under the appropriate circumstances, disclosure may be made to the Office of Personnel Management, the Equal Employment Opportunity Commission, the General Accounting Office and others. Failure to furnish any or all of this information may result in your supervisors preparing your evaluation, or conducting a progress review, without considering information you may feel is relevant or significant. [End of form] The following two-page form is used to evaluate performance for IRS frontline employees, including revenue agents and revenue officers. Performance Appraisal and Retention Standard Rating: (Review instructions on the reverse before completing form): 1. Name of employee (Last, first, middle): 2. POI: 3. Period covered: From: To: 4. Reason for Appraisal: - Annual Rating: - Departure Rating: - Merit Promotion: - Other: 5. SSN: 6. Organization code: 7. Position title: 8. SPD or PD#: 9. Pay plan, series/grade: 10. Critical Job Elements: (Job Element Rating: N/A; 5; 4; 3; 2; 1): I. Employee Satisfaction - Employee Contribution: II. Customer Satisfaction - Knowledge: III. Customer Satisfaction - Application: IV. Business Results - Quality: Measured; Unmeasured: V. Business Results - Efficiency: Measured; Unmeasured: 11. Retention Standard Rating: - Not applicable: - Met: - Not Met: 12. Average of Critical Elements: 13. Summary Level: - Outstanding: - Exceeds Fully Successful: - Fully Successful: - Minimally Successful: - Unacceptable: 14. Annual Rating: Rater: Name/title/signature/date (I certify that records of tax enforcement results were not used to prepare this appraisal): Reviewing Official: Name/title/signature/date (I certify that records of tax enforcement results were not used to prepare this appraisal): This appraisal has been discussed with me and I have been given a copy. Employee signature/date: 15. Revalidation of Rating of Record: (See instructions for revalidation). Period covered: From: To: Rater: Name/title/signature/date (I certify that records of tax enforcement results were not used to prepare this appraisal): 16. Merit Promotion Revalidation (See instructions for Merit Promotion revalidation). Name/title/signature/date of revalidation: Instructions: Appraise the employee against the critical job elements of his/her position for the rating period. Information is available online at [hyperlink, http://shr.web.irs.qov/cje]. Blocks 1 and 3 to 9 - Self-explanatory. Block 2. POI (Personnel Office Identifier) - Contact your Transactional Processing Branch for the correct number. Block 10. Critical Job Elements: The five (5) critical job elements for all employees are listed. If performance of the duties/responsibilities reflected by a critical job element has not been observed, identify the critical job element as Not Applicable (NA). Reasons for not appraising critical job element must be documented as part of the appraisal. Indicate as applicable, if performance is measured by TEPS or unmeasured (critical job elements IV and V). The rating for each critical job element will be based upon a review and consideration of all aspects of the critical job element, using the following scale: * Outstanding - Exceeds all performance aspects of the critical job element: (5); * Exceeds Fully Successful - Exceeds more than half of the performance aspects of the critical job element and meets the remaining performance aspects: (4); * Fully Successful - Meets all performance aspects of the critical job element: (3); * Minimally Successful - Fails one performance aspect of the critical job element: (2); * Unacceptable - Fails two or more performance aspects of the critical job element: (1); Block 11. Retention Standard Rating - Narrative is mandatory if rating is Not Met. Block 12. Average Of Critical Job Elements - Used to determine performance awards. Block 13. Summary Level - On the basis of the rating on the individual critical job elements and the Retention Standard, assign an overall Rating using the following scale: * Outstanding - Employee is rated Outstanding in more than half of the critical job elements and exceeds Fully Successful in the remainder of the critical job elements. * Exceeds Fully Successful - Employee is rated Exceeds Fully Successful or above in more than half of the critical job elements and Fully Successful in the remainder of the critical job elements. * Fully Successful - Employee is rated Fully Successful or above in all of the critical job elements. * Minimally Successful - Employee is rated Minimally Successful in one or more critical job element but not Unacceptable in any critical job element. * Unacceptable - Employee is rated Unacceptable in one or more critical job elements or receives a "Not Met" on the Retention Standard. Block 14. Annual Rating - Signatures as indicated. The rater and reviewing official must certify that records of tax enforcement results were not used to prepare the appraisal. Block 15. Revalidation Of Rating Of Record - If a manager determines that a journey level or above employee, in at least the second year of their position, would receive a Rating of Record for the current appraisal period identical to the Rating of Record for the most previous period, the manager may certify that the most recent Rating of Record is valid for performance in the current appraisal period. A manager may revalidate a Rating of Record only once. Block 16. Merit Promotion Revalidation - Place a check in the box if you are revalidating a Rating of Record that is more than six (6) months old but less than 12 months old on or before the opening date of the vacancy announcement. This applies to bargaining unit employees only. Narratives are required with all Merit Promotion evaluations. Privacy Act Statement: The Privacy Act of 1974 requires that when we ask you to provide information about yourself, we must tell you: our legal right to ask for the information; the principal purpose(s) for which the information is intended to be used, what could happen if we do not receive any or all of the information and whether your response is voluntary or mandatory. Our legal right to ask you to acknowledge receipt of performance appraisal is derived from 5 USC 9508, General Workforce Performance Management System and 26 CFR Part 801, Balanced System for Measuring Organizational and Employee Performance within the Internal Revenue Service. Your signature will acknowledge that you received the performance appraisal and it was discussed with you. Your supervisory officials will consider the information you furnish in preparing an assessment of your performance or conducting periodic progress reviews. The information contained in your performance assessment may be disclosed to IRS employees who have a need for the record in their official duties. Disclosures may also be made when appropriate, under routine uses published in the Federal Register for Privacy Act system of records, Treasury/IRS 36.003, General Personnel and Payroll Records. Under the appropriate circumstances, disclosure may be made to the Office of Personnel Management, the Equal Employment Opportunity Commission, the General Accounting Office and others. Failure to furnish any or all of this information may result in your supervisors preparing your appraisal or conducting a progress review without considering information you may feel is relevant or significant. [End of form] [End of section] Appendix IV: Examples of Evaluations Addressing Group Managers’ Responsibilities: This appendix provides examples of critical job responsibilities, supporting behaviors, and statements from narratives that we found in our review of the enforcement group managers’ evaluations. Table 11: Examples of Critical Job Responsibilities, Supporting Behaviors, and Evaluation Narratives for Group Managers: Critical job responsibility: Leadership; Key supporting behavior: Uses sound judgment to make effective and timely decisions. Behavior described in evaluation narrative: “As a leader, you have demonstrated sound judgment in your decisions organizing the second group offer.” Critical job responsibility: Leadership; Key supporting behavior: Develops, prioritizes and aligns strategies, objectives and goals, taking into account key influences on organizational performance. Behavior described in evaluation narrative: “He led his employees to develop realistic personal and group goals that easily meshed into the needs of the service and the general public.” Critical job responsibility: Customer satisfaction; Key supporting behavior: Constantly listens to customers, analyzes their feedback to identify their needs and expectations, and acts continuously to improve products and services. Behavior described in evaluation narrative: “You take the time to speak with the taxpayers and their representative about their concerns and try to accommodate their needs.” Critical job responsibility: Customer satisfaction; Key supporting behavior: Ensures that employees do the same (as immediately above) and that they are prompt, fair, and responsive to the circumstances of individual customers to the extent permitted by law and regulation. Behavior described in evaluation narrative: “He insists his employees take the same approach with their customers and that they address each case with the rights of the taxpayers up front.” Critical job responsibility: Employee satisfaction; Key supporting behavior: Effectively uses ongoing feedback, coaching, and timely evaluations of performance to promote cooperation, teamwork, knowledge/skill sharing and goal accomplishment. Behavior described in evaluation narrative: “Throughout the year you provided employees with regular, specific performance feedback geared to improve and enhance performance, in both verbal and written form.” Critical job responsibility: Employee satisfaction; Key supporting behavior: Acts on employee concerns, values, ideas and perspectives of people from diverse backgrounds. Behavior described in evaluation narrative: “He heartily endorsed Survey 2000 participation and was rewarded by increased levels of employee satisfaction as reflected in survey results.” Critical job responsibility: Business results; Key supporting behavior: Develops and executes plans to achieve organizational goals, leveraging available resources (e.g., human, financial, etc.) to maximize efficiency and produce high-quality results. Behavior described in evaluation narrative: “In addition to your responsibilities to the offer program, you have ensured your employees have supported the walk-in program when needed.” Critical job responsibility: Business results; Key supporting behavior: Learns about current and emerging issues/developments in own field of expertise and applies knowledge to make technically sound operational decisions. Behavior described in evaluation narrative: “Because of his recognized technical expertise, he took an active part in training during the year and his employees and peers relied upon him to clarify, explain, and reinforce complicated or newly enacted tax law.” Critical job responsibility: Equal employment opportunity; Key supporting behavior: Promptly responds to allegations of discrimination and/or harassment and initiates appropriate action to address the situation. Behavior described in evaluation narrative: “You are also sensitive to the work environment and bring any potential problems or issues to the attention of the appropriate people.” Critical job responsibility: Equal employment opportunity; Key supporting behavior: Supports staff participation in special emphasis programs. Behavior described in evaluation narrative: “He makes sure that his employees have the opportunity to participate in EEO and diversity programs, and demonstrates his commitment by attending himself.” [End of table] [End of section] Appendix V: Distribution of Raters’ Comments for Each Supporting Behavior: This appendix provides information on the distribution of raters’ comments among the behaviors supporting each critical job responsibility. The supporting behaviors listed for each critical job responsibility are taken directly from IRS’s evaluation form, shown in appendix III. Table 12: Distribution of Raters’ Comments: Critical job responsibilities and supporting behaviors: Customer satisfaction: Communicates to employees the importance of customer focus as a critical component of IRS’ mission; Percentage of managers evaluated on each supporting behavior, FY 2000: 25; Percentage of managers evaluated on each supporting behavior, FY 2001: 19. Customer satisfaction: Constantly listens to customers, analyzes their feedback to identify their needs and expectations, and acts to continuously improve products and services; Percentage of managers evaluated on each supporting behavior, FY 2000: 46; Percentage of managers evaluated on each supporting behavior, FY 2001: 56. Customer satisfaction: Insures employees do the same (as immediately above) and that they are prompt, professional, fair, and responsive to the circumstances of individual customers to the extent permitted by law and regulation; Percentage of managers evaluated on each supporting behavior, FY 2000: 47; Percentage of managers evaluated on each supporting behavior, FY 2001: 47. Customer satisfaction: Identifies and utilizes policies, and economic, political and social trends in an effort to improve organizational performance; Percentage of managers evaluated on each supporting behavior, FY 2000: 1; Percentage of managers evaluated on each supporting behavior, FY 2001: 0. Customer satisfaction: Builds strong alliances, involving stakeholders (for example NTEU, internal customers, suppliers, etc.) in making decisions, and gaining cooperation to achieve mutually satisfying solutions; Percentage of managers evaluated on each supporting behavior, FY 2000: 37; Percentage of managers evaluated on each supporting behavior, FY 2001: 55. Customer satisfaction: Initiates actions and manages risks to develop new products and services within or outside the organization; Percentage of managers evaluated on each supporting behavior, FY 2000: 9; Percentage of managers evaluated on each supporting behavior, FY 2001: 7. Customer satisfaction: Shares innovations with others; Percentage of managers evaluated on each supporting behavior, FY 2000: 0; Percentage of managers evaluated on each supporting behavior, FY 2001: 1. Business results: Pursues business excellence through effective process management and the application of balanced measures; Percentage of managers evaluated on each supporting behavior, FY 2000: 78; Percentage of managers evaluated on each supporting behavior, FY 2001: 87. Business results: Develops and executes plans to achieve organizational goals, leveraging available resources (human, financial, etc.) to maximize efficiency and produce high-quality results; Percentage of managers evaluated on each supporting behavior, FY 2000: 68. 64; Percentage of managers evaluated on each supporting behavior, FY 2001: Business results: Identifies and analyzes problems to resolve individual and organizational issues in accordance with law, regulation, and IRS policy; Percentage of managers evaluated on each supporting behavior, FY 2000: 18; Percentage of managers evaluated on each supporting behavior, FY 2001: 15. Business results: Continuously seeks to improve business processes, sharing those efforts with other units to better overall IRS performance; Percentage of managers evaluated on each supporting behavior, FY 2000: 21; Percentage of managers evaluated on each supporting behavior, FY 2001: 25. Business results: Takes steps to prevent waste, fraud and abuse and instill public trust; Percentage of managers evaluated on each supporting behavior, FY 2000: 3; Percentage of managers evaluated on each supporting behavior, FY 2001: 7. Business results: Learns about current and emerging issues/developments in own field of expertise and applies knowledge to make technically sound operational decisions; Percentage of managers evaluated on each supporting behavior, FY 2000: 3; Percentage of managers evaluated on each supporting behavior, FY 2001: 2. Business results: Understands and uses organizational realities, networks and accepted practices to achieve desired results; Percentage of managers evaluated on each supporting behavior, FY 2000: 2; Percentage of managers evaluated on each supporting behavior, FY 2001: 3. Employee satisfaction: Ensures that a safe, healthy work environment is maintained; Percentage of managers evaluated on each supporting behavior, FY 2000: 8; Percentage of managers evaluated on each supporting behavior, FY 2001: 16. Employee satisfaction: Demonstrates importance of employee satisfaction in accomplishing IRS’ mission; Percentage of managers evaluated on each supporting behavior, FY 2000: 7; Percentage of managers evaluated on each supporting behavior, FY 2001: 2. Employee satisfaction: Motivates employees to achieve high performance through empathetic, open and honest communication, by involving them in decision making, and ensuring that they have the tools and training to perform their jobs; Percentage of managers evaluated on each supporting behavior, FY 2000: 82; Percentage of managers evaluated on each supporting behavior, FY 2001: 85. Employee satisfaction: Effectively uses ongoing feedback, coaching, and timely evaluations of performance to promote cooperation, teamwork, knowledge/skill sharing and goal accomplishment; Percentage of managers evaluated on each supporting behavior, FY 2000: 78; Percentage of managers evaluated on each supporting behavior, FY 2001: 81. Employee satisfaction: Develops and recognizes employees so that they realize their full potential; Percentage of managers evaluated on each supporting behavior, FY 2000: 62; Percentage of managers evaluated on each supporting behavior, FY 2001: 80. Employee satisfaction: Acts on employee concerns, values, ideas and perspectives of people from diverse backgrounds; Percentage of managers evaluated on each supporting behavior, FY 2000: 65; Percentage of managers evaluated on each supporting behavior, FY 2001: 63. Employee satisfaction: Ensures all employees are treated in a fair and equitable manner; Percentage of managers evaluated on each supporting behavior, FY 2000: 11; Percentage of managers evaluated on each supporting behavior, FY 2001: 12. Employee satisfaction: Creates an environment for continuous learning, pursuing development opportunities for self and others, with the intent to increase individual and organizational effectiveness; Percentage of managers evaluated on each supporting behavior, FY 2000: 38; Percentage of managers evaluated on each supporting behavior, FY 2001: 38. Leadership: Demonstrates integrity and the highest ethical standards of public service; Percentage of managers evaluated on each supporting behavior, FY 2000: 11; Percentage of managers evaluated on each supporting behavior, FY 2001: 7. Leadership: Develops, prioritizes and aligns strategies, objectives and goals, taking into account key influences on organizational performance; Percentage of managers evaluated on each supporting behavior, FY 2000: 10; Percentage of managers evaluated on each supporting behavior, FY 2001: 5. Leadership: Successfully leads organizational change, effectively communicating IRS’ mission, core values, and strategic goals to employees and other stakeholders and responding creatively to changing circumstances; Percentage of managers evaluated on each supporting behavior, FY 2000: 52; Percentage of managers evaluated on each supporting behavior, FY 2001: 57. Leadership: Creates and sustains a positive workplace that inspires others to support IRS’ mission and goals; Percentage of managers evaluated on each supporting behavior, FY 2000: 15; Percentage of managers evaluated on each supporting behavior, FY 2001: 12. Leadership: Shows a commitment to public service and citizenship; Percentage of managers evaluated on each supporting behavior, FY 2000: 3; Percentage of managers evaluated on each supporting behavior, FY 2001: 3. Leadership: Uses sound judgment to make effective and timely decisions; Percentage of managers evaluated on each supporting behavior, FY 2000: 6; Percentage of managers evaluated on each supporting behavior, FY 2001: 3. Equal employment opportunity: Takes steps to implement EEO and affirmative employment goals established by IRS; Percentage of managers evaluated on each supporting behavior, FY 2000: 32; Percentage of managers evaluated on each supporting behavior, FY 2001: 28. Equal employment opportunity: Supports staff participation in special emphasis programs; Percentage of managers evaluated on each supporting behavior, FY 2000: 31; Percentage of managers evaluated on each supporting behavior, FY 2001: 34. Equal employment opportunity: Promptly responds to allegations of discrimination and/or harassment and initiates appropriate action to address situation; Percentage of managers evaluated on each supporting behavior, FY 2000: 7; Percentage of managers evaluated on each supporting behavior, FY 2001: 5. Equal employment opportunity: Cooperates with EEO counselors, investigators, and other officials responsible for conducting inquires into EEO complaints; Percentage of managers evaluated on each supporting behavior, FY 2000: 8; Percentage of managers evaluated on each supporting behavior, FY 2001: 9. Equal employment opportunity: Assigns work and makes employment decisions in areas such as hiring, promotion, training and developmental assignments without regard to sex, race, color, national origin, religion, age, disability, sexual orientation or prior participation in EEO process; Percentage of managers evaluated on each supporting behavior, FY 2000: 26; Percentage of managers evaluated on each supporting behavior, FY 2001: 25. Equal employment opportunity: Monitors work environment to prevent instances of prohibited discrimination and/or harassment; Percentage of managers evaluated on each supporting behavior, FY 2000: 18; Percentage of managers evaluated on each supporting behavior, FY 2001: 14. Note: Percentages are rounded to the nearest percent. Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ written performance evaluations. [End of table] [End of section] Appendix VI: Examples of Group Managers’ Commitments: This appendix provides examples of commitments written by enforcement group managers in our sample. We found that almost all commitments met IRS’s criteria of being clear and achievable, but about 85 percent did not meet the criteria of being specific, outcome or output oriented, or easily monitored. The examples of commitments are grouped into two categories, those that were specific, outcome or output oriented, and easily monitored, and those that were not. Commitments that Were Specific, Outcome or Output Oriented, or Easily Monitored: We judged that the following commitments were specific, outcome or output oriented, or easily monitored. “Use of Balanced Measures Matrix will be demonstrated on one case or situation each quarter.” “I will meet on a quarterly basis with the Office Assistant to discuss issues relevant to his job performance.” “I will have monthly group meetings to discuss case issues, Survey 99 updates, RRA updates, FY 2000 Operating Plan activities, and group concerns.” “I will review and update IDPs (Individual Development Plans) on a semi- annual basis to monitor accomplishments and modify the existing development plans.” “I will give a presentation to Group __ and/or Branch II managers about what I have learned about organizing.” “By 03/31/2000 I will conduct refresher training/briefings for my employees in Group __ on TFRP (Trust Fund Recovery Penalty) procedures.” “I will continue to conduct monthly meetings with the employees to communicate changes and any impact.” “I will conduct quarterly reviews of casework focusing on IRM (Internal Revenue Manual) requirements and CQMS (Collection Quality Measurement System) measures to ensure timely closing of cases.” “To improve employee satisfaction, I will keep employees informed of their performance by preparing timely mid-year and annual appraisals supported with sufficient recordation.” “Walk-in customer service will be provided at four outreach sites; Sioux City, IA, Storm Lake, IA, O’Neill, NE and Columbus, NE. These sites will be staffed for one day every other week from January through April 2000.” To improve Employee Satisfaction, I will ensure I provide timely and constructive feedback to employees. I will do sample case reviews semimonthly and quarterly bundle reviews and/or other written recordation concerning individual performance and timely completion of mid-year and annual appraisals.” Commitments that Were Not Specific, Outcome or Output Oriented, or Easily Monitored: The following commitments were determined not to be specific, outcome or output oriented, or easily monitored. “Support and develop employees in a manner which encourages participation by all.” “Provide leadership to my group in achieving organizational goals by the use of effective group meetings, effective review and evaluation practices, and the Individual Development process.” “I will work with the municipal governments within the State. Relationships built here will continue to homogenize State employment tax administration efforts.” “I will provide leadership by increased managerial involvement in my employee’s casework.” “I will use required group meetings to provide guidance on relevant issues.” “I will support the partnership with NTEU [National Treasury Employees Union] so that positive employee relations are maintained.” “The identification and development of fraud cases will be encouraged in my work group by creating an environment of fraud awareness.” “Provide leadership and direction to the field staff to regain appropriate levels of compliance on a case by case basis by conducting reviews, timely entity queries and follow ups.” “I will support the Field Branches as needed.” “I will ensure sufficient resources and managerial attention to support the walk-in program in my post of duty.” “I will continue to have open and honest communications with all employees.” “I will continue to be supportive of employees’ efforts to advance their careers.” “I will lead Field Group #__ through the organizational changes demanded by the modernization of the IRS by providing leadership, planning, and support to effectively transition the work and human resources of the current district structure to the new operating units.” [End of section] Appendix VII: Comments from the Internal Revenue Service: Department Of The Treasury: Commissioner: Internal Revenue Service: Washington, D.C. 20224: July 5, 2002: Mr. Michael Brostek: Director, Tax Issues U.S. General Accounting Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Brostek: Thank you for the opportunity to review and comment on your draft report entitled "Performance Management Systems: IRS's Systems for Front-line Employees and Managers Align with Strategic Goals but Improvements Can Be Made." Upon review, we were pleased the report recognized the great strides the IRS has made in redesigning and modernizing its Performance Management System to complement strategic goals and objectives. This is nothing short of a major cultural change for IRS. While your study was underway, we implemented many improvements. For example, as part of recently completed negotiations with the National Treasury Employees Union (NTEU) over a new Servicewide collective bargaining agreement, we dramatically reduced the administrative burden associated with the employee appraisal process and provided a streamlined dispute resolution mechanism for appraisal grievances. Beginning with the FY 2002 appraisal cycle, we established new Critical Job Elements for every IRS employee that align directly with our Balanced Measures System and implemented a new National Performance Awards Agreement that eliminates over 100 local awards agreements negotiated over the last decade. In addition, we issued a new guide to managers on writing commitments and self-assessments and established an electronic "Performance Management Resource Center" on the Service's Intranet that includes detailed guidance on all of the above initiatives. Your report cites two primary recommendations. First you suggested that we improve the linkage between front-line employees' critical job responsibilities, the supporting behaviors, and organizational unit performance measures. Second, you recommended that we establish a plan for monitoring and assessing whether the new employee performance management systems are operating as intended and take the necessary actions to resolve any identified problems. As a general matter, we concur with your recommendations and will include them as we continuously improve our Performance Management System. In this regard, we currently use Performance Review Boards to ensure that the performance agreements and evaluations of our executives and senior managers align with and reflect the Service's strategic goals, annual business plans, and organizational performance measures. While the numbers involved make a comparable process less practical at lower levels of the organization, we will explore other means of insuring this alignment. As you know, any impact on bargaining unit employees will require negotiations with the NTEU. In addition, you can rest assured that we will continue to monitor the overall efficacy of our Performance Management System to determine what works and what needs refinement. For your information, this assessment occurs as a matter of course, as part of the Service's comprehensive annual Strategic Planning and Budgeting process (that process includes a strategic human capital assessment), as well as in the regular and recurring Business Performance Reviews of our corporate and operating division strategic human resources functions. By providing for such oversight and continuous program improvement, we believe that we are well on our way to developing and deploying a "best practice" Performance Management System that truly distinguishes between high and low performers at all levels of the organization. If you have any further questions, you should feel free to contact Ronald P. Sanders, the Service's Chief Human Resource Officer at (202) 283-9200. Signed by: [Illegible] for: Charles O. Rossotti: [End of section] Appendix VIII: GAO Contacts and Staff Acknowledgments: GAO Contacts: Michael Brostek (202) 512-9039: Ralph Block (415) 904-2150: Jonda Van Pelt (415) 904-2186: Acknowledgments: In addition to those named above, Michelle Bowsky, Maya Chakko, Elizabeth Fan, Tre Forlano, Diana Hu, Ann Lee, and Samuel Scrutchins made key contributions to this report. [End of section] Footnotes: [1] The mission of IRS is to “provide American taxpayers top quality service by helping them understand and meet their tax responsibilities, and by applying the tax laws with integrity and fairness to all.” [2] See Appendix IV for examples of evaluations addressing group managers’ critical job responsibilities. [3] Enforcement group managers are those managers in IRS’s field offices that supervise frontline employees, such as revenue agents who directly deal with taxpayers when auditing tax returns and revenue officers who deal with taxpayers when collecting unpaid taxes. [4] We express the precision of the results of our analysis of group managers’ commitments and evaluations, and the survey (that is, the sampling errors associated with these estimates) as 95 percent confidence intervals. All percentage estimates dealing with our review of evaluations have sampling errors of +/- 11 percentage points or less, unless otherwise stated. All percentage estimates dealing with the group manager survey have sampling errors of +/- 12 percentage points or less, unless otherwise stated. [5] Office of Personnel Management regulations define performance management as the integrated processes agencies use to (1) communicate and clarify organizational goals, (2) identify accountability for accomplishing organizational goals, (3) identify and address developmental needs, (4) assess and improve performance, (5) measure performance for recognizing and rewarding accomplishments, and (6) prepare appraisals. [6] In July 1999 IRS incorporated into the evaluation process a new retention standard relating to the fair and equitable treatment of taxpayers that managers and frontline employees must meet at a passing level to retain their jobs. When assessing managers and employees, raters are to first determine whether employees met the standard and, if the employees did, then proceed to prepare a written evaluation of how the employees met their critical job responsibilities. [7] In our analysis, we only counted the number of commitments that aligned with each of the critical job responsibilities. We did not try to assess the significance or relative importance of each commitment. [8] IRS’s strategic guidance calls for periodic evaluations. For the purposes of this report, we are using the term assessment. [9] We express the precision of the results of our analysis of group managers’ commitments and evaluations (that is, the sampling errors associated with these estimates) as 95 percent confidence intervals. [10] Our assessment was limited to a count of the number of commitments, not the significance or the relative challenge of accomplishing them. [11] As with our analysis of managers’ commitments and performance evaluations, we express the precision of the results of our survey (that is, the sampling errors associated with these estimates) as 95 percent confidence intervals. [End of section] GAO’s Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site [hyperlink, http://www.gao.gov] contains abstracts and fulltext files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to [hyperlink, http://www.gao.gov] and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov: (202) 512-4800: U.S. General Accounting Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: