Tax Administration:

IRS Needs Better Strategic Planning and Evaluation of Taxpayer Assistance Training

GAO-05-782: Published: Jul 11, 2005. Publicly Released: Aug 11, 2005.

Additional Materials:

Contact:

James R. White
(202) 512-5594
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Millions of taxpayers ask IRS questions about tax law each year. While the accuracy of IRS's answers has improved in some cases, it is still not always what taxpayers or Congress expect. Concerns about accuracy have raised questions about the adequacy of the training IRS provides to its taxpayer assistance staff. Because of these questions, GAO was asked to assess the extent to which IRS's planning and evaluation of its taxpayer assistor training conformed to guidance published by GAO and others. Planning and evaluation are part of a feedback loop whereby lessons from one year can be applied to making improvements in future years.

IRS devotes significant resources to training its tax law assistors to answer questions, by telephone and at walk-in sites, and prepare tax returns. Although IRS cannot separate the costs of training tax law assistors from other assistance staff, the thousands of staff devoted to providing tax law assistance receive training each year. The training includes classroom and computer-based training on such subjects as tax law and communication. While IRS has some data on travel and course development costs associated with training, it does not have data on what is likely the largest cost component, the value of staff time devoted to tax law training. Responsibility for training IRS taxpayer assistance staff is decentralized. IRS's Human Capital Office provides guidance and sets policy. The two divisions responsible for tax law assistance each have a human resources office with technical staff that are assigned to the various taxpayer assistance programs. Generally speaking, the taxpayer assistance programs share responsibility for planning training with human resource staff. The human resource staff are responsible for training evaluations. IRS's planning for taxpayer assistor training could be enhanced by a more strategic approach. To their credit, some taxpayer assistance programs clearly communicated the importance of training to staff and had knowledge and skills inventories. All the units had analyses of some of the individual factors that affect accuracy, such as the quality and use of their taxpayer assistance guidance. However, none of the programs had long-term goals for either accuracy or training or had benchmarked their training efforts against those of other organizations. Nor had they done a combined analysis of the major factors that affect accuracy in order to determine their relative importance. Setting long-term goals and analyzing training needs and relative impacts are important steps in strategic planning. Goals can provide a yardstick for measuring progress and benchmarking can help identify best practices. Analyses of relative impacts can help IRS make informed decisions about a strategy for improving accuracy, including the importance of training compared to other factors in that strategy. The taxpayer assistance programs routinely conducted evaluations of their training efforts. However, with one exception, the evaluations did not include analyses of the impact of training on the accuracy of assistance. Instead, the units conducted less sophisticated analyses of more immediate impacts, such as trainees' satisfaction. Given the importance of accurate answers to taxpayers' questions and the resources spent on training, the four assistance programs would benefit from more sophisticated evaluations of the effectiveness of training. One program had recognized the potential value of a more sophisticated evaluation of training and pilot tested an analysis in 2004. The value of evaluation is that it provides feedback about the effectiveness of one year's training that can be used to plan improvements to future training.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: Based on our recommendation, IRS established a long-term goal for the accuracy of taxpayer assistance as of November 15, 2006. Based on information from IRS, the goal will be tracked through IRS's Wage and Investment Division's Accounts Management and Assistance program and includes fiscal year targets for customer accounts resolved, toll-free tax law, and toll-free accounts.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should establish a long-term goal for the accuracy of taxpayer assistance.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  2. Status: Closed - Implemented

    Comments: Based on our recommendation, IRS established a long-term goal for the accuracy of taxpayer assistance, including fiscal year targets and measures for customer accounts resolved, toll-free tax law, and toll-free accounts. The goal and related targets and measures were completed as of November 15, 2006.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should establish goals and measures for training and development logically linked to accuracy.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  3. Status: Closed - Implemented

    Comments: In July 2008, IRS expanded the use of its Embedded Quality Reports System (EQRS)to Taxpayer Service functions. Using EQRS allows for a more direct assessment of training effectiveness as it relates to performance of specific applications and case attributes. EQRS includes links to key course content objectives for a more direct assessment of training effectiveness as it relates to performance of specific applications and case types.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should determine and track the relative importance of the various factors, including training, that affect accuracy.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  4. Status: Closed - Implemented

    Comments: In May 2009, IRS released its benchmarking study on training and development. The vast majority of the data collected was obtained through research conducted by the Corporate Leadership Council. The Council conducted a comprehensive search of published materials, augmented by IRS's Quality and Performance Management office. Data was collected from approximately 13 organizations determined to be most closely aligned with IRS. Significant findings for IRS to attain the next level of training effectiveness include IRS to continue to (1) focus on measurable training impact, (2)link strategic business objectives to training, and (3) collect customer satisfaction data.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should benchmark training and development programs against high-performing organizations.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  5. Status: Closed - Implemented

    Comments: IRS implemented the field assistance technical assistance battery (FA TAB) in June 2006; all employees are expected to complete the assessment. The FA TAB helps to identify gaps in training and/or reference materials based on compiled FA TAB results so that gaps may be filled to meet the needs of employees, thus improving quality. According to IRS, the implementation of the FA TAB indicates that the values of such assessments are inculcated into its near-term goals and long-term strategies.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should conduct skills and knowledge gap analyses for all taxpayer assistance programs, to include identifying and comparing current skills to long-term skill needs.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  6. Status: Closed - Implemented

    Comments: According to IRS, its annual training plan formulation process allows business divisions to identify their training priorities based on strategic plans and the level of funding appropriated for that year. When the Service-wide long-term goals are finalized and approved, the training plan will be developed considering costs, benefits, risk mitigation, and appropriate levels of investment for training and development efforts.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should consider costs, benefits, ways to mitigate risks, and the appropriate level of investment for training and development efforts.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  7. Status: Closed - Implemented

    Comments: IRS has taken steps to meet the level 4 requirement. Specifically, IRS completed a level 3 evaluation prototype utilizing the embedded quality review system. IRS also used industry experts, Donald and Jim Kirkpatrick to review the evaluation. Based on feedback from the Kirkpatrick's review, the level 3 prototype was completed using a training course conducted at a variety of locations. Collectively the actual training and instructor feedback moving through the first three levels of evaluation demonstrated training value through compelling evidence. This meets the level 4 requirement.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. In the area of evaluation, IRS should continue to pursue the level 4 pilot in TEC and, if that analysis is shown to be feasible, conduct level 4 evaluations for its other taxpayer assistance training and development programs. The evaluations should include an analysis of the feasibility and cost effectiveness of alternative level 4 methodologies and a data collection and analysis plan.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  8. Status: Closed - Implemented

    Comments: IRS has taken steps to meet the level 4 requirement. Specifically, IRS completed a level 3 evaluation prototype utilizing the embedded quality review system. IRS also used industry experts, Donald and Jim Kirkpatrick to review the evaluation. Based on feedback from the Kirkpatrick's review, the level 3 prototype was completed using a training course conducted at a variety of locations. Collectively the actual training and instructor feedback moving through the first three levels of evaluation demonstrated training value through compelling evidence. This meets the level 4 requirement.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. In the area of evaluation, IRS should continue to pursue the level 4 pilot in TEC and, if that analysis is shown to be feasible, conduct level 4 evaluations for its other taxpayer assistance training and development programs. The evaluations should include a comparison of the accuracy benefits to the costs of the training.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  9. Status: Closed - Implemented

    Comments: In May 2009, IRS released its benchmarking study on training and development. The vast majority of the data collected was obtained through research conducted by the Corporate Leadership Council. The Council conducted a comprehensive search of published materials, augmented by IRS's Quality and Performance Management office. Data was collected from approximately 13 organizations determined to be most closely aligned with IRS and included benchmarking of the analytical methods and the results of the data analysis against high-performing organizations.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. In the area of evaluation, IRS should continue to pursue the level 4 pilot in TEC and, if that analysis is shown to be feasible, conduct level 4 evaluations for its other taxpayer assistance training and development programs. The evaluations should include benchmarking of the analytical methods and the results of the data analysis against high-performing organizations.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  10. Status: Closed - Implemented

    Comments: IRS has taken steps to meet the level 4 requirement. Specifically, IRS completed a level 3 evaluation prototype utilizing the embedded quality review system. IRS also used industry experts, Donald and Jim Kirkpatrick to review the evaluation. Based on feedback from the Kirkpatrick's review, the level 3 prototype was completed using a training course conducted at a variety of locations. Collectively the actual training and instructor feedback moving through the first three levels of evaluation demonstrated training value through compelling evidence. This meets the level 4 requirement.

    Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. In the area of evaluation, IRS should continue to pursue the level 4 pilot in TEC and, if that analysis is shown to be feasible, conduct level 4 evaluations for its other taxpayer assistance training and development programs. The evaluations should include an analysis of stakeholder assessments of the impact of training on accuracy.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  11. Status: Closed - Implemented

    Comments: In October 2007, IRS implemented a process to collect level 2 data and store it in the Enterprise Learning Management System (ELMS), which replaces ACES. In addition, IRS's Human Capital's Office of Learning Technology asked IRS training directors to review and update items in the ELMS course catalog to ensure they accurately reflect content which requires a level 2 evaluation.

    Recommendation: IRS should replace the defunct Administrative Corporate Education System database, which had been used to store level 2 data, with another database for this purpose.

    Agency Affected: Department of the Treasury: Internal Revenue Service

 

Explore the full database of GAO's Open Recommendations »

Sep 18, 2014

Aug 11, 2014

Jul 29, 2014

Jul 22, 2014

Jul 18, 2014

Jul 7, 2014

Jul 2, 2014

Jun 13, 2014

May 30, 2014

May 20, 2014

Looking for more? Browse all our products here