Tax Administration:

IRS Needs to Further Refine Its Tax Filing Season Performance Measures

GAO-03-143: Published: Nov 22, 2002. Publicly Released: Nov 22, 2002.

Additional Materials:

Contact:

James R. White
(202) 512-5594
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The tax-filing season, roughly January 1 through April 15, is when most taxpayers file their returns, receive refunds, and call or visit IRS offices or the IRS Web site with questions. To provide better information about the quality of filing season services, IRS is revamping its suite of filing season performance measures. Because the new measures are part of a strategy to improve service and because filing season service affects so many taxpayers, GAO was asked to assess whether the new measures have the four characteristics of successful performance measures graphically depicted below.

In assessing 53 performance measures across IRS's four program areas, GAO found that IRS has made significant efforts to improve its performance measurement system. Many of the measures satisfied some of the four key characteristics of successful performance measures established in earlier GAO work. Although improvements are ongoing, GAO identified instances where measures showed weaknesses including the following: (1) The objectivity and reliability of some measures could be improved so that they will be reasonably free from significant bias and produce the same result under similar circumstances. For example, survey administrators may notify Telephone Assistance's customer service representatives (CSR) too soon that their call was selected to participate in the customer satisfaction survey, which could bias CSR behavior towards taxpayers and adversely affect the measure's objectivity. In addition, the measure Electronic Filing and Assistance uses to determine the number of Web site hits was not reliable because it did not represent the actual number of times the Web site is accessed. (2) The clarity of some performance information was affected when that measure's definition and formula were not consistent. For example, the definition for "CSR response level" measure is the percentage of callers who receive service from a CSR within a specified period of time, but the measure did not include callers who received a busy signal or hung up. (3) Some suites of measures did not cover government-wide priorities such as quality, timeliness, and cost of service. For example, Field Assistance was missing measures for timeliness and cost of service.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: During their fiscal year 2003 Leadership Conference, IRS managers reviewed the agencywide and operating division goals and measures, ensured there was alignment among them, and documented their relationship--beginning in the fiscal year 2005 Strategy and Program Plan. According to the IRS, the Strategy and Program Plan was shared throughout the organization to ensure communication at all levels. By implementing this recommendation, the IRS has created a bridge between long-term strategic goals and short-term daily operational goals, which can, among other things, be used for holding the IRS and the key filing season program areas accountable for achieving its goals. Further, the link to specific measures provides additional information needed to clearly communicate the alignment of goals and measures throughout the agency, as well as the level of performance the operating division hopes to achieve.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to take steps to ensure that agencywide goals clearly align with operating division goals and performance measures for each of the four areas reviewed. Specifically, (1) clearly document the relationship among agencywide goals, operating division goals, and performance measures (the other three program areas may want to consider developing a template similar to the one field assistance developed, shown in figure 4) and (2) ensure that the relationship among goals and measures is communicated to staff at all levels of the organization.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  2. Status: Closed - Implemented

    Comments: In its 2003 report, GAO recommended that the IRS make the name and definition of several field assistance measures (i.e., "return preparation contacts," "return preparation units," "TACS total contact," "form contacts," "tax law contacts," "other contacts," "tax law accuracy," "accounts/notices accuracy," and "return preparation accuracy") clearer to indicate what is and is not included in the formula. Field Assistance implemented GAO's recommendation by updating its "data dictionary," which is used in reporting measures to all levels of the organization and specifically identifies what is or is not included in the formulas. In addition, the data dictionary now includes the following categories: operating division, measure name, type of measure, program category, related strategic goal, definition, reporting level(s), reporting data source, reports, formula/methodology, data source/measurement tools, reliability of data, frequency of data availability/reporting, purpose of measure, data limitations, calculation changes, complete description of the process, critical path, and management controls for items on critical path. The measures also include alternate sites in the definition of a Taxpayer Assistance Center, such as a mobile unit. These modifications are substantial and provide significant additional information about the measures. As a result, the measures have clarity and can be meaningful to stakeholders.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to make the name and definition of several field assistance measures (i.e., "geographic coverage," "return preparation contacts," "return preparation units," "TACs total contacts," "forms contacts," "tax law contacts," "account contacts," "other contacts," "tax law accuracy," "accounts/notices accuracy," and "return preparation accuracy") more clear to indicate what is and is not included in the formula.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  3. Status: Closed - Implemented

    Comments: Initially, IRS disagreed with this recommendation, stating that the formula accurately reflected the percentage of customers that wanted to speak to a Customer Service Representative (CSR) and subsequently received service. However, automated services have increased as an alternative to CSR services and taxpayers have selected automated services before entering a CSR queue. Thus, while automated calls answered remains part of the CSR Level of Service formula, the number of these calls included in the computation is minimal, if any, according to a December 2006 interview with IRS officials. As a result, the measure's clarity has improved because the name is more closely aligned to the measure's definition since the number of automated calls is minimal. Also, transparency to users and stakeholders has improved, leading to fewer misinterpretations of results or failures to take proper action to resolve performance problems.

    Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: remove automated calls from the formula for the "CSR level of service" measure.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  4. Status: Closed - Implemented

    Comments: IRS initially disagreed with this recommendation because, according to officials, the Customer Service Representative (CSR) Response Level measure was in accordance with industry standards. However, IRS took alternative corrective action by eliminating the measure from Strategy and Program plan and limited its use to an internal diagnostic tool. IRS officials confirmed this change in a December 2006 interview stating that they have no plans to put it back in the Strategy and Program Plan. By removing the measure from this important document, which is presented or available to external stakeholders such as the Congress, IRS has effectively limited external reliance or possible misinterpretation of this information, while still making it available to IRS.

    Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: revise the "CSR response level" measure to include calls from taxpayers who tried to reach a customer service representative (CSR) but did not, such as those who (1) hung-up while waiting to speak to a CSR, (2) were provided access only to automated services and hung up, and (3) received a busy signal.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  5. Status: Closed - Implemented

    Comments: IRS agreed with our recommendation, but instead of revising the Customer Service Representative (CSR) Services Provided measure, it took alternative corrective action by making scripting changes that have reduced transfer rates between fiscal years 2002 and 2006 by an average of almost 10 percent. IRS also better explained how transferred calls were counted in its fiscal year 2005 data dictionary. As a result, the measure is clearer and IRS has been able to reduce inefficiencies associated with transferred calls.

    Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: analyze and use new or existing data to determine why calls are transferred and use the data to revise the "CSR services provided" measure so that it only reflects transferred calls in which the caller received help from more than one CSR (i.e., exclude calls in which a CSR simply transferred the call and did not provide service.)

    Agency Affected: Department of the Treasury: Internal Revenue Service

  6. Status: Closed - Implemented

    Comments: IRS has implemented two new indicators to gauge Website traffic: "Number of visits" shows Website access by counting as "visits" a series of actions that begins when a visitor views his/her first page on IRS.gov and ends when the visitor leaves the site or remains idle beyond 30 minutes, while "Number of page views" counts whenever any given web page is accessed by a visitor to IRS.gov. Information on Website hits is still collected, but is used by IRS only for load balancing of servers.

    Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: either discontinue use of the "number of IRS digital daily Web site hits" measure or revise the way "hits" are calculated so that the measure more accurately reflects usage.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  7. Status: Closed - Implemented

    Comments: While IRS initially agreed that there was a need to revise this measure, the Service eventually eliminated the measure under IRS's Taxpayer Assistance Center Closure Project. The FY 2006-2007 Wage and Investment Strategy and Program Plan and 2006 data dictionary do not contain this measure. By eliminating the measure from these important external documents, IRS has effectively addressed GAO's underlying concern that it could be providing misleading information about agency performance to members of Congress or other external stakeholders.

    Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: revise field assistance's "geographic coverage" measure by ensuring that the formula better reflects (1) the various types of field assistance facilities, including alternate sites and kiosks; (2) the types of services provided by each facility; and (3) the facility's operating hours.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  8. Status: Closed - Implemented

    Comments: IRS agreed that the Submission Processing productivity measure needed improvement. While IRS has not revised the measure as GAO specifically recommended, Submission Processing officials reported that they have taken steps to address the underlying problems with this measure, which were that field managers did not understand it and therefore did not use it. They did this by holding multiple productivity meetings and conference calls with managers during fiscal year 2006. As a result, field managers have a better understanding of the measure and can better use it to help alert them to problems and/or respond when problems arise.

    Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: revise submission processing's "productivity" measure so it provides more meaningful information to users.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  9. Status: Closed - Implemented

    Comments: The IRS concurred with GAO's recommendation that it should only make changes to official targets under extenuating circumstances. Further it agreed that disclosing such changes is appropriate.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to refrain from making changes to official targets, such as electronic filing and assistance did in fiscal year 2001, unless extenuating circumstances arise. Disclose any extenuating circumstances in the Strategy and Program Plan and other key documents.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  10. Status: Closed - Implemented

    Comments: GAO found that the customer satisfaction survey, which is the basis of the telephone customer service measure, was not administered properly. Specifically, potential bias existed because the survey administrators did not follow the sampling procedures, thus distorting the sample's randomness and the overall objectivity of the measure. The IRS has taken corrective action by (1) instructing all administering sites to use the sampling procedures and (2) revising the procedures to make them easier to use. Further, GAO recommended that the IRS modify its procedures for the toll-free customer satisfaction survey, possibly by requiring that administrators listen to the entire call, to better ensure that administrators (1) notify Customer Service Representatives that their call was selected for the survey as close to the end as possible and (2) can accurately answer the questions they are responsible for on the survey. In August 2003, the IRS notified GAO that it had instructed the administrators to listen to each call from its beginning to as close the conclusion as practical. They concurred that formalizing this practice would enable the administrators to accurately answer the survey questions.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to modify procedures for the toll-free customer satisfaction survey, possibly by requiring that administrators listen to the entire call, to better ensure that administrators (1) notify CSRs that their call was selected for the survey as close to the end of a call as possible and (2) can accurately answer the questions they are responsible for on the survey.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  11. Status: Closed - Implemented

    Comments: In 2003, IRS revised four of its five external quality measures for telephone assistance into two measures: "customer accuracy for toll-free tax law" and "customer accuracy for toll-free accounts." The fifth measure, a timeliness measure, was changed to an internal measure. At the time of these changes, the Service also put in place numerous checks and controls for the revised measures to ensure the accuracy of the data collected. Controls include daily and weekly validity checks, a "rebuttal system whereby call sites can challenge the review and coding of individual calls, and unique consistency reviews. IRS has, therefore, addressed GAO's underlying concern behind this recommendation, that there could be problems with the reliability of the data collected to report on the measures.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to implement annual effectiveness studies to validate the accuracy of the data collection methods used for the five telephone measures ("toll-free tax law quality," "toll-free accounts quality," "toll-free tax law correct response rate," "toll-free account correct response rate," and "toll-free timeliness") subject to potential consistency problems. The studies could determine the extent to which variation exists in collecting data and recognize the associated impact on the affected measures. For those measures, and for the five submission processing measures that already have effectiveness studies in place ("refund timeliness individual (paper)," "notice error rate," "refund error rate-- individual (paper)," "letter error rate," and "deposit error rate"), IRS should establish goals for improving consistency, as needed.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  12. Status: Closed - Implemented

    Comments: In fiscal year 2003, IRS officials reviewed all measures to verify that they were relevant and specific. Of the 10 overlapping measures GAO identified, four were removed from the Strategy and Program Plan, including "return preparation units," "automated completion rate," " toll-free tax law correct response rate," and "toll-free account correct response rate." Based on IRS management's judgment, measures that provided basically the same information as another measure were removed. Consequently, managers no longer have to sort through redundant, costly information that might not add value or cloud the bottom line.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to ensure that plans to remove overlapping measures in telephone and field assistance are implemented.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  13. Status: Closed - Implemented

    Comments: IRS responded that the development of cost of service measures was dependent on the Servicewide deployment of the Integrated Financial System (IFS). In 2007, IRS was still developing cost estimates for the Chief Financial Officer's IFS. However, as part of its 2007 Taxpayer Assistance Blueprint, IRS prepared unit cost per contact estimates based on fiscal year 2005 data for the major IRS taxpayer services and channels. While IRS reported that these estimates do not fully allocate all indirect overhead and support costs such as headquarters staffing, information technology modernization projects, and centralized real estate facilities procurement and employee services, they represent the best and most comprehensive estimates currently available. As a result of these initial steps, IRS has better achieved governmentwide priorities and balance of its measures.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: in the spirit of provisions in the Chief Financial Officers Act of 1990 and Financial Accounting Standards Number 4, develop a cost of services measure using the best information currently available for each of the four areas discussed in this report, recognizing data limitations as prescribed by GPRA. In doing so, adhere to guidance, such as Office of Management and Budget Circular A-76, and consider seeking outside counsel to determine best or industry practices.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  14. Status: Closed - Not Implemented

    Comments: IRS has not developed a customer satisfaction survey or measure for its automated telephone assistance and does not intend to develop one. While IRS has supported this recommendation for the past 7 years, it now is reporting that such a survey to be a non-critical enhancement to its taxpayer services and thus the funding has not and is not available to implement it.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: given the importance of automated telephone assistance, develop a customer satisfaction survey and measure for automated assistance.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  15. Status: Closed - Implemented

    Comments: Although IRS did not put the measure back into the Strategy and Program Plan (SPP), it made enhancements to TeleTax, IRS's automated telephone information system, which resulted in the ability to capture the number of automated calls that were completed. IRS tracks this enhancement as part of its Automated Calls Answered measure, which is in the SPP. The fiscal year 2005 through 2006 SPP confirms that this enhancement was effective. As a result, taxpayer behavior is tracked and reported in an important document available to Congress and external stakeholders, which better gauges the automated service IRS provides to taxpayers.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: put the "automated completion rate" measure back in the Strategy and Program Plan after revising the formula so that calls for recorded tax law information are not counted as completed when taxpayers hang up before receiving service.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  16. Status: Closed - Not Implemented

    Comments: IRS disagreed with this recommendation, noting that adding one or more quality measures to electronic filing and assistance's suite of measures in the Strategy and Program Plan would not enhance the electronic filing program. Accordingly, the most recent editions of Wage and Investment's Data Dictionary (FY 2006) and Strategy and Program Plan (FY 2006- FY 2007) do not contain a quality measure(s) on electronic filing (such as "processing accuracy," "refund timeliness, electronically filed," or "number of electronic returns rejected").

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: add one or more quality measures to electronic filing and assistance's suite of measures in the Strategy and Program Plan. Possible measures include "processing accuracy," "refund timeliness, electronically filed," and "number of electronic returns rejected."

    Agency Affected: Department of the Treasury: Internal Revenue Service

  17. Status: Closed - Implemented

    Comments: By the 2006 tax filing season, IRS had a system in place for monitoring and managing timeliness of customer service at walk-in sites, which provided coverage for 81 percent of all walk-in site traffic. Managers at sites equipped with the "full" Q-Matic system can monitor traffic in real-time, specifically wait-times. With this additional information and capability, IRS is better able to balance customers' need for accurate information against customers' needs for timely service at walk-in sites. IRS has, therefore, addressed GAO's underlying concern behind this recommendation, that field assistance could be emphasizing customers' need for accurate service at the expense of their need for timely service.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: re-implement field assistance's timeliness measure.

    Agency Affected: Department of the Treasury: Internal Revenue Service

  18. Status: Closed - Implemented

    Comments: IRS implemented a diagnostic tool to monitor Field Assistance's efficiency. We still believe that an organization performance measure that would be visible throughout IRS and to Treasury and congressional stakeholders would be preferable. However, IRS officials confirmed in September 2007 that they find the efficiency diagnostic tool useful and that they monitor frequently. As a result, we agree that IRS's implementation of an efficiency diagnostic tool is a good step for direct mangers to monitor Field Assistance's productivity and to use to identify its strengths, weakness, and areas for improvement.

    Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: develop a measure that provides information about field assistance's efficiency.

    Agency Affected: Department of the Treasury: Internal Revenue Service

 

Explore the full database of GAO's Open Recommendations »

Nov 19, 2014

Sep 22, 2014

Sep 18, 2014

Aug 11, 2014

Jul 29, 2014

Jul 22, 2014

Jul 18, 2014

Jul 7, 2014

Jul 2, 2014

Jun 13, 2014

Looking for more? Browse all our products here