IRS Needs to Strengthen Its Approach for Evaluating the SRFMI Data-Sharing Pilot Program
GAO-09-45: Published: Nov 7, 2008. Publicly Released: Dec 8, 2008.
The State Reverse File Match Initiative (SRFMI) is one of the Internal Revenue Service's (IRS) data-sharing strategies to reduce the estimated $345 billion gross federal tax gap. SRFMI matches federal and state taxpayer data to identify noncompliant taxpayers--individuals and businesses who do not file tax returns or do not report all of their income. IRS's document-matching program has proven to be a highly cost-effective way of identifying underreported income, thereby bringing in billions of dollars of tax revenue while boosting voluntary compliance. Based on concerns about whether IRS is fully using data from state and local governments to reduce the tax gap, GAO was asked to assess IRS's plan for evaluating the effectiveness of the SRFMI pilot taxpayer data-sharing program. To address these objectives, GAO consulted social science and evaluation literature, published GAO guidance, and IRS guidance; reviewed and analyzed SRFMI planning and evaluation documents; and interviewed IRS officials about IRS's plans to evaluate SRFMI.
IRS is developing a plan for evaluating SRFMI data but does not currently have a documented plan even as the agency enters phase III of the pilot program and is less than 1 year away from rolling out the pilot to 45 states. According to IRS officials, the SRFMI pilot includes plans to evaluate program results to make decisions about expanding data sharing with states and using compliance data to assess whether SRFMI cases are more or less productive than other cases. IRS has little documentation on its evaluation. A well-developed and documented plan can help ensure that the evaluation generates performance information needed to make effective management decisions. A sound evaluation approach should also include criteria or standards for determining pilot-program performance. However, IRS has no criteria or standards for determining where the pilot program performs adequately to be incorporated into normal IRS compliance processes. IRS officials told GAO that they plan to use research results to help formulate standards to determine pilot success but did not provide a target date when this will occur. Moreover, IRS has not completely articulated its methodology to evaluate the pilot program. IRS officials have action plans and testing sample plans for individual units. The action plans primarily contained project-management-type information such as actions or tasks to be taken by individuals, start date, completion date, and comments rather than a comprehensive description of the methodological approach for the overall pilot project. The testing sample plans were questionnaires soliciting information about compliance measures for the individual unit and the resources available for testing SRFMI data. None of the plans outline the methods, timing, or frequency of data collection. While IRS officials have begun formulating plans for developing a sampling approach and determining appropriate sample size, they encountered challenges such as delays in information-technology assistance and time limits for using taxpayer data that have impeded progress in moving forward on its evaluation methodology. The need to evaluate the program is underscored because obtaining and using SRFMI data imposes costs not only on IRS but also on the states. Without a sound assessment of pilot program results, IRS may make poor decisions about the program's future.
Recommendation for Executive Action
Status: Closed - Implemented
Comments: In January 2010, IRS developed a comprehensive evaluation plan for the State Reverse File Match Initiative (SRFMI) that addresses all components of the pilot program and includes key evaluation features. By following this documented plan, IRS will be able to determine the benefits of using state taxpayer data for compliance purposes and make better decisions about the program's future.
Recommendation: The IRS's Commissioner of Internal Revenue should ensure that IRS develops an evaluation plan to accurately and reliably assess the SRFMI pilot program's results. This plan should address all components of the pilot program and include key evaluation features of a sound plan: (1) well-defined, clear, and measurable objectives; (2) criteria or standards for determining pilot program performance; (3) clearly articulated methodology, including sound sampling methods, determination of appropriate sample size for the evaluation design, and a strategy for comparing the pilot results with other efforts; (4) a clear plan that details the type and source of data necessary to evaluate the pilot, methods for data collection, and the timing and frequency of data collection; and (5) a detailed data-analysis plan to track the program's performance and evaluate the final results of the project.
Agency Affected: Department of the Treasury: Internal Revenue Service