This is the accessible text file for GAO report number GAO-03-486 
entitled 'Tax Administration: IRS Should Reassess the Level of 
Resources for Testing Forms and Instructions' which was released on 
April 11, 2003.



This text file was formatted by the U.S. General Accounting Office 

(GAO) to be accessible to users with visual impairments, as part of a 

longer term project to improve GAO productsí accessibility. Every 

attempt has been made to maintain the structural and data integrity of 

the original printed product. Accessibility features, such as text 

descriptions of tables, consecutively numbered footnotes placed at the 

end of the file, and the text of agency comment letters, are provided 

but may not exactly duplicate the presentation or format of the printed 

version. The portable document format (PDF) file is an exact electronic 

replica of the printed version. We welcome your feedback. Please E-mail 

your comments regarding the contents or accessibility features of this 

document to Webmaster@gao.gov.



Report to the Chairman and Ranking Minority Member, Committee on 

Finance, U.S. Senate:



United States General Accounting Office:



GAO:



April 2003:



TAX ADMINISTRATION:



IRS Should Reassess the Level of Resources for Testing Forms and 

Instructions:



GAO-03-486:



GAO Highlights:



Highlights of GAO-03-486, a report to Chairman and Ranking Member of 

the Senate Committee on Finance 





Why GAO Did This Study:



Taxpayers rated the Internal Revenue Serviceís (IRS) ability to provide 

clear and easy-to-use forms and instructions among the lowest of 27 

indicators of service in 1993.  Due to continuing concerns about 

unclear forms and instructions, GAO was asked to determine



* whether and how often IRS tests the clarity of new and revised 

individual income tax forms and instructions;

* the benefits, if any, of testing forms and instructions for clarity 

prior to their use; and

whether any factors limit IRSís ability to do more tests and if so, how 

they can be addressed.



What GAO Found:



IRS used taxpayers and its employees to test revisions to five 

individual income tax forms and instructions from July 1997 through 

June 2002. According to IRS officials, they revised about 450 tax forms 

and instructions in 2001, many of which were for individual income tax 

returns.



Testing forms and instructions can help ensure their clarity and 

thereby benefit taxpayers and IRS by, for instance, reducing taxpayersí 

time to understand and complete tax forms, reducing calls to IRS for 

assistance, and reducing taxpayer errors. Due to similar benefits, 

federal agencies we contacted that routinely collect information from 

the public test their questionnaires.  Quantifying benefits due to 

testing is difficult, but IRSís experience in revising and testing 

Earned Income Credit and Child Tax Credit forms and instructions 

suggests that benefits of testing in some cases can considerably exceed 

the cost of testing. If taxpayers who did their own tax returns needed 

1 less minute to understand these two credits due to testing, their 

time saved, valued at the minimum wage, would be worth $1.2 million; 

IRSís contracting cost for the two tests was $56,000.



Although IRS officials recognized that testing could be beneficial, 

they cited tight time frames and constrained resources as limiting 

their ability to do more tests.  While IRS faces time constraints when 

making some changes to forms and instructions due to the passage of new 

laws, not all changes are time constrained. IRS does not have 

procedures specifying which versions of draft forms and instructions 

should be tested with taxpayers or when in its annual forms development 

process testing should occur. Resources currently available for testing 

are limited but the office responsible for testing has not developed 

data on missed testing opportunities and has limited data on the 

benefits that have been realized when testing occurred.  IRSís planning 

and budgeting process uses such data to support resource allocation 

decisions.



What GAO Recommends:



This report makes a series of recommendations that, if fully 

implemented, would improve IRSís annual process for creating and 

revising individual income tax forms and instructions by helping to 

ensure that its scarce testing resources are consistently applied to 

the highest priorities, impediments to testing are lessened, and 

appropriate information is developed so IRS management can better 

ensure that adequate resources are available to support testing in view 

of the potential benefits to taxpayers and IRS.



IRS agreed with our recommendations and plans to implement all but one 

of them in time for the 2004 forms development cycle.



www.gao.gov/cgi-bin/getrpt?GAO-03-486.



To view the full report, including the scope

and methodology, click on the link above.

For more information, contact Michael Brostek at (202) 512-9110 or 

brostekm@gao.gov.



Contents:



Letter:



Results in Brief:



Background:



IRS Tested Five Tax Forms and Instructions Over 5 Years:



Testing Written Documents Helps Ensure Clarity and Benefits Taxpayers 

and IRS:



IRS Has Not Addressed Constraints to Increased Testing:



Conclusions:



Recommendations for Executive Action:



Agency Comments and Our Evaluation:



Appendix I: Objectives, Scope, and Methodology:



Appendix II: Use of Testing in NCHS, Census, and BLS:



Appendix III: Comments from the Internal Revenue Service:



Tables:



Table 1: Forms and Instructions IRS Tested Using Individual Taxpayers 

Between July 1997 and June 2002:



Table 2: Percentage Decrease in the Number of EIC and Child Tax Credit 

Errors before and after Revising and Testing:



Table 3: Potential Benefits if Testing Helps Ensure the Clarity of Tax 

Forms and Instructions:



Table 4: Illustration of Potential Savings to IRS Alone in 2000 from 

Testing EIC and Child Tax Credit Forms and Instructions:



Table 5: Illustration of Potential Savings to Taxpayers and IRS in 2000 

from Error Reductions Alone from Testing EIC and Child Tax Credit Forms 

and Instructions:



Table 6: Illustration of Potential Benefit to Taxpayers in 2000 if 

Testing Reduced the Time Needed to Understand and Complete EIC and 

Child Tax Credit Forms:



Table 7: Illustration of Potential Cost Savings to IRS in 2002 if the 

Rate Reduction Credit Instructions Had Been Tested and Errors Were 

Reduced:



Table 8: Illustration of Potential Savings to Taxpayers and IRS in 2002 

from Potential Error Reductions if Testing the Rate Reduction Credit 

Had Reduced Taxpayer Errors:



Table 9: Illustration of Potential Benefit to Taxpayers in 2002 if 

Testing Reduced the Time Needed to Understand and Complete the Rate 

Reduction Credit:



Figures:



Figure 1: Simplified Overview of IRSís Annual Tax Forms Development 

Process:



Figure 2: Simplified Overview of Where Testing Can Occur in IRSís 

Annual Tax Forms Development Process:



This is a work of the U.S. Government and is not subject to copyright 

protection in the United States. It may be reproduced and distributed 

in its entirety without further permission from GAO. It may contain 

copyrighted graphics, images or other materials. Permission from the 

copyright holder may be necessary should you wish to reproduce 

copyrighted materials separately from GAOís product.



Abbreviations:



BLS: Bureau of Labor Statistics:



EIC: Earned Income Credit:



ERS: Error Resolution System:



FTE: full-time equivalent:



IRS: Internal Revenue Service:



NCHS: National Center for Health Statistics:



OMB: Office of Management and Budget:



TIGTA: Treasury Inspector General for Tax Administration:



W&I: Wage and Investment:



United States General Accounting Office:



Washington, DC 20548:



April 11, 2003:



The Honorable Charles E. Grassley

Chairman,

Committee on Finance

United States Senate:



The Honorable Max Baucus

Ranking Minority Member,

Committee on Finance

United States Senate:



Internal Revenue Service (IRS) tax forms and instructions that are not 

clear and understandable have been a long-standing frustration of 

taxpayers.[Footnote 1] For example, respondents to a 1993 customer 

satisfaction survey rated IRSís ability to provide clear and easy-to-

use tax forms and instructions and to minimize taxpayerís burden as the 

lowest of 27 indicators of service.[Footnote 2] More recently, we cited 

the instructions for the tax year 2001 rate reduction credit as 

confusing and one factor contributing to millions of taxpayer 

errors.[Footnote 3]



To be fair, IRS faces a challenge in clearly communicating with 

taxpayers due to the complexity of the Internal Revenue Code. However 

the codeís complexity also underscores the importance of tax forms and 

instructions that are as clear and understandable as possible.



One approach with recognized potential for aiding in the development of 

clearer forms and instructions is testing. By testing draft forms and 

instructions with taxpayers, IRS might be able to detect and revise 

language that is unnecessarily confusing. The Treasury Inspector 

General for Tax Administration (TIGTA) and we have both previously 

recommended that IRS do more to, for instance, identify what individual 

taxpayers find difficult to understand about tax forms or 

publications.[Footnote 4]



Because of your interest in ensuring that taxpayers have clearer, 

understandable tax forms and instructions and because of the potential 

for testing to contributing to that end, you asked us to determine:



* whether and how often IRS used taxpayers to test the clarity of new 

or revised individual income tax forms and instructions;



* the benefits, if any, of having taxpayers test forms and instructions 

for clarity prior to their use by the public; and



* whether any factors limit IRSís ability to test more forms and 

instructions for clarity and if so, how these factors can be addressed.



To address these objectives, we interviewed officials (1) in IRSís Wage 

and Investment (W&I) Divisionís Tax Forms and Publications Division, 

(2) from three federal agencies that use private citizens extensively 

in testing research and data collection activities, and (3) a private 

research firm that specializes in testing written documents, such as 

forms and surveys, for a variety of public and private sector clients. 

We also developed illustrations of the potential benefits to taxpayers 

and IRS of testing forms and instructions for clarity. Our scope and 

methodology are discussed in greater detail in appendix I to this 

report.



Results in Brief:



From July 1997 through June 2002, IRS used taxpayers or its employees, 

primarily in focus groups, to test 5 individual income tax forms or 

instructions. According to IRS officials, they revised about 450 forms 

and instructions in 2001, many of which were for individual income tax 

returns.



Testing written documents, such as forms and instructions, prior to 

their use helps ensure that they communicate clearly. Testing thereby 

helps (1) lessen the burden to respondents of understanding and 

completing forms and (2) reduce errors made by respondents. To the 

extent that taxpayers find IRSís tax forms and instructions that were 

tested easier to understand, IRS is less likely to be contacted by 

taxpayers for assistance in fulfilling their tax obligations and is 

likely to have fewer taxpayer errors to detect and correct. Benefits 

like these may be difficult to quantify, but form the basis for several 

agencies we contacted, as well as ourselves, to make testing a standard 

practice when developing or modifying documents used to collect 

information from the public. IRSís recent, limited experience with 

testing forms and instructions used by individuals suggests that 

testing is beneficial. At least in some cases the potential benefits of 

testing IRSís forms and instructions are likely to be substantially 

greater than the costs of testing to IRS.



Although IRS officials said that making greater use of testing to 

improve clarity of forms and instructions could be beneficial, they 

cited tight time frames for making changes and limited resources in the 

Tax Forms and Publications Division as the primary factors that limit 

their ability to conduct more tests. Although IRS does face time 

constraints when making some changes, especially when changes are due 

to new tax laws, time constraints do not apply to all changes IRS makes 

to forms and instructions for individual taxpayers. Further, IRS does 

not have clear procedures specifying which draft versions of forms and 

instructions should be tested with taxpayers or when testing should 

occur during the annual forms update process. Resources available 

within the division responsible for developing forms and instructions 

are limited. For example, one employee has responsibility for 

organizing testing efforts. However, the division has not developed 

information that IRS uses when making resource allocation decisions. 

The office has not developed data on the universe of changes for which 

testing is most likely to yield significant benefits to taxpayers and 

IRS and it has only limited data on the benefits that have been 

realized when testing has occurred.



This report makes a series of recommendations to improve IRSís annual 

process for creating and revising individual income tax forms and 

instructions through ensuring that (1) its scarce testing resources are 

consistently applied to the highest priorities, (2) impediments to 

testing are lessened, and (3) appropriate information is developed so 

IRS managers can ensure that adequate resources are available for 

testing in view of the potential benefits to taxpayers and IRS. IRS 

agreed with our recommendations and plans to implement all but one of 

them in time for the 2004 forms development cycle. The remaining 

recommendation--to ensure that an appropriate range of evaluations is 

conducted of tests--would take more time to put into practice.



Background:



Although most organizational components within IRS are involved, the 

Tax Forms and Publications Division within IRSís Media and Publications 

Division of the W&I Division is primarily responsible for creating and 

improving tax forms, instructions, and other documents. One goal of the 

Tax Forms and Publications Division is to make tax forms and 

instructions as clear and understandable as possible. It is divided 

into three branches--Individual Forms and Publications, Business Forms 

and Publications, and Tax Exempt/Government Entities and Specialty 

Forms and Publications. As of late January 2003, 103 persons were 

assigned to the Tax Forms and Publication Division, including about 15 

persons whose primary responsibility was creating, revising, and 

reviewing individual income tax forms and instructions.



Many tax forms and instructions are revised annually, often with short 

turnaround times and in response to tax law changes. IRS also 

periodically reviews tax forms and, when appropriate, schedules them 

for revision. According to IRSís estimate, it revised about 450 tax 

forms and instructions in 2001 that affected individual and business 

tax returns. In addition to tax law changes, revisions to tax forms and 

instructions generally reflected procedural changes, legal rulings, and 

feedback from internal and external stakeholders about the 

understandability of forms and instructions.



As illustrated in figure 1, the annual tax forms development process 

generally starts with a review of the current tax forms. IRSís tax law 

specialists review the existing forms and instructions to determine 

what changes, if any, may be needed to reflect tax law changes and 

other requirements. The tax law specialists consider comments from a 

variety of sources both within and outside IRS. For example, comments 

may be obtained from IRS customer service staff in toll-free call 

centers who answer calls from taxpayers and have firsthand knowledge of 

particular forms or instructions that were confusing to taxpayers. 

IRSís Taxpayer Advocate Service staff may also provide comments useful 

to the tax law specialists. The Tax Forms Coordinating Committee, 

comprised of representatives from all of IRSís key components, the 

Department of the Treasury, and IRSís Chief Counsel, reviews draft 

forms to help ensure that they are not overly burdensome and that they 

conform to legal and technical requirements. Draft forms are generally 

posted to IRSís external website so that external stakeholders and 

taxpayers may review and comment on them.



The Office of Management and Budget (OMB) is responsible for approving 

each form once every 3 years. The purpose of OMBís approval is to 

assess IRSís compliance with the Paperwork Reduction Act, which, among 

other things, requires agencies to assess the extent of burden the 

information they collect imposes on the public. Under the Paperwork 

Reduction Act, OMB must approve new forms and major revisions to 

existing ones. After OMBís approval, the forms and instructions are 

sent to IRSís vendors to be printed. Generally, IRS needs to have 

approved forms ready for printing by early October to ensure that they 

can be printed and distributed to the public when the tax filing season 

starts the following January.



Figure 1: Simplified Overview of IRSís Annual Tax Forms Development 

Process:



[See PDF for image]



Note: GAO analysis of IRS data.



[End of figure]



IRS Tested Five Tax Forms and Instructions Over 5 Years:



As shown in table 1, IRS used taxpayers or IRS employees to test the 

clarity of five individual income tax forms and instructions from July 

1997 through June 2002.[Footnote 5] IRS relied primarily on focus 

groups to do the testing. Contractors using private citizens did three 

of the tests (i.e., Earned Income instructions, Child Tax Credit 

instructions, and Schedule D) and IRS did the other two using IRS 

employees.



Table 1: Forms and Instructions IRS Tested Using Individual Taxpayers 

Between July 1997 and June 2002:



Forms and instructions: Earned Income Credit schedule; Year tested: 

1998; Number of participants: 50; Testing method: 5 focus groups.



Forms and instructions: Earned Income Credit instructions; Year tested: 

1999; Number of participants: 104; Testing method: 10 focus groups.



Forms and instructions: Child Tax Credit instructions; Year tested: 

1999; Number of participants: 104; Testing method: 10 focus groups.



Forms and instructions: Schedule D Form-Capital Gains and Losses; Year 

tested: 2001; Number of participants: 48; Testing method: 6 focus 

groups; 8 one-on-one interviews.



Forms and instructions: Innocent Spouse Application form; Year tested: 

2002; Number of participants: 52; Testing method: 6 focus groups.



Source: IRS.



Note: Forms and instructions may also refer to schedules and 

worksheets.



[End of table]:



The two testing methods that IRS used--focus groups and one-on-one 

interviews--are among the commonly used methods to obtain data from 

individuals on whether documents such as forms and instructions are 

clear and understandable. Focus groups generally consist of a small 

number of participants--about 8 to 12 persons---and are usually 

selected and organized around the focus group topic. Focus groups are a 

form of group interviewing that relies on interaction within the group 

to obtain the impressions of a group of people but not necessarily the 

impressions of each participant. One-on-one interviews, which aim to 

obtain individual attitudes, beliefs, and feelings, are used to probe 

individuals about specific difficulties they may have with completing a 

form or reading instructions. In some instances, these methods may be 

used in combination depending upon the particular circumstances of the 

test. As noted in table 1, IRS used both focus groups and one-on-one 

interviews in testing the Schedule D form.



Testing Written Documents Helps Ensure Clarity and Benefits Taxpayers 

and IRS:



Testing written documents such as tax forms and instructions helps 

ensure they are clear, thereby benefiting taxpayers and IRS. 

Researchers from three federal agencies and a private research firm 

said testing leads to clearer documents as well as more accurate 

responses. Our guidance on developing and using questionnaires also 

recommends testing prior to distribution.[Footnote 6] IRSís experience 

indicates that testing likely improves the clarity of tax forms and 

instructions and may therefore help reduce the number of errors 

taxpayers make. Although limited data were available on the costs and 

benefits of testing IRS forms and instructions, recent changes to 

Earned Income Credit (EIC) and Child Tax Credit forms illustrate the 

potential for benefits to significantly exceed the costs of testing.



Experiences of Three Federal Agencies, a Private Research Firm, and IRS 

Support Use of Testing:



Researchers from the National Center for Health Statistics (NCHS), the 

Census Bureau, and the Bureau of Labor Statistics (BLS) said that 

testing helps ensure that their documents are clear, and therefore 

users are more likely to understand them and complete them correctly. 

Consequently, researchers from the agencies said they routinely test 

written documents, such as forms or surveys, prior to public 

distribution. A representative of a private research firm also said 

that testing ensures that documents communicate clearly and that his 

firm and many others perform such testing for a wide variety of private 

and public clients.



The researchers from the federal agencies told us that the benefits of 

testing are difficult to quantify and, in some instances, may not be 

quantifiable. NCHS officials stated that the experience they have 

gained over time from testing and revising documents has helped them 

develop clearer forms from the outset, a benefit that may not be 

quantifiable. A Census researcher also agreed that testing is 

beneficial but difficult to quantify. Nevertheless, the Census 

researcher said that testing documents prior to public use helps ensure 

that they are clear and understandable, which gives the agency a 

greater chance of receiving accurate responses and which lessens the 

need for follow-up interviews. Similarly, a BLS researcher noted that, 

though difficult to quantify, testing benefits the agency by reducing 

errors made by respondents and those reductions can result in savings 

of time, money, and effort for BLS. See appendix II for additional 

information on the use of testing in these three agencies.



The representative of the private research firm also provided some 

perspective of its experiences in conducting tests for public and 

private sector customers. This firm, like many similar companies, 

provides a variety of testing and data collection services to public 

and private sector customers. The firm arranges and conducts focus 

groups, one-on-one interviews, and other tests in order to ensure the 

clarity of forms, instruction manuals, surveys, and Web sites among 

other things. Focus groups generally involve 12 participants and cost 

around $6,700 to $7,600, excluding costs to develop the item to be 

tested but including incentive pay for participants that could range 

from $25 to $50 per participant. These costs may be higher, the 

representative said, if the participants come from special groups. For 

example, if the participants are medical doctors, incentive pay could 

be as high as $250 per participant.



The researcher also said that private firms vary in how much they spend 

to test their documents. He estimated that firms generally spend 

between $300,000 and $500,000 to ensure that a form or document is 

clear and will meet the firmís needs. In some cases, firms spend more 

than $500,000 to do a series of tests, making revisions between each 

test, before they arrive at a final version of a form or document. He 

also added that most firms do not spend at levels that would allow them 

to test all their forms and documents.



While maintaining that testing is beneficial, researchers also stated 

that testing is not fail-safe. It can help identify particular parts of 

a form that are not clear, researchers said, but it cannot ensure that 

subsequent changes to the form will entirely resolve the clarity issue. 

In addition, testing may identify problems participants have in 

completing written documents, but the participantsí problems may be 

related to other issues, such as poor math skills, rather than 

confusing or unclear documents.



Testing questionnaires before distribution is also recommended as a 

quality assurance measure in our guidance on developing and using 

questionnaires. According to our guidance, testing questionnaires 

before they are used is one of the best ways to ensure that the 

document actually communicates what it was intended to communicate and 

that users will uniformly interpret it. Testing increases the 

likelihood that respondents will provide the information needed and 

helps to alleviate inaccurate responses. Our guidance is also 

consistent with professional literature on survey design. According to 

professional literature, ďreducing measurement error through better 

question design is one of the least costly ways to improve survey 

estimates. For any survey, it is important to attend to careful 

question design and pretesting.Ē[Footnote 7]



IRSís recent limited experience with testing indicates that testing may 

help ensure the clarity of tax forms and instructions. In 1999, IRS 

revised the forms and instructions related to the EIC and the Child Tax 

Credit, tested the revised forms and instructions, and revised them 

again based on the test results. The following year, when the revised 

and tested forms and instructions were used by taxpayers, the error 

rates for the EIC and the Child Tax Credit decreased by 28 and 35 

percent, respectively, as shown in table 2.[Footnote 8]



Table 2: Percentage Decrease in the Number of EIC and Child Tax Credit 

Errors before and after Revising and Testing:



Forms/Instructions: Earned Income Credit; Number of errors on tax 

returns filed in 1999 (before testing): 1,797,162; Number of errors on 

tax returns filed in 2000 (after testing): 1,296,095; Percentage 

decrease in errors from

1999 to 2000: 28.



Forms/Instructions: Child Tax Credit; Number of errors on tax returns 

filed in 1999 (before testing): 1,430,394; Number of errors on tax 

returns filed in 2000 (after testing): 934,289; Percentage decrease in 

errors from

1999 to 2000: 35.



Source: IRS data.



[End of table]:



IRS officials told us they attribute part of the decrease in EIC errors 

to a new approach officials developed for structuring EIC forms and 

instructions and part to the improvements in the draft documents that 

resulted from testing the revised forms and instructions. Before the 

EIC forms and instructions were revised, the instructions included a 

definition and example describing a qualifying child that taxpayers had 

to interpret.[Footnote 9] Incorrectly claimed qualifying children have 

been a major source of EIC errors. IRS revised the instructions so that 

taxpayers would answer a series of ďyes/noĒ questions to determine if 

they have a qualifying child instead of relying on their 

interpretations of the definition or example of a qualifying child. 

They then tested the old format and the new format. The number of 

errors decreased substantially when taxpayers used the new format. IRS 

then made some final changes to clarify the instructions based on test 

results. In IRS officialsí opinions, the ďyes/noĒ format made it 

clearer for taxpayers to determine if they had a qualifying child.



Benefits of Testing Forms and Instructions Can Far Exceed IRSís Costs:



The benefits of testing some changes to IRSís forms and instructions 

can considerably exceed IRSís costs to do tests, especially because so 

many taxpayers can be affected by improvements in clarity that may 

result from testing. IRSís contract costs, including travel, for 

testing changes to the EIC and Child Tax Credit forms and instructions 

were about $56,000 and these costs may have been offset within IRS 

alone in the year that the change was implemented. More significantly, 

if testing changes to forms and instructions for these credits led to a 

1-minute reduction, on average, in the time taxpayers needed to 

understand and complete the forms during the 2000 tax filing season, 

affected taxpayers would have saved 240,000 hours worth $1.2 million 

valued at minimum wage.



Taxpayers and IRS Can Realize a Wide Range of Benefits and Costs From 

Testing:



Testing has the potential to yield a wide range of benefits to 

taxpayers and IRS. Table 3 summarizes some of the potential benefits 

that could result if testing helps clarify tax forms and instructions. 

If the form or instruction that has been revised and tested remains 

unchanged, some potential benefits could recur annually for the life of 

the form or instruction.



Table 3: Potential Benefits if Testing Helps Ensure the Clarity of Tax 

Forms and Instructions:



IRSís benefits: 



* Fewer errors that need to be corrected before 

processing returns; 

* Reduction in audits due to clarity-related taxpayer errors and 

redirection of audit resources to other noncompliant taxpayers;

 * Reduction in demand for taxpayer assistance at local IRS offices 

may allow IRS to provide better service to other taxpayers;

 * Reduction in demand for assistance via IRSís toll-free telephone 

service related to clarity issues may enable IRS to answer some calls 

that would otherwise have not been answered;

 * Experience and knowledge gained through testing may lead to 

creation of clearer forms and instructions in the future; Taxpayersí 

benefits: 

* Increased clarity of IRSís forms and instructions reduces; 

-time to complete tax forms; 

-expenses associated with obtaining assistance in completing tax forms;

 * Reduced burden from not having to deal with IRS notices, such as 

reduced time to; 

-open and read IRSís notices; 

-decide what to do; 

-research tax records; 

-prepare response to IRS; 

-copy and mail response, if necessary, to IRS; 

-call IRS for assistance.



Source: GAO.

:

[End of table]



From the taxpayerís perspective, benefits from testing could include 

avoiding the burdens associated with (1) interacting with IRS if they 

make mistakes due to unclear forms and instructions and (2) 

understanding and complying with unclear forms and instructions. From 

IRSís perspective, benefits are generally in the form of opportunities 

to use its resources better serving other taxpayers and enforcing the 

tax laws.[Footnote 10]



Because IRS makes many changes to forms and instructions affecting 

individual taxpayers every year, ranging from very simple to more 

complex changes, the benefits can vary according to the type of changes 

made. Some forms or instructions may change simply to update certain 

dollar thresholds based on inflation and these changes may be unlikely 

to be confusing or unclear to taxpayers. However, in other cases 

changes may introduce new requirements or concepts to taxpayers, such 

as when new rules are established through legislation or regulation. 

Changes intended to address situations like these may be more likely to 

be confusing or unclear to taxpayers, which could result in a burden on 

taxpayers to understand their obligations in preparing their tax forms 

and, possibly, to errors that lead to subsequent interactions with IRS 

to correct their returns.



To the extent that a form or instruction is unclear and the lack of 

clarity leads to taxpayer errors, the method IRS uses to detect the 

errors can affect the costs IRS incurs as a consequence. If a 

taxpayerís error can be detected by IRS and corrected under its ďmath 

errorĒ procedures, which rely extensively on automated processes, the 

cost to IRS to correct the error is likely to be small.[Footnote 11] On 

the other hand, if unclear forms or instructions lead to compliance 

errors that are detected and addressed through audits conducted through 

the mail, in IRS offices, or in the taxpayerís location, the costs to 

IRS are likely to be higher in part because these processes are more 

labor intensive.



The burden and costs taxpayers might avoid if testing helps clarify 

forms and instructions, and thereby helps taxpayers avoid errors, can 

vary substantially just as IRSís costs can vary. In general, because 

taxpayers need only respond if they disagree with an IRS notice stating 

that it has corrected an error under its math error procedures, the 

taxpayerís burden and cost are likely to be lower than if IRS contacts 

the taxpayer as part of an audit since audits require taxpayer 

responses and reviews of taxpayersí books and records.[Footnote 12]



Testing Benefits Can Far Exceed IRSís Costs:



Illustrations we developed of the potential benefits and costs of 

testing forms and instructions show that at least in some cases 

benefits can be substantially greater than the costs to IRS to do 

tests. The benefits of testing to IRS alone can potentially exceed its 

testing costs in the first year a change is implemented. But, primarily 

because a small change in the time required of taxpayers to understand 

their tax obligations can total to a large aggregate benefit, taking 

taxpayersí benefits into account can yield total benefits substantially 

above IRSís costs.



IRS officials have not attempted to develop quantitative estimates of 

the benefits to taxpayers and IRS that may result from testing forms 

and instructions and the costs IRS incurs to achieve those benefits. 

IRS officials did believe that because taxpayers made fewer errors when 

using the revised EIC and the Child Tax Credit forms and instructions 

as shown in table 2, IRS spent less time and money correcting errors 

related to them.[Footnote 13] The officials said they could not 

quantify the cost savings because IRS does not track error correction 

costs by type of error.



To provide some perspective on the potential magnitude of benefits and 

costs that may be realized due to testing changes to forms and 

instructions, we analyzed the changes IRS made to EIC and Child Tax 

Credit forms and instructions. Our analyses are illustrations and not 

actual assessments of benefits and costs that were associated with 

testing these forms and instructions because complete data were not 

available on the potential benefits and costs. Further, in constructing 

our illustrations we sought to be conservative in estimating benefits, 

in part because we did not have information on the full range of costs 

IRS incurred to test forms and instructions. Our illustrations focus on 

(1) a narrow set of benefits to IRS alone due to potential reductions 

in taxpayer errors, (2) those benefits plus certain benefits to the 

taxpayers from reduced errors, and (3) potential benefits to taxpayers 

in reduced time to do their taxes. See appendix I for details on the 

methodology we used in developing our illustrations.



Our first illustration quantifies a narrow set of benefits to IRS alone 

from testing EIC and Child Tax Credit forms and instructions--that is, 

the benefits IRS may have realized due to reduced numbers of errors 

that are handled under its math error procedures. It is likely that to 

the extent testing contributed to better taxpayer understanding of 

these two credits, IRS would have obtained other benefits. For 

instance, because improperly claimed qualifying children is one of the 

leading causes of the EICís high noncompliance rate,[Footnote 14] if 

clarified EIC forms and instructions lead fewer taxpayers to improperly 

claim the EIC, IRS would likely be able to free some of its EIC-related 

audit resources for other audits or to audit EIC returns that it might 

otherwise have had insufficient resources to cover. In fiscal year 

2002, IRS used about 1,400 full-time equivalent (FTE) staff 

years[Footnote 15] for correspondence audits of EIC issues.



Our analysis in table 4 shows the amount by which IRSís potential cost 

savings from not having to correct EIC and Child Tax Credit errors may 

have exceeded its testing costs given differing assumptions about how 

much testing alone may have contributed to reduced taxpayer errors. As 

illustrated, IRS would have saved more in cost avoidance (thereby 

freeing resources to work elsewhere) in the first year of the change 

alone than it spent on the contracted testing of the forms and 

instructions if half of the reduction in errors was due to 

testing.[Footnote 16] If only 10 or 25 percent of the reduction was due 

to testing, then IRS would not have saved more than it spent on the 

testing contract in the first year. However, some of the benefits of a 

change in forms or instructions continue to be realized in future 

years. Again, the illustration does not consider other benefits IRS may 

have realized.



Table 4: Illustration of Potential Savings to IRS Alone in 2000 from 

Testing EIC and Child Tax Credit Forms and Instructions:



Form and instruction: EIC; Differing assumed levels of errors avoided: 

50%; Assumed number of errors eliminated due to testing: 250,534; 

Estimated direct labor cost to correct an error[A]: $0.36; Estimated 

cost to correct eliminated errors: $90,000; Potential IRS cost savings 

assuming $56,000 cost to perform test[B]: $34,000.



Differing assumed levels of errors avoided: 25%; Assumed number of 

errors eliminated due to testing: 125,267; Estimated direct labor cost 

to correct an error[A]: 0.36; Estimated cost to correct eliminated 

errors: 45,000; Potential IRS cost savings assuming $56,000 cost to 

perform test[B]: (11,000).



Differing assumed levels of errors avoided: 10%; Assumed number of 

errors eliminated due to testing: 50,107; Estimated direct labor cost 

to correct an error[A]: 0.36; Estimated cost to correct eliminated 

errors: 18,000; Potential IRS cost savings assuming $56,000 cost to 

perform test[B]: (38,000).



Form and instruction: Child tax credit; Differing assumed levels of 

errors avoided: 50%; Assumed number of errors eliminated due to 

testing: 248,053; Estimated direct labor cost to correct an error[A]: 

0.36; Estimated cost to correct eliminated errors: 89,000; Potential 

IRS cost savings assuming $56,000 cost to perform test[B]: 33,000.



Differing assumed levels of errors avoided: 25%; Assumed number of 

errors eliminated due to testing: 124,026; Estimated direct labor cost 

to correct an error[A]: 0.36; Estimated cost to correct eliminated 

errors: 45,000; Potential IRS cost savings assuming $56,000 cost to 

perform test[B]: (11,000).



Differing assumed levels of errors avoided: 10%; Assumed number of 

errors eliminated due to testing: 49,611; Estimated direct labor cost 

to correct an error[A]: 0.36; Estimated cost to correct eliminated 

errors: 18,000; Potential IRS cost savings assuming $56,000 cost to 

perform test[B]: (38,000).



Source: IRS and GAO.



Note: GAO analysis of IRS data.



[A] IRS officials provided (1) the number of FTEs, including labor-

related overhead costs such as training and leave, for operating the 

Error Resolution System (ERS) and (2) the number of errors corrected by 

ERS during fiscal year 2002. According to IRS officials, ERS personnel 

are hired at GS-4 and the journeyman level is GS-6. Using the number of 

ERS FTEs, the number of errors corrected by ERS staff, and the GS-5, 

step 1 salary effective January 2002, as the salary for ERS staff 

members, we estimated that IRS incurred $0.36 in costs per error. (IRS 

officials were unable to provide the actual costs for ERS staff. We 

chose to use GS-5, step 1, because beginning ERS staff are GS-4s and 

more experienced ERS staff are GS-6s.) Total error correction costs may 

be higher because the $0.36 estimate does not include various other 

costs such as nonlabor-related overhead for equipment and supplies.



[B] IRS contracted with a firm to conduct focus groups to test EIC and 

Child Tax Credit forms and instructions. The contract was for $54,000 

and IRS spent an additional $2,000 on related travel costs. Because one 

contractor conducted the tests for both credits, IRS officials said it 

would be difficult to allocate the $56,000 between the tests for the 

two credits.



[End of table]:



To provide some perspective on how the potential benefits to taxpayers 

from testing EIC and Child Tax Credit forms could affect the overall 

benefits and costs of testing, we next looked at potential reduced 

burden from credit claimants receiving fewer notices due to reduced 

errors. First, we assumed that on average all taxpayers receiving an 

error notice from IRS take 2 or 5 minutes to deal with the notice. 

Based on those assumptions, we calculated the value to taxpayers of the 

time saved (using minimum wage levels) from not having to deal with IRS 

error notices.[Footnote 17] We used the same assumed reductions in 

errors due to testing that we made for table 4 and we netted taxpayersí 

savings with the savings shown in table 4 for IRS alone.



As table 5 shows, including testing-related benefits to taxpayers from 

decreased errors suggests that in the first year following testing of 

EIC and Child Tax Credit forms and instructions, the net benefit to 

taxpayers and IRS combined could have been positive except for our 

lowest assumption about the degree to which testing may have reduced 

taxpayer errors--our 10 percent assumption.



Table 5: Illustration of Potential Savings to Taxpayers and IRS in 2000 

from Error Reductions Alone from Testing EIC and Child Tax Credit Forms 

and Instructions:



Form and instruction: EIC; Assumed number of errors eliminated due to 

testing: 250,534; Value to taxpayers in time saved (2 minutes per 

notice): $43,000; Value to taxpayers in time saved (5 minutes per 

notice): $108,000; Potential IRS cost savings assuming $56,000 cost to 

perform test: $34,000; Potential net savings to taxpayers and IRS from 

error reductions: 2 minutes: $77,000; Potential net savings to 

taxpayers and IRS from error reductions: 5 minutes: $142,000.



Assumed number of errors eliminated due to testing: 125,267; Value to 

taxpayers in time saved (2 minutes per notice): 22,000; Value to 

taxpayers in time saved (5 minutes per notice): 54,000; Potential IRS 

cost savings assuming $56,000 cost to perform test: (11,000); Potential 

net savings to taxpayers and IRS from error reductions: 2 minutes: 

11,000; Potential net savings to taxpayers and IRS from error 

reductions: 5 minutes: 43,000.



Assumed number of errors eliminated due to testing: 50,107; Value to 

taxpayers in time saved (2 minutes per notice): 9,000; Value to 

taxpayers in time saved (5 minutes per notice): 22,000; Potential IRS 

cost savings assuming $56,000 cost to perform test: (38,000; Potential 

net savings to taxpayers and IRS from error reductions: 2 minutes: 

(29,000); Potential net savings to taxpayers and IRS from error 

reductions: 5 minutes: (16,000).



Form and instruction: Child tax credit; Assumed number of errors 

eliminated due to testing: 248,053; Value to taxpayers in time saved (2 

minutes per notice): 43,000; Value to taxpayers in time saved (5 

minutes per notice): 106,000; Potential IRS cost savings assuming 

$56,000 cost to perform test: 33,000; Potential net savings to 

taxpayers and IRS from error reductions: 2 minutes: 76,000; Potential 

net savings to taxpayers and IRS from error reductions: 5 minutes: 

139,000.



Assumed number of errors eliminated due to testing: 124,026; Value to 

taxpayers in time saved (2 minutes per notice): 21,000; Value to 

taxpayers in time saved (5 minutes per notice): 53,000; Potential IRS 

cost savings assuming $56,000 cost to perform test: (11,000; Potential 

net savings to taxpayers and IRS from error reductions: 2 minutes: 

10,000; Potential net savings to taxpayers and IRS from error 

reductions: 5 minutes: 42,000.



Assumed number of errors eliminated due to testing: 49,611; Value to 

taxpayers in time saved (2 minutes per notice): 9,000; Value to 

taxpayers in time saved (5 minutes per notice): 21,000; Potential IRS 

cost savings assuming $56,000 cost to perform test: (38,000); Potential 

net savings to taxpayers and IRS from error reductions: 2 minutes: 

(29,000); Potential net savings to taxpayers and IRS from error 

reductions: 5 minutes: (17,000).



Source: GAO.



Note: GAO analysis.



[End of table]:



Finally, to illustrate the potential benefits if testing EIC and Child 

Tax Credit forms and instructions made them clearer and thereby reduced 

taxpayersí time needed to understand and complete the credit forms, we 

calculated the value of time saved by taxpayers (using minimum wage 

levels) in understanding and completing EIC and Child Tax Credit forms 

assuming the time saved was 1 minute. Unlike for tables 5 and 6, all 

taxpayers who used the form or instructions to determine whether they 

qualified for either credit may have saved time if testing contributed 

to clearer EIC and Child Tax Credit forms and instructions. However, 

because we did not know how many taxpayers might have used the forms 

and instructions for this purpose, in calculating the value of time 

taxpayers may have saved we used only the number of taxpayers who 

claimed these credits and did not use paid preparers to prepare their 

tax returns.



Table 6 shows that if testing the creditsí forms and instructions 

helped clarify them and that led taxpayers to take 1 minute less to 

understand and complete the forms, credit claimants would have saved a 

total of about 240,000 hours worth $1.2 million at minimum wage levels.



Table 6: Illustration of Potential Benefit to Taxpayers in 2000 if 

Testing Reduced the Time Needed to Understand and Complete EIC and 

Child Tax Credit Forms:



Form and Instruction: EIC; Number of returns claiming the credit 

(millions)[A]: 6.2; Total hours saved assuming 1 minute less per 

taxpayer: 100,000; Potential value to taxpayers of time saved 

(millions): $0.5.



Form and Instruction: Child Tax Credit; Number of returns claiming the 

credit (millions)[A]: 8.4; Total hours saved assuming 1 minute less per 

taxpayer: 140,000; Potential value to taxpayers of time saved 

(millions): 0.7.



Form and Instruction: Totals; Number of returns claiming the credit 

(millions)[A]: 14.6; Total hours saved assuming 1 minute less per 

taxpayer: 240,000; Potential value to taxpayers of time saved 

(millions): 1.2.



Source: GAO.



Note: GAO analysis.



[A] This is the number of tax returns prepared by taxpayers claiming 

the credits. Since taxpayers who used a preparer might not need to read 

and understand the forms and instructions, we did not include them in 

this table. Both the number of EIC and Child Tax Credit returns are 

based on the percentage of returns prepared by taxpayers who claimed 

EIC since data on the percentage of returns prepared by taxpayers who 

claimed the Child Tax Credit were not available.



[End of table]:



To provide another perspective on the potential magnitude of benefits 

and costs associated with testing changes to forms and instructions, we 

also looked at IRSís experience with the rate reduction credit. This 

one-time credit was enacted in June 2001.[Footnote 18] When 2001 tax 

returns were processed during 2002, over 8 million returns had errors 

related to the credit.[Footnote 19] IRS did not test the instructions 

for computing the rate reduction credit that was included on the Form 

1040 for tax year 2001. According to IRS officials, they did not test 

the instructions because the credit was a one-time event, and in their 

judgment, they had insufficient time to test it. We reported that some 

of the taxpayersí errors were probably due to taxpayers not 

understanding IRSís instructions on how to compute the credit. We also 

reported that the demand for telephone assistance related to the credit 

was significant during the 2002 filing season, and that some of these 

calls, based on Taxpayer Advocate Service information, were made 

because taxpayers did not understand how to compute the 

credit.[Footnote 20]



Using the same approach to illustrate whether IRS alone may have 

realized benefits in excess of its testing costs as we did for EIC and 

Child Tax Credit changes, we developed the illustration shown in table 

7.[Footnote 21] As shown, considering only IRSís cost and assuming that 

all errors were corrected by IRS using its math error procedures and 

assuming IRS would have spent the same amount to test the rate 

reduction credit instructions as it did for EIC and Child Tax Credit 

tests, IRS may have been able to save between $233,000 and $666,000. 

Although this case is somewhat atypical since the rate reduction credit 

affected essentially all individual taxpayers and the number of errors 

related to the credit was unusually high, these figures illustrate that 

the potential for savings to IRS alone from testing instructions at 

times can substantially exceed its testing costs.



Table 7: Illustration of Potential Cost Savings to IRS in 2002 if the 

Rate Reduction Credit Instructions Had Been Tested and Errors Were 

Reduced:



Number of returns with rate reduction credit errors: 8,025,851; 

Differing assumed levels of errors avoided: 25%; Assumed number of 

errors eliminated by testing: 2,006,646; Estimated direct labor cost to 

correct an error: $0.36; Estimated cost to correct eliminated errors: 

$722,000; Potential IRS cost savings assuming $56,000 cost to perform 

test: $666,000.



Differing assumed levels of errors avoided: 15%; Assumed number of 

errors eliminated by testing: 1,203,878; Estimated direct labor cost to 

correct an error: 0.36; Estimated cost to correct eliminated errors: 

433,000; Potential IRS cost savings assuming $56,000 cost to perform 

test: 377,000.



Differing assumed levels of errors avoided: 10%; Assumed number of 

errors eliminated by testing: 802,585; Estimated direct labor cost to 

correct an error: 0.36; Estimated cost to correct eliminated errors: 

289,000; Potential IRS cost savings assuming $56,000 cost to perform 

test: 233,000.



Source: IRS and GAO.



Note: GAO analysis of IRS data.



[End of table]:



However, just as with EIC and Child Tax Credits, taxpayers would have 

benefited if testing had been done, led to clearer instructions and 

consequently led to fewer taxpayer errors for the rate reduction 

credit. Using the same approach we used for EIC and Child Tax Credits, 

table 8 shows the potential taxpayersí savings from dealing with fewer 

rate reduction credit error notices and the net savings to taxpayers 

and IRS.



Table 8: Illustration of Potential Savings to Taxpayers and IRS in 2002 

from Potential Error Reductions if Testing the Rate Reduction Credit 

Had Reduced Taxpayer Errors:



Number of returns with rate reduction credit errors: 8,025,851; Assumed 

number of errors eliminated by testing: 2,006,646; Value to taxpayers 

of time saved (2 minutes per notice): $344, 000; Value to taxpayers of 

time saved (5 minutes per notice): $861,000; Potential IRS cost savings 

assuming $56,000 cost to perform test: $666,000; Potential net savings 

to taxpayers and IRS from error reductions: 2 minutes: $1,010,000; 

Potential net savings to taxpayers and IRS from error reductions: 5 

minutes: $1,527,000.



Assumed number of errors eliminated by testing: 1,203,878; Value to 

taxpayers of time saved (2 minutes per notice): 207,000; Value to 

taxpayers of time saved (5 minutes per notice): 517,000; Potential 

IRS cost savings assuming $56,000 cost to perform test: 377,000; 

Potential net savings to taxpayers and IRS from error reductions: 2 

minutes: 584,000; Potential net savings to taxpayers and IRS from 

error reductions: 5 minutes: 894,000.



Assumed number of errors eliminated by testing: Assumed number of 

errors eliminated by testing: 802,585; Value to taxpayers of time saved 

(2 minutes per notice): Value to taxpayers of time saved (2 minutes per 

notice): 138,000; Value to taxpayers of time saved (5 minutes per 

notice): Value to taxpayers of time saved (5 minutes per notice): 

344,000; Potential IRS cost savings assuming $56,000 cost to perform 

test: Potential IRS cost savings assuming $56,000 cost to perform test: 

233,000; Potential net savings to taxpayers and IRS from error 

reductions: 2 minutes: 2 minutes: 371,000; Potential net savings to 

taxpayers and IRS from error reductions: 5 minutes: 5 minutes: 577,000.



Source: GAO.



Note: GAO analysis.



[End of table]:



Table 9 shows that if testing had been done and it improved the clarity 

of the instructions enough to save taxpayers, on average, 30 seconds in 

understanding whether and how they needed to complete the credit line 

on their tax returns, the savings would have been larger than savings 

to IRS and taxpayers from avoided errors alone.[Footnote 22]



Table 9: Illustration of Potential Benefit to Taxpayers in 2002 if 

Testing Reduced the Time Needed to Understand and Complete the Rate 

Reduction Credit:



Number of returns affected by the credit[A]: 50,000,000; Total hours 

saved assuming 30 seconds less per taxpayer: 415,000; Potential value 

to taxpayers of time saved (millions): $2.1.



Source: GAO.



Note: GAO analysis.



[A] This is the number of paper returns prepared by taxpayers. Since 

taxpayers who used a preparer might not need to read and understand the 

instructions, we did include them in this table. We only included the 

number of paper returns because information was not readily available 

on the number of returns filed electronically that were completed by 

taxpayers rather than a preparer.



[End of table]



IRS Has Not Addressed Constraints to Increased Testing:



Although IRS officials said that making greater use of testing to 

improve clarity of forms and instructions could be beneficial, 

officials have not addressed the two constraints--time and resources--

that they state limit their ability to do more testing of changes to 

forms and instructions. Time constraints are not binding for some 

changes IRS considers to forms and instructions, although IRS cannot 

realistically test the unknown portion of the changes that are due to 

laws passed shortly before, or even after, the effective dates for the 

forms. Also, IRSís procedures for developing and revising forms (1) do 

not clearly specify which draft version of forms and instructions 

should be tested with taxpayers or (2) when in the annual forms 

development cycle testing should occur. In addition to tight time 

frames, officials also say that limited resources, such as only one 

person responsible for coordinating all testing efforts in the Forms 

and Publications Division, preclude them from increasing tests of forms 

and instructions. However, IRS has not documented which changes to 

forms and instructions likely would benefit from testing or 

demonstrated the benefits that are gained when testing is done. IRSís 

planning and budgeting process uses such information in determining the 

level of resources to be allocated to various units.



Tight Time Frames Do Not Always Exist and IRSís Procedures Do Not 

Always Facilitate Testing:



IRS officials told us that when new tax laws are enacted during the 

year that require IRS to create or revise tax forms and instructions in 

time to distribute them to taxpayers by January 1, the start of the 

tax-filing season, they lack time to test the forms and instructions 

before distributing them to taxpayers. However, not all changes to 

forms and instructions are time constrained and IRSís procedures lack a 

clear target for which version of forms and instructions should be 

tested with taxpayers.



While sufficient data were not available to determine the portion of 

changes IRS makes to forms and instructions that cannot be tested due 

to time constraints, not all changes are time constrained. Due to the 

variability in the time that may be required to test a form or 

instruction and in the amount of time IRS needs to develop the initial 

form or instruction to be tested, we cannot say definitively when IRS 

may or may not have sufficient time to conduct tests. In some cases, 

IRS likely could have sufficient time to do testing when it identifies 

a needed change to forms or instructions itself since it largely 

controls the scheduling of this work. Similarly, when the Congress 

passes a law that is not effective until a future tax year, or that 

contains provisions that are not effective until a future tax year, IRS 

may have sufficient time to conduct tests. For example, the Economic 

Growth and Tax Reconciliation Act of 2001 was passed on June 7, 2001, 

with some provisions effective for tax year 2001, but others with later 

effective dates. The provisions modifying education Individual 

Retirement Accounts were effective for taxable years beginning after 

December 31, 2001. This gave IRS approximately 16 months to develop and 

test any modifications to tax forms and instructions and make final 

revisions before those forms and instructions needed to go to printing 

for distribution by January 2003. When a law affects the current tax 

year, i.e., changes how taxpayers will need to calculate their taxes in 

the next tax-filing season, IRS is less likely to have sufficient time 

to test. Even in such a case, however, the new law may be passed early 

enough to allow testing.



IRSís current procedures for developing and revising forms and 

instructions do not clearly specify which draft version of forms and 

instructions should be tested with taxpayers or when in the annual 

forms development cycle testing should occur. Officials said that draft 

forms may be tested with taxpayers either before or after they are 

posted to IRSís website for external comments by the public, tax 

practitioners, software developers, and others. Tax Forms and 

Publications Division officials said that they consider the particular 

circumstances surrounding the development of each form and instruction 

when deciding which version of a draft form or instruction they should 

test. However, because IRS does not have a clear targeted time for 

testing, IRSís ability to plan and conduct tests maybe constrained. If 

IRSís procedures defined a point in the annual forms development cycle 

where a version of a draft form or instruction would be available for 

testing, IRS would be able to establish processes and deadlines 

designed to ensure that the opportunity for testing is realized.



To the extent that a draft version of a form or instruction is 

available for testing early in the process, it would give IRS a fuller 

range of options for testing. For example, if IRS tested draft versions 

of forms and instructions before or during the approximately 3-week 

period that the form is available on its Web site, this would minimize 

any additional calendar time that testing might otherwise add to IRSís 

forms development process.



Figure 2 shows the points in IRSís annual forms development process 

where testing can occur. As illustrated, testing may be conducted early 

in the process and late in the process.



Figure 2: Simplified Overview of Where Testing Can Occur in IRSís 

Annual Tax Forms Development Process:



[See PDF for image]



Note: GAO analysis of IRS data.



[End of figure]:



Testing earlier drafts of forms and instructions would also enable 

officials to select from various testing alternatives depending on how 

early a draft is available for testing. We did not find a uniform 

amount of time needed to test a change to a form or instruction. At the 

low end of the spectrum, an official from NCHS said that it takes about 

7 weeks to test that agencyís questionnaires using one-on-one 

interviews. IRS officials estimated that when IRS employees are used as 

focus group participants it requires about 8 to 12 weeks to schedule 

and conduct the tests, analyze the data, and prepare a report 

summarizing the results. IRS officials estimated that when they 

contract with a private firm to conduct focus groups using private 

citizens, 24 to 32 weeks are required to obtain a contract, recruit 

participants, conduct the tests, analyze the results, and prepare a 

report. This time frame is based on using regular contracting processes 

involving developing a statement of work, soliciting bids, and 

selecting a contractor. Contract options exist that enable agencies to 

identify a firm or group of firms qualified to undertake work so that 

an expedited task order procedure can be used to select a firm for when 

needs arise. According to IRS officials, they recently entered into a 

multiyear contract with two vendors that will enable them to issue task 

orders when work is needed.



Data Are Not Collected That Could Be Used by IRS Management to 

Determine the Proper Allocation of Resources to Support Testing:



Although IRSís Tax Forms and Publications Division officials believe 

current resources are insufficient to support more testing of forms and 

instructions, they do not have some of the information needed to 

determine whether to allocate additional resources. This information is 

not available at least in part because division guidelines and policies 

do not require that it be gathered.



Officials said that because they have so few staff available to conduct 

tests and have a limited budget to contract for testing, they could not 

increase the number of tests they perform. According to the officials, 

currently only 1 of 103 persons in the division is trained in testing 

methods. In addition to other duties, this person coordinates the tests 

for the division, such as the test of EIC forms and instructions 

completed in 1999 by a private vendor and the test of the innocent 

spouse application form completed by IRS in 2002. Officials also told 

us some staff who are primarily responsible for creating and revising 

tax documents may occasionally assist in conducting tests, such as the 

three persons involved in testing the innocent spouse form. Officials 

also said the total budget for contract support for the division was 

$150,000 in fiscal year 2002, $185,000 in fiscal year 2001, and 

$130,000 in fiscal year 2000.



As part of its annual planning and budgeting process, IRS management 

determines what resources will be needed to accomplish strategies and 

implement programs. IRSís planning and budget guidance requires that 

each operating unit prepare a business plan that, among other things, 

clearly defines priorities and resource requirements. Requests in the 

business plan for resources must be substantiated with evidence that 

allocating additional resources is justified.



However, the division does not systematically identify when testing 

would be beneficial and does not routinely demonstrate the benefits to 

taxpayers and IRS that have been gained from such testing. Officials do 

not identify which of the many changes it makes to forms and 

instructions each year would most likely benefit from testing. Thus, 

the officials cannot tell IRS management how many opportunities to 

improve forms and instructions may be lost due to current resource 

levels. Further, when tests are performed, officials do not identify, 

quantitatively or qualitatively, the benefits that taxpayers and IRS 

may have realized.



One reason that IRS does not have data on forgone testing opportunities 

is that the division lacks formal, written guidelines and procedures 

for determining when testing would be beneficial. Currently, testing is 

an optional step in the process for developing forms. IRSís Tax Forms 

and Publications officials said that they decide which forms to test 

based on informal guidelines and procedures and input from officials in 

IRSís four operating division program offices and the Taxpayer Advocate 

Service. The informal guidelines and procedures call for officials to 

weigh, among other things, whether a form or instruction (a) affects a 

large number of taxpayers, (b) has a high error rate based on 

taxpayersí prior use of the form, (c) is perceived as complex, and (d) 

will be used for several filing seasons. Also, according to IRS, the 

amount of time available to perform tests is factored into testing 

decisions.



These informal guidelines do not require officials to consider in all 

cases whether testing would be beneficial and to document the decisions 

made. Accordingly, even if the informal guidelines are applied, and 

officials judge that some forms or instructions could benefit from 

testing but cannot be tested due to scarce resources, those decisions 

are not made systematically and documented.



Further, although the factors the guidelines suggest taking into 

account appear to have evolved from officialsí experience and therefore 

should be useful, they do not consider some pertinent factors that 

could affect the benefits likely to be realized from testing. For 

instance, the guidelines suggest taking the number of affected 

taxpayers into account but not the likely amount of burden they would 

face due to unclear forms or instructions. They also do not clearly 

call for officials to consider the costs to test forms and instructions 

and the benefits that may accrue throughout IRS, such as in telephone 

service centers. In addition, these informal guidelines and procedures 

automatically exclude testing forms and instructions that will be used 

only one time. Also, according to IRS officials, the time frame between 

the passage of new tax laws and when the newly created or revised forms 

and instructions must be finalized may preclude some forms and 

instructions from being tested. Even if one-time-use forms meet other 

testing criteria, such as affecting a large number of taxpayers who may 

perceive them as complex, IRS will not consider testing them. As the 

rate reduction credit situation discussed earlier illustrates, such 

automatic exclusions may not be appropriate in all situations.



IRS officials do not have information on the results achieved when 

forms and instructions are tested in part because the division does not 

have policies that require such evaluations. When IRS obtained 

information on the reduction in error rates following testing of EIC 

and Child Tax Credit forms and instructions, the studies did not 

include collecting other information on the benefits that may have 

resulted for taxpayers and for IRS. For instance, the studies did not 

estimate the savings IRS may have realized in its telephone and walk-in 

service due to increased form clarity.



Capturing fuller information on the results of testing would be 

consistent with IRSís strategic planning and budgeting process, which 

emphasizes assessing the impact of current programs to efficiently 

allocate resources. Further, by evaluating results of testing 

decisions, IRS officials would be able to determine if their testing 

guidelines and procedures lead to good decisions about when testing is 

most likely to be beneficial. They may also be able to see if the 

methods they use to test--for example, focus groups formed by IRS 

employees or one-on-one interviews with individuals--yield the most 

effective test results.



Conclusions:



IRS continually faces the daunting task of developing and revising tax 

forms and instructions to administer our ever-changing set of federal 

tax laws. Taxpayers rely on IRS for forms and instructions that are as 

clear and easy to understand as possible given the complexity of the 

tax laws and providing clear materials is a key goal of IRSís Tax Forms 

and Publications Division.



In attempting to meet this goal, IRS has tested an average of one set 

of forms and instructions each year over the last 5 years. In contrast, 

officials from three federal agencies that routinely collect 

information from the public say that testing documents for clarity 

before using them is their standard practice. They do so because they 

believe testing will ensure that their data collection documents are 

clear and that individuals will understand them and complete them 

accurately.



Although it is difficult to gauge how much testing alone contributes to 

the clarity of tax forms and instructions, IRS officials believe 

testing has contributed to significant declines in taxpayer errors. 

Illustrations we developed based on IRSís experience in testing forms 

and instructions suggest that IRS can completely recover its testing 

costs in the first year following testing in some circumstances and 

that when savings to taxpayers from more understandable forms and 

instructions are considered, total benefits even in the first year 

following tests can be several times IRSís testing costs.



Although they recognize that testing is beneficial, officials say time 

constraints and limited resources preclude more testing. However, IRSís 

procedures do not clearly specify when draft versions of forms and 

instructions should be available for testing. Having a clearly defined 

point where testing would be performed would facilitate establishing 

procedures and deadlines to better ensure that testing could be done 

even within IRSís annual forms update cycle. Further, IRS officials do 

not have information that would help IRS management to determine 

whether to allocate additional resources to support enhanced testing. 

Because IRS lacks standard written procedures for testing, officials 

have not documented cases where testing would likely be beneficial and 

have not demonstrated the benefits that are gained from testing.



Recommendations for Executive Action:



Because testing could potentially yield clearer and more understandable 

tax forms and instructions, thereby producing benefits both to 

taxpayers and IRS, we recommend that the Acting Commissioner of 

Internal Revenue take the following actions.



* Develop written criteria for determining which changes to tax forms 

and instructions should be tested with taxpayers before publication.

:



* Develop official written guidance that incorporates those criteria 

and ensure that the guidance requires staff that develop new or revised 

forms and instructions to document which changes would merit testing 

and why.

:



* Clarify procedures by designating when in the annual forms 

development process that a draft version of forms and instructions 

should be available for testing with taxpayers.

:



* Ensure that an appropriate range of evaluations are conducted of 

tests that are performed to better establish the costs and benefits of 

performing tests and to refine IRSís approach to testing on the basis 

of lessons learned.

:



* Use information gained from documenting when changes to forms or 

instructions likely would be beneficial and from evaluations of tests 

to reassess an appropriate level of resources to perform testing.



Agency Comments and Our Evaluation:



The Acting Commissioner of Internal Revenue provided written comments 

on a draft of this report in an April 7, 2003, letter, which is 

reprinted in appendix III. The Acting Commissioner agreed with our 

recommendations. We are encouraged that IRS plans to implement all but 

one of our recommendations in time for the 2004 forms development 

cycle. Understandably, the remaining recommendation to ensure that an 

appropriate range of evaluations is conducted of tests would take more 

time to put into practice. The Acting Commissioner also provided 

additional comments and observations on our draft report.



The Acting Commissioner commented that the crux of our report is that 

we do not believe IRS has performed adequate testing on new and revised 

tax forms and instructions due to a lack of resources. He said that 

resources for testing forms and instructions have been adequate for the 

testing IRS wanted to perform. While not questioning whether resources 

were adequate for the testing IRS performed, we concluded that IRS 

officials do not have information needed to determine the level of 

resources that should be allocated to testing forms and instructions. 

Accordingly, we recommended that IRS systematically identify 

opportunities to improve forms and instructions through testing and to 

evaluate the costs and benefits when testing is done.



Although agreeing that testing is beneficial, the Acting Commissioner 

also said that there are significant staff costs associated with 

testing that are not included in our cost analysis. We recognize that 

our analysis excluded staff costs and as stated in our draft report we 

sought to be conservative in estimating the benefits of testing, in 

part because we did not have information on the full range of costs IRS 

incurs when undertaking projects to tests forms and instructions. 

During the course of our work, we requested estimates of staff costs 

for testing but IRS was unable to provide them. Nevertheless, at least 

in the cases we illustrated the potential benefits of testing were so 

much greater than the costs that including staff costs likely would not 

have substantially changed the results of our illustrations.



The Acting Commissioner expressed concern about whether IRS could have 

forms and instructions ready for the filing season if testing was done 

late in the forms development cycle as shown in our figure 2 depicting 

IRSís process. We agree with the Acting Commissionerís concern; 

however, our figure shows the various points at which testing can occur 

in IRSís current processes based on interviews with IRS officials and 

the documentation they provided us. As IRS implements our 

recommendation to clarify when testing should be done, selecting a 

point as early as possible would help maximize the number of changes 

that can be tested during the annual forms update cycle.



The Acting Commissioner also said that he disagreed with our conclusion 

that IRSís experience with obtaining feedback on its products is 

limited or recent. He said IRS uses various methods to obtain customer 

feedback. We agree that IRS uses methods other than testing to obtain 

feedback on its forms and instructions. However, our report describes 

the potential benefits and costs of testing as a feedback method. In 

terms of testing, IRS has only tested five forms and instructions 

during July 1997 through June 2002; in our view, this is a limited 

number of tests that were conducted during the recent past.



The Acting Commissioner also disagreed that testing would result in 

reduced demand for walk-in and toll-free assistance. He said that IRS 

lacks data to support such a conclusion and, based on its experience, 

new forms generate requests for assistance and error rates on them tend 

to be higher. We recognize that there will always be a demand for 

taxpayer customer assistance. However, we believe that reduced demand 

for assistance is a potential benefit of testing. We note, for example, 

that IRS officials seek input from telephone assistors when deciding 

which forms or instructions need to be clarified, apparently believing 

that clarifying the forms and instructions may help reduce calls to 

assistors. Finally, testing is one means for ensuring that even for new 

forms and instructions requests for assistance and errors made by 

taxpayers will be minimized.



We are sending copies of this report to the Chairman and Ranking 

Minority Member of the House Committee on Ways and Means and its 

Subcommittee on Oversight; the Secretary of the Treasury; the Acting 

Commissioner of Internal Revenue; the Director of the Office of 

Management and Budget; and other interested parties. We will make 

copies available to others on request. In addition, the report will be 

available at no charge on the GAO Web site at http://www.gao.gov.



This report was prepared under the direction of Charlie Daniel, 

Assistant Director. If you have any questions regarding this report, 

please contact him or me at (202) 512-9110. Key contributors to this 

report were David Alexander, Christopher Currie, Ronald La Due Lake, 

Anne Laffoon, Veronica Mayhand, Edward Nannenhorn, and Shellee Soliday.



Michael Brostek:



Director, Tax Issues:



Signed by Michael Brostek:



[End of section]



Appendix I: Objectives, Scope, and Methodology:



To determine how often IRS has used taxpayers to test the clarity of 

new or revised individual income tax forms and instructions, we 

interviewed officials in its W&I Divisionís Tax Forms and Publications 

Division in Washington, D.C. We sought to obtain (a) an understanding 

of the process IRS used to develop and revise individual income tax 

forms and instructions and (b) gather information on the forms and 

instructions IRS tested using taxpayers for the 5-year period between 

July 1997 and June 2002. We requested information for a 5-year span to 

help ensure that the information collected would reflect the amount of 

testing usually done by IRS. Our work did not include assessing IRSís 

processes for developing and revising notices or publications or 

assessing the clarity of any specific tax forms or instructions.



To obtain insights on the benefits of testing written documents, we 

interviewed officials from three federal agencies in the metropolitan 

Washington, D.C., area that perform extensive research and data 

collection using private citizens. We contacted NCHS, BLS, and the 

Census Bureau because they have broad experience in conducting tests of 

the clarity of written documents, such as forms, surveys, and 

instructions. We also contacted a private research firm that 

specializes in testing forms and surveys for a variety of public and 

private sector clients. We also reviewed our own guidance for 

developing and using questionnaires.[Footnote 23]3:



To determine the benefits to taxpayers and IRS of testing tax forms and 

instructions with taxpayers prior to their use by the public, we took 

several steps. First, we interviewed IRS officials including Tax Forms 

and Publications Division officials to obtain information on whether 

they perceived testing as being beneficial to taxpayers and IRS. We 

also interviewed IRSís W&I Research Division officials in Indianapolis 

to obtain their views on whether testing tax forms and instructions 

benefits taxpayers and IRS. The research division officials, among 

other things, collect information on taxpayer errors that the Tax Forms 

and Publications Division uses when deciding which forms and 

instructions to test. These officials provided us with data on changes 

in error rates for EIC and Child Tax Credit forms and instructions 

before and after IRS revised and tested them.



Second, we developed illustrations of the potential benefits and costs 

of IRSís testing EIC and Child Tax Credit forms and instructions by 

analyzing the data on the changes in error rates obtained from IRS 

research officials. To construct our illustrations, we used readily 

available data and made certain assumptions. Data were not available on 

many of the potential benefits of the changes and on the full range of 

costs IRS incurred to conduct the tests. We developed similar 

illustrations of potential benefits to taxpayers and IRS if the rate 

reduction credit instructions had been tested. In all cases, the 

illustrations we developed are not actual assessments of the costs and 

benefits that were associated with testing forms and instructions, or 

that would have resulted if testing had occurred. In developing the 

illustrations we sought to be conservative in estimating benefits, in 

part because we did not have information on the full range of costs IRS 

incurred to test forms and instructions. Because we had to make various 

assumptions, our illustrations undoubtedly vary from actual costs and 

benefits.



To determine the potential cost savings to IRS of testing these forms 

and instructions, we first estimated IRSís costs to correct a taxpayer 

error. Our estimates of IRSís cost to correct errors were limited to 

the labor cost associated with correcting errors in IRSís math error 

program. IRS provided us the number of FTE staff years for operating 

the ERS that was used to detect and correct math errors and the number 

of errors corrected by ERS during fiscal year 2002. We used this 

information to calculate an average labor cost to correct math errors 

that we then applied to reductions in EIC and Child Tax Credit errors 

as well as to error reductions that might have resulted from testing 

the rate reduction credit. Total error correction costs may be higher 

because we did not include a number of other costs associated with 

correcting errors. For example, we excluded printing and postage costs 

for the notices sent to taxpayers. IRS notices often cover more than 

one issue associated with a tax return and data were not readily 

available to determine what portion of the postage cost might be 

attributable to EIC or Child Tax Credit issues alone. In addition, the 

estimate does not include costs such as equipment and rent. We did not 

specifically test the accuracy of the cost information provided; 

however, our audits of IRSís annual financial statements have raised 

concerns regarding IRSís ability to identify all costs associated with 

a given program or activity.[Footnote 24]4:



Because data were not available on the extent that testing reduced 

errors related to EIC and Child Tax Credit forms and instructions, we 

assumed different percentage reduction rates in errors due to testing 

these forms and instructions. Our illustrations show potential savings 

based on assumed percentage reduction in errors attributable to testing 

of 50, 25, and 10 percent. Using our estimate of IRSís costs to correct 

an error and the assumed number of errors eliminated by testing, we 

arrived at the costs that would have been incurred to correct those 

errors. The difference between costs to correct eliminated errors and 

the costs to test is the potential cost savings from testing. Cost 

savings due to reduced errors likely would not mean reductions to IRSís 

budget. Further, savings likely would mean that IRS would provide 

services to other taxpayers or would pursue other compliance or tax 

collection activities that it would otherwise have been unable to do.



To determine IRSís costs to test forms and instructions, we used IRSís 

actual contract costs for testing EIC and Child Tax Credits in 1999. 

IRS spent a total of about $56,000 to test both EIC and Child Tax 

Credit forms and instructions with focus groups. Although the cost of 

contracted support for testing each form and instruction individually 

likely would have been somewhat lower than this, we applied the total 

cost in each case.[Footnote 25]5 Because IRS could not provide data, we 

did not include the costs associated with IRS letting and managing the 

contract or the cost for Tax Forms and Publications Division staff to 

work with the contractor in conducting and managing the tests or the 

cost of any other IRS staff that were involved in these tests.



To determine the potential benefits to taxpayers from testing EIC and 

Child Tax Credit forms and instructions, we calculated estimated values 

to taxpayers of the time saved if testing improved form and instruction 

clarity, thereby reducing taxpayer errors and the burden of dealing 

with IRS error notices. We also estimated taxpayersí time saved if 

testing reduced the time needed to understand and complete tax forms 

and instructions. We developed similar illustrations for potential 

benefits to taxpayers if the rate reduction credit had been tested. Our 

benefit illustrations were based on a series of assumptions. For 

example, to estimate the value to taxpayers of time saved from not 

having to deal with an IRS notice, we applied the minimum wage rate 

that was in effect after IRS revised and tested the forms and 

instructions to estimates of time taxpayers might save by not having to 

deal with an IRS notice.[Footnote 26]6 For these illustrations we 

assumed, on average, that taxpayers who received an IRS error notice 

might spend either 2 or 5 minutes to deal with it. Data were 

unavailable on how much time taxpayers actually spend dealing with 

IRSís notices; however, according to Taxpayer Advocate Service 

information, IRSís notices are difficult for taxpayers to understand. 

Further, taxpayers who decide to contest an IRS notice may take time to 

call or write letters to IRS or to contact and work with a tax 

preparer. Data were not readily available to determine what portion of 

taxpayers who received an EIC or Child Tax Credit error notice 

contested IRSís change to their tax returns.



For our illustrations of the estimated value of time saved by taxpayers 

if testing reduced the time needed to understand and complete EIC and 

Child Tax Credit forms and instructions, we assumed that taxpayers 

would save on average 1 minute by using clearer forms and instructions. 

We excluded from our illustrations those tax returns prepared by 

preparers because the taxpayers might not have had to read and 

understand the forms and instructions. Because data were not available 

on the number of tax returns claiming the Child Tax Credit that were 

prepared by paid preparers, we reduced the number of returns claiming 

the credit by the same percentage of EIC returns that were prepared by 

preparers. The percentage of EIC claimants using paid preparers exceeds 

the average for all taxpayers. For the rate reduction credit, our 

illustration is based on a 30-second time savings and the number of 

taxpayers who filed on paper and did not use preparers. We chose a 30-

second potential savings for this illustration because many taxpayers 

would have had to read only part of the instructions to determine what 

to do. We aggregated the times for all taxpayers and multiplied the 

total hours saved by the prevailing minimum wage rate to arrive at 

estimated benefits to taxpayers.



To determine whether any factors limited IRSís ability to use 

individual taxpayers to test forms and instructions and, if so, how 

these factors can be addressed, we interviewed IRSís Tax Forms and 

Publications Divisionís officials and analyzed supporting data they 

provided us. Regarding officialsí view that they lacked sufficient time 

to do more testing, we reviewed information on the amount of time IRS, 

NCHS, and the private research firm we contacted took to perform 

various types of tests. We also reviewed IRSís process for developing 

new and revised forms and instructions and determined how many weeks 

were available between the dates that various laws were enacted or 

their provisions became effective and IRSís normal October 1st deadline 

for printing. Finally, regarding officialsí view that they lacked 

sufficient resources to do more testing, we obtained information on the 

resources available within IRS for testing.



We performed our work from May 2002 through March 2003 in accordance 

with generally accepted government auditing standards.



[End of section]



Appendix II: Use of Testing in NCHS, Census, and BLS:



Like IRS, the three federal agencies we contacted create written 

documents to be completed by the public. The agencies create documents 

such as forms, surveys, and questionnaires that they use to collect 

information from the public to fulfill their missions. Unlike IRS, 

described below, these agencies routinely test their forms, surveys, 

and questionnaires prior to distribution to the public.



* The National Center for Health Statistics, the nationís principal 

health statistics agency, compiles statistical information to guide 

actions and policies relevant to public health and health policy. 

According to NCHS, obtaining accurate and usable health information is 

crucial to successfully fulfilling its mission to provide reliable 

information to the Centers for Disease Control and Prevention and the 

Department of Health and Human Services. NCHS collects information 

through various sources including questionnaires that it develops and 

administers. Researchers at NCHS told us that in support of their 

research they administer surveys and questionnaires each year in 

addition to developing questionnaires used by the Centers for Disease 

Control and Prevention. They test each questionnaire using one-on-one 

interviews. When documents pertain to a particular rather than a 

general population, the researchers recruit participants with 

characteristics similar to those persons who might be completing the 

forms or questionnaires. For example, researchers recruited asthmatics 

to test a questionnaire related to asthma. NCHS tests forms in one-on-

one settings in which a participant may be asked to work through a form 

while a moderator observes and then later interviews the participant. 

This approach allows the researcher to identify specific points at 

which the forms were confusing or problematic and learn why the 

participant had difficulty. NCHS prefers to use one-on-one interviews 

when conducting tests because this method closely resembles the ways in 

which individuals will be completing the documents since individuals 

will likely complete the documents by themselves.

:



* The Census Bureau is the principal agency responsible for collecting 

and providing data about the people and the economy of the United 

States. An accurate census is important because census results are used 

to reapportion seats in the House of Representatives, redraw 

congressional districts and other political boundaries, and address 

countless other public and private data needs. The Census Bureau 

collects information through short-form and long-form questionnaires 

that it develops, tests, and administers. In preparation for the 2000 

Census, the Congress budgeted millions of dollars to develop and test 

questionnaires during the 1990s. The Census Bureauís policy requires 

that demographic survey questionnaires be tested. It has used focus 

groups and one-on-one interviews to test its questionnaires and forms. 

For example, in fiscal year 1996, the Census Bureau decided to make 

fundamental changes to the traditional census design such as shortening 

census questionnaires. In that year, it budgeted funds to test, among 

other things, respondentsí understanding of race and ethnicity 

questions. The Census Bureau has also conducted detailed cost-benefit 

analyses of alternative designs; in 1992 it tested the simplified 

questionnaire in order to gauge whether the new form would increase 

response rates and reduce costly follow up with households that did not 

respond to the census.[Footnote 27]7 

:



* The Bureau of Labor Statistics, the principal fact-finding agency for 

the federal government in the broad field of labor economics and 

statistics, also depends on clear and understandable written documents 

to collect accurate information from the public. According to BLSís 

policy, testing documents such as forms and surveys prior to use by the 

general public should be undertaken to help identify factors that may 

impede usersí ability to understand forms or surveys. Then these 

factors can be addressed in order to improve the clarity of written 

documents and increase the accuracy of responses. Testing should be 

done in the early stages of document development so that any problems 

with clarity can be identified early. BLS routinely tests its written 

documents using focus groups and one-on-one interviews, and uses the 

results of the tests to make improvements to the documents. 

:



[End of section]



Appendix III: Comments from the Internal Revenue Service:



DEPARTMENT OF THE TREASURY INTERNAL REVENUE SERVICE WASHINGTON, D.C. 

20224:



COMMISSIONER:



April 7, 2003:



Mr. Michael Brostek Director, Strategic Issues United States General 

Accounting Office 441 G Street, N.W.



Washington, D.C. 20548:



Dear Mr. Brostek:



We appreciate the opportunity to review and provide comments on the 

draft report titled ďTax Administration: IRS Should Reassess the Level 

of Resources for Testing Forms and Instructions (GAO-03-486).Ē We value 

the input of the General Accounting Office on our tax forms and 

publication development process. As you know, providing clear and 

understandable tax forms and instructions is a goal the IRS has always 

pursued to help simplify taxpayersí tax-related obligations and reduce 

taxpayer burden in complying with the tax laws. Along with the general 

comments we provided on the report to your staff, we would like to 

offer the following observations:



* We appreciate that changes were made to the draft report as a result 

of preliminary comments we provided. The crux of the report is that GAO 

does not believe IRS has performed adequate testing on new and revised 

tax forms and instructions due to lack of resources. We would like for 

the record to reflect the resources for testing forms and instructions 

have been adequate for the testing we wanted to do. However, as stated 

in our response to the recommendations, we agree to establish a means 

of documenting how decisions are made about testing. We also agree that 

developing a cost-benefit analysis to assess the results of testing 

will help address this issue.



* We fully agree it is beneficial to test certain new or redesigned 

products. However, there are significant costs in staff resources 

(including training, travel, salaries, etc.) for creating or 

redesigning the tested product that are not included by GAO in the cost 

analysis. In the future, we will assess possible cost analysis methods 

that take all of the associated factors into account when designing new 

products.



We agree with GAO that other issues outside of our control, such as a 

taxpayerís poor math skills, will continue to cause errors. These all 

contribute to the difficulty in developing a cost-benefit analysis on 

the value of any testing.



* We currently use a number of sources to identify materials to be 

tested. In conjunction with Wage and Investment Research Group 3, Tax 

Forms and Publications can identify the products on which the greatest 

number of errors occur and those impacting the largest number of 

taxpayers. These criteria are of great value in selecting materials to 

be redesigned and tested.



There are situations when late legislation severely limits our time to 

develop materials for public use. The rate reduction credit in 2001 is 

a prime example. However, these situations are relatively uncommon.



* We recognize the benefits of obtaining feedback from customers on our 

products and strive to obtain feedback through a variety of methods in 

addition to conducting focus groups. We agree that we conducted five 

tests of forms and instructions in focus groups, and one-on-one 

interviews between July 1997 and June 2002. We also obtain customer 

feedback on our products (forms, instructions, and publications) in 

other ways as well. We conduct town hall meetings with the public, 

participate in annual Tax Forums, solicit e-mail suggestions from 

taxpayers and IRS employees, and regularly meet with professional 

groups such as the American Payroll Association, and the American 

Institute of Certified Public Accountants. We use feedback from these 

sources to make many changes to improve the clarity and 

understandability of our products. Our use of these tools to obtain 

customer feedback dates back to the late 1970ís. Therefore, we would 

like to go on record as disagreeing with GAOís conclusion that IRSí 

experience with obtaining feedback on its products is limited or 

recent.



The report reflects GAOís conclusion that testing a product will result 

in reduced demand for IRSí walk-in or toll-free telephone assistance. 

Since we currently do not have the data to support such a conclusion, 

we disagree with it. While testing a product may indicate where 

problems may occur, it cannot predict the number of contacts that will 

actually result. Our experience has shown that a new product is likely 

to generate requests for assistance because taxpayers are unfamiliar 

with it. Similarly, error rates on new forms tend to be higher the 

first year a product is issued.



We still are concerned with the box in Figure 2 for considering a test 

as changes are made to finalize a form. The amount of time in that 

stage of the development process is quite limited. We feel it would not 

be wise to conduct testing at the ďMake changes and finalize formĒ 

stage of development. It is too late in the forms development process 

to test the form, review the suggested changes, and then incorporate 

them in the final product. The printing and distribution of the form in 

time for the filing season would be compromised.



My responses to your recommendations for Executive Action are enclosed. 

If you need any additional information, please let me know or contact 

Floyd Williams, Director, Office of Legislative Affairs at 202-622-

3720.



Sincerely,



Bob Wenzel:



Acting Commissioner:



Signed by 

Bob Wenzel:



Enclosure:



Enclosure:



Recommendation:



Develop written criteria to be used in determining which changes to tax 

forms and instructions should be tested with taxpayers before 

publication.



Response:



We agree with the recommendation and will develop written criteria for 

determining which changes to tax forms and instructions should be 

tested. These written criteria will be in place for the 2004 

development cycle.



Recommendation:



Develop official written guidance that incorporates those criteria and 

ensure that the guidance requires staff that develop new or revised 

forms and instructions to document which changes would merit testing 

and why.



Response:



We agree with the recommendation and will provide official written 

guidance for using criteria when developing new and revised forms and 

instructions so that documentation is available for decisions made. The 

written guidance will be in place for the 2004 development cycle.



Recommendation:



Clarify procedures by designating when in the annual forms development 

process that a draft version of forms and instructions should be 

available for testing with taxpayers.



Response:



We agree with the recommendation and are revising our operating 

procedures to designate when in the annual forms development process a 

draft version of forms and instructions should be available for 

testing. The revised operating procedures will be in place for the 2004 

development cycle.



Recommendation:



Ensure that an appropriate range of evaluations are conducted of tests 

that are performed to better establish the costs and benefits of 

performing tests and to refine IRSí approach to testing on the basis of 

lessons learned.



Response:



We agree with the recommendation and will proceed with plans to 

establish a means for cost-benefit analyses to be performed to assess, 

to the extent possible, the benefits from testing tax forms and 

instructions. This will require a major project to assess the reduction 

in errors and the impact on processing, customer assistance, compliance 

activities, etc., to isolate the benefits from testing tax products 

from other assessment:



tools that create benefits (e.g., practitioner feedback). The scope of 

the recommendation will most likely require a phased-in approach to 

implement the processes that ultimately are agreed to by all affected 

organizations within the IRS. We anticipate that analyses can begin 

during 2004.



Recommendation:



Use information gained from documenting when changes to forms or 

instructions likely would be beneficial and from evaluations of tests 

that are performed to reassess an appropriate level of resources to 

perform testing.



Response:



We agree with the recommendation. Once the written criteria, official 

written guidance, and revised operating procedures are implemented, we 

will use the information gathered from testing to reassess the level of 

resources available to perform testing. We plan to implement this 

recommendation starting in 2004.



[End of section]





FOOTNOTES



[1] Tax forms and instructions as referred to in this report also 

include related schedules and worksheets. 



[2] 1993 IRS Customer Satisfaction Survey: Public Perceptions of IRS 

Service Quality, Contract number TIR 90-0002 (Aug. 27,1993). 



[3] U.S. General Accounting Office, Tax Administration: Advance Tax 

Refund Program Was a Major Accomplishment, but Not Problem Free, GAO-

02-827 (Washington, D.C.: Aug. 2, 2002).



[4] The Process of Developing Tax Forms for Individual Taxpayers Should 
be 

Further Improved, TIGTA 2000-40-060 (March 2000); U.S. General 

Accounting Office, Tax Administration: IRS Efforts to Improve Forms and 

Publications, GAO/GGD-95-34 (Washington, D.C.: Dec. 7, 1994).



[5] Because our review was limited to forms and instructions used by 

individual taxpayers, table 1 includes only tests IRS did on these 

documents during this period. IRS conducted some other tests during 

this period. For example, IRS used focus groups to test 5 publications. 

Of the 5 publications, 3 related to individual taxpayers, 1 to business 

taxpayers, and one related to both individual and business taxpayers. 

IRS estimates that it will revise over 90 publications during 2003.



[6] U.S. General Accounting Office, Developing and Using 

Questionnaires, GAO/PEMD-10.1.7 (Washington, D.C.: October 1993).



[7] Floyd J Fowler, 2002; ďSurvey Research MethodsĒ; Third Edition; 

Sage Publications, Applied Social Research Methods Series, volume 1 

(2002). Also see R. Tourangeau, L.J. Rips, and K. Rasinski, The 

Psychology of Survey Response (2000).



[8] As noted in table 1, IRS tested five forms during July 1997 through 

June 2002. IRS did not determine the effect of testing on error rates 

after testing the EIC Schedule in 1998 according to IRS Research 

officials. IRS tested Schedule D in 2001. Any change in error rates 

would be based on the 2002 filing season, and IRS has not yet analyzed 

2002 figures. IRS tested the Innocent Spouse form in 2002 and any 

change in error rates will not occur until the 2003 filing season. 



[9] Most taxpayers eligible for the EIC have a qualifying child.



[10] Generally, because IRS has more demand for assistance from 

taxpayers than it can handle and identifies more potentially 

noncompliant tax returns than it can address, benefits from testing are 

unlikely to include reductions in IRSís overall budget.



[11] In the math error program, IRS uses computer comparisons and 

calculations to correct clerical and mathematical errors on tax 

returns. These corrections involve addition and subtraction errors; 

incorrect social security numbers, filing status and exemptions; and 

missing schedules or forms.



[12] Although in many cases taxpayers would not need to respond to a 

math error notice, in some cases, such as when IRS has recalculated EIC 

or other benefits due to its detection of an apparently invalid social 

security number, the taxpayer may be able to provide IRS additional 

information to substantiate the original return.



[13] Error correction costs relate to the error resolution process. 

When a taxpayer makes an error, in some cases, IRS will correct the 

error. For example, if a taxpayer transposes information from the 

Schedule D to an incorrect line on the form 1040, IRS can correct this 

mistake for the taxpayer. IRS cannot correct certain errors made by 

taxpayers such as claiming EIC for a child whose social security number 

on the tax return does not match Social Security Administration 

records. In this case, because it cannot determine the correct social 

security number for the child, IRS will disallow the claim and send an 

error notice to the taxpayer that explains what IRS did and what the 

taxpayer can do if he/she disagrees with IRSís actions.



[14] Department of the Treasury, Internal Revenue Service: Compliance 

Estimates for Earned Income Tax Credit Claimed on 1999 Returns 

(Washington, D.C.: Feb. 28, 2002).



[15] An FTE consists of one or more employees who collectively work 1 

year. For example, one full-time employee or two half-time employees 

equal one FTE.



[16] Because IRS officials attributed reduced errors both to their new 

approach to structuring EIC guidance and to improvements to the new 

approach that resulted from testing, we chose to illustrate reductions 

due to testing alone that accounted for half or less of the decline in 

errors.



[17] We used minimum wage to value taxpayersí time to be conservative 

in estimating potential benefits because EIC is targeted to low-income 

taxpayers. Minimum wage is a conservative estimate of taxpayersí time 

even for EIC in that some taxpayers can qualify for EIC at incomes that 

are three times minimum wage. For purposes of the Paperwork Reduction 

Act, OMB values time for dealing with paperwork requirements like tax 

forms at $30 per hour. 



[18] Taxpayers who were eligible to receive an advance tax refund in 

2001 but who (1) did not receive a check because IRS did not have their 

current addresses or (2) did not have enough taxable income in 2000 to 

qualify for the maximum amount allowable, may have been entitled to a 

rate reduction credit when filing their tax year 2001 returns. In 

addition, taxpayers who were not eligible for an advance tax refund, 

such as those who did not have taxable income in 2000, may have been 

entitled to a rate reduction credit provided they had taxable income in 

2001.



[19] This total only includes rate reduction credit errors corrected by 

IRS under its math error authority as of September 27, 2002. Some 

electronically filed tax returns that IRS rejected during screening for 

processing also had rate reduction credit errors.



[20] GAO-02-827.



[21] Consistent with our effort to be conservative in estimating 

testing benefits, we assumed that testing would have reduced errors by 

no more than 25 percent because many taxpayers only needed to review a 

sentence or two of the instructions and did not need to do any 

calculations.



[22] We assumed an average taxpayer time savings of 30 seconds because 

many taxpayers only needed to review a sentence or two of the 

instructions and did not need to do any calculations.



[23] [3] GAO/PEMD-10.1.7.



[24] [4] U.S. General Accounting Office, Major Management Challenges 

and Program Risks: Department of the Treasury, GAO-03-109 (Washington, 

D.C.: January 2003).



[25] [5] IRS contracted with a firm to conduct focus groups to test EIC 

and Child Tax Credit forms and instructions. The contract was for 

$54,000 and IRS spent an additional $2,000 on related travel costs. 

Because one contractor conducted both tests, IRS officials said it 

would be difficult to allocate the $56,000 between the two credits.



[26] [6] Our estimates of the value to taxpayers of time saved from 

using clearer forms and instructions are conservative estimates. The 

minimum wage rate of $5.15 per hour that we used in our illustrations 

is substantially less than OMBís estimate of $30 per hour as the 

average value of time for individuals and entities to provide 

information to the government. Since EIC claimants are lower-income 

taxpayers, we used the minimum wage rate. However, some taxpayers would 

qualify for EIC even at three times the income of a full time minimum 

wage level employee.



[27] [7] U.S. General Accounting Office, Census Reform: Questionnaire 

Test Shows Simplification Holds Promise, GAO/T-GGD-92-59 (Washington, 

D.C.: July 1, 1992) and U.S. General Accounting Office, Decennial 

Census: 1990 Results Show Need for Fundamental Reform, GAO/GGD-92-94 

(Washington, D.C.: June 9, 1992) address recommendations that the 

Census Bureau simplify its questionnaire in order to improve its 

response rate and thereby reduce the number of follow-ups with 

nonrespondents and costs associated with follow-up activities.



GAOís Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAOís commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAOís Web site ( www.gao.gov ) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as ďTodayís Reports,Ē on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select ďSubscribe to daily E-mail alert for newly 

released productsĒ under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office



441 G Street NW,



Room LM Washington,



D.C. 20548:



To order by Phone: 	



	Voice: (202) 512-6000:



	TDD: (202) 512-2537:



	Fax: (202) 512-6061:



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov



Automated answering system: (800) 424-5454 or (202) 512-7470:



Public Affairs:



Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.



General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.



20548: