This is the accessible text file for GAO report number GAO-05-130 
entitled 'Medicare: Accuracy of Responses from the 1-800-MEDICARE Help 
Line Should Be Improved' which was released on December 08, 2004.

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

Report to Congressional Committees:

United States Government Accountability Office:

GAO:

December 2004:

Medicare:

Accuracy of Responses from the 1-800-MEDICARE Help Line Should Be 
Improved:

GAO-05-130:

GAO Highlights:

Highlights of GAO-05-130, a report to congressional committees

Why GAO Did This Study:

In March 1999, the Centers for Medicare & Medicaid Services (CMS) 
implemented a telephone help line—1-800-MEDICARE—to provide information 
about program eligibility, enrollment, and benefits. The Medicare 
Prescription Drug, Improvement, and Modernization Act of 2003 (MMA) 
directed GAO to examine several issues related to this 24-hour help 
line and the customer service representatives (CSRs) who staff it. In 
this report, GAO evaluated (1) the accuracy of the information the help 
line provides, (2) the training given to CSRs, and (3) CMS’s efforts to
monitor the accuracy of information provided through the help line.

What GAO Found:
The 1-800-MEDICARE help line provided accurate answers to 61 percent of 
the 420 calls we made and inaccurate answers to 29 percent. We were not 
able to obtain any answers for the remaining 10 percent of our calls at 
the time we placed them. Most of these calls were not answered because 
they were transferred to other contractors responsible for processing 
Medicare claims that were not open for business at the time we called 
or these calls were inadvertently disconnected. To facilitate accurate 
responses, the 1-800-MEDICARE help line provides CSRs with written 
answers—called “scripts”—that CSRs use during a call. When CSRs 
provided inaccurate information, it was largely because they did not 
seem to access and effectively use a script that answered our 
questions. CMS and its contractor do not routinely pretest the scripts 
to ensure that they are understandable to CSRs or potential callers.

Percentage of Calls with Accurate, Inaccurate, and No Answers: 

[See PDF for image]

Note: Based on 420 calls placed in July 2004.

[End of figure]

The training for CSRs meets CMS’s requirements, but it is not 
sufficient to ensure that CSRs are able to answer questions accurately 
on the help line. Before handling calls, CSRs must complete about 2 
weeks of classroom training; accurately answer two simulated calls 
consecutively out of six; and score at least 90 percent on a written 
exam. In addition, all CSRs receive ongoing training. However, the 
results from our calls indicate that the testing and simulated call 
answering did not sufficiently measure whether CSRs were prepared to 
answer questions accurately. 

CMS delegates most accuracy monitoring to one of its contractors and 
reviews the results. The bulk of the monitoring focuses on how 
accurately individual CSRs answer questions. However, this monitoring 
does not systematically track questions answered inaccurately by CSRs 
as a group, which could help target training and script improvement. 
Through two smaller studies that measured how accurately specific 
questions were answered, CMS was able to identify areas to improve 
scripts and training.

What GAO Recommends:

To improve the accuracy of the information the help line provides, GAO 
recommends that CMS 
(1) revise procedures so that calls are not transferred to other 
contractors that are closed, 
(2) assess current scripts and pretest new and revised scripts to 
ensure that they are understandable, (3) provide more testing of CSRs’ 
ability to accurately answer questions and use the results to target 
training efforts as needed, and (4) monitor the accuracy rate for each 
frequently asked question and use the results to modify scripts or 
provide training, if necessary. CMS agreed with the recommendations.

www.gao.gov/cgi-bin/getrpt?GAO-05-130.

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Leslie G. Aronovitz at 
(312) 220-7600.

[End of section]

Contents:

Letter:

Results in Brief:

Background:

1-800-MEDICARE Provided Accurate Information for Less Than Two-thirds 
of the Calls We Placed:

CSR Training Met CMS's Requirements, But Was Not Sufficient to Ensure 
Accurate Responses:

Monitoring Generally Focused on Performance of Individual CSRs, Rather 
than Accuracy Rates for Specific Questions:

Conclusions:

Recommendations:

Agency Comments and Our Evaluation:

Appendix I: Scope and Methodology:

Appendix IIPrescription Drug Discount Card Questions and Information 
Used to Develop Accuracy Criteria:

Appendix III: $600 Credit Question and Information Used to Develop 
Accuracy Criteria:

Appendix IV: Medigap Question and Information Used to Develop Accuracy 
Criteria:

Appendix V: Power Wheelchair Question and Information Used to Develop 
Accuracy Criteria:

Appendix VI: Medicare Part B Question and Information Used to Develop 
Accuracy Criteria:

Appendix VII: Eye Glasses/Exam Question and Information Used to Develop 
Accuracy Criteria:

Appendix VIII: Comments from the Department of Health & Human Services:

Appendix IX: GAO Contact and Staff Acknowledgments:

GAO Contact:

Acknowledgments:

Tables:

Table 1: Questions and Criteria Used to Determine Answer Accuracy:

Table 2: Summary of Unanswered Calls by Question:

Figure:

Figure 1: Percentage of Calls with Accurate, Inaccurate, and No 
Answers, by Question:

Abbreviations:

CDC: Centers for Disease Control and Prevention: 
CMS: Centers for Medicare & Medicaid Services: 
CSR: customer service representative: 
IRS: Internal Revenue Service: 
MMA :Medicare Prescription Drug, Improvement, and Modernization Act of 
2003:

United States Government Accountability Office:

Washington, DC 20548:

December 8, 2004:

The Honorable Charles E. Grassley: 
Chairman: 
The Honorable Max Baucus: 
Ranking Minority Member: 
Committee on Finance: 
United States Senate:

The Honorable Joe Barton: 
Chairman: 
The Honorable John D. Dingell: 
Ranking Minority Member: 
Committee on Energy and Commerce: 
House of Representatives:

The Honorable William M. Thomas: 
Chairman: 
The Honorable Charles B. Rangel: 
Ranking Minority Member: 
Committee on Ways and Means: 
House of Representatives:

Medicare is a federal program that helps pay for a variety of health 
care services and items for approximately 41 million elderly and 
disabled beneficiaries. One of the responsibilities of the Centers for 
Medicare & Medicaid Services (CMS), the federal agency that administers 
Medicare, is to provide beneficiaries and other members of the public 
with clear, accurate, and timely information about the program. To help 
do so, in March 1999, CMS implemented a nationwide toll-free telephone 
help line--1-800-MEDICARE--which Medicare beneficiaries, their 
families, and other members of the public can call to ask questions 
about program eligibility, enrollment, and benefits. By 2001, the help 
line had customer service representatives (CSR) answering calls 24 
hours a day, 7 days a week. For information about coverage and payment 
for medical services and items, beneficiaries and others have also been 
able to call the companies under contract with CMS to process and pay 
Medicare claims.[Footnote 1]

In 2004, the 1-800-MEDICARE help line significantly expanded its 
operations in order to handle an increased number of calls. This 
increase occurred after the enactment of the Medicare Prescription 
Drug, Improvement, and Modernization Act of 2003 (MMA) in December 
2003, which established a prescription drug discount card program for 
Medicare beneficiaries and required that information on this new 
program be made available through the 1-800-MEDICARE help line. During 
the 6 months following the enactment of MMA, the 1-800-MEDICARE help 
line handled over 9 million calls, more than triple the number handled 
in the previous 6 months. Many of these callers asked about the 
prescription drug benefit that will be available beginning in 2006, the 
prescription drug discount cards available in the interim, and the $600 
credit for prescription drugs purchased by low-income beneficiaries 
with the cards. In response to the increased call volume, in the first 
half of 2004, CMS added over 800 CSRs--more than doubling the number of 
staff who had previously been available to respond to 1-800-MEDICARE 
help line inquiries. In June 2004, CMS also increased the number of 
contractors managing the 1-800-MEDICARE help line from one to two.

MMA also required that GAO examine the accuracy and consistency of 
answers provided through the 1-800-MEDICARE help line and the training 
and education given to its CSRs.[Footnote 2] To address this 
requirement, as discussed with the congressional committees with 
jurisdiction over Medicare, we evaluated (1) the accuracy of 
information that 1-800-MEDICARE provides, (2) the training given to 1-
800-MEDICARE CSRs, and (3) CMS's efforts to monitor the accuracy of 
information provided through the 1-800-MEDICARE help line.

To evaluate the accuracy of information provided by 1-800-MEDICARE, we 
made 420 calls to the 1-800-MEDICARE help line during July 2004 and 
posed 1 of 6 questions about the Medicare program during each 
call.[Footnote 3] For each question, we randomly placed calls at 
different times of the day and days of the week to match the typical 
pattern of calls reported by the 1-800-MEDICARE help line in April 
2004. To develop our 6 questions, we initially formulated 20 questions 
from the 100 topics most frequently addressed by the 1-800-MEDICARE 
help line's CSRs in April 2004. For each of our questions we developed 
criteria that we used to define an accurate response, using information 
on the Medicare Web site's frequently asked questions section.[Footnote 
4] CMS confirmed the responses we provided, and provided us with the 
written answers that the CSRs would be expected to use to respond to 
each question. From the list of 20 questions, we chose 6 questions, 
asked each one a total of 70 times, and evaluated the accuracy of the 
responses using information provided by CMS. If the CSR provided some 
information on the topic, but did not provide sufficient and complete 
information to meet our criteria, we considered the answer to be 
inaccurate. To evaluate the training provided to CSRs, we interviewed 
officials at the primary contractor[Footnote 5] and CMS staff, reviewed 
the instructional materials used in CSR training, and observed a 
training session in June 2004. In addition, we reviewed previous GAO 
reports on the training provided by help lines, including the training 
provided to the CSRs answering calls on the Internal Revenue Service's 
(IRS) help line.[Footnote 6] To evaluate CMS's efforts to monitor the 
accuracy of information provided through the help line, we interviewed 
CMS officials and officials from the primary 1-800-MEDICARE contractor. 
In addition, we reviewed related documents and information about call 
centers in other industries. We did not verify the reliability of CMS's 
monitoring data. Appendix I includes a more detailed discussion of our 
scope and methodology. Our work was conducted from May 2004 through 
December 2004 in accordance with generally accepted government auditing 
standards.

Results in Brief:

In response to our 420 calls to the 1-800-MEDICARE help line, CSRs 
accurately answered the questions posed in 61 percent of the calls we 
placed. The accuracy rate varied significantly among the six questions 
we posed. Twenty-nine percent of the calls were answered inaccurately. 
We were not able to obtain any answers to our questions for the 
remaining 10 percent of our calls at the time we placed them. This 
generally occurred when our morning, evening, and weekend calls were 
transferred to claims administration contractors that were not open for 
business at the time we called or when our calls were inadvertently 
disconnected. To facilitate accurate and consistent responses, the 1-
800-MEDICARE help line provides CSRs with information in the form of 
written answers--known as "scripts"--that CSRs can access by typing in 
key words on their computers during a call. The scripts are designed to 
address the help line's frequently asked questions or provide links to 
additional information. However, we found that CSRs provided inaccurate 
information largely because they did not always understand enough about 
the Medicare program to access a script that answered the question or 
could not clearly explain the material in the scripts that they were 
using. For example, one question about income eligibility for the $600 
prescription drug credit was answered inaccurately in 55 out of 70 
calls, generally because the CSRs did not seek other needed information 
from a second script to correctly answer the question. In addition, 
CSRs sometimes did not understand the language used in the scripts or 
other written material available to them. For example, in answering a 
question on power wheelchair payment, one CSR confused "trunk 
strength"--which is upper body strength--with car trunk space and 
incorrectly explained that Medicare would only cover a power wheelchair 
if a beneficiary had adequate space to put it in the trunk of his car. 
As this example shows, when CSRs do not understand the scripts, they 
provide inaccurate answers. Nevertheless, CMS and its contractor do not 
routinely pretest the scripts to ensure that they are understandable to 
the CSRs or potential callers.

The training for CSRs meets CMS's requirements, but it is not 
sufficient to ensure that CSRs are able to answer questions accurately 
on the 1-800-MEDICARE help line. As required by CMS, all CSRs receive 
training. The primary contractor, which is responsible for training all 
newly hired CSRs, provides them with a 2-week session of classroom 
instruction on accessing and using scripts, customer service etiquette, 
and general information about the Medicare program. As part of the 
training, CMS requires newly hired CSRs to score 90 percent or higher 
on a written exam before they are allowed to answer questions on the 
help line. CMS requires CSRs to pass an exercise answering six 
simulated calls. As required by CMS, both contractors also provide 
their CSRs with periodic refresher training on Medicare program changes 
and other issues. Although the 1-800-MEDICARE contractors meet CMS's 
training requirements by providing instruction and testing, the number 
of inaccurate responses we received on our calls indicates that not all 
CSRs had the necessary knowledge and skills to answer our questions 
accurately. Testing how accurately CSRs answer frequently asked 
questions using scripts provides a better measure of their ability to 
handle calls. However, in their testing, CMS and its contractors do not 
emphasize CSRs' ability to answer questions accurately using scripts. 
For example, to pass the simulated call handling exercise, new CSRs 
have to demonstrate their ability to use scripts to answer calls 
accurately on only two consecutive calls out of six possible attempts. 
Similarly, on the written exam that newly hired CSRs must pass before 
they work on the help line, 24 of the exam's 52 questions ask CSRs to 
identify scripts with information to answer specific questions while 
the remaining 28 questions target other skills needed by help line 
staff. Further, while all CSRs receive continuing training, they are 
not required to demonstrate that they have effectively mastered new 
material in answering calls.

CMS delegates most accuracy monitoring to its primary contractor and 
reviews the results, but this monitoring is not designed to identify 
and systematically track responses to questions that CSRs as a group 
commonly answer incorrectly. The primary contractor listens to the 
conversations CSRs have with callers, observes the scripts they select 
on their computers to answer questions, and then scores the CSRs' 
performance in several categories, including customer service etiquette 
and accuracy. Calls can be evaluated as acceptable if CSRs provide some 
correct information, even if they do not provide enough information to 
accurately answer the question. As required by CMS, the primary 
contractor evaluates four calls per month for each CSR--which is a 
standard amount for the help line industry--and reports the results to 
CMS. Although the primary contractor's monitoring includes an 
assessment of CSRs' accuracy, the monitoring does not categorize and 
identify accuracy rates by question, which could help CMS target 
training and other improvement efforts. In contrast to most of the 1-
800-MEDICARE help line monitoring efforts, on two occasions CMS 
contracted for studies that examined accuracy rates by question. Like 
our methodology, these studies used specific criteria to evaluate an 
answer's completeness and accuracy. The findings from these studies 
were similar to ours--accuracy rates varied by question and were 
sometimes lower than the accuracy rates found through the primary 
contractor's monitoring. For example, one study found that CSRs 
accurately answered between 39 and 69 percent of the questions asked 
about Medicare prescription drug discount card and benefit; in the same 
month the subcontractor found an overall accuracy rate for all 
questions to be about 94 percent. CMS used information from these 
smaller studies that focused on the accuracy rates for specific 
questions to improve the scripts and training for CSRs.

To improve the accuracy of responses provided to the 1-800-MEDICARE 
help line callers, we are making several recommendations. Specifically, 
we are recommending that CMS revise procedures so that calls are not 
transferred to claims administration contractors that are closed for 
business at that time; assess current scripts and pretest new and 
revised scripts before approving them to ensure that they are 
understandable to CSRs and potential callers; provide more testing of 
CSRs' ability to use scripts to accurately answer the most frequently 
asked questions, and use the results to target additional training 
efforts as needed; and monitor the accuracy rate of each frequently 
asked question and use the results to modify scripts or provide 
training, if necessary.

In its comments on a draft of this report, CMS agreed with our 
recommendations and provided additional detail on the challenges it 
faced in administering 1-800-MEDICARE after the massive call volume 
increase that followed MMA. CMS also described the steps that it was 
considering, or had begun, to address our recommendations. CMS 
expressed concern that we considered answers from 1-800-MEDICARE CSRs 
to be inaccurate if they were incomplete. However, we believe that our 
accuracy criteria included the minimum detail needed to avoid 
misleading callers.

Background:

CMS administers the 1-800-MEDICARE help line to answer beneficiaries' 
questions about Medicare eligibility, enrollment, and benefits. The 
help line currently operates 24 hours a day, 7 days a week, and has 
eight call center locations that are run by two contractors. As of 
October 2004, the primary contractor managed 1,332 of the 2,137 CSRs 
and operated seven of the eight 1-800-MEDICARE call centers. In 
addition, the primary contractor is responsible for other activities, 
such as distributing program material requested by callers, training 
all new CSRs before they handle calls on the 1-800-MEDICARE help line, 
and researching answers to more complex questions some callers may 
have. Prior to 2004, one contractor managed the 1-800-MEDICARE help 
line. In June 2004, in response to increasing call volume,[Footnote 7] 
CMS hired a second contractor, which in October 2004 managed 805 CSRs 
and operated one of the eight 1-800-MEDICARE call centers.

A call placed to 1-800-MEDICARE is answered initially by an interactive 
voice response system. This is an automated system that, depending on 
the caller's responses to the system's automated prompts, routes a call 
to a CSR or to other information sources. These other information 
sources can include the other help lines maintained by Medicare's 
claims administration contractors or recorded information.

All CSRs must have a high school diploma or its equivalent, but they 
are not required to be knowledgeable about the Medicare program at the 
time they are hired. To help provide clear, accurate, and timely 
answers to callers' questions, CMS expects the CSRs to use written 
scripts, which contain information about the program.[Footnote 8] CSRs 
listen to a caller's question and then type in related keywords to 
generate a list of suggested scripts that could be used to answer the 
question. The CSRs select the script they consider best suited to 
answer the question and read either excerpts or the entire script. CSRs 
can also consult other information sources. For example, CSRs can use 
Web-based tools available on the Medicare Web site[Footnote 9] to help 
beneficiaries select a prescription drug discount plan, nursing home, 
or home health agency.

Because the types of questions frequently posed to 1-800-MEDICARE 
change in response to program or other policy changes, new scripts may 
need to be created or existing ones updated. Either CMS or the primary 
contractor may decide to develop a new script or update an existing one 
for clarification or in response to program changes. CMS officials 
approve scripts that are developed by the primary contractor, and check 
them for accuracy and completeness.

1-800-MEDICARE Provided Accurate Information for Less Than Two-thirds 
of the Calls We Placed:

The 1-800-MEDICARE help line provided accurate answers to 61 percent of 
the 420 calls we made. The accuracy rate varied significantly among the 
six questions we posed. Overall, 29 percent of our calls were answered 
inaccurately. In general, CSRs erred because they did not understand 
enough about the Medicare program to access the script with information 
to answer the question or clearly explain the material in it. In 
addition, for 10 percent of the calls we placed, we were unable to get 
a response to our question at the time we contacted the 1-800-MEDICARE 
help line, mainly due to problems when CSRs transferred calls.

Accuracy Rate Varied Significantly by Question:

In response to our calls to the 1-800-MEDICARE help line, CSRs answered 
our questions accurately for 256 out of 420 calls, a rate of 61 
percent. The criteria we developed to determine the information that 
constituted an accurate answer for each question are shown in table 1. 
(A more detailed version of the questions and information to answer 
them are in apps. II through VII.) The criteria were based on answers 
we developed from information on the Medicare Web site and were 
confirmed by CMS, which provided us with the scripts that contained 
information to answer the questions.[Footnote 10] We considered all 
calls we placed to the 1-800-MEDICARE help line to be part of our test 
of its accuracy, even if the call was transferred to a claims 
administration contractor to provide the answer.

Table 1: Questions and Criteria Used to Determine Answer Accuracy:

Medicare prescription drug discount card questions: 

Question 1: Drug card; 
Question: Medicare prescription drug discount card questions: What drug 
card can my father-in-law get that will cover all of his drugs at a 
particular pharmacy and have his drugs cost the least amount? 
Criteria: An accurate response would include at least one prescription 
drug discount card recommendation with the lowest total prices for the 
drugs the beneficiary used.

Question 2: $600 credit; 
Question: Medicare prescription drug discount card questions: Can my 
mother qualify for the $600 credit with three specified sources and 
amounts of income? 
Criteria: An accurate response would indicate that a beneficiary with 
the amount of income from the specified sources could receive the $600 
credit.

Question 3: Medigap[A]; 
Question: Medicare prescription drug discount card questions: Can my 
grandmother get a Medicare-approved prescription drug discount card if 
she has Medicare and a Medigap policy? 
Criteria: An accurate response would indicate that a beneficiary could 
have a Medigap policy and still receive a Medicare-approved 
prescription drug discount card.

Other Medicare coverage or eligibility questions: 

Question 4: Power wheelchair; 
Question: Medicare prescription drug discount card questions: Will 
Medicare pay for a power wheelchair for my father? 
Criteria: An accurate response would indicate that (1) a physician must 
prescribe the power wheelchair or determine it to be medically 
necessary, and (2) a power wheelchair requires a copayment on the part 
of the Medicare beneficiary.

Question 5: Medicare part B[B] enrollment; 
Question: Medicare prescription drug discount card questions: Should my 
husband sign up for part B if I am still working and we have health 
insurance coverage from my employer? 
Criteria: An accurate response would indicate that a beneficiary could 
wait to enroll and then sign up for Medicare part B during a special 
enrollment period.

Question 6: Eye exam and glasses; 
Question: Medicare prescription drug discount card questions: Will 
Medicare pay for an eye exam and a new pair of eyeglasses if my 
mother's prescription has changed? 
Criteria: An accurate response would indicate that Medicare does not 
pay for routine eye exams and eyeglasses. 

Source: GAO analysis.

Notes: The analysis is based on a report of the most frequently 
accessed 1-800-MEDICARE scripts in May 2004 provided by CMS, scripts 
related to these questions provided by CMS on July 1, 2004, and 
information from Medicare's Web site on frequently asked questions and 
their answers, http://medicare.custhelp.com/cgi-bin/medicare.cfg/php/
enduser/std_alp.php, accessed on May 21, 2004.

[A] Medigap is a privately purchased health insurance policy that 
supplements Medicare by paying for some of the health care costs that 
are not covered by the program.

[B] Eligible individuals can choose to enroll in part B and pay a 
monthly premium. Medicare part B services include physician and 
outpatient hospital services, diagnostic tests, mental health services, 
outpatient physical and occupational therapy, ambulance services, some 
home health services, durable medical equipment, prosthetics, 
orthotics, and medical supplies.

[End of table]

The percentage of calls CSRs answered accurately varied by question, as 
shown in figure 1. For example, CSRs accurately answered 81 percent of 
the calls asking whether a beneficiary could receive a prescription 
drug discount card if they had a Medigap policy. The answer to the 
Medigap question was clearly described in a script, which allowed CSRs 
to respond with the highest accuracy rate for all of our questions. 
Similarly, for question 1--choosing a prescription drug discount card-
-CSRs answered accurately 76 percent of the time. By July 2004, when we 
placed our calls, a large number of CSRs had been recently hired and 
trained specifically to answer this question, using a script and a Web-
based tool.[Footnote 11] In contrast, for question 2 calls about the 
$600 prescription drug credit, CSRs answered inaccurately 79 percent of 
the time. CSRs scored poorly on this question primarily because they 
based their answers on the beneficiary's total income without 
considering that some specific types of income should not be included 
in the calculation of eligibility for the credit. CSRs would have had 
to access two scripts to correctly answer the question, because the 
more general script on the topic did not contain all of the needed 
information. Question 5, which addressed Medicare part B enrollment, 
also had a relatively high inaccuracy rate--41 percent. We were not 
able to obtain an answer to some of our questions at the time that we 
called, most commonly when CSRs or the interactive voice response 
system transferred calls concerning questions 4 and 6 to other help 
lines.

Figure 1: Percentage of Calls with Accurate, Inaccurate, and No 
Answers, by Question:

[See PDF for image]

Note: Numbers may not add to 100 percent due to rounding.

[End of figure]

Inaccurate Responses Were Largely Due to Ineffective Use of Scripts:

CSRs responded inaccurately to our questions largely because they did 
not seem to know enough to effectively use the scripts. According to a 
CMS official and the primary contractor's staff, CSRs are expected to 
use scripts to guide their discussion with callers; they are not 
supposed to rely solely on acquired knowledge of Medicare to answer 
questions. We found, however, that in responding to our questions CSRs 
usually had one of four problems using scripts. The CSRs (1) did not 
seem to access a script, even when one was available; (2) did not seem 
to access a script with the right information to answer the question; 
(3) did not obtain enough information from the script; or (4) 
misunderstood some of the words in the scripts.

We found instances when CSRs did not seem to access scripts when 
responding to calls. For example, when responding to our calls 
concerning the prescription drug discount card question, 2 CSRs 
indicated that they were not able to inform the caller about which card 
had the lowest drug prices--even though 53 other CSRs successfully used 
a script and a Web-based tool to answer this question. One other CSR 
referred our caller to AARP for an answer, rather than respond with the 
appropriate script and Web-based tool.[Footnote 12] These 3 CSRs did 
not seem to know how to correctly answer this question, which was 
addressed by one of the most commonly accessed script for the first 
half of the year. During 20 of the calls to answer our question on 
whether a spouse should enroll in Medicare part B if he had current 
employment-based health insurance, CSRs told our callers that enrolling 
in Medicare was a personal decision and they could not answer the 
question, which we classified as an inaccurate answer. They did not 
seem to recognize that they could access a script that contained 
information designed to answer that question.

CSRs sometimes seemed to be accessing the wrong script to answer our 
question. For example, in answering the question on whether a 
beneficiary could receive a prescription drug discount card if she had 
a Medigap policy, one CSR incorrectly stated that the caller needed to 
complete a survey before receiving an answer. There is a script 
available that provides the answer to the Medigap question, but the 
script does not mention a survey. This CSR seemed to be using a 
different script about the prescription drug discount card, which has 
the right information to answer our question on the best prescription 
drug discount card to choose.

In some cases, CSRs did not obtain enough information from the scripts 
they were using to accurately answer the question we asked. For 
example, to answer our question concerning whether a beneficiary could 
qualify for the $600 credit toward prescription drug purchases, the CSR 
should consider the source, as well as the amount of the beneficiary's 
income. Some sources of income are not counted in determining a 
beneficiary's eligibility for the $600 credit. According to CMS, to 
answer this question accurately the CSR would have to access two 
different scripts. The first script provides general information about 
the $600 credit and refers CSRs to the second script, which lists the 
sources of income that are not included in the eligibility calculation. 
However, the CSRs who answered this question incorrectly in 55 calls--
or 79 percent of the time--focused on the total amount of income and 
did not seem to seem to consider the sources of the income or to access 
and use information from the second script. In 14 of the calls--or 20 
percent of the time--CSRs were able to answer this question correctly, 
because they did consider the sources and amounts of income that we 
indicated the beneficiary had.

Finally, CSRs sometimes misinterpreted or did not understand the words 
they were reading from the scripts or other written materials. For 
example, to answer our Medigap question, a CSR incorrectly told the 
caller that the beneficiary would automatically receive a prescription 
drug discount card if enrolled in a Medigap plan. The CSR may have been 
confusing Medigap policies with Medicare managed care plans, because 
both are discussed in the script that answered this question. In 
another example, for our question related to power wheelchair coverage, 
a CSR misread the requirement that a beneficiary must have adequate 
trunk--or upper body--strength. The CSR mistakenly informed us that a 
Medicare beneficiary needs to have adequate "trunk space" in order to 
qualify for a power wheelchair. When we asked for clarification, the 
CSR stated that Medicare requires a qualifying beneficiary to have 
adequate trunk space in his or her car to hold a power wheelchair. 
Similarly, during one of our calls about eye exam and glasses payment, 
the CSR informed us that an eye exam would be covered and then stated, 
"the only part of the exam that is not covered is 'refraction,' but I 
don't know exactly what that is." Because the CSR did not understand 
that a typical eye exam would be considered a refraction, she gave the 
caller the incorrect impression that Medicare would pay for a routine 
eye exam.

CMS and contractor staff acknowledged that scripts for the 1-800-
MEDICARE help line are not routinely pretested to ensure that both the 
CSR and the caller can understand the script before it is used to 
answer callers' questions.[Footnote 13] On occasion, the 1-800-MEDICARE 
contractor has obtained CSRs' feedback on draft scripts before they are 
used on the 1-800-MEDICARE help line to ensure that scripts can be 
easily read and understood. But this is not done as a routine step 
before new and revised scripts are used in handling calls. In addition, 
even if the CSRs consider the script understandable, it may still be 
confusing to Medicare beneficiaries. We found that pretesting to ensure 
that written material is understandable to its intended audience is a 
standard practice used to develop effective communications materials. 
For example, prior to issuing the first Medicare & You 
handbook[Footnote 14] nationwide to beneficiaries, CMS conducted 
consumer testing of its publication to evaluate its effectiveness as a 
communication tool. CMS has revised subsequent editions of the handbook 
to make it easier to read and use, based on feedback from 
beneficiaries. Moreover, other HHS agencies, such as the Centers for 
Disease Control and Prevention[Footnote 15] and the Substance Abuse and 
Mental Health Services Administration,[Footnote 16] have developed 
guidance on steps for ensuring that written material is understandable 
for intended readers and pretesting the materials before use.[Footnote 
17]

Ten Percent of Calls Were Not Answered, Often When Calls Were 
Transferred to Other Contractors' Help Lines:

We did not receive answers to our questions for 10 percent (42) of the 
420 calls we placed at the time we originally called. Several reasons 
accounted for this, as table 2 shows. For half (21) of the unanswered 
calls, the CSRs or the interactive voice response system transferred 
the calls placed during morning, evening, and weekend hours to claims 
administration contractors that were not open for business at the time 
of our call. Although the 1-800-MEDICARE help line is open 24 hours a 
day, 7 days a week, these other help lines are not. The transferred 
calls pertained to our questions concerning Medicare coverage about 
power wheelchairs and eye exams and glasses. The 1-800-MEDICARE CSRs or 
the interactive voice response system transfer such questions to the 
claims administration contractors' help lines because these contractors 
generally have greater knowledge about Medicare coverage 
issues.[Footnote 18] Once our calls were transferred to closed help 
lines, we generally heard recordings that stated the contractors' 
regular business hours and suggested calling back at that time. 
However, for 7 of those 21 calls, the contractors' recorded messages 
did not provide a telephone number to use to call back during the 
stated business hours.[Footnote 19]

Table 2: Summary of Unanswered Calls by Question:

Medicare prescription drug discount card questions: 

Question 1: Drug card; 
Call transferred to closed help line: 0; 
Call disconnected: 3; 
Web-based tool inoperative: 4; 
Call routed to a wrong telephone number: 0; 
Total unanswered calls: 7.

Question 2: $600 credit; 
Call transferred to closed help line: 0; 
Call disconnected: 1; 
Web-based tool inoperative: 0; 
Call routed to a wrong telephone number: 0; 
Total unanswered calls: 1.

Question 3: Medigap; 
Call transferred to closed help line: 0; 
Call disconnected: 2; 
Web-based tool inoperative: 0; 
Call routed to a wrong telephone number: 0; 
Total unanswered calls: 2.

Other Medicare coverage or eligibility questions: 

Question 4: Power wheelchair; 
Call transferred to closed help line: 10; 
Call disconnected: 5; 
Web-based tool inoperative: 0; 
Call routed to a wrong telephone number: 0; 
Total unanswered calls: 15.

Question 5: Medicare part B enrollment; 
Call transferred to closed help line: 0; 
Call disconnected: 0; 
Web-based tool inoperative: 0; 
Call routed to a wrong telephone number: 0; 
Total unanswered calls: 0.

Question 6: Eye exam/glasses; 
Call transferred to closed help line: 11; 
Call disconnected: 5; 
Web-based tool inoperative: 0; 
Call routed to a wrong telephone number: 1; 
Total unanswered calls: 17.

Total number of responses by category; 
Call transferred to closed help line: 21; 
Call disconnected: 16; 
Web-based tool inoperative: 4; 
Call routed to a wrong telephone number: 1; 
Total unanswered calls: 42. 

Source: GAO analysis.

[End of table]

The second most common reason we did not receive answers to our calls 
was that our calls were disconnected. Sixteen of the 42 unanswered 
calls were disconnected. For example, calls were disconnected before we 
were able to obtain a response to our questions. In one instance, the 
call was placed on hold for 30 minutes and then was disconnected. Four 
calls made on the same day did not receive a response because computer 
maintenance prevented the CSRs from accessing the Web-based tool they 
needed to use to answer our question about the prescription drug 
discount card. Finally, one other call was unanswered because the call 
was routed to a wrong telephone number.

CSR Training Met CMS's Requirements, But Was Not Sufficient to Ensure 
Accurate Responses:

As required by CMS, both newly hired and experienced CSRs receive 
training to help them answer questions posed on the 1-800-MEDICARE help 
line. The training for newly hired CSRs includes instruction on 
accessing and using scripts, customer service etiquette, and 
information about the Medicare program. As part of the training, CMS 
requires newly hired CSRs to score 90 percent or higher on a written 
exam before they handle calls on the help line. All CSRs also receive 
continuing training, and take written quizzes on the new material. 
Although the 1-800-MEDICARE contractors met CMS's training requirements 
by providing instruction and testing, the testing does not fully 
measure CSRs' ability to accurately answer real questions from callers.

The primary contractor develops and conducts the training new CSRs 
receive. Most of the training consists of 2 weeks of classroom 
instruction. In general, the instruction introduces CSRs to scripts and 
provides general information about the Medicare program. For example, 
in a training session we observed for newly hired CSRs in June 2004, 
the instructors helped the CSRs prepare for the types of inquiries that 
might be expected from callers on the 1-800-MEDICARE help line. The 
instructors posed different questions to the class, and each CSR 
attempted to identify and access a script with the right information to 
answer the instructor's question.[Footnote 20] One CSR would be 
selected to read the script that they chose, and participants discussed 
whether they thought this was the script with the best information to 
answer the question. After completing their initial instruction, CMS 
requires the new CSRs to score at least 90 percent on a written exam 
and successfully complete a call handling simulation exercise before 
they answer calls on the help line without supervision. To successfully 
complete the call handling simulation exercise, CSRs must accurately 
answer two consecutive simulated help line calls out of six possible 
attempts. In addition, new CSRs generally spend about 4 hours listening 
to calls answered by an experienced CSR.

In addition to the initial training for newly hired CSRs, CMS requires 
all CSRs to receive continuing training.[Footnote 21] Continuing 
training is delivered through three methods: refresher classes, online 
broadcast announcements, and small group meetings. Weekly refresher 
classes provide a means of instructing CSRs about Medicare program 
changes. Following each refresher training class, CSRs complete a short 
quiz to show that they understand the new information. While there is 
no minimum score that CSRs must achieve on the short quiz, CMS staff 
informed us that help line supervisors review each quiz to ensure that 
any questions that posed problems for CSRs would be addressed with 
further training. To provide CSRs with information quickly, the primary 
contractor sends online broadcast announcements to each CSR's computer 
workstation. These online announcements usually contain information 
that may affect CSRs' responses to help line questions, such as news 
about a change in a specific script. Lastly, small group meetings of 
about 12 CSRs and their supervisor are held for 30 minutes each week so 
that CSRs can discuss topics that can help them improve their call 
handling skills.

After gaining experience in answering calls, some CSRs receive 4 
additional days of special training and are promoted to a senior 
position. These CSRs receive classroom training on using Web-based 
computer programs that can assist Medicare beneficiaries in selecting a 
managed care plan, a nursing home, or other Medicare-related services. 
Like other CSRs, they must score 90 percent or higher on a written 
exam, and successfully complete a simulated call handling exercise 
before they can handle calls using the Web-based computer programs. 
Currently, about 200 senior CSRs answer calls on the 1-800-MEDICARE 
help line.

Although all CSRs receive training and are tested as required, the 
responses we received indicate that not all CSRs had the necessary 
knowledge and skills to answer our questions accurately. In our 
opinion, testing how effectively CSRs use scripts to answer frequently 
asked questions provides the best measure of their preparation to do 
so. While 24 of the exam's 52 questions ask CSRs to identify scripts 
that could be used to answer specific inquiries, the remaining 28 
questions target other skills.[Footnote 22] In addition to the written 
test, new CSRs must appropriately answer questions posed in two 
consecutive simulated calls before they staff the help line.[Footnote 
23] This simulated call handling and some of the written exam questions 
are the only measures of the CSRs' ability to accurately answer calls 
using scripts. In combination, the test and the two simulated calls do 
not appear to be a sufficient measure of new CSRs' ability to 
accurately answer the most frequently asked questions, given our 
findings on the accuracy of their responses. Further, while all CSRs 
receive continuing training, they are not required to demonstrate that 
they have effectively mastered the new material in handling calls.

Developing a more targeted assessment of where CSRs need to augment 
their skills helped focus another help line's training efforts and 
allowed it to meet its accuracy goals. In 2001, we assessed the 
telephone help line maintained by the IRS to answer taxpayers' 
questions and found that it had not met the agency's goals for 
accurately answering general questions about tax law and specific 
questions about individuals' tax returns in 2001.[Footnote 24] In 
response, the IRS analyzed the specific types of inquiries within the 
area of tax law and individual returns that were answered inaccurately 
and identified the knowledge and skills its CSRs needed to answer 
questions more accurately. The IRS also identified the CSRs most in 
need of training to improve accuracy in those knowledge and skill areas 
and provided additional training to them before call volume increased 
for the 2002 tax season. By the end of the training period, these CSRs 
were required to be certified by their managers as capable of providing 
correct responses to taxpayer questions. The IRS also assigned 
responsibility for selected tax law topics to individual call center 
managers, making them accountable for ensuring that CSRs were trained 
and could accurately address inquiries on these topics. After these 
initiatives were complete, we found that the help line had improved its 
accuracy enough to meet its 2001 goals. In the span of 1 year, the 
accuracy rate on answering tax law questions increased from 79 to 85 
percent and the accuracy rate for answering questions about 
individuals' tax returns increased from 88 to 91 percent.[Footnote 25]

Monitoring Generally Focused on Performance of Individual CSRs, Rather 
than Accuracy Rates for Specific Questions:

CMS monitors the 1-800-MEDICARE help line mostly by requiring its 
primary contractor to evaluate four individual conversations that each 
CSR has with callers each month. Based on these conversations, the 
primary contractor evaluates the performance of individual CSRs in 
several categories, including accuracy, and reports the overall results 
to CMS. CMS also occasionally directly monitors a small number of 
individual CSRs' calls. However, the contractor's and CMS's monitoring 
does not systematically track the accuracy rates for commonly asked 
questions. As a result, the monitoring does not assess how accurately 
CSRs as a group answer particular questions, which could help CMS 
target additional training efforts. Two smaller evaluation efforts did 
focus on specific questions answered inaccurately, and these targeted 
monitoring efforts provided information that CMS used to improve CSR 
training and the scripts used on the help line.

CMS's Primary Contractor Monitors Performance of Individual CSRs:

At the time of our review, CMS had delegated most of the responsibility 
for monitoring the accuracy of the 1-800-MEDICARE help line to the 
primary contractor, while maintaining oversight by reviewing the 
primary contractor's results. To monitor 1-800-MEDICARE, the primary 
contractor focuses on the performance of individual CSRs, evaluating 
four calls per month for each person. The primary contractor evaluates 
either live conversations--known as blind monitoring--or recorded 
conversations on the help line, while viewing displays of the CSRs' 
computer activity during calls. Viewing the CSRs' computer activity 
allows the primary contractor's staff to observe the scripts or other 
materials that CSRs access to answer callers' questions. After 
monitoring a call, the primary contractor's supervisory staff uses a 
checklist to evaluate the CSR's response to the caller. Help line 
supervisors share the results with each CSR to help improve 
performance. The primary contractor provides monthly reports to CMS on 
the results of its monitoring.[Footnote 26] The primary contractor has 
a subcontractor, which is responsible for conducting some independent 
call monitoring, as well as reviewing the results of some of the 
primary contractor's call monitoring. In addition to the four calls per 
CSR per month that CMS requires the primary contractor to monitor, the 
subcontractor monitors up to one additional call per month per CSR. The 
subcontractor reports its monitoring results monthly to CMS and the 
primary contractor.

In addition to meeting CMS requirements, the amount of call monitoring 
per CSR approximates industry standards. A survey of 735 North American 
call centers that represent help lines in various industries, including 
telecommunications, financial services, and health care, found a wide 
variance in the number of calls monitored per month. The most commonly 
reported monthly monitoring frequencies were 4 to 5 calls per CSR or 10 
or more calls per CSR.[Footnote 27]

The evaluation checklist used by CMS's contractor for monitoring calls 
indicates that a CSR's performance should include certain components--
such as using an appropriate greeting, showing respect to the caller, 
actively listening to the caller, responding accurately to the 
question, providing a complete response, using appropriate tone and 
speed, offering to provide additional information if necessary, and 
ending the call politely. The primary contractor's staff uses the 
checklist to evaluate both the customer service skills and knowledge 
skills demonstrated during a call and classify these into one of four 
categories--"unacceptable," "needs improvement," "achieves 
expectations," and "exceeds expectations."

CMS requires the primary contractor to reach a quality rating of 
"achieves expectations" or higher for at least 90 percent of the total 
number of CSR calls evaluated each month. The primary contractor 
evaluates a call as either "accurate" or "inaccurate," and because 
accuracy is weighted more heavily than other components, a CSR cannot 
provide inaccurate information on a call and still have the call scored 
as "achieves expectations." However, in contrast to our methodology for 
this report, a CSR can provide incomplete information--information that 
is correct but does not fully answer the question--and still have the 
call scored as "achieves expectations." In addition, the evaluation 
checklist does not indicate the specific criteria used to determine a 
call's accuracy.

Although CMS's main role in monitoring the help line is to review the 
efforts of the primary contractor, the agency also conducts some 
monitoring of CSRs on its own. Like the primary contractor, CMS 
occasionally uses blind monitoring to evaluate the performance of 
individual CSRs--listening to real-time calls and watching the scripts 
and other materials the CSRs use. CMS does not conduct blind monitoring 
routinely and document the results, and therefore, CMS staff could not 
provide us any information on the extent of this monitoring.[Footnote 
28] According to a CMS official, the agency conducts blind monitoring 
on a "limited and as-needed" basis.

CMS Has Tracked Accuracy Rates by Question on Occasion:

Although CMS's primary 1-800-MEDICARE contractor monitors the accuracy 
of individual CSRs, CMS does not use the regular monthly monitoring to 
identify trends in inaccurate responses by question. Specifically, the 
primary contractor does not routinely classify or categorize CSRs' 
answers by specific question to identify the questions that 
collectively were answered less accurately. While routine information 
about a question's accuracy rate could be used to target improvement 
efforts, CMS has only taken this approach twice in recent years. Both 
of these efforts were small compared to the primary contractor's 
monitoring. The larger effort to monitor accuracy by question lasted 29 
months and involved 300 calls a month, whereas the primary contractor 
evaluated about 7,350 calls in July 2004.[Footnote 29]

CMS contracted for a study to evaluate the 1-800-MEDICARE help line's 
accuracy in answering specific questions, but did not receive results 
quickly enough to immediately address problems. Beginning in January 
2002 and until May 2004, the CMS contractor hired to assess the 
"Medicare & You" program[Footnote 30] placed about 300 calls per month 
to the 1-800-MEDICARE help line. These callers used a set of 
hypothetical scenarios to assess how specific questions were answered. 
This study also established criteria specifying the information an 
accurate answer should provide[Footnote 31] and made a distinction 
between fully responsive answers--in other words, complete and accurate 
answers--or partially responsive answers--not complete but providing 
some accurate information.[Footnote 32] For the first 19 months 
studied, the average percentage of calls that received fully responsive 
answers ranged from under 40 percent to over 90 percent, depending on 
the question and the period of time studied.

The study helped CMS identify questions that the CSRs were answering 
less accurately. However, CMS staff told us that the agency received 
the reports 4 to 5 months after monitoring occurred, which did not 
allow CMS to immediately address any identified problems. Nevertheless, 
CMS staff indicated to us that the results of these evaluations were 
used to identify areas where CSR refresher training was needed. Due to 
funding constraints, this project was terminated in May 2004. CMS told 
us it planned to resume a similar project in November 2004 around the 
time when the next cycle for the "Medicare & You" program contract 
begins.

In another study, CMS measured 1-800-MEDICARE's accuracy by question 
rather than individual CSR and found that certain questions were 
answered more accurately than others. In April 2004, a private 
consultant group that was under contract with CMS placed 49 calls to 1-
800-MEDICARE to determine whether CSRs were relaying accurate 
information about Medicare's new prescription drug discount card and 
benefit.[Footnote 33] The study established specific criteria on the 
information that CSRs should include for an answer to be 
accurate.[Footnote 34] Evaluating accuracy by question, the study found 
that CSRs accurately answered between 39 percent and 69 percent of the 
questions asked about the new Medicare prescription drug discount card 
and benefit. In contrast, in that same month the subcontractor--using 
its evaluation checklist--determined that the overall accuracy rate for 
the calls it measured on all topics to be about 94 percent.

As a result of this study, CMS improved its scripts and related 
training. The private consultant's report indicated that CSRs were 
having difficulty distinguishing between the Medicare prescription drug 
coverage benefit that will be in effect in 2006 with the Medicare 
prescription drug discount card that is currently available. CMS 
responded by clarifying the scripts used to answer these questions and 
improving the related materials used to train CSRs. For example, CMS 
worked with the contractor to rename the titles of the different 
scripts to include the term "benefit" or "card" as a method of 
differentiating between them. When the subcontractor noted about 3 
months later that a few CSRs were continuing to confuse the 
prescription drug discount card and prescription drug benefit, CMS 
further clarified the scripts and its primary contractor conducted 
additional refresher training to attempt to correct the problem.

Conclusions:

Each year, millions of Medicare beneficiaries, their family members, 
and other callers rely on the 1-800-MEDICARE help line for information. 
Providing them with accurate answers is critical to keeping them 
informed about Medicare's benefits. However, we found that 6 out of 10 
calls were answered accurately, 3 out of 10 calls were answered 
inaccurately, and we were not able to get a response for 1 out of 10 
calls.

To answer inquiries accurately, CSRs have to be able to effectively 
access and use scripts. Given the lack of prior Medicare knowledge 
among CSRs, the 1-800-MEDICARE help line's script-based approach is a 
reasonable means to facilitate accurate and consistent responses to 
caller's questions. However, this approach makes CSRs--and thus the 
help line they support--dependent on the clarity and accuracy of the 
scripts available. Pretesting scripts might have identified ones that 
were difficult to understand by either the CSR or by potential callers, 
but this is not routinely done. Further, the training that CSRs receive 
on using scripts is also essential to their ability to answer questions 
accurately. However, the written exam that newly hired CSRs must pass 
and the continuing training quizzes do not measure the ability to use 
information in scripts to provide accurate and complete answers on the 
help line.

Monitoring the help line could identify areas where CSRs' knowledge and 
skills are lacking. Although CMS ensures that the amount of the 
contractor's monitoring per CSR falls within industry standards, the 
bulk of the monitoring methods are not designed to systematically 
assess how accurately CSRs as a group answer particular questions. 
Evaluating how accurately particular questions are addressed is an 
important step to improving scripts and CSR training for those topics.

Finally, 1-800-MEDICARE is advertised as providing information 24 hours 
a day, 7 days a week, but we could not always obtain answers to our 
questions when we called. When we called with questions about Medicare 
payment for power wheelchairs and coverage of eye exams and glasses, 
the help line frequently transferred our calls to claims administration 
contractors that were closed at the time. For a third of these 
transferred calls, we were not given a call-back number. This practice 
of transferring calls to claims administration contractors that are 
closed, in effect, reduces the benefit of a 24-hour help line to a 
business-hour help line for many beneficiaries.

Recommendations:

In order to improve the accuracy of responses made on the 1-800-
MEDICARE help line and callers' ability to have their questions 
answered, we recommend that the CMS Administrator take four actions:

* Assess the current scripts for the most commonly asked questions to 
ensure that they are understandable to CSRs and potential callers and 
if not, revise them as needed and pretest new and revised scripts to 
ensure that CSRs can effectively use them to accurately answer callers' 
questions.

* Enhance testing of CSRs' skills in accurately answering the most 
commonly asked questions using scripts and, if needed, provide 
additional training to improve the accuracy and completeness of their 
responses.

* Supplement current monitoring efforts to include a systematic review 
of the accuracy of information provided by the CSRs as a group for the 
most frequently asked questions and use the results to modify scripts 
or provide more training, as needed.

* Revise routing procedures and technology to ensure that calls are not 
transferred or referred to claims administration contractors during 
nonbusiness hours.

Agency Comments and Our Evaluation:

In its written comments on a draft of this report, which are reprinted 
in appendix VIII, CMS agreed with our recommendations and stated that 
it had begun several efforts to address them. CMS also provided more 
detail on the challenges it faced in administering 1-800-MEDICARE due 
to the massive increase in call volume that occurred after the passage 
of the MMA.

CMS agreed with our recommendation to assess current scripts and 
pretest new and revised scripts to ensure that they are understandable. 
In its comments, CMS stated that the written information used to 
develop 1-800-MEDICARE scripts often comes from Medicare publications 
that have been consumer tested as part of the publication preparation 
process. Language that has undergone some consumer testing is often 
incorporated into the scripts to improve clarity. While this step may 
be helpful, we believe that pretesting scripts verbally is also 
important, as consumer testing of material intended for written 
publication may not be adequate to determine whether the scripts are 
understandable to CSRs and the public. CMS also stated that it is 
considering implementing an editorial board to review scripts, which we 
believe would be another positive step to help assure the scripts' 
clarity.

CMS agreed with our recommendation to enhance testing of CSRs' ability 
to accurately answer questions and provide additional training, as 
needed. CMS indicated that it was reassessing its testing requirements 
to determine better ways to ensure that CSRs are prepared to handle 
calls, once they are certified. The agency stated that it planned to 
benchmark its efforts against industry standards to determine more 
effective approaches. In its comments, CMS expressed concern that we 
had characterized customer service skills as less meaningful than 
knowledge skills. While we believe both are important, in keeping with 
our congressional mandate, this report focused on the accuracy of 
information provided by 1-800-MEDICARE and did not address the quality 
of its CSRs' customer service skills.

CMS agreed with our recommendation to supplement its current monitoring 
efforts by including a systematic review of the accuracy of information 
provided by CSRs as a group for the most frequently asked questions and 
using the results to modify scripts and provide more training, as 
needed. CMS indicated that it believed it had done a good job 
developing a quality assurance program that focuses on the most 
important requirements for both accuracy and customer service skills 
needed to answer calls from the elderly population. We agree that CMS 
has focused on quality assurance for 1-800-MEDICARE. Our recommendation 
did not deal with changing, but with enhancing, its quality assurance 
efforts. To address our recommendation, CMS indicated that it would 
implement a plan to develop trend information on the results of its 
quality assurance activities and would focus on improving the accuracy 
of responses to frequently asked questions.

CMS agreed with our recommendation to revise procedures so that calls 
are not transferred to claims administration contractors during their 
nonbusiness hours. The agency indicated that although the 1-800-
MEDICARE CSRs are available 24 hours a day, 7 days a week, they do not 
have access to claim-specific information. Therefore, the 1-800-
MEDICARE CSRs would have to direct callers asking questions about 
specific claims to contact the claims administration contractors during 
their normal business hours. Our report focused on questions for which 
the CSRs had scripted responses and did not need to access claim-
specific information. Nevertheless, for 7 of the 21 calls that were 
routed to claims administration contractors that were closed at the 
time we called, the contractors' recorded messages did not provide a 
telephone number to call back during stated business hours. In its 
comments, CMS indicated that it had implemented additional routing 
plans to address this problem and is expanding access to claims data 
that will help reduce this problem in the future. CMS also raised 
concerns that we did not release the detailed audit documentation on 
our test calls while our work was still in progress. GAO's policy does 
not allow us to provide audit documentation to an agency while work is 
ongoing. At the time of CMS's request, we described our policy and 
offered to verbally provide more detail on telephone disconnections, 
but CMS did not follow up with us to obtain this information. After our 
report is published, we will address this request.

Finally, CMS expressed concern that we did not describe the criteria we 
used to evaluate the accuracy of responses to our six questions and 
stated that incomplete answers should not be considered inaccurate 
responses. As noted in the draft report, table 1 lists the criteria 
that we established for each of our six questions. We developed these 
criteria so that we could objectively evaluate responses received from 
CSRs. For four of these questions (numbers 1, 2, 3, and 6), there was 
only one element in the correct response, so an incomplete response was 
not possible.[Footnote 35] For the remaining two questions, we 
considered the answer accurate if it included two elements. We believe 
that by not including both elements for each of these questions, 
callers would be left with a false impression, rather than with an 
accurate answer. For example, in evaluating the response to the 
question of whether Medicare would pay for a power wheelchair, we 
thought it was important for the caller to know that (1) the wheelchair 
needed to be prescribed by a physician and (2) the beneficiary would be 
responsible for a copayment. Because the copayment for a power 
wheelchair is at least $1,000, we believe that it would be misleading 
not to mention either a copayment or cost sharing when a caller asks 
whether Medicare pays for this item. Likewise, needing to have a 
physician prescribe the power wheelchair is a Medicare requirement, and 
we did not think a response could be accurate without mentioning it. 
For the question on Medicare part B enrollment, we thought that it was 
important to know that a beneficiary could wait to enroll but, once 
other health insurance coverage ended, had a limited time period to 
enroll in part B without incurring higher premiums. Without knowing 
both elements of this answer, beneficiaries would not have enough 
information to guide their decision on part B enrollment and, 
therefore, the answer provided would be misleading.

We are sending copies of this report to the Administrator of CMS, 
appropriate congressional committees, and other interested parties. We 
will also make copies available to others upon request. This report is 
also available at no charge on GAO's Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please call 
me on (312) 220-7600. An additional GAO contact and other staff who 
made contributions to this report are listed in appendix IX.

Signed by: 

Leslie G. Aronovitz: 
Director, Health Care--Program Administration and Integrity Issues:

[End of section]

Appendix I: Scope and Methodology:

To determine the accuracy of information provided, we placed a total of 
420 calls to the 1-800-MEDICARE help line. We made 420 calls in order 
to have a sample that was large enough to determine if differences in 
accuracy were significant. We selected six questions about Medicare--
three related to the Medicare prescription drug discount card and three 
related to Medicare coverage or eligibility for benefits.[Footnote 36] 
We asked each of the selected questions a total of 70 times. We 
randomly placed calls at different times of the day and different days 
of the week between July 8 and July 30, 2004, to match the daily and 
hourly pattern of calls reported by 1-800-MEDICARE in April 2004.

To select the 6 questions, we initially chose 20 questions that related 
to the 100 topics most frequently addressed by the 1-800-MEDICARE help 
line's CSRs in May 2004 and developed criteria for an accurate response 
from information on the Medicare Web site's frequently asked questions 
section.[Footnote 37] We then presented the 20 questions and answers to 
Centers for Medicare & Medicaid Services (CMS) officials, who provided 
us with a script number and text for each question. CMS officials did 
not object to using any of the 6 questions that we ultimately chose, or 
suggest that the answers that we had provided for these questions were 
incorrect. We informed both CMS and one of the 1-800-MEDICARE 
contractors that we would be placing these calls. However, we did not 
tell them which 6 of the 20 questions we selected, or the specific 
dates and times when we would be placing our calls.

Before placing our calls, we created a scenario with fictional names 
and zip codes for each of the six questions to make them sound more 
realistic. (Appendices II to VII contain the scenarios that we used.) 
We made pretest calls for each question before we finalized its 
wording. During our actual calls, the CSRs were not aware that their 
responses would be included in a research study. We recorded the length 
of, and routing process for, each call. We evaluated the accuracy of 
the responses by CSRs to the 420 calls we placed by whether key 
information was provided.

The results from our 420 calls are limited only to those calls and are 
not generalizable to the population of calls routinely made to call 
centers by beneficiaries or other callers. Although the six questions 
we posed were among topics most often accessed by CSRs, they do not 
encompass all of the questions callers might ask. In addition, we did 
not verify the reliability of CMS's monitoring data.

To examine the training provided to 1-800-MEDICARE CSRs, we interviewed 
officials representing CMS and the 1-800-MEDICARE contractor 
responsible for training CSRs. We reviewed the primary contractor's 
training requirements and the instructional materials that are used to 
educate new CSRs. We also observed a training session for new CSRs at a 
1-800-MEDICARE call center. In addition, we reviewed previous GAO 
reports on the operations of other help lines, including the training 
provided to the CSRs answering calls on the Internal Revenue Service's 
help line.

To evaluate CMS's role in overseeing the accuracy of information 
provided through the 1-800-MEDICARE help line, we interviewed officials 
from CMS and one of the two 1-800-MEDICARE contractors about their 
monitoring and oversight activities. For our first objective, we 
focused on the accuracy of information provided by the CSR regardless 
of the contractor who managed their work. For our other two objectives, 
we relied on information provided by one of the two contractors--which 
we refer to as the primary contractor. We also identified CMS 
requirements for call center operations and reviewed contractor reports 
to identify the types of problems encountered through the help line. We 
performed our work from May 2004 through December 2004 in accordance 
with generally accepted government auditing standards.

[End of section]

Appendix II: Prescription Drug Discount Card Questions and Information 
Used to Develop Accuracy Criteria:

For question 1, which is about choosing a prescription drug discount 
card, we used scenarios with different combinations of prescription 
drugs and one of four different locations, in order to ensure our 
anonymity. If a CSR named one of the two prescription drug discount 
cards with the lowest cost for the combination of prescription drugs in 
the scenario posed, we considered it to be a correct response to our 
question. To ensure that we obtained the correct answer for each 
question, we periodically checked the prescription drug prices using 
the prescription drug tool on the Medicare.gov site. This is the same 
tool CSRs used to answer our questions. The answers shown in this 
appendix were accurate as of July 15, 2004.

Question 1a posed to CSRs:

My father-in-law lives in Wayne, Pennsylvania, and wants to continue to 
shop at Yorke Apothecary (located at 110 S. Wayne Ave., Wayne, 
Pennsylvania, 19087). What drug card can he get that will cover all of 
his drugs at Yorke Apothecary, and cost the least amount?

He takes the following drugs:

Hydrochlorothiazide; 25MG; 30 TABS. 
Lipitor; 20MG; 30 TABS.
Warfarin sodium; 5MG; 30 TABS.

[End of table]

Other information to provide to the CSR if asked:

* He is single.

* He lives in Wayne, Pennsylvania, in Delaware County. His zip code is 
19087.

* He currently has fee-for-service Medicare with no other drug 
benefits.

* He does not use an American Indian Health pharmacy.

* He does not live in a long-term care facility.

* He has $20,000 in annual income and is not interested in any drug 
assistance programs, including the $600 credit.

* His sources of income are a pension and Social Security, but the 
amount from each is unknown.

* He has some bank accounts, but their value is unknown.

* The amount he currently pays for drugs is unknown.

* Default answer for other questions: "I don't know."

Two prescription drug discount cards listed on Medicare.gov with the 
lowest prices for the combination of drugs in our scenario:

myPharmaCare Prescription Drug Discount Card; 
1-800-601-3002; 
Monthly drug costs: $116.69; 
Annual enrollment fee: $25.00.

U Share Prescription Drug Discount Card; 
1-800-707-3914; 
Monthly drug costs: $115.59; 
Annual enrollment fee: $19.95.

Question 1b posed to CSRs:

My father lives in Homewood, Illinois, and wants to continue to shop at 
the K-Mart Pharmacy in Homewood, Illinois (located at 17550 Halsted 
Rd., Homewood, Illinois, 60430). What drug card can he get that will 
cover all of his drugs at the K-Mart Pharmacy, and cost the least 
amount?

He takes the following drugs:

Aricept; 10 MG; 30 TABS; 
Celebrex; 100 MG; 30 TABS.

[End of table]

Other information to provide to the CSR if asked:

* He is single.

* He lives in Homewood, Illinois, in Cook County. His zip code is 
60430.

* He currently has fee-for-service Medicare with no other drug 
benefits.

* He does not use an American Indian Health pharmacy.

* He does not live in a long-term care facility.

* He has $20,000 in annual income and is not interested in any drug 
assistance programs, including the $600 credit.

* His sources of income are a pension and Social Security, but the 
amount from each is unknown.

* He has some bank accounts, but their value is unknown.

* The amount he currently pays for drugs is unknown.

* Default answer for other questions: "I don't know."

Two prescription drug discount cards listed on Medicare.gov with the 
lowest prices for the combination of drugs in our scenario:

U Share Prescription Drug Discount Card; 
1-800-707-3914; 
Monthly drug costs: $174.91; 
Annual enrollment fee: $19.95:

Any of several prescription drug discount cards available with this 
combination of drugs priced at $182.80.[Footnote 38]

Question 1c posed to CSRs:

My father lives in Cincinnati, Ohio, and wants to continue to shop at 
the CVS Pharmacy (located at 3195 Linwood Ave., Cincinnati, Ohio). What 
drug card can he get that will cover all of his drugs at the CVS 
Pharmacy, and cost the least amount?

He takes the following drugs:

Lipitor; 10 MG; 30 TABS. 

Ambien; 5 MG; 30 TABS.

Lipitor: Vioxx; 25 MG; 30 TABS.

[End of table]

Other information to provide to the CSR if asked:

* He is single.

* He lives in Cincinnati, Ohio, in Hamilton County. His zip code is 
45226.

* He currently has fee-for-service Medicare with no other drug 
benefits.

* He does not use an American Indian Health pharmacy.

* He does not live in a long-term care facility.

* He has $20,000 in annual income and is not interested in any drug 
assistance programs, including the $600 credit.

* His sources of income are a pension and Social Security, but the 
amount from each is unknown.

* He has some bank accounts, but their value is unknown.

* The amount he currently pays for drugs is unknown.

* Default answer for other questions: "I don't know."

Two prescription drug discount cards listed on Medicare.gov with the 
lowest prices for the combination of drugs in our scenario:

myPharmaCare Prescription Drug Discount Card; 
1-800-601-3002; 
Monthly drug costs: $202.79; 
Annual enrollment fee: $25.00:

Anthem Prescription Drug Discount Card; 
1-800-730-2804; 
Monthly drug costs: $209.87; 
Annual enrollment fee: $14.95:

Question 1d posed to CSRs:

My father lives in Brooklyn, New York, and wants to continue to shop at 
the Neergaard Pharmacy (located at 454 Fifth Avenue, in Brooklyn, New 
York). What drug card can he get that will cover all of his drugs at 
the Neergaard Pharmacy, and cost the least amount?

He takes the following drugs:

Cardura: 2 MG; 30 TABS.

Hydrochlorothiazide; 25 MG; 30 TABS.

Lisinopril; 5 MG; 30 TABS.

[End of table]

Other information to provide to the CSR if asked:

* He is single.

* He lives in Brooklyn, New York, in Kings County. His zip code is 
11215.

* He currently has fee-for-service Medicare with no other drug 
benefits.

* He does not use an American Indian Health pharmacy.

* He does not live in a long-term care facility.

* He has $20,000 in annual income and is not interested in any drug 
assistance programs, including the $600 credit.

* His sources of income are a pension and Social Security, but the 
amount from each is unknown.

* He has some bank accounts, but their value is unknown.

* The amount he currently pays for drugs is unknown.

* Default answer for other questions: "I don't know."

Two prescription drug discount cards listed on Medicare.gov with the 
lowest prices for the combination of drugs in our scenario:

EnvisionRx Plus Prescription Drug Discount Card; 
1-866-250-2005; 
Monthly drug costs: $46.33; 
Annual enrollment fee: $30.00:

Any of several prescription drug discount cards available with this 
combination of drugs priced at $50.45.[Footnote 39]

[End of section]

Appendix III: $600 Credit Question and Information Used to Develop 
Accuracy Criteria:

Question 2 posed to CSRs:

I've heard about the $600 credit that can help pay for prescriptions 
and wanted to know if my mother was eligible for it. Could she qualify 
for the credit? I know she has three sources of income.

* She has about $765 per month from Social Security.

* She also gets $250 each month in rental income from the apartment in 
the downstairs part of her house. She has a tenant that pays rent to 
her.

* She's also getting a payout from my father's life insurance policy of 
$70 each month.

Other information to provide to the CSR if asked:

* She is single and lives alone.

* She only has fee-for-service Medicare as health insurance.

* She owns her house.

* She lives in Miami, Florida, 33129.

* Default answer for other questions: "I don't know."

Information from Medicare.gov that GAO used to develop accuracy 
criteria:

If your annual gross income is currently no more than $12,569 ($1,048 
per month) as a single person or no more than $16,862 ($1,406 per 
month) for a married couple, you might qualify for a $600 credit to 
help pay for your prescription drugs and Medicare may pay your annual 
enrollment fee. If you and your spouse both qualify for the credit, the 
credit will be put on each of your cards.

You cannot qualify for the credit if you already have outpatient 
prescription drug coverage from Medicaid, TRICARE for Life,[Footnote 
40] Federal Employees Health Benefit Plan, or other health coverage 
that includes outpatient prescription drugs such as an employer group 
health plan. Note: If your other coverage is provided through a 
Medicare Advantage plan or a Medigap plan, you may still qualify.

The following sources of income should be included when calculating 
your gross income for your $600 credit enrollment form:

* Employee compensation (salary, wages, tips, bonuses, awards, etc.)

* Unemployment compensation:

* Pensions and annuities:

* Social Security benefits (including Social Security Equivalent 
portion of Railroad Retirement):

* Railroad Retirement benefits:

* Veterans Affairs benefits:

* Military and government disability pensions - armed forces, Public 
Health Service, National Oceanic and Atmospheric Administration, 
Foreign Service (based on date pension began, combat-related pension, 
etc.)

* Individual Retirement Account distributions:

* Interest (savings accounts, checking accounts, etc.)

* Ordinary dividends (stocks, bonds, etc.)

* Refunds, credits, or offsets of state and local income taxes:

* Alimony received:

* Business income:

* Capital gains:

* Farm income:

* Rental real estate, royalties, partnerships, trusts, etc.

* Other gains (sale or exchange of business property):

* Other income (lottery winnings, awards, prizes, raffles, etc.)

The following sources of income should not be included when calculating 
your income for $600 credit enrollment form:

* Inheritances and gifts (taxed to estate or giver if not under limits 
for exemption):

* Interest on state and local government obligations (e.g., bonds):

* Workers compensation payments:

* Federal Employees Compensation Act payments:

* Supplemental Security Income benefits:

* Income from national senior service corps programs:

* Public welfare and other public assistance benefits:

* Proceeds from sale of a home:

* Lump sum life insurance benefits paid upon death of insured:

* Life insurance benefits paid in installments:

* Accelerated life insurance death benefit payments (e.g., viatical 
settlements, terminal illness, chronic illness):

* Medical Savings Accounts withdrawals for medical expenses:

* Payments from long-term care insurance policies (subject to 
limitation):

* Accident or health insurance policy benefits:

* Accident compensatory damages:

* Child support payments received:

* Most foster care provider payments received:

* Disaster Relief grants:

* Disability payments as the result of a terrorist attack:

[End of section]

Appendix IV: Medigap Question and Information Used to Develop Accuracy 
Criteria:

Question 3 posed to CSRs:

I'm calling with a question about my grandmother. She is 69 and she has 
Medicare, and she also has a Medigap policy. Could you please tell me 
if she can still get a Medicare-approved drug discount card?

Other information to provide to the CSR if asked:

* She is single and lives alone.

* She lives in Miami, Florida. I don't know the zip code off-hand.

* She is not in a long-term care facility.

* She is enrolled only in Medicare fee-for-service. She doesn't have a 
Medicare managed care plan.

* She is not enrolled in Medicaid.

* I don't think she's interested in the $600 credit right now; I was 
just wondering if she could get the prescription drug discount card.

* Default answer for other questions: "I don't know."

Information from Medicare.gov that GAO used to develop accuracy 
criteria:

Having a Medigap policy does not preclude a Medicare beneficiary from 
being eligible for a Medicare prescription drug discount card.

[End of section]

Appendix V: Power Wheelchair Question and Information Used to Develop 
Accuracy Criteria:

For question 4 about power wheelchairs, we provided the CSRs with one 
of four different city and state combinations, as shown below. The four 
city and state combinations were randomly assigned to different power 
wheelchair calls. We did this to ensure that if our call was 
transferred to one of the four claims administrator contractors that 
administer Medicare's durable medical equipment claims--including 
power wheelchair claims--we were not biasing our results toward any 
particular claims administrator.

Question 4 posed to CSRs:

My father is having trouble getting around. He has a hard time walking 
and doesn't have much upper body strength. Could you please tell me if 
Medicare will pay for a power wheelchair for him?

Other information to provide to the CSR if asked:

* He is enrolled in Medicare, both parts A and B.

* He lives in [select one, based on random assignment]:

* Philadelphia, Pennsylvania. His zip code is 19105.

* Detroit, Michigan. His zip code is 48209.

* Pensacola, Florida. His zip code is 32516.

* Scottsdale, Arizona. His zip code is 85262.

* His doctor has suggested he get a power wheelchair to improve his 
mobility.

* He doesn't have enough strength to use a manual wheelchair.

* He lives alone and is not married.

* Default answer for other questions: "I don't know."

Information from Medicare.gov that GAO used to develop accuracy 
criteria:

Power wheelchairs and/or scooters are covered if they are medically 
necessary based on Medicare's criteria for coverage. In order for 
Medicare to cover a power wheelchair/scooter, the beneficiary's doctor 
must provide a prescription or certificate of medical 
necessity[Footnote 41] that states that he needs it because of his 
medical condition.

If your father qualifies for coverage, Medicare will pay 80 percent of 
the Medicare-allowed amount.[Footnote 42],[Footnote 43]

[End of section]

Appendix VI: Medicare Part B Question and Information Used to Develop 
Accuracy Criteria:

Question 5 posed to CSRs:

Should my husband sign up for part B if I am still working and we have 
health insurance coverage from my employer?

Other information to provide to the CSR if asked:

* My husband is about to turn 65 next January.

* If asked whether working for a large or small employer: I work for 
the federal government.

* I have full medical coverage, including dental and vision.

* My husband is fully covered under my insurance plan.

* Neither of us is disabled.

* The city/zip code information that corresponds with the location of 
the caller.

* Default answer for other questions: "I don't know."

Information from Medicare.gov that GAO used to develop accuracy 
criteria:

Your husband might want to wait to sign up for part B, because he would 
have to pay the monthly part B premium and the benefits may be of 
limited value as long as the group health plan [spouse's insurance] is 
the primary payer. You could save on monthly premiums by waiting to 
sign up.

If your husband doesn't sign up for part B when first eligible because 
he has group health coverage through an employer, he can sign up for 
Medicare part B during a special enrollment period. This can be anytime 
he is still covered by the employer's group health plan or during the 8 
months following the month when either the coverage or the employment 
ends--whichever is first.

Most people who sign up for Medicare part B benefits during a special 
enrollment period do not pay higher premiums.

[End of section]

Appendix VII: Eye Glasses/Exam Question and Information Used to Develop 
Accuracy Criteria:

Question 6 posed to CSRs:

My mother is 66 and is enrolled in Medicare. She has been complaining 
lately that she is having trouble reading the paper and thinks she may 
need new eyeglasses. Will Medicare pay for an eye exam and a new pair 
of eyeglasses if her prescription has changed?

Other information to provide to the CSR if asked:

* The city/zip code information that corresponds with the location of 
the caller.

* She is not married.

* She is enrolled in Medicare fee-for-service only.

* I do not know the name of the county she lives in.

* Default answer for other questions: "I don't know."

Information from Medicare.gov that GAO used to develop accuracy 
criteria:

Medicare does not pay for routine eye exams, eyeglasses, or contact 
lenses. The beneficiary must pay 100 percent of these services.

[End of section]

Appendix VIII: Comments from the Department of Health & Human Services: 

DEPARTMENT OF HEALTH & HUMAN SERVICES: 
Centers for Medicare & Medicaid Services:
Administrator: 
Washington, DC 20201:

NOV 30 2004:

TO: Leslie G. Aronovitz:

Director, Health Care -Program Administration and Integrity Issues: 
Government Accountability Office:

FROM: Mark B. McClellan, M.D., Ph.D.: 
Administrator:

SUBJECT: Government Accountability Office (GAO) Draft Report: MEDICARE: 
Accuracy of Response from the 1-800-MEDICARE Help Line Should Be 
Improved, (GAO-05-130):

The Centers for Medicare & Medicaid Services (CMS) thanks GAO for it's 
analysis and recommendations to help improve 1-800-MEDICARE. We concur 
with the recommendations and have taken steps to implement further 
improvements. We appreciate the work GAO has done and hope that we will 
be able to benefit from more of their specific work products in 
developing our improvement strategies. In addition to the 
recommendations, we are asking for additional information from GAO (see 
Attachment A). We hope GAO will provide us with the information we have 
discussed with GAO regarding the findings on disconnected calls, so we 
can determine if these are, as we suspect, the result of telephone 
network problems.

While we concur fully with the goals specified by GAO, we also want to 
stress some basic points related to their work and findings. The 1-800-
MEDICARE has experienced astounding growth since it was first launched 
at the end of 1998. In 2000, the first full calendar year of nationwide 
operations, the call center received 3.6 million calls asking about 
Medicare benefits, and health plan information and coverage, handled by 
approximately 300 customer service representatives (CSRs). In four 
short years, CMS has expanded the service to be available 24 hours a 
day, 7 days a week, added 3,000 CSRs, provided additional training on 
such important topics as choosing a nursing home, long-tern care, and 
choosing a drug discount card. We were faced with an unprecedented 
volume of calls about a new part of the Medicare program that required 
new training efforts and many new CSRs, and we believe we responded as 
well as we reasonably could given the unique and demanding 
circumstances. During the month of May 2004 alone, 1-800-MEDICARE 
received 3.8 million calls as a result of the overwhelming interest in 
the Prescription Drug Discount Card. The Prescription Drug Discount 
Card and the subsequent drug benefit, which takes effect January 2006, 
are historic improvements to the Medicare program.

We are continually making improvements to the services provided through 
1-800-MEDICARE and were already implementing steps to make enhancements 
prior to the GAO study. The GAO findings complement the work we are 
doing to reduce the number of agent transfers and ensure that callers 
reach the CSR who can help them with their Medicare question. While 1-
800-MEDICARE is open 24 hours a day, 7 days a week, the claims 
contractors are not, and we have addressed the problem of calls being 
routed to offices that are closed. In addition, we are currently 
reassessing various elements of our testing requirements and training 
protocol.

We are concerned about GAO's criteria for designating answers as 
"inaccurate," particularly because they consider an "incomplete" answer 
to be inaccurate. We train our CSRs to be attentive to the caller's 
needs and tailor the response to the caller's specific inquiry, as well 
as, the caller's interest and understanding level.

We have structured our responses to make these key points, and we have 
provided further details regarding the GAO report below. It is 
important to provide some additional general background information 
that we believe is relevant to the study. This supplementary 
information provides valuable information that will help the readers of 
the report understand some of the broader circumstances that 
contributed to the findings. We then respond specifically to each of 
the four findings in the order they have been presented by the GAO. For 
ease of review, our response to each finding will address four distinct 
subject areas:

1. Our concurrence or non-concurrence with the finding;

2. Additional information that we believe is pertinent in understanding 
why the specific finding was identified;

3. Questions and/or concerns we have with the methodology used in the 
study including terminology issues, the need for additional 
information, etc.; and:

4. The steps we are taking to implement the recommendations made 
including timeframes where appropriate.

Background Information:

Based on our analysis of past call volumes and planned programmatic 
impacts to 1-800-MEDICARE, we anticipated and were staffing for an 
annual call volume of approximately seven million calls in Fiscal Year 
2004. The Medicare Prescription Drug, Improvement, and Modernization 
Act of 2003 (MMA) and the news surrounding it caused the call volume to 
increase exponentially. For the month of May 2004 alone, 1-800-MEDICARE 
received approximately 3.8 million calls. In this single 30-day period, 
our call volume exceeded 50 percent of the call volume estimated for an 
entire year and it was not even during our regularly busy Fall ad 
campaign. We were not prepared to handle the extreme increase in calls 
received as a result of the legislation. To further illustrate the 
sheer explosion in call volume we experienced. we were notified in 
early May by MCI, our telecommunications provider, that they had to 
suspend delivery of calls because of concerns that the 1-800-MEDICARE 
calls traffic would crash their entire voice network. To meet the 
unprecedented high call volume, CMS rapidly worked to increase 1-800-
MEDICARE capacity (both staffing and network infrastructure) and 
improve the level of service provided to callers.

In response to this urgent and compelling need, we prepared a scope of 
work, negotiated a contract, hired, trained, and installed the 
necessary infrastructure to immediately add 1,000 additional CSRs to
the I-800-MEDICARE team. This was all accomplished in approximately 15 
business days. In addition, we aggressively worked with the existing 1-
800-MEDICARE contractor (Pearson GS) to recruit, hire, train, and place 
on the telephones additional CSRs and network routing improvements to 
ensure we were doing everything possible to meet the needs of our 
callers. To enable us to have more CSRs on the telephones faster, CMS 
implemented an expedited training program that focused on the use of 
the Prescription Drug Assistance Program (PDAP) tool, and responded to 
the top drug card questions with specific CSR groups. Callers with 
questions that went 'beyond the scope of these basic topics were 
transferred to a more experienced CSR for further assistance. This 
effort immediately raised our CSR workforce by 30 percent to 3,000 
CSRs, which is now our baseline staffing level for ongoing operations.

The purpose of this background section is to set the context of the 
call center environment in which GAO was tasked to conduct a study. It 
is not our intention to justify less than top quality service with 
circumstances that were outside of our control. It is our goal to help 
readers understand that we were doing everything possible to react to 
unprecedented call volume and some of the decisions we were making in 
the areas of staffing; training, content review, and network 
enhancements were made while considering many factors, the first of 
which has always been how we can get the best possible service to our 
callers under very difficult circumstances.

GAO Recommendation:

Revise routing procedures and technology to ensure that calls are not 
transferred or referred to claims administration contractors during 
non-business hours.

CMS Response:

We concur with the recommendation.

Relevant Information:

Enactment of the MMA in December 2003, resulted in the immediate 
legislative requirement to consolidate each of the 97 individual 
contractor telephone numbers into a single number (1-800-MEDICARE). It 
was always our goal as part of the Virtual Call Center Strategy (VCS) 
to rebuild the beneficiary telephone network to allow for a single 
entry point that callers would use to contact Medicare. The legislation 
that required the single 1-800 number was effective upon enactment. fhe 
initial planning estimates for consolidating to a single 1-800 number 
indicated that it would take between 12 to 18 months to consolidate the 
network. To respond to the legislative requirements, we rebuilt the 
entire network and routing plans so that a single point of entry could 
be routed to one of 97 different lines of business. We built and 
implemented the first Medicare speech automated system to help callers 
more easily access the desired service. We developed an extremely 
complex solution behind the scenes ensuring that each caller would, 
after answering a few simple questions, be routed to the contractor 
best suited to answer the caller's specific question. Due to the 
compressed timeframes, we were not able to implement all aspects of the 
original plan. As a result, some calls were routed to contractor 
locations after their normal hours of operation. In this situation, a 
message was played indicating that the contractor was closed for the 
day.

Current Approach:

We have since implemented additional routing plans that ensure callers 
are not transferred to a site that is closed, but it is important to 
note that while the base 1-800-MEDICARE CSRs are available 24 hours a 
day, 7 days a week, the claims processing contractor's CSRs are only 
available during normal business hours. With this in mind, it should be 
understood that while we have amended the routing plans to ensure that 
callers are sent to a general 1-800-MEDICARE CSR in situations where 
the claims processing contractor is closed, those CSRs do not have 
access to the claim detail that exists in the contractor's systems. The 
result is that while we will be answering calls with a "live" person in 
all instances, there will be times when we are not able to provide the 
information a caller needs, depending on the hour of the day. We are 
expanding the availability of access to claims data that will help us 
reduce this problem in the future.

Concerns:

It was never our goal to implement the specific routing solution that 
was in place during the study as a long-term application. We understand 
the impact this process has on customer service, and we would never be 
satisfied with that level of service on a permanent basis. However, we 
found it necessary to make short-term adjustments to the plans in order 
to comply with legislative requirements. Given the scope of work 
associated with consolidating the vast network of phone numbers and 
associated applications into a single phone number, speech routing 
application, and the timeframes associated with the legislative 
requirement, it was not technically possible to develop and implement a 
solution that would address all of the requirements to complete the 
work within the required mandate timeframe. Since implementation, we 
have improved the functionality of the application and the specific 
accomplishments we have made are outlined below under "Action Plan."

More importantly, it was not our intent to implement a system where 
claims processing contractors would be available 24 hours a day, 7 days 
a week concurrent, with the implementation of a single I-800 number 
network. We believe that the calls identified as receiving "No Answers" 
in the report would more appropriately be referred to as "Calls routed 
to closed locations."

The report indicates that there were some portions of this subset of 
calls that were "inadvertently disconnected." We find this portion of 
the report particularly troublesome, and it is one of the reasons why 
we immediately contacted GAO to request more detailed information on 
this issue. It is disheartening for CMS to learn that GAO is not 
willing to immediately release detailed information to CMS that would 
aid in our troubleshooting in what appears to be a network problem 
during an ongoing audit. We believe you understand from this and past 
audits that telephone networks are vastly complex. There are literally 
thousands of places where a problem with a single line of code or 
hardware can result in some sort of service degradation. It appears, 
based on the statements made in the report, that some portions of the 
calls are being affected by a network issue that we have not yet 
identified. As we explained to the GAO representative immediately upon 
receiving the draft report, specific call data from the audit would 
save extensive CMS staff resources and valuable time locating the 
problem independent of having your data. More importantly, it would 
allow CMS to respond more quickly with a solution, thus, reducing the 
number of callers who will now continue to have some percentage of 
calls "inadvertently disconnected." It is our sincere request that you 
provide us with the crucial information upon final completion of your 
report.

Action Plan:

We have deployed an Advanced Routing Database in the speech application 
that reviews the hours of operations for each of the 47 claims call 
centers that are now connected to 1-800-MEDICARE. The database checks 
to make sure that the call center is open before calls are routed to 
one of the claims call centers. If the call center is closed for any 
reason, the call will be routed to one of the general Medicare CSRs, 
and the CSR will try to assist the caller with general information or 
instruct the caller to call back during business hours when the claims 
system is available.

We have developed new information tools for the general Medicare CSRs 
to use which makes it easy for them to find the correct Medicare claims 
center for their caller and its correct hours of operation, prior to 
transferring the call.

We have implemented weekly business and technical process meetings with 
each of the call centers (both general and claims) to improve process 
and technologies. As a part of these meetings, the need for all CSR 
refresher training is identified and scheduled. We have built failsafe 
software into the transfer technology that will not allow the CSR to 
drop the caller into closed or invalid call center's queues. If the 
queue cannot accept the call when the CSR transfers it, the network 
forces the call back to the transferring CSR. This software helps to 
prevent CSRs from "dropping" calls and keeps the caller from getting 
lost in the transfer process.

GAO Recommendation:

Assess the current scripts for the most commonly asked questions to 
ensure that they are understandable to CSRs and potential callers and 
if not, revise them as needed and pretest new and revised scripts to 
ensure that CSRs can effectively use them to accurately answer callers' 
questions.

CMS Response:

We concur with the recommendation.

Relevant Information:

We are always interested in improving the way we develop and implement 
scripts in the Desktop application. However, it is inaccurate, as the 
report implies, to say that there is no testing of the scripts used by 
1-800-MEDICARE CSRs. We based the majority of our scripts on content 
that has been consumer tested in the development of Medicare 
publications. In addition, we hold regular focus groups to identify 
ways to explain concepts to beneficiaries in our static publications as 
well as our communication messages. We often incorporate that language 
into scripts; thus, there has been some degree of beneficiary testing.

It is also true that 1-800-MEDICARE is the front-line of our customer 
service process and, as such, it is the very first place people go when 
"hot topics" need to be addressed. Scripts are often developed to 
address urgent or emergent issues, and there are situations where 
sufficient time to fully pre-test/focus tests does not exist.

Concerns:

No concerns identified.

Action Plan:

We will continue our focused feedback sessions with CSRs that center on 
pre-testing scripts. CSRs participate in content workgroups to develop 
the MMA scripts and job aids.

We continue to hold weekly meetings with the call centers to discuss 
how to make the scripts more user-friendly, complete, understandable, 
and comprehensive.

Additionally, we have a desktop tool that allows CSRs to make 
recommended changes to scripts. This feedback is usually the result of 
caller feedback indicating the script is confusing, not helpful, etc. 
CSR feedback is compiled weekly and the content is shared with CMS 
staff. call center management, the learning and development team, as 
well as other 1-800-MEDICARE functional areas. CSR feedback is reviewed 
as part of the weekly meetings to determine call trends, the need for 
additional scripts or script modifications, identify training 
opportunities, and assist with overall continuous improvement at 1-800-
MEDICARE.

Based on feedback received, we are considering the implementation of an 
editorial board that would review words and phases to make 
recommendations such as changing "trunk strength" to "upper body 
strength."

GAO Recommendation:

Enhance testing of CSR's skills in accurately answering the most 
commonly asked questions using scripts and, if needed, provide 
additional training to improve the accuracy and completeness of their 
responses.

CMS Response:

We concur with the recommendation presented.

Relevant Information:

Addressed in Concerns section.

Concerns:

We believe the report reflects an incorrect understanding of the 
training requirements. The specific contractual requirement is for a 
CSR to "Achieve Expectations on or above two practice calls in a row." 
The contractor addresses this by using a series of six simulated calls, 
and the class trainers sit with the individual CSRs to determine, on a 
ease-by-case basis, their success at handling the simulated calls. If 
the CSR does not do well on the skill assessments on the simulated 
calls, whether they pass or fail, the trainers provide the CSR with 
additional coaching and mentoring. The intent is to provide the 
necessary training to ensure that the CSR is ready for real-time 
operations. In many of the cases where a CSR "fails" at a practice call 
scenario, it may be due to the CSR being nervous, and the extra 
tutoring helps to alleviate failure. This is not a situation where the 
trainers just run through the six simulated calls until the CSR gets 
two in a row correct, but is intended to provide the CSR with any 
additional training they need in areas of weakness.

In the case of customer service, we disagree with the recommendation 
that the questions targeted for the skills other than knowledge are not 
as meaningful. It is very important that the CSRs can handle themselves 
in all areas of customer service and even if they know how to get to 
the right script and provide the correct answer, they will not have 
credibility if they do not sound and act professionally.

Action Plan:

We are reassessing the testing requirements currently in place to 
determine ways to better ensure that the CSRs are prepared to handle 
calls at the point that they are certified. We are adjusting the 
training protocol to spread testing of the CSR's ability to handle 
calls over the length of the training period rather than just at the 
conclusion and to focus on categories of calls, especially on the most 
frequently asked questions. This approach allows more time for the 
trainers to coach and mentor the CSRs prior to the next phase of their 
job experience, which includes side-by-side work with an experienced 
CSR We will also benchmark against industry standards to determine more 
effective approaches, ensuring that the CSRs are prepared to respond to 
calls upon completion of the classroom training.

GAO Recommendation:

Supplement current monitoring efforts to include a systematic review of 
the accuracy of information provided by the CSRs as a group for the 
most frequently asked questions and use the results to modify scripts 
or provide more training, as needed.

CMS Response:

We concur with the recommendation to pursue trend analysis. However, we 
cannot concur with the comment on the observations made in this 
section, because we do not have access to the data GAO used to arrive 
at their findings. (Noted below under "Concerns"):

Relevant Information:

We agree that more analysis could be conducted to better identify 
global deficiencies across our CSR base. At the same time, we believe 
that we have done a very good job developing a quality assurance 
program that focuses on the important requirements for both accuracy 
and customer service skills needed to answer calls from an elderly 
population.

We require our contractor to monitor each CSR's call performance on a 
monthly basis using a national quality scorecard. The accuracy rate for 
the months July through October 2004 is as follows: July 89 percent; 
August 91 percent; September 90 percent; October 92 percent. Our 
national quality assurance scorecard measures the entire customer 
service experience across three broad categories-accuracy, 
completeness, and call action-and a subset of criteria within each of 
these categories. For example, accuracy is assessed on the basis of 
factors such as whether the CSR understood the question or whether they 
used the correct script(s).

In order to have an independent assessment of performance we conduct 
"mystery shopping" through a specialized quality evaluation contractor. 
A response is classified as "fully responsive" only if all key points 
are conveyed to the caller. For example, for the mystery shopping on 
prescription drugs, the calls scored 85 percent or above as "fully 
responsive."

In addition, we require our contractor to conduct 1,000 beneficiary 
satisfaction surveys each month to determine customer satisfaction with 
the services provided. For the month of July, 92 percent were satisfied 
with the service they received. This is consistent with other months. 
We also believe that we have done a very good job implementing 
technology to help support this effort and keep costs under control. We 
have not gotten to the point of regularly taking a more global review 
of the data we collect or looking at it from a higher perspective; 
(i.e., trends).

Concerns:

In the report, GAO identified 29 percent of the calls made to 1-800-
MEDICARE resulted in "inaccurate answers." The methodology used in 
identifying an answer as inaccurate was "if the CSR provided 
information on the topic but did not provide sufficient and complete 
information to meet our criteria, we considered the answer to be 
inaccurate." Consequently, it is difficult for:

IS to understand specifically what the specific criteria for a correct 
answer. As mentioned in he report, CMS provided GAO with a broad list 
of questions and various scripts that may have been used in answering 
the questions. It is important to note, it is not the policy of CMS to 
require all related scripts to be read to a caller, nor is it CMS 
policy to require the entire text of any single script to be read. CSRs 
are instructed to read only those portions of the scripts that pertain 
to the callers' specific inquiry. Furthermore, there are judgment 
decisions the CSRs must make with regard to how much information the 
caller really needs and their level of interest and understanding. In 
addition, CMS requires the CSRs to remain objective and to not provide 
opinions to the callers. Based on the information provided in the draft 
report it is not possible for CMS to identify what portion of the 
responses deemed "inaccurate" by GAO. We hope to receive call details 
and specific measurement criteria from the list of calls made to 
further investigate this issue. Additionally, we question the GAO's use 
of the word "inaccurate" and "incomplete" answers to classify a 
grouping of responses. We do not consider an "incomplete" answer to be 
an "inaccurate" answer. Again, we thank you for the opportunity to 
review and comment on this report.

Action Plan:

CMS has deployed trained staff to monitor CSR calls themselves and 
provide recommendations on changes and improvements for content. We 
have implemented ongoing analysis of trending reports from all 
monitoring sources to ensure that issues are identified and changes 
made as a result of the findings of these groups.

We appreciate the feedback provided and we will implement a plan that 
ensures that results of our Quality Assurance (QA) activities are 
trended and that we have a process in place that takes the results of 
that trending to the next step. Each month we will focus efforts on a 
specific Frequently Asked Questions activity based on QA findings from 
our other measures. This will enable us to better ensure that the CSRs 
handling these questions, which make up a large majority of the issues 
our callers want to discuss, can do so more accurately and effectively.

Attachment:

ATTACHMENT A:

The CMS is requesting additional information from the call sample that 
was conducted. This information will be extremely helpful to us in our 
efforts to troubleshoot and correct network problems and it will also 
support our efforts to improve the consistency and accuracy of the 
responses we provide to callers:

For each of the calls in the Study we are requesting the following 
information:

Date of call Time of call Number that the call was placed from:

The phone number of the switch (in the event calls were not made from 
individual's homes) The name that the tester used on the call:

The specific question asked How each call was scored The answer that 
was considered to be correct The answer that was received:

In addition, it would be very helpful if the following information 
could be provided separately for the subset of calls that were 
identified as "Inadvertently disconnected" in the report:

Date of call Time of call Number that the call was placed from:

The phone number of the switch (in the event calls were not made from 
individual's homes): 

[End of section]

Appendix IX: GAO Contact and Staff Acknowledgments:

GAO Contact:

Sheila K. Avruch, (202) 512-7277:

Acknowledgments:

Shaunessye D. Curry, Joy L. Kraybill, Krister P. Friday, Sari B. 
Shuman, Mary W. Reich, Ramsey L. Asaly, Alexis Chaudron, Perry G. 
Parsons, and Leslie Spangler made key contributions to this report.

FOOTNOTES

[1] These companies, known as Medicare claims administration 
contractors, maintain other toll-free help lines that are open during 
normal business hours. In June 2004, CMS began a process to discontinue 
listing these separate help line numbers and have 1-800-MEDICARE serve 
as the only number that callers would use to access information about 
Medicare eligibility, enrollment, benefits, coverage, and payment. As 
of September 30, 2004, this process was complete.

[2] Pub. L. No. 108-173, § 923(d)(2), 117 Stat. 2066, 2395.

[3] We made 420 calls in order to have a sample that was large enough 
to determine if differences in accuracy were significant.

[4] http://medicare.custhelp.com/cgi-bin/medicare.cfg/php/enduser/
std_alp.php.

[5] For our first objective, we focused on the accuracy of information 
provided by the CSRs, regardless of the contractor that managed their 
work. For our next two objectives, we relied on information provided by 
one of the two contractors--which we refer to as the primary 
contractor--because this contractor provided the training for all new 
CSRs and managed CSRs who answered 80 percent of the calls on the help 
line in July 2004. CMS does not use the term "primary contractor," 
because both companies contract directly with CMS. We are using the 
term primary contractor to distinguish the one that had a larger role 
during the time that we were conducting our calls. 

[6] GAO, IRS's 2002 Tax Filing Season: Returns and Refunds Processed 
Smoothly; Quality of Assistance Improved, GAO-03-314 (Washington, D.C.: 
Dec. 20, 2002); GAO, IRS Telephone Assistance: Limited Progress and 
Missed Opportunities to Analyze Performance in the 2001 Filing Season, 
GAO-02-212 (Washington, D.C.: Dec. 7, 2001).

[7] In May 2004, 1-800-MEDICARE received over 3 and a half million 
calls, which was more calls received than in any prior month. 
Throughout the summer of 2004, 1-800-MEDICARE received between 1 
million and 2 million calls each month.

[8] There are over 670 scripts for 1-800-MEDICARE in use.

[9] www.Medicare.gov.

[10] For some topics, such as the prescription drug discount card, 
there are several scripts that deal with the topic, but generally one 
script provides the best information to accurately answer a specific 
question, according to CMS. For each of our proposed questions, CMS 
provided the name and number of the script that it considered as having 
the best information for CSRs to use in formulating a response.

[11] This tool is also available on Medicare's Web site, at http://
www.medicare.gov/AssistancePrograms/home.asp.

[12] AARP is an association for individuals 50 years of age and older. 
It provides information, such as tips for healthy living and retirement 
planning; advocacy on issues affecting the elderly; and services, such 
as supplemental health insurance. 

[13] In its comments on a draft of this report, CMS informed us that 
the written information used to develop 1-800-MEDICARE scripts often 
comes from Medicare publications that have been consumer tested as part 
of the publication preparation process. Language that has undergone 
some consumer testing is often incorporated into the scripts to improve 
clarity. 

[14] CMS distributes a handbook called Medicare & You to all Medicare 
beneficiary households. The Medicare & You handbook describes the 
Medicare program and its benefits. 

[15] The Centers for Disease Control and Prevention is an agency of the 
U.S. Department of Health and Human Services that provides federal 
leadership to develop and apply disease prevention and control, 
environmental health, and health promotion and education activities 
designed to improve the health of people of the United States. 

[16] The Substance Abuse and Mental Health Services Administration is 
an agency of the U.S. Department of Health and Human Services 
established to focus attention, programs, and funding on improving the 
lives of people with, or at risk for, mental and substance abuse 
disorders.

[17] U.S. Department of Health and Human Services, Centers for Disease 
Control and Prevention, Scientific and Technical Information: Simply 
Put (Atlanta, Ga.: 1999) (at http://www.cdc.gov/publications.htm, 
downloaded Sept. 21, 2004), and U.S. Department of Health and Human 
Services, Substance Abuse and Mental Health Services Agency, Technical 
Assistance Bulletin: Pretesting is Essential; You Can Choose From 
Various Methods. (1994, http://www.health.org/_usercontrols/
printpage.aspx?from downloaded Sept. 21, 2004).

[18] Each of our six questions was posed during evening and weekend 
hours, as well as during normal business hours, using the same random 
pattern of calling during certain hours of each day over the course of 
3 weeks. However, because questions 4 and 6 were often transferred or 
referred more often to other contractors' help lines, these were the 
questions that were not answered when the contractors were closed for 
business. In contrast, the 1-800-MEDICARE help line did not routinely 
transfer questions 1, 2, 3, and 5 to other contractors. 

[19] For 14 transferred calls, our callers were provided with a 
telephone number and were able to complete the calls later. For 10 of 
these 14 calls completed later (71 percent), claims administration 
contractors' CSRs answered our questions accurately. 

[20] There can be more than one script on a particular topic, but often 
one of the scripts is the best one to answer a specific question about 
that topic.

[21] While the primary contractor provided all training for newly hired 
CSRs, both the primary and secondary contractors provided continuing 
training for their CSRs. 

[22] The test measures several types of information and skills that 
CSRs need. It consists of 52 questions: 14 questions on choosing a 
script that can be used to answer a question; 10 matching questions to 
choose definitions of Medicare terms; 10 true or false questions where 
CSRs were instructed to use a script to answer a question about the 
Medicare program; 6 multiple choice questions on call handling 
etiquette; and 12 matching questions to identify the organizations to 
which CSRs could refer certain questions--for example, the Social 
Security Administration for questions about enrolling in Medicare. The 
guide for scoring each question on scripts indicates several scripts 
that could be named as possible responses, including the script that 
would best address the question posed. If the CSR chooses any of the 
scripts that are listed as possible responses, the answer is scored 
correctly, even if the CSR does not choose the script with the best 
information to answer the question.

[23] The call handling exam for new CSRs consists of up to six 
questions. In order to successfully pass, the new CSR must adequately 
handle two of these calls consecutively. 

[24] GAO-02-212.

[25] GAO-03-314.

[26] The primary contractor also surveys callers to assess their 
satisfaction with the information they received from 1-800-MEDICARE. 
These customer satisfaction survey results are also reported to CMS in 
standard reports.

[27] Incoming Calls Management Institute, Call Center Monitoring Study 
II Final Report (Annapolis, Md.: 2002).

[28] This monitoring occurs in addition to the four calls per CSR per 
month that are monitored by the primary contractor, and the calls 
monitored by the subcontractor.

[29] In addition, the subcontractor monitored about 1,400 calls in July 
2004.

[30] The National Medicare Education Program publishes the Medicare & 
You handbook and maintains other activities to communicate with 
Medicare beneficiaries. The 1-800-MEDICARE help line is one of the 
activities in the program. 

[31] For example, for one question related to Medicare's coverage of a 
piece of durable medical equipment, the study defined a fully 
responsive answer as one that informed the caller that a prescription 
would be needed for the item, specified that the beneficiary needed a 
supplier who accepts Medicare, provided an overview of the Medicare 
billing process, shared information on how to contact the contractors 
that process durable medical equipment claims, and indicated that 
supplemental insurance might pay part of the costs.

[32] If CSRs provided partially correct but incomplete information that 
did not fully address our criteria, we scored the calls as 
"inaccurate." 

[33] This research was part of a CMS effort to determine whether CSRs 
were prepared to handle questions about program changes resulting from 
the MMA. 

[34] The April 2004 test calls used a scorecard that included up to 
nine different elements that measured accuracy, depending on the 
question the caller used. For example, the call could be scored on 
whether the CSR accurately informed the caller which Medicare 
beneficiaries would be eligible for a prescription drug discount card, 
how a beneficiary could apply for a prescription drug discount card, 
and when a beneficiary could start to use their prescription drug 
discount card. 

[35] For question 1, the name of either one of the two prescription 
drug plans with the lowest cost for the drugs was the correct answer. 
For questions 2 and 3, the correct answer was "yes," and for question 
6, the correct answer was "no."

[36] Except for a script used when transferring calls, each of the top 
15 scripts accessed in May 2004 by 1-800-MEDICARE CSRs was designed to 
answer various questions about the prescription drug discount card 
program. 

[37] http://medicare.custhelp.com/cgi-bin/medicare.cfg/php/enduser/
std_alp.php.

[38] If a CSR mentioned any prescription drug discount card that cost 
$182.80 per month, we considered the response to be accurate.

[39] If a CSR mentioned any prescription drug discount card that cost 
$50.45 per month, we considered the response to be accurate.

[40] TRICARE for Life provides secondary military health coverage 
available for Medicare-eligible uniformed services beneficiaries, 
their eligible family members, and survivors enrolled in Medicare part 
B.

[41] A certificate of medical necessity is a standardized form that 
physicians complete to indicate that an item is needed for medical 
reasons.

[42] The Medicare.gov Web site notes that Medicare will pay 80 percent 
after the $100.00 part B deductible is met, but we did not require CSRs 
to mention the deductible for our accuracy criteria.

[43] According to the HHS Office of the Inspector General, the median 
Medicare reimbursement for a purchased power wheelchair was $5,297 in 
2003.

GAO's Mission:

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony:

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading.

Order by Mail or Phone:

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548:

To order by Phone:

	

Voice: (202) 512-6000:

TDD: (202) 512-2537:

Fax: (202) 512-6061:

To Report Fraud, Waste, and Abuse in Federal Programs:

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470:

Public Affairs:

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: