This is the accessible text file for GAO report number GAO-03-809 
entitled 'Child Welfare: Most States Are Developing Statewide 
Information Systems, but the Reliability of Child Welfare Data Could Be 
Improved' which was released on August 12, 2003.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

On November 16, 2003, this document was revised to add various 
footnote references missing in the text of the body of the document.

Report to Congressional Requesters:

United States General Accounting Office:

GAO:

July 2003:

Child Welfare:

Most States Are Developing Statewide Information Systems, but the 
Reliability of Child Welfare Data Could Be Improved:

GAO-03-809:

GAO Highlights:

Highlights of GAO-03-809, a report to Congressional Requesters 

Why GAO Did This Study:

To better monitor children and families served by state child welfare 
agencies, Congress authorized matching funds for the development of 
statewide automated child welfare information systems (SACWIS) and 
required that the Department of Health and Human Services (HHS) 
compile information on the children served by state agencies.

This report reviews (1) states’ experiences in developing child 
welfare information systems and HHS’s role in assisting in their 
development, (2) factors that affect the reliability of data that 
states collect and report on children served by their child welfare 
agencies and HHS’s role in ensuring the reliability of those data, and 
(3) practices that child welfare agencies use to overcome challenges 
associated with SACWIS development and data reliability.

What GAO Found:

HHS reported that 47 states are developing or operating a SACWIS, but 
many continue to face challenges developing their systems. Most state 
officials said they recognize the benefit their state will achieve by 
developing SACWIS, such as contributing to the timeliness of child 
abuse and neglect investigations; however, despite the availability of 
federal funds since 1994, states reported a median delay of 2-½ years 
beyond the timeframes they set for completion. States reported that 
they encountered some difficulties during SACWIS development, such as 
challenges receiving state funding and creating a system that 
reflected their work processes. In response to some of these 
challenges, HHS has provided technical assistance to help states 
develop their systems and conducted on-site reviews of SACWIS to 
verify that the systems meet federal requirements.

Despite efforts to implement comprehensive information systems, 
several factors affect the states’ ability to collect and report 
reliable adoption, foster care, and child abuse and neglect data. 
States responding to GAO’s survey and officials in the 5 states GAO 
visited reported that insufficient caseworker training and inaccurate 
and incomplete data entry affect the quality of the data reported to 
HHS. In addition, states reported technical challenges reporting data. 
Despite HHS’s assistance, many states report ongoing challenges, such 
as the lack of clear and documented guidance on how to report child 
welfare data. In addition, although states were mandated to begin 
reporting data to the Adoption and Foster Care Analysis and Reporting 
System (AFCARS) in 1995, few reviews of states’ AFCARS reporting 
capabilities have been conducted to assist states in resolving some of 
their reporting challenges.

Some states are using a variety of practices to address the challenges 
associated with developing SACWIS and improving data reliability. For 
example, 44 states included caseworkers and other system users in the 
design and testing of SACWIS, and 28 states reported using approaches 
to help caseworkers identify and better understand the data elements 
that are required for federal reporting.

What GAO Recommends:

In order to improve the reliability of state-reported child welfare 
data, GAO recommends that the Secretary of HHS consider ways to 
enhance the guidance and assistance offered to states to help them 
overcome the key challenges in collecting and reporting child welfare 
data. In commenting on this report, HHS generally agreed with GAO’s 
findings and commented that the report provides a useful perspective 
of the problems states face in collecting data and of HHS’s effort to 
provide ongoing technical assistance to improve child welfare data.

www.gao.gov/cgi-bin/getrpt?GAO-03-809.

To view the full product, including the scope and methodology, click 
on the link above. For more information, contact Cornelia M. Ashby at 
(202) 512-8403 or AshbyC@gao.gov.

[End of section]

Contents:

Letter:

Results in Brief:

Background:

Most States Are Developing SACWIS, but Challenges Remain Despite HHS's 
Oversight and Technical Assistance:

Several Factors Affect the States' Ability to Ensure Reliable Data on 
Children's Experiences, and Some of HHS's Oversight and Assistance Is 
Problematic:

States Are Using Various Practices to Overcome System Development 
Challenges and Improve Data on Children's Experiences:

Conclusion:

Recommendation to the Secretary of Health and Human Services:

Agency Comments:

Appendix I: Scope and Methodology:

Appendix II: State SACWIS Stages of Development:

Appendix III: Comments from the Department of Health and Human 
Services:

Appendix IV: GAO Contacts and Acknowledgments:

GAO Contacts:

Staff Acknowledgments:

Bibliography:

Related GAO Products:

Tables:

Table 1: Significant Child Welfare Information and Data Systems:

Table 2: Number of States in Various Stages of SACWIS Development:

Table 3: Selected SACWIS Child Welfare and Administrative Services:

Table 4: Commonly Used SACWIS Development Participants and Their Level 
of Helpfulness:

Table 5: State-Reported Benefits of SACWIS Development:

Table 6: Number of Months States Delayed in SACWIS Development:

Table 7: Common SACWIS Development Challenges:

Table 8: Selected AFCARS Elements and Six States' Levels of Compliance 
in Meeting Reporting Requirements:

Figures:

Figure 1: Most Common Caseworker Issues That Affect Data Quality:

Figure 2: Federal Practices That Affect Data Quality:

Abbreviations:

ACF: Administration for Children and Families: 

ADP: advance planning document: 

AFCARS: Adoption and Foster Care Analysis and Reporting System: 

CAPTA: Child Abuse Prevention and Treatment Act: 

CRSR: Child and Family Services Reviews: 

FFP: federal financial participation: 

HHS: Department of Health and Human Services: 

IT: information technology: 

NCANDS: National Child Abuse and Neglect Data System: 

NYTD: National Youth in Transition Database: 

SACWIS: Statewide Automated Child Welfare Information System: 

VCIS: Voluntary Cooperative Information System:

United States General Accounting Office:

Washington, DC 20548:

July 31, 2003:

The Honorable Charles E. Grassley 
Chairman, Committee on Finance United States Senate:

The Honorable Tom DeLay 
Majority Leader 
House of Representatives:

Recent news reports of tragedies involving child welfare agencies 
across the country highlight the long-standing problems states have had 
monitoring children in their care. Given that hundreds of thousands of 
children are found to be victims of abuse and neglect and are estimated 
to spend some time in foster care each year, the Congress required that 
the Department of Health and Human Services (HHS) compile information 
on the children served by state agencies and authorized federal funds 
to match those of states for use in the development of state child 
welfare information systems. Since 1994, designated federal matching 
funds have been available to states to develop and implement 
comprehensive case management systems--Statewide Automated Child 
Welfare Information Systems (SACWIS)--to manage their child welfare 
cases as well as to report child abuse and neglect, foster care, and 
adoption information to the federal government. States have the option 
to implement a SACWIS or develop different information systems without 
using SACWIS funds to support their child welfare agencies and collect 
information on their child welfare cases. Regardless of the type of 
system a state develops, child welfare caseworkers at the county or 
local level are the key personnel who collect and document information 
on children and families served by child welfare agencies, in addition 
to performing a wide range of services to protect children--such as 
investigating child abuse or neglect reports or providing support 
services to maintain the children in their homes. Currently, HHS 
compiles state-reported child welfare data in two databases: the 
Adoption and Foster Care Analysis and Reporting System (AFCARS) and the 
National Child Abuse and Neglect Data System (NCANDS). HHS relies on 
the information available in its databases to analyze and track 
children's experiences in the child welfare system, to determine 
states' performance on federal child welfare outcome measures, and to 
report to Congress on children's well being and child welfare 
experiences.

Because of your interest in how states have developed systems to 
collect and report information on the children they serve and the 
reliability of that information, you asked us to determine (1) states' 
experiences in developing child welfare information systems and HHS's 
role in assisting in their development; (2) factors that affect the 
reliability of data that states collect and report on children served 
by their child welfare agencies, and HHS's role in ensuring the 
reliability of those data; and (3) practices that child welfare 
agencies use to overcome challenges associated with SACWIS development 
and data reliability.

To conduct our work, we surveyed all 50 states and the District of 
Columbia regarding their experiences in developing and using 
information systems and their ability to report data to HHS. We 
received responses from 49 states and the District of 
Columbia,[Footnote 1] although all states did not respond to every 
question. We also reviewed a variety of HHS documents, including the 
protocol and reports for its reviews of SACWIS systems and states' 
AFCARS reporting capabilities. In addition, we visited 5 states--
Colorado, Iowa, New York, North Carolina, and Oklahoma--to obtain 
first-hand information on their experiences developing SACWIS and 
reporting data to HHS. We selected these states to represent geographic 
diversity and different stages of SACWIS implementation. Finally, we 
interviewed HHS officials and child welfare and data experts and 
reviewed relevant literature. We conducted our work between June 2002 
and June 2003 in accordance with generally accepted government auditing 
standards. A more detailed discussion of our scope and methodology 
appears in appendix I.

Results in Brief:

HHS reported that 47 states are developing or operating a SACWIS, but 
many continue to face challenges despite HHS's oversight and technical 
assistance. Most states are using federal SACWIS funds and are in 
various stages of development. States reported in our survey that they 
have spent approximately $2.4 billion in federal, state, and local 
funding on SACWIS. Most state officials said they recognize the benefit 
their state will achieve by developing SACWIS, such as contributing to 
the timeliness of child abuse and neglect investigations; however, 
despite the availability of federal funds since 1994, many child 
welfare agencies lag behind the timeframes they set for completion, 
with delays ranging from 2 months to 8 years. Forty-two of the 46 
states responding to our survey that they are developing SACWIS 
reported at least some challenge obtaining state funding. In Iowa, for 
example, state officials reported that insufficient state funds delayed 
them in making the necessary modifications to meet federal requirements 
for system completion. Some states had difficulties developing a system 
that met the state child welfare agency's needs statewide. For example, 
state officials in New York--a state where the counties are responsible 
for administering child welfare services--said that building a uniform 
system was stalled when significant frustration with the system's 
design led some county officials to request that the state stop SACWIS 
development. In addition, 32 states reported at least some challenge 
securing information technology contractors with knowledge of child 
welfare practice to develop their SACWIS. In response to some of these 
challenges, HHS has provided technical assistance to help states 
develop their systems and conducted on-site reviews of SACWIS to verify 
that the systems meet all federal requirements. For example, at the 
time of our review, HHS had conducted on-site reviews in 26 states with 
operational SACWIS to ensure that the systems met all federal 
requirements and to offer assistance to states that faced challenges 
completing the development of their SACWIS.

Several factors affect states' ability to collect and report reliable 
data on children served by state child welfare agencies, and some 
problems exist, such as a lack of clear and documented guidance, with 
HHS's oversight and technical assistance. Almost all of the states 
responding to our survey reported that insufficient caseworker training 
and inaccurate and incomplete data entry into their information system 
affect the quality of the data reported to HHS. Although most states 
reported these as separate factors, HHS and the states we visited found 
that insufficient training and inaccurate and incomplete data entry are 
often linked. In addition, 36 of the 50 states that responded to our 
survey reported that technical challenges, such as matching their state 
data element definitions to HHS's data categories, affected the quality 
of the data that they report to the federal government. Similarly, 
during assessments of 6 states' compliance with AFCARS reporting 
standards, HHS found that these issues affect data reliability. Despite 
HHS's assistance in helping states improve their data, such as testing 
state data quality and providing the results to the states to aid them 
in resubmitting data, states report ongoing challenges. For example, 41 
of the 50 states responding to our survey reported that a lack of clear 
and documented guidance from HHS affects the quality of the data they 
report to AFCARS, and 25 states said the lack of clear, documented 
guidance also affected the data reported to NCANDS. In addition, 
although states were mandated to begin reporting AFCARS in 1995, few 
reviews of states AFCARS reporting capabilities have been conducted to 
assist states in resolving some of their challenges.

Some states are using a variety of practices to address the challenges 
associated with developing SACWIS and improving data reliability, 
although no formal evaluations are available on their effectiveness. To 
overcome the challenge of developing a system to meet statewide needs, 
many states relied on caseworkers and other system users for input on 
design and testing of SACWIS. Few states reported that they devised 
strategies to overcome the other challenges, such as limited funding 
and finding contractors with knowledge of child welfare. However, 
Oklahoma child welfare officials--in order to maximize the limited 
state funding for maintaining their SACWIS--reported saving $1 million 
each year by hiring some of the contractors who developed their SACWIS 
as permanent staff. To improve data reliability, the 5 states we 
visited routinely review their data to identify data entry errors so 
that managers can ensure that the missing data are entered 
appropriately. In addition, some states reported that frequent use of 
the data, such as publishing periodic management reports detailing 
local offices' performance on outcome measures, helps caseworkers 
understand the importance of entering timely information.

In order to improve the reliability of state-reported child welfare 
data, we are recommending that the Secretary of HHS consider ways to 
enhance the guidance and assistance offered to states to help them 
overcome the key challenges in collecting and reporting child welfare 
data. In commenting on this report, HHS's Administration for Children 
and Families (ACF) generally agreed with our findings and commented 
that the report provides a useful perspective of the problems states 
face in collecting data and of ACF's effort to provide ongoing 
technical assistance to improve the quality of child welfare data. In 
response to our recommendation, ACF said that we did not recognize the 
long-term efforts it has taken to provide AFCARS and NCANDS guidance. 
ACF also noted that the data definitions need to be updated and revised 
and said it is currently in the process of revising the AFCARS 
regulations to further standardize the information states are to 
report--which we acknowledge in our report. Further, ACF added that 
although staff turnover in state child welfare agencies is a 
significant contributor to data quality issues, we did not focus on 
this as a significant factor. However, because we recently issued a 
detailed report on a variety of caseworker issues, we primarily focused 
in this report on the key data entry challenges caseworkers face and 
refer readers to our previous work for additional information on 
challenges related to caseworker recruitment and retention and their 
affect on child welfare agencies. ACF commented that it is firmly 
committed to continue to support the states and to provide technical 
assistance and other guidance as its resources will permit. We believe 
that the recent activities to formally obtain, document, and 
incorporate feedback from the states with regard to collection and 
reporting adoption and foster care data represent are a step towards 
improving states' data. Our recommendation encourages HHS to consider 
ways to enhance the guidance and assistance already offered to states 
as a step to helping them better comply with the reporting 
requirements.

Background:

ACF's Children's Bureau is responsible for the administration and 
oversight of federal funding to states for child welfare services under 
Titles IV-B and IV-E of the Social Security Act. However, the 
monitoring of children served by state child welfare agencies is the 
responsibility of the state agencies that provide the services to these 
children and their families. Child welfare caseworkers at the county or 
local level are the key personnel responsible for documenting the wide 
range of services offered to children and families, such as 
investigations of abuse and neglect; treatment services offered to 
families to keep them intact and prevent the need for foster care; and 
arrangements made for permanent or adoptive placements when children 
must be removed from their homes. Caseworkers are supported by 
supervisors who typically assign new cases to workers and monitor 
caseworkers' progress in achieving desired outcomes, analyzing and 
addressing problems, and making decisions about cases.

A number of efforts at the national level have been taken to implement 
comprehensive data systems that capture, report, and analyze the child 
welfare information collected by the states (see table 1 for 
information on national data systems as well as information on state 
systems).

Table 1: Significant Child Welfare Information and Data Systems:

System: State information systems: 

System: Statewide Automated Child Welfare Information System (SACWIS); 
History: The Omnibus Budget Reconciliation Act (OBRA) of 1993 
authorized the use of an enhanced federal financial participation (FFP) 
rate of 75 percent to assist states develop uniform automated 
information systems--SACWIS--that support the administration of 
services offered under their child welfare programs.[A] The enhanced 
FFP was available initially from federal fiscal years 1994 through 1996 
and subsequently extended through federal fiscal year 1997. After 1997, 
states receive a 50 percent match for SACWIS-related activities. 
Funding approval for SACWIS is based on states' estimated costs for 
development and operation, and no time limits are placed on the receipt 
of federal funding; Use: A SACWIS is designed and developed for use by 
states' caseworkers and other personnel for the purposes of 
establishing an electronic case file for children and families served 
by the state child welfare agency. Some of the data captured in SACWIS 
are reported to HHS.

System: National data systems: 

System: Voluntary Cooperative Information System (VCIS); History: With 
funding from HHS, the American Public Welfare Association--now known as 
the American Public Human Services Association--established VCIS in an 
effort to compile state-specific child welfare data. States voluntarily 
reported aggregate-level data on the characteristics of children in 
foster care and those adopted from state child welfare agencies. VCIS 
compiled data between 1982 and 1994; Use: The data available in VCIS 
was used to produce reports at the national level on the 
characteristics of children in foster care and those adopted from state 
child welfare agencies. However, the data reported to VCIS were 
inconsistent.[B] Following the implementation of AFCARS, states no 
longer reported to VCIS.

System: Adoption and Foster Care Analysis and Reporting System 
(AFCARS); History: In 1986, Congress added Section 479 to Title IV-E of 
the Social Security Act, which required HHS to establish and implement 
a mandatory foster care and adoption data collection system. Despite 
the legislative requirement that HHS complete final regulations for the 
system by the end of calendar year 1988, regulations were not issued 
until December 1993. States began reporting to AFCARS in 1995 on the 
demographic characteristics of adoptive and foster children and their 
parents as well as foster children's type of placement and permanency 
goals. States are required to report 66 foster care and 37 adoption 
data elements for each child in care during 6-month periods. Between 
1998 and 2001, states not meeting certain reporting standards were 
subject to penalties. The penalties were rescinded in January 2002 
following an appeal filed by 12 states; Use: Some of the key uses of 
AFCARS data include: Calculating the national standard for five 
federal outcome measures--incidence of child abuse and/or neglect in 
foster care, foster care re-entries, stability of foster care 
placements, length of time to achieve reunification, and length of time 
to achieve adoption; The publication of the annual Child Welfare 
Outcomes Report; Determining the allocation of funds in the adoption 
incentive program and Chafee Foster Care Independence Program.[C].

System: National Child Abuse and Neglect Data System (NCANDS); History: 
In 1988, Congress amended the Child Abuse Prevention and Treatment Act 
(CAPTA) by directing HHS to establish a national data collection and 
analysis system consisting of state child abuse and neglect 
information. Walter R. McDonald & Associates was awarded the contract 
to compile and analyze the state-reported data. States began 
voluntarily reporting annually in 1990; however, the 1996 CAPTA 
amendments directed states to report certain data to receive CAPTA 
grant funding. States submit either child-specific records or 
aggregate-level data. In 2001--the most recently available data from 
HHS--39 states and the District of Columbia submitted child-specific 
data, which includes the demographics of the children and their 
perpetrators, the types of maltreatment, investigation or assessment 
findings, risk factors, and services provided as a result of the 
investigation or assessment. The remaining 11 states submitted 
aggregate data; Use: Some of the key uses of NCANDS data include: 
Calculating the national standard for two federal outcome measures on 
child safety--recurrence of maltreatment and incidence of child abuse 
and/or neglect in foster care; The publication of the annual Child 
Maltreatment Report; The publication of the annual Child Welfare 
Outcomes Report.

System: National Youth in Transition Database (NYTD); History: In 
response to requirements included in the Foster Care Independence Act 
of 1999, HHS has proposed to develop the NYTD to capture more detailed 
data than is currently reported to AFCARS on older foster children who 
receive independent living services. Such services include life-skills 
training in financial management or career planning. HHS reported to 
Congress in September 2001 that nationwide implementation of NYTD was 
planned for October 2003, but recent HHS information indicates that the 
actual implementation date will likely be 2005 or 2006. Once 
implemented, states will be required to report to NYTD; Use: In 
response to the legislation, data available in the NYTD will be used, 
in part, to track (1) the number and characteristics of older youth 
receiving independent living services, (2) the type and quantity of 
services states provided, and (3) state performance on the outcome 
measures required by the legislation.

Source: GAO analysis based on federal legislation, regulations, and 
other documents.

[A] The allowable costs under the 75 percent FFP included planning, 
design, development, and installation of a SACWIS. Other costs, such as 
operation of the SACWIS, were covered at 50 percent.

[B] See www.acf.hhs.gov/programs/cb/dis/vcis/maintoc.htm for more 
complete details on the shortcomings of the data available in VCIS.

[C] The total number of finalized adoptions reported to AFCARS are used 
to determine the amount, if any, of the adoption incentive payments 
awarded to states. In addition, AFCARS data are used to determine the 
ratio of the number of children in foster care in each state to the 
total number of children in foster care in all states to calculate 
state allotments under the Chafee Foster Care Independence Program.

[End of table]

To qualify for federal funding for SACWIS, states must prepare and 
submit an advance planning document (APD) to ACF's Children's Bureau, 
in which they describe the state's plan for managing the design, 
development, implementation, and operation of a SACWIS that meets 
federal requirements and state needs in an efficient, comprehensive, 
and cost-effective manner.[Footnote 2] In addition, the state must 
establish SACWIS and program performance goals in terms of projected 
costs and benefits in the APD. States are required to submit separate 
APDs for the planning and development phases, in addition to periodic 
updates.

Since the administration and structure of state child welfare agencies 
vary across the nation, states can design their SACWIS to meet their 
state needs, as long as states meet certain federal requirements. 
Federal funding is available to states for SACWIS that:

* meet the requirements for reporting AFCARS data to HHS;

* to the extent practicable, are capable of linking with the state data 
collection system that collects information on child abuse and neglect;

* to the extent practicable, are capable of linking with, and 
retrieving information from, the state data collection system that 
collects information on the eligibility of individuals under Title IV-
A--Temporary Assistance for Needy Families; and:

* provides for more efficient, economical, and effective administration 
of the programs carried out under a state's plans approved under Titles 
IV-B and IV-E of the Social Security Act.

A SACWIS must operate uniformly as a single system in each state and 
must encompass all entities that administer programs provided under 
Titles IV-B and IV-E. In some cases, HHS will allow the statewide 
system to link to another state system to perform required functions, 
such as linking to financial systems to issue and reconcile payments to 
child welfare service providers. The state's APD must describe how its 
SACWIS will link to other systems to meet the requirements in the 
SACWIS regulations.

In addition to monitoring the APDs of the states that are developing 
SACWIS, HHS reviews state information systems through formal SACWIS 
assessment reviews and the Child and Family Services Reviews (CFSR)--a 
federal review process to monitor states' compliance with child welfare 
laws and federal outcome measures. The formal SACWIS reviews are 
conducted by ACF's Children's Bureau to determine if a state has 
developed and implemented all components detailed in the state's APD 
and if the system adheres to federal requirements. The CFSR assesses 
statewide information systems, along with other systemic factors, to 
determine if the state is operating a system that can readily identify 
the status, demographic characteristics, location, and goals for 
placement of every child who is in foster care. This systemic factor is 
reviewed in all states, regardless of whether the state is developing a 
SACWIS or the stage of system development. According to results from 
the fiscal years 2001 and 2002 CFSRs, 4 of the 32 states in which HHS 
reviewed were not in substantial conformity on the statewide 
information system indicator. These 4 states must address how they will 
come into conformity with this factor in a program improvement plan. 
HHS has also conducted SACWIS reviews in 2 of these states.

Most States Are Developing SACWIS, but Challenges Remain Despite HHS's 
Oversight and Technical Assistance:

While 47 states are developing or operating a SACWIS, many challenges 
remain despite HHS's oversight and technical assistance. Since 1994, 
states reported that they have spent approximately $2.4 billion in 
federal, state, and local funding on SACWIS. While most state officials 
we interviewed and those responding to our survey said that they 
recognize the benefits their state will achieve by developing a 
statewide system, many states reported that the development of their 
SACWIS is delayed between 2 months and 8 years beyond the time frames 
the states set for completion, with a median delay of 2-½ years. Most 
states responding to our survey faced challenges, such as obtaining 
state funding and developing a system that met the child welfare 
agency's needs statewide. In response to some of these challenges, HHS 
has provided technical assistance to help states develop their systems 
and conducted on-site SACWIS reviews to verify that the systems meet 
all federal requirements.

States Are Using Federal and State Funds and Various Participants to 
Develop MultiComponent SACWIS:

Currently, 47 states are developing or operating a SACWIS and are in 
various stages of development--ranging from planning to complete. The 
states responding to our survey reported using approximately $1.3 
billion in federal funds[Footnote 3] and approximately $1.1 billion in 
state and local funds[Footnote 4] for their SACWIS. However, HHS 
estimated that it allocated approximately $821 million between fiscal 
years 1994 and 2001 in SACWIS developmental funds[Footnote 5] and $173 
million between fiscal years 1999 and 2001 in SACWIS operational 
funds.[Footnote 6] The total amount of federal funding provided to 
states for SACWIS is unknown because states claimed operational costs 
as a part of their Title IV-E administrative expenses prior to 
1999.[Footnote 7] Although the federal government matched state funding 
at an enhanced rate of 75 percent beginning in 1994, many states did 
not apply for federal funding or begin SACWIS development until 1996 
and 1997 when more than $467 million--the bulk of federal funds--were 
allocated. Most states were still developing their SACWIS by the time 
enhanced funding expired in 1997, after which states could receive a 50 
percent FFP for SACWIS development and operation. Although 47 states 
are currently developing or operating a SACWIS, all states except 
Hawaii received some federal SACWIS funds. For example, according to 
figures provided by HHS, North Carolina received approximately $9.6 
million in developmental funds and North Dakota received approximately 
$2.4 million in developmental funds and $245,000 in operational funds 
for SACWIS, but both states encountered difficulties that prevented 
them from completing their systems. In these situations, HHS entered 
into negotiations with the states about the amount of money that the 
states must return to the federal government.

In order to track states' SACWIS development, HHS places them in six 
categories that identify their stage of development (see table 2). 
States are required to submit APD updates periodically, which inform 
HHS of their progress in developing SACWIS. See appendix II for a 
complete list of states' phases of development. Although most states 
continue to advance in the development of their systems, some encounter 
problems that cause HHS to recategorize them into a lower stage of 
development. In Pennsylvania, for example, the child welfare agency 
encountered difficulties, such as inadequate computer software to 
support a comprehensive SACWIS, after attempting to implement its 
SACWIS in 2000. Due to these problems, the state is in the process of 
shutting down the system and has resubmitted an APD for a new system to 
HHS for review and approval for further federal funding. According to 
figures provided by HHS, Pennsylvania has received approximately $9.7 
million in federal funding thus far. In addition, while HHS may 
classify a state system as complete following an assessment of their 
SACWIS, a state may make additional changes to the system since SACWIS, 
like other computer systems, continually evolve as technology and child 
welfare practices change. States can claim federal funding for these 
changes as operational expenses. For example, Oklahoma's SACWIS was the 
first system to be determined complete, but it has made enhancements to 
its system since HHS found the system in compliance with federal 
requirements in 1998. In addition, Oklahoma is currently considering 
moving to a Web-based system. An HHS official reported that such 
changes do not need prior approval unless they are in excess of $5 
million.

Table 2: Number of States in Various Stages of SACWIS Development:

Stage: Complete[A]; Number of states: 5.

Stage: Operational[B]; Number of states: 24.

Stage: Partially operational[C]; Number of states: 9.

Stage: Implementation[D]; Number of states: 2.

Stage: Planning[E]; Number of states: 7.

Stage: No SACWIS[F]; Number of states: 4.

Source: HHS.

Note: Status is as of May 22, 2003.

[A] SACWIS assessment process is completed, and all functional 
requirements and specifications set forth in the APD are either 
included in the system or in an accepted corrective action plan.

[B] All functional requirements and specifications in APD are included 
in system, and system is functional statewide, but state has not 
completed SACWIS assessment or is working on other issues.

[C] State is still rolling out system to field sites or still adding 
functions to systems that are operational statewide.

[D] In active design and development, even if delayed while waiting to 
resolve problems such as funding.

[E] Working through options for a SACWIS.

[F] Have never pursued SACWIS funding or have abandoned plans to 
develop a system.

[End of table]

In developing a system, states have considerable flexibility in the 
design of their SACWIS. According to HHS officials, a state should be 
using its SACWIS as a case management tool that uses automation to 
support the various aspects of state child welfare programs, such as 
recording child protection, out of home care, and foster care and 
adoption services. To further assist child welfare practice, states 
have designed their systems to follow the natural flow of child welfare 
practice in their state and have added design features to help track 
key events during a case. For example, in Iowa child welfare work is 
divided between child abuse and neglect investigations and ongoing case 
management for children brought into the care of the child welfare 
agency. As a result, Iowa designed a SACWIS to reflect this work 
process by linking two databases--one to record child abuse and neglect 
information and one to record ongoing case records--that share 
information with one another.[Footnote 8] In Rhode Island, the SACWIS 
was designed to alert caseworkers if an alleged perpetrator has been 
the subject of three reports of abuse or neglect within 1 year. 
Regardless of the findings of each report, this alert notifies the 
caseworker to initiate an investigation when a third report is 
received.

Since many states are in different phases of SACWIS development, their 
systems currently support to varying degrees a variety of child welfare 
and administrative components (see table 3). According to HHS, while 
the components represented in table 3 are required for a state's SACWIS 
to be considered compliant with federal guidance--either through an 
interface or built within the system--some of the subcomponents, such 
as a function that helps caseworkers manage their caseloads, are 
optional. HHS has encouraged states to automate as many functions as 
possible in the SACWIS in an effort to cut down on the additional 
paperwork or duplicative steps inherent in manual data collection. One 
of these services, tracking independent living, is becoming more 
important for states as HHS decides how to implement the Foster Care 
Independence Act of 1999 and considers the development of the 
NYTD.[Footnote 9] Some states have already started collecting data on 
older youth and the services they receive. Currently, 27 states 
reported in our survey that they are at some stage of using their 
SACWIS to track independent living services, and an additional 14 
states plan to include this component in their system in preparation 
for the requirements. However, 21 of the 46 states reporting to our 
survey that they are developing or operating a SACWIS reported that 
they would have to make substantial changes to their SACWIS in order to 
capture this information.

Table 3: Selected SACWIS Child Welfare and Administrative Services:

Service: Child Welfare Services: 

Service: Child protection[A]; Fully or partially operational in SACWIS: 
38; Planned for SACWIS: 5.

Service: Out of home care[B]; Fully or partially operational in SACWIS: 
35; Planned for SACWIS: 8.

Service: Adoption; Fully or partially operational in SACWIS: 34; 
Planned for SACWIS: 9.

Service: Independent living; Fully or partially operational in SACWIS: 
27; Planned for SACWIS: 14.

Service: Intensive home-based services[C]; Fully or partially 
operational in SACWIS: 27; Planned for SACWIS: 13.

Service: Administrative Services: 

Service: Workload management; Fully or partially operational in SACWIS: 
32; Planned for SACWIS: 8.

Service: IV-E eligibility[D]; Fully or partially operational in SACWIS: 
29; Planned for SACWIS: 14.

Service: Foster care maintenance payments; Fully or partially 
operational in SACWIS: 28; Planned for SACWIS: 14.

Service: Adoption assistance payments; Fully or partially operational 
in SACWIS: 25; Planned for SACWIS: 17.

Service: Contract provider payment; Fully or partially operational in 
SACWIS: 24; Planned for SACWIS: 15.

Source: GAO survey.

Note: Based on responses from 46 states developing or operating a 
SACWIS. The rows for the columns "fully or partially operational" and 
"planned" do not add to 46 because the respondents may have answered 
"not supported," "don't know," or "no answer.":

[A] Child protection includes services such as intake and screening, 
investigation, and disposition.

[B] Out of home care includes things such as foster care, group homes, 
and residential placement.

[C] Intensive home-based services include efforts to avoid placing a 
child in foster care.

[D] IV-E funding is available for foster care, adoption, and 
independent living services.:

[End of table]

To assist with the design of their SACWIS, states relied on a number of 
different participants including internal users, such as caseworkers 
and managers, information technology (IT) staff, and contractors. Most 
states found these participants to be extremely or very helpful in the 
process (see table 4). In Oklahoma, for example, 150 child welfare 
staff from the field worked closely with the contractor in intensive 
work group sessions to design and test the system. To complement the 
caseworkers' knowledge of child welfare practice, 43 states relied on 
IT staff. In Colorado, for example, IT staff said that during SACWIS 
design and development, they shared office space with program staff 
that had been assigned to help with SACWIS development. This co-
location of staff aided in the exchange of information pertaining to 
the development of the system. Finally, 42 states reported that they 
hired private contractors to conduct a large part of SACWIS design and 
development. The contractors helped states meet federal requirements, 
designed the system with state specific options, wrote the necessary 
software, tested and implemented the system, and trained users.

Table 4: Commonly Used SACWIS Development Participants and Their Level 
of Helpfulness:

SACWIS development participant: Internal users; Number of states using 
participant: 44; Number of states rating participant as extremely or 
very helpful: 41.

SACWIS development participant: IT staff; Number of states using 
participant: 43; Number of states rating participant as extremely or 
very helpful: 31.

SACWIS development participant: Private contractors; Number of states 
using participant: 42; Number of states rating participant as extremely 
or very helpful: 37.

Source: GAO survey.

Note: Based on responses from 46 states developing or operating a 
SACWIS.:

[End of table]

At the time of our review, HHS reported that 4 states were not pursuing 
SACWIS development and most of these states reported various reasons in 
our survey for not developing a system. In Hawaii, the child welfare 
agency chose not to pursue SACWIS because it already had a statewide 
system in place that it believed was adequately meeting its needs and 
which was collecting and reporting federal child welfare data. After an 
attempt to develop a system, North Carolina cancelled its efforts 
because it could not build consensus across its 100 counties on the 
design of a uniform system. On our site visit to North Carolina, child 
welfare officials reported that they are currently working on a 
statewide information system that will encompass a number of social 
services, such as food stamps and mental health services, but an HHS 
official reported that North Carolina is not seeking federal SACWIS 
funding to support the development of this system. Vermont officials 
reported that they did not pursue SACWIS because the legislature 
declined to provide the matching state funds. In retrospect, they 
believe that the choice not to develop SACWIS was best for the state 
because they found the SACWIS requirements too restrictive to enable 
the state to design a system to meet its needs. Officials said that the 
state would not use a number of the required SACWIS components, such as 
developing all the required electronic links to other agencies' 
systems, especially since the state has a small child welfare 
population. Another state--North Dakota--did not report in our survey 
the reason for stopping SACWIS development; however, HHS officials 
reported that the state had attempted to develop a SACWIS, but faced a 
variety of problems, such as receiving state funding.:

States Accrue Benefits from Using SACWIS, but Several Issues Create 
Delays in Completing States' Systems:

While most state child welfare agency officials said they recognize the 
benefits the state will achieve by developing SACWIS, such as enhancing 
their ability to track the whereabouts of foster children, 31 state 
agencies lag behind the time frames they set for completion, with 26 
states reporting delays ranging from 2 months to 8 years. State 
officials reported in our survey and during site visits that SACWIS has 
contributed to more efficient and effective agency functioning, which 
can improve states' capabilities to manage their child welfare cases, 
including keeping track of where the children are living and the 
services they are receiving. Child welfare officials in Colorado 
reported that automation has improved agency functioning by making 
child welfare case information available statewide, which is especially 
helpful when families move from one county to another. In Oklahoma, 
caseworkers and state officials noted that they believe their children 
are safer since the implementation of SACWIS simply because the 
information on the children is easily accessible to the caseworkers and 
their supervisors. According to survey results, automated systems 
provided easier access to data and allowed caseworkers to better 
monitor children in their care, which may contribute to additional 
child welfare and administrative benefits, such as decreased incidences 
of child abuse and neglect, shortened length of time to achieve 
adoption, timeliness of payments to foster families, and timeliness of 
payments to foster facilities (see table 5). New Jersey reported in our 
survey that its goal in developing a SACWIS is to integrate the more 
than 40 stand-alone systems that currently capture information on the 
children served by their child welfare agency. By pulling all of these 
systems together into a uniform SACWIS, the state hopes to improve the 
recording of casework activities in a timely manner and to develop a 
tool to better target resources and services. Effectively integrating 
these systems will require the state to use a disciplined IT management 
approach that includes (1) detailed analyses of users' needs and 
requirements, (2) a clearly defined strategy for addressing information 
needs, and (3) sufficient technical expertise and resources to support 
the effort.

Table 5: State-Reported Benefits of SACWIS Development:

Child welfare benefits: Decreased incidence of child abuse and neglect; 
Number of states identifying SACWIS as extremely or very effective in 
achieving measure: 25.

Child welfare benefits: Shortened length of time to achieve adoption; 
Number of states identifying SACWIS as extremely or very effective in 
achieving measure: 24.

Child welfare benefits: Decreased recurrence of child maltreatment; 
Number of states identifying SACWIS as extremely or very effective in 
achieving measure: 23.

Child welfare benefits: Shortened length of time to achieve 
reunification; Number of states identifying SACWIS as extremely or very 
effective in achieving measure: 22.

Management benefits: 

Management benefits: Timeliness of payment to foster families; 
Number of states identifying SACWIS as extremely or very effective in 
achieving measure: 36.

Management benefits: Timeliness of payment to foster facilities; 
Number of states identifying SACWIS as extremely or very effective in 
achieving measure: 34.

Management benefits: Overall case management for children and 
families; Number of states identifying SACWIS as extremely or very 
effective in achieving measure: 33.

Management benefits: Timeliness of child abuse and neglect 
investigations; Number of states identifying SACWIS as extremely or 
very effective in achieving measure: 33.

Source: GAO survey.

Note: Based on responses from 46 states developing or operating SACWIS. 
States not included answered "moderately effective," "somewhat 
effective," "not at all effective," "not a state goal," "system doesn't 
support," "don't know," or "no answer.":

[End of table]

Despite the benefits that many states have accrued with SACWIS, 31 
states reported in our survey that they have been delayed in system 
completion beyond their initial deadline and identified a number of 
challenges that have led to the delay (see table 6).[Footnote 10] Some 
of the common difficulties states reported in developing SACWIS 
included receiving state funding approval, reaching internal agreement 
on system development, and creating a system that reflects child 
welfare work processes and is user friendly (see table 7).

Table 6: Number of Months States Delayed in SACWIS Development:

State: Alabama; Length of delay in months[A]: 36.

State: Arkansas; Length of delay in months[A]: 6.

State: California; Length of delay in months[A]: 36.

State: Colorado; Length of delay in months[A]: 26.

State: Connecticut; Length of delay in months[A]: 96.

State: District of Columbia; Length of delay in months[A]: 36.

State: Georgia; Length of delay in months[A]: 25.

State: Idaho; Length of delay in months[A]: 21.

State: Illinois; Length of delay in months[A]: 79.

State: Indiana; Length of delay in months[A]: 6.

State: Kansas; Length of delay in months[A]: 72.

State: Louisiana; Length of delay in months[A]: 12.

State: Maryland; Length of delay in months[A]: 12.

State: Michigan; Length of delay in months[A]: 26.

State: Minnesota; Length of delay in months[A]: 12.

State: Mississippi; Length of delay in months[A]: 12.

State: New Jersey; Length of delay in months[A]: 42.

State: New Mexico; Length of delay in months[A]: 3.

State: Ohio; Length of delay in months[A]: 36.

State: Oregon; Length of delay in months[A]: 70.

State: Rhode Island; Length of delay in months[A]: 14.

State: South Carolina; Length of delay in months[A]: 47.

State: Tennessee; Length of delay in months[A]: 36.

State: Utah; Length of delay in months[A]: 48.

State: Virginia; Length of delay in months[A]: 2.

State: Washington; Length of delay in months[A]: 36.

Source: GAO survey.

Note: While 31 states reported in the survey that they have experienced 
a delay in SACWIS development, only 26 states reported the length of 
their delay. The survey was issued in October 2002 and completed by 
states as late as December 2002.

[A] States were asked to report the number of months the delays 
exceeded the time line outlined in their APD.

[End of table]

Table 7: Common SACWIS Development Challenges:

SACWIS development challenges: Receiving state funding approval; Number 
of states experiencing at least some challenge[A]: 42.

SACWIS development challenges: Reaching internal agreement on system 
development; Number of states experiencing at least some challenge[A]: 
41.

SACWIS development challenges: Creating a system that reflects work 
processes; Number of states experiencing at least some challenge[A]: 
40.

SACWIS development challenges: Creating a user friendly system; Number 
of states experiencing at least some challenge[A]: 39.

SACWIS development challenges: Insufficient state funding allocation; 
Number of states experiencing at least some challenge[A]: 32.

SACWIS development challenges: Securing contractors with knowledge of 
child welfare; Number of states experiencing at least some 
challenge[A]: 32.

Source: GAO survey.

Note: Based on responses from 46 states developing or operating a 
SACWIS.:

[A] States were asked the extent to which certain factors were a 
challenge in developing SACWIS using the following scale: very great, 
great, moderate, some, or no challenge. The number of states reported 
in our analysis of challenges represents the total number of states 
reporting any extent the factor posed a challenge. States not included 
answered "no challenge," "have not yet reached this stage," "don't 
know," or "no answer.":

[End of table]

Forty-two states reported challenges receiving funding approval, and 32 
states reported that insufficient state funding allocations for SACWIS 
development were a challenge in developing a comprehensive system. For 
example, Vermont officials reported that the state legislature declined 
to provide the matching state funds needed to secure federal funding 
for SACWIS. As a result, the state could not pursue development. In 
addition to the development challenges reported in our survey, 2 of the 
5 states we visited reported that insufficient funding affected ongoing 
SACWIS maintenance. In Colorado, state agencies have received a series 
of budget cuts, which child welfare officials report have impacted 
their ability to respond to child welfare caseworkers' needs for system 
improvements. In Iowa, child welfare officials reported that the state 
legislature appropriated $17,000 for state fiscal year 2002 for all 
child welfare automated systems activity, which they noted was an 
insufficient amount to maintain and upgrade systems as well as to pay 
staff. They reported that, as a result, the child welfare agency 
provided the information systems department with approximately $1 
million from other parts of the agency's budget.

Despite user involvement in system design, some states still faced 
challenges trying to reach internal agreement among agency officials 
and caseworkers on the design of a system, resulting in a delay in 
development. In New York--a state where the counties are responsible 
for administering child welfare services--the development of SACWIS was 
stalled when significant frustration with the system's design led 
Commissioners from five large counties and New York City to request 
that the state stop SACWIS development until a reassessment of the 
design and plans for the implementation of the system was completed. 
After a thorough evaluation of the project, the state made changes to 
the project plan and developed statewide work groups to ensure all 
counties were in agreement with the system design. In addition, they 
hired a contractor to monitor system development and ensure that all 
users' requirements are seriously considered.

Similarly, despite states' heavy reliance on contractors, many reported 
that securing contractors with knowledge of child welfare practice was 
a challenge for timely SACWIS development. Contractors are hired by the 
state for their system development knowledge but often are unfamiliar 
with child welfare policies and practices, especially since they vary 
from state to state. Officials in Colorado, for example, said they 
encountered difficulties with their contractors because of high 
turnover among the contractor staff and their lack of knowledge of 
child welfare policies. A contractor who has worked with 7 states to 
develop their SACWIS reported that contractors are asked to learn the 
child welfare business practices of a state in a short amount of time 
and that states cannot devote many resources, such as caseworkers, to 
help in the design process because caseworkers need to devote their 
time to providing services to children and families. Therefore, 
contractors often have to acquire knowledge on their own.

Many states reported that creating a system that reflects child welfare 
work processes and is user friendly was a challenge in developing 
SACWIS. These issues were also identified in the federal reviews of 
states' SACWIS. For example, one state explained in the SACWIS review 
that it had designed a system to meet the caseworkers' needs and 
reflect the nature of the child welfare work processes by developing a 
system that required events to be documented as they occurred. However, 
this design limited the SACWIS's functionality because it did not allow 
the caseworkers to go back and enter information after an event 
happened. The state explained that caseworkers do not use the system in 
real time, but provide services to the children and families and then 
record the information in the system. The state had to redesign the 
system to correct for this design flaw. In addition, the 14 states 
reporting that they have adapted a system from another state have 
experienced some challenge modifying the systems to reflect their work 
processes. While HHS advises states to consider adapting another 
state's system if it requires few changes, states report that they and 
their contractors were not always aware of the changes that would need 
to be made to adapt a system. Although Colorado and New York decided to 
modify another state's SACWIS instead of designing a new system, child 
welfare officials in these states reported that adapting a SACWIS from 
another state has created more problems than anticipated. Colorado and 
New York adapted systems from state-administered child welfare 
agencies, which required extensive modifications to meet their needs as 
county-administered states. For example, Colorado needed a system that 
supported its administrative structure and could handle a larger number 
of cases.

HHS Provides Some Assistance to Help States Meet SACWIS Requirements:

HHS has assisted states in a variety of ways in developing and 
completing their SACWIS.[Footnote 11] As a part of its regulatory 
responsibilities, HHS must review, assess, and inspect the planning, 
design, development, installation, and operation of SACWIS. In addition 
to reviewing and monitoring states' APDs, HHS conducts on-site SACWIS 
reviews to comply with these responsibilities. HHS officials told us 
that these reviews are a detailed and thorough assessment of state 
systems to ensure the systems' compliance with SACWIS requirements. In 
addition, officials reported that they provide verbal technical 
assistance during the on-site review to help states that do not fully 
conform with the applicable regulations and policies. At the time of 
our review, HHS had conducted 26 SACWIS reviews--5 of which were 
determined as meeting all the requirements and classified as complete. 
HHS officials told us that since states have the flexibility to build a 
SACWIS that meets their needs, a large portion of the formal reviews 
concentrate on ensuring that the systems conform to state business 
practices. For example, while SACWIS regulations require that a state 
report all AFCARS data from their SACWIS, one state HHS reviewed relied 
on a separate state system to report data on the children served by the 
juvenile justice agency who are eligible for IV-E foster care funds. 
The state proved it had developed an automated process to merge data 
from both systems to compile a single AFCARS report that included 
children captured in both their SACWIS and juvenile justice systems. 
Therefore, HHS recognized that this process best met the state's needs 
and determined the SACWIS to be complete and meeting all requirements.

Few systems have been determined complete after an on-site review 
because of unresolved issues, such as not being able to build links to 
other state information systems or not implementing certain eligibility 
determination functions. To help states address some of these 
development challenges, the SACWIS review team provides the state with 
recommendations for complying with SACWIS requirements. For example, 
HHS observed during a review in one state that the SACWIS was available 
statewide, but information collected in one county was not available to 
caseworkers in other counties. The federal officials offered 
recommendations to the state to meet the SACWIS requirement that all 
information be available statewide. In addition, HHS officials reported 
that once the draft report with the results of the SACWIS review are 
completed, federal staff schedule a conference call with the state 
officials to walk through the system's deficiencies and offer guidance 
on how the state can move forward.

HHS facilitates the sharing of information between states developing 
SACWIS through an automated system users group that allows state and 
federal officials to exchange information, ideas, and concerns. 
According to some state and HHS officials, the trust level at these 
meetings is very high, which promotes open discussions and also creates 
an atmosphere for informal dialogue with HHS. The systems users group 
developed out of another active group--the child welfare users group--
when HHS solicited state representatives to help HHS define a model 
child welfare information system, which was later used as the basis for 
the SACWIS functional requirements after the passage of the 1993 
legislation authorizing enhanced federal funding. State officials in 
Iowa and New York reported that the systems users group continues to 
play an important role in providing a forum for the honest exchange of 
information on SACWIS development. For example, child welfare and 
technical officials in New York said that the systems users group has 
been very beneficial because they have learned from other states' 
positive and negative experiences in developing SACWIS, as well as the 
experiences unique to states with county-administered agencies. In 
addition to the users group, HHS officials also sponsor a listserv--an 
electronic mailing list--that allows state officials to exchange 
information, and a monthly conference call with state information 
technology directors. Iowa child welfare information technology 
officials said that they find the monthly SACWIS telephone conference 
call helpful because project managers discuss issues such as promising 
practices and new regulations.[Footnote 12]

Technical assistance for SACWIS development is also available to states 
through the National Resource Center for Information Technology in 
Child Welfare (Resource Center). According to survey results, 9 states 
said they used the Resource Center for assistance in developing SACWIS 
and 14 states reported using it for help with SACWIS maintenance and 
improvements. According to Resource Center officials, they assist 
states with SACWIS development by helping states understand the 
technology that is available for use, providing information on the 
automation of child welfare work and converting data, and reviewing the 
APD documentation. For example, the Resource Center offered technical 
assistance to Pennsylvania to help the state decide if it should 
continue development of its current SACWIS, abandon the SACWIS project 
and allow the counties to operate individual systems, or design a 
different SACWIS. The Resource Center evaluated the current SACWIS to 
determine if it could capture information based on the SACWIS 
regulations and if it was user friendly for the caseworker. Following 
the Resource Center's analysis, Pennsylvania decided to discontinue the 
existing SACWIS and develop a new SACWIS. When the Resource Center 
opened in 1999--5 years after many states started developing SACWIS--
staff were not very familiar with many of the efforts states made 
during development. In an attempt to remedy this lack of knowledge on 
states' issues developing SACWIS, Resource Center staff participated in 
some of the on-site SACWIS reviews conducted by HHS. Both HHS and 
Resource Center officials believe this exposure to the SACWIS systems 
enhanced the availability of technical assistance resources and 
knowledge available to the states.

Several Factors Affect the States' Ability to Ensure Reliable Data on 
Children's Experiences, and Some of HHS's Oversight and Assistance Is 
Problematic:

Several factors affect states' ability to collect and report 
reliable[Footnote 13] data on children served by state child welfare 
agencies, and some problems exist, such as a lack of clear and 
documented guidance, with HHS's oversight and technical assistance. 
Almost all of the states responding to our survey reported that 
insufficient caseworker training and inaccurate and incomplete data 
entry affect the quality of the data reported to HHS.[Footnote 14] In 
addition, 36 of the 50[Footnote 15] states that responded to our survey 
reported that technical challenges, such as matching their state data 
element definitions to HHS's data categories, affected the quality of 
the data that they report to the federal government. For example, North 
Carolina officials told us that while state policy mandates that they 
count every location in which a child resides, including hospital 
stays, AFCARS regulations say that hospital stays and other short-term 
placements should not be included in the count of foster care 
placements. In cases where state policy differs from federal policy, 
state officials must carefully re-format their data in order to meet 
federal reporting requirements. Similarly, during assessments of 6 
states' compliance with AFCARS reporting standards, HHS found that 
these issues affect data reliability. Despite the assistance that HHS 
offers to states, such as testing state data quality and providing the 
results to states to aid them in resubmitting data, states report 
ongoing challenges receiving clear and documented guidance and 
accessing technical assistance.

Insufficient Caseworker Training and Inaccurate and Incomplete Data 
Entry Are the Most Common Factors That Affect Data Reliability:

Almost every state responding to our survey and all the states we 
visited reported that insufficient training for caseworkers and 
inaccurate and incomplete data entry affect the quality of the data 
reported to AFCARS and NCANDS (see fig. 1). Although most states 
reported these as separate factors, HHS and the states we visited found 
that insufficient training and inaccurate and incomplete data entry are 
often linked. For example, in official reviews of states' information 
systems capability to capture data and report them to AFCARS, HHS 
advised states to offer additional training to caseworkers on several 
AFCARS data elements, such as recording the reasons for a child leaving 
foster care, to improve the accuracy of the data submitted. Similarly, 
Oklahoma reported that the state found that caseworkers were 
misinterpreting reports of policy violations by foster parents and 
inaccurately recording them as abuse or neglect allegations. However, 
state officials told us that training is typically one of the first 
programs cut when states face tight budget restrictions. For example, 
Iowa officials told us that training has been significantly reduced in 
recent years because of budget cuts and new workers may wait 2 to 3 
months before being trained how to enter data appropriately into their 
SACWIS.

Figure 1: Most Common Caseworker Issues That Affect Data Quality:

[See PDF for image]

Notes: Based on responses from 50 states.

[End of figure]

The results reported in the figure are a sum of the states that 
reported the issue had a very great affect, great affect, moderate 
affect, or some affect on the quality of state data submitted to HHS. 
Very great and great affect responses are represented in the top 
section of each bar. Moderate and some affect responses are represented 
in the bottom section of each bar. States not included answered "no 
affect," "don't know," or "no answer.":

Inaccurate and incomplete data entry can also result from a number of 
other factors, such as caseworkers' hesitation to ask families for 
sensitive information. For example, caseworkers in Oklahoma reported 
that they did not feel comfortable asking if a child's mother was 
married at the time of birth or if a child is of Hispanic origin--both 
of which are required AFCARS data elements. In commenting on a draft of 
this report, Oklahoma added that caseworkers did not understand why the 
data elements were required and how the federal government used the 
information. In addition, Iowa state officials said that caseworkers 
may guess the racial backgrounds of children in their care or record 
them as unknown, especially when children come from mixed racial 
backgrounds, rather than asking the family for the information. HHS 
noted similar issues in 5 states that have had an AFCARS 
review.[Footnote 16] Caseworkers were inaccurately recording a child's 
race as "unable to determine" even though this option should be 
selected only if the child's parents or relatives cannot provide the 
information, such as when a child is abandoned.[Footnote 17]

Caseworkers, supervisors, and managers in the 5 states we visited 
reported that additional factors, such as difficulties balancing data 
entry with the time that they spend with the families and children, 
contributed to inaccurate or incomplete data entry. In addition, our 
recent work on caseworker recruitment and retention found that 
caseworkers struggle to balance the time they spend with children and 
data entry, and reportedly spend at least 50 percent of their time 
documenting case records.[Footnote 18] Supervisors in Iowa explained 
that since caseworkers are responsible for ensuring that children and 
their families receive the services they need, the caseworkers tend to 
initially limit data entry to the information that is necessary to 
ensure timely payment to foster care providers, and complete all other 
data elements when the caseworkers have time. In addition, caseworkers 
in Colorado said that they are between 30 and 60 days behind in their 
data entry, so the information in the automated system may not 
accurately reflect the current circumstances of children in care. The 
caseworkers reported that they tend to concentrate only on entering 
data that will allow them to open a case in their SACWIS. HHS's 
Inspector General recently issued a report in which more than two-
thirds of the states reported that caseworkers' workloads, turnover, a 
lack of training, and untimely and incomplete data entry affected the 
reporting of AFCARS data.[Footnote 19]

Technical Challenges, such as Matching State Definitions to Federal 
Definitions, Affect Data Reliability:

In addition to data quality being affected by caseworker issues, many 
states experienced technical challenges reporting their data to HHS. 
The problems reported by states are typically a result of challenges 
associated with data "mapping"--matching state data elements to the 
federal data elements. For example, 36 states reported in our survey 
that matching their state-defined data to HHS's definitions affected 
the quality of the data reported to NCANDS and AFCARS. Similarly, 24 
states reported that matching the more detailed data options available 
in their states' information systems to the federal data elements 
affected the quality of the data reported to NCANDS. Twenty-nine states 
reported that this issue created challenges in reporting data to 
AFCARS. For example, following an AFCARS assessment, HHS instructed a 
state that collects detailed information on children's disabilities, 
such as Downs Syndrome, Attention Deficit Disorder, and eating 
disorders, to map the information to the more limited options in 
AFCARS, such as mental retardation and emotionally disturbed. The 
Inspector General's report found that states faced similar challenges 
mapping their data to meet the AFCARS reporting requirements.

In many cases, states have to balance state policy with federal 
requirements to ensure that they are reporting accurate data to AFCARS 
and NCANDS, but are not contradicting their state policies. For 
example, Texas officials reported that although the findings of their 
AFCARS review instructed them to modify their SACWIS to collect, map, 
and extract data on guardianship placements, the state does not support 
guardianship arrangements.[Footnote 20] In addition, a recent report 
from the Child Welfare League of America (CWLA) found that when 
reporting the number of times children move from one foster care 
placement to another, states varied in the type of placements included 
in that count.[Footnote 21] For example, 29 percent of the states 
responding to CWLA's survey included respite,[Footnote 22] 25 percent 
included runaways, and 16 percent included trial home visits when 
reporting the number of placements a child had during the AFCARS report 
period. According to federal guidance, the "number of placements" 
element is meant to gather information on the number of times the child 
welfare agency found it necessary to move a child while in foster care 
and that by including runaways or trial home visits, a state is 
inflating the number of moves a child experienced. However, North 
Carolina officials told us that although the federal definition for 
placements instructs states not to include such stays when counting the 
number of children's foster care placements, the state instructs them 
to count each time a child is sleeping in a different place as a new 
placement. The Inspector General reported that the placement 
definitions were the most commonly cited source of confusion among the 
states surveyed.[Footnote 23]

In addition to the challenges reported in our survey, HHS reported that 
transferring data from older data systems into SACWIS affects the 
quality of the data reported to AFCARS and NCANDS. HHS officials 
reported that they have observed that states experience the biggest 
change in data quality when they begin reporting from their SACWIS. In 
general, the first data submissions are of low quality because of the 
time it takes states to transfer data or the system re-sets the 
information for data elements. For example, in 1 state, 65 percent of 
the records reviewed by HHS during an AFCARS assessment recorded the 
date the children were removed from their homes as July 28, 1997--the 
date the SACWIS came on-line; however, the actual dates of removal for 
these children ranged from 1988 to 1997.

Although HHS Has Taken Steps to Help States Improve Their Data, Some 
Problems with Its Efforts Exist:

HHS provides technical assistance for AFCARS and NCANDS reporting 
through a number of resources. HHS officials in the central office and 
NCANDS contractor staff serve as the points of contact for states to 
ask questions and seek guidance on reporting child welfare data. HHS 
officials reported that assistance is offered in a number of ways, 
including telephone and e-mail communication. The officials in 3 of the 
5 states that we visited said that the one-on-one focused technical 
assistance was useful when provided in a timely fashion. Most state 
officials found the NCANDS data easier to report, in part because more 
people were available for consultation and they were more accessible 
and responsive. For example, states have access to four NCANDS 
specialists and staff in the contractor's central office when they need 
assistance reporting child abuse and neglect information. However, some 
of the states we visited reported that only one or two staff in HHS's 
central office are available to assist with AFCARS reporting.

In addition, the Resource Center offers states assistance with 
improving data quality; however, Resource Center staff reported that 
the assistance is geared more towards improving the limited data used 
in the federal review process to monitor states' compliance with child 
welfare laws and federal outcome measures--CFSR--rather than all the 
data reported to HHS. The Resource Center also sponsors an annual 
information technology conference during which sessions covering all 
data-related issues are held, including practices for ensuring data 
quality and outcome evaluation in child welfare.

In conjunction with the national data conference, the HHS officials and 
the contractors that operate NCANDS hold an annual technical assistance 
meeting for states to share ideas with one another, discuss data 
elements that pose difficulties, and explore ways to address these 
problems. For example, at a recent technical assistance meeting, 
approximately 43 state representatives attended sessions on preparing 
the calendar year 2002 NCANDS data submissions and received a detailed 
explanation of how the NCANDS staff test states' data submissions for 
quality. In addition, an NCANDS state advisory group meets annually to 
talk with HHS officials about NCANDS data and their experiences 
reporting data. From these meetings, the state advisory group proposes 
changes or improvements to NCANDS. HHS and state officials reported 
that this partnership has helped ease some of the challenges in 
reporting child abuse and neglect data.

In addition to the direct assistance through consultation with HHS 
officials and the Resource Center, HHS has made available to states the 
software it uses to examine states' AFCARS and NCANDS submissions for 
inconsistencies and invalid data. Officials in all the states we 
visited said that they regularly use this software, and an HHS official 
said that nearly every state has used the software at least once. When 
the data are submitted to HHS, they are run through the same software, 
and HHS notifies the states of areas where data are missing or 
inconsistent and allows the states to resubmit the data after errors 
are corrected. For example, HHS officials said that they worked with 
one state that was trying to determine the source of data errors in 
reporting to AFCARS the race or ethnicity of children in their care. 
The state was not able to determine the source of the problem, so an 
HHS official examined the state's submissions and helped correct the 
data errors. The officials reported that these tests help them to 
identify some data quality errors, such as missing data, and said that 
they believe that, in general, data have improved in recent years. 
However, they indicated that the tests cannot pinpoint the underlying 
problems contributing to these errors. Furthermore, one official 
reported that no specific efforts have been conducted to track the 
individual data elements and, therefore, HHS cannot report on how data 
quality has changed over time. The results of these quality tests had 
been the basis for penalties levied against states that submitted low 
quality AFCARS data before the penalties were rescinded. HHS officials 
reported that the penalties served as an effective motivation to states 
to correct their data. Although HHS was not able to report how the lack 
of penalties might be affecting recent data quality, an official 
reported that the agency plans to conduct this analysis in the future.

In an attempt to help states comply with the reporting standards and 
address some of the factors that contribute to data quality problems, 
HHS performs comprehensive reviews of state information systems' 
ability to capture AFCARS data to identify problems associated with 
data collection and reporting, and to ensure that the information in 
the automated system correctly reflects children's experiences in care. 
The assessments include a technical review of the states' computer 
code, a comparison of the data from selected cases available in the 
information system to the case files, and an improvement plan to 
resolve any errors. In addition, HHS officials offer guidance to the 
states on improvements that can be made to the information system and 
changes to program code used to report the AFCARS data. HHS conducted 
pilot reviews in eight states between 1996 and 2000. By March 2003, HHS 
had conducted eight official reviews--even though states began 
reporting to AFCARS in 1995. According to results from six of the eight 
official AFCARS assessments we reviewed, no state met the reporting 
requirements for all AFCARS data elements. Table 8 shows a selection of 
the data elements and the states' ratings. The problems noted in the 
reviews are similar to those we heard from states responding to our 
survey and those we visited. For example, most states received ratings 
of 2 or 3, indicating technical and/or data entry errors that affect 
the AFCARS data quality.[Footnote 24]

Table 8: Selected AFCARS Elements and Six States' Levels of Compliance 
in Meeting Reporting Requirements:

Data Elements: Foster Care: 

Data Elements: Child race (American Indian or Alaska native, Asian, 
Black or African American, Native Hawaiian or other Pacific Islander, 
White, unable to determine); Rating factor[A]: (# of states); 2 (4 
states); 3 (1 state); 4 (1 state).

Data Elements: Has the child been diagnosed with a disability? (yes, 
no, not yet determined); Rating factor[A]: (# of states); 2 (6 states).

Data Elements: Mental retardation, visually/hearing impaired, 
physically disabled, emotionally disturbed, other diagnosed 
condition[B[(DOES NOT APPLY, APPLIES)] es); Rating factor[A]: (# of 
states); 2 (5 states); 3 (1 state).

Data Elements: Has child ever been adopted? (yes, no, unable to 
determine); Rating factor[A]: (# of states); 1 (1 state); 2 (3 states); 3 
(2 states).

Data Elements: Total number of removals from home[C]; Rating factor[A]: 
(# of states); 2 (3 states); 3 (2 states); 4 (1 state).

Data Elements: Date of discharge from previous episode[C]; Rating 
factor[A]: (# of states); 2 (3 states); 3 (2 states); 4 (1 state).

Data Elements: Date of latest removal[C]; Rating factor[A]: (# of 
states); 2 (3 states); 3 (3 states).

Data Elements: Date of placement in current setting; Rating factor[A]: 
(# of states); 2 (5 states); 4 (1 state).

Data Elements: Number of previous placement settings in this 
episode[C]; Rating factor[A]: (# of states); 2 (6 states).

Data Elements: Current placement setting (pre-adoptive home, foster 
family home-relative, foster family home-nonrelative, group home, 
institution, supervised independent living, runaway, trial home visit); 
Rating factor[A]: (# of states); 2 (4 states); 3 (1 state); 4 (1 state);.

Data Elements: Out of state placement (yes, no); Rating factor[A]: (# 
of states); 2 (4 states); 3 (1 state); 4 (1 state).

Data Elements: Most recent case plan goal (reunify with parents or 
principal care takers, live with other relative(s), adoption, long term 
foster care, emancipation, guardianship, case plan goal not yet 
established); Rating factor[A]: (# of states); 1 (1 state); 2 (5 
states).

Data Elements: Date of discharge[C]; Rating factor[A]: (# of states); 2 
(2 states); 3 (2 states); 4 (2 states).

Data Elements: Reason for discharge[C] (NOT APPLICABLE, REUNIFICATION 
WITH PARENT(S) OR PRIMARY CARETAKER(S), LIVING WITH OTHER 
relative(s), adoption, emancipation, guardianship, transfer to another 
agency, runaway, death of child); Rating factor[A]: (# of states); 2 (4 
states); 3 (1 state); 4 (1 state).

Data Elements: Adoption: 

Data Elements: Child race (American Indian or Alaska native, Asian, 
Black or African American, Native Hawaiian or other Pacific Islander, 
White, unable to determine); Rating factor[A]: (# of states); 2 (4 
states); 3 (1 state); 4 (1 state).

Data Elements: Primary basis for determining special needs (not 
applicable; racial/original background; age; membership in a sibling 
group; medical conditions or mental, physical, or emotional 
disabilities; other); Rating factor[A]: (# of states); 2 (6 states).

Data Elements: Mental retardation, physically disabled, emotionally 
disturbed[B[(DOES NOT APPLY, APPLIES)] es); Rating factor[A]: (# of 
states); 2 (5 states); 3 (1 state).

Data Elements: Visually/hearing impaired (does not apply, applies); 
Rating factor[A]: (# of states); 2 (4 states); 3 (2 states).

Data Elements: Other diagnosed condition (does not apply, applies); 
Rating factor[A]: (# of states); 2 (6 states).

Source: HHS.

Note: Analysis based on results of AFCARS reviews in 6 states. States 
are rated on 66 foster care elements and 37 adoption elements.

[A] HHS rates each data element using a four-point scale: (1) the 
AFCARS requirement(s) has not been implemented in the information 
system; (2) the technical system requirements for AFCARS reporting do 
not fully meet the standards; (3) the technical system requirements for 
AFCARS reporting are in place, but there are data entry problems 
affecting the quality of the data; (4) all of the AFCARS requirements 
have been met.

[B] States are rated on each category. The data elements were combined 
for purposes of this analysis.

[C] AFCARS data element used in the CFSR.:

[End of table]

State officials in these 6 states reported that they found the reviews 
useful for improving their AFCARS data submissions. In particular, they 
valued the thorough review by HHS officials of the computer code states 
use to report the data. Some of these officials reported that if all 
states were reviewed, the quality of data available in AFCARS would 
improve tremendously. However, HHS officials reported that they are not 
mandated to conduct the AFCARS reviews and that priority is placed on 
other reviews, such as the CFSR and SACWIS reviews. In addition, 
officials explained that the AFCARS reviews are not conducted in states 
developing SACWIS until the systems are operational. HHS expects to 
complete approximately four reviews each year depending on available 
resources and has scheduled states through 2006. Similar to the SACWIS 
reviews, HHS officials offer recommendations and technical assistance 
to states during the review on how they can improve the quality of the 
data reported to AFCARS.

Although the states we visited appreciated some of HHS's efforts to 
assist with improving state data quality, they and most states 
responding to our survey agreed that the assistance is not always 
consistent or easily accessible (see fig. 2). States reported similar 
information to the Inspector General--AFCARS data elements were not 
clearly and consistently defined and technical assistance is effective 
but difficult to access.

Figure 2: Federal Practices That Affect Data Quality:

[See PDF for image]

Notes: Based on responses from 50 states.

[End of figure]

The results reported in the figure are a sum of the states that 
reported the issue had a very great affect, great affect, moderate 
affect, or some affect on the quality of state data submitted to HHS. 
Very great and great affect responses are represented in the top 
section of each bar. Moderate and some affect responses are represented 
in the bottom section of each bar. States not included answered "no 
affect," "don't know," or "no answer.":

The primary concerns reported by the states we visited were delays in 
receiving clear, written guidance on defining and reporting certain 
data elements and the lack of state input in suggesting changes to 
AFCARS. Despite the written guidance available to states in the form of 
regulations and an on-line policy manual, states reported that the 
variation in state policies and practices make it difficult to 
interpret how to apply the general guidance. As a result, states 
consult with HHS to ensure they are applying the regulations 
appropriately. However, in commenting on a draft of this report, 
officials in Oklahoma told us that a common concern among the states is 
the lack of timely response from HHS when seeking guidance on how to 
report data. In addition, officials in New York explained they have 
made it a practice to check the HHS Web site on a regular basis for 
current guidance, but have not found it a useful tool, and may turn to 
other states for guidance on AFCARS reporting. In commenting on a draft 
of this report, HHS explained that it first refers states to its Web 
site for information and believes that the available guidance addresses 
states' concerns in most instances. In addition, the states that have 
had an AFCARS review experienced delays in obtaining guidance on how to 
proceed following the on-site review. Although they found the review to 
be very helpful, some states reported that HHS officials are delayed in 
responding to their questions. For example, Texas officials reported 
that the state sought clarification on its improvement plan and 
submitted additional questions to HHS following the review; however, 
when we spoke with the state officials, they said that they had been 
waiting 3 months for a response on how to proceed. An HHS official told 
us that since the review process is relatively new, the agency is still 
developing a process to respond to the states and recognizes that it 
has not been responsive to the states already reviewed. In addition, 
HHS is taking steps to gather feedback from states and other users of 
AFCARS data to determine how to improve the system to make the data 
more accurate and useable. As a part of these efforts, HHS has 
published a Federal Register notice soliciting comments and held focus 
group meetings at national conferences.

The difficulties states face in receiving federal guidance and 
assistance, as well as the other challenges they face in reporting 
data, may negatively impact the reliability of the data available in 
AFCARS and NCANDS. As a result, states are concerned that the national 
standards used in the CFSR are based on unreliable data and should not 
be used as a basis of comparison and potential financial penalty. The 
variation in states' reporting practices may affect the validity of the 
measures and may place some states at a disadvantage. For example, the 
CWLA and Inspector General studies found that approximately half the 
states include the juvenile justice population in their AFCARS reports, 
while the other states do not.[Footnote 25] Child welfare experts and 
some state officials believe that the states that include children 
served by the juvenile justice agency in their AFCARS report may report 
a higher number of re-entries into the child welfare system or a higher 
number of moves within the system when compared to states that do not 
have IV-E agreements[Footnote 26] with their juvenile justice 
systems.[Footnote 27] As a result, a state that includes such children 
in their AFCARS report are likely to fare less favorably when compared 
to the national standard than other states on two outcome measures--
foster care re-entries and stability of foster care placements--and may 
face financial penalties associated with the CFSR.

States Are Using Various Practices to Overcome System Development 
Challenges and Improve Data on Children's Experiences:

Some states are using a variety of practices to address the challenges 
associated with developing SACWIS and improving data reliability, 
although no formal evaluations are available on their effectiveness. To 
address the challenge of developing a system to meet statewide needs, 
states relied on caseworkers and supervisors from local offices to 
assist in the design and testing of the system. Few states reported in 
our survey strategies to overcome the other key challenges, such as 
limited funding and securing knowledgeable contractors, but some states 
we visited have devised some useful approaches. For example, Oklahoma 
child welfare officials--in order to maximize the limited state funding 
for maintaining their SACWIS--reported saving $1 million each year by 
hiring some of the contractors who developed their SACWIS as permanent 
staff. To improve data reliability, the 5 states we visited routinely 
review their data to identify data entry errors so that managers can 
ensure that the missing data are entered appropriately. In addition, 
some states reported that frequent use of the data, such as publishing 
periodic management reports detailing local offices' performance on 
outcome measures, helps caseworkers understand the importance of 
entering timely information.

States Are Primarily Relying on SACWIS Users to Overcome Some of the 
Challenges to Completing Their Systems:

To overcome development challenges, survey respondents emphasized the 
importance of including system users in the various phases of 
completing SACWIS--planning, design, development, testing, and 
implementation. Past GAO work and other research efforts have 
determined similar approaches as best practices in building information 
systems.[Footnote 28] Forty-four of the 46 states responding to our 
survey that they are developing or operating a SACWIS indicated that 
they relied on internal users, such as caseworkers and supervisors, in 
the development of their systems and 34 of these states said that they 
were extremely helpful participants. The extent to which the users were 
involved in development differed across the states. For example, in 
Texas, caseworkers from all of their child welfare regions were 
recruited to provide input on design and development, as well as during 
initial testing, pilot testing, and implementation of the system. 
Arkansas reported establishing a committee made up of users to review 
the work plan and sign off on recommended changes. In addition, states 
reported that their system users served a number of purposes, including 
serving as experts on the different specialties within child welfare, 
such as child abuse, foster care, or adoption, and as representatives 
from local or county offices to assist in identifying the diverse 
approaches to capturing information across the state. For example, 
Indiana reported that caseworkers involved in SACWIS development 
represented the unique needs of the different geographical areas of the 
state and helped design a uniform statewide system to meet the diverse 
needs of large, intermediate, and small local offices.

Ten states noted that user input should not be limited to frontline 
workers, such as caseworkers, but should include representatives from 
other areas of the agency, such as the financial staff, and other 
agencies that serve children, such as child support 
enforcement.[Footnote 29] Since many SACWIS link with other state 
information systems, states advised that developing a collaborative 
relationship with other state agencies will help the development of the 
system. While not one of the most common challenges reported in our 
survey, New Hampshire reported that one of its challenges with meeting 
its SACWIS timeframe was not working collaboratively with other 
agencies, such as Temporary Assistance for Needy Families 
(TANF)[Footnote 30] and child support enforcement, to develop the 
payment component of SACWIS. Similarly, we previously reported that the 
difficulty developing linkages between social services agencies limits 
the effectiveness of all the programs to serve families.[Footnote 31] 
To attempt to overcome this challenge, 26 of the 46 states responding 
to our survey that they are developing or operating a SACWIS indicated 
that they included external public agency users and 23 reported using 
representatives from other state agencies that serve children in 
developing their SACWIS. Indiana said that a task force made up of 
representatives from the TANF and child support enforcement agencies 
was developed to design the linkages between the systems. In addition, 
Colorado officials reported that they are working with the Department 
of Youth Corrections--an agency that shares the SACWIS with child 
welfare--to ensure that the shared screens use the same definitions.

In addition to seeking input from caseworkers and other system users 
while developing SACWIS, many states continue to include users as a 
part of the implementation teams, to serve as contacts in the field and 
provide ongoing assistance, and to provide input on system 
enhancements. Alabama responded in our survey that the state had 
"mentors" in each county to help caseworkers adjust to the new system. 
These mentors continue to provide ongoing support now that the system 
is implemented. Similarly, Oklahoma developed Field Implementation 
Teams consisting of one contractor and one child welfare staff person. 
During system implementation, the teams went to field offices to 
provide on-site assistance with using SACWIS and becoming accustomed 
with the new method of recording child welfare information. 
Furthermore, Oklahoma recruits experienced child welfare field staff 
for its SACWIS help desk because of their knowledge of the system and 
child welfare policy and practice.

Although states faced other challenges in completing their SACWIS, few 
reported implementing approaches to overcome the barriers. According to 
survey results, a common problem states faced in developing SACWIS was 
receiving insufficient state funding for development. However, in our 
previous work on managing information technology, we found that the IT 
products can become obsolete in a matter of months rather than years, 
calling for more frequent investments in upgrades and 
enhancements.[Footnote 32] In addition, officials in Iowa told us that 
maintaining systems takes just as much money as building them. States 
did not report in our survey approaches for obtaining more funding for 
developing SACWIS, and few states reported developing strategies in an 
attempt to overcome the challenges associated with tight budgets for 
maintaining their systems. For example, Iowa officials engaged in 
careful planning with system users to ensure that they addressed the 
highest priorities when enhancing the system. In particular, the 
officials reported that maintaining tight control over the development 
and maintenance processes helps them avoid investing inordinate amounts 
of resources to make corrections to the system. In Oklahoma, child 
welfare officials reported that they relied on the contractors who 
developed their SACWIS to conduct ongoing maintenance activities until 
the contract expired in 2001. At that time, the agency hired some of 
the contract staff as full-time state employees to continue with the 
maintenance activities. State officials explained that this approach 
ensured continuity of service, in addition to saving the agency 
approximately $1 million each year. Similarly, few states reported on 
approaches to overcome the challenge of finding contractors with 
knowledge of child welfare practice. However, Iowa officials explained 
that once the contract staff are hired, they are required to attend the 
same training as new caseworkers to ensure that they are familiar with 
the state's child welfare policies and to familiarize themselves with 
casework practices.

States Use Strategies, such as Producing Reports That Identify Missing 
Data, in an Attempt to Improve the Reliability of the Data Reported to 
HHS:

Twenty-eight states reported using approaches to help caseworkers 
identify the data elements that are required for federal reporting and 
to help them better understand the importance of entering timely and 
accurate data. Ten states responding to our survey reported reviewing 
the federal reporting requirements in training sessions as a promising 
approach they use to improve data quality or as a lesson learned. For 
example, Tennessee reported that the state added a component about 
AFCARS to the initial and ongoing training workers receive about using 
SACWIS. The curriculum addresses the AFCARS report in general and the 
individual data elements to help the caseworkers better understand the 
purpose of collecting the information. In Nebraska, a "desk aid" that 
explains the data elements and where and why to enter them in the 
system is available on the caseworkers' computer desktops. In addition, 
New York has developed a step-by-step guide explaining to workers how 
NCANDS data should be entered, with references to the policy or statute 
requiring the information.

To improve data reliability, some states have designed their 
information systems with special features to encourage caseworkers to 
enter the information. Four states responding to our survey and 3 
states we visited designed their SACWIS with color-coded fields to draw 
attention to the data elements that caseworker are required to enter. 
For example, the AFCARS data fields in Oklahoma's system are coded red 
until the data are entered, after which the fields change to blue. In 
addition, workers can look at a single screen in the Oklahoma system to 
see what AFCARS data elements need to be completed without having to 
scroll through the entire case record. Colorado, Iowa, New York, and 
Oklahoma have built into their systems alerts--also known as 
"ticklers"--to remind caseworkers and supervisors of tasks that they 
need to complete. For example, in Iowa, alerts are sent to supervisors 
if a caseworker fails to enter the data necessary to complete a payment 
to a foster care provider. Whereas, in Oklahoma, a stoplight icon on 
the caseworker's computer desktop reminds the worker when tasks are 
due. A green light indicates that nothing is due within 5 days; a 
yellow light means that something is due within 5 days; and a red light 
means that something is overdue. Caseworkers and supervisors in the 
states we visited had mixed responses about the usefulness and 
effectiveness of the alerts. Some caseworkers found them to be a 
nuisance, while other caseworkers and supervisors found them to be 
useful tools in managing workloads and prioritizing daily tasks.

Six states reported that the best way to improve data quality was to 
use the data in published reports and hold the caseworkers and 
supervisors accountable for the outcomes of the children in their care. 
In addition, 6 states responding to our survey reported using the data 
available in their information systems to measure state outcomes 
similar to the CFSR. State officials reported that this approach is an 
effective way to get local offices invested in the quality of the data. 
For example, North Carolina publishes monthly reports for each county 
comparing their performance on state data indicators, such as the 
length of time children spend in care, to counties of similar size and 
the state as a whole. County officials reported that these reports 
encourage workers to improve the quality of the data collected and 
entered into the state system since their performance is being widely 
published and compared to other counties.

In addition, all the states we visited reported that frequent review of 
their data, such as using software from HHS to test their AFCARS and 
NCANDS data to pin-point data entry errors prior to submitting them to 
HHS, has helped improve data quality. When the states identify poor 
data, they alert the caseworkers and supervisors of needed corrections 
and data entry improvements. For example, Colorado runs these reports 
about 4 to 5 times a year, with one run occurring approximately 6 weeks 
before each AFCARS submission. When the data specialists find errors, 
they notify the caseworker to clean up the data. New York officials 
told us that they incorporate the results from these tests in training 
if a consistent pattern of errors is identified.

Conclusion:

While most states are developing statewide information systems, 
challenges with data reliability remain. Although SACWIS development is 
delayed in many states, state officials recognize the benefits of 
having a uniform system that enhances the states' ability to monitor 
the services provided and the outcomes for children in their care. 
Although states began reporting to NCANDS in1990 and were mandated to 
begin reporting to AFCARS in 1995, most states continue to face 
challenges providing complete, accurate, and consistent data to HHS. In 
addition, the results of more recent HHS efforts, such as conducting 
AFCARS-related focus groups, are unknown. Reliable data are essential 
to the federal government's development of policies that address the 
needs of the children served by state child welfare agencies and its 
ability to assist states in improving child welfare system 
deficiencies. Without well-documented, clearer guidance and the 
completion of more comprehensive reviews of states' AFCARS reporting 
capabilities, states are limited in overcoming challenges that affect 
data reliability. Because these challenges still remain, HHS may be 
using some questionable data as the foundation for national reports and 
national standards for the CFSR and may not have a clear picture of how 
states meet the needs of children in their care.

Recommendation to the Secretary of Health and Human Services:

To improve the reliability of state-reported child welfare data, we are 
recommending that the Secretary of HHS consider, in addition to HHS's 
recent efforts to improve AFCARS data, ways to enhance the guidance and 
assistance offered to states to help them overcome the key challenges 
in collecting and reporting child welfare data. These efforts could 
include a stronger emphasis placed on conducting AFCARS reviews and 
more timely follow-up to help states implement their improvement plans 
or identifying a useful method to provide clear and consistent guidance 
on AFCARS and NCANDS reporting.

Agency Comments:

We obtained comments on a draft of this report from the Department of 
Health and Human Services' Administration for Children and Families 
(ACF). These comments are reproduced in appendix III. ACF also provided 
technical clarifications, which we incorporated when appropriate.

ACF generally agreed with our findings and commented that the report 
provides a useful perspective of the problems states face in collecting 
data and of ACF's effort to provide ongoing technical assistance to 
improve the quality of child welfare data. In response to our 
recommendation, ACF said that we categorized its efforts as "recent" 
and did not recognize the long-term efforts to provide AFCARS and 
NCANDS related guidance to the states. Although we did not discuss each 
effort in depth, we do mention the agency's ongoing efforts in our 
report. However, we refer to the recent efforts in the recommendation 
in recognition of the agency's current activities to formally obtain, 
document, and incorporate feedback from the states with regard to 
collecting and reporting adoption and foster care data. ACF also noted 
in its comments that the data definitions need to be updated and 
revised and said it is currently in the process of revising the AFCARS 
regulations to further standardize the information states are to 
report--which we acknowledge in our report. In addition to the steps 
HHS is taking to further improve the AFCARS data, our recommendation 
encourages HHS to consider ways to enhance the ongoing guidance and 
assistance offered to states to help them overcome the key challenges 
in collecting and reporting child welfare data. ACF requested specific 
recommendations on approaches to overcome the difficulty of collecting 
and merging information from multiple state and county programs into a 
single national database. While there may be additional methodologies 
that the agency could use to overcome such challenges, our 
recommendation focuses on improving the guidance already offered to the 
states as a step to helping them better comply with the reporting 
requirements.

In addition, ACF added that although staff turnover in state child 
welfare agencies is a significant contributor to data quality issues, 
we did not focus on this as a significant factor. ACF also commented 
that it is firmly committed to continue to support the states and to 
provide technical assistance and other guidance as its resources will 
permit. However, because we recently issued a detailed report on a 
variety of caseworker issues,[Footnote 33] we primarily focused in this 
report on the key data entry challenges caseworkers face and refer 
readers to our previous work for additional information on challenges 
related to caseworker recruitment and retention and their affect on 
child welfare agencies. In commenting on our previous work prior to its 
release, HHS indicated that it does not have the authority to require 
states to address factors that contribute to staff turnover, such as 
high caseloads and said that it has limited resources to assist the 
states in the area of staff recruitment and retention.

ACF commented that it provided increased funding to the National 
Resource Centers in fiscal year 2003, which they believe will improve 
ACF's ability to provide assistance to the states. After receiving the 
draft report for comment, HHS separately provided information on an 
additional service the National Resource Center for Information 
Technology in Child Welfare provides to states. In an effort to assist 
states with improving the quality of their AFCARS data, the Resource 
Center will review states' programming code used for AFCARS data. As of 
June 2003, HHS reported that the Resource Center provided this 
assistance to Arkansas, Louisiana, Mississippi, North Carolina, Nevada, 
New Jersey, and Rhode Island, and 3 states--Maryland, Michigan, and 
Wisconsin--and the District of Columbia have requested the assistance.

In response to our survey methodology, ACF requested that we explain 
why the territory of Puerto Rico was not included in the state survey. 
Although Puerto Rico receives federal child welfare funds, we 
traditionally focus on the states and therefore do not include the U.S. 
territories, including American Samoa, the Commonwealth of the Northern 
Mariana Islands, Guam, Puerto Rico, and the Virgin Islands, in the 
scope of our reviews.

Finally, in response to our discussion of the AFCARS review process, 
ACF provided a few clarifications. During the course of our review, an 
HHS official characterized the AFCARS review process as relatively new 
and explained that the agency is still developing a process to respond 
to the states following the completion of the on-site review. When 
responding to a draft of this report, ACF disagreed with this 
characterization. ACF commented that the review process has been in 
place since 1996, pointing to the pilot reviews as evidence that the 
agency has a defined process. However, when we requested AFCARS reports 
for review, HHS explained that the states undergoing pilot reviews 
would be re-reviewed and that the official process was formalized in 
2001 with the release of an AFCARS review guide and the start of the 
official reviews. In addition, ACF commented that SACWIS reviews do not 
take priority over AFCARS reviews. However, officials had previously 
explained that although SACWIS and AFCARS reviews can happen at the 
same time, in practice, the AFCARS reviews are scheduled to occur in 
the states that are developing SACWIS after they have participated in a 
SACWIS review. Furthermore, ACF explained that states do not develop 
their improvement plan following the conclusion of the AFCARS review. 
Instead, ACF officials draft the plan for the state. Although state 
representatives had described a challenge in receiving timely feedback 
on their improvement plan, we have changed the language in the report 
to reflect ACF's comment.

We also provided a copy of our draft to child welfare officials in the 
5 states we visited--Colorado, Iowa, North Carolina, New York, and 
Oklahoma. Iowa and New York had technical clarifications, which we in 
incorporated when appropriate. Oklahoma provided additional 
information, which was incorporated. Colorado had no suggested 
corrections or edits. North Carolina did not provide any comments.

As agreed with your offices, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days 
after its issue date. At that time, we will send copies of this report 
to the Secretary of Health and Human Services, state child welfare 
directors, and other interested parties. We will make copies available 
to others on request. In addition, the report will be available at no 
charge on GAO's Web site at http://www.gao.gov. If you or your staff 
have any questions or wish to discuss this material further, please 
call me at (202) 512-8403 or Diana Pietrowiak at (202) 512-6239. Key 
contributors to this report are listed in appendix IV.

Cornelia M. Ashby: 
Director, Education, Workforce, and Income Security Issues:

Signed by Cornelia M. Ashby: 

[End of section]

Appendix I: Scope and Methodology:

To determine the progress states have made in developing Statewide 
Automated Child Welfare Information Systems (SACWIS), we surveyed all 
50 states and the District of Columbia through a Web-based survey. We 
pretested the survey instrument in Maryland and the District of 
Columbia. We received responses from 49 states and the District of 
Columbia. The state of Nevada did not respond to the survey. We 
discarded a question that asked states to report the date their advance 
planning document (APD) was approved by the Department of Health and 
Human Services (HHS). Due to a technical error, the date was truncated 
and a valid answer was not stored in the responses. Of the 50 survey 
responses, 46 were from states that are pursuing SACWIS development. 
The 4 states not developing SACWIS were asked to skip sections of the 
survey that asked about SACWIS development, system modifications, and 
supported services and links. We did not independently verify the 
survey responses. In addition, we visited 5 states to obtain more 
detailed and qualitative information regarding states' experiences 
developing SACWIS. We conducted site visits in Colorado, Iowa, New 
York, North Carolina, and Oklahoma. We selected these states to 
represent a range of SACWIS development stages, sizes of foster care 
populations, and geographic locations. During our site visits, we 
interviewed state and local child welfare staff, state and local staff 
that regularly exchange information with the child welfare agency, and 
private contractors. We also spoke with HHS staff in the central and 
regional offices, National Resource Center officials, contractors 
involved in SACWIS development, and child welfare experts from the 
Child Welfare League of America and the American Public Human Services 
Association.

To determine how states and HHS ensure reliable data exist on children 
served by child welfare agencies we surveyed states using the above-
mentioned survey instrument. In addition, we interviewed state and HHS 
officials on their efforts to analyze and compile data and HHS's role 
in providing technical assistance to states. We spoke with state 
officials during our site visits and HHS officials in the central and 
regional offices and attended the 6th National Child Welfare Data 
Conference. We obtained and reviewed available SACWIS and Adoption and 
Foster Care Analysis and Reporting System (AFCARS) reports. At the time 
of our review, HHS had conducted 26 SACWIS reviews. We obtained and 
reviewed 23 reports. The remaining reports were not available for 
review because HHS has not yet completed the report or shared the 
results with the state. Most of the SACWIS reports were considered 
drafts since many states are in the process of resolving issues with 
completing their systems. We reviewed AFCARS assessment reports from 6 
of the 8 states assessed by HHS--Arkansas, Connecticut, New Mexico, 
Texas, Vermont, and Wyoming. HHS conducted reviews in Delaware and West 
Virginia after we completed our analysis. We did not review any of the 
eight pilot review reports since these were not final reports and HHS 
plans to conduct official reviews in these states. These AFCARS 
assessment reports were analyzed to understand the breadth of on-site 
assistance HHS provides to states during the review and to identify 
common data collection and reporting difficulties among states. 
Finally, we talked with officials in 6 of the 8 states that had an 
AFCARS review about their experiences during the review and child 
welfare experts.

To identify practices state and local child welfare agencies are using 
to help ensure the accuracy, timeliness, and completeness of child 
welfare data we interviewed state and local child welfare officials on 
our site visits and inquired about the practices they are employing. We 
also included questions on practices and lessons learned in our survey. 
In addition, we spoke with numerous child welfare experts, including 
individuals from the National Resource Center for Information 
Technology in Child Welfare, the Child Welfare League of America, and 
the American Public Human Services Association[Footnote 34].:

[End of section]

Appendix II: State SACWIS Stages of Development:

[See PDF for image]

Source: HHS.

Note: Status is as of May 22, 2003.

[End of table]

[End of section]

Appendix III: Comments from the Department of Health and Human Services:

DEPARTMENT OF HEALTH & HUMAN SERVICES:

ADMINISTRATION FOR CHILDREN AND FAMILIES Office of the Assistant 
Secretary, Suite 600:

370 L'Enfant Promenade, S.W. Washington, D.C. 20447:

DATE: JUL 2 2003:

TO: Cornelia M. Ashby:

Director, Education, Workforce, and Income Security Issues:

FROM: Wade F. Horn, Ph.D. /,4Z 72 Assistant Secretary:

for Children and Families:

[See PDF for image]

[End of figure]

SUBJECT: Comments on the GAO Draft Report: "Most States are Developing 
Statewide Information Systems, but the Reliability of Child Welfare 
Data Could be Enhanced" (GAO-03-809):

Attached are the Administration for Children and Families' comments on 
the GAO Draft Report: "Most States are Developing Statewide Information 
Systems, but the Reliability of Child Welfare Data could be Enhanced" 
(GAO-03-809).

We appreciate the opportunity to comment on this report. Should you 
have any questions or need additional information, please contact Susan 
Orr, Associate Commissioner of the Children's Bureau at (202) 205-8618.

Attachment:

COMMENTS OF THE ADMINISTRATION FOR CHILDREN AND FAMILIES (ACF) ON THE 
GENERAL ACCOUNTING OFFICE'S (GAO) DRAFT REPORT: "MOST STATES ARE 
DEVELOPING STATEWIDE INFORMATION SYSTEMS, BUT THE RELIABILITY OF CHILD 
WELFARE DATA COULD BE ENHANCED," GAO-03-809:

The Administration for Children and Families (ACF) appreciates the 
opportunity to comment on the GAO's draft report.

GAO Recommendation:

To improve the reliability of state-reported child welfare data, GAO 
recommends that the Secretary of HHS consider, in addition to HHS's 
recent efforts to improve AFCARS data, ways to enhance the guidance and 
assistance offered to states to help them overcome the key challenges 
in collecting and reporting child welfare data. These efforts could 
include a stronger emphasis placed on conducting the Adoption and 
Foster Care Analysis and Reporting System (AFCARS) reviews and more 
timely follow-up to help states implement their improvement plans or 
identifying a useful method to provide clear and consistent guidance on 
AFCARS and the National Child Abuse and Neglect Data System (NCANDS) 
reporting.

ACF Comments:

The background provided in the report was useful in giving a 
perspective of the problem states face in collecting data and ACF's 
effort to provide ongoing technical assistance to improve the quality 
of that data. However, the GAO categorizes ACF's efforts as "recent" 
and does not recognize the long-term efforts to provide AFCARS and 
NCANDS related guidance to the states. We recommend that the term 
"ongoing" be substituted for the term "recent" in the "Recommendation" 
section of the report.

ACF has provided extensive guidance on how states can improve the 
quality of their AFCARS data since before the first AFCARS report was 
submitted. To categorize this effort as "recent" might be misconstrued 
by the reader. What follows is a short list of some of the guidance 
provided by this agency:

* State Technical Advisory Group:

March 1994 
May 1994 
March 1995:

* Statewide Automated Child Welfare Information Systems (SACWIS) 
Conferences: AFCARS sessions were held during these conferences to 
provide assistance and updates to states:

September 1994 
September 1995:

* ACF User Group Meetings: Held two times a year since 1994. AFCARS 
workshops were conducted as well as one-on-one meetings with states to 
provide technical assistance.

* AFCARS Conferences/Meetings:

State Technical Assistance Workshops: San Francisco and Washington D.C. 
- July & September 1994:

State Technical Assistance Workshops: San Francisco and Washington D.C. 
- July & September 1995:

SACWIS/AFCARS Conference - 1996 AFCARS Data Conference -1997 AFCARS 
Data Conference -1998 AFCARS Data Conference -1999 AFCARS Data 
Conference -2000 AFCARS Data Conference -2001 AFCARS Data Conference -
2002 AFCARS Data Conference -2003:

* Technical Assistance Documents:

AFCARS: Part I - Strategic Planning Guide: A guide for state systems 
planning and development; Part II - Data Dictionary.

AFCARS Guide to the Elements - September 1995 (Rescinded):

NRC-ITCW AFCARS Toolkit (Available in draft form, final version to be 
released summer of 2003):

Disability Code Table - Last updated 10/30/01:

State Data Compliance Software - First issued in 1995 State Data 
Quality Software - First issued in 1996 Statistical package of 
programming logic for the State Data Profile - 1998:

* Policy Guidance:

Policy Interpretation Questions (PIQ) March 1994 Policy Interpretation 
Questions (PIQ March 1995 ACYF-PI-CB-95-09 (REVISED), issued 5/23/95: 
Methods of Extracting Data for Submission to AFCARS:

ACYF-CB-IM-00-04, issued March 30, 2000: Adoption and Foster Care 
Analysis and Reporting System Re-issuance of Technical Bulletins ACYF-
CB-PI-01-07, issued 5/4/2001: Change in the State AFCARS File Name 
Child Welfare Policy Manual: September 24, 2001 (Replaces the two PIQs 
for AFCARS):

ACYF-CB-IM-02-03, issued April 8, 2002: Decision regarding AFCARS 
Penalties, AFCARS File Name:

ACYF-CB-11\1-02-05, issued April 22, 2002: AFCARS Assessment Reviews:

ACF has issued several Technical Bulletins since 1994. These have been 
updated and modified as necessary.

[See PDF for image]

[End of table]

As noted in the GAO report, we concur with the general finding that 
states face significant challenges as they strive to improve the 
quality of the data that they report to the federal government. 
However, we are concerned that the report does not focus more attention 
on the continuing problem that the turnover of state and contractor 
staff has on the quality of data reported to the federal government. 
The GAO report briefly mentions the impact that the loss of social 
workers and contractor technical staff has on the quality of data, but 
the report does not focus on this as a significant factor. ACF has 
found that staff turnover to be a significant contributor to data 
quality issues. New social workers must be trained on how to 
effectively use the systems and what elements are important from a data 
requirement perspective, while new technical staff must re-learn the 
intricacies of the national reporting requirements. ACF remains firmly 
committed to continue to support state technical staff and to provide 
technical assistance and other guidance as our resources will permit. 
We are continuously exploring new opportunities to extend our limited 
and shrinking resources to provide the types of technical assistance 
suggested in the GAO report to states and their staffs.

Other Comments:

Page 1, first paragraph, third sentence.

"Through these federal matching funds, states can develop and implement 
Statewide Automated Child Welfare Information Systems (SACWIS) to 
manage their child welfare cases as well as to report child abuse and 
neglect, foster care, and adoption information to the federal 
government.":

This sentence could be enhanced by noting that states have the option 
of building SACWIS or a non-SACWIS system to support their business 
processes. The GAO should consider replacing the current sentence with 
the following:

"Through these federal matching funds, states can develop and implement 
information systems to manage their child welfare cases as well as to 
report child abuse and neglect, foster care, and adoption information 
to the federal government. If the state elects to pursue implementing a 
comprehensive case management system to support the different child 
welfare programs that are administered by the state, the state may 
elect to develop a system that meets the requirement for a Statewide 
Automated Child Welfare Information System (SACWIS). If a state elects 
to implement a SACWIS-compliant system, the state is eligible to 
receive additional funding to build and operate that system. Regardless 
of the type of system that a state elects to implement, child welfare 
caseworkers at the county or local level are the key personnel who 
collect and document information on children and families served by 
child welfare agencies, in addition to performing a wide range of 
services to protect children, such as investigating child abuse and 
neglect reports or providing support services to maintain the children 
in their homes.":

* Page 3, second paragraph, last sentence and page 4, first sentence. We 
are concerned about the example cited at the bottom of the page, which 
indicates that one state waited a year for technical assistance.

"For example, state officials in Oklahoma said that it took HHS close 
to one year to provide written guidance on how to report the amount of 
monthly adoption assistance subsidies. As a result, the state may have 
misreported the amount of money the state awarded in subsidy payments 
to adoptive families.":

We question whether the summary of the concern really conveys what 
happened. This is particularly true because issues or questions 
concerning the adoption assistance payment data elements are not 
generally complex or difficult to answer.

Page 12, first paragraph, second complete sentence.

"As a result, Iowa designed a SACWIS to reflect this work process by 
linking two databases - one to record child abuse and neglect 
information and one to record ongoing case records - that share 
information with one another.":

We recommend that GAO modify this sentence to read as follows:

"As a result, Iowa designed its SACWIS to have an interface with the 
state child abuse and neglect data system so that information can be 
shared between the two systems. Most states have built functionality 
that supports the investigation of child abuse and neglect allegations 
into their SACWIS.":

ACF does not view the child abuse and neglect system in Iowa as part of 
its SACWIS. However, the state has met the SACWIS requirements in this 
area by building an interface between the two systems.

* Page 12, second paragraph, first sentence.

"Forty-three states reported in our survey that they have or are 
planning to incorporate child welfare and administrative components 
into their SACWIS (see Table 3).":

This sentence is somewhat confusing since many of the functional areas 
noted in the reference on page 13, Table 3, must be included in a 
SACWIS for it to be considered SACWIS-compliant. The issue is further 
complicated because some of the required functional areas identified in 
the table may be addressed through an interface to another system, 
while some must be included within the SACWIS application. The way this 
section is written, it may leave a reader with the impression that 
some states are building SACWIS systems that do not 
meet all of the SACWIS requirements. We recommend that the first 
sentence in this paragraph be modified as follows:

"Forty three states reported ... they are planning to incorporate 
optional child welfare and administrative components into their SACWIS 
(see Table 3).":

This approach would probably necessitate the addition of a footnote 
explaining that some SACWIS components are mandatory while others are 
optional. Also, please see our comments below related to Table 3.

* Page 12, second paragraph, second sentence.

"Some of these components, such as child protection and determining IV-
E eligibility, are required by SACWIS regulations, to the extent 
practicable.":

The phrase "to the extent practicable" is used in the regulation and 
statute in relation to certain mandatory interfaces. The use of the 
phrase here is incorrect and it should be deleted.

* Page 12, second paragraph, second sentence from bottom.

"Currently, 27 states reported in our survey that they are at some 
stage of using their SACWIS to track independent living services, and 
an additional 14 states plan to include this component in their system 
in preparation for the legislative requirements.":

The word "legislative" should be deleted. ACF is responsible for 
promulgating any National Youth in Transition Database (NYTD) 
requirements.

* Page 13, Table 3. As previously noted, this table may leave the reader 
with the misunderstanding that some states are building non-compliant 
SACWIS systems. We provide the table below to identify our concerns.

At the very least, we recommend that the GAO add the following 
disclaimer to this table: "HHS has indicated that a state's failure to 
support most of these functional components would be a problem that the 
state would need to correct.":

[See PDF for image]

[End of figure]

Page 14, last sentence. This sentence states that North Dakota did not 
report in the GAO survey the reason for stopping SACWIS development. It 
is our understanding that North Dakota stopped its SACWIS initiative 
due to a combination of issues, including programmatic structure.

Page 17, Table 6, titled, "Number of Months States Delayed in SACWIS 
Development," indicates states were asked to report the number of 
months the delays exceeded the time line outlined in their Advance 
Planning Document (ADP). Based on our experience with the states 
identified in Table 6, we believe that the respondents may have replied 
to the survey using different definitions of what "delayed" meant and 
different methodologies for counting the number of months that they 
were delayed. ACF recommends that GAO explore ways to enhance the 
reliability of the data presented in its report.

Page 23, footnote 13. This footnote states, "The analysis of survey 
responses about reporting data to HHS is based on responses from 49 
states and the District of Columbia." The GAO should consider 
explaining why Puerto Rico was excluded from the survey.

Page 24. The GAO should consider enhancing this section of the report 
to clarify the concerns being described. For example, the first two 
sentences state:

"Almost every state responding to our survey and all the states we 
visited 
reported that insufficient training for caseworkers and inaccurate and 
incomplete data entry affect the quality of the data reported to AFCARS 
and NCANDS (see fig. 1.). Although most states did not report that 
insufficient training contributed to inaccurate and incomplete data 
entry, HHS and the states we visited found that these issues are often 
linked.":

The first and second sentences appear to be contradictory. In sentence 
one the report indicates that the states reported that insufficient 
training for caseworkers affects the quality of the data, while the 
second sentence notes that most states did not report this to be the 
case. We recommend that the GAO enhance this area of its report to 
describe what was found. Furthermore, if this conclusion is primarily 
supported by empirical analysis, the GAO should consider downplaying 
the importance of this finding.

Page 25, Figure 1 and Page 33, Figure 2. To clarify the survey results, 
we recommend that the bar chart include demarcation lines for states 
indicating the various factors had "some" or a "moderate" affect on the 
quality of data. The current presentation approach appears to present a 
more negative view of the impact these factors have on the quality of 
data than the survey questions may have solicited.

* Page 26, first paragraph, first sentence.

"Caseworkers were inaccurately recording a child's race as unknown even 
though this option should be selected only if the child's parents or 
relatives cannot provide the information, such as when a child is 
abandoned.":

The finding from AFCARS reviews indicates that information is often 
defaulted to the response of "unable to determine" in order for the 
element not to fail the missing data standard, not that workers are 
recording "unknown.":

Pages 27 and 28. Having identified the difficulty of collecting 
information from multiple state and county programs into a single 
national database, ACF requests that GAO recommend solutions for this 
concern. Specifically, we request that the GAO describe methodologies 
on how to merge divergent data from multiple state and county programs 
with different program requirements and data definitions into a single 
common database in a manner that will result in better data. We welcome 
suggestions for an approach that would be more efficient, effective and 
economical than the approach currently used.

* Pages 28 through 35. ACF has increased the accessibility of technical 
assistance resources available to the states through its support of the 
National Resource Centers (NRCs), specifically the National Resource 
Center for Information Technology in Child Welfare (NRC-ITCW). The 
Children's Bureau (CB) increased funding to the NRC for FY 2003. The 
NRC-ITCW provides substantial technical assistance to states in data-
related matters such as programming, input, interpretation, analysis, 
and reporting for AFCARS and NCANDS data. The increased funding will 
improve ACF's ability to provide this assistance to our state partners.

Concerning the states' comment on the lack of state input in suggesting 
changes to AFCARS, it should be noted that AFCARS is defined in 
regulation. Therefore, it cannot be changed or modified without a 
process of public comment and a regulation change.

All publicly available AFCARS documentation is accessible through ACF's 
web page. In order to enable self-sufficiency and to sustain the 
state's learning curve, state staff are asked to first read the 
guidance available through ACF's web page. In most instances, the 
available guidance addresses their concern and we do not hear back from 
them.

* Pages 32 and 33. ACF issued policy clarifications regarding placement 
information on July 5, 2002. Additionally, the AFCARS report is in the 
regulation process to further standardize the information that states 
are to report. We recognize that the definitions need to be updated and 
revised.

* Page 32, first paragraph, fourth sentence. The statement, "However, 
HHS officials reported that they are not mandated to conduct the AFCARS 
reviews and that priority is placed on other reviews, such as the CFSR 
and SACWIS review," is incorrect. SACWIS reviews do not take precedence 
over AFCARS reviews.

* Page 34. ACF wishes to advise the GAO that information pertaining to 
the AFCARS review process, specifically the improvement plan, is 
incorrect. States do not submit a proposed AFCARS improvement plan 
(AIP). The AIP is included in the final report to the state following 
an AFCARS assessment review (AAR). The AIP is based on the findings of 
the AAR and indicates how states are to proceed to correct the errors. 
States are given an opportunity to review the preliminary findings on-
site and make corrections, negotiate changes, and ask for clarification 
at that time. The findings in the final report reflect the preliminary 
findings. In some instances, additional findings are made regarding the 
quality of the data after an analysis of the case file review is 
completed. States are to provide ACF with estimated due dates for the 
tasks listed in the AIP.

Page 34, first paragraph, fifth sentence. This sentence, indicating 
that AFCARS reviews are relatively new, is incorrect. Reviews have been 
conducted either as pilots or actual reviews since 1996. The statement 
that the agency is still developing a process to respond to the states 
is also not accurate. There is a defined process for assessing state 
progress during the AFCARS improvement phase. This process was not part 
of the pilot program; therefore, it is being fine tuned and is a fluid 
process. Since AFCARS reviews are not mandatory, work related to the 
reviews on the federal level has to be adjusted to accommodate work on 
mandated priority items.

Page 34, first paragraph, sixth sentence. This section discusses HHS 
taking steps to gather comments from states and should be a separate 
paragraph. This is a separate topic than what is being discussed in the 
paragraph where it is located.

* Page 35, footnote 22 has a few errors.

1) The first sentence should be changed, as the juvenile justice agency 
does not claim federal title IV-E funds. The title IV-E agency claims 
costs incurred by the juvenile justice agency.

2) The second sentence indicates that delinquents who meet IV-E 
eligibility criteria and "present with child protection and/or 
dependency issues, in addition to their delinquent status, may be 
eligible for title IV-E foster care." This sentence should be revised 
to read that delinquents who meet title N-E eligibility criteria are 
eligible for the program.

3) The following clarification should be added at the end of the last 
sentence: "for children who are receiving title IV-E funding or for 
children in foster care under the placement and care responsibility of 
the child welfare agency." As the sentence is written, it indicates 
that all adjudicated delinquents must receive IV-E/IV-B protections, 
but that is only true if they are receiving IV-E funds or meet our 
definition of foster care.

There are a couple of states that have an umbrella agency where the IV-
B/IV-E agency has responsibility for all juvenile justice children. 
Therefore, although those children may not receive a foster care 
maintenance payment, the state may still have to meet the state plan 
requirements and protections for those youth if they are in foster 
care.

[End of section]

Appendix IV: GAO Contacts and Acknowledgments:

GAO Contacts:

Diana Pietrowiak, (202) 512-6239 Sara L. Schibanoff, (202) 512-4176:

Staff Acknowledgments:

In addition to those named above, Leah DeWolf and Rachel Seid made key 
contributions to this report. Avrum Ashery, Patrick DiBattista, Barbara 
Johnson, Valerie Melvin, and Rebecca Shea also provided key technical 
assistance.

[End of section]

Bibliography:

The American Public Welfare Association. Statewide Automated Child 
Welfare Information Systems: Survey of State Progress. Washington, 
D.C., July 1997.

The American Public Welfare Association. Child Welfare Information 
Systems: Some Concepts and Their Implications. Washington, D.C., July 
1994.

The American Public Welfare Association. Survey of State Child Welfare 
Information Systems: Status of AFCARS and SACWIS. Washington, D.C., 
April 1995.

Caliber Associates, Analysis of State Child Welfare Data: VCIS Survey 
Data from 1990 through 1994, May 1998, Department of Health and Human 
Services.

Center for Technology in Government, University of Albany, SUNY. Tying 
a Sensible Knot: A Practical Guide to State-Local Information Systems. 
Albany, N.Y., June 1997.

Child Welfare League of America. National Working Group Highlights, 
"Child Maltreatment in Foster Care: Understanding the Data." 
Washington, D.C., October 2002.

Child Welfare League of America. National Working Group Highlights, 
"Placement Stability Measure and Diverse Out-of-Home Care Populations." 
Washington, D.C., April 2002.

U.S. Department of Health and Human Services, Administration for 
Children and Families, Administration on Children, Youth and Families, 
Children's Bureau. Child Maltreatment 2001. Washington, D.C., 2003.

U.S. Department of Health and Human Services, Administration for 
Children and Families, Administration on Children, Youth and Families, 
Children's Bureau. Child Welfare Outcomes 1999: Annual Report. 
Washington, D.C., n.d.

U.S. Department of Health and Human Services, Office of Inspector 
General. Adoption and Foster Care Analysis and Reporting System 
(AFCARS): Challenges and Limitations. Washington, D.C., March 2003.

[End of section]

Related GAO Products:

Child Welfare and Juvenile Justice: Federal Agencies Could Play a 
Stronger Role in Helping States Reduce the Number of Children Placed 
Solely to Obtain Mental Health Services. GAO-03-397. Washington, D.C.: 
April 21, 2003.

Child Welfare: HHS Could Play a Greater Role in Helping Child Welfare 
Agencies Recruit and Retain Staff. GAO-03-357. Washington, D.C.: March 
31, 2003.

Human Services: Federal Approval and Funding Processes for States' 
Information Systems. GAO-02-347T. Washington, D.C.: July 9, 2002.

Foster Care: Recent Legislation Helps States Focus on Finding Permanent 
Homes for Children, but Long-Standing Barriers Remain. GAO-02-585. 
Washington, D.C.: June 28, 2002.

Human Services Integration: Results of a GAO Cosponsored Conference on 
Modernizing Information Systems. GAO-02-121. Washington, D.C.: January 
31, 2002.

District of Columbia Child Welfare: Long-Term Challenges to Ensuring 
Children's Well-Being. GAO-01-191. Washington, D.C.: December 29, 2000.

Child Welfare: New Financing and Service Strategies Hold Promise, but 
Effects Unknown. GAO/T-HEHS-00-158. Washington, D.C.: July 20, 2000.

Welfare Reform: Improving State Automated Systems Requires Coordinated 
Federal Effort. GAO/HEHS-00-48. Washington, D.C.: April 27, 2000.

Foster Care: States' Early Experiences Implementing the Adoption and 
Safe Families Act. GAO/HEHS-00-1. Washington, D.C.: December 22, 1999.

Foster Care: HHS Could Better Facilitate the Interjurisdictional 
Adoption Process. GAO/HEHS-00-12. Washington, D.C.: November 19, 1999.

Foster Care: Effectiveness of Independent Living Services Unknown. GAO/
HEHS-00-13. Washington, D.C.: November 5, 1999.

Foster Care: Kinship Care Quality and Permanency Issues. GAO/HEHS-99-
32. Washington, D.C.: May 6, 1999.

Juvenile Courts: Reforms Aim to Better Serve Maltreated Children. GAO/
HEHS-99-13. Washington, D.C.: January 11, 1999.

Child Welfare: Early Experiences Implementing a Managed Care Approach. 
GAO/HEHS-99-8. Washington, D.C.: October 21, 1998.

Foster Care: Agencies Face Challenges Securing Stable Homes for 
Children of Substance Abusers. GAO/ HEHS-98-182. Washington, D.C.: 
September 30, 1998.

Managing Technology: Best Practices Can Improve Performance and Produce 
Results. GAO/T-AIMD-97-38, January 31, 1997.

Child Welfare: HHS Begins to Assume Leadership to Implement National 
and State Systems. GAO/AIMD-94-37. Washington, D.C.: June 8, 1994.

Executive Guide: Improving Mission Performance Through Strategic 
Information Management and Technology. GAO/AIMD-94-115. May 1, 1994.

FOOTNOTES

[1] Throughout this report, references to state survey responses 
include the District of Columbia. Forty-six of these states reported 
that they are developing or operating a SACWIS. Nevada, which HHS 
reported has an operational SACWIS, did not respond to our survey.

[2] When states choose to develop information systems that include 
other human services, such as food stamps, child support enforcement, 
or Medicaid, states must submit APDs to each cognizant federal agency. 
In a hearing held last year before the Subcommittee on Technology and 
Procurement Policy, House Committee on Government Reform, we testified 
that the federal agencies do not have systems to monitor states 
requests for federal approval and funding through the life cycle of a 
state request. (See U.S. General Accounting Office, Human Services: 
Federal Approval and Funding Processes for States' Information System, 
GAO-02-347T (Washington, D.C: July 9, 2002.))

[3] Forty-four states provided information on the total amount of 
federal funds they received to develop and operate SACWIS. Alaska, 
Hawaii, Missouri, North Carolina, Texas, and Vermont did not report 
federal funding information. Nevada did not respond to our survey. 
State-reported figures may include some funding allocated in fiscal 
year 2003 since the survey was issued in October 2002 and completed as 
late as December 2002. 

[4] Forty-four states provided information on the total amount of state 
funds used to develop and operate SACWIS. Arkansas, Hawaii, Missouri, 
North Carolina, Texas, and Vermont did not report state funding 
information. Nevada did not respond to our survey. State-reported 
figures may include some funding allocated in fiscal year 2003 since 
the survey was issued in October 2002 and completed as late as December 
2002. 

[5] This figure includes developmental funds allocated by HHS to 49 
states and the District of Columbia. Hawaii did not take any federal 
money for SACWIS development. 

[6] This figure includes operational funds allocated to 35 states. 
States begin claiming operational costs when some or all components of 
their SACWIS are operating in local offices. Operational activities 
include routine maintenance, minor enhancements, and other changes that 
do not significantly increase or modify the functionality of the 
system.

[7] According to HHS officials, prior to fiscal year 2000, states 
reported SACWIS operational expenses as part of their Title IV-E 
administrative expenses because the claims sheet states used for 
reporting did not have a separate column for SACWIS operational 
expenditures. In fiscal year 2000, states were required to use a claims 
sheet that was reformatted to provide space for SACWIS operational 
expenditures. In addition, an HHS official explained that the 
difference between the state-reported figures and the federal figures 
may be due to states claiming some SACWIS expenses under different 
programs, such as Title IV-E administrative funds, rather than 
separately as SACWIS expenses.

[8] Although the Iowa state officials described their SACWIS as 
including the child abuse and neglect system, HHS commented on a draft 
of this report that it does not view the child abuse and neglect system 
as part of the state's SACWIS. However, HHS said that the state has met 
the SACWIS requirement in this area by building an interface between 
the two systems. 

[9] The Foster Care Independence Act of 1999 increased federal support 
to states for independent living programs--programs designed to assist 
youth who are identified as likely to remain in foster care until age 
18. Independent living services can include education or training 
necessary for the youth to obtain employment. 

[10] Twelve of the 46 states reporting that they are developing or 
operating a SACWIS reported that they have not experienced delays in 
developing their systems. In response to the length of the delays 
reported by 26 states in our survey, ACF commented on a draft of this 
report that these states may be using different definitions in defining 
their delays. However, ACF did not provide further information on how 
the delays represented in this report differ from its perception of 
states' experiences. In our survey, we asked states to report on the 
delays that exceeded the time line outlined in their initial APD. 

[11] With regard to the budget difficulties that states reported 
facing, since 1994 the federal government has made a commitment to help 
states develop and maintain their SACWIS by matching 75 percent of 
states' development funds through 1997 and providing an ongoing match 
of 50 percent of state funding for the development and maintenance of 
their systems. However, since the states' legislatures must make the 
initial commitment to fund SACWIS, the federal government cannot assist 
state child welfare agencies with this challenge.

[12] In commenting on a draft of this report, HHS indicated that a Web 
resource is available to states interested in learning about other 
states' efforts to develop human services--child welfare, food stamps, 
Temporary Assistance to Needy Families, child care, and child support 
enforcement--information systems at http://www.acf.hhs.gov/nhsitrc.

[13] Data are reliable when they are complete and accurate. A 
subcategory of accuracy is consistency. Consistency refers to the need 
to obtain and use data that are clear and well-defined enough to yield 
similar results in similar analysis. See U.S. General Accounting 
Office, Assessing the Reliability of Computer-Processed Data, 
GAO-02-15G (Washington, D.C.: Sept. 2002).

[14] States were asked the extent to which certain problems may 
decrease the quality of the data submitted to AFCARS and NCANDS using 
the following scale: very great, great, moderate, some, and no affect. 

[15] The analysis of survey responses about reporting data to HHS is 
based on responses from 49 states and the District of Columbia. All 
states, regardless of SACWIS development, were asked to complete these 
questions. 

[16] We reviewed AFCARS reports from 6 of the 8 states assessed by HHS-
-Arkansas, Connecticut, New Mexico, Texas, Vermont, and Wyoming. HHS 
conducted reviews in Delaware and West Virginia after we completed our 
analysis.

[17] In commenting on a draft of this report, ACF said that the finding 
from the AFCARS reviews indicates that information is often defaulted 
to the response "unable to determine" in order for the element not to 
fail the missing data standard, not that workers are recording 
"unknown"; however, the report findings we used in this analysis 
instruct states to fix the defaults and address caseworker practice by 
enhancing training on the correct use of "unable to determine" when 
noting a child's race.

[18] See U.S. General Accounting Office, Child Welfare: HHS Could Play 
a Greater Role in Helping Child Welfare Agencies Recruit and Retain 
Staff, GAO-03-357 (Washington, D.C.: Mar. 31, 2003).

[19] Department of Health and Human Services, Office of Inspector 
General, Adoption and Foster Care Analysis and Reporting System 
(AFCARS): Challenges and Limitations, OEI-07-01-00660 (Washington, 
D.C.: Mar. 2003).

[20] Guardianship arrangements occur when permanent legal custody of a 
child is awarded to an individual, such as a relative, but the child is 
not legally adopted.

[21] Child Welfare League of America. National Working Group 
Highlights, "Placement Stability Measure and Diverse Out-of-Home Care 
Populations" (Washington, D.C., Apr. 2002).

[22] Respite care provides temporary childcare for children away from 
their caretakers.

[23] Although the findings from the Inspector General's report and our 
study are more recent, ACF commented on a draft of this report that it 
issued policy clarifications regarding placement information on July 5, 
2002.

[24] A rating of 2 indicates that the state's information system does 
not fully meet the requirements for AFCARS reporting, whereas if a 
state receives a 3, the information system requirements for AFCARS 
reporting are in place, but there are data entry problems affecting the 
quality of data. According to an HHS official, data elements that have 
a combination of technical and data entry problems are rated as 2 until 
the technical issues are resolved. HHS will then rate the element as a 
3 until the data entry practices are changed. A state receives a rating 
of 4 if in compliance with the AFCARS requirements or a 1 if the 
requirement is not addressed in the state's information system.

[25] Federal guidance states that children who are (1) removed from 
their home and initially placed in a juvenile justice facility are not 
to be included in AFCARS reporting; (2) in a foster care setting who 
are moved to a juvenile justice facility and who are expected to be 
returned to a foster care setting should continue to be included in the 
AFCARS reporting population; and (3) in a foster care setting and are 
moved to a juvenile justice facility and who become the responsibility 
of another agency should not be included in the AFCARS reporting 
population. However, these studies show that the arrangements between 
child welfare and juvenile justice agencies vary, which affect the 
population of children reported to AFCARS. For example, some state 
child welfare agencies have responsibility for all children in the 
juvenile justice system and include these children in their AFCARS 
reports, while other states only report children who are in the custody 
of the juvenile justice system, but receiving Title IV-E funding. 

[26] According to HHS, state child welfare agencies can claim Title IV-
E foster care maintenance funds for eligible children for costs 
incurred by juvenile justice agencies that have entered into an 
agreement with the child welfare agencies. Delinquent children served 
by these agencies who meet the Title IV-E eligibility criteria are 
eligible for Title IV-E foster care maintenance funds. States must meet 
all Titles IV-B and IV-E program and/or eligibility requirements with 
respect to the children who are adjudicated delinquent and are 
receiving Title IV-E funding or for children in foster care under the 
placement and care responsibility of the child welfare agency. 

[27] Children in the juvenile justice system may enter care because of 
behavior problems and return home when the behavior is controlled. 
However, the problem may arise again and the child could re-enter state 
custody. Also, the children may experience a greater number of 
placements while in foster care because of their behavior problems or a 
"step-down" approach that some states use to gradually decrease the 
level of security required to care for the children.

[28] See U.S. General Accounting Office, Executive Guide: Improving 
Mission Performance Through Strategic Information Management and 
Technology, GAO/AIMD-94-115 (Washington, D.C.: May 1, 1994); Center for 
Technology in Government, University of Albany, SUNY. Tying a Sensible 
Knot: A Practical Guide to State-Local Information Systems. Albany, 
N.Y., June 1997.

[29] The Child Support Enforcement Program is a joint federal, state, 
and local partnership that was established in 1975 under Title IV-D of 
the Social Security Act. Each state runs a child support program, which 
provides four major services: locating non-custodial parents, 
establishing paternity, establishing child support obligations, and 
collecting child support for families.

[30] In 1996, the Congress created the block grant Temporary 
Assistance for Needy Families program replacing the Aid to Families 
with Dependent Children (AFDC) and related welfare programs. States 
were given increased flexibility in designing the eligibility criteria 
and benefit rules, which require work in exchange for time-limited 
benefits.

[31] See U.S. General Accounting Office, Welfare Reform: Improving 
State Automated Systems Requires Coordinated Federal Effort, GAO/
HEHS-00-48 (Washington, D.C.: Apr. 27, 2000).

[32] See U.S. General Accounting Office, Managing Technology: Best 
Practices Can Improve Performance and Produce Results, GAO/T-AIMD-97-38 
(Washington, D.C.: Jan. 31, 1997).

[33] See U.S. General Accounting Office, Child Welfare: HHS Could Play 
a Greater Role in Helping Child Welfare Agencies Recruit and Retain 
Staff. GAO-03-357. (Washington, D.C.: Mar. 31, 2003).

[34] The Child Welfare League of America is an association of almost 
1,200 public and private nonprofit agencies that assist abused and 
neglected children and their families with a wide range of services. 
The American Public Human Services Association is an organization of 
individuals and agencies concerned with human services, including state 
and local human service agencies and individuals who work in or 
otherwise have an interest in human service programs.

GAO's Mission:

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony:

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading.

Order by Mail or Phone:

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:

U.S. General Accounting Office

441 G Street NW,

Room LM Washington,

D.C. 20548:

To order by Phone: 	

	Voice: (202) 512-6000:

	TDD: (202) 512-2537:

	Fax: (202) 512-6061:

To Report Fraud, Waste, and Abuse in Federal Programs:

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470:

Public Affairs:

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.

General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.

20548: