This is the accessible text file for GAO report number GAO-04-267T 
entitled 'Child Welfare: States Face Challenges in Developing 
Information Systems and Reporting Reliable Child Welfare Data' which 
was released on November 19, 2003.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

Testimony:

Before the Subcommittee on Human Resources, Committee on Ways and 
Means, House of Representatives:

United States General Accounting Office:

GAO:

For Release on Delivery Expected at 2:00 p.m. EST:

Wednesday, November 19, 2003:

CHILD WELFARE:

States Face Challenges in Developing Information Systems and Reporting 
Reliable Child Welfare Data:

Statement of Cornelia M. Ashby, Director, Education, Workforce, and 
Income Security Issues:

GAO-04-267T:

GAO Highlights:

Highlights of GAO-04-267T, a testimony for the Subcommittee on Human 
Resources, Committee on Ways and Means, House of Representatives 

Why GAO Did This Study:

To better monitor children and families served by state child welfare 
agencies, Congress authorized matching funds for the development of 
statewide automated child welfare information systems (SACWIS) and 
required that the Department of Health and Human Services (HHS) 
compile information on the children served by state agencies. This 
testimony is based on our July 2003 report and addresses the 
following: (1) states’ experiences in developing child welfare 
information systems and HHS’s role in assisting in their development, 
(2) factors that affect the reliability of data that states collect 
and report on children served by their child welfare agencies and 
HHS’s role in ensuring the reliability of those data, and (3) 
practices that child welfare agencies use to overcome challenges 
associated with SACWIS development and data reliability. For the July 
2003 report, we surveyed all 50 states and the District of Columbia 
regarding their experiences developing and using information systems 
and their ability to report data to HHS. We also reviewed a variety of 
HHS documents and visited five states to obtain firsthand information. 
Finally, we interviewed HHS officials and child welfare and data 
experts and reviewed relevant literature.

What GAO Found:

HHS reported that 47 states are developing or operating a SACWIS, but 
many states continue to face challenges developing their systems. Most 
state officials said they recognize the benefit their state will 
achieve by developing SACWIS, such as contributing to the timeliness 
of child abuse and neglect investigations; however, despite the 
availability of federal funds since 1994, states reported a median 
delay of 2½ years beyond the time frames they set for completion. 
States reported that they encountered some difficulties during SACWIS 
development, such as challenges receiving state funding and creating a 
system that reflected their work processes. In response to some of 
these challenges, HHS has provided technical assistance to help states 
develop their systems and conducted on-site reviews of SACWIS to 
verify that the systems meet federal requirements.

Despite efforts to implement comprehensive information systems, 
several factors affect the states’ ability to collect and report 
reliable adoption, foster care, and child abuse and neglect data. 
States responding to GAO’s survey and officials in the five states GAO 
visited reported that insufficient caseworker training and inaccurate 
and incomplete data entry affect the quality of the data reported to 
HHS. In addition, states reported technical challenges reporting data. 
Despite HHS assistance, many states report ongoing challenges, such as 
the lack of clear and documented guidance on how to report child 
welfare data. In addition, although states were mandated to begin 
reporting data to the Adoption and Foster Care Analysis Reporting 
System (AFCARS) in 1995, few reviews of states’ AFCARS reporting 
capabilities have been conducted to assist states in resolving some of 
their reporting challenges.

Some states are using a variety of practices to address the challenges 
associated with developing SACWIS and improving data reliability. For 
example, 28 states reported using approaches to help caseworkers 
identify and better understand the data elements that are required for 
federal reporting.

In a related report, we recommended that the Secretary of HHS consider 
ways to enhance the guidance and assistance offered to states to help 
them overcome the key challenges in collecting and reporting child 
welfare data. These efforts could include a stronger emphasis placed 
on conducting AFCARS reviews and timelier follow-up to help states 
implement their improvement plans or identifying a useful method to 
provide clear and consistent guidance. HHS generally agreed with our 
findings but, in response to our recommendation, said that we did not 
recognize the long-term efforts to provide AFCARS and National Child 
Abuse and Neglect Data System related guidance. HHS also noted that 
the data definitions need to be updated and revised and said it was in 
the process of revising regulations. HHS added that it is firmly 
committed to continue to support the states and to provide technical 
guidance and assistance as resources permit.


www.gao.gov/cgi-bin/getrpt?GAO-04-267T.

To view the full product, including the scope and methodology, click 
on the link above. For more information, contact Cornelia M. Ashby at 
(202) 512-8403 or ashbyc@gao.gov.

[End of section]

Mr. Chairman and Members of the Subcommittee:

Thank you for inviting me here today to discuss states' development of 
automated child welfare information systems. As you are aware, the 
Congress required that the Department of Health and Human Services 
(HHS) compile information on the children served by state agencies and 
authorized federal funds to match those of states for use in the 
development of state child welfare information systems. Since 1994, 
designated federal matching funds have been available to states to 
develop and implement comprehensive case management systems--statewide 
automated child welfare information systems (SACWIS)--to manage their 
child welfare cases as well as to report child abuse and neglect, 
foster care, and adoption information to the federal government. States 
have the option to implement a SACWIS or develop different information 
systems without using SACWIS funds to support their child welfare 
agencies and collect information on their child welfare cases. 
Regardless of the type of system a state develops, child welfare 
caseworkers at the county or local level are the key personnel who 
collect and document information on children and families served by 
child welfare agencies, in addition to performing a wide range of 
services to protect children--such as investigating child abuse or 
neglect reports or providing support services to maintain the children 
in their homes.

Currently, HHS compiles state-reported child welfare data in two 
databases: the Adoption and Foster Care Analysis and Reporting System 
(AFCARS) and the National Child Abuse and Neglect Data System (NCANDS). 
HHS relies on the information available in its databases to analyze and 
track children's experiences in the child welfare system, to determine 
states' performance on federal child welfare outcome measures, and to 
report to Congress on children's well being and child welfare 
experiences.

My testimony today will focus on three key issues: (1) states' 
experiences in developing child welfare information systems and HHS's 
role in assisting in their development; (2) factors that affect the 
reliability of data that states collect and report on children served 
by their child welfare agencies, and HHS's role in ensuring the 
reliability of those data; and (3) practices that child welfare 
agencies use to overcome challenges associated with SACWIS development 
and data reliability. My comments are based on the findings from our 
July 2003 report, Child Welfare: Most States Are Developing Statewide 
Information Systems, but the Reliability of Child Welfare Data Could Be 
Improved (GAO-03-809, July 31, 2003). Those findings were based on our 
survey of all 50 states and the District of Columbia regarding their 
experiences in developing and using information systems and their 
ability to report data to HHS. We received responses from 49 states and 
the District of Columbia,[Footnote 1] although some states did not 
respond to every question. We also reviewed a variety of HHS documents, 
including the protocol and reports for its reviews of SACWIS systems 
and states' AFCARS reporting capabilities and visited five states--
Colorado, Iowa, New York, North Carolina, and Oklahoma--to obtain 
firsthand information on their experiences developing SACWIS and 
reporting data to HHS. We selected these states to represent geographic 
diversity and different stages of SACWIS implementation. Finally, we 
interviewed HHS officials and child welfare and data experts and 
reviewed relevant literature.

In summary, HHS reported that 47 states were developing or operating a 
SACWIS, but many states continue to face challenges developing their 
systems. Most state officials said they recognize the benefit their 
state will achieve by developing SACWIS, but added that they have 
encountered difficulties in receiving state funding and in creating a 
system that reflected their work processes. Despite the availability of 
federal funds since 1994, states reported a median delay of 2½ years 
beyond the time frames they set for completion. Several factors affect 
the states' ability to collect and report reliable adoption, foster 
care, and child abuse and neglect data. For example, insufficient 
caseworker training and inaccurate and incomplete data entry affect the 
quality of data reported to HHS. States also reported technical 
challenges reporting data. Despite HHS's assistance, many states 
reported ongoing challenges, such as the lack of clear and documented 
guidance from HHS on how to report child welfare data. In addition, 
although states were mandated to begin reporting data to AFCARS in 
1995, few reviews of states' AFCARS reporting capabilities have been 
conducted. Some states are using a variety of practices to address the 
challenges they face in developing SACWIS and improving data 
reliability. For example, 28 states reported using approaches to help 
caseworkers identify and better understand the data elements that are 
required for federal reporting. To improve the reliability of state-
reported child welfare data, we recommended in our July 2003 report 
that the Secretary of HHS consider ways to enhance the guidance and 
assistance offered to states to help them overcome the key challenges 
in collecting and reporting child welfare data.

Background:

ACF's Children's Bureau is responsible for the administration and 
oversight of federal funding to states for child welfare services under 
Titles IV-B and IV-E of the Social Security Act. However, the 
monitoring of children served by state child welfare agencies is the 
responsibility of the state agencies that provide the services to these 
children and their families. Child welfare caseworkers at the county or 
local level are the key personnel responsible for documenting the wide 
range of services offered to children and families, such as 
investigations of abuse and neglect, treatment services offered to keep 
families intact and prevent the need for foster care, and arrangements 
made for permanent or adoptive placements when children must be removed 
from their homes. Caseworkers are supported by supervisors, who 
typically assign new cases to workers and monitor caseworkers' progress 
in achieving desired outcomes, analyzing and addressing problems and 
making decisions about cases.

To qualify for federal funding for SACWIS, states must prepare and 
submit an advance planning document (APD) to the Children's Bureau, in 
which they describe the state's plan for managing the design, 
development, implementation, and operation of a SACWIS that meets 
federal requirements and state needs in an efficient, comprehensive, 
and cost-effective manner. In addition, the state must establish SACWIS 
and program performance goals in terms of projected costs and benefits 
in the APD. States are required to submit separate APDs for the 
planning and development phases, in addition to periodic updates.

Since the administration and structure of state child welfare agencies 
vary across the nation, states can design their SACWIS to meet their 
state needs, as long as states meet certain federal requirements. 
Federal funding is available to states for SACWIS that:

* meet the requirements for reporting AFCARS data to HHS;

* to the extent practicable, are capable of linking with the state data 
collection system that collects information on child abuse and neglect;

* to the extent practicable, are capable of linking with, and 
retrieving information from, the state data collection system that 
collects information on the eligibility of individuals under Title IV-
A--Temporary Assistance for Needy Families; and:

* provides for more efficient, economical, and effective administration 
of the programs carried out under a state's plans approved under Titles 
IV-B and IV-E of the Social Security Act.

A SACWIS must operate uniformly as a single system in each state and 
must encompass all entities that administer programs provided under 
Titles IV-B and IV-E. In some cases, HHS will allow the statewide 
system to link to another state system to perform required functions, 
such as linking to financial systems to issue and reconcile payments to 
child welfare service providers. The state's APD must describe how its 
SACWIS will link to other systems to meet the requirements in the 
SACWIS regulations.

In addition to monitoring the APDs of the states that are developing 
SACWIS, HHS reviews state information systems through formal SACWIS 
assessment reviews and the Child and Family Services Reviews (CFSR)--a 
federal review process to monitor states' compliance with child welfare 
laws and federal outcome measures. The formal SACWIS reviews are 
conducted by the Children's Bureau to determine if a state has 
developed and implemented all components detailed in the state's APD 
and if the system adheres to federal requirements. The CFSR assesses 
statewide information systems, along with other systemic factors, to 
determine if the state is operating a system that can readily identify 
the status, demographic characteristics, location, and goals for 
placement of every child who is in foster care. This systemic factor is 
reviewed in all states, regardless of whether the state is developing a 
SACWIS or the stage of system development. For the 40 CFSR reports that 
are available, HHS found that four states were not in substantial 
conformity on the statewide information system indicator.[Footnote 2] 
These four states must address how they will come into conformity with 
this factor in a program improvement plan. HHS has also conducted 
SACWIS reviews in two of these states.

Most States Are Developing SACWIS, But Challenges Remain Despite HHS's 
Oversight and Technical Assistance:

While 47 states are developing or operating a SACWIS, many challenges 
remain despite HHS's oversight and technical assistance. Since 1994, 
states reported that they have spent approximately $2.4 billion in 
federal, state, and local funding on SACWIS. While most state officials 
we interviewed and those responding to our survey said that they 
recognize the benefits their state will achieve by developing a 
statewide system, many states reported that the development of their 
SACWIS is delayed between 2 months and 8 years beyond the time frames 
the states set for completion, with a median delay of 2½ years. Most 
states responding to our survey faced challenges, such as obtaining 
state funding and developing a system that met the child welfare 
agency's needs statewide. In response to some of these challenges, HHS 
has provided technical assistance to help states develop their systems 
and conducted on-site SACWIS reviews to verify that the systems meet 
all federal requirements.

States Are Using Federal and State Funds and Various Participants to 
Develop Multicomponent SACWIS:

Currently, 47 states are developing or operating a SACWIS and are in 
various stages of development--ranging from planning to complete. The 
states responding to our survey reported using approximately $1.3 
billion in federal funds[Footnote 3] and approximately $1.1 billion in 
state and local funds[Footnote 4] for their SACWIS. However, HHS 
estimated that it allocated approximately $821 million between fiscal 
years 1994 and 2001 in SACWIS developmental funds[Footnote 5] and $173 
million between fiscal years 1999 and 2001 in SACWIS operational 
funds.[Footnote 6] The total amount of federal funding provided to 
states for SACWIS is unknown because states claimed operational costs 
as a part of their Title IV-E administrative expenses prior to 
1999.[Footnote 7] Although the federal government matched state funding 
at an enhanced rate of 75 percent beginning in 1994, many states did 
not apply for federal funding or begin SACWIS development until 1996 or 
1997, when more than $467 million--the bulk of federal funds--were 
allocated. Most states were still developing their SACWIS by the time 
enhanced funding expired in 1997, after which states could receive a 50 
percent federal financial participation for SACWIS development and 
operation. Although 47 states are currently developing or operating a 
SACWIS, all states except Hawaii received some federal SACWIS funds. 
For example, according to figures provided by HHS, North Carolina and 
North Dakota received some developmental funds but encountered 
difficulties that prevented them from completing their systems.

In order to track states' SACWIS development, HHS places them in six 
categories that identify their stage of development (see table 1). HHS 
sometimes recategorizes states into a lower stage of development when 
problems are encountered. In addition, while HHS may classify a state 
system as complete following an assessment of the state's SACWIS, a 
state may make additional changes to the system since SACWIS, like 
other computer systems, continually evolve as technology and child 
welfare practices change. States can claim federal funding for these 
changes as operational expenses. An HHS official reported that such 
changes do not need prior approval unless they are in excess of $5 
million.

Table 1: Number of States in Various Stages of SACWIS Development:

Stage: Complete[A]; Number of states: 5.

Stage: Operational[B]; Number of states: 24.

Stage: Partially operational[C]; Number of states: 9.

Stage: Implementation[D]; Number of states: 2.

Stage: Planning[E]; Number of states: 7.

Stage: No SACWIS[F]; Number of states: 4.

Source: HHS.

Note: Status is as of October 13, 2003.

[A] The SACWIS assessment process is completed, and all functional 
requirements and specifications set forth in the APD are either 
included in the system or in an accepted corrective action plan.

[B] All functional requirements and specifications in the APD are 
included in the system, and the system is functional statewide, but 
state has not completed a SACWIS assessment or is working on other 
issues.

[C] The state is still rolling out a system to field sites or still 
adding functions to systems that are operational statewide.

[D] In active design and development, even if delayed while waiting to 
resolve problems such as funding.

[E] Working through options for a SACWIS.

[F] Have never pursued SACWIS funding or have abandoned plans to 
develop a system.

[End of table]

States have considerable flexibility in the design of their SACWIS. 
According to HHS officials, a state should be using its SACWIS as a 
case management tool that uses automation to support the various 
aspects of state child welfare programs, such as recording child 
protection, out-of-home care, and foster care and adoption services. To 
further assist child welfare practice, states have designed their 
systems to follow the natural flow of child welfare practice in their 
state and have added design features to help track key events during a 
case. For example, in Iowa child welfare work is divided between child 
abuse and neglect investigations and ongoing case management for 
children brought into the care of the child welfare agency. As a 
result, Iowa designed a SACWIS to reflect this work process by linking 
two databases--one to record child abuse and neglect information and 
one to record ongoing case records--that share information with each 
other.[Footnote 8]

Since many states are in different phases of SACWIS development, their 
systems currently support to varying degrees a variety of child welfare 
and administrative components (see table 2). According to HHS, while 
the components listed in table 2 are required for a state's SACWIS to 
be considered compliant with federal guidance--either through an 
interface or built within the system--some of the subcomponents, such 
as a function that helps caseworkers manage their caseloads, are 
optional. HHS has encouraged states to automate as many functions as 
possible in the SACWIS in an effort to cut down on the additional 
paperwork or duplicative steps inherent in manual data collection.

Table 2: Selected SACWIS Child Welfare and Administrative Services:

Service: Child welfare services: 

Service: Child protection[A]; Fully or partially operational in SACWIS: 
38; Planned for SACWIS: 5.

Service: Out-of-home care[B]; Fully or partially operational in SACWIS: 
35; Planned for SACWIS: 8.

Service: Adoption; Fully or partially operational in SACWIS: 34; 
Planned for SACWIS: 9.

Service: Independent living; Fully or partially operational in SACWIS: 
27; Planned for SACWIS: 14.

Service: Intensive home-based services[C]; Fully or partially 
operational in SACWIS: 27; Planned for SACWIS: 13.

Service: Administrative services: 

Service: Workload management; Fully or partially operational in SACWIS: 
32; Planned for SACWIS: 8.

Service: IV-E eligibility[D]; Fully or partially operational in SACWIS: 
29; Planned for SACWIS: 14.

Service: Foster care maintenance payments; Fully or partially 
operational in SACWIS: 28; Planned for SACWIS: 14.

Service: Adoption assistance payments; Fully or partially operational 
in SACWIS: 25; Planned for SACWIS: 17.

Service: Contract provider payment; Fully or partially operational in 
SACWIS: 24; Planned for SACWIS: 15.

Source: GAO survey.

Note: This table is based on responses from 46 states developing or 
operating a SACWIS. The rows for the columns "fully or partially 
operational" and "planned" do not add to 46 because the respondents may 
have answered "not supported," "don't know," or "no answer.":

[A] Child protection includes services such as intake and screening, 
investigation, and disposition.

[B] Out-of-home care includes things such as foster care, group homes, 
and residential placement.

[C] Intensive home-based services include efforts to avoid placing a 
child in foster care.

[D] IV-E funding is available for foster care, adoption, and 
independent living services.

[End of table]

To assist with the design of their SACWIS, states relied on a number of 
different participants, including internal users, such as caseworkers 
and managers, information technology (IT) staff, and contractors. In 
Oklahoma, for example, 150 child welfare staff from the field worked 
closely with the contractor in intensive work group sessions to design 
and test the system. To complement the caseworkers' knowledge of child 
welfare practice, 43 states relied on IT staff. Finally, 42 states 
reported that they hired private contractors to conduct a large part of 
SACWIS design and development.

At the time of our review, HHS reported that four states were not 
pursuing SACWIS development, and most of these states reported various 
reasons in our survey for not developing a system. In Hawaii, for 
example, the child welfare agency chose not to pursue SACWIS because it 
already had a statewide system in place that it believed was adequately 
meeting its needs and which was collecting and reporting federal child 
welfare data.

States Accrue Benefits from Using SACWIS, but Several Issues Create 
Delays in Completing States' Systems:

While most state child welfare agency officials said they recognize the 
benefits the state will achieve by developing SACWIS, such as enhancing 
their ability to track the whereabouts of foster children, 31 state 
agencies lag behind the time frames they set for completion, with 26 
states reporting delays ranging from 2 months to 8 years. According to 
survey results, automated systems provided easier access to data and 
allowed caseworkers to better monitor children in their care, a fact 
that may contribute to additional child welfare and administrative 
benefits, such as decreased incidences of child abuse and neglect, 
shortened length of time to achieve adoption, timeliness of payments to 
foster families, and timeliness of payments to foster facilities. New 
Jersey, which is in the planning stage, reported in our survey that its 
goal in developing a SACWIS is to integrate the more than 40 stand-
alone systems that currently capture information on the children served 
by their child welfare agency.[Footnote 9] By pulling all of these 
systems together into a uniform SACWIS, the state hopes to improve the 
recording of casework activities in a timely manner and to develop a 
tool to better target resources and services. Effectively integrating 
these systems will require the state to use a disciplined IT management 
approach that includes (1) detailed analyses of users' needs and 
requirements, (2) a clearly defined strategy for addressing information 
needs, and (3) sufficient technical expertise and resources to support 
the effort.

Despite the benefits that many states have accrued with SACWIS, 31 
states reported in our survey that they have been delayed in system 
completion beyond their initial deadline and identified a number of 
challenges that have led to the delay (see table 3).[Footnote 10] Some 
of the common difficulties states reported in developing SACWIS 
included receiving state funding approval, reaching internal agreement 
on system development, and creating a system that reflects child 
welfare work processes and is user-friendly. Vermont officials, for 
example, reported that the state legislature declined to provide the 
matching state funds needed to secure federal funding for SACWIS. As a 
result, the state could not pursue development.

Table 3: Number of Months States Delayed in SACWIS Development:

State: Alabama; Length of delay in months[A]: 36.

State: Arkansas; Length of delay in months[A]: 6.

State: California; Length of delay in months[A]: 36.

State: Colorado; Length of delay in months[A]: 26.

State: Connecticut; Length of delay in months[A]: 96.

State: District of Columbia; Length of delay in months[A]: 36.

State: Georgia; Length of delay in months[A]: 25.

State: Idaho; Length of delay in months[A]: 21.

State: Illinois; Length of delay in months[A]: 79.

State: Indiana; Length of delay in months[A]: 6.

State: Kansas; Length of delay in months[A]: 72.

State: Louisiana; Length of delay in months[A]: 12.

State: Maryland; Length of delay in months[A]: 12.

State: Michigan; Length of delay in months[A]: 26.

State: Minnesota; Length of delay in months[A]: 12.

State: Mississippi; Length of delay in months[A]: 12.

State: New Jersey; Length of delay in months[A]: 42.

State: New Mexico; Length of delay in months[A]: 3.

State: Ohio; Length of delay in months[A]: 36.

State: Oregon; Length of delay in months[A]: 70.

State: Rhode Island; Length of delay in months[A]: 14.

State: South Carolina; Length of delay in months[A]: 47.

State: Tennessee; Length of delay in months[A]: 36.

State: Utah; Length of delay in months[A]: 48.

State: Virginia; Length of delay in months[A]: 2.

State: Washington; Length of delay in months[A]: 36.

Source: GAO survey.

Note: While 31 states reported in the survey that they have experienced 
a delay in SACWIS development, only 26 states reported the length of 
their delay. The survey was issued in October 2002 and completed by 
states as late as December 2002.

[A] States were asked to report the number of months the delays 
exceeded the time line outlined in their APD.

[End of table]

Despite user involvement in system design, some states still faced 
challenges trying to reach internal agreement among agency officials 
and caseworkers on the design of a system, resulting in a delay in 
development. In New York--a state where the counties are responsible 
for administering child welfare services--the development of SACWIS was 
stalled when significant frustration with the system's design led 
commissioners from five large counties and New York City to request 
that the state stop SACWIS development until a reassessment of the 
design of and plans for the implementation of the system was completed.

Similarly, despite states' heavy reliance on contractors, many reported 
that securing contractors with knowledge of child welfare practice was 
a challenge for timely SACWIS development. Contractors are hired by the 
state for their system development knowledge but often are unfamiliar 
with child welfare policies and practices, especially since they vary 
from state to state. A contractor who has worked with seven states to 
develop their SACWIS reported that contractors are asked to learn the 
child welfare business practices of a state in a short amount of time 
and that states cannot devote many resources, such as caseworkers, to 
help in the design process because caseworkers need to devote their 
time to providing services to children and families.

Many states reported that creating a system that reflects child welfare 
work processes and is user-friendly was a challenge in developing 
SACWIS. These issues were also identified in the federal reviews of 
states' SACWIS. For example, one state explained in the SACWIS review 
that it had designed a system to meet the caseworkers' needs and 
reflect the nature of the child welfare work processes by developing a 
system that required events to be documented as they occurred. However, 
this design limited the SACWIS's functionality because it did not allow 
the caseworkers to go back and enter information after an event 
happened. The state explained that caseworkers do not use the system in 
real time, but provide services to the children and families and then 
record the information in the system. The state had to redesign the 
system to correct for this design flaw.

HHS Provides Some Assistance to Help States Meet SACWIS Requirements:

HHS has assisted states in a variety of ways in developing and 
completing their SACWIS.[Footnote 11] As a part of its regulatory 
responsibilities, HHS must review, assess, and inspect the planning, 
design, development, installation, and operation of SACWIS. In addition 
to reviewing and monitoring states' APDs, HHS conducts on-site SACWIS 
reviews to comply with these responsibilities. HHS officials told us 
that these reviews are a detailed and thorough assessment of state 
systems to ensure the systems' compliance with SACWIS requirements. In 
addition, officials reported that they provide technical assistance 
during the on-site review to help states that do not fully conform with 
the applicable regulations and policies. As of October 2003, HHS had 
reviewed 27 SACWIS--5 of which were determined as meeting all the 
requirements and classified as complete. HHS officials told us that 
since states have the flexibility to build a SACWIS that meets their 
needs, a large portion of the formal reviews concentrate on ensuring 
that the systems conform to state business practices. For example, 
while SACWIS regulations require that a state report all AFCARS data 
from their SACWIS, one state HHS reviewed relied on a separate state 
system to report data on the children served by the juvenile justice 
agency who are eligible for IV-E foster care funds. The state proved it 
had developed an automated process to merge data from both systems to 
compile a single AFCARS report that included children captured in both 
their SACWIS and juvenile justice systems. Therefore, HHS recognized 
that this process best met the state's needs and determined the SACWIS 
to be complete and meeting all requirements.

Few systems have been determined complete after an on-site review 
because of unresolved issues, such as not being able to build links to 
other state information systems or not implementing certain eligibility 
determination functions. To help states address some of these 
development challenges, the SACWIS review team provides the state with 
recommendations for complying with SACWIS requirements. In addition, 
HHS officials reported that once the draft report with the results of 
the SACWIS review is completed, federal staff schedule a conference 
call with the state officials to walk through the system's deficiencies 
and offer guidance on how the state can move forward.

HHS facilitates the sharing of information between states developing 
SACWIS through an automated system users' group that allows state and 
federal officials to exchange information, ideas, and concerns. In 
addition to the users' group, HHS officials also sponsor a Listserv--an 
electronic mailing list--that allows state officials to exchange 
information and a monthly conference call with state information 
technology directors. [Footnote 12] Technical assistance for SACWIS 
development is also available to states through the National Resource 
Center for Information Technology in Child Welfare (Resource Center), 
which opened in 1999. According to survey results, 9 states said they 
used the Resource Center for assistance in developing SACWIS and 14 
states reported using it for help with SACWIS maintenance and 
improvements. According to Resource Center officials, they assist 
states with SACWIS development by helping states understand the 
technology that is available for use, providing information on the 
automation of child welfare work and converting data, and reviewing the 
APD documentation.

Several Factors Affect the States' Ability to Ensure Reliable Data on 
Children's Experiences, and Some of HHS's Oversight and Assistance Is 
Problematic:

Several factors affect states' ability to collect and report 
reliable[Footnote 13] data on children served by state child welfare 
agencies, and some problems exist, such as a lack of clear and 
documented guidance, with HHS's oversight and technical assistance. 
Almost all of the states responding to our survey reported that 
insufficient caseworker training and inaccurate and incomplete data 
entry affect the quality of the data reported to HHS.[Footnote 14] In 
addition, 36 of the 50[Footnote 15] states that responded to our survey 
reported that technical challenges, such as matching their state data 
element definitions to HHS's data categories, affected the quality of 
the data that they report to the federal government. Despite the 
assistance that HHS offers to states, such as testing state data 
quality and providing the results to states to aid them in resubmitting 
data, states report ongoing challenges receiving clear and documented 
guidance and obtaining technical assistance.

Insufficient Caseworker Training and Inaccurate and Incomplete Data 
Entry Are the Most Common Factors That Affect Data Reliability:

Almost every state responding to our survey and all the states we 
visited reported that insufficient training for caseworkers and 
inaccurate and incomplete data entry affect the quality of the data 
reported to AFCARS and NCANDS (see fig. 1). Although most states 
reported these as separate factors, HHS and the states we visited found 
that insufficient training and inaccurate and incomplete data entry are 
often linked. In official reviews of states' information systems' 
capability to capture data and report them to AFCARS, HHS advised 
states to offer additional training to caseworkers on several AFCARS 
data elements, such as recording the reasons for a child leaving foster 
care, to improve the accuracy of the data submitted. However, state 
officials told us that training is typically one of the first programs 
cut when states face tight budget restrictions. For example, Iowa 
officials told us that training has been significantly reduced in 
recent years because of budget cuts and new workers may wait 2 to 3 
months before being trained how to enter data appropriately into their 
SACWIS.

Figure 1: Most Common Caseworker Issues That Affect Data Quality:

[See PDF for image]

Notes: Based on responses from 50 states.

[End of figure]

The results reported in the figure are a sum of the states that 
reported the issue had a very great affect, great affect, moderate 
affect, or some affect on the quality of state data submitted to HHS. 
Very great and great affect responses are represented in the top 
section of each bar. Moderate and some affect responses are represented 
in the bottom section of each bar. States not included answered "no 
affect," "don't know," or "no answer.":

Inaccurate and incomplete data entry can also result from a number of 
other factors, such as caseworkers' hesitation to ask families for 
sensitive information. For example, caseworkers in Oklahoma reported 
that they did not feel comfortable asking if a child's mother was 
married at the time of birth or if a child is of Hispanic origin--both 
of which are required AFCARS data elements. In commenting on a draft of 
this report, Oklahoma added that caseworkers did not understand why the 
data elements were required and how the federal government used the 
information. HHS noted similar issues in five states that have had an 
AFCARS review.[Footnote 16] Caseworkers were inaccurately recording a 
child's race as "unable to determine" even though this option should be 
selected only if the child's parents or relatives cannot provide the 
information, such as when a child is abandoned.[Footnote 17]

Caseworkers, supervisors, and managers in the 5 states we visited 
reported that additional factors, such as difficulties balancing data 
entry with the time that they spend with the families and children, 
contributed to inaccurate or incomplete data entry. Supervisors in Iowa 
explained that since caseworkers are responsible for ensuring that 
children and their families receive the services they need, the 
caseworkers tend to initially limit data entry to the information that 
is necessary to ensure timely payment to foster care providers and 
complete all other data elements when they have time. In addition, 
caseworkers in Colorado said that they are between 30 and 60 days 
behind in their data entry, so the information in the automated system 
may not accurately reflect the current circumstances of children in 
care. HHS's Inspector General recently issued a report in which more 
than two-thirds of the states reported that caseworkers' workloads, 
turnover, a lack of training, and untimely and incomplete data entry 
affected the reporting of AFCARS data.[Footnote 18]

Technical Challenges, such as Matching State Definitions to Federal 
Definitions, Affect Data Reliability:

In addition to data quality being affected by caseworker issues, many 
states experienced technical challenges reporting their data to HHS. 
The problems reported by states are typically a result of challenges 
associated with data "mapping"--matching state data elements to the 
federal data elements. For example, 36 states reported in our survey 
that matching their state-defined data to HHS's definitions affected 
the quality of the data reported to NCANDS and AFCARS. Similarly, 24 
states reported that matching the more detailed data options available 
in their states' information systems to the federal data elements 
affected the quality of the data reported to NCANDS. Twenty-nine states 
reported that this issue created challenges in reporting data to 
AFCARS. For example, following an AFCARS assessment, HHS instructed a 
state that collects detailed information on children's disabilities, 
such as attention deficit disorder and eating disorders, to map the 
information to the more limited options in AFCARS, such as mental 
retardation and emotionally disturbed.

In many cases, states have to balance state policy with federal 
requirements to ensure that they are reporting accurate data to AFCARS 
and NCANDS, but are not contradicting their state policies. For 
example, Texas officials reported that although the findings of their 
AFCARS review instructed them to modify their SACWIS to collect, map, 
and extract data on guardianship placements, the state does not support 
guardianship arrangements.[Footnote 19] In addition, a recent report 
from the Child Welfare League of America (CWLA) found that when 
reporting the number of times children move from one foster care 
placement to another, states varied in the type of placements included 
in that count.[Footnote 20] For example, 29 percent of the states 
responding to CWLA's survey included respite,[Footnote 21] 25 percent 
included runaways, and 16 percent included trial home visits when 
reporting the number of placements a child had during the AFCARS report 
period. According to federal guidance, the "number of placements" 
element is meant to gather information on the number of times the child 
welfare agency found it necessary to move a child while in foster care 
and that by including runaways or trial home visits, a state is 
inflating the number of moves a child experienced.

Although HHS Has Taken Steps to Help States Improve Their Data, Some 
Problems with Its Efforts Exist:

HHS provides technical assistance for AFCARS and NCANDS reporting 
through a number of resources. HHS officials in the central office and 
NCANDS contractor staff serve as the points of contact for states to 
ask questions and seek guidance on reporting child welfare data. The 
officials in three of the five states that we visited said that the 
one-on-one focused technical assistance was useful when provided in a 
timely fashion. Most state officials found the NCANDS data easier to 
report, in part because more people were available for consultation and 
they were more accessible and responsive. For example, states have 
access to four NCANDS specialists and staff in the contractor's central 
office when they need assistance reporting child abuse and neglect 
information. However, some of the states we visited reported that only 
one or two staff in HHS's central office are available to assist with 
AFCARS reporting.

In addition, the Resource Center offers states assistance with 
improving data quality. However, Resource Center staff reported that 
the assistance is geared more toward improving the limited data used in 
the federal review process to monitor states' compliance with child 
welfare laws and federal outcome measures--CFSR--rather than all the 
data reported to HHS. The Resource Center also sponsors an annual 
information technology conference during which sessions covering all 
data-related issues are held, including practices for ensuring data 
quality and outcome evaluation in child welfare. In conjunction with 
this conference, the HHS officials and the contractors that operate 
NCANDS hold an annual technical assistance meeting for states to share 
ideas with one another, discuss data elements that pose difficulties, 
and explore ways to address these problems. In addition, an NCANDS 
state advisory group meets annually to talk with HHS officials about 
NCANDS data and their experiences reporting data. From these meetings, 
the state advisory group proposes changes or improvements to NCANDS. 
HHS and state officials reported that this partnership has helped ease 
some of the challenges in reporting child abuse and neglect data.

HHS has also made available to states the software it uses to examine 
states' AFCARS and NCANDS submissions for inconsistencies and invalid 
data. Officials in all the states we visited said that they regularly 
use this software, and an HHS official said that nearly every state has 
used the software at least once. When the data are submitted to HHS, 
they are run through the same software, and HHS notifies the states of 
areas where data are missing or inconsistent and allows the states to 
resubmit the data after errors are corrected. HHS officials reported 
that these tests help them to identify some data quality errors, such 
as missing data, and said that they believe that, in general, data have 
improved in recent years. However, they indicated that the tests cannot 
pinpoint the underlying problems contributing to these errors. 
Furthermore, one official reported that no specific efforts have been 
conducted to track the individual data elements, and therefore HHS 
cannot report on how data quality has changed over time.

In an attempt to help states comply with the reporting standards and 
address some of the factors that contribute to data quality problems, 
HHS performs comprehensive reviews of state information systems' 
ability to capture AFCARS data to identify problems associated with 
data collection and reporting and to ensure that the information in the 
automated system correctly reflects children's experiences in care. The 
assessments include a technical review of the states' computer code, a 
comparison of the data from selected cases available in the information 
system to the case files, and an improvement plan to resolve any 
errors. In addition, HHS officials offer guidance to the states on 
improvements that can be made to the information system and changes to 
program code used to report the AFCARS data. HHS conducted pilot 
reviews in eight states between 1996 and 2000. By October 2003, HHS had 
conducted 11 official reviews--even though states began reporting to 
AFCARS in 1995. According to results from 6 of the 11 official AFCARS 
assessments we reviewed, no state met the reporting requirements for 
all AFCARS data elements. The problems noted in the reviews are similar 
to those of states responding to our survey and those we visited. For 
example, most states received ratings of 2 or 3, indicating technical 
and/or data entry errors that affect the AFCARS data quality.[Footnote 
22] For the current placement setting data element,[Footnote 23] for 
instance, 4 states received a rating of 2, 1 state received a rating of 
3, and 1 state received a rating of 4. In Connecticut, which received a 
rating of 2, HHS found that, among other things, workers were not 
consistently entering placement information in a timely way. It also 
found that workers entered placement data only into a narrative field, 
which resulted in placement history gaps and incomplete AFCARS reports.

State officials in the six states for which we reviewed the HHS AFCARS 
assessments reported that they found the reviews useful for improving 
their AFCARS data submissions. In particular, they valued the thorough 
review by HHS officials of the computer code states use to report the 
data. Some of these officials reported that if all states were 
reviewed, the quality of data available in AFCARS would improve 
tremendously. However, HHS officials reported that they are not 
mandated to conduct the AFCARS reviews and that priority is placed on 
other reviews, such as the CFSR and SACWIS reviews. In addition, 
officials explained that the AFCARS reviews are not conducted in states 
developing SACWIS until the systems are operational. HHS expects to 
complete approximately four reviews each year, depending on available 
resources, and has scheduled states through 2006. Similar to the SACWIS 
reviews, HHS officials offer recommendations and technical assistance 
to states during the review on how they can improve the quality of the 
data reported to AFCARS.

Although the states we visited appreciated some of HHS's efforts to 
assist with improving state data quality, they and most states 
responding to our survey agreed that the assistance is not always 
consistent or easily accessible (see fig. 2). States reported similar 
information to the Inspector General--AFCARS data elements were not 
clearly and consistently defined and technical assistance is effective 
but difficult to access.

Figure 2: Federal Practices That Affect Data Quality:

[See PDF for image]

Notes: Based on responses from 50 states.

[End of figure]

The results reported in the figure are a sum of the states that 
reported the issue had a very great affect, great affect, moderate 
affect, or some affect on the quality of state data submitted to HHS. 
Very great and great affect responses are represented in the top 
section of each bar. Moderate and some affect responses are represented 
in the bottom section of each bar. States not included answered "no 
affect," "don't know," or "no answer.":

The primary concerns reported by the states we visited were delays in 
receiving clear written guidance on defining and reporting certain data 
elements and the lack of state input in suggesting changes to AFCARS. 
Despite the written guidance available to states in the form of 
regulations and an online policy manual, states reported that the 
variation in state policies and practices makes it difficult to 
interpret how to apply the general guidance. As a result, states 
consult with HHS to ensure they are applying the regulations 
appropriately. However, in commenting on a draft of this report, 
officials in Oklahoma told us that a common concern among the states is 
the lack of timely response from HHS when seeking guidance on how to 
report data. In addition, officials in New York explained they have 
made it a practice to check the HHS Web site on a regular basis for 
current guidance but have not found it a useful tool, and may turn to 
other states for guidance on AFCARS reporting. In commenting on a draft 
of this report, HHS explained that it first refers states to its Web 
site for information and believes that the available guidance addresses 
states' concerns in most instances. In addition, the states that have 
had an AFCARS review experienced delays in obtaining guidance on how to 
proceed following the on-site review. For example, Texas officials 
reported that the state sought clarification on its improvement plan 
and submitted additional questions to HHS following the review. 
However, when we spoke with the state officials, they said that they 
had been waiting 3 months for a response on how to proceed. An HHS 
official told us that since the review process is relatively new, the 
agency is still developing a process to respond to the states and 
recognizes that it has not been responsive to the states already 
reviewed. In addition, HHS is taking steps to gather feedback from 
states and other users of AFCARS data to determine how to improve the 
system to make the data more accurate and usable. As a part of these 
efforts, HHS has published a Federal Register notice soliciting 
comments and held focus group meetings at national conferences. The 
difficulties states face in receiving federal guidance and assistance, 
as well as the other challenges they face in reporting data, may 
negatively affect the reliability of the data available in AFCARS and 
NCANDS.

States Are Using Various Practices to Overcome System Development 
Challenges and Improve Data on Children's Experiences:

Some states are using a variety of practices to address the challenges 
associated with developing SACWIS and improving data reliability, 
although no formal evaluations of their effectiveness are available. To 
address the challenge of developing a system to meet statewide needs, 
states relied on caseworkers and supervisors from local offices to 
assist in the design and testing of the system. Few states reported in 
our survey strategies to overcome the other key challenges, such as 
limited funding and the difficulty of securing knowledgeable 
contractors, but some states we visited have devised some useful 
approaches. To improve data reliability, the five states we visited 
routinely review their data to identify data entry errors so that 
managers can ensure that the missing data are entered appropriately.

States Are Primarily Relying on SACWIS Users to Overcome Some of the 
Challenges to Completing Their Systems:

To overcome development challenges, survey respondents emphasized the 
importance of including system users in the various phases of 
completing SACWIS--planning, design, development, testing, and 
implementation. Past GAO work and other research efforts have 
determined similar approaches as best practices in building information 
systems.[Footnote 24] Forty-four of the 46 states responding to our 
survey that they are developing or operating a SACWIS indicated that 
they relied on internal users, such as caseworkers and supervisors, in 
the development of their systems and 34 of these states said that they 
were extremely helpful participants. The extent to which the users were 
involved in development differed across the states. For example, in 
Texas, caseworkers from all of their child welfare regions were 
recruited to provide input on design and development, as well as during 
initial testing, pilot testing, and implementation of the system. 
Arkansas reported establishing a committee made up of users to review 
the work plan and sign off on recommended changes.

Ten states noted that user input should not be limited to frontline 
workers, such as caseworkers, but should include representatives from 
other areas of the agency, such as the financial staff, and other 
agencies that serve children, such as child support 
enforcement.[Footnote 25] While not one of the most common challenges 
reported in our survey, New Hampshire reported that one of its 
challenges with meeting its SACWIS timeframe was not working 
collaboratively with other agencies, such as Temporary Assistance for 
Needy Families (TANF)[Footnote 26] and child support enforcement, to 
develop the payment component of SACWIS. To attempt to overcome this 
challenge, 26 of the 46 states responding to our survey that they are 
developing or operating a SACWIS indicated that they included external 
public agency users, and 23 reported using representatives from other 
state agencies that serve children in developing their SACWIS.

In addition to seeking input from caseworkers and other system users 
while developing SACWIS, many states continue to include users as a 
part of the implementation teams, to serve as contacts in the field and 
provide ongoing assistance, and to provide input on system 
enhancements. Alabama responded in our survey that the state had 
"mentors" in each county to help caseworkers adjust to the new system. 
These mentors continue to provide ongoing support now that the system 
is implemented. Oklahoma recruits experienced child welfare field staff 
for its SACWIS help desk because of their knowledge of the system and 
child welfare policy and practice.

Although states faced other challenges in completing their SACWIS, few 
reported implementing approaches to overcome the barriers. According to 
survey results, a common problem states faced in developing SACWIS was 
receiving insufficient state funding for development. States did not 
report in our survey, however, approaches for obtaining more funding 
for developing SACWIS, and few states reported developing strategies in 
an attempt to overcome the challenges associated with tight budgets for 
maintaining their systems. For example, Iowa officials engaged in 
careful planning with system users to ensure that they addressed the 
highest priorities when enhancing the system. In particular, the 
officials reported that maintaining tight control over the development 
and maintenance processes helps them avoid investing inordinate amounts 
of resources to make corrections to the system. Similarly, few states 
reported on approaches to overcome the challenge of finding contractors 
with knowledge of child welfare practice. However, Iowa officials 
explained that once the contract staff are hired, they are required to 
attend the same training as new caseworkers to ensure that they are 
familiar with the state's child welfare policies and to familiarize 
themselves with casework practices.

States Use Strategies, such as Producing Reports That Identify Missing 
Data, in an Attempt to Improve the Reliability of the Data Reported to 
HHS:

Twenty-eight states reported using approaches to help caseworkers 
identify the data elements that are required for federal reporting and 
to help them better understand the importance of entering timely and 
accurate data. Ten states responding to our survey reported reviewing 
the federal reporting requirements in training sessions as a way to 
improve data quality. For example, Tennessee reported that the state 
added a component about AFCARS to the initial and ongoing training 
workers receive about using SACWIS. The curriculum addresses the AFCARS 
report in general and the individual data elements to help the 
caseworkers better understand the purpose of collecting the 
information. In Nebraska, a "desk aid" that explains the data elements 
and where and why to enter them in the system is available on the 
caseworkers' computer desktops. In addition, New York has developed a 
step-by-step guide explaining to workers how NCANDS data should be 
entered, with references to the policy or statute requiring the 
information.

To improve data reliability, some states have designed their 
information systems with special features to encourage caseworkers to 
enter the information. Four states responding to our survey and three 
states we visited designed their SACWIS with color-coded fields to draw 
attention to the data elements that caseworkers are required to enter. 
Colorado, Iowa, New York, and Oklahoma have built into their systems 
alerts--also known as "ticklers"--to remind caseworkers and supervisors 
of tasks that they need to complete. For example, in Oklahoma, a 
stoplight icon on the caseworker's computer desktop reminds the worker 
when tasks are due. A green light indicates that nothing is due within 
5 days; a yellow light means that something is due within 5 days; and a 
red light means that something is overdue. Caseworkers and supervisors 
in the states we visited had mixed responses about the usefulness and 
effectiveness of the alerts. Some caseworkers found them to be a 
nuisance, while other caseworkers and supervisors found them to be 
useful tools in managing workloads and prioritizing daily tasks.

Six states reported that the best way to improve data quality was to 
use the data in published reports and hold the caseworkers and 
supervisors accountable for the outcomes of the children in their care. 
In addition, six states responding to our survey reported using the 
data available in their information systems to measure state outcomes 
similar to the CFSR. State officials reported that this approach is an 
effective way to get local offices invested in the quality of the data. 
For example, North Carolina publishes monthly reports for each county 
comparing their performance on state data indicators, such as the 
length of time children spend in care, to counties of similar size and 
the state as a whole. County officials reported that these reports 
encourage workers to improve the quality of the data collected and 
entered into the state system since their performance is being widely 
published and compared with that of other counties.

In addition, all the states we visited reported that frequent review of 
their data, such as using software from HHS to test their AFCARS and 
NCANDS data to pinpoint data entry errors prior to submitting them to 
HHS, has helped improve data quality. When the states identify poor 
data, they alert the caseworkers and supervisors of needed corrections 
and data entry improvements. For example, Colorado runs these reports 
about four to five times a year, with one run occurring approximately 6 
weeks before each AFCARS submission. When the data specialists find 
errors, they notify the caseworker to clean up the data.

Concluding Observations:

While most states are developing statewide information systems, 
challenges with data reliability remain. Although SACWIS development is 
delayed in many states, state officials recognize the benefits of 
having a uniform system that enhances the states' ability to monitor 
the services provided and the outcomes for children in their care. 
Although states began reporting to NCANDS in 1990 and were mandated to 
begin reporting to AFCARS in 1995, most states continue to face 
challenges providing complete, accurate, and consistent data to HHS. In 
addition, the results of more recent HHS efforts, such as conducting 
AFCARS-related focus groups, are unknown. Reliable data are essential 
to the federal government's development of policies that address the 
needs of the children served by state child welfare agencies and its 
ability to assist states in improving child welfare system 
deficiencies. Without well-documented, clearer guidance and the 
completion of more comprehensive reviews of states' AFCARS reporting 
capabilities, states are limited in overcoming challenges that affect 
data reliability. Because these challenges still remain, HHS may be 
using some questionable data as the foundation for national reports and 
may not have a clear picture of how states meet the needs of children 
in their care.

To improve the reliability of state-reported child welfare data, we 
recommended in our July 2003 report that the Secretary of HHS consider, 
in addition to HHS's recent efforts to improve AFCARS data, ways to 
enhance the guidance and assistance offered to states to help them 
overcome the key challenges in collecting and reporting child welfare 
data. These efforts could include a stronger emphasis placed on 
conducting AFCARS reviews and more timely follow-up to help states 
implement their improvement plans or identifying a useful method to 
provide clear and consistent guidance on AFCARS and NCANDS reporting. 
ACF generally agreed with our findings and commented that the report 
provides a useful perspective of the problems states face in collecting 
data and of ACF's effort to provide ongoing technical assistance to 
improve the quality of child welfare data. In response to our 
recommendation, ACF said that we categorized its efforts as "recent" 
and did not recognize the long-term efforts to provide AFCARS-and 
NCANDS-related guidance to the states. Although we did not discuss each 
effort in depth, we did mention the agency's ongoing efforts in our 
report. ACF also noted in its comments that the data definitions need 
to be updated and revised and said it is currently in the process of 
revising the AFCARS regulations to further standardize the information 
states are to report--which we acknowledged in our report. ACF also 
commented that it is firmly committed to continue to support the states 
and to provide technical assistance and other guidance as its resources 
will permit. ACF commented that it provided increased funding to the 
National Resource Centers in fiscal year 2003, and it believed that 
this increase will improve ACF's ability to provide assistance to the 
states. After receiving the draft report for comment, HHS separately 
provided information on an additional service the National Resource 
Center for Information Technology in Child Welfare provides to states. 
More recently, HHS said that it would be creating policy guidance that 
will delineate what will happen if a state fails to complete its SACWIS 
within a reasonable time frame. For example, funding may become 
contingent on successful completion of specific milestones.

Mr. Chairman, this concludes my prepared statement. I would be pleased 
to respond to any questions that you or other members of the 
Subcommittee may have.

GAO Contact and Acknowledgments:

For further contacts regarding this testimony, please call Cornelia M. 
Ashby at (202) 512-8403. Individuals making key contributions to this 
testimony include Diana Pietrowiak and Sara Schibanoff.

[End of section]

Related GAO Products:

Child Welfare: Enhanced Federal Oversight of Title IV-B Could Provide 
States Additional Information to Improve Services. GAO-03-956. 
Washington, D.C.: September 12, 2003.

Child Welfare: Most States Are Developing Statewide Information 
Systems, but the Reliability of Child Welfare Data Could Be Improved. 
GAO-03-809. Washington, D.C.: July 31, 2003.

Child Welfare and Juvenile Justice: Several Factors Influence the 
Placement of Children Solely to Obtain Mental Health Services. GAO-03-
865T. Washington, D.C.: July 17, 2003.

Child Welfare and Juvenile Justice: Federal Agencies Could Play a 
Stronger Role in Helping States Reduce the Number of Children Placed 
Solely to Obtain Mental Health Services. GAO-03-397. Washington, D.C.: 
April 21, 2003.

Child Welfare: HHS Could Play a Greater Role in Helping Child Welfare 
Agencies Recruit and Retain Staff. GAO-03-357. Washington, D.C.: March 
31, 2003.

Human Services: Federal Approval and Funding Processes for States' 
Information Systems. GAO-02-347T. Washington, D.C.: July 9, 2002.

Foster Care: Recent Legislation Helps States Focus on Finding Permanent 
Homes for Children, but Long-Standing Barriers Remain. GAO-02-585. 
Washington, D.C.: June 28, 2002.

Human Services Integration: Results of a GAO Cosponsored Conference on 
Modernizing Information Systems. GAO-02-121. Washington, D.C.: January 
31, 2002.

District of Columbia Child Welfare: Long-Term Challenges to Ensuring 
Children's Well-Being. GAO-01-191. Washington, D.C.: December 29, 2000.

Child Welfare: New Financing and Service Strategies Hold Promise, but 
Effects Unknown. GAO/T-HEHS-00-158. Washington, D.C.: July 20, 2000.

Welfare Reform: Improving State Automated Systems Requires Coordinated 
Federal Effort. GAO/HEHS-00-48. Washington, D.C.: April 27, 2000.

Foster Care: States' Early Experiences Implementing the Adoption and 
Safe Families Act. GAO/HEHS-00-1. Washington, D.C.: December 22, 1999.

Foster Care: HHS Could Better Facilitate the Interjurisdictional 
Adoption Process. GAO/HEHS-00-12. Washington, D.C.: November 19, 1999.

Foster Care: Effectiveness of Independent Living Services Unknown. GAO/
HEHS-00-13. Washington, D.C.: November 5, 1999.

Foster Care: Kinship Care Quality and Permanency Issues. GAO/HEHS-99-
32. Washington, D.C.: May 6, 1999.

Juvenile Courts: Reforms Aim to Better Serve Maltreated Children. GAO/
HEHS-99-13. Washington, D.C.: January 11, 1999.

Child Welfare: Early Experiences Implementing a Managed Care Approach. 
GAO/HEHS-99-8. Washington, D.C.: October 21, 1998.

Foster Care: Agencies Face Challenges Securing Stable Homes for 
Children of Substance Abusers. GAO/ HEHS-98-182. Washington, D.C.: 
September 30, 1998.

Managing Technology: Best Practices Can Improve Performance and Produce 
Results. GAO/T-AIMD-97-38, January 31, 1997.

Child Welfare: HHS Begins to Assume Leadership to Implement National 
and State Systems. GAO/AIMD-94-37. Washington, D.C.: June 8, 1994.

Executive Guide: Improving Mission Performance Through Strategic 
Information Management and Technology. GAO/AIMD-94-115. Washington, 
D.C.: May 1, 1994.


FOOTNOTES

[1] Throughout this testimony, references to state survey responses 
include the District of Columbia. Forty-six of these states reported 
that they are developing or operating a SACWIS. Nevada, which HHS 
reported has an operational SACWIS, did not respond to our survey.

[2] We are currently conducting an engagement on states' and HHS's 
experiences in conducting the CFSRs.

[3] Forty-four states provided information on the total amount of 
federal funds they received to develop and operate SACWIS. Alaska, 
Hawaii, Missouri, North Carolina, Texas, and Vermont did not report 
federal funding information. Nevada did not respond to our survey. 
State-reported figures may include some funding allocated in fiscal 
year 2003, since the survey was issued in October 2002 and completed as 
late as December 2002. 

[4] Forty-four states provided information on the total amount of state 
funds used to develop and operate SACWIS. Arkansas, Hawaii, Missouri, 
North Carolina, Texas, and Vermont did not report state funding 
information. Nevada did not respond to our survey. State-reported 
figures may include some funding allocated in fiscal year 2003 since 
the survey was issued in October 2002 and completed as late as December 
2002. 

[5] This figure includes developmental funds allocated by HHS to 49 
states and the District of Columbia. Hawaii did not take any federal 
money for SACWIS development. 

[6] This figure includes operational funds allocated to 35 states. 
States begin claiming operational costs when some or all components of 
their SACWIS are operating in local offices. Operational activities 
include routine maintenance, minor enhancements, and other changes that 
do not significantly increase or modify the functionality of the 
system.

[7] According to HHS officials, prior to fiscal year 2000, states 
reported SACWIS operational expenses as part of their Title IV-E 
administrative expenses because the claims sheet states used for 
reporting did not have a separate column for SACWIS operational 
expenditures. In fiscal year 2000, states were required to use a claims 
sheet that was reformatted to provide space for SACWIS operational 
expenditures. In addition, an HHS official explained that the 
difference between the state-reported figures and the federal figures 
may be due to states claiming some SACWIS expenses under different 
programs, such as Title IV-E administrative funds, rather than 
separately as SACWIS expenses.

[8] Although the Iowa state officials described their SACWIS as 
including the child abuse and neglect system, HHS commented on a draft 
of the July 2003 report that it does not view the child abuse and 
neglect system as part of the state's SACWIS. However, HHS said that 
the state has met the SACWIS requirement in this area by building an 
interface between the two systems. 

[9] New Jersey reported in our survey that it had spent approximately 
$9 million in federal funds and $4 million in state and local funds on 
system development. According to HHS, New Jersey first received federal 
funds in 1996.

[10] Twelve of the 46 states reporting that they are developing or 
operating a SACWIS reported that they have not experienced delays in 
developing their systems. In response to the length of the delays 
reported by 26 states in our survey, ACF commented on a draft of the 
July 2003 report that these states may be using different definitions 
in defining their delays. However, ACF did not provide further 
information on how the delays represented in that report differ from 
its perception of states' experiences. In our survey, we asked states 
to report on the delays that exceeded the time line outlined in their 
initial APD. 

[11] With regard to the budget difficulties that states reported 
facing, since 1994 the federal government has made a commitment to help 
states develop and maintain their SACWIS by matching 75 percent of 
states' development funds through 1997 and providing an ongoing match 
of 50 percent of state funding for the development and maintenance of 
their systems. However, since the states' legislatures must make the 
initial commitment to fund SACWIS, the federal government cannot assist 
state child welfare agencies with this challenge.

[12] In commenting on a draft of the July 2003 report, HHS indicated 
that a Web resource is available to states interested in learning about 
other states' efforts to develop human services--child welfare, food 
stamps, Temporary Assistance to Needy Families, child care, and child 
support enforcement--information systems at http://www.acf.hhs.gov/
nhsitrc.

[13] Data are reliable when they are complete and accurate. A 
subcategory of accuracy is consistency. Consistency refers to the need 
to obtain and use data that are clear and well defined enough to yield 
similar results in similar analysis. See U.S. General Accounting 
Office, Assessing the Reliability of Computer-Processed Data, 
GAO-02-15G (Washington, D.C.: Sept. 2002).

[14] States were asked the extent to which certain problems may 
decrease the quality of the data submitted to AFCARS and NCANDS using 
the following scale: very great, great, moderate, some, and no affect. 

[15] The analysis of survey responses about reporting data to HHS is 
based on responses from 49 states and the District of Columbia. All 
states, regardless of SACWIS development, were asked to complete these 
questions. 

[16] For the July 2003 report, we reviewed AFCARS reports from six of 
the eight states that had been assessed by HHS--Arkansas, Connecticut, 
New Mexico, Texas, Vermont, and Wyoming. HHS conducted reviews in 
Delaware and West Virginia after we completed our analysis. As of 
October 2003, HHS had completed three additional reviews for North 
Dakota, Rhode Island, and Washington.

[17] In commenting on a draft of the July 2003 report, ACF said that 
the finding from the AFCARS reviews indicates that information is often 
defaulted to the response "unable to determine" in order for the 
element not to fail the missing data standard, not that workers are 
recording "unknown"; however, the report findings we used in this 
analysis instruct states to fix the defaults and address caseworker 
practice by enhancing training on the correct use of "unable to 
determine" when noting a child's race.

[18] Department of Health and Human Services, Office of Inspector 
General, Adoption and Foster Care Analysis and Reporting System 
(AFCARS): Challenges and Limitations, OEI-07-01-00660 (Washington, 
D.C.: Mar. 2003).

[19] Guardianship arrangements occur when permanent legal custody of a 
child is awarded to an individual, such as a relative, but the child is 
not legally adopted.

[20] Child Welfare League of America. National Working Group 
Highlights, "Placement Stability Measure and Diverse Out-of-Home Care 
Populations" (Washington, D.C., Apr. 2002).

[21] Respite care provides temporary child care for children away from 
their caretakers.

[22] HHS rates each data element using a four-point scale: (1) the 
AFCARS requirement(s) has not been implemented in the information 
system; (2) the technical system requirements for AFCARS reporting do 
not fully meet the standards; (3) the technical system requirements for 
AFCARS reporting are in place, but there are data entry problems 
affecting the quality of the data; (4) all of the AFCARS requirements 
have been met. According to an HHS official, data elements that have a 
combination of technical and data entry problems are rated as 2 until 
the technical issues are resolved. HHS will then rate the element as a 
3 until the data entry practices are changed.

[23] Current placement setting refers to a pre-adoptive home, foster 
family home-relative, foster family home-nonrelative, group home, 
institution, supervised independent living, runaway, or trial home 
visit.

[24] See U.S. General Accounting Office, Executive Guide: Improving 
Mission Performance Through Strategic Information Management and 
Technology, GAO/AIMD-94-115 (Washington, D.C.: May 1, 1994); Center for 
Technology in Government, University of Albany, SUNY. Tying a Sensible 
Knot: A Practical Guide to State-Local Information Systems. Albany, 
N.Y., June 1997.

[25] The Child Support Enforcement Program is a joint federal, state, 
and local partnership that was established in 1975 under Title IV-D of 
the Social Security Act. Each state runs a child support program, which 
provides four major services: locating noncustodial parents, 
establishing paternity, establishing child support obligations, and 
collecting child support for families.

[26] In 1996, the Congress created the block grant Temporary Assistance 
for Needy Families program replacing the Aid to Families with Dependent 
Children (AFDC) and related welfare programs. States were given 
increased flexibility in designing the eligibility criteria and benefit 
rules, which require work in exchange for time-limited benefits.