This is the accessible text file for GAO report number GAO-12-312 entitled 'Foster Care Program: Improved Processes Needed to Estimate Improper Payments and Evaluate Related Corrective Actions' which was released on March 28, 2012. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Requesters: March 2012: Foster Care Program: Improved Processes Needed to Estimate Improper Payments and Evaluate Related Corrective Actions: GAO-12-312: GAO Highlights: Highlights of GAO-12-312, a report to congressional requesters. Why GAO Did This Study: Each year, hundreds of thousands of the nation’s most vulnerable children are removed from their homes and placed in foster care. While states are primarily responsible for providing safe and stable out-of- home care for these children, Title IV-E of the Social Security Act provides federal financial support. The Administration for Children and Families (ACF) in the Department of Health and Human Services (HHS) is responsible for administering and overseeing federal funding for Foster Care. Past work by the HHS Office of Inspector General (OIG), GAO, and others have identified numerous financial deficiencies associated with the Title IV-E Foster Care program. GAO was asked to determine the extent to which (1) ACF’s estimation methodology generated a reasonably accurate and complete estimate of improper payments across the Foster Care program and (2) ACF’s corrective actions reduced Foster Care program improper payments. To complete this work, GAO reviewed HHS’s fiscal year 2010 improper payments estimation procedures, conducted site visits, and met with cognizant ACF officials. What GAO Found: Although ACF has established a process to calculate a national improper payment estimate for the Foster Care program, the estimate is not based on a statistically valid methodology and consequently does not reflect a reasonably accurate estimate of the extent of Foster Care improper payments. In addition, the estimate deals with only about one-third of the federal expenditures for Foster Care, and is therefore incomplete. ACF’s methodology for estimating Foster Care improper payments was approved by the Office of Management and Budget (OMB) in 2004 with the understanding that continuing efforts would be taken to improve the accuracy of ACF’s estimates of improper payments in the ensuing years. ACF, however, continued to generally follow its initial methodology which, when compared to federal statistical guidance and internal control standards, GAO found to be deficient in all three phases of ACF’s estimation methodology—planning, selection, and evaluation. These deficiencies impaired the accuracy and completeness of the Foster Care program improper payments estimate of $73 million reported for fiscal year 2010. Table: Deficiencies in ACF’s Methodology to Estimate Foster Care Improper Payments: Estimation methodology phase: Planning; Deficiencies by phase: * Methodology is limited to identifying improper payments for only one- third of the total federal share of foster care expenditures— maintenance payments. * The case-level population data ACF used to derive its foster care improper payment estimate does not contain the associated payment data needed for a direct estimate of the payment error rate and the total amount of dollars that were improperly paid. Estimation methodology phase: Selection; Deficiencies by phase: * ACF has not established up-front data quality procedures over the case-level population data that states report and that ACF uses for improper payments estimation. * Sample selection process includes a high percentage of replacement cases due to inaccurate information contained in the case-level population data. Estimation methodology phase: Evaluation; Deficiencies by phase: * Methodology does not include procedures on how to identify payment errors related to underpayments and duplicate payments during the review of sampled cases across states. * Methodology used to aggregate state-level improper payment data does not take into account each state’s margin of error, which is needed to calculate an overall program improper payment estimate with a 90 percent confidence level generally required by OMB guidance. Source: GAO analysis of ACF’s methodology to estimate Foster Care improper payments. [End of table] ACF has reported significantly reduced estimated improper payments for its Foster Care maintenance payments, from a baseline of 10.33 percent for fiscal year 2004 to a 4.9 percent error rate for fiscal year 2010. However, the validity of ACF’s reporting of reduced error rates is questionable. GAO found that ACF’s ability to reliably assess the extent to which its corrective actions reduced improper payments was impaired by weaknesses in its requirements for state-level corrective actions. For example, ACF used the number of cases found in error rather than the dollar amount of improper payments identified to determine whether or not a state was required to implement corrective actions. As such, some states with higher improper payment dollar error rates were not required to implement actions to reduce these rates. GAO also found deficiencies in ACF’s Audit Resolution Tracking and Monitoring System that limited its ability to efficiently track and compare trends across states’ Single Audit findings. What GAO Recommends: GAO is making seven recommendations to help improve ACF’s methodology for estimating improper payments for the Foster Care program and its corrective action process. HHS agreed that its improper payment estimation efforts can and should be improved, and generally concurred with four of the recommendations and agreed to continue to study the remaining three recommendations. GAO reaffirms the need for all seven recommendations. View [hyperlink, http://www.gao.gov/products/GAO-12-312]. For more information, contact Susan Ragland at (202) 512-8486 or raglands@gao.gov. [End of table] Contents: Letter: Background: ACF's Methodology for Estimating Foster Care Program Improper Payments Is Not Statistically Valid or Complete: Validity of Reported Foster Care Program Improper Payment Reductions is Questionable: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: Title IV-E Expenditures by Type and by State for Fiscal Year 2010: Appendix III: Estimated Improper Payment Error Rates by State for Fiscal Year 2010 Reporting: Appendix IV: Comments from the Department of Health and Human Services: Appendix V: GAO Contact and Staff Acknowledgments: Tables: Table 1: Eligibility Criteria for the Title IV-E Foster Care Program: Table 2: Deficiencies in ACF's Methodology to Estimate Foster Care Improper Payments: Figures: Figure 1: Title IV-E Federal Share of Expenditures by Type for Fiscal Year 2010: Figure 2: Title IV-E Eligibility Review 3-Year Cyclical Process: Figure 3: Foster Care Program Outlays for Fiscal Year 2010 Covered under IPIA Reporting: Figure 4: ACF's Calculation to Derive the National Improper Payment Estimate for the Foster Care Program: Abbreviations: ACF: Administration for Children and Families: ACR: Administrative Cost Review: AFCARS: Adoption and Foster Care Analysis and Reporting System: ARTMS: Audit Resolution Tracking and Monitoring System: HHS: Department of Health and Human Services: IPERA: Improper Payments Elimination and Recovery Act of 2010: IPIA: Improper Payments Information Act of 2002: OIG: Office of Inspector General: OMB: Office of Management and Budget: PIP: Program Improvement Plan: SACWIS: Statewide Automated Child Welfare Information System: [End of section] United States Government Accountability Office: Washington, DC 20548: March 7, 2012: The Honorable Thomas R. Carper: Chairman: The Honorable Scott Brown: Ranking Member: Subcommittee on Federal Financial Management, Government Information, Federal Services, and International Security: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable John McCain: United States Senate: Each year, hundreds of thousands of our nation's most vulnerable children are removed from their homes and placed in foster care, often due to abuse or neglect. While states are primarily responsible for providing safe and stable out-of-home care for these children until they are returned safely home, placed with adoptive families, or placed in other arrangements, Title IV-E of the Social Security Act provides states some federal financial support in this area.[Footnote 1] The Administration for Children and Families (ACF) under the Department of Health and Human Services (HHS) is responsible for administering this program and overseeing Title IV-E funds. HHS's reported fiscal year 2010 outlays to states for their Foster Care programs under Title IV-E totaled more than $4.5 billion serving over 408,000 children as of September 30, 2010. Past work by the HHS Office of Inspector General (OIG), GAO, and others have identified numerous deficiencies in state claims associated with the Title IV-E Foster Care program. In particular, the HHS OIG found hundreds of millions of dollars in unallowable claims associated with Title IV-E funding.[Footnote 2] A 2006 GAO report also found variations in costs states claimed under the Title IV-E program and recommended a number of actions HHS should take to better safeguard federal resources.[Footnote 3] In addition, annual state- level audits have identified weaknesses in states' use of federal funds, such as spending on unallowed activities or costs and inadequate state monitoring of federal funding.[Footnote 4] As required under the Improper Payments Information Act of 2002 (IPIA), as amended,[Footnote 5] HHS has identified its programs that are susceptible to significant improper payments, including the Foster Care program.[Footnote 6] HHS has reported annually on estimated improper payment amounts for the Foster Care program since 2005. [Footnote 7] In its fiscal year 2010 agency financial report, HHS reported estimated improper payments for Foster Care of about $73 million,[Footnote 8] which contributed to the total governmentwide improper payment estimate of over $125 billion for that year.[Footnote 9] Given continuing concerns about financial vulnerabilities in the Foster Care program, you asked us to conduct a review of the program's improper payment estimate and reduction strategies. Specifically, our objectives were to (1) determine the extent to which ACF's estimation methodology generated a reasonably accurate and complete estimate of Foster Care improper payments and (2) determine the extent to which ACF's corrective actions reduced improper payments. To address these objectives, we reviewed applicable improper payment legislation, Title IV-E of the Social Security Act, as amended, and HHS regulations on Title IV-E eligibility reviews, related Office of Management and Budget (OMB) guidance and ACF's internal guidance, results from Title IV-E eligibility reviews, past GAO and OIG reports, and internal control standards. We also reviewed improper payment information reported in the Improper Payments Section of HHS's fiscal year 2010 agency financial report. To further address the first objective, we performed independent assessments of ACF's methodology, including statistical sampling analysis using OMB's statistical guidance, GAO guidance, and federal internal control standards as criteria to determine the accuracy and completeness of ACF's reported fiscal year 2010 improper payment estimate for the Foster Care program.[Footnote 10] As part of this assessment, we conducted interviews with ACF officials and its contractor to clarify our understanding of the methodology. We also obtained information about ACF's pilot to develop a methodology for estimating improper payments related to administrative costs for the Title IV-E Foster Care program. The scope of our review did not include an assessment of individual states' processes or payment systems. To further address the second objective, we reviewed agency policies and procedures and states' Program Improvement Plans (PIP) used to address the root causes of improper payments identified from the Title IV-E eligibility reviews, which ACF conducts to ensure that states are claiming federal reimbursement only for eligible children; evaluated the compliance thresholds ACF uses to require states to implement corrective actions; and conducted interviews with officials from ACF's central office. We also obtained information about other ACF monitoring activities over states, such as its tracking and monitoring of states' efforts to address the findings identified in the Single Audits.[Footnote 11] In addition, we conducted site visits at 3 of ACF's 10 regional offices (Philadelphia, PA; Chicago, IL; and San Francisco, CA) where we interviewed ACF staff on how Title IV-E eligibility reviews are conducted and how they work with states to implement corrective actions and follow up on Single Audit findings. We selected these three regional offices for site visits because they provided oversight of states that collectively claimed over half of the total federal share of Foster Care maintenance payments made in fiscal year 2009, the most recent data available at the time of our review for site visit selection. We also selected these regional offices to achieve variation in the numbers of error cases and amount of disallowed claims found during Title IV-E eligibility reviews. We conducted this performance audit from February 2011 through March 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. See appendix I for additional details on our objectives, scope, and methodology. Background: Foster Care Program: Title IV-E of the Social Security Act provides for states to obtain federal reimbursement for the costs of their Foster Care programs. [Footnote 12] While states may provide foster care services to a range of children outlined by state laws and regulations, they may only claim Title IV-E Foster Care funds for children meeting eligibility criteria outlined in the Social Security Act (see table 1). Table 1: Eligibility Criteria for the Title IV-E Foster Care Program: Title IV-E Eligibility Criteria: The state, Indian tribe, tribal organization, or tribal consortium has a foster care plan approved by the Secretary of Health and Human Services that meets defined statutory criteria; Social Security Act Sections: 471(a), (b); 479c(b). Title IV-E Eligibility Criteria: The child was placed in a state- licensed foster family home or child-care institution in accordance with either a voluntary placement agreement between the state agency and the child's legal guardian(s), or a judicial determination that conditions in the home from which the child was removed were contrary to the child's welfare; Social Security Act Sections: 472(a)(1)-(2), (b), (c), and (f). Title IV-E Eligibility Criteria: A judicial determination for removal of a child must include a determination that: * reasonable efforts have been made to preserve and reunify families by preventing or eliminating the need for removing the child or making it possible for the child to safely return, or; * other reasonable efforts have been made in accordance with a plan for the permanent placement of the child, if efforts to preserve the family are inappropriate (e.g., in instances of violent crime) or determined to be inconsistent with the permanency plan; that conditions in the home from which the child was removed were contrary to the child's welfare and reasonable efforts were made to prevent removal; Social Security Act Sections: 472(a)(1) and (2); 471(a)(15). Title IV-E Eligibility Criteria: The child's placement and care are the responsibility of a state agency (or other public agency acting under an agreement with a state agency), Indian tribe, tribal organization, or tribal consortium operating under a plan approved by the Secretary of Health and Human Services; Social Security Act Sections: 472(a)(1) and (2)(B). Title IV-E Eligibility Criteria: But for the removal of the child from the home, the child would have received aid under the Aid to Families with Dependent Children program as it was in effect on July 16, 1996, had it been applied for in the month in which the voluntary agreement or judicial determination was made; and; Social Security Act Sections: 472(a)(1) and (3). Title IV-E Eligibility Criteria: If removal is the result of a voluntary placement agreement, eligibility terminates 180 days after removal unless a state obtains a judicial determination that continued placement is in the child's best interest; Social Security Act Sections: 472(d), (e), and (f). Source: Title IV-E of the Social Security Act., codified, as amended, at 42 U.S.C. §§ 670-679c. [End of table] Title IV-E authorizes states to receive federal reimbursement for "maintenance payments" to support expenses for a foster care child, such as food, clothing, shelter, and school supplies. The federal government matches the amounts states pay for maintenance costs under the Medicaid rate. The Medicaid rate varies by state and by year and, for fiscal year 2010, ranged from 50 to 83 percent. In addition to maintenance costs, Title IV-E authorizes states to receive reimbursement for other costs incurred to manage the program. Those other costs and the allowable reimbursement rates[Footnote 13] fall under the following three main categories: * Child placement services and other administrative activities (administrative costs), which generally cover expenses states incur in identifying eligible children, referring them to services, and planning for permanent placement. These can also include administrative costs used to serve foster care "candidate" children, who are at-risk for foster care but still reside in the home. These costs are matched at 50 percent. * State and local training costs (training costs), which are matched at 75 percent. * Statewide Automated Child Welfare Information System (SACWIS) development, installation, and operation costs (SACWIS costs). SACWIS helps states manage their child welfare cases and report related information to the federal government. These SACWIS costs are matched at 50 percent. Since 2002, HHS has also approved states to receive federal reimbursement for demonstration project costs involving the waiver of certain provisions of Title IV-E.[Footnote 14] The waivers grant states flexibility in the use of Title IV-E foster care funds for "demonstration projects" of alternative services that promote safety, permanency, and well-being for children in the foster care system, so long as the projects do not cost the federal government more than the states would have received under Title IV-E. As of June 2010, nine states have active Title IV-E waiver agreements.[Footnote 15] Program Trends and Costs: Data from HHS shows that the average number of children served by Title IV-E Foster Care funds has declined, from over 197,000 in fiscal year 2008 to 181,000 in fiscal year 2010. HHS and child welfare experts have cited a number of reasons for this decline. For example, they noted that a child is required to qualify for the Aid to Families with Dependent Children program (a means- tested program based on a federally defined poverty line) as it was in effect on July 16, 1996, in order to be eligible for Title IV-E. Because income limits for the program have remained static while inflation has raised nominal incomes for some families, fewer children are eligible. For example, to be considered as below the federal poverty line, a family comprised of 4 persons, including 2 children, had to have an annual income below $15,911 in 1996, as compared to $22,113 in 2010. However, the $15,911 threshold continues to be used each year to determine eligibility for the Title IV-E Foster Care program. In addition, states sometimes use other federal programs for children who could otherwise have been claimed under Title IV-E, because the other programs, such as Title XX's Social Services Block Grants, Medicaid, and Temporary Assistance to Needy Families, provide federal reimbursement for a broader range of services.[Footnote 16] Of the $4.5 billion in total Title IV-E Foster Care funds paid to states in fiscal year 2010, ACF reported that maintenance costs made up 34 percent of the total while administrative costs accounted for the largest share of the costs at 44 percent. Figure 1 shows federal outlays, as reported by HHS, by type of expenditure for fiscal year 2010. Title IV-E expenditures by type and by state for fiscal year 2010 are presented in appendix II. Figure 1: Title IV-E Federal Share of Expenditures by Type for Fiscal Year 2010: [Refer to PDF for image: pie-chart] Administrative costs: 44%; Maintenance costs: 34%; Demonstration project costs: 15%; Training costs: 4%; SACWIS costs: 3%. Source: GAO analysis of unaudited Foster Care expenditure data for fiscal year 2010. [End of figure] Oversight Activities: ACF is responsible for the administration and oversight of Title IV-E funding to states. ACF staff are located in ACF's headquarters (Central Office) and its 10 regional offices.[Footnote 17] Collectively, these ACF offices oversee states' financial internal control processes for the Title IV-E program and monitor their performance and compliance with federal child welfare laws. One key oversight activity related to state Foster Care programs is ACF's Title IV-E eligibility reviews, as required under the Social Security Act and HHS regulations.[Footnote 18] ACF has conducted these reviews since 2000. They are intended to help evaluate whether state claims for federal reimbursement for Foster Care maintenance costs are valid and accurate. Title IV-E eligibility reviews are to be conducted by teams composed of both federal and state staff, and are to include (1) desk reviews to ensure that the correct amount of maintenance costs was claimed on behalf of foster care children during the review period and (2) site visits to states to ensure that maintenance costs were claimed only for children who were eligible for the Title IV-E program. As required by the Social Security Act and HHS regulations, there are two stages of Title IV-E eligibility reviews, a primary and secondary review. During a primary review, HHS regulations specify that the review team is to examine a sample of 80 cases per state, selected from the Adoption and Foster Care Analysis and Reporting System (AFCARS).[Footnote 19] Each case represents a child for whom a Title IV-E Foster Care maintenance payment was made. If a primary review finds fewer than 5 cases with errors in either the amounts paid on behalf of a child or in a child's eligibility for Title IV-E funds (5 percent of the cases reviewed or fewer), ACF determines that the state is in substantial compliance with the regulations. At that point, the state is scheduled to have another primary review in 3 years.[Footnote 20] On the other hand, if a primary review finds 5 or more cases are in error (exceeding 5 percent of the number of cases reviewed), ACF determines that the state is not in substantial compliance with the regulations. In those instances, ACF requires states to develop a Program Improvement Plan (PIP) designed to correct the areas of noncompliance identified, such as payments to unlicensed providers or incomplete criminal record checks. Any improper payments the review teams identify during these reviews are classified as disallowed costs that, in general, are to be returned to ACF or withheld from future reimbursement claims.[Footnote 21] States required to develop a PIP generally have 1 year to implement the corrective actions specified in the PIP, after which a secondary review is to be conducted.[Footnote 22] During the secondary review, the review team is to examine a sample of 150 cases, as outlined in HHS regulations. If 10 percent of cases or fewer are found to be in error and if the total dollar amount found to be in error is less than 10 percent of the total dollar amount reviewed, then ACF determines that the state is in substantial compliance. Further, if the state exceeded only one of these secondary review error thresholds, then ACF would also determine that the state is in substantial compliance. Only in instances where the state exceeds both the case percentage and dollar percentage error thresholds of 10 percent would ACF determine that the state is not in substantial compliance. HHS regulations require such a state to repay a disallowance percentage applied to its Title IV-E claims during the review period. After conducting a secondary review, ACF would then schedule another primary review in 3 years (see figure 2). Figure 2: Title IV-E Eligibility Review 3-Year Cyclical Process: [Refer to PDF for image: process illustration] 1) Team (ACF/state) conducts primary review (80 cases). 2) Does the number of cases with errors exceed the 5% threshold? If yes, go to #3; If no, State is deemed substantially compliant; go to #11. 3) State is deemed not to be in substantial compliance. 4) Within 30 days of completion of review, ACF publishes report and notifies state of non-compliance. 5) Within 90 days of notification, state develops and submits Program Improvement Plan (PIP). 6) Within 30 days of receipt of PIP, ACF approves or rejects PIP and notifies state. 7) State has one year to implement PIP. 8) Team (ACF/state) conducts secondary review (150 cases). 9) Do both he number of cases with errors and the total dollar value of errors exceed the 10% threshold? If yes, go to #10; If no, go to #11. 10) ACF extrapolates disallowance against state's claims. 11) Within 30 days of completion of review, ACF publishes report and notifies state of compliance determination; Return to #1. Source: GAO analysis of ACF's process for Title IV-E eligibility reviews. [End of figure] Another key ACF oversight activity related to state Foster Care programs is the monitoring of findings from state-level audits conducted under the Single Audit Act and OMB Circular No. A-133, known as Single Audits.[Footnote 23] The Single Audit Act requires an annual audit of states, local governments, and non-profit organizations that expend $500,000 or more of federal funds in a given year. ACF regional offices are to work with states to resolve Single Audit findings related to the Foster Care program to help ensure that states are using funds in accordance with program requirements and addressing financial management weaknesses. ACF started using the Audit Resolution Tracking and Monitoring System (ARTMS) in 2010 to provide online processing and real-time tracking of ACF's audit follow-up process. The National External Audit Review Center is a specialized function of the HHS OIG that serves as a clearinghouse to determine which state Single Audit report findings HHS is responsible for resolving. When ACF receives Single Audit finding data from the OIG's National External Audit Review Center, HHS headquarters staff upload the data into ARTMS and assign the audit finding data to the appropriate ACF regional office staff for resolution. Consistent with OMB Circular No. A-50, ACF considers an audit finding resolved when the auditor and the state agree on action to be taken.[Footnote 24] ARTMS provides users with notifications of tasks to be performed, such as when an audit is assigned to a financial management specialist for follow-up, and allows them to submit and view all audit resolution information online. Improper Payments Information Act of 2002 and Implementing Guidance: The Improper Payments Information Act (IPIA) was enacted in November 2002 to enhance the accuracy and integrity of federal payments. [Footnote 25] IPIA requires agencies to: * review all programs and activities and identify those that are susceptible to significant improper payments; * obtain a statistically valid estimate of the annual amount of improper payments, including the gross total of over-and underpayments, in those susceptible programs and activities; * report to the Congress estimates of the annual amount of improper payments in their susceptible programs and activities and; * for estimates exceeding $10 million, implement a plan to reduce improper payments. OMB's implementing guidance for IPIA, in effect for fiscal year 2010 reporting, required that for any programs and activities identified as susceptible to significant improper payments, agencies must develop a statistically valid methodology, or other methodology approved in advance by OMB, to estimate the annual amount of improper payments, including a gross total of both underpayments and overpayments. [Footnote 26] The Foster Care program was deemed a risk susceptible program and therefore required to address the IPIA reporting requirements.[Footnote 27] OMB guidance also requires that, as part of their plan to reduce improper payments for all programs and activities with improper payments exceeding $10 million, agencies identify the reasons their programs and activities are at risk of improper payments (also known as root causes), set reduction targets for future improper payment levels and a timeline within which the targets will be reached, and ensure that agency managers and accountable officers are held accountable for reducing improper payments. ACF annually reports to HHS--for inclusion in its agency financial report used to report to the Congress--an improper payment estimate for Foster Care program maintenance payments based on results of Title IV-E eligibility reviews required under the Social Security Act. For programs administered at the state level such as Foster Care, OMB guidance provides that statistically valid estimates of improper payments may be provided at the state level either for all states or for all sampled states annually. These state-level improper payment estimates should then be used to generate a national dollar estimate and improper payment rate. With prior OMB approval, ACF has taken its existing Title IV-E eligibility review process, already in place under the Social Security Act, and leveraged it for IPIA estimation. OMB granted this approval in December 2004 with the expectation that continuing efforts would be taken to improve the accuracy of ACF's estimates of improper payments in the ensuing years. ACF provides a national estimated error rate based on a rolling average of error rates identified in states examined on a 3-year cycle. As a result, ACF's IPIA reporting for each year is based on new data for about one- third of the states and previous years' data for the remaining two- thirds of states. While each state sample represents a distinct 6- month period under review, the national "composite sample" reflects a composite period under review that encompasses a 3-year period (the Title IV-E eligibility review 3-year improper payment cyclical process). To calculate a national estimate of improper payments, ACF uses error rates that span a 3-year period of Title IV-E eligibility reviews in the 50 states, the District of Columbia, and Puerto Rico. ACF applies the percentage dollar error rate from the sample to the total payments for the period under review for each state. Improper payment error rates by state for fiscal year 2010, as calculated by ACF, can be found in appendix III. On July 22, 2010, the Improper Payments Elimination and Recovery Act of 2010 (IPERA) was enacted. IPERA amended IPIA, and established additional requirements related to federal agency management accountability, compliance and noncompliance determinations based on an Inspector General's assessment of an agency's adherence to IPERA requirements and reporting that determination, and an opinion on internal controls over improper payments. Specifically, one new IPERA provision calls for federal agencies' Inspectors General to annually determine whether their respective agencies are in compliance with key IPERA requirements such as meeting annual reduction targets for each program assessed to be at risk of and measured for improper payments, and to report on their determinations to the agency head, the Congress, and the Comptroller General. ACF's Methodology for Estimating Foster Care Program Improper Payments Is Not Statistically Valid or Complete: ACF's methodology, which resulted in a reported $73 million (or 4.9 percent) estimate of improper payments in the Foster Care program for fiscal year 2010,[Footnote 28] had deficiencies in all three phases of its estimation methodology--planning, selection, and evaluation--when compared to OMB's statistical guidance, GAO guidance, and federal internal control standards, as summarized in table 2. Specifically, ACF's estimation methodology (1) did not consider nearly two-thirds of reported federal Foster Care program payments for fiscal year 2010, (2) was not based on a probability sample of payments, (3) lacked specific procedures for identifying underpayments and duplicate payment errors, and (4) used a flawed process for aggregating state- level data into an overall national error rate. As a result, ACF's methodology is not statistically valid or complete, and these deficiencies impair the accuracy and completeness of its reported Foster Care program improper payment estimate. Table 2: Deficiencies in ACF's Methodology to Estimate Foster Care Improper Payments: Estimation methodology phase: Planning; Deficiencies by phase: * Methodology is limited to identifying improper payments for only one- third of the total federal share of foster care expenditures-- maintenance payments; * The case-level population data used to derive the foster care improper payment estimate does not contain the associated payment data needed for a direct estimate of the payment error rate and the total amount of dollars that were improperly paid. Estimation methodology phase: Selection; Deficiencies by phase: * ACF has not established up-front data quality procedures over the case-level population data, self-reported by states, prior to sample selection; * Sample selection process includes a high percentage of replacement cases due to inaccurate information contained in the case-level population data. Estimation methodology phase: Evaluation; Deficiencies by phase: * Methodology does not include procedures on how to identify payment errors related to underpayments and duplicate payments during the review of sampled cases across states; * Methodology used to aggregate state-level improper payment data does not take into account each state's margin of error, which is needed to calculate an overall program improper payment estimate with a 90 percent confidence level generally required by OMB guidance. Source: GAO analysis of ACF's methodology to estimate Foster Care improper payments. [End of table] Methodology Excludes Most Foster Care Program Expenditures from Estimate: ACF's annual IPIA reporting for the Foster Care program is incomplete, as it is limited to identifying improper payments for only one type of program payment activity--maintenance payments. For fiscal year 2010, as shown in figure 1 of this report, maintenance payments represented 34 percent of the total federal share of expenditures for the Foster Care program. Administrative and other payments were not considered in ACF's IPIA estimation process and thus, not included in the Foster Care program improper payment estimate of about $73 million for fiscal year 2010.[Footnote 29] Administrative costs accounted for 44 percent of the total federal share of expenditures for the Foster Care program, while other costs accounted for the remaining 22 percent.[Footnote 30] These other costs include operational and development costs associated with SACWIS; training costs; and state demonstration projects to provide alternative services and support for children in the Foster Care system. Figure 3 shows the portion of Foster Care program outlays considered in ACF's methodology for estimating associated improper payments for IPIA reporting. Figure 3: Foster Care Program Outlays for Fiscal Year 2010 Covered under IPIA Reporting: [Refer to PDF for image: illustration] 2010 Total program outlays: $4.5 billion. Portion of 2010 total subjected to IPIA reporting (34%): Maintenance costs: $1.5 billion. Portion of 2010 total not subjected to IPIA reporting (66%): Administrative costs: $2.0 billion; SACWIS costs: $0.1 billion; Training costs: $0.2 billion; Demonstration project costs: $0.7 billion. Source: GAO analysis of Foster Care expenditure data for fiscal year 2010. [End of figure] Because ACF's methodology does not include an estimate for improper payments related to its administrative payment activity, the related payment errors that meet the definition of improper payments were not accounted for or included in the reported estimate for the Foster Care program. OMB's December 2004 approval of ACF's proposed methodology included an expectation that ACF would develop a plan and timetable to test administrative expenses by April 2005. Consistent with this expectation, in order to begin exploring the issues of accounting for and including administrative costs, ACF established a working group in 2006. Then, in 2007, ACF initiated an Administrative Cost Review (ACR) pilot to examine how certain state agencies accumulate costs that are included in their expenditure claims for federal financial participation[Footnote 31] and to identify improper administrative payments within those pilot states.[Footnote 32] Seven states volunteered for pilots held from fiscal years 2007 through 2011, and two more states are scheduled for fiscal year 2012.[Footnote 33] Pilot reports for two states have provided estimates for a gross improper payment total of $11.3 million for the period October 1, 2008, through March 31, 2009. These amounts were not included in ACF's estimated amounts for improper payments. According to ACF, it will use the results of the ACR pilots to determine the feasibility of developing a methodology to estimate an administrative error rate as part of the calculation of the national Foster Care improper payment error rate. However, as of December 15, 2011, ACF had not yet made a decision with respect to when these reviews would be implemented and, ultimately, whether to establish a methodology to estimate improper payments related to administrative costs. Although ACF did not consider Foster Care administrative expenditures in its fiscal year 2010 IPIA estimation process, its Title IV-E eligibility reviews identified disallowed administrative costs (or improper payment amounts) which were added to the amount of any claims disallowances. For fiscal year 2010, disallowed administrative costs that ACF documented from the Title IV-E eligibility reviews totaled $2.4 million; however, this amount was not included in its Foster Care improper payment estimate. According to ACF, administrative payments are not currently included as part of the reported improper payment estimate because this disallowed amount is based on a calculation and not directly determined from a case file review. ACF calculates the administrative cost disallowance by allocating an average administrative cost for any ineligible time period identified during a Title IV-E eligibility review. ACF's Methodology for Selecting Cases to Review Is Also Deficient: The methodology ACF used to estimate improper maintenance payments was not based on a probability sample of payments, which is needed for a direct estimate of the payment error rate and total amount of dollars that were improperly paid.[Footnote 34] In 2004, ACF proposed three options to OMB as approaches for estimating improper payments in the Foster Care program. OMB approved ACF's plan to derive the estimate using error dollars per case from state review samples for its base error rate calculation, with the expectation that continuing attention to the statistical processes used would be needed to obtain the best estimate of erroneous payment rates. OMB's approval reflected the idea that the methods initially used would incorporate annual improvements to the accuracy of improper payment estimates. However, other than a change in 2008 to derive the estimate using sample state dollar error rates, ACF generally continues to use the same methodology outlined in 2004. We found that ACF selected a sample from a universe of all cases receiving Title IV-E Foster Care payments during the period under review. This population of Foster Care cases is drawn from AFCARS. However, AFCARS does not contain any Title IV-E financial data that links a Title IV-E payment amount to each case file. Lacking such payment data, ACF relies on states to provide payment histories for all cases selected for review. According to ACF, it was already using AFCARS to select samples for the Title IV-E eligibility reviews prior to IPIA implementation. Consistent with OMB's approval of its methodology, ACF opted to use AFCARS data instead of creating or identifying a new data source for meeting IPIA requirements. ACF officials stated that utilizing this existing source of data reduced the burden on states by not requiring them to draw their own samples and employed the AFCARS database in a practical manner. Notwithstanding ACF's objective of leveraging existing data to estimate foster care improper payments for addressing IPIA requirements, if ACF is to make an estimate that accurately represents the target population of Title IV-E payments, a more direct statistical method would be needed to select a probability sample of Title IV-E payments. OMB's Standards and Guidelines for Statistical Surveys documents the professional principles and practices that federal agencies are required to adhere to and the level of quality and effort expected in all statistical activities.[Footnote 35] According to these standards and guidelines, probabilistic methods for survey sampling are one of a variety of methods for sampling that give a known, non-zero, probability of selection to each member of the target population. The advantage of probabilistic sampling methods is that sampling error can be calculated. For the purpose of making a valid estimate with a measurable sampling error that represents a population, the sample must be selected using probabilistic methods. The sample results can then be used to make an inference about the target population, in this instance, foster care cases that received a maintenance payment. While it is possible for ACF to estimate a payment error rate and the total amount of dollars improperly paid for maintenance payments using a combination of AFCARS and supplemental payment data from the states, this would require a more complex estimation methodology than ACF currently uses. Based on our review of the sampling documentation provided, ACF did not consider key factors such as variation in volume of payments and dollars of payments across cases and states. In addition, the population of data that ACF used to select its sample from is not reliable because ACF's sampling methodology did not provide for up-front data quality control procedures to (1) ensure that the population of cases was complete prior to its sample selection and (2) identify inaccuracies in the data field used for sample selection[Footnote 36].: During our review, we found that the population of Foster Care cases from AFCARS contained inaccurate information on whether a case had actually received a Title IV-E Foster Care maintenance payment during the period under review, reflecting continuing concerns regarding the accuracy and completeness of AFCARS data. Specifically, ACF had to replace a high percentage of cases sampled from the database of Foster Care cases for the fiscal year 2010 reporting period due to inaccurate information in AFCARS. To ensure that a sufficient number of relevant sample items are available for review, ACF routinely selects an "over- sample" of cases--cases selected in addition to the required 80 or 150 cases initially selected for the primary and secondary reviews. Of the original 4,570 sample cases ACF selected for testing in its primary and secondary reviews for fiscal year 2010, 298 cases (almost 7 percent) had to be replaced with substitutes taken from the "over- sampled" cases because the selected cases had not received Title IV-E Foster Care maintenance payments during the period under review. Of the 298 over-sampled cases used to replace the cases initially selected, 63 cases (more than 21 percent) then had to be replaced again because those cases had also not received Title IV-E Foster Care maintenance payments during the period under review. Although we were able to determine how many sampled (or over-sampled) cases had to be replaced because available records showed no Title IV-E payment was received during the reporting period, neither we nor ACF were able to determine the extent to which the opposite occurred--the extent to which cases that had received a payment (and therefore should have been included in the sample population) had not been coded as receiving Title IV-E payments. As part of its sampling methodology, ACF has not established procedures to identify any such occurrences. Therefore, ACF could not determine whether its sampling universe was complete, i.e., whether all of the cases receiving a Foster Care payment were included in the universe of cases from which it selected sample cases for review. According to GAO's Assessing the Reliability of Computer-Processed Data, reliable data are defined as data that are reasonably complete and accurate, meet intended purposes, and are not subject to inappropriate alteration.[Footnote 37] "Completeness" refers to the extent to which relevant records are present and the fields in each record are populated appropriately. "Accuracy" refers to the extent to which recorded data reflect the actual underlying information. GAO's Internal Control Management and Evaluation Tool provides that reconciliations should be performed to verify data completeness.[Footnote 38] Also, data validation and editing should be performed to identify erroneous data. Erroneous data should be captured, reported, investigated, and promptly corrected. ACF officials told us they are aware that AFCARS does not contain Title IV-E payment data and acknowledged that they do not perform procedures to identify incorrect or missing information in the population prior to sample selection. However, ACF officials said they continue to use the data to meet IPIA reporting requirements because it is the only database that contains case-level information on all children in foster care for whom the state child welfare agencies have responsibility for placement, care, or supervision. However, without developing a statistically valid sampling methodology that incorporates up-front data quality controls to ensure complete and accurate information on the population, including payment data, ACF cannot provide assurance that its reported improper payment estimate accurately and completely represents the extent of improper maintenance payments in the Foster Care program. ACF's Methodology Does Not Include Procedures to Identify the Full Extent of Errors Related to Underpayments and Duplicate or Excessive Payments: In its fiscal year 2010 agency financial report, ACF reported that underpayments and duplicate or excessive payments represented 25 percent of the errors that caused improper payments.[Footnote 39] While ACF's methodology for performing Title IV-E eligibility reviews included written guidance and a data collection instrument that focused on eligibility errors, it did not include procedures on how to search for and identify payment errors related to underpayments and duplicate or excessive payments during case reviews. Rather, ACF's procedures only provided that any observed underpayments and duplicate or excessive payments are to be disclosed as findings in the state's final eligibility review report. Without detailed procedures to guide review teams in the identification of underpayments and duplicate or excessive payments, ACF's methodology cannot effectively assure its review team identifies the full extent to which any such underpayments or duplicate or excessive payments exist in the Foster Care program. As defined in IPIA, improper payments include both overpayments and underpayments, and an agency's estimate should reflect both types of errors. IPIA also includes examples of improper payments, one of which is duplicate payments. According to GAO's Standards for Internal Control in the Federal Government, operational information is needed to determine whether the agency is achieving its compliance requirements under various laws and regulations.[Footnote 40] Information is required on a day-to-day basis, to make operating decisions, monitor performance, and allocate resources. Pertinent information should be identified, captured, and distributed in a form and time frame that permits people to perform their duties efficiently. ACF compiles the results of all state eligibility reviews to determine the most common types of payment errors. ACF officials told us that all review team members receive the same training and the results of the state reviews are analyzed to ensure consistency and reliability. For fiscal year 2010, underpayments were the largest percentage of payment errors (19 percent--as a percentage of all Title IV-E maintenance payment errors identified in sampled cases). Duplicate or excessive payments comprised 6 percent of the payment errors. However, the extent of underpayments and duplicate or excessive payment errors identified varied widely by state, and in some instances were not identified at all.[Footnote 41] The lack of detailed procedures for identifying any such payment errors may have contributed to the variation or whether the teams found any errors. For example, our analysis of the Title IV-E eligibility reviews that comprised the fiscal year 2010 foster care improper payment estimate identified underpayments in 21 of 51 state reviews. Of the 21 states where reviewers had identified underpayments, such payments ranged from 1.3 percent to 12.0 percent of cases reviewed. Similarly, duplicate or excessive payments were identified in only 16 of 51 states. Of the 16 states that had this type of error, these payments ranged from 1.3 percent to 5.0 percent. During our site visits, ACF regional officials told us that states have differing claiming practices for certain expenses. Specifically, officials in one regional office said that if a child became eligible during a specific month, a state could have claimed through the first day of the month for that child, but chose not to so as to not risk a potential error on a future Title IV-E eligibility review. According to these regional officials, the regional offices operate under the presumption that "if a state made the decision not to claim certain expenses, then the failure to claim is not considered an underpayment." These types of decisions would be discussed during the reviews but ACF guidance does not call for decisions and their rationale to be formally documented. According to ACF's Title IV-E eligibility review guide, potential underpayments are to be identified during a review of the case record and payment history. ACF's Title IV- E eligibility review guide provides that payment histories should be submitted, but it does not specify what criteria reviewers are to look for in order to determine instances of underpayments or duplicate or excessive payments. In August 2011, ACF issued a new attachment to the Title IV-E eligibility review guide to provide a tool for calculating and reporting underpayments identified during eligibility reviews. This new attachment provides a template for recording underpayments for the period under review, but it does not provide guidance on how to identify underpayments. Instances of duplicate or excessive payments are to be reported on other existing attachments in the Title IV-E eligibility review guide. However, none of these attachments offer additional guidance for how to identify national underpayments or duplicate or excessive payments. ACF's Aggregation Methodology Does Not Generate a National Foster Care Program Improper Payment Estimate: ACF's fiscal year 2010 Foster Care program improper payment estimate did not appropriately aggregate state improper payment data to derive a national improper payment estimate (dollars and error rate). ACF calculated the national estimate of improper payments each year using data collected in the most recent eligibility review for each of 50 states, the District of Columbia, and Puerto Rico. According to the information ACF presented to OMB in December 2004, ACF's methodology would calculate the standard error of each state estimate, and of the national estimate, to examine the extent to which the precision requirements as specified in OMB's implementing guidance for IPIA are met. However, the methodology ACF actually used to aggregate this state-level improper payment data does not take into account each state's margin of error, which is needed to calculate an overall program improper payment estimate with a 90 percent confidence level generally required by OMB guidance. Figure 4 depicts, at a high level, ACF's calculation to derive the national improper payment estimate for the Foster Care program. Figure 4: ACF's Calculation to Derive the National Improper Payment Estimate for the Foster Care Program: [Refer to PDF for image: illustration] Calculation used for new reviews for one-third of states: Total sample improper maintenance payments: divided by: Total sample maintenance payments: equals: Error rate for state (1); multiplied by: Total maintenance payment dollars for the state; equals: Improper maintenance payment dollars for the state (1). Roll-up of state data for all reviews: Aggregate new review year data of 1/3 states' improper maintenance payment dollar estimate with 2/3 states' previous years estimates; divided by: Aggregate of each states' maintenance payment claim for the respective years of their estimate; equals: Current year national Foster Care error rate (2); multiplied by: Current year Foster Care outlays maintenance payments; equals: Current year national Foster Care improper maintenance payment dollars. (1) The calculation of each state's improper payment estimate (dollars and error rate) does not incorporate the margin of error inherent in statistical sampling. The margin of error is often expressed as a confidence interval, which is a range of values that is expected to contain the population value with a specified confidence level.[A] Instead of calculating a confidence interval for each state, ACF calculated an overall, program-wide confidence interval based on the aggregate, 3-year period composite sample results from the state reviews without knowing the margin of error. ACF did not calculate the margin of error surrounding each states' improper payment estimate. Because the state margin of error is not incorporated into the overall estimate, the margin of error of the overall program estimate is unknown. OMB requires agencies to calculate 90 percent confidence intervals for improper payment estimates, absent advance approval of an alternate sampling method.[B] (2)Second, the national foster care dollar error rate is not adjusted for inflation. For fiscal year 2010, ACF estimated improper payments related to its maintenance payments, based on nominal amounts (also called current dollar or then-year values). Our analysis of the state reviews ACF used to develop its fiscal year 2010 improper payment estimate showed that the review dates ranged from March 2007 to July 2010. When using more than 1 year of dollar amounts to develop a statistical estimate, the nominal amounts and the real or inflation- adjusted amounts should be used to generate the estimate.[C] Source: GAO analysis of ACF's calculation to derive the improper payment estimate for the Foster Care program. [A] A confidence interval is a statistical sample-based estimate expressed as a range of values, for example, $100 plus or minus $5. The sample is designed such that there is a specified confidence level for which the population value is being estimated is expected to be located within the interval. [B] OMB's implementing guidance for IPIA requires agencies to consult a statistician to ensure the validity of their sample design, sample size, and measurement methodology. Estimates are to be based on a statistically random sample of sufficient size to yield an estimate with a 90 percent confidence interval of plus or minus 2.5 percentage points. Agencies may alternatively use a 95 percent confidence interval of plus or minus 3 percentage points. However, the guidance also allows agencies to propose alternate sampling approaches for OMB approval prior to implementation. [C] GAO, Inflation Adjustments, (Washington, D.C.: Sept. 14, 2011). [End of figure] Validity of Reported Foster Care Program Improper Payment Reductions is Questionable: ACF has reported significantly reduced estimated improper maintenance payments, from a baseline error rate of 10.33 percent for 2004 to a 4.9 percent error rate for 2010, but the validity of ACF's reporting of reduced improper payment error rates is questionable. Examples of corrective actions ACF has identified include reviews, the requirement for state improvement plans, on-site training and technical assistance to states, and outreach to judicial organizations to educate them as to their role in addressing Foster Care eligibility issues. However, the significant weaknesses discussed previously concerning ACF's estimation methodology impaired the accuracy and completeness of ACF's reported improper payment estimate for the Foster Care program. Further, we found that ACF's ability to reliably assess the extent to which its corrective actions reduced Foster Care program improper payments was impaired by deficiencies in (1) its method for requiring when states implement corrective actions and (2) information technology limitations related to monitoring states' Foster Care program-related Single Audit findings. Deficiencies in ACF's Requirements for State Foster Care Program Improper Payment Corrective Actions: We identified three deficiencies in ACF's process for implementing plans to reduce improper payments. * ACF did not use reported improper payment error rates--which are based on the dollar amount of improper payments identified in a sample of state Foster Care cases--to determine whether or not a state is required to implement corrective actions. * ACF's measure for assessing corrective action effectiveness is through performance on its secondary review of Title IV-E cases, which has a more lenient passing standard than the primary review. * Not all types of payment errors are required to be addressed in the PIP. OMB's implementing guidance for IPIA requires that agencies put in place a corrective action plan to reduce improper payments. In addition, ACF's internal guidance requires states to implement corrective actions through a PIP if, during the Title IV-E primary eligibility review, a state is found to have 5 or more cases in error (exceeding 5 percent of the number of cases reviewed). While ACF identifies state PIPs as a corrective action strategy, it does not use the dollar-based estimated improper payments to determine when a state is required to develop a PIP. Instead, ACF uses the number of sample cases found in error to determine which states should develop a PIP. Therefore, some states with improper payment dollar error rates exceeding 5 percent were not required to implement corrective actions to reduce these rates. For fiscal year 2010 reporting, ACF used the results of 44 primary eligibility reviews and 7 secondary reviews. [Footnote 42] Of the 44 state primary reviews, 13 had dollar-based estimated improper payments greater than 5 percent; however, because ACF uses case error rates as the determining factor for states' compliance with their primary reviews, not all states were required to complete a PIP. Of the 13 states, ACF determined 7 were noncompliant in their primary eligibility reviews because the case error rate exceeded ACF's threshold of 5 percent (more than 4 of the 80 cases were found in error) and thus, were required to complete a PIP. The remaining 6 states were found substantially compliant in their primary reviews as their case error rate was below the established 5 percent threshold (4 or fewer cases were found in error). The dollar-based improper payment rates for those 6 states ranged from 5.1 to 19.8 percent--based on the percentage of improper payment dollars found in the sample. Because improper payment rates are not used in applying the PIP corrective action strategy, ACF's method cannot effectively measure states' progress over time in reducing improper payments. It also cannot effectively help determine whether further action is needed to minimize future improper payments. This limits the extent to which states are held accountable for the reduction of improper payments in the Foster Care program. Upon a state's implementation of its PIP, ACF conducts a secondary review to determine whether errors found during the primary review have been addressed. The secondary review is ACF's principal tool to measure a state's success in implementing actions to reduce Foster Care program improper payments. These reviews carry the potential financial penalty of an extrapolated disallowance of the state's federal share of Title IV-E expenditures if the state is found to be noncompliant. However, because ACF's error threshold to be found noncompliant with a secondary review is twice as high as that of the primary review (10 percent versus 5 percent), it limits ACF's ability to provide an effective incentive for states to focus continuing attention on causes of improper payments. Based on our analysis of ACF's Title IV-E eligibility reviews, 27 states have had at least one secondary review between 2002 and 2010. Of the 27 states that received a secondary review, 26 states passed this review (meaning that the error rates were below the ten percent threshold) and only 1 state failed. Of the 26 states that passed, 13 states (50 percent) would have failed if the primary review error threshold of 5 percent was in effect. Of those 13 states, we found at least 3 states that passed the secondary review with a case error rate over 10 percent because the reported improper payment dollar based error rate was below 10 percent.[Footnote 43] The one state that failed its secondary review in 2003 received an extrapolated disallowance in accordance with HHS regulations. Since the eligibility reviews began in 2000, this is the only state found to be noncompliant with its secondary review. While the extrapolated disallowance is a financial penalty intended to encourage states to address causes of improper payments, after incurring an extrapolated disallowance, this state was again found to be noncompliant based on ACF's subsequent review in 2006. As such, the state was again required to develop and implement a PIP to address the causes of errors identified in this review. After implementing this PIP, the state was subject to another secondary review and was found to be compliant with a case error rate of 6.67 percent and dollar error rate of 2.84 percent. However, this state would have failed if the primary review error threshold of 5 percent was in effect. According to ACF officials, it established the 10 percent threshold for compliance with secondary reviews in 2000 based on states' error rates at that time which were between 15 percent and 17 percent (in terms of both cases and dollars). ACF officials told us the 10 percent appeared to be a target that states could meet to demonstrate reductions in improper payments over time. Also, the baseline estimated improper payment error rate reported for the Foster Care program in 2004 was 10.33 percent. Since establishing the 10 percent threshold in 2000, ACF has not conducted a review to validate the continuing propriety of the performance metric. GAO's Internal Control Management and Evaluation Tool provides that an agency should periodically review and validate the propriety and integrity of both organizational and individual performance measures and indicators.[Footnote 44] According to this tool, performance measurement factors are to be evaluated to ensure they are linked to mission, goals, and objectives, and that they are balanced and set appropriate incentives for achieving goals while complying with law, regulations, and ethical standards. For fiscal year 2010, ACF reported that error rates for most of the states (33 of 51) were less than 5 percent.[Footnote 45] In addition, ACF's process for overseeing states' implementation of improper payment reduction actions has other weaknesses. ACF's guidance only requires that the PIP--required if a state has more than four cases found in error in its primary review--address areas that the eligibility review identified as needing improvement. Consequently, states' corrective action plans may not address all types of previously identified payment errors. There is nothing in the guidance to prevent states from addressing other areas in the PIP. However, based on our review of the guidance for developing PIPs and discussions with Central Office and regional office staff, underpayments and other non-eligibility errors and eligibility errors outside of the period under review might not be addressed in the PIP if these types of errors were not a factor in a state's compliance. Not including all types of errors in states' corrective action plans reduces their effectiveness for addressing the causes of payment errors as required under IPIA. Limitations of ACF's Audit Tracking System Impair Monitoring Corrective Actions: In 2010, ACF began using a departmentwide system, ARTMS, to track and monitor the resolution of audit findings for programs it administers, including audit findings concerning Foster Care program payment errors.[Footnote 46] ACF utilizes ARTMS as its primary tool for monitoring states' resolution of reported Single Audit findings. Single Audit findings for states' Foster Care programs have included, among other issues, deficiencies in state oversight over subrecipients of federal funds, lack of training of state personnel on program eligibility requirements, and potential for unauthorized access to information systems to create and approve cases. Single Audit reports generally include a summary of prior audit findings describing any recurring issues and any related corrective actions undertaken by the state agency. Single Audits also generally provide information about any deficiencies in state agencies' systems and processes that can be useful for ACF in monitoring federal expenditures and identifying and reducing improper payments in the Foster Care program. According to ACF officials, ARTMS is designed to track and monitor the resolution of Single Audit findings by audit report number, but the system does not enable users to search for specific audit findings by type of finding, grantee, state, region, or across years. As a result, regional offices could not use ARTMS to examine trends in the types of findings in their states in order to ensure that any systemic issues are addressed. Limitations with ARTMS decrease ACF's ability to leverage existing agency data to identify reoccurring issues and other vulnerabilities such as inadequate state monitoring of federal funding that might not be identified during the 3-year eligibility review process and could lead to improper payments. This lack of information could impair ACF's and regional offices' ability to effectively monitor states' efforts to reduce improper payments and the effectiveness of corrective action strategies implemented. According to GAO's Standards for Internal Control in the Federal Government, information should be recorded and communicated to management and others within the entity who need it and in a form and within a time frame that enables them to carry out their internal control and other responsibilities.[Footnote 47] ACF regional office officials acknowledged limitations with ARTMS related to functionality in tracking findings. Specifically, a regional office official told us that ARTMS was not designed to be able to generate reports of all audit findings for an entire ACF region. Lacking such capability, a user interested in aggregating the findings would have to obtain the audit findings from each state and manually combine them outside of ARTMS. This regional office used a separate internal spreadsheet to track information related to the audit findings for all states in its purview. Otherwise, staff would need to view each state's Single Audit findings individually within ARTMS, which could be time-consuming. Another regional office we visited used a similar spreadsheet, which also included information on the time it takes to close findings. A third regional office we visited also utilized an off-line spreadsheet as a means to track the clearance process for closing audit findings. Conclusions: A statistically valid approach for estimating improper payments would help ensure that Foster Care program improper payment estimates are reasonably accurate and complete, reflecting all types of program payments including administrative costs, based on complete and accurate payment data, and aggregated using state-level margins of error. Developing and implementing a sound methodology is a critical program management tool for understanding and addressing financial vulnerabilities in the Foster Care program through approaches such as identifying underpayments and duplicate or excessive payment errors consistently across states. While ACF has reported an improper payment estimate and related reductions for the Foster Care program, the statistical validity of both is questionable. Further, ACF's method for evaluating the effectiveness of states' implementation of their corrective action plans has several significant weaknesses, including reliance on ineffective and dated metrics that do not consider states' improper payment dollar error rates in conjunction with targets that have not been reassessed since 2000. Similarly, deficiencies in its system for monitoring Single Audit findings limit ACF's ability to efficiently track and compare trends across states. This includes ACF's ability to measure states' progress in reducing their improper payment errors, as well as its ability to reliably and completely identify and correct vulnerabilities at the state level that could lead to improper payments. Although OMB's approval reflected a stated plan for ACF to implement a process to annually improve the accuracy of its improper payment estimate, this has not resulted in substantial changes to the process ACF outlined in 2004. Given the financial accountability challenges reported for state- administered programs, the ongoing imbalance between revenues and outlays across the federal government, and increasing demands for accountability over taxpayer funds, improving ACF's ability to identify, reduce, and recover improper payments is critical. It will be important for ACF to work closely with OMB in examining and updating its statistical procedures to help assure the validity of ACF's estimates. Recommendations for Executive Action: In order to more accurately and completely estimate improper payments for the Foster Care program and ensure that its methodology is statistically valid, we recommend that the Secretary of Health and Human Services direct the Assistant Secretary for the Administration for Children and Families to take the following four actions: * augment procedures for estimating and reporting Foster Care program improper payments, to include administrative costs; * develop and implement procedures to provide a statistically valid methodology for estimating and reporting Foster Care program improper payments based on complete and accurate payment data; * augment guidance to teams gathering state-level Foster Care program improper payment estimate data to include specific procedures to follow in identifying and reporting any underpayments and duplicate or excessive payment errors; and: * revise existing procedures for calculating a national improper payment estimate for the Foster Care program to include a statistically valid method for aggregating state-level margins of error to derive an overall, inflation adjusted, program estimate. To help ensure corrective action strategies effectively reduce Foster Care program improper payments, we recommend that the Secretary of Health and Human Services direct the Assistant Secretary for the Administration for Children and Families to take the following three actions: * develop and implement procedures requiring states to implement and report on corrective actions whenever a state's estimated improper payment dollar error rate exceeds a specified target level for the program; * establish and implement procedures requiring periodic assessments of state-level improper payment target levels, including targets associated with Title IV-E secondary reviews, for the Foster Care program for which states are to implement and report on corrective actions; and: * enhance ARTMS reporting capabilities to provide data on the status of actions taken to address Single Audit findings concerning states' Foster Care program payments, such as providing reporting capabilities to allow ARTMS users to search for specific audit findings by type of finding, grantee, state, region, or across years. Agency Comments and Our Evaluation: We provided a draft of this report to the Secretary of Health and Human Services for comment. In its written comments, reprinted in appendix IV, HHS agreed that its improper payment estimation efforts can and should be improved. HHS provided a summary of refinements that it had made to its improper payment estimation methodology over the years and also provided information on additional steps it planned to take. HHS stated that our analysis would be a helpful resource as it continued to improve its process. With regard to our seven recommendations to help improve ACF's methodology to estimate improper payments and its corrective action process, HHS generally concurred with four of the recommendations and agreed to continue to study the remaining three recommendations. HHS also provided technical comments that we incorporated, as appropriate. HHS generally concurred with three recommendations we made related to improving the improper payment estimation methodology for the Foster Care program. Specifically, HHS generally agreed to (1) estimate and report improper payments related to administrative costs, (2) provide specific procedures to identify and report any underpayments and duplicate or excessive payment errors, and (3) revise procedures for calculating the aggregate state-level margins of error to derive an overall, inflation adjusted, program estimate. HHS described several actions currently under way to address these recommendations. Regarding the first recommendation, HHS noted that it was continuing to pilot test the Administrative Cost Reviews, described in this report, in fiscal year 2012; however, HHS's response did not indicate when it expects these reviews will be fully implemented. In its response to the second recommendation, HHS stated that additional guidance for identifying and reporting underpayments and duplicate or excessive payment errors will be included in the updated Eligibility Review Guide and review instrument during the fiscal year 2012 review cycle. For the third recommendation, HHS stated that it can and will adjust its calculation to incorporate individual state margins of error in aggregating the state-level estimates into the national program estimate. However, HHS also stated that it will seek to determine whether making this revision would add sufficient value given that the estimate spans 3 years and that inflation is relatively low. We maintain that both aggregating state-level margins of error and factoring for inflation are needed to implement a statistically valid method for estimating improper payments. For the other recommendation we made related to the improper payment estimation methodology for the Foster Care program, HHS stated it would continue to study our recommendation to develop and implement a statistically valid Foster Care improper payment methodology based on complete and accurate payment data. HHS agreed that it should use the best data available. In its comments on our draft report, HHS acknowledged that it would be optimal to conduct a separate data collection to obtain a universe of Title IV-E payments, but stated that it needs to balance the goal of appropriate measurement with the cost and burden placed on states. HHS described the quality controls in place over the AFCARS data to help ensure the information is complete and accurate prior to selecting case samples for its Title IV- E eligibility reviews, which form the basis for its Foster Care improper payment estimate. Examples of such controls include automated system edit checks within AFCARS, AFCARS Assessment Reviews, and other outreach efforts to improve state AFCARS reporting. Our report describes some of the steps ACF has taken to address AFCARS data quality, but we also point out limitations in these efforts. For example, the AFCARS Assessment Reviews are not conducted annually for all states and do not address verifying the accuracy or completeness of the specific data element that ACF uses to develop its population of foster care cases for estimating improper payments. HHS also stated that its use of oversample cases demonstrated that its sampling and oversampling process is working properly to exclude cases that do not meet the selection criteria. While this process would identify some cases that did not meet ACF's selection criteria, our point in this report is that ACF's extensive reliance on the use of oversampling in its methodology is indicative that the population of cases could contain additional inaccuracies that may not be identified through its existing process. Further, as we stated in our report, neither we nor ACF were able to determine the completeness of the universe of cases used to estimate Foster Care improper payments, that is, whether all cases that had actually received a Title IV-E payment were properly coded as such. Thus, the issues we identified with ACF's sampling methodology, provides limited assurance that the reported improper payment estimate accurately and completely represents the extent of improper maintenance payments in the Foster Care program. With respect to our three recommendations to help ensure corrective strategies effectively reduce Foster Care improper payments, HHS concurred with one recommendation related to establishing and implementing procedures for periodic assessments of state-level improper payment target levels. HHS also agreed to consider another recommendation to develop and implement corrective action procedures whenever a state's estimated improper payment dollar error rate exceeds a specified target level for the program in conjunction with the recommendation it concurred with to implement periodic assessments. HHS highlighted several actions it plans to take to enhance its efforts to reduce improper payments, such as taking steps to reexamine and explore the feasibility for lowering the error rate threshold and considering ways to enhance existing Eligibility Review Guide instructions to address any eligibility review findings that involve improper payments not specifically requiring development of a corrective action plan. HHS stated that it plans to further study our recommendation to enhance ARTMS reporting capabilities to provide data on the status of actions taken to address Single Audit findings concerning states' Foster Care program payments. HHS stated that the agency would study the value of potential enhancements to ARTMS in light of the significant relevant data already available (such as audit information by federal program, grantee name, audit resolution status, and audit periods). Although these data elements are currently available in ARTMS, search results are presented by individual audit reports and include limited information. For example, the search results provide only the number of findings associated with a specific audit report but do not provide details of the individual audit findings that would allow program managers to analyze, for example, trends in the types of findings in states in order to ensure that any systemic issues are addressed. We reaffirm our recommendation to ensure that ACF is able to fully utilize ARTMS as a tool to analyze Single Audit findings. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees; the Secretary of Health and Human Services; and other interested parties. In addition, the report will be available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staffs have any questions about this report, please contact me at (202) 512-8486 or raglands@gao.gov. Contact points for our Offices of Public Affairs and Congressional Relations may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V. Signed by: Susan Ragland: Director Financial Management and Assurance: [End of section] Appendix I: Objectives, Scope, and Methodology: The objectives of this report were to (1) determine the extent to which the Administration for Children and Families' (ACF) estimation methodology generated a reasonably accurate and complete estimate of improper payments across the Foster Care program and (2) determine the extent to which ACF's corrective actions reduced improper payments. To address these objectives, we reviewed the Improper Payments Information Act of 2002 (IPIA) requirements[Footnote 48] and related Office of Management and Budget (OMB) guidance effective for fiscal year 2010,[Footnote 49] Department of Health and Human Services (HHS) regulations on Title IV-E eligibility reviews,[Footnote 50] and ACF's internal guidance including policies and procedures on conducting Title IV-E Foster Care eligibility reviews, computing improper payments, implementing corrective action plans for reducing improper payments, and monitoring and resolving audit findings in the Foster Care program. We also reviewed results from the Title IV-E eligibility reviews for periods 2000 through 2010, and prior GAO[Footnote 51] and HHS Office of Inspector General (OIG) reports.[Footnote 52] In addition, we reviewed improper payment information reported in HHS's fiscal year 2010 agency financial report, Improper Payments Section. We reviewed these documents to understand ACF's efforts to address IPIA requirements and to identify previously reported issues with ACF's improper payment reporting. To further determine the extent to which ACF's methodology generated a reasonably accurate and complete estimate of improper payments across the Foster Care program, we: * Performed an independent analysis of ACF's sampling methodology, including a review of the sampling plan and other underlying documentation, as well as evaluated whether ACF's sampling methodology complied with OMB statistical guidance, GAO guidance, and federal internal control standards[Footnote 53] as criteria to determine the accuracy and completeness of ACF's reported fiscal year 2010 improper payment estimate for the Foster Care program. The scope of our review did not include an assessment of individual states' processes or payment systems that are the underlying data that ACF uses to support the national estimate of Foster Care improper payments. * Interviewed ACF officials such as the Acting Associate Commissioner for the Children's Bureau, its contractor, and staff at selected regional offices such as program managers and financial specialists to gain an understanding of (1) the methodology that ACF uses to estimate improper payments in the Foster Care program in accordance with IPIA requirements[Footnote 54] and (2) the administrative cost review pilot at five states to develop a methodology for estimating related administrative improper payments. We reviewed available reports for the five pilot reviews to identify what information ACF obtained from these reviews. To further determine the extent to which ACF's corrective action process reduced improper payments, we: * Reviewed ACF policies and procedures to gain an understanding of reported corrective action strategies, including the Title IV-E eligibility review process and development of states' Program Improvement Plans (PIP) used to address the root causes of improper payments, which are identified from the Title IV-E eligibility reviews. We reviewed applicable states' PIPs for the period 2001 through 2010. We also inquired of ACF officials from the Program Implementation Division within the Children's Bureau about other monitoring activities in place for states that did not have a PIP in place for IPIA reporting in fiscal year 2010. * Assessed compliance thresholds ACF uses to require states to implement corrective actions against actual performance data to assess the propriety of established performance measures. As part of this analysis, we reviewed our internal control standards as guidance to assess ACF's evaluation of states' efforts to implement corrective actions.[Footnote 55] * Conducted a walkthrough of ACF's Audit Resolution Tracking and Monitoring System (ARTMS) to obtain an understanding of ACF's monitoring activities to track and resolve state's Single Audit findings for the Foster Care program. In addition, we interviewed officials in ACF's Office of Information Services and the Division of Financial Integrity located in the Central Office and selected representative regional offices such as Regional Program Managers, and program and fiscal specialists to determine how ARTMS is used to identify and correct vulnerabilities that could lead to improper payments. * Examined states' reported Single Audit[Footnote 56] findings for fiscal years 2008 through 2010 from ARTMS and a listing of HHS OIG reports on the Foster Care program to identify vulnerabilities or weaknesses in states' operations that may not have been identified through ACF's Title IV-E eligibility reviews. * Reviewed agency policies and procedures such as ACF's Title IV-E Foster Care Eligibility Review Guide issued in March 2006 which includes the Title IV-E Foster Care Eligibility On-Site Review Instrument and Instructions; ACF's user guide for ARTMS, version 1.2; and ACF's FY 2010 Corrective Action Plan to Reduce the Estimate Rate of Improper Payments in the Foster Care Program, dated November 12, 2010. * In addition, we conducted site visits to three of ACF's ten regional offices (Philadelphia, PA; Chicago, IL; and San Francisco, CA). These three regional offices provided oversight of states who collectively claimed over half of the total federal share of Foster Care payments made in fiscal year 2009, the most recent data available at the time of our review for site visit selection. We also selected these regional offices to achieve variation in the numbers of error cases and amount of disallowed claims found during Title IV-E eligibility reviews, which ACF conducts to help ensure that states are claiming federal reimbursement only for eligible children. One region represented the highest number of error cases found in the Title IV-E eligibility reviews and the highest maintenance payment disallowance. Another region had the largest amount of foster care maintenance payments in fiscal year 2009 and the states within this region had high improper payment rates. Another region had a low number of error cases and improper payment issues relative to the high amount of maintenance payments it made to states in its purview. During these site visits, we interviewed agency personnel such as program managers, regional grants officers, and financial specialists to gain an understanding of how Title IV-E eligibility reviews are conducted and how the regional offices work with states on corrective actions and follow up on Single Audit findings. We also inquired about other ACF monitoring activities over states to address financial management weaknesses. We conducted this performance audit from February 2011 through March 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions, based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Title IV-E Expenditures by Type and by State for Fiscal Year 2010: State: Alabama; Average monthly number of children: 2,097; Net maintenance payments: $6,151,654; Child placement services and administration (administrative costs): $22,659,900; SACWIS (operations and development): $2,234,570; Training: $$898,679; Demonstrations: [Empty]; Total: $31,944,803. State: Alaska; Average monthly number of children: 655; Net maintenance payments: $2,541,413; Child placement services and administration (administrative costs): $9,670,033; SACWIS (operations and development): $1,294,613; Training: $308,245; Demonstrations: [Empty]; Total: $13,814,304. State: Arizona; Average monthly number of children: 4,403; Net maintenance payments: $34,920,745; Child placement services and administration (administrative costs): $25,433,283; SACWIS (operations and development): $1,625,049; Training: $10,213,887; Demonstrations: $4,056; Total: $72,197,020. State: Arkansas; Average monthly number of children: 1,789; Net maintenance payments: $13,298,257; Child placement services and administration (administrative costs): $19,613,926; SACWIS (operations and development): [Empty]; Training: $7,629,291; Demonstrations: [Empty]; Total: $40,541,474. State: California; Average monthly number of children: 33,188; Net maintenance payments: $189,058,341; Child placement services and administration (administrative costs): $434,338,308; SACWIS (operations and development): $41,470,987; Training: $74,006,011; Demonstrations: $464,542,684; Total: $1,203,416,331. State: Colorado; Average monthly number of children: 2,041; Net maintenance payments: $21,048,905; Child placement services and administration (administrative costs): $37,310,491; SACWIS (operations and development): $2,833,886; Training: $1,432,169; Demonstrations: [Empty]; Total: $62,625,451. State: Connecticut; Average monthly number of children: 1,670; Net maintenance payments: $33,837,323; Child placement services and administration (administrative costs): $26,140,861; SACWIS (operations and development): $494,901; Training: $1,008,418; Demonstrations: [Empty]; Total: $61,481,503. State: Delaware; Average monthly number of children: 201; Net maintenance payments: $1,415,097; Child placement services and administration (administrative costs): $2,336,241; SACWIS (operations and development): $251,308; Training: $45,551; Demonstrations: [Empty]; Total: $4,048,197. State: District of Columbia; Average monthly number of children: 902; Net maintenance payments: $17,088,155; Child placement services and administration (administrative costs): $13,730,035; SACWIS (operations and development): $2,639,434; Training: $216,026; Demonstrations: [Empty]; Total: $33,673,650. State: Florida; Average monthly number of children: 6,127; Net maintenance payments: $(662,979); Child placement services and administration (administrative costs): [Empty]; SACWIS (operations and development): $8,246,087; Training: [Empty]; Demonstrations: $164,026,443; Total: $171,609,551. State: Georgia; Average monthly number of children: 2,755; Net maintenance payments: $35,226,100; Child placement services and administration (administrative costs): $38,876,701; SACWIS (operations and development): $6,928,247; Training: $642,342; Demonstrations: [Empty]; Total: $81,673,390. State: Hawaii; Average monthly number of children: 473; Net maintenance payments: $3,215,746; Child placement services and administration (administrative costs): $12,858,071; SACWIS (operations and development): [Empty]; Training: $2,188,051; Demonstrations: [Empty]; Total: $18,261,868. State: Idaho; Average monthly number of children: 963; Net maintenance payments: $3,148,911; Child placement services and administration (administrative costs): $4,823,821; SACWIS (operations and development): $322,585; Training: $1,213,690; Demonstrations: [Empty]; Total: $9,509,007. State: Illinois; Average monthly number of children: 13,292; Net maintenance payments: $67,605,231; Child placement services and administration (administrative costs): $99,941,388; SACWIS (operations and development): [Empty]; Training: $1,458,872; Demonstrations: $18,734,590; Total: $187,740,081. State: Indiana; Average monthly number of children: 3,087; Net maintenance payments: $37,968,599; Child placement services and administration (administrative costs): $43,019,507; SACWIS (operations and development): $416,181; Training: $5,291,285; Demonstrations: $4,610,780; Total: $91,306,352. State: Iowa; Average monthly number of children: 1,471; Net maintenance payments: $10,575,979; Child placement services and administration (administrative costs): $10,057,790; SACWIS (operations and development): $634,089; Training: $1,393,149; Demonstrations: $1,672,279; Total: $24,333,286. State: Kansas; Average monthly number of children: 1,245; Net maintenance payments: $13,946,950; Child placement services and administration (administrative costs): $13,549,938; SACWIS (operations and development): [Empty]; Training: $98,523; Demonstrations: [Empty]; Total: $27,595,411. State: Kentucky; Average monthly number of children: 2,921; Net maintenance payments: $27,357,551; Child placement services and administration (administrative costs): $12,950,173; SACWIS (operations and development): $1,823,554; Training: $4,863,147; Demonstrations: [Empty]; Total: $46,994,425. State: Louisiana; Average monthly number of children: 2,562; Net maintenance payments: $26,969,310; Child placement services and administration (administrative costs): $23,512,183; SACWIS (operations and development): $(2,529,684); Training: $8,187,171; Demonstrations: [Empty] ; Total: $56,138,980. State: Maine; Average monthly number of children: 957; Net maintenance payments: $5,963,933; Child placement services and administration (administrative costs): $11,220,480; SACWIS (operations and development): $502,557; Training: $385,164; Demonstrations: [Empty]; Total: $18,072,134. State: Maryland; Average monthly number of children: 2,145; Net maintenance payments: $41,022,794; Child placement services and administration (administrative costs): $29,899,462; SACWIS (operations and development): $1,463,167; Training: $2,372,090; Demonstrations: [Empty]; Total: $74,757,513. State: Massachusetts; Average monthly number of children: 2,191; Net maintenance payments: $26,328,104; Child placement services and administration (administrative costs): $33,629,458; SACWIS (operations and development): $526,268; Training: [Empty]; Demonstrations: [Empty]; Total: $60,483,830. State: Michigan; Average monthly number of children: 4,165; Net maintenance payments: $27,686,604; Child placement services and administration (administrative costs): $65,368,746; SACWIS (operations and development): [Empty]; Training: $620,506; Demonstrations: [Empty]; Total: $93,675,856. State: Minnesota; Average monthly number of children: 1,800; Net maintenance payments: $11,218,971; Child placement services and administration (administrative costs): $17,590,313; SACWIS (operations and development): $5,550,119; Training: $4,900,152; Demonstrations: $3,617,755; Total: $42,877,310. State: Mississippi; Average monthly number of children: 999; Net maintenance payments: $8,190,833; Child placement services and administration (administrative costs): $6,493,427; SACWIS (operations and development): [Empty]; Training: [Empty]; Demonstrations: [Empty]; Total: $14,684,260. State: Missouri; Average monthly number of children: 3,166; Net maintenance payments: $16,858,234; Child placement services and administration (administrative costs): $30,177,210; SACWIS (operations and development): $4,083,378; Training: $4,950,719; Demonstrations: [Empty]; Total: $56,069,541. State: Montana; Average monthly number of children: 627; Net maintenance payments: $4,679,312; Child placement services and administration (administrative costs): $5,282,883; SACWIS (operations and development): $943,474; Training: $1,358,777; Demonstrations: [Empty]; Total: $12,264,446. State: Nebraska; Average monthly number of children: 1,369; Net maintenance payments: $7,861,664; Child placement services and administration (administrative costs): $9,467,519; SACWIS (operations and development): $169,953; Training: $2,499,675; Demonstrations: [Empty]; Total: $19,998,811. State: Nevada; Average monthly number of children: 2,083; Net maintenance payments: $11,695,544; Child placement services and administration (administrative costs): $19,428,724; SACWIS (operations and development): $1,925,270; Training: $2,261,741; Demonstrations: [Empty]; Total: $35,311,279. State: New Hampshire; Average monthly number of children: 436; Net maintenance payments: $3,424,675; Child placement services and administration (administrative costs): $11,753,640; SACWIS (operations and development): $470,028; Training: $1,298,133; Demonstrations: [Empty]; Total: $16,946,476. State: New Jersey; Average monthly number of children: 4,226; Net maintenance payments: $34,725,549; Child placement services and administration (administrative costs): $54,478,552; SACWIS (operations and development): $2,525,021; Training: $3,321,363; Demonstrations: [Empty]; Total: $95,050,485. State: New Mexico; Average monthly number of children: 1,092; Net maintenance payments: $5,367,458; Child placement services and administration (administrative costs): $14,420,493; SACWIS (operations and development): $1,562,400; Training: $3,368,379; Demonstrations: [Empty]; Total: $24,718,730. State: New York; Average monthly number of children: 12,724; Net maintenance payments: $201,743,181; Child placement services and administration (administrative costs): $213,684,975; SACWIS (operations and development): $5,193,320; Training: $9,882,339; Demonstrations: [Empty]; Total: $430,503,815. State: North Carolina; Average monthly number of children: 3,197; Net maintenance payments: $20,787,787; Child placement services and administration (administrative costs): $53,681,394; SACWIS (operations and development): [Empty]; Training: $874,011; Demonstrations: [Empty]; Total: $75,343,192. State: North Dakota; Average monthly number of children: 375; Net maintenance payments: $5,962,358; Child placement services and administration (administrative costs): $5,115,417; SACWIS (operations and development): $17,411; Training: $385,436; Demonstrations: [Empty]; Total: $11,480,622. State: Ohio; Average monthly number of children: 7,446; Net maintenance payments: $105,128,723; Child placement services and administration (administrative costs): $81,031,793; SACWIS (operations and development): $2,907,758; Training: $5,625,728; Demonstrations: $300,500; Total: $194,994,502. State: Oklahoma; Average monthly number of children: 3,308; Net maintenance payments: $11,848,998; Child placement services and administration (administrative costs): $16,665,705; SACWIS (operations and development): $2,181,593; Training: $3,531,191; Demonstrations: [Empty]; Total: $34,227,487. State: Oregon; Average monthly number of children: 3,190; Net maintenance payments: $23,579,566; Child placement services and administration (administrative costs): $58,623,327; SACWIS (operations and development): $8,347,283; Training: $1,275,803; Demonstrations: $6,422,206; Total: $98,248,185. State: Pennsylvania; Average monthly number of children: 14,690; Net maintenance payments: $125,564,749; Child placement services and administration (administrative costs): $124,439,736; SACWIS (operations and development): [Empty]; Training: $5,255,437; Demonstrations: [Empty]; Total: $255,259,922. State: Puerto Rico; Average monthly number of children: [Empty]; Net maintenance payments: [Empty]; Child placement services and administration (administrative costs): [Empty]; SACWIS (operations and development): [Empty]; Training: [Empty]; Demonstrations: [Empty]; Total: [Empty]. State: Rhode Island; Average monthly number of children: 592; Net maintenance payments: $6,611,638; Child placement services and administration (administrative costs): $5,288,952; SACWIS (operations and development): $1,495,342; Training: $75,712; Demonstrations: [Empty]; Total: $13,471,644. State: South Carolina; Average monthly number of children: 1,174; Net maintenance payments: $12,273,512; Child placement services and administration (administrative costs): $22,002,174; SACWIS (operations and development): $1,024,325; Training: $1,223,697; Demonstrations: [Empty]; Total: $36,523,708. State: South Dakota; Average monthly number of children: 590; Net maintenance payments: $3,749,543; Child placement services and administration (administrative costs): $2,628,588; SACWIS (operations and development): $98,542; Training: $216,286; Demonstrations: [Empty]; Total: $6,692,959. State: Tennessee; Average monthly number of children: 2,981; Net maintenance payments: $20,262,663; Child placement services and administration (administrative costs): $12,901,854; SACWIS (operations and development): $9,376,444; Training: $3,107,492; Demonstrations: $(490,734); Total: $45,157,719. State: Texas; Average monthly number of children: 11,971; Net maintenance payments: $142,903,432; Child placement services and administration (administrative costs): $104,371,896; SACWIS (operations and development): [Empty]; Training: $9,969,190; Demonstrations: [Empty]; Total: $257,244,518. State: Utah; Average monthly number of children: 902; Net maintenance payments: $5,054,187; Child placement services and administration (administrative costs): $12,228,891; SACWIS (operations and development): $1,250,832; Training: $1,704,787; Demonstrations: [Empty]; Total: $20,238,697. State: Vermont; Average monthly number of children: 528; Net maintenance payments: $6,851,819; Child placement services and administration (administrative costs): $2,485,472; SACWIS (operations and development): [Empty]; Training: $1,203,723; Demonstrations: [Empty]; Total: $10,541,014. State: Virginia; Average monthly number of children: 2,870; Net maintenance payments: $27,490,610; Child placement services and administration (administrative costs): $27,827,516; SACWIS (operations and development): [Empty]; Training: $701,434; Demonstrations: [Empty]; Total: $56,019,560. State: Washington; Average monthly number of children: 4,159; Net maintenance payments: $20,774,711; Child placement services and administration (administrative costs): $59,527,687; SACWIS (operations and development): $5,207,194; Training: $5,952,490; Demonstrations: [Empty]; Total: $91,462,082. State: West Virginia; Average monthly number of children: 1,012; Net maintenance payments: $27,171,706; Child placement services and administration (administrative costs): $4,255,352; SACWIS (operations and development): $2,014,995; Training: $352,446; Demonstrations: [Empty]; Total: $33,794,499. State: Wisconsin; Average monthly number of children: 2,151; Net maintenance payments: $20,032,342; Child placement services and administration (administrative costs): $27,653,539; SACWIS (operations and development): $2,527,137; Training: $2,009,693; Demonstrations: $792,644; Total: $53,015,355. State: Wyoming; Average monthly number of children: 120; Net maintenance payments: $500,589; Child placement services and administration (administrative costs): $1,176,256; SACWIS (operations and development): $502,913; Training: $212,042; Demonstrations: [Empty]; Total: $2,391,800. State: Totals; Average monthly number of children: 181,078; Net maintenance payments: $$1,538,027,077; Child placement services and administration (administrative costs): $$1,995,624,081; SACWIS (operations and development): $$130,552,531; Training: $$201,990,143; Demonstrations: $664,233,203; Total: $$4,530,427,035. Source: ACF's Title IV-E Foster Care Fiscal Year 2010 Expenditures as Reported by States (May 4, 2011). [End of table] [End of section] Appendix III: Estimated Improper Payment Error Rates by State for Fiscal Year 2010 Reporting: State: Alabama; Review date: 08/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.96%; Sample error rate: 0.96%. State: Alaska; Review date: 07/2009; Sample underpayment rate: 0.53%; Sample overpayment rate: 4.42%; Sample error rate: 4.95%. State: Arizona; Review date: 03/2010; Sample underpayment rate: 0.00%; Sample overpayment rate: 1.16%; Sample error rate: 1.16%. State: Arkansas; Review date: 08/2009; Sample underpayment rate: 0.38%; Sample overpayment rate: 7.19%; Sample error rate: 7.57%. State: California; Review date: 09/2009; Sample underpayment rate: 1.77%; Sample overpayment rate: 6.20%; Sample error rate: 7.97%. State: Colorado; Review date: 06/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 4.04%; Sample error rate: 4.04%. State: Connecticut; Review date: 04/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 8.09%; Sample error rate: 8.09%. State: Delaware; Review date: 04/2009; Sample underpayment rate: 2.58%; Sample overpayment rate: 7.69%; Sample error rate: 10.27%. State: District of Columbia; Review date: 09/2009; Sample underpayment rate: 0.70%; Sample overpayment rate: 3.06%; Sample error rate: 3.76%. State: Florida; Review date: [A]; Sample underpayment rate: [Empty]; Sample overpayment rate: [Empty]; Sample error rate: 0.00%. State: Georgia; Review date: 09/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 6.88%; Sample error rate: 6.88%. State: Hawaii; Review date: 06/2010; Sample underpayment rate: 3.08%; Sample overpayment rate: 0.57%; Sample error rate: 3.66%. State: Idaho; Review date: 04/2010; Sample underpayment rate: 0.92%; Sample overpayment rate: 18.88%; Sample error rate: 19.80%. State: Illinois; Review date: 08/2007; Sample underpayment rate: 0.00%; Sample overpayment rate: 10.87%; Sample error rate: 10.87%. State: Indiana; Review date: 01/2009; Sample underpayment rate: 2.33%; Sample overpayment rate: 8.84%; Sample error rate: 11.17%. State: Iowa; Review date: 09/2007; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.00%; Sample error rate: 0.00%. State: Kansas; Review date: 08/2008; Sample underpayment rate: 0.40%; Sample overpayment rate: 1.37%; Sample error rate: 1.77%. State: Kentucky; Review date: 11/2007; Sample underpayment rate: 0.48%; Sample overpayment rate: 1.66%; Sample error rate: 2.14%. State: Louisiana; Review date: 09/2007; Sample underpayment rate: 0.61%; Sample overpayment rate: 0.72%; Sample error rate: 1.33%. State: Maine; Review date: 06/2010; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.81%; Sample error rate: 0.81%. State: Maryland; Review date: 06/2008; Sample underpayment rate: 0.00%; Sample overpayment rate: 14.39%; Sample error rate: 14.39%. State: Massachusetts; Review date: 11/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 17.74%; Sample error rate: 17.74%. State: Michigan; Review date: 06/2010; Sample underpayment rate: 2.56%; Sample overpayment rate: 1.78%; Sample error rate: 4.34%. State: Minnesota; Review date: 03/2010; Sample underpayment rate: 1.64%; Sample overpayment rate: 3.49%; Sample error rate: 5.13%. State: Mississippi; Review date: 07/2008; Sample underpayment rate: 0.00%; Sample overpayment rate: 16.80%; Sample error rate: 16.80%. State: Missouri; Review date: 09/2008; Sample underpayment rate: 0.00%; Sample overpayment rate: 4.68%; Sample error rate: 4.68%. State: Montana; Review date: 06/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.00%; Sample error rate: 0.00%. State: Nebraska; Review date: 08/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 1.96%; Sample error rate: 1.96%. State: Nevada; Review date: 06/2008; Sample underpayment rate: 0.15%; Sample overpayment rate: 3.84%; Sample error rate: 3.99%. State: New Hampshire; Review date: 12/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.18%; Sample error rate: 0.18%. State: New Jersey; Review date: 08/2008; Sample underpayment rate: 1.97%; Sample overpayment rate: 3.11%; Sample error rate: 5.08%. State: New Mexico; Review date: 07/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 3.62%; Sample error rate: 3.62%. State: New York; Review date: 08/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 1.50%; Sample error rate: 1.50%. State: North Carolina; Review date: 06/2008; Sample underpayment rate: 0.00%; Sample overpayment rate: 1.46%; Sample error rate: 1.46%. State: North Dakota; Review date: 08/2008; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.69%; Sample error rate: 0.69%. State: Ohio; Review date: 07/2010; Sample underpayment rate: 3.16%; Sample overpayment rate: 1.94%; Sample error rate: 5.10%. State: Oklahoma; Review date: 01/2010; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.00%; Sample error rate: 0.00%. State: Oregon; Review date: 07/2008; Sample underpayment rate: 0.69%; Sample overpayment rate: 2.48%; Sample error rate: 3.17%. State: Pennsylvania; Review date: 07/2007; Sample underpayment rate: 0.15%; Sample overpayment rate: 3.04%; Sample error rate: 3.19%. State: Puerto Rico; Review date: 05/2007; Sample underpayment rate: 0.00%; Sample overpayment rate: 10.76%; Sample error rate: 10.76%. State: Rhode Island; Review date: 09/2007; Sample underpayment rate: 0.00%; Sample overpayment rate: 14.31%; Sample error rate: 14.31%. State: South Carolina; Review date: 05/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 1.24%; Sample error rate: 1.24%. State: South Dakota; Review date: 05/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.00%; Sample error rate: 0.00%. State: Tennessee; Review date: 06/2009; Sample underpayment rate: 0.00%; Sample overpayment rate: 8.79%; Sample error rate: 8.79%. State: Texas; Review date: 07/2009; Sample underpayment rate: 0.15%; Sample overpayment rate: 0.00%; Sample error rate: 0.15%. State: Utah; Review date: 09/2008; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.19%; Sample error rate: 0.19%. State: Vermont; Review date: 05/2008; Sample underpayment rate: 0.00%; Sample overpayment rate: 1.97%; Sample error rate: 1.97%. State: Virginia; Review date: 03/2007; Sample underpayment rate: 1.48%; Sample overpayment rate: 4.09%; Sample error rate: 5.57%. State: Washington; Review date: 08/2007; Sample underpayment rate: 0.82%; Sample overpayment rate: 0.75%; Sample error rate: 1.58%. State: West Virginia; Review date: 04/2008; Sample underpayment rate: 0.00%; Sample overpayment rate: 4.44%; Sample error rate: 4.44%. State: Wisconsin; Review date: 08/2008; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.00%; Sample error rate: 0.00%. State: Wyoming; Review date: 03/2010; Sample underpayment rate: 0.00%; Sample overpayment rate: 0.00%; Sample error rate: 0.00%. Source: ACF's Title IV-E Foster Care Program Fall 2010 Error Rate Update. [A] According to ACF, Florida IV-E Reviews suspended pending completion of a statewide Foster Care demonstration project and were therefore, not included in the national error rate. [End of table] [End of section] Appendix IV: Comments from the Department of Health and Human Services: Department Of Health and Human Services: Office of The Secretary: Assistant Secretary for Legislation: Washington, DC 20201: February 24, 2012: Susan Ragland, Director: Financial Management & Assurance: U.S. Government Accountability Office: 441 G Street NW: Washington, DC 20548: Dear Ms. Ragland: Attached are comments on the U.S. Government Accountability Office's (GAO) report entitled, "Foster Care Program: Improved Processes Needed to Estimate Improper Payments and Evaluate Related Corrective Actions" (GAO-12-312). The Department appreciates the opportunity to review this draft section of the report prior to publication. Sincerely, Signed by: Jim R. Esquea: Assistant Secretary for Legislation: Attachment: [End of letter] General Comments Of The Department Of Health and Human Services (HHS) On The Government Accountability Office's (GAO) Draft Report Entitled, "Foster Care Program: Improved Processes Needed To Estimate Improper Payments And Evaluate Related Corrective Actions" (GAO-12-312): The Department appreciates the opportunity to review and comment on this draft report. GAO Recommendations: In order to more accurately and completely estimate improper payments for the Foster Care program and ensure that its methodology is statistically valid, we recommend that the Secretary of Health and Human Services direct the Assistant Secretary for the Administration for Children and Families to take the following four actions: * augment procedures for estimating and reporting Foster Care program improper payments to include administrative costs; * develop and implement procedures to provide a statistically valid methodology for estimating and reporting Foster Care program improper payments based on complete and accurate payment data; * augment guidance to teams gathering state-level Foster Care program improper payment estimate data to include specific procedures to follow in identifying and reporting any underpayments and duplicate or excessive payment errors; and; * revise existing procedures for calculating a national improper payment estimate for the Foster Care program to include a statistically valid method for aggregating state-level margins of error to derive an overall, inflation adjusted, program estimate. To help ensure corrective action strategies effectively reduce Foster Care program improper payments, we recommend that the Secretary of Health and Human Services direct the Assistant Secretary for the Administration of Children and Families to take the following three actions: * develop and implement procedures requiring states to implement and report on corrective actions whenever a state's estimated improper payment dollar error rate exceeds a specified target level for the program; * establish and implement procedures requiring periodic assessments of state-level improper payment target levels, including targets associated with Title IV-E secondary reviews, for the Foster Care program for which states are to implement and report on corrective actions; and; * enhance ARTMS reporting capabilities to provide data on the status of actions taken to address Single Audit findings concerning states' Foster Care program payments, such as providing reporting capabilities to allow ARTMS users to search for specific audit findings by type of finding, grantee, state, region, or across years. Administration for Children and Families (ACF) Response: In this report, GAO suggests that improved processes are needed to estimate improper payments and evaluate related corrective actions for the foster care program. ACF agrees that improper payments estimation efforts can and should be improved. GAO has identified a set of significant issues concerning limitations to the current methodology. ACF has reviewed, and will further examine GAO's recommendations to identify feasible improvements. In doing so, it is important to keep in mind that ACF's methodology has been developed and modified over time in a manner that has sought to minimize additional costs and administrative burdens for States. As we take additional steps to improve this methodology, we also will seek to avoid creating significant new costs or administrative burdens for States. As we do so, we will benefit from GAO's analysis. While we agree that the methodology would benefit from additional improvements, we note that the current methodology has been improved and refined over time. On page 17, the GAO report states that OMB's approval of ACF's methodology in 2004 "reflected the idea that the 'short term' methods initially used would incorporate annual improvements to the accuracy of improper payments estimates." In fact, ACF did not commit to "annual" changes to the underlying title IV-E Foster Care program improper payments estimation method (and indeed, annual changes to this methodology could undermine program efforts to track performance across time). ACF has, however, made refinements to its methodology, in consultation with OMB, and has regularly updated OMB on the methodology, refinements to the methodology, and the results of improper payment estimation. ACF instituted several refinements to the existing methodology following baseline reporting to improve the quality of improper payments estimation and reporting. These refinements focused on a few important and specified changes to maintain as much reliability in reporting as possible. The GAO report mentions one of these refinements: "a change in 2008 to derive the estimate using sample state dollar error rates." ACF has also made several additional refinements: * Shifted from case-based to payment-based extrapolation in 2008, and maintained dual reporting for several years to permit comparisons across time. * Examined individual eligibility errors and other payment errors contributing to annual improper payments performance, beginning in 2006. * Included underpayment reporting in reviews conducted since 2006. * Developed methodology to review and test administrative payments and received approval to proceed with pilot tests in 2007. * Continue to conduct pilot tests of a protocol for assessing improper administrative claims and for incorporating administrative payments in improper payment reporting to ensure feasibility and support the reliability of the resulting review process. Finally, as described in more detail below, ACF is continuing to make additional improvements to this methodology. When we complete the pilot test and refinement of the administrative cost review process and begin carrying out such reviews nationwide, we will incorporate administrative payments into improper payment reporting program-wide. In addition, in August 2011, we issued new staff guidance to clarify procedures for calculating and reporting underpayments. Additional guidance on improper payments will be operational during the FY 2012 review cycle. GAO's analysis will be a helpful resource as we continue to improve this process. Following is ACF's response to the specific recommendations in the report: GAO Recommendation #1: Augment procedures for estimating and reporting Foster Care program improper payments to include administrative costs. We concur with caveats and, as GAO indicates in its report, we are in the process of developing a methodology for doing so. * During development of initial methodology options for estimating Foster Care program improper payments, the ACF team did explore and discuss with OMB the possibility of incorporating consideration of administrative payments into program error rate estimation. Although title IV-E Foster Care Eligibility Reviews do impose disallowance of administrative payments for cases with eligibility errors (computed as average administrative costs per month, per case), the reviews do not directly measure and test administrative payments. Consequently, OMB approved ACF's methodology for estimating and reporting on improper maintenance payments and suggested that ACF explore ways to incorporate consideration of administrative payments into its improper payments efforts. In 2005, ACF submitted and, with OMB approval, implemented a plan for incorporating consideration of administrative payments in the reporting of improper payments and has engaged in pilot tests of a method for testing administrative payments. * In response to feedback and suggestions provided by OMB, ACF has worked to develop and pilot test an Administrative Cost Review (ACR) process to identify and appropriately address any improper payments claimed as title 1V-E Foster Care administration. This was necessary because no model methodology for administrative cost consideration was apparent in other improper payments reporting across similar state- administered, federally reimbursed programs. The ACR methodology developed by ACF represents a unique innovative approach for reviewing and testing administrative payments allocated and assigned to title IV- E Foster Care in an intensive and fairly comprehensive way. * To ensure that the ACR review process is feasible and yields sufficiently reliable and valid data without imposing undue burden on States, ACF is continuing to pilot test this review in FY 2012 and have been working to assure that information on improper payments for administrative costs is captured through ACR findings in a manner that will facilitate its incorporation into program improper payments reporting. Once data are available, program improper payments reporting will incorporate estimates of improper payments for (a) maintenance payments; (b) administrative payments; and (c) combined administrative and maintenance payments and report on all three components. It is our expectation that this important element of information on improper payments will be included in the title IV-E Foster Care IPIA improper payment estimate calculation beginning in the fiscal year when the ACR is fully implemented. At that point, using FY 2010 title IV-E claiming information, the ACF Foster Care program improper payments estimation methodology will address 78% of annual program claims. * The GAO report notes that ACF calculated associated administrative payment disallowances of $2.4 million in FY 2010 based on title IV-E eligibility review findings, but did not report these amounts as part of its IPIA improper payments estimate. This approach is based on our ongoing efforts to develop an overall administrative component for the title IV-E Foster Care program improper payments rather than to report only one piece of that amount associated with cases determined as not title W-E eligible. Nevertheless, it is important to note that eligibility reviews do encompass some recovery of improper administrative payments through this imposition of disallowances for administrative costs associated with cases determined to be ineligible for title IV-E funds. GAO Recommendation #2: Develop and implement procedures to provide a statistically valid methodology for estimating and reporting Foster Care program improper payments based on complete and accurate payment data. This recommendation stems from GAO's assessment that rather than relying on random samples of eligible foster care cases, eligibility review samples should be drawn from the universe of title IV-E payments. We agree that we should use the best data available. In considering these issues, we need to balance the goal of appropriate measurement with the cost and burden placed on states. We're continuing to study this recommendation. We agree that, putting aside resource constraints and the burden imposed on States by Federal data collection activities, it would be optimal to conduct a separate data collection for improper payments purposes by drawing the sample from a universe of title IV-E payments. However, from the outset Federal policy has permitted or encouraged reliance on existing sources of data to reduce the administrative costs and burdens on states. Sampling from the universe of payments was considered in developing the initial proposed estimation methodology options; however, investigation into available data sources at the outset of this project revealed no national transaction- level data sources for title IV-E Foster Care payments, and such broad variability across state financial systems as to mitigate the feasibility of instituting automated aggregation of transactional payment information. Thus, ACF proposed and received approval to develop program estimates of improper payments and payment en-or rate based on eligibility review sample data. We are committed to continuing to improve our improper payment estimates and reporting. ACF will take this recommendation under consideration. This recommendation also reflects a position noted in the GAO report questioning the quality of AFCARS data, citing as evidence frequent use of replacement cases from oversamples to replace sampled cases that are discovered to have received no payments during the review period. GAO concludes that this jeopardizes the validity of the resulting sample. The bullets below provide further information on ACF's ongoing efforts to ensure that this data is valid. * The AFCARS system captures data entered by State title IV-E agencies into selected standardized fields in each State agency's child welfare information system(s). The sample for foster care eligibility reviews is drawn from the universe of cases indicated in AFCARS as eligible to receive a title IV-E payment during the particular review period for a given State. More specifically, the sample is randomly selected from the universe of cases in AFCARS for which data element #59 (i.e., Foster Care) is coded, "1". For AFCARS reporting purposes, foster care element #59 is reported as "applies" if at any time during the six- month report period a child was eligible for title IV-E. This means that if the child was eligible for even 1 day in the report period, #59 should be coded with a "1". If a title IV-E case receives payment for only one day in the PUR, the case remains in the sample. * ACF has instituted several procedures to validate aspects of AFCARS data. All State semi-annual AFCARS data submissions undergo extensive edit-checks for internal reliability. The results of the AFCARS edit- checks for each of the six-month data submissions are automatically generated and sent back to each State, to support State efforts to improve data quality. Many States submit revised data, often for more than one prior submission period, to ensure that AFCARS reflects accurate State data. The Children's Bureau in ACF conducts several AFCARS compliance reviews each year, which typically yield comprehensive AFCARS Improvement Plans (AIP). Additionally, ACF funds the National Resource Center for Child Welfare Data and Technology, which provides States with technical assistance to improve reporting to AFCARS, improve statewide information systems, and make better use of child welfare data. Finally, ACF hosts the National Child Welfare Data and Technology Conference to further support efforts to improve data accuracy and integrity across child welfare systems. * In spite of dedicated efforts to qualify or validate AFCARS data before eligibility review samples are drawn, coding errors are inevitable. For instance, a State may reclassify a case from title IV- E eligible to non-IV-E eligible, but necessary coding updates may not transmit to the State system before the AFCARS report is produced. Coding cannot be changed once the sample has been drawn for a State's eligibility review. At that point, ACF conducts a quality check once the sample has been drawn to verify the accuracy of the information and ensure that cases included in the review sample are correctly coded as "1" in data element 59 for the review period. If the child was not eligible at any point in the six-month review period, with proper agency documentation, the case is removed from the eligibility review sample and replaced with one from the oversample. However, if a case receives a title 1V-E payment even for just one day in the review period, the case remains in the sample. * GAO cites what it considers high use of oversample cases to replace sample cases that are discovered to have received no payment during the review period as a threat to validity. In communications during the course of this inquiry, ACF staff members cited several examples to illustrate instances in which cases that are eligible for payment during a review period, might legitimately be found to receive no title IV-E payments during the review period as, for example, if a child comes into care in the last days of the review period and a payment is not made until the following month. In such instances, when cases are discovered in review samples, a case from the oversample is used to replace the original case. * The finding that oversample cases are used to replace cases from the original sample in such instances, indicates that the sampling and oversampling process is working properly to exclude cases that do not meet the selection criteria (i.e., eligible and receive a payment during the review period) from the review sample and ensure that the review sample contains the required number of cases for the review. Oversample cases are also drawn at random from the same universe of cases as is the original sample, and use of an oversample case does not alter the probability of selection of any case in the universe. All cases in the universe (i.e., cases with a IV-E payment in the review period) are still equally likely to be selected for the sample. Thus, the resulting sample is a random sample of all cases that received a title IV-E payment(s) during the review period for foster care services. GAO Recommendation #3: Augment guidance to teams gathering state-level Foster Care program improper payment estimate data to include specific procedures to follow in identifying and reporting any underpayments and duplicate or excessive payment errors. We concur. * In fulfilling our oversight responsibilities and working to reduce improper payments to States, ACF continues to refine its efforts to identify underpayments and overpayments (including ineligible payments, duplicate payments, and otherwise unallowable program payments). * We appreciate GAO's recommendation to take steps to support identification of underpayments and duplicate or excessive payments. It is consistent with our action plan, currently underway, to provide additional guidance on identifying and reporting all types of improper payments. ACF has already undertaken a number of activities that address many of the concerns raised by GAO concerning review of underpayments. For example, new guidance was released to staff in August, 2011, to further clarify procedures for calculating and reporting underpayments. This guidance was supplemented through technical assistance and conference calls with program and fiscal staff to help them understand an additional eligibility review worksheet related to case-level underpayments. Additional guidance on improper payments, including underpayments, overpayments and duplicate payments, will be fully operationalized with the release of the updated Eligibility Review Guide and review instrument during the FY2012 review cycle. This new documentation was not considered in the GAO Inquiry, which examined 2010 measurement and reporting; however, we believe it addresses a good deal of this recommendation to enhance procedures for identifying and reporting underpayments and duplicate or excessive payments. GAO Recommendation #4: Revise existing procedures for calculating a national improper payment estimate for the Foster Care program to include a statistically valid method for aggregating state-level margins of error to derive an overall, inflation adjusted, program estimate. We concur with the recommendation to incorporate individual state margins of error in the national program estimate, and will further study the recommendation to adjust the estimate for inflation. * According to the information ACF presented to OMB in December 2004 we would calculate standard error estimates for each State estimate to be used in developing the confidence interval for the national program estimate. The program did, in fact, develop confidence intervals using State-level estimates of variance, aggregated into the national estimate as well, for the baseline estimate in 2005 and for subsequent estimates that followed the original estimation methodology of case- based extrapolation (i.e., through 2008, when the program began reporting based on the agreed-upon shift to a refined dollar-based extrapolation). * The dollar-based extrapolation methodology examined by GAO for this report considered variability across rather than within the State estimates in developing the 90 percent confidence interval for the national estimate. We can and will readily adjust that calculation in accordance with the GAO recommendation to incorporate individual state margins of error (i.e., standard error) into the aggregation of State- level estimates into the national program estimate. While this will not change the point estimate for the national figure, it will likely result in a different range (i.e., confidence interval) for the national estimate. We concur with this preferred approach. * The second element in GAO Recommendation #4 calls for the incorporation of an adjustment for inflation. We will further study this recommendation. Given that the program rate is based on a rolling estimate that spans just three years, and that inflation is relatively low, we will seek to determine whether revising the methodology to incorporate inflation would add sufficient value as to justify removing the estimate from the "real-time" dollars reflected in state review data. GAO Recommendation #5: Develop and implement procedures requiring states to implement and report on corrective actions whenever a state's estimated improper payment dollar error rate exceeds a specified target level for the program. We think this recommendation must be considered in conjunction with recommendation #6 and agree to consider the recommendations accordingly. * GAO correctly states that ACF does not base the determination of review compliance and requirement for a State to undertake corrective actions on the payment error rate identified in the Eligibility Review sample. Eligibility reviews were initially developed to examine proper application of title IV-E foster care eligibility criteria, to recover title IV-E funds claimed by States for ineligible cases and, in conjunction with a required program improvement plan required for States determined to be non-compliant, to support performance improvement to be reflected in future reviews. The foster care eligibility review effort supports greater accuracy in Federal financial assistance to States and provides timely and specific feedback to States to improve program administration and implementation. As approved by OMB, eligibility review data are also used for improper payments estimation in order to minimize undue burden on States; nevertheless, standards for review compliance are based on eligibility review criteria rather than on improper payment error rates. * It is important to note that although compliance thresholds determine whether States must develop and implement Performance Improvement Plans and whether extrapolated disallowances are taken following a secondary review, all final eligibility review reports include complete lists and descriptions of all eligibility and other payment errors identified (with associated disallowances taken), and ACF Regional Office staff work with States to improve performance in those areas specified in their review reports. * ACF will consider ways to enhance existing Eligibility Review Guide instructions to ensure that appropriate attention is paid to addressing any eligibility review findings involving improper payments not specifically requiring development of a corrective action plan. GAO Recommendation #6: Establish and implement procedures requiring periodic assessments of state-level improper payment target levels, including targets associated with Title IV-E secondary reviews, for the Foster Care program for which states are to implement and report on corrective actions. We concur. * The clear majority of States reviewed to date have taken action to comply with applicable regulatory and eligibility provisions as they apply to foster care programs and have successfully performed within the performance level established for substantial compliance. * ACF is committed to continuing to improve program performance and will reexamine the error rate threshold for substantial compliance and explore the feasibility of lowering the threshold. The expectation is that a lowering of the threshold could further program efforts to reduce improper payments and serve as strong encouragement to States to continue to implement effective practices to reduce the number of cases in error and improve their child welfare systems. It is important to note that implementation of such changes cannot occur until such time as existing regulations have been modified. GAO Recommendation #7: Enhance ARTMS reporting capabilities to provide data on the status of actions taken to address Single Audit findings concerning states' Foster Care program payments, such as providing reporting capabilities to allow ARTMS users to search for specific audit findings by type of finding, grantee, state, region, or across years. We plan to further study this recommendation, to balance the value of such potential enhancements in light of the significant relevant data already available to ACF. * Information on title IV-E program findings from each audit cleared by ACF is detailed in an audit determination letter sent to the State by ACF. These letters reference any recurring findings, corrective actions taken to date and any needed follow-up actions. Each ACF Regional Office is responsible to maintain this correspondence in an official grant file. Additionally, ACF's Audit Resolution Tracking and Monitoring System (ARTMS) currently provides the capacity to search for audit information by Federal program, grantee name, audit resolution status, and audit periods. These search criteria should permit the identification of all related audit findings. [End of section] Appendix V: GAO Contact and Staff Acknowledgments: GAO Contact: Susan Ragland, (202) 512-8486 or raglands@gao.gov: Staff Acknowledgments: In addition to the contact named above, Carla Lewis, Assistant Director; Betty Ward-Zukerman, Assistant Director; Sharon Byrd, Assistant Director; Sophie Brown; Gabrielle Fagan; Vincent Gomes; and Nhi Nguyen made key contributions to this report. Also contributing to this report were Kay Brown; Wilfred Holloway; Francine DelVecchio; Jason Kirwan; and Karen O'Conor. [End of section] Footnotes: [1] Codified, as amended, at 42 U.S.C. §§ 670-679c. [2] Examples of HHS OIG reports include: HHS OIG, Audit of Allegheny County Title IV-E Foster Care Claims From October 1997 Through September 2002, A-03-08-00554 (Jan. 4, 2011); Review of Title IV-E Foster Care Costs Claimed on Behalf of Delinquent Children in Georgia, A-04-07-03519, (June 17, 2010); Review of California's Title IV-E Claims for Payments Made by Los Angeles County to Foster Homes of Relative Caregivers, A-09-06-00023 (Oct. 2, 2009); and Philadelphia County's Title IV-E Claims Based on Contractual Per Diem Rates of $300 or Less for Foster Care Services from October 1997 Through September 2002, A-03-07-00560 (May 19, 2008). [3] GAO, Foster Care and Adoption Assistance: Federal Oversight Needed to Safeguard Funds and Ensure Consistent Support for States' Administrative Costs, [hyperlink, http://www.gao.gov/products/GAO-06-649] (Washington, D.C.: June 15, 2006). [4] Examples of state-level audit reports include: California State Auditor, State of California Internal Control and State and Federal Compliance Audit Report for the Fiscal Year Ended June 30, 2010 (Sacramento, CA: Mar. 29, 2011); KPMG, Government of the District of Columbia Schedule of Expenditures of Federal Awards and Reports Required by Government Auditing Standards and OMB Circular A-133, Year Ended September 30, 2010 (Washington, D.C.: Jan. 27, 2011); and State of Indiana, State Board of Accounts, State of Indiana Single Audit Report July 1, 2009 to June 30, 2010 (Indianapolis, Ind.: Feb. 25, 2011). [5] Pub. L. No. 107-300, 116 Stat. 2350 (Nov. 26, 2002). IPIA was amended by the Improper Payments Elimination and Recovery Act of 2010, Pub. L. No. 111-204, 124 Stat. 2224 (July 22, 2010). The changes made by this law to IPIA estimation and reporting were first implemented for fiscal year 2011 reporting. [6] The act defines an improper payment as any payment that should not have been made or that was made in an incorrect amount (including overpayments and underpayments) under statutory, contractual, administrative, or other legally applicable requirements. It includes any payment to an ineligible recipient, any payment for an ineligible service, any duplicate payment, payment for services not received, and any payment that does not account for credit for applicable discounts. Office of Management and Budget (OMB) guidance also instructs agencies to report payments for which insufficient or lack of documentation was found as improper payments. [7] In its fiscal year 2005 performance and accountability report, HHS reported an improper payment estimate for the Foster Care program for fiscal years 2004 and 2005. According to HHS, the fiscal year 2004 error rate had not been finalized prior to the issuance of its fiscal year 2004 performance and accountability report, and thus was not reported in that publication. [8] The reported estimate of improper payments in the Foster Care program for fiscal year 2011 is $72 million or 5.3 percent. [9] GAO, Status of Fiscal Year 2010 Federal Improper Payments Reporting, [hyperlink, http://www.gao.gov/products/GAO-11-443R] (Washington, D.C.: Mar. 25, 2011). [10] OMB, Standards and Guidelines for Statistical Surveys (September 2006); OMB Memorandum M-06-23, Issuance of Appendix C to OMB No. Circular A-123 (Aug. 10, 2006); GAO, Inflation Adjustments, (Washington, D.C.: Sept. 14, 2011); Assessing the Reliability of Computer-Processed Data, [hyperlink, http://www.gao.gov/products/GAO-09-680G] (Washington, D.C.: July 2009); Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999); and Using Statistical Sampling, [hyperlink, http://www.gao.gov/products/GAO/PEMD-10.1.6] (Washington, D.C.: May 1992). [11] States, local governments, and non-profit organizations that expend $500,000 or more in a year in federal awards are to have an audit conducted under the provisions of the Single Audit Act, 31 U.S.C. §§ 7501-7507. Typically, this takes the form of a "Single Audit," which includes audit coverage of both the entity's financial statements and the federal awards it receives. [12] 42 U.S.C. §§ 670-679c. Title IV-E also authorizes funds to states for Adoption Assistance programs, which provide financial support to families who adopt eligible children with special needs from the foster care system. [13] See 42 U.S.C. §§ 672(i), 674(a)(3). [14] Section 208 of the Social Security Act Amendments of 1994, Pub. L. No. 103-432, § 208, 108 Stat. 4398, 4457 (Oct. 31, 1994), codified, as amended, at 42 U.S.C. § 1320a-9, authorized state demonstration programs for which the Secretary was permitted to waive some Title IV- E requirements. The Adoption and Safe Families Act of 1997, Pub. L. No. 105-89, § 301, 111 Stat. 2115, 2127 (Nov. 19, 1997) extended and expanded HHS's authority to use waivers for child welfare programs, allowing up to 10 new demonstration projects each year. [15] The nine states that have active Title IV-E waiver agreements are California, Florida, Illinois, Indiana, Iowa, Minnesota, Ohio, Oregon, and Wisconsin. [16] See [hyperlink, http://www.gao.gov/products/GAO-06-649]. [17] ACF regional offices are located in Boston, MA; New York, NY; Philadelphia, PA; Atlanta, GA; Chicago, IL; Dallas, TX; Kansas City, MO; Denver, CO; San Francisco, CA; and Seattle, WA. [18] 42 U.S.C. § 1320a-2a; 45 C.F.R. § 1356.71. [19] AFCARS is the federal information system that collects and processes data on children in foster care and those who have been adopted under the auspices of state child welfare agencies. AFCARS serves as central depository of various nationwide data on the foster care program, as required by the Title IV-E legislation. ACF uses this system for, among other purposes, determining and assessing outcomes for children and families, budget planning and projections, and targeting areas for greater or potential technical assistance efforts. The data in AFCARS are self-reported and maintained by the states, and are subject to information system assessment reviews and federally mandated edit checks by ACF. [20] According to ACF, HHS employs a 10 percent error threshold as part of its sampling methodology to determine the level of state compliance in meeting the federal requirements in the Foster Care program. If during a primary review, in which 80 cases are reviewed, 4 or fewer cases are found to be in error, HHS can be 91 percent certain that no more than 10 percent of the entire population of Title IV-E Foster Care cases will be in error. [21] States have the right to appeal disallowance decisions to HHS's Departmental Appeals Board. HHS's Departmental Appeals Board is responsible for providing impartial, independent review of disputed decisions in a wide range of HHS programs. It generally issues final decisions on behalf of HHS, which may then be appealed to federal court. [22] Given the time it takes to develop and implement a PIP, a secondary review is typically conducted 2 to 3 years after the primary review. [23] OMB Circular No. A-133, Audits of States, Local Governments and Non-Profit Organizations (includes revisions published in the Federal Register on June 27, 2003, and June 26, 2007). [24] OMB Circular No. A-50, Audit Followup (Sept. 29, 1982). [25] Pub. L. No. 107-300, 116 Stat. 2350 (Nov. 26, 2002). [26] OMB Memorandum M-06-23, Issuance of Appendix C to OMB Circular A- 123 (Aug. 10, 2006). OMB has updated this guidance to reflect changes made by the Improper Payments Elimination and Recovery Act of 2010, Pub. L. No. 111-204, 124 Stat. 2224 (July 22, 2010); these updates first took effect during the fiscal year 2011 reporting cycle. [27] Prior to IPIA, OMB identified seven agency programs within HHS, including the Foster Care program, as being susceptible to significant improper payments. The other six high risk programs within HHS were Head Start, Medicare, Medicaid, Temporary Assistance for Needy Families, State Children's Insurance Program, and the Child Care and Development Fund. Because OMB deemed these programs as high risk, HHS was required to report estimated improper payments internally to OMB. After IPIA became effective in fiscal year 2004, HHS began reporting improper payment information on Foster Care and the other programs in its annual agency financial report. [28] The $73 million estimate of improper payments in the Foster Care program for fiscal year 2010 consists of $9 million in underpayments and $64 million in overpayments. [29] According to ACF, the range for the national improper payment estimate for the Foster Care program is $68.0 to $77.3 million, at a confidence interval of 90 percent. [30] Administrative payments (which are based on a state's average monthly Title IV-E administrative cost claims per case) make up a large portion of Title IV-E expenditures and can vary greatly across states. For example, for fiscal year 2010, administrative payments represent 71 percent of the total federal reimbursements for Alabama and North Carolina compared to 13 percent for West Virginia. Administrative costs are not subject to a maximum limit for reimbursement. The federal government generally reimburses the states for 50 percent of eligible administrative costs with no limit. [31] Federal financial participation is the federal government's share of expenditures made by an entity in implementing a program administered by the agency. [32] ACF does not currently have any initiatives under way to estimate improper payments related to SACWIS, training, or demonstration projects. [33] The seven states where ACF has conducted ACR pilots are Arizona, Texas, and New York in fiscal year 2007; Iowa and Wisconsin in fiscal year 2009; and New Mexico and North Carolina in fiscal year 2011; the two states scheduled for fiscal year 2012 are New Hampshire and Mississippi. [34] A probability sample is defined as a sample from a population selected by some random method such that each item in the population has a known, nonzero probability of being drawn that can be calculated. [35] OMB, Standards and Guidelines for Statistical Surveys (Sept. 2006). [36] The data contained in AFCARS is self-reported by the states. ACF relies on existing AFCARS Assessment Reviews which evaluate states' automated information systems for collecting, extracting, and submitting AFCARS data, and assess the accuracy of data collected in accordance with the regulation requirements as well as ACF policy and technical guidance. However, these reviews are not conducted annually for all states and do not address verifying the accuracy or completeness of the data element--#59 Title IV-E (Foster Care)--ACF uses to develop its population of foster care cases for estimating improper payments. [37] [hyperlink, http://www.gao.gov/products/GAO-09-680G]. [38] GAO, Internal Control Management and Evaluation Tool, [hyperlink, http://www.gao.gov/products/GAO-01-1008G] (Washington, D.C.: August 2001). [39] The other types of errors identified related to eligibility. These included providers not licensed or approved, ineligible payments (e.g., therapy), a child not being eligible under the Aid to Families with Dependent Children program at the time of removal, criminal records check not completed, judicial determination regarding reasonable efforts to finalize permanency plan not timely, and no judicial determination of reasonable efforts to prevent removal. [40] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [41] ACF did not identify underpayments in 31 of 51 state eligibility reviews and did not identify duplicate or excessive payments in 36 of 51 state eligibility reviews. While we did not assess the validity of the reported findings in these states, total absence of such errors seems inconsistent with the general distribution of errors reported elsewhere. Also, Florida was excluded from review due to a statewide waiver demonstration project. [42] According to ACF, Title IV-E eligibility reviews for Florida were suspended pending completion of the statewide Foster Care demonstration project, and therefore the results were not included for fiscal year 2010 reporting. [43] Our analysis was limited to only identifying 3 states because ACF did not identify the dollar error rate for 9 of the other 24 states. Therefore, there may be other states that passed secondary reviews with an error rate over 10 percent because their dollar error rate was below 10 percent. [44] [hyperlink, http://www.gao.gov/products/GAO-01-1008G]. [45] Florida was excluded from review due to a statewide waiver demonstration project. [46] ARTMS, intended to be a HHS-wide system, is being rolled out in phases. Phase 1 was rolled out in 2010 to include ACF. ARTMS was designed by ACF's Office of Administration (Office of Financial Services--Division of Financial Integrity; Office of Grants Management; and Office of Information Services). With each phase of the roll out, the agency will include upgrades to enhance the system. [47] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [48] Pub. L. No. 107-300, 116 Stat. 2350 (Nov. 26, 2002). IPIA was amended by the Improper Payments Elimination and Recovery Act of 2010, Pub. L. No. 111-204, 124 Stat. 2224 (July 22, 2010). The changes made by this law to IPIA estimation and reporting were first implemented for fiscal year 2011 reporting. [49] OMB Memorandum M-06-23, Issuance of Appendix C to OMB Circular A- 123 (Aug. 10, 2006). [50] See 45 C.F.R. §§ 1355.35, 1355.36, 1356.71. [51] GAO, Foster Care and Adoption Assistance: Federal Oversight Needed to Safeguard Funds and Ensure Consistent Support for States' Administrative Costs, 06-649 (Washington D.C.: June 15, 2006); and Foster Care: State Practices for Assessing Health Needs, Facilitating Service Delivery, and Monitoring Children's Care, [hyperlink, http://www.gao.gov/products/GAO-09-26] (Washington, D.C.: Feb. 6, 2009). [52] Examples of HHS OIG reports include: HHS OIG, Audit of Allegheny County Title IV-E Foster Care Claims From October 1997 Through September 2002, A-03-08-00554 (Jan. 4, 2011); Review of Title IV-E Foster Care Costs Claimed on Behalf of Delinquent Children in Georgia, A-04-07-03519 (June 17, 2010); Review of California's Title IV-E Claims for Payments Made by Los Angeles County to Foster Homes of Relative Caregivers, A-09-06-00023 (Oct. 2, 2009); and Philadelphia County's Title IV-E Claims Based on Contractual Per Diem Rates of $300 or Less for Foster Care Services from October 1997 Through September 2002, A-03-07-00560 (May 19, 2008). [53] OMB Memorandum M-06-23, Issuance of Appendix C to OMB Circular A- 123 (Aug. 10, 2006); OMB, Standards and Guidelines for Statistical Surveys (September 2006); and GAO, Assessing the Reliability of Computer-Processed Data, [hyperlink, http://www.gao.gov/products/GAO-09-680G] (Washington, D.C.: July 2009); Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999); and Using Statistical Sampling, [hyperlink, http://www.gao.gov/products/GAO/PEMD-10.1.6] (Washington, D.C.: May 1992). [54] The scope of our review did not include an assessment of individual states' processes or payment systems. [55] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999); and GAO, Internal Control Management and Evaluation Tool, [hyperlink, http://www.gao.gov/products/GAO-01-1008G] (Washington, D.C.: August 2001). [56] States, local governments, and non-profit organizations that expend $500,000 or more in a year in federal awards are to have an audit conducted under the provisions of the Single Audit Act, 31 U.S.C. §§ 7501-7507. Typically, this takes the form of a "Single Audit," which includes both the entity's financial statements and the federal awards it receives. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E- mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548. GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E- mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548.