This is the accessible text file for GAO report number GAO-06-140 entitled 'Defense Management: Actions Are Needed to Improve the Management and Oversight of the National Guard Youth Challenge Program' which was released on November 29, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: United States Government Accountability Office: GAO: November 2005: Defense Management: Actions Are Needed to Improve the Management and Oversight of the National Guard Youth Challenge Program: GAO-06-140: GAO Highlights: Highlights of GAO-06-140, a report to congressional committees: Why GAO Did This Study: The fiscal year 1993 National Defense Authorization Act established the National Guard Youth Challenge Program as a pilot program to evaluate the effectiveness of providing military based training to improve the life skills of high school dropouts. The Assistant Secretary of Defense for Reserve Affairs, under the authority of the Under Secretary of Defense for Personnel and Readiness, is responsible for overall policy for the program. The National Guard Bureau (NGB) provides direct management and oversight. In 1998, Congress permanently authorized the program and began decreasing the federal cost share until it reached its current level of 60 percent in 2001. Conference Report 108-767 directed GAO to review the program. Specifically, GAO reviewed (1) historical trends of the program; (2) the extent of analyses performed to determine program costs and the need to adjust the federal and state cost share; and (3) NGB oversight of the program. GAO is also providing information on Reserve Affairs’ and states’ efforts to obtain funding from alternative sources. What GAO Found: Between fiscal years 1998 and 2004, total expenditures for the Challenge Program, including funds spent to cover the federal and state cost shares and federal management expenses, have increased from about $63 million to $107 million. During this same period, participation in the program has grown from 10 sites in 10 states to 29 sites in 24 states and Puerto Rico. Since the program’s inception, NGB has reported positive performance outcomes in academic performance, community service activities, and post-residential placements. For example, in 2004, NGB reported graduating 7,003 students, or 79 percent of those enrolled, with 70 percent of those graduates earning a high school equivalent diploma. While Reserve Affairs and NGB have expressed concern about the current program funding level and have suggested increasing both the cost basis used to determine funding needs and the federal cost share, neither has performed analyses to support the need for such changes. Federal financial standards state that reliable cost information is crucial for effective management of government operations. Since 1993, NGB has used $14,000 per student as the basis for determining the amount of funds needed to cover program operating costs, and applied the federal-state cost share to this amount. To keep pace with inflation, NGB has suggested increasing the per student cost to $18,000. Reserve Affairs has reported some states are having difficulty meeting their share and, in 2004, recommended the federal share be increased from 60 percent to 75 percent. However, neither Reserve Affairs nor NGB has compiled or analyzed data on actual program costs, states’ financial situations, or the impact of adjusting the federal and state cost-share. Without better cost and financial information, the Department of Defense (DOD) cannot justify future funding requests or a change in the cost-share ratio. Although NGB uses various oversight mechanisms, it lacks a complete oversight framework, making it difficult to measure program effectiveness and to adequately address audit and review findings. Also, some audits have not been performed as required. The Government Performance and Results Act suggests a complete oversight framework including goals and measures against which to objectively evaluate performance. While NGB requires states to report certain performance outcomes, it does not require states to establish performance goals in these areas, and therefore does not have a firm basis for evaluating program outcomes and DOD’s return on investment. Existing agreements require state programs to be audited at least every three years. However these audits have not been conducted as required and no provisions exist for submitting audit results to NGB. Without regular audits and access to results, NGB cannot be assured that programs are using federal funds appropriately and that audit findings are addressed. What GAO Recommends: GAO recommends a number of actions designed to improve the management and oversight of the National Guard Youth Challenge Program. DOD concurred with GAO’s recommendations. www.gao.gov/cgi-bin/getrpt?GAO-06-140. To view the full product, including the scope and methodology, click on the link above. For more information, contact Sharon Pickup at (202) 512-9619 or pickups@gao.gov. [End of section] Contents: Letter: Results In Brief: Background: Challenge Program Expenditures and Participation Have Increased and NGB Has Reported Positive Performance Outcomes: Reserve Affairs and NGB Have Not Analyzed Data to Determine Actual Program Costs or the Need to Adjust the Federal and State Cost Share: The National Guard Bureau Lacks a Complete Framework for Oversight of the Challenge Program, Making It Difficult to Measure Program Effectiveness: Reserve Affairs Lacks a Formal Approach for Securing Alternative Funding Support for the Challenge Program, While Some Participating States Have Obtained Alternative Funding Support as a Result of Their Efforts: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: National Guard Youth Challenge Program Sites: Appendix III: Comments from the Department of Defense: Appendix IV: GAO Contact and Staff Acknowledgment: Tables: Table 1: Hypothetical Examples Showing How Changing the Cost Basis or Cost-Share Ratio Would Affect Yearly Challenge Program Funding Amounts: Table 2: States Selected for Site Visits or Interviews: Figures: Figure 1: Annual Expenditures for the National Guard Youth Challenge Program: Figure 2: Number of National Guard Youth Challenge Program Sites by Reporting Year: Figure 3: Youth Participation in National Guard Youth Challenge Program: Figure 4: Challenge Program Sites by 2004 Graduation Results: Figure 5: Number of GEDs Earned by Challenge Students: Figure 6: Post-Residential Placement Data, 2003-2005: Abbreviations: DMARS: Data Management and Reporting System: DOD: Department of Defense: DOJ: Department of Justice: DOL: Department of Labor: ED: Department of Education: GED: general educational development: NGB: National Guard Bureau: United States Government Accountability Office: Washington, DC 20548: November 29, 2005: The Honorable John Warner: Chairman: The Honorable Carl Levin: Ranking Minority Member: Committee on Armed Services: United States Senate: The Honorable Duncan L. Hunter: Chairman: The Honorable Ike Skelton: Ranking Minority Member: Committee on Armed Services: House of Representatives: After Operation Desert Storm and the end of the Cold War, Congress began to reevaluate the focus of U.S. military activities and proposed using some military assets and training to help address critical domestic challenges such as drugs, poverty, and unemployment. In particular, some policymakers saw an opportunity to use the military's capabilities to address domestic needs by providing role models for youth and training and educational opportunities for the disadvantaged. Section 1091 of the National Defense Authorization Act for Fiscal Year 1993 established the National Guard Youth Challenge Program (Challenge Program) as a pilot program to evaluate the feasibility and effectiveness of military based training to improve the life skills and employability of high school dropouts. After the program was permanently authorized in 1998, Congress began limiting the share of state operating costs covered by the federal government through the Department of Defense (DOD), which initially paid for 100 percent of these costs. DOD now pays 60 percent of state operating costs, and states provide 40 percent. In addition, the authorizing legislation for the Challenge Program allows the Secretary of Defense to use nondefense funding sources in support of the program. The Assistant Secretary of Defense for Reserve Affairs, under the authority of the Under Secretary of Defense for Personnel and Readiness, is responsible for management oversight of the Challenge Program, including policy and funding issues, while the National Guard Bureau (NGB) is responsible for the administration of the program, including daily management and oversight. For example, NGB maintains frequent contacts with state programs and conducts program evaluations through a contractor, while Reserve Affairs prepares the program budget based on input from NGB and ensures that states are able to provide their required share. To participate in the program, states must sign a cooperative agreement with NGB that lays out roles and responsibilities among the states and NGB, and also provides guidance on the content of the program. Since 1993, NGB has used a cost of $14,000 per student as a basis for providing funds to state programs. Conference Report 108-767 directed the Comptroller General to conduct a study of the Challenge Program.[Footnote 1] Specifically, we examined: (1) historical trends of the Challenge Program, including program expenditures, participation, and performance; (2) the extent to which Reserve Affairs and NGB have determined actual program costs and the consequent need to adjust the federal and state cost share; and (3) the extent to which NGB has provided oversight of the program. We also determined the extent to which Reserve Affairs and participating states have made an effort to obtain alternative funding support for the Challenge Program. In performing our work, we interviewed officials from the Office of the Assistant Secretary of Defense for Reserve Affairs; NGB; the Departments of Education, Labor, and Justice; the contractor that monitors and evaluates state programs; the National Guard Youth Foundation; and selected states. We also reviewed the program's annual reports and other documents provided by Reserve Affairs and NGB as well as the data management system used by states to collect Challenge Program student information and track individual and program activities. We concluded that the data were sufficiently reliable for our purposes. We analyzed program evaluations, resource management reviews, and audits, in addition to reviewing existing policies and procedures for managing the program. We conducted a survey of all 29 established programs to collect information on their budgets and expenditures, federal and state funding levels, each state's ability to fund their individual program, and the availability of funding from other sources. We also visited or contacted eight Challenge Program sites in seven states and asked them about any alternative sources of funding for their programs. A detailed description of our scope and methodology is presented in appendix I. We conducted our work from January 2005 to October 2005 in accordance with generally accepted government auditing standards. Results In Brief: According to NGB, program expenditures and state participation in the Challenge Program have increased since the program began, and the program has achieved positive performance outcomes. Between fiscal years 1998 and 2004, total expenditures for the Challenge Program, including funds spent to cover the federal and state cost shares and federal management expenses, have increased from about $63 million to about $107 million. For fiscal year 2004, for example, NGB spent approximately $61.6 million for the federal share of the program and $5.8 million for NGB management costs, while states contributed approximately $40.5 million. Ten states participated in the first year of the pilot program in 1993; and today there are 29 Challenge Program sites in 24 states and Puerto Rico. Additionally, 9 states remain on a waiting list to start new programs, and several states have expressed an interest in expanding their existing programs. Some common performance outcomes that NGB reports include the number of graduates earning a general educational development credential, changes in program students' scores on standardized math and reading tests, and the number of community service hours performed by Challenge Program students. For example, the Challenge Program reported that 70 percent of graduates in 2004 earned a general education development credential. Another performance outcome that NGB reports is the number of placements of program graduates in continuing education, the military, or the labor force at the time they complete their 12-month post- residential phase. For example, in 2004, 3,698 graduates were placed in one of these categories by the end of the post-residential phase of the program. Although Reserve Affairs and NGB have expressed concern about the current program funding level and have suggested increasing both the cost basis used to determine funding needs and the federal cost share, we found that neither Reserve Affairs nor NGB has performed analyses to support the need for such changes. Good budget practices, included in the Office of Management and Budget's Federal Financial Accounting Standards, state that agencies should determine actual costs of their activities on a regular basis and that reliable cost information is crucial for effective management of government operations. Since 1993, NGB has used a cost of $14,000 per student as the basis for determining the amount of funds needed to cover program operating costs. However, due to cost variations between states, program officials that we surveyed reported that they actually spent between $9,300 and $31,031 per graduate. According to NGB, all 29 programs are providing the services required by the cooperative agreements, and several states have added program enhancements such as field trips or vocational classes. However, some of these states reported that they reduced nonrequired services to stay within their budgets. For example, they implemented staff pay and hiring freezes, eliminated the student stipend, and eliminated some enrichment activities such as field trips and vocational classes. In addition, Reserve Affairs, NGB, and participating states have suggested that the cost-share ratio be changed from its current 60 percent federal share to a 75 percent federal share because they believe that the current required 40 percent state share is sometimes difficult for states to meet. However, neither Reserve Affairs nor NGB has compiled or analyzed data on actual program costs, states' financial situations, or the impact of adjusting the federal and state cost share. Reserve Affairs has asked the National Guard Bureau to determine a new funding formula for the program based on individual state needs, but, at the time of our review, NGB had not yet done so and Reserve Affairs has not given NGB a deadline for completion. Without better cost and financial information, DOD cannot justify future funding requests or a change in the cost share ratio. Although NGB has several mechanisms in place for overseeing the Challenge Program, it lacks a complete oversight framework, making it difficult to measure the effectiveness of the program and to adequately address audit and review findings. NGB conducts several oversight activities, including informally communicating with state program directors and having an outside evaluator conduct yearly operational evaluations and biennial resource management reviews of the Challenge Program, and these reports have prompted some changes to the program. However, until recently, NGB did not have a formal mechanism for tracking the findings of these reports. In response to our review, NGB recently implemented a mechanism for tracking these findings. As part of its oversight activities, NGB does require each state program to report certain performance outcome measures, such as academic test scores. However, it does not require states to establish performance goals in these areas. A complete oversight framework includes performance goals and measures against which to objectively measure performance as well as a mechanism for tracking findings of audits or reviews and responding to those findings. Without clear and agreed upon performance goals, there is no objective yardstick against which to fully measure program performance and effectiveness and thereby assess DOD's return on investment. In addition, Standards for Internal Control in the Federal Government states that agencies need to ensure that the findings of audits and reviews are promptly resolved. Under the cooperative agreements governing the Challenge Program, United States property and fiscal officers in each state are required to conduct full audits of the individual Challenge Programs at least once every three years. However, according to property and fiscal officers that we spoke with, all of these audits have not been conducted as required, due to the low priority placed on these audits and lack of staff for the property and fiscal officer. If these audits are not conducted, it may be difficult to ensure that federal interests are adequately protected. Copies of these audits are not currently being provided to program managers at NGB because, according to the Chief of Property and Fiscal Affairs at NGB, there is no specific requirement to do so. If the National Guard does not see the audits, it cannot ensure that the findings of these audits are promptly resolved. The authorizing legislation for the Challenge Program allows the Secretary of Defense to use nondefense funding sources in support of the program, and allows states to supplement program funds from other resources. However, Reserve Affairs has not adopted a formal strategy for pursuing nondefense funding, while some states have been successful in securing alternative funding support for their programs. Reserve Affairs has primarily adopted informal strategies to contact agencies outside DOD to inform them about the Challenge Program and seek opportunities for partnerships, but has not been successful in securing funds from these agencies. We found that because Reserve Affairs has not made a formal business case to request funds from these nondefense agencies, these agencies are unable to determine whether or not they are specifically able to fund the Challenge Program. Officials from some federal agencies told us that that their agencies have general authority to provide funds to other programs if those programs are consistent with their agency's interests. However, officials at the Department of Justice, the Department of Labor, and the Department of Education stated that Reserve Affairs needed to present more specific information to them before they could determine whether funds could be provided to the Challenge Program. According to those officials, at a minimum, any such request should contain the amount of funding sought and a sufficiently detailed description of the program to allow the agency receiving the request to determine whether it would be an appropriate use of funds. Until Reserve Affairs makes a more formal request for funding, other agencies will be unable to determine if they can provide funds for the Challenge Program. At the state level, some states we contacted had made efforts to obtain alternative funding support to enhance or maintain their program's operations. For example, some state programs organized into charter schools to obtain funding from local school districts, applied for National School Lunch Program funding, obtained grants from nonprofit organizations, and received additional funding from the state government beyond the required match. Some states reported obstacles in securing supplemental funding, such as the prohibition against using Challenge funds to hire grant writers. We are making recommendations designed to improve DOD's management and oversight of the Challenge Program and to strengthen efforts in obtaining alternative funding support for the program. In commenting on a draft of this report, DOD concurred with our recommendations. DOD's comments and our evaluation of them appear later in this report. Background: The National Guard Youth Challenge Program is a 17-month program that serves at-risk youth at 29 sites in 24 states and Puerto Rico. The purpose of the program is to improve the education, life skills, and employment potential of students by providing military-based training, supervised work experience, and knowledge in eight core program components.[Footnote 2] Students must be 16 to 18 years old, drug-free, unemployed, high school dropouts, and not in trouble with the law. NGB reports that more than 59,000 students have been graduated from the Challenge Program since it began as a pilot in 1993. The program was authorized by 32 U.S.C. §509 on a permanent basis in fiscal year 1998, at which time states were to begin paying a share of operating costs. Challenge Program Structure: Each Challenge Program site operates two residential classes per year, one of which begins in January and the other around July.[Footnote 3] A typical graduation goal is 100 students per class, or 200 per year, although several programs graduate more students. In 2004, for example, Illinois graduated almost 800 students in 2004, and Louisiana's three sites combined graduated more than 950 students. The residential phase of the program runs 22 weeks and includes a 2-week Pre-Challenge phase. During Pre-Challenge, applicants are assessed for their ability and motivation to complete the remaining 20 weeks of the residential program. Those who successfully complete Pre-Challenge are then formally enrolled in the Challenge Program in numbers that equal each program's graduation target plus normal program attrition rate. In the residential phase, students receive military-based training and supervised work experience. Additionally, each state develops a curriculum that incorporates the eight core components and the tasks, conditions, and standards that students must complete to demonstrate progress in those components.[Footnote 4] Each student must receive a score of at least 80 percent on each core component to be graduated from the program. During the 12-month post-residential phase, individuals who have successfully completed the residential phase are involved in placement activities, which include employment, education, volunteer activities, or any combination of the three or military service. The graduates work with adult mentors who were matched with them during the residential phase. These mentors provide guidance and support to the graduates and are required to contact the youths twice each month at a minimum. Program staff use the written, post-residential action plan that each student prepares and updates during the residential phase to monitor placement activities. Mentors also use this plan during their interactions with graduates. The Challenge Program reports youth placement activities at the end of the 12-month follow-up period. To further assess the long-term impact of the program, NGB has contracted with AOC Solutions to conduct a retrospective longitudinal study of program graduates as well as students who did not complete the program. Program Funding: Prior to 1998, the federal government, through DOD, completely funded the Challenge Program. In fiscal year 1998, Congress began requiring states to provide a minimum of 25 percent of their programs' operating costs. The state cost share increased 5 percent each year until fiscal year 2001, when it reached the current funding requirement of 40 percent.[Footnote 5] Although some states had provided more funds than required in the past, program funding each year is now determined by the 40 percent share, which is based on $14,000 for each youth targeted for graduation. In addition to the federal and state funds used to operate the program sites, DOD also provides funds for NGB management expenses such as program evaluations, contractor-provided training, and travel for training and workshops. These NGB program management costs are not subject to the federal/state 60/40 cost share requirement. Each state submits a budget to NGB that is based on that state's target for number of graduates. Since the program's inception, the funding provided by NGB has been based on a cost per student of $14,000.[Footnote 6] For example, if a state has a target of 100 students per class (200 per year) to graduate, the estimated program costs would be $2.8 million. The federal contribution, or 60 percent of the total, would be $1.68 million, while the state contribution would be $1.12 million. To receive federal funding, a state must certify that it has sufficient funds to provide its 40 percent share. State funds can be composed of cash, noncash supplies, services, or a combination of these sources. States are allowed to provide additional funding (over and above the 40 percent share) to the program from sources such as individual and corporate donations, additional moneys from the state general fund or other state revenue sources, or other federal funding. Some Challenge Program sites, for example, operate as alternative schools and are reimbursed by their state education agencies for portions of their program costs. Management, Oversight, and Support: Reserve Affairs, under the authority of the Under Secretary of Defense for Personnel and Readiness, is responsible for preparing the annual budget and reviewing state budgets and funding certifications. Reserve Affairs is to monitor program compliance with DOD policy, issue supplemental policy guidance, and submit the Challenge Program annual report to Congress. NGB provides day-to-day administration and oversight of the Challenge Program, issuing regulations, and submitting budgets and annual report drafts to Reserve Affairs. NGB has contracted with AOC Solutions to assist with the oversight of the Challenge Program. AOC Solutions performs the annual operational evaluations and the biennial resource management reviews. This contractor also pulls together the program information for the annual report and maintains and oversees the Data Management and Reporting System (DMARS), which is used to collect student data and report on individual and program activities. NGB has also contracted with Dare Mighty Things to provide training and technical assistance to Challenge Program staff.[Footnote 7] Finally, NGB has United States property and fiscal officers in each state who are responsible for receiving and accounting for all federal Challenge Program funds and property under control of the National Guard in that state. The property and fiscal officers are also responsible for ensuring that federal funds are properly obligated and expended. NGB enters into cooperative agreements with governors of states approved to participate in the Challenge Program. The cooperative agreements describe the responsibilities of the states and NGB as well as the funding, costs, and regulations for operating National Guard Youth Programs.[Footnote 8] The cooperative agreements also define the eight core components and provide guidance on how to run the residential and post-residential phases and other aspects of the Challenge Program. Each Challenge Program state is also required to submit state plans and budget estimates for their state. These state plans must include details on the state's procedures and be consistent with overall program guidance provided by DOD. For example, state plans include information on application and selection procedures, staffing and staff training, and a detailed budget. Challenge Program Expenditures and Participation Have Increased and NGB Has Reported Positive Performance Outcomes: According to NGB, Challenge Program expenditures and state participation have increased since the program began, and the program has achieved positive program performance outcomes over time. Since the program's inception, total expenditures have increased from about $63 million to about $107 million per year. The number of states participating in the Challenge Program has also increased, and several states have expressed interest in adding a program or expanding existing ones. Challenge sites must account for their activities throughout the year, and NGB has reported positive performance outcomes over time. Program Expenditures Have Increased Over Time: NGB reports that overall federal expenditures for the Challenge Program have increased over time, but states have also increased their expenditures since the program was permanently authorized in fiscal year 1998. Between fiscal years 1998 and 2004, total expenditures for the Challenge Program, including funds spent to cover the federal and state cost shares and federal management expenses, have increased from about $63 million to about $107 million. For fiscal year 2004, for example, NGB expenses included $61.6 million for the federal cost share and $5.8 million for NGB management costs, while states contributed approximately $40.5 million.[Footnote 9] In addition, in 2000 and 2001, the Challenge Program received $5 million and $7,483,500 respectively from the Department of Justice. Reserve Affairs stated that the primary use of these funds has been to start new Challenge Program sites. Since 2001, four programs were established using these funds and three programs' operations were maintained in 2002. In total, approximately $5.97 million remain unspent in a nonexpiring account. Officials at Reserve Affairs and NGB told us that these funds remain unspent because no new Challenge Programs have started. According to these officials, new programs have not been established because state governments have not committed the required 40 percent match. (See fig. 1 for total program expenditures from fiscal year 1998 to 2004, broken down by federal and state cost share and NGB management expenses.) Figure 1: Annual Expenditures for the National Guard Youth Challenge Program: [See PDF for image] Note: Federal cost shares for fiscal years 2001 and 2002 include $6.5 million from Department of Justice. State cost share amounts are the required state matches only and do not include any additional funding or in-kind support the states might have provided. [End of figure] The Number of States Participating in the Challenge Program Has Increased: When the pilot program began in 1993, there were 10 Challenge Program sites in 10 states. The program has now grown to 29 sites in 24 states and Puerto Rico.[Footnote 10] In fiscal year 2005, Wyoming received funds to start up a program site. According to NGB, Wyoming will begin its first class in January 2006. In addition to those states currently operating Challenge Program sites, there are also nine states that have expressed interest in establishing new programs.[Footnote 11] For example, according to NGB officials, representatives from Washington and Indiana National Guard units have visited some existing program sites and are in the process of developing state programs. Other states are interested in expanding their programs to serve more youth at existing sites or to open new locations. On the other hand, for various reasons including difficulty meeting the state match requirement, lack of state support, and substandard facilities, four states have discontinued their Challenge Programs. Connecticut, a pilot program state, dropped its program in 1994 after two classes. Colorado discontinued its program after 1999, Missouri after 2002, and New York after 2003. Figure 2 describes the number of Challenge Program sites for each year since the program began. Appendix II identifies the individual states with Challenge Program sites. Figure 2: Number of National Guard Youth Challenge Program Sites by Reporting Year: [See PDF for image] Note: Reporting year is the time frame covered in annual reports. It includes the July class from the previous year and the January class from the current year. [End of figure] Student participation in and graduation from the Challenge Program have also increased over time. States are required to track the number of youth who have applied to the Challenge Program, enrolled in the third week of the program (after the 2-week Pre-Challenge phase), and were graduated from the residential phase. According to NGB, the target graduation rate for 2004 was 6,961; the actual number of enrollees was 8,920; and 7,003 students were graduated, or 79 percent of those enrolled, from the Challenge Program. Figure 3 shows the target numbers, the actual number of students who were enrolled in the residential phase at week 3, and the number that graduated from the program from 2000 through 2004. Figure 3: Youth Participation in National Guard Youth Challenge Program: [See PDF for image] Note: Reporting year is the time frame covered in annual reports. It includes the July class from the previous year and the January class from the current year. [End of figure] Although all Challenge Programs graduate two classes per year, the number of graduates per class varies. In addition, some states have multiple programs. For example, Louisiana has three Challenge Programs and, in 2004, graduated a total of 952 students. Figure 4 identifies those states currently participating in the Challenge Program by the number of graduates they reported for 2004. Figure 4: Challenge Program Sites by 2004 Graduation Results: [See PDF for image] Note: The 2004 reporting year covers NGB classes 21 and 22, which began in July 2003 and January 2004. [End of figure] NGB Has Reported Positive Performance Outcomes: NGB has reported positive performance outcomes in academic performance, community service activities, and post-residential placements. Program performance information is tracked by each Challenge site and submitted to NGB. Each year, the Challenge Program reports on outcomes for the two classes completing the 22-week residential phase during that reporting year and for the two preceding classes as they complete their 1-year post-residential follow-up phase. The Challenge Program sites use the same automated system, DMARS, to collect information on students and report on their progress and activities. The information collected in DMARS is reviewed by the contractor through weekly and monthly reports and during random checks of source documents during operational evaluation site visits. Some residential phase outcomes of the Challenge Program, such as the number of graduates earning a general educational development (GED) credential or high school degree and changes in scores on standardized math and reading tests, are tied to the core component of Academic Excellence.[Footnote 12] For example, NGB reported that 70 percent of graduates in 2004 earned a GED. Figure 5 illustrates the outcomes of GED attainment for the past 5 reporting years. Students also take the Tests of Adult Basic Education, a series of tests that identify individual education levels in various academic subject areas. Each state program tests its students early in the residential phase and then toward the end of the 22-week period, and it reports the changes in test scores. In 2004, for example, NGB reported that graduating students improved 1.7 grade levels in reading and 1.8 grade levels in math during the residential phase. Another core component, Service to the Community, requires each student to perform a minimum of 40 hours of service to the community or conservation project activities. The number of community service hours performed by each student is tracked, and the total number of hours for each site is another outcome that the Challenge Program reports annually. For example, in 2004, NGB reported that Challenge Program students performed more than 590,000 hours of community service, such as maintaining historical cemeteries and parks and supporting organizations such as Special Olympics and Habitat for Humanity. Figure 5: Number of GEDs Earned by Challenge Students: [See PDF for image] [End of figure] Figure 6: Each month of the post-residential phase, each Challenge Program graduate, or that individual's mentor, reports on the graduate's post-residential activities. Following the 12-month post- residential phase, each Challenge Program site reports graduate placements in continuing education, the military, or the labor force. These placements are verified with schools, the military, and employers and are documented. Program representatives are not always able to contact all graduates for placement information and therefore placement data reflect only the students contacted, not all graduates. Education placements include returning to high school or going to a post- secondary or vocational-training institution, which students may be attending full-or part-time. Some Challenge Program graduates also enter the military, into either the active or the reserve forces. Post- residential employment placements can be full-or part-time, and they include those graduates who are self-employed. Graduates can have placements in more than one of these categories. For example, an individual might be attending school and working part-time. Challenge Program sites continue to update their placement records after the 12- month follow-up period when they come in contact with former students. During the longitudinal study, for example, the contractor has been able to update placement data based on information received from state program officials and graduates. Figure 6 shows post-residential placement trends for the past 3 reporting years. The total numbers of graduates placed in these 3 years are 2,407 in 2003; 3,698 in 2004; and 4,086 in 2005. Figure 7: Post-Residential Placement Data, 2003-2005: [See PDF for image] Notes: Program representatives are not always able to contact all graduates for placement information and therefore placement data reflect only the students contacted, not all graduates. [A] According to DOD, data for 2005 are preliminary, may change, and have not yet been officially published in an annual report. [B] Miscellaneous placements include caregivers, volunteers, homemakers, incarcerated, or individuals who are unable to hold an approved placement due to disability and/or hospitalization. Graduates must care for an individual or work as a volunteer for a minimum of 30 hours per week to be considered placed as a caregiver or volunteer. [End of figure] Reserve Affairs and NGB Have Not Analyzed Data to Determine Actual Program Costs or the Need to Adjust the Federal and State Cost Share: Although Reserve Affairs and NGB have expressed concern about the current program funding level and have suggested increasing both the cost basis used to determine funding needs and the federal cost share, we found that neither Reserve Affairs nor NGB has performed analyses to support the need for such changes. Good budget practices, included in the Office of Management and Budget's Federal Financial Accounting Standards,[Footnote 13] state that agencies should determine actual costs of their activities on a regular basis and that reliable cost information is crucial for effective management of government operations. Without better cost and financial information, DOD cannot justify future funding requests or a change in the cost-share ratio. Other than calculating how inflation has affected program costs, NGB has not analyzed data on actual program costs. Since 1993, NGB has used a cost of $14,000 per student as the basis for determining the amount of funds needed to cover program operating costs. In 2003, NGB calculated that if that amount were adjusted for inflation, it would be $18,000. The results of our survey of all Challenge Programs showed that in 2004, states actually spent between $9,300 and $31,031 per graduate with an average of $15,898 per graduate.[Footnote 14] In addition, our survey showed that, on average, states estimated that the program should be funded at approximately $16,900 per target graduate to cover all of the services in the cooperative agreements, although the estimates ranged from $14,000 to $31,800.[Footnote 15] Most Challenge Program officials also told us that increasing the cost per student funding level for the program without increasing the federal cost share would negatively impact their programs because their states would be unlikely to come up with the additional state match money. Because costs vary between states due to regional differences in salary levels, staff benefits, and facility costs, Reserve Affairs has asked NGB to determine a new funding formula for the program based on individual state needs. At the time of our review, NGB had not yet done this and Reserve Affairs has not given NGB a deadline for completion. In addition to expressing a desire to change the amount of funding per student, Reserve Affairs, NGB, and participating states have suggested that the cost-share ratio be changed from its current 60 percent federal share to a 75 percent federal share because they believe that the current 40 percent state share is sometimes difficult for states to meet; however, neither Reserve Affairs nor NGB has analyzed states' financial situations or the impact of adjusting the federal and state cost share. Challenge Program officials told us that increasing the federal cost share of the program would be beneficial because it would enable states to expand their existing programs; give states more flexibility in funding their programs; and allow programs to restore to students some services that had been eliminated due to budgetary constraints. Although we did not analyze how changing the cost basis or the cost-share ratio would affect specific states, we prepared hypothetical examples for illustrative purposes. Table 1 shows how changing the cost basis and the cost-share ratio would affect federal and state required funding levels. Table 1: Hypothetical Examples Showing How Changing the Cost Basis or Cost-Share Ratio Would Affect Yearly Challenge Program Funding Amounts: Current funding situation of $14,000 per student and 60 percent federally funded; Per individual program: Typical number of students per program: 200; Per individual program: State cost share per program: $1,120,000; Per individual program: Federal cost share per program: $1,680,000; Per individual program: Total number of students in program[A]: 7,000; Total Challenge Program: Total required state cost share: $39,200,000; Total Challenge Program: Total required federal cost share: $58,800,000; Total Challenge Program: Total program cost[B]: $98,000,000. Change in cost basis to $16,900[C] per student and 60 percent federally funded; Per individual program: Typical number of students per program: 200; Per individual program: State cost share per program: $1,352,000; Per individual program: Federal cost share per program: $2,028,000; Per individual program: Total number of students in program[A]: 7,000; Total Challenge Program: Total required state cost share: $47,320,000; Total Challenge Program: Total required federal cost share: $70,980,000; Total Challenge Program: Total program cost[B]: $118,300,000. Change in funding ratio to 75 percent federally funded with cost basis of $14,000 per student; Per individual program: Typical number of students per program: 200; Per individual program: State cost share per program: $700,000; Per individual program: Federal cost share per program: $2,100,000; Per individual program: Total number of students in program[A]: 7,000; Total Challenge Program: Total required state cost share: $24,500,000; Total Challenge Program: Total required federal cost share: $73,500,000; Total Challenge Program: Total program cost[B]: $98,000,000. Change in cost basis to $16,900[C] per student and 75 percent federally funded; Per individual program: Typical number of students per program: 200; Per individual program: State cost share per program: $845,000; Per individual program: Federal cost share per program: $2,535,000; Per individual program: Total number of students in program[A]: 7,000; Total Challenge Program: Total required state cost share: $29,575,000; Total Challenge Program: Total required federal cost share: $88,725,000; Total Challenge Program: Total program cost[B]: $118,300,000. Source: GAO analysis. [A] Approximate number of students currently in program. [B] This amount does not include NGB overhead costs. [C] This amount is based on the average amount that states estimated in our survey would cover all of the services outlined in the cooperative agreements. [End of table] In our survey, states reported varying views on whether they were experiencing difficulty in meeting their share of program costs. We did not verify the basis for their responses. Our survey showed that some states are able to provide funds above the required match; some states provide only the required match; and some states are unable to provide a match based on $14,000 per student and therefore fund the program at a lower level and receive less money from NGB. For example, California is able to provide additional support beyond the required match through additional money provided from the state general fund and funding from the program's local school district. In 2004, California spent approximately $20,200 per graduate. Oregon, on the other hand, funds its program primarily through state education money. Due to recent state budget difficulties, Oregon cannot fund the program at $14,000 per student. In 2004, Oregon spent approximately $12,600 per graduate. According to the National Guard Bureau, all 29 programs are providing the services required by the cooperative agreements, and several states have added program enhancements such as field trips or vocational classes. However, some states reported that they reduced nonrequired services to stay within their budgets. For example, they implemented pay and hiring freezes; eliminated the student stipend; and eliminated program enrichment activities, such as field trips and vocational classes. Some states told us that additional funding, provided by a change in the cost-share ratio or an increase in the per student funding amount, would allow them to restore some of these services. The National Guard Bureau Lacks a Complete Framework for Oversight of the Challenge Program, Making It Difficult to Measure Program Effectiveness: Although NGB has several mechanisms in place for overseeing the Challenge Program, it lacks a complete oversight framework, making it difficult to measure the effectiveness of the program. A complete oversight framework, as suggested by the Government Performance and Results Act of 1993 (the Results Act)[Footnote 16] and Standards for Internal Control in the Federal Government,[Footnote 17] includes performance goals and measures against which to objectively measure performance as well as a mechanism for tracking findings of audits or reviews and responding to those findings. Currently, NGB does not require participating states to establish performance goals for individual programs and therefore does not have a firm basis for evaluating program outcomes and DOD's return on investment. In accordance with the cooperative agreements, U.S. property and fiscal officers in each state are required to conduct full audits of state Challenge Programs at least every 3 years. However, these audits have not been conducted as required; and, when audits are conducted, copies of the results are not provided to NGB for review. Without regular audits and access to results, NGB cannot be assured that programs are using federal funds appropriately and that audit findings are addressed. The National Guard Bureau Conducts Several Oversight Activities and Recently Implemented a Mechanism for Tracking Audit Findings: NGB conducts several oversight activities for the Challenge Program. In accordance with the DOD Instruction[Footnote 18] and cooperative agreements governing the Challenge Program, NGB is responsible for the overall administration of the program, including program oversight. NGB uses both informal and formal mechanisms to oversee the program. On an informal basis, NGB frequently communicates with state program directors via e-mail and telephone calls. In addition, according to NGB, if a program director has a problem or an issue that he or she feels NGB needs to be involved with, he or she will initiate contact. Formal oversight of the program is conducted by NGB through AOC Solutions with yearly operational evaluations and biennial resource management reviews of all 29 Challenge Programs. The purpose of the operational evaluations is to assess the programs' compliance with the cooperative agreements and the implementation of the eight core components. The resource management reviews focus on assessing programs' financial accountability and reviewing resources including staffing levels and salaries; food service costs; and physical inventory of property. Both the operational evaluations and the resource management reviews have identified areas for improvement and, according to NGB, changes were made to the program. For example, the operational evaluation of one state program conducted in fiscal year 2004 reported that program staff was calling the students inappropriate names. According to NGB, a staff member was dismissed as a result of this finding. Another operational evaluation of a different program conducted in fiscal year 2005 found that over 90 percent of the community service hours accumulated by the students were for kitchen patrol. The cooperative agreements state that work in the dining facility may not be counted towards community service hours. According to NGB, this program has completely revamped its community service program and students now participate in such activities as visiting with residents of the local veterans' home and caring for a historic cemetery. Until recently, NGB did not have a formal mechanism for tracking the findings of the reports conducted by AOC Solutions. During our review, we discussed with NGB the importance of keeping track of review findings in order to adequately respond to these findings, in accordance with Standards for Internal Control in the Federal Government. In response to our review, in October 2005, NGB provided new guidance to AOC Solutions regarding the operational evaluations and resource management reviews, which required, among other things, AOC Solutions to review findings from previous evaluation reports to determine whether corrective actions have been taken where warranted. In addition, officials at NGB told us that they currently monitor responses to audit and review findings informally with individual program directors. The National Guard Bureau Does Not Require Participating States to Establish Performance Goals for the Challenge Program: Although NGB requires state Challenge Programs to report on certain outcome measures, such as GED attainment and number of graduates, NGB does not require states to establish any performance goals in these areas to measure the effectiveness of the program. The establishment of performance goals is consistent with the principles of effective management as set forth in the Results Act and would allow NGB to better evaluate the overall performance of the program and to assess DOD's return on its investment. Without clear and agreed upon performance goals, there is no objective yardstick against which to fully measure program performance and thereby assess DOD's return on investment. Although it may not be reasonable to have the same performance goal for all state programs, it would be appropriate for each state program to negotiate a performance goal for defined performance areas such as increases in standardized test scores or physical fitness levels. Similar state programs, overseen by the Department of Labor, set individual negotiated levels of performance for specified core performance measures. These measures are used to provide information for systemwide reporting and evaluation for program improvement. For example, state Workforce Investment Act[Footnote 19] programs negotiate performance measures for youth ages 14 to 18 in three areas: attainment of basic skills; attainment of high school diplomas or their equivalents; and placement in education, the workforce, or the military. For the area of diploma attainment, the goals range from 42.8 percent of participants in Louisiana to 68 percent of participants in New Hampshire. Currently, Challenge Program states are required to submit state plans annually. These state plans are required to contain long-term and annual performance goals and are to be updated annually. However, NGB has not provided guidance on specific performance areas where states should focus their goals; therefore, states may not have goals in the same performance areas, making it difficult for NGB to compare performance across programs. For example, California's state plan contains a goal to acquire additional sources of funding through grants and charitable contributions. Oregon's state plan, on the other hand, contains a goal stating that 80 percent of graduates from the residential portion of the program will be placed in education, the military, or employment but does not contain any goal related to acquiring additional sources of funding. In addition, states are not currently held accountable to the goals that they do set since the evaluation process does not measure the states' performance against their goals. NGB told us that beginning in January 2006, states would be held accountable to the goals outlined in their state plans. U.S. Property and Fiscal Officers Have Not Conducted Required Audits and Audit Results Are Not Provided to NGB: Although the cooperative agreements governing the Challenge Program currently require U.S. property and fiscal officers to perform full audits of state Challenge Programs at least every 3 years, and prior to January 2005, the cooperative agreements required full audits every year, these audits have not been conducted as required. For example, according to NGB, out of a required 29 audits, only 14 were conducted in 2003, and only 7 audits were conducted in 2004. According to property and fiscal officers that we spoke with, audits were not conducted due to the lower priority placed on these audits compared to other audits that needed to be conducted within the state and a lack of staff to conduct the audits. Because the property and fiscal officers are responsible for ensuring that federal dollars are appropriately spent, if these audits are not conducted, it may be difficult to ensure that federal interests are adequately protected. When the property and fiscal officers do conduct audits of the Challenge Program, they are not currently providing copies of the audit results to NGB because, according to the Chief of Property and Fiscal Affairs at NGB, there is no specific requirement to do so. Standards for Internal Control in the Federal Government states that agencies need to ensure that the findings of audits and reviews are promptly resolved. If NGB does not review these audits, it cannot ensure that audit findings are resolved or identify trends across programs that may require action at a programwide level. According to officials at NGB, the audits remain internal to the state and the property and fiscal officer works directly with the state Challenge Program site to resolve any issues. In addition, if any audit findings require action from NGB, the state property and fiscal officer will contact NGB and ask for assistance. NGB needs to be aware of all audit findings, including those reported by the property and fiscal officers to effectively manage the Challenge Program. We reviewed the most recent property and fiscal officer audits for each Challenge Program participating state and found that they did identify areas that needed improvement. For example, an audit conducted of one program in fiscal year 2004 found unspent program dollars totaling approximately $180,000 that needed to be returned to the NGB. According to a follow-up report by the property and fiscal officer, this money was returned to NGB. Another audit conducted during fiscal year 2003 found that oversight of budget expenditures for one program was not adequate to ensure accurate, timely, and complete accounting of expenditures and recommended that key management controls over the program be identified, documented, published and tested. A follow-up audit of the program found that this recommendation was implemented. Reserve Affairs Lacks a Formal Approach for Securing Alternative Funding Support for the Challenge Program, While Some Participating States Have Obtained Alternative Funding Support as a Result of Their Efforts: Reserve Affairs has not adopted a formal strategy for pursuing nondefense funding, while some participating states have obtained alternative funding support for their programs as a result of their own efforts. Because Reserve Affairs has not made a formal business case to request funds from nondefense agencies, these agencies are unable to determine whether or not they are able to fund the Challenge Program. As a result, Reserve Affairs is potentially missing out on additional sources of funding that could enhance the program. Some participating states have obtained alternative funding support to enhance or maintain services provided to Challenge Program students. Nondefense Agencies Are Unable to Determine If They Can Fund the Challenge Program without a More Formal Request for Funding: Reserve Affairs has not made a formal business case to request funds from nondefense agencies and therefore these agencies are unable to determine whether or not they are able to fund the Challenge Program. Although the authorizing legislation for the Challenge Program allows the Secretary of Defense to use nondefense funding sources in support of the program, Reserve Affairs has not adopted a formal strategy for pursuing nondefense funding. Rather, Reserve Affairs has primarily adopted informal strategies to contact agencies outside DOD to inform them about the Challenge Program and seek opportunities for partnerships. For example, Reserve Affairs officials have sent information via e-mail to officials at federal agencies with programs targeting at-risk youth (i.e. Department of Justice (DOJ), Department of Labor (DOL), and Department of Education (ED)) to inform them about the Challenge Program. These e-mails did not include specific requests for program funding. In addition, in July 2003, Reserve Affairs presented senior DOL officials with a proposal for forming an interagency partnership with the Challenge Program, but did not ask for a commitment of funding from DOL in a specified amount. Additionally, officials from these agencies have been invited to events sponsored by the National Guard Youth Foundation in support of the Challenge Program. Lastly, Reserve Affairs officials told us that they participate or have participated on a number of interagency councils and working groups that represent at-risk youth such as the President's Crime Prevention Council, the White House Federal Interagency Working Group on Service, the Corporation for National and Community Service, the National Civilian Community Corps, the Math Science Initiative, and the Juvenile Justice and Delinquency Prevention Council.[Footnote 20] Officials from DOJ, DOL, and ED noted that their agencies have general authority to provide funds to other programs if the transferred funds were used to support a program consistent with their agency's interests. It is not unprecedented for an agency to transfer funds to the Challenge Program. In fiscal years 2000 and 2001, DOD and DOJ signed an interagency agreement under which DOJ agreed to provide $5 million in 2000 and $7,483,500 in 2001 to the Challenge Program.[Footnote 21] As DOJ concluded in this situation, and as officials at DOJ, DOL and ED stated, the transfer of funds to another agency must be consistent with the purpose and legal requirements of the program from which funds would be transferred. Moreover, these officials stated that sufficient funding must be available in the program from which the funds would be transferred, and making such a transfer must be in the interest of the agency. Officials at DOJ, DOL, and ED stated that Reserve Affairs needed to present more specific information to them before they could make a determination as to whether funds could be provided to the Challenge Program. These officials stated that an executive branch agency that wishes to make a request for funding from another executive branch agency could make such a request in any number of ways, either formally or informally. At a minimum, they noted, any such request should contain the amount of funding sought and a sufficiently detailed description of the program to allow the agency receiving the request to determine whether it would be an appropriate use of funds. At the time of our review, none of the agency officials we met with were aware of any specific request from Reserve Affairs concerning this matter. Until Reserve Affairs makes a more formal request for funding, other agencies will be unable to determine if they are able to provide funds for the Challenge Program. Some Participating States Have Obtained Alternative Funding Support to Enhance or Maintain Services Provided to Challenge Program Students: Under cooperative agreements with NGB, states are not required to seek funding support beyond the required federal and state contributions; however, some states have made efforts to obtain alternative funding support for their programs to enhance or maintain the services provided to Challenge Program students. Under authorizing legislation for the Challenge Program, states may accept, use, and dispose of gifts or donations of money, other property, or services for the Challenge Program. Reserve Affairs and NGB officials said that they encouraged states to seek out additional funding sources for their programs if they want to enhance the services provided to program students. To assist state programs in their efforts to obtain that additional support, NGB, through its contractor Dare Mighty Things, shares information among state programs on strategies for successfully organizing a 501 (c) (3) corporation,[Footnote 22] developing a fundraising policy that focuses on the long-term vision of the Challenge Program, accessing and sharing a grant writer with other programs, applying for National School Lunch Program funds, and educating state legislators to secure funding support. By providing information and examples of programs that are currently implementing these strategies, other state Challenge Programs have the information available to them on how to obtain additional funding support for their programs. In our review of the program, we found that some states had identified strategies for soliciting additional funding support from nonprofit organizations. For example, in the annual plans submitted to NGB, some states developed specific strategies for obtaining additional funding support such as setting a goal for the number of grants they would apply for and establishing a 501 (c) (3) corporation to raise funds on behalf of the program. We saw further evidence of states' efforts to obtain additional funding support through our site visits and in our survey of the 29 Challenge Program sites. For example, in one state we visited, the Challenge site was also a charter school, which allowed the program to receive additional funding from the local school district. Moreover, in four states we visited, the state legislatures provided additional funding beyond the required state match of 40 percent to pay for additional staffing costs or facilities. According to our survey of all Challenge Programs, 28 out of 29 programs identified receiving some type of funding support beyond the required federal and state contributions to pay for program expenditures incurred during 1999 through 2004. These programs identified receiving additional funding from the states' general fund beyond the required state match as well as support from other state agencies such as the state Department of Education. Additionally, the programs we surveyed relied on funding assistance from other federal agencies such as the Department of Agriculture, which provides funding under the National School Lunch Program; nonprofit organization grants; the programs' private 501 (c) (3) corporation; and donations from private individuals. Results from our site visits and national survey also showed that state Challenge Programs rely on donations of goods and services to support the program. For example, states we visited relied on donations from private citizens, corporations, schools, and the states' National Guard units for such items as computers and software, exercise equipment, books, uniforms, and shoes. Some programs also received support from the surrounding community in the form of donated services such as transportation assistance and medical services from local doctors and nurses. Our survey further showed that 21 out of 29 programs received some form of donated goods and services. Program officials we met stated that donations of goods and services were vital to programs because many times they do not have funds available to acquire equipment or services for the program. Donations of goods and services can enhance the basic program beyond the military-based training and academics required under the cooperative agreements adding little or no cost to the program. Despite the success that many states had in obtaining additional funding support and donations of goods and services for their programs, some states reported obstacles in securing supplemental funding. For example, the cooperative agreements governing the Challenge Program do not allow states to use Challenge Program funds to hire full-time grant writers. Lastly, some programs we visited expressed concern that if they received funding support outside the federal and state required contributions, then their state governments might reduce their allocations. Conclusions: Although Reserve Affairs, NGB, and participating states have suggested the current cost basis of $14,000 per student is not sufficient to sustain the program and that the cost-share ratio should be changed, Reserve Affairs and NGB have done little analysis to show what the actual costs of the program are and how changing the cost-share ratio for the program would impact participating states. Without better cost and financial information, DOD cannot justify future funding requests or a change in the cost-share ratio. Until NGB establishes clear and agreed upon performance goals, there is no objective yardstick against which to fully measure program performance. Without these performance goals in place, NGB does not have a firm basis for evaluating program outcomes and DOD's return on investment. Although property and fiscal officers are required to conduct full audits of the Challenge Program at least every 3 years, these audits have not been conducted due to competing priorities at the state level and a lack of staff. Without these audits, it may be difficult to ensure that federal interests are adequately protected. Also, because NGB does not receive copies of audits conducted by property and fiscal officers, it cannot know what the findings of these audits were and if changes were made to the programs based on these findings. Until Reserve Affairs makes a more formal request for funding, other agencies will be unable to determine if they can provide funds for the Challenge Program. Without a more formal process that outlines the amount of funding needed and a detailed description of the Challenge Program and its specific funding needs, Reserve Affairs is potentially missing out on additional sources of funding that could enhance the program. Recommendations for Executive Action: To improve the management and oversight of the National Guard Youth Challenge Program, we recommend that the Secretary of Defense, direct the Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant Secretary of Defense for Reserve Affairs and the Chief of the National Guard Bureau, to take the following three actions: * Determine the actual costs of the Challenge Program, including states' ability to fund their share of the program, and use this information, as appropriate, to support funding requests or a request to change the cost-share ratio. * Establish performance goals to measure the effectiveness of the Challenge Program. * Direct U.S. property and fiscal officers to conduct audits as required and require that copies of audit results are provided to the appropriate office at the National Guard Bureau in order to ensure that the results of audits are promptly reviewed and resolved. To strengthen efforts at obtaining alternative funding in support of the National Guard Youth Challenge Program, we recommend that the Secretary of Defense direct the Undersecretary of Defense for Personnel and Readiness, in consultation with the Assistant Secretary of Defense for Reserve Affairs, to develop more formal strategies for requesting alternative funding support for the Challenge Program. Such strategies may include submitting requests for funding that include the amount of funding requested and a sufficiently detailed description of the proposed program to allow potential providers of funds, such as nondefense agencies, to determine whether it would be an appropriate use of their funds. Agency Comments and Our Evaluation: In written comments on a draft of this report, DOD concurred with our recommendations. In its overall comments, DOD asserts that our report reaffirms that the program met its congressional mandate to improve the life skills and employment potential of participants by providing military-based training and supervised work experience. Further, DOD's comments state that our report validates the core program components of assisting participants to receive a high school diploma or its equivalent; developing leadership skills; promoting fellowship and community service; developing life coping and job skills; and improving physical fitness, health and hygiene. DOD also states that it appreciates GAO's confirmation that it is properly executing the program. We do not agree with DOD's characterization of our report. Whereas our report provides background data on program performance and mentions that DOD has reported positive outcomes from the Challenge Program, we do not assess whether or not DOD met its congressional mandate; we do not validate the program's core components, nor do we confirm that DOD is properly executing the program, as DOD's comments suggest. Rather our objectives were to examine: (1) historical trends of the National Guard Youth Challenge Program, including program expenditures, participation, and performance; (2) the extent to which Reserve Affairs and the NGB have determined actual program costs and the need to adjust the federal and state cost-share; and (3) the extent to which the NGB has provided oversight of the program. We also determined the extent to which Reserve Affairs and participating states have made an effort to obtain alternative funding support for the Challenge Program. In response to our recommendation to determine the actual costs of the Challenge Program, including the ability of states to fund their share, DOD concurred, but claims that the matching fund requirement makes the program vulnerable to state budget cuts. DOD also contends that until recently, sluggish state revenues made the cost share requirement more burdensome and inhibited the ability of both the NGB and the states to focus on determining the actual costs of the Challenge Program. DOD claims that the budget shortfalls required the states to reduce and/or eliminate program services and that, with state revenues slowly closing their budget gaps, restoring funding for needed program services is being considered. DOD also notes that it is currently working on a new funding formula for the program. Although DOD's comments suggest that adequate funding was not available for needed services, through our survey and other work, we found that all 29 programs were able to provide the services required by the cooperative agreements, and several states reported they added program enhancements such as field trips or vocational classes. However, some states reported that they reduced nonrequired services to stay within their budgets. Furthermore, the basis for DOD's comment regarding the effect of the matching fund requirement and the level of state revenues on the program is unclear. As we reported, neither Reserve Affairs nor NGB has compiled or analyzed data on actual program costs, states' financial situations, or the impact of adjusting the federal and state cost share. We continue to believe that any future funding request, or a request to change the cost share requirement, should be based on an analysis of the actual costs of the program. DOD's comments are printed in their entirety in appendix III. We are sending copies of this report to the Secretary of Defense and interested congressional committees. We will also make copies available to others upon request. This report will be available at no charge on GAO's Web site at http://www.gao.gov. If you or your staff have any questions regarding this report, please contact me at (202) 512-9619 or pickups@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. Signed by: Sharon L. Pickup: Director, Defense Capabilities and Management: [End of section] Appendix I: Scope and Methodology: To examine historical trends of the National Guard Youth Challenge Program, including program expenditures, participation, and performance, we interviewed officials from the Office of the Assistant Secretary of Defense for Reserve Affairs; the National Guard Bureau (NGB); and the contractor that monitors and evaluates state programs, AOC Solutions. We reviewed other documentation provided by Reserve Affairs and NGB, such as program funding summaries. We also reviewed the Challenge Program's annual reports submitted to Congress from 1994 through the present, excluding 1997,[Footnote 23] which provided us background information on the program. For data on participation and performance outcomes, including general education development credential attainment, community service hours, and post-residential placements, we relied on the Data Management and Reporting System (DMARS) that is used by NGB to collect participant information and track individual and program activities. DMARS was implemented for the 2003 reporting year, and the contractor has performed procedures to clean up the data as far back as 2000. We conducted data reliability tests on the program's annual reports and data management system to conclude that the data are sufficiently reliable for our purposes. We did not compare the costs or outcomes of this program to other similar youth programs currently funded by the federal government since that was not within the scope of this engagement. To assess the extent to which Reserve Affairs and NGB determined actual program costs and analyzed states' ability to fund their share of the program, we interviewed officials at Reserve Affairs and NGB as well as officials at state programs. We reviewed and analyzed internal budget documents prepared by state programs and submitted to NGB annually as well as documents prepared by individual program directors that discussed the funding situation in their state. We also reviewed and analyzed good budget practices as described in federal financial accounting standards. Using a semi-structured questionnaire, we interviewed eight state Challenge Program directors and their budget officers in seven states and discussed sources of funding for their programs, actual costs of operating their programs, and the types of services that are provided to program participants given the available funding. States selected for site visits or telephone interviews are shown in table 2. There were a number of factors which affected the judgmental sampling of these state programs. States were chosen because of their length of time in the program, size, geographic location, experience of the program's director, and whether or not there were multiple programs in the state. Additionally, in July 2005, we surveyed all 29 participating Challenge Programs and asked them about the actual costs of their programs and their assessment of their states' ability to fund the programs. This survey had a response rate of 100 percent. Table 2: States Selected for Site Visits or Interviews: State: Arizona; Program Name: Arizona Project Challenge. State: California; Program Name: California Grizzly Youth Academy. State: Illinois; Program Name: Illinois Lincoln's Challenge Academy. State: Maryland; Program Name: Maryland Freestate Challenge Academy. State: Oregon; Program Name: Oregon National Guard Youth Challenge Program. State: South Carolina; Program Name: Camp Long. State: South Carolina; Program Name: Columbia. State: Wisconsin; Program Name: Wisconsin National Guard Challenge Academy. Source: GAO. [End of table] To assess the extent to which NGB has provided oversight of the Challenge Program, we interviewed officials at Reserve Affairs and NGB as well as program officials at the state level. To determine the oversight responsibilities of federal and state officials involved in the program, we reviewed and analyzed pertinent laws, the Department of Defense's (DOD) Instruction governing the program, and the NGB Master Cooperative Agreement, which defines the terms and conditions of the program in each state. We also interviewed the independent contractor hired by NGB to conduct yearly and biennial on-site program reviews, AOC Solutions. We obtained access to its data management system and analyzed the program evaluations and resource management reviews it completed on behalf of NGB since 2003. We also reviewed and analyzed individual state program's annual plans and NGB's strategic plan to identify the types of goals set by the Challenge program. We consulted previous GAO work regarding performance measurement and evaluations, identified best practices for establishing and measuring performance goals, and reviewed Standards for Internal Control in the Federal Government and the Government Performance and Results Act of 1993. We interviewed representatives of the U.S. Property and Fiscal Officer's Office in six states as well as the NGB Chief of Property and Fiscal Affairs to determine the time frame for completing audits and mechanisms for reporting, tracking, and resolving issues that arise out of the PFO audits. To determine the extent to which Reserve Affairs and participating states have obtained alternative funding support for the program, we reviewed relevant laws, policies, and reports to determine the relevant authorities for receiving and transferring funds between federal agencies. We also interviewed federal officials at Reserve Affairs and the Departments of Labor, Education, and Justice to determine the extent to which these agencies discussed the possibility of transferring funds to DOD in support of the Challenge program. To determine states' efforts at obtaining alternative funding support for their program's operations, we conducted a survey of all 29 participating programs to collect information about their sources of funding for the program, how funding is distributed across different program operational functions, the sources and types of donated goods and services, and descriptions of strategies programs use to obtain alternative funding support. Using a semi-structured questionnaire, we interviewed eight state Challenge Program directors and members of their staff in seven states and discussed sources of funding for their programs, the strategies these specific programs use for obtaining alternative funding support and donations, and the difficulties these states face in seeking out other funding. We provided a draft of this report to officials at DOD for their review and incorporated their comments where appropriate. We conducted our work from January 2005 to October 2005 in accordance with generally accepted government auditing standards. [End of section] Appendix II: National Guard Youth Challenge Program Sites: State: Alaska; Program name: Alaska Military Youth Academy; Calendar year first class began: 1994; Number of graduates in 2004: 192. State: Arizona; Program name: Arizona Project Challenge; Calendar year first class began: 1993; Number of graduates in 2004: 182. State: Arkansas; Program name: Arkansas National Guard Youth Challenge Academy; Calendar year first class began: 1993; Number of graduates in 2004: 181. State: California; Program name: California Grizzly Youth Academy; Calendar year first class began: 1998; Number of graduates in 2004: 204. State: Florida; Program name: Florida Youth Challenge Academy; Calendar year first class began: 2001; Number of graduates in 2004: 227. State: Georgia; Program name: Georgia - Fort Gordon Youth Challenge Academy; Calendar year first class began: 2000; Number of graduates in 2004: 276. Program name: Georgia - Fort Stewart Youth Challenge Academy; Calendar year first class began: 1993; Number of graduates in 2004: 444. State: Hawaii; Program name: Hawaii National Guard Youth Challenge Academy; Calendar year first class began: 1994; Number of graduates in 2004: 207. State: Illinois; Program name: Illinois Lincoln's Challenge Academy; Calendar year first class began: 1993; Number of graduates in 2004: 785. State: Kentucky; Program name: Kentucky Bluegrass Challenge Academy; Calendar year first class began: 1999; Number of graduates in 2004: 160. State: Louisiana; Program name: Louisiana Youth Challenge Program - Camp Beauregard; Calendar year first class began: 1993; Number of graduates in 2004: 407. Program name: Louisiana Youth Challenge Program - Camp Minden; Calendar year first class began: 2002; Number of graduates in 2004: 214. Program name: Louisiana Youth Challenge Program - Gillis W. Long; Calendar year first class began: 1999; Number of graduates in 2004: 331. State: Maryland; Program name: Maryland Freestate Challenge Academy; Calendar year first class began: 1993; Number of graduates in 2004: 224. State: Michigan; Program name: Michigan Youth Challenge Academy; Calendar year first class began: 1999; Number of graduates in 2004: 174. State: Mississippi; Program name: Mississippi National Guard Youth Challenge Program; Calendar year first class began: 1994; Number of graduates in 2004: 404. State: Montana; Program name: Montana Youth Challenge Program; Calendar year first class began: 1999; Number of graduates in 2004: 144. State: New Jersey; Program name: New Jersey Youth Challenge Program; Calendar year first class began: 1994; Number of graduates in 2004: 178. State: New Mexico; Program name: New Mexico Youth Challenge Academy; Calendar year first class began: 2001; Number of graduates in 2004: 197. State: North Carolina; Program name: North Carolina Tarheel Challenge Academy; Calendar year first class began: 1994; Number of graduates in 2004: 200. State: Oklahoma; Program name: Oklahoma Thunderbird Youth Academy; Calendar year first class began: 1993; Number of graduates in 2004: 221. State: Oregon; Program name: Oregon National Guard Youth Challenge Program; Calendar year first class began: 1999; Number of graduates in 2004: 221. State: Puerto Rico; Program name: Puerto Rico Youth Challenge Program; Calendar year first class began: 1999; Number of graduates in 2004: 208. State: South Carolina; Program name: South Carolina Youth Challenge Academy - Camp Long; Calendar year first class began: 2002; Number of graduates in 2004: 108. Program name: South Carolina Youth Challenge Academy - Columbia; Calendar year first class began: 1998; Number of graduates in 2004: 257. State: Texas; Program name: Texas Seaborne Challenge Corps; Calendar year first class began: 1999; Number of graduates in 2004: 188. State: Virginia; Program name: Virginia Commonwealth Challenge Program; Calendar year first class began: 1994; Number of graduates in 2004: 140. State: West Virginia; Program name: West Virginia Mountaineer Challenge Academy; Calendar year first class began: 1993; Number of graduates in 2004: 175. State: Wisconsin; Program name: Wisconsin National Guard Challenge Academy; Calendar year first class began: 1998; Number of graduates in 2004: 154. Total Graduates: 7,003. Source: GAO analysis of data from AOC Solutions, Inc. for NGB Classes 21 and 22. Note: The Wyoming Youth Challenge Program received start-up funds in 2005, and will begin its first class in January 2006. Nevada does not have a Challenge Program site but sends approximately 24 youths per year to the Arizona site. Until fiscal year 2005, the District of Columbia sent 60 youths per year to the Maryland site. [End of table] [End of section] Appendix III: Comments from the Department of Defense: ASSISTANT SECRETARY OF DEFENSE: RESERVE AFFAIRS: WASHINGTON, DC 20301-1500: NOV 15 2005: Ms. Sharon L. Pickup: Director, Defense Capabilities and Management: U.S. Government Accountability Office: 441 G Street, N.W.: Washington, D.C. 20548: Dear Ms. Pickup: This is the Department of Defense (DoD) response to the Government Accountability Office (GAO) draft report, "DEFENSE MANAGEMENT: Actions are Needed to Improve the Management and Oversight of the National Guard Youth Challenge Program," dated October 24, 2005 (GAO Code 350643/GAO-06-140). The report reaffirms that the Department met its congressional mandate to improve life skills and employment potential of participants by providing military-based training and supervised work experience. It also validated the core program components of assisting participants to receive a high school diploma or its equivalent, leadership development, promoting fellowship and community service, developing life coping skills and job skills, and improving physical fitness and health and hygiene. A more appropriate title might be "The National Guard Youth Challenge Program: Statutory Requirements Being Met - Management and Oversight Can Be Strengthened." The Department appreciates GAO's confirmation that the DoD is properly executing the National Guard Youth Challenge Program, and the suggestions for improvement. The DoD comments on the draft report findings and recommendations are enclosed. The Department of Defense appreciates the opportunity to comment on the draft report. If there are any questions, my point of contact is Mr. Ernie Gonzales, (703) 693-8630. Sincerely, Signed by: T. F. Hall: Enclosure: As stated: GOVERNMENT ACCOUNTABILITY OFFICE DRAFT REPORT DATED OCTOBER 24, 2005 (GAO CODE 350643/GAO-06-140): "DEFENSE MANAGEMENT: Actions Are Needed to Improve the Management and Oversight of the National Guard Youth Challenge Program" DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS: RECOMMENDATION 1: The GAO recommended that the Secretary of Defense, direct the Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant Secretary of Defense for Reserve Affairs and the Chief of the National Guard Bureau to determine the actual costs of the Challenge Program, including states' ability to fund their share of the program, and use this information, as appropriate, to support funding requests or a request to change the cost-share ratio. (p. 34/GAO Draft Report): DOD RESPONSE: Concur, but the matching fund requirement makes the Challenge Program vulnerable to state budget cuts. Until recently, sluggish state revenues made the cost-share burden more burdensome and prohibited both the National Guard Bureau and the states to focus on determining the actual costs of the Challenge Program. The budget shortfalls required the states to reduce and/or eliminate program services. With state revenues slowly closing their budget gaps, restoring funding for needed program services are being considered. In the report, the GAO affirmed that the Department is currently working on a new funding formula for the program based on individual state needs. This new formula will adequately address and determine funding for required program services. RECOMMENDATION 2: The GAO recommended that the Secretary of the Defense, direct the Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant Secretary of Defense for Reserve Affairs and the Chief of the National Guard Bureau to establish performance goals in order to measure the effectiveness of the Challenge Program. (p. 34/GAO Draft Report): DOD RESPONSE: Concur, but although the Department does not specifically define program objectives, the GAO noted that the Department does report annually to Congress positive performance outcomes in academic performance, community service activities, and post-residential placements. RECOMMENDATION 3: The GAO recommended that the Secretary of the Defense, direct the Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant Secretary of Defense for Reserve Affairs and the Chief of the National Guard Bureau to direct U.S. Property and Fiscal Officers to conduct audits as required and require that copies of audit results are provided to the appropriate office at the National Guard Bureau in order to ensure that the results of audits are promptly reviewed and resolved. (pp. 34 and 35/GAO Draft Report): DOD RESPONSE: Concur. RECOMMENDATION 4: The GAO recommended that the Secretary of the Defense direct the Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant Secretary of Defense for Reserve Affairs to develop more formal strategies for requesting alternative funding support for the Challenge Program. Such strategies may include submitting requests for funding that include the amount of funding requested and a sufficiently detailed description of the proposed program in order to allow potential providers of funds, such as non- defense agencies, to determine whether it would be an appropriate use of their funds. (p. 35/GAO Draft Report): DOD RESPONSE: Concur. The Department acknowledges that it will need to take more formal actions for requesting supplemental funding and recognized that the informal meetings, presentations and discussions with other federal agencies did not result in formal commitments. [End of section] Appendix IV: GAO Contact and Staff Acknowledgment: GAO Contact: Sharon L. Pickup (202) 512-9619: Acknowledgments: In addition to the individual named above, Harold Reich, Karyn Angulo, Leslie Bharadwaja, Susan Ditto, K. Nicole Harms, Wilfred Holloway, Jessica Kaczmarek, Stanley Kostyla, Julia Matta, Renee McElveen, and John Van Schaik made key contributions to this report. FOOTNOTES [1] Conference Report 108-767 accompanied the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375 (2004). [2] The eight core program components are: leadership/followership, job skills, responsible citizenship, health and hygiene, service to the community, academic excellence, life-coping skills, and physical fitness. [3] NGB reports on site activities by NGB class numbers. The first class under the pilot program, NGB-1, began in July 1993, and NGB-25 started in July 2005. [4] A task is a clearly defined and measurable activity accomplished by an individual. Condition describes the training situation, environment, or field conditions under which the cadet must perform the task. The standard establishes criteria for how well a task must be performed. [5] 32 U.S.C. §509(d)(4). [6] Funding allocations to the states have historically been based on target enrollments; that is, the maximum number of students per class that are funded annually in a program's budget. Beginning with fiscal year 2004, funding allocations are now based on target graduation, which is the maximum number of students identified in a program's budget for participation in and graduation from the program. According to NGB officials, although allocations were originally based on a site's target enrollment, this figure was treated as a graduation goal. These numbers differ by state. [7] Contracts with AOC Solutions and Dare Mighty Things were competitively awarded for a 1-year base period with four 1-year options. [8] The Master Youth Programs Cooperative Agreement covers two youth programs: the National Guard Youth Challenge Program and the National Guard STARBASE Program. The STARBASE Program was established to improve the knowledge and skills of at-risk youth in math, science and technology. [9] At the time of our review, final expenditure reports for fiscal year 2005 were unavailable. [10] Louisiana has three Challenge Program sites, and Georgia and South Carolina each operate two sites. [11] According to NGB, as of August 2005, the District of Columbia and eight states had expressed interest in starting a Challenge Program: Alabama, Colorado, Connecticut, Delaware, Indiana, Nevada, Pennsylvania, and Washington. [12] In some states, younger youths are not eligible to take the GED due to age restrictions, and some youths are not academically prepared to take the tests. Some programs offer alternative degrees, such as adult education diplomas; some programs are accredited high schools and can offer high school diplomas or credits; and many youths return to their home high schools to earn their diplomas. [13] Office of Management and Budget, Statement of Federal Financial Accounting Standards, Number 4, Managerial Cost Accounting Concepts and Standards for the Federal Government, (Washington, D.C.: July 31, 1995). [14] We calculated this number with information provided to us by individual states on our survey. According to our calculations, in 2004, the Fort Gordon, Georgia Program spent $9,300 per graduated student and Alaska spent $31,031 per graduated student. In order to cover the high personnel costs in Alaska and to provide program enhancements, the state contributes more than its required 40 percent share. [15] Five programs said that the current funding level of $14,000 per graduate was sufficient: Ft. Gordon, Georgia; Camp Beauregard, Louisiana; Montana; New Mexico; and Puerto Rico. [16] Pub. L. No. 103-62 (1993). [17] GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). [18] Department of Defense, Instruction 1025.8, National Guard Challenge Program, March 20, 2002. [19] The Workforce Investment Act of 1998 was passed with the goal of consolidating, coordinating, and improving employment, training, literacy, and vocational rehabilitation programs in the United States, Pub. L. No. 105-220 (1998). Youth programs under the Workforce Investment Act seek to assist youth with employment and educational goals and the services needed to achieve those goals. [20] As of 1997 and 2000, respectively, the President's Crime Prevention Council and the White House Federal Interagency Working Group on Service no longer hold meetings. [21] Specifically, the interagency agreement was signed between the Department of Defense and the Department of Justice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention. The authority for this transfer of funds was found in the Juvenile Justice and Delinquency Prevention Act of 1974, as amended (codified at 42 U.S.C. §5665). According to the interagency agreement signed in 2001, the funds were to be used to fund up to eight new Challenge Programs; reduce negative behaviors of program youth including delinquency, violence, substance use, school suspensions and expulsions, and unplanned teen pregnancies; and to engage up to 200 youth in each of the program sites in constructive programming. [22] According to the Internal Revenue Code, a 501 (c) (3) corporation is an organization exempt from taxation, and organized and operated exclusively for religious, charitable, scientific, testing for public safety, literary, or educational purposes (26 U.S.C. §501 (c) (3)). [23] According to NGB, there was no annual report published for 1997 because program officials were not sure whether or not Congress would permanently authorize the program. GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: