This is the accessible text file for GAO report number GAO-08-422 entitled 'Native Hawaiian Education Act: Greater Oversight Would Increase Accountability and Enable Targeting of Funds to Areas with Greatest Need' which was released on March 25, 2008. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Requesters: United States Government Accountability Office: GAO: March 2008: Native Hawaiian Education Act: Greater Oversight Would Increase Accountability and Enable Targeting of Funds to Areas with Greatest Need: Native Hawaiian Education Act: GAO-08-422: GAO Highlights: Highlights of GAO-08-422, a report to congressional requesters. Why GAO Did This Study: The Native Hawaiian Education Act (NHEA) seeks to develop innovative educational programs to assist Native Hawaiians. The Department of Education (Education) administers NHEA and has provided grants for a wide range of activities. Education is authorized to establish a Native Hawaiian Education Council and seven island councils to help implement NHEA. To inform reauthorization, GAO was asked to analyze (1) what is known about NHEA’s impact on Native Hawaiian education, (2) Education’s efforts to oversee NHEA grants, and (3) the extent to which Education and the Native Hawaiian councils have fulfilled their roles and responsibilities. To do this, GAO reviewed federal laws and regulations and departmental documents, and interviewed Education officials, council members, grantees, and experts in Native Hawaiian education. What GAO Found: Little is known about the NHEA’s impact on Native Hawaiian education. Education has not evaluated NHEA due to its relatively small funding level, about $30 million annually, and academic research that shows educational improvements for Native Hawaiians does not indicate whether NHEA was a contributor to reported gains, such as in preschool enrollment and high school and college graduation rates. Some grantees have reported positive outcomes in education and other areas for Native Hawaiians served by NHEA grants. However, the extent to which NHEA has contributed to these outcomes has not been determined. Education has made efforts in recent years to improve NHEA grant management and oversight, and plans to address other weaknesses in the future. Education has established three performance measures that grantees are expected to use to assess their activities, conducted community outreach, and improved reporting requirements for grantees. However, these performance measures are not applicable to some of the educational outcomes that could result from NHEA’s many activities. Moreover, Education has yet to establish a method to track grantee activities, such as how funds have been distributed across activities or islands, and some grantees said that in the past they have received little direction or guidance from Education. Education officials said they plan to address these weaknesses. They plan to reexamine the performance measures and provide additional guidance and technical assistance to grantees this year. Officials also stated that they would like to develop a tracking system to better manage grants funded by NHEA, but could not say specifically when this would occur. The parties responsible for administering NHEA—Education, the Education Council, and the island councils—have not fulfilled many of their respective roles and responsibilities under the act. Education has not provided sufficient oversight to ensure that the Education Council carries out many of its responsibilities under the law, or provided guidance to the Education Council on how to assist island councils represent local interests. The Education Council has not fulfilled its primary duty to collect data on the status of Native Hawaiian education, assess current needs in the different communities, make recommendations for the future use of grant funds, or, until recently, provided Education with the required annual reports on its activities. In regard to the island councils, some have not ensured the adequate representation of local interests, or supported the Education Council in efforts to assess and prioritize local needs and make recommendations to Education. In the absence of complete information on NHEA activities and local needs, Education has not targeted grant funds or reported to Congress on NHEA activities, as required. The overall lack of information about NHEA leaves participants and policymakers in a knowledge vacuum, unable to make informed decisions that will help meet and advance NHEA’s goals. What GAO Recommends: GAO recommends that, among other things, the Secretary of Education report to Congress on the activities under the act, establish additional performance measures, track grant activities, and provide more guidance and assistance to grantees and the Education Council. Education agreed with the majority of recommendations, but not fully or in part with two regarding establishing performance measures and tracking grant activities. GAO believes these recommendations remain valid. To view the full product, including the scope and methodology, click on [hyperlink, http://www.GAO-08-422]. For more information, contact Cornelia M. Ashby @ (202) 512-7215 or AshbyC@gao.gov. [End of section] Contents: Letter: Appendix I: Briefing Slides: Appendix II: Comments from the Department of Education: Appendix III: GAO Contact and Staff Acknowledgments: Abbreviations: NCLBA: No Child Left Behind Act: NHEA: Native Hawaiian Education Act: PREL: Pacific Resources for Education and Learning: United States Government Accountability Office: Washington, DC 20548: March 25, 2008: The Honorable Byron L. Dorgan: Chairman: Committee on Indian Affairs: United States Senate: The Honorable Daniel K. Akaka: United States Senate: The Honorable Daniel K. Inouye: United States Senate: The Native Hawaiian Education Act (NHEA), originally enacted in 1988 and last reauthorized in 2002 as part of the No Child Left Behind Act (NCLBA), was designed to develop innovative educational programs for Native Hawaiians, who often experience academic disparities. Since 2002, the Department of Education (Education) has provided about $30 million annually for grants under the act to fund a wide range of prenatal through postsecondary education programs for Native Hawaiians. The act also authorized Education to establish a Native Hawaiian Education Council (Education Council) to--in conjunction with seven island councils--represent local interests, coordinate existing services, identify unmet needs, and make recommendations to Education for focusing future grant funds. The act also requires Education to report to Congress on activities conducted under the act. To respond to your interest in knowing whether implementation of NHEA has been effectively managed, we answered the following questions: * What is known about the impact of NHEA on Native Hawaiian education? * What is the status of Education's efforts to manage and oversee NHEA grants? * To what extent have Education, the Education Council, and island councils fulfilled their roles and responsibilities in implementing NHEA? On November 6, 2007, we briefed requesters' staff on the interim results of our work. This report formally conveys the information provided during that briefing and additional results developed after the briefing. (See app. I.) In summary, due to the lack of research linking activities under NHEA to outcomes for Native Hawaiians, we found that little is known about the act's impact on Native Hawaiian education. With respect to the management, oversight, and implementation of NHEA activities, Education and the councils have not fulfilled many of their roles and responsibilities. Education has, in the past, provided limited management and oversight of grant activities and little guidance to the Education Council, and the Education Council and island councils have not assessed the educational needs of Native Hawaiians as required. However, Education has plans for improving its management and oversight of NHEA grants, and Education and the Education Council are now collaborating on several efforts. More specifically: * We did not find any impact evaluations, studies, or other types of research linking activities under NHEA to changes in Native Hawaiian educational outcomes. While some grantees reported positive outcomes in education and other areas for Native Hawaiians served by NHEA grants, the extent to which grant activities contributed to these outcomes has not been determined. * Education established three performance measures that grantees are expected to use to assess their programs, conducted outreach to Native Hawaiian communities, and strengthened grantee reporting requirements. However, these performance measures are not applicable to all grant- funded activities. Moreover, Education has not tracked these activities, and some grantees told us they need more guidance and assistance from the department. Education plans, however, to reexamine the performance measures and provide more guidance and technical assistance to grantees this year. Education also plans to develop a tracking system to improve grant management, although department officials could not say specifically when this would occur. * Education did not report to Congress in 2006 as required, and although the department funds the Education Council's activities, it did not provide the council with sufficient guidance and oversight. Similarly, the Education Council--in conjunction with the island councils--has not fulfilled its duties to collect data on the status of Native Hawaiian education, coordinate available resources, assess current needs in the different communities, or make recommendations for the future use of grant funds. Nor has it, until recently, provided Education with past and current required annual reports on its activities. Further, we found that there is some concern about the composition of the Education Council and that some of the island councils have not adequately represented local interests. We are making the following seven recommendations to the Secretary of Education. To improve oversight of NHEA grantees, assess program activities, and fulfill Education's reporting responsibilities, we recommend that the Secretary: * establish additional or broader performance measures to cover the range of educational outcomes that could result from activities funded by NHEA and include in its report to Congress a time frame for when such action will be completed; * expedite development of a method to track how grant funds are allocated and spent across islands and activities, and include in its report to Congress a time frame for when development will be completed; and: * provide additional direction and guidance to NHEA grantees to help them fulfill their responsibilities and meet their stated goals, including their reporting requirements. To enable the department to provide grant funds aligned with local needs and priorities for improving educational outcomes for Native Hawaiians, we recommend that the Secretary: * work with the Education Council to identify and coordinate existing services for each of the islands; determine what needs remain unmet through a transparent, evidence-based process; and ensure that the Education Council provides the department the required annual reports on its activities; * provide guidance to the Education Council on actions it can take to best help the island councils represent local interests and determine local needs and priorities; and: * help ensure appropriate representation on the Education Council by reviewing the composition of the Education and island councils and, if necessary, advising them to appoint members who can ensure that needs are adequately identified across all communities. In addition, to provide Congress with information useful for program accountability and policy direction, we recommend that the Secretary fulfill the statutory responsibility to report to Congress on the activities of the Education Council and the allocation and use of grant funds, with policy recommendations for advancing the education of Native Hawaiians. If necessary, the report could explain why certain reporting requirements could not be met and what steps will be taken to provide the missing information in the future. To develop our findings, we first reviewed relevant federal laws, regulations, and existing literature and interviewed agency officials from Education. To further our understanding, we conducted site visits on the islands of Oahu, Hawaii, Kauai, and Molokai. During our site visits, we interviewed various stakeholders, including members of the Education Council and the six island councils that are currently in operation, officials from the Hawaii Department of Education, researchers and experts in Native Hawaiian education, and community members such as parents, teachers, and students. Additionally, we met with a judgmental sample of 22 current and former grantees selected for the range of services they provided, for priorities noted in the law, and for their geographic locations. Some of these grantees had received the largest grant amounts, and some were recommended by Education and the Education Council because they were considered particularly innovative or effective. We also reviewed a judgmental sample of performance reports of 25 grant projects for many of the grantees we visited to assess the extent to which grantees were reporting on the current performance measures established by Education.[Footnote 1] Further, we reviewed grant abstracts for 93 NHEA grants awarded between 2003 and 2007 to determine what types of activities have been funded, and other documents from Education. We conducted our work between July 2007 and March 2008 in accordance with generally accepted government auditing standards. We provided a draft of this report to the U.S. Department of Education for review and comment. Education's written comments are reprinted in appendix II. In written comments on our draft report, Education concurred with five of our seven recommendations and disagreed with the remaining recommendations in whole or in part. Specifically, Education concurred with our third, fourth, and fifth recommendations that grantees and the Education Council should receive additional direction and guidance to fulfill their responsibilities under the act, and has taken several actions, such as increasing department staff assigned to the Native Hawaiian program, holding workshops for grantees, and working with the Education Council's executive director on plans for conducting a needs assessment across islands to identify local needs and priorities. Education also agreed with our sixth recommendation to help ensure appropriate representation on the Education Council, and has begun a review of the council's bylaws and membership. Education also concurred with our seventh recommendation and said it will submit a report to Congress, as required by law, saying it would do so upon receipt of requested information from the Education Council. Education agreed, in part, with our first recommendation, that the department establish additional or broader performance measures to cover the range of educational outcomes that could result from NHEA activities. Education acknowledged that the current performance measures cover only a subset of authorized activities and agreed with the need to determine whether the department should modify or adopt additional measures that cover a broader range of activities. Education stated that it was working through the department's Data Quality Initiative to help refine performance measures and data collection practices. While Education said it expected to identify broader or alternative measures for the program by the end of this calendar year, it questioned the feasibility of developing measures that would cover each allowable activity. We applaud the department's efforts to improve its performance measures; however, we continue to believe that the department needs to be able to assess the educational outcomes for all funded activities. Education disagreed with our second recommendation, to track how grant funds are allocated and spent across islands and activities, saying that it would be burdensome to the grantee, and would require data and a tracking system that other programs do not require. Education described the current grants administration and payment system that tracks grant funds by administrative categories, such as personnel and supplies. Our recommendation was not intended to have the department track the fiscal expenditures of grantees. To clarify, our recommendation was intended to have the department track how funds are allocated across islands and the 11 activity categories authorized by the law, such as postsecondary education and curriculum development. As discussed in this report, Education officials could not tell us the extent to which federal funding supported the various activities under NHEA and where grantees were providing services. Our recommendation is in response to these conditions. Given that Education is required under the law to report to Congress describing the allocation and use of funds under NHEA, we continue to believe that tracking funds across NHEA activity categories and islands is necessary for Education to fulfill its statutory responsibilities. We are sending copies of this report to relevant congressional committees, the Secretary of Education, and other interested parties. We will also make copies available to others upon request. In addition, this report will be available at no charge on GAO's Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-7215. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributions to this report are listed in appendix III. Signed by: Cornelia M. Ashby: Director, Education, Workforce, and Income Security Issues: [End of section] Appendix I: Briefing Slides: Native Hawaiian Education Act: Greater Oversight Would Increase Accountability and Enable Targeting of Funds to Areas with Greatest Need: Briefing for Staff of Chairman Byron L. Dorgan and Senators Daniel K. Inouye and Daniel Akaka Senate Committee on Indian Affairs: November 6, 2007*: * Updated March 2008: Introduction: The Native Hawaiian Education Act (NHEA), last reauthorized in 2002 as part of the No Child Left Behind Act of 2001 (NCLBA), was enacted to develop innovative educational programs to assist Native Hawaiians* by providing grant funds for Native Hawaiian educational activities and fostering community participation in strategies for improvement. Since 2002, Congress has designated about $30 million annually for NHEA. The NHEA authorized the Department of Education (Education) to: * make competitive grant awards to eligible organizations; * establish and fund a statewide Native Hawaiian Education Council (Education Council) to carry out the purposes of NHEA through the coordination of educational resources, assessment of local needs, and issuance of reports and recommendations to the department on issues related to Native Hawaiian education; and: * facilitate the establishment of seven individual island councils to carry out NHEA’s purposes and to ensure adequate representation of island and community interests within the Education Council (see map of islands on the next slide). (* As defined in the Native Hawaiian Education Act, Native Hawaiians are citizens of the United States who are descendants of the aboriginal people who occupied and exercised sovereignty in the area that now constitutes the state of Hawaii prior to 1778, as evidenced by genealogical or birth records or verification by elders or long-term 2 unity residents. According to the 2000 Census, which is based on self- reported data, Native Hawaiians alone or in combination with one or more other race, make up percent of the total U.S. population and 20 percent of the state of Hawaii’s population.) Figure: Hawaii: This figure is a map of Hawaii. [See PDF for image] Source: GAO based on Map Resources. Note: Kahoolawe is an uninhabited island and does not have an island council. [End of table] Objectives: In view of pending reauthorization of NCLBA and due to concerns about whether NHEA is fostering better educational outcomes and is being administered and managed effectively to meet the most pressing needs of Native Hawaiians, GAO was asked to answer the following questions: 1. What is known about the impact of NHEA on Native Hawaiian education? 2. What is the status of Education’s efforts to manage and oversee NHEA grants? 3. To what extent have Education, the Education Council, and island councils fulfilled their roles and responsibilities in implementing NHEA? Scope and Methodology: To answer these questions, we: reviewed relevant federal laws and regulations and existing literature on Native Hawaiian education and needs; conducted site visits on the islands of Oahu, Hawaii, Kauai, and Molokai, covering each of Hawaii’s four major counties; reviewed a judgmental sample of performance reports of 25 grant projects for many of the grantees we visited,* abstracts of 93 grants awarded between 2003 and 2007 and other documents from Education; and: interviewed officials from the U.S. Department of Education. (* Some grantees had multiple grant projects. GAO reviewed at least one performance report for each of the current grantees we visited. However, we generally did not review the performance reports of former grantees because their grant projects had ended prior to the establishment of the current performance measures in 2004.) During our site visits, we met with: * twenty-two current or former NHEA grantees that reflect a diversity of geographic locations and services, including those receiving some of the largest grant amounts in recent years, and those considered by the department and the Education Council to be innovative or effective; * officials from the Hawaii Department of Education; * the Education Council and island council members from the islands of Hawaii, Molokai, Lanai, Oahu, Kauai, and Niihau (Maui’s council disbanded in 2006); * researchers and experts in Native Hawaiian education from Kamehameha Schools, Pacific Resources for Education and Learning (PREL), and the University of Hawaii at Manoa and Hilo; and: * community members such as parents, teachers, and students. We conducted this performance audit from July 2007 to March 2008 in accordance with generally accepted government auditing standards. Summary of Findings: There are few data available for determining the impact of NHEA, although research indicates progress for Native Hawaiians in some areas, and some grantees have reported positive changes among those served by NHEA grants. Education has taken limited steps to improve grant management and oversight, but plans to address some weaknesses in the future. Administering parties have not fulfilled many of their respective roles and responsibilities to fully implement NHEA, such as ensuring that needs are identified so funds can be appropriately targeted. Background: History and Purpose of NHEA: Originally enacted in 1988 and last reauthorized in 2002 as part of NCLBA, the NHEA was designed to develop innovative educational programs for Native Hawaiians, who often experience academic disparities. As noted in the 2001 law: Native Hawaiian students score below national norms on standardized education achievement tests at all grade levels. They are also overrepresented in special education, but underrepresented in gifted and talented programs and postsecondary institutions. NHEA Funding for Educational Activities: From 2002 through 2007: Nearly $30 million to over $33 million was designated annually for grants to organizations in Hawaii to develop innovative educational programs or expand existing ones. * Education reports awarding over 100 different grants to more than 30 organizations in Hawaii. * Entities eligible for NHEA funds include Native Hawaiian educational or community-based organizations, and other public and private nonprofit organizations that have experience developing or operating Native Hawaiian programs, or a consortia of these entities. A total of $500,000 of the funds designated annually is reserved for a direct grant to the Education Council to carry out its responsibilities under the law. Grant Activities: The NHEA, as amended by NCLBA, authorizes grantees to carry out a wide range of activities:* 1. development and maintenance of a statewide Native Hawaiian early education care system; 2. operation of family-based education centers to deliver, conduct research on, and assess early childhood and preschool programs for Native Hawaiians; 3. beginning reading and literacy in Hawaiian or English for Native Hawaiian students in kindergarten through third grade and addressing combined English and Hawaiian literacy for Hawaiian students in fifth and sixth grades; 4. meeting the special needs of Native Hawaiian students with disabilities; 5. meeting the special needs of Native Hawaiian students who are gifted and talented (list continued on next slide); (* In addition, competitive preference is given to project grants that address one of the four priorities in the law: (1) beginning reading and literacy among students in grades K-3, (2) needs of at-risk youth, (3) needs of fields or disciplines in which Native Hawaiians are underemployed, and (4) use of Native Hawaiian language in instruction.) 6. academic and vocational curriculum development to address the needs of Native Hawaiian children and adults; 7. professional development for educators of Native Hawaiian students; 8. community-based learning centers that address the needs of Native Hawaiian families and communities; 9. postsecondary education for Native Hawaiians; 10. research, data collection, or evaluation on Native Hawaiian’s educational status and needs and programs under NHEA; and: 11. other activities consistent with the purposes of the program to meet the educational needs of Native Hawaiian children and adults. Roles and Responsibilities: As specified in the law, Education: * is authorized to award direct grants to eligible organizations; * is authorized to establish the Education Council and facilitate the establishment of seven island councils; * appoints members of the Education Council based on recommendations from the Native Hawaiian community;* and: * was required to submit to Congress by January 2006 a report that summarizes annual reports of the Education Council, describes the allocation and use of NHEA funds, and recommends policy changes to advance the purposes of the program. As grantor of NHEA funds to grantees and the Education Council, Education is responsible for: * monitoring compliance with grant requirements and overseeing grantee performance and: * overseeing and providing guidance to the Education Council and island councils. (* Additional conditions and terms relating to Education Council membership, including term lengths and term renewals, are required to be determined by a majority of the Education Council.) The law specifies the duties and composition of the Education Council. Duties: * coordinate educational and related services and programs available to Native Hawaiians and assess the extent to which these meet the needs of Native Hawaiians; * collect data on the status of Native Hawaiian education; * make recommendations to appropriate federal, state, and local agencies to focus and improve the use of resources related to Native Hawaiian education; * prepare and submit annual reports to Education on its activities; and: * provide administrative and financial assistance to the island councils, as determined by Education to be appropriate. Composition: * Unless otherwise determined by a majority of the Education Council, no more than 21 members, serving for 3 years. * At least 10 members shall be Native Hawaiian education service providers; 10 members shall be Native Hawaiians or Native Hawaiian education consumers; and a representative of the State of Hawaii Office of Hawaiian Affairs shall also serve as a member. The law specifies the purpose and composition of the island councils. Purpose: * Assist with advancement of NHEA’s purposes. * Ensure the adequate representation of island and community interests within the Education Council. Duties: * Meet at least four times each calendar year. Composition of island councils: * Each island council shall consist of parents, students, and other community members who have an interest in the education of Native Hawaiians, and shall be representative of individuals concerned with the educational needs of all age groups, and three-fourths of the members must be Native Hawaiians. Objective 1: Availability of Impact Information Few data exist to determine NHEA's impact, though some grantees have reported positive outcomes for those served by their projects. No study or evaluation has been conducted to determine the impact of NHEA on educational outcomes. Some research does show improvements for Native Hawaiians, but these study results cannot be linked to NHEA. Grantees’ performance reports showed some positive outcomes for those served by their projects, but the extent to which NHEA has contributed to these outcomes has not been determined. Little data exist to determine NHEA’s impact, though some grantees have reported positive outcomes for those served by their projects. No study or evaluation has been conducted to determine the impact of NHEA on educational outcomes. Education has not conducted evaluations of NHEA’s educational outcomes. * Education has not employed the Program Assessment Rating Tool—the Office of Management and Budget’s (OMB) governmentwide method for evaluating the effectiveness of federal programs—because OMB generally does not require programs under $40 million to conduct such an evaluation, according to department officials. * Education has not conducted or sponsored any other research to determine NHEA’s impact. Other organizations performing research in the area of Native Hawaiian education have not conducted studies evaluating the overall impact of NHEA. Academic research comparing Census data from 1990 and 2000 showed some improvements in school enrollment for Native Hawaiians.* * Preschool enrollment of Native Hawaiians increased and was about proportionate to their numbers in the general population of 3– and 4–year olds. * High school graduation and college enrollment rates increased slightly, though they were still below Hawaii statewide averages. The same research examining state achievement test data between 1992 and 2000 showed some improvements in reading and math. * The gap between Native Hawaiian and non-Native Hawaiian middle school students in reading proficiency decreased by 20 percent. * The gap between Native Hawaiian and non-Native Hawaiian high school students in math proficiency decreased by 24 percent. This research, however, does not indicate whether the NHEA was a contributor to these improvements for Native Hawaiians. (* See Shawn K. Kana’iaupuni, Nolan Malone, and Koren Ishibashi. “Ka huaka’i: 2005 Native Hawaiian Educational Assessment” (Honolulu, Hawaii: Kamehameha Schools, Pauahi Publications, 2005). Grantees’ performance reports showed some positive outcomes for those served by their projects. Grantees’ performance reports showed some positive outcomes for those served by their projects. Grantees’ annual and final performance reports measuring student progress showed some positive outcomes across various levels of education. improvements in vocabulary for students in early education programs; improvements in reading scores among students in elementary school and math and reading scores among those in high school; greater interest in science among students participating in a science career mentoring program; and: increases in the number of students enrolling in college through the provision of scholarships to those in financial need. Interviewed grantees described some other benefits: * building program capacity to serve more people and provide better services; * revitalizing Hawaiian language and culture; * engaging parents and the community; and: * improving students’ sense of self-identity and self-esteem. However, the extent to which NHEA has contributed to these outcomes among students receiving services has not been determined. Objective 2: Grant Management and Oversight: Education has taken limited steps to improve grant management and oversight, and plans to address remaining weaknesses in the future. Steps to improve grant management and oversight include establishing some performance measures, conducting community outreach, and strengthening grantee reporting requirements. Education has yet to establish performance measures for all activities, develop a system to track grant activities, and provide sufficient guidance and assistance to grantees. Education plans to address these weaknesses in the future. Education has established some performance measures. To meet government standards for performance and accountability, in 2004, Education established three GPRA* performance measures of program performance: * improvements in school readiness and literacy for children in early education programs; * increases in the number of students meeting or exceeding proficiency standards in math, science, or reading; and: * increases in the percentage of teachers who participate in professional development relating to Native Hawaiian education. Grantees are expected to use these measures to the extent they apply when reporting on their progress and outcomes. (* The Government Performance and Results Act of 1993 (GPRA) was enacted to improve federal program effectiveness, accountability, and service delivery and enhance congressional decision making by providing more objective information on program performance.) Education has taken other steps to improve the grant process. Since 2004, the department has: * performed community outreach to encourage a greater number of grant applications; * conducted information sessions on what is required in the grant application; and: * improved the rigor of grantee reporting requirements to include quantifiable baseline data, yearly goals, and annual progress. Education has not established performance measures relevant to some activities funded by the NHEA. Established measures do not always apply, and are not relevant to educational outcomes that could result from some grant activities: * Grantees are not required to use established measures if they do not apply to their project activities and may instead establish their own measures. * Our review of a sample of 25 performance reports covering reporting periods between 2005 and 2007 found that 11 of 25 grant projects did not report on any of the three GPRA measures. * Our review of 93 grant abstracts indicated that established performance measures may not be relevant to activities frequently funded over the past 5 years, such as higher education and curriculum development* (see next slide). Education officials noted that it has been difficult to establish a set of relevant performance measures for such a broad range of activities. As a result of the broad range of allowable activities, Education’s established measures cannot be used to measure performance of all activities. (* GPRA measures may have been relevant to other activities funded under the same grants.) Grants awarded from 2003 through 2007 fall within 11 categories of activities. Figure: This figure is a horizontal bar graph. The X axis represents number of activities, and the X axis represents the activity categories. Activity category: Development of a statewide early education and care system; Number of Activities 3. Activity category: Family-based education centers to deliver early childhood and preschool programs; Number of Activities 20. Activity category: Beginning reading and literacy in Hawaiian or English; Number of Activities 12. Activity category: Specific needs related to disabilities; Number of Activities 3. Activity category: Gifted and talented programs; Number of Activities 5. Activity category: Curriculum development; Number of Activities 34. Activity category: Professional development for educators; Number of Activities 27. Activity category: Community-based learning criteria that address the needs of families and communities; Number of Activities 13. Activity category: Postsecondary education; Number of Activities 17. Activity category: Research and data on Native Hawaiian' educational status and needs or on programs under NHEA; Number of Activities 10. Activity category: Other activities consistent with NHEA's purpose; Number of Activities 4. [See PDF for image] Source: GAO analysis of abstracts from Education of grants awarded under the NHEA program between fiscal years 2003 and 2007. Note: Many of the 93 grant abstracts we reviewed addressed multiple categories of activities. [End of program] Stakeholders cited other weaknesses in the current performance measures. While some grantees were satisfied with the current measures, other grantees, as well as some Education Council and island council members, and researchers said they were not always appropriate or adequate: lacked a component to support programs that integrate and help preserve Native Hawaiian language and culture; failed to consider outcomes such as improvements in attendance rates, grades, personal self-esteem, or community involvement; and: prematurely asked for outcomes that may not be realized until after the 3-year grant cycle. Education also lacks a system to track funded activities. Education lacks a method to track grant activities. In response to our request for information, officials: did not have a list available showing grantees that provide services on the different islands and: could not provide a summary of how funds had been spent across activity categories, such as early education or higher education. Education’s guidance and assistance to grantees have been limited. Some grantees we interviewed stated that they have received limited assistance from Education: * lack of feedback on project evaluations submitted by grantees in their performance reports and: * limited technical assistance for preparing annual reports and ensuring fiscal accountability. Grantees wanted support in several areas to improve their performance: * information on promising practices or related research to use resources more efficiently and avoid “reinventing the wheel;” * guidance and technical assistance to establish performance measures, collect and analyze data, and conduct evaluations; and: * local assistance for first-time grantees to help them meet grant requirements. Education plans to address weaknesses. This year, Education plans to: * Provide more guidance and technical assistance. According to officials, the plan is to: - review all grantees’ budgets and performance reports to identify grantees that are at risk of not meeting their financial and project goals or complying with program requirements and: - provide more guidance to grantees, such as in reporting, as well as technical assistance to those at risk. * Reexamine the three performance measures and seek input from grantees and others. However, officials did not know how or when measures would be changed if they determined this was appropriate. At some point in the future, Education would like to develop a tracking system to better manage grants funded by NHEA. Responsible parties have not fulfilled many of their respective roles and responsibilities under NHEA. Objective 3: Fulfillment of Roles and Responsibilities: Education has not provided sufficient direction and guidance to the Native Hawaiian Education Council and island councils, or submitted the required report to Congress. The Education Council has not conducted a needs assessment, provided recommendations for targeting grant funds, or, until very recently, provided annual reports to Education. Island councils may not have adequately represented local interests. Education has not provided sufficient direction and guidance to the Education Council and island councils or submitted its report to Congress. Education officials said that other priorities and lack of complete information have precluded the department from: providing sufficient direction and guidance to the Education Council on how to fulfill its responsibilities, including assisting the island councils to represent local interests; and: reporting to Congress, as required in January of 2006, with a summary of the annual reports from the Education Council, a description of the allocation and use of funds, and recommendations for policy changes to advance NHEA. The Education Council has not performed its duties to identify and report on local needs. The Education Council has not: * conducted needs assessments; * coordinated available resources; * collected data on the status of Native Hawaiian education in Hawaii; * made recommendations to Education for targeting grant funds; * provided sufficient assistance to the island councils to help them fulfill their roles and responsibilities; and: * submitted timely annual reports on its activities to Education. Lack of relevant grantee information from the department may have affected the councils ability to perform some of its duties, according to some Education Council members. Concerns were raised by grantees and council members about the appropriateness of having current grantees sit on the Education Council insofar as this body has responsibility for making recommendations for tie use of future grant funds. Island councils may not have adequately represented local interests or helped further the advancement of NHEA. Lack of direction and capacity may have affected some island councils’ ability to: ensure adequate representation of local interest within the Education Council: * One island, Maui, lacks any representation because the council disbanded in 2006, in disagreement with the Education Council over the use of funds. * Island councils may not be representative of all communities’ interests, according to some grantees. assist the Education Council in advancing purposes of NHEA: * Island councils, except Kauai, have not assessed and reported on community needs and priorities. * Until recently, most island councils were spending their monies for direct services and other activities, which the Education Council does not view as appropriate in advancing the purposes of NHEA. Parties administering NHEA are taking steps to fulfill their roles and responsibilities. Education has conducted some outreach to the Education Council and grantee communities with a few site visits, and has plans to work more closely with the Education Council to prioritize its activities. Over the last 2 years, the Education Council has started to build relationships with the department and strategic partners in the community, and has established some fiscal control over island councils’ use of NHEA funds. Most island councils are recognizing the need to identify available resources and assess local needs. A few island councils have: * hosted meetings with grantees and: * informally surveyed community members to identify community needs. Program participants also cited additional "impediments" to meeting NHEA'goals. Lack of consensus among Education, grantees, and island council members as to whether funds should be awarded primarily as seed money or for project expansion. Absence of a grant requirement for collaboration and community involvement that could promote effectiveness and sustainability. Disagreement about cultural preservation as a goal of NHEA. Lack of information sharing, such as through a Web-based clearinghouse for use by the Education Council and grantees. Perceptions of traditional hierarchies, patronage, and conflicts of interest among the council members. Concerns about the law’s low ceiling on administrative costs and its impact on the financial viability of small organizations. Conclusions: While some individual grantees have reported successes, the lack of research and evaluations has made it difficult to determine the overall impact of NHEA on Native Hawaiian education. The relatively low funding level of NHEA activities may not warrant a comprehensive impact evaluation. However, Education has not established a method to capture basic information related to grantee activities. Without sufficient and relevant performance measures in place to appropriately assess all the activities funded by NHEA, or a method to track how funds have been spent, Education is unable to effectively assess activities and outcomes, manage and oversee grants, strategically target future resources, and report to Congress on NHEA’s activities, as required. The size of the funding levels under NHEA is small relative to other Education programs, and until recently, Education has paid little attention to the management and oversight of NHEA grants. However, moving forward, it will be important for Education to fulfill its stated intention to reexamine the performance measures, implement a system to track and monitor grantee activities, and provide additional direction and guidance to grantees. Without appropriate oversight to ensure that the Education Council carries out many of its responsibilities—primarily determining the most urgent unmet needs across the state and providing the department with annual reports on its activities—Education will continue to lack the information it needs to best target funding for future grant cycles and report to Congress on the activities under NHEA as required. * In view of the wide range of activities allowed under NHEA, an evidence-based assessment of local needs and priorities is critical for leveraging the program’s limited funds. * Concerns of Education Council and island council members as well as grantees about appropriate representation suggest that more guidance from Education may be needed to ensure that appointed members can adequately represent local interests. Recommendations for Executive Action: To improve oversight of NHEA grantees, assess program activities, and fulfill Education’s reporting responsibilities, we recommend that the Secretary of Education: establish additional or broader performance measures to cover the range of educational outcomes that could result from activities funded by NHEA, and include in its report to Congress a time frame for when such action will be completed; expedite development of a method to track how grant funds are allocated and spent across islands and activities, and include in its report to Congress a time frame for when development will be completed; and: provide additional direction and guidance to NHEA grantees to help them fulfill their responsibilities and meet their stated goals, including their reporting requirements. To enable the department to provide grant funds aligned with local needs and priorities for improving educational outcomes for Native Hawaiians, we recommend that the Secretary: work with the Education Council to identify and coordinate existing services for each of the islands, determine what needs remain unmet through a transparent and evidence-based process, and ensure that the Education Council provides the department the required annual reports on its activities; provide guidance to the Education Council on actions it can take to best help the island councils represent local interests and determine local needs and priorities; and: help ensure appropriate representation on the Education Council by reviewing the composition of the Education and island councils and, if necessary, advising them to appoint members who can ensure that needs are adequately identified across all communities. To provide Congress with information useful for program accountability and policy direction, we recommend that the Secretary fulfill Education’s statutory requirement by reporting to Congress: * the activities of the Education Council, * the allocation and use of grant funds, and: * policy recommendations to advance the education of Native Hawaiians. The report could contain reasons why certain reporting requirements could not be met, and what steps will be taken to provide the missing information in the future. [End of section] Appendix II: Comments from the Department of Education: United States Department Of Education: Office Of Elementary And Secondary Education: 600 Independence Ave. S.W.: Washington. D.C. 20202-6100: Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation. The Assistant Secretary: March 13, 2008: Ms. Cornelia M. Ashby: Director: Education, Workforce and Income Security Issues: Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Ms. Ashby: I am writing in response to your request for comments on the Government Accountability Office (GAO) draft report (GAO 08-422), dated March 2008, entitled "Native Hawaiian Education Act Greater Oversight Would Increase Accountability and Enable Targeting of Funds to Areas with Greatest Need." I appreciate the opportunity to comment on the draft report. The following are responses to the specific recommendations in the report calling for the Department to take actions to increase oversight and accountability in order to target funds to areas with the greatest need: Recommendation I. Establish additional or broader performance measures to cover the range of educational outcomes that could result from activities funded by the Native Hawaiian Education Act (NHEA) and include in its report to Congress a time frame for when such action will be completed. The Department agrees in part with the recommendation that the performance measures need to be re-examined to capture the actual performance of the program. Section 7205(a)(3) of the Elementary and Secondary Education Act (ESEA) of 1965, as amended by the No Child Left Behind Act (NCLB) of 2001, authorizes a wide range of very diverse activities and services for Native Hawaiian grantees, When we first developed performance measures for this program, we faced the challenge of identifying indicators that would cover the full spectrum of funded activities. At the time, we selected indicators that we believed would best capture the success and progress of the overall program. However, the current performance measures cover only a subset of the authorized activities. We acknowledge the need to determine whether the Department should adopt additional performance measures that cover a broader range of the activities carried out under the program, or modify the current measures. However, given the wide range of allowable activities and services under the Act, it would be extremely difficult, if not impossible, and not meaningful to develop measures that would cover each allowable activity. We are currently working through the Department's Data Quality Initiative (which has been successful in helping other Department programs refine their GPRA measures and their performance data collection practices) on this effort. We are hopeful that this endeavor will result in refined, or possibly additional, measures that will allow for improved and consistent reporting on the programs and services funded by NHEA grants. We expect to identify broader or alternative measures for the program by the end of this calendar year. Recommendation 2. Expedite development of a method to track how grant funds are allocated and spent across islands and activities, and include in its report to Congress a time frame for when development will be completed. To the extent that GAO is recommending that the Department develop a method to track the fiscal expenditures of Native Hawaiian grantees by activity or services, we do not concur with this recommendation. When any applicant for a discretionary grant submits a proposal to the Department, it includes a standard budget form (Standard Form (SF) 424) that is divided into ten general line item categories: personnel, fringe benefits, travel, equipment, supplies, contractual, construction, other, indirect costs, and training stipends. Costs for some grant activities may include expenditures that fit into several categories. From the SF 424, we would have information on how grant funds will be spent in each of the ten categories to achieve the project's goals and objectives, but we would not be able to know how much a grantee has requested for each activity in its application. To request information for each activity would be very burdensome on the grantee, and would require data that other programs do not require. After we make the grant award, we monitor a grantee's general draw-down activity through our Grants Administration and Payment System (GAPS). However, the GAPS system does not give us information on how much is spent on each activity. We can examine the total amount that the grantee has drawn down, but we do not know what activity – or budget line item – the expenditure covers. If we have concerns about the size of the draw-down request, we contact the grantee for additional information. Our best tools for monitoring expenditures are our annual performance report, grant monitoring, and the single annual audit. The annual performance report requires grantees to report actual budget expenditures and to identify unexpended (carryover) funds. Again, this information is based on the original budget line items and categories of expenditures; however, because grantees are reporting actual expenditures and are linking those expenditures to grant performance, we are likely to have a better understanding of how grant funds were used to support grant activities. Additionally, when we conduct on-site monitoring we are able to review the grantee's fiscal records to ensure that the expenditures are allowable and support the activities described in the grant proposal. Further, the annual single audits provide additional oversight on the expenditures made by grantees and whether they are allowable. In sum, the Native Hawaiian program is using the tools and resources that all other discretionary grant programs use, and requiring the development of a separate system to track how grant funds are allocated and spent across islands and activities in greater detail would be extremely burdensome and not practical. Recommendation 3. Provide additional direction and guidance to NHEA grantees to help them fulfill their responsibilities and meet their stated goals, including their reporting requirements. The Department acknowledges the importance of adequate guidance and direction, and concurs with the recommendation that grantees should receive additional guidance, especially related to reporting requirements. This is a recommendation that is already being implemented. Over the past several years, we have worked closely with the program applicants and grantees to identify areas to provide needed technical assistance and support. In 2005, program staff conducted application workshops for program applicants in Hawaii. This technical assistance was provided to applicants – both first-time and veteran – in helping them apply for grant funds and design more effective programs. In 2006, program staff held a grantee meeting in Hawaii, and provided technical assistance on topics including the following: reporting requirements, performance indicators, carryover, no-cost time extensions, and other general budget management issues. Most recently, in January 2008, program staff held a project directors' meeting in Oahu with over 100 grantees in attendance. Staff members held sessions on topics including reporting, performance monitoring and evaluation, and budget management. As a result of these efforts, and through increased grant monitoring, we have identified additional areas of needed technical assistance. We have recently increased the number of Department personnel assigned to the Native Hawaiian program and are committed to continuing our technical assistance efforts, particularly in the areas of data collection, analysis, and the synthesis of data for program improvement. Recommendation 4. Work with the Education Council to identify and coordinate existing services for each of the islands, determine what needs remain unmet through a transparent and evidence-based process, and ensure that the Education Council provides the department the required annual reports on its activities. Recommendation 5. Provide guidance to the Education Council on actions it can take to best help the island councils represent local interests and determine local needs and priorities. The Department agrees with Recommendations 4 and 5, and has already begun to implement these recommendations. In the fall of 2007, we met with the executive director of the Education Council to discuss how the Council could conduct a needs assessment across the islands in order to better identify the local needs and priorities. We requested that the Council include this plan with its other annual reports that were due to the Department by December 2007. We have reviewed the plans and reports and have had a follow-up discussion with the Council regarding its submission. We recently sent the Council a written request for additional information so that we can fully analyze the plan, and so that it can be enhanced if necessary. Once we have received the additional information we will be able to determine whether the Council's plans, among other things, adequately include and address local interests and priorities. Recommendation 6. Help ensure appropriate representation on the Education Council by reviewing the composition of the Education and island councils and, if necessary, advising them to appoint members who can ensure that needs are adequately identified across all communities. The Department agrees with the importance of ensuring appropriate representation on the Council, but notes that its ability to help the Council in selecting members is restricted by the authorizing statute. The NHEA provides significant flexibility to the Council with regard to member composition, membership, and term length. Although the statute provides certain conditions and terms related to membership, most of the requirements may be overridden by a majority of the Council (See §§ 7240(b) & (c)(4)). In December 2007, the Department received a copy of the bylaws of the Council, as well of a list of the current members. We are reviewing the bylaws to ensure that the Council is complying with the rules and laws it has established for itself. Additionally, we are reviewing the list of current members and will provide guidance to the Council on the appointment of new and continuation council members. The review of the Council's submission under Recommendations 4 and 5 above will also help us ensure that there is adequate representation across the islands. Recommendation 7: To provide Congress with information useful for program accountability and policy direction, we recommend that the Secretary fulfill Education's statutory requirement by reporting to Congress the activities of the Education Council, the allocation and use of grant funds, and policy recommendations to advance the education of Native Hawaiians. The report could contain reasons why certain reporting requirements could not be met, and what steps will be taken to provide the missing information in the future. The Department agrees with this recommendation, and is already taking steps to implement this recommendation. As noted above, we have thoroughly reviewed the Council's annual plan and reports and have recently sent the Council a written request for additional information so that we can complete our review of the materials. We will complete the report to Congress as soon as we have sufficient responses to our requests for additional information from the Council. We appreciate the opportunity to share our comments and plans. If you need additional information, please contact Ms. Sylvia Lyles at 202-260- 2551 or by email at: Sylvia.Lyles@.ed.gov. Sincerely, Signed by: Kerri L. Briggs, Ph. D. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Cornelia M. Ashby, Director, at (202) 512-7215 or AshbyC@gao.gov. Staff Acknowledgments: Lacinda Ayers, Assistant Director, and Susan Chin, Analyst-in-Charge, managed this assignment. Theresa Lo made significant contributions to this report. In addition, Brian Tremblay provided assistance in data analysis; Susan Bernstein provided writing assistance; Sheila McCoy and Doreen Feldman provided legal assistance; and Jay Smale assisted in methodology. [End of section] Footnotes: [1] Some grantees had multiple grant projects. GAO reviewed at least one performance report for each of the current grantees we visited. However, we generally did not review the performance reports of former grantees because their grant projects had ended prior to the establishment of the current performance measures in 2004. GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: