This is the accessible text file for GAO report number GAO-12-228R entitled 'Race to the Top: Characteristics of Grantees' Amended Programs and Education's Review Process' which was released on January 9, 2012. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-12-228R: United States Government Accountability Office: Washington, DC 20548: December 8, 2011: The Honorable John P. Kline: Chairman: Committee on Education and the Workforce: House of Representatives: The Honorable Martha Roby: House of Representatives: Subject: Race to the Top: Characteristics of Grantees' Amended Programs and Education's Review Process: The U.S. Department of Education (Education) established the Race to the Top (RTT) grant fund to encourage states to support school reform efforts. In 2010, Education awarded 12 RTT grants totaling nearly $4 billion, making RTT the largest competitive grant fund ever administered by Education. Individual grantee awards ranged from $75 million to approximately $700 million.[Footnote 1] The size of each grantee's award was based in part on its population, among other factors, and each grantee has 4 years to use grant funds. We recently reported on grantees' plans to implement a variety of school reform efforts and how Education has provided extensive support to states. [Footnote 2] Our report also discussed implementation challenges with RTT. Specifically, we reported that grantees have had difficulty finding and hiring qualified personnel to run their grant-funded activities, have encountered difficulty complying with state procedures for awarding contracts, and that Education took more time than anticipated to review grantees' RTT implementation plans. Moreover, the 12 RTT grantees requested amendments to their approved proposals. In an effort to ensure that states are held accountable for the terms of their awards, you asked us to provide information on the following questions: (1) What amendments have grantees requested to their RTT plans, and what reasons have grantees given for their amendment requests, particularly those that significantly changed projects, project timelines, or budgets? (2) What process has Education established to review and approve grantees' requests for amendments to their RTT plans? To describe the types of and reasons for the amendments that grantees made to their RTT plans, we reviewed the 37 RTT amendment decision letters issued by Education through August 31, 2011. We compared the amended activities described in the decision letters with the activities outlined in the grantees' approved applications and budgets. We also interviewed Education officials about the reasons that grantees requested amendments as well as Education's rationale for approving the requests. To describe Education's process to review and approve RTT amendments, we examined Education's legal authority to approve amendments to its discretionary grant proposals and interviewed Education officials about the implementation of the department's process. We conducted this performance audit from October 2011 through December 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions on our audit objectives. In summary, we found that: * According to Education officials, most amendments consisted of minor adjustments to grant budgets, activities, and timelines, and some amendments involved significant changes to the grant award. Grantees have cited a variety of reasons for these amendments, such as timeline delays and difficulty finding qualified staff. * Education established a review process in which Education officials consider amendment requests on a case-by-case basis. In addition, the department distinguishes significant amendment requests from minor requests based on how the amendment would change project timelines, budgets, performance measures, and the implementation of other related projects. Education reportedly applied greater scrutiny to requests that involved significant changes to grantees' planned activities, often by requiring that grantees provide additional information or seek consultation from issue-area experts within the department. Rather than reject amendment requests, Education officials explained that they generally asked grantees to resubmit requests with more information. Background: As part of the American Recovery and Reinvestment Act's State Fiscal Stabilization Fund (SFSF), Congress required Education to make grants to states that reform their education systems.[Footnote 3] To implement this requirement, Education established the $4 billion RTT competitive grant fund to encourage states to reform their elementary and secondary education systems (K-12) and to reward states that have improved student outcomes, such as high school graduation rates. RTT provides incentives for reform across multiple areas of K-12 education, including reforms related to developing effective teachers and leaders, improving the lowest-achieving schools, expanding student data systems, and enhancing standards and assessments. In 2011, Congress provided funds for additional RTT grants. Specifically, the Department of Defense and Full-Year Continuing Appropriations Act, 2011, provided up to $700 million for the Race to the Top Fund. [Footnote 4] On May 25, 2011, Education announced that approximately $500 million of these funds would support the new Race to the Top Early Learning Challenge program and that approximately $200 million would be made available to some or all of the nine unfunded finalists from the 2010 RTT competition. Under Education's regulations, grantees are required to obtain the department's prior approval before making programmatic changes, such as revising the scope or objectives of the project.[Footnote 5] However, these regulations allow grantees to make certain changes without prior approval. For example, grantees can generally transfer up to 10 percent of funds from one grant activity to another without Education's approval.[Footnote 6]Education officials told us that almost every discretionary grant program within the department has a process by which grantees can request revisions or amendments to their grant awards. In addition to the general rules that typically govern grantees' requests for amendments, Education provided specific guidance to RTT grantees on the circumstances for and types of allowable amendment requests for the program. According to this guidance, an RTT grantee must submit an amendment request for (1) a proposed revision that constitutes a substantial change in activities from the approved grant project, regardless of budgetary impact; (2) budgetary changes, including transfers among categories or programs, that exceed $500,000 of the current approved budget; or (3) changes to the list of local educational agencies (LEAs) participating with the grantee's RTT plan. The department will not consider amendment requests that would change the overall scope and objectives of the approved proposal, fail to comply with the terms of the award or the statutory and regulatory provisions of the program, or violate the general principles of the RTT program.[Footnote 7] As part of the request, a grantee must include several key pieces of information. In particular, the request must include the grant area that would be affected by the proposed change, a description of the change, and the rationale for making the change. It must also include a description of how the proposed change aligns with RTT principles, the impact of that change on any performance measures or outcomes, budget documentation, and relevant official signatures.[Footnote 8] The department requires both Education and the grantee to mutually agree in writing to the changes. Grantees Made Both Minor and Significant Amendments to Their RTT Plans, Citing a Variety of Reasons: In general, amendments approved as of August 31, 2011, were for minor changes to project timelines, activities, and budgets, according to Education officials. For example, * The District of Columbia delayed the start date by 6 months for a project that delivers training to teachers on new academic standards, while still maintaining the original end date. The District of Columbia requested the delay because it needed additional time to develop training materials for the new standards and to contract with a vendor to deliver training on those standards. * Florida changed the type of organization that administers an RTT subgrant. Florida officials decided to contract directly with community-based organizations rather than with LEAs as originally planned. These organizations will provide volunteers from local businesses to work in the state's lowest-achieving schools and to arrange business shadowing opportunities for students, among other things. * North Carolina reallocated savings from a project aimed at improving the state's lowest-achieving schools. North Carolina had fewer schools that qualified as lowest-achieving than when it applied for RTT, so the state reduced project budgets to reflect the smaller number of schools and redirected those funds to a new pilot project. The new pilot project provides retention bonuses for staff in the state's lowest-achieving schools. In contrast, some approved amendments significantly changed project timelines or budgets. Grantees requested those changes for a variety of reasons, including timeline delays and difficulty finding qualified staff. Examples of and reasons for significant amendments are as follows: * Massachusetts delayed time frames to develop parts of the state's data systems that are used to support instruction by 1 year because of difficulties finding qualified staff and coordinating RTT funds with other federal funds. Massachusetts reported having difficulty finding staff qualified to manage its data projects at the salary levels available through the state human resources system. In previous work, Massachusetts officials told us that they did not have the staff to administer their planned data projects and would rely on hiring contract staff with RTT funds.[Footnote 9] Massachusetts officials also explained that they leveraged funding from other federal grants to build the state's longitudinal data system,[Footnote 10] and that they encountered difficulty establishing contracts for those grants, which delayed timelines for related RTT projects.[Footnote 11] Although Education approved Massachusetts' request to delay time frames, the department is requiring Massachusetts to provide additional information to program officers in order to better track the state's progress on its data systems. * New York requested to reduce an assessment project from $41.5 million to $20 million, because the state received funding from another federal grant for English and mathematics assessments. When New York submitted its RTT application, the state did not know whether its assessment projects would be funded as part of a separate federal grant competition.[Footnote 12] During the RTT application process, Education officials advised applicants not to assume funding from other federal discretionary grants that were not yet awarded. New York plans to use the $20 million remaining in this project to develop science and social studies assessments. In addition, New York plans to use the $21.5 million as follows: $18.3 million to develop curriculum resources for grades prekindergarten through 12 and $3.2 million for a pilot program to assess the quality of early learning programs in high- need communities. * Georgia submitted an amendment request to delay timelines by 1 to 2 years for at least 28 activities because of errors in its approved RTT application and a transition in state leadership. According to Education, Georgia had not completely updated project timelines that it included in its application for an earlier round of the RTT grant competition. Also, shortly after receiving the RTT grant, the state elected a new governor and a new state superintendent of education, who, according to Georgia officials, needed more time to review the RTT plan. As a result, Georgia pushed back deadlines for some of its projects, such as its longitudinal data system, teacher evaluation system, and alternative licensure paths for principals. In its decision letter, Education officials approved these amendments, stating that they were confident that Georgia would meet its grant objectives within the grant period despite these interim delays. Education Established a Review Process That Applies Greater Scrutiny to More Complex Amendment Requests: Education uses a two-tiered strategy to review and approve RTT amendment requests on a case-by-case basis. Since the grants were awarded, the department has tracked grantee activities and challenges by assigning a program officer to each grantee, communicating with grantees via monthly phone calls, reviewing their monthly progress reports, and reviewing other documentation, such as implementation plans. When appropriate, department program officials work closely with grantees to help them clarify the implications of any proposed change. Education officials told us that this ongoing communication allows the department to learn about potential changes to grantee proposals and to advise grantees on information that the department requires in order to consider and potentially approve such changes. Upon receiving a request for an amendment, Education officials categorize the request as either significant or minor based on the effect of the proposed amendment on RTT project timelines, project budgets, performance measures, and related projects[Footnote 13]. According to department officials, Education classifies an amendment request as significant if it would shift project timelines by several months or more, change activity budgets by several hundred thousand dollars or more, alter performance measures for the activity, or require changes in the implementation of related projects. Amendment requests that do not involve changes of such magnitude are classified as minor. Education officials reported providing greater scrutiny to significant requests: * If Education classifies the amendment as significant, senior managers take a more active role in reviewing the request by meeting with grantees or by consulting subject area experts within the department. For example, as part of a New York request to expand quality assessments of early learning programs in high-need communities, Education facilitated several conversations with New York officials to learn more about the proposed change. Education also directed New York officials to speak with Education experts on programs for early childhood and disadvantaged students to obtain advice on the proposed project. * If a grantee requests an amendment that Education classifies as minor, the RTT program officer will consult with other program officers when appropriate, and submit the request to senior RTT management and the Office of the General Counsel for approval. According to Education officials, the department generally does not reject amendment requests, but rather asks grantees to resubmit the requests with more detail when needed, particularly when there is reasonable doubt about the proposed amendment. For example, as a part of its new teacher evaluation system, Delaware submitted a request to change the way the state would use student growth to determine which teachers to rate "highly effective." In its amendment decision letter, Education stated that Delaware had not provided enough information about the new "highly effective" rating for the department to consider the request fully. Rather than reject Delaware's amendment request, Education asked Delaware to resubmit the request with additional information. Specifically, Education asked Delaware to convene a technical advisory group to review changes to the evaluation system and report back to Education. As of early November 2011, Delaware had not resubmitted the amendment with this information. Further, Education placed stipulations or conditions on its approval of certain amendments. For example, Rhode Island asked to delay the adoption of new educator certification regulations for approximately 3 to 7 months to allow more time for public hearings and comment on the regulations. Education officials agreed, but told the state that the department would withhold approximately $18 million in funds associated with this project if the state missed the new timeline. According to Education officials, the department tracks the changes that grantees make to their RTT plans, but does not quantify the number of individual amendments requested or approved. Most of Education's amendment decision letters include more than one change to a grantee's activities, and some letters contain many changes. However, Education officials explained that attempting to count the number of changes would not accurately reflect the grantees' work or Education's process to oversee that work. Specifically, Education officials clarified that the majority of requests were interdependent, meaning that a change in one activity would lead to a change in another activity. Because Education requires grantees to describe all activities affected by a proposed change, an amendment with a single purpose might appear as multiple changes in the amendment decision letter descriptions, and therefore trying to delineate the exact number of distinct amendments is difficult. For example, Tennessee reallocated savings from one project to two different projects because the state was able to reduce some of its contracting costs. This amendment appears as three changes in the decision letter since it affected three projects. Further, Education approved Ohio's amendment request to push back timelines for a few months for over 40 individual activities because the state received its full RTT award several months later than anticipated.[Footnote 14] Although there is a single reason for the amendment request, the decision letter includes descriptions of 40 separate changes. Agency Comments: We provided a draft of this letter to Education for review and comment. In addition to providing technical comments, Education said we should improve our description of the RTT grant program and clarify the case-by-case nature of its amendment review process. We have incorporated these comments as appropriate, and Education's comments have been reprinted in the enclosure. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to appropriate congressional committees and the Department of Education. This letter also will be available on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions concerning this report, please contact me at (202) 512-7215 or scottg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this correspondence. Key contributors to this report include Elizabeth Morrison, Rebecca Rose, and Jason Palmer. In addition, Susan Bernstein provided writing assistance, Sheila McCoy and Alexander G. Galuten provided legal support, and Nancy Cosentino verified our findings. Signed by: George A. Scott: Director, Education Workforce, and Income Security Issues: Enclosure: [End of section] Enclosure: Comments from the Department of Education: United States Department Of Education: Washington, DC 20202: November 29, 2011: Mr. George A. Scott: Director: Education, Workforce, and Income Security Issues: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Scott: I am writing in response to the Government Accountability Office (GAO) draft correspondence, "Race to the Top: Characteristics of Grantees' Amended Programs and Education's Review Process" (GAO-12-228R). We appreciate the time that your office devoted to this study and, in particular, the efforts that were made to describe the Department's process to review amendment requests by Race to the Top grantees. We also appreciate the opportunity to provide comments on this draft document. We recommend modifying a few areas in the correspondence to present a more accurate description of the implementation of the program, the process for reviewing amendments, and the referenced amendments. The following are our substantive comments on the draft: 1. Page 1 — The description of the implementation of the Race to the Top program beginning with the sentence that starts "Specifically" is incomplete, overly negative, and does not accurately reflect the more balanced position on the implementation of the Race to the Top program that is reported in GAO's earlier report (GAO 11-685). The Race to the Top program supports bold, comprehensive, and innovative reform at the state and local levels, and the Department is providing technical assistance and monitoring that also is bold, comprehensive, and innovative, with Department program officials addressing broad and challenging program issues in a direct manner. The level of interaction between federal and state officials is a fresh step forward in providing a high degree of effective monitoring and technical assistance on a very timely basis. The shorthand comment about the earlier report does not appropriately recognize these efforts and is a misleading summary of the earlier GAO report. We request that GAO modify the description to provide a more balanced summary of the earlier report and the Department's efforts in implementing the Race to the Top program. 2. The draft correspondence also does not provide an appropriate perspective of the amendment process. In general, amendments are considered on a case-by-case basis and, while there are systems and procedures that apply (as explained in our earlier comments sent on November 18), the Department's response to the amendments and its ultimate determinations are very dependent on the individual circumstances of each case. A two-month shift in one project may have no impact on project activities, while a two-month shift in another may mean the difference between launching an activity in year 2 versus year 3 of the grant. When appropriate, Department program officials work closely with the grantees to help them clarify the implications of any proposed change, and the program officers work with the Race to the Top Team leadership to determine the appropriate next steps. If there is reasonable doubt about the proposed amendment, the program team often requests that the state provide clarifying information and works to ensure that there is an appropriate level of further internal review. In the attached document, we also provide technical comments on some of the discussions in the draft document on particular state amendments to help provide a more accurate understanding of those amendments. We hope you find these comments helpful. Please let us know if you need more information or have questions about these comments or our earlier comments submitted on November 18. Thank you again for your work and the opportunities we have had to provide clarifying comments. Sincerely, Signed by: Ann Whalen: Director, Policy and Program Implementation: Implementation and Support Unit: Enclosure: [End of section] Footnotes: [1] Grantees included Delaware, the District of Columbia, Florida, Georgia, Hawaii, Maryland, Massachusetts, New York, North Carolina, Ohio, Rhode Island, and Tennessee. [2] GAO, Race to the Top: Reform Efforts Are Under Way and Information Sharing Could Be Improved, [hyperlink, http://www.gao.gov/products/GAO-11-658] (Washington, D.C.: June 30, 2011). [3] Pub. L. No. 111-5, §14006, 123 Stat. 115, 283. [4] Pub. L. No. 112-10, § 1832(a)(2), 125 Stat. 38, 163. [5] 34 C.F.R. § 80.30. [6] Although Education's regulations generally permit grantees to transfer up to 10 percent of funds from one grant activity to another without prior approval from the department, the regulations also allow Education to waive the approval requirement for those grantees transferring more than 10 percent. Education officials told us that they originally chose to set the budgetary threshold at $100,000 (as opposed to 10 percent) for RTT because of the high dollar amount of each grant and the high-profile nature of the grant program. However, in October 2011, Education relaxed that budgetary threshold to $500,000 in response to grantee concerns about administrative burden. [7] Education established several principles to determine whether a proposed change is still consistent with the overall purpose of the RTT grant program. According to these principles, amended activities must, among other things, continue to (1) support increases in student outcomes such as high school graduation rates and workforce success; (2) make progress on or meet annual performance targets; (3) support a comprehensive reform platform in all reform areas of the grant; (4) maintain LEA participation, particularly LEAs with high numbers of disadvantaged students; and (5) use and monitor funds responsibly. [8] Until Education updated its amendment submission guidance in October 2011, grantees were not specifically required to include the rationale for making the request or a description of how the amended activity would align with RTT principles in the request submission. [9] [hyperlink, http://www.gao.gov/products/GAO-11-658]. [10] Longitudinal data systems track individual student progress over time, based on student records. The systems are intended to enhance the ability of states to efficiently and accurately manage, analyze, and use education data and to make data-driven decisions to improve student learning, as well as facilitate research to increase student achievement and close achievement gaps. Massachusetts has received two Statewide Longitudinal Data Systems Grants, which are competitive grants that Education awards to states to develop data systems that track student progress over time, based on individual student records. [11] Massachusetts officials told us that the new parts of the state longitudinal data system funded with other federal grants laid the foundation for the work funded with RTT. [12] In September 2010, Education awarded Race to the Top Assessment Program grants to two consortia of states to develop advanced academic assessment systems. New York was a member of one of the winning consortia. [13] Education officials told us that about half of the grantee submitted a request that the department classified as significant. [14] Education gave each RTT grantee access to its full grant award once the department approved the grantee's implementation plan. Prior to the approval, the grantee was able to access part of the total award. According to Education, Ohio procurement law prohibits the state from beginning work on a grant project until it receives the full award. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E- mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548.