This is the accessible text file for GAO report number GAO-14-855 entitled 'DOD Education Benefits: Action Is Needed to Ensure Evaluations of Postsecondary Schools Are Useful' which was released on September 8, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Committees: September 2014: DOD Education Benefits: Action Is Needed to Ensure Evaluations of Postsecondary Schools Are Useful: GAO-14-855: GAO Highlights: Highlights of GAO-14-855, a report to congressional committees. Why GAO Did This Study: DOD's military Tuition Assistance Program includes partnership agreements with about 3,000 schools through which service members can pursue a postsecondary education. Through this program, service members' tuition is paid directly to participating schools and in fiscal year 2013, the program spent $540 million. The program also provides service members with education advisors, and conducts evaluations of schools to assess quality. Congress mandated that GAO provide information on the role of these advisors and on the DOD contractor evaluations of schools participating in the program. GAO examined (1) the number of advisors and the type of advice they provide, and (2) the information collected through evaluations of schools participating in the military Tuition Assistance Program. For this work, GAO analyzed DOD data on the program from fiscal year 2011 through 2013; reviewed all DOD contractor evaluations for fiscal years 2012 and 2013; and interviewed officials from DOD and the military services, contractor staff responsible for the evaluations, and advisors at Joint Base Andrews, Maryland. GAO visited this base because many of its service members participate in the program, and some of the participating schools were evaluated by DOD's contractor in 2013. What GAO Found: In fiscal year 2013, 571 Department of Defense (DOD) education advisors were available to provide information and educational support to the nearly 280,000 service members taking courses funded through the military Tuition Assistance Program. This program accommodates service members, who may regularly be reassigned to another location (including overseas), by allowing them to take classes online, directly on base, or at nearby schools. DOD advisors offer a range of services to service members such as helping them understand the types of degrees and courses schools offer and helping them develop educational goals and plans. DOD used a contractor to conduct evaluations of schools participating in the Tuition Assistance Program, however, according to DOD, the evaluations did not provide the agency the information it needed to assess schools. This is because DOD lacked a specific plan to frame the evaluations, which according to federal standards, should clearly define the evaluation questions and methodology and address the collective knowledge, skills, and experience needed by the entity conducting the evaluations. According to DOD's contract, evaluations were to assess school quality, but the 15 areas DOD provided the contractor for evaluation were often not clearly defined and it was not clear what the contractor was to evaluate. For example, one of the areas was the “degree of congruence” among various entities involved in delivering educational services, which DOD provided the contractor without further specificity. Further, because DOD's contract did not specify all the skills needed by the contractor, DOD had to modify its contract to require such skills. However, still lacking information it needs, DOD recently decided not to renew the contract. DOD has suspended the evaluations and is exploring alternative options for evaluating schools, but does not yet have a plan to guide future efforts. Absent a plan, it will be difficult for DOD to have all of the information it needs to effectively evaluate schools. Figure: A Participant in a DOD Education Program: [Refer to PDF for image: photograph] Source: U.S. Army. [End of figure] What GAO Recommends: GAO recommends that DOD develop a plan for future school evaluations. DOD agreed with the recommendation. View [hyperlink, http://www.gao.gov/products/GAO-14-855]. For more information, contact Melissa Emrey-Arras, (617) 788-0534, emreyarrasm@gao.gov. [End of section] Contents: Letter: Background: In Fiscal Year 2013, 571 Advisors Provided Information on Programs and Educational Support to Nearly 280,000 Eligible Service Members: Evaluations of Schools Participating in the Programs Do Not Provide DOD with Information Needed for Assessment: Conclusions: Recommendation for Executive Action: Agency Comments and Our Evaluation: Appendix I: Selected Statistics Related to Active Duty Service Members and Advisors for the Military Tuition Assistance Program, Fiscal Years 2011 through 2013: Appendix II: DOD's Required Basic Qualifications of Military Service Advisors: Appendix III: Information on Selected Guidance the Services Provided About the Role of DOD Advisors: Appendix IV: Fifteen Areas Covered by the DOD Contractor Evaluations: Appendix V: Evaluation Contract Modifications to Address Skills Required: Appendix VI: Comments from the Department of Defense: Appendix VII: GAO Contact and Staff Acknowledgments: Figures: Figure 1: Number of Advisors and Service Members Participating in the Tuition Assistance Program, Fiscal Year 2013: Figure 2: Sources of Information Available to Service Members in the Tuition Assistance Program: Abbreviations: DANTES: Defense Activity for Non-Traditional Education Support: DOD: Department of Defense: Education: Department of Education: FAFSA: Free Application for Federal Student Aid: FTE: Full Time Equivalent: GPA: grade point average: MOU: memorandum of understanding: VA: Department of Veterans Affairs: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: September 8, 2014: Congressional Committees: Through the Department of Defense's (DOD) Tuition Assistance Program, active duty service members can pursue a postsecondary education and improve their skills. In fiscal year 2013, DOD spent $540 million on the program, which allows service members to take courses from more than 3,000 participating schools.[Footnote 1] Service members can attend classes online or, in some cases, at schools located on the military base or in close proximity to the base. As part of this program, education advisors within each of the military services are made available to service members to help them with decisions regarding their education. DOD uses a contractor to evaluate the quality of participating schools and submit to DOD reports of schools with recommendations to improve the educational services. Congress mandated[Footnote 2] that GAO provide information on the role and function of the services' advisors and the nature of the findings from the school evaluations. This report addresses (1) the number of advisors and the type of advice they provide, and (2) the information collected by DOD's contractor in assessing and reporting on the schools participating in the Tuition Assistance Program. For our first objective, we analyzed, for each of the four services, trend data from fiscal years 2011 through 2013 on the number of advisors available (full-time equivalents or FTEs).[Footnote 3] We collected these data directly from each service.[Footnote 4] For the same time period, we also analyzed information from DOD's Voluntary Education Management Information System on the number of service members enrolled in the Tuition Assistance Program. To assess the reliability of the data on advisors and service members, we reviewed relevant documents and interviewed program officials. We determined that they were sufficiently reliable for the purposes of this report. Further, for each service, we met with officials to discuss the advisors' role and any guidance provided to advisors and methods of delivering advice to service members (i.e., in person, and on websites). In addition, we visited Joint Base Andrews, Maryland--with 7,134 service members assigned to the base during fiscal year 2013, of which 2,223 were enrolled in classes under the Tuition Assistance Program--where we met with advisors to better understand the nature of the advice they provide service members and how education services are delivered. We selected this installation because (1) a large number of its service members were enrolled in classes, (2) five schools offered classes on base, and (3) the schools providing classes to service members on the base and the installation were recently evaluated by DOD's contractor. For our second objective, we reviewed the 23 evaluations of schools conducted for DOD under the current contract, covering fiscal years 2012 through 2013.[Footnote 5] We also reviewed the information that DOD provided to the contractor and interviewed the contractor's staff, and reviewed the evaluation contract and related contract documentation. We also compared the contract and related documentation to federal evaluation standards and principles established by the American Evaluation Association.[Footnote 6] In addition, for fiscal years 2012 through 2013, we reviewed the contractor's recommendations resulting from these evaluations. To determine how the evaluations and recommendations were used, we interviewed DOD officials and officials for the four services and reviewed relevant federal laws and regulations, as well as other relevant documents. We conducted this performance audit from April through August 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: DOD's military Tuition Assistance Program helps active duty service members--some of whom may regularly be reassigned to another location, including overseas--pursue an education.[Footnote 7] Through partnership agreements between DOD and more than 3,000 schools, during off-duty hours, service members are able to take undergraduate, graduate, vocational, licensure, certificate, and language courses. They may also complete their high school education, if necessary.[Footnote 8] The majority of service members who participate in the program are enrolled in undergraduate courses. The Undersecretary of Defense for Personnel and Readiness within DOD is responsible for implementing the Tuition Assistance Program, which includes the provision of educational counseling for service members. [Footnote 9] However, each military service is responsible for establishing and operating its own program. Through 195 education centers located on U.S. military bases, advisors are available to provide assistance and information to service members pursuing their education. These bases also make classroom space available for service members to take classes on base, although the majority of service members enroll in online classes. To participate in the program, service members must meet certain requirements. In consultation with an advisor, they must develop an education goal and education plan, and maintain a 2.0 grade point average (GPA) for undergraduate-level courses and a 3.0 GPA for graduate-level courses. Service members can receive up to $250 in tuition assistance per credit hour, with a maximum of $4,500 each year. If the cost of tuition exceeds the amount that the program provides, service members are eligible for other federal financial aid, such as federal grants and loans, to cover their expenses. Tuition is paid directly to the schools by DOD, and if a service member fails to complete a course or receives a failing grade, the student must pay back the money for those courses. Schools participating in the military Tuition Assistance Program must sign DOD's Voluntary Education Partnership memorandum of understanding (MOU),[Footnote 10] which requires, among other things, that the schools (1) be accredited by a national or regional accrediting agency recognized by the Department of Education (Education);[Footnote 11] (2) comply with state authorization requirements consistent with Education regulations;[Footnote 12] (3) be certified to participate in federal student aid programs authorized under Title IV of the Higher Education Act of 1965;[Footnote 13] (4) disclose basic information about the school's programs and costs, including tuition, fees, and other charges to service members; and (5) undergo, when requested, an evaluation of the quality of the education programs it is providing to service members.[Footnote 14] DOD contracts with an independent entity to assess the quality of postsecondary educational programs and services used by service members to assist in the improvement of these educational programs and services. In accordance with contract requirements, the contractor was to conduct evaluations of individual postsecondary schools. Also, for selected military bases that have a school on the base, the contractor was to evaluate all of the schools located on a single base and the facilities and operations of that base that support these schools and the delivery of education services to service members. Specifically, each year DOD required: four evaluations of military bases (with limited scope evaluations of the schools located on the base); two evaluations of distance learning (or online) schools; and four evaluations of schools located in close proximity to the base. Education is responsible for the administration of all federal student aid under Title IV[Footnote 15] of the Higher Education Act of 1965. Under that act, Education has oversight of the more than 7,200 postsecondary schools[Footnote 16] that participate in federal student aid programs, including those that participate in DOD's Tuition Assistance Program (but not with respect to compliance with DOD's requirements). Specifically, Education must certify a school's eligibility to participate in federal student aid by determining that the school is accredited by an accrediting agency it recognizes, is authorized to operate within a state, and meets certain administrative and financial requirements.[Footnote 17] In addition, postsecondary schools that provide federal student aid are subject to program reviews by Education, which are made available on its website. Education also maintains websites that provide publicly available information about schools that participate in federal student aid programs authorized under Title IV of the Higher Education Act of 1965, including graduation rates, default rates, and costs.[Footnote 18] In addition to Education reviews, these schools are subject to compliance and financial audits by independent auditors. Lastly, the Department of Veterans Affairs' (VA) Post-9/11 Veterans Educational Assistance Program (Post-9/11 GI Bill) is similar to DOD's program, but larger.[Footnote 19] About 755,000 eligible veterans and service members participated in the VA program in fiscal year 2013.[Footnote 20] After students enroll, VA pays schools directly for tuition and fees and sends housing and book payments to eligible students.[Footnote 21] As part of this program, VA is required to conduct annual compliance reviews to assess whether postsecondary schools that receive VA educational benefits adhere to applicable laws and regulations.[Footnote 22] Postsecondary schools that participate in Education, VA, and DOD education benefits programs include (1) public schools, which are operated and funded by state or local governments; (2) nonprofit schools, which are owned and operated by nonprofit organizations whose net earnings do not benefit any shareholder or individual; and (3) for- profit schools, which are privately owned and whose net earnings can benefit individuals or shareholders. In Fiscal Year 2013, 571 Advisors Provided Information on Programs and Educational Support to Nearly 280,000 Eligible Service Members: Since Fiscal Year 2011, the Number of Advisors Increased Slightly Overall, with the Air Force Driving Most of the Increase: In fiscal year 2013, 571 advisors were available to provide a range of information on programs and educational support to nearly 280,000 service members[Footnote 23] taking courses funded under the Tuition Assistance Program (see figure 1). Each military service determines the number of advisors it will allocate to support service members' education, based on competing priorities that balance education support with readiness needs. Across all four services, there was a net increase of 27 advisors from fiscal year 2011 through 2013. [Footnote 24] That increase, however, was driven primarily by the Air Force, which added 48 advisors to its program. According to one Air Force official, the increase was largely the result of provisions in law requiring that service members separating from the military be provided with transition assistance services (including education advice).[Footnote 25] Conversely, the Navy reduced the number of its advisors by 25. A Navy official said this decrease was due to budget cuts resulting from sequestration.[Footnote 26] This official said that the decrease in the number of advisors also reflects a shift within the Navy to reduce support for off-duty education and direct it towards training that directly affects military readiness--such as training in operating and maintaining complex radar and fire control systems for ballistic missile defense. (See appendix I for statistics on service members and advisors participating in the Tuition Assistance Program, fiscal years 2011 through 2013.) Figure 1: Number of Advisors and Service Members Participating in the Tuition Assistance Program, Fiscal Year 2013: [Refer to PDF for image: 2 pie-charts] Advisors (571 total): Air Force: 37% (211); Army: 37% (210); Marine Corps: 11% (62) Navy: 15% (88). Service members[A] (279,399 total): Air Force: 36% (100,972); Army: 38% (106,635); Marine Corps: 9% (25,867); Navy: 16% (45,925). Source: GAO analysis of Department of Defense data. GAO-14-664. Note: Total percentage of service members does not sum to 100 percent due to rounding. [A] For the Army and Air Force, the data on service members include activated National Guard and Reserve personnel. For the Navy and the Marine Corps, the data on service members include activated Reserve personnel; these two services do not have a National Guard component. [End of figure] From fiscal year 2011 through 2013, the number of service members taking courses using the Tuition Assistance Program declined by 8,819 (about 3 percent), according to DOD data. DOD officials said the decrease was due in part to a temporary suspension of the program [Footnote 27] resulting from the automatic spending reductions to federal budgets as part of the 2013 sequestration. Further, according to DOD officials, the drawdown of forces in 2013 accounted for some of this decrease. As the forces continue to draw down, the demand for tuition assistance could drop as service members leave active duty service and are no longer eligible for the program. Decreases in the number of active duty service members could potentially affect the number of advisors needed in the future. Service Members Receive Program Information and Support from DOD Advisors and from School Personnel, as well as from Online Sources: To help service members pursue educational opportunities, DOD advisors are available to provide a range of assistance and information and service members can also receive assistance from other sources (see figure 2). DOD advisors, who are required to have specific training in educational advising,[Footnote 28] are available to help service members determine their educational goals, advise them about the range and types of schools they can attend, and the types of courses and degrees these schools offer. (See appendix II for a description of DOD advisor qualifications.) In addition, advisors provide information on possible sources of funding for education, such as federal grants and loans, and discuss the program requirements. Each of the services has guidance that describes the roles and responsibilities of their respective DOD advisors. (See appendix III for information on guidance the services provided about the role of DOD advisors.) Figure 2: Sources of Information Available to Service Members in the Tuition Assistance Program: [Refer to PDF for image: illustration] Sources of Information: DOD advisor; School counselor; Online; Newsletters and brochures. Source: GAO analysis of Department of Defense data. GAO-14-664. [End of figure] At Joint Base Andrews--where 2,223 service members were enrolled in courses during fiscal year 2013--the two advisors we interviewed said that as service members arrive on base they are provided with information about Tuition Assistance Program funding, eligibility, and the schools available. These advisors reported holding information sessions, distributing brochures and pamphlets, posting information on the military base's Facebook™ page, and holding office hours for service members who want to meet in person to receive this overview. Once a service member decides to register for the Tuition Assistance Program, the advisors said they are available to answer questions about the schools, financing education, and transfer of credits if reassigned to another location.[Footnote 29] In addition, they said that they might also explain the differences in the types of programs schools offer and share information such as graduation rates and other school statistics available from Education websites. Further, the advisors we spoke with said they also discuss a range of financing options available, including other military aid, federal financial aid (grants and loans), and private student loans. With respect to private student loans, which sometimes have significantly higher interest rates than federal student loans, the advisors told us they advise service members to exhaust all federal options to cover college costs before pursuing private loans. Once a service member has completed a certain number of course hours, as required by the program, advisors we interviewed told us they work with that individual to develop, and then approve, an education plan for completing a specific course of study. These plans are to help students and advisors track the student's progress toward fulfilling course requirements that will ultimately lead to a degree, certificate, or license. All services require the development of an education plan for service members. When service members are registering for classes, advisors we interviewed said they are the busiest. They also told us that requests for their services vary by service member, with some requiring little assistance. Advisors added that other support staff at the education center on the base are available to assist service members when an advisor is not needed. For example, support staff may answer routine questions about the program; mail or e-mail the service member materials; and confirm that the service member has dropped or withdrawn from a course, or that course grades have been entered into the system. All of the services have similar staff to perform these functions. Further, as required by DOD, all participating schools have their own counselors available to help service members, for example, by recommending courses and explaining how to register for classes. At Joint Base Andrews, counselors representing each of the five schools operating on that base told us that they help students decide which courses they will need to complete a degree program and help transfer the students' credits if they are reassigned to another duty station. These counselors also told us that while they are able to answer routine questions about the Tuition Assistance Program, they refer the students to the DOD advisors for more detailed information. Lastly, we found that DOD and each of the services provide online information and tools about the Tuition Assistance Program through multiple websites. For example, one of the DOD websites[Footnote 30] provides a list of all of the schools eligible to participate in the program. In addition, all of the services report that they are increasingly using online tools to deliver information and support to service members. Also, the Navy relies heavily on its online resources to help answer service members' questions. In addition, the Army website allows service members to register for classes, and access information about other financial aid programs, such as federal grants, loans, and GI bill funding, which provides funding for veterans' education.[Footnote 31] Evaluations of Schools Participating in the Programs Do Not Provide DOD with Information Needed for Assessment: The evaluations of schools by DOD's contractor have provided a range of different information on schools participating in the Tuition Assistance Program, according to DOD officials, but they do not provide the information DOD needs to assess the schools. This is because DOD lacked a specific plan to frame and guide the evaluations, and did not require the contractor to develop one. According to federal standards,[Footnote 32] an evaluation plan should clearly define the evaluation questions and methodology and address the collective knowledge, skills, and experience needed by the entity conducting the evaluations. According to DOD's contract, evaluations were to assess school quality. However, our review of the contract found that the 15 areas for evaluation that DOD provided to the contractor were often broad, not clearly defined, and lacked specificity (see appendix IV for list of 15 areas). For example, one area was simply stated as "the methods whereby academic institutions are invited on the installation." Another simply stated "the degree of congruence among various missions (the military, installation, and the institutions), the education plan of the installation, the educational programs provided by the institutions which have a MOU with the installation and the distance learning providers." Based on our review of these 15 areas, it was not always clear what DOD was asking the contractor to evaluate and how the 15 areas would be measured. For example, in asking the contractor to assess the "degree of congruence," it was not clear what DOD meant and how the contractor would measure this area. According to the contractor, DOD did not clearly define quality and the 15 areas to be assessed were the only formal guidance provided to them. Further, evaluation questions are critical because they frame the scope of the assessment and drive the evaluation design, the selection of data to collect, and the study results. For this reason, evaluation questions must be clear and specific and use terms that can be readily defined and measured. Had DOD developed a specific plan to frame and guide the evaluations, such a plan could have better positioned DOD to fully assess the skills needed by the contractor before awarding the evaluation contract. Specifically, DOD did not initially include requirements in its contract to ensure that the contractor provided personnel with the requisite education and experience needed to conduct the evaluations. Thus, DOD had to modify its contract to obtain the needed skills (see appendix V for contract modifications). For example, in a 2013 modification, DOD required that the contractor acquire staff with a working knowledge of measurement methods and tools; sufficient experience in postsecondary education; and expertise in education theories, principles, and practices; among other skills. Although the contractor hired additional staff in time for the fiscal year 2014 evaluations, we identified problems in some fiscal year 2013 evaluation reports that were also present in the more recent 2014 evaluation reports we reviewed, where data from student surveys were misinterpreted and erroneously reported. Specifically, the contractor made broad generalizations about student satisfaction with their school based on survey responses from a non-representative sample of students whose responses may not have been the prevailing view of the other students who did not respond.[Footnote 33] In providing technical comments on a draft of this report, the contractor said that they made no attempt to draw general conclusions about the student body as a whole based on the limited survey responses. We found, however, that several of the reports did contain broad generalizations. For example, a recent evaluation report stated that there was a "high overall degree of student satisfaction" with the institution based on responses from a non-representative sample of students. Lastly, in some instances, we found that the contractor made recommendations that only tangentially related to quality. For example, a fiscal year 2012 report of the schools on one military base recommended replacing the artwork in the classrooms with pictures of students in the hallways to foster a positive climate. In addition, a fiscal year 2013 evaluation of one school included a recommendation that for clarity and consistency, the school change the names of the academic terms to read "mid-Fall," "mid-Winter," "mid-Spring," and "mid-Summer." In a fiscal year 2014 report, the contractor recommended that the school change the term "Military Assistance" to "Tuition Assistance" in the listing of funding sources within the school's online application process. In providing technical comments on a draft of this report, the contractor stated that their contract with DOD required them to evaluate all aspects of the course offerings provided by educational institutions under review. As a result, the contractor said in total it made over 300 recommendations, which were both tangential and more central to quality.[Footnote 34] For example, the contractor cited several recommendations which they viewed as more central to quality including recommendations that the close working relationship between one education office and a school be maintained; additional professional development opportunities be offered to adjunct faculty members; and an advisor position that had recently been cut be reinstated and filled with an experienced advisor. By not fully considering the skills and expertise needed by the contractor before awarding the contract, DOD risked receiving information provided from the contractor that would not meet its needs. According to the American Evaluation Association, agencies should ensure that contractors conducting evaluations possess the education, skills, and experience necessary to undertake the evaluations.[Footnote 35] After concluding that the evaluations are not providing the information needed, DOD decided not to renew its school evaluation contract.[Footnote 36] The agency is suspending the evaluations and plans to refocus its evaluations in accordance with a recently issued Executive Order.[Footnote 37] Under that Executive Order, DOD, along with Education and VA, are to ensure that schools provide meaningful information to service members about the cost and quality of programs to help them make informed choices about how to use their tuition benefits; prevent abusive and deceptive recruiting practices that target service members; and ensure that schools provide high-quality academic and student support services to service members, among other things. Further, the Order requires that DOD, Education, and VA work collaboratively to develop a comprehensive strategy for developing student outcome measures (e.g., graduation and retention rates) and strengthen their oversight of schools to protect service members participating in educational programs, such as the Tuition Assistance Program. According to DOD officials, they are meeting with Education and VA to discuss how they might coordinate efforts and leverage information from these agencies. DOD officials have been discussing a process for determining the number of school evaluations DOD will conduct each year and how they will select the schools for evaluation. DOD officials are also exploring whether these evaluations will be conducted under contract or through an interagency agreement. Although DOD has several efforts in place, it does not have a plan guiding these efforts. Fundamentally, the agency has not yet developed a plan that includes specific questions to frame the evaluations and the qualification requirements for those conducting the evaluations. Unless DOD addresses these issues, it risks receiving future evaluations that will not provide all of the information necessary to evaluate schools. Conclusions: Although the federal investment in service member education is substantial, DOD's current approach has left the department without the information it needs about the quality of schools that served about 280,000 service members in fiscal year 2013. By not having a plan in place to guide the evaluation of schools, DOD's ability to effectively assess the schools has been limited. Without a plan, it will be difficult for DOD to obtain information on the quality of the schools and determine whether any adjustments are needed in the program. Recommendation for Executive Action: To improve the usefulness of information from school evaluations, we recommend that the Secretary of Defense direct the Undersecretary of Defense for Personnel and Readiness to develop a plan for future school evaluations that includes, among other things, clearly-defined evaluation questions and an assessment of the experience, expertise, and skills needed by the personnel from the entity or entities conducting the school evaluations. Agency Comments and Our Evaluation: We provided a draft copy of this report to the Department of Defense for review and comment. We received written comments from DOD, which are reproduced in appendix VI. In its comments, DOD agreed with our recommendation that a plan was needed to guide future school evaluations. However, DOD expressed concern that we had not made recommendations about whether the number of advisors was sufficient relative to the number of service members they serve. As stated in our report, each military service determines the number of advisors it will allocate to support service members' education, based on competing priorities that balance education support with readiness needs. We made no recommendations in this area in acknowledgment of the difficulty inherent in weighing readiness needs against education needs. Further, with respect to the advisors, DOD stated that the numbers cited in our report can be misleading because its advisors perform other education functions unrelated to the Tuition Assistance Program. Our report, however, already acknowledges that advisors also support additional service members who are participating in DOD's other voluntary education programs. We also provided a draft copy of this report to the Department of Education and the DOD contractor for review and comment, and received technical comments, which we incorporated into the report, as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretaries of Defense and Education, and other interested parties. In addition, this report will also be available at no charge on GAO's website at [Hyperlink, http://www.gao.gov.] If you or your staff have any questions about this report, please contact me at (617) 788-0534 or emreyarrasm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VII. Signed by: Melissa Emrey-Arras: Director: Education, Workforce, and Income Security Issues: List of Committees: The Honorable Carl Levin: Chairman: The Honorable James M. Inhofe: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Richard J. Durbin: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Howard P. "Buck" McKeon: Chairman: The Honorable Adam Smith: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable Rodney Frelinghuysen: Chairman: The Honorable Peter J. Visclosky: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Appendix I: Selected Statistics Related to Active Duty Service Members and Advisors for the Military Tuition Assistance Program, Fiscal Years 2011 through 2013: Table 1: : Total number of active duty service members: FY 2011: 1,411,425; FY 2012: 1,368,428; FY 2013: 1,370,329. Army: FY 2011: 561,437; FY 2012: 530,164; FY 2013: 528,070. Navy: FY 2011: 320,141; FY 2012: 312,734; FY 2013: 319,838. Air Force: FY 2011: 328,821; FY 2012: 328,812; FY 2013: 326,573. Marine Corps: FY 2011: 201,026; FY 2012: 196,718; FY 2013: 195,848. Total number of active duty service members participating in DOD's Military Tuition Assistance Program[A]: FY 2011: 288,218; FY 2012: 288,722; FY 2013: 279,399. Army: FY 2011: 113,794; FY 2012: 113,590; FY 2013: 106,635. Navy: FY 2011: 47,105; FY 2012: 46,512; FY 2013: 45,925. Air Force: FY 2011: 98,597; FY 2012: 101,852; FY 2013: 100,972. Marine Corps: FY 2011: 28,722; FY 2012: 26,768; FY 2013: 25,867. Total number of advisors: FY 2011: 544; FY 2012: 557; FY 2013: 571. Army: FY 2011: 205; FY 2012: 216; FY 2013: 210. Navy: FY 2011: 113; FY 2012: 106; FY 2013: 88. Air Force: FY 2011: 163; FY 2012: 172; FY 2013: 211. Marine Corps: FY 2011: 63; FY 2012: 63; FY 2013: 62. Sources: For (1) total number of active duty service members and (2) total number of service members participating in DOD's Military Tuition Assistance Program, the source is the Department of Defense Voluntary Education Management Information System. For total number of advisors, the source is each of the military services. GAO-14-855. [A] For the Army and Air Force, the data on service members include activated National Guard and Reserve personnel. For the Navy and the Marine Corps, the data on service members include activated Reserve personnel; these two services do not have a National Guard component. [End of table] [End of section] Appendix II: DOD's Required Basic Qualifications of Military Service Advisors: Each service requires that a military service advisor has obtained a degree that included or was supplemented by at least 24 semester hours in one or a combination of the following areas: * Tests and measurement: Study of the selection, evaluation, administration, scoring, interpretation, and uses of group and individual aptitude, proficiency, interest, and other tests. * Adult education: Study of the adult as a learner, teaching-learning theories for adults, models and procedures for planning, designing, managing, and evaluating adult learning activities. * Educational program administration: Study of the foundation and methods in organizing for adult and continuing education programs. * Curriculum development or design: Study of the principles and techniques for development of curricula for adult or vocational education programs. * Teaching methods: Study of teaching strategies and learning styles of the adult learner. * Guidance and counseling: Study of the purposes and methods in counseling and guidance, the role of the counselor in various settings, approaches to counseling, and the uses of tests in the counseling situation. * Career planning: Study of career development, learning activities, systems, approaches, program coordination, use of educational and community resources, and vocational counseling systems. * Occupational information: Study of theories of occupational choice and vocational development and their application to the guidance process. Identification and utilization of various types of occupational information and resources. * College or university-sponsored practicum in counseling. * At least one course in Test and Measurement or Adult Education. Source: Department of Defense. GAO-14-855. [A] Each service requires that advisors be professionally qualified, subject matter experts, or program managers in the Education Services Series, GS-1740 or possess equivalent qualifications. [End of table] [End of section] Appendix III: Information on Selected Guidance the Services Provided About the Role of DOD Advisors: Service: Army; Guidance to advisors: * Assist soldiers in determining appropriate educational goals (for example, occupational certificates/diplomas, terminal, or transferable college degrees); * Counsel (face-to-face or electronic medium) all soldiers before Tuition Assistance is initially approved to ensure that soldiers understand their degree plan and their responsibilities regarding Tuition Assistance use; * Counsel all soldiers about the requirement to have a documented degree plan after completion of nine semester hours of college credit with one school; * Provide information on alternative funding, such as the use of in- service GI Bill benefits and federal financial aid. Approve an active duty soldier's request to use Tuition Assistance for more than 8 semester hours at one time when the soldier's prior academic history indicates likelihood for success; * Discuss comparative cost effectiveness of similar programs when assisting the soldier in choosing a degree program; * Explain the Tuition Assistance reimbursement requirement; * Advise active duty soldiers of their responsibility to accomplish all Tuition Assistance-related actions through the GoArmyEd portal; * Advise soldiers with regard to enrollment with schools not accredited by a regional accrediting body recognized by the Department of Education. Credits earned may not be accepted in transfer by regionally accredited colleges. Service: Navy; Guidance to advisors: * Assist personnel in establishing an educational goal based on the individual's academic background, aptitudes, work experience, and career objectives; * Establish education plans to enable sailors to pursue their educational goals by providing information on available education institutions, degrees, and courses; * Brief sailors on requirements for using Tuition Assistance benefits. * Recommend and/or administer appropriate or required examinations; * Assist with enrollment in schools and programs; * Provide information on financial aid programs and procedures to include assisting sailors in applying on line to the Free Application for Federal Student Aid, [hyperlink, http://www.fafsa.ed.gov]; * Conduct regularly scheduled Education Service Officers' education workshops for the purpose of training military education services personnel, career counselors, and command master chiefs. Service: Air Force; Guidance to advisors: The Education Training Section Chief ensures counseling is provided on the following programs and that counseling entries are annotated in Air Force Automated Education Management System for each counseling session, whether conducted in-person or via telephone or via e-mail. Enlisted and officer airmen will be advised on academic and career development from the time they enter active duty until the time they retire or separate. * Academic Programs. Academic counseling includes, but is not be limited to: high school completion; adult education for college preparation; technical, vocational, and professional certificates/licenses; associate degrees; bachelor's degrees; and master's degrees; * Counseling/Advisement is provided by qualified and trained staff; * The Education Training Section Chief ensures enlisted airmen are provided Community College of the Air Force degree status at initial orientation/in-processing briefings; * Military Tuition Assistance Program. Counseling must be conducted prior to authorization of Military Tuition Assistance for first time use on a specific education goal. Additional counseling will be provided to meet specific needs as they arise during an airman's progress toward an education goal; * Counsel and assist students with application for the education benefits programs available under existing Department of Veterans Affairs programs of title 38 of the U.S. Code to include, but not limited to, the following: Chapter 30, Montgomery GI Bill; Chapter 32, Post-Vietnam Era Veterans Educational Assistance Program; and Chapter 33, Post-9/11 GI Bill. Further assistance with GI Bill-related issues should be referred to the Regional Department of Veterans Affairs Office, as appropriate; * Provide counseling in both group and individual venues so students can make informed decisions on their eligibility for, and use of, GI Bill benefits. Counseling is appropriate when requested, but may also be provided prior to separation or retirement in, or apart from, Transition Assistance briefings. Emphasize potential benefits available to airmen who are scheduled to be involuntarily separated. Service: Marine Corps; Guidance to advisors: Officers and enlisted marines appointed as top echelon education officers at installations, division or wing level will provide educational guidance and counseling as follows: * Provide counseling at the first permanent duty station, at each new duty station, prior to separation, and at other suitable intervals during their military career; * Identify and counsel, individually, those enlisted marines who do not possess a high school credential and those officers who do not possess a baccalaureate degree; * Identify and screen all eligible Military Academic Skills Program personnel and provide for enrollment opportunity; * Provide assistance to marines applying for Military Academic Skills Program; * Maintain official Lifelong Learning program files, records, and data; * Prepare a Lifelong Learning program education plan for all Lifelong Learning program participants; * Publicize and promote the opportunities available through the Lifelong Learning program, using a variety of appropriate media. Source: Army, Navy, Air Force, and Marine Corps guidance. GAO-14-855. [End of table] [End of section] Appendix IV: Fifteen Areas Covered by the DOD Contractor Evaluations: Evaluation Area: 1. The character of the academic partnership of the institution with the service to include but not limited to programs offered, demographics, and facilities (for non-distance learning delivery), student services for all delivery learning programs, learning resource support, library resources, institutional support for military education needs, coordination of academic institution's satellite offices with home campus, and the working relationship between the service's education staff and the academic institution personnel. Evaluation Area: 2. The educational needs assessment by the service. Evaluation Area: 3. The methods whereby academic institutions are invited on the installation. Evaluation Area: 4. The methods by which the installation and the academic institution assess institutional effectiveness in meeting the educational needs of the installation and monitoring institutional compliance with the MOU or contract (if overseas). Evaluation Area: 5. The support provided by various levels of the military, the Undersecretary of Defense for Personnel and Readiness, service headquarters, major commands and installations for voluntary education. Evaluation Area: 6. The degree of congruence among various missions (the military, installation, and the institutions), the education plan of the installation, the educational programs provided by the institutions which have a MOU with the installation and the distance learning providers. Evaluation Area: 7. The degree of consistency of the distance learning programs for both military and civilian students and traditional in-the-classroom courses, if offered, and the consistency of satellite campus courses to home campus courses. Evaluation Area: 8. The responsiveness and flexibility toward service members with regard to programmatic, administrative, and academic processes--such as flexibility in areas of admissions, credit transfer, and academic residency requirements. Evaluation Area: 9. The means by which students are given the opportunity to evaluate the learning they receive and how the institutions respond to those evaluations. Evaluation Area: 10. The institution's disclosure in its marketing and communication regarding its mission, accreditation, courses and programs, services, transfer credit, tuition and fees, recruitment incentives/commissions in its enrollment and recruitment processes. Evaluation Area: 11. The institutional outcomes, on-campus students compared to military students, regarding retention, graduation, subject course completion, and withdrawal rates of the students. 12. Evaluation Area: The resources institutions offer to support student learning. Evaluation Area: 13. All faculty members meet the same academic qualifications and standards in accord with the institution's accrediting and oversight bodies. Evaluation Area: 14. The institution has clear, consistent policies, measures, and procedures to evaluate the performance and needs of faculty members. Evaluation Area: 15. The institution's academic and administrative student services-- standards and processes to ensure timely response to student's questions and concerns. Source: Department of Defense. GAO-14-855. [End of table] [End of section] Appendix V: Evaluation Contract Modifications to Address Skills Required: Description of DOD's Modifications to the Contract for School Evaluations: * Required that one member on the team evaluating schools have postsecondary or tuition assistance program experience; * Working knowledge of measurement methods and tools; * Expert in education theories, principles, and practices and in roles of federal and state governments sufficient to plan, evaluate, and advise DOD voluntary education agencies and other relevant stakeholders; * 2 to 3 years experience in post-secondary adult education, including one of following: voluntary education for the military, accreditation assessment or evaluation, institutional self-studies, academic counseling or administration on postsecondary level from an accredited institution and academic instruction, with a degree in adult education or instructional/curriculum design, with an emphasis on designing Distance Learning Coursework; * One assessor on each assessment team must have two to three years experience in voluntary education with the military. Assessors who have worked on previous evaluations but do not meet the qualification requirements may have requirements waived on a case-by-case basis. Source: Department of Defense. GAO-14-855. Notes: The first item was modified on October 19, 2012; the next four items were modified on September 25, 2013. The contract for assessment services was awarded August 31, 2011, at a cost of $645,438. At the time of our report, DOD had extended the contract twice, each time for a year, before it allowed the contract to expire. As of June 2014, DOD had paid a total of $2,069,329 for evaluation services provided by the contractor. [End of table] [End of section] Appendix VI: Comments from the Department of Defense: Office of The Assistant Secretary Of Defense: Readiness and Force Management: 4000 Defense Pentagon: Washington, D.C. 20301-4000: August 21, 2014: Ms. Melissa Emrey-Arras: Director, Education Workforce and Income Security Team: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Ms. Emrey-Arras: Thank you for the opportunity to provide you with the Department of Defense (DOD) response to the GAO Draft Report GAO-14-664, "DOD Education Benefits: Action is Needed to Ensure Evaluations of Postsecondary Schools Are Useful," dated July 18, 2014 (GAO Code 131292). [Now GAO-14-855] DOD concurs with the report and is providing official written comments for inclusion (enclosed). However, we are providing additional comments regarding other issues relevant to the report as noted below. The Department is concerned that there are few recommendations in the report regarding the number of education advisors in relation to the number of Service members they support. Although the report cites the number of advisors and the number of Service members participating in the Tuition Assistance (TA) program, it does not provide analysis as to whether this number is sufficient or supportive in relation to the need. Additionally, as education advisors perform other education- related functions in addition to providing TA program counseling, the number of advisors cited in the report can be misleading as it does not represent a cadre of advisors who provide TA program counseling 100% of their duty time. Other key issues that the report docs not address include: * The significant increase in program requirements (Transition Assistance Program, Military Life Cycle, Postsecondary Education Complaint System, additional requirements of DOD Instruction 1322.25, "Voluntary Education Programs," etc.) and associated workload of education advisors with little to no increase in advisor capacity. * The important distinction between the unbiased guidance regarding program and school options provided by DOD education advisors and the specific guidance provided by counselors employed by academic institutions. * The suggestion that the number of counselors could be affected by the decline in Service member TA participation resulting from drawdown of the Total Force without consideration for the two above concerns. The Department appreciates the support of and thanks the GAO for reporting on these benefits which are so critical to the welfare of Service members and their families and also acknowledges the time and effort required to compile and process this report. However, the Department would recommend that the GAO conduct additional analysis and provide recommendations as to whether the number of education advisors as compared to the number of Service members they serve is adequate or must be examined further. Sincerely, Signed by: Rosemary Feitas Williams: Deputy Assistant Secretary of Defense (Military Community and family Policy): Enclosure: As stated. GAO Draft Report Dated July 18, 2014: GAO-14-664 (GAO Code 131292): "DOD Education Benefits: Action Is Needed To Ensure Evaluations Of Postsecondary Schools Are Useful" Department Of Defense Comments To The GAO Recommendation: Recommendation: The GAO recommends that the Secretary of Defense direct the Undersecretary of Defense for Personnel and Readiness to develop a plan for future school evaluations that includes, among other things, clearly-defined evaluation questions and an assessment of the experience, expertise, and skills needed by the personnel from the entity or entities conducting the school evaluations. DOD Response: Concur. The Department is committed not only to sustaining but also to enhancing oversight of the educational programs offered to our Service members and their families. We take our responsibility to act as good stewards of tax payer dollars and to provide quality opportunities to facilitate educational goal attainment very seriously. To this end, we purposefully and intentionally engaged in efforts to review compliance initiatives across the Department including, but not limited to, third-party review, complaint resolution, and information sharing. One such effort already underway is the development of a more efficient, focused, and all-encompassing quality review process of the educational programs and services provided to our military community (estimated completion date is March 2015). The Department's plans for enhanced compliance initiatives include myriad too ls aimed at enhancing oversight, such as a signed DOD Voluntary Education Partnership Memorandum of Understanding (MOU) and compliance with its terms, centralized complaint and feedback systems, third-party education assessment/compliance reviews, program assessment and outcome measures, and information sharing among federal agencies. It is important to note that the impetus for increased oversight of education programs across the Federal government arose with President Obama signing Executive Order 13607, "Establishing Principles of Excellence for Educational Institutions Serving Service Members, Veterans, Spouses, and Other Family Members," on April 27, 2012. The Principles of Excellence as described in the Executive Order address misleading or predatory practices involving members of the military community who are pursuing higher education and prompted several initiatives to better equip students with comprehensive information to make school and program choices that meet their educational goals. Many of these initiatives are still in development today. As a part of this effort, DOD issued a revised Department of Defense Instruction (DODI) 1322.25, "Voluntary Education Programs," increasing protections to Service members and their families through a requirement for participating educational institutions to sign an enhanced MOU with the Department. The latest revision of the DODI incorporates the remaining Principles of Excellence in support of the President's Executive Order 13607 to include additional provisions surrounding unfair, deceptive, and abusive recruiting practices, as well as procedures for accessing DOD installations. Any educational institution that wishes to participate in the DOD Tuition Assistance (TA) Program must sign an MOU conveying the commitments and agreements between the educational institution and DOD prior to receiving TA funds. By signing the MOU, all educational institutions, whether traditional or on-line, for-profit or not-for- profit, agree to participate in our third-party review process, the Third Party Education Assessment. Additionally, the enhanced requirements and strengthened provisions of DODI 1322.25 necessitated changes to the Third Party Education Assessment contract that were deemed by DOD to be outside of the current contract's existing scope. During the next nine months, DOD plans to develop a more efficient, focused, and all-encompassing quality review process of the educational programs and services provided to our military community. The Department has already improved its oversight capacity in numerous areas, as represented by the launch of the centralized Postsecondary Education Complaint System (PECS) on January 30, 2014. The PECS empowers our military students and their families to report negative experiences with educational institutions, to provide the federal government with the information it needs to identify and address unfair, deceptive and misleading practices. In addition, better interagency coordination in the name of voluntary education is a reality. The Departments of Defense, Education, Veterans Affairs, Justice, the Consumer Financial Protection Bureau, and the Federal Trade Commission have been actively participating in cooperative working groups, frequently recurring meetings, and information sharing agreements that address the Principles of Excellence and common issues with federal education benefit programs. Efforts such as these will continue to supplement and better inform Departmental oversight. [End of section] Appendix VII: GAO Contact and Staff Acknowledgments: GAO Contact: Melissa Emrey-Arras, (617) 788-0534 or emreyarrasm@gao.gov. Staff Acknowledgments: In addition to the contact named above, Sherri Doughty (Assistant Director), Sandra Baxter, Kurt Burgeson, and Linda Siegel made key contributions to this report. Also contributing to this report were Susan Bernstein, Deborah Bland, Jessica Botsford, Holly Dye, Mimi Nguyen, Michael Silver, and Craig Winslow. [End of section] Footnotes: [1] The authorizing statute uses the term "educational institutions" (10 U.S.C. § 2007(a)) and various additional terms are used in other contexts, but for ease of readability throughout this report we generally use the term "schools." [2] Senate Report No.113-85 at 35 (2013). [3] We analyzed DOD data for the Air Force, Army, Navy, and Marine Corps. Because data on the Coast Guard is not collected by DOD, we did not include them in our study. We are using "advisors" to refer to three types of positions that provide some type of advisory support to service members--Education Specialists, Counselors (or Advisors), and Education Service Officers. [4] We collected these data directly from each military service because we determined that data on advisors from DOD's Voluntary Education Management Information System were not sufficiently reliable for the purposes of this report. [5] At the time of our review, only three evaluations from fiscal year 2014 had been completed, which we included in our review of evaluations. [6] See American Evaluation Association Guiding Principles for Evaluators, July 2004. The American Evaluation Association is an international professional association of evaluators whose mission is to improve evaluation practices and methods, increase evaluation use, and promote evaluation as a profession. Also, see Quality Standards for Inspection and Evaluation, Councils for the Inspectors General on Integrity and Efficiency, 2011; Centers for Disease Control and Prevention, Program Performance and Evaluation Office, Program Evaluation Steps, Aug. 3, 2011; GAO, Designing Evaluations: 2012 Revision, [hyperlink, http://www.gao.gov/products/GAO-12-208G] (Washington, D.C.: Jan. 2012); and GAO, Internal Control Standards: Internal Control Management and Evaluation Tool, [hyperlink, http://www.gao.gov/products/GAO-01-1008G] (Washington, D.C.: Aug. 2001). [7] Service members who are deployed overseas are able to participate in the Tuition Assistance Program by, for example, taking classes online. [8] Voluntary education programs are authorized by 10 U.S.C. §§ 2005 and 2007. [9] 32 C.F.R. § 68.4(c). [10] The MOU requirement was introduced through a final DOD instruction on March 15, 2011. DoDI 1322.25, Voluntary Education Programs. The instruction was then included in a final regulation published December 7, 2012 (Voluntary Education Programs, 77 Fed. Reg. 72,941), but participating schools with an MOU under the March 2011 instructions could continue to operate under it or sign a new MOU. Participating schools without an MOU under the March 2011 instructions that wished to participate in the program were ultimately given until March 1, 2013 to sign a new MOU. On May 15, 2014, DOD published another final regulation revising the MOU, and all schools wishing to participate in the program were generally given 60 days to sign a revised one. Voluntary Education Programs, 79 Fed. Reg. 27,732. [11] Institutional accreditation is a peer review evaluative process that compares a school against its accrediting agency's standards regarding such matters as school resources, admissions requirements, and student achievement. Institutions must be accredited by an accrediting agency recognized by the Secretary of Education in order to participate in financial aid programs administered by Education. [12] In order to participate in Title IV federal student aid programs, schools must be legally authorized within a state to provide postsecondary education. 20 U.S.C. § 1001. On October 29, 2010, Education published final regulations describing the statutory requirements for state authorization. Program Integrity Issues, 75 Fed. Reg. 66,832. These regulations generally provide, among other things, that schools are legally authorized in a state where they are located if the state has a process to review and appropriately act on complaints concerning the school, including enforcing applicable state laws. 34 C.F.R. § 600.9. The original effective date was July 1, 2011, but it was later delayed until July 1, 2014, for any school whose state authorization does not meet the requirements of the regulations but is taking steps to meet them. Institutional Eligibility Under the Higher Education Act of 1965, as Amended; Delay of Implementation Date, 78 Fed. Reg. 29,652. Finally, on June 24, 2014, Education further delayed implementation for those schools until July 1, 2015. Institutional Eligibility Under the Higher Education Act of 1965, as Amended; Delay of Implementation Date, 79 Fed. Reg. 35,692. [13] 20 U.S.C. §§ 1070-1099d and 42 U.S.C. §§ 2751 - 2756b. This particular requirement was added by the final rule published May 15, 2014. Voluntary Education Programs, 79 Fed. Reg. 27,732. While schools were generally given 60 days to sign revised MOUs, those with current MOUs that are not certified to participate in federal student aid programs authorized under Title IV of the Higher Education Act, but have initiated their applications with the Department of Education, were given 18 months to meet this requirement. [14] 32 C.F.R. pt. 68, appendix A. [15] Title IV of the act authorizes federal financial assistance to help students and families pay for postsecondary education through student grants and loans, such as Pell Grants for low-income students, Stafford loans, and PLUS loans. [16] This number was derived from Education's Integrated Postsecondary Education Data System, which collects information on U.S. postsecondary institutions and excludes postsecondary institutions in U.S. territories and jurisdictions. See U.S. Department of Education, Postsecondary Institutions and Cost of Attendance in 2013-14; Degrees and Other Awards Conferred, 2012-13; and 12-Month Enrollment, 2012-13 Report, July 2014, NCES 2014-066rev. [17] 20 U.S.C. §§ 1099c. [18] See U.S. Department of Education, College Navigator, accessed June 3, 2014 [hyperlink, http://nces.ed.gov/collegenavigator/]. [19] 38 U.S.C. §§ 3301-3325. [20] Veterans and service members who served on active duty for at least 90 days after September 10, 2001, were eligible for the Post-911 GI Bill Program. 38 U.S.C. § 3311(b)(8). [21] 38 U.S.C. § 3313(a), (c) and (h). [22] 38 U.S.C. § 3693. [23] These advisors also supported additional service members who were participating in DOD's other voluntary education programs, which include basic academic skills classes. They also administered tests, such as the Scholastic Achievement Test, Graduate Record Examinations, and General Educational Development. [24] The Army increased the number of its advisors by five and the Marine Corps decreased the number of its advisors by one. [25] According to an Air Force official, the increase in the number of advisors resulted from provisions within the VOW to Hire Heroes Act of 2011. Among other things, the act mandated that service members generally be provided with transition assistance services (including education advice) and that pre-separation counseling also include those services. Pub. L. No. 112-56, § 221(a) and (b), 125 Stat. 711, 715-16 (codified at 10 U.S.C. §§ 1144(c) and 1142(a)(2)). [26] Sequestration is the process of presidentially-directed, largely across-the-board spending reductions under which budget authority is reduced in the interest of advancing certain budget policy goals. [27] In March 2013, the Army, Marine Corps, and Air Force stopped approving new tuition assistance applications for their members as a cost-saving measure due to sequestration. According to DOD, the enactment of the Consolidated and Furthering Continuing Appropriations Act, 2013 (Pub. L. No. 113-6, 127 Stat. 198) funded the Tuition Assistance Program for the remainder of fiscal year 2013 and tuition assistance was reinstated in April 2013. [28] 32 C.F.R. § 68.3. [29] A service member assigned to a new location may be able to transfer credits earned to a different school in the new location. See GAO, Transfer Students: Postsecondary Institutions Could Promote More Consistent Consideration of Coursework by Not Basing Determinations on Accreditation, [hyperlink, http://www.gao.gov/products/GAO-06-22] (Washington, D.C.: Oct. 18, 2005). [30] For a list of participating schools, see DOD MOU: [hyperlink, http://www.dodmou.com/]. Other DOD websites are Defense Activity for Non-Traditional Education Support (DANTES): [hyperlink, http://www.dantes.doded.mil/index.html] and Military OneSource: [hyperlink, http: //www.militaryonesource.mil/voluntary-education]. [31] GAO, VA Education Benefits: VA Should Strengthen Its Efforts to Help Veterans Make Informed Education Choices, [hyperlink, http://www.gao.gov/products/GAO-14-324] (Washington, D.C.: May 13, 2014). [32] See Quality Standards for Inspection and Evaluation, Councils for the Inspectors General on Integrity and Efficiency, 2011; Centers for Disease Control and Prevention, Program Performance and Evaluation Office, Program Evaluation Steps, Aug. 3, 2011; GAO, Designing Evaluations: 2012 Revision, [hyperlink, http://www.gao.gov/products/GAO-12-208G] (Washington, D.C.: Jan. 2012); and GAO, Internal Control Standards: Internal Control Management and Evaluation Tool, [hyperlink, http://www.gao.gov/products/GAO-01-1008G] (Washington, D.C.: Aug. 2001). [33] Many of the surveys in the reports we reviewed had response rates of about 10-20 percent, which suggests that service members who did not respond might have answered differently than those who did. Results from such surveys cannot be generalized to the entire population of those surveyed unless the respondents were a statistically representative sample and/or the results were weighted during analyses to estimate results for the full population, neither of which was the case in any of the surveys we reviewed. [34] We reviewed 23 evaluation reports that contained a total of 281 recommendations. In providing comments on our report, the contractor included three other evaluation reports (not included in the scope of our review), which contained additional recommendations. [35] See American Evaluation Association Guiding Principles for Evaluators, July 2004. The American Evaluation Association is an international professional association of evaluators whose mission is to improve evaluation practices and methods, increase evaluation use, and promote evaluation as a profession. [36] DOD is not renewing the contract, and the current contract will expire on September 30, 2014. [37] Establishing Principles of Excellence for Educational Institutions Serving Service Members, Veterans, Spouses, and Other Family Members (Exec. Order No. 13,6074), 77 Fed. Reg. 25,861 (May 2, 2012). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]