This is the accessible text file for GAO report number GAO-10-476 entitled 'Elections: DOD Can Strengthen Evaluation of Its Absentee Voting Assistance Program' which was released on July 21, 2010. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Requesters: United States Government Accountability Office: GAO: June 2010: Elections: DOD Can Strengthen Evaluation of Its Absentee Voting Assistance Program: GAO-10-476: GAO Highlights: Highlights of GAO-10-476, a report to congressional requesters. Why GAO Did This Study: Under the Uniformed and Overseas Citizens Absentee Voting Act (UOCAVA), citizens covered are permitted to register and vote absentee. The Secretary of Defense has the primary responsibility for federal UOCAVA functions, and the Department of Defense’s (DOD) Federal Voting Assistance Program (FVAP) facilitates absentee voters’ participation in federal elections. Since 2001, the DOD Office of Inspector General and GAO have reviewed FVAP’s efforts and recommended improvements to its procedures and the direction FVAP provides to the services. In response to a congressional request, this report evaluates how FVAP (1) addresses its mission and evaluates efforts to conduct it, (2) aligns budget priorities with strategic goals, and (3) implemented DOD Office of Inspector General and GAO recommendations made from 2001 through 2009. GAO analyzed FVAP’s performance measures, relevant DOD directives, FVAP’s strategic plans, budgets, and past audit reports. Also, GAO interviewed agency officials. What GAO Found: FVAP has efforts under way to address requirements identified in DOD guidance, but FVAP’s evaluation of those efforts yields data of varying quality. FVAP’s efforts to address requirements include obtaining and distributing state-specific voting information and developing forms to request absentee voting materials. To assess the effectiveness of FVAP’s efforts, GAO used criteria (e.g., data credibility, reliability, and consistency) that it had previously identified. While FVAP improved some of its evaluative methodologies during this decade, GAO identified concerns with findings for FVAP’s postelection surveys and “measures of success.” The concerns include low response rates and not following governmentwide guidance to conduct nonresponse analyses as well as credible but limited measures to assess some efforts. Also, FVAP has not evaluated its Voting Assistance Officer training even though Voting Assistance Officers are crucial to FVAP’s voter outreach efforts. GAO noted the need for training programs to have an evaluative component. FVAP’s new Director commented on reasons for GAO findings, explaining that many organizations focus on implementation instead of impact, but he plans to add more evaluations in the future. Better evaluative information on its efforts and an assessment of its Voting Assistance Officer training would allow FVAP to enhance its efforts to be effective and efficient. While a detailed analysis was not possible because FVAP does not budget by strategic goals, FVAP’s recent budgets and current strategic goals appear to be generally aligned. GAO’s review found linkages between FVAP’s recent annual budgets that have averaged about $4 million and its strategic goals. The strategic plan for 2008 and 2009 contained four general goals, including encouraging adoption of FVAP legislative initiatives in order to facilitate UOCAVA voting and improving marketing and outreach efforts. The linkages to the goals were most readily apparent for the budget categories of travel and contracts. For example, FVAP’s travel budget was higher during years with federal elections, in large part due to travel to conduct Voting Assistance Officer training workshops and in support of FVAP’s objective to enhance training products and services. In addition, FVAP’ s contracts budget for products and services, such as improving the Web site and exploring new technology methods for Internet voting, are linked to goals involving the improvement of the FVAP’s marketing and outreach efforts with stakeholders. FVAP and the Under Secretary of Defense for Personnel and Readiness fully or partially implemented most UOCAVA-related audit recommendations from 2001 through 2009. FVAP and the Under Secretary fully or partially implemented 11 of 16 DOD Office of Inspector General and GAO recommendations for improvements in the oversight and direction it provides to the services. These included several changes to DOD guidance to specify the number of Voting Assistance Officers needed and emphasize the services’ responsibilities in monitoring and overseeing their programs. Also, FVAP fully or partially implemented six of the seven recommendations for improvements in its processes, such as improving the security of its electronic initiatives. What GAO Recommends: GAO recommends that DOD (1) assess its current evaluation methodologies, (2)implement additional methodologies, and (3) evaluate methods for training voting assistance officers. DOD concurred or partially concurred with GAO’s recommendations. View [hyperlink, http://www.gao.gov/products/GAO-10-476] or key components. For more information, contact Jack Edwards (202) 512-8246 or edwardsj@gao.gov. [End of section] Contents: Letter: Background: FVAP Has Taken Steps to Address DOD Requirements, but the Quality of the Evaluative Information Varies: FVAP's Budget Priorities Appear to Be Generally Aligned with Its Strategic Plan Goals: FVAP and USD (P&R) Fully or Partially Implemented Most of the Recommendations: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope & Methodology: Appendix II: DOD Office of Inspector General and GAO Reports Containing DOD Voting Assistance Program Recommendations: Appendix III: Comments from the Department of Defense: Appendix IV: GAO Contact and Staff Acknowledgments: Related GAO Products: Tables: Table 1: FVAP's Efforts to Address the Requirements in DOD Directive 1000.04: Table 2: FVAP's Budgets for Fiscal Years 2005 through 2009 Broken into Six Major Categories: Table 3: FVAP's and USD (P&R)'s Implementation of DOD Office of Inspector General and GAO Recommendations: 2001 through 2009: Table 4: DOD Office of Inspector General and GAO Recommendations to Improve Oversight of and Direction to the Services and FVAP's and USD (P&R)'s Actions Taken to Implement the Recommendations: 2001 through 2009: Table 5: DOD Office of Inspector General and GAO Recommendations to Improve FVAP Processes and Actions Taken by FVAP to Implement the Recommendations: 2001 through 2009: Abbreviations: DOD: Department of Defense: FVAP: Federal Voting Assistance Program: UOCAVA: Uniformed and Overseas Citizens Absentee Voting Act: USD (P&R): Under Secretary of Defense for Personnel and Readiness: VAO: Voting Assistance Officer: [End of section] United States Government Accountability Office: Washington, DC 20548: June 17, 2010: Congressional Requesters: Members of the military, their dependents of voting age, and other eligible U.S. citizens living abroad are permitted to participate by absentee ballot in all federal elections under the Uniformed and Overseas Citizens Absentee Voting Act (UOCAVA).[Footnote 1] According to the Department of Defense (DOD), the act covers over 6 million U.S. citizens. The Secretary of Defense is the presidential designee with the primary responsibility for federal UOCAVA functions; and through the Under Secretary of Defense for Personnel and Readiness (USD (P&R)), the Federal Voting Assistance Program (FVAP) provides support to potential overseas absentee voters in order to facilitate their participation in federal elections. Absentee voters may, however, experience difficulties accessing voting materials and then returning them in time to meet state-specific deadlines. According to FVAP's 2008-2009 strategic plan, the program's mission is to inform and educate U.S. citizens worldwide about their right to vote; foster voter participation; and protect the integrity of and enhance the electoral process at the federal, state, and local levels. Among other things, FVAP staff train voting assistance officers (VAOs) from the services, the Department of State, and overseas citizen organizations to help these entities implement their voting assistance programs. Since 2001, the DOD Office of Inspector General and GAO have reviewed FVAP's efforts and made recommendations to USD (P&R) to improve the oversight and direction FVAP provides to the services and to improve FVAP's processes. Following the 2008 presidential election, you requested that we assess FVAP's efforts to provide assistance to absentee voters. In response to your request, we evaluated the extent to which FVAP has (1) addressed its mission and evaluated the effectiveness of its efforts to conduct that mission, (2) put processes in place to help ensure that budget priorities are aligned with the strategic plans and goals, and (3) implemented recommendations made from 2001 through 2009 by the DOD Office of Inspector General and GAO. This report contributes to a larger GAO body of work on DOD's overseas absentee voting program (see the Related GAO Products section at the end of this report). To evaluate the extent to which FVAP has addressed its mission and evaluated the effectiveness of its efforts to conduct that mission, we analyzed FVAP's documentation of its efforts, such as communications with VAOs and state election officials, and the relationship of those efforts to FVAP's mission requirements outlined in UOCAVA and related DOD guidance. We corroborated our findings during interviews with FVAP officials. To assess the performance measures that FVAP used to evaluate the extent to which its efforts addressed its mission, we reviewed FVAP's data gathering procedures and available measures obtained from FVAP and published reports (e.g., ours and those of the DOD Office of Inspector General and non-governmental organizations). We then compared FVAP's procedures and measures against criteria on performance measurement that GAO and others had published in previous reports.[Footnote 2] For example, we reviewed the methodology and questions used in FVAP's 2008 postelection surveys of servicemembers, VAOs, overseas citizens, and local election officials and compared them against survey-related criteria in guidance from the Office of Management and Budget.[Footnote 3] To evaluate the extent to which FVAP has processes in place to align its budget priorities with its strategic plans and goals, we reviewed FVAP's budget processes and compared its budgets for fiscal years 2005 through 2009 with the goals described in its strategic plan for 2008 through 2009. To determine the extent to which the DOD Office of Inspector General's and GAO's recommendations were implemented, we reviewed DOD's Office of Inspector General and GAO's reports issued from 2001 through 2009. We also interviewed the DOD Office of Inspector General officials who had conducted the evaluations of FVAP's and USD (P&R)'s actions to implement recommendations and reviewed DOD directives, a DOD draft instruction, USD (P&R) memorandums, and DOD Office of Inspector General follow-up records of these evaluations to help determine the extent to which FVAP and USD (P&R) had implemented the DOD Office of Inspector General's recommendations. Similarly, we reviewed our reports and follow-up records to determine the extent to which GAO recommendations had been implemented. For each of our objectives, we assessed the reliability of the data we analyzed by reviewing existing documentation related to the data sources and interviewing knowledgeable agency officials about the data that we used. We found the data sufficiently reliable for the purposes of this report. Appendix I explains our scope and methodology in greater detail. We conducted this performance audit from June 2009 through May 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: FVAP relies heavily on the military services and the Department of State to carry out mission requirements to help ensure that U.S. citizens covered by UOCAVA understand their voting rights and how to register and vote by absentee ballot. Each military service is also required to designate a senior service voting representative to manage the service's voting assistance program. Similarly, the Department of State--through its Bureau of Consular Affairs, embassies, and consulates--carries out a portion of its voter assistance responsibilities through staff designated to provide assistance. To accomplish its mission, FVAP also coordinates and interacts with a number of other stakeholders including the Election Assistance Commission (Commission), state and local election officials, and non- governmental organizations. For example, the Commission was directed to establish electronic absentee-voting system guidelines for DOD and FVAP to use in establishing an electronic absentee-voting demonstration project as directed by section 1604 of the National Defense Authorization Act for Fiscal Year 2002.[Footnote 4] FVAP also works with state and local election officials to promote, among other things, the adoption of legislative initiatives to facilitate absentee voting under UOCAVA and to help ensure mutual understanding of state- specific absentee-voting procedures. Finally, FVAP coordinates with advocacy groups and other non-governmental organizations like the Overseas Vote Foundation and the Pew Charitable Trusts to leverage the outreach capacity of those organizations for reaching the diffuse population of overseas U.S. citizens. Findings from our past reports have documented opportunities for improving FVAP efforts to meet mission requirements. For example, in 2001, we reported on opportunities for DOD to improve outreach to absentee voters and improve the oversight of FVAP. In 2007, we reported on opportunities for DOD to improve electronic absentee voting initiatives--one opportunity's being for DOD to develop action plans to identify safeguards for the security and privacy of DOD's electronic and Internet-accessible voting systems. FVAP Has Taken Steps to Address DOD Requirements, but the Quality of the Evaluative Information Varies: FVAP Has Taken Steps to Address Requirements in the DOD Directive That Implements UOCAVA: FVAP has efforts under way to address requirements set out in the DOD guidance implementing the executive order assigning primary responsibility for federal UOCAVA functions to the Secretary of Defense.[Footnote 5] DOD Directive 1000.04 (DOD's guidance) specifies that USD (P&R) is responsible for managing, coordinating, and performing the responsibilities assigned to the presidential designee under UOCAVA, and for administering FVAP.[Footnote 6] The directive also requires USD (P&R) to designate a Director for FVAP who is to be responsible for all aspects of the program, and the directive contains a number of requirements that address the responsibilities assigned in UOCAVA. Table 1 describes requirements in DOD's Directive 1000.04 and identifies FVAP's efforts to address them. For example, to address the requirement to publicize the right of citizens to participate in the electoral process under UOCAVA, FVAP prints and issues voter literature (e.g., posters) and has downloadable versions of literature and other information on FVAP's Web site. Among other things, these materials provide the absentee voter with information on how to contact FVAP's ombudsman service, as well as methods and a timeline for both requesting an application for absentee voting and returning an absentee ballot. Table 1: FVAP's Efforts to Address the Requirements in DOD Directive 1000.04: DOD-specified requirements: Prescribe an official postcard form, containing both an absentee voter registration and absentee ballot registration application for use by the states, as part of the UOCAVA responsibilities referenced in the directive; FVAP's efforts to address requirements: Developed Federal Post Card Application (Standard Form-76A) that all states accept as an application for the absentee voter registration and ballot. DOD-specified requirements: Prescribe a federal write-in absentee ballot for use in general elections for federal office by absent uniformed services voters and overseas voters who make timely application for, and do not receive, states' absentee ballots as part of the UOCAVA responsibilities referenced in the directive; FVAP's efforts to address requirements: Developed Federal Write-In Absentee Ballot (Standard Form-186A) that all states accept for absentee voting in federal elections. DOD-specified requirements: Prescribe the standard oath to be used with any UOCAVA documents[A]; FVAP's efforts to address requirements: Prepared the standard oath and prints it on official absentee voting application and ballot. DOD-specified requirements: Designate a biennial Armed Forces Voters Week to encourage military personnel and their family members to exercise their right to vote; FVAP's efforts to address requirements: Identifies and communicates the dates of the Armed Forces Voters Week to the services. DOD-specified requirements: Obtain current voting information from each state, and disseminate it to other federal executive departments, agencies, and DOD components; FVAP's efforts to address requirements: Updates deadlines and other state-specific requirements in the Voting Assistance Guide, provides it to VAOs, and posts the guide on the FVAP Web site. DOD-specified requirements: Publicize the right of citizens to participate in the electoral process under UOCAVA; FVAP's efforts to address requirements: Issues voter literature (e.g., posters), posts downloadable literature and other information on the FVAP Web site, and issues press releases that publicize information on UOCAVA-covered citizens' right to vote. DOD-specified requirements: Provide an ombudsman service for all UOCAVA-covered persons and state and local officials; FVAP's efforts to address requirements: Provides a toll-free telephone ombudsman service to assist VAOs, absentee voters, and state and local officials with their questions and paperwork. DOD-specified requirements: Ensure voters receive information about registration, voting procedures, and materials pertaining to upcoming elections; FVAP's efforts to address requirements: Distributes Voting Assistance Guides, Federal Post Card Applications, and other materials to absentee voters and VAOs; and posts these materials on the FVAP Web site. DOD-specified requirements: Establish and maintain liaison with officials of state legislatures and with state and local government officials; FVAP's efforts to address requirements: Maintains contact with state and local officials, including issuing legislative initiative letters and newsletters to identify how a state can meet requirements and incorporate technology into the absentee voting process. DOD-specified requirements: Encourage and assist states and other U.S. jurisdictions to adopt the mandatory and recommended UOCAVA provisions, and advise them on how federal laws and regulations apply to their individual electoral systems; FVAP's efforts to address requirements: Provides state-specific legislative initiative letters detailing UOCAVA requirements and recommendations to accommodate technological advances in absentee voting. DOD-specified requirements: Conduct voting assistance workshops for VAOs in even-numbered years worldwide; FVAP's efforts to address requirements: Provides in-person and online training workshops to train VAOs on how to help potential absentee voters understand their voting rights. DOD-specified requirements: Survey military and other U.S. citizens covered by UOCAVA to gather information for the quadrennial report to the President and Congress required by UOCAVA; FVAP's efforts to address requirements: Conducts postelection surveys of servicemembers, VAOs, overseas citizens, and local election officials; and issues quadrennial report to the President and Congress. Source: GAO analysis of DOD data. [A] The standard oath requires voters to "affirm that a material misstatement of fact in the completion of such a document may constitute grounds for conviction for perjury." [End of table] FVAP's Evaluations of Its Efforts to Address the Requirements Yield Data of Varying Quality: The evaluative information on FVAP's efforts varies in terms of quality. In a prior report, we noted that agencies need reliable information on the performance of agency programs, the financial condition of programs and their operations, and the costs of programs and operations in order to efficiently and effectively operate, manage, and oversee activities.[Footnote 7] We also have identified the reliability, consistency, and credibility of data as some of the characteristics that might be examined in assessing the quality of information.[Footnote 8] During the last decade, we and the DOD Office of Inspector General have identified opportunities for FVAP to enhance the evaluative information that the agency has for monitoring and adjusting its efforts, and FVAP took steps to improve its evaluative information. For example, in 2001, we recommended that FVAP develop a methodology to gather nationally projectable data on disqualified military and overseas absentee ballots and reasons for disqualification. In response, FVAP redesigned its postelection survey of local election officials to obtain the number of absentee ballots not counted and the reasons for the disqualification of absentee ballots from servicemembers and other overseas citizens. Similarly, as part of its evaluation of the services' voting assistance programs for 2007, the DOD Office of Inspector General found that the services were not using consistent metrics and targets when reporting on the effectiveness and compliance of their programs. In response to this concern, FVAP established with the services some performance metrics that were published in the DOD Office of Inspector General's subsequent annual assessment of the services' voting assistance programs.[Footnote 9] FVAP refers to these new performance measures as its "measures of success." During our discussions with FVAP officials about how the agency evaluates its efforts to address the requirements in the DOD directive, the officials most frequently cited findings from FVAP's postelection surveys and measures of success. As a result of our review of documents and discussions with DOD staff knowledgeable about the methodologies, we identified concerns regarding some of the resulting evaluative information. * Postelection surveys. Following each presidential election, FVAP administers variations of this survey to six categories of individuals covered or impacted by UOCAVA: uniformed servicemembers, federal civilians overseas, non-federal civilians overseas, unit VAOs, State Department VAOs, and local election officials. Respondents complete a survey and submit the form directly to FVAP, which then compiles the results. As part of recent improvements to its survey process, FVAP collaborated with survey professionals in another part of DOD to, among other things, re-word some survey questions and adjust sampling methods in an effort to improve the statistical validity and generalizability of the survey results. Despite these improvements, concerns remain regarding the evaluative information resulting from the surveys. For example, even though the response rates for some of the surveys are low, FVAP does not conduct a non-response bias analysis that the guidance from the Office of Management and Budget has identified as a necessary step in determining whether survey findings are biased.[Footnote 10] Not conducting such an analysis limits data reliability, a data collection characteristic that GAO previously identified as affecting information quality. We also talked about credibility of data, another GAO-identified characteristic that might be examined in assessing the quality of the survey findings. We noted that potentially important content information was not included as survey questions. For example, FVAP's survey of election officials does not ask whether the state-specific legislative initiative letters that FVAP sends were useful or how FVAP may assist states in better meeting the provisions of UOCAVA. Without incorporating a wider selection of credible, valid questions into its surveys, FVAP cannot be assured that its efforts are effectively and efficiently addressing stakeholders' concerns. Together, not conducting the additional analyses to improve data reliability and leaving out some content in its surveys may result in FVAP obtaining incomplete evaluative information from its postelection surveys. * Measures of success. The measures of success include the number of VAOs trained, number of Federal Post Card Applications and Voting Assistance Guides distributed, and number of times voting messages were included on leave and earning statements, among other measures. Such measures are one type of information that FVAP uses to assess two requirements identified in the DOD directive: (1) ensure voters receive information about registration, voting procedures, and materials pertaining to upcoming elections and (2) conduct voting assistance workshops for VAOs in even-numbered years worldwide. While the measures of success are credible metrics for evaluating some of FVAP's efforts, FVAP did not conduct a reliability assessment of the data for the measures before the DOD Office of Inspector General used the information in the 2009 report. In fact, we found that some of the measures may be artificially high or low, indicating reliability concerns. For example, because FVAP uses multiple methods to deliver the Federal Post Card Application for absentee ballots, asking VAOs to report the number of copies distributed could result in (1) over- counting of servicemembers if they received multiple copies of the post card through multiple outreach efforts or (2) under-counting of servicemembers if some servicemembers only obtained a copy of the post card from FVAP's Web site. We identified additional inconsistencies in the FVAP-reported data, such as one service's total number of trained VAOs was reported as approximately 2,500 persons greater than its total number of VAOs. Also, even though the data are self-reported, FVAP did not instruct either the VAOs or the services how to collect the evaluative information in order to promote data consistency. In addition, while these measures were developed in concert with the services, they were developed according to the data that were readily available to the services, rather than using established guidance on developing performance measures to thoroughly assess FVAP's effectiveness and efficiency. Due to the lack of data verification, the inconsistencies we identified in the data submitted to the DOD Office of Inspector General, and credibility concerns, the extent to which the data accurately measure the effectiveness or efficiency of FVAP's efforts is unknown. When we discussed FVAP's evaluative information with FVAP's new Director, he noted that he intends to continue improving the methods that FVAP uses to gather evaluative information. For example, he is reaching out to obtain evaluative assistance both within DOD and from outside sources. He also indicated that he intends to add performance measures to examine the impact of FVAP's efforts and supplement the evaluative information (e.g., numbers of materials distributed and VAOs trained) that the agency currently gathers on its processes. We also found that FVAP is not systematically collecting evaluative information on all aspects of its VAO training program. While VAO outreach to potential voters is integral to several of FVAP's efforts to achieve its mission, FVAP has not formally evaluated its in-person or online training course that each VAO receives. FVAP does, however, obtain ad-hoc feedback from VAOs while conducting its in-person training workshops and through a general feedback question on the postelection survey to VAOs. In a 2004 report, we noted that training programs should have an evaluative component to help ensure the training is effective and improves performance and results. We also stated that agencies need evaluation processes that systematically track the cost and delivery of training.[Footnote 11] The cost associated with completing the online VAO training is primarily indirect (e.g., personnel time to complete the course). In contrast, in-person VAO training results in travel costs for the FVAP trainers and possibly some VAOs who must travel to another location to be trained. The locations of the 2-hour in-person VAO training further illustrate the higher cost for the in-person training. In 2008, FVAP offered in-person VAO training workshops in a variety of locations: 20 in the area around Washington, D.C., 28 in other U.S. locations, and 62 in foreign locations. Without a formal evaluation of the training, it is not possible to know (1) how well either training method conveys knowledge to VAOs, (2) steps needed to enhance the training, (3) whether in-person training provides added benefits to outweigh its additional costs, and (4) whether new technology--such as Webinars-- could result in more effective, efficient, or timely delivery of the training. Since the VAO training is given only in even numbered years, delaying a formal evaluation of the two methods of training until 2012 would mean that there would be no empirical basis for adjusting the training until 2014. In discussing potential reasons why some of the concerns about the evaluative information remain, FVAP's new Director said that these types of concerns exist in many organizations where the past emphasis focuses only on whether something was implemented, rather than on impact. However, in later discussions, FVAP's new Director mentioned that the agency will perform more reviews of evaluative information in the future. Continued use of the evaluative information without adding steps such as prescribing procedures for the self-reporting of information and conducting non-response analyses could limit (1) FVAP's ability to enhance the effectiveness and efficiency of its overall program and (2) DOD's ability to provide a more complete picture of FVAP's performance, which could impact the quality of future reports to Congress submitted in response to section 586 of the National Defense Authorization Act for Fiscal Year 2010.[Footnote 12] FVAP's Budget Priorities Appear to Be Generally Aligned with Its Strategic Plan Goals: While we could not perform detailed analyses because FVAP does not budget by strategic goals, FVAP's budgets for fiscal years 2005 through 2009 and the goals in its strategic plans for the same period appear to be generally aligned. FVAP uses major budget categories and budget directions from the Defense Human Resource Agency. FVAP's budget averaged approximately $4 million for the 5 years. Table 2 shows the amounts for each fiscal year and the average across the 5 years for each of six major budget categories. Table 2: FVAP's Budgets for Fiscal Years 2005 through 2009 Broken into Six Major Categories: Budget category: Personnel; Average[A]: $1,822,000; Fiscal year: 2005: $1,672,000; Fiscal year: 2006: $1,710,000; Fiscal year: 2007: $1,800,000; Fiscal year: 2008: $1,727,000; Fiscal year: 2009: $2,200,000. Budget category: Contracts; Average[A]: $1,665,000; Fiscal year: 2005: $1,787,000; Fiscal year: 2006: $1,497,000; Fiscal year: 2007: $1,642,000; Fiscal year: 2008: $2,306,000; Fiscal year: 2009: $1,091,000. Budget category: Operations; Average[A]: $234,000; Fiscal year: 2005: $285,000; Fiscal year: 2006: $231,000; Fiscal year: 2007: $176,000; Fiscal year: 2008: $158,000; Fiscal year: 2009: $322,000. Budget category: Travel; Average[A]: $145,000; Fiscal year: 2005: $35,000; Fiscal year: 2006: $220,000; Fiscal year: 2007: $60,000; Fiscal year: 2008: $350,000; Fiscal year: 2009: $60,000. Budget category: Information technology; Average[A]: $103,000; Fiscal year: 2005: $55,000; Fiscal year: 2006: $125,000; Fiscal year: 2007: $78,000; Fiscal year: 2008: $105,000; Fiscal year: 2009: $150,000. Budget category: Training; Average[A]: $33,000; Fiscal year: 2005: $25,000; Fiscal year: 2006: $14,000; Fiscal year: 2007: $15,000; Fiscal year: 2008: $50,000; Fiscal year: 2009: $60,000. Budget category: Total; Average[A]: $4,001,000; Fiscal year: 2005: $3,859,000; Fiscal year: 2006: $3,797,000; Fiscal year: 2007: $3,771,000; Fiscal year: 2008: $4,696,000; Fiscal year: 2009: $3,883,000. Source: GAO analysis of FVAP data. Note: Totals may not include congressional adjustments. [A] The averages in this column are rounded to the nearest thousand dollars. [End of table] According to FVAP's budget requests to Congress for fiscal year 2009, the program's overall budget generally supports the achievement of its mission and strategic goals. The strategic plan for 2008 and 2009 contained the following four general goals: (1) encourage adoption of FVAP legislative initiatives in order to facilitate UOCAVA voting, (2) maximize use of existing and emerging technologies, (3) improve marketing and outreach efforts with customers and stakeholders, and (4) enhance resourcing of FVAP programs. FVAP's travel budget varies in election years versus non-election years, and the increased budgets in election years are related to travel associated with in-person training. In the non-election years of 2005, 2007, and 2009 when FVAP staff members conducted very few VAO workshops, the amounts budgeted for travel in those fiscal years were $35,000, $60,000, and $60,000, respectively. FVAP officials told us that the travel budget during an odd-numbered fiscal year consists primarily of staff members' trips to state conferences to meet with state and local election officials. Such efforts generally correspond to FVAP's goal 1 objective of engaging the states and territories in an effort to pass recommended legislative initiatives. In contrast, the travel budgets for fiscal years 2006 and 2008 were much higher-- $220,000 and $350,000, respectively. The higher amounts reflect FVAP's traveling for the previously mentioned meetings with election officials and to locations in the U.S. and foreign countries in order to conduct VAO training workshops during the fiscal years that directly precede federal elections. This latter purpose corresponds to the goal 2 objective to enhance training products and services. Our analysis of other documents found that FVAP's budget for contracts also appears to contribute to the agency's efforts to achieve its strategic goals and objectives. Funding in this category is particularly important in supporting goal 2 and goal 3 objectives. We will use the fiscal year 2008 contracts budget to illustrate some linkages between FVAP's budget and its strategic goals. For example, * Some of FVAP's contract funds were linked to goal 2 objectives such as improving the Web site and exploring new technology methods for Internet voting. For example, in 2008 FVAP used funds from their contracts budget for the Electronic Transmission Service,[Footnote 13] absentee ballot tracking equipment for the Military Postal Service Agency, an electronic voter registration and ballot delivery system, and redesign of the FVAP Web site. These services and technologies are used to reach UOCAVA voters and state and local election officials, FVAP's target populations. FVAP officials told us that some expenses, such as funds spent on the Electronic Transmission Service and publication printing (e.g., the Voting Assistance Guide, motivational posters, and election dates posters), reoccurred between fiscal years 2005 through 2009. * Funds spent from the contracts budget related to the postelection surveys generally link to goal 2 and 3 and involve the use of existing and emerging technologies and the improvement of the program's marketing and outreach efforts with stakeholders. According to FVAP officials, these initiatives aid in the design and execution of FVAP's postelection surveys. Resources for postelection survey planning are required in the odd-numbered fiscal year prior to a presidential election and additional resources for planning and executing the survey and analyzing the results continue into at least the next 2 fiscal years. Although it was not possible for us to determine how much of the personnel budget could be linked directly to a specific strategic goal, our review found that accomplishment of each of FVAP's strategic objectives required actions by the agency's staff. Among other things, the first goal of encouraging adoption of FVAP legislative initiatives is accomplished by FVAP staff developing the initiatives that will facilitate UOCAVA-related voting and coordinating with state and local election officials to accept the initiatives. Moreover, the goal 4 objective that is related to employing improved budgeting, staffing, and training is accomplished by FVAP staff. FVAP officials told us that the remaining three budget categories-- operations, information technology, and training for FVAP employees-- support FVAP's daily operations. The operations budget includes the cost of utilities for the program's office space, supplies, and forms purchases (e.g., Federal Write-in Absentee Ballot and Federal Post Card Application). Information technology includes basic Web site expenses and computer equipment purchases. Our analysis and discussion with FVAP officials revealed that these budget categories provide resources for FVAP staff to perform their work which helps them to meet FVAP's strategic goals. FVAP and USD (P&R) Fully or Partially Implemented Most of the Recommendations: FVAP and USD (P&R) fully or partially implemented 17 of 23 DOD Office of Inspector General and GAO recommendations made from 2001 through 2009.[Footnote 14] The 23 recommendations involve improvements in the (1) oversight and direction FVAP and USD (P&R) provide to the services' respective voting assistance programs through formal policy and implementation guidance and (2) processes for carrying out FVAP's mission. FVAP and USD (P&R) fully implemented 8 of 16 DOD Office of Inspector General and GAO recommendations for improving the oversight and direction that FVAP provides to the services and partially implemented three of the other eight recommendations. Additionally, FVAP fully implemented three GAO recommendations for improvements in the processes it uses to carry out its mission and partially implemented three of the four remaining recommendations. Table 3 summarizes FVAP's and USD (P&R)'s implementation of DOD Office of Inspector General's and GAO's recommendations. Table 3: FVAP's and USD (P&R)'s Implementation of DOD Office of Inspector General and GAO Recommendations: 2001 through 2009: Source of recommendation and recommendation status: DOD Office of Inspector General recommendations: Implemented; Area of improvement: Oversight and direction to the services: 5; Area of improvement: FVAP processes: 0; Area of improvement: Total: 5. Source of recommendation and recommendation status: DOD Office of Inspector General recommendations: Partially implemented; Area of improvement: Oversight and direction to the services: 2; Area of improvement: FVAP processes: 1; Area of improvement: Total: 3. Source of recommendation and recommendation status: DOD Office of Inspector General recommendations: Not implemented; Area of improvement: Oversight and direction to the services: 5; Area of improvement: FVAP processes: 1; Area of improvement: Total: 6. Source of recommendation and recommendation status: Subtotal: DOD Inspector General; Area of improvement: Oversight and direction to the services: 12; Area of improvement: FVAP processes: 2; Area of improvement: Total: 14. Source of recommendation and recommendation status: GAO recommendations: Implemented; Area of improvement: Oversight and direction to the services: 3; Area of improvement: FVAP processes: 3; Area of improvement: Total: 6. Source of recommendation and recommendation status: GAO recommendations: Partially implemented; Area of improvement: Oversight and direction to the services: 1; Area of improvement: FVAP processes: 2; Area of improvement: Total: 3. Source of recommendation and recommendation status: GAO recommendations: Not implemented; Area of improvement: Oversight and direction to the services: 0; Area of improvement: FVAP processes: 0; Area of improvement: Total: 0. Source of recommendation and recommendation status: Subtotal: GAO; Area of improvement: Oversight and direction to the services: 4; Area of improvement: FVAP processes: 5; Area of improvement: Total: 9. Source of recommendation and recommendation status: Combined DOD Inspector General and GAO recommendations: Implemented; Area of improvement: Oversight and direction to the services: 8; Area of improvement: FVAP processes: 3; Area of improvement: Total: 11. Source of recommendation and recommendation status: Combined DOD Inspector General and GAO recommendations: Partially implemented; Area of improvement: Oversight and direction to the services: 3; Area of improvement: FVAP processes: 3; Area of improvement: Total: 6. Source of recommendation and recommendation status: Combined DOD Inspector General and GAO recommendations: Not implemented; Area of improvement: Oversight and direction to the services: 5; Area of improvement: FVAP processes: 1; Area of improvement: Total: 6. Source of recommendation and recommendation status: Total: Combined DOD Inspector General and GAO; Area of improvement: Oversight and direction to the services: 16; Area of improvement: FVAP processes: 7; Area of improvement: Total: 23. Source: GAO analysis of DOD Office of Inspector General, FVAP, and GAO data. [End of table] FVAP and USD (P&R) Fully or Partially Implemented Most of the Recommendations to Improve Oversight and Direction: FVAP and USD (P&R) fully implemented 8 of 16 DOD Office of Inspector General and GAO recommendations and partially implemented 3 recommendations to improve the oversight and direction FVAP provides to the services to guide their implementation of service-and installation-level voting assistance programs. However, 5 of the 16 recommendations have not been implemented. In recent years, several revisions have been made to DOD Directive 1000.04, Federal Voting Assistance Program (FVAP), and many of the changes are related to DOD Office of Inspector General and GAO recommendations. For example, the directive was revised in 2002 to direct the services to develop written policies to support all eligible military personnel and their family members, following a 2001 GAO recommendation that DOD direct service secretaries to develop voting assistance directives that fully reflect the requirements of DOD Directive 1000.04. DOD Directive 1000.04 was also revised in 2002 to require the heads of DOD components to assign one Unit VAO within each unit of 25 or more permanently assigned members, and was revised again in 2004 to require assignment of an additional Unit VAO for each additional 50 members above the 25 member base. Prior to these revisions, we reported that VAOs were sometimes responsible for too many voters as a result of the lack of DOD-wide criteria for determining how many voters a VAO should serve and recommended that DOD consult with the services and revise DOD Directive 1000.04 to establish recommended ratios of VAOs to population served. In addition to changes made through the directive, FVAP and USD (P&R) directed the revision of service-level voting assistance regulations and provided additional clarifying guidance to the military services through the biennial FVAP voting action plans and memorandums. Table 4 summarizes the DOD Office of Inspector General's and GAO's recommendations to improve oversight of and direction to the services and lists the actions that FVAP and USD (P&R) took to implement the recommendations. Table 4: DOD Office of Inspector General and GAO Recommendations to Improve Oversight of and Direction to the Services and FVAP's and USD (P&R)'s Actions Taken to Implement the Recommendations: 2001 through 2009: Recommendation (source, year): Direct service secretaries to develop voting assistance directives that fully reflect the requirements of DOD Directive 1000.04. (GAO, 2001); Actions taken: Implemented: USD (P&R) revised Directive 1000.04 in 2002 to direct the services to develop written policies to support all eligible military personnel and their family members. Recommendation (source, year): Consult with the services and revise DOD Directive 1000.04 to establish recommended ratios of VAOs to population served. (GAO, 2001); Actions taken: Implemented: USD (P&R) revised Directive 1000.04 in 2002 to require that the heads of DOD components assign one unit VAO within each unit of 25 or more permanently assigned members, and added language in 2004 stating that an additional unit VAO should be assigned for each additional 50 members above the 25 member base. Recommendation (source, year): Actively collect and share best practices identified by service and Department of State voting assistance programs. (GAO, 2001); Actions taken: Implemented: DOD's biennial voting action plans require FVAP to collect and share best practices identified by the services and Department of State. FVAP published best practices on its Web site and in the Voting Information News newsletter that it distributes monthly to VAOs, and discussed them at voting workshops. Recommendation (source, year): Direct service secretaries to require that the senior service voting representatives monitor installations' voting assistance programs, periodically provide briefings to FVAP on the services' efforts to comply with DOD policy, and submit a final report by June 30 of odd-numbered years. (GAO, 2001); Actions taken: Partially implemented: DOD Directive 1000.04 requires that the services file an after-action report to FVAP, and the biennial voting action plans require that these reports be filed by January 15 of odd-numbered years. In late 2003, the USD (P&R) began holding monthly meetings with FVAP and the senior service voting representatives to discuss the status of service voting assistance programs. Neither the DOD Directive nor the voting action plans explicitly require the senior service voting representatives to monitor installation-level voting assistance programs. Recommendation (source, year): Provide oversight to ensure that service voting assistance program regulations are consistent with DOD Directive 1000.04 requirements. (DOD Office of Inspector General, 2003); Actions taken: Implemented: According to follow up completed by the Office of Inspector General, USD (P&R) fulfilled this recommendation by issuing a memorandum to require that the service Secretaries revise their policies to correct discrepancies DOD Office of Inspector General found between the policies and DOD Directive 1000.04. Recommendation (source, year): Expedite the revision and issuance of DOD Directive 1000.04 and the DOD Voting Action Plan for 2004-2005. (DOD Office of Inspector General, 2004); Actions taken: Implemented: DOD issued the revised Directive 1000.04 on April 14, 2004. USD (P&R) and FVAP released the Voting Action Plan for 2004-2005 on February 11, 2004. Recommendation (source, year): Provide oversight to ensure that service voting assistance program regulations and service voting plans are consistent with the requirements established by the new guidance. (DOD Office of Inspector General, 2004); Actions taken: Implemented: FVAP reviewed the revised service policies to ensure compliance with DOD Directive 1000.04. Recommendation (source, year): Revise DOD Directive 1000.04 to require FVAP and the services to collect and analyze metrics on a more frequent basis. (DOD Office of Inspector General, 2005); Actions taken: Not implemented: The current version of the draft DOD Instruction 1000.04, if finalized, will require the director of FVAP to establish minimum voting program metrics to be used by the DOD components and the Uniformed Services in evaluating their individual voting assistance programs, and report on the results of those metrics.[A] Recommendation (source, year): Revise DOD Directive 1000.04 to require that VAO training include all program objectives outlined in DOD Directive 1000.04 and good ideas and best practices as tools for accomplishing objectives, and that VAOs should complete FVAP training within 60 days of their appointment. (DOD Office of Inspector General, 2005); Actions taken: Not implemented: FVAP collects and shares best practices, and discusses them at VAO training workshops. However, based on our observation, VAO training does not address all program objectives of DOD Directive 1000.04, and the Directive does not require that all objectives be addressed in the training. The current version of draft DOD Instruction 1000.04, if finalized, would require VAOs to complete training within 90 days of their appointment.[A] Recommendation (source, year): Revise DOD Directive 1000.04 to permit heads of the DOD components and the services to exercise flexibility in how many unit VAOs they assign and at what level. (DOD Office of Inspector General, 2006); Actions taken: Not implemented: USD (P&R) issued a memorandum to the Secretaries of the Military Departments stating that the guidance and direction found in DOD Directive 1000.04, which includes guidance regarding the ratio of VAOs to assigned members within a unit and the desired rank or pay grade of VAOs, should be used as a baseline and may be adjusted to meet specific requirements and cultures within each service. In addition, the memorandum describes revisions to the relevant provisions in the current version of the draft DOD Instruction 1000.04 and emphasizes the flexibility of those provisions.[A] Recommendation (source, year): Direct the services to establish a "one- click" Web site access from its home page to a voting page that provides service and FVAP-related information and materials. (DOD Office of Inspector General, 2006); Actions taken: Partially implemented: The current version of draft DOD Instruction 1000.04, if finalized, will require each service to have a direct link from its Web site home page to the service's voting assistance program home page. Recommendation (source, year): Publish guidance to the services clarifying DOD Directive 1000.04 on the commander's flexibility in establishing the ratio of unit VAOs to unit members. (DOD Office of Inspector General, 2007); Actions taken: Implemented: USD (P&R) issued a memorandum to the Secretaries of the Military Departments stating that the guidance and direction found in DOD Directive 1000.04, which includes guidance regarding the ratio of VAOs to assigned members within a unit, should be used as a baseline and may be adjusted to meet specific requirements and cultures within each service. In addition, the memorandum describes revisions to the relevant provisions in the current version of the draft DOD Instruction 1000.04 and emphasizes the flexibility of those provisions.[A] Recommendation (source, year): Revise DOD Directive 1000.04 to provide flexibility for pay grades required of personnel assigned VAO duties. (DOD IG, 2007); Actions taken: Not implemented: USD (P&R) issued a memorandum to the Secretaries of the Military Departments stating that the guidance and direction found in DOD Directive 1000.04, which includes guidance regarding the desired rank or pay grade of VAOs, should be used as a baseline and may be adjusted to meet specific requirements and cultures within each service. In addition, the memorandum describes revisions to the relevant provisions in the current version of the draft DOD Instruction 1000.04, and emphasizes the flexibility of those provisions.[A] Recommendation (source, year): Publish guidance to the services emphasizing the importance and benefits of appointing civilian personnel as installation VAOs. (DOD Office of Inspector General, 2007); Actions taken: Implemented: USD (P&R) issued a memorandum to the Secretaries of the Military Departments stating that the guidance and direction found in DOD Directive 1000.04, which says that where possible, Installation VAOs should be a civilian GS-12 or higher, should be used as a baseline and may be adjusted to meet specific requirements and cultures within each service. In addition, the memorandum describes revisions to the relevant provisions in the current version of the draft DOD Instruction 1000.04.[A] Recommendation (source, year): Direct the services to comply with a recommendation from the 2005 report to establish one-click links from their home page to its voting assistance site. (DOD Office of Inspector General, 2007); Actions taken: Partially implemented: The current version of draft DOD Instruction 1000.04, if finalized, will require each service to have a direct link from its Web site home page to the service's voting assistance program home page. Recommendation (source, year): Revise DOD Directive 1000.04 to authorize appointment of civilian personnel as unit VAOs where feasible. (DOD Office of Inspector General, 2007); Actions taken: Not implemented: USD (P&R) determined that this recommendation may not be practicable because civilian government personnel within the United States may not be authorized by law to administer oaths. Source: GAO analysis of DOD Office of Inspector General, USD (P&R), FVAP, and GAO information. [A] Although the DOD Instruction 1000.04, which will replace the current DOD Directive 1000.04, was still in draft during our review, FVAP officials told us that the services had been informed of the new provisions in the policy for the purpose of implementing their respective voting assistance programs. [End of table] FVAP and USD (P&R) partially implemented three other recommendations. In 2001, GAO recommended that the service Secretaries should (1) require the senior service voting representatives to monitor installations' voting assistance programs, (2) periodically update FVAP on efforts to meet the requirements of DOD Directive 1000.04, and (3) submit a final report by June 30 of odd-numbered years. In response, USD (P&R) began holding monthly meetings in late 2003 with FVAP staff and senior service voting representatives to discuss the status of service voting assistance programs. However, neither DOD Directive 1000.04 nor FVAP's biennial voting action plans explicitly required senior service voting representatives to monitor installations' voting assistance programs. In our 2001 report, we noted that such monitoring would enable the services to identify and address weaknesses in their programs. In addition, FVAP partially addressed a 2006 and 2007 DOD Office of Inspector General recommendation in its the current draft that includes a requirement that each service have a direct link from its Web site home page, to the service's voting assistance program home page. FVAP did not implement five of the 16 recommendations. While FVAP has created a draft version of DOD Instruction 1000.04, it is still under development and has not been finalized. The draft of DOD Instruction 1000.04--if finalized as currently written--would replace the existing directive and include several new provisions responding to DOD Office of Inspector General recommendations that call for revisions to Directive 1000.04. For example, the DOD Office of Inspector General recommended that DOD revise Directive 1000.04 to require FVAP and the services to collect and analyze metrics on a more frequent basis. While FVAP and the services recently developed metrics, and the draft DOD Instruction, if finalized, would require the director of FVAP to (1) establish minimum voting program metrics to be used by the DOD components and the Uniformed Services in evaluating their individual voting assistance programs, and (2) report on the results of those metrics, the draft guidance has not been finalized and the directive has not been revised as recommended. In addition, recommendations were made to revise DOD Directive 1000.04 to (1) require that VAO training include all program objectives, (2) permit heads of the DOD components and the services to exercise flexibility in how many unit VAOs they assign, and (3) provide more flexibility for pay grades required of personnel assigned VAO duties. FVAP also did not implement a 2005 DOD Office of Inspector General recommendation to revise DOD Directive 1000.04 to require that VAO training cover all program objectives in the directive and best practices, and that the training be completed within 60 days of a VAO's appointment. The current draft DOD Instruction 1000.04 if finalized will require that VAO training be completed within 90 days of an individual's appointment. Further, while FVAP does collect best practices among VAOs and shares some of these in VAO training workshops, the training workshops we observed did not cover all the program objectives from DOD Directive 1000.04. For example, the workshops did not cover the requirement that the DOD components and uniformed services are to establish and publicize a special telephone service, a "Voting Action Line," to link unit voting officers with their respective Service or Departmental Voting Action Officer. In addition, FVAP did not implement a 2007 DOD Office of Inspector General recommendation to revise Directive 1000.04 to authorize appointment of civilian personnel as unit VAOs in order to increase continuity in the unit VAO positions.[Footnote 15] In the written response to this recommendation, USD (P&R) stated that this recommendation may not be practicable. By way of explanation, it cited potential limitations on the authority of civilian government employees within the United States to administer oaths. FVAP Fully or Partially Implemented All But One Recommendation to Improve Its Processes: FVAP fully implemented three of seven recommendations pertaining to changes in the processes that it uses to execute its mission, and it partially implemented another three such recommendations. For example, in 2001 we recommended that FVAP revise its postelection survey methodology to gather nationally projectable data on disqualified overseas and military absentee ballots from local election officials. In response, FVAP redesigned the postelection survey to obtain the number of absentee ballots from UOCAVA-eligible voters not counted and the reasons for disqualifying the ballots. At the time of our recommendation, we concluded that the collection and analysis of this information over time could be useful in helping assess whether efforts to improve voting assistance and work with states have a positive impact on military and overseas citizens being able to vote. Moreover, such information should help Congress assess the extent to which military and overseas ballots are disqualified and evaluate whether a legislative remedy is needed. FVAP also took steps to implement two GAO recommendations from 2007 to improve the security and accuracy of its electronic and Internet initiatives. The steps included (1) the addition of a cautionary statement to FVAP's electronic blank ballot request and delivery system to warn UOCAVA voters to remove personal data from their computers and (2) the development of a process to review online UOCAVA voting guidance at the FVAP Web site by creating a checklist with steps that FVAP staff must take when making changes to the Web site. Table 5 summarizes the DOD Office of Inspector General's and GAO's recommendations to improve FVAP processes and lists the actions taken by FVAP to implement the recommendations. Table 5: DOD Office of Inspector General and GAO Recommendations to Improve FVAP Processes and Actions Taken by FVAP to Implement the Recommendations: 2001 through 2009: Recommendation (source, year): Develop a methodology to gather nationally projectable data on disqualified military and overseas absentee ballots and reasons for disqualification. (GAO, 2001); Action taken: Implemented: FVAP redesigned the methodology of its postelection survey of local election officials to obtain nationally projectable data on the number of absentee ballots not counted and the reasons for the disqualification of absentee ballots from military and overseas citizens. Recommendation (source, year): Develop an automated electronic delivery and reporting system that would be used to disseminate information to voters. (DOD Office of Inspector General, 2005); Action taken: Not implemented: According to FVAP officials, they determined, along with the services, that it was infeasible because some servicemembers do not have e-mail or Internet access. Recommendation (source, year): Incorporate lessons learned into plans for future systems such as those we identified, including adding cautionary statements to future ballot request and receipt systems to warn UOCAVA voters to remove personal data from their computers. (GAO, 2007); Action taken: Implemented: FVAP added a cautionary statement to its electronic blank ballot request and delivery system. Recommendation (source, year): Institutionalize a process to review online UOCAVA guidance to ensure that DOD provides accurate and consistent information to UOCAVA voters. (GAO, 2007); Action taken: Implemented: FVAP has developed a process to review online voting guidance at its Web site by creating a checklist with steps that FVAP staff must take when changes to the Web site are made. Recommendation (source, year): Comply with the information security requirements in the DOD Certification and Accreditation Process guidance. (GAO, 2007); Action taken: Partially implemented: As of February 2010, FVAP officials told us they plan to ensure that the Electronic Transmission Service for the November 2010 federal election complies with the DOD requirements. A contract for the service has not yet been awarded. Recommendation (source, year): Create an integrated, comprehensive, long-term, results-oriented plan for future electronic voting programs. (GAO, 2007); Action taken: Partially implemented: To date, FVAP has developed some high-level tasks and estimated timeframes for such a program, but FVAP officials told us they are waiting for the Election Assistance Commission to issue electronic absentee voting guidelines in order to finish their plan. Recommendation (source, year): Develop metrics and standards for assessing effectiveness and compliance of services' and DOD's Voting Assistance Programs. (DOD Office of Inspector General, 2008); Action taken: Partially implemented: FVAP established some performance metrics for implementation beginning in the 2008 election year, but we found that some of these metrics have weaknesses. Source: GAO analysis of DOD Office of Inspector General, FVAP, and GAO information. [End of table] FVAP partially implemented three of the remaining four DOD Office of Inspector General's and GAO's recommendations for improvements to its processes. First, in response to a 2008 DOD Office of Inspector General recommendation that USD (P&R) and the service Secretaries should develop metrics for assessing their voting assistance programs, FVAP established some performance metrics for implementation beginning in the 2008 election year. However, as we noted earlier in our report, we found that some of FVAP's evaluative information can continue to be strengthened. Second, FVAP officials told us they cannot develop a long-term plan for future electronic absentee voting programs, which we recommended in 2007, without the establishment of electronic absentee voting guidelines by the Election Assistance Commission. [Footnote 16] According to a Commission official, voting guidelines are not yet complete because, among other reasons, National Institute of Technology officials--who are providing assistance to the Commission in developing the guidelines--do not believe that a secure electronic absentee voting system is technologically feasible at this time. In the interim, however, FVAP has established some preliminary, high-level tasks and estimated time frames for the development of an electronic absentee voting system. In our 2007 report, we concluded that until FVAP develops the type of plan we recommended, DOD is not in a position to address congressional expectations to establish secure and private electronic and Internet-based voting initiatives. Third, FVAP officials told us they will ensure their Electronic Transmission Service complies with DOD's information security requirements--as we recommended in 2007--prior to the November 2010 federal election. As we reported in 2008, until FVAP fulfills this recommendation by performing and documenting security assessments and certifications for its Electronic Transmission Service, it has not taken all the necessary measures to secure its system and comply with DOD's information security requirements. FVAP did not implement one DOD Office of Inspector General recommendation for improving its processes. In 2005, DOD Office of Inspector General recommended that FVAP develop an automated electronic delivery and reporting system that could be used to disseminate information to voters via targeted mass electronic communication and track the extent to which the information had reached the entire population of servicemembers. FVAP officials told us they determined that this recommendation is infeasible because not all servicemembers have ready access to e-mail or the Internet. Alternatively, FVAP officials told us they have improved the information available to UOCAVA voters at the FVAP Web site, disseminated voting assistance alerts through the heads of the services, and used the comments section on servicemembers' leave and earnings statements to inform them about absentee voting procedures. Conclusions: Members of Congress, advocacy groups, and others have called for improved absentee-voting opportunities for the estimated 6-million U.S. citizens covered by UOCAVA while living in other countries. The Secretary of Defense is the presidential designee with the primary responsibility for federal UOCAVA functions, and through the Under Secretary of Defense for Personnel and Readiness (USD (P&R)), the Federal Voting Assistance Program provides support to potential overseas absentee voters in order to facilitate their participation in federal elections. Toward that end, FVAP took many actions to (1) address the requirements identified in DOD Directive 1000.04, (2) implement recommendations that DOD Office of Inspector General and GAO made to improve FVAP's oversight of and direction to the services as well as to enhance its own processes, and (3) evaluate its efforts to accomplish its mission. Program evaluation is an area where FVAP can continue to improve. Even though FVAP added new performance measures and more rigor for some evaluative methods during recent years, our analyses identified some continuing challenges that FVAP faces in obtaining reliable, consistent, and credible evaluative information for monitoring and adjusting its efforts to address requirements in the DOD directive. Our work showed that these challenges are the result of such things as not following survey-related guidance from the Office of Management and Budget, gathering data on credible but limited content issues, and not checking the reliability of self- reported information. The new FVAP Director has reached out to sources in DOD and externally to obtain evaluative expertise that FVAP has not previously used, but the outreach has been too recent to determine its impact on FVAP's evaluative processes. However, delays in improving the evaluation of FVAP activities could result in missed opportunities to both improve the data that FVAP will have following the 2010 federal election and make data-guided improvements before the 2012 election. Delay in improving the evaluation of the VAOs' training is particularly important because VAOs occupy such a critical role in helping FVAP to address the directive-specified requirements. For example, the absence of data comparing the performance of VAOs who had online versus in-person training raises questions about: * the relative effectiveness of the two training methods, * whether the online training produces enough added benefit to warrant its much higher cost than remote training, and: * whether newer technological alternatives should replace one or both of the current training methods. * If FVAP were to address these areas that need improvements promptly, the agency would also have better data to provide to Congress in response to the new requirement for an annual report which was mandated in the National Defense Authorization Act for Fiscal Year 2010. Recommendations for Executive Action: To improve the effectiveness of DOD's absentee voting assistance efforts, we recommend that the Secretary of Defense direct the Under Secretary of Defense for Personnel and Readiness to take the following three actions: * Assess the methodologies currently in use to gather evaluative information on FVAP's efforts to address the requirements in DOD Directive 1000.04. * Implement, where needed, improved and supplemental program evaluation methodologies to address concerns such as those that we identified in this report. * Evaluate current and alternative methods for training VAOs as part of federal elections. Agency Comments and Our Evaluation: In written comments on a draft of this report, DOD concurred with two of our recommendations and partially concurred with a third recommendation. The department's written comments are reprinted in appendix III. DOD concurred with our recommendation to assess the methodologies currently in use to gather evaluative information on FVAP's efforts to address the requirements in DOD Directive 1000.04. DOD stated that it is already developing, with the military services, new metrics to evaluate the program's effectiveness in meeting the requirements of DOD Directive 1000.04, as well as the new responsibilities identified in the Military and Overseas Voters Empowerment Act. While developing these metrics is a step in the right direction, it is too soon to determine how well and fully the new metrics will evaluate the program's effectiveness to meet its requirements. DOD partially concurred with our recommendation to implement, where needed, improved and supplemental program-evaluation methodologies to address concerns such as those that we identified in this report. DOD discussed its survey methodologies only and therefore did not indicate how FVAP might (1) address the non-survey concerns that we identified with its measures of success and (2) supplement its current methodologies to enhance the agency's program evaluation efforts. For instance, FVAP did not identify plans to conduct reliability assessments or other evaluations to promote data consistency and accuracy. With regard to its surveys, DOD stated that the "exceptionally low" response rate for the overseas civilian survey highlights the problems in defining the actual size and composition of the overseas American population. Given the very low response rate, DOD said FVAP is not considering any of the results of that survey in its post-2008 elections report to Congress. Additionally, DOD said that FVAP and the Defense Manpower Data Center are, however, attempting to refine and improve that survey and will include nonresponse analysis for its 2010 survey. DOD further agreed with GAO's concern regarding survey content and reported that FVAP has been in close discussions with the Defense Manpower Data Center to include appropriate questions in the 2010 Post-Election Survey. DOD also noted that it does not believe the post-election surveys of active duty military members and the local election officials suffer in the same way as other surveys do from low response rates. DOD added that a lower response rate is not indicative of a flawed study, nor does the lack of a specific nonresponse bias analysis indicate that the original survey results are not statistically valid. The department supported its assertions by noting that the Defense Manpower Data Center has much data on the demographics of the people that it surveys and by stating that an academic researcher found "no strong empirical relationship between response rates and nonresponse bias." Additional information needs to be considered in evaluating DOD's assertions. First, GAO raised the issue of nonresponse bias analysis because 2006 OMB-issued guidance for executive branch agencies states "Agencies must design the survey to achieve the highest practical rates of response, commensurate with the importance of survey uses, respondent burden, and data collection costs, to ensure that survey results are representative of the target population so that they can be used with confidence to inform decisions. Nonresponse bias analyses must be conducted when unit or item response rates or other factors suggest the potential for bias to occur."[Footnote 17] Thus, DOD's position that nonresponse bias analysis is not warranted is contrary to an OMB- prescribed standard for data collection. Second, if the Defense Manpower Data Center does not use its extensive demographic data to analyze differences in response rates between groups, DOD cannot draw conclusions about how well the demographics of the respondents and nonrespondents actually agreed. Third, DOD's summarization of the academic researcher's findings is somewhat misleading.[Footnote 18] The researcher said that the relationship between nonresponse rate and bias was "modest." More importantly, other points in the researcher's study noted that: "A synthesis of research studies estimating nonresponse bias shows the bias often present," and that: "As response rates decline, researchers face a growing obligation to mount nonresponse bias studies in order to inform the evaluation of survey estimates." We reiterate our call for FVAP and Defense Manpower Data Center to follow OMB guidance to conduct the nonresponse bias analysis. Continued use of the evaluative information without quality- assurance steps such as assessing data reliability and potential nonresponse bias could limit DOD's ability to provide a more complete picture of FVAP's performance, which could impact the quality of future reports to Congress submitted in response to section 586 of the National Defense Authorization Act for Fiscal Year 2010. DOD concurred with our recommendation to evaluate current and alternative methods for training VAOs as part of federal elections. DOD stated that it is pursuing a different approach by transforming voting assistance into a direct-to-the-voter assistance program of centralized online tools, training, and assistance provided by FVAP. For the 2010 election cycle, FVAP said it is most prudent to continue the traditional VAO program and supplement it with direct-to-the-voter assistance. With regard to the focus of our recommendation on the evaluation of VAO training, FVAP indicated that it is implementing post-training evaluation of its VAO workshops and will include more detailed questions in its FVAP's surveys of voters and VAOs following the 2010 election. Reliance on findings from those post-election surveys is problematic because of concerns such as response rates and identification of the population. Additional evaluation methods could be useful, such as testing samples of participants on how much knowledge they gained from the training. Similarly, structured gathering of participants' perceptions (e.g., which training topics need additional explanation) and analysis of that information could offer insights regarding the relative advantages of current and alternative methods of VAO training. As we agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution of it until 30 days from its date. At that time, we will send copies to other interested parties. The report will also be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. Should you or your staff have any questions about this report, please contact me at (202) 512-8246. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV. Signed by: Jack E. Edwards: Director, Defense Capabilities and Management: List of Requesters: The Honorable Robert Brady: Chairman: The Honorable Daniel Lungren: Ranking Member: Committee on House Administration: House of Representatives: The Honorable Susan Davis: Chairwoman: Subcommittee on Military Personnel: Committee on Armed Services: House of Representatives: The Honorable Mike Honda: House of Representatives: The Honorable Carolyn Maloney: House of Representatives: [End of section] Appendix I: Scope & Methodology: To evaluate the extent to which the Federal Voting Assistance Program (FVAP) has addressed its mission and evaluated the effectiveness of its efforts to conduct that mission, we reviewed and analyzed relevant laws (e.g., Uniformed and Overseas Citizens Absentee Voting Act-- UOCAVA[Footnote 19]), directives (e.g., Department of Defense (DOD) Directive 1000.04), and other guidance (e.g., memorandums). We reviewed UOCAVA and DOD Directive 1000.04 to identify specific federal responsibilities for absentee voting and compared these responsibilities with actions taken by FVAP. We also reviewed relevant reports prepared by GAO, FVAP, DOD Office of Inspector General, Election Assistance Commission, Overseas Vote Foundation, and Pew Charitable Trusts to gain a thorough understanding of prior reviews conducted on FVAP and the challenges involved in the absentee voting process. To assess the effectiveness of FVAP's efforts to conduct its mission, we reviewed FVAP's data gathering procedures and available measures obtained from FVAP and published reports (e.g., ours and those of the DOD Office of Inspector General and non-governmental organizations). We then compared FVAP's procedures and measures against criteria on performance measurement, program evaluation, and related issues that GAO and others published in reports.[Footnote 20] We additionally reviewed the methodology and questions used in FVAP's 2008 postelection surveys of servicemembers, VAOs, overseas citizens, and local election officials and compared FVAP-related information against criteria in guidance from the Office of Management and Budget. [Footnote 21] We interviewed key survey program officials from the Defense Manpower Data Center to obtain information and methodologies for FVAP's postelection surveys. We also interviewed officials from FVAP, the Department of State, the Election Assistance Commission, Overseas Vote Foundation, and Pew Charitable Trusts to obtain their perspectives on FVAP's efforts. Additionally, we reviewed FVAP's 2010- 2011 Voting Assistance Guide and the agency's Web site to determine the types of information provided to UOCOVA-covered citizens considering or attempting to vote absentee. To specifically assess the requirement that FVAP train VAOs, we held interviews with key officials and with a nonprobability sample of VAOs from the services to obtain their perspective of FVAP's VAO training. Because this was a nonprobability sample of VAOs, the results of these interviews are not generalizeable to the population of VAOs. To select the VAOs, we obtained a list of VAOs for the most recent voting cycle from the Army, Navy, Marine Corps, and Air Force and generated a non- probability random sample from each list. We observed in-person VAO training workshops and completed the online training that can also be used for VAO training. For both types of training, we compared the methodological and content-related information against requirements specified in DOD Directive 1000.04 and criteria specified in training standards that we published.[Footnote 22] To evaluate the extent to which FVAP has processes in place to align its budget priorities with its strategic plans and goals, we reviewed FVAP's budget development procedures and compared its budgets for fiscal years 2005 through 2009 with the goals described in its strategic plan for 2008 through 2009. We began with fiscal year 2005 because this is when FVAP's budget reporting requirement shifted from the Washington Headquarters Services to the Defense Human Resource Activity. Additionally, we interviewed key FVAP and Defense Human Resource Activity officials to clarify the program's planning and budgeting processes. After gaining an understanding of the budget, budget process, and contents of FVAP's strategic plan, we worked with FVAP staff to identify which parts of the budget were primarily used to address each of the goals in the strategic plan. To determine the extent to which DOD Office of Inspector General and GAO recommendations made from 2001 through 2009 were implemented by FVAP, we reviewed DOD Office of Inspector General and GAO reports on absentee voting to identify any recommendations related to this program.[Footnote 23] We also interviewed DOD Office of Inspector General officials who conducted the evaluations of FVAP's and the Under Secretary of Defense for Personnel and Readiness's implementation efforts and reviewed documentation on those efforts. Additionally, we discussed our findings with DOD Office of Inspector General officials to clarify the recommendations' status, actions taken by FVAP and the Under Secretary, and any positive outcomes from implementation. Similarly, we reviewed our reports with UOCAVA-related recommendations as well as information later obtained to document the efforts to implement our recommendations. In cases where a recommendation was not implemented or partially implemented, we sought additional information from FVAP officials, for example, confirmation that our finding was correct and the reason why the recommendation was not fully implemented. For each of our objectives, we assessed the reliability of the data we analyzed by reviewing existing documentation related to the data sources and interviewing knowledgeable agency officials about the data that we used. We found the data sufficiently reliable for the purposes of this report. We conducted this performance audit from June 2009 through May 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: DOD Office of Inspector General and GAO Reports Containing DOD Voting Assistance Program Recommendations: Department of Defense, Office of the Inspector General. 2008 Evaluation of the DOD Voting Assistance Program. Report Number IE-2009- 005. Arlington, Virginia: April 30, 2009. Department of Defense, Office of the Inspector General. 2007 Evaluation of the Federal Voting Assistance Program in the Department of Defense. Report Number IE-2008-002. Arlington, Virginia: March 31, 2008. GAO. Elections: Action Plans Needed to Fully Address Challenges in Electronic Absentee Voting Initiatives for Military and Overseas Citizens. [hyperlink, http://www.gao.gov/products/GAO-07-774]. Washington, D.C.: June 14, 2007. Department of Defense, Office of the Inspector General, 2006 Evaluation of the Federal Voting Assistance Program in the Department of Defense. Report Number IE-2007-004. Arlington, Virginia: March 31, 2007. Department of Defense, Office of the Inspector General, Evaluation of the Voting Assistance Program. Report Number IE-2006-001. Arlington, Virginia: March 31, 2006. Department of Defense, Office of the Inspector General, Evaluation of the Voting Assistance Program. Report Number IE-2005-001. Arlington, Virginia: March 31, 2005. Department of Defense, Office of the Inspector General, DOD Implementation of the Voting Assistance Program. Report Number D-2004- 065. Arlington, Virginia: March 31, 2004. Department of Defense, Office of the Inspector General, DOD Compliance With the Uniformed and Overseas Citizens Absentee Voting Act. Report Number D-2003-072. Arlington, Virginia: March 31, 2003. GAO. Elections: Voting Assistance to Military and Overseas Citizens Should Be Improved. [hyperlink, http://www.gao.gov/products/GAO-01-1026]. Washington, D.C.: September 28, 2001. [End of section] Appendix III: Comments from the Department of Defense: Under Secretary Of Defense: Personnel And Readiness: 4000 Defense Pentagon: Washington, D.C. 20301-4000: May 24, 2010: Mr. Jack E. Edwards: Director, Defense Capabilities and Management: U.S. Government Accountability Office: 441 G Street, N.W. Washington, DC 20548: Dear Mr. Edwards: This is the Department of Defense (DoD) response to the GAO draft report, Elections: DoD Can Strengthen Evaluation of Its Absentee Voting Assistance Program, dated April 20, 2010 (GAO Code 351362/GA0- 10-476). The Federal Voting Assistance Program (FVAP) will make strategic and operational improvements based on the findings of its 2008 post- election surveys of Active Duty Military and Local Election Officials, customer and stakeholder feedback, as well as input from the GAO and DoD Inspector General reports. GAO only had the 2008-09 FVAP Strategic Plan available for analysis. FVAP has since initiated a strategic planning effort based upon new ideas for voting assistance improvements, the results of the 2008 post-election surveys, and inputs from your staff during this audit. This new effort will substantially address the issues raised in this report. The 2008 post-election surveys show military voter registration rates exceed those of the national electorate, and that military voter participation rates, when adjusted for the substantial age and gender differences between military and civilian populations, also exceed the national voter participation rates. Where military voter success differs sharply from the general electorate is in absentee ballot return rates: only 63% of military absentee ballots were returned for the November 2008 general election, compared to 91% of the general electorate. It appears the primary reason for this failure is that ballots were not sent to military voters far enough before the election for military voters to receive, complete, and return them by State-imposed absentee ballot receipt deadlines. This is outside the control of the military Voting Assistance Officers, of course, but significantly impacts overall military voter success rates. The requirement imposed upon States by the MOVE Act to transmit ballots to military and overseas voters at least 45 days before federal elections, and to transmit them electronically, should substantially improve the absentee ballot return rates for military and overseas voters. We believe the FVAP surveys show the FVAP program is successfully assisting military voters in exercising their right to vote. We agree we can do even better, and are committed to further studies to monitor and improve what we do. Thank you for your cooperation and the opportunity to comment on this report. Sincerely, Signed by: Clifford L. Stanley: Attachment: Comments to the Recommendations: [End of letter] GAO Draft Report — Dated April 20, 2010: GAO Code 351362/GAO-10-476: "Elections: DoD Can Strengthen Evaluation of Its Absentee Voting Assistance Program" Department Of Defense Comments To The Recommendations: Recommendation 1: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense for Personnel and Readiness (USD(P&R)) to assess the methodologies currently in use to gather evaluative information on Federal Voting Assistance Program's (FVAP's) efforts to address the requirements in DoD Directive 1000.04. DOD Response: Concur. The Federal Voting Assistance Program (FVAP) is already developing, with the military Services, the new metrics to evaluate the program's effectiveness in meeting the requirements of DoD Directive 1000.04, to include the new responsibilities put forward in the Military and Overseas Voters Empowerment Act. As noted in the GAO report, the current draft of the revised DoD Instruction 1000.04 will address many of the shortcomings GAO identified in the prior year program evaluation information. Further, the combined Unit Voting Assistance Officer and direct-to-the-voter assistance program FVAP has undertaken during the 2010 election cycle will present an opportunity to evaluate both programs against each other. Recommendation 2: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense for Personnel and Readiness (USD(P&R)) to implement, where needed, improved and supplemental program evaluation methodologies to address concerns such as those that we identified in this report. DOD Response: Partially Concur. The exceptionally low response rate for the overseas civilian survey highlights the problems the Federal Voting Assistance Program (FVAP) and others in both the overseas community and federal government (Department of State and Census Bureau) face in defining the actual size and composition of the overseas American population. Given that very low response rate, FVAP is not considering any of the results of that survey in its post-2008 election report. This is a departure from previous FVAP practice which would report such survey results, regardless of the survey population coverage or response rates, no matter how low. To determine possible remedies, FVAP and the Defense Manpower Data Center are examining a number of alternative methods to refine and improve the overseas civilian survey, including revised survey methods and new outreach programs through overseas citizen advocacy groups. Regardless, until the overall overseas, non-federal, U.S. citizen population is better defined, it will be impossible to fully evaluate the effectiveness of the voting assistance program provided to overseas Americans; voter registration and voter participation rates are impossible to calculate if the denominator of the total overseas American population is unknown. In contrast, FVAP does not believe the active duty military and the local election official post-election surveys suffer in the same way from lower response rates. The Department understands GAO's concerns with response rates and lack of non-response bias studies during the 2008 Post-Election Survey, but a lower response rate, in and of itself, is not indicative of a flawed study, nor does the lack of specific non-response analysis indicate that the original survey results are not statistically valid. Groves (2006) finds no strong empirical relationship between response rates and nonresponse bias. [Footnote 24] FVAP believes that the active duty survey likely has lower nonresponse bias than surveys with much higher response rates because generally survey organizations know very little about survey nonrespondents. For instance, in telephone surveys, for 'ring-no-answer' cases the surveyor may only know limited geographic data based on the telephone exchange. For household interview surveys, the surveyor may only have outdated knowledge (usually Census data) of block characteristics (e.g., percent Hispanic). For the FVAP survey of active duty military, DMDC has exceptionally high quality and timely demographic data available for both respondents and nonrespondents, including Service, paygrade, age, gender, education, deployment history, and much more. DMDC statisticians use knowledge of nonrespondents' characteristics to create sophisticated statistical adjustments designed specifically to reduce nonresponse bias, including non-response and post- stratification weighting adjustments. For the FVAP survey of local election officials (LEO), the 55 percent response rate is generally considered high for a survey that does not include high-cost survey methods like face-to-face interviewing. For the LEO survey, we also have considerable information on nonrespondent LEOs, including state, jurisdiction size in number of registered voters, and type of governmental unit (county versus local). These data were used both for statistical imputations for item-missing data and weighting adjustments to compensate for unit nonresponse, and again both these procedures reduce the nonresponse bias in survey estimates. For both these surveys, FVAP and DMDC's statisticians consider estimates to be representative of their respective populations and, therefore, the results can be used in program evaluation, policy decisions, and program planning and execution. Regardless, the Department will include non-response analysis in its 2010 survey. The Department also agrees with GAO's specific concern regarding survey content, such as local election official views on the effectiveness of FVAP's Legislative Initiatives program, and has been in close discussions with the Defense Manpower Data Center to include such questions in the 2010 Post-Election Survey. Recommendation 3: The GAO recommends that the Secretary of Defense direct the Under Secretary of Defense for Personnel and Readiness (USD(P&R)) to evaluate current and alternative methods for training voting assistance officers (VAOs) as part of federal elections. DOD Response: Concur. The Federal Voting Assistance Program (FVAP) also agrees with the Department of Defense Inspector General's 2005 assessment of the Voting Assistance Officer program: "voting assistance will always be a secondary duty [for military unit Voting Assistance Officers, and] senior leadership can expect improvement only if a radically different approach is applied."[Footnote 25] To that end, FVAP is pursuing a different approach by transforming voting assistance into a direct to the voter assistance program of centralized online tools, training, and assistance provided by FVAP. However, for the 2010 election cycle, given the apparent performance based success of the Voting Assistance Officer program,[Footnote 26] FVAP felt it most prudent to continue the traditional VAO program and layer the additional direct-to-the-voter assistance on top of that unit-and installation-level VAO assistance, in order to evaluate both against each other. Additionally, the Military and Overseas Voter Empowerment Act requires extensive voting assistance at installation voting assistance offices, including additional voting assistance through the National Voter Registration Act at every installation. Coupled with increased help desk assistance, the FVAP should begin to decrease the demands on the Unit VAOs to only those most in need of assistance. Regardless, based on GAO's initial communications, FVAP is already implementing post-training evaluation of VAO workshops, and will include more detailed contextual and evaluative questions in its voter and Voting Assistance Officer 2010 Post-Election Surveys. [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: Contact: Jack Edwards, (202) 512-8246 or EdwardsJ@gao.gov: Acknowledgments: In addition to the individual named above, Carleen Bennett, Assistant Director; Bonita Anderson; Melissa Blanco; Chanee Gaskin; Nicole Harms; Mae Jones; Jennifer Madison; Amanda Miller; Terry Richardson; Michael Silver; Susan Tindall; and Yong Song made key contributions to this report. [End of section] Related GAO Products: Elections: Action Plans Needed to Fully Address Challenges in Electronic Absentee Voting Initiatives for Military and Overseas Citizens. [hyperlink, http://www.gao.gov/products/GAO-07-774]. Washington, D.C.: June 14, 2007. Elections: All Levels of Government Are Needed to Address Electronic Voting System Challenges. [hyperlink, http://www.gao.gov/products/GAO-07-576T]. Washington, D.C.: March 7, 2007. Elections: DOD Expands Voting Assistance to Military Absentee Voters, but Challenges Remain. [hyperlink, http://www.gao.gov/products/GAO-06-1134T]. Washington, D.C.: September 28, 2006. Elections: The Nation's Evolving Election System as Reflected in the November 2004 General Election. [hyperlink, http://www.gao.gov/products/GAO-06-450]. Washington, D.C.: June 6, 2006. Elections: Absentee Voting Assistance to Military and Overseas Citizens Increased for the 2004 General Election, but Challenges Remain. [hyperlink, http://www.gao.gov/products/GAO-06-521]. Washington, D.C.: April 7, 2006. Election Reform: Nine States' Experiences Implementing Federal Requirements for Computerized Statewide Voter Registration Lists. [hyperlink, http://www.gao.gov/products/GAO-06-247]. Washington, D.C.: February 7, 2006. Elections: Views of Selected Local Election Officials on Managing Voter Registration and Ensuring Eligible Citizens Can Vote. [hyperlink, http://www.gao.gov/products/GAO-05-997]. Washington, D.C.: September 27, 2005. Elections: Federal Efforts to Improve Security and Reliability of Electronic Voting Systems Are Under Way, but Key Activities Need to Be Completed. [hyperlink, http://www.gao.gov/products/GAO-05-956]. Washington, D.C.: September 21, 2005. Elections: Additional Data Could Help State and Local Elections Officials Maintain Accurate Voter Registration Lists. [hyperlink, http://www.gao.gov/products/GAO-05-478]. Washington, D.C.: June 10, 2005. Department of Justice's Activities to Address Past Election-Related Voting Irregularities. [hyperlink, http://www.gao.gov/products/GAO-04-1041R]. Washington, D.C.: September 14, 2004. Elections: Electronic Voting Offers Opportunities and Presents Challenges. [hyperlink, http://www.gao.gov/products/GAO-04-975T]. Washington, D.C.: July 20, 2004. Elections: Voting Assistance to Military and Overseas Citizens Should Be Improved. [hyperlink, http://www.gao.gov/products/GAO-01-1026]. Washington, D.C.: September 28, 2001. Elections: The Scope of Congressional Authority in Election Administration. [hyperlink, http://www.gao.gov/products/GAO-01-470]. Washington, D.C.: March 13, 2001. [End of section] Footnotes: [1] Pub. L. No. 99-410 (1986), codified as subsequently amended at 42 U.S.C. §§1973ff et seq. [2] GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season Performance Measures, [hyperlink, http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: Nov. 22, 2002). [3] Office of Management and Budget, Guidance on Agency Survey and Statistical Information Collection, (Washington, D.C.: January 2006). [4] The National Defense Authorization Act for Fiscal Year 2002, Pub. L. No. 107-107, §1604 (2001) directed the Secretary of Defense to carry out a demonstration project under which absent uniformed services voters are permitted to cast ballots through an electronic voting system. Initially, the project was to be carried out in the general election for federal office in either November 2002 or November 2004. The Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375, §567 (2004) amended that requirement by requiring the project to be implemented during the first general election for federal office that occurs after the Election Assistance Commission establishes electronic absentee voting guidelines and certifies that it will assist the Secretary of Defense in carrying out the project. In 2007, we reported that DOD had not moved forward with the project because the Commission had not developed the guidelines that DOD needs to develop the demonstration project. [5] 42 U.S.C. §1973ff, et seq. [6] DOD Directive 1000.04, Federal Voting Assistance Program (FVAP) (Apr. 14, 2004). [7] GAO, Managing for Results: Critical Issues for Improving Federal Agencies' Strategic Plans, [hyperlink, http://www.gao.gov/products/GAO/GGD-97-180] (Washington, D.C.: Sept. 16, 1997). [8] GAO, Program Evaluation: An Evaluation Culture and Collaborative Partnerships Help Build Agency Capacity, [hyperlink, http://www.gao.gov/products/GAO-03-454] (Washington, D.C.: May 2, 2003). [9] DOD Office of Inspector General, 2008 Evaluation of the DOD Voting Assistance Program, Report No. IE-2009-005 (Arlington, Va.: Apr. 30, 2009). [10] Office of Management and Budget's January 2006 Guidance on Agency Survey and Statistical Information Collections says, among other things, that information collection requests for "surveys with expected response rates lower than 80 percent need…a description of plans to evaluate nonresponse bias." In explaining the reason for such analyses, the guidance notes that the lower the response rates are and the greater the differences between respondents and nonrespondents, the greater the nonresponse bias. [11] GAO, Human Capital: A Guide for Assessing Strategic Training and Developmental Efforts in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO-04-546G] (Washington, D.C.: March 2004). [12] National Defense Authorization Act for Fiscal Year 2010, Pub. L. No. 111-84, §586 (2009), requires the presidential designee to submit an annual report to the President and to the relevant committees of Congress. That report is to include an assessment of the effectiveness of activities carried out under 42 U.S.C. §1973ff-2B, including the activities and actions of FVAP, a separate assessment of voter registration and participation by overseas voters who are not members of the uniformed services, and a description of the cooperation between states and federal governments in carrying out section 1973ff- 2B. Additionally, the report is to contain a description of the utilization of voter registration assistance under 10 U.S.C. §1533a, including a description of the specific programs implemented by each military department of the Armed Forces pursuant to section 1533a and the number of absent uniformed services voters who utilized voter registration assistance provided under that section. [13] The Electronic Transmission Service is a resource that allows fax- to-email and email-to-fax conversion of election materials during transmission between state and local election officials and UOCAVA voters. [14] See appendix II for DOD Office of Inspector General and GAO reports reviewed from 2001 to 2009 related to these recommendations. [15] In general, civilians serve for a longer period of time than servicemembers before moving to a different unit. [16] The National Defense Authorization Act for Fiscal Year 2002, Pub. L. No. 107-107, §1604 (2001) directed the Secretary of Defense to carry out a demonstration project under which absent uniformed services voters are permitted to cast ballots through an electronic voting system. Initially, the project was to be carried out in either the general election for federal office in either November 2002 or November 2004. The Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375, §567 (2004) amended that requirement by requiring the project to be implemented during the first general election for federal office that occurs after the Election Assistance Commission establishes electronic absentee voting guidelines and certifies that it will assist the Secretary of Defense in carrying out the project. In 2007, we reported that DOD had not moved forward with the project because the Commission had not developed the guidelines that DOD needs to develop the demonstration project. [17] Office of Management and Budget, Standards and Guidelines for Statistical Surveys (September 2006). [18] Robert M. Groves, "Nonresponse Rates and NonResponse Bias in Household Surveys," Public Opinion Quarterly, Vol. 70, No. 5, Special Issue, pp. 646-675 (2006). The cited quotes are on pages 646, 657, and 659. [19] Pub. L. No. 99-410 (1986), codified at 42 U.S.C. §§1973ff et seq. [20] GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season Performance Measures, [hyperlink, http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: Nov. 22, 2002). [21] Office of Management and Budget, Guidance on Agency Survey and Statistical Information Collections (Washington, D.C.: January 2006). [22] GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO-04-546G] (Washington, D.C.: Mar. 1, 2004). [23] The DOD Office of Inspector General reports were completed in response to 10 U.S.C. §1566. [24] Groves, Robert M. "Nonresponse Rates and Nonresponse Bias in Household Surveys." Public Opinion Quarterly, 70(5):646-675. [25] DoD IG, 2004 Evaluation, p. 17, 26. [26] For the 2008 November general election, 77% of the military was registered to vote as compared to 71% of the national Citizen Age Voting Population. When adjusted for the substantial age and gender differences between the military and the general electorate, 73% of the military Citizen Voting Age Population voted as compared to 63.6% of the national Citizen Voting Age Population. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: