This is the accessible text file for GAO report number GAO-14-59 entitled '2020 Census: Bureau Needs to Improve Scheduling Practices to Enhance Ability to Meet Address List Development Deadlines' which was released on December 19, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Chairman, Committee on Homeland Security and Governmental Affairs, U.S. Senate: November 2013: 2020 Census: Bureau Needs to Improve Scheduling Practices to Enhance Ability to Meet Address List Development Deadlines: GAO-14-59: GAO Highlights: Highlights of GAO-14-59, a report to the Chairman, Committee on Homeland Security and Governmental Affairs, U.S. Senate. Why GAO Did This Study: According to the Bureau, it is committed to limiting its per household cost for the 2020 Census to that of the 2010 Census, and believes that reducing the cost of updating the MAF can be of significant help. Because of tight deadlines and the involvement of several different Bureau units in this effort, effective scheduling and collaboration practices are important for the entire process to stay on track. GAO was asked to examine scheduling and collaboration in the Bureau’s efforts to develop a more cost-effective MAF. GAO (1) assessed the reliability of the schedules for two key MAF development programs, and (2) examined the extent to which the Bureau is following leading practices for collaboration for its MAF development work. GAO analyzed the schedules for the two programs most relevant to developing the address list, and reviewed strategic plans and other documents establishing coordination mechanisms and compared them to leading practices for intra-agency collaborative efforts. What GAO Found: The Census Bureau (Bureau) is not producing reliable schedules for the two programs most relevant to building the Master Address File (MAF)— the 2020 Research and Testing program and the Geographic Support System Initiative. * The Bureau did not include all activities in either schedule. The schedules appeared to have reasonable durations for most activities, but they did not include information about required resources. * For both schedules, the Bureau logically linked many activities in a sequence. Yet in both schedules the Bureau did not identify the preceding and following activity for a significant number of activities. Without this logic, the effect of a change in one activity on future activities cannot be seen in the schedule, potentially resulting in unforeseen delays. * The Bureau is not in a position to carry out a quantitative risk analysis on the schedules. As a result of these issues, the schedules are producing inaccurate dates, which could mislead Bureau managers to falsely conclude that all of the work is on schedule when it may not be. Without reliable schedule information, such as valid forecasted dates and the amount of flexibility remaining in the schedule, management faces challenges in assessing the progress of MAF development efforts and determining what activities most need attention. Staff managing the schedules said that they had not received thorough training or certification on scheduling best practices, and, according to schedule managers, staff turnover contributed to the issues GAO identified. Workforce planning and training can help the Bureau have the skills in place to ensure that characteristics of a reliable schedule are met to support key management decisions. The Bureau has documented collaboration activities that follow many leading practices for collaboration. Because several divisions are involved in efforts to develop the MAF, collaboration across these divisions is critical. In recent months, the Bureau has put in place a variety of mechanisms to aid coordination, such as crosscutting task teams. For example, research projects relevant to developing the MAF have representation from multiple divisions. The Bureau has also established memorandums of understanding across divisions to provide a broad framework for working together. Continued management attention to collaboration practices will help to ensure that collaboration across units is occurring as MAF development continues. What GAO Recommends: GAO recommends that the Census Director take a number of actions to improve the reliability of its schedules, including steps to ensure that all relevant activities are included in the schedules, complete scheduling logic is in place, and a quantitative risk assessment is conducted. In addition, GAO recommends a robust workforce planning effort to identify and address gaps in scheduling skills for staff that work on schedules. The Department of Commerce concurred and suggested several clarifications, which GAO included in the report as appropriate. View [hyperlink, http://www.gao.gov/products/GAO-14-59]. For more information, contact Robert Goldenkoff at (202) 512-2757 or goldenkoffr@gao.gov. [End of section] Contents: Letter: Background: Because Schedules Are Not Reliable, Management Lacks Valid Information to Assess Progress and Manage Risk: The Bureau Generally Documented Leading Practices for Collaboration in Its Master Address File Plans: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: Description of Scheduling Best Practices: Appendix III: Assessment of the Extent to Which the Bureau Followed Scheduling Best Practices: Appendix IV: Comments from the Department of Commerce: Appendix V: GAO Contact and Staff Acknowledgment: Tables: Table 1: The Bureau's Schedules for 2020 Research and Testing and the Geographic Support System Initiative Met the Characteristics of a Reliable Schedule to Varying Extents: Table 2: The Bureau Generally Documented Leading Collaboration Practices in Its Plans for Developing a Cost-Effective Master Address File: Figure: Figure 1: Planned Research and Testing Will Help Inform Future Design Decisions: Abbreviations: Bureau: U.S. Census Bureau: MAF: Master Address File: TIGER: Topologically Integrated Geographic Encoding and Referencing: GSS-I: Geographic Support System Initiative: [End of section] GAO: United States Government Accountability Office: 441 G St. N.W. Washington, DC 20548: November 21, 2013: The Honorable Thomas R. Carper: Chairman: Committee on Homeland Security and Governmental Affairs: United States Senate: Dear Chairman Carper: The U.S. Census Bureau (Bureau) relies on a complete and accurate address list, along with precise maps, as the fundamental building blocks of a successful census. An accurate address list is critical because it both identifies all households that are to receive a census questionnaire and serves as the control mechanism for following up with households that fail to respond to the initial questionnaire. Precise maps are critical for counting the population in the proper locations--the basis of congressional reapportionment and redistricting. If the Bureau's address list and maps are inaccurate, people can be missed, counted more than once, or counted at the wrong location. However, our prior work has shown that developing an accurate address list has been both labor intensive and costly. In the 2000 and 2010 Censuses, for example, Bureau field staff verified every existing Master Address File (MAF) housing unit address in the nation and added missing housing units by going door-to-door. The 2010 effort, known as address canvassing, required 140,000 temporary workers to verify 145 million addresses at a cost of $444 million. [Footnote 1] The Bureau felt that in the absence of research demonstrating where it could rely on something less than door-to-door visits to update its address list, it needed to proceed with full canvassing to develop a quality address list and maps. In addition, census workers used handheld computers for the first time in the 2010 address canvassing operation to capture global positioning system information to improve Bureau maps and address lists. With a life cycle cost around $13 billion, the 2010 Census was the most expensive population count in U.S. history, costing over 50 percent more than the $8.1 billion 2000 Census.[Footnote 2] According to the Bureau, it is committed to limiting its per household cost for the 2020 Census to not more than that of the $98 per household cost of the 2010 Census, in constant 2010 dollars. The Bureau believes that reducing the cost of address canvassing can help limit costs. The Bureau's 2020 Research and Planning Office along with four other principal Bureau units[Footnote 3] have several projects under way researching options intended to more cost-effectively build the MAF and the map system that supports it, called the Topologically Integrated Geographic Encoding and Referencing (TIGER). (Hereafter, we refer to these as MAF). Primary among the efforts is the Geographic Support System Initiative (GSS-I), being carried out by the Bureau's Geography Division. A central goal of GSS-I is to enable the Bureau to use "targeted" address canvassing to replace the more expensive "full" address canvassing from prior censuses. Targeted address canvassing would limit the operation only to areas where the Bureau believes door- to-door verification is needed to develop an accurate and complete address list, such as areas where there has been significant new housing development. In selecting an approach to address canvassing, the Bureau is balancing cost and quality trade-offs. According to the Bureau, MAF development must begin to transition from conducting research to inform the design of the 2020 Census to the operational development phase in 2014 and 2015. Several decisions about design options for the 2020 Census need to be made by 2015 so that the Bureau can begin implementing them in time for the 2020 Census. Effective scheduling and collaboration practices are important for the entire effort to stay on track. In our prior work, we recommended the Bureau improve scheduling practices by including estimates of the resources in the schedule.[Footnote 4] The Department of Commerce did not respond to the recommendation at that time. Effective collaboration could present a management challenge for the Bureau. For example, in 2011 the Bureau's Director testified about the need to break down internal barriers to coordination. You asked us to examine the status of scheduling and collaboration in the Bureau's efforts to develop a more cost-effective MAF. We (1) assessed the reliability of the schedules for two key MAF development programs, and (2) examined the extent to which the Bureau is following leading practices for collaboration for its MAF development work. To meet the first objective, we analyzed the GSS-I schedule and the 2020 Research and Testing schedule and their supporting documents against the characteristics of a reliable schedule.[Footnote 5] We also spoke with relevant Bureau officials regarding the GSS-I and 2020 Research and Testing schedules. For the second objective, we reviewed documents related to collaboration including the Bureau's 2020 Strategic Plan and 2013-2017 Strategic Plan, as well as other documents establishing coordination mechanisms and compared them to leading practices for intra-agency collaborative efforts.[Footnote 6] Additionally, we spoke with relevant officials regarding collaboration in the Geography and 2020 Research and Planning Office divisions as well as in the Center for Administrative Records Research and Applications, Decennial Statistical Studies Division, and Field divisions, which are participating in some MAF development activities with the two primary divisions. For more details on our scope and methodology, see appendix I. We conducted our work from April 2013 to November 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: The MAF is a data file that contains a list of all known living quarters in the United States and Puerto Rico. The Bureau uses the MAF to support the decennial census as well as the American Community Survey and other ongoing demographic surveys.[Footnote 7] The MAF contains address information, census geographic location codes, and source and history data. In conjunction with the MAF, the TIGER contains spatial geographical information that allows information from the MAF to be mapped. For the 2010 Census, the Bureau updated the MAF through a complete address canvassing that verified virtually every existing MAF address and added new addresses and deleted those that no longer exist. While full address canvassing helped ensure the accuracy of the address list, we believe it was also very costly. According to Bureau decision documents leading up to the 2010 Census, the Bureau canceled planned research on the feasibility of targeting its canvassing when prioritizing its research agenda early in the decade given its funding levels. As part of the Bureau's effort to conduct the 2020 Census at a cost lower than the 2010 Census, the Bureau is researching the feasibility of conducting targeted address canvassing, verifying addresses in only select areas that are more likely to require updates to the address list. To support targeted address canvassing, the Bureau plans to increase its reliance on other previously used sources of updates, including U.S. Postal Service files, commercial database files, and significant input from state and local governments. For example, GSS-I is working to allow government agencies at all levels to more regularly share and update their address lists with the Bureau throughout the decade (rather than solely 2 years prior to the decennial, as had been the case in prior decennial censuses) so that fewer areas need to be fully canvassed. The life cycle for 2020 Census preparation is divided into five phases, as illustrated in figure 1. The Bureau intends to use the early research and testing phase through fiscal year 2015 to develop a proposal for conducting targeted address canvassing that considers both cost and quality implications. By the end of the early research and testing phase, the Bureau plans to complete decisions about preliminary operational designs rather than continuing critical research and testing until the end of the decade as it did for the 2010 Census. Figure 1: Planned Research and Testing Will Help Inform Future Design Decisions: [Refer to PDF for image: timeline] FY 2009–FY 2011: Options analysis. FY 2012–FY 2015: Early research and testing. June 30, 2015: Submit targeted address canvassing decision to Congress. September 30, 2015: Determine operational design of 2020 Census. FY 2015–FY 2018: Supplemental research and testing; Operational development and systems testing. April 1, 2020: Census Day. December 31, 2020: Delivery of apportionment counts to the President. FY 2019–FY 2023: Readiness testing, execution and close out. Source: GAO analysis of Census Bureau data. Note: FY indicates fiscal year. [End of figure] Because Schedules Are Not Reliable, Management Lacks Valid Information to Assess Progress and Manage Risk: The Bureau faces legally mandated deadlines for delivering census tabulations.[Footnote 8] Effective scheduling is critical for ensuring that the Bureau adheres to a timeline that meets these deadlines. The Bureau relies on schedules to help monitor progress of its many interdependent activities. The schedules are essential to help manage the risks to preparing and implementing a successful decennial census. Certain dates within the schedule could be subject to change or activities may be canceled as a result of time or budget constraints. As dates change from the original schedule or there are significant changes to the work planned, there could be an associated increase in risk as the Bureau may have less time than originally planned to complete future activities in time to make decisions needed to execute the 2020 Census. We determined that a schedule not only provides a road map for systematic execution of a program, but also provides a means by which to gauge progress, identify and address potential problems, and promote accountability. In the GAO Schedule Assessment Guide,[Footnote 9] we identified four characteristics of a reliable schedule. A schedule should be: * Comprehensive: The schedule should identify all activities and resources necessary to accomplish the project. The schedule should cover the scope of work to be performed so that the full picture is available to managers. * Well constructed: Activities should be logically sequenced and critical activities that would affect the timelines of the schedule should be identified. * Credible: All schedules should be linked to a complete master schedule for managers to reference and analyzed for how risk impacts the outcome of the schedule. * Controlled: There should be a documented process for changes to the schedule so that the integrity of the schedule is assured. For a schedule to be reliable, it must substantially or fully meet all criteria for these four characteristics. These characteristics and their criteria are described in more detail in appendix II. We found that the Bureau's 2020 Research and Testing and GSS-I schedules exhibit some of the characteristics of a reliable schedule, yet important weaknesses remain. Each of the schedules substantially met one of the four characteristics (controlled) and minimally or partially met the other three characteristics (comprehensive, well constructed, and credible) (see table 1). Table 1: The Bureau's Schedules for 2020 Research and Testing and the Geographic Support System Initiative Met the Characteristics of a Reliable Schedule to Varying Extents: Characteristic: Comprehensive; 2020 Research and Testing Schedule: Partially Met; GSS-I Schedule: Partially Met. Characteristic: Well constructed; 2020 Research and Testing Schedule: Minimally Met; GSS-I Schedule: Minimally Met. Characteristic: Credible; 2020 Research and Testing Schedule: Minimally Met; GSS-I Schedule: Minimally Met. Characteristic: Controlled; 2020 Research and Testing Schedule: Substantially Met; GSS-I Schedule: Substantially Met. Legend: Fully Met: The Bureau provided complete evidence that satisfies the entire characteristic. Substantially Met: The Bureau provided evidence that satisfies a large portion of the characteristic. Partially Met: The Bureau provided evidence that satisfies about half of the characteristic. Minimally Met: The Bureau provided evidence that satisfies a small portion of the characteristic. Not Met: The Bureau provided no evidence that satisfies any of the characteristic. Source: GAO analysis of Census Bureau schedule data. [End of table] Examples of the extent to which these characteristics were met are provided below. For a more detailed explanation, see appendix III. Comprehensive-Partially Met: The Bureau is using a work breakdown structure to guide the activities of the 2020 Research and Testing and GSS-I schedules. A work breakdown structure defines in detail the work necessary to accomplish a program's objectives. However, not all activities listed in the work breakdown structure are included in the 2020 Research and Testing and GSS-I schedules. For example, in the GSS-I schedule, 20 of the 28 projects have very few activities in them, indicating a lack of detail. Additionally, two MAF-related projects in the 2020 Research and Testing schedule, the MAF Business Rules Improvement project and the Frame Extract Evaluation project, do not have activities assigned to them. If research activities--or any other activities relevant to developing the MAF--are not listed in a schedule, managers may not be able to readily identify causes of delay. According to the Bureau, the schedules are still evolving, these two projects have not yet started or been staffed, and activities and detail will continue to be added to the schedule. For both schedules, the Bureau appeared to record reasonable durations for most activities, helping to ensure that managers can understand the time activities are expected to take and can hold staff who are executing these activities accountable for meeting deadlines. However, neither schedule included information about what levels of resources are required to complete the planned work. Information on resource needs and availability in each work period assists with forecasting the likelihood that activities will be completed as scheduled. Bureau officials stated that they hope to begin the exercise of identifying the resources needed for each activity in both schedules by early 2014 and are waiting for decisions and guidance from the Bureau's effort to standardize cost estimation practices enterprise-wide. In 2012 we recommended, and the Bureau agreed, that the Bureau establish and communicate a timeline for all enterprise activity so that decennial managers can plan accordingly.[Footnote 10] However, the Bureau has not yet produced this timeline. Well Constructed-Minimally Met: In both of the schedules, the Bureau logically linked many of the activities in a sequence. This helps staff identify next steps as they progress through MAF development activities and helps managers identify the impact of changes in one activity on subsequent activities. Yet in both schedules, the Bureau did not identify the preceding and following activity for a number of activities (20 percent for the GSS-I schedule and 9 percent for the 2020 Research and Testing schedule). Scheduling staff were unable to explain why this information was missing. Without this logic, the effect of a change in one activity on future activities cannot be seen in the schedule. For example, in the GSS-I schedule, the "delivery of the targeted address canvassing recommendation report" to managers has no predecessor. According to the Bureau, this report is to outline the research findings, impacts, operational considerations, and benefits of conducting a targeted address canvassing. For those activities that lack predecessors in the schedule, the real effects of changes or delays in preceding activities would not be visible in the schedule, potentially resulting in unforeseen delays in the recommendation report. The Bureau used a large number of constraints which, if used inappropriately, can affect the reliability of the schedule. Activities for which constraints would be justified are Census Day and the delivery of the Apportionment Count, because they have legally mandated deadlines. But, for example, the Bureau also placed a constraint on the delivery of a draft targeted address canvassing report. Such an activity would likely not need a constraint because delays in preceding activities could affect the actual timing of the delivery of the draft report. Placing a constraint on this type of activity would mask in the schedule the effects of any changes or delays that would affect the true delivery date. While our schedule guide states that documenting the justification for constraints is important, the Bureau has not provided justifications in the schedule for its use of constraints. The Bureau told us that justifications are in meeting notes and e-mails, rather than the schedule. If this information is not included in the schedule, the justification for constraints remains unclear to those who did not have access to the meeting notes or e-mails. Also, leaving constraints within the schedule beyond when the schedule is being tested can make the schedule unreliable for other purposes. Additionally, inappropriately used constraints make it difficult to identify the schedules' "critical path"--the sequence of steps needed to achieve the end goal that, if they slip, could negatively affect the overall project completion date. The absence of a critical path or a poorly constructed one calls into question the reliability of the calculated schedule dates, such as estimates of when research results will be available. When certain constraints are placed on an activity, this can automatically trigger the schedule software to place an activity on the calculated critical path when it might otherwise not be. Because the Bureau used so many constraints and the schedule is missing logic about preceding and following activities, it is possible that the calculated critical path includes activities that are not necessarily germane to the true critical path. Eliminating the unnecessary constraints and including additional logic would provide a more accurate picture of the degree of criticality in the schedule. Until the Bureau can produce a true critical path, it will not be able to provide reliable timeline estimates of effects of schedule changes. This undermines the Bureau's ability to focus on activities that will have detrimental effects on the progress of designing targeted address canvassing and other 2020 Census decisions. Finally, a critical path with so many activities appearing on it is not useful to managers in identifying what is truly necessary to develop the MAF in a timely manner. For example, within the 2020 Research and Testing schedule, 52 percent of activities not yet completed appear on the calculated critical path. Similarly, for the GSS-I schedule, 19 percent of the activities appear on the calculated critical path, almost half of which could be because constraints are placed on them. Such a large share of activities appearing on the critical path can reduce the flexibility managers have to complete activities in parallel with each other or to reallocate resources when the same resource is needed for multiple activities on the path. Credible-Minimally Met: The schedules have shortcomings with (1) the integration into management reporting and (2) the ability to automatically change as activities within the schedule change. First, management documents from the 2020 Research and Planning Office indicate the Bureau does not always derive information on milestones from the schedule. For example, two documents dated July 2013 cite the same baseline date from the schedule list major milestones, but the documents indicate a different date for the same part of the research and testing schedule; one states that the research and testing milestones for the current phase will be complete in September 2014, while the other states that these milestones will be completed in September 2015. Bureau managers acknowledged that the planning milestones within the schedule had not been updated to reflect ongoing Bureau management decisions about reprioritizing research and testing plans in light of budget uncertainty during fiscal year 2013. Without keeping the schedule current and using the most recent information to derive information for management such as schedule milestones, there are limited assurances that management is receiving reliable information. Second, we tested the schedules to determine how they changed when dates within the schedule were changed. In our test, the Research and Testing schedule responded automatically to changes in dates of activities, following best practices. However, the GSS-I schedule did not respond in the same way: When we adjusted the date of an activity, subsequent related activities appearing necessary to achieve the milestone did not change, even though the ultimate milestone date changed based on the date shift. More importantly, though, the Bureau is not in a position to carry out systematic quantitative risk analysis on its schedule. A quantitative risk analysis relies on statistical simulation to predict the level of confidence in meeting a program's completion date. The Bureau has identified risks to MAF development efforts, but a quantitative risk analysis would have the advantage of illustrating the impact of risks on the schedule and how that would affect the Bureau's ability to meet milestones and provide a measure of how much time contingency should be built in the schedule to help manage certain risks. Bureau officials said they were waiting for decisions about scheduling software before making decisions about conducting a schedule risk analysis. Without a more credible schedule, the Bureau cannot determine the likelihood that information will be available in time to inform decisions about building the MAF; moreover, the Bureau may not be able to fully understand which risks could affect when information will be available to make decisions and the likelihood that the risks could occur. Controlled-Substantially Met: Both schedules were baselined--creating a comparison schedule to measure, monitor, and report the project's progress--in March 2013, and there is evidence the Bureau has a schedule management process in place and a method for logging changes to the schedule that is in line with best practices. By baselining the schedule, the Bureau helps provide some accountability and transparency to the measurement of the program's progress. The Bureau has implemented a formal change control process which helps ensure the measurement of meaningful progress through comparisons to past versions of the schedule. The Bureau clearly documented its criteria for justifying changes. A team of senior managers is to approve the change and Bureau teams are to acknowledge the change's effect if the schedule indicates they will be affected by the change. The Bureau provides narratives that go along with some schedule updates and includes these in monthly status reports, ensuring that management are informed of schedule changes on a regular basis in accordance with leading practices. This practice helps Bureau officials use their schedules to produce reports that can be used to identify work that should have started or finished by that time. Bureau managers acknowledged that not all changes reflecting Bureau decisions on dealing with budget uncertainty have been processed and reflected yet in the schedule. Yet with processes in place--and being used--that ensure the schedule is updated, management can be reasonably assured that it is looking at current data when examining the schedule, contingent upon the accuracy of the updates. Scheduling Challenges Demonstrate Lack of Expertise among Staff: In conversations with Bureau officials responsible for managing the 2020 Research and Testing and GSS-I schedules, they said that they had not received training or certification in scheduling practices,[Footnote 11] although staff have received training in the software they are using for scheduling and many staff have been trained in project management. The scheduling managers referred to GAO's Schedule Assessment Guide as a key resource for their efforts; however, staff answers to interview questions about leading practices demonstrated a lack of knowledge of the practices. For example, staff explained the presence of the large number of constraints in the schedule they provided to us was related to their occasional "testing" of the schedule, but guidelines for a baselined schedule state that it represents the original configuration of the program plan, and would, thus, not include temporary changes such as the staff described. Both the 2020 Research and Planning Office and the Geography Division have contracted for scheduling support in recent years, and maintain that their contractors have a number of certifications in the advanced use of appropriate software and project management methods. Further, Bureau officials described high turnover and extended vacancies in the management team over the 2020 Research and Planning Office's scheduling contractors and staff until shortly before we began our audit and obtained a copy of their schedule to review. After we completed our audit work at the Bureau, officials told us that subject to the availability of funding, schedule team members will pursue professional certification to further develop and refine their project scheduling skills. Geography Division managers also stressed to us their commitment to schedule management. Having staff and their managers knowledgeable in scheduling practices will likely help avoid many of the deficiencies identified in these schedules. In our past work,[Footnote 12] we found that strategic workforce planning can help align an organization's workforce with the competencies needed to achieve programmatic goals. A key principle for strategic workforce planning includes systematically identifying gaps in competencies in staff with the goal of minimizing or eliminating these gaps. Our prior work has shown that organizations can use methods such as training, contracting, staff development, and hiring to help align skills in order to eliminate gaps in competencies needed for mission success. By conducting a workforce planning process that includes an analysis of skills and training needed, such as what the Bureau describes for its scheduling staff in the future, and the identification of gaps to be addressed, the Bureau can better ensure that staff who manage the schedules understand the leading practices and the importance of adhering to them. Thus, the Bureau can better ensure it has the capacity to develop schedules able to support key management decisions. The Bureau Generally Documented Leading Practices for Collaboration in Its Master Address File Plans: Several divisions are involved in efforts to build the 2020 MAF, making collaboration critical to ensuring that participating divisions work together to achieve the Bureau's goals. In our past work,[Footnote 13] we identified leading practices to foster collaborative relationships across organizational boundaries. We determined that four of these practices were directly relevant to the Bureau's internal efforts to build its MAF. Table 2 identifies and describes these four practices and shows our assessment of the extent to which Bureau documentation demonstrates the Bureau engaged in these leading practices. Table 2: The Bureau Generally Documented Leading Collaboration Practices in Its Plans for Developing a Cost-Effective Master Address File: Leading practice: Define and articulate a common outcome; Characteristics: Collaborating entities must have a clear and compelling rationale to work together, requiring participating staff to define and articulate the common purpose they are seeking to achieve that is consistent with any of their respective divisional goals; Extent documented: Generally documented. Leading practice: Establish mutually reinforcing or joint strategies; Characteristics: To achieve a common outcome, collaborating entities need to establish strategies that work in concert with those of other participants in the collaborative effort or are joint in nature. For the collaborative effort, such strategies help in aligning the participating division's activities, core processes, and resources to accomplish the common outcome; Extent documented: Generally documented. Leading practice: Agree on roles and responsibilities; Characteristics: Collaborating entities should work together to define and agree on their respective roles and responsibilities including how the collaborative effort will be led. In doing so, participating entities can clarify who will do what, organize their joint and individual efforts, and facilitate decision making; Extent documented: Partially documented. Leading practice: Reinforce individual accountability for collaborative efforts through performance management systems; Characteristics: Entities link personal accountability to collaboration by adding a collaboration-related competency or performance standard against which individual performance can be evaluated; Extent documented: Generally documented. Source: GAO analysis of Census Bureau data. [End of table] Define and Articulate a Common Outcome: The Bureau has documented its goals for building a more cost-effective MAF as part of its strategic plans. The Bureau's 2020 Census Strategic Plan set forth Bureau-wide goals for the MAF and the 2020 Census. These goals provide a common rationale for Bureau teams to work across organizational boundaries. Specifically, the Bureau has documented its intention to: * improve the coverage and accuracy of the address list; * continuously update the address list through the decade; and: * improve the cost-effectiveness of the address list. The Geography Division and the 2020 Research and Planning Office have incorporated these common outcomes as part of their individual efforts. Officials in these units indicated an understanding of these goals and communicated them to us. Each also documented these goals in their planning documents. For example, the Geography Division's governance document for GSS-I connects its purpose to working towards building the 2020 MAF. Moreover, in the 2020 Research and Planning Office's Research and Testing management plan, the Bureau sets goals for the division's research associated with improving the accuracy and cost-effectiveness of the MAF. The Bureau, through its strategic plan, has set goals for the 2020 MAF, communicating these to organizational units in a way that will help focus the work in support of upcoming design decisions. Establish Mutually Reinforcing or Joint Strategies: Similar to its goals, the Bureau has established and documented joint strategies as part of its strategic plans. These strategies help outline how the Bureau will achieve the goals of an accurate, continuously updated and cost-effective MAF. The Bureau's 2013-2017 Strategic Plan and 2020 Census Strategic Plan specifically identify these strategies for achieving its goals for the MAF: * implementing targeted address canvassing; * defining components of error in the MAF; * assessing how rules for using addresses contained in the MAF should change to accommodate new address sources; and: * identifying more effective approaches to incorporate addresses from state, local, and tribal governments. The Bureau is executing five research projects related to these strategies. The Bureau is also reinforcing implementation of these strategies by creating new coordination groups as well as placing staff from relevant units on MAF-related research projects. For example, the MAF error model research team has representation from the Geography Division, the Decennial Statistical Studies Division, and the Field Division, among others. The importance of having joint strategies clearly documented is underscored by the differences in perspectives that these divisions can bring to common challenges they may work on together, such as developing a proposal for targeted address canvassing. For example, the Geography Division is responsible for, among other things, administering geographic and cartographic activities needed for the 2020 Census. Its research when working with other teams will focus on geographic concepts, methods, and standards needed for the 2020 Census. Meanwhile, the Field Division, with its responsibility for effectively deploying field personnel to support efficient field data collection, will focus on the "on the ground" feasibility and challenge of targeting certain types of housing units for address canvassing. In addition, coordination bodies are working to share information. In May 2013, relevant Bureau officials began to meet regularly to discuss issues related to implementing targeted address canvassing. Bureau officials involved in these meetings said that the team acts as a vehicle to provide status updates across organizational boundaries. Another team working to identify models to predict where addresses were most in need of being updated was chartered in May 2013. The charter indicated that membership was to include representation from staff in the Field Division and would work with relevant research projects. By defining strategies and reinforcing the collaborative nature of these strategies through such actions as coordination groups and matrixed research teams, the Bureau is helping to align the activities and resources of various divisions to achieve the goals of the 2020 MAF. Agree on Roles and Responsibilities and Participants: The Bureau's 2013-2017 Census Strategic Plan identifies relevant divisions within the Bureau with responsibilities related to developing a more cost-effective 2020 MAF and implementing targeted address canvassing. The 2020 Research and Planning Office has identified the relevant divisions participating in active research projects and coordination groups through documents such as charters. For example, the Targeted Address Canvassing Research, Model, and Area Classification team--a coordination team headed by the Geography Division--was chartered in May 2013 and defines what is both in and out of the scope of its activities. Members are responsible for analyzing potential datasets to be used for targeted address canvassing, but are not responsible for analyzing the costs of targeted address canvassing. In addition, the Bureau recently established memorandums of understanding between the 2020 Research and Planning Office and other relevant divisions, generally finalizing them in May 2013 and signing them in June and July 2013.[Footnote 14] These agreements are not limited to MAF building efforts, but they provide the broad framework for working together and defining coordination. The agreements define the responsibilities of the 2020 Research and Planning Office and the relevant divisions and include provisions for communication between the two organizational units, resource sharing, and modifying agreements as changes in work dictate. However, the Bureau has not taken advantage of some opportunities to use its schedules to reinforce roles or clarify responsibilities. Detailed schedules for 2020 Research and Testing and GSS-I do not completely reflect roles and responsibilities of other divisions or organizational units, such as by reflecting dependencies of activities or handing off to each other. Information on dependencies between projects is available in the project plans for research projects, but such dependencies are not reflected in the schedules. Bureau officials said they would address this by directing project teams to more clearly identify dependencies on various divisions, and review activities to be flagged as having "external" dependencies within the Research and Testing schedule. Reinforce Collaborative Efforts through Performance Management Systems: The Bureau reinforces individual accountability for collaborative activities through individual performance expectations including both broad ones and others specific to MAF development efforts. Bureau- wide, individuals are rated on their "customer service," a work competency that includes their performance working in collaboration with those outside of their division to respond to internal and external needs. Managers we spoke with said that collaboration across units within the Bureau is assessed within this competency. Bureau officials also provided examples of performance management plans where staff were to be rated specifically on collaboration. For example, one staff member was expected to attend interdivisional coordination meetings and to implement new projects based on these meetings. The inclusion of specific performance expectations and metrics dependent on collaborative activity can reinforce synergy across organizational boundaries within the Bureau. This should help ensure that individuals with responsibility for developing the MAF have a vested interest in achieving the overall goals set by the Bureau. As the Bureau moves to testing and implementation, roles and responsibilities will change, and the respective roles of divisions may also change in prominence. Continued management attention to follow leading practices for collaboration will help to ensure that collaboration across units is occurring as the Bureau strives to achieve its goal of a more cost-effective 2020 MAF and Census. Conclusions: Planning efforts related to targeted address canvassing and building a more cost-effective MAF are important to the Bureau's efforts to control the costs of the 2020 decennial. As key design decisions are to be made in the coming years, it is important that the Bureau has a reliable schedule in place upon which management can depend to make those decisions. Our analysis of two Bureau schedules key to MAF development efforts indicates that there are problems with the schedules' reliability. It will be important to ensure that schedules are comprehensive in order for management to be reasonably assured that they have complete information to make decisions. Similarly, problems with the schedules' construction mean that the progression of critical events could be unclear to management. Finally, the schedules lack credibility, meaning that risks, including those the Bureau has already identified, could impact the schedules in ways not yet considered. Some of the identified deficiencies indicate that staff and managers have not been available and prepared to sufficiently construct and maintain the schedules. Conducting a workforce planning process of staff working on MAF schedules could help the Bureau to identify staff skills needed to help ensure related gaps are addressed. Without staff knowledge of the leading practices and the importance of adhering to them, the schedules may prove problematic for decennial managers' ability to assess progress, make decisions, identify future risks, or anticipate potential delays. With its planning documents, memorandums of understanding, and various charters, the Bureau has put in place a framework to support collaborative efforts following leading practices, particularly in recent months, which will aid the efforts. These methods could be bolstered by building collaboration into the schedule. By improving practices in the area of constructing a schedule, the Bureau can help address these gaps. As the Bureau continues its implementation efforts up to and beyond key decisions about how to build a cost-effective MAF, it is vital to ensure that the practices incorporated into Bureau planning documents and processes thus far are continued. Recommendations for Executive Action: To help maintain a more thorough and insightful 2020 Census development schedule in order to better manage risks to a successful 2020 Census, the Secretary of Commerce and Undersecretary of Economic Affairs should direct the U.S. Census Bureau to improve its scheduling practices in three areas: * the comprehensiveness of schedules, including ensuring that all relevant activities are included in the schedule; * the construction of schedules, including ensuring complete logic is in place to identify the preceding and subsequent activities as well as a critical path that can be used to make decisions; and: * the credibility of schedules, including conducting a quantitative risk assessment. In addition, we recommend that the Director of the U.S. Census Bureau initiate a robust workforce planning process for those working on schedules related to the Master Address File, including actions such as an analysis of skills needed, to identify and address gaps in scheduling skills. Agency Comments and Our Evaluation: We provided a draft of this report to the Department of Commerce and received the department's written comments on November 5, 2013. The comments are reprinted in appendix IV. The Department of Commerce concurred with our findings and recommendations and provided several clarifications, which are reflected in this report as appropriate. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of Commerce, the Under Secretary of Economic Affairs, the Director of the U.S. Census Bureau, and interested congressional committees. The report also is available at no charge on GAO's website at [hyperlink, http://www.gao.gov]. If you have any questions about this report please contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. The GAO staff that made major contributions to this report are listed in appendix V. Sincerely yours, Signed by: Robert Goldenkoff: Director: Strategic Issues: [End of section] Appendix I: Objectives, Scope, and Methodology: This report (1) assesses the reliability of the schedules for two key Master Address File (MAF) development programs, and (2) examines the extent to which the Census Bureau (Bureau) is following leading practices for collaboration for its MAF development work. To determine the extent to which the Bureau is following leading practices for scheduling as identified in the GAO Schedule Assessment Guide,[Footnote 15] we analyzed the Geographic Support System Initiative (GSS-I) schedule and the 2020 Research and Planning Office (Research and Testing) schedule. We scored each scheduling best practice on a five-point scale ranging from "not met" to "fully met." To determine the extent to which the Bureau's key efforts to build a cost-effective MAF/Topologically Integrated Geographic Encoding and Referencing (MAF/TIGER) incorporate leading practices for collaboration, we identified leading practices to apply to intra- agency collaborative efforts based on our past work on leading collaboration practices.[Footnote 16] We identified organizational units and activities relevant to building a cost-effective MAF in consultation with the Bureau. We also identified documentation of their research projects. We reviewed key management documents for content pertaining to collaboration, including the Bureau's strategic plan for the decennial and current (2013-2017) strategic plan. In addition, we reviewed documents directly addressing coordination efforts, such as charters and meeting minutes from coordination groups and memorandums of understanding between divisions. We compared the documented plans and activities to best practices in order to rate the extent to which leading practices were incorporated or were intended to be incorporated into Bureau documents. We rated each practice on a three-point scale from "Not Documented" to "Generally Documented." * Not Documented: The Bureau provided no documentary evidence that satisfies any of the criteria. * Partially Documented: The Bureau provided documentary evidence that satisfies a portion of the criteria. * Generally Documented: The Bureau provided documentary evidence that satisfies all or nearly all of the criteria. We then interviewed Bureau officials in the Geography and 2020 Research and Planning Office divisions to discuss schedules and their collaboration efforts. Additionally, regarding scheduling and collaboration, we spoke with relevant officials in the Center for Administrative Records Research and Applications, Decennial Statistical Studies Division, and Field Division. These divisions are participating in some MAF development activities with the Geography and 2020 Research and Planning Office divisions. Our review of scheduling and collaboration practices was limited to 2020 Decennial Census activities and focused on MAF development activities and cannot be generalized to other, non-decennial Bureau activities and operations. [End of section] Appendix II: Description of Scheduling: Characteristic: Comprehensive: Best practice: Capturing all activities; Description: A schedule should reflect all activities defined in the project's work breakdown structure and include all activities to be performed by the government and contractor. Best practice: Assigning resources to all activities; Description: The schedule should realistically reflect the resources (i.e., labor, material, and overhead) needed to do the work, whether all required resources will be available when needed, and whether any funding or time constraints exist. Best practice: Establishing the durations of all activities; Description: The schedule should reflect how long each activity will take to execute. Characteristic: Well constructed: Best practice: Sequencing all activities; Description: The schedule should be planned so that all activities are logically sequenced in the order they are to be carried out. Best practice: Confirming that the critical path is valid; Description: The schedule should identify the critical path, or those activities that, if delayed, will negatively impact the overall project completion date. The critical path enables analysis of the effect delays may have on the overall schedule. Best practice: Ensuring reasonable total float; Description: The schedule should identify float--the amount of time an activity can slip in the schedule before it affects other activities-- so that flexibility in the schedule can be determined. As a general rule, activities along the critical path have the least amount of float. Characteristic: Credible: Best practice: Verifying that the schedule is traceable horizontally and vertically; Description: The detailed schedule should be horizontally traceable, meaning that it should link products and outcomes associated with other sequenced activities. The integrated master schedule should also be vertically traceable--that is, varying levels of activities and supporting subactivities can be traced. Such mapping or alignment of levels enables different groups to work to the same master schedule. Best practice: Conducting a schedule risk analysis; Description: The schedule should include a schedule risk analysis that uses statistical techniques to predict the probability of meeting a completion date. A schedule risk analysis can help management identify high priority risks and opportunities. Characteristic: Controlled: Best practice: Updating the schedule with actual progress and logic; Description: Progress updates and logic provide a realistic forecast of start and completion dates for program activities. Maintaining the integrity of the schedule logic at regular intervals is necessary to reflect the true status of the program. To ensure that the schedule is properly updated, people responsible for updating should be trained in critical path method scheduling. Best practice: Maintaining a baseline schedule; Description: A baseline schedule represents the original configuration of the program plan and is the basis for managing the project scope, the time period for accomplishing it, and the required resources. Comparing the current status of the schedule to the baseline can help managers target areas for mitigation. Source: Exposure Draft GAO-12-120G. [End of table] [End of section] Appendix III: Assessment of the Extent to Which the Bureau Followed Scheduling Best Practices: Assessment of the Bureau's Research and Testing Program Schedule: Characteristic: Comprehensive; Overall assessment: Partially Met; Best practice: Capturing all activities; Individual assessment: Partially Met. Best practice: Assigning resources to all activities; Individual assessment: Not Met. Best practice: Establishing the durations of all activities; Individual assessment: Substantially Met. Characteristic: Well constructed; Overall assessment: Minimally Met; Best practice: Sequencing all activities; Individual assessment: Partially Met. Best practice: Confirming that the critical path is valid; Individual assessment: Minimally Met. Best practice: Ensuring reasonable total float; Individual assessment: Minimally Met. Characteristic: Credible; Overall assessment: Minimally Met; Best practice: Verifying that the schedule is traceable horizontally and vertically; Individual assessment: Partially Met. Best practice: Conducting a schedule risk analysis; Individual assessment: Not Met. Characteristic: Controlled; Overall assessment: Substantially Met; Best practice: Updating the schedule with actual progress and logic; Individual assessment: Substantially Met. Best practice: Maintaining a baseline schedule; Individual assessment: Substantially Met. Legend: Fully Met: The Bureau provided complete evidence that satisfies the entire criteria. Substantially Met: The Bureau provided evidence that satisfies a large portion of the criteria. Partially Met: The Bureau provided evidence that satisfies about half of the criteria. Minimally Met: The Bureau provided evidence that satisfies a small portion of the criteria. Not Met: The Bureau provided no evidence that satisfies any of the criteria. Source: GAO analysis of Census Bureau schedule data. [End of table] Assessment of the Bureau's Geographic Support System Program Schedule: Characteristic: Comprehensive; Overall assessment: Partially Met; Best practice: Capturing all activities; Individual assessment: Partially Met. Best practice: Assigning resources to all activities; Individual assessment: Not Met. Best practice: Establishing the durations of all activities; Individual assessment: Substantially Met. Characteristic: Well constructed; Overall assessment: Minimally Met; Best practice: Sequencing all activities; Individual assessment: Partially Met. Best practice: Confirming that the critical path is valid; Individual assessment: Minimally Met. Best practice: Ensuring reasonable total float; Individual assessment: Minimally Met. Characteristic: Credible; Overall assessment: Minimally Met; Best practice: Verifying that the schedule is traceable horizontally and vertically; Individual assessment: Partially Met. Best practice: Conducting a schedule risk analysis; Individual assessment: Not Met. Characteristic: Controlled; Overall assessment: Substantially Met; Best practice: Updating the schedule with actual progress and logic; Individual assessment: Substantially Met. Best practice: Maintaining a baseline schedule; Individual assessment: Substantially Met. [End of table] Legend: Fully Met: The Bureau provided complete evidence that satisfies the entire criteria. Substantially Met: The Bureau provided evidence that satisfies a large portion of the criteria. Partially Met: The Bureau provided evidence that satisfies about half of the criteria. Minimally Met: The Bureau provided evidence that satisfies a small portion of the criteria. Not Met: The Bureau provided no evidence that satisfies any of the criteria. Source: GAO analysis of Census Bureau schedule data. [End of table] [End of section] Appendix IV: Comments from the Department of Commerce: The Deputy Secretary of Commerce: Washington, D.C. 20230: November 5, 2013: Mr. Robert Goldenkoff: Director: Strategic Issues: United States Government Accountability Office: Washington, DC 20548: Dear Mr. Goldenkoff: The U.S. Department of Commerce appreciates the opportunity to comment on the United States Government Accountability Office's draft report titled 2020 Census: Bureau Needs to Improve Scheduling Practices to Enhance Ability to Meet Address List Development Deadlines (GA0-14- 59). The Department's comments on this report are enclosed. Sincerely, Signed by: Patrick Gallagher: Acting Deputy Secretary: Enclosure: U.S. Department of Commerce: Comments on the United States Government Accountability Office Draft Report titled 2020 Census: Bureau Needs to Improve Scheduling Practices to Enhance Ability to Meet Address List Development Deadlines (GA0-14-59): October 2013: The U.S. Census Bureau appreciates the opportunity to review this draft report. We have no substantive disagreements with the findings or recommendations, but do have the following comments and suggestions for wording changes. 1. On page 4, the report states, "As part of the Bureau's effort to conduct the 2020 Census at a cost lower than the 2010 Census, the Bureau plans to adopt targeted address canvassing ..." To say we plan to do so may be too emphatic; we would appreciate if GAO could rephrase the statement to "the Census Bureau is currently conducting research into the feasibility of conducting a targeted address canvassing, and the GSS-I and its goal of continuously improving the MAF are intended to support the feasibility of a targeted address canvassing." 2. On Page 5, Figure 1 should show a design decision deadline of September 2015, not September 2016. 3. The wording at the top of page 10 currently reads, "As we were completing our audit, Geography Division officials stated that they have begun reviewing constraints in the GSS-I schedule for the purpose of reducing their number." We request this wording be revised to state, "As we were completing our audit, the Geography Division has begun reviewing their use of constraints within the schedule and indicate this may, or may not, lead to a reduced level of what the GAO has identified as 'unnecessary' constraints." 4. On page 7, the draft states that the Frame Extract Evaluation and MAF/TIGER Business Rules Improvement projects do not have any activities assigned to them in the schedule. While this statement is true, the lack of assigned activities is due to the fact that those projects do not start up until FY 2014; therefore, no team has yet been established to develop an activity schedule. While the high-level project plans show some dates, these dates are preliminary and have not been reviewed or analyzed by experts. Additionally, these dates have not been reassessed based on recent budget cuts and program changes. 5. On pages 11 - 12 of the draft report, the updated explanation as to training provided in the section titled "Scheduling challenges demonstrate lack of expertise among staff' is accurate. However, the wrap-up statement in the "What GAO Found" introduction does not convey the same meaning as to the Census Bureau's commitment to training. * Currently, the wording reads as follows: "Staff managing the schedules said that they had not received training or certification in scheduling practices, and, according to the schedule manager, staff turnover contributed to the issues we identified." * We request this wording be modified to "The Census Bureau has provided program management training that emphasizes schedule management, as well as additional schedule management tool-specific training, but staff still lack a thorough knowledge of scheduling best practices." 6. Page 19 refers to the Acting Secretary of Commerce and the Acting Director of the U.S. Census Bureau. The Department no longer has acting officials in these positions, but now has both a Secretary and a Director of the Census Bureau. [End of section] Appendix V: GAO Contact and Staff Acknowledgment: GAO Contact: Robert Goldenkoff, (202) 512-2757 or goldenkoffr@gao.gov: Staff Acknowledgments: Other key contributors to this report include Ty Mitchell, Assistant Director; Tom Beall; Juaná Collymore; Rob Gebhart; David Hulett; Andrea Levine; Jeffrey Niblack; Karen Richey; and Timothy Wexler. [End of section] Footnotes: [1] GAO, 2020 Census: Initial Research Milestones Generally Met but Plans Needed to Mitigate Highest Risks, [hyperlink, http://www.gao.gov/products/GAO-13-53] (Washington, D.C.: Nov. 7, 2012). [2] In constant 2010 dollars. [3] The four additional units are the Center for Administrative Records Research and Applications, the Decennial Statistical Studies Division, the Geography Division, and the Field Division. [4] GAO, 2010 Census: Census Bureau Has Made Progress on Schedule and Operational Control Tools, but Needs to Prioritize Remaining System Requirements, [hyperlink, http://www.gao.gov/products/GAO-10-59] (Washington, D.C.: Nov. 13, 2009). [5] GAO, GAO Schedule Assessment Guide: Best Practices for Project Schedules, Exposure Draft [hyperlink, http://www.gao.gov/products/GAO-12-120G] (Washington, D.C.: May 2012). Underlying these characteristics are 10 leading practices, which are described in appendix II. These characteristics and leading practices were developed in 2012 based on our practices for creating a reliable cost estimate and in consultation with experts from the scheduling community. [6] GAO, Managing for Results: Key Considerations for Implementing Interagency Collaborative Mechanisms, [hyperlink, http://www.gao.gov/products/GAO-12-1022] (Washington, D.C.: Sept. 27, 2012); GAO, Results-Oriented Government: Practices That Can Help Sustain Collaboration Among Federal Agencies, [hyperlink, http://www.gao.gov/products/GAO-06-15] (Washington, D.C.: Oct. 21, 2005). In 2012, we developed key considerations for collaboration using our past work, including GAO-06-15, as well as literature reviews and interviews with government experts. [7] The American Community Survey (ACS) is an ongoing statistical survey that provides data every year. Information from the survey generates data that help determine how federal and state funds are distributed each year. [8] The Bureau's legally mandated deadlines include Census Day, which is April 1, and the delivery of apportionment counts to the President, which must be done by December 31. 13 U.S.C. § 141. [9] Exposure Draft [hyperlink, http://www.gao.gov/products/GAO-12-120G]. [10] GAO, 2020 Census: Additional Steps Are Needed to Build on Early Planning, [hyperlink, http://www.gao.gov/products/GAO-12-626] (Washington, D.C.: May 17, 2012). [11] Several scheduling certifications are available by professional organizations. Examples include the Planning and Scheduling Professional certificate from the Association for the Advancement of Cost Engineering or the Scheduling Professional certification from the Project Management Institute. [12] GAO, Human Capital: Key Principles for Effective Strategic Workforce Planning, [hyperlink, http://www.gao.gov/products/GAO-04-39] (Washington, D.C.: Dec. 11, 2003). We developed the key principles of workforce planning by reviewing documents from organizations with expertise in workforce planning models and federal agencies with promising workforce planning practices, as well as our past work. [13] GAO, Managing for Results: Key Considerations for Implementing Interagency Collaborative Mechanisms, [hyperlink, http://www.gao.gov/products/GAO-12-1022] (Washington, D.C.: Sept. 27, 2012); GAO, Results-Oriented Government: Practices That Can Help Sustain Collaboration Among Federal Agencies, [hyperlink, http://www.gao.gov/products/GAO-06-15] (Washington, D.C.: Oct. 21, 2005). [14] The 2020 Research and Planning Office separately finalized and signed a memorandum of understanding with the Director for Information Technology and the Chief Information Officer in October 2012. [15] GAO, GAO Schedule Assessment Guide: Best Practices for Project Schedules, Exposure Draft [hyperlink, http://www.gao.gov/products/GAO-12-120G] (Washington, D.C.: May 2012). Underlying these characteristics are 10 leading practices, which are described in appendix II. These characteristics and leading practices were developed in 2012 based on our practices for creating a reliable cost estimate and in consultation with experts from the scheduling community. [16] GAO, Managing for Results: Key Considerations for Implementing Interagency Collaborative Mechanisms, [hyperlink, http://www.gao.gov/products/GAO-12-1022] (Washington, D.C.: Sept. 27, 2012); Results-Oriented Government: Practices That Can Help Sustain Collaboration Among Federal Agencies, [hyperlink, http://www.gao.gov/products/GAO-06-15] (Washington, D.C.: Oct. 21, 2005). In 2012, we developed key considerations for collaboration using our past work, [hyperlink, http://www.gao.gov/products/GAO-06-15], as well as literature reviews and interviews with government experts. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]