This is the accessible text file for GAO report number GAO-05-888 
entitled 'Department of Homeland Security: Strategic Management of 
Training Important for Successful Transformation' which was released on 
September 23, 2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Chairman, Subcommittee on Oversight of Government 
Management, the Federal Workforce, and the District of Columbia, 
Committee on Homeland Security and Governmental Affairs, U.S Senate: 

United States Government Accountability Office: 

GAO: 

September 2005: 

Department of Homeland Security: 

Strategic Management of Training Important for Successful 
Transformation: 

GAO-05-888: 

GAO Highlights: 

Highlights of GAO-05-888, a report to the Chairman, Subcommittee on 
Oversight of Government Management, the Federal Workforce, and the 
District of Columbia, U.S. Senate: 

Why GAO Did This Study: 

Training can play a key role in helping the Department of Homeland 
Security (DHS) successfully address the challenge of transformation and 
cultural change and help ensure that its workforce possesses the 
knowledge and skills needed to effectively respond to current and 
future threats. This report discusses (1) how DHS is addressing or 
planning to address departmentwide training and the related challenges 
it is encountering; (2) examples of how DHS training practices, 
specifically those related to planning and evaluation, reflect 
strategic practices; and (3) examples of how DHS uses training to 
foster transformation and cultural change. 

What GAO Found: 

DHS has taken several positive steps toward establishing an effective 
departmentwide approach to training, yet significant challenges remain. 

Progress made in addressing departmentwide training issues, but efforts 
are still in the early stages and face several challenges. Actions 
taken by DHS include issuing its first training strategic plan in July 
2005, establishing training councils and groups to increase 
communication across components, and directly providing training for 
specific departmentwide needs. However, several challenges may impede 
DHS from achieving its departmental training goals. First, the sharing 
of training information across components is made more difficult by the 
lack of common or compatible information management systems and a 
commonly understood training terminology. Second, authority and 
accountability relationships between the Office of the Chief Human 
Capital Officer and organizational components are not sufficiently 
clear. Third, DHS’s planning may be insufficiently detailed to ensure 
effective and coordinated implementation of departmentwide training 
efforts. Finally, according to training officials, DHS lacks resources 
needed to implement its departmental training strategy. 

Examples of planning and evaluation of training demonstrate some 
elements of strategic practice. Specific training practices at both the 
component and departmental levels may provide useful models or insights 
to help others in DHS adopt a more strategic approach to training. We 
found that some components of DHS apply these practices, while others 
do not. For example, Customs and Border Protection (CBP) aligns 
training priorities with strategic goals through planning and budgeting 
processes. In the area of evaluation, the Federal Law Enforcement 
Training Center obtains feedback from both the trainee and the 
trainee’s job supervisor to inform training program designers in order 
to make improvements to the program curriculum. 

Training has been used to help DHS’s workforce as it undergoes 
transformation and cultural change. The creation of DHS from different 
legacy organizations, each with its own distinct culture, has resulted 
in significant cultural and transformation challenges for the 
department. At the departmental level, one of the ways DHS is 
addressing these challenges is by encouraging the transformation to a 
shared performance-based culture through the implementation of its new 
human capital management system, MAXHR. DHS considers training to be 
critical to effectively implementing this initiative and defining its 
culture. Toward that end, the department is providing a wide range of 
training, including programs targeted to executives, managers, and 
supervisors. For example, at the component level, CBP has developed 
cross-training to equip employees with the knowledge needed to 
integrate inspection functions once carried out by three different 
types of inspectors at three separate agencies. 

What GAO Recommends: 

We recommend that the Secretary of Homeland Security (1) adopt 
additional good planning practices, specifically, creating a clearer 
crosswalk between training goals and DHS’s organizational and human 
capital strategic goals and developing appropriate performance measures 
and targets; (2) specify authority/accountability relationships 
between CHCO and components on training; (3) ensure the department and 
components develop detailed training implementation plans; and (4) when 
setting funding priorities, give appropriate attention to supporting 
training councils and groups. DHS generally agreed with the report’s 
recommendations. 

www.gao.gov/cgi-bin/getrpt?GAO-05-888. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact George Stalcup at (202) 
512-9490 or stalcupg@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

DHS Has Made Progress in Addressing Departmentwide Training Issues, but 
Challenges May Impede Its Efforts: 

DHS's Approach to Planning and Evaluation of Training Demonstrates Some 
Elements of Strategic Practice, but Is Still Evolving: 

DHS Has Used Training in an Effort to Help Its Workforce as the 
Department Undergoes Transformation and Cultural Change: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments: 

Appendix I: Scope and Methodology: 

Appendix II: "One Face at the Border" Training at U.S. Customs and 
Border Protection: 

Appendix III: Comments from the Department of Homeland Security: 

Table: 

Table 1: Responsibilities and Selected Training Characteristics of DHS 
Components Included in Our Review: 

Figures: 

Figure 1: DHS Organizational Structure: 

Figure 2: Departmental Training Councils and Subgroups at DHS: 

Figure 3: Timeline for MAXHR Training Scheduled Through the End of 
2005: 

Figure 4: Levels of Training Evaluation: 

Figure 5: DHS Components Included in Our Review: 

Abbrevations: 

ADL: Advanced Distributed Learning: 

ADLG: Advanced Distributed Learning Group: 

CBP: U.S. Customs and Border Protection: 

CHCO: Chief Human Capital Officer: 

CIS: U.S. Citizenship and Immigration Services: 

DHS: Department of Homeland Security: 

FLETC: Federal Law Enforcement Training Center: 

FTE: full-time equivalent: 

HPT: Human Performance Technology: 

ICE: U.S. Immigration and Customs Enforcement: 

LTDG: Leadership Training and Development Group: 

NTEP: National Training Evaluation Program: 

SES: Senior Executive Service: 

TLC: Training Leaders Council: 

TSA: Transportation Security Administration: 

United States Government Accountability Office: 

Washington, DC 20548: 

September 23, 2005: 

The Honorable George V. Voinovich: 
Chairman: 
Subcommittee on Oversight of Government Management, the Federal 
Workforce, and the District of Columbia: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

Dear Mr. Chairman: 

Addressing an organization's culture--that is, its underlying 
assumptions, beliefs, values, attitudes, and expectations--is at the 
heart of any serious organizational transformation or change management 
initiative. As the Department of Homeland Security (DHS) strives to 
protect the nation from terrorism, it faces significant challenges in 
transforming from 22 separate agencies and programs to a single 
coordinated department, requiring the integration of approximately 
180,000 employees as well as multiple management systems and processes. 
In recognition of these challenges, we have designated the 
implementation and transformation of the department as high 
risk.[Footnote 1]

Training and development can play a key role in helping DHS 
successfully address the challenge of transformation and cultural 
change and help ensure that its workforce possesses the knowledge, 
skills, and competencies needed to effectively respond to current and 
future threats.[Footnote 2] In addition, a strategic approach to the 
management of training can help to effectively target limited resources 
and further the achievement of its organizational goals. To this end, 
our framework for assessing training management provides a way for DHS 
to recognize and develop such an approach.[Footnote 3]

You asked us to examine how DHS, as a federal agency undergoing 
transformation, uses training to help achieve its organizational goals. 
In response to your request, this report discusses (1) how DHS is 
addressing or planning to address departmentwide training and the 
related challenges it is encountering; (2) examples of how DHS training 
practices, specifically those related to planning and evaluation, 
reflect strategic practices; and (3) examples of how DHS uses training 
to foster transformation and cultural change. 

To achieve our objectives, we reviewed internal training at both the 
departmental and component levels, focusing on the systems and 
processes in place to manage, plan, and evaluate training for DHS's 
workforce. To this end, we analyzed training, management, and planning 
documents and interviewed numerous officials responsible for training 
issues in the Office of the Chief Human Capital Officer (CHCO office) 
and at six organizational components collectively responsible for 
training 95 percent of the DHS workforce. To determine whether DHS used 
a strategic approach in managing, planning, and evaluating its training 
activities, we drew on our previous work regarding strategic planning 
and effective management practices, as well as criteria contained in 
our guide for assessing strategic training and development efforts in 
the federal government. We recognize that DHS provides a significant 
amount of training to external audiences, such as state and local 
governments; however, given the nature of your request and interest in 
examples of how DHS is using training to foster its organizational 
transformation, we did not include training intended for audiences 
external to DHS within the scope of our review. For more information on 
our scope and methodology, see appendix I. 

We conducted our work from November 2004 through July 2005 in 
accordance with generally accepted government auditing standards. 

Results in Brief: 

DHS has made progress in addressing departmentwide training issues, but 
these efforts are still in the early stages and challenges may impede 
these efforts. Thus far, DHS has taken several steps toward 
establishing a departmentwide approach to training, including the 
following: 

* DHS has issued its first training strategic plan providing a 
strategic vision for departmental training. It used a collaborative and 
inclusive process to develop the plan, consulting with component 
training leaders and others. 

* It also has established training councils and groups with the goal of 
increasing communication across components and fostering greater 
collaboration and coordination. 

* Finally, it has provided training targeted to address specific 
departmentwide needs. Examples of areas where DHS has directly provided 
or supported training on the departmental level include (1) 
implementation of MAXHR, DHS's new human capital management 
system;[Footnote 4] (2) leadership development; and (3) civil rights 
and liberties. 

However, we identified four challenges that may impede the department 
from achieving departmental training goals. First, the sharing of 
training information across components is hampered by the lack of a 
common or compatible information management infrastructure and the 
absence of a commonly understood terminology. Officials told us that 
the lack of compatible information technology systems complicated their 
efforts to make the most efficient use of training resources across 
components. Second, authority and accountability relationships between 
the CHCO office and the organizational components are not sufficiently 
clear. A clear and agreed-upon understanding of the specific 
responsibilities and authorities of the key organizations involved in 
training should significantly improve DHS's ability to effectively 
implement its training strategies. The department recognizes this need 
to clarify the responsibilities and authorities of the CHCO office and 
the components and has addressed this need in its training strategic 
plan. Third, DHS's planning may be insufficiently detailed to ensure 
effective and coordinated implementation of departmentwide training 
efforts. Because they share authority for training, the department and 
the components need to develop detailed implementation plans to help 
ensure that departmentwide training initiatives are coordinated and 
effectively implemented. Fourth, according to training officials, DHS 
lacks resources needed to implement its departmental training strategy. 

While still evolving, some of DHS's training practices at both the 
component and department levels demonstrate strategic elements in the 
areas of planning and evaluation and may provide useful models or 
insights to help others in DHS adopt a more strategic approach to 
training. We have reported previously that as part of the planning 
process, agencies demonstrating a strategic approach to training align 
their training efforts with overall organizational strategic 
priorities; some components of DHS apply these practices, while others 
do not. U.S. Customs and Border Protection (CBP), for example, employs 
practices intended to align training priorities with strategic goals 
through planning and budgeting processes. The U.S. Coast Guard also 
demonstrates a strategic approach by using a process for determining 
whether training is the appropriate intervention to address a specific 
performance problem. 

With respect to evaluation, we have reported that agencies 
demonstrating a strategic approach to training employ a variety of 
practices, such as systematically evaluating training, actively 
incorporating feedback during training design, and obtaining feedback 
from multiple perspectives. Several components and programs we examined 
at DHS demonstrate these practices, while others do not. For example, 
the Federal Law Enforcement Training Center (FLETC) obtains feedback 
from both the trainee and the trainee's job supervisor to inform 
training program designers making improvements to the curriculum. CBP 
and the CHCO office also use various strategies to evaluate training 
programs and take appropriate actions. 

DHS also has used training to help foster transformation and cultural 
change as the department moves from multiple distinct organizational 
cultures to a new culture that endeavors to be increasingly integrated 
and performance focused. This is not an easy process, and the creation 
of DHS from 22 different agencies and programs has resulted in 
considerable cultural and transformation challenges. At the department 
level, DHS has addressed these challenges by encouraging the 
transformation to a shared performance-based culture through the 
adoption of a new human capital management system, known as MAXHR. As 
an essential part of implementing this initiative, DHS developed 
targeted training for executives, managers, and supervisors, providing 
these groups with the tools and information needed to champion the 
benefits of a performance-based culture and successfully implement 
MAXHR in their components. In another example, this time at the 
component level, in order to improve coordination and communication 
across inspection functions and enhance flexibility of the workforce, 
CBP created the new positions of CBP officer and CBP agriculture 
specialist. Cross-training of employees in these new positions helped 
CBP to integrate the inspection functions of three former agencies. In 
addition, CBP designed and piloted a training module that specifically 
targeted how supervisors could more effectively understand the value 
and perspective of staff coming from legacy organizational cultures. 

To help DHS further establish and implement an effective and strategic 
approach to departmental training, we recommend that the Secretary of 
Homeland Security take the following actions: (1) adopt additional good 
strategic planning and management practices to enhance the department's 
training strategic plan by creating a clearer crosswalk between 
specific training goals and objectives and DHS's organizational and 
human capital strategic goals and developing appropriate performance 
measures and targets; (2) clearly specify authority and accountability 
relationships between the CHCO office and organizational components 
regarding training, as a first step toward addressing issues DHS has 
identified for fiscal year 2006; (3) ensure that the department and 
component organizations develop detailed implementation plans and 
related processes for training initiatives; and (4) when setting 
funding priorities, give appropriate attention to providing resources 
to support training councils and groups to further DHS's capacity to 
achieve its departmentwide training goals. 

We provided a draft of this report to the Secretary of Homeland 
Security for comment. DHS generally agreed with our recommendations. 
The department provided technical comments that we incorporated where 
appropriate. DHS's written response is reprinted in appendix III. 

Background: 

The Homeland Security Act of 2002 created DHS, bringing together 22 
agencies and programs responsible for important aspects of homeland 
security.[Footnote 5] The intent behind the creation of a single 
department was to improve coordination, communication, and information 
sharing among these previously separate entities, thereby increasing 
their effectiveness in protecting the nation's security. Each of these 
organizations brought with it the capacity and expertise to provide 
training for its particular aspect of homeland security. For example, 
in several cases such as the Coast Guard and FLETC, this training 
capacity, as well as the management systems supporting it was 
transferred intact with the creation of the new department. In other 
cases, such as CBP and U.S. Immigration and Customs Enforcement (ICE), 
the training functions of legacy organizations were merged. Table 1 
presents information on selected training characteristics of components 
in our review, including the origin of each component's training 
function. In addition, the Act led to the creation of the CHCO position 
in DHS responsible for, among other human capital topics, oversight and 
planning of the training of employees.[Footnote 6] The CHCO, who 
reports directly to the department's Under Secretary for Management, 
has primary responsibility for defining and developing the department's 
role regarding training. Figure 1 depicts these positions as well as 
the department's major components in the context of DHS's overall 
organizational structure. 

Training both new and current staff to fill new roles and work in 
different ways will play a crucial part in the ability of federal 
departments and agencies, such as DHS, as they work to successfully 
transform their organizations. In 2004, we issued an assessment guide 
that introduces a framework for evaluating the management of training 
in the federal government.[Footnote 7] As presented in our guide, the 
training process can be segmented into four broad, interrelated phases: 
(1) planning/front-end analysis, (2) design/development, (3) 
implementation, and (4) evaluation. For each of these phases, we 
summarize key attributes of effective training programs and offer 
related issues and questions. Using this framework, this report 
identifies selected strategic training practices, with a focus on the 
planning and evaluation phases, that may offer an opportunity for 
others in DHS to build on experiences and practices discussed below. 

Table 1: Responsibilities and Selected Training Characteristics of DHS 
Components Included in Our Review: 

U.S. Citizenship and Immigration Services (CIS); 10,207 FTEs; 
Overview of component responsibilities: 
* Administer immigration and naturalization adjudication functions, 
including immigrant visa petitions, naturalization petitions, and 
asylum and refugee applications; 
* Establish and administer policies for immigrant services and 
benefits; 
Selected characteristics of training: 
* Training function was transferred from former Immigration and 
Naturalization Service with the exception of law enforcement and soft 
skills training; 
* Conducts extensive field training through district offices and 
service centers; 
* Operates CIS Academy at FLETC site in Glynco, GA, for basic training 
of new employees; 
* Provides refresher training for adjudicators at field sites; 
* Supervisory training provided by ICE's Leadership Development Center 
in Dallas, TX, through a shared services agreement between CIS, ICE, 
and CBP. 

U.S. Coast Guard; 46,847 FTEs; 
Overview of component responsibilities: 
* Protect the public, environment, and U.S. economic interests in 
nation's ports, waterways, coasts, and international waters; 
* Specific responsibilities include: Maritime safety (e.g., search and 
rescue), maritime mobility (e.g., aids to navigation and waterways 
management), protection of natural resources, maritime security (e.g., 
drug interdiction), and national defense; 
Selected characteristics of training: 
* Training function transferred as a whole along with the rest of the 
Coast Guard from Department of Transportation; 
* Operates multiple training programs for (1) indoctrination for new 
employees, (2) apprenticeship after indoctrination course is completed, 
and (3) specialized skills (e.g., law enforcement inspections); 
* Operates special leadership training program at Leadership 
Development Center, New London, CT; 
* Uses some Department of Defense training courses for specialized 
skills. 

U.S. Customs and Border Protection (CBP); 40,761 FTEs; 
Overview of component responsibilities: 
* Protect U.S. borders from terrorism, at and between official ports of 
entry, and foster legitimate trade and travel; 
Selected characteristics of training: 
* Training function structure and processes largely carried over from 
former U.S. Customs organization; 
* Basic CBP officer training provided at CBP Academy at FLETC site in 
Glynco, GA, and basic CBP agricultural specialist training at CBP 
Academy at U.S. Department of Agriculture-Professional Development 
Center in Frederick, MD; 
* Provides basic Border Patrol training at Border Patrol Academy in 
Artesia, NM; 
* Provides extensive field training at ports throughout the United 
States; 
* Supervisory training provided at ICE's Leadership Development Center 
in Dallas, TX, through a shared services agreement between CIS, ICE, 
and CBP. 

Federal Law Enforcement Training Center (FLETC); 959 FTEs; 
Overview of component responsibilities: 
* Train and prepare law enforcement professionals across government; 
Selected characteristics of training: 
* Training function transferred as a whole from Department of the 
Treasury with formation of DHS; 
* Consolidates law enforcement training at five FLETC academies for 
multiple DHS components; 
* FLETC budget covers most of the basic training provided to DHS 
components; auxiliary costs, such as room and board, are reimbursed by 
components. 

U.S. Immigration and Customs Enforcement (ICE); 14,950 FTEs; 
Overview of component responsibilities: 
* Prevent acts of terrorism by targeting people, money, and materials 
that support terrorist and criminal activities focusing on the nation's 
border, economic, transportation, and infrastructure security; 
Selected characteristics of training: 
* Training function partly carried over from legacy Immigration and 
Naturalization Service and legacy Customs Inspections and includes a 
separate training organization for the Federal Protective Service; 
* Federal Protective Service trains its uniformed officers at FLETC 
site in Glynco, GA, and ICE Academy is located at same FLETC site; 
* ICE's Leadership Development Center in Dallas, TX, provides 
supervisor and manager training for multiple DHS components. 

U.S. Secret Service; 6,526 FTEs; 
Overview of component responsibilities: 
* Protect the President and other designated personnel; 
* Protect the country's currency and financial infrastructure; 
* Provide security for designated national events; 
* Selected characteristics of training; 
* Training function transferred as a whole along with the rest of the 
Secret Service from Department of the Treasury; 
* Trains both special agents and uniformed law enforcement officers 
with most instruction taking place at Rowley Training Center in 
Beltsville, MD; 
* Basic training for new employees takes place at FLETC site in Glynco, 
GA. 

Transportation Security Administration (TSA); 52,467 FTEs; 
Overview of component responsibilities: 
* Protect the nation's transportation systems to ensure freedom of 
movement for people and commerce; 
Selected characteristics of training: 
* Training function greatly expanded after transfer from Department of 
Transportation with formation of DHS; separate training organization in 
place for the Federal Air Marshal Service, which was recently 
transferred as a result of DHS's Second Stage Review; 
* Some training delivered through TSA headquarters office in Arlington, 
VA, which also oversees contracted training activities through its 
quality assurance unit; 
* Most training delivered in field (airport) sites through contractors 
and approved instructors; 
* Operates an academy in Artesia, NM, and an academy at the FLETC site 
in Glynco, GA; 
* Federal Air Marshal Service trains new hires at an initial program in 
Artesia, NM, and then they receive specialized training in Atlantic 
City, NJ; Federal Air Marshal Service field offices also conduct 
ongoing training. 

Sources: GAO presentation of DHS information and the President's Fiscal 
Year 2006 Budget. 

Note: Figures showing full-time equivalents (FTE) for components 
reflect FTEs listed under "current services" from the President's 
Fiscal Year 2006 Budget. 

[End of table]

The results of a governmentwide survey conducted by the Office of 
Personnel Management in 2004 on human capital practices and employee 
attitudes suggest that efforts to identify and build upon examples of 
good training practice within DHS may be particularly 
relevant.[Footnote 8] For each of the eight questions in the 2004 
Federal Human Capital Survey that focused on training related topics, 
the percentage of DHS respondents providing positive responses 
(typically the top two options on a five-point scale) was lower than 
the governmentwide average. In fact, the DHS response ranged from 5 to 
20 percentage points lower than the governmentwide average for the same 
questions. For example, 54 percent of respondents at DHS indicated that 
they received the training they needed in order to perform their jobs, 
compared to 60 percent governmentwide. Half (50 percent) of DHS 
respondents said that they were either satisfied or very satisfied with 
the training they received for their present jobs, as opposed to 55 
percent that expressed these levels of satisfaction governmentwide. The 
largest difference involved having electronic access to learning and 
training programs, where 51 percent of DHS respondents responded 
positively, compared to 71 percent governmentwide. A DHS official told 
us that the department is aware of the challenges reflected in these 
data and is currently exploring options with the Office of Personnel 
Management to conduct further analysis. The aim of this work would be 
to identify areas where DHS might target additional attention as well 
as provide a baseline for future attitude measures. 

Figure 1: DHS Organizational Structure: 

[See PDF for image]

Note: This organization chart shows the expected end state resulting 
from a reorganization announced by the department in July 2005 at the 
conclusion of DHS's Second Stage Review process. 

[End of figure]

DHS Has Made Progress in Addressing Departmentwide Training Issues, but 
Challenges May Impede Its Efforts: 

DHS has made progress in addressing departmentwide training issues and 
these efforts reflect some of the elements of a strategic approach 
toward training as described in our previous work.[Footnote 9] Most 
training-related activities at DHS--such as planning, delivery, and 
evaluation--primarily take place at the component level and relate to 
mission issues. Therefore, any successful approach regarding 
departmentwide training issues will require the concerted and 
coordinated efforts of multiple components within DHS as well as the 
ability of the CHCO to effectively lead a network of different training 
organizations. The department's current efforts, although promising, 
are still in the early stages and they face significant challenges. 
Unless these challenges are successfully addressed they may impede 
DHS's ability to achieve its departmentwide training goals. 

DHS Recently Developed a Departmental Training Strategy: 

DHS recently developed a coordinated departmental training strategy 
that supports broader human capital and organizational goals and 
objectives. We have previously reported that effective organizations 
establish clear goals with an authority structure able to carry out 
strategies and tactics, that is, the day-to-day activities needed to 
support the organization's vision and mission. By so doing, a well- 
designed training function can be directly linked to the organization's 
strategic goals and help to ensure that the skills and competencies of 
its workforce enable the organization to perform its mission 
effectively. 

DHS's department-level training strategy is presented in its human 
capital and training strategic plans. Issued in October 2004, its human 
capital strategic plan includes selected training strategies, such as 
developing a leadership curriculum to ensure consistency of 
organizational values across the department and using training to 
support the implementation of the new DHS human capital management 
system, MAXHR. In July 2005, DHS issued its first departmental training 
plan, Department of Homeland Security Learning and Development 
Strategic Plan, which provides a strategic vision for departmentwide 
training. This plan is a significant and positive step toward 
addressing departmentwide training challenges. The plan identifies four 
short-term goals for fiscal year 2006 and one long-term goal for fiscal 
years 2006 through 2010. Among the short-term goals are such tasks as 
defining the scope of training activities and improving the governance 
process between the CHCO office and individual organizational 
components, supporting the rollout of MAXHR, identifying/implementing 
best practices, and addressing specific concerns regarding DHS's 
training facilities and advanced distributed learning studies. The plan 
also articulates a long-term goal for DHS to "become a recognized world-
class learning organization where managers and supervisors effectively 
lead people."

Each of these goals is followed by supporting strategies and tactics. 
For example, to achieve its goal of ensuring the best use of training 
resources through the identification and implementation of best 
practices, the plan identifies specific strategies, one of which is to 
improve the awareness of ongoing DHS training activities among 
organizational components. This strategy is, in turn, supported by 
still more specific tactics such as developing a site on the DHS 
Interactive system to facilitate the sharing of information across the 
training community. 

More significant than the fact that DHS issued a training strategic 
plan document is the fact that DHS followed an inclusive and 
collaborative process while developing it. We have previously reported 
that for high-performing, results-oriented organizations, a strategic 
plan is not simply a paper-driven exercise or onetime event, but rather 
the result of a dynamic and inclusive process wherein key stakeholders 
are consulted and involved in the identification of priorities and the 
formation of strategies.[Footnote 10] When creating its plan, DHS 
consulted training leaders at components throughout the department, in 
addition to others, to help develop and review its content. Several 
training leaders we spoke with thought highly of this process and the 
extent to which it provided them opportunities to contribute and 
comment on the draft plan. 

DHS has made considerable progress in addressing departmentwide 
training issues through the development of its first training strategic 
plan. However, there are areas where future efforts can be improved. 

Linkage to DHS organizational and human capital strategic plans. Our 
past work on strategic planning and management practices shows that 
effective strategic plans describe the alignment between an agency's 
long-term goals and objectives and the specific strategies planned to 
achieve them. Clearly linking training tactics with particular 
organizational objectives creates a direct line of sight that can both 
facilitate the ability of staff to work toward mission goals and enable 
stakeholders to provide meaningful oversight. In the introduction to 
the DHS training strategic plan, the department's CHCO highlights the 
value of this practice stating that "the key purpose of [the] plan is 
to align our education, training and professional development efforts 
with the President's Management Agenda and the Department's vision, 
mission, core values and strategic plan."

The DHS training strategic plan contains examples of goals, strategies, 
and tactics that align with and support goals found in the department's 
human capital and organizational strategic plans; however, these 
linkages are never actually identified or discussed in the plan itself. 
For example, the DHS training strategic plan contains a goal and 
several tactics related to MAXHR training. These, in turn, support a 
MAXHR goal and strategy in the department's human capital strategic 
plan as well as the "organizational excellence" goal of the DHS 
strategic plan. However, the training strategic plan does not show 
these linkages. Identifying such linkages, either in the training plan 
itself or in an appendix, would more clearly communicate to both 
internal and external stakeholders the connections and justifications 
for specific training goals, strategies, and tactics. 

DHS's own human capital strategic plan provides an illustration of one 
way to communicate linkages between goals and strategies contained in 
the plan and the broader organizational goals they are intended to 
support. For example, in an appendix, the DHS human capital strategic 
plan contains a matrix that directly links strategies, such as 
developing a new Senior Executive Service (SES) performance management 
system, with specific objectives contained in the DHS strategic plan as 
well as the President's Management Agenda human capital standards for 
success. 

Usefulness of performance measures. We have previously reported several 
key characteristics of effective strategic and management plans, 
including the need for performance measures.[Footnote 11] Appropriate 
performance measures along with accompanying targets are important 
tools to enable internal and external stakeholders to effectively track 
the progress the department is making toward achieving its training 
goals and objectives. To this end, organizations may use a variety of 
performance measures--output, efficiency, customer service, quality, 
and outcome--each of which focuses on a different aspect of 
performance. 

The DHS training strategic plan contains few specific performance 
measures for its goals or strategies and all of these are output 
measures. For example, the plan makes use of output measures in its 
requirement that certain actions, such as the development of a new 
management directive or the chartering of a team, be completed by the 
end of fiscal year 2006, and in establishing a deadline for when 
reports need to be completed in order to be included in the 2007 plan. 
In contrast to output measures like these, which gauge the level of 
activity or effort by measuring whether a particular thing is produced 
or service performed, other types of measures, such as measures of 
customer satisfaction or program outcomes, focus on the impact or 
results of activities. By appropriately broadening the mix of measures 
it uses and more clearly identifying targets against which DHS can 
assess its performance, DHS can improve the usefulness of its plan. 
After we completed our audit work, DHS training officials informed us 
that they decided to delay the development of performance measures 
until the rollout of the plan, when they could be developed by 
individual teams, as needed. They subsequently informed us that these 
teams will be held accountable to establish further performance 
measures that are outcome based and results oriented. 

DHS's human capital strategic plan again provides an illustration of 
how the department's training strategic plan might begin to work toward 
the inclusion of different types of performance measures. For example, 
accompanying the strategy that DHS assess the feasibility of 
establishing a 21st Century Leadership Training and Development 
Institute, the plan identifies two performance measures--customer 
satisfaction and cost of delivery--along with specific targets for 
each. For the customer satisfaction measure, the plan establishes a 
target of 4.5 on a scale from 1 to 5. The plan also includes specific 
tactics to achieve the strategy, such as developing and obtaining cross-
organizational support, developing measures and methodologies for 
leadership training, and implementing a learning management system, 
along with key milestone dates for completing them. 

The department may benefit from considering the experiences of leading 
organizations regarding the development of results-oriented performance 
measurement. In general, results-oriented organizations we have studied 
that were successful in measuring their performance developed measures 
that were: 

* tied to program goals and demonstrated the degree to which the 
desired results were achieved,

* limited to the vital few that were considered essential to producing 
data for decision making,

* responsive to multiple priorities, and: 

* responsibility linked to establish accountability for 
results.[Footnote 12]

Similar to the consultative process DHS followed when developing the 
goals and strategies contained in its training strategic plan, 
decisions concerning the selection of an appropriate set of performance 
measures should also be based on input from key stakeholders to 
determine what is important to them when assessing the department's 
performance regarding training. Clear and appropriate performance 
measures, developed in this way, can also provide DHS with valuable 
information, especially significant in the current fiscal environment, 
when it seeks to justify requests for resources from Congress. 

Training Councils and Groups Created by DHS Increase Communication 
across Components: 

Under the overall direction of the CHCO office, DHS has established a 
structure of training councils and groups that cover a wide range of 
issues and include representatives from each organizational component 
within DHS. The department is in the process of using these bodies to 
facilitate communication and the sharing of information within its 
diverse training community. In some instances, these councils and 
groups foster greater collaboration and coordination on training 
policies, programs, and the sharing of training opportunities. We have 
previously reported that agencies with a strategic approach to training 
recognize the importance of having training officials and other human 
capital professionals work in partnership with other agency leaders and 
stakeholders on training efforts.[Footnote 13]

The Training Leaders Council (TLC) plays a vital role in DHS's efforts 
to foster communication and interchange among the department's various 
training communities. This council consists of senior training leaders 
from each of the department's components as well as representatives 
from several department-level headquarters staff and support 
organizations with an interest in training-related issues.[Footnote 14] 
Started in October 2004 and formally chartered by the CHCO in March 
2005, the TLC's mission is to establish and sustain a collaborative 
community with the aim of promoting high-quality training, education, 
and development throughout DHS. To this end, it functions as a convener 
of training leaders from throughout the department and provides an 
overarching framework for several preexisting training groups and 
councils that were reestablished as standing committees of the TLC. 
Membership of the TLC consists of senior training leaders from each DHS 
component. In addition, most of these leaders as well as other training 
staff serve on one or more of its subgroups. See figure 2 for 
descriptions of the TLC and each of its subgroups. 

Figure 2: Departmental Training Councils and Subgroups at DHS: 

[See PDF for image]

[End of figure]

One key function of the TLC and these other training groups is to serve 
as a "community of practice" wherein officials can discuss common 
training challenges and share knowledge and best practices. For 
example, the Training Evaluation and Quality Assurance Group, composed 
of DHS training professionals responsible for evaluating and ensuring 
the quality of DHS training programs, conducted an informal survey of 
evaluation practices in various components with the intent of 
identifying effective evaluation approaches. A training official 
involved in the group told us that this survey was particularly 
important for the department's newer organizations, such as the 
Directorate for Information Analysis and Infrastructure Protection, 
which need to establish new practices from scratch. According to this 
official, his directorate and other organizations within DHS plan to 
use the group as a way to tap into the experience of other components 
within the department, such as CBP and FLETC, which have considerably 
more experience with training evaluation. 

In addition to sharing information about training practices, these 
groups can also provide a forum for exchanging practical information 
with the goal of making more efficient use of existing resources. For 
example, one training official told us that as a result of information 
obtained at TLC meetings, the official became aware of the existence of 
free training space available at facilities of two other components 
located in the Washington, D.C., metro area. Also, as a result of 
participating in these meetings, the official's organization was able 
to send an additional person to the Federal Executive Institute after 
becoming aware that another component had surplus spaces and was 
offering them at a reduced price to other components within 
DHS.[Footnote 15]

Another role carried out by the TLC is to collaborate on the 
formulation of training policies and advise the department's CHCO 
accordingly. For example, the TLC, in cooperation with staff from the 
CHCO office and an external contractor, conducted a survey of training 
sites throughout the department in 2004. This study cataloged available 
physical resources and site capacities with the aim of identifying 
potential opportunities to share these resources more efficiently, 
consolidate unneeded or duplicative sites, and identify other 
opportunities to increase training collaboration and effectiveness. 
This effort resulted in a series of recommendations that were 
subsequently incorporated into the department's training strategic 
plan. 

The activities of the department's Advanced Distributed Learning Group 
(ADLG) provides another example of how training officials from 
different components have worked together to develop proposals for 
solutions to departmentwide challenges. This group identified several 
issues in the area of technology and learning, including the need for a 
compatible IT infrastructure across components and the fact that some 
components lacked established systems with which to coordinate and 
manage training opportunities and attendance. Working with an outside 
consultant, the ADLG's efforts resulted in a proposal that DHS create a 
new Advanced Distributed Learning (ADL) Program Management Office to 
oversee the process of setting common standards. This proposal was 
subsequently included as part of the department's training strategic 
plan. In addition, the ADLG's work also led to DHS entering into a 
memorandum of understanding with the Office of Management and Budget 
and the Office of Personnel Management to create a DHS headquarters 
learning management system. Throughout this process, the ADLG 
represented the interests of the DHS training community as it worked 
with representatives from the Chief Information Officer's office and 
other functions within the department, as well as outside consultants. 

Despite these positive steps, DHS's effort to foster communication and 
coordination through departmentwide training councils and groups is at 
a relatively early stage and so far has produced varied results. Some 
training organizations, such as the TLC and ADLG, have met regularly 
leading to tangible results, while others such as the Training 
Evaluation and Quality Assurance Group, have met a few times and have 
only begun to set the groundwork for substantive coordination and 
collaboration in these areas. In addition, a training official told us 
that even active organizations like the TLC have encountered 
difficulties related to the relative lack of staff support for these 
efforts. As a result, additional burdens sometimes fall to the leaders 
and members of these groups who, in addition to serving on one or more 
departmental training groups or councils, must carry out full-time 
training positions at their home components. 

DHS Provides Training to Address Selected Departmentwide Needs: 

Another way DHS addresses departmentwide training issues is to directly 
provide training interventions or resources that address selected 
departmentwide needs, goals, or objectives. Three examples of the areas 
where DHS has worked to directly provide or support training on the 
departmental level are the following: (1) training related to the 
implementation of MAXHR, (2) DHS leadership development, and (3) 
training related to civil rights and civil liberties. 

Training for MAXHR implementation. DHS's new human capital management 
system, known as MAXHR, represents a fundamental change in many of the 
department's human capital policies and procedures that will affect a 
large majority--approximately 110,000--of its civilian employees. MAXHR 
covers many key human capital areas, such as pay, performance 
management, classification, labor relations, adverse actions, and 
employee appeals, and will be implemented in phases affecting 
increasing numbers of employees over the next several years. 

DHS correctly recognizes that a substantial investment in training is a 
key aspect of effectively implementing MAXHR, and in particular, the 
new performance management system it establishes. The need for in-depth 
performance management and employee development training is further 
supported by the department's results on the 2004 Federal Human Capital 
Survey. In this survey, just over half of DHS respondents--51 percent-
-believe supervisors or team leaders in their work units encourage 
their development at work, significantly less than the governmentwide 
response of 64 percent. DHS officials said they plan to educate all 
affected DHS employees on the details of the new system, how it will 
affect them, and the purpose of the changes. To do this, the department 
decided to develop, coordinate, and manage MAXHR training centrally 
through the CHCO office and offered its first training in May 2005. DHS 
plans to continue to provide its workforce with MAXHR training over the 
next several years following a phased approach that takes into account 
both when individual provisions of the new regulations take effect as 
well as the different audiences that exist within the DHS community, 
including human capital personnel, supervisors, and general employees. 
See figure 3 for a depiction of planned training during 2005 and its 
intended audiences. 

Figure 3: Timeline for MAXHR Training Scheduled through the End of 
2005: 

[See PDF for image]

Note: This training schedule reflects the department's plans as of 
early August 2005. However, we were told by a DHS official that these 
dates are subject to change. 

[End of figure]

The department has worked with contractors to develop training that 
uses a variety of approaches, including classroom instruction, ADL, 
handbooks, manuals, and quick reference guides, depending on specific 
needs. For example, in May 2005, labor relations/employee relations 
specialists and attorneys in the department received 2-½ days of 
training on the provisions of the new regulations and the major 
difference between them and previous programs. Structured as a "train 
the trainer" type intervention intended to prepare participants to 
conduct supervisor briefings in their own components, this was an 
instructor-led course held at sites across the country. In addition to 
educating individuals about the regulations, procedures, and systems 
associated with MAXHR and the adoption of a new performance management 
system, the department also plans to offer training specifically 
targeted to developing the skills and behaviors that will be necessary 
for its successful implementation. For example, in July 2005 
supervisors began to receive training on techniques for providing 
meaningful feedback to, coaching, and mentoring employees. 

DHS leadership development training. Leadership development is another 
area top management in DHS acknowledged as appropriate for 
departmentwide training to supplement existing component-level 
offerings. In 2004, the Secretary of Homeland Security announced the 
"One DHS" policy that identified the need to establish a common 
leadership competency framework for the department, as well as a 
unified training curriculum for current and future leaders. The purpose 
of this framework was to identify the skills, abilities, and attributes 
necessary for success as a DHS leader and to establish measurable 
standards for evaluation. 

To this end, the CHCO established the DHS Leadership Training and 
Development Group (LTDG), comprising training officials from each DHS 
component who combined an expertise in leadership development with 
personal knowledge of the missions and unique aspects of their 
particular organizational components. The LTDG met regularly from late 
2003 to mid-2004. During this time, the group developed a set of new 
core leadership competencies for DHS supervisors, managers, and 
executives, which it issued in April 2004. According to a DHS official, 
since the development of these new competencies they have been used by 
one component as part of its own leadership development plan and they 
have also helped to guide and inform current MAXHR leadership 
development efforts. 

DHS has recently taken steps regarding another facet of its leadership 
development initiative--its SES Candidate Development Program. In June 
2005, DHS issued a management directive establishing the SES Candidate 
Development Program, which included a rigorous selection process and 
critical leadership development opportunities, such as mentoring, 
developmental assignments, and action learning designed to give SES 
candidates experience in different job roles. DHS initially announced 
that it planned to implement the program in fiscal year 2005, but now 
may delay doing so until fiscal year 2006. 

Civil rights/civil liberties training. A third area in which DHS has 
taken steps to provide or support departmental training involves civil 
rights and civil liberties. FLETC's Behavioral Science Division and 
Legal Division, working with the DHS Office of Civil Rights and Civil 
Liberties, produced several training interventions, including Web- 
based, CD-ROM, and in-person programs designed to increase sensitivity 
and understanding in protecting human and constitutional rights. As 
part of this effort, FLETC held diversity seminars that focused on 
promoting understanding and respect of religious practices, 
particularly involving those of the Arab and Muslim communities. In 
another example of this effort, the Office of Civil Rights and Civil 
Liberties produced Web-based training on current policies regarding 
racial profiling. 

Our interviews with DHS training leaders suggest that further 
improvements can be made in communicating the availability of selected 
departmentwide training programs and resources. Staff at the Office of 
Civil Rights and Civil Liberties provided copies of its civil rights 
and liberties programs to training offices at each component in the 
department. While some senior training officials told us that their 
components actively disseminated this material by placing it on the 
component's training Web site or incorporating it into preexisting 
courses, other senior training officials we spoke with were unaware of 
any departmental training on these topics. In addition, other officials 
told us that their component's training office had independently 
developed its own material on Arabic sensitivity training, wholly apart 
from similar efforts undertaken by others in the department. More 
specifically, they told us that their development of certain training 
modules predated the development of very similar modules later prepared 
by DHS's Office of Civil Rights and Civil Liberties and FLETC, leading 
these officials to conclude that they may have been able to assist 
departmental efforts by sharing their work had they been aware of them. 

Challenges May Impede Achievement of Departmentwide Training Goals: 

As DHS moves forward, it faces challenges to achieving departmentwide 
training goals. These challenges include lack of common management 
information systems, the absence of commonly understood training 
terminology across components, the lack of specificity in authority and 
accountability relationships between the CHCO office and components, 
insufficient planning for effective implementation, and insufficient 
resources for ensuring effective implementation of training strategies. 

Sharing of Training Information across Components Made More Difficult 
by Lack of Compatible Infrastructure and Commonly Understood 
Terminology: 

The formation of DHS from 22 legacy agencies and programs has created 
challenges to achieving departmentwide training goals. Of particular 
concern to the training officials we spoke with are the lack of common 
management information systems and the absence of commonly understood 
training terminology across components. The training functions at DHS's 
components largely operate as they did before the creation of the 
department, with many of the same policies, practices, and 
infrastructures of their former organizations, and within these 
organizations are, for the most part, the same training leaders. It 
will take time for these organizations to evolve into a coordinated, 
integrated department. We have previously reported that successful 
transformations of large organizations, even those faced with less 
strenuous reorganizations and pressure for immediate results than DHS, 
can take from 5 to 7 years to fully take hold.[Footnote 16]

One issue DHS officials raised was the lack of common or compatible 
management information systems, such as information technology or 
financial management, which can inform decision makers' efforts to make 
efficient use of training resources across components. For example, DHS 
officials stated that a key challenge they encountered involved the 
difficulty of knowing what others were doing outside their particular 
offices or components. DHS lacks any unified sourcebook that employees 
could consult for the names, telephone numbers, and other relevant 
information of key contact persons in areas such as acquisition. 
Obtaining accurate information about resources and products available 
in the marketplace as well as data on users, vendors, and kinds of work 
has been a challenge to that effort. Another issue cited by officials 
concerned the lack of compatibility between learning management systems 
across components. In addition, some training officials expressed 
concerns about the accuracy or timeliness of some training data, which 
can limit or at least considerably delay their ability to track and 
fully account for funds spent on training and training-related travel. 
DHS has several efforts under way to address these issues, including 
the development of an online training facilities inventory intended to 
increase awareness of existing resources across the department and its 
decision to begin developing common ADL policies and standards. 

Officials also told us that there was little or no common understanding 
among DHS organizational components regarding the meaning of such basic 
terms as "subject matter expert," "orientation," and even "training." 
The lack of commonly understood terminology has presented challenges 
when officials from different components, including those participating 
in departmental training councils and groups, try to share practices 
with each other. These officials told us that the lack of commonly 
understood terminology can also affect their interactions with outside 
entities, such as contractors and state and local agencies. Besides 
facilitating communications and enabling components to share practices, 
a DHS official told us that a common nomenclature would increase the 
transparency of training practices to external contractors as well as 
the internal DHS training community. The department's training 
strategic plan calls for the creation of a common training language and 
glossary of terms in fiscal year 2006, and officials told us that they 
are currently in the early stages of creating such a glossary. 

Authority and Accountability Relationships between CHCO Office and 
Components Are Not Sufficiently Specific: 

An effective management control environment appropriately assigns 
authority and delegates responsibility to the proper personnel to 
achieve organizational goals and objectives.[Footnote 17] In such an 
environment, staff members who are delegated responsibility are given 
corresponding authority. In light of this, DHS's adoption of a "dual 
accountability" governance structure in 2004 presents certain 
challenges. Under this concept, heads of organizational components and 
the CHCO share responsibility for effective training in DHS. With a 
shared responsibility for DHS training, both the CHCO and component 
heads should have appropriate authority for making decisions regarding 
training.[Footnote 18] DHS does not specify how authority for training 
matters will be shared between the CHCO office and components for 
budgeting, staffing, and policy (e.g., determining which training 
functions, if any, should remain with components or be performed by DHS 
headquarters). The DHS management directive on training currently in 
place is a high-level two-page document that provides very few 
specifics on policies, procedures, and authorities for the CHCO office 
and the components. 

The department recognizes the need to clarify the responsibilities and 
authorities of the CHCO office and the components, as indicated by its 
inclusion in the DHS training strategic plan. Many of the tactics 
included in the plan would be difficult to successfully implement 
without first having a clear understanding of the responsibilities and 
authorities of the key organizations involved. More specifically, in 
the absence of clear authority relationships, decisions regarding how 
particular component training goals and strategies are to be 
incorporated in the DHS training strategic plan, or which training 
facilities should be consolidated to achieve departmental efficiencies, 
will be difficult to make. Without moving ahead with this effort in a 
timely fashion and completing the process of specifying how the CHCO 
office and components will share authority over training matters, it 
will be difficult for DHS to make the progress necessary on its 
departmentwide training agenda if it is to effectively implement the 
many strategies and tactics planned for fiscal year 2006. 

In addition, DHS's efforts at coordinating training across components 
and clarifying roles and relationships between departmental functions 
and organizational components may be further hampered by the fact that 
the management directive governing the integration of the human capital 
function claims that the Coast Guard[Footnote 19] and the Secret 
Service[Footnote 20] are statutorily exempt from its application. We 
found no reasonable basis to conclude that the directive could not be 
made applicable to them and are not aware of any explicit statutory 
exemption that would prevent the application of this directive. 
Moreover, exempting the Coast Guard and the Secret Service from the 
provisions of this directive casts doubt on the authority and 
accountability relationships between these components and the CHCO, 
potentially complicating the department's objective of clarifying the 
responsibilities, accountability, and authorities of the CHCO office 
and the components set forth in DHS's training strategic plan. 

Planning May Be Insufficiently Detailed to Ensure Effective 
Implementation in "Dual Accountability" Environment: 

In and of itself, DHS's dual accountability authority structure is not 
an obstacle to implementation of departmentwide training efforts. 
However, without detailed implementation plans, it presents potentially 
significant challenges. Because of this shared authority, DHS will need 
to take great care when planning for departmentwide training 
initiatives involving multiple organizational components to ensure that 
resources are aligned with organizational units performing activities, 
especially related to cross-organizational sharing of training and 
delivery of common training. The lack of comprehensive and rigorous 
planning can lead to confusion over responsibilities, lack of 
coordination, and missed deadlines. Regular and rigorous use of 
detailed implementation plans is necessary to implement decisions and 
carry out activities in a coordinated manner. 

After we completed our audit work, DHS informed us that it plans to 
establish 31 tactic teams to take ownership of each of the tactics 
presented in the DHS training strategic plan to be completed by the end 
of fiscal year 2006. As of mid-August 2005, DHS provided us with 
documentation indicating that 3 of these teams have been established to 
date. These teams appear to have taken promising steps toward the 
establishment of detailed plans for implementing their respective 
training tactics by developing draft objectives, deliverables, and 
closure criteria. But as fiscal year 2006 approaches, time is short for 
the CHCO office and the components to establish the remaining teams and 
then take the actions necessary to develop and put in place the 
detailed plans that will be critical for effectively implementing DHS's 
many training tactics by the end of the coming fiscal year. The TLC's 
ADLG has made use of this type of detailed approach in a report 
proposing a distance learning architecture for the department. Appended 
to its report is a detailed plan outlining the major activities, 
milestones, resources, and components needed to support the successful 
implementation of the proposal. 

Resources Needed to Implement Departmental Training Strategy: 

Several training officials told us they were concerned about the lack 
of dedicated resources and related capacity to carry out departmental 
initiatives. At the time we started our review, the CHCO office had 
only one full-time permanent employee dedicated to carrying out these 
activities; consequently, both training leaders and staff from 
organizational components were relied on to contribute to 
departmentwide efforts. After we concluded our audit work, a DHS 
official told us that the CHCO office had recently hired two additional 
full-time training staff: an ADL program manager and a staffer to 
oversee a recently approved SES candidate development program and 
headquarters operational leadership development. Individual components 
have also provided some assistance to departmentwide efforts through 
the appointment of temporary personnel. In late 2004, CBP and FLETC 
each detailed a staff member to the CHCO office to work on training- 
related projects. In addition, DHS has contracted for services to 
address selected departmentwide issues, such as setting common 
standards for ADL and reviewing DHS training facilities. 

DHS's departmental training councils and groups are almost exclusively 
staffed by component training leaders who already have full-time 
training commitments. The department's training strategic plan 
identifies many tactics for fiscal year 2006--including creating a 
common training language and glossary of training terms, establishing a 
repository for course catalog information, and developing a DHS 
training Web site--that will require considerable staff support to 
implement. Successful and timely completion of these and other 
initiatives will depend on sufficient resources being provided. 

DHS's Approach to Planning and Evaluation of Training Demonstrates Some 
Elements of Strategic Practice, but Is Still Evolving: 

It is essential for federal agencies to ensure that their training 
efforts are part of--and are driven by--their organizational strategic 
and performance planning processes. We have reported that aligning 
training with strategic priorities and systematically evaluating 
training activities play key roles in helping agencies to ensure that 
training is strategically focused on improving performance and meeting 
overall organizational goals.[Footnote 21] Strategic training practices 
in several DHS components or programs may provide models or insights to 
others in the department regarding ways to improve training 
practices.[Footnote 22] In areas where some components employed 
strategic practices, other components did not. 

Aligning Training with Organizational Priorities Is a Strategic 
Practice: 

We have previously reported that agencies demonstrating a strategic 
approach to training align their training efforts with overall 
strategic priorities. To do this, agencies can employ a variety of 
practices, such as linking training activities to strategic planning 
and budgeting and performing front-end analysis to ensure that training 
activities are not initiated in an ad hoc, uncoordinated manner, but 
rather are focused on improving performance toward the agency's 
goals.[Footnote 23] Some components in DHS applied the strategic 
practice of aligning training with organizational priorities, while 
others did not. 

CBP links its new and existing training activities to its strategic 
priorities when planning for its strategic initiatives and 
expenditures. Importantly, the head of training at CBP is at the 
decision-making table with other CBP leaders to help establish training 
priorities consistent with the priorities of the CBP Commissioner. 
Relevant program managers are asked, "What training do you need to 
achieve the goals in your strategic plan?" Such discussions took place 
during planning for CBP's custom trade pact initiative. 

During each budget cycle, CBP's central training office issues a "call 
for training" to its mission and mission support customers to estimate 
CBP's training needs for existing training activities and prioritize 
these needs based upon the Commissioner's priorities. Prior to 
establishing this process, training was mostly decided on a first come, 
first serve basis without clear and transparent linkages to 
organizational priorities. CBP's current process results in an annual 
training plan in which training needs are identified by priority as 
well as major occupational type, such as border patrol agent or CBP 
officer. Training decisions are based on whether training requested is 
critical, necessary, or "nice to have." During fiscal year training 
plan implementation, CBP tracks actual training activity through a 
central database to determine whether CBP is using its planned training 
resources. By tracking plan usage through a centrally managed database, 
CBP is able to reallocate unused training funds prior to the end of the 
fiscal year for either training activities that were not included in 
its original plan because of capacity constraints, or for emerging 
training priorities. 

The Coast Guard has adopted a strategic and analytic approach to 
training through its use of the Human Performance Technology (HPT) 
model--a front-end training assessment process to determine the cause 
of performance problems. The process starts with the assumption that 
many factors influence individual and unit performance and it is 
important to determine what the factors are before concluding that 
training is the solution. From its HPT analysis, the Coast Guard 
determines whether training is needed or whether another type of 
solution, such as a policy change, would be more appropriate. For 
example, in addressing a problem in aviation maintenance, a Coast Guard 
working group looked at likely causes of its performance problems and 
concluded that focusing on making aviation maintenance training better 
was not the only solution. More specifically, training officials 
encountered problems with job dissatisfaction and subpar performance 
from aviation chief warrant officers. In this case, training officials 
used HPT to analyze the nature of work performed by those responsible 
for aviation maintenance and concluded that there was not a good match 
between job skills and responsibilities. Specifically, over the last 20 
years, the scope and nature of the work performed by chief warrant 
officers changed significantly from maintaining components to managing 
aircraft systems. Performance problems were mainly caused by 
significant changes in the job functions of these officers over the 
years rather than by a lack of adequate training. 

In cases where the HPT analysis concludes that training is warranted, a 
training analysis is performed to determine the specific training 
interventions. For example, in implementing activities related to the 
Maritime Transportation Security Act, the Coast Guard analyzed its 
training needs through the HPT process to determine training necessary 
to help maritime inspectors reduce the exposure of ports and waterways 
to terrorist activities. The analysis identified the skills and 
knowledge necessary for new maritime inspector tasks and provided 
training interventions, such as developing job aids and targeted 
classes, to prepare inspectors for the tasks most relevant to support 
their new role. New courses were piloted and then subjected to 
multilevel evaluations to assess their effectiveness and potential 
impact on employee performance. 

Systematic Evaluation of Training Is a Strategic Practice: 

Agencies demonstrating a strategic approach to training employ a 
variety of practices, such as systematically evaluating training, 
actively incorporating feedback during training design, and using 
feedback from multiple perspectives.[Footnote 24] Several components 
and programs we examined at DHS used these practices, while others did 
not. 

One commonly accepted model used for assessing and evaluating training 
programs consists of five levels of assessment (see fig. 4).[Footnote 
25] In our review, virtually all components captured Level I data 
focusing on end-of-course reactions, while several also collected Level 
II data focusing on changes in employee skill, knowledge, or abilities. 
Several components evaluated, or were planning to evaluate, the impact 
of selected training programs on individual behavior, represented by 
Level III evaluations. 

Figure 4: Levels of Training Evaluation: 

[See PDF for image]

Note: GAO presentation based on information from Donald L. Kirkpatrick, 
Evaluating Training Programs: The Four Levels (San Francisco: Berrett- 
Koehler Publishers, Inc., 1998), and Jack J. Phillips, ed., 
Implementing Evaluation Systems and Processes (Alexandria, Va.: 
American Society for Training and Development, 1998). 

[End of figure]

To measure the real impact of training, however, agencies need to move 
beyond data focused primarily on inputs and outputs and develop 
additional indicators that help determine how training efforts 
contribute to the accomplishment of agency goals and objectives. At a 
couple of components, DHS officials told us they conducted Level IV 
evaluations, which assess the effectiveness of training interventions. 
We found no examples of the department or its components measuring the 
return on investment of training activities (Level V).[Footnote 26] 
Training effectiveness should be measured against organizational 
performance; however, not all levels of training evaluation require or 
are suitable for return on investment analysis. Determining whether 
training programs merit the cost of using such an approach depends upon 
the programs' significance and appropriateness. 

CBP takes a systematic approach to evaluating its training activities 
through its National Training Evaluation Program (NTEP) to help program 
managers and trainers make more informed decisions on the effectiveness 
of training courses and their delivery. Despite the fact that CBP is a 
large and decentralized organization, NTEP has enabled it to collect 
course evaluation information and make this information available to a 
wide range of users in a timely manner. NTEP has also standardized 
evaluation data to allow for comparison of training throughout various 
field locations. Before the rollout of NTEP, CBP did not use a standard 
mechanism for collecting evaluation data, which, according to a CBP 
official, made it difficult to gather evaluation data nationally. 

CBP focuses on collecting both end-of-course student reactions (Level 
I) and supervisor assessments of student on-the-job performance after 
attending the training (Level III). Electronic or paper-based 
evaluations are entered into the NTEP information system. The "close to 
real time" online data enables supervisors to perform trend analysis on 
training quality and provides opportunities for them to troubleshoot 
training deficiencies and identify high-performing courses. The NTEP 
online system allows CBP employees access to evaluation data on a need 
to know basis with four levels of access, while enabling them to locate 
evaluation data for any training class by date. Evaluation reports are 
aggregated for review by senior CBP officials. 

A CBP official told us that collecting course evaluation data is labor 
intensive, especially since many field operations still use paper 
processes. In addition, CBP has experienced a relatively low submission 
rate for Level I evaluation data for many of its training classes. The 
official told us that this was especially true for end-of-course 
reactions from staff in the field, where only about one-third of 
officer-related course participants submit evaluation forms. Given cost 
and labor challenges, CBP has targeted areas for evaluation that it 
believes are important, such as training related to its "One Face at 
the Border" initiative. 

In addition, agencies with a strategic approach to training do not wait 
until the conclusion of a training intervention to conduct evaluations. 
Rather, they approach evaluation through an iterative process capable 
of informing all stages of training.[Footnote 27] DHS's CHCO office 
used multiple forms of feedback from employees to develop its training 
strategy for MAXHR. From February through April 2005, the department 
administered surveys and conducted focus groups to obtain information 
on the needs, attitudes, and reactions of different communities 
affected by MAXHR. Shortly after issuing its new human capital 
regulations, the department provided basic information to all employees 
on the nature and timeline of changes they could expect under MAXHR 
through a Web broadcast. After the broadcast an online survey was used 
to obtain feedback from employees regarding the broadcast itself and 
their general feelings and concerns about the MAXHR initiative. 

DHS followed this initial survey with a larger survey to gather 
additional feedback on how information regarding MAXHR had been 
communicated, as well as specific areas where employees wanted 
additional information. Concerns about the need for training were 
prominent among the more than 9,000 responses received, with 
respondents ranking training as the second most serious challenge to 
the successful implementation of MAXHR. According to a senior DHS 
official, the survey results will inform subsequent training and 
communication efforts. 

DHS also collected evaluative feedback by conducting a series of focus 
groups held in locations across the country. The aim of these sessions 
was to validate the design of the performance management program 
established under MAXHR and identify concerns that would inform the 
development of additional training. Consistent with the strategic 
training practice of seeking out different perspectives when 
redesigning and assessing training efforts, DHS staff held separate 
focus groups for bargaining unit employees, non-bargaining unit 
employees, and supervisors and managers at all of the locations 
visited. This enabled them to identify issues of particular concern to 
each of these groups as well as issues common to all three. For 
example, both the bargaining unit and non-bargaining unit employee 
focus groups raised concerns about supervisors having inadequate skills 
for fairly administering the new performance management system. This 
concern was also shared by supervisors and managers themselves who 
expressed the need for additional skills training in areas such as goal 
setting and providing performance feedback. The sessions validated the 
CHCO office's plans to offer performance management training to 
supervisors and managers before the implementation of the new system 
and assisted in refining issues for future training. 

FLETC's methods for evaluating its major training programs include 
feedback from multiple perspectives when examining the benefits of 
training on actual employee job performance. FLETC's Level III 
evaluations obtain feedback from both trainees and their supervisors to 
inform future improvements to training curricula. Evaluation results 
are compiled into a comprehensive report used during FLETC's periodic 
curriculum reviews on its major training programs, such as the Natural 
Resource Police Training Program. The report contains detailed feedback 
from both the trainee and supervisor perspectives 6 months to 1 year 
after the trainee has attended the training program. For example, for 
the Natural Resource Police Training Program, FLETC analyzed how well 
the program prepared trainees in all aspects of their jobs. In this 
case, analysis identified those courses that had benefited program 
trainees the least--including determining speed from skid marks and 
death notification. Training designers can use report information to 
improve program curricula and refocus training on knowledge and skill 
areas most critical to performing the job. In addition to Level III 
evaluation results, its training designers make program and individual 
class changes by using other methods of evaluation, such as direct 
student feedback after classes and trainee examinations, which 
determine how well the trainees understood the course material 
immediately following the program. 

DHS Has Used Training in an Effort to Help Its Workforce as the 
Department Undergoes Transformation and Cultural Change: 

The creation of DHS resulted in significant cultural and 
transformational challenges for the department. We have previously 
reported that training is one way organizations successfully address 
cultural issues while simultaneously facilitating new ways to work 
toward the achievement of organizational goals.[Footnote 28] Among the 
DHS components in our review, some merged cultures from different 
legacy organizations (CBP, ICE), another component came as a small 
organization that greatly expanded when joining DHS (Federal Air 
Marshal Service), while others joined DHS intact (Secret Service, Coast 
Guard, FLETC), and still another was previously a part of a larger 
legacy organization (CIS). Each component faces the need to find a way 
to identify itself as part of the larger DHS organization, that is, 
with a sense of affiliation rather than as an outsider looking in. At 
the same time, components must either maintain their existing cultures 
or develop new cultures to adapt to changing missions and needs. The 
key is to build upon positive aspects of the components' cultures as 
the larger organization develops its own culture. 

Agencies that undergo successful transformations change more than just 
their organizational charts, they also make fundamental changes in 
basic operations, such as how they approach strategic human capital 
management. DHS understands this, and the MAXHR initiative is part of 
an effort by the department to fundamentally change its approach to 
human capital management by establishing a personnel system that is 
flexible, performance oriented, and market based. 

The Secretary of Homeland Security and other top officials have 
actively supported the role of training in implementing these changes 
by making it a leadership expectation that all DHS executives, 
managers, and supervisors be personally involved as both participants 
in and supporters of MAXHR training efforts. The CHCO office, working 
with the assistance of outside contractors, has developed several 
training interventions aimed at providing these groups with the tools 
and information needed to champion the benefits of a performance-based 
culture and successfully implement MAXHR in their components. 

In August 2005, DHS sponsored a 2-½ day training program for 350 to 400 
of the department's senior executives and flag officers. The program 
covered a range of topics, including an update on current DHS 
priorities; techniques and best practices for how senior leaders can 
effectively support and implement these priorities; as well as specific 
management, communication, and training approaches that can be used to 
support the creation of a performance-based culture. The Secretary, 
Deputy Secretary, and Under Secretary for Management all participated 
in the program, which also featured presentations from human capital 
and organizational change experts from outside the department. In 
addition to its focus on MAXHR implementation, which included both 
large and small group sessions wherein participants could discuss 
performance management and share information on practices, the course 
also provided a forum for the department's top leadership and senior 
executives to review the then recently issued recommendations resulting 
from the Secretary's Second Stage Review process. 

Another training intervention sponsored by the department directly 
targets managers and supervisors who will be responsible for carrying 
out many of the key behaviors associated with the new system and whose 
active support is viewed by DHS as critical for achieving the 
transformation to a performance-based culture. The 2-½ day program 
focuses on developing and improving interpersonal, managerial, and 
other so-called soft skills. DHS expects to provide the training to 
approximately 12,000 managers and supervisors throughout the 
department. 

On the component level, training has also played an important role in 
CBP's effort to transform from the traditional, largely siloed approach 
used by its legacy agencies when protecting our borders to a new 
integrated concept that it believes is more in line with its current 
needs. Officials noted that the merger into CBP led to some resistance 
from employees who had not yet understood or accepted the reasons for 
the merger. These same officials acknowledged that they must continue 
to work at informing employees why changes were made and provide 
vehicles for better integration through training. For example, in the 
"One Face at the Border" initiative, supervisory training has 
incorporated some elements of cultural integration by including a 
session on bridging the culture gap. Officials at CBP designed and 
piloted a training module to be added to the supervisory curriculum 
specifically targeting how they can more effectively understand the 
value and perspective of staff coming out of the legacy organizational 
cultures. 

In addition, training played a key role in facilitating the transition 
of CBP's workforce from its three legacy organizations. Training for 
the new CBP officer and CBP agriculture specialist positions aimed to 
improve coordination and communication across inspection functions and 
enhance the flexibility of CBP's workforce. Specifically, CBP created a 
series of training courses to provide former Customs and former 
Immigration and Naturalization Service officers with the knowledge and 
skills necessary to carry out the responsibilities of this new 
position. To develop this training, CBP-wide working groups identified 
and validated critical tasks for the new frontline CBP officer to 
perform. A mix of training delivery methods were used (i.e., e-learning 
and instructor led), and classroom knowledge and skills were reinforced 
with on-the-job training. CBP provided extensive train-the-trainer 
courses so that trainers could return to their field sites and instruct 
officers there. (See app. 2.)

Conclusions: 

DHS must continue to make progress on three important aspects of 
training as it moves forward: (1) forging an effective role for 
training at the departmental level and implementing its departmentwide 
training strategy; (2) taking a strategic approach to training 
practices, in part by building upon examples of good practice to be 
found among its former organizations, as well as considering other 
examples of strategic practices; and (3) finding ways that training can 
help to foster organizational transformation and cultural change within 
the department. To date, DHS has taken positive steps in these areas, 
yet significant challenges lie ahead. 

The ability to make decisions from a departmentwide perspective and 
then effectively implement them will help determine whether training in 
DHS achieves its intended results. Strong leadership will play a 
critical role in this process. To be successful, DHS will need to have 
both a clear plan of action as well as the ability to anticipate and 
overcome several implementation challenges. The creation of the TLC and 
the development of the department's first training strategic plan both 
represent a good start in this process. Better performance measures, 
more specific milestones, and the inclusion of performance targets 
would make DHS's strategic training plan a more useful tool for both 
internal and external stakeholders to use in tracking the department's 
progress toward achieving its training objectives. Clarifying authority 
relationships between the CHCO and component heads, developing detailed 
implementation plans, and giving appropriate attention to providing 
resources to implement training initiatives when setting funding 
priorities are also likely to be critical factors in building and 
sustaining an effective role for department-level training at DHS. 

A strategic approach toward training is also very important as DHS 
seeks to build on its current efforts and strives to move forward. As 
we have noted, some programs and components in DHS already use specific 
strategic training practices, and other components within the 
department can benefit from their example. As DHS implements new 
training programs, such as the large-scale, multistage training being 
developed to support the implementation of MAXHR, it has a valuable 
opportunity to reflect the lessons learned from these experiences in 
subsequent departmentwide training efforts. Finally, the transition to 
a new department has brought with it cultural challenges, and training 
can play a role in both defining and refining an effective DHS culture 
without sacrificing the cultural history of its components. 

Recommendations for Executive Action: 

To help DHS establish and implement an effective and strategic approach 
to departmentwide training, we recommend that the Secretary of Homeland 
Security take the following actions: 

* adopt additional good strategic planning and management practices to 
enhance the department's training strategic plan by (1) creating a 
clearer crosswalk between specific training goals and objectives and 
DHS's organizational and human capital strategic goals and (2) 
developing appropriate training performance measures and targets;

* clearly specify authority and accountability relationships between 
the CHCO office and organizational components regarding training as a 
first step to addressing issues DHS has identified for fiscal year 2006;

* ensure that the department and component organizations develop 
detailed implementation plans and related processes for training 
initiatives; and: 

* when setting funding priorities, give appropriate attention to 
providing resources to support training councils and groups to further 
DHS's capacity to achieve its departmentwide training goals. 

Agency Comments: 

We provided a draft of this report to the Secretary of Homeland 
Security for comment and received written comments from DHS that are 
reprinted in appendix III. In addition, we received technical comments 
and clarifications, which we incorporated where appropriate. DHS 
generally agreed with our recommendations. 

We will provide copies of this report to the Secretary of Homeland 
Security and other interested parties. Copies will also be provided to 
others upon request. In addition, this report is available at no charge 
on the GAO Web site at http://www.gao.gov. 

If you have any questions about this report, please contact me at (202) 
512-9490 or stalcupg@gao.gov. Major contributors to this report were 
Kimberly Gianopoulos, Assistant Director; Peter J. Del Toro; Robert 
Yetvin; and Gerard Burke. 

Sincerely yours,

Signed by: 

George H. Stalcup: 
Director, Strategic Issues: 

[End of section]

Appendix I: Scope and Methodology: 

To achieve our objectives, we reviewed training at the Department of 
Homeland Security (DHS) at both the departmental and component levels. 
When examining training at the departmental level, we collected, 
reviewed, and analyzed the department's training rules, procedures, 
policies, and organizational charts; departmental, human capital, and 
training strategic plans; human capital and training management 
directives; Internet and intranet Web pages; and other relevant 
documents. To further our understanding of training at DHS and the 
issues and challenges involved, we interviewed training and human 
capital officials in the Office of the Chief Human Capital Officer and 
the leaders and coleaders of DHS's training councils and groups. We 
also observed the January 2005 meeting of the Training Leaders Council. 
We supplemented our review of departmental training at DHS by examining 
the department's effort to use training related to MAXHR to foster 
transformation and cultural change in the department. 

In addition, we reviewed training at major organizational components in 
DHS and selected the six largest components based on staff size and 
budget. Using these criteria, we reviewed training at Customs and 
Border Protection (CBP), Citizenship and Immigration Services, 
Immigration and Customs Enforcement (including the Federal Air Marshal 
Service, the Federal Protective Service, and the Leadership Development 
Center), the Coast Guard, the Secret Service, and the Transportation 
Security Administration. See figure 5 for a depiction of the DHS 
organizational structure in place during the time of our review. These 
components collectively represent about 95 percent of the total staff 
at DHS. We also included the Federal Law Enforcement Training Center 
because of the special role it plays in training employees from other 
DHS components. 

Figure 5: DHS Components Included in Our Review: 

[See PDF for image]

Note: This figure does not depict all components and entities in the 
department, but is intended to provide a general framework within which 
to present the components and centers included in our review. The 
organizational structure depicted above was in place during our review, 
and does not reflect actual or proposed changes related to DHS's Second 
Stage Review. 

[End of figure]

When examining training at selected components, we reviewed component- 
level strategic, human capital, and training plans when available; 
training budget requests and expenditure documents; training 
procedures, policies, and organizational charts; rules and policies for 
identifying and prioritizing training programs; Internet and intranet 
Web pages; selected training course materials; and other relevant 
documents produced by these components. To further our understanding of 
training at the component level, we also interviewed training officials 
at each of the selected components and identified these individuals 
based on their knowledge, experience, and leadership roles. We 
conducted our interviews at component headquarters or field offices 
located in the Washington, D.C., area. In addition, as part of our 
review of DHS's efforts to foster transformation and cultural change, 
we observed training related to CBP's "One Face at the Border" 
initiative in northern Virginia. 

To help determine whether DHS used a strategic approach in planning and 
evaluating its training activities at the departmental or component 
levels, we referenced criteria contained in our guide for assessing 
strategic training and development efforts in the federal 
government.[Footnote 29] This guide outlines a framework for assessing 
training efforts, consisting of a set of principles and key questions 
that federal agencies can use to ensure that their training investments 
are targeted strategically and not wasted on efforts that are 
irrelevant, duplicative, or ineffective. We selected our case examples 
based on their suitability for demonstrating specific strategic 
training practices. Other components within DHS may, or may not, be 
engaged in similar practices. To determine whether DHS followed leading 
management practices in planning and implementing departmentwide 
training, we also drew on our previous work on strategic planning and 
effective management practices. 

We did not include within our scope training intended for audiences 
external to DHS, and we generally covered training and training 
management in effect during the period in which we did our work. We 
conducted our work from November 2004 through July 2005 in accordance 
with generally accepted government auditing standards. We requested 
comments on a draft of this report from DHS, which are reprinted in 
appendix III. The comments are addressed in the Agency Comments section 
of this report. 

[End of section]

Appendix II: "One Face at the Border" Training at U.S. Customs and 
Border Protection: 

One of the initial goals for creating DHS was to better protect the 
United States from terrorists entering the country, and ports of entry 
are the means through which terrorists can enter. The creation of CBP 
within DHS merged border inspection functions at U.S. ports of entry, 
which had previously been performed by three separate agencies. Known 
as "One Face at the Border," this initiative created the positions of 
CBP officer and CBP agriculture specialist that combined aspects of 
three former inspector functions. This initiative aimed to improve 
coordination and communication of inspections to better protect the 
nation's borders from terrorists as well as to improve entry for 
legitimate travel and trade. 

To successfully make the transition to these new positions, significant 
training was needed. Specifically, CBP created a series of training 
courses to provide former U.S. Customs and former Immigration and 
Naturalization Service officers with the knowledge and skills necessary 
to carry out the responsibilities of this new position. In addition, 
CBP officers received training to meet CBP's new mission priority of 
terrorism prevention. Although the emphasis was on cross-training 
legacy officers, the new curriculum was also geared to new hires. 
Because agricultural inspections are more specialized, CBP officers 
receive training sufficient to enable them to identify potential 
agricultural threats, make initial regulatory decisions, and determine 
when to make referrals to CBP agriculture specialists. More detailed 
agricultural inspections are performed by these specialists who have 
substantial training and background in agricultural issues. 

A variety of training delivery methods were used (e.g., e-learning and 
classroom) and these training methods were reinforced with extensive on-
the-job training. In addition to traditional content areas (e.g., cross-
training for former U.S. Customs officers includes courses on 
immigration fundamentals and immigration law), training courses also 
covered CBP's new priority mission of preventing terrorism (e.g., 
training in detecting possible terrorists and fraudulent documents, 
honing interviewing skills, and making appropriate referrals to staff 
for additional inspection). CBP emphasizes on-the-job training in an 
effort not to place inspectors on the job without direct supervisory 
and tutorial backup. Training for new recruits has also been modified 
to include a preacademy orientation program at the port location where 
the recruit will eventually work before he or she receives academy 
training. This is a 72-day course for CBP officers and a 46-day course 
for CBP agriculture specialists. 

CBP's main strategy to prepare for field delivery of training was to 
provide extensive train-the-trainer courses so that trainers could 
return to their field sites and instruct officers there. Training 
priorities were established with the idea of spacing the training out 
so that field offices would not be overwhelmed. For example, CBP rolled 
out its primary cross-training to airports, while antiterrorism 
training was rolled out to land borders. 

Officials reported that cross-training benefited CBP officers since 
they have gained more knowledge by learning both immigration and 
customs laws and procedures. This increase in knowledge has the 
potential benefit of providing more variety in job tasks as well as 
increasing the opportunities for advancement since an officer can now 
apply for supervisory-level positions that had previously been open 
only to former U.S. Customs or Immigration and Naturalization Service 
officers. 

Change has not come about without challenges, however, as many officers 
were reported to have resisted changes to their responsibilities, 
mainly related to the difficulties in learning a new set of procedures 
and laws. Officials noted that there has been an enormous amount of 
required training for CBP officers, and it can sometimes be 
overwhelming. For former officers, in addition to completing an 
extensive cross-training schedule and new training related to terrorism 
prevention, there are many other required courses related to their 
mission. For example, training modules are required in areas such as 
body scanning, hazardous materials, cargo inspection, and seized 
assets. 

Although staffing challenges may ultimately be relieved with trained 
officers able to perform dual inspections, officials noted that it has 
been extremely difficult to take staff off-line to complete the "One 
Face at the Border" training. One official said that classes have been 
very difficult to schedule because of the constant pressure to staff 
operations. For example, in one case, a class was canceled right after 
it began because the trainees were pulled out to staff their inspection 
booths. This official also noted that trainers have had to be very 
flexible to accommodate staff schedules to ensure that training occurs. 

[End of section]

Appendix III: Comments from the Department of Homeland Security: 

U.S. Department of Homeland Security: 
Washington, DC 20528: 

September 7, 2005: 

Mr. George H. Stalcup: 
Director:
Strategic Issues:
U.S. Government Accountability Office (GAO): 
441 G Street, NW, 2440C:
Washington, DC 20548: 

Dear Mr. Stalcup: 

RE: GAO-05-888, Department of Homeland Security (DHS): Strategic 
Management of Training Important for Successful Transformation (GAO Job 
Code 450353). 

Thank you for the opportunity to review the draft report. The report is 
a positive statement of the initial actions DES has taken to ensure 
that training is used as a resource for transformation of the 
Department. We concur that the creation of the Training Leaders Council 
(TLC) and the development of the initial DHS Learning and Development 
Strategic Plan, "is a significant and positive step towards addressing 
department wide training challenges." Further, we agree that 
implementation of specific training programs, such as training for 
MAX^HR is important to our efforts to meet our goal for organizational 
excellence. 

The Chief Human Capital Office was pleased to provide the GAO with 
reports about our training organizations and to coordinate interviews 
with DHS training leaders. Although, the training enterprise within the 
Department is a major undertaking with limited resources, we believe 
that external review and input provides us the opportunity for 
improvement. We encourage GAO to use the information gathered while 
creating this report as a baseline to streamline the process for future 
reviews. Technical and substantive comments were provided separately 
for your use in updating the final report. 

We concur with the intent of the recommendations in the report and have 
already taken steps to ensure that measures are appropriately used when 
implementing the Learning and Development Strategic Plan. The TLC 
designed the plan to link to the President's Management Agenda and to 
the DES Strategic Plan, but will take further steps to make those links 
more transparent to external audiences. We anticipate that the 
authority and the role of the CHCO will be further clarified as we work 
towards functional integration and as the TLC develops a full 
accounting of current training authorities and requirements. Our plan 
is to ensure that component organizations complete annual training 
plans by March, 2006. We recognize that additional resources would 
enhance the TLC's ability to implement training strategies but will 
need to balance that issue with on-going mission requirements. 

We look forward to the final report and plan to provide additional 
comments at that time regarding the substance and recommendations 
contained in the report. 

Sincerely,

Signed by: 

Steven J. Pecinovsky: 
Director:
Departmental GAO Audit Liaison Office: 

[End of section] 

FOOTNOTES

[1] GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.: 
January 2005). 

[2] In previous GAO reports we have defined "training" as making 
available to employees planned and coordinated educational programs of 
instruction in professional, technical, or other fields that are or 
will be related to their job responsibilities. Similarly, we have 
defined "development" to generally include aspects of training, as well 
as structured on-the-job learning experiences (such as coaching, 
mentoring, or rotational assignments), and education. For the purposes 
of this report, "training" will be used as a shorter substitute for 
"training and development."

[3] GAO, Human Capital: A Guide for Assessing Strategic Training and 
Development Efforts in the Federal Government, GAO-04-546G (Washington, 
D.C.: March 2004). 

[4] According to DHS, the name MAXHR was chosen to convey the intent of 
the new system to foster "maximizing results, rewarding excellence." 
MAXHR covers key human capital areas, including pay, performance 
management, classification, labor relations, adverse actions, and 
employee appeals. 

[5] Homeland Security Act of 2002, Pub. L. No. 107-296 (Nov. 25, 2002). 

[6] This section of the Homeland Security Act of 2002, also known as 
the Chief Human Capital Officers Act of 2002, led to the creation of 
chief human capital officers in each federal department. Pub. L. No. 
107-296, §§ 1301-1305 (Nov. 25, 2002). 

[7] GAO-04-546G. 

[8] According to the Office of Personnel Management, both the survey's 
governmentwide findings and DHS-specific findings are generalizable to 
their respective populations. For additional findings from the 2004 
Federal Human Capital Survey on these and other training and non- 
training-related questions, see www.fhcs2004.opm.gov. 

[9] GAO-04-546G. 

[10] GAO, Executive Guide: Effectively Implementing the Government 
Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June 
1996). 

[11] GAO, Managing for Results: Next Steps to Improve the Federal 
Government's Management and Performance, GAO-02-439T (Washington, D.C.: 
Feb. 15, 2002). 

[12] GAO/GGD-96-118. 

[13] GAO-04-546G. 

[14] The TLC includes representatives from the following entities: CBP, 
CIS, FLETC, ICE, TSA, Emergency Preparedness and Response/Federal 
Emergency Management Agency, Information Analysis and Infrastructure 
Protection, Science and Technology, the Coast Guard, the Secret 
Service, and US-VISIT. The TLC also includes representatives from the 
following DHS department level organizations: CHCO office, Office of 
the Chief Information Officer, Office of the Inspector General, Office 
of the General Counsel, Office of Civil Rights and Civil Liberties, 
Office of Counter Narcotics, and Office of State and Local Coordination 
and Preparedness/Center for Domestic Preparedness. 

[15] The Government Employees Training Act provides that an agency can 
extend its training courses to employees of other government agencies 
(5 U.S.C. § 4104(2)). A Comptroller General decision reviewed the 
legislative history of this provision and concluded that training can 
be provided on a reimbursable or a nonreimbursable basis, at the 
discretion of the agency offering the training (B-193293, Nov. 13, 
1978). A DHS component that is offering training is authorized to make 
its courses available to other employees of DHS or of other government 
agencies. 

[16] GAO, Results-Oriented Cultures: Implementation Steps to Assist 
Mergers and Organizational Transformations, GAO-03-669 (Washington, 
D.C.: July 2, 2003). 

[17] GAO, Internal Control: Standards for Internal Control in the 
Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 
1999). 

[18] The DHS management directive entitled "Human Capital Line of 
Business Integration and Management" specifies several roles and 
responsibilities for the CHCO and component heads. Roles for the CHCO 
include advising and assisting top DHS officials on training issues and 
designing processes and systems to achieve departmentwide training 
goals. For example, component heads have the role of recruiting, 
hiring, and reviewing human capital officials, including training 
staff. However, DHS has not yet specified, in detail, the 
responsibility, accountability, and authority of the CHCO and the 
component heads, although officials have indicated that they plan to do 
so during fiscal year 2006. 

[19] While several provisions in the Homeland Security Act require the 
Coast Guard to be maintained as a distinct entity and would limit the 
range of management initiatives regarding the Coast Guard, none of them 
would appear to be applicable in this case. We find nothing in the DHS 
management directive on the integration of human capital that 
contravenes these limitations and nothing in the directive would 
reasonably appear to threaten the status of the Coast Guard as a 
distinct entity or otherwise impair its ability to perform statutory 
missions. We have reported on a similar situation with respect to the 
department's acquisition function. See GAO, Homeland Security: 
Successes and Challenges in DHS's Efforts to Create an Effective 
Acquisition Organization, GAO-05-179 (Washington, D.C.: Mar. 29, 2005). 

[20] Similarly, DHS's management directive on human capital integration 
also asserts that the Secret Service is exempted by statute. As with 
the Coast Guard, we are unaware of any specific statutory exemption 
that would prevent the application of the DHS management directive 
regarding human capital, and given the nature of the management 
directive, we do not see any apparent reason to exempt the Secret 
Service. 

[21] GAO-04-546G. 

[22] We have previously reported that organizations undergoing 
successful transformations look for and implement best practices 
wherever they may be found. See GAO-03-669 and GAO, Highlights of a GAO 
Forum, Mergers and Transformation: Lessons Learned for a Department of 
Homeland Security and Other Federal Agencies, GAO-03-293SP (Washington, 
D.C.: Nov. 14, 2002). 

[23] GAO-04-546G. 

[24] GAO-04-546G. 

[25] Donald L. Kirkpatrick, author of Evaluating Training Programs: The 
Four Levels (San Francisco: Berrett-Koehler Publishers, Inc., 1998), 
developed a commonly recognized four-level model for evaluating 
training and development efforts. The fourth level is sometimes split 
into two levels with the fifth level, return on investment, 
representing a comparison of costs and benefits quantified in dollars. 
See Jack J. Philips, Implementing Evaluation Systems and Processes. 

[26] Higher levels of evaluation, and in particular Level V, can be 
challenging to conduct because of the difficulty and costs associated 
with data collection and the complexity in directly linking training 
programs to improved individual and organizational performance. Factors 
to consider when deciding the appropriate level of evaluation include 
estimated costs of the training effort, size of the training audience, 
management interest, program visibility, and the anticipated "life 
span" of the effort. In light of these considerations, an agency may 
decide to evaluate participants' reactions (Level I) for all of its 
training programs, while conducting a return on investment analysis 
(Level V) for only a very few. Each agency will need to consider the 
feasibility and cost-effectiveness of conducting these in-depth 
evaluations, along with budgetary and staffing circumstances that may 
limit the agency's ability to complete such evaluations. 

[27] GAO-04-546G. 

[28] See GAO, Organizational Culture: Techniques Companies Use to 
Perpetuate or Change Beliefs and Values, GAO/NSIAD-92-105 (Washington, 
D.C.: Feb. 27, 1992), and Organizational Culture: Use of Training to 
Help Change DOD Inventory Management Culture, GAO/NSIAD-94-193 
(Washington, D.C.: Aug. 30, 1994). The first of these reports examined 
techniques used by nine large private sector firms to affect their 
organizational cultures. We found that company officials identified two 
techniques that were of very great importance to a successful culture 
change: (1) total commitment of top management and (2) training that 
promotes and develops skills in line with the desired culture. More 
recently, we have reported on the cultural changes and key practices 
necessary for successful transformation. See, GAO, 21st Century 
Challenges: Transforming Government to Meet Current and Emerging 
Challenges, GAO-05-830T (Washington, D.C.: July 13, 2005). 

[29] GAO-04-546G. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: