This is the accessible text file for GAO report number GAO-06-171 
entitled 'DOD Systems Modernization: Uncertain Joint Use and Marginal 
Expected Value of Military Asset Deployment System Warrant Reassessment 
of Planned Investment' which was released on December 15, 2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Subcommittee on Readiness and Management Support, 
Committee on Armed Services, U.S. Senate: 

December 2005: 

DOD Systems Modernization: 

Uncertain Joint Use and Marginal Expected Value of Military Asset 
Deployment System Warrant Reassessment of Planned Investment: 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-171] 

GAO Highlights: 

Highlights of GAO-06-171, a report to the Subcommittee on Readiness and 
Management Support, Committee on Armed Services, U.S. Senate: 

Why GAO Did This Study: 

Because of the importance of the Department of Defense’s (DOD) 
adherence to disciplined information technology (IT) acquisition 
processes in successfully modernizing its business systems, GAO was 
asked to determine whether the Transportation Coordinators’ Automated 
Information for Movements System II (TC-AIMS II) program is being 
managed according to important aspects of DOD’s acquisition policies 
and guidance, as well as other relevant acquisition management best 
practices. TC-AIMS II was initiated in 1995 as a joint services system 
to help manage force and equipment movements within the United States 
and abroad. The U.S. Department of the Army has the lead responsibility 
for managing the system’s acquisition and estimates its life-cycle cost 
to be $1.7 billion over 25 years. 

What GAO Found: 

The Army has managed the TC-AIMS II program in accordance with some, 
but not all, key aspects of DOD’s system acquisition management 
policies and related guidance. These policies and guidance are intended 
to reasonably ensure that investment in a given IT system represents 
the right solution to fill a mission need—and, if it does, that 
acquisition and deployment of the system are handled in a manner that 
maximizes the chances of delivering defined system capabilities on time 
and within budget. The Army has not managed the program in accordance 
with those DOD policies and related guidance, including related federal 
and other best practice guidance, that are intended to reasonably 
ensure that a proposed system is the right solution to meet mission 
needs. Specifically: 

* The Army has not economically justified its investment in TC AIMS II 
on the basis of reliable estimates of costs and benefits. For example, 
the most recent economic justification included cost and benefit 
estimates predicated on all four military services using the system. 
However, two services (U.S. Department of the Air Force and U.S. Marine 
Corps) have stated that they do not intend to use it. 

* The Army has not invested in TC-AIMS II within the context of a well-
defined enterprise architecture, which is an institutional blueprint to 
control program investment decisions in a way that promotes 
interoperability and reduces redundancy among systems. The Army has 
instead focused on aligning TC-AIMS II with its logistics architecture; 
this means that even though TC-AIMS II is intended to be a DOD-wide 
program, it has been based on a service-specific architecture rather 
than a DOD-wide architecture. As a result, it may not properly fit 
within departmentwide plans. 

To its credit, the Army has largely managed the program in accordance 
with key policies and related guidance that are intended to reasonably 
ensure that the acquisition and deployment of a given system are 
handled in a manner that maximizes the chances of delivering defined 
capabilities on time and within budget. However, some aspects of this 
policy and guidance have not been followed. For example, the Army has 
not fully implemented risk management and has not adhered to a key 
feature of performance-based contracting. 

Reasons the Army cited for not following policies and guidance ranged 
from management inattention to lack of training. As a result, the Army, 
among other things, does not know whether the system is the right 
solution. Until this uncertainty and the previously discussed problems 
are addressed, it will remain unclear whether further planned 
investment in TC AIMS II is warranted, and certain aspects of the 
program’s management will be limited. 

What GAO Recommends: 

GAO is making recommendations to the Secretary of Defense to, among 
other things, develop the analytical basis needed to determine if 
continued investment in TC-AIMS II, as planned, represents prudent use 
of limited defense resources. In written comments on a draft of this 
report, DOD concurred or partially concurred with GAO’s 
recommendations. It also described planned actions that are largely 
consistent with GAO’s recommendations. 

www.gao.gov/cgi-bin/getrpt?GAO-06-171. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Randolph C. Hite at (202) 
512-3439 or hiter@gao.gov.

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

TC-AIMS II Has Not Been Managed in Accordance with Certain DOD System 
Acquisition Policies and Related Guidance: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendixes: 

Appendix I: Objective, Scope, and Methodology: 

Appendix II: Assessment of Problem Reports: 

Appendix III: Comments from the Department of Defense: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Table: 

Table 1: TC-AIMS II Management Entities and Their Roles and 
Responsibilities: 

Figures: 

Figure 1: TC-AIMS II Acquisition, Operations, and Maintenance Timeline, 
by Fiscal Year

Figure 2: Number of Problem Reports, Fiscal Years 1999 through 2005: 

Abbreviations: 

DOD: Department of Defense: 

IT: information technology: 

TC-AIMS II: Transportation Coordinators' Automated Information for 
Movements System II: 

Letter December 15, 2005: 

The Honorable John Ensign: 
Chairman: 
The Honorable Daniel K. Akaka: 
Ranking Minority Member: 
Subcommittee on Readiness and Management Support: 
Committee on Armed Services: 
United States Senate: 

Because it is so important that the Department of Defense (DOD) adhere 
to disciplined information technology (IT) acquisition processes in 
order to successfully modernize its business systems, you requested 
that we determine whether the department is following its own IT 
acquisition policies and guidance, which were recently 
revised.[Footnote 1] As part of our response to your request, we agreed 
to review the Transportation Coordinators' Automated Information for 
Movements System II (TC-AIMS II) program. TC-AIMS II was initiated in 
1995 by DOD as a joint services system with the goal of helping to 
manage the movement of forces and equipment within the United States 
and abroad. Assigned the lead responsibility for acquiring the system, 
the U.S. Department of the Army estimates the system has a 25-year, 
life-cycle cost of $1.7 billion. 

Our objective was to determine whether TC-AIMS II is being managed 
according to important aspects of DOD's acquisition policies and 
guidance as well as other relevant acquisition management best 
practices. We focused on the program's (1) economic justification; (2) 
architectural alignment; (3) risk management; (4) requirements 
development and management; (5) commercial component management; and 
(6) contract management, including contractor oversight and performance-
based contracting. 

We conducted our work between October 2004 and October 2005, in 
accordance with generally accepted government auditing standards. 
Details of our objective, scope, and methodology are included in 
appendix I. 

Results in Brief: 

The Army has managed the TC-AIMS II program in accordance with some, 
but not all, key aspects of DOD's system acquisition management 
policies and related guidance. Collectively, DOD's acquisition 
management policies and guidance are intended to reasonably ensure that 
investment in a given IT system represents the right solution to fill a 
mission need--and, if it does, that acquisition and deployment of the 
system are handled in a manner that maximizes the chances of delivering 
defined system capabilities on time and within budget. In particular, 
the Army has not managed TC-AIMS II in accordance with DOD policy and 
related guidance, including related federal and other best practice 
guidance, that are intended to reasonably ensure that a proposed system 
is the right solution to meeting mission needs. 

* The Army has not economically justified its investment in TC-AIMS II 
on the basis of reliable estimates of costs and benefits. The most 
recent economic justification included cost and benefit estimates 
predicated on all four military services using the system. However, two 
services (U.S. Department of the Air Force and U.S. Marine Corps) have 
stated that they do not intend to use it. Even with costs and benefits 
for all four services included, the analysis showed a marginal return 
on investment; that is, for each dollar spent on the system, slightly 
less than one dollar of benefit would be returned. 

* The Army has not invested in TC-AIMS II within the context of a well-
defined enterprise architecture, which is an institutional blueprint to 
guide and constrain program investment decisions in a way that promotes 
interoperability and reduces redundancy among related and dependent 
systems. As we recently reported,[Footnote 2] the version of the 
department's business enterprise architecture that has been available 
to the program has not contained sufficient context (depth and scope of 
operational and technical requirements) to effectively guide and 
constrain system investments. The Army has instead focused on aligning 
TC-AIMS II with the Army's logistics architecture; this means that even 
though TC-AIMS II was to be a DOD-wide program, it has been based on a 
service-specific architecture, rather than a DOD-wide architecture. As 
a result, the system, as defined, may not properly fit within a DOD-
wide enterprise architecture. 

At the same time, the Army has largely managed the program in 
accordance with key policies and related guidance that are intended to 
reasonably ensure that the acquisition and deployment of a given system 
solution are handled in a manner that maximizes the chances of 
delivering defined capabilities on time and within budget. However, 
aspects of this policy and guidance have not been followed. 

* The Army has not fully implemented the department's risk management 
policy and guidance. While the program office has developed a risk 
management plan, it has not effectively implemented it. For example, 
the program office identified 22 active risks to the program but only 
developed mitigation strategies for 16 of these risks, increasing the 
probability that program risks will lead to cost, schedule, and 
performance problems. 

* The Army has not implemented a key feature of performance-based 
contract management. Specifically, the most recent contract to acquire 
TC-AIMS II does not specify effective disincentives or penalties for 
the contractor if it fails to meet performance criteria. Among other 
things, this limits the Army's ability to effectively influence 
contractor performance. 

Reasons that the Army cited for not following relevant policies and 
guidance include lack of management attention and training. By not 
following them, the Army does not have an adequate basis to know 
whether TC-AIMS II is the right systems solution to meet DOD's asset 
deployment support needs, and, thus, it is unclear whether planned 
investment in TC-AIMS II is warranted. Even if the uncertainty is 
adequately addressed, the manner in which the system is being acquired 
can be strengthened. Accordingly, we are making recommendations to the 
Secretary of Defense that are aimed at first developing the basis for 
determining whether continued investment in TC-AIMS II is a prudent use 
of limited resources. We are also making recommendations, conditional 
upon a decision by DOD to proceed with further investment in TC-AIMS 
II, to further enhance the department's adherence to other acquisition 
practices. 

The Office of the Assistant Secretary of Defense for Networks and 
Information Integration provided written comments on a draft of this 
report. In its comments, the department stated that it either concurred 
or partially concurred with our recommendations. It also described 
efforts initiated or planned to bring the program into compliance with 
applicable guidance. In our view, the department's comments are largely 
consistent with our report. These comments, along with our responses, 
are discussed in detail in the Agency Comments and Our Evaluation 
section of this report. The DOD comments are also reprinted in their 
entirety in appendix III. 

Background: 

As one of the largest and most complex organizations in the world, DOD 
spends billions of dollars each year to operate, maintain, and 
modernize IT systems that support core business operations. These 
business operations consist of various interrelated and interdependent 
business functions, including logistics management, procurement, health 
care management, and financial management. Further, execution of these 
business operations span a wide range of defense organizations, 
including the military services and their respective major commands and 
functional activities, defense agencies and field activities, and 
combatant and joint operational commands that are responsible for 
military operations in geographic regions or theaters. For fiscal year 
2005, DOD requested approximately $13 billion to operate, maintain, and 
modernize about 4,150 business systems and related IT infrastructure. 
The Army's share of this funding and systems is about $2.7 billion and 
727 systems. 

In 1995, we designated DOD's business systems modernization efforts as 
a high-risk program, and we continue to designate it as such 
today[Footnote 3] for several reasons, including the department's 
challenges in implementing effective IT investment management 
structures and processes, developing and implementing an enterprise 
architecture, and implementing effective IT system acquisition and 
development processes. 

Overview of TC-AIMS II Genesis, Purpose, and Acquisition Strategy: 

In the early 1990s, DOD recognized the need to integrate defense IT 
systems by ordering the consolidation of systems that serve 
multiservice purposes. Accordingly, it directed that approximately 150 
operating transportation systems be evaluated for consolidation 
opportunities. In 1995, DOD initiated TC-AIMS II as one of a number of 
programs that were commenced following this evaluation to consolidate 
and replace existing systems, and to automate transportation management 
function areas for all services. In doing so, DOD designated the Army 
as the "executive agent" responsible for managing TC-AIMS II, including 
acquiring the system. 

Expected TC-AIMS II benefits included enhancing and improving the 
efficiency and effectiveness of the support planning needed to deploy 
and redeploy forces and equipment, both within the United States and 
abroad; improving the visibility of assets; and enhancing cargo and 
passenger receiving, controlling, and shipping. The system is also 
expected to enable decision makers at various command levels to access 
information about unit movement and installation transportation, and is 
to provide key shared functionality related, for example, to load 
planning. 

The Army defined the following five-block acquisition and 
implementation strategy for TC-AIMS II. According to the strategy, each 
block is to address a functional portion of the requirements. 

* Block 1: This block is to import data about assets (people, 
equipment, and supplies) from the services' existing systems and is to 
manage these data to (1) plan and execute organizational moves (i.e., 
provide documentation, packing and shipping labels, and orders) and (2) 
request and schedule strategic and local transportation vehicles (e.g., 
trucks, ships, airplanes, motor pools, and trucks). 

* Block 2: This block is to add Web access to permit automated data 
exchange with related systems, such as airlift load planning systems 
and global transportation networks. This increment is to link and track 
cargo and personnel (via DOD's standard smart cards), and their 
corresponding transports. It also is to generate and manage the 
documentation associated with these functions. 

* Block 3: This block is to add interfaces to both supply and personnel 
systems to facilitate control over "in theater" operations, such as 
reception (unit arrival and off-loading in theater), staging 
(preparation for departure to assigned locations), onward movement 
(transit to assigned locations), and integration (coordinating and 
linking the movement of multiple related units). It also is to generate 
and manage documentation associated with these functions. 

* Block 4: This block is to add the capability to manage maritime 
forces associated with prepositioned equipment and cargo (e.g., tank 
ammunition for an armored cavalry unit), including the ability to 
manage cargo logistics (loading and off-loading), as well as greater 
capability to manage theater operations, including the availability, 
distribution, and dispatch of vehicle fleets and drivers. It also is to 
generate and manage documentation associated with these functions. 

* Block 5: This block is to automate the management of transportation 
assets and traffic operations at military installations both in the 
United States and worldwide. At this point, TC-AIMS II is to be able to 
generate and manage all data and documentation related to moving units 
within DOD's transportation system. As part of this block, the 
interfaces to systems that are to be integrated with TC-AIMS II are to 
be upgraded or created, as required, in response to changes in 
technology. 

Figure 1 shows the Army's timeline for acquiring the five TC-AIMS II 
blocks. In particular, the Army reports that it is currently acquiring 
Block 3 with the goal of making it operational by late 2006. 

Figure 1: TC-AIMS II Acquisition, Operations, and Maintenance Timeline, 
by Fiscal Year: 

[See PDF for image] 

[End of figure] 

The Army estimates the total life-cycle cost of TC-AIMS II to be $1.7 
billion over 25 years, including $569 million for acquisition and 
$1,168 million for operations and maintenance. The Army reports that it 
has spent approximately $751 million on TC-AIMS II since its inception. 

Description of TC-AIMS II Roles and Responsibilities: 

A number of DOD and Army organizations are involved in managing the TC-
AIMS II program and acquisition. Briefly, they are as follows: 

* The Assistant Secretary of Defense for Network Information and 
Integration, who has overall responsibility for the program. 

* The Army's Program Executive Office for Enterprise Information 
Systems, Transportation Information Systems, oversees the management of 
TC-AIMS II, which is managed on a day-to-day basis by the Project 
Manager, Joint Program Management Office. 

* The Joint Requirements Board represents TC-AIMS II users and includes 
stakeholders from the military services and other DOD organizations. 
The board is responsible for managing TC-AIMS II requirements 
development activities. 

These and other key organizations and their roles and responsibilities 
are described in more detail in table 1. 

Table 1: TC-AIMS II Management Entities and Their Roles and 
Responsibilities: 

Entity: Assistant Secretary of Defense for Network Information and 
Integration; 
Roles and responsibilities: Serves as the chair of the TC-AIMS II IT 
integrated product team and is the milestone decision authority. 
Assigned overall responsibility for the TC-AIMS II program; approves 
the program to proceed through its acquisition cycle on the basis of a 
review of key documents, such as an acquisition plan, an independently 
evaluated life-cycle cost-and-benefit estimate, Acquisition Program 
Baseline documents, and Defense Acquisition Executive Summary reports. 

Entity: Department of the Army, Program Executive Office for Enterprise 
Information Systems, Transportation Information Systems; 
Roles and responsibilities: Serves as the program executive office. 
Assigned overall responsibility for TC-AIMS II program oversight; 
reviews the component cost analysis, acquisition strategy, and 
Acquisition Program Baseline prior to approval by the milestone 
decision authority. 

Entity: Joint TC-AIMS II Management Board; 
Roles and responsibilities: Provides guidance and vision for TC-AIMS 
II. This board includes representatives from the Joint Command, 
Transportation Command, and the Military Services, including the U.S. 
Departments of the Air Force, Army, and Navy and the U.S. Marine Corps. 

Entity: Joint Program Management Office; 
Roles and responsibilities: Serves as the program office. Assigned 
responsibility for day-to-day program management of TC-AIMS II and, as 
such, is the single point of accountability for managing the program's 
objectives through development, production, and sustainment, including 
risk management, management of commercial components, and contractor 
tracking and oversight. Manages cost, schedule, and performance 
reporting. Prepares and updates the acquisition strategy, component 
cost analysis, and acquisition program baselines. Coordinates all 
testing activities with requirements, including measuring system 
quality and managing problem reports[A] and change requests. 

Entity: Joint Requirements Board; 
Roles and responsibilities: Represents the system users. Participates 
in the process of establishing functional requirements. Consists of key 
system stakeholders and representatives and is chaired by the Joint 
Forces Command. The board is responsible for managing requirements 
development activities for TC-AIMS II. Among other things, the board's 
duties include defining, reviewing, validating, prioritizing, and 
approving the system's requirements. 

Source: GAO analysis of DOD data. 

[A] Program officials provided data on, among other things, the number 
and type of problem reports that the program has encountered since 
system initiation. For details, see appendix II. 

[End of table] 

The Army uses a system integration contractor to support the program 
office in managing the acquisition of TC-AIMS II. Most recently, it 
awarded a firm, fixed-price contract to a contractor in April 2004 to 
deliver the Block 3 solution. The contract has 5 option years, which 
are 1 year in duration. To help it oversee contractor activities, the 
program office has employed the help of the General Service 
Administration's Federal Systems Integration and Management Center, 
whose mission is to provide acquisition management expertise and 
support to DOD and other federal agencies on large, complex IT 
projects. 

Prior Review Identified Strengths and Weaknesses in DOD's Acquisition 
Policies and Guidance: 

In July 2004, we reported[Footnote 4] that DOD's revised systems 
acquisition policies and guidance incorporated many best practices for 
acquiring business systems, such as (1) justifying system investments 
economically, on the basis of costs, benefits, and risks, and (2) 
continually measuring an acquisition's performance, cost, and schedule 
against approved baselines. However, the revised policies and guidance 
did not incorporate a number of other best practices, particularly 
those associated with acquiring commercial component-based business 
systems, and DOD did not have documented plans for incorporating these 
additional best practices into its policies. We also reported that the 
department's revised acquisition policies did not include sufficient 
controls to ensure that military services and defense agencies would 
appropriately follow these practices. We concluded that until these 
additional best practices were incorporated into DOD's acquisition 
policies and guidance, there was increased risk that department system 
acquisitions would not deliver planned capabilities and benefits on 
time and within budget, and increased risk that DOD organizations would 
not adopt and use best practices that were defined. Accordingly, we 
made 14 recommendations to the Secretary of Defense that were aimed at 
strengthening DOD's acquisition policy and guidance by including 
additional IT systems acquisition best practices and controls for 
ensuring that these best practices were followed. DOD agreed with most 
of our recommendations and has since issued additional system 
acquisition guidance. 

TC-AIMS II Has Not Been Managed in Accordance with Certain DOD System 
Acquisition Policies and Related Guidance: 

The Army, as DOD's acquisition agent for TC-AIMS II, has not managed 
this program in accordance with certain department acquisition 
policies, and related guidance, that are intended to reasonably ensure 
that a proposed system is the right solution to meet mission needs. In 
particular, the Army has not economically justified the program on the 
basis of reliable estimates of life-cycle costs and benefits, and it 
has not ensured that this program, which is intended to produce a 
departmentwide military deployment management system, is aligned with a 
departmentwide business enterprise architecture or an integrated 
blueprint for business systems modernization. As a result, the Army 
does not know that investment in TC-AIMS II as planned is warranted, 
represents a prudent use of limited DOD resources, and will be 
interoperable with and not be duplicative of related systems. 

Even if TC-AIMS II happens to be the right system, both economically 
and architecturally, the Army has not fully implemented risk management 
and has not fully employed performance-based contracting practices. 
However, it has largely managed system requirements, commercial 
components, and contractor tracking and oversight in accordance with 
DOD policies and related guidance. If the program can be economically 
justified and architecturally defined, then fully addressing these key 
acquisition management areas will be essential to the program's 
success. 

Investment in TC-AIMS II Has Not Been Adequately Justified on the Basis 
of Costs and Benefits: 

DOD's acquisition management policy and related guidance[Footnote 5] 
recognize the importance of developing reliable economic analyses to 
support investment decision making. Accordingly, they describe how 
these analyses should be developed and used. For TC-AIMS II, the Army 
has not adhered to important aspects of departmental policy and related 
guidance, such as updating analyses to reflect material changes in 
program scope and justifying investment in large, multiyear programs on 
an incremental basis. As a result, the Army currently lacks an adequate 
basis to justify its planned investment in TC-AIMS II. 

Latest Economic Analysis Does Not Recognize System's Uncertain Joint 
Use and Shows Marginal Expected Value: 

DOD policy and guidance state that projects should be economically 
justified on the basis of reliable analyses of expected life-cycle 
costs and benefits and that cost and benefit estimates should be based 
on realistic plans for the program's scope. Federal guidance also 
advocates economically justifying proposed investments on the basis of 
a benefit-to-cost ratio that is greater than 1, basing that ratio on 
reliable analyses of expected, quantifiable life-cycle benefits, costs, 
and risks. The guidance also promotes the calculation and consideration 
of qualitative benefits in preparing an economic analysis. Further, the 
guidance calls for updating the analysis when significant program 
changes occur. These practices help to continually ensure a positive 
return on investment--even as program changes occur. 

The program office developed economic analyses to justify investment in 
the system in August 2002 and again in December 2003. The program 
office was supported in its efforts by the Office of the Deputy 
Assistant Secretary of the Army's Office of Cost and Economics and the 
Office of the Secretary of Defense's Office of Program Analysis and 
Evaluation, whose missions include verifying and validating the 
reliability of cost and benefit estimates found in Army and other 
services' economic analyses. However, this most recent economic 
analysis is not based on reliable estimates of life-cycle costs and 
benefits. Specifically, the analysis was based on the assumption that 
TC-AIMS II would be used by all four military services, and, thus, it 
included life-cycle cost estimates showing all of the services' data--
as well as the benefits that would accrue through its use by all four. 
However, this assumption is not valid. Specifically, the Air Force's 
Assistant Deputy Chief of Staff, Installations and Logistics, informed 
the program office in a September 2002 correspondence that it would not 
use the system, stating that the Air Force would rely instead on its 
existing system capabilities. Similarly, in March 2004, the Marine 
Corps' Deputy Commandant for Installations and Logistics informed the 
program office that the Corps would not use the system because of 
concerns about whether the system would, among other things, deliver 
expected capabilities. Program officials told us that the latest 
economic analysis includes the costs and benefits for all services 
because, despite communications to the contrary from the Air Force and 
Marine Corps, the program office still considers TC-AIMS II to be a 
multiservice program and has not yet been directed to assume and plan 
otherwise. 

In addition to the unreliable cost and benefit estimates, the latest 
analysis does not show a benefit-to-cost ratio that is greater than 1. 
Specifically, the analysis identified present value, total expected 
benefits of $928 million versus present value, and total estimated 
costs of $929 million, for a benefit-to-cost ratio of slightly less 
than 1. This means that for each dollar spent on the system, slightly 
less than one dollar of benefit is returned. In addition, the analysis 
cited various qualitative benefits (e.g., increasing commander 
visibility of assets and enhancing worker productivity) that would be 
derived from the system. However, the analysis did not provide any 
underlying support or analysis for these qualitative benefits. 

Program officials said that they recognize that the system's expected 
return on investment is marginal, but they nevertheless have been 
directed by the milestone decision authority--the Assistant Secretary 
for Defense, Networks and Information Integration--to deliver the 
system's capabilities as planned. According to the officials in the 
Assistant Secretary's office, the decision to continue investing in TC-
AIMS II was based on quantitative and qualitative benefits, which they 
stated together exceeded costs, thus making the investment justified. 
However, these officials did not provide supporting justification for 
their qualitative benefit claims. In addition, their position does not 
recognize the intended change in the program's scope that is not 
reflected in the latest analysis. Thus, the Army lacks the basis for 
knowing whether its planned investment in TC-AIMS II is justified. 

Investments Have Not Been Incrementally Justified: 

DOD policy and guidance, as well as other related federal and best 
practice guidance,[Footnote 6] state that large projects, such as TC-
AIMS II, should be divided into a series of smaller, incremental 
subprojects or releases so that investment decisions can be made on 
each increment. By doing this, the tremendous risk associated with 
investing large sums of money over many years in anticipation of 
delivering system capabilities and associated expected business value 
far into the future can be spread across project increments that are 
smaller, of shorter duration, and capable of being more reliably 
justified and more effectively managed against cost, benefit, and risk 
expectations. 

The Army has neither analyzed TC-AIMS II costs and benefits, nor made 
associated investment decisions, on an incremental basis. Specifically, 
while the program office divided the system's acquisition into five 
smaller increments (blocks), the economic analyses addressed life-cycle 
costs and benefits for the entire system. The analyses did not identify 
separately the costs and benefits for each of the five increments, and, 
thus, this information was not considered as program officials made 
decisions about TC-AIMS II at key milestones. Program officials stated 
that they have not analyzed costs and benefits on an incremental basis 
because once the program had been approved and initiated, they did not 
see the need thereafter to justify each increment. By not determining 
the costs and benefits of each of the system's increments, the Army 
runs the risk of discovering too late (i.e., after it has invested 
hundreds of millions of dollars) that TC-AIMS II is not cost-
beneficial. 

TC-AIMS II Has Not Been Defined and Developed within the Context of a 
DOD Enterprise Architecture: 

DOD's acquisition policies and guidance,[Footnote 7] as well as federal 
and best practice guidance,[Footnote 8] recognize the importance of 
investing in IT business systems within the context of an enterprise 
architecture. Our research and experience in reviewing federal agencies 
show that not doing so often results in systems that are duplicative, 
are not well integrated, are unnecessarily costly to interface and 
maintain, and do not optimally support mission outcomes.[Footnote 9] TC-
AIMS II has not been defined and developed in the context of a DOD 
enterprise architecture. Instead, the Army has pursued the system on 
the basis of an Army logistics-focused architecture. This means that TC-
AIMS II as a DOD-wide program is based on a service-specific 
architecture and not a DOD-wide architecture, thus increasing the risk 
that TC-AIMS II, as defined, will not properly fit within the context 
of future DOD enterprisewide business operations and IT environments. 

Well-defined DOD Enterprise Architecture to Guide and Constrain TC-AIMS 
II Has Not Existed: 

A well-defined enterprise architecture provides a clear and 
comprehensive picture of an entity, whether it is an organization 
(e.g., a federal department) or a functional or mission area that cuts 
across more than one organization (e.g., personnel management). This 
picture consists of snapshots of both the enterprise's current or "As 
Is" environment and its target or "To Be" environment, as well as a 
capital investment road map for transitioning from the current to the 
target environment. These snapshots consist of integrated "views," 
which are one or more architecture products that describe, for example, 
the enterprise's business processes and rules; information needs and 
flows among functions, supporting systems, services, and applications; 
and data and technical standards and structures. 

DOD has long operated without a well-defined enterprise architecture 
for its business environment. In 2001, we first reported that DOD did 
not have such an architecture, and we recommended that it develop one 
to guide and constrain IT business systems, such as TC-AIMS 
II.[Footnote 10] Over the next 4 years, we have reported that DOD's 
architecture development efforts were not resulting in the kind of 
business enterprise architecture that could effectively guide and 
constrain business system investments,[Footnote 11] largely because the 
department did not have in place the architecture management structures 
and processes described in federal guidance. In particular, we most 
recently reported in July 2005[Footnote 12] that despite spending about 
$318 million producing eight versions of its architecture, DOD's latest 
version still did not have, for example, a clearly defined purpose that 
could be linked to the department's goals and objectives, and a 
description of the "As Is" environment and a transition plan. Further, 
we reported that the description of the "To Be" environment was still 
missing important content (depth and scope of operational and technical 
requirements) relative to, for example, the actual systems to be 
developed or acquired to support future business operations and the 
physical infrastructure (e.g., hardware and software) that would be 
needed to support the business systems. Over the last several years, we 
have also reported that DOD's efforts for determining whether ongoing 
investments were aligned to its evolving architecture were not 
documented and independently verifiable.[Footnote 13] On September 28, 
2005, DOD's Under Secretary of Defense for Acquisition, Technology, and 
Logistics and Under Secretary for Defense (Comptroller) issued the next 
version of its business enterprise architecture,[Footnote 14] which we 
are required to review, along with other things, such as the 
department's efforts to review certain investments' alignment with the 
architecture, pursuant to the Fiscal Year 2005 National Defense 
Authorization Act.[Footnote 15] 

According to program officials, the system has not been assessed 
against DOD's business enterprise architecture because one has yet to 
be completed. Instead, as described in more detail below, program 
officials told us they have aligned TC-AIMS II with the Army's 
enterprise architecture for logistics. However, without a well-defined 
architecture that sets the DOD-wide context within which a DOD-wide 
system is to fit, and a firm understanding of the extent to which TC-
AIMS II, as defined, fits within that context, the program office risks 
acquiring a system that does not meet DOD business needs and introduces 
redundancies and incompatibilities that require costly and time-
consuming rework to fix. 

Army Reports That TC-AIMS II Is Aligned with an Army-Specific 
Architecture: 

In the absence of a DOD-wide architecture, program officials told us 
they have developed the system in the context of the Army's enterprise 
architecture for logistics--the Single Army Logistical Enterprise, 
which is managed by the Army's Chief Information Officer. For example, 
in 2004, program officials requested the Army's Chief Information 
Officer to determine whether TC-AIMS II was aligned with the logistic 
architecture. The Office of the Chief Information Officer conducted an 
analysis during the fall of 2004 that focused on, among other things, 
tracing TC-AIMS II business processes and requirements to those 
specified in the logistics architecture. The Chief Information Officer 
reported in March 2005 that TC-AIMS II, as defined in Blocks 1 to 3, 
was aligned with the logistics architecture. We did not review, and 
thus do not question, this analysis because TC-AIMS II is intended to 
be a joint system, and thus aligning it solely with the Army's 
logistics architecture will not provide the program with sufficient 
basis for managing the risk of not being able to adequately ensure that 
a DOD-wide system aligns with a DOD-wide architecture. 

Risk Management Has Not Been Effectively Implemented: 

Effective risk management is vital to the success of any system 
acquisition. Accordingly, DOD acquisition management policies and 
guidance,[Footnote 16] as well as other relevant best practice 
guidance,[Footnote 17] advocate proactively identifying facts and 
circumstances that can increase the probability of an acquisition's 
failing to meet cost, schedule, and performance commitments and then 
taking steps to reduce the probability of their occurrence and impact. 
Effective risk management includes: 

* developing written policies and procedures; 

* assigning roles and responsibilities for managing risk; 

* developing a risk management plan that provides for (1) identifying 
and prioritizing risks, (2) developing and implementing the appropriate 
risk mitigation strategies, and (3) tracking and reporting on progress 
in implementing the strategies; and: 

* executing the plan. 

To its credit, the program office has satisfied the first three of 
these four key practices. First, it has adopted and uses the risk 
policies and procedures specified in DOD's major IT systems acquisition 
guidance.[Footnote 18] Second, the program office has assigned risk 
management roles and responsibilities to the Government Risk Manager, 
who chairs the program's Risk Management Board. The board, which 
consists of key program managers and stakeholder 
representatives,[Footnote 19] is responsible for ensuring that the risk 
management program is implemented according to the risk management 
plan. Among other things, this includes assigning individuals who are 
to develop and implement risk mitigation plans for all newly identified 
risks. Third, the program office has a risk management plan, dated 
October 2004, and this plan provides for (1) identifying and 
prioritizing risks, (2) developing and implementing the appropriate 
mitigation strategies, and (3) tracking and reporting on progress in 
implementing the strategies. For example, the plan defines procedures 
for program staff and the board to follow in prioritizing risks, 
including assessing its "risk exposure" on the basis of a probability 
of occurrence and potential impact. On the basis of the assessment, 
"risk exposure" is designated as either high, medium, or low.[Footnote 
20] As of May 2005, the TC-AIMS II program office had identified 22 
active risks in its risk inventory with the following exposures--1 
high, 18 medium, and 3 low. 

Fourth, since the program office has not fully implemented its risk 
management plan, it has not satisfied the fourth risk management 
practice--execution. For example, of the 22 active risks, 6 did not 
have mitigation strategies. Of these 6, 1 is high exposure and 5 are 
medium exposure risks. As another example, the program office's 
inventory does not include all key risks, such as the lack of reliable 
economic justification and alignment to a DOD-wide architecture as 
previously described in this report. 

Program office officials stated that these risk management practices 
have not been fully implemented because of other competing priorities. 
Further, officials attributed the lack of risk mitigation strategies to 
staff turnover, which resulted in relatively few staff receiving 
training on how to use the automated tool employed by the office to 
document and track risks. In particular, program officials stated that 
only about 20 percent of the program staff (40 individuals) has 
attended the program's training on how to use the automated tool. 

When we concluded our audit work, program officials acknowledged the 
missing risk mitigation strategies and stated that they are in the 
process of addressing them. Until the program fully implements 
effective risk management practices, there is increased probability 
that program risks will become actual problems leading to program cost, 
schedule, and performance shortfalls. 

System Requirements Are Being Effectively Managed: 

DOD policy and guidance, as well as other relevant best 
practices,[Footnote 21] recognize the importance of effectively 
developing and managing system requirements. According to the policy 
and guidance, well-defined requirements are important because they 
establish agreement among the various stakeholders on what the system 
is to do, how well it is to do it, and how it is to interact with other 
systems. Having this agreement is key to acquiring a system that meets 
end-user needs and performs as intended. 

Effective requirements development and management involve, among other 
things, (1) establishing a written policy on establishing and 
conducting requirements development and management activities; (2) 
eliciting desired and required system capabilities from users and 
translating them into system requirements; (3) documenting the 
requirements, including having users validate that they have been 
accurately captured; (4) ensuring that requirements changes are 
controlled as the system is developed and implemented; and (5) 
maintaining bidirectional traceability, meaning that a given 
requirement can be traced backward to its source and forward to both 
the component or increment that will satisfy the requirement and the 
test that will verify that it is satisfied. 

The program office is managing TC-AIMS II requirements in accordance 
with these five practices. First, the program office adopted DOD 
Instruction 5000.2 and related guidance,[Footnote 22] as the program's 
policy for requirements development and management activities. The 
directive and guidance are also incorporated in program office plans 
that detail how the office develops and manages requirements during 
their life cycle--that is, from when requirements are first documented 
to when system responsibility is transferred to operations and 
maintenance support. According to program officials responsible for 
managing requirements, they use the guidance and plans to perform their 
work. 

Second, program officials elicited required system capabilities from 
end users through the program's Joint Requirements Board, which 
consists of key system stakeholders and representatives and is chaired 
by the Joint Forces Command. The board is responsible for managing 
requirements development activities. Among other things, the board's 
charter states that its duties include defining, reviewing, validating, 
prioritizing, and approving the system's requirements. Board meeting 
minutes show evidence of it performing these duties, including, for 
example, reviewing the Block 3 baseline requirements document on 
September 8, 2004. 

Third, program officials showed us how they document the requirements, 
using an automated tool. Further, the Joint Requirements Board approved 
the requirements baseline for each of the three blocks. 

Fourth, to manage system requirement changes, the program office 
established a change control structure (configuration control board) 
and supporting processes, which are documented in its 2004 
configuration management plan. Consistent with the plan, the board has 
six voting members and includes representatives from the four services. 
As shown in board meeting minutes and supporting documentation, the 
board meets monthly to consider and decide on change requests. 

Fifth, the program maintains bidirectional traceability for the 
requirements using an automated tool.[Footnote 23] Specifically, 
program officials demonstrated to us how each requirement can be traced 
back and to its source (e.g., TC-AIMS II's operational requirements 
document) and forward to the block in which the requirement is to be 
satisfied. 

Important Steps Have Been Taken to Effectively Leverage Commercial 
Components: 

The quality of the processes and practices followed in acquiring 
software-intensive systems greatly influences the quality of the 
systems produced. Moreover, acquiring custom-developed system solutions 
is sufficiently different from acquiring commercial component-based 
systems that adherence to certain practices unique to the latter is key 
to their success. As we have previously reported,[Footnote 24] DOD's 
revised systems acquisition policies and guidance[Footnote 25] require 
that commercial components be used to the maximum extent that is 
feasible. Collectively, they also provide for the following commercial 
component acquisition management best practices: 

* modifying components only after a thorough analysis of life-cycle 
costs and benefits, 

* ensuring that integration contractors are explicitly evaluated on 
their ability to implement commercial components, and: 

* ensuring that project plans provide for the time and resources needed 
to integrate commercial components with existing (legacy) systems. 

The Army has largely complied with these practices for TC-AIMS II. 
First, program officials stated that it is their policy to discourage 
component modifications, which they have communicated in meetings and 
other program management directives. In addition, the program has 
established a joint configuration control board and supporting 
processes to limit and control changes to the commercial products being 
used. As stated in the program's configuration management plan (dated 
June 30, 2005), the board, which consists of key program managers and 
stakeholder representatives, only approves proposed changes to those 
products if the change is justified by a thorough analysis of costs, 
benefits, and risks. The board meets monthly to discuss and decide upon 
such changes, and, according to program officials, this approach has 
resulted in no major modifications to date. 

Second, the program office evaluation of competing contractors included 
consideration of their ability to implement commercial components. For 
example, the TC-AIMS II request for proposal specified that the 
contractor's ability to implement commercial products was to be a 
significant factor in evaluating overall performance. 

Third, the program's work breakdown schedule (dated Nov. 9, 2004) 
specifies the time and resources needed for integrating commercial 
components with existing systems. 

By following these best practices associated with acquiring commercial-
based systems, the program is increasing the likelihood that TC-AIMS II 
will be successfully implemented and effectively used. 

Effective Contractor Management Practices Have Not Been Fully Employed: 

DOD policy and guidance and other relevant system acquisition 
management best practices recognize the importance of effectively 
managing contractor activities. To this end, policy and guidance call 
for, among other things, performing contractor tracking and oversight 
activities and using performance-based contracts to the maximum extent 
practicable. To its credit, the Army has performed key activities 
related to tracking and oversight. However, it has not fully 
implemented one key aspect of performance-based contracting. By not 
doing so, the program office has limited its ability to effectively 
influence contractor performance. 

Effective Contractor Tracking and Oversight Practices Have Been 
Followed: 

Department policy and guidance and related best practices 
guidance[Footnote 26] identify effective contractor tracking and 
oversight as a key contract management activity and define a number of 
associated practices to follow, including: 

* establishing a written policy on contract tracking and oversight; 

* designating responsibility for contract tracking and oversight 
activities and having contracting specialists perform these activities; 
and: 

* using approved contractor planning documents as part of periodic 
reviews and interchanges with the contractor, during which actual cost 
and schedule performance is compared with the contractor's planned 
budgets and schedules to identify variances and issues. 

The program office's tracking and oversight of the systems integration 
contractor from Block 3 largely satisfies these three practices. First, 
the program office has adopted the department's acquisition management 
polices and guidance as its written program-specific policy on 
contractor tracking and oversight. 

Second, the program office assigned responsibility for contract 
tracking and oversight to its Budget Management Division, which has 
three staff devoted to these duties. The devoted staff are contracting 
specialists on detail from the General Services Administration's 
Federal Systems Integration and Management Center. 

Third, the program office uses approved contractor planning documents 
as the basis for overseeing the contractor. These planning documents 
are specified in the contract and include a program management plan, a 
master project schedule, and program status reports. Collectively, 
these documents delineate when and how products and services are to be 
delivered. According to program officials, these planning documents are 
used during monthly status meetings with the contractor to track 
progress. In particular, they are used to compare actual cost and 
schedule performance to date with planned budgets and schedules to 
identify potential variances. 

Program officials attribute the state of their contract tracking and 
oversight capability to two factors. First, the program has been in 
existence for about 10 years, thus allowing time for the office to 
develop disciplined processes in this area. Second, in hiring the 
Federal Systems Integration and Management Center contract specialists, 
the program has subject matter experts who know the importance of, and 
who actively advocate, rigorously following the processes. By employing 
key contract tracking and oversight practices, the program office is 
increasing the likelihood that the contractor will perform as expected. 

Performance-Based Contracting Approach Is Not Fully Consistent with 
Policy and Best Practices: 

Department policy, federal acquisition regulations, and other relevant 
best practices[Footnote 27] also advocate the use of performance-based 
contracting when acquiring services. Effective performance-based 
contracting includes: 

* defining clearly the work to be performed, 

* specifying performance standards (quality and timeliness) that are 
tied to contractual requirements, 

* establishing positive and negative incentives that are tied to the 
contractor's performance, and: 

* having a quality assurance plan that describes how the contractor's 
performance in meeting requirements will be measured against standards. 

The program office's approach to managing the contract is largely 
consistent with these four practices. First, the contract for Block 3 
(dated Apr. 30, 2004) includes a statement of objectives that define 
the work to be performed. For example, the contract identifies specific 
project tasks (e.g., establishing a Block 3 development help desk for 
system users) and associated project management products (e.g., program 
management plan, program schedule, and program status reports) that the 
contractor is to perform and deliver, respectively, in accordance with 
the contract. 

Second, the contract includes performance standards (measures) that are 
contractually required. For example, it states that the help desk is to 
be staffed 24 hours per day, 7 days a week, with help operators capable 
of resolving 80 percent of user calls within 1 hour, while limiting 
caller on-hold time to 3 minutes or less. 

Third, the contract provides positive incentives that are tied to 
contractor performance. That is, in addition to the base contract (cost 
plus fixed fee) amount, the contract includes an additional 8 percent 
fee that can be awarded to the contractor for satisfying specified 
performance criteria. 

Fourth, the latest quality assurance plan, dated August 2004, describes 
how the contractor's performance will be measured and what remedial 
steps are to be taken in case the contractor does not meet contractual 
requirements. 

However, the contract does not specify effective disincentives or 
penalties for failing to meet the performance criteria. More precisely, 
the contract does not provide for forfeiting all or part of the fee if 
the contractor fails to perform as expected. Instead, program officials 
stated that their practice of withholding the additional award fee when 
expectations are not met, constitutes a disincentive. For example, they 
said that they withheld the award fee for the contractor's first 
performance period (which totaled about $900,000) because the 
contractor failed to develop two deliverables on time (the program 
management plan and the performance metrics). 

We do not agree that this practice constitutes an effective negative 
incentive because it merely withholds an incentive that is tied to 
exceeding expectations. It does not penalize performance that fails to 
meet expectations. Moreover, program officials told us that in the 
above example, withholding the award fee was not a forfeiture, because 
it was actually deferred to the second performance period when the 
contractor had another opportunity to earn it. According to program 
officials, during the second performance period, the contractor earned 
almost $780,000 of the withheld fee from the first period by 
completing, among other things, first period tasks. 

In the absence of specific negative incentives that are tied to 
intended performance, the program is at increased risk in the case of 
contractor nonperformance. This practice puts the program at risk of 
taking more time and spending more money than necessary to acquire 
contract deliverables and services. 

Conclusions: 

It is unclear whether the Army's planned investment in TC-AIMS II is 
warranted. Of critical concern is the absence of reliable analysis 
showing that further investment will produce future mission benefits 
commensurate with estimated costs. This is due in large part to the 
evident change in the scope of the program, which was initiated on the 
basis of the four military services using TC-AIMS II but has not been 
revised to reflect that two services have stated their intentions not 
to use the system. Compounding this uncertainty is the inherent risk of 
defining and developing this multiservice system outside the context of 
a well-defined DOD-wide enterprise architecture. Without this 
information, the Army cannot currently determine whether TC-AIMS II, as 
defined and as being developed, is the right solution to meet its 
strategic business and technological needs. 

If these uncertainties are addressed and the Army demonstrates that TC-
AIMS II plans are the right course of action, the department largely 
has the capabilities that are essential to successful system 
acquisition and deployment, although there is room for strengthening 
its practices related to risk management and performance-based 
contracting. 

It is vital that Army and DOD authorities responsible and accountable 
for ensuring prudent use of limited resources first reassess whether 
allowing TC-AIMS II to continue as planned is warranted, and that the 
decision on how to proceed be based on reliable data about program 
costs, benefits, risk, and status. 

Recommendations for Executive Action: 

We recommend that the Secretary of Defense direct the Secretary of the 
Army to determine if continued investment in TC-AIMS II as planned 
represents a prudent use of the department's limited resources. To 
accomplish this, the Secretary of the Army should take the following 
three actions: 

* collaborate with the Office of the Assistant Secretary of Defense for 
Networks and Information Integration/Chief Information Officer, the 
Office of Program Analysis and Evaluation, and the Army Cost Analysis 
Division to prepare a reliable economic analysis; 

* ensure that development of this economic analysis (1) complies with 
cost-estimating best practices and relevant Office of Management and 
Budget cost benefit guidance, (2) incorporates available data on 
whether deployed TC-AIMS II capabilities are actually producing 
benefits, and (3) addresses that the Air Force and Marines are not 
planning to use the system; and: 

* collaborate with the Undersecretary of Defense for Acquisition, 
Technology, and Logistics and the Under Secretary of Defense 
(Comptroller) to ensure that TC-AIMS II is adequately aligned with the 
evolving DOD business enterprise architecture. 

In addition, we recommend that the Secretary of Defense direct the 
Secretary of the Army to present the results of these analyses to the 
Deputy Secretary of Defense, or his designee, and seek a departmental 
decision on how best to proceed with the program. Until this is done, 
we recommend that the Secretary of Defense direct the Secretary of the 
Army to limit future investment in already deployed applications to 
essential operation and maintenance activities and only developmental 
activities deemed essential to national security needs. 

If--on the basis of reliable data--a decision is made to further 
develop TC-AIMS II, we recommend that the Secretary of Defense direct 
the Secretary of the Army to ensure that the program implements 
effective program management activities related to risk management and 
performance-based contracting. 

Agency Comments and Our Evaluation: 

In its written comments on our draft report, signed by the Deputy to 
the Assistant Secretary of Defense for Networks and Information 
Integration (Command, Control, Communications, Intelligence, 
Surveillance, and Reconnaissance and Information Technology 
Acquisition) and reprinted in appendix III, DOD either agreed or 
partially agreed with our recommendations, and it stated that some of 
our findings have merit and that it has significant efforts ongoing to 
comply with applicable guidance. For example, while DOD agreed that TC-
AIMS II was originally defined and implemented without a complete and 
formal enterprise architecture and that the program had not identified 
the lack of a mature architecture as a program risk, it stated that the 
program is taking steps to ensure alignment to relevant enterprise 
architectures and is thereby mitigating this risk. In addition, 
although the department agreed that the latest economic analysis shows 
a negative return on investment, it said that other intangible benefits 
not included in the analysis, such as improved planning and equipment 
tracking, bolster the value of TC-AIMS II. Further, it agreed that the 
system was originally envisioned as a single DOD-wide solution and that 
the Air Force and Marine Corps are no longer intending to use the 
system, but added that it was now evaluating whether these two 
services' existing (legacy) systems should be enabled to share data 
with TC-AIMS II, rather than be replaced in their entirety by a joint 
system as was originally envisioned. According to DOD, the results of 
this evaluation are to be considered at the next milestone review, 
which is planned for early to mid-2006. 

These comments are largely consistent with our report. Specifically, 
the report makes clear that DOD is managing TC-AIMS II in accordance 
with some key practices and has efforts under way, such as those 
previously cited, intended to bring the program into compliance with 
other key practices. However, the report also points out noncompliance 
in areas intended to provide sufficient information to determine how 
the TC-AIMS II program should proceed. Accordingly, it contains 
recommendations to ensure that the economic analysis is updated to 
(among other things) reflect current operating assumptions, and that 
the program is adequately aligned with the department's evolving 
enterprise architectures. It is precisely because the program does not 
have this information that, despite having invested $751 million over 
11 years, the department does not know whether the program as defined 
is the right solution to meet DOD-wide asset deployment needs. 

DOD offered five detailed comments on our four recommendations. The 
department's comments and our responses are as follow: 

1. DOD stated that the TC-AIMS II program has used best practices and 
followed all economic analysis requirements specified in relevant DOD 
and OMB guidance, and has successfully passed milestone reviews at 
which it was required to show costs and benefits for both the entire 
system and its increments. In addition, the department stated that even 
though its economic analysis showed a slight negative return on 
investment, the system's intangible benefits bolster the program's 
overall value. According to DOD, the program identified these 
intangible benefits to obtain department milestone approval, and this 
documentation was provided to us on October 31, 2005. This 
notwithstanding, the department stated the program will (1) conduct, in 
coordination with the Office of the Secretary of Defense's Program 
Analysis and Evaluation, an analysis to determine the relationship of 
TC-AIMS II and other programs, such as the Global Combat Support 
System, that may provide similar functionality to ensure optimal return 
on investment and (2) brief the Networks and Information Integration 
Overarching Integrated Product Team in preparation for the next major 
milestone decision review. It also stated that these results will be 
used to justify further investment. 

We do not agree that the program has used all best practices and 
followed all requirements specified in relevant guidance. In 
particular, the Army did not economically justify its TC-AIMS II 
investment on the basis of reliable estimates of costs and benefits and 
did not invest in TC-AIMS II within the context of a well-defined 
architecture, both of which are recognized best practices as well as 
DOD and OMB requirements. In addition, while it is appropriate to 
consider intangible benefits in economically justifying an investment, 
these benefits are not in the economic analysis, and the separate 
document provided on October 31, 2005, did not contain sufficient 
rationale and justification for claimed qualitative benefits. Rather, 
the information provided was a restatement of the quantitative benefits 
in the economic analysis. Further, by stating that it is to conduct an 
analysis of the relationships of TC-AIMS II to other programs that 
provide similar functionality and use the results of this analysis to 
justify further system development, DOD is recognizing that it does not 
have the information it needs to make informed investment decisions. If 
done thoroughly, the analysis, which the department committed to 
conduct, should help to inform future program decisions. 

2. Consistent with our report, DOD stated that TC-AIMS II was 
established as a joint military service program. However, in explaining 
why the Air Force and Marine Corps are not intending to use TC-AIMS II, 
it added that changes to operations throughout the world since then 
have necessitated changes to the services' deployment processes. In 
light of this, the department stated it will evaluate other options for 
data exchange among Air Force and Marine Corps existing asset 
deployment systems and TC-AIMS II, instead of having all the services 
use a single, joint system. Further, the department said it will have 
the program, in coordination with Program Analysis and Evaluation, 
develop an economic analysis for TC-AIMS II that is based on the system 
being used solely by the Army and the Navy. DOD also commented that our 
report assumed a positive cost-benefit impact from including the Air 
Force and Marine Corps in the program. 

We support the department's decision to develop an economic analysis 
that reflects this material change in the program's scope, which is 
what we recommended. However, we would note that DOD's comments do not 
explain how unspecified changes to operations throughout the world 
caused unspecified changes to each of the services' deployment 
processes, which in turn required DOD to have to interface multiple 
service-unique systems, rather than adopting a joint system. We would 
expect such an explanation to be an integral part of updating the 
economic analysis, as will the costs of pursuing service-unique courses 
of action. 

With regard to the comment that we assumed the inclusion of the Air 
Force and Marine Corps would have a positive cost-benefit impact, we do 
not agree. We made no assumption about the impact of the program's 
scope on the cost-to-benefit or return on investment ratios. Rather, 
our point is that DOD policy and related guidance state that programs 
should be economically justified by, among other things, cost and 
benefit estimates that are based on realistic plans for the program's 
scope. Such realistic plans did not exist because DOD's justification 
was based on the four services using the system, when in reality the 
Air Force and Marine Corps do not intend to do so. Relevant guidance 
calls for updating economic analyses when significant program changes 
such as these occur. Accordingly, we recommended that the department 
follow this guidance to ensure it had adequate information for informed 
decision making. 

3. DOD commented that it has worked over the past several years to 
develop an enterprise architecture and is on a path to fully defining 
and effectively managing its enterprise architecture. It also said that 
it is working to align TC-AIMS II with relevant evolving architectures. 
In particular, DOD stated that Blocks 1 through 4 are aligned with the 
Joint Deployment Operational Architecture and the program office 
continues to work to align the program with the Joint Deployment and 
Distribution Architecture, which it said is currently aligning to the 
evolving Business Enterprise Architecture. 

We do not take issue with the comments that work has been and is under 
way in this area. Our point is that the Business Enterprise 
Architecture is still a work in progress and what has been available to 
guide and constrain business system investments, such as TC-AIMS II, 
has not been sufficient. Moreover, program officials told us that TC-
AIMS II has not been assessed against the Business Enterprise 
Architecture because this architecture has not yet been completed. 
Without a well-defined DOD-wide architecture that sets the context 
within which such a joint military service system is to fit, and a firm 
understanding of the extent to which TC-AIMS II, as defined, fits 
within that context, the program office risks, among other things, 
acquiring a system that does not meet DOD's strategic business needs. 

4. DOD stated that while comprehensive mitigation and contingency 
strategies were not developed for all of the 22 risks cited in our 
report, the program office has subsequently developed strategies for 
them. DOD also commented that subsequent risk management data, along 
with evidence that risks are continuously reviewed and updated, was 
provided to us on October 28 and November 3, 2005. 

We acknowledge that program officials orally stated that this had 
occurred. However, we have yet to receive any documentation to 
substantiate these statements. 

5. DOD stated that while the current TC-AIMS II contract does not 
include direct disincentives for contractor failure to meet performance 
criteria, its semiannual award fee mechanism serves a similar purpose. 
Nonetheless, DOD commented that it will review its award fee 
implementation on future TC-AIMS II contract actions. 

We do not agree that this semiannual award fee approach is an effective 
disincentive because award fees are only being withheld when contractor 
performance does not exceed certain targets; conversely, direct 
disincentives penalize poor performance that falls short of meeting 
contractually required levels of performance. Without direct 
disincentives, the program is at increased risk of contractor 
nonperformance, and thus taking more time and spending more money than 
necessary to acquire contract deliverables and services. We support the 
department's stated commitment to address this issue. 

We are sending copies of this report to interested congressional 
committees. We are also sending copies to the Director, Office of 
Management and Budget; the Secretary of Defense; the Deputy Secretary 
of Defense; the Undersecretary of Defense for Acquisition, Technology, 
and Logistics; the Assistant Secretary of Defense (Networks and 
Information Integration)/Chief Information Officer; the Secretaries of 
the Air Force, Army, and Navy; the Commandant of the Marine Corps; the 
Department of the Army's Chief Information Officer; and the Program 
Executive Officer, Enterprise Information Systems, Transportation 
Information Systems, Department of the Army. We are also sending copies 
to other interested parties. This report will also be available at no 
charge on our Web site at http://www.gao.gov. 

Should you have any questions about matters discussed in this report, 
please contact me at (202) 512-3439 or [Hyperlink, hiter@gao.gov]. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. A GAO contact and 
staff who made major contributions to this report are listed in 
appendix IV. 

Signed by: 

Randolph C. Hite: 
Director, Information Technology Architecture and Systems Issues: 

[End of section] 

Appendixes: 

Appendix I: Objective, Scope, and Methodology: 

Our objective was to determine whether the Transportation Coordinators' 
Automated Information for Movements System II (TC-AIMS II) was being 
managed according to important aspects of the Department of Defense's 
(DOD) acquisition policies and guidance, as well as other relevant 
acquisition management best practices. To accomplish this, we focused 
on the program's (1) economic justification; (2) architectural 
alignment; (3) risk management; (4) requirements development and 
management; (5) commercial components; and (6) contract management, 
including contractor oversight and performance-based contracting. 

To determine whether the Department of the Army had economically 
justified its investment in TC-AIMS II, we reviewed the latest economic 
analysis to determine the basis for the cost and benefit estimates and 
net present value calculations. This included evaluating the analysis 
against DOD policies and guidance[Footnote 28] as well as other 
relevant best practices guidance.[Footnote 29] It also included 
interviewing responsible program officials, including the program and 
deputy program managers, regarding their respective roles, 
responsibilities, and actual efforts in developing and/or reviewing the 
economic analysis. In addition, we also interviewed the program and 
deputy program manager and the Office of the Assistant Secretary of 
Defense for Network Information and Integration about the purpose and 
use of the analysis for managing the Army's investment in the TC-AIMS 
II program, including the extent to which measures and metrics showed 
that benefits projected in the economic analysis were actually being 
realized. 

To determine whether the Army had aligned TC-AIMS II to the DOD 
business enterprise architecture,[Footnote 30] we relied on our prior 
reports addressing DOD and Army architecture development and 
implementation efforts and documents on the Army's efforts to align TC-
AIMS II with its logistics architecture (the Single Army Logistics 
Enterprise), as well as documents on the Army's efforts to align the 
logistics architecture with DOD's business enterprise architecture. We 
also interviewed officials from the TC-AIMS II joint program management 
office, the Office of the Assistant Secretary of Defense for Networks 
and Information Integration/Chief Information Officer, and the Office 
of the Army's Chief Information Officer about DOD and Army architecture 
efforts and TC-AIMS II's alignment to them. 

To determine whether the Army was effectively managing key system 
acquisition activities, namely risk management, requirements 
development and management, commercial components, and contract 
management, we did the following: 

* To assess risk management, we reviewed the program's risk policies, 
processes, management plan, and related documentation. We also observed 
how the program used an automated tool (database) to document and track 
risks, including developing risk mitigation strategies. In addition, we 
analyzed a version of the database to determine, among other things, 
the state of risk mitigation strategies. We then compared these 
activities with DOD policy, guidance, and related best practices to 
determine whether variances existed, and if so, why. Moreover, we 
interviewed Army officials involved with risk management, including the 
program manager and risk manager, to discuss their roles and 
responsibilities for managing risk associated with acquiring and 
implementing TC-AIMS II. 

* To assess requirements development and management capabilities, we 
reviewed program documentation, such as the official list of 
requirements, and system specifications and evaluated them against 
these relevant best practices[Footnote 31] for several characteristics, 
including traceability and prioritization. We observed program office 
staff trace requirements to both higher level documents and lower level 
specifications using an automated tool. We interviewed Army officials 
involved in the requirements development process, including the program 
and deputy program managers, to discuss their roles and 
responsibilities for developing and managing requirements. 

* To evaluate the management of commercial components, we reviewed a 
variety of documents, such as program and contractor studies of 
commercial components' products, as well as the program office's 
project plans, and contract task order documentation. Additionally, we 
conducted interviews with the program officials, including the program 
and deputy program managers, responsible for selecting the commercial 
components for planned TC-AIMS II blocks, to determine whether the 
commercial component products had been acquired in accordance with DOD 
policy, guidance, and related best practices. 

* To assess contract management, we focused on the program office's 
contract tracking and oversight activities as well as its use of 
performance-based contracting. For tracking and oversight, we reviewed 
program management plans, contractor monthly status reports, and 
related documentation of the program office's policies, processes, and 
activities for overseeing its system integration contractor for Block 
3. We then compared these with DOD policies, guidance, and related best 
practices. We also interviewed program offices responsible for managing 
these activities. For performance-based contracting, we reviewed the 
Block 3 contract (issued in April 2004) and related documentation, such 
as contract modifications, quality assurance plan, and reports on 
determining contractor award fee, and then compared them with 
department policy and guidance, federal acquisition regulations, and 
related best practices. We also interviewed program management 
officials, including the contract specialist from the General Service 
Administration's Federal Systems Integration and Management Center, who 
are responsible for performing these activities. 

We did not independently validate information on the program's cost and 
budget as well as the number of problem reports. 

We conducted our work at DOD headquarters in the Washington, D.C., 
metropolitan area and at the Army's Joint Program Management Office in 
Northern Virginia. We performed our work from October 2004 through 
October 2005, in accordance with generally accepted government auditing 
standards. 

[End of section] 

Appendix II: Assessment of Problem Reports: 

One indicator of system quality is defect density, which can be 
measured in a number of ways, including the trend in the number of 
system defects being reported and resolved. The program office prepares 
problem reports to document and prioritize system defects, and it 
established a database to manage and track these reports. Further, the 
program office assigns priority (i.e., criticality) levels to problem 
reports, with category 1 being the most critical, and category 5 being 
the least critical. The program office's definition of each category is 
as follows: 

Category 1: The system or function does not work and the problem must 
be addressed immediately, before any further work can be completed. 

Category 2: The system or function may work, but the problem is severe 
and must be addressed before the end of the current block. 

Category 3: Essentially, this is a category 2 problem with a known 
solution that would provide a temporary fix. 

Category 4: A routine problem that does not pose a significant threat 
and thus no immediate action is required. 

Category 5: A minor problem that does not pose a significant threat to 
system functionality. 

As depicted in figure 2, the data show that the number of open (yet to 
be resolved) problem reports over the first two development blocks has 
decreased. In particular, the total number of open problem reports has 
decreased by 321 (or 75 percent), from 426 in fiscal year 1999 to 105 
at the end of fiscal year 2004. During the same period, categories 1 
and 2 problem reports decreased by 216 (or about 100 percent), from 217 
to 1. This downward trend reversed for the less critical defects in 
fiscal year 2005. Specifically, as of September 30, 2005, there were 
534 total defects, 25 of which were categories 1, 2, and 3. Program 
officials attributed this reversal of categories 4 and 5 defects to 
current Block 3 testing activities and stated that they anticipated the 
increase and are actively managing it. In addition, they stated that 
while overall defects have recently increased, the number of critical 
defects has increased only slightly, and program staff are working to 
resolve these increases, with the goal of having permanent solutions 
implemented before Block 3 is deployed, which is scheduled to begin in 
October 2006. 

Figure 2: Number of Problem Reports, Fiscal Years 1999 through 2005: 

[See PDF for image] 

[End of figure] 

[End of section] 

Appendix III: Comments from the Department of Defense: 

OFFICE OF THE ASSISTANT SECRETARY OF DEFENSE: 
NETWORKS AND INFORMATION INTEGRATION: 
6000 DEFENSE PENTAGON: 
WASHINGTON, DC 20301-6000: 

DEC 06 2005: 

Mr. Randolph C. Hite: 
Director, Information Technology Architecture and Systems Issues: 
U.S. General Accounting Office: 
441 G Street, N.W. 
Washington, D.C. 20548: 

Dear Mr. Hite: 

This is the Department of Defense (DoD) response to the GAO Draft 
Report, GAO-06-171, "DOD SYSTEMS MODERNIZATION: Uncertain Joint Use and 
Marginal Expected Value of Military Asset Deployment System Warrant 
Reassessment of Planned Investment," dated November 15, 2005 (GAO Code 
310295). 

We appreciate the opportunity to comment on the draft report and the 
time your staff afforded us during their preparation of the report. 

The Transportation Coordinators'-Automated Information for Movements 
System II(TC-AIMS II) is an Acquisition Category I AM program. It was 
established as a Joint Program in 1999. The lead service is the United 
States Army. Program oversight is the responsibility of the Assistant 
Secretary of Defense, Networks and Information Integration. An 
oversight process is being followed that includes an Overarching 
Integrated Product Team chaired by the undersigned. Day-to-day 
management of the program falls within the authority of the Army 
Program Executive Officer for Enterprise Information Systems, (PEO 
EIS). 

The Army, as lead service, provided significant analysis and comment 
regarding the report. While our response recognizes that some of the 
report's findings have merit, we point out significant effort that is 
ongoing to comply with applicable guidance and direction. Although TC-
AIMS II was originally defined and implemented without complete and 
formal enterprise architecture design, the Army Program Management 
Office has made an extraordinary effort to ensure maximum alignment to 
the DoD Business Enterprise and various secondary architectures. 
Further, while the strict economic analysis shows a slight negative 
return, improved planning and equipment tracking, which are command and 
control benefits, bolster the value of TC-AIMS II to the Army and Navy. 
Finally, while the TC-AIMS II was originally envisioned as the single 
solution, the Joint Forces Command (JFCOM) and the Joint Staff J4 is 
now reviewing this concept. Given the significant investment by the Air 
Force and the Marine Corps in their legacy deployment systems and the 
unique deployment processes that exist, JFCOM is investigating if the 
various deployment systems can be joined electronically through a data 
centric approach versus single system centric. The results of JFCOM's 
and J4 's investigation will be reviewed by the OIPT as a part of the 
next milestone review. 

Rather than provide detailed comments on the report's findings, we 
chose to focus our response on the report's recommendations. Our reply 
to each recommendation is enclosed. Our point of contact is Dean 
Carsello at 703-602-2720, ext 116. 

Sincerely, 

Signed by: 

John R. Landon: 
Deputy to the ASD(NII) (C3ISR & IT Acquisition): 

Enclosure: As stated: 

GAO DRAFT REPORT DATED NOVEMBER 15, 2005: 
GAO-06-171 (GAO CODES 310295): 

"DOD SYSTEMS MODERNIZATION: UNCERTAIN JOINT USE AND MARGINAL EXPECTED 
VALUE OF MILITARY ASSET DEPLOYMENT SYSTEM WARRANT REASSESSMENT OF 
PLANNED INVESTMENT" 

DEPARTMENT OF DEFENSE COMMENTS TO THE GAO RECOMMENDATION: 

RECOMMENDATION 1: The GAO recommended that the Secretary of Defense 
direct the Secretary of the Army to collaborate with the Office of the 
Assistant Secretary of Defense for Networks and Information 
Integration/Chief Information Officer, the Office of Program Analysis 
and Evaluation (PA&E), and the Army Cost Analysis Division to prepare a 
reliable economic analysis. (p. 28/GAO Draft Report): 

DoD RESPONSE: Partially Concur. 

The GAO recommendation is based on findings that: 

"investment in TC-AIMS II has not been adequately justified on the 
basis of costs and benefits " (Page 11) and "The Army has not analyzed 
TC AIMS II cost and benefits, and made associated investment decisions, 
on an incremental basis." (Page 13). 

The TC-AIMS II Program has followed all requirements for the 
development of an Economic Analysis (EA) defined by DoD Instruction 
(DoDI) 5000.2 and OMB Circular A-94. As prescribed by DoDI 5000.2, Army-
level and Office of Secretary of Defense (OSD)-level reviews have been 
conducted at least twice for each incrementally developed block of 
capability. These reviews included an EA for the specific Block under 
review as well as a review of the overall program. At each milestone 
review, the TC-AIMS II program office is required to show costs and 
benefits at the system and incremental Block level. EA reviews by the 
Office of the Deputy Assistant Secretary of the Army (Cost and 
Economics) (ODASA-CE) and PA&E ensure that the EA is prepared in 
accordance with cost estimating best practices and relevant Office of 
Management and Budget (OMB) cost benefit guidance. 

After review of the Army's initial EA for TC-AIMS II, PA&E requested 
additional support for the intangible benefits that the Army's EA 
assumed would accrue. This documented rationale was provided to GAO in 
response to the out brief results on 31 October, 2005. 

The rationale provided to PA&E demonstrated the potential cost savings 
resulting from better inventory tracking and control generated from 
developing TC-AIMS II Block 3. This rationale resulted in PA&E 
endorsing MS B approval for Block 3. Additionally, TC-AIMS II provides 
benefits directly to a war fighting mission. For example, TC-AIMS II 
will provide improved tracking that will help locate missing critical 
items such as ammunition. Strict economics cannot necessarily offset 
this intangible opportunity cost benefit. 

Notwithstanding, we will direct the Army to (1) conduct, in 
coordination with PA&E, an analysis to determine the relationship of TC-
AIMS II, the Army's Global Combat Support System and other programs 
that may provide similar functionality to ensure optimal return on 
investment and (2) brief the Networks and Information Integration (NII) 
Overarching Integrated Product Team in preparation for the Block 4 MS B 
milestone decision review. The results will be used to justify further 
development. 

RECOMMENDATION 2: The GAO recommended that the Secretary of Defense 
direct the Secretary of the Army to ensure that development of this 
economic analysis: (1) complies with cost estimating best practices and 
relevant OMB cost benefit guidance; (2) incorporates available data on 
whether deployed Transportation Coordinators Automated Information for 
Management System II capabilities are actually producing benefits; and 
(3) addresses that the Air Force and Marines are not planning to use 
the system. (p. 28/GAO Draft Report): 

DoD RESPONSE: Partially Concur. 

The GAO recommendation is based on a finding that: 

"The Army has not economically justified its investment in TC-AIMS II 
on the basis of reliable estimates of costs and benefits. The most 
recent economic justification included cost and benefit estimates 
predicated on all four military services using the system. However, two 
services (Air Force and Marines) have stated that they do not intend to 
use it. Even with costs and benefits for all four services included, 
the analysis showed a marginal return on investment; that is, for each 
dollar spent on the system, slightly less than one dollar of benefit 
would be returned." (Page 2): 

TC-AIMS II was established as a joint program involving the 
participation of all Services (Army, Navy, Air Force, and Marine Corps) 
by the Joint Requirements Oversight Counsel (JROC) on March 25, 1999. 
However, changes to operations throughout the world have necessitated 
changes to each of the Services' deployment processes. JFCOM initiated 
a study, at the direction of the Milestone Decision Authority, to 
determine whether or not a single solution is still viable. The Air 
Force and Marine Corps have invested into legacy programs that match 
their unique processes. To accommodate unique processes, improvements 
in technology have enabled focusing on a data centric versus a single 
system centric approach. The Deputy Director for Strategic Logistics, 
Joint Chief of Staff-J4, has initiated an effort to provide 
interoperability among several force movement systems, including TC-
AIMS II. This effort will compensate for the mission-unique 
capabilities that precluded the Air Force and Marine Corps from 
adopting TC-AIMS II if the Department determines TC-AIMS II will not be 
the joint system solution for force movement. 

As such, we will direct that the Army, in coordination with PA&E, 
develop an EA that includes Army and Navy use only as an option. This 
EA information will be used in combination with the analysis that 
determines whether the Department implements a data centric approach 
and if further investment into TC-AIMS II is justified. It should be 
noted that the GAO comment regarding the inclusion of Air Force and 
Marine Corps factors in the economic analysis assumes a positive impact 
on TC-AIMS II benefit-to-cost and return on investment ratios. This is 
not necessarily the case. The most recent EA for TC-AIMS II appears to 
show that the ROI may actually improve if the Department implements an 
Army and Navy only program The EA continues to be refined in 
preparation for the next milestone scheduled next summer. 

RECOMMENDATION 3: The GAO recommended that the Secretary of Defense 
direct the Secretary of the Army to collaborate with the Under 
Secretary of Defense (Acquisition, Technology, and Logistics) and the 
Under Secretary of Defense (Comptroller) to ensure that TC-AIMS II is 
adequately aligned with the evolving DoD Business Enterprise 
Architecture. (p. 28/GAO Draft Report): 

DoD RESPONSE: Partially concur. 

The GAO recommendation is based on a finding that: 

"TC-AIMS II has not been defined and developed in the context of a DoD 
enterprise architecture. Instead, the Army has pursued the system based 
on an Army logistics-focused architecture. This means that TC-AIMS II 
as a DoD-wide program is based on a service-specific architecture and 
not a DoD-wide architecture, thus increasing the risk that TC-AIMS II, 
as defined, will not properly fit within the context of future DoD 
enterprise-wide business operations and IT environments.. DoD has long 
operated without a well defined enterprise architecture for its 
business environment." (Pages 14-15): 

The Department has worked over the past several years to develop a 
comprehensive enterprise architecture and management structure. The 
implementation of domain owners and portfolio management tied to 
investment as outlined in DoD Directive 8115.01 signed 10 October 2005 
will place the Department on a path to fully defining and effectively 
managing its enterprise architecture. TC-AIMS II was initiated before 
the Department's formulation of a net-centric strategy. The program 
crosses over both the Deployment and Distribution Enterprises. The 
Program Management Office is working to ensure alignment to these 
evolving enterprise architectures. Blocks 1 through 4 align with the 
Joint Forces Command (JFCOM) Joint Deployment System (JDS) Joint 
Deployment Operational Architecture (JDOA) and that alignment is 
demonstrated in the DoD Architecture Framework (DoDAF) operational and 
system view products provided with system documentation. Additionally, 
the Program Management Office continues to work with the Distribution 
Process Owner (DPO), United States Transportation Command (USTRANSCOM), 
to align the program architecture to the Joint Deployment and 
Distribution Architecture, which is currently aligning to the evolving 
DoD Business Enterprise Architecture (BEA). 

RECOMMENDATION 4: The GAO recommended that the Secretary of Defense 
direct the Secretary of the Army to present the result of the analysis 
outlined in Recommendation 1 to the Deputy Secretary of Defense, or his 
designee, and seek a departmental decision on how best to proceed with 
the program. Until this is done, the GAO recommended that the Secretary 
of Defense direct the Secretary of the Army to limit future investment 
in already deployed applications to essential operation and maintenance 
activities and only developmental activities deemed essential to 
national security needs. (p. 28/GAO Draft Report): 

DoD RESPONSE: Concur. 

The Department procedures require each block of new capability to 
undergo a milestone review. Prior to entering Block 4 development, the 
Milestone Decision Authority (MDA) must assess the economic benefit of 
the investment. The Department's Investment Review Board assesses 
compliance with the BEA and certifies program investment strategies 
annually. The Department makes programmatic decisions throughout the 
development life cycle of the program. Based on a number of factors to 
include the economic analysis, the Department makes a determination for 
each Block to begin development, to assess readiness to enter 
operational testing and to determine if the capability should be 
fielded. The Block 3 development is virtually complete. Prior to 
entering its operational testing, the MDA has directed the Army conduct 
a review of the status of the program. Further, prior to final 
fielding, the program the MDA will conduct the Full Rate Production 
Decision Review including compliance with Title 40 requirements. 
Finally, as stated above, prior to initiating Block 4 development, the 
program must demonstrate its capability to satisfy requirements 
including its overall return on investment. 

TC-AIMS II was originally envisioned as the Department's single system 
solution for force movement. However, technology advancement and 
changes to deployment processes in response to the changing global 
security environment have necessitated and facilitated a data centric 
approach to meeting the needs of the war fighter. In addition to the 
JFCOM initiative stated above, The Assistant Deputy Under Secretary of 
Defense for Transportation Policy and the Joint Staff J4, in 
collaboration with DoD Components, and other OSD staff, are 
coordinating efforts for the DPO to perform a mission requirements 
analysis for future investment into TC-AIMS II. The process has just 
begun. This analysis will support advancement of the data centric 
approach if the Department decides to follow that course of action. 
Prior to any additional investment into TC-AIMS 11 development, the MDA 
through the OIPT will reassess the program considering the milestone 
requirements, the Joint Deployment Process Owner study, and the DPO 
analysis: 

RECOMMENDATION 5: The GAO recommended that if based on reliable data, a 
decision is made to further develop TC-AIMS 11, the Secretary of 
Defense direct the Secretary of the Army to ensure that the program 
implements effective program management activities, related to risk 
management and performance based contracting. (p. 29/GAO Draft Report): 

DoD RESPONSE: Partially concur. 

The GAO recommendation is based on a finding that: 

"The Army has not fully implemented the department's risk management 
policy and guidance. While the program office developed a risk 
management plan, it has not effectively implemented it. For example, 
the program office identified 22 active risks to the program but only 
developed mitigation strategies for 16 of these risks, increasing the 
probability that program risks will lead to cost, schedule, and 
performance problems." (Page 3). 

The above finding was based on a snapshot in time where, although risks 
to the program had been identified, comprehensive mitigation and 
contingency strategies had not been developed. 

The program office possesses an active risk management program. In 
fact, subsequent risk management data was provided to the GAO on Oct 28 
and Nov 3, 2005, reflecting contingency and mitigation strategies for 
all 22 entries and along with evidence that risks are continuously 
reviewed and updated. As the GAO review noted, the program office has 
implemented a strong process review for risk management. While a few 
risks were noted and not all personnel are trained in the risk tool, it 
should be understood that all key personnel in decision-making 
positions have been trained. The program office is comprised of 
Government, non-Government advisor, and the prime contractor's 
personnel. Contractor employees receive internal training as well as 
the augmentation that the program office program provides. Both 
personnel in the program and the actual risks themselves are dynamic in 
nature requiring constant emphasis in this area. Despite the fact that 
the GAO noted that the lack of a mature DoD architecture was not a 
documented risk, the program office has implemented a strong mitigation 
strategy as part of its day-to-day operations. 

The GAO commented that the contract does not provide for disincentives 
or penalties for failing to meet performance criteria (Page 26). In 
question is a Performance-based Cost-Plus-Award-Fee type contract. 
Award fees are determined for each option year with semi-annual 
evaluations. Part or all of an award fee not awarded in the first 
evaluation period may be carried to the second evaluation period; 
however, unearned award fees may not be carried from one option year to 
the next. The two-period assessment within a contract year provides 
interim feedback to the contractor. While the contract lacks direct 
disincentives, typically, failing to achieve substantial award fees is 
harshly received within a company. This personal disincentive for the 
direct employees is a powerful motivating factor. Further, egregious 
failures can result in a negotiated reduction in the base fee. The 
complexity of the effort and the inherent risks of development as noted 
by the GAO review drove the cost type contract. Using a strong award 
fee program encourages proper performance. By using a two-review within 
a contract period process, we believe that correct incentives are in 
place to encourage maximum results. We will address the contract award 
fee implementation in future reviews to ensure contractor performance 
is optimal. 

[End of section] 

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Randolph C. Hite, (202) 512-3439 or [Hyperlink, hiter@gao.gov]: 

Staff Acknowledgments: 

In addition to the contact named above, Gary Mountjoy, Assistant 
Director; Justin Booth; Hal Brumm Jr; Joanne Fiorino; Anh Le; Kush K. 
Malhotra; Teresa M. Neven; Dr. Rona Stillman; Amos Tevelow; and Bill 
Wadsworth made key contributions to this report. 

(310295): 

FOOTNOTES 

[1] DOD, Department of Defense Directive Number 5000.1, The Defense 
Acquisition System (May 12, 2003); Department of Defense Instruction 
Number 5000.2, Operation of the Defense Acquisition System (May 12, 
2003); and Defense Acquisition Guidebook, Version 1.0 (Oct. 17, 2004). 

[2] GAO, DOD Business Systems Modernization: Long-standing Weaknesses 
in Enterprise Architecture Development Need to Be Addressed, GAO-05-702 
(Washington, D.C. July 22, 2005). 

[3] GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C. 
January 2005). 

[4] GAO, Information Technology: DOD's Acquisition Policies and 
Guidance Need to Incorporate Additional Best Practices and Controls, 
GAO-04-722 (Washington, D.C. July 30, 2004). 

[5] For example, see DOD, Department of Defense Directive 5000.1; 
Department of Defense Instruction 5000.2; Department of Defense 
Instruction 7041.3, Economic Analysis for Decision Making (Nov. 7, 
1995); and Office of Management and Budget, Circular No. A-94, 
Guidelines and Discount Rates for Benefit-Cost Analysis of Federal 
Programs (Oct. 29, 1992). 

[6] See, for example, the Clinger-Cohen Act of 1996, 41 U.S.C. §§ 434, 
and Office of Management and Budget, Circular No. A-130, Management of 
Federal Information Resources (Nov. 30, 2000). 

[7] DOD, Department of Defense Directive Number 5000.1 and Department 
of Defense Architecture Framework, Version 1.0, Volume 1 (February 
2004). 

[8] Clinger-Cohen Act of 1996, 40 U.S.C. §§ 11312 and 11315(b)(2); E-
Government Act of 2002, Public Law 107-347 (Dec. 17, 2002); GAO, 
Information Technology: A Framework for Assessing and Improving 
Enterprise Architecture Management (Version 1.1), GAO-03-584G 
(Washington, D.C. April 2003); Chief Information Officer Council, A 
Practical Guide to Federal Enterprise Architecture, Version 1.0 
(February 2001); and Institute of Electrical and Electronics Engineers, 
Standard for Recommended Practice for Architectural Description of 
Software-Intensive Systems 1471-2000 (Sept. 21, 2000). 

[9] See, for example, GAO, Information Technology: FBI Is Taking Steps 
to Develop an Enterprise Architecture, but much Remains to Be 
Accomplished, GAO-05-363 (Washington, D.C. Sept. 9, 2005); Homeland 
Security: Efforts Under Way to Develop Enterprise Architecture, but 
Much Work Remains, GAO-04-777 (Washington, D.C. Aug. 6, 2004); DOD 
Business Systems Modernization: Limited Progress in Development of 
Business Enterprise Architecture and Oversight of Information 
Technology Investments, GAO-04-731R (Washington, D.C. May 17, 2004); 
Information Technology: Architecture Needed to Guide NASA's Financial 
Management Modernization, GAO-04-43 (Washington, D.C. Nov. 21, 2003); 
DOD Business Systems Modernization: Important Progress Made to Develop 
Business Enterprise Architecture, but Much Work Remains, GAO-03-1018 
(Washington, D.C. Sept. 19, 2003); Business Systems Modernization: 
Summary of GAO's Assessment of the Department of Defense's Initial 
Business Enterprise Architecture, GAO-03-877R (Washington, D.C. July 7, 
2003); Information Technology: DLA Should Strengthen Business Systems 
Modernization Architecture and Investment Activities, GAO-01-631 
(Washington, D.C. June 29, 2001); and Information Technology: INS Needs 
to Better Manage the Development of Its Enterprise Architecture, 
GAO/AIMD-00-212 (Washington, D.C. Aug. 1, 2000). 

[10] GAO, Information Technology: Architecture Needed to Guide 
Modernization of DOD's Financial Operations, GAO-01-525 (Washington, 
D.C. May 17, 2001). 

[11] GAO, DOD Business Systems Modernization: Improvements to 
Enterprise Architecture Development and Implementation Efforts Needed, 
GAO-03-458 (Washington, D.C. Feb. 28, 2003); Information Technology: 
Observations on Department of Defense's Draft Enterprise Architecture, 
GAO-03-571R (Washington, D.C. Mar. 28, 2003); GAO-03-877R; GAO-03-1018; 
and GAO-04-731R. 

[12] GAO-05-702. 

[13] GAO, DOD Business Systems Modernization: Billions Being Invested 
Without Adequate Oversight, GAO-05-381 (Washington, D.C. Apr. 29, 
2005). 

[14] The Under Secretary of Defense for Acquisition, Technology, and 
Logistics and the Under Secretary of Defense (Comptroller) are 
responsible for overseeing the development of DOD's business enterprise 
architecture. 

[15] Ronald W. Reagan National Defense Authorization Act for Fiscal 
Year 2005, Public Law 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct. 
28, 2004) (codified in part at 10 U.S.C. § 2222). 

[16] DOD, Department of Defense Instruction Number 5000.2. 

[17] Software Engineering Institute, Software Acquisition Capability 
Maturity Model® version 1.03, CMU/SEI-2002-TR-010 (Pittsburgh, PA: 
March 2002). 

[18] DOD, Department of Defense Instruction Number 5000.2. 

[19] The Risk Management Board consists of key program officials, 
including the deputy program manager, system development manager, 
operations manager, user support and deployment manager, performance 
assurance manager, and business manager as well as the contractor's 
program manager. 

[20] According to the risk plan, risk exposure is a function of a 
risk's projected impact and its probability of occurrence. It can be 
expressed as a relative function--the higher the level of exposure, the 
greater the risk to the program. Risk exposure is calculated by 
multiplying the impact by the probability of occurrence. 

[21] See, for example, DOD, Department of Defense Instruction Number 
5000.2, and Software Engineering Institute, CMU/SEI-2002-TR-010. 

[22] DOD, Department of Defense Instruction Number 5000.2, and Defense 
Acquisition Guidebook, Version 1.0. 

[23] Dynamic Object Oriented Requirements System. 

[24] GAO-04-722. 

[25] DOD, Department of Defense Instruction Number 5000.2. 

[26] See, for example, DOD, Department of Defense Instruction Number 
5000.2, and Software Engineering Institute, CMU/SEI-2002-TR-010. 

[27] See, for example, Federal Acquisition Regulation, section 
37.102(a); DOD, Department of Defense Directive Number 5000.1; and 
Software Engineering Institute, CMU/SEI-2002-TR-010. 

[28] These policies and guidance included DOD, Department of Defense 
Directive Number 5000.1, The Defense Acquisition System (May 12, 2003); 
Department of Defense Instruction Number 5000.2, Operation of the 
Defense Acquisition System (May 12, 2003); and Defense Acquisition 
Guidebook, Version 1.0 (Oct. 17, 2004). 

[29] See, for example, Office of Management and Budget, Circular No. A-
94: Guidelines and Discount Rates for Benefit-Cost Analysis of Federal 
Programs (Oct. 29, 1992); and GAO, Information Technology: DOD's 
Acquisition Policies and Guidance Need to Incorporate Additional Best 
Practices and Controls, GAO-04-722 (Washington, D.C. July 30, 2004). 

[30] GAO, DOD Business Systems Modernization, Long-standing Weaknesses 
in Enterprise Architecture Development Need to Be Addressed, GAO-05-702 
(Washington, D.C. July 22, 2005); Information Technology: Enterprise 
Architecture Use across the Federal Government Can Be Improved, GAO-02-
6 (Washington, D.C. Feb. 19, 2002); and Information Technology: 
Leadership Remains Key to Agencies Making Progress on Enterprise 
Architecture Efforts, GAO-04-40 (Washington, D.C. Nov. 17, 2003). 

[31] Software Engineering Institute, Software Acquisition Capability 
Maturity Model® version 1.03, CMU/SEI-2002-TR-010 (Pittsburgh, PA: 
March 2002). 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, 

NelliganJ@gao.gov 

(202) 512-4800 

U.S. Government Accountability Office, 

441 G Street NW, Room 7149 

Washington, D.C. 20548: