This is the accessible text file for GAO report number GAO-07-860 
entitled 'DOD Business Transformation: Lack of an Integrated Strategy 
Puts the Army's Asset Visibility System Investments at Risk' which was 
released on July 27, 2007. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Subcommittee on Readiness and Management Support, 
Committee on Armed Services, U.S. Senate: 

United States Government Accountability Office: 

GAO: 

July 2007: 

DOD Business Transformation: 

Lack of an Integrated Strategy Puts the Army's Asset Visibility System 
Investments at Risk: 

GAO-07-860: 

GAO Highlights: 

Highlights of GAO-07-860, a report to
the Subcommittee on Readiness and Management Support, Committee on 
Armed Services, U.S. Senate 

Why GAO Did This Study: 

The Department of Defense (DOD) established a goal to achieve total 
asset visibility (TAV) over 30 years ago, but to date it has been 
unsuccessful. GAO was requested to (1) determine whether the Army has a 
systems strategy for achieving TAV, (2) determine if the Army’s 
business system investment governance structure is consistent with DOD 
guidance, and (3) evaluate the Army’s effort to correct previously 
reported problems with the Logistics Modernization Program (LMP). GAO 
obtained an understanding of the Army’s efforts to achieve TAV, oversee 
and manage its business system investments, and address previously 
reported LMP problems. 

What GAO Found: 

Supply chain management has been on GAO’s high-risk list since 1990. 
One area that has contributed to this long-standing problem has been 
DOD’s inability to maintain control and accountability over hundreds of 
billions of dollars of assets. DOD plans to improve its asset 
management through its business system modernization. In this regard, 
GFEBS, the Global Combat Support System-Army (GCSS-Army), and LMP are 
aimed at achieving TAV within the Army. The Army estimates that it will 
invest approximately $5 billion to develop and implement these systems. 
However, this investment is being made without a clear integrated 
strategy. 

* GFEBS, GCSS-Army, and LMP are not being developed in the context of a 
well-defined Army-wide enterprise architecture. As a result, the Army 
does not have an informed basis for determining if these systems will 
fit within the context of future Army business operations and will 
efficiently and effectively address the Army’s long-standing weaknesses 
associated with the lack of asset visibility. 

* The Army lacks a concept of operations that would describe, at a high 
level, (1) how the three business systems relate to each other in 
achieving the Army’s TAV goal, and (2) how information flows from and 
through these systems. Moreover, GAO found that the Army's lack of a 
concept of operations has contributed to its failure to take full 
advantage of business process reengineering opportunities that are 
available when using an enterprise resource planning solution. 

Without these key foundational elements, the Army is at risk of 
investing about $5 billion in business systems and still not achieving 
DOD’s and the Army’s goal of TAV. 

Furthermore, while the Army has established a governance structure that 
is consistent with DOD guidance, its processes are still maturing. The 
Army’s governance structure is designed to certify and review 
individual business systems rather than to evaluate these investments 
from a portfolio perspective. Such a perspective permits investments to 
be viewed in a comprehensive manner to help ensure that the 
organization’s missions and objectives are achieved. GAO also found 
that the Army did not have reliable processes and analyses, such as an 
independent validation and verification function or economic analyses, 
to support its oversight of individual business systems. Until the 
Army's investment processes mature, it runs the risk of investing in 
business systems that do not provide the desired functionality and 
efficiency. 

Additionally, LMP continues to be plagued by problems that have beset 
the system since its implementation in July 2003. LMP continues to 
experience problems with accurately recognizing revenue and billing 
customers, which can, in part, be attributed to ineffective system 
testing. 

What GAO Recommends: 

GAO makes five recommendations to DOD and the Army: (1) develop a 
concept of operations for the Army; (2) develop policies, procedures, 
and processes to manage investments from a portfolio perspective; (3) 
establish an independent verification and validation function; (4) 
require that any future General Fund Enterprise Business System (GFEBS) 
economic analysis is prepared in accordance with applicable policies; 
and (5) direct that LMP use an independent system test team. Overall, 
DOD concurred with the recommendations and stated that it will work 
diligently to close them. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-860]. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact McCoy Williams at (202) 
512-9095 or Keith Rhodes at (202) 512-6412. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Army Lacks an Integrated Strategy for Achieving TAV: 

Army's Ability to Effectively Oversee Portfolios of Business Systems 
Investment Is Not Yet Fully Developed: 

LMP Illustrates Continuing Problems in Implementing Business Systems on 
Time, within Budget, and with the Promised Capability: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Defense: 

Appendix III: Army Strategy Does Not Fully Utilize Capabilities of ERP 
Solution: 

Appendix IV: LMP Problems Continue: 

Appendix V: GAO Contacts and Staff Acknowledgments: 

Tables: 

Table 1: Unbilled and Credit Customer Orders at the Tobyhanna Army 
Depot for November 2006 through January 2007: 

Table 2: DFAS Billings to the Tobyhanna Army Depot for Accounting 
Services: 

Figures: 

Figure 1: LMP Time Line: 

Figure 2: GCSS-Army Time Line: 

Figure 3: GFEBS Time Line: 

Figure 4: Overview of Tiered Accountability for Army Business System 
Investments: 

Figure 5: Example of an ERP Vision for Accountability of the Army's 
PP&E: 

Figure 6: Planned GFEBS and GCSS-Army Fund Control Process: 

Figure 7: GCSS-Army Funds Control Process Utilizing ERP Capabilities: 

Abbreviations: 

BEA: business enterprise architecture: 

BTA: Business Transformation Agency: 

CIO: Chief Information Officer: 

COTS: commercial off-the-shelf: 

CSC: Computer Sciences Corporation: 

DBSMC: Defense Business Systems Management Committee: 

DFAS: Defense Finance and Accounting Service: 

DOD: Department of Defense: 

DPAS: Defense Property Accountability System: 

EA: enterprise architecture: 

EAMMF: Enterprise Architecture Management Maturity Framework: 

ERP: enterprise resource planning: 

FFMIA: Federal Financial Management Improvement Act: 

FOC: full operational capability: 

GCSS-Army: Global Combat Support System-Army: 

GFEBS: General Fund Enterprise Business System: 

IRB: investment review board: 

IT: information technology: 

IV&V: independent verification and validation: 

LMP: Logistics Modernization Program: 

MAIS: major automated information system: 

MDAP: major defense acquisition program: 

OMB: Office of Management and Budget: 

PBUSE: Property Book Unit Supply Enhanced: 

PCA: Pre-Certification Authority: 

PLM+: Product Lifecycle Management Plus: 

PP&E: property, plant, and equipment: 

SALE: Single Army Logistics Enterprise: 

TAV: total asset visibility: 

V&V: verification and validation: 

United States Government Accountability Office: 
Washington, DC 20548: 

July 27, 2007: 

The Honorable Daniel K. Akaka: 
Chairman: 
The Honorable John Ensign: 
Ranking Member: 
Subcommittee on Readiness and Management Support: 
Committee on Armed Services: 
United States Senate: 

The Department of Defense (DOD) has continually struggled to achieve 
and maintain efficient and effective management over the hundreds of 
billions of dollars it has invested in tangible assets, including 
inventory, supplies, and materials (inventory and related property) and 
property, plant, and equipment (PP&E).[Footnote 1] DOD was responsible 
for almost 72 percent ($697 billion) of the total $970 billion reported 
governmentwide value for these assets, as of September 30, 
2006.[Footnote 2] The nature and severity of DOD's financial and 
business management system deficiencies impede the ability of DOD 
managers to receive the full range of information needed to effectively 
manage day-to-day operations. Of the 27 areas on GAO's high-risk 
list,[Footnote 3] DOD has 8 high-risk areas of its own[Footnote 4] and 
shares responsibility for 7 governmentwide high-risk areas.[Footnote 5] 

Visibility over its assets has been a DOD concern for decades. If the 
information contained in the asset accountability systems is not 
accurate, complete, and timely, DOD's day-to-day operations could be 
adversely affected by investing in inventory, for example, that is not 
needed to meet current needs. When this occurs, the department may 
obligate funds unnecessarily, which could lead to not having sufficient 
obligational authority to purchase needed items. In recognition of the 
importance of asset accountability to successful operations, the 
department established a goal to achieve total asset visibility (TAV) 
over 30 years ago. DOD defines TAV as the ability to provide timely and 
accurate information on the location, movement, status, and identity of 
units, personnel, equipment, and supplies and having the ability to act 
on that information. Over the years, the military services and defense 
components have undertaken numerous initiatives to achieve TAV. Within 
the Army, one such initiative has been the Logistics Modernization 
Program (LMP). In May 2004 and June 2005, we reported that LMP was not 
providing the Army the promised capability.[Footnote 6] DOD's current 
estimate for achieving TAV is 2010. 

This report provides information in support of your continuing 
oversight of DOD's progress towards resolving the department's long- 
standing problems in achieving TAV. As you requested, our initial 
effort was directed at the Army. In September 2006, the Army reported 
inventory and related property of about $57 billion and PP&E over $85 
billion.[Footnote 7] The Army has identified three primary system 
initiatives directed at achieving TAV within the service: (1) LMP, (2) 
Global Combat Support System-Army Field/Tactical[Footnote 8] (GCSS- 
Army), and (3) General Fund Enterprise Business System (GFEBS). Our 
objectives were to (1) determine whether the Army has developed a 
business system strategy for achieving TAV, (2) determine if the Army 
has effectively implemented a governance structure to oversee and 
manage its business system investments in accordance with DOD guidance, 
and (3) evaluate the extent to which the Army has made progress in 
correcting the previously reported problems regarding LMP's 
implementation. 

To address the first objective, we met with Army program office 
officials for GFEBS, GCSS-Army, and LMP and obtained briefings on the 
intended purpose of each system. In addition, we conducted walkthroughs 
and reviewed documentation related to various transactions to obtain an 
understanding of how the systems would exchange data and to assess how 
the Army intended to use these systems individually and collectively to 
achieve TAV. To address the second objective, we reviewed guidance 
issued by DOD, the Army, and the Business Transformation Agency (BTA) 
related to investment management. We also obtained an understanding of 
the Army's business system investment governance structure and process 
for ensuring compliance with the certification and annual system review 
processes required by the fiscal year 2005 National Defense 
Authorization Act.[Footnote 9] To address the third objective, we 
interviewed and obtained briefings from LMP program management office 
officials and others, and reviewed and analyzed LMP system requirement 
and testing documentation to assess the extent to which corrective 
actions had been taken or are planned to address our prior 
recommendations. We determined that the documentation Army prepared and 
submitted through its business system governance process to the 
investment review boards (IRBs) and the Defense Business Systems 
Management Committee (DBSMC) as a basis for approving individual Army 
business system investments was sufficiently reliable for our purposes. 
Our work was performed from May 2006 through June 2007 in accordance 
with U. S. generally accepted government auditing standards. Details on 
our scope and methodology are included in appendix I. We requested 
comments on a draft of this report from the Secretary of Defense or his 
designee. We received written comments from the Deputy Under Secretary 
of Defense (Business Transformation), which are reprinted in appendix 
II. 

Results in Brief: 

The Army's current approach for developing GFEBS, GCSS-Army, and LMP 
lacks several elements that are critical to the successful 
implementation of integrated business systems, such as an Army-level 
enterprise architecture (EA),[Footnote 10] a concept of operations, and 
a portfolio-based rather than individual-project-based business system 
investment review process. Without these key foundational elements, the 
risk that the Army's efforts to achieve TAV will not be successful is 
greatly increased. While the Army's efforts to develop these systems 
and transform its logistics operations may result in incremental 
improvements, without these three essential elements, they are unlikely 
to achieve the efficiencies that can be attained through an integrated 
business system solution. Instead, if the Army continues on its current 
path, it runs the risk of investing significant resources to simply 
automate its existing inefficient business processes using more current 
technology. 

As it now stands, the Army plans to invest about $5 billion over the 
next several years to develop and implement GFEBS, GCSS-Army, and LMP 
without the benefit of a well-defined Army EA. We reported in August 
2006,[Footnote 11] that the Army was in the initial stages of 
developing an EA. As of May 2007, this was still the case.[Footnote 12] 
A well-defined EA is an essential tool for leveraging information 
technology (IT) in the transformation of business and mission 
operations. Our experience with federal departments and agencies has 
shown that attempting to modernize systems without an EA to guide and 
constrain investments often results in operations and systems that are 
duplicative, not well integrated, unnecessarily costly to maintain and 
interface, and ineffective in supporting mission goals.[Footnote 13] 
Moreover, the development, implementation, and maintenance of an EA are 
widely recognized as hallmarks of successful public and private 
organizations, and their use is required by the Clinger-Cohen Act of 
1996[Footnote 14] and the related guidance from the Office of 
Management and Budget (OMB).[Footnote 15] 

In addition to the Army's lack of an EA, the Army also lacked a concept 
of operations, which outlines the Army's strategy for achieving TAV, 
including how the three systems it has identified as key in attaining 
TAV will interoperate. A concept of operations would provide the Army a 
forum for interchange among stakeholders--such as oversight entities, 
program managers, developers, and users--on major technical and 
programmatic issues related to achieving TAV. Without a concept of 
operations for achieving TAV, the Army is hindered in its ability to 
apply an enterprise view in (1) making decisions as to how GFEBS, GCSS- 
Army, and LMP will individually and collectively enhance the Army's 
asset accountability, including providing TAV; and (2) determining what 
changes are needed in its related business processes. Additionally, the 
Army's inability to achieve TAV hinders its and DOD's efforts to 
resolve the long-standing problems associated with supply chain 
management, which has been on our high-risk list since 1990. Asset 
visibility is one of the focus issues critical to successfully 
addressing this high-risk area. Further, the Army's lack of a concept 
of operations has resulted in its failure to take full advantage of 
business process reengineering opportunities that are available when 
using an enterprise resource planning (ERP)[Footnote 16] solution. 
Rather, the Army's existing strategy perpetuates some of the cumbersome 
and ineffective business processes that are currently used in its 
existing legacy system environment. The benefits of an ERP solution 
include streamlining of business processes and elimination of data 
redundancy. 

Furthermore, while the Army has established a business system 
investment management governance structure that is consistent with DOD 
guidance, its overall investment management approach is still maturing. 
Currently, the Army's investment review process is designed to ensure 
the completion of certifications and annual reviews of individual 
business systems rather than to evaluate business system investments 
from a portfolio management perspective. A portfolio-based perspective 
permits an organization to view its business system investments in a 
comprehensive manner to help ensure that the organization's missions, 
strategic goals, and objectives are achieved. Moreover, we found that 
the Army did not have reliable processes or analyses, such as 
independent verification and validation functions or economic analyses, 
to support its oversight of program management office efforts to 
develop and implement business systems. Until the Army adopts a 
business system investment management approach that provides for 
reviewing groups of systems and making enterprise decisions regarding 
how these groups will collectively interoperate to provide a desired 
capability, it runs the risk of investing significant resources in 
business systems that do not provide the desired functionality and 
efficiency. 

LMP continues to be plagued by operational problems that have beset the 
system virtually since its initial implementation in July 2003. While 
from a "big picture" perspective, an EA ,concept of operations, and 
effective IT portfolio management are essential elements in an entity's 
efforts to transform its operations, it is equally important for the 
entity to have the disciplined processes needed to actually implement 
individual business systems on time, within budget, and with the 
promised capability. As we have previously reported, historically DOD 
has had difficulty in accomplishing this goal, and LMP has been no 
exception. As of September 2006, the Army reported that it had 
obligated approximately $452 million to develop and implement LMP. In 
May 2004 and June 2005, we reported on operational issues related to 
LMP, for example, the inability to accurately recognize revenue and 
bill customers--a problem that continues today. We recommended, and the 
Army agreed, that the implementation of LMP should be delayed until the 
operational problems we identified were resolved. While the Army is 
working to resolve LMP operational issues, we continue to have concerns 
about the adequacy of LMP's system testing given that the continuing 
problems with LMP can, in part, be attributed to ineffective and 
nonindependent system testing. Until an effective LMP testing process 
is implemented, the Army will have little assurance that the corrective 
actions it takes (1) are properly developed, and (2) do not introduce 
additional defects into the system. 

We are making five recommendations to the Secretary of Defense to 
improve the department's efforts to achieve TAV and further enhance its 
efforts to improve its control and accountability over business system 
investments. More specifically, we recommend that the Secretary of 
Defense (1) develop a concept of operations for the Army; (2) develop 
policies, procedures, and processes to manage investments from a 
portfolio perspective; (3) establish an independent verification and 
validation function; (4) ensure the GFEBS economic analysis update is 
prepared in accordance with applicable guidance; and (5) direct that 
LMP use an independent system test team. 

We received written comments on a draft of this report from the Deputy 
Under Secretary of Defense (Business Transformation), which are 
reprinted in appendix II. Overall, DOD concurred with our 
recommendations and stated that it would work diligently to implement 
them. The comments included two sets of specific responses to our 
recommendations--one set provided by BTA and another provided by the 
Army. In its comments, the Army concurred with each of the 
recommendations. BTA stated that it fully agreed with our observations. 
BTA, though, partially concurred with all the recommendations on the 
basis that they were directed jointly to the Secretary of the Army and 
the Director, BTA. BTA's comments noted that it has neither the 
authority nor the responsibility to direct the actions of the Army. 

We appreciate the department's willingness to address our 
recommendations. With regard to BTA's concern, our recommendations do 
not direct BTA to oversee or direct the Army. Rather, the 
recommendations stated that the specific actions should be undertaken 
jointly at the direction of the Secretary of Defense. We continue to 
believe that a cooperative and effectively coordinated BTA and Army 
approach to addressing our recommendations is most likely to achieve 
the fundamental business system transformation necessary to achieve the 
department's TAV objective. 

Background: 

TAV has been elusive within DOD. Timely, reliable information on the 
location, quantity, and status of the department's tangible assets 
could significantly improve its ability to more efficiently and 
effectively deliver needed items to DOD operating forces and thereby 
enhance military readiness. The department has recognized the 
importance of achieving TAV and included it as part of its overall 
business transformation initiative, which includes the development and 
implementation of a business enterprise architecture (BEA). The Joint 
Chiefs of Staff also identified TAV as one of four fundamental changes 
needed to transform the department's logistics operations.[Footnote 17] 

One of the primary factors contributing to DOD's inability to provide 
management with TAV is DOD's outdated and ineffective management 
information system environment.[Footnote 18] The Federal Financial 
Management Improvement Act (FFMIA) of 1996[Footnote 19] and other 
financial management reform legislation have emphasized the importance 
of improving financial management, which necessarily encompasses proper 
inventory management, across the federal government. Built upon the 
foundation laid by the Chief Financial Officers Act of 1990,[Footnote 
20] FFMIA emphasizes the need for agencies to have integrated financial 
management systems that can generate timely, accurate, and useful 
information to make informed decisions and to ensure accountability on 
a continuous basis.[Footnote 21] Lacking such critical information, 
government leaders will not be positioned to invest resources, reduce 
costs, oversee programs, and hold agency managers accountable for the 
manner in which government programs are operated. 

Army Initiatives Aimed at Achieving TAV: 

To improve control and accountability over its assets, the Army has 
embarked on a multisystem integration effort that is intended to 
leverage commercial ERP software and processes. This integration is the 
focus of the Single Army Logistics Enterprise (SALE) initiative. SALE 
is designed to integrate information technology requirements, business 
processes, business rules, and data from the Army's logistics, 
financial, and acquisition transactions for planning and supporting 
warfighting logistics operations. According to the Army, SALE is to 
provide a coordinated ERP solution built around two individual 
logistics system development efforts: (1) LMP and (2) GCSS-Army. Under 
the SALE vision, GCSS-Army and LMP will be integrated into a single 
solution to provide an Army-wide logistics environment spanning from 
"the factory to foxhole." On the financial side, the Army is developing 
GFEBS to provide financial visibility over its assets. Each of these 
efforts is described below. 

LMP. In February 1998, the U.S. Army Materiel Command began an ERP 
effort--LMP--to replace its legacy materiel and maintenance management 
systems--the Commodity Command Standard System and the Standard Depot 
System--with LMP. The Army has been using the existing legacy systems 
for over 30 years. LMP is intended to transform the Army's Working 
Capital Fund logistics operations in six core processes: order 
fulfillment, demand and supply planning, procurement, asset management, 
materiel maintenance, and financial management. LMP became operational 
at the U.S. Army Communications and Electronics Command, Fort Monmouth, 
New Jersey, and Tobyhanna Army Depot, Tobyhanna, Pennsylvania, in July 
2003. The initial deployment of LMP consisted of inventory items such 
as electronics; electronic repair components; and communications and 
intelligence equipment such as night vision goggles, electronic 
components such as circuit boards, and certain munitions such as 
guidance systems included in missiles. Figure 1 shows the LMP time line 
as of March 2007. 

Figure 1: LMP Time Line: 

[See PDF for image] 

Source: DOD, LMP program office. 

[End of figure] 

The Communications-Electronics Life Cycle Management Command,[Footnote 
22] Tobyhanna Army Depot, and the Defense Finance and Accounting 
Service (DFAS) are the primary LMP users. When LMP is fully 
implemented, its capacity is expected to include more than 17,000 users 
at 149 locations and it will be populated with 6 million Army-managed 
inventory items valued at about $40 billion. LMP is scheduled to reach 
full operational capability (FOC)[Footnote 23] in fiscal year 2010. As 
of September 30, 2006, the Army reported that approximately $452 
million had been obligated for this system effort and estimates that it 
will invest at least another $895 million in LMP.[Footnote 24] 

GCSS-Army. The GCSS-Army program was initiated in 1997 to overcome 
duplicative databases, poor asset visibility, and stovepiped 
communications between numerous existing Army logistics systems. The 
goal of GCSS-Army is to integrate multiple logistic functions by 
replacing numerous legacy systems and interfaces. Since the program's 
inception, it has undergone several revisions, including a change from 
a custom software development to a commercial ERP solution, as well as 
a change in the prime contractor. According to Army officials, it 
invested approximately $95 million in the previous efforts before 
adopting the ERP approach. The existing ERP effort was started in 
September 2003 and is currently estimated to reach FOC during fiscal 
year 2014. Figure 2 shows the time line for GCSS-Army as of March 2007. 

Figure 2: GCSS-Army Time Line: 

[See PDF for image] 

Source: DOD, GCSS-Army program office. 

[End of figure] 

GCSS-Army is intended to replace 16 stovepiped, legacy logistics 
systems that cover certain types of inventory and PP&E. The system is 
intended to be operational at all deployable Army units, and provide 
asset visibility for accountable items down to the foxhole. As of 
September 30, 2006, the Army reported that it had obligated 
approximately $203 million for the ERP version of GCSS-Army. 
Additionally, the Army estimates that another $2.1 billion will be 
invested in GCSS-Army. 

Product Lifecycle Management Plus (PLM+). This initiative is the 
technical enabler to integrate LMP and GCSS-Army. PLM+ is the means by 
which the Army intends to achieve the SALE vision of integrated 
logistics systems. PLM+ is intended to provide a single point of entry 
and exit for interfaces to external systems. Furthermore, PLM+ is 
intended to eliminate duplicative and costly system interfaces. 
Additionally, the PLM+ implementation schedule and related funding are 
aligned with that of GCSS-Army. PLM+ does not stand alone as an 
independent capability. As of September 2006, the Army reported that 
approximately $31 million had been obligated for PLM+. 

GFEBS. While LMP is intended to provide financial control for the Army 
Working Capital Fund, GFEBS is intended to provide this important 
control over all non-working capital fund inventory, including that 
which is reported in GCSS-Army. GFEBS is an ERP solution that was 
initiated in October 2004 and is intended to serve as the Army's 
general ledger system for its general fund accounting.[Footnote 25] As 
such, GFEBS is intended to improve the reliability of the Army's 
financial information and thereby enhance the Army's management 
decision-making process. GFEBS is expected to replace 87 legacy 
systems, including the 30-year old Standard Army Finance System. Figure 
3 shows the GFEBS time line as of June 2007. 

Figure 3: GFEBS Time Line: 

[See PDF for image] 

Source: DOD, GFEBS program office. 

[End of figure] 

The Army estimates that GFEBS will reach FOC by 2010. As of September 
30, 2006, the Army reported that it had obligated $123 million for the 
development of GFEBS. In addition, the Army currently estimates that it 
will invest another $1.3 billion to implement GFEBS. 

Overview of DOD's Investment Management Practices: 

In 2005, DOD adopted a "tiered accountability" approach to improve 
control and accountability over the billions of dollars it invests 
annually in DOD business systems. Under this approach, executive 
leadership for the direction, oversight, and execution of DOD 
investments is the responsibility of several entities within DOD and 
its components. As shown in figure 4 and described below, the 
investment control process begins at the component level and works its 
way up through a hierarchy of review and approval authorities, 
depending on the size and significance of the investment.[Footnote 26] 

Figure 4: Overview of Tiered Accountability for Army Business System 
Investments: 

[See PDF for image] 

Source: Army, DOD. 

[End of figure] 

At the DOD enterprise level, key entities involved in maintaining 
control and accountability over Army business system investments with 
systems modernizations over $1 million include the DBSMC, which serves 
as the highest ranking governance body for business systems 
modernization activities; the Principal Staff Assistants, who serve as 
the certification authorities for business system modernizations in 
their respective core business missions; the investment review boards 
(IRBs), which form the review and decision-making bodies for business 
system investments in their respective areas of responsibility and 
review each investment for BEA compliance; and BTA, which provides 
support to the DBSMC and the IRBs and is responsible for leading and 
coordinating business transformation efforts across the department. The 
BTA is organized into seven directorates, one of which is the Defense 
Business Systems Acquisition Executive--the component acquisition 
executive for DOD enterprise-level (DOD-wide) business systems and 
initiatives. This directorate is responsible for developing, 
coordinating, and integrating DOD enterprise-level projects, programs, 
systems, and initiatives--including managing resources such as funding, 
personnel, and contracts for assigned systems and programs. 

To implement tiered accountability within the Army, the Army designated 
its Chief Information Officer (CIO) as the Army's Pre-Certification 
Authority (PCA) for certification and annual reviews of business system 
investments.[Footnote 27] The PCA is accountable for the component's 
business system investments and acts as the component's principal point 
of contact for communication with the IRBs. As such, the PCA (1) 
validates that the system information for all business systems 
modernizations over $1 million is complete and accessible to the IRBs, 
(2) reviews development/modernization investments with total cost of $1 
million or less, (3) reviews system compliance with DOD's BEA and 
enterprise transition plan, (4) verifies the investment's economic 
viability analysis, (5) asserts the status and validity of the 
investment information by submitting a component precertification 
letter to the appropriate IRB for its review, and (6) provides IT 
portfolio management policy guidance and oversight of mission area/ 
domain IT portfolios. Below the Army enterprise level, the Army has 
established six functional area domains within its business mission 
area.[Footnote 28] The domains are responsible for implementing the IT 
portfolio management process developed by DOD and the Army to define 
and justify the portfolio of planned IT expenditures consistent with 
strategic objectives and operational requirements. 

GAO Has Previously Reported LMP Issues: 

Our May 2004 report[Footnote 29] pointed out that the Army had not 
effectively managed its implementation of LMP. The report noted that 
after LMP was deployed in July 2003, operational difficulties at the 
Tobyhanna Army Depot resulted in inaccurate financial management 
information. More specifically, the depot was not (1) producing 
accurate workload planning information, (2) generating accurate 
customer bills, and (3) capturing all repair costs, necessary for the 
Army to calculate accurate future repair prices. The report also 
pointed out that LMP requirements (1) lacked the specific information 
necessary to understand the required functionality that was to be 
provided, and (2) did not describe how to determine quantitatively, 
through testing or other analysis, whether the system would meet the 
Army's needs. Subsequently, in June 2005,[Footnote 30] we reported that 
the problems with LMP continued to prevent the Tobyhanna Army Depot 
from accurately reporting on its financial operations, including gains 
and losses, which adversely affected the depot's ability to accurately 
set customer sales prices. Further, the report pointed out that 
problems persisted with recognizing revenue and billing customers. We 
recommended, and the Army agreed, that the implementation of LMP should 
be delayed until the operational problems we identified were resolved. 
The Army is continuing to resolve the outstanding issues. 

Army Lacks an Integrated Strategy for Achieving TAV: 

The Army has estimated that it will invest about $5 billion over the 
next several years to complete development and implementation of GFEBS, 
GCSS-Army, and LMP. However, the Army is making this significant 
investment without a clear integrated strategy for how these systems 
will be used to achieve TAV over hundreds of billions of dollars of 
assets. As we reported in December 2004,[Footnote 31] because DOD has 
not developed a clear long-range strategy, the military services will 
be exposed to the risk of spending billions of dollars on duplicative, 
stovepiped systems that do not support the department's business 
transformation goals, including attaining TAV. More specifically, we 
found that the Army's current strategy for achieving TAV, including the 
implementation of these three systems, does not embrace two key 
foundational elements that we had identified as essential to achieving 
successful transformation of business systems and processes: an EA and 
a concept of operations. Without these key foundational elements, the 
Army is at risk of investing billions of dollars in business systems 
that may not achieve DOD's and the Army's goal of achieving TAV. 
Further, the Army's planned strategy perpetuates some of the cumbersome 
and ineffective business processes and data redundancies that are 
currently being used in the existing legacy system environment. 

Army's Efforts to Achieve TAV Lack the Benefit of an EA: 

The Army has yet to develop and implement an Army business EA to help 
guide its system efforts, including those aimed at achieving TAV. DOD's 
acquisition policies and guidance,[Footnote 32] as well as federal and 
best practice guidance,[Footnote 33] recognize the importance of 
investing in business systems within the context of an EA. GFEBS, GCSS- 
Army, and LMP are not being managed and developed in the context of a 
well-defined Army-wide EA. In August 2006,[Footnote 34] we reported 
that the Army was in the very early stages of developing an EA. We 
recommended that the Secretary of Defense ensure that the DOD 
architecture programs[Footnote 35] we reviewed develop and implement 
plans for fully satisfying each of the conditions in our Enterprise 
Architecture Management Maturity Framework (EAMMF) for assessing and 
improving EA management.[Footnote 36] In commenting on the report, DOD 
agreed with our recommendation. 

As detailed in our August 2006 report, we assessed the Army's efforts 
to develop an EA against the criteria specified in EAMMF. Our EAMMF is 
a five-stage architecture framework for managing the development, 
maintenance, and implementation of an architecture and understanding 
the extent to which effective architecture management practices are 
being performed and where an organization is in its progression toward 
having a well-managed architecture program. In short, the framework 
consists of 31 core elements that relate to architecture governance, 
content, use, and measurement. These elements reflect research by us 
and others showing that architecture programs should be founded upon 
institutional architecture commitment and capabilities, and measured 
and verified products and results. Our analysis of information provided 
by the Army indicated that it had satisfied only 3 percent of all 
framework elements. In essence, this means that the Army is at stage 1 
of developing and implementing an EA. While stage 1 agencies may have 
initiated some enterprise architecture activity, these agencies' 
efforts are ad hoc and unstructured, lack institutional leadership and 
direction, and do not provide the management foundation necessary for 
successful enterprise architecture development. 

In May 2007, we reported that the Army still had not developed an 
EA.[Footnote 37] As a result, the Army does not have a well-informed 
basis for determining if these systems will fit within the context of 
future Army business operations and will most efficiently and 
effectively address the Army's long-standing weaknesses associated with 
the lack of asset visibility. Improving asset visibility is critical to 
addressing the problems associated with supply chain management, which 
has been on our high-risk list since 1990.[Footnote 38] Asset 
visibility is one of the focus areas that are a critical part of supply 
chain management.[Footnote 39] 

In addition, without a clear understanding of its current systems and 
business processes--commonly referred to as the "As Is" environment-- 
and the business operations it envisions for the future--commonly 
referred to as the "To Be" environment--the Army will increase the risk 
that misalignments can occur that can introduce redundancies and 
incompatibilities that can produce inefficiencies and require costly 
and time-consuming rework to fix. GFEBS, GCSS-Army, and LMP have 
proceeded without a common, institutional frame of reference (for 
example, EA) that can be used to effectively manage their relationships 
and dependencies. In November 2006, DOD's internal review of GFEBS by 
BTA noted that the Army lacked a strategy for integrating GFEBS, GCSS- 
Army, and LMP. 

Army Strategy to Achieve TAV Does Not Utilize an Enterprisewide 
Perspective: 

The Army also lacks a concept of operations for how GFEBS, GCSS-Army, 
and LMP will collectively achieve TAV. An effective concept of 
operations would describe, at a high level, (1) how the three business 
systems relate to each other in achieving the Army's TAV goal, and (2) 
how information flows from and through these systems. Moreover, we 
found that the Army's lack of a concept of operations has contributed 
to its failure to take full advantage of business process reengineering 
opportunities that are available when using an ERP solution. Without a 
clear long-term strategy, the Army lacks a key management control for 
ensuring that time frames, results-oriented performance measures, and 
accountability mechanisms are established and monitored to help achieve 
TAV. 

Specifically, we noted that the Army's strategy perpetuates some of the 
cumbersome and inefficient business processes that are currently being 
used in the existing legacy system environment. One of the key benefits 
of an ERP is that it can be used to streamline business processes and 
eliminate data redundancy. However, some of the basic business 
processes being used or expected to be used by GFEBS, GCSS-Army, and 
LMP are based on the Army's existing business processes--many of which 
are error prone, labor intensive, and redundant. Considering the 
billions of dollars the Army is investing in these systems, it is 
incumbent upon the Army to embrace and utilize the most efficient and 
streamlined ERP processes to the fullest extent possible in its effort 
to achieve TAV. 

For example, the Army's planned strategy to improve control and 
accountability over its PP&E does not take advantage of the 
capabilities of an ERP solution. (See app. III for additional examples 
in which the Army's TAV systems initiatives have not effectively 
streamlined legacy processes related to funds control and 
disbursements.) Financial data, including acquisition cost and 
depreciation, for most of the Army's PP&E are currently maintained in 
the Defense Property Accountability System (DPAS). PP&E accountability 
data, such as location and quantity, are maintained in a variety of 
legacy systems such as the Property Book Unit Supply Enhanced (PBUSE) 
system--which is 1 of the 16 systems that is expected to be replaced by 
GCSS-Army. In March 2007, Army Program Executive Office Enterprise 
Information Systems officials informed us that the financial data--such 
as acquisition cost and depreciation--for the Army's PP&E will be 
transferred from DPAS to GFEBS. This maintains the existing accounting 
data in an accounting system. On the other hand, the accountability 
data--such as location, condition, and number of units--for PP&E assets 
will initially be transferred from the current legacy system to PBUSE 
or GCSS-Army, depending on whether the items belong to an installation 
or an Army unit that may be deployed. The transferring of items to 
PBUSE appears to be an interim solution since, according to DOD's 
enterprise transition plan GCSS-Army is to replace PBUSE. However, GCSS-
Army is not expected to assume the control and accountability over 
items that are recorded in PBUSE associated with the Army Working 
Capital Fund or other Army organizational entities that are not 
considered to be units that can be deployed (field/tactical units) in 
military operations. 

The Army's decision to segregate its PP&E financial and accountability 
data into two separate system solutions reflects its lack of an 
enterprise concept of operations and enterprise view for achieving TAV. 
The Army's strategy of moving PP&E data from DPAS to PBUSE, and 
ultimately to another system, means that one data conversion will take 
place to move the data from DPAS to PBUSE--a system the Army is 
supposed to eliminate with the implementation of GCSS-Army--followed by 
another data conversion from PBUSE to yet another system, not yet 
identified. Further, the Army is continuing to limit the capabilities 
of LMP by transferring all information on PP&E from DPAS to PBUSE-- 
which as noted before is considered a legacy business system. As a 
result, information on PP&E that could enhance Tobyhanna's operations, 
especially in the area of workload planning, will not be available 
within the LMP environment. 

Thus, the approach being taken by the Army is simply to adapt the 
processes used by the legacy systems rather than use the streamlined 
processes inherent in the ERP solution. If the functionality of an ERP 
solution was fully utilized by the Army, it would increase the 
likelihood that TAV could be accomplished within each of the three 
systems for the specific type of PP&E items each was responsible for. 
For example, GFEBS would contain both the financial and accountability 
information related to nontactical/non-working capital fund PP&E, GCSS- 
Army would have all the information associated with tactical PP&E, and 
LMP would have the data related to Working Capital Fund PP&E. Figure 5 
illustrates one way that an ERP approach could be utilized to achieve 
physical and financial accountability control of the Army's PP&E. 

Figure 5: Example of an ERP Vision for Accountability of the Army's 
PP&E: 

[See PDF for image] 

Source: GAO. 

[End of figure] 

While the Army's approach may result in incremental improvements in its 
asset accountability, it does not take full advantage of the (1) 
benefits of adopting enterprise processes and (2) functionality that is 
available in the ERP solution it has selected to support those 
processes. Fundamental to a successful ERP implementation is the 
reengineering of an organization's business processes in a manner that 
helps ensure that the right resources (people, material, machinery, and 
funds) are available in the correct quantities when needed.[Footnote 
40] While Army has stated that its use of ERP software would help 
reengineer its business processes, we found that at least some business 
processes that are being used or expected to be used in the future do 
not reflect reengineered processes necessary to most effectively 
implement an ERP solution. By perpetuating the use of cumbersome, error-
prone, and ineffective business processes in its asset accountability 
operations, the Army will diminish its capability to achieve TAV and 
improve accountability over its assets. 

Army's Ability to Effectively Oversee Portfolios of Business Systems 
Investment Is Not Yet Fully Developed: 

While the Army has established a governance structure to oversee its 
business system investments--including its asset accountability system 
investments--that is consistent with DOD guidance, additional actions 
are needed to enhance oversight, control, and accountability. The 
Army's business system investment oversight efforts to date have 
primarily focused on ensuring that business systems modernizations over 
$1 million are reviewed in accordance with the criteria specified in 
the fiscal year 2005 National Defense Authorization Act. Both DOD and 
Army officials acknowledged that the department's and Army's investment 
review processes, particularly related to their ability to review 
business system investments from a portfolio perspective, are in the 
early stage of maturity. Portfolio management is a conscious, 
continuous, and proactive approach to allocating limited resources 
among competing initiatives in light of the investments' relative 
benefits. Taking an enterprise view enables an organization to consider 
its investment comprehensively, so that collectively the investments 
optimally address the organization's mission, goals, and objectives. In 
addition, we found that the Army did not have reliable processes and 
analyses to support its oversight of individual systems modernization 
program efforts intended to improve asset visibility. Specifically, (1) 
the Army has not established or implemented an independent verification 
and validation (IV&V) process for the three systems--GFEBS, GCSS-Army, 
or LMP and (2) the August 2004 GFEBS economic analysis was not prepared 
in accordance with DOD and OMB guidance.[Footnote 41] 

Army's Oversight of Its Business System Investments Lacks a Portfolio 
Perspective: 

The Army's oversight of its business system investments--including 
system investments intended to achieve TAV--continues to be focused on 
the review and approval of individual business system investments. In 
May 2007, we reported that DOD needed to improve its policies and 
procedures associated with managing its business system investments as 
portfolios.[Footnote 42] More specifically, we found that DOD had not 
yet progressed from project-based processes to portfolio-based 
processes, a key element for effectively managing business system 
transformation efforts. DOD informed us that DOD components are 
responsible for developing and managing their own portfolio management 
processes. 

Army IT guidance for managing IT investments requires portfolios to be 
managed and monitored using established quantifiable outcome-based 
performance measures to determine whether to recommend continuations, 
modification, or termination of individual investments within a 
portfolio.[Footnote 43] However, the Army has not implemented processes 
or procedures that facilitate an enterprise view toward management and 
oversight of portfolios of business system investments intended to 
collectively provide a specific capability or functionality. Rather, 
the Army's current business system investment process focuses primarily 
on reviewing and approving investment packages submitted by the 
responsible program management office for individual systems 
improvement efforts that have development/modernization funding over $1 
million. Individual business system investments that exceed the $1 
million systems modernization threshold are required to be reviewed by 
the appropriate IRB and approved by the DBSMC. 

Army officials acknowledged that the Army's ability to provide 
portfolio-based investment review and oversight is in the early stages 
of development. For example, while the Army has begun a process to 
share investment package information, including cost, schedule, and 
performance data, across its mission areas and domains, Army officials 
acknowledged that the information provided by a program management 
office, as well as the associated oversight and review, is centered 
largely on the functionality and capabilities of individual systems. 
For example, annual certifications for the three systems, LMP, GCSS- 
Army, and GFEBS, which the Army intends to utilize collectively in 
achieving TAV, are not submitted or reviewed as a portfolio, but rather 
as individual system investments. The Army has not yet implemented the 
processes to evaluate and improve its progress toward achieving TAV 
using portfolio-based projects such as costs, schedule, performance, 
and risks. As a result, the Army's ability to effectively oversee the 
development of the portfolio of systems intended to collectively 
provide the Army with TAV is limited. Until the Army's governance 
process, policies, and procedures mature, including its ability to 
apply a corporate portfolio perspective in managing and overseeing 
subportfolios of business system investments intended to provide a 
desired capability or function, the Army is at risk of implementing 
systems that (1) do not provide the desired capability, including TAV; 
(2) are stovepiped in their functionality; (3) do not interoperate in 
an efficient manner; and (4) are not supported by efficient and 
effective business processes. 

Reliable Processes and Analyses Are Needed to Facilitate Management 
Oversight of Business System Investments: 

The Army has not implemented reliable processes and analyses needed to 
enhance its management oversight of program management office efforts 
to develop and implement its TAV, as well as other business systems on 
time, within budget, and with the intended capability. Both DOD's and 
the Army's business system investment oversight processes are highly 
dependent upon the reliability of investment information provided by 
Army program management offices. Thus, the reliability of that business 
system investment information is critical to both DOD's and the Army's 
ability to effectively oversee, manage, and redirect--if necessary--the 
services' business system development and modernization efforts. 
However, we found that the Army had not yet established an independent 
verification and validation function for any of the three systems we 
reviewed. In addition, the economic analysis submitted by the program 
management office for GFEBS was not prepared in accordance with DOD or 
OMB guidance. As a result, the economic analysis submitted by the 
program management office to DOD and the Army oversight entities 
justifying the Army's investment in GFEBS was based on questionable 
costs and benefits. 

Independent Verification and Validation Would Enhance Program 
Oversight: 

The Army had not yet established an IV&V function[Footnote 44] for any 
of the three systems--GFEBS, GCSS-Army, and LMP. While the Army has 
established a verification and validation function for LMP, it was not 
an independent review because the reviewer reports directly to the LMP 
program management office. Independence is a key component to reliable 
verification and validation function. 

Best business practices have demonstrated that use of an IV&V function 
is an invaluable means to providing management reasonable assurance 
that a planned system, or the portfolio of systems, will satisfy its 
planned use and users. An effective IV&V review process should provide 
an independent assessment to DOD and Army management of the overall 
status of the project, including a discussion of any existing or 
potential revisions to the project with respect to cost, schedule, and 
performance. The IV&V reports should identify to management the issues 
or weaknesses that increase the risks associated with the project or 
portfolio to senior management so that they can be promptly addressed. 
These assessments involve reviewing project documentation, 
participating in meetings at all levels within the project, and 
providing periodic reports and recommendations, if deemed warranted, to 
senior management. The IV&V function should report on every facet of a 
system project such as: 

Testing program adequacy. Testing activities would be evaluated to 
ensure they are properly defined and developed in accordance with 
industry standards and best practices. 

Critical-path analysis. A critical path defines the series of tasks 
that must be finished in time for the entire project (or portfolio of 
projects) to finish on schedule. Each task on the critical path is a 
critical task. A critical-path analysis helps to identify the impact of 
various project events, such as delays in project deliverables, and 
ensures that the impact of such delays is clearly understood by all 
parties involved with the project(s). 

System strategy documents. Numerous system strategy documents that 
provide the foundation for the system development and operations are 
critical aspects of an effective system project. These documents are 
used for guidance in developing documents for articulating the plans 
and procedures used to implement a system. Examples of such documents 
include the Life-cycle Test Strategy, Interface Strategy, and 
Conversion Strategy. 

Validity of GFEBS's Economic Analysis Questionable: 

Our analysis of the August 2004 GFEBS economic analysis found that it 
was not prepared in accordance with OMB and DOD guidance.[Footnote 45] 
The purpose of an economic analysis is to give the decision maker 
insight into economic factors that have a bearing on accomplishing the 
stated objective of the system investment, for example implementation 
of GFEBS to improve the Army's financial accountability. As such, it is 
important that the economic analysis reliably identifies factors, such 
as cost and performance risks and drivers that can be used to establish 
priorities and allocate resource allocations. While it may be 
appropriate to invest in a particular IT investment, such as GFEBS, for 
reasons other than estimated economic benefits, nonetheless, the issues 
we identified raise questions as whether the funds invested in GFEBS 
were economically justified. Our specific concerns with the August 2004 
GFEBS economic analysis are highlighted below. 

* Inappropriate cost savings. At least $142 million of estimated 
savings were not savings, but rather, should have been classified as 
transfers--which do not equate to economic benefits. Transfers 
represent shifts of control over resource allocation from one group to 
another that do not result in a net change in the value of the 
resources involved in the transfer. For example, we found that the Army 
claimed over $88 million of savings related to transferring real 
property management from the Army's legacy system to GFEBS. Since the 
Army's real property management responsibilities were not eliminated, 
only transferred from the legacy system environment to GFEBS, this 
represents a transfer rather than an economic benefit for the Army. 
Additionally, the Army claimed benefits of about $54 million related to 
GFEBS reducing the length of time funds are borrowed by the Treasury to 
meet the Army financial obligations by facilitating the Army's ability 
to make just-in-time disbursements. OMB guidance provides that there 
are no economic gains from a pure transfer payment because the benefits 
to those government entities that receive such a transfer are matched 
by the costs borne by those government entities that provide the 
transfer.[Footnote 46] The Army's inappropriate classification of 
transfers resulted in an overstatement of the economic benefits that 
would be achieved by implementing GFEBS. Additionally, in November 
2006, BTA's enterprise risk assessment management review of GFEBS 
questioned justifying the GFEBS program on factors not fully within the 
program's control. 

* Lack of sensitivity analysis. The analysis did not include an 
assessment of the effects of the uncertainty inherent in estimates of 
GFEBS benefits and costs, as stipulated in OMB and DOD 
guidance.[Footnote 47] Because an economic analysis uses estimates and 
assumptions, it is critical that a sensitivity analysis be performed to 
understand the effects of the imprecision in both underlying data and 
modeling assumptions. This analysis is required since the estimates of 
future benefits and costs are subject to varying degrees of 
uncertainty. Sensitivity analysis refers to changing the value of a 
given variable in a model to gauge the effect of change on model 
results. It varies a single data element or assumption while holding 
the other data elements and assumptions constant to determine what 
amount of change in that element is required to raise or lower the 
resulting benefit and cost elements. In this way, GFEBS data and 
assumptions can be risk-ranked for decisionmaking and auditing. The 
Army's failure to conduct a sensitivity analysis of GFEBS, as part of 
its economic analysis, to identify the effect of uncertainties 
associated with different assumptions increases the chance that 
decisions regarding GFEBS will be made without a clear understanding of 
the possible impact on GFEBS estimates of costs and benefits. Army 
officials informed us that they will consider including a sensitivity 
analysis of the effect of uncertainties when they update GFEBS's 
economic analysis. 

* Lack of documentation supporting benefits and cost assumptions. The 
GFEBS program management office could not provide us with benefits and 
costs data needed to replicate their analyses. As a result, we were 
unable to trace the cost and benefit estimates reported in the GFEBS 
economic analysis to original source documentation to assess and 
validate the reliability and applicability of the data. OMB and DOD 
guidance stress the importance of maintaining documentation supporting 
assumptions used in a cost-benefit analysis. 

LMP Illustrates Continuing Problems in Implementing Business Systems on 
Time, within Budget, and with the Promised Capability: 

LMP continues to be plagued by operational problems that have beset the 
system virtually since its initial implementation in July 2003. While, 
an EA, concept of operations, and effective IT portfolio management are 
essential elements in the Army's efforts to transform its operations, 
it is equally important for the Army to have disciplined processes in 
place to implement its business systems on time, within budget, and 
with the promised capability. Many of DOD's approximately 2,980 
business systems are nonintegrated, stovepiped, and not capable of 
providing departmental management and the Congress accurate and 
reliable information on DOD's day-to-day operations. LMP is no 
exception. As previously noted, LMP was to have reached FOC in fiscal 
year 2005, but currently FOC is estimated for fiscal year 2010--a 
slippage of 5 years. Further, LMP's estimated program costs have also 
increased. Tobyhanna Army Depot--the only depot that is operating LMP-
-and DFAS personnel have stated that improvements have been made in 
LMP's operating efficiency. However, problems in the areas of revenue 
recognition and billing continue. These problems can, in part, be 
attributed to ineffective system testing. The operational issues 
confronting LMP and our concerns about the effectiveness of the system 
testing are highlighted below and discussed in more detail in appendix 
IV. 

Operational Issues Continue: 

Tobyhanna Army Depot continues to experience problems with LMP 
accurately recognizing revenue and billing customers. These problems 
have existed virtually since the implementation of LMP in July 2003. 
While the Army and its contractor--Computer Sciences Corporation (CSC)-
-have made numerous attempts to fix the problem, they have not been 
successful. Additionally, the accuracy of LMP financial reports 
continues to be questionable. For instance, the DOD Inspector General's 
audit of the Army's Working Capital Fund financial statements for 
fiscal year 2006 noted that LMP was not properly recording transactions 
in accordance with the U.S. Government Standard General Ledger 
requirements. 

For the 3-month period ending January 31, 2007, we found that LMP's 
continuing billing problems resulted in (1) customers not being billed 
for costs incurred that should have been billed and (2) customers being 
billed too much. Based upon the billing information provided by the LMP 
program management office, there were 146 customer orders valued at 
approximately $5.4 million that were not billed (or recognized as 
revenue) during the January 31, 2007, billing cycle. Customer billings 
and the associated revenue provide the means by which the depot 
finances its day-to-day operations. Similarly, during the January 31, 
2007, billing cycle about 308 customer orders, amounting to about $5.8 
million, were shown as being overbilled and a credit should have been 
provided to the customer. Moreover, the continuing billing and revenue 
recognition problem may be a factor contributing to higher depot bills 
from DFAS. According to DFAS personnel, the recurring billing problems 
have resulted in DFAS personnel processing Tobyhanna Army Depot's bills 
manually. Based on information provided by DFAS, the number of hours 
spent to provide DFAS accounting services--including billings for the 
Tobyhanna Army Depot--has increased from approximately 17,800 hours in 
fiscal year 2004 to over 22,600 hours in fiscal year 2006. 

System Testing Remains a Concern: 

Our prior audits[Footnote 48] of LMP identified significant weaknesses 
with the LMP program management office's efforts to effectively 
implement the requirements management and testing processes needed to 
reduce risks to acceptable levels. During our current audit, we found 
that the requirements management processes have improved--a critical 
first step--but concern remains with respect to the adequacy of the 
testing. 

Our analysis of 10 selected LMP corrective actions identified specific 
testing weaknesses in each of the corrective actions reviewed. For 
example, none of the test scripts provided adequate information that 
linked the items tested to the specific requirement being tested. This 
linking is commonly referred to as traceability and is characteristic 
of a disciplined testing process. The test scripts reviewed contained 
headings for information on the scenario, key data requirements, 
expected results, actual results, and whether the test was considered 
successful. However, the actual test script did not contain the level 
of specificity that clearly delineated how a specific requirement 
identified in the requirement document(s) and the associated 
requirement(s) in that document were being tested. As a result, it was 
not possible to determine if all LMP system requirements were properly 
tested. Without linking a given requirement to the tests designed to 
exercise that requirement, it was impossible to obtain reasonable 
assurance that (1) all the requirements were tested and (2) the applied 
test provided adequate coverage for each requirement. 

After discussing our concerns with the LMP program management office 
and its contractor in February 2007, the LMP program office requested 
that its verification and validation (V&V) contractor perform an 
assessment of the LMP testing process. According to information 
provided by the LMP program office and the V&V contractor, in March 
2007, the review found that the LMP testing process was adequately 
planned but that it did not support a clear understanding of the (1) 
requirement being tested and (2) tests used to determine whether a 
requirement was adequately implemented. The V&V contractor's assessment 
substantiates our analysis of the LMP testing process. According to the 
V&V contractor, these areas need to be addressed before the Army Test 
and Evaluation Command conducts its review of LMP. 

According to LMP program management office officials, the CSC is 
developing and implementing a testing process that is designed to 
address the concerns raised by us and the Army V&V contractor. CSC is 
expected to have a testing program that clearly links the requirements 
to the tests being conducted and to establish an independent test group 
to ensure that the testing process is following best practices. 
However, until the Army has reasonable assurance that an effective 
testing process has been properly implemented, we are reiterating our 
June 2005 recommendation that LMP not be deployed to additional 
locations until the Army has assurance that LMP is providing the 
intended functionality and, more specifically, that LMP can accurately 
bill its customers and recognize revenue. 

Conclusions: 

The Army's efforts to develop and implement GFEBS, GCSS-Army, and LMP 
may result in incremental improvements in the Army's accountability and 
visibility over its billions of dollars in tangible assets. However, 
the Army has not developed or utilized key management tools and 
concepts necessary to successfully transform its business processes to 
achieve TAV. The primary question that the Army has yet to effectively 
consider and address is how these systems and associated processes, 
individually and collectively, will provide the desired functionality 
necessary to achieve TAV. Until the Army develops and implements an 
Army EA, concept of operations, and portfolio-based management and 
oversight processes, it continues to risk (1) investing billions of 
dollars in asset accountability systems that may not enhance the 
department's and the Army's goal of achieving TAV, and (2) further 
delay DOD's efforts to remove supply chain management from our high- 
risk list. 

It is also important that the Army take action to implement the 
disciplined processes necessary to implement asset accountability 
systems on time, within budget, and with the promised capability. 
Continuing problems with LMP are illustrative of the consequences in 
failing to address these issues. Although LMP became operational 4 
years ago, it continues to encounter operational problems. While there 
have been some improvements in LMP, key functionality, such as 
accurately billing customers and recognizing revenue, remains 
problematic. The Army's inability to resolve these persistent LMP 
problems can, in part, be attributed to inadequate system testing. 
Until LMP implements an effective testing process, it will have little 
assurance that the development and corrective actions it takes (1) are 
properly developed, and (2) do not introduce additional defects into 
the system. 

Recommendations for Executive Action: 

To improve the department's efforts to achieve TAV and further enhance 
its efforts to improve the control and accountability over business 
system investments, we recommend that the Secretary of Defense direct 
the Secretary of the Army and the Director, BTA, to jointly take the 
following five actions: 

* Develop a concept of operations that (1) clearly defines the ERP 
vision for accomplishing total asset visibility within the Army; (2) 
addresses how its business systems and processes, individually and 
collectively, will provide the desired functionality to achieve TAV; 
and (3) determines the desired functionality among the selected 
systems. 

* Develop policies, procedures, and processes to support the oversight 
and management of selected groupings of business systems that are 
intended to provide a specific capability or functionality, such as TAV 
from a portfolio perspective, utilizing indicators such as costs, 
schedule, performance, and risks. 

* Establish an IV&V function for GFEBS, GCSS-Army, and LMP. 
Additionally, direct that all IV&V reports for each system be provided 
to Army management, the appropriate IRB, and BTA. 

* Require that any future GFEBS economic analysis identify costs and 
benefits in accordance with the criteria specified by DOD and OMB 
guidance, to include a sensitivity analysis. 

* Direct that LMP utilize systems testers that are independent of the 
LMP system developers to help ensure that the system is providing the 
users of the system the intended capabilities. 

Agency Comments and Our Evaluation: 

We received written comments on a draft of this report from the Deputy 
Under Secretary of Defense (Business Transformation), which are 
reprinted in appendix II. Overall, DOD concurred with our 
recommendations and stated that it would work diligently to implement 
them. The comments included two sets of specific responses to our 
recommendations--one set provided by BTA and another provided by the 
Army. In regard to the response provided by the Army, it concurred with 
each of the recommendations. BTA stated that it fully agreed with our 
observations. BTA, though, partially concurred with all the 
recommendations on the basis that they were directed jointly to the 
Secretary of the Army and the Director, BTA. BTA stated that under U.S. 
Code Title 10 and in accordance with DOD's tiered accountability, the 
Army has complete authority for execution of its programs and 
responsibility for implementing the recommendations. BTA's comments 
noted that it has neither the authority nor the responsibility to 
direct the actions of the Army. 

We appreciate the department's willingness to address our 
recommendations. With regard to BTA's concern, our recommendations do 
not direct BTA to oversee or direct the Army. Rather, the 
recommendations stated that the specific actions should be undertaken 
jointly at the direction of the Secretary of Defense. We continue to 
believe that a cooperative and effectively coordinated BTA and Army 
approach to addressing our recommendations is most likely to achieve 
the fundamental business system transformation necessary to achieve the 
department's TAV objective. Further, the involvement of the Office of 
the Secretary of Defense, including BTA, in the Army's investment 
management practices is consistent with 10 U.S.C. §§ 186 and 2222, 
which provide for DOD policymaking and oversight of Army business 
system functions. 

In regard to the responses provided by the Army, it acknowledged the 
importance of a concept of operations in achieving its goal of TAV and 
in optimizing its business system investments. Further, the Army 
acknowledged that there are risks involved in attempting to effectively 
integrate the various system efforts. Going forward, as we recommended, 
it will be important that the Army develop a clear, long-term strategy 
to help ensure that time frames, results-oriented performance measures, 
and accountability mechanisms are established and monitored to achieve 
TAV. 

We are sending copies of this report to the Secretary of Defense; 
Acting Secretary of the Army; Under Secretary of Defense (Acquisition, 
Technology, and Logistics); Under Secretary of Defense (Comptroller); 
Deputy Under Secretary of Defense (Business Transformation); Deputy 
Under Secretary of Defense (Financial Management); Assistant Secretary 
of Defense (Networks and Information Integration); Assistant Secretary 
of the Army (Acquisition, Logistics, and Technology); Assistant 
Secretary of the Army (Financial Management and Comptroller); Army 
Chief Information Officer; Army Deputy Chief of Staff (Logistics); 
Commander, U.S. Army Materiel Command; and other interested 
congressional committees and members. Copies of this report will be 
made available to others upon request. In addition, this report is 
available at no charge on the GAO Web site at http://www.gao.gov. 

Please contact McCoy Williams at (202) 512-9095 or williamsm1@gao.gov, 
Keith A. Rhodes at (202) 512-6412 or rhodesk@gao.gov, or William M. 
Solis at (202) 512-8365 or solisw@gao.gov if you or your staff have 
questions on matters discussed in this report. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. Key contributors to this report are 
listed in appendix V. 

Signed by: 

McCoy Williams: 
Director: 
Financial Management and Assurance: 

Signed by: 

Keith A. Rhodes: 
Chief Technologist: 
Applied Research and Methods: 
Center for Technology and Engineering: 

Signed by: 

William M. Solis: 
Director: 
Defense Capabilities and Management: 

[End of section] 

Appendix I: Scope and Methodology: 

To determine whether the Army has developed a business system strategy 
for achieving total asset visibility (TAV), we met with program office 
officials for the General Fund Enterprise Business System (GFEBS), the 
Global Combat Support System-Army (GCSS-Army), and the Logistics 
Modernization Program (LMP). We obtained briefings on the intended 
purpose of each system and walkthroughs of various scenarios that 
described how the systems would transmit data between them. We also met 
with officials from the Office of Program Analysis and Evaluation, the 
Office of the Assistant Secretary of Defense (Network and Information 
Integration)/the Department of Defense (DOD) Chief Information Officer, 
and the Business Transformation Agency (BTA) to identify and discuss 
any issues/concerns they have related to the implementation of GFEBS 
and GCSS-Army. Furthermore, we reviewed reports by the Army Audit 
Agency and the DOD Inspector General to ascertain if they had 
previously reported upon system weaknesses related to the three systems 
and how those weaknesses would affect the Army's vision of an 
integrated systems strategy. 

In order to determine if the Army has effectively implemented a 
governance structure to oversee and manage its business system 
investment in accordance with DOD guidance, we reviewed guidance issued 
by the Office of the Secretary of Defense, the Army, and BTA related to 
investment management. We also obtained an understanding of the Army's 
process to comply with the certification and annual system review 
processes required by the fiscal year 2005 National Defense 
Authorization Act.[Footnote 49] We also obtained and analyzed 
documentation related to the certification and annual system review 
process. Furthermore, we reviewed the Selected Capital Investment 
Review reports as of February 2006 and February 2007 to ascertain if 
there were any cost, schedule, and performance issues discussed in 
these reports for the GFEBS, GCSS-Army, and LMP. We also reviewed 
Defense Acquisition Executive Summary reports for GFEBS to ascertain if 
there were any cost, schedule, and performance issues discussed in 
these reports. Similar reports were not required for GCSS-Army and LMP. 
Additionally, we met with BTA officials to obtain an understanding of 
how the Enterprise Risk Assessment Model is to be utilized as part of 
the department's investment management criteria. 

To evaluate the extent to which the Army has made progress in 
correcting the previously reported problems regarding LMP's 
implementation, we met with officials from the LMP program office and 
the Computer Sciences Corporation (CSC). Briefings were provided that 
detailed the specific corrective actions taken in response to the 
various issues discussed in our two previous reports.[Footnote 50] To 
substantiate the corrective actions taken to address the accounting 
issues, we reviewed the Defense Finance and Accounting Service (DFAS) 
1307 accounting reports[Footnote 51] and discussed the issues detailed 
in the reports with DFAS personnel responsible for preparing the 
reports. We also reviewed the fiscal year 2006 Army Working Capital 
Fund Financial Statements to ascertain the specific issues related to 
LMP. Further, we met with DOD Inspector General personnel and reviewed 
their workpapers to obtain an understanding of the deficiencies 
discussed in the fiscal year 2006 Army Working Capital Fund Financial 
Statements. 

To assess whether the Army had established and implemented disciplined 
processes related to requirements management and testing we: 

* obtained an understanding of the Army's revised procedures for 
defining requirements management frameworks and compared these 
procedures to its current practices; 

* reviewed guidance published by the Institute of Electrical and 
Electronics Engineers and the Software Engineering Institute and 
publications by experts to determine the attributes that should be used 
for developing good requirements; and: 

* selected 10 requirements and performed an in-depth review and 
analysis to determine whether they had the attributes normally 
associated with good requirements and whether these requirements traced 
between the various process documents. These requirements were to have 
followed the revised requirements management process. 

To augment these document reviews and analyses, we discussed our 
analysis with LMP program office and CSC officials. We also met with 
officials from the Office of Program Analysis and Evaluation, the 
Office of the Assistant Secretary of Defense (Network and Information 
Integration)/DOD Chief Information Officer (CIO), and the Business 
Transformation Agency to identify any issues/concerns they have related 
to the implementation of LMP. 

We performed our audit work from May 2006 through June 2007 in 
accordance with U. S. generally accepted government auditing standards. 
We conducted our work at the Office of the Assistant Secretary of 
Defense (Networks and Information Integration)/DOD Chief Information 
Officer; the U.S. Army Materiel Command; the Army CIO; and the Business 
Transformation Agency. We visited the Army Program Executive Office 
Enterprise Information Systems and GFEBS program office at Ft. Belvoir, 
Virginia. We also visited the Army contractor's site for LMP, Computer 
Sciences Corporation in Marlton, New Jersey--and for GCSS-Army-- 
Northrop Grumman in Chester, Virginia. We also visited the Tobyhanna 
Army Depot, Tobyhanna, Pennsylvania and DFAS Indianapolis, Indiana. We 
requested comments on a draft of this report from the Secretary of 
Defense or his designee. We received written comments from the Deputy 
Under Secretary of Defense (Business Transformation), which are 
reprinted in appendix II. 

[End of section] 

Appendix II: Comments from the Department of Defense: 

Note: GAO comments supplementing those in the report text appear at the 
end of this appendix. 

Office Of The Under Secretary Of Defense: 
9000 Defense Pentagon: 
Washington, DC 20301-3000: 
Acquisition, Technology And Logistics: 

Jul 6 2007: 

Mr. McCoy William: 
Director, Financial Management and Assurance: 
U.S. Government Accountability Office: 
441 G Street, N.W. 
Washington, DC 20548: 

Dear Mr. William: 

This is the Department of Defense (DoD) response to the GAO draft 
report 07-860, "DOD Business Transformation: Lack of an Integrated 
Strategy Puts the Army's Asset Visibility System Investments at Risk", 
dated June 6, 2007 (GAO Code 195087). 

DoD hereby acknowledges receipt of the draft report. We have reviewed 
the draft and appreciate the GAO's support in identifying areas of 
improvement as it relates to the Department of Army's total asset 
visibility. Overall, DoD concurs with the GAO's recommendations, except 
where it is recommended that the Business Transformation Agency (BTA) 
jointly direct the Army to implement a recommendation. Under Title 10 
U.S. Code, the BTA neither has the authority nor responsibility to 
direct the actions of the Army. Attached, you will find responses to 
these recommendations from both the BTA and the Army. 

Still, DoD agrees with the core of each recommendation, and we will 
work diligently to close them. Where appropriate, the BTA will provide 
assistance and support to the Army in achieving this objective. 
Additionally, it is DoD's intent, at an enterprise level, to continue 
maturing our portfolio management policies and procedures such that 
they align with and assist the Army in addressing the recommendations 
presented in the draft report. 

Again, thank you for your support. DoD strongly values our partnership 
with the GAO, and we appreciate your continued strides to assist us 
with our business transformation efforts. 

Signed for: 

Paul A. Brinkley: 
Deputy Under Secretary of Defense (Business Transformation): 

GAO Draft Report Dated June 6, 2007 GAO-07-860 (GAO code 195087): 

"DOD Business Transformation: Lack Of An Integrated Strategy Puts The 
Army's Asset Visibility System Investments At Risk" 

Department Of Defense Comments To The GAO Recommendations Provided By 
The Business Transformation Agency: 

Recommendation 1: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of Business 
Transformation Agency (BTA), to jointly develop a concept of operations 
that: 

a. clearly defines the enterprise resource planning (ERP) vision for 
accomplishing total asset visibility within the Army; 

b. addresses how its business systems and processes, individually and 
collectively, will provide the desired functionality to achieve total 
asset visibility (TAV); and: 

c. determines the desired functionality among the selected systems. 
(Page 42/GAO Draft Report): 

DOD Response: Partially Concur: 

The Department concurs with the Army's position. However, the overall 
Department position is "partially concur" due to the recommendation 
being directed jointly to the Secretary of the Army and the Director, 
BTA. While we fully agree with GAO's observations, under U.S. Code 
Title 10 and in accordance with the Department's tiered accountability 
approach, the Army has complete authority for execution of its programs 
and responsibility for implementing the recommendation. BTA will 
continue to provide assistance and support to the Army from an 
enterprise standpoint as needed. 

BTA has developed the BEA and has provided specific recommendations to 
the Army in addressing TAV and other related issues, and continues to 
work with the Army for the integration of the proper functionality. BTA 
has also evaluated the deployment of the ERP solutions in accordance 
with commercial best practices and recommendations to the Army which 
are designed to achieve TAV and proper financial controls. 

Recommendation 2: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of BTA, to jointly 
develop policies, procedures, and processes to support the oversight 
and management of selected groupings of business systems that are 
intended to provide a specific capability or functionality, such as TAY 
from a portfolio perspective, utilizing indicators such as costs, 
schedule, performance, and risks. (Page 43/GAO Draft Report): 

DOD Response: Partially Concur: 

The Department concurs with the Army's position. However, the overall 
Department position is "partially concur" due to the recommendation 
being directed jointly to the Secretary of the Army and the Director, 
BTA. While we fully agree with GAO's observations, under U.S. Code 
Title 10 and in accordance with the Department's tiered accountability 
approach, the Army has complete authority for execution of its programs 
and responsibility for implementing the recommendation. BTA will 
continue to provide assistance and support to the Army from an 
enterprise standpoint as needed. 

It is the Department's intent, at an enterprise level, to continue 
maturing our portfolio management policies and procedures. 

Recommendation 3: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of BTA jointly 
establish an independent verification and validation (IV&V) function 
for General Fund Enterprise Business System (GFEBS), Global Combat 
Support System-Army Field/Tactical (GCSS-Anny) and Logistics 
Modernization Program (LMP). Additionally, direct that all IV&V reports 
for each system be provided to Army management, the appropriate 
investment review board (IRB) and BTA. (Page 43/GAO Draft Report): 

DOD Response: Partially Concur: 

The Department concurs with the Army's position. However, the overall 
Department position is "partially concur" due to the recommendation 
being directed jointly to the Secretary of the Army and the Director, 
BTA. While we fully agree with GAO's observations, under U.S. Code 
Title 10 and in accordance with the Department's tiered accountability 
approach, the Army has complete authority for execution of its programs 
and responsibility for implementing the recommendation. BTA will 
continue to provide assistance and support to the Army from an 
enterprise standpoint as needed. 

Recommendation 4: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of BTA, to jointly 
require that any future GFEBS economic analysis identify costs and 
benefits in accordance with the criteria specified by DoD and OMB 
guidance, to include a sensitivity analysis. (Page 43/GAO Draft 
Report): 

DOD Response: Partially Concur: 

The Department concurs with the Army's position. However, the overall 
Department position is "partially concur" due to the recommendation 
being directed jointly to the Secretary of the Army and the Director, 
BTA. While we fully agree with GAO's observations, under U.S. Code 
Title 10 and in accordance with the Department's tiered accountability 
approach, the Army has complete authority for execution of its programs 
and responsibility for implementing the recommendation. BTA will 
continue to provide assistance and support to the Army from an 
enterprise standpoint as needed. 

Recommendation 5: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of BTA, to jointly 
direct the LMP utilize system testers that are independent of the LMP 
system developers to help ensure that the system is providing the users 
of the system the intended capabilities. (Page 43/GAO Draft Report): 

DOD Response: Partially Concur: 

The Department concurs with the Army's position. However, the overall 
Department position is "partially concur" due to the recommendation 
being directed jointly to the Secretary of the Army and the Director, 
BTA. While we fully agree with GAO's observations, under U.S. Code 
Title 10 and in accordance with the Department's tiered accountability 
approach, the Army has complete authority for execution of its programs 
and responsibility for implementing the recommendation. BTA will 
continue to provide assistance and support to the Army from an 
enterprise standpoint as needed. 

GAO Draft Report Dated June 6, 2007 GAO-07-860 (GAO code 195087): 

"DOD Business Transformation: Lack Of An Integrated Strategy Puts The 
Army's Asset Visibility System Investments At Risk" 

Department Of Defense Comments To The GAO Recommendations Provided By 
The Army: 

Recommendation 1: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of Business 
Transformation Agency (BTA), to jointly develop a concept of operations 
that: 

a. clearly defines the enterprise resource planning (ERP) vision for 
accomplishing total asset visibility within the Army; 

b. addresses how its business systems and processes, individually and 
collectively, will provide the desired functionality to achieve total 
asset visibility (TAV); and: 

c. determines the desired functionality among the selected systems. 
(Page 42/GAO Draft Report): 

DOD Response: Concur: 

* The Army is amenable to ideas that will facilitate our optimization 
goals to achieve TAV. We believe that a concept of operations that 
includes the three components described in Recommendation 1 will help 
us attain our optimization goals. We have blueprinted our systems based 
on requirements and priorities and divided that functionality among the 
three ERP systems. Above all, our goal is an efficient and effective 
deployment of automated business capabilities given the realities of 
where the Army is today, strategically, financially, and 
programmatically. 

* The Army agrees that we should optimize our IT business investments 
to achieve the goal of TAV. At program initiation, the Army 
functionally aligned its financial and logistics ERP solutions. We 
further recognize the synchronization risk between ERP programs and 
manage accordingly. Hence, the Army has a single materiel developer 
organization, PEO Enterprise Information Systems (PEO EIS), and a 
single manager to whom the ERP Product Managers report. A recent 
realignment of the Logistics Modernization Program (LMP) from an Army 
Command to the PEO EIS facilitated unity of purpose, helping to 
synchronize the acquisition aspects of a solution that enables TAV for 
the Army. Moreover, we have established a functional cross-domain 
working group between the Single Automated Logistics Enterprise (SALE) 
and the Single Automated Financial Enterprise (SAFE) that meets 
regularly to decide optimal implementations of business processes to 
achieve visibility of Army assets that transcend Domains and the 
tactical and national levels of logistics. 

Recommendation 2: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of BTA, to jointly 
develop policies, procedures, and processes to support the oversight 
and management of selected groupings of business systems that are 
intended to provide a specific capability or functionality, such as TAV 
from a portfolio perspective, utilizing indicators such as costs, 
schedule, performance, and risks. (Page 43/GAO Draft Report): 

DOD Response: Concur. 

While we are an Army at war, men and women are working diligently to 
develop and institutionalize an oversight system for business systems 
that is austere, yet effective; defined and repeatable but not 
bureaucratic; and focused on the Army's ability to man, equip, train, 
and maintain our Forces in the field. In this regard, the Army Business 
Mission Area has allocated resources to assist in the maturation of 
Army Portfolio Management (PfM). 

Recommendation 3: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of BTA jointly 
establish an independent verification and validation (IV&V) function 
for General Fund Enterprise Business System (GFEBS), Global Combat 
Support System-Army Field/Tactical (GCSS-Army) and Logistics 
Modernization Program (LMP). Additionally, direct that all IV&V reports 
for each system be provided to Army management, the appropriate 
investment review board (IRB) and BTA. (Page 43/GAO Draft Report): 

DOD Response: Concur. 

As previously stated, the Army has charged the PEO EIS to manage the 
cost, schedule, and performance aspects of the GFEBS, GCSS-Army, and 
LMP. The organization is governed by and accountable to the Milestone 
Decision Authority according to the program baselines established. The 
Army recognizes the need for a enterprise view of the risks associated 
with each program. Therefore, the PEO EIS will ensure establishment of 
an Independent Validation and Verification (IV&V) process for each 
program that provides for risk management at a level where management 
responsibility exists for each of the three ERP systems and independent 
of the programs. All PEO EIS IV&V results will be provided to the 
Department of the Army and relevant management organizations as 
required. 

Recommendation 4: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of BTA, to jointly 
require that any future GFEBS economic analysis identify costs and 
benefits in accordance with the criteria specified by DoD and OMB 
guidance, to include a sensitivity analysis. (Page 43/GAO Draft 
Report): 

DOD Response: Concur. 

* The GAO report cites $142M of questionable benefits, comprised of 
"transfers". While the economics of the transfers was debated with GAO, 
the real point should be that these are $142M questionable benefits of 
$6,667M total benefits. GAO takes issue with not adequately justifying 
2.1 % of the benefits. Assuming that the 2.1% is not legitimate, 
eliminating these benefits at the very worst lowers the Benefit-Cost 
Ratio from 2.35 to 2.28, which still would justify the investment. 

* The GAO report cites the lack of a sensitivity analysis. The Cost 
Review Board Working Group included the sensitivity analysis by adding 
into the estimate the uncertainty surrounding the costs of interfaces. 
While this does not show up as a titled section in the EA called 
"Sensitivity", the results of the analysis were in fact included in the 
GFEBS EA. 

Recommendation 5: The GAO recommends that the Secretary of Defense 
direct the Secretary of the Army and the Director of BTA, to jointly 
direct the LMP utilize system testers that are independent of the LMP 
system developers to help ensure that the system is providing the users 
of the system the intended capabilities. (Page 43/GAO Draft Report): 

DOD Response: Concur. 

The following are GAO's comments on DOD's letter dated July 6, 2007. 

GAO Comments: 

1. See the "Agency Comments and Our Evaluation" section of this report. 

2. In regard to the GFEBS economic analysis, the program office did not 
provide the necessary support for us to replicate the estimated program 
savings. 

3. In regard to the sensitivity analysis, the Army's approach of adding 
the uncertainty surrounding the costs of interfaces into the estimate 
is not consistent with OMB, DOD, and Army guidance, as discussed in the 
report. Rather, a sensitivity analysis is accomplished by changing the 
numerical value of a given variable to gauge the effect of that change 
on model results, such as the benefit-to-cost ratio. A sensitivity 
analysis identifies key assumptions and varies a single assumption 
while holding the others constant to determine what amount of change in 
that assumption is required to raise or lower the resulting dominant 
benefit or cost estimates by a set amount. 

[End of section] 

Appendix III: Army Strategy Does Not Fully Utilize Capabilities of ERP 
Solution: 

Fundamental to successful enterprise resource planning (ERP) 
implementation is the reengineering of an organization's business 
processes in a manner that helps ensure that the right resources are 
available when needed. While the Army has stated that through the 
successful implementation of the General Fund Enterprise Business 
System (GFEBS), the Global Combat Support System-Army (GCSS-Army), and 
the Logistics Modernization Program (LMP) will reengineer its business 
processes, we found some business processes that have been adopted or 
are expected to be adopted have not embraced the basic concept of an 
ERP solution. Details concerning the Army's strategy for handling funds 
control and disbursements are highlighted below. 

Funds Control: 

The funds control concept that will be utilized by GCSS-Army and GFEBS 
is that all funds control activities will be handled by GFEBS. 
Accordingly, an interface must be developed to transmit the appropriate 
information between the two systems in order to ensure that all of the 
information is in agreement. Figure 6 illustrates how the process is 
intended to work. 

Figure 6: Planned GFEBS and GCSS-Army Fund Control Process: 

[See PDF for image] 

Source: Army. 

[End of figure] 

As shown in figure 6, information will have to be transmitted from GCSS-
Army each and every time there is a transaction that is associated with 
funds control. This approach does not take full advantage of the 
capabilities within the ERP software that will be utilized by GCSS- 
Army. Figure 7 shows an example of more efficient and streamlined funds 
control process based on ERP concepts that eliminates these various 
transactions having to be processed between GFEBS and GCSS-Army. 

Figure 7: GCSS-Army Funds Control Process Utilizing ERP Capabilities: 

[See PDF for image] 

Source: Army. 

[End of figure] 

We were informed by GFEBS program officials that funds control for the 
activities associated with GCSS-Army needed to be maintained in GFEBS 
because (1) GCSS-Army was not considered an official accounting system, 
(2) the version of the software initially selected by GCSS-Army could 
not meet the requirements of the Federal Financial Management 
Improvement Act of 1996 (FFMIA),[Footnote 52] and (3) the Army wanted 
to centrally manage general fund activities. In effect, the Army's 
approach mirrors the functionality in the stovepiped processes of the 
legacy systems, dedicated processes to obtain information for specific 
reasons rather than ensuring a given process provided the information 
needed from an enterprise point of view. Furthermore, the concern that 
the version of software being used by GCSS-Army had not been certified 
as compliant with federal requirements can be easily addressed. Army 
officials have stated that the software vendor is committed to 
obtaining the necessary certifications and expects that these will be 
granted before GCSS-Army is deployed. From a corporate perspective, 
whether one system or two systems is the proper approach is not 
necessarily the question that the Army needs to answer. Rather, the 
Army needs to address how best to utilize the capabilities of the ERP 
to achieve an effective funds control system. 

Disbursements: 

The Department of Defense (DOD) Inspector General recently reported 
that LMP has been implemented without the functionality to match 
proposed disbursements with corresponding obligations before making any 
payments.[Footnote 53] This process is referred to as prevalidation, 
which was mandated with the passage of the fiscal year 1995 Department 
of Defense Appropriations Act[Footnote 54] because of DOD's long- 
standing difficulty in properly matching disbursements with 
corresponding obligations. Implementing LMP without this functionality 
is an example of suboptimizing the system, resulting in the failure to 
address a long-standing weakness in matching obligations with proposed 
disbursements for the Army Working Capital Fund. This functionality is 
part of the basic software package that is being used by LMP. We noted 
that the Army expects to use the same ERP software package for GFEBS. 

The DOD Inspector General reported that, rather than reengineering its 
business processes, the Army decided to follow its existing business 
processes that were inefficient and precluded the Army from being in 
compliance with the fiscal year 1995 legislative requirement. As a 
result of this decision, the Army has not taken full advantage of the 
capabilities within the ERP software package being used to implement 
LMP. Considering that the Army has already invested approximately $452 
million in LMP, and as previously noted is expecting to invest hundreds 
of million more, taking full advantage of its capabilities is also a 
prudent financial decision. Because the Army failed to reengineer its 
business processes, it continues to use an off-line database, managed 
by the Defense Finance and Accounting Service (DFAS), to compare 
entitlement and accounting data and provide approval for prevalidation 
requests. The DOD Inspector General report noted that the database 
contained errors in the accounting data, which resulted in 
prevalidation failures and the need for additional research and rework 
by DFAS. 

A more efficient and streamlined process could be achieved by adopting 
ERP processes for disbursements. For example, by utilizing ERP concepts 
LMP would be the accounting system of record and responsible for 
ensuring that the applicable federal requirements governing fund 
control and disbursements were effectively implemented. Further, the 
disbursements would be made directly by the Department of the Treasury-
-much like the process used by civilian agencies--rather than going 
through DFAS. We recognize that this approach involves some technical 
issues, such as building the necessary interfaces with the Department 
of the Treasury. However, the commercial-off-the-shelf (COTS) package 
adopted by LMP is already required by its Office of Management and 
Budget certification to provide the necessary support for these 
interfaces. The elimination of the DFAS interfaces, which are not 
inherently supported by the COTS package, should help reduce the cost 
and risk associated with this change. 

[End of section] 

Appendix IV: LMP Problems Continue: 

The Tobyhanna Army Depot continues to experience financial management 
problems because significant operational and developmental issues 
related to the Logistics Modernization Program (LMP) have not been 
resolved. More specifically, LMP continues to experience problems with 
accurately recognizing revenue and billing customers. This problem has 
existed virtually since the implementation of LMP in July 2003. While 
the Army and its contractor have made numerous attempts to fix the 
problem, they have not been successful. While we found that the 
requirements management processes have improved--a critical first step 
in reducing risk to acceptable levels[Footnote 55]--we continue to have 
reservations about the adequacy of LMP's testing. 

Billing Problems Continue to Plague LMP: 

LMP's continuing billing problems have resulted in (1) customers not 
being billed for costs incurred that should have been billed and (2) 
customers being billed too much. According to information provide by 
the LMP program office, 146 customer orders valued at approximately 
$5.4 million were not billed during the January 31, 2007, billing 
cycle. Customer orders that were not recognized as revenue and billed 
represent funds that Tobyhanna Army Depot is entitled to, because the 
work requested by the customer was performed. It is essential that 
revenue is properly recognized and customers billed for work performed 
because it is the means by which the depot finances its day-to-day 
operations. Similarly, during the January 31, 2007, billing cycle about 
308 customer orders amounting to about $5.8 million were shown as being 
overbilled and a credit should have been issued to the customer. It is 
critical that the credit be provided to the customer as soon as 
possible since the customer may be able to use these funds to meet 
other funding demands. 

Our analysis of the January 2007 billings for Tobyhanna disclosed that, 
of the 146 unbilled customer orders, 61 (about 42 percent) related to 
fiscal years 2006 and 2007. Similarly, for the credits, 178 of 308 
customer orders (about 58 percent) were for fiscal years 2006 and 2007. 
From a dollar perspective, most of these billings also related to 
fiscal years 2006 and 2007 transactions. The fact that these billing 
problems have persisted clearly indicates that the various efforts by 
the Army to resolve this critical aspect of the system have failed. 
Table 1 shows the dollar value and the number of customer orders 
related to unbilled work and credits for 3 months--November and 
December 2006 and January 2007. 

Table 1: Unbilled and Credit Customer Orders at the Tobyhanna Army 
Depot for November 2006 through January 2007: 

Month: November 2006; 
Unbilled customer orders: 84; 
Dollar value of unbilled customer orders: (3,161,490); 
Customer orders with credits: 291; 
Dollar value of customer orders with credits: 3,156,533. 

Month: December 2006; 
Unbilled customer orders: 92; 
Dollar value of unbilled customer orders: (2,637,903); 
Customer orders with credits: 301; 
Dollar value of customer orders with credits: 4,772,884. 

Month: January 2007; 
Unbilled customer orders: 146; 
Dollar value of unbilled customer orders: (5,367,798); 
Customer orders with credits: 308; 
Dollar value of customer orders with credits: 5,792,851. 

Source: LMP program office. 

[End of table] 

The continuing billing problem at Tobyhanna is also a factor 
contributing to higher depot bills from the Defense Finance and 
Accounting Service (DFAS)--the Department of Defense's (DOD) 
centralized finance and accounting organization. According to DFAS 
personnel, the recurring billing and other problems have resulted in 
DFAS processing Tobyhanna's bills manually and performing other manual 
actions that are not required by the other depots that do not use LMP. 
According to the information provided by DFAS, the number of hours 
spent on DFAS accounting services billings for Tobyhanna has increased 
between fiscal years 2004 and 2006. Table 2 shows the number of hours 
and related amounts DFAS billed Tobyhanna in each year. 

Table 2: DFAS Billings to the Tobyhanna Army Depot for Accounting 
Services: 

Fiscal year: 2004; 
Number of hours: 17,737; 
Amount billed (dollars): 1,105,168. 

Fiscal year: 2005; 
Number of hours: 21,197; 
Amount billed (dollars): 1,360,407. 

Fiscal year: 2006; 
Number of hours: 22,653; 
Amount billed (dollars): 1,432,777. 

Source: GAO based upon data provided by DFAS. 

[End of table] 

Furthermore, according to DFAS personnel, while the accounting needs of 
other depots requires the support of one full-time DFAS staff, DFAS 
uses three full-time personnel to service Tobyhanna due to the numerous 
manual workarounds associated with LMP. The manual workarounds required 
to address LMP's billing issues results in higher DFAS bills for 
Tobyhanna, which in turn increases the prices charged by Tobyhanna to 
its customers for the work performed. Since Tobyhanna is part of the 
Army Working Capital Fund, the costs incurred for accounting services 
are passed on to the customer in the form of higher depot prices. 
During the course of our audit, LMP program officials stated that they 
were finalizing efforts to revise the billing process. At completion of 
our field work, the LMP program office stated that it anticipated 
having the revised process in place in the August 2007 time frame. 

Since billing and revenue recognition have been such a long-standing 
issue, and given that the entire process is fundamental to the 
efficient and effective operation of the Working Capital Fund, it is 
incumbent upon the LMP program office to closely monitor the planned 
corrective actions and obtain reasonable assurance that the revised 
process is meeting the Army's expectations. System testing is a 
critical process that should help improve an entity's confidence that 
the system will satisfy the requirements of the end user and will 
operate as intended. If the planned corrective actions do not resolve 
the existing problem, Tobyhanna's operations will continue to be 
adversely affected. As we have previously recommended, LMP should not 
be deployed to any additional locations.[Footnote 56] 

Accuracy of Financial Information Reported by LMP Still a Concern: 

LMP continues to experience difficulty in accurately reporting on the 
results of operations. For example, the DOD Inspector General's fiscal 
year 2006 financial statement audit of the Army Working Capital Fund 
identified instances in which LMP was not properly recording 
transactions in accordance with the U.S. Government Standard General 
Ledger requirements. Further, our analysis of the DFAS monthly report 
on depot operations--commonly referred to as the 1307 report[Footnote 
57]--found that inaccurate account balances continue to persist in LMP. 
These issues have presented many challenges that are reflected 
throughout the Army Working Capital Fund financial statements as 
adjustments, corrections, and footnote disclosures. More specific 
details on the problems are highlighted below. 

* The DOD Inspector General Report on Internal Controls and Compliance 
with Laws and Regulations noted that the nonfederal accounts payable 
account was misstated by approximately $175 million at the end of 
fiscal year 2006. This misstatement arose because Army entities using 
LMP had not resolved approximately $89 million in abnormal accounts 
payable balances reported by the Supply Management business area. An 
abnormal account balance is one in which the recorded amount has been 
incorrectly classified. During fiscal year 2006, the LMP program office 
initiated systemic corrections to reduce the reported abnormal 
balances. Our analysis of the first quarter fiscal year 2007 financial 
statements noted that the reported balance had been reduced to 
approximately $81 million. The LMP program office was continuing to 
research the issue. 

* Our analysis of the DFAS 1307 reports found that abnormal balances in 
accounts payable have been reported since fiscal year 2004. LMP program 
officials stated that abnormal balances were primarily due to the 
migration of remaining obligations on open contracts from the legacy 
systems. Substantial effort was made in fiscal year 2006 to correct the 
abnormal accounts payable balances that were reported by LMP, which 
resulted in a reduction from $334 million at the end of fiscal year 
2005 to $88 million at the end of fiscal year 2006. The Army's efforts 
to clean up abnormal account balances continued in fiscal year 2007. 

The Army has developed a specific plan to address the financial 
management problems recognized in these statements. For example, a 
Tobyhanna Army Depot on-site support team has been assembled to 
complete the data cleanup, validate that all system and procedural 
issues have been identified, document business processes and 
procedures, and train Tobyhanna users. However, until these problems 
are corrected, LMP will not be able to provide reliable and accurate 
financial information to the Congress or Army management. 

Adequacy of LMP Testing Continues to Be a Concern: 

Our prior audits[Footnote 58] of LMP identified significant weaknesses 
with LMP's efforts to effectively implement the requirements management 
and testing processes needed to reduce risks to acceptable levels. 
During our current audit, we found that the requirements management 
processes have improved--a critical first step--but we continue to have 
reservations about the adequacy of the testing. LMP program officials 
stated that these testing weaknesses are being addressed and expect 
that an improved testing process will be effectively implemented to 
reduce risks to acceptable levels. 

LMP's Requirements Management Practices Have Improved: 

During the current review, Army and contractor Computer Sciences 
Corporation (CSC) officials stated that they have acted upon our prior 
recommendations and taken steps to improve the program's requirements 
management practices. In setting forth the revised requirements 
management process, LMP and CSC officials noted that the new process 
generally envisioned having several documents that outlined the 
requirements that needed to be addressed with each document containing 
specificity needed for the intended audience. 

To ascertain if the Army's stated corrective actions were being adhered 
to, we selected 10 requirements for detailed review. Our analysis found 
that the LMP documentation in support of the requirements management 
practices was generally in compliance with the stated process. More 
specifically, the requirements reviewed were generally consistent 
between the documents and appeared to contain the necessary specificity 
to reduce requirement-related defects to acceptable levels. Our 
observations were consistent with the conclusions reached by a 
subsequent review performed by the LMP verification and validation 
contractor, who examined the same documentation that we analyzed. 

Concerns regarding LMP Testing Processes Remain: 

Our review of the stated testing processes found that many of the 
attributes associated with a disciplined testing process were present. 
However, based upon our review and analysis of the 10 selected 
corrective actions, we still have concerns that the LMP testing process 
may not be adequate since we identified specific weaknesses in the 10 
corrective actions reviewed. System testing is a critical process 
utilized by disciplined organizations and improves an entity's 
confidence that the system will satisfy the requirements of the end 
user and operate as intended. The stated testing process included the 
following: 

* Documenting scenario testing. A scenario is a business process that 
typically consists of several events. For example, a scenario could be 
developed for small purchases, which would include events such as (1) 
initiating a purchase request, (2) approving the request, (3) 
obligating the necessary funds, (4) ordering the item, (5) receiving 
the item, and (6) paying for the item. One benefit of testing scenarios 
is that it helps to ensure that information is accurately passed from 
one process to another. For instance, the denial of a requisition 
prevents the purchase of the item. 

* Linking requirements to the test cases. Linking requirements and test 
cases helps ensure that either (1) all requirements are adequately 
tested or (2) the risks associated with not testing a requirement can 
be properly evaluated. Because it is not economically feasible to 
develop a testing program that can find every defect, it is critical 
that an approach be developed to ensure that testing resources are 
focused on the areas of highest risk. Accordingly, linking the 
requirements makes it easier to determine which combinations of items 
are not tested, thereby enabling the Army to evaluate the risks of not 
conducting those tests. 

* Regression testing. Regression testing is a process that helps ensure 
that changes made to the system have not adversely affected 
functionality that was working prior to a change being made to the 
system. In essence, it is a process that retests the entire system to 
ensure that the problem fixed did not have an adverse effect on other 
functions within the system. 

While the processes described above are an improvement over what we 
have seen in the past, we continue to identify weaknesses in the 
testing process. For example, none of the test scripts provided 
adequate information that linked the items in the test script to a 
specific requirement. This linking is commonly referred to as 
traceability and is characteristic of a disciplined testing process. 
The test scripts reviewed contained headings for information on the 
scenario, key data requirements, expected results, actual results, and 
whether the test was considered successful. However, the actual test 
script failed to contain the level of specificity that clearly 
delineated how a specific requirement identified in the requirement 
document(s) and the associated requirement(s) in that document were 
being tested. As a result, it is virtually impossible to ascertain if 
the requirement was properly tested. For example, several requirements 
defined rules that were expected to be implemented. Without linking a 
given requirement to the tests designed to exercise that requirement, 
it was impossible to obtain reasonable assurance that (1) all the 
requirements had been tested and (2) the test provided adequate 
coverage for each requirement. 

A review of the test script for one corrective action showed that the 
identified expected results were "output of report should show open 
accounts receivable items for customers in the range noted." Our 
analysis of the documentation disclosed that the test scripts only 
tested to make sure that a report could be printed for (1) a specific 
customer, (2) a consecutive range of customers (e.g., customers 1 
through 10), (3) federal and nonfederal customers (the exact condition 
tested was not specified since the user was only required to pick one 
of the conditions), and (4) receivables associated with a specified 
general ledger account. While this testing addressed the requirements 
that the report be printed using these conditions, the scripts did not 
provide evidence that other requirements were tested and the data 
presented were valid. For example, the report was expected to put 
receivables into 11 different age categories (for example 0 to 30 days, 
31 to 60 days, etc.); however, no evidence was provided to show that 
the balances shown on these reports (1) were consistent with known 
results and (2) the accounts selected provided adequate coverage of the 
ranges specified. The latter is especially important since two of the 
ranges overlap. LMP and CSC officials agreed that the documentation did 
not always provide the level of detail that was needed to (1) document 
which requirements contained in applicable requirement documents were 
being tested and (2) ensure that all of the specific tests that were 
required to provide reasonable assurance that the application worked as 
expected were completed. 

Furthermore, we were informed that the system testing was conducted by 
the developers and subject matter experts. Using developers and subject 
matter experts, and depending on those individuals' comprehensive body 
of knowledge as the foundation for a testing effort, carries 
significant risks since this is not a recognized best practice. A basic 
testing principle is that a developer should not test his or her own 
work, nor should a development organization test its own work. These 
testing principles are based on the concept that testing is the process 
of executing a program with the intent of finding errors. Testing is 
normally considered a "destructive" process while the development 
activities are normally considered a "constructive" progress. It is 
very difficult for a developer to change the perspective required to 
successfully develop a program into the perspective necessary to 
adequately test a process.[Footnote 59] This does not mean that the 
developers are not involved in the testing process since a best 
practice is to have them responsible for testing the code they develop 
to ensure that it is ready for the next stage of testing.[Footnote 60] 

Further, as noted in our May 2004 report,[Footnote 61] our analysis of 
LMP's December 2003 and January 2004 project status reports identified 
numerous instances in which the Army continued to experience problems 
with the accuracy of data related to budgeting; workload planning and 
forecasting and depot maintenance operations; and accounting records 
such as customer orders, purchase orders and requisitions, obligations, 
and disbursements. DFAS and Army officials acknowledged that these 
problems were attributable to relying on subject matter experts to 
develop tests for their respective functional areas, and not performing 
end-to-end testing across the various functional areas. 

After discussing our concerns with the LMP program office and CSC 
officials in February 2007, the LMP program office requested the 
verification and validation (V&V) contractor to perform an assessment 
of the LMP testing process. According to information provided by the 
LMP program office and the V&V contractor, in March 2007, the review 
found that the LMP testing process was adequately defined in the 
planning documentation and noted that the LMP testing philosophy was 
governed by the concepts that (1) the change should operate as 
intended, (2) the change should not be harmful to the existing 
functional environment, (3) testing should be practical, and (4) the 
testing efforts should be cost efficient. The V&V contractor analysis 
of the LMP testing process also found that the testing documentation-- 
which was the same documentation that we analyzed did not support a 
clear understanding of the (1) requirement being tested and (2) tests 
used to determine whether a requirement was adequately implemented. 
According to the V&V contractor, these key areas need to be addressed. 
The V&V contractor assessment substantiates our analysis of the LMP 
testing process. 

Improved Testing Process Planned: 

LMP program office officials and the V&V contractor agreed that the 
testing process currently being utilized could be improved. They also 
noted that the LMP program had begun implementing an initiative to 
improve the testing program that should address the weaknesses we 
identified. The following outlines two key components of LMP's improved 
testing efforts. 

* Establishment of an independent test group. This is a critical step 
and, if effectively implemented, can serve as a strong foundation for 
building an effective testing process. An independent testing group is 
a best practice. 

* Development and implementation of an automated testing program. CSC 
is developing an automated testing process which was expected to reduce 
the testing burden and eliminate user errors in the testing process. 
For example, the user may not detect that the application did not 
provide the expected result or record an error when the application did 
operate as expected. These are commonly referred to as testing defects 
and are to be expected when manual testing activities are conducted. 

LMP program officials are of the opinion that these two actions will 
help address the problems of adequately linking the requirements to the 
testing and documenting the testing that is actually performed. These 
actions, if effectively implemented, are a significant step forward and 
can help reduce the risks associated with testing to acceptable levels. 
However, the effective implementation will have to overcome a number of 
past problems, including ensuring that the tests provide adequate 
coverage. While developing automated tests can improve the reliability 
of the testing efforts and help facilitate an effective regression 
testing program, the key is whether the automated tests are testing the 
correct items. In our discussions with CSC, we were informed that the 
existing test scripts were the basis for developing the new automated 
test. Whether this approach will provide an effective testing process 
is yet to be seen. As we noted above, the test scripts that will be 
used as the basis for developing the automated test scripts do not 
provide the necessary link to the requirements that are being tested or 
provide the detailed information necessary to write the detailed test 
scripts called for in the automated process. 

[End of section] 

Appendix V: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

McCoy Williams, (202) 512-9095 or williamsm1@gao.gov: 

Keith A. Rhodes, (202) 512-6412 or rhodesk@gao.gov: 

William M. Solis, (202) 512-8365 or solisw@gao.gov: 

Acknowledgments: 

In addition to the above contacts, the following individuals made key 
contributions to this report: J. Christopher Martin, Senior-Level 
Technologist; Darby Smith, Assistant Director; Evelyn Logue, Assistant 
Director; F. Abe Dymond, Assistant General Counsel; Beatrice Alff; 
Sunny Chang; Harold Brumm, Jr; Francine DelVecchio; K. Eric Essig; 
Jason Kelly; Jason Kirwan; Alyson Mahan; and Debra Rucker. 

FOOTNOTES 

[1] Federal Accounting Standards and the Joint Financial Management 
Improvement Program (JFMIP) define inventory, supplies, and materials 
as consisting of three subclassifications: (1) inventory--tangible 
personal property that is (a) held for sale, (b) in the process of 
production for sale, or (c) to be consumed in the production of goods 
for sale or in the provision of services for a fee; (2) operating 
materials and supplies--tangible personal property to be consumed in 
normal operations; and (3) stockpile materials--strategic and critical 
materials held due to statutory requirements for use in national 
defense, conservation, or national emergencies. Property, plant, and 
equipment is defined as tangible assets that have an estimated useful 
life of 2 or more years, are not intended for sale in the ordinary 
course of business, and are intended to be used or available for use by 
the entity. 

[2] The reported amounts are net of allowances and depreciation, as 
applicable. 

[3] GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.: 
January 2007). 

[4] The eight specific DOD high-risk areas are (1) business 
transformation, (2) business systems modernization, (3) contract 
management, (4) financial management, (5) personnel security clearance 
program, (6) supply chain management, (7) support infrastructure 
management, and (8) weapon systems acquisition. 

[5] The seven high-risk areas that DOD shares responsibility for are 
(1) disability programs, (2) information sharing for homeland security, 
(3) information systems and critical infrastructures, (4) interagency 
contracting, (5) human capital, (6) real property, and (7) technologies 
critical to national security interests. 

[6] GAO, DOD Business Systems Modernization: Billions Continue to Be 
Invested with Inadequate Management Oversight and Accountability, GAO-
04-615 (Washington, D.C.: May 27, 2004) and Army Depot Maintenance: 
Ineffective Oversight of Depot Maintenance Operations and System 
Implementation Efforts, GAO-05-441 (Washington, D.C.: June 30, 2005). 

[7] As of September 30, 2006, the Army reported net inventory and 
related net property valued at about $39 billion for the Army general 
fund and about $18 billion for the Army working capital fund. For the 
same period, Army also reported net PP&E valued at over $84 billion for 
the Army general fund and over $1 billion for the Army working capital 
fund. 

[8] Field/Tactical refers to Army units that are deployable to 
locations around the world such as Iraq or Afghanistan. 

[9] Ronald W. Reagan National Defense Authorization Act for Fiscal Year 
2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-56 (Oct. 28, 
2004) (codified, in part, at 10 U.S.C. §§ 186, 2222). 

[10] An enterprise architecture is a blueprint for organizational 
change defined in models that describe (in both business and technology 
terms) how the entity operates today and how it intends to operate in 
the future; it also includes a plan for transitioning to this future 
state. 

[11] GAO, Enterprise Architecture: Leadership Remains Key to 
Establishing and Leveraging Architectures for Organizational 
Transformation, GAO-06-831 (Washington, D.C.: Aug. 14, 2006). 

[12] GAO, DOD Business Systems Modernization: Progress Continues to Be 
Made in Establishing Corporate Management Controls, but Further Steps 
Are Needed, GAO-07-733 (Washington, D.C.: May 14, 2007). 

[13] See, for example, GAO, Information Technology: FBI Is Taking Steps 
to Develop an Enterprise Architecture, but Much Remains to Be 
Accomplished, GAO-05-363 (Washington, D.C.: Sept. 9, 2005); Homeland 
Security: Efforts Under Way to Develop Enterprise Architecture, but 
Much Work Remains, GAO-04-777 (Washington, D.C.: Aug. 6, 2004); DOD 
Business Systems Modernization: Limited Progress in Development of 
Business Enterprise Architecture and Oversight of Information 
Technology Investments, GAO-04-731R (Washington, D.C.: May 17, 2004); 
Information Technology: Architecture Needed to Guide NASA's Financial 
Management Modernization, GAO-04-43 (Washington, D.C.: Nov. 21, 2003); 
and DOD Business Systems Modernization: Important Progress Made to 
Develop Business Enterprise Architecture, but Much Work Remains, GAO-03-
1018 (Washington, D.C.: Sept. 19, 2003). 

[14] Pub. L. No. 104-106, div. E, tit. LI, §§ 5122, 5125, 110 Stat. 
186, 683, 685 (Feb. 10, 1996) (codified at 40 U.S.C. §§ 11312, 11315). 

[15] OMB Circular No. A-130, Management of Federal Information 
Resources, § 8(b) (Nov. 28, 2000). 

[16] An ERP solution is an automated system using commercial off-the- 
shelf (COTS) software consisting of multiple, integrated functional 
modules that perform a variety of business-related tasks such as 
payroll, general ledger accounting, and supply chain management. 

[17] DOD identified the following four fundamental changes in logistics 
practices that are needed to transform its logistics operations: (1) 
customer wait time, (2) time-definite delivery, (3) TAV, and (4) Web- 
based, shared data environment. 

[18] GAO, Defense Inventory: DOD Could Improve Total Asset Visibility 
Initiative with Results Act Framework, GAO/NSIAD-99-40 (Washington, 
D.C.: Apr. 12, 1999). 

[19] Pub. L. No. 104-208, div. A, §101(f), title VIII, 110 Stat. 3009, 
3009-389 (Sept. 30, 1996). 

[20] Pub. L. No. 101-576, 104 Stat. 2838 (Nov. 15, 1990). 

[21] FFMIA requires the 24 Chief Financial Officers Act departments and 
agencies to implement and maintain financial management systems that 
comply substantially with (1) federal financial management systems 
requirements, (2) applicable federal accounting standards, and (3) the 
U.S. Government Standard General Ledger at the transaction level. 

[22] The name of the command changed from Communications and 
Electronics Command to the Communications-Electronics Life Cycle 
Management Command in February 2005. 

[23] Full operational capability means that the system has been 
deployed to all intended locations. 

[24] The contractor has submitted claims totaling $850 million to the 
Army. According to the LMP Program Office, these claims are being 
reviewed by the Army contracting officer, who is expected to make a 
decision by June 2007. 

[25] The Army Corps of Engineers will continue to use its existing 
financial system--Corps of Engineers Financial Management System. 

[26] There are four tiers of business systems. Tier 1 systems include 
all large, expensive system programs classified as a "major automated 
information system" (MAIS) or a "major defense acquisition program" 
(MDAP) and subject to the most extensive statutory and regulatory 
reporting requirements. Tier 2 systems include those with modernization 
efforts of $10 million or greater but that are not designated as MAIS 
or MDAP or programs that have been designated as IRB interest programs 
because of their effect on DOD transformation objectives. Tier 3 
systems include those with modernization efforts that have anticipated 
costs greater than $1 million but less than $10 million. Tier 4 systems 
are those with modernization efforts that have anticipated costs of up 
to $1 million. 

[27] Certification reviews apply to business system development/ 
modernization programs with a total cost of over $1 million. This 
certification review focuses on program alignment with the business 
enterprise architecture and must be completed before components 
obligate funds for programs. The annual review applies to all business 
programs. The focus of the annual review is to determine whether the 
system development effort is in compliance with the business enterprise 
architecture, meeting its milestones, and addressing IRB certification 
conditions. 

[28] DOD defines a mission area as a defined area of responsibility 
with functions and processes that contribute to mission accomplishment. 
The Army maps its mission areas to the four Global Information Grid 
Enterprise Services Mission Areas delineated by DOD, including: 
Warfighting, Business, Enterprise Information Environment, and the DOD 
portion of National Intelligence. The Business Mission Area includes 
six domains: (1) acquisition, (2) financial management, (3) human 
resources management, (4) logistics, (5) installations and environment, 
and (6) civil works. 

[29] GAO-04-615. 

[30] GAO-05-441. 

[31] GAO, Defense Inventory: Improvements Needed in DOD's 
Implementation of Its Long-Term Strategy for Total Asset Visibility of 
Its Inventory, GAO-05-15 (Washington, D.C.: Dec. 6, 2004). 

[32] DOD, Department of Defense Directive Number 5000.1 (May 12, 2003); 
Department of Defense Architecture Framework, Version 1.0, Volume 1 
(February 2004). 

[33] Pub. L. No. 104-106, div. E, tit. LI, §§ 5122, 5125; E-Government 
Act of 2002, Pub. L. No. 107-347 § 101, 116 Stat. 2899, 2901-2910 (Dec. 
17, 2002) codified at 44 U.S.C. §§ 3601-06); OMB Circular No. A-130, § 
8(b); GAO, Information Technology: A Framework for Assessing and 
Improving Enterprise Architecture Management (Version 1.1), GAO-03-584G 
(Washington, D.C.: April 2003); Chief Information Officer Council, A 
Practical Guide to Federal Enterprise Architecture, Version 1.0 
(February 2001); and Institute of Electrical and Electronics Engineers, 
Standard for Recommended Practice for Architectural Description of 
Software-Intensive Systems 1471-2000 (Sept. 21, 2000). 

[34] GAO-06-831. 

[35] The five DOD architecture programs that were included in our audit 
were the Departments of the Army, Navy, and Air Force, and the DOD 
business enterprise architecture and the DOD global information grid. 

[36] GAO, Information Technology: A Framework for Assessing and 
Improving Enterprise Architecture Management (Version 1.1), GAO-03-584G 
(Washington, D.C.: April 2003). 

[37] GAO-07-733. 

[38] GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.: 
January 2007). 

[39] The other two focus areas related to supply chain management are 
requirements forecasting and materiel distribution. 

[40] Thomas F. Wallace and Michael H. Kremzar, ERP: Making It Happen, 
(New York, N.Y.: John Wiley & Sons, Inc. 2001). 

[41] An economic analysis of Army's LMP initiative is not required 
because it is considered a procurement of a service as opposed to an 
actual system, and therefore is not subject to the criteria established 
by Department of Defense Directive 5000.1 and Department of Defense 
Instruction 5000.2. Subsequent to the draft of this report being 
submitted to DOD for comment, we were informed that the Office of the 
Deputy Assistant Secretary of the Army for Cost and Economics had 
determined that GCSS-Army is economically viable. We have not evaluated 
the results of that analysis. 

[42] GAO, Business Systems Modernization: DOD Needs to Fully Define 
Policies and Procedures for Institutionally Managing Investments, GAO-
07-538 (Washington, D.C.: May 11, 2007). 

[43] U.S. Army, Army Knowledge Management (AKM) Guidance Memorandum, 
Capabilities-Based Information Technology (IT) Portfolio Governance 
Implementing Guidance (Jan. 5, 2006). 

[44] According to the Institute of Electrical and Electronics 
Engineers, verification and validation processes for projects such as 
the Army ERPs can be used to determine whether (1) the products of a 
given activity conform to the requirements of that activity and (2) the 
software satisfies its intended use and user needs. This determination 
may include analyzing, evaluating, reviewing, inspecting, assessing, 
and testing software products and processes. The verification and 
validation processes should assess the software in the context of the 
system, including the operational environment, hardware, interfacing 
software, operators, and users. 

[45] OMB Circular No. A-94, Guidelines and Discount Rates for Benefit- 
Cost Analysis of Federal Programs (Oct. 29, 1992); Department of 
Defense Instruction 7041.3, Economic Analysis for Decisionmaking (Nov. 
7, 1995); Department of the Army, U.S. Army Cost and Economic Analysis 
Center, Economic Analysis Manual (February 2001). 

[46] OMB Circular No. A-94, § 6(a)(4). 

[47] OMB Circular No. A-94, § 9; Department of Defense Instruction 
7041.3, § E3.2.2. 

[48] GAO-04-615 and GAO-05-441. 

[49] Ronald W. Reagan National Defense Authorization Act for Fiscal 
Year 2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-56 (Oct. 
28, 2004) (codified, in part, at 10 U.S.C. §§ 186, 2222). 

[50] GAO, DOD Business Systems Modernization: Billions Continue to Be 
Invested with Inadequate Management Oversight and Accountability, GAO-
04-615 (Washington, D.C.: May 27, 2004) and Army Depot Maintenance: 
Ineffective Oversight of Depot Maintenance Operations and System 
Implementation Efforts, GAO-05-441 (Washington, D.C.: June 30, 2005). 

[51] We reviewed the 1307 reports for March 2004, September 2004, March 
2005, September 2005, March 2006, September 2006, December 2006, and 
February 2007. 

[52] Pub. L. No. 104-208, div. A, §101(F), title VIII, 110 Stat. 3009, 
3009-389 (Sept. 30, 1996). FFMIA requires the 24 Chief Financial 
Officers Act departments and agencies to implement and maintain 
financial management systems that comply substantially with (1) federal 
financial management system requirements, (2) applicable federal 
accounting standards, and (3) the U.S. Government Standard General 
Ledger at the transaction level. 

[53] DOD Inspector General, Controls Over the Prevalidation of DOD 
Commercial Payments, Report Number D2007-065 (Arlington, Va.: Mar. 2, 
2007). 

[54] Pub. L. No. 103-335, § 8137, 108 Stat. 2599, 2654-55 (Sept. 30, 
1994). 

[55] Acceptable levels refer to the fact that any systems acquisition 
effort will have risks and will suffer the adverse consequences 
associated with defects in the processes. However, effective 
implementation of disciplined processes reduces the possibility of the 
potential risks actually occurring and prevents significant defects 
from materially affecting the cost, timeliness, and performance of the 
project. 

[56] GAO, Army Depot Maintenance: Ineffective Oversight of Depot 
Maintenance Operations and System Implementation Efforts, GAO-05-441 
(Washington, D.C.: June 30, 2005). 

[57] We reviewed the 1307 reports for March 2004, September 2004, March 
2005, September 2005, March 2006, September 2006, December 2006, and 
February 2007. 

[58] GAO, DOD Business Systems Modernization: Billions Continue to Be 
Invested with Inadequate Management Oversight and Accountability, GAO-
04-615 (Washington, D.C.: May 27, 2004) and GAO-05-441. 

[59] Glenford J. Myers, The Art of Software Testing (New York, N.Y.: 
John Wiley & Sons, Inc., 1979). 

[60] Rex Black, Critical Testing Processes: Plan, Prepare, Perform, 
Perfect (Boston, Mass.: Addison-Wesley, 2004). 

[61] GAO-04-615. 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm: 

E-mail: fraudnet@gao.gov: 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400: 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800: 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: