This is the accessible text file for GAO report number GAO-07-1139R 
entitled 'Military Personnel: The Navy Has Not Provided Adequate 
Justification For Its Decision to Invest in MCTFS' which was released 
on July 26, 2007. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

July 25, 2007: 

Congressional Committees: 

Subject: Military Personnel: The Navy Has Not Provided Adequate 
Justification For Its Decision to Invest in MCTFS: 

In June 2006, the Navy completed a feasibility assessment of both the 
Marine Corps Total Force System (MCTFS)--the Marine Corps' integrated 
personnel and payroll system--and the Defense Integrated Military Human 
Resources System (DIMHRS)--the department's solution to provide a 
joint, integrated, standardized military personnel and pay system 
across all military components. Based on the results of this 
assessment, the Navy expressed a preference to deploy MCTFS instead of 
DIMHRS. In August 2006, the Defense Business Systems Management 
Committee (DBSMC)[Footnote 1] made a business decision to accept the 
Navy's plan to proceed with the development and implementation of MCTFS 
instead of DIMHRS. However, the Department of Defense (DOD) had already 
committed to deploy DIMHRS . According to the department, as of 
September 2006, it had spent over $668 million[Footnote 2] on the 
DIMHRS program. 

The John Warner National Defense Authorization Act for Fiscal Year 
2007, Pub. L. No. 109-364, 324 (2006) directed the Secretary of the 
Navy to prepare a report about MCTFS, including (1) an analysis of 
alternatives to MCTFS, including a comparison between the costs of 
deploying and operating MCTFS within the Navy and the cost of including 
the Navy in DIMHRS; (2) a business case analysis of the costs and 
benefits to both the Navy and DOD of the alternatives to MCTFS 
considered in the first objective; and (3) an analysis of the 
compatibility of MCTFS with the department's business enterprise 
architecture.[Footnote 3] The Navy concluded in its report--which we 
received on April 25, 2007--that (1) either MCTFS or DIMHRS could 
provide basic personnel and pay capability for the Navy uniformed force 
at approximately equivalent cost; (2) the DIMHRS alternative has 
substantially higher risks on cost, schedule, and function because 
MCTFS is already operational; and (3) MCTFS is fully compatible and 
compliant with the department's business enterprise architecture. The 
Act directs the DBSMC to determine--not sooner than 120 days after 
April 25, 2007--whether the deployment of MCTFS by the Navy is in the 
best interest of DOD. 

The Act directed us to review the Navy's report and provide a written 
assessment to the congressional defense committees and the Chairman of 
the DBSMC within 90 days of our receipt of the Navy's report. The DBSMC 
is to consider our report in making its decision. Specifically, we 
determined whether the Navy in its congressionally mandated report has 
justified its decision to invest in MCTFS. As part of our review, we 
also determined whether the department's business enterprise 
architecture was sufficient to guide and constrain the acquisition of 
MCTFS. 

Scope and Methodology: 

To perform our assessment, we reviewed the Navy report,[Footnote 4] 
which summarized the results of the analysis of alternatives, the 
business case analysis of the costs and benefits of the alternatives, 
and the analysis of the compatibility of MCTFS with the department's 
enterprise architecture. Using our Cost Assessment Guide and relevant 
Office of Management and Budget (OMB) criteria,[Footnote 5] we 
evaluated the analysis of alternatives and the business case analysis. 
According to these criteria, the decision to invest in any system 
should be based on reliable analyses of estimated costs and expected 
benefits over the life of the system. Best practices state that 
agencies should prepare a business case analysis--a comparative 
analysis that presents facts and supporting details about competing 
alternatives (i.e., analysis of alternatives). A business case analysis 
should consider all life-cycle costs and nonquantifiable as well as 
quantifiable benefits. It should consider all possible alternatives and 
should not be developed solely to support a predetermined solution. A 
business case analysis should be rigorous enough that independent 
auditors can review it and clearly understand why a particular 
alternative was chosen. We evaluated the cost estimates that the Navy 
used in the analysis of alternatives and the business case analysis to 
determine whether these estimates were comprehensive, accurate, well- 
documented, and credible. Our Cost Assessment Guide considers an 
estimate to be: (1) 'comprehensive' if it includes all relevant cost 
elements; (2) 'accurate' if it is unbiased and neither overly 
conservative nor overly optimistic and is based on an assessment of the 
costs most likely to be incurred; (3) 'well-documented' if it can be 
traced back to the documentation, which should include the source of 
the data, detailed calculations and results, and explanations of why a 
particular method or reference was chosen; and (4) 'credible' if 
limitations of the data and assumptions are documented. 

Using applicable guidance,[Footnote 6] we evaluated the efforts taken 
by the Navy to show the extent to which MCTFS is compatible with the 
architecture. This guidance states that agencies should develop and use 
enterprise architectures to guide and constrain their investment 
decisions. We obtained and analyzed the documents provided--such as 
MCTFS architecture products, the results of the Navy's analysis of 
MCTFS using the Architecture Compliance and Requirements Traceability 
Tool, and a Gartner, Inc. report[Footnote 7]--the technology standards, 
and version 4.0 of DOD's business enterprise architecture. 

We interviewed officials to discuss the results of our work and to 
obtain explanations for the discrepancies we identified. Specifically, 
we interviewed officials from the following: 

* Several organizations within the Department of the Navy, including 
the Office of the Chief of Naval Operations; the Assistant Secretary of 
the Navy Financial Management and Comptroller; the Manpower and Reserve 
Affairs, U. S. Marine Corps; and the Naval Center for Cost Analysis. 
These officials were responsible for developing the analysis of 
alternatives, the business case analysis, and the analysis of the 
compatibility of MCTFS with the department's business enterprise 
architecture. 

* The Enterprise Program Management Office-DIMHRS, which is responsible 
for acquiring and implementing DIMHRS, including the DIMHRS Enterprise 
Deputy Program Manager. 

* The Personnel and Readiness Information Management Office, which is 
responsible for developing the DIMHRS architecture, including staff 
supporting the investment management process. 

* The Business Transformation Agency, which is responsible for 
maintaining and updating the department's business enterprise 
architecture, including the chief architect for the Business Mission 
Area. 

Our work was conducted from April through July 2007 in accordance with 
generally accepted government auditing standards. 

Summary: 

In its report, the Navy has not adequately justified its decision to 
invest in MCTFS. Specifically, the analysis of alternatives and the 
business case analysis of both MCTFS and DIMHRS were unreliable, 
because the cost and benefit estimates used in these analyses were not 
sufficiently comprehensive, accurate, documented, or credible. The 
Navy's analysis shows that MCTFS is sufficiently compatible with DOD's 
evolving business enterprise architecture to allow the Navy to begin 
acquisition.[Footnote 8] However, we found that the architecture lacks 
sufficient content to be used effectively to guide and constrain the 
acquisition of MCTFS. DOD officials from the Navy, the Marine Corps, 
and the Business Transformation Agency provided oral comments on the 
facts presented in this report and we have incorporated these comments 
as appropriate. 

Analysis of Alternatives and Business Case Analysis Were Unreliable: 

The Navy's report included an analysis of alternatives and a business 
case analysis of MCTFS and DIMHRS; however, these analyses were 
unreliable because the cost and benefit estimates used for these 
analyses were not sufficiently comprehensive, accurate, documented, or 
credible. First, the cost estimates were not comprehensive, because 
they did not include all possible alternatives. For example, the Navy's 
report included a third alternative that involves deploying MCTFS 
initially and then deploying DIMHRS after the Army and Air Force have 
successfully implemented it. However, the analysis of alternatives and 
the business case analysis did not include cost estimates for this 
alternative. In March 2007, DBSMC tasked the Navy to prepare the cost 
and benefit estimates for this alternative, after which Navy officials 
said that they discussed it with the Secretary of the Navy. These 
estimates were based on those that had been used for both the MCTFS and 
DIMHRS alternatives and we had concluded that these estimates were 
unreliable. Second, the cost estimates were not accurate, because the 
Navy excluded the third alternative, which was more costly than the 
MCTFS and DIMHRS alternatives. This exclusion skewed the results of the 
analyses by showing that the MCTFS alternative was the least costly 
solution. Third, the cost estimates were not well-documented, because 
source data typically was not provided and the calculations performed 
for several cost elements were not explained. Finally, the cost 
estimates were not credible, because there were neither cross-checks of 
these estimates nor a current independent cost estimate; both of these 
are necessary in order to determine whether a different estimating 
method would have produced similar results. 

Moreover, the analyses based on these cost estimates did not take into 
account the following risks: (1) the potential impact of moving MCTFS 
development operations from Kansas City to Indianapolis as part of the 
Base Realignment and Closure 2005 actions and (2) the potential impact 
of the contractor discontinuing its support of DIMHRS after 2013, as 
planned. A Navy official responsible for MCTFS development operations 
stated his intent to delay the closing of the Kansas City site as long 
as possible in order to mitigate its impact on MCTFS. However, this 
official did not provide documentation of his authority to carry out 
this intent or of the Navy's plans for doing so within the time 
limitations established for the Base Realignment and Closure 2005 
actions. After the Navy issued its report, the contractor changed its 
policy and stated its intent to support DIMHRS indefinitely; however, 
this support will be limited. For example, 'sustaining support' does 
not include key services, such as new updates, product fixes, security 
alerts, critical patch and regulatory updates, and certification of 
DIMHRS with new products developed by the contractor. It is unclear 
what the costs, if any, will be for this sustaining support. However, 
the contractor's change in policy is not a binding legal agreement; the 
contractor could later decide to discontinue its support of DIMHRS or 
increase the cost prohibitively for sustaining support. Since the 
accuracy of the cost estimates is in question, the benefits or cost 
savings to be realized are also in question. 

According to best practices,[Footnote 9] the decision to invest in any 
system should be based on reliable analyses of estimated costs and 
expected benefits over the life of the system. Best practices state 
that agencies should prepare a business case analysis--a comparative 
analysis that presents facts and supporting details about competing 
alternatives. A business case analysis considers all life-cycle costs 
and quantifiable and nonquantifiable benefits. It should consider all 
possible alternatives and should not be developed solely to support a 
predetermined solution. A business case analysis should be rigorous 
enough that independent auditors can review it and clearly understand 
why a particular alternative was chosen. 

Navy officials said that the report did not include the strategic 
alternative showing the Navy's intent to ultimately transition to 
DIMHRS because the mandate did not require them to do so. However, best 
practices require that an agency consider all possible alternatives 
when preparing a business case analysis, so this alternative should 
have been included in the report. In November 2006, the Navy had tasked 
the Naval Center for Cost Analysis to develop cost estimates for the 
business case analysis to be included in the report by February 2007. 
Naval Center for Cost Analysis officials said that this deadline give 
them adequate time to meet best practices for developing comprehensive, 
accurate, well-documented, and credible cost estimates. These officials 
stated, and we agree, that it typically takes 6 months to develop 
reliable cost estimates. DIMHRS program officials stated that the cost 
and benefit estimates the Navy used in its analyses were outdated and 
did not reflect the current DIMHRS program. They also stated that they 
had not been contacted by the Naval Center for Cost Analysis to provide 
updated information for the Navy report. However, officials from the 
Naval Center for Cost Analysis provided documentation showing that they 
had in fact contacted DIMHRS program officials and that these officials 
had refused to answer programmatic and technical questions about the 
DIMHRS program. Because the analyses were unreliable, Congress and the 
DBSMC cannot be assured that the most cost-effective solution will be 
selected. 

MCTFS Is Compatible with the Business Enterprise Architecture, but the 
Architecture Is Insufficient to Guide Investment Decisions: 

The Navy's analysis of the compatibility of MCTFS with the department's 
business enterprise architecture was sufficient for the Navy to begin 
acquisition of the system. However, we found that the architecture 
lacks sufficient content to be used effectively to guide and constrain 
investments for systems such as MCTFS. As a result of this lack of 
architectural constraint on systems, the Navy determined the 
architecture requirements MCTFS will need to meet to fit within the 
department's vision as outlined in the evolving architecture. According 
to best practices and DOD guidance,[Footnote 10] agencies should 
develop and use enterprise architectures to guide and constrain 
investment decisions. Senior DOD and Navy officials acknowledged that 
the architecture cannot be used at this time to effectively guide and 
constrain component-level system investments such as MCTFS. As a 
result, there is a risk that the department will not be able to achieve 
its vision to deploy standard systems such as DIMHRS. By deploying 
standard systems, DOD may be able to address the weaknesses in its 
current system environment, which is comprised of numerous systems that 
are duplicative in functionality, are not interoperable, are 
unnecessarily costly to maintain and interface, and do not optimize 
mission performance and accountability. 

Conclusion: 

Our assessment of the Navy's report shows that the analyses are based 
on flawed estimates and an insufficient architecture. If the DBSMC 
relies solely on the Navy's report in deciding whether or not to deploy 
MCTFS, it risks recommending an investment decision that would not be 
in the best interest of DOD. As it deliberates, the DBSMC should 
consider the issues we have identified. 

We are sending copies of this report to selected congressional 
committees and the Chairman of the Defense Business Systems Management 
Committee. We will also make copies available to others upon request. 
In addition, the report will be available at no charge on GAO's Web 
site at http://www.gao.gov. 

If your offices or staffs have any questions concerning this report, 
please contact me at (202) 512-3604 or by e-mail at farrellb@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who 
made major contributions to the report are listed in enclosure I. 

Signed by: 

Brenda S. Farrell: 
Director, Defense Capabilities and Management: 

Enclosure - 1: 

List of Committees: 

The Honorable Daniel Inouye: 
Chairman: 
The Honorable Ted Stevens: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
United States Senate: 

The Honorable Carl Levin: 
Chairman: 
The Honorable John McCain: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

The Honorable John P. Murtha: 
Chairman: 
The Honorable C.W. Bill Young: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
House of Representatives: 

The Honorable Ike Skelton: 
Chairman: 
The Honorable Duncan Hunter: 
Ranking Member: 
Committee on Armed Services: 
House of Representatives: 

[End of section] 

Enclosure I: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Brenda S. Farrell (202) 512-3604 or farrellb@gao.gov: 

Acknowledgments: 

In addition to the contact above, Cynthia Jackson, Assistant Director; 
Brian Bothwell; Harold J. Brumm, Jr; Jennifer K. Echard; Joanne 
Landesman; Sonya Phillips; Karen A. Richey; Matthew S. Spiers; Randolph 
Tekeley; Joseph J. Watkins; and Angela D. Watson made key contributions 
to this report. 

(351049): 

FOOTNOTES 

[1] The Deputy Secretary of Defense established the DBSMC, which 
oversees DOD's business transformation, and tasked it with 
responsibility for overall departmental transformation pursuant to 10 
U.S.C.  186. 

[2] The cost figure includes costs incurred by the Enterprise Program 
Management Office-DIMHRS, the Army, the Air Force, the U. S. Marine 
Corps, and the Defense Finance and Accounting Office; it does not 
include costs incurred by the Navy since program inception. 

[3] An enterprise architecture provides a clear and comprehensive 
picture of an entity, whether it is an organization (e.g., a federal 
department) or a functional or mission area that cuts across more than 
one organization (e.g., human resources). It is an essential tool for 
effectively and efficiently engineering business processes and for 
implementing and evolving systems that support the business processes. 

[4] Department of the Navy, Report from the Secretary of the Navy to 
the Congressional Defense Committees and the Comptroller General on the 
Marine Corps Total Force System (MCTFS), April 2007. 

[5] GAO, Cost Assessment Guide: Best Practices for Developing and 
Managing Program Costs, July 2007; Office of Management and Budget 
(OMB) Circular A-11, Preparation, Submission, and Execution of the 
Budget, Part 7: Planning, Budgeting, Acquisition, and Management of 
Capital Assets (June 2006); and OMB Circular A-94, Guidelines and 
Discount Rates for Benefit-Cost Analysis of Federal Programs (Revised 
Oct. 29, 1992). 

[6] Office of Management and Budget (OMB) Capital Programming Guide, 
Version 2.0 (June 2006); GAO, Information Technology: A Framework for 
Assessing and Improving Enterprise Architecture Management (Version 
1.1), GAO-03-584G (Washington, D.C.: April 2003); Department of 
Defense, DOD IT Business Systems Investment Review Process, Business 
Enterprise Architecture (BEA) Compliance Guidance, Version 1.0 (Apr. 
10, 2006); and Department of Defense, Department of Defense 
Architecture Framework, Version 1.5, Volume 1 (Apr. 23, 2007). 

[7] Gartner, Inc., A Report for U. S. Marine Corps and Defense Finance 
and Accounting Service: Marine Corps Total Force System Program 
Evaluation (June 19, 2006). 

[8] Our analysis determined the extent to which MCTFS was compatible 
with version 4.0 of the department's business enterprise architecture. 

[9] See GAO, Cost Assessment Guide: Best Practices for Developing and 
Managing Program Costs, OMB Circular A-11, and OMB Circular A-94. 

[10] Office of Management and Budget (OMB) Capital Programming Guide, 
Version 2.0 ; GAO-03-584G; Department of Defense, DOD IT Business 
Systems Investment Review Process, Business Enterprise Architecture 
(BEA) Compliance Guidance; and Department of Defense, Department of 
Defense Architecture Framework, Version 1.5, Volume 1. 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: