This is the accessible text file for GAO report number GAO-04-731R 
entitled 'DOD Business Systems Modernization: Limited Progress in 
Development of Business Enterprise Architecture and Oversight of 
Information Technology Investments' which was released on May 17, 2004.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

May 17, 2004:

Congressional Defense Committees:

Subject: DOD Business Systems Modernization: Limited Progress in 
Development of Business Enterprise Architecture and Oversight of 
Information Technology Investments:

The Department of Defense's (DOD) long-standing business systems 
problems adversely affect the economy, effectiveness, and efficiency of 
its business operations and have resulted in a lack of adequate 
transparency and appropriate accountability across all of its major 
business areas. To help the department transform its operations, we 
recommended[Footnote 1] in 2001 that DOD develop an enterprise 
architecture to guide and constrain its almost $20 billion annual 
investment in business systems and that it establish the investment 
controls needed to implement this architecture. In July 2001, DOD 
initiated a program[Footnote 2] to, among other things, develop a DOD 
business enterprise architecture (architecture). This effort is an 
essential part of the Secretary of Defense's broad initiative to 
"transform the way the department works and what it works on.":

Because DOD is one of the largest and most complex organizations in the 
world, overhauling its business operations and supporting systems 
represents a huge management challenge. In fiscal year 2003, DOD 
reported that its operations involved over $1 trillion in assets, 
nearly $1.6 trillion in liabilities, approximately 3.3 million military 
and civilian personnel, and disbursements of over $416 billion. To 
support its business operations, DOD reported that it relies on about 
2,300 business systems, including accounting, acquisition, logistics, 
and personnel systems. The department requested about $19 billionóabout 
$4.8 billion for business systems modernization and about $14 billion 
for operation and maintenance of these systemsóin fiscal year 2004.

ecognizing the importance of DOD's efforts to transform its business 
operations and systems through the use of an enterprise architecture, 
the Congress included provisions in the National Defense Authorization 
Act for Fiscal Year 2003[Footnote 3] that were aimed at developing and 
effectively implementing a well-defined architecture. Specifically, 
section 1004 of this act required that DOD (1) develop, by May 1, 2003, 
a financial management enterprise architecture[Footnote 4] and a 
transition plan for implementing the architecture that meets certain 
requirements and (2) review financial system improvements with proposed 
obligations of funds in amounts exceeding $1 million to determine if 
those system improvements meet specific conditions that are called for 
in the act. The act also directed us to assess actions that DOD has 
taken to comply with these requirements. In July[Footnote 5] and 
September 2003,[Footnote 6] we reported on DOD's actions and made a 
number of recommendations to assist DOD in its efforts to effectively 
develop and implement an architecture and to guide and constrain its 
business systems investments.

The act further requires that the Secretary of Defense submit an annual 
report not later than March 15 of each year from 2004 through 2007 to 
congressional defense committees on its progress in implementing the 
architecture, including the transition plan. Additionally, the act 
directs us to submit to congressional defense committees, within 60 
days of DOD's report submission, an assessment of DOD's actions taken 
to comply with these requirements. (See enc. I for a copy of section 
1004 of the act.) DOD submitted its first annual report on March 15, 
2004. This report is our assessment of DOD's March 15, 2004 report. As 
agreed with your offices, we determined (1) the actions DOD has taken 
to address our previous recommendations regarding the development and 
implementation of the architecture and (2) the actions DOD is taking to 
ensure its ongoing and planned investments will be consistent with its 
evolving architecture.

We performed our work from December 2003 through April 2004 in 
accordance with U.S. generally accepted government auditing standards. 
Details on our scope and methodology are in enclosure II.

Results in Brief:

Since our last review--and after 3 years of effort and over $203 
million in obligations--we have not seen any significant change in the 
content of DOD's architecture or in DOD's approach to investing 
billions of dollars annually in existing and new systems. Few actions 
have been taken to address the recommendations we made in our September 
2003[Footnote 7] report, which were aimed at improving DOD's plans for 
developing the next version of the architecture and implementing the 
institutional means for selecting and controlling both planned and 
ongoing business systems investments. To DOD's credit, it has 
established, for example, a group under the Business Management 
Modernization Program (program) steering committe[Footnote 8]eE to 
facilitate communication and coordination across the domai[Footnote 
9]ns for modernization program activities, including extending and 
evolving the architecture. However, DOD has not yet adopted key 
architecture management best practi[Footnote 10]ces and has not added 
the scope and detail to its architecture that we previously identified 
as missing. Further, DOD has not yet implemented an effective 
management structure and processes to provide adequate control and 
accountability over its $5 billion annual investment in business 
systems modernization. Additionally, the department does not have 
reasonable assurance that it is in compliance with the National Defense 
Authorization Act for Fiscal Year 2003, which requires DOD's 
Comptroller to review all system improvements with obligations 
exceeding $1 million.

Regarding architecture management best practices, DOD has not yet 
implemented key elements, such as assigning accountability and 
responsibility for directing, overseeing, and approving the 
architecture. In addition, it has not explicitly defined performance 
metrics to evaluate the architecture's quality, content, and utility. 
With respect to the architectural content, DOD's latest version of the 
architecture does not include many of the key elements of a well-
defined "As Is," "To Be," and transition plan that we previously 
reported as not being satisfied.[Footnote 11] For example, the "To Be" 
environment does not provide sufficient descriptive content related to 
future business operations and supporting technology to permit 
effective acquisition and implementation of system solutions and 
associated operational change. Similarly, DOD's verification and 
validation contractor has concluded that this latest version of the 
architecture retains most of the limitations that the initial version 
had. Moreover, DOD has not yet defined specific plans, including 
milestones, detailing how it intends to extend and evolve the 
architecture to incorporate this missing content. For example, the 
program's first objective for increment one--to enable asset 
accountability, total force visibility, and an unqualified audit 
opinion on DOD's fiscal year 2007 consolidated financial statements--is 
not supported by a DOD-wide plan of action, and the individual 
component plans for accomplishing this objective may not be linked to 
the program's activities, including the architecture. According to DOD, 
it will not have the plans on how the capabilities for increment one 
will be achieved until August 2004.

DOD has also made limited progress in addressing our recommendations 
aimed at establishing and implementing effective investment management 
processes and, therefore, continues to lack effective management 
oversight and control over ongoing business systems modernization 
investments. While DOD has recently issued a policy that assigns 
investment management responsibilities to the domains, the policy has 
not yet been implemented, and DOD has not clearly defined the roles and 
responsibilities of the domains, established common investment 
criteria, and conducted a comprehensive review of its existing business 
systems to ensure that they are consistent with the architecture. 
Further, each of the DOD components continues to receive its own 
funding and make its own parochial investment decisions. Moreover, DOD 
has not yet established and implemented an effective process for 
ensuring that system improvements with obligations exceeding $1 million 
are submitted to DOD's Comptroller for review and to determine whether 
they are consistent with the architecture, as required by the National 
Defense Authorization Act for Fiscal Year 2003. Based upon a comparison 
of limited information provided by the military services and defense 
agencies, we identified a total of $863 million in obligations for 
improvements that exceeded $1 million each but had not been submitted 
to DOD's Comptroller for review and determination.

The department acknowledges that it still has much more to do, 
including developing the architecture to a necessary level of detail, 
defining specific performance metrics, and clarifying the roles and 
responsibilities associated with managing the domains' portfolios of 
business systems and ensuring that these systems comply with the 
architecture. The limited progress that DOD has made is due, in part, 
to the lack of clearly assigned, accountable, and sustained program 
leadership and to changes in the program direction and priorities. For 
example, from May 2003 to February 2004, there was no program manager 
to identify, direct, and execute program activities. Our experience in 
reviewing other challenged architecture efforts shows that these 
efforts have suffered from limited senior management understanding of 
and commitment to an architecture and from cultural resistance to 
having and using one. Cultural resistance to change, organization 
component parochialism, and stovepiped operations are all evident at 
DOD.

Because many of our prior recommendations--for managing architecture 
development, maintenance, and implementation and for controlling 
ongoing and planned investments in business systems--remain open, we 
are not making any new recommendations in this report, but we are 
reiterating the 22 open recommendations that we made in our May 
2001,[Footnote 12] February 2003,[Footnote 13] and September 
2003[Footnote 14] reports. (Enc. III contains details on the status of 
all of our prior recommendations, including our assessment of DOD's 
actions.) It is imperative that DOD act swiftly to implement these 
recommendations. If it does not, the prognosis for this program is 
bleak, which in turn puts the department's business transformation 
efforts in jeopardy.

In written comments, which are reprinted in enclosure IV, DOD agreed 
with our assessment that the department's long-standing business 
systems problems have resulted in a lack of adequate transparency and 
appropriate accountability across all major business areas and that 
development and use of an architecture is necessary to transforming its 
business operations and supporting systems. In response to our 
characterization that progress has been limited, DOD stated that, over 
the past 3 years, progress has been slower than either GAO or DOD would 
prefer but that it has been significant, despite appearances to the 
contrary. In support of its view, DOD provided an extensive package of 
information. We considered this information, most of which we had 
previously evaluated. The willingness of DOD's leadership to tackle the 
department's decades-old problems with its business operations and 
supporting systems represents a major step forward. However, we remain 
convinced that it is fair to characterize DOD's progress in developing 
a well-defined architecture and implementing effective management 
oversight and control over its business systems as limited. As a 
result, we did not make any changes to this report.

Background:

Prior Reviews of DOD's Architecture Efforts Have Identified Challenges 
and Weaknesses:

Over the last 3 years, we have reported[Footnote 15] that one of the 
key elements to successfully meeting DOD's financial and related 
business management challenges is establishing and implementing an 
enterprise architecture, or modernization blueprint. An enterprise 
architecture provides a clear and comprehensive picture of an entity, 
whether it is an organization (e.g., federal department or agency) or a 
functional or mission area that cuts across more than one organization 
(e.g., financial management). This picture consists of snapshots of 
both the enterprise's current or "As Is" operational and technological 
environment and its target or "To Be" environment, as well as a capital 
investment road map for transitioning from the current to the target 
environment. These snapshots further consist of "views," which are 
basically one or more architecture products that provide conceptual or 
logical representations of the enterprise. We have made numerous 
recommendations to assist DOD in successfully developing the 
architecture and using it to gain control over its ongoing business 
systems investments. Enclosure III contains details on the status of 
all of our prior recommendations, including our assessment of DOD's 
actions.

In May 2001[Footnote 16], we reported that the department did not have 
an architecture for its financial and financial-related business 
operations, nor the management structures, processes, and controls in 
place to effectively develop and implement one. We reported that if the 
department continued to spend billions of dollars on new and modified 
systems, independently from one another and outside the context of an 
architecture, this would result in more processes and systems that are 
duplicative, not interoperable, and unnecessarily costly to maintain 
and interface. We made eight recommendations aimed at providing the 
means for effectively developing and implementing an architecture. The 
Secretary of Defense established a program in July 2001 to develop and 
implement an architecture. In April 2002, DOD entered into an agreement 
with International Business Machines (IBM) pursuant to a governmentwide 
General Services Administration contract, under which DOD issued task 
orders for services to begin developing the architecture.

During the first year of DOD's architecture development, in 2002, we 
reviewed the department's efforts and recognized that it was 
undertaking a challenging and ambitious task and following some 
architecture best practices and information technology (IT) investment 
management processes and controls. We also identified challenges and 
weaknesses in DOD's architecture efforts. For example, in a February 
2003 report,[Footnote 17] we pointed out that DOD had not yet (1) 
established a governance structure and process controls needed to 
ensure ownership of and accountability for the architecture across the 
department, (2) clearly communicated to intended stakeholders its 
purpose, scope, and approach for developing the architecture, and (3) 
defined and implemented an independent quality assurance process. We 
also reported that DOD had yet to establish the necessary departmental 
investment governance structure and process controls needed to 
adequately align ongoing investments with its architectural goals and 
direction. We made six recommendations aimed at enhancing DOD's ability 
to further develop its architecture and guide and constrain its 
business systems modernization investments.

Our March 2003[Footnote 18] report noted that the draft version of the 
architecture did not include a number of items recommended by relevant 
architectural guidance and that DOD's plans would not fully satisfy the 
requirements of the National Defense Authorization Act for Fiscal Year 
2003. For example, the draft architecture did not include a "To Be" 
security view, which defines the security requirements, including 
relevant standards to be applied in implementing security policies, 
procedures, and controls. DOD officials agreed with our preliminary 
assessment of the architecture and stated that subsequent versions of 
the architecture would provide these missing details.

In July and September 2003,[Footnote 19] we reported that although DOD 
had expended tremendous effort and resources in complying with 
statutory requirements for developing and implementing a well-defined 
architecture, the initial version of its architecture, including the 
transition plan, did not adequately address the statutory requirements 
and other relevant architectural requirements. For example, the "As Is" 
environment did not include descriptions of the current business 
operations in terms of entities and people who perform the functions, 
processes, and activities and the locations where these are performed. 
The "To Be" environment did not include descriptions of actual systems 
to be developed or acquired to support future business operations and 
the physical infrastructure that would be needed to support the 
business systems. The transition plan did not include time frames for 
phasing out existing systems within DOD's reported current inventory of 
about 2,300 business systems. Overall, the department's initial version 
of the architecture did not contain sufficient scope and detail either 
to satisfy the act's requirements or to effectively guide and constrain 
the departmentwide business transformation and systems modernization. 
In our September 2003 report[Footnote 20], we reiterated the open 
recommendations that we had made in our May 2001[Footnote 21] and 
February 2003[Footnote 22] reports, and we made 10 new recommendations 
aimed at improving DOD's plans for developing the next version of the 
architecture and implementing the institutional means for selecting and 
controlling both planned and ongoing business systems investments. To 
date, DOD has yet to address 22 of our recommendations.

Funding Status of Architecture Program:

As of March 12, 2004, about $258 million had been appropriated to 
support the program, of which the department reported having obligated 
over $203 million and having made disbursements of $111 million since 
the program began in 2002. Table 1 shows the reported status of program 
funding by appropriation and fiscal year.

Table 1: Reported Funding Status of DOD's Business Management 
Modernization Program as of March 12, 2004:

(Dollars in millions):

:

Appropriation; Appropriated; Obligated; Unobligated[A]; Disbursed; 
Unliquidated[B].

Fiscal year 2002/2003 Research, Development, Test, and 
Evaluation (RDT&E) -Defensewide (DW)[C]; 
Appropriated: $94.5; 
Obligated: $94.5; 
Unobligated[A]: $0.0; 
Disbursed: $83.0; 
Unliquidated[B]: $11.5.

Fiscal year 2003/2004 RDT&E -DW[D]; 
Appropriated: $67.2; 
Obligated: $67.2; 
Unobligated[A]: $0.0; 
Disbursed: $11.9; 
Unliquidated[B]: $55.3.

Fiscal year 2004/2005 RDT&E - DW[E]; 
Appropriated: $45.1; 
Obligated: $8.5; 
Unobligated[A]: $36.6; 
Disbursed: $0.0; 
Unliquidated[B]: $8.5.

Fiscal year 2003 Operations and Maintenance (O&M) -DW[F]; 
Appropriated: $24.9; 
Obligated: $24.9; 
Unobligated[A]: $0.0; 
Disbursed: $16.1; 
Unliquidated[B]: $8.8.

Fiscal year 2004 O&M-DW[G]; 
Appropriated: $26.1; 
Obligated: $8.3; 
Unobligated[A]: $17.8; 
Disbursed: $0.0; 
Unliquidated[B]: $8.3.

Total; 
Appropriated: $257.8; 
Obligated: $203.4; 
Unobligated[A]: $54.4; 
Disbursed: $111.0; 
Unliquidated[B]: $92.4. 

Source: Unaudited funding information contained in DOD's March 15, 2004 
report to congressional defense committees. We did not validate the 
accuracy and reliability of the funding information reported.

[A] Unobligated balances are the differences between funds appropriated 
and obligated; they represent funds that have not been obligated for 
expenditure. GAO's calculations are based on DOD's reported data.

[B] Unliquidated balances are the differences between the funds 
obligated and disbursed; they represent funds that have been obligated 
but have not yet been paid. GAO's calculations are based on DOD's 
reported data.

[C] The fiscal year 2002/2003 RDT&E funds supported the initial 
delivery of the architecture, transition plan, and change management 
and communications initiatives.

[D] The fiscal year 2003/2004 RDT&E funds were used to refine the 
architecture through business process modeling/reengineering for the 
program's first increment and to fund test and evaluation activities, 
verification and validation efforts, and engineering support.

[E] The fiscal year 2004/2005 RDT&E funds will be used to complete the 
program's first increment and begin work on the second increment and to 
fund the integration of the architecture, engineering support, and test 
and evaluation activities.

[F] The fiscal year 2003 O&M funds were used for salaries, facilities, 
supplies, and program management support contracts.

[G] The fiscal year 2004 O&M funds are being used for salaries, 
facilities, supplies, and program management support contracts.

[End of table]

DOD Has Taken Few Actions to Address Our Previous Recommendations:

DOD has taken some steps since our last review, but it has not yet (1) 
implemented key architecture management best practices,[Footnote 23] 
such as assigning accountability and responsibility for directing, 
overseeing, and approving the architecture to a committee or group 
comprised of representatives from across DOD's major organization 
components, (2) revised the architecture products to include the 
missing scope and detail needed to guide and constrain the 
implementation of system solutions, and (3) clearly defined near-term 
and long-term plans for guiding its transformation activities. Until 
the department addresses these key elements to fully satisfy relevant 
architectural guidance, it will be challenged in its ability to produce 
an architecture that is sufficient to guide and constrain its business 
operations and systems modernization efforts. In response to our 
recommendations, DOD has taken some actions, including establishing a 
group to facilitate communication and coordination across the domains 
for program activities; beginning to establish a configuration change 
management process; recently issuing a policy governing the 
development, maintenance, and implementation of the architecture; and 
updating the initial version of the architecture. However, these 
actions are not sufficient and do not fully address our concerns.

DOD Has Not Yet Implemented Key Architecture Management Best Practices:

DOD's actions have not been adequate to address architecture management 
best practices or our previous recommendations. Since our last report, 
the department has taken some actions in response to our 
recommendations concerning the need to implement an effective 
architecture management program. First, in September 2003, it formally 
established the Domain Owners Integration Team (DO/IT), which reports 
to the steering committee. The DO/IT is comprised of the various senior 
executives from each domain and the Business Modernization and Systems 
Integration office. The DO/IT is responsible for facilitating 
communication and coordination across the domains for program 
activities, including extending and evolving the architecture. 
According to program officials, the DO/IT's role is continuing to 
evolve. In particular, the specific actions or tasks it needs to 
perform to carry out this responsibility have not been defined in 
detail.

Second, DOD has taken steps to establish and implement a configuration 
management process to ensure that all changes to the architecture 
products are justified and are accounted for in a manner that maintains 
documentation integrity. Specifically, the department has established a 
configuration control board (CCB), developed a charter, and written 
procedures governing this process. However, the CCB has been tasked 
with reviewing changes to only some, but not all, of the architecture 
products. For example, the CCB is not responsible for tracking changes 
that are being made to the transition plan. In addition, both the 
charter and the procedures governing this process are still in draft. 
According to DOD officials, the charter is to be approved in June 2004, 
but no time frame has been provided for final approval of the 
procedures.

Third, DOD has recently issued an IT portfolio management policy 
governing the development, maintenance, and implementation of the 
architecture. This policy addresses, among other things, the roles, 
responsibilities, and relationships of key players and program 
participants, the value of an architecture, and the scope of the 
architecture. However, the policy does not address accountability for 
and approval of updates to the architecture, as called for by best 
practices, nor does it address the issuance of waivers in those 
instances when exceptions to the architecture are justified on the 
basis of documented analysis.

Finally, DOD has developed high-level performance measures that are to 
be used to develop the more specific results-oriented performance 
metrics needed to enable it to evaluate program progress and benefits. 
This means that DOD has yet to establish measurable, results-oriented 
goals to evaluate and track, on an ongoing basis, specific program 
progress, outcomes, and results. For example, it has not explicitly 
defined performance measures to evaluate the quality, content, and 
utility of subsequent major updates to its initial architecture. Given 
that DOD has reported obligations of over $203 million since 
architecture development efforts began 3 years ago, this is a serious 
performance management weakness. It is critical that the department 
establishes meaningful, tangible, and measurable program goals and 
objectives--short-term and long-term--to enable the department to 
determine what value it is receiving for its investment.

In addition, the department has yet to address other prior 
recommendations related to architecture management best practices. Two 
examples are provided below.

* DOD has not assigned accountability and responsibility for directing, 
overseeing, and approving the architecture to a committee or group 
comprised of representatives from across the department. Although it 
previously established an executive and a steering committee, these 
committees are advisory in nature, and they are currently not 
accountable or responsible for directing, overseeing, and approving the 
architecture. According to the department, it expects to revise the 
committees' charters in June 2004 to include these responsibilities.

* DOD does not have an independent verification and validation function 
to review the architecture products and management processes. The 
department's current verification and validation contractor is not 
independent[Footnote 24] and is responsible for reviewing the 
architecture products and only selected program deliverables. According 
to program officials, the department is planning to acquire the 
services of a new contractor to perform this work, who will be tasked 
with reviewing all the architecture products and all other program 
deliverables. However, DOD officials also said that no decision has 
been made to change the reporting structure of this function to make it 
independent or to have this contractor review architecture management 
processes. In addition, no milestones have been set for any of these 
planned actions.

Several factors have contributed to the department's limited progress 
in implementing an effective architecture management program, such as 
changes to and a lack of accountability in program leadership, 
direction, and priorities. As we previously stated, DOD has not yet 
formalized responsibility for directing, overseeing, and approving the 
architecture. Further, from May 2003 to February 2004, there was no 
program manager to identify, direct, and execute program activities. 
DOD hired a new program manager in February 2004. Initially, DOD had 
planned to develop and implement its architecture in 1 year, but later 
it adopted an incremental approach to developing the architecture. 
About 1 year after adopting this approach, DOD has assigned labels to 
these increments, but it has not yet defined the purpose of these 
increments and plans of action to implement them.

Further, our research of successful organizations and experience in 
reviewing other challenged enterprise architecture efforts show that 
senior management's understanding of and commitment to an enterprise 
architecture and overcoming cultural resistance to having and using one 
are critical success factors. Cultural resistance to change, military 
service parochialism, and stovepiped operations have all contributed 
significantly to the failure of previous attempts to implement broad-
based management reforms at DOD. Until such barriers are addressed, and 
effective architecture management structures and processes are 
established, it is unlikely that DOD will be able to produce and 
maintain a complete and enforceable architecture or implement 
modernized systems in a way that minimizes overlap and duplication and 
maximizes integration and mission support.

DOD's Architecture Products Remain Incomplete, with Minimal Content 
Change:

Since our last review, DOD has not made significant changes to the 
content of its architecture. The department has an updated version of 
the architecture (version 2.0), which is currently being reviewed, but 
it has not been approved. According to DOD, this version of the 
architecture differs from the initial architecture (version 1.0) in 
that it incorporates integrated "baseline reference business process 
models," which, according to DOD, constitute a high-level process 
framework consisting of processes that provide a business perspective 
on the department's operations (e.g., execute budget and performance 
plans) that cut across functions and organizations. DOD also stated 
that version 2.0 includes the previously excluded requirements (e.g., 
revenue requirements) for the Joint Financial Management Improvement 
Program (JFMIP)[Footnote 25] and partially addresses 21 of the 62 
missing architecture content elements that we recommended be added. 
According to DOD, the domains are currently validating all of the 
requirementsóincluding the JFMIP requirements:

Further, with regard to the 21 missing elements, documentation provided 
by DOD shows that it is either just beginning to initiate activities or 
was planning to develop plans to address these missing elements. This 
documentation also did not specify either how or when any of the 21 
elements would be fully addressed. Instead, it showed that the 
department, to date, has made only cosmetic changes to the 
architecture, such as correcting typographical (i.e., misspelled words) 
and grammatical errors, as well as deleting unnecessary text from 
various architecture products. DOD has yet to incorporate the missing 
scope and detail that we previously identified. For example, the "To 
Be" environment does not provide sufficient descriptive content related 
to future business operations and supporting technology to permit 
effective acquisition and implementation of system solutions and the 
associated operational change.

Program officials also stated that version 2.0 addresses some of the 
recommendations made by its verification and validation contractor. 
However, DOD did not provide documentation showing that it had 
addressed any of these recommendations. For example, the verification 
and validation contractor had previously reported that the "As Is" 
information was insufficient to support realistic transition planning. 
In April 2004, this contractor further reported that, while there have 
been some improvements in both the overall completeness of definitions 
and the structure of the architecture products, version 2.0[Footnote 
26] retains most of the critical problems previously cited for version 
1.0. Some examples are below.

* The utility of the architecture to stakeholders and the ability of 
the architecture to support acquisition and portfolio investment 
management decisions remain unclear.

* Supporting documentation that describes the analysis and rationale 
for architecture choices represented within many of the architecture 
products remains incomplete in many areas and, when available, is 
poorly linked or referenced.

* The "As Is" environment, including business processes and existing 
business application systems and supporting technology, is inadequate, 
making it difficult for DOD to perform a gap analysis to support 
development of a transition plan.

* The information assurance (security) representation remains spotty 
and difficult to find even at the enterprise level, and it has some 
unnecessary deviations from the department's accepted practices.

In addition, the verification and validation contractor stated that the 
reference business process models included in version 2.0, which DOD 
cited as the major difference from version 1.0, lacked the underlying 
data and linkage to make it useful in reengineering existing processes. 
The contractor also stated that these models would likely need to be 
reworked based on the results of efforts that the domains currently 
have under way.

According to DOD and contractor officials responsible for updating the 
architecture products, the changes made to date have primarily affected 
architecture management processes and not the architecture's content; 
and this is the result of the change in the program's focus and 
priorities, as discussed above. However, these officials also stated 
that the Architecture Integration Teams (AIT), comprised of 
representatives from the domains, are currently focused on developing 
needed architectural contentóprimarily business processes, business 
rules, and regulations for increment one.

Until the architecture is sufficiently complete and includes the 
missing scope and detail, the department remains at risk for not 
achieving its intended business transformation goals and of not having 
an architecture that can be used to guide and constrain ongoing and 
planned business systems investments to prevent duplicative and 
noninteroperable systems.

DOD Has Yet To Explicitly Define Near-term and Long-term Plans for 
Guiding Its Transformation Activities:

DOD has not yet developed either near-term or long-term plans for 
developing the architecture that explicitly identify and establish a 
baseline for the actions to be taken, milestones to be achieved, cost 
estimates to be met, and targeted outcomes to be achieved. As we 
previously stated, DOD has adopted an incremental approach to 
developing the architecture, including the transition plan, and plans 
to refine and extend the architecture in three increments. The 
increments, as defined by DOD, are:

* Increment one. To enable asset accountability, total force 
visibility, and an unqualified audit opinion of DOD's consolidated 
fiscal year 2007 financial statements.

* Increment two. To focus on reducing acquisition cycle time and 
streamlining the Planning, Programming, Budgeting, and Execution System 
process between fiscal years 2004 and 2009.

* Increment three. To focus on providing total asset visibility and 
total force management between fiscal years 2004 and 2010.

However, it is unclear what the increments individually or collectively 
mean, and what they will provide or allow DOD to achieve in the near-
term and long-term, because DOD does not have detailed plans that 
include performance measures for the quality, content, and utility of 
the architecture. Although the three increments were identified in 
November 2003, program officials do not expect to have a plan for 
increment one until the next version of the transition plan is 
completed in August 2004. According to program officials, the goals and 
scope for the second and third increments were only recently approved 
by the steering committee and, therefore, detailed plans of action and 
milestones for developing these plans do not yet exist.

Currently, DOD has three initiatives under way to support increment 
one. First, the program office is developing a plan of action for 
increment one and intends to complete the plan by August 2004. Second, 
the accounting and finance domain is conducting workshops to develop 
needed business rules and requirements for extending and evolving 
version 2.0 of the architecture. This domain also has two ongoing pilot 
initiatives to acquire and implement production accounting systems by 
the end of 2005 at an estimated cost of $135 million, which are 
intended to provide JFMIP-compliant financial management systems to 
support the department's general and working capital fund activities. 
Last, DOD components are developing individual plans detailing their 
respective efforts for supporting increment one. However, there is no 
evidence that the program office is coordinating with the components 
and that the components are coordinating amongst themselves. According 
to a Defense Logistics Agency (DLA) official, there is currently no 
direct link between DLA's plan and modernization program activities. 
Because there are not yet detailed plans guiding the program's 
activities, it is unclear whether and how these activities support each 
other and whether they support the department's goal of achieving an 
unqualified audit opinion in 2007.

DOD recognizes that it needs to develop detailed plans and establish 
performance metrics to measure and track program progress in order to 
determine (1) what it wants to accomplish by a certain point in time, 
(2) what it has actually accomplished by that point in time, and (3) 
how much it has spent. Changes in the direction of the program and a 
lack of sustained leadership have hindered DOD's ability to do so. In 
its March 15, 2004 progress report, DOD reported that it plans to 
establish an initial approved program metrics baseline to evaluate the 
cost, schedule, and performance of the program and that, beginning with 
the fourth quarter of fiscal year 2004, it plans to begin formal 
tracking and reporting of specific program goals, objectives, and 
measures.

The lack of explicitly defined program plans also has made it difficult 
to define measurable tasks that are or will be assigned to DOD 
employees and the 289 program contractor employees who are involved in 
architecture development, domain support, and program support 
functions. In reviewing the contractor's work statements for 
architecture development, we found that many of the tasks lacked the 
specificity necessary to use them effectively to monitor the 
contractor's progress. For example, tasks included such broad 
statements as "maintain, extend, and integrate" the architecture. Since 
DOD is acquiring services on the basis of direct labor hours at 
specified fixed hourly rates and cost of materials, it is essential 
that the department use efficient methods and effective cost controls 
to measure the work being performed.[Footnote 27] It is also essential 
because the proposed work statement that DOD plans to approve, at an 
estimated cost of $43 million, explicitly states that work is 
considered complete when the hours allotted have been expended, rather 
than defining completion as the contractor's delivery of products that 
meet predefined quality standards. According to DOD, to enable it to 
better monitor the contractor's progress, it plans to reduce future 
tasks to 15 staff-day increments (from 6 week increments) and to 
require the contractor to provide more frequent, informal products to 
program officials so they can monitor and track progress more 
rigorously.

Without an explicitly defined program baseline, detailed plans, and 
performance measures, it is difficult to validate or justify the $122 
million that DOD has requested for fiscal year 2005 and the $494 
million the department estimates it will need for fiscal years 2006 
through 2009. In fact, DOD has been unable to show measurable progress 
and meaningful utility of the architecture products to DOD stakeholders 
for the $203.4 million that had been obligated for this program as of 
March 2004.

DOD Has Taken Few Actions to Control Ongoing and Planned Investments in 
Business Systems:

DOD continues to lack effective investment management oversight and 
control over its numerous investments in business systems. While the 
domains have been designated to oversee business systems investments, 
the actual funding continues to be spread among DOD's components, which 
continue to make their own parochial decisions regarding those 
investments, including obligations in excess of $1 million for system 
improvements, without having received the scrutiny of DOD's Comptroller 
as required by the National Defense Authorization Act for Fiscal Year 
2003.[Footnote 28] Because DOD lacks a departmentwide focus and 
effective management oversight and control of its business systems 
investments, it continues to invest billions of dollars in systems that 
potentially will fail to deliver the integrated business systems 
outcomes that are needed to provide DOD management with timely and 
reliable financial information.

We previously recommended[Footnote 29] that DOD establish investment 
review boards to better control its business systems investments (with 
each board comprised of representatives from across the department) and 
that the boards, consistent with recognized best practices, use a 
standard set of investment review and decision-making criteria to 
ensure compliance and consistency with the architecture. DOD agreed 
with our recommendations and, in response, it issued in March 2004 an 
IT portfolio management policy that assigns the domains responsibility 
for IT portfolio management. However, the procedures to be followed to 
implement the policy are still under development and no time frames for 
completion have been provided. According to the IT portfolio management 
policy, the department has 180 days (until mid-September 2004) to 
develop these procedures. In addition, the department has yet to 
formalize specific roles and responsibilities of the domains, develop 
standard criteria for performing the system reviews, and assign 
explicit authority for fulfilling roles and responsibilities. DOD 
recognizes the need to clarify the roles and responsibilities 
associated with managing the domains' portfolios of business systems 
and ensure compliance with the architecture. However, it has yet to 
establish time frames for completing these activities.

DOD has not yet implemented an effective investment management 
structure and processes for controlling ongoing and planned business 
systems investments, including one that meets the act's requirements 
for ensuring that system improvements with obligations in excess of $1 
million are consistent with the architecture. In an attempt to 
substantiate that the obligations for business systems modernization 
were in accordance with the National Defense Authorization Act for 
Fiscal Year 2003, we requested that DOD agencies provide us with a list 
of obligations greater than $1 million for fiscal year 2003[Footnote 
30] and fiscal year 2004, as of December 2003. To ascertain whether 
DOD's Comptroller had made the determination required by the act, we 
compared a list of systems approvals provided by the program office 
with the obligational data (by system) provided by the DOD activities. 
As we previously testified,[Footnote 31] we identified $479 million in 
obligations for business systems modernization reported by the DOD 
military services that were not submitted to DOD's Comptroller for 
review. Subsequently, we requested information from the defense 
agencies and found $384 million in reported obligations for business 
systems modernization that had not been submitted to DOD's Comptroller 
for review. Examples of DOD system improvements with obligations in 
excess of $1 million that were not submitted include the following:

* The Defense Finance and Accounting Service (DFAS) obligated about $19 
million in fiscal year 2003 for the DFAS Corporate Database/DFAS 
Corporate Warehouse. We previously reported[Footnote 32] that DOD had 
yet to provide economic justification that its investment in this 
system would result in tangible improvements to DOD's financial 
management operations. As of April 2004, an economic analysis has yet 
to be approved but, according to DOD officials, about $129 million had 
been spent on the program through January 2004.

* The Defense Information Systems Agency obligated about $7 million in 
fiscal year 2003 and about $2 million in fiscal year 2004 for the Wide 
Area Workflow.[Footnote 33]

* DFAS obligated about $7 million in fiscal year 2003 for the Defense 
Standard Disbursing System before it was terminated in December 2003 
after approximately 7 years of effort and a reported investment of 
about $53 million. We previously reported[Footnote 34] that continued 
investment in this system had not been justified, because an economic 
analysis had not been updated to reflect significant schedule delays. 
DFAS noted that this system was terminated because a valid business 
case for continuing the effort could not be made.

* DLA obligated about $5 million in fiscal year 2003 and about $2 
million in fiscal year 2004 for the Defense Medical Logistics Standard 
Support Program.

These and other examples demonstrate that DOD does not have reasonable 
assurance that it is in compliance with the National Defense 
Authorization Act for Fiscal Year 2003, which provides that obligations 
in excess of $1 million for system improvements may not be made unless 
DOD Comptroller makes a determination that the improvement is in 
accordance with the criteria specified in the act. The act places 
limitations on the legal authority of individual program and government 
contracting officials to obligate funds in support of the systems for 
which they are responsible, but DOD has yet to proactively manage its 
investments to avoid violations of the limitations and to review and 
approve investments in any meaningful way in order to comply with these 
statutory limitations.

DOD acknowledges that the department does not have a systematic means 
to identify and determine which system improvements should be submitted 
to DOD's Comptroller for review and, in essence, is dependent upon 
system owners coming forward to the domains and requesting approval. 
Also, as we have testified,[Footnote 35] DOD components continue to 
receive direct funding for their business systems and continue to make 
their own parochial decisions regarding investments without the 
scrutiny of DOD's Comptroller that is required by the act. The current 
funding process has contributed to the proliferation of duplicative, 
nonintegrated, and stovepiped business systems that are highly prone to 
error and are unable to provide DOD management with timely, reliable 
financial information. In March 2004,[Footnote 36] we offered a 
suggestion for legislative action to address this issue, consistent 
with our open recommendations to DOD that funds for its business 
systems be appropriated directly to the domains in order to provide for 
accountability, transparency, and the ability to prevent the continued 
parochial approach to systems investments that exists today. The 
legislation we envisioned--for which we will make a legislative 
proposal in a related report to be issued soon--would define the scope 
of the domains and establish functional responsibility for managing the 
portfolio of business systems to the domains. The domains would 
establish business systems investment review boards with DOD-wide 
representation, including the military services and defense agencies.

Until DOD strengthens its process for selecting and controlling 
business systems investments and sufficiently develops a well-defined 
architecture that can be used to guide and constrain investment 
decisions, it will likely continue to spend billions of dollars on 
duplicative, nonintegrated, stovepiped systems that do not optimize 
mission performance and accountability and, therefore, do not support 
the department's transformation goals.

Conclusions:

Having and effectively using a well-defined architecture is essential 
for guiding and constraining DOD's business transformation efforts and 
moving the department away from nonintegrated business systems 
development efforts. DOD's efforts to date to address our prior 
recommendations aimed at accomplishing this have not been sufficient. 
Specifically, despite 3 years of effort and over $203 million in 
reported obligations, DOD's architecture remains insufficiently 
defined, and the way in which the department makes business systems 
investments decisions remains largely unchanged. As a result, billions 
of dollars continue to be at risk of being spent on more systems that 
are duplicative, are not interoperable, cost more to maintain than 
necessary, and do not optimize mission performance and accountability.

The future of DOD's architecture development and implementation 
activities is at risk; this in turn places the department's business 
transformation effort in jeopardy of failing as other efforts by the 
department have in the past. Therefore, it is imperative that DOD move 
swiftly to implement our 22 open recommendations, which are aimed at 
strengthening architecture management activities, adding missing 
content to architecture products, and implementing investment oversight 
and control measures.

Agency Comments and Our Evaluation:

In commenting on a draft of this report (reprinted in enc. IV), DOD 
agreed with our assessment that its long-standing business systems 
problems have resulted in a lack of adequate transparency and 
appropriate accountability across all major business areas. It 
confirmed that development and use of an architecture is necessary to 
transforming its business operations and supporting systems. However, 
in response to our characterization that progress has been limited, DOD 
stated that, over the past 3 years, progress has been slower than 
either GAO or DOD would prefer but that it has been significant, 
despite appearances to the contrary. The willingness of DOD's 
leadership to tackle the department's decades-old problems with its 
business operations and supporting systems represents a major step 
forward. However, we continue to believe that our characterization of 
DOD's progress in developing a well-defined architecture and 
implementing effective management oversight and control over its 
business systems as limited is fair. Therefore, we are not making any 
changes to our report.

DOD stated that we did not adequately consider or incorporate in our 
report important completed and ongoing activities, plans, and 
documentation, and provided an extensive package of information. We 
disagree that we did not adequately consider or incorporate in our 
report important completed and ongoing activities, plans, and 
documentation. While our assessment focused primarily on the 
department's completed, ongoing, and planned activities that it 
reported to Congress on March 15, 2004, we also considered and 
incorporated various activities, plans, and documentation through April 
27, 2004, which included the vast majority of the information in the 
package.

In addition, we acknowledged in this report most, if not all, of the 
actions DOD represents in their comments as having been completed such 
as issuing the IT portfolio management policy and establishing the DO/
IT. We also recognized many of DOD's ongoing activities, such as the 
workshops being held by the accounting and finance domain and the 
department's establishment of the AIT, which are currently focused on 
developing needed architectural content for increment one. To gain an 
understanding of the specific AIT activities, which began in January 
2004, we met with the domains during our review and discussed various 
collaborative efforts, such as their participation in validating 
business rules and requirements to support the development of end-to-
end business processes. Regarding planned activities, we recognized 
DOD's intent to revise the executive and steering committees' charters 
to include responsibilities for directing, overseeing, and approving 
the architecture, and to develop the procedures for implementing the IT 
portfolio management policy. In its response, DOD stated that it plans 
to complete these actions in June and December 2004, respectively.

Because we received information about certain activities after we 
completed our fieldwork--or not at all--these activities are not 
included in our evaluation. For example, DOD stated that the department 
plans to release a management initiative during May 2004; establish an 
architecture and program management maturity plan and supporting 
metrics to address the quality, content, and utility of the 
architecture; and establish a balanced scorecard and supporting metrics 
focused on the organization's internal management processes. While we 
inquired about the management initiative during our review, program 
officials stated that they were unable to provide us a copy because DOD 
does not allow the release of such draft documents outside the 
department. In addition, program officials did not provide a 
characterization of the focus or scope of the management initiative; 
therefore we were unable to include it in our report. We made inquiries 
about other initiatives during the course of our review, but were not 
advised of the maturity plan and balanced scorecard. However, it is 
important to note that carrying out activities does not necessarily 
guarantee results, and thus progress, unless these activities are based 
on an approved program plan that can be used to measure progress.

In addition, we disagree with DOD's other comments. Specifically, DOD 
stated that the Comptroller is accountable for approval of updates to 
the architecture as called for by best practices. As we previously 
reported[Footnote 37] and reiterated in this report, best practices 
recommend that the accountability and responsibility for directing, 
overseeing, and approving the architecture be assigned to a committee 
or group comprised of representatives responsible for the core business 
areas across the department--not a single individual. The purpose of 
assigning the accountability and responsibility to such a committee is 
to obtain and ensure continued enterprisewide support of the 
architecture effort, thereby, increasing the department's chances for 
success.

In addition, DOD stated that its verification and validation function 
is independent because the contractor responsible for these activities 
currently reports to the Deputy Director, Program Support, who is not 
directly involved with developing the architecture. DOD also stated 
that to ensure this independence, reports prepared by this contractor 
are provided directly to GAO and the steering committee. We disagree 
that this function is sufficiently independent. According to best 
practices, the contractor performing this role should report the 
results of its work directly to the steering committee, as well as the 
program office. Since the Deputy Director, Program Support reports 
directly to the program management office and not the steering 
committee, the function is not independent. In addition, the 
verification and validation contractor agreed with us that its reports 
are not provided directly to GAO or the steering committee. The 
contractor also concurred with our statement that, in the one instance 
in which it briefed the steering committee, the program office modified 
the content of the presentation.

DOD disagreed with our prior position that funds for DOD business 
systems be appropriated directly to the domains. We are reaffirming 
that position as stated by the Comptroller General in his March 2004 
testimony[Footnote 38] before the Subcommittee on Readiness and 
Management Support, Senate Armed Services Committee. This matter for 
congressional consideration and related changes in responsibility will 
be discussed in additional detail in a report to be issued shortly.

We are sending copies of this report to interested congressional 
committees; the Director, Office of Management and Budget; the 
Secretary of Defense; the Under Secretary of Defense (Comptroller); the 
Assistant Secretary of Defense (Networks and Information Integration)/
DOD Chief Information Officer; the Under Secretary of Defense 
(Acquisition, Technology, and Logistics); the Under Secretary of 
Defense (Personnel and Readiness); and the Director, Defense Finance 
and Accounting Service. This report will also be available at no charge 
on our Web site at http://www.gao.gov.

If you have any questions concerning this information, please contact 
Gregory Kutz at (202) 512-9095 or kutzg@gao.gov or Randolph Hite at 
(202) 512-3439 or hiter@gao.gov. GAO contacts and key contributors to 
this report are listed in enclosure V.

Signed by: 

Gregory D. Kutz:

Director, Financial Management and Assurance:

Signed by: 

Randolph C. Hite:

Director, Information Technology Architecture and Systems Issues:

Enclosures:

List of Committees:

The Honorable John W. Warner:
Chairman:
The Honorable Carl Levin:
Ranking Minority Member:
Committee on Armed Services:
United States Senate:

The Honorable Ted Stevens:
Chairman:
The Honorable Daniel K. Inouye:
Ranking Minority Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:

The Honorable Duncan Hunter:
Chairman:
The Honorable Ike Skelton:
Ranking Minority Member:
Committee on Armed Services:
House of Representatives:

The Honorable Jerry Lewis:
Chairman:
The Honorable John P. Murtha:
Ranking Minority Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:

Enclosure I:

SEC. 1004. [of Public Law 107-314] DEVELOPMENT AND IMPLEMENTATION OF 
FINANCIAL MANAGEMENT ENTERPRISE ARCHITECTURE:

(a) REQUIREMENT FOR ENTERPRISE ARCHITECTURE AND FOR TRANSITION PLAN--

Not later than May 1, 2003, the Secretary of Defense shall develop--

(1) a financial management enterprise architecture for all budgetary, 
accounting, finance, enterprise resource planning, and mixed 
information systems of the Department of Defense; and:

(2) a transition plan for implementing that financial management 
enterprise architecture.

(b) COMPOSITION OF ENTERPRISE ARCHITECTURE--

(1) The financial management enterprise architecture developed under 
subsection (a)(1) shall describe an information infrastructure that, at 
a minimum, would enable the Department of Defense to--

(A) comply with all Federal accounting, financial management, and 
reporting requirements;

(B) routinely produce timely, accurate, and reliable financial 
information for management purposes;

(C) integrate budget, accounting, and program information and systems; 
and:

(D) provide for the systematic measurement of performance, including 
the ability to produce timely, relevant, and reliable cost information.

(2) That enterprise architecture shall also include policies, 
procedures, data standards, and system interface requirements that are 
to apply uniformly throughout the Department of Defense.

(c) COMPOSITION OF TRANSITION PLAN--The transition plan developed under 
subsection (a)(2) shall include the following:

(1) The acquisition strategy for the enterprise architecture, including 
specific time-phased milestones, performance metrics, and financial and 
nonfinancial resource needs.

(2) A listing of the mission critical or mission essential operational 
and developmental financial and nonfinancial management systems of the 
Department of Defense, as defined by the Under Secretary of Defense 
(Comptroller), consistent with budget justification documentation, 
together with--

(A) the costs to operate and maintain each of those systems during 
fiscal year 2002; and:

(B) the estimated cost to operate and maintain each of those systems 
during fiscal year 2003.

(3) A listing of the operational and developmental financial management 
systems of the Department of Defense as of the date of the enactment of 
this Act (known as 'legacy systems') that will not be part of the 
objective financial and nonfinancial management system, together with 
the schedule for terminating those legacy systems that provides for 
reducing the use of those legacy systems in phases.

(d) CONDITIONS FOR OBLIGATION OF SIGNIFICANT AMOUNTS FOR FINANCIAL 
SYSTEM IMPROVEMENTS--An amount in excess of $1,000,000 may be obligated 
for a defense financial system improvement only if the Under Secretary 
of Defense (Comptroller) makes a determination regarding that 
improvement as follows:

(1) Before the date of an approval specified in paragraph (2), a 
determination that the defense financial system improvement is 
necessary for either of the following reasons:

(A) To achieve a critical national security capability or address a 
critical requirement in an area such as safety or security.

(B) To prevent a significant adverse effect (in terms of a technical 
matter, cost, or schedule) on a project that is needed to achieve an 
essential capability, taking into consideration in the determination 
the alternative solutions for preventing the adverse effect.

(2) On and after the date of any approval by the Secretary of Defense 
of a financial management enterprise architecture and a transition plan 
that satisfy the requirements of this section, a determination that the 
defense financial system improvement is consistent with both the 
enterprise architecture and the transition plan.

(e) CONGRESSIONAL REPORTS--Not later than March 15 of each year from 
2004 through 2007, the Secretary of Defense shall submit to the 
congressional defense committees a report on the progress of the 
Department of Defense in implementing the enterprise architecture and 
transition plan required by this section. Each report shall include, at 
a minimum--

(1) a description of the actions taken during the preceding fiscal year 
to implement the enterprise architecture and transition plan (together 
with the estimated costs of such actions);

(2) an explanation of any action planned in the enterprise architecture 
and transition plan to be taken during the preceding fiscal year that 
was not taken during that fiscal year;

(3) a description of the actions taken and planned to be taken during 
the current fiscal year to implement the enterprise architecture and 
transition plan (together with the estimated costs of such actions); 
and:

(4) a description of the actions taken and planned to be taken during 
the next fiscal year to implement the enterprise architecture and 
transition plan (together with the estimated costs of such actions).

(f) COMPTROLLER GENERAL REVIEW--Not later than 60 days after the 
approval of an enterprise architecture and transition plan in 
accordance with the requirements of subsection (a), and not later than 
60 days after the submission of an annual report required by subsection 
(e), the Comptroller General shall submit to the congressional defense 
committees an assessment of the extent to which the actions taken by 
the Department comply with the requirements of this section.

(g) DEFINITIONS--In this section:

(1) The term 'defense financial system improvement' means the 
acquisition of a new budgetary, accounting, finance, enterprise 
resource planning, or mixed information system for the Department of 
Defense or a modification of an existing budgetary, accounting, 
finance, enterprise resource planning, or mixed information system of 
the Department of Defense. Such term does not include routine 
maintenance and operation of any such system.

(2) The term 'mixed information system' means an information system 
that supports financial and non-financial functions of the Federal 
Government as defined in Office of Management and Budget Circular A-127 
(Financial Management Systems).

(h) REPEAL--(1) Section 2222 of title 10, United States Code, is 
repealed. The table of sections at the beginning of chapter 131 of such 
title is amended by striking the item relating to such section.

(2) Section 185(d) of such title is amended by striking 'has the 
meaning given that term in section 2222(c)(2) of this title' and 
inserting 'means an automated or manual system from which information 
is derived for a financial management system or an accounting system'.

Enclosure II: Scope and Methodology:

Consistent with the act and as agreed with congressional defense 
committees' staffs, our review focused on (1) the actions the 
Department of Defense (DOD) has taken to address our previous 
recommendations regarding the development and implementation of the 
architecture and (2) the actions DOD is taking to ensure its ongoing 
and planned investments will be consistent with its evolving 
architecture.

To determine what actions DOD has taken to address our previous 
recommendations that relate to the development and implementation of 
the architecture, we met with DOD and contractor officials to obtain an 
update on the status of our prior recommendations. We used our 
Enterprise Architecture Management Maturity Framework,[Footnote 39] 
which describes the stages of management maturity, to update the status 
of key elements of architecture management best practices that DOD had 
not adopted. To make this determination, we reviewed program 
documentation, such as program policies and procedures, communications 
plan, and charters for the governance bodies; and we compared them to 
the elements in the framework. We reviewed program documentation, such 
as requests documenting proposed and actual changes to the architecture 
and external[Footnote 40] requirements extracted from the updated 
architecture. We also reviewed contractor task orders to determine the 
work performed and planned by the prime contractor associated with 
extending and evolving the architecture.

To determine what actions DOD is taking to ensure its ongoing and 
planned business systems investments are consistent with its evolving 
architecture, we met with DOD officials to obtain an update on the 
status of our prior recommendations. We met with appropriate officials 
from the offices of the Under Secretary of Defense (Comptroller) and 
the Assistant Secretary of Defense (Networks and Information 
Integration)/DOD Chief Information Officer, and with representatives 
from the domains to discuss the status of various draft policies and 
guidance that are aimed at improving the department's control of and 
accountability for business systems investments. We also reviewed and 
analyzed DOD's budget request for fiscal year 2004 to identify the 
business systems investments that could be subject to the requirements 
of the National Defense Authorization Act for Fiscal Year 2003, which 
requires that all financial system improvements with obligations in 
excess of $1 million be reviewed by DOD's Comptroller, who must make a 
determination on whether the system improvements are in accordance with 
criteria specified in the act. To assess DOD's compliance with the act, 
we also obtained and reviewed departmental guidance, memorandums, DOD 
Comptroller review decisions, and other documentation provided by the 
program office. Additionally, we requested that DOD provide us with 
data on obligations in excess of $1 million for business systems 
modernization for fiscal years 2003 and 2004, as of December 2003. We 
also received obligational data from some of the defense agencies. We 
then compared the obligational data with the information we had 
received from the program office to ascertain if the systems 
modernization had been reviewed.

To augment our document reviews and analyses, we interviewed officials 
from various DOD organizations and contractors, including the Office of 
the Under Secretary of Defense (Comptroller); Office of the Under 
Secretary of Defense (Acquisition, Technology, and Logistics); Office 
of the Under Secretary of Defense (Personnel and Readiness); 
International Business Machines; and MITRE Corporation.

We did not validate the accuracy and reliability of the funding 
information contained in DOD's March 15, 2004, report. Also, the 
unliquidated and unobligated calculations that we made were based on 
DOD reported data. Further, we did not validate the accuracy and 
reliability of the obligational data provided by the military services 
and the defense agencies for systems modernization.

We requested comments on a draft of this report from the Secretary of 
Defense or his designee. We received written comments from the Acting 
Under Secretary of Defense (Comptroller), which we printed in enclosure 
IV. We considered, but did not reprint the voluminous attachments that 
DOD provided with its written comments.

We conducted our work primarily at DOD headquarters offices in 
Washington, D.C., and Arlington, Virginia, from December 2003 through 
April 2004, in accordance with U.S. generally accepted government 
auditing standards.

Enclosure III: Status of Prior Recommendations on DOD's Business 
Enterprise Architecture (BEA):

GAO-01-525: Information Technology: Architecture Needed to Guide 
Modernization of DOD's Financial Operations. May 17, 2001.

Recommendation: (1) The Secretary of Defense immediately designate DOD 
financial management modernization a departmental priority and 
accordingly direct the Deputy Secretary of Defense to lead an 
integrated program across the department for modernizing and optimizing 
financial management operations and systems; 
Status of recommendation: Implemented; 
DOD’s comments and our assessment: DOD’s comments and our assessment: 
[Empty].

Recommendation: (2) The Secretary immediately issue a DOD policy that 
directs the development, implementation, and maintenance of a financial 
management enterprise architecture; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: DOD recently issued its Information 
Technology (IT) Portfolio Management policy. This policy, in 
conjunction with the overarching Global Information Grid[A] policy, 
assigns responsibilities for the development, implementation, and 
maintenance of the BEA.b However, it does not provide for having 
accountability for and approval of updates to the BEA, processes for 
BEA oversight and control, and review and validation of the BEA.

Recommendation: (3) The Secretary immediately modify the Senior 
Financial Management Oversight Council's charter to; 
designate the Deputy Secretary of Defense as the Council Chair and the 
Under Secretary of Defense (Comptroller) as the Council vice-Chair; 
* empower the Council to serve as DOD's financial management enterprise 
architecture steering committee, giving it the responsibility and 
authority to ensure that a DOD financial management enterprise 
architecture is developed and maintained in accordance with the DOD 
C4ISR Architecture Framework; 
* empower the Council to serve as DOD's financial management investment 
review board, giving it the responsibility and authority to (1) select 
and control all DOD financial management investments and (2) ensure 
that its investment decisions treat compliance with the financial 
management enterprise architecture as an explicit condition for 
investment approval that can be waived only if justified by a 
compelling written analysis; 
and; 
* expand the role of the Council's System Compliance Working Group to 
include supporting the Council in determining the compliance of each 
system investment with the enterprise architecture at key decision 
points in the system's development or acquisition life cycle; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: The department had previously 
established the executive and steering committees, which we reported 
were advisory in nature. The department recently established the Domain 
Owners Integration Team[C] and stated that these three bodies are 
responsible for governing the program. However, these three groups have 
not been assigned the accountability and responsibility for directing, 
overseeing, and approving the BEA. According to DOD, it expects to 
revise and finalize the executive and steering committees' charters to 
include these responsibilities in June 2004; DOD issued an IT portfolio 
management policy (March 2004) that assigns the domains[D] 
responsibility for IT portfolio management. However, the procedures to 
be followed to implement the policy are still under development and no 
time frames for completion have been provided. According to the IT 
portfolio management policy, the department has 180 days (until mid-
September 2004) to develop these procedures. In addition, the 
department has yet to formalize specific roles and responsibilities of 
the domains, develop standard criteria for performing the system 
reviews, and assign explicit authority for fulfilling roles and 
responsibilities. DOD has yet to establish time frames for completing 
these activities.

Recommendation: (4) The Secretary immediately make the Assistant 
Secretary of Defense (Command, Control, Communications & Intelligence), 
in collaboration with the Under Secretary of Defense (Comptroller), 
accountable to the Senior Financial Management Oversight Council for 
developing and maintaining a DOD financial management enterprise 
architecture; In fulfilling this responsibility, the Assistant 
Secretary appoint a Chief Architect for DOD financial management 
modernization and establish and adequately staff and fund an enterprise 
architecture program office that is responsible for developing and 
maintaining a DOD-wide financial management enterprise architecture in 
a manner that is consistent with the framework defined in the CIO 
Council's published guide for managing enterprise architectures. In 
particular, the Assistant Secretary should take appropriate steps to 
ensure that the Chief Architect; 
* obtains executive buy-in and support; 
* establishes architecture management structure and controls; 
* defines the architecture process and approach; 
* develops the baseline architecture, the target architecture, and the 
sequencing plan; 
* facilitates the use of the architecture to guide financial management 
modernization projects and investments; 
and; 
* maintains the architecture; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: The Assistant Secretary of Defense 
(Networks and Information Integration)/DOD Chief Information Officer is 
a member of the executive and steering committees; however, as 
discussed previously, members' roles and responsibilities are advisory 
in nature; DOD established a Financial Management Modernization Program 
Office in July 2001. DOD has also appointed a chief architect and, 
according to the department, it has adequate program funding and staff 
for developing and maintaining its BEA. However, DOD has not yet 
defined the roles and responsibilities for the Chief Architect.

Recommendation: (5) The Assistant Secretary of Defense (Command, 
Control, Communications & Intelligence) reports at least quarterly to 
the Senior Financial Management Oversight Council on the Chief 
Architect's progress in developing a financial management enterprise 
architecture, including the Chief Architect's adherence to enterprise 
architecture policy and guidance from OMB, the CIO Council, and DOD; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: The Assistant Secretary of Defense 
(Networks and Information Integration)/ DOD Chief Information Officer 
is a member of the steering committee, which is briefed monthly by the 
program office on various program activities; 
but not necessarily on the development and maintenance of the BEA's 
content. The Chief Architect is a member of the program office.

Recommendation: (6) The Senior Financial Management Oversight Council 
report to the Secretary of Defense every 6 months on progress in 
developing and implementing a financial management enterprise 
architecture; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: The Deputy Chief Financial Officer 
briefed the Secretary of Defense in November 2003 on behalf of DOD's 
Comptroller, who chairs the executive committee.

Recommendation: (7) The Secretary reports every 6 months to the 
congressional defense authorizing and appropriating committees on 
progress in developing and implementing a financial management 
enterprise architecture; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: Senate Report 107-213 directs that 
the department report every 6 months on the status of the BEA effort. 
DOD submitted status reports on January 31 and July 31, 2003 and on 
January 31, 2004. However, these reports were submitted by DOD's 
Comptroller and were not signed by the members of the executive or 
steering committees.

Recommendation: (8) Until a financial management enterprise 
architecture is developed and the Council is positioned to serve as 
DOD's financial management investment review board as recommended 
above, the Secretary of Defense limit DOD components' financial 
management investments to; 
* deployment of systems that have already been fully tested and involve 
no additional development or acquisition cost; 
* stay-in-business maintenance needed to keep existing systems 
operational; 
* management controls needed to effectively invest in modernized 
systems; 
and; 
* new systems or existing system changes that are congressionally 
directed or are relatively small, cost effective, and low risk and can 
be delivered in a relatively short time frame; 
Status of recommendation: Not implemented; 
DOD’s comments and our assessment: DOD has not yet defined and 
implemented an approach for selecting and controlling business systems 
investments. In March 2004, the Deputy Secretary of Defense signed the 
IT portfolio management policy and assigned responsibility to the 
domains for IT portfolio management. However, the procedures to be 
followed to implement the policy are still under development and no 
time frames for completion have been provided. According to the IT 
portfolio management policy, the department has 180 days (until mid-
September 2004) to develop these procedures. In addition, the 
department has yet to formalize specific roles and responsibilities of 
the domains, develop standard criteria for performing the system 
reviews, and assign explicit authority for fulfilling roles and 
responsibilities. DOD has yet to establish time frames for completing 
these activities.

GAO-03-458: DOD Business Systems Modernization: Improvements to 
Enterprise Architecture Development and Implementation Efforts Needed. 
February 28, 2003.

Recommendation: (1) The Secretary of Defense ensure that the enterprise 
architecture executive committee members are singularly and 
collectively made explicitly accountable to the Secretary for the 
delivery of the enterprise architecture, including approval of each 
version of the architecture; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: The department had previously 
established the executive and steering committees, which we reported 
were advisory in nature. The department recently established the Domain 
Owners Integration Team and stated that these three bodies are 
responsible for governing the program. However, these groups have not 
been assigned the accountability and responsibility for directing, 
overseeing, and approving the BEA. According to DOD, it expects to 
revise and finalize the executive and steering committees' charters to 
include these responsibilities in June 2004.

Recommendation: (2) The Secretary of Defense ensure that the enterprise 
architecture program is supported by a proactive marketing and 
communication program; 
Status of recommendation: Not implemented; 
DOD’s comments and our assessment: DOD has a strategic communications 
plan; 
however, the plan has not yet been executed. According to DOD, the plan 
is scheduled to be executed between June and December 2004.

Recommendation: (3) The Secretary of Defense ensure that the quality 
assurance function; 
* includes the review of adherence to process standards and reliability 
of reported program performance, 
* is made independent of the program management function, and; 
* is not performed by subject matter experts involved in the 
development of key architecture products; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: According to DOD, the quality 
assurance function does not yet address process standards and program 
performance, nor is it yet an independent function. Further, DOD 
subject matter experts continue to be involved in the quality assurance 
function.

Recommendation: (4) The Secretary gain control over ongoing IT 
investments by establishing a hierarchy of investment review boards, 
each responsible and accountable for selecting and controlling 
investments that meet defined threshold criteria, and each composed of 
the appropriate level of executive representatives, depending on the 
threshold criteria, from across the department; 
Status of recommendation: Not implemented; 
DOD’s comments and our assessment: DOD has not yet defined and 
implemented an approach for selecting and controlling business systems 
investments. In March 2004, the Deputy Secretary of Defense signed the 
IT portfolio management policy and assigned responsibility to the 
domains for IT portfolio management. However, the procedures to be
followed to implement the policy are still under development and no 
time frames for completion have been provided. According to the IT 
portfolio management policy, the department has 180 days (until mid-
September 2004) to develop these procedures. In addition, the 
department has yet to formalize specific roles and responsibilities of 
the domains, develop standard criteria for performing the system 
reviews, and assign explicit authority for fulfilling roles and 
responsibilities. DOD has yet to establish time frames for completing 
these activities.

Recommendation: (5) The Secretary gain control over ongoing IT 
investments by establishing a standard set of criteria to include (a) 
alignment and consistency with the DOD enterprise architecture and; 
(b) our open recommendations governing limitations in business systems 
investments pending development of the architecture; 
Status of recommendation: Not implemented; 
DOD’s comments and our assessment: DOD has not developed standard 
criteria for evaluating business systems investments and has not 
established time frames for completing this activity.

Recommendation: (6) The Secretary gain control over ongoing IT 
investments by directing these boards to immediately apply these 
criteria in completing reviews of all ongoing IT investments, and to 
not fund investments that do not meet these criteria unless they are 
otherwise justified by explicit criteria waivers; 
Status of recommendation: Not implemented; 
DOD’s comments and our assessment: In March 2004, the Deputy Secretary 
of Defense signed the IT portfolio management policy and assigned 
responsibility to the domains for IT portfolio management. However, the 
procedures to be followed to implement the policy are still under 
development and no time frames for completion have been provided. 
According to the IT portfolio management policy, the department has 180 
days (until mid- September 2004) to develop these procedures. In 
addition, the department has yet to formalize specific roles and 
responsibilities of the domains, develop standard criteria for 
performing the system reviews, and assign explicit authority for 
fulfilling roles and responsibilities. DOD has yet to establish time 
frames for completing these activities. Further, the policy also does 
not address the issuance of waivers in those instances when exceptions 
to the BEA are justified on the basis of documented analysis.

GAO-03-1018: DOD Business Systems Modernization: Important Progress 
Made to Develop Business Enterprise Architecture, but Much Work 
Remains. September 19, 2003.

Recommendation: (1) The Secretary of Defense or his appropriate 
designee; 
define and implement an effective investment management process to 
proactively identify, control, and obtain DOD Comptroller review and 
approval of expenditures for new and ongoing business systems 
investments exceeding $1 million while the architecture is being 
developed and after it is completed, and which includes clearly defined 
domain owners' roles and responsibilities for selecting and controlling 
ongoing and planned 
system investments; 
Status of recommendation: Not implemented; 
DOD’s comments and our assessment: As discussed in this report, DOD has 
not yet defined and implemented an effective investment management 
process to proactively identify and control system improvements 
exceeding $1 million in obligations. DOD officials have acknowledged 
that the department does not have a systematic means to identify and 
determine which system improvements should be submitted to the 
Comptroller for review and, are dependent upon system owners coming 
forward to the domains and requesting approval.

Recommendation: (2) The Secretary of Defense or his appropriate 
designee implement the core elements in our Enterprise Architecture 
Framework for Assessing and Improving Enterprise Architecture 
Management that we identify in this report as not satisfied, including 
ensuring that minutes of the meetings of the executive body charged 
with directing, overseeing, and approving the architecture are prepared 
and maintained; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: DOD has taken some actions, but 
these actions are not sufficient to fully address our previous 
concerns. For example, DOD has; 
* established the Domain Owners Integration Team; 
* begun to establish a configuration management process; 
* recently issued an IT portfolio management policy governing the 
development, maintenance, and implementation of the BEA; 
and; 
* developed high-level performance measures that are to be used to 
develop the more specific results-oriented performance metrics that are 
needed to enable it to evaluate program progress and benefits; 
However, as discussed previously, accountability and responsibility 
have not been assigned to a committee or group comprised of 
representatives from across the DOD component organizations--which 
would be responsible for directing, overseeing, and approving the BEA. 
DOD plans to update the executive and steering committees' charters to 
include these responsibilities in June 2004. Also, the configuration 
control board has been tasked with reviewing changes to only some, not 
all BEA products (e.g., the board is not responsible for tracking 
changes that are being made to the transition plan as part of the BEA 
effort). In addition, both the charter and the procedures governing the 
configuration management process are still in draft. According to DOD, 
the charter is to be approved in June 2004, but no time frame has been 
provided for final approval of the procedures. The portfolio management 
policy does not address accountability for and approval of updates to 
the BEA--as called for by best practices. In addition, the policy does 
not address the issuance of waivers in those instances when exceptions 
to the BEA are justified on the basis of documented analysis. The 
department's current verification and validation contractor is not 
independent and is responsible for reviewing the BEA products and only 
selected program deliverables.

Recommendation: (3) The Secretary of Defense or his appropriate 
designee update version 1.0 of the architecture to include the 340 
Joint Financial Management Improvement Program requirements that our 
report identified as omitted or not fully addressed; 
Status of recommendation: Implemented; 
DOD’s comments and our assessment: [Empty].

Recommendation: (4) The Secretary of Defense or his appropriate 
designee update version 1.0 of the architecture to include the 29 key 
elements governing the "As Is" architectural content that our report 
identified as not being fully satisfied; 
Status of recommendation: Not implemented; 
DOD’s comments and our assessment: Of the 29 elements, DOD stated that 
it plans to address 26, but it did not have detailed plans on how and 
when they would be fully addressed. According to DOD, it is currently 
addressing 2 of the 26--system and process inventories--which will be 
part of the transition plan. With respect to the system inventory, 
DOD's Comptroller recently testified that the actual number of DOD 
systems could be twice as many as is currently reported; 
2,274. For the 3 elements that DOD did not consider to be relevant to 
the program, it did not provide any explicit analysis as to why the 3 
would be irrelevant. Until DOD provides such an analysis, we consider 
all 29 elements to be critical for the "As Is" architectural content.

Recommendation: (5) The Secretary of Defense or his appropriate 
designee update version 1.0 of the business enterprise architecture to 
include the 30 key elements governing the "To Be" architectural content 
that our report identified as not being fully satisfied; 
Status of recommendation: Not implemented; 
DOD’s comments and our assessment: According to DOD, it has partially 
addressed 21 elements, but it plans to address all 30. However, DOD did 
not provide any detailed plans showing how and when it would fully 
address these elements. In addition, the information provided for the 
21 elements shows that it had not addressed them but was planning to 
address them or had just begun to take needed actions.

Recommendation: (6) The Secretary of Defense or his appropriate 
designee update version 1.0 to ensure that "To Be" architecture 
artifacts are internally consistent, to include addressing the 
inconsistencies described in this report, as well as including user 
instructions or guidance for easier architecture navigation and use; 
Status of recommendation: Partially implemented; 
DOD’s comments and our assessment: DOD did not have information readily 
available to show that it had addressed inconsistencies identified in 
our report.

Recommendation: (7) The Secretary of Defense or his appropriate 
designee update version 1.0 of the architecture to include (a) the 3 
key elements governing the transition plan content that our report 
identified as not being fully satisfied and (b) those system 
investments that will not become part of the "To Be" architecture, 
including time frames for phasing out those systems; 
Status of recommendation: Not implemented;
DOD’s comments and our assessment: According to DOD, the next 
transition plan will be available in August 2004. However, DOD did not 
have information readily available to show how or when it would fully 
address our recommendations.

Recommendation: (8) The Secretary of Defense or his appropriate 
designee update version 1.0 of the architecture to address comments 
made by the verification and validation contractor; 
Status of recommendation: Not implemented;
DOD’s comments and our assessment: According to DOD, it has addressed 
some of its verification and validation contractor's comments and plans 
to address the majority of the others. However, DOD did not have 
documentation readily available to show that it had addressed some of 
the contractor's comments, and was unable to provide plans detailing 
how and when it would fully address the remaining comments. According 
to DOD, the verification and validation contractor's comments would be 
addressed in BEA updates after version 2.0.

Recommendation: (9) The Secretary of Defense or his appropriate 
designee develop a well-defined near-term plan for extending and 
evolving the architecture and ensure that this plan includes addressing 
our recommendations, defining roles and responsibilities of all 
stakeholders involved in extending and evolving the architecture, 
explaining dependencies among planned activities, and defining measures 
of activity progress; 
Status of recommendation: Not implemented;
DOD’s comments and our assessment: As discussed in this report, DOD has 
not developed well-defined near-term or long-term plans to be used to 
guide day-to-day program activities to enable it to evaluate its 
progress.

Recommendation: (10) The Secretary of Defense or his appropriate 
designee limit the pilot projects to small, low-cost, low-risk 
prototype investments that are intended to provide knowledge needed to 
extend and evolve the architecture, and are not to acquire and 
implement production version system solutions or to deploy an 
operational system capability; 
Status of recommendation: Not implemented;
DOD’s comments and our assessment: The accounting and finance domain 
has two ongoing pilot initiatives to acquire and implement production 
accounting systems by the end of 2005--at an estimated cost of $135 
million. This domain is also currently conducting workshops to develop 
needed business rules and requirements for extending and evolving the 
BEA in support of DOD's achieving objective of increment one, which 
includes achieving an unqualified audit opinion on DOD's fiscal year 
2007 consolidated financial statements. However, the department did not 
provide detailed plans that showed how or if the workshop results would 
serve as the basis for acquiring these production systems. 

Source: GAO analysis of DOD data.

[A]DOD DEFINES THE GLOBAL INFORMATION GRID AS THE GLOBALLY 
INTERCONNECTED, END-TO-end set of information, capabilities, 
associated processes, and personnel for collecting, processing, 
storing, disseminating, and managing information on demand to 
warfighters, policymakers, and support personnel.

[B]IN MAY 2003, THE DOD COMPTROLLER CHANGED THE ARCHITECTURE NAME FROM 
THE Financial Management Enterprise Architecture to the Business 
Enterprise Architecture to reflect the transformation of departmentwide 
business operations and supporting systems, including accounting and 
finance, budget formulation, acquisition, inventory management, 
logistics, personnel, and property management systems.

[C]IN SEPTEMBER 2003, DOD ESTABLISHED THE DOMAIN OWNERS INTEGRATION 
TEAM, comprised of domain representatives, to facilitate communication 
and coordination across the domains.

[D]DOD HAS SIX DEPARTMENTAL DOMAINS, WHICH ARE (1) ACCOUNTING AND 
FINANCE, (2) acquisition, (3) human resources management, (4) 
installations and environment, (5) logistics, and (6) strategic 
planning and budgeting. It also has one enterprise information 
environment mission area. The domains and the mission area, comprised 
of the Under Secretaries of Defense and the Assistant Secretary of 
Defense for Networks and Information Integration/DOD Chief Information 
Officer, have authority, responsibility, and accountability within 
their domains for business transformation, implementation of the BEA, 
development and execution of the transition plan, portfolio management, 
and establishment of a structure to ensure representation of the DOD 
components and the appropriate federal agencies.

[End of table]

Enclosure IV:

Comments from the Department of Defense:

UNDER SECRETARY OF DEFENSE 
1100 DEFENSE PENTAGON 
WASHINGTON, DC 20301-1100:

MAY 10 2004

COMPTROLLER:

Mr. Gregory Kutz Director:

Financial Management and Assurance 
United States General Accounting Office
Washington, DC 20548:

Dear Mr. Kutz:

Enclosed is the proposed response by the Department of Defense (DoD) to 
the General Accounting Office (GAO) Draft Report, "DoD Business Systems 
Modernization: Limited Progress in Development of Business Enterprise 
Architecture and Oversight of Information Technology Investments," 
(GAO-04-731R), dated April 30, 2004.

The referenced draft General Accounting Office (GAO) report accurately 
describes the size and scope of the Business Management Modernization 
Program (BMMP). As stated in the opening paragraphs of the draft 
report, "DoD is one of the largest and most complex 
organizations in the world, overhauling its business operations and 
supporting systems represents a huge management challenge." 
Furthermore, the draft report pointedly describes the adverse effect of 
long-standing business system problems, which have resulted in "a lack 
of adequate transparency and appropriate accountability across all 
major business areas.":

The DoD agrees with that assessment and concurs with the urgent need to 
undertake the largest ongoing transformation in the public sector. 
Since July 2001, we have been actively engaged in efforts to correct 
these long-standing problems. However, GAO's draft report characterizes 
DoD's progress so far as "limited." Though progress over the past 3 
years admittedly has been slower than either GAO or DoD would prefer, 
it has been significant, despite appearances to the contrary. The 
report also underestimates development methodology of the Business 
Enterprise Architecture (BEA), as well as the significant growth in 
program maturity. The Department remains committed to working with GAO 
to address these issues.

My point of contact for this matter is Dr. Paul Tibbits, Director for 
Business Modernization and Systems Integration (BMSI). Dr. Tibbits may 
be contacted by email: Paul.Tibbits@osd.mil or by telephone at (703) 
607-3370.

Sincerely,

Signed by: 

Lawrence J. Lanzillotta:

Acting 

Enclosure: As stated:

Department of Defense (DoD) Response to General Accounting Office (GAO) 
Draft Report, "DoD Business Systems Modernization: Limited Progress in 
Development of Business Enterprise Architecture and Oversight of 
Information Technology Investments" (GAO-04-731R):

DoD Response: The referenced draft General Accounting Office (GAO) 
report accurately describes the size and scope of the Business 
Management Modernization Program (BMMP). As stated in the opening 
paragraphs of the draft report, "...DoD is one of the largest and most 
complex organizations in the world, overhauling its business operations 
and supporting systems represents a huge management challenge." 
Furthermore, the draft report pointedly describes the adverse effect of 
long-standing business system problems, which have resulted in "a lack 
of adequate transparency and appropriate accountability across all 
major business areas." The Department of Defense (DoD) agrees with that 
assessment and concurs with the urgent need to undertake the largest 
ongoing transformation in the public sector. Since July, 2001, we have 
been actively engaged in efforts to correct these long-standing 
problems. However, GAO's draft report characterizes DoD' s progress so 
far as "limited." Though progress over the past 3 years admittedly has 
been slower than either GAO or DoD would prefer, it has been 
significant, despite appearances to the contrary. The report also 
underestimates development methodology of the Business Enterprise 
Architecture (BEA), as well as the significant growth in program 
maturity.

GAO's report states that DoD has little to show for its three-year 
effort and $203 million investment. However, GAO's assessment does not 
adequately consider or incorporate in their report important completed 
and ongoing activities, plans, and documentation. In addition to the 
scope and complexity of the BMMP, DoD must solve difficult, persistent, 
and tenacious culture, management and partnership barriers to defining 
and using the BEA to drive business transformation, conditions for 
which the GAO report seems to offer no allowances, although it 
certainly acknowledges them.

The draft report asserts that GAO has not observed significant change 
since their last review, which seems improbable considering the level 
of effort expended during the past year. However, it is possible that 
GAO's observations overlooked ongoing architecture development 
activity or governance deliberations and therefore did not recognize 
relevant improvements in their report. For example, the report should 
highlight the significance of the Information Technology (IT) Portfolio 
Management policy memorandum issued March 22, 2004. This document lays 
the foundation of the Department's portfolio management strategy and 
will be supplemented by the IT Portfolio Management Instruction, 
currently under development, and other management initiatives.

In 2003-2004, DoD Chief Financial Officer (CFO) and Chief Information 
Officer (CIO) collaborated on the DoD portfolio management policy. The 
BMMP and its associated governance also enabled development of a 
comprehensive directive to leverage the BEA and portfolio management 
for near term IT investment decisions.

The DoD IT budget request for FY05 totals $29B. This budget includes 
business applications, warfighting systems, shared infrastructure, 
information assurance and related 
technical activities. The business applications portion of the DoD IT 
budget totals $5.0313 in FY05, which includes BMMP systems and IT 
activities that support business functions of the Department. 
Warfighting systems account for $7.813, shared infrastructure accounts 
for $12.813, and Information Assurance and related technical activities 
account for $3.113. DoD Domains clearly have responsibility for 
developing and implementing business transition plans and portfolios 
that will assure that the $4.813 annual business system expenditures 
cited in the referenced GAO report are focused on the established 
business transformation goals of the Department.

DoD's proposed Management Initiative is a key document that lays out 
specific Domain responsibility and authority for DoD business 
transformation. It is expected to be released in May 2004. The GAO 
report does not it recognize the magnitude of the BMMP governance 
accomplishments. For the first time in its history, DOD's business 
executives are taking an integrated view of the business environment 
and making collaborative decisions based on an enterprise view. In 
addition, a major accomplishment and strong indicator of DoD 
architecture management maturity is the series of completed and ongoing 
Domain-led workshops and the subsequent collaborative Domain-led 
Architecture Integration Team (AIT) sessions.

The portfolio management policy (Attachment 1), draft portfolio 
management instruction and other related documents, provide clear 
direction and accountability for management and oversight of IT 
investment decisions. This accountability structure, organized along 
functional business owner lines, is the key mechanism for achieving DoD 
investment control frequently called for in GAO and other audit and 
management reports. These policy documents will provide the Department 
the visibility and information necessary to make informed investment 
decisions.

The Department's BEA, containing the baseline "blueprint" of the 
Department's vast, worldwide business activities (e.g., business 
policies and rules, business process controls, information flows, and 
business operational data collection and use), is one of the largest, 
most complex, and most pervasive business architectures developed to 
date, in either the public or private sectors. The DoD progress in the 
past three years exceeds any experience in the private sector and 
government.

There is no proven roadmap to guide our actions thus far. Despite the 
absence of a direct precedent or example to follow, DoD continues to 
take full advantage of guidelines, lessons learned and consultant 
expertise in addition to the guidance provided by GAO.

BEA 5coue and Detail:

The BEA 1.1 Release provided the baseline and also illustrated the need 
to approach architecture extensions in an incremental manner on 
creating additional detail. As such, DoD is deliberately delivering 
these extensions in increments, each scoped to take approximately a 
year to define in the BEA. DoD identified and approved the focus areas 
of the additional two increments (Two and Three) at the Domain Owner 
Integration Team (DO/IT) meeting on March 22, 2004 and the BMMP 
Steering Committee meeting on March 25, 2004 (Attachments 2 and 3). The 
scope of increment one had already been approved.

* Increment One: Unqualified audit opinion (UAO), asset accountability 
and total force visibility.

* Increment Two: Acquisition best practices, total asset visibility, 
improved force management, improved military health care, and improved 
environmental safety and occupational health.

* Increment Three: Improved planning, programming, budgeting and 
execution, integrated total force management, and improved force 
management.

The scope for each increment includes the processes, information, 
requirements and business rules pertinent to accomplishing each of the 
major focus areas. For the BEA, the Executive Summary Concept of 
Operations (Attachment 4) defines the needed level of detail: "The BEA 
shall depict architecture products in sufficient detail to provide an 
unambiguous interpretation of cross-Domain business transactions".

Plans to Extend and Evolve BEA:

The BEA is an integrated plan for Departmental business transformation. 
It includes work products consisting of models, diagrams, tables, and 
narratives, all of which translate DoD's business activities into 
meaningful representations of business processes with comprehensive 
integration of relevant rules and regulations. These include the 
initial all views (AV), system views (SV), technical views (TV) and 
operational views (OV) of the business processes of the six key DoD 
business Domains. The Domains encompass the following business areas; 
Accounting and Finance, Acquisition, Human Resource Management, 
Installations and Environment, Logistics, and Strategic Planning and 
Budget. For the first time in the history of DoD, each Domain Owner is 
now serving as the single authoritative source of interpretation of 
rules and regulations for the Department in each of their respective 
areas of cognizance. In addition to the Business Domains, the Assistant 
Secretary of Defense (Networks and Information Integration)/Chief 
Information Officer - ASD(NII)/CIO - is the leader of the Enterprise 
Information Environment (EIE) Mission Area. This Mission Area is 
composed of equipment, software and common information capabilities or 
services for Enterprise use.

In the Fall of 2003, the BEA was extended to include the Reference 
Business Process Model (RBPM), which proposed a desired condition for 
the enterprise-level business processes for BEA 2.0. This Release was 
published in February 2004.

For Increment One, all Domains conducted preparatory workshops to 
define supporting business processes more fully. From October 2003 to 
January 2004, the Accounting and Finance Domain conducted 26 workshops 
(Attachment 5) to specifically analyze DoD business processes and 
associated financial management requirements. These formed the basis 
for extended analysis by the increment one Architecture Integration 
Team (AITs), which focus on the following process categories:

* Team #1: Apportionment:

* Team #2: Receipts & Acceptance:

* Team #3: Commitments & Obligations:

* Team #4: Receivables, Payables & Disbursements:

* Team #5: Asset Accountability:

* Team #6: Purchase Card/Travel Card:

Each of the AITs is comprised of 8 to 21 team members consisting of 
Domain Action Officers (DAOs) serving as a Team Lead, domain Subject 
Matter Experts (SMEs) and Business Modernization Systems Integration 
(BMSI) staff supporting the teams with process modelers, data modelers, 
requirements analysts and accountants. Although they did not take 
advantage of the DoD offer, GAO was invited to participate in any and 
all the AIT sessions to witness the significance and depth of this 
collaborative work. The AITs are key to assuring cross domain 
integration, unambiguous interpretation of requirements across the 
Department, and coherent representation of end-to-end processes that 
will serve as the roadmap for subsequent IT investment and system 
configuration decisions.

As described in the AIT methodology, DoD is developing the Enterprise 
Business Process Model (EBPM), and end-to-end descriptions of DoD 
business processes. This model will accurately identify financially 
relevant requirements and business rules necessary to resolve all known 
material weaknesses to support an Unqualified Audit Opinion and asset 
accountability. These rules and requirements, along with their 
associated data objects, are being linked to DoD business processes. 
The EBPM will update the RBPM previously published in BEA Release 2.0. 
The initial EBPM (in HTML) and updated Overview and Summary (AV-1) were 
released on April 30, 2004 (BEA 2. 1) as scheduled and approved. This 
Release represents a significant improvement of the BEA and can be 
found on the BMMP Portal (portal navigation instructions are at 
Attachment 6).

The near term schedules for completing Release 2.2 are shown in 
Attachments 7 and 8. The EBPM defines the complete end-to-end process 
of doing business in the Department, from planning to disposal. The 
activity model developed for the baseline BEA will be aligned with the 
enterprise processes (EBPM) to ensure architectural completeness, and 
to facilitate its use. BEA Release 2.1 Overview and Summary (AV-1, 
Attachment 9) explains that the content of BEA Release 2.2, scheduled 
for July 2004, will support IT investment decisions for the FY06 
Budget. Release 2.3 (October 2004) and 2.4 (January 2005) will augment 
BEA 2.2 for enhanced portfolio management acquisition decision support 
and business process re-engineering.	Attachment 10 maps architecture 
products to business utility and will serve to prioritize and focus 
development of BEA products.

In BEA Release 2.2, DoD will concentrate on completeness and 
correctness of the requirements repository, business rules, the 
enterprise business process model, the conceptual data model, and 
system functions, entities and information exchanges related to 
Increment One. Release 2.3 will add the relevant realigned activity 
model, and updated organizational nodes, roles, information and 
activities.

In summary, since the publication of BEA Release 1.0 in May 2003, the 
Department has:

* Completed an initial Transition Plan that identifies BMMP concept 
strategy and delineates steps for future business transformation.

* Established the enterprise-wide DoD Portfolio Management policy.

Page 4 of 10:

* Established a DoD-wide BMMP Governance Structure, which defines 
program strategies, performs strategic decision-making, and provides 
oversight and control of portfolio management responsibilities with 
respect to IT business system investments. 

* Established an incremental 
approach for extending the BEA into implementable parts that will drive 
transformed business operations into DoD. BMMP Increments One, Two, and 
Three have been approved by the BMMP Steering and Executive Committees.

* Established initial BMMP and BMSI program office performance metrics 
(i.e., goals, objectives, measures and targets), and is near completion 
of an integrated set of enterprise and domain business transformation 
metrics. DoD plans to begin data collection and reporting in the 4`h 
Quarter of FY04.

* Established the initial inventory of business IT systems.

* Identified the template and program entries to instruct the 
implementation of a Standard General Ledger (SGL).

* Developed a schedule for completion of initial business process 
reengincering and modeling for integration with the next extension of 
the BEA.

Independent Verification and Validation Contractor:

GAO states that "DOD does not have an independent verification and 
validation function to review the architecture products and management 
processes. The department's current verification and validation 
contractor is not independent and only reviews selected architecture 
products." The current independent verification and validation (IV&V) 
contractor is a neutral third party, a Federally Funded Research and 
Development Center (FFRDC), free of organizational conflict of interest 
(OCI) issues. The contractor is not directly involved with the BEA or 
the Transition Plan development; hence they provide the requisite 
independence. Under the recently revised BMSI organization structure, 
the 1V&V contractor is not within the BEA or Transition Planning chain 
of command. IV&V contractor activities are included in a separate 
directorate with Quality Assurance and Risk Management. To assure 
independence, reports of the IV&V contractor are provided directly to 
GAO and the Steering Committees. (The list of IV&V items reviewed since 
April 2003 and provided to GAO is included in Attachment 11):

The IV&V contractor provides input to evaluation criteria for major 
deliverables and reviews the significant architecture-related 
documentation developed by the BMSI office. These include; architecture 
methodology, requirements management methodology, BEA Releases and 
other significant documents. The BMSI office is in the process of 
extending the IV&V support through a competitive acquisition process. 
The contract award is anticipated by June 2004.

GAO and Verification and Validation Contractor Comments:

Attachment 12 is the verification and validation contractor's report on 
BEA Release 2.0 as submitted to DoD and GAO. It supports DoD statements 
that some of the contractor's recommendations were addressed in Release 
2.0. Specific comments are on pages 17, 18, 19,

21, 22, and 23. DoD expects Release 2.2 and 2.3 of the architecture to 
address the remaining IV&V comments. Attachment 13 is a matrix showing 
consolidated comments mapped to disposition.

DoD is also addressing the 61 content elements identified by GAO. 
Attachment 14 shows the evaluation criteria for the detailed action 
plan for developing AS-IS BEA products. This action plan, scheduled for 
completion June 28, 2004, will provide the timeline to reconcile and 
address GAO content elements. Status for addressing the TO-BE 
architecture products is at Attachment 15. Attachment 16, BEA 
Information Assurance (IA) Guidance, illustrates DoD's plans to address 
IA and Security and Attachment 17 contains the BEA Configuration 
Management (CM) Plan as approved and implemented on January 28, 2004. 
To ensure that BEA considers and implements all of the GAO comments 
concerning the maturity of the architecture, DoD is using the GAO 
Framework for Enterprise Architecture Management Maturity to measure 
BMSI performance, as briefed to the BMMP Steering Committee.

DoD will measure BMSI performance relative to the quality of BEA 
content, and its alignment with the Federal Enterprise Architecture 
(FEA), using the Office of Management and Budget (OMB) Guidelines for 
Enterprise Architecture Assessment Framework. To ensure consistent 
mapping to FEA, DoD has developed a set of DoD enterprise reference 
models (see Attachment 18) that map to each of the FEA reference 
models.

BMMP Governance Structure:

The Department has established a BMMP Governance structure that 
institutionalizes the participation of senior civilian and military 
leaders from the Department's key business areas. We have created, and 
are using, a hierarchy of governance committees to address and resolve 
business transformation issues in the most efficient manner, at the 
lowest level possible.

Through the Program's Executive Committee, the Secretary's senior 
political leadership provides strategic direction and guidance. The 
BMMP Executive Committee and Steering Committee are jointly chaired by 
the DoD CFO and DoD CIO, and include representatives from both the 
Office of the Secretary of Defense and the Military Services. The 
Domain Owners Integration Team (DO/IT) are responsible for business 
transformation by establishing and maintaining intra-Domain governance 
processes, managing domain-specific Information Technology (IT) 
investment portfolios, ensuring systems compliance with the BEA, and 
assisting in the extension of the enterprise architecture..

The Governance Structure, at the appropriate level, has actively 
engaged in developing the Department's enterprise level Portfolio 
Management policies, and now is developing the guidance to the DoD 
Components regarding the submission of IT exhibits for the FY06 budget. 
With the release of the Department's Management Initiative, expected in 
May 2004, greater detail regarding roles and responsibilities will be 
explicitly articulated and Domain Owners will be provided additional 
direction to manage DoD IT investments as portfolios. DOD's Management 
Initiative will establish specific time frames (30 days from signature 
date) for the issuance of initial Portfolio Management guidance, to 
include the criteria for performing reviews of proposed IT investments.

Architectural compliance is noted as one of the criteria for decisions 
on what investments to make, and to modify or terminate a system in the 
recently issued Information Technology 
Portfolio Management policy memorandum. This policy memorandum states 
that the Under Secretary of Defense (Comptroller)(USD(C))/CFO is 
responsible for developing and maintaining the DoD BEA as a component 
of the overarching Global Information Grid (GIG) architecture. The 
USD(C) is therefore accountable for the approval of updates to the BEA, 
as called for by best practices.

The issue of waivers is addressed in DoD Directive 8100.1, "Global 
Information Grid (GIG) Overarching Policy", dated September 19, 2002. 
This directive is referenced in the Information Technology Portfolio 
Management policy memorandum. DoD Directive 8100.1 states that the DoD 
Chief Information Officer is responsible for establishing GIG 
compliance and enforcement mechanisms to minimize duplication of IT. 
The BEA Conops (Attachment 4) clarifies that the BEA is a component of 
the GIG and, as such, the GIG waiver policy applies to the BEA as well.

Timeframes have been established for the development and implementation 
of a DoD directive and instruction on IT Portfolio Management. The IT 
Portfolio Management policy memorandum was signed by the Deputy 
Secretary of Defense, March 22, 2004. This policy memorandum will be 
issued as a directive within 180 days (mid-September 2004). An 
accompanying IT Portfolio Management Instruction is under development, 
and as stated earlier, will be issued 30 days from signature of the 
Management Initiative. Coordination of this instruction will begin in 
June 2004, with a scheduled completion date of December 2004, The 
Department is not waiting for the publication of these documents to 
begin the review process. The Steering Committee completed its first 
round of IT Portfolio Management reviews for all Domains in March 2004. 
Decisions from these and subsequent reviews this Summer and Fall will 
guide the FY06 budget preparation process.

The Domains have a strong oversight responsibility for guiding 
investment in IT. However, the GAO has recommended that funds for DoD 
business systems be appropriated directly to the Domains in order to 
provide for accountability, transparency, and the ability to prevent 
"...a continued parochial approach to systems investments that exists 
today.":

The Department does not concur with this recommendation, and does not 
envision a governance structure in which Domain owners are assigned the 
management responsibility for the planning, design, acquisition, 
deployment, operation, maintenance, and modernization of individual DoD 
business systems. Similarly, DoD does not envision a governance or 
program management strategy in which funds are appropriated to Domain 
leaders rather than the military service and Defense agencies to 
operate, maintain, and modernize DoD business systems. The Department 
does not consider this recommendation to be advisable for several 
reasons.

The GAO's recommendation appears to have been made without 
consideration of the Department's long standing recognition of the need 
for operational expertise and perspective in the development of 
business IT systems. The recommendation was made without regard for 
collaborative consultation before advancing a concept that is radically 
different from the portfolio management concepts being implemented in 
DoD today. Finally, funds management by the Domains would require 
creation of a completely new layer of bureaucracy to assure proper 
management of appropriated funds.

Beginning with the FY06 DoD program budget development process, the 
Domains will be acting as an Investment Review Board, and will provide 
strong management oversight to the Department's business IT investment 
decisions. Therefore the Domains will oversee, coordinate and guide the 
business system investment decisions made by the DOD Components. IT 
investments will be based on Domain approval. The DoD demonstrated in 
last year's Program Review how the Logistics Domain realigned resources 
from lower priority efforts into programs that better support the BEA. 
In this case, the DoD moved $121M within the Army, $87M within the 
Navy, and the $139M within the DLA - clearly a successful demonstration 
of portfolio management. We are expanding that effort by working with 
all the Domains to identify opportunities for similar realignments in 
the FY06 budget.

DOD has the strongest requirements, acquisition and budgeting processes 
of any Federal Agency. Our efforts to govern BMMP within these existing 
processes, suggests that we will have an even more robust oversight 
process for controlling portfolio investments. All Major 
Automated Information Systems (MAIS) programs are subjected to the same 
acquisition oversight as Major Defense Acquisition Programs (MDAP), 
including adherence to Clinger-Cohen Act requirements.	IT Capital 
Planning and Investment Control have been instituted in DoD through 
longstanding processes and executive-level forums. The Department uses 
the Planning, Programming, Budgeting, and Execution System and the 
Defense Acquisition System to select, manage, and evaluate IT 
investments. DoD uses these existing management processes in lieu of 
creating parallel processes to manage IT investments - ensuring the 
integration of IT investments into the total DoD investment portfolio 
in compliance with the BEA.

Several decision-making forums govern the movement of proposed IT 
investments from one phase to the next, and ultimately determine 
whether programs will be approved for funding. These bodies include the 
Defense Resources Board, Defense Acquisition Board, IT Overarching 
Integration Product Teams (OIPT), the DoD CIO Executive Board, and the 
GIG Architecture Integration Panel. The roles of these bodies include 
ensuring proposed IT investments support core missions, allocating 
resources, overseeing and controlling IT investments, and overseeing 
the development and configuration management of integrated 
architectures.

Systems Review and Legislative Compliance:

The Department acknowledges that a more proactive approach is required 
to ensure certification compliance with the legislative requirement of 
the FY03 National Defense Authorization Act. To that end, we are 
currently finalizing and issuing guidance that will significantly 
improve the process by clearly delineating roles and responsibilities, 
establishing time frames and making key system modifications to better 
align budgets with actual IT expenditures. The guidance directs the 
Business Domains to develop and submit business 1T system certification 
compliance plans to the USD(C) within thirty days of issuance of 
Management Initiative. The plan must include a schedule to accomplish 
the reviews for all the systems comprising at least 80 percent of the 
total business system funding by August 23, 2004. To facilitate 
development of the plans the Department will use the Information 
Technology Application (ITMA) and system repositories of the 
Components, which serve as key components 
in developing the Department's IT budget requirements, as the primary 
source to identify Business IT systems needing USD(C) certification.

The completion of these reviews by Fall 2004 is required so that 
Domains will have the opportunity to review component budget 
submissions during the Program/Budget review cycle and make necessary 
funding adjustments to the FY06 budget.

Both the BEA and the Business IT investment governance structure depend 
significantly on the business system portfolio management process being 
established within the Department's key business Domains. Beginning 
with the FY06 budget development process, the Domains will provide 
strong management oversight to the Department's near-term business 
system investment decisions. Additionally, the Department has developed 
standard architecture criteria by which the business systems can be 
evaluated, to ensure they are being developed and managed in accordance 
with BMMP. These criteria will be enhanced as the BEA matures.

Metrics:

The Department understands the importance of establishing meaningful, 
tangible, and measurable program goals and objectives to guide and 
measure the value of business transformation investments. The 
establishment of a comprehensive performance measurement process for 
the transformation of DoD business operations, the largest business 
operations in the world, takes substantial time, resources, and 
extensive cross-organizational coordination. BMMP has made substantial 
progress establishing metrics (i.e., goals, objectives, measures, and 
targets) to include business transformation metrics and BMSI Program 
Management Office management metrics.

As approved by the Steering Committee and the DOAT, DoD is nearing 
completion of an initial integrated set of enterprise and domain 
business transformation metrics and plans to begin data collection and 
reporting in 4 Quarter FY04. BMMP expects the integrated set of metrics 
to encompass approximately 16 to 20 goals, 30 to 50 objectives, and 90 
to 120 measures. These metrics reflect a major accomplishment because 
they provide the basis for demonstrating the alignment of business 
transformation investments with strategic outcomes.

Additionally, BMSI has established Architecture Integration Team (AIT) 
performance metrics that track the number of requirements dispositioned 
relative to the total number of requirements and the percent of process 
steps reviewed relative to the total number of process steps. BMSI 
tasking to contractors is quite clear, serving as the basis for use of 
rigorous Earned Value Management metrics for BEA development. BMSI is 
establishing an Architecture and Program Management Maturity Plan and 
supporting metrics to address the quality, content, and utility of the 
BEA. BMSI is establishing a balanced scorecard (BSC) and supporting 
metrics focused on the organization's internal management processes.

Contracting Issues:

Under the General Services Administration (GSA) schedule through which 
the BEA Blanket Purchase Agreement (BPA) was awarded, only two types of 
contracts are permitted, 
Time and Materials (T&M) and Firm Fixed Price (FFP). Due to the 
complexity and evolutionary nature of the BEA development effort, it is 
not prudent acquisition strategy to use a firm, fixed price vehicle. 
Maintaining, extending and integrating the architecture can be 
accomplished in a variety of ways using many different approaches, such 
as the use of AITs.

The Government and the prime contractor establish acceptance criteria 
prior to work beginning on each deliverable. The Government uses the 
established criteria to evaluate the deliverables, after providing a 
clear understanding of the work expectations. Government personnel 
monitor the progress on a daily basis.

In addition, the Government holds weekly reviews with the Contractor to 
monitor deliverables, performance (Earned Value Management System-
EVMS). The attached work statement for Call 0009 (Attachment 19) 
includes additional EVMS requirements the Government is instituting to 
more closely monitor performance.

Pilot Initiatives:

The Accounting and Finance Domain is serving as the program advocate 
and oversight entity, to assure compliance with the BEA, in accordance 
with the responsibilities assigned to the Domains, for two Component 
acquisitions of Joint Financial Management Improvement 
Program (JFMIP)-compliant financial management systems (Defense 
Enterprise Accounting and Management System (DEAMS) and General Fund 
Enterprise Business System (GFEBS)) to support the Department's general 
and working capital fund activities. These initiatives are being funded 
by the individual Components, not the Accounting and Finance Domain. 

The following are GAO's comments on the Department of Defense's (DOD) 
letter dated May 10, 2004.

GAO Comments:

See the "Agency Comments and Our Evaluation" section of this report.

We have reviewed multiple versions of the architecture development 
methodology, including the latest version, which is currently draft. 
Based on our review of the latest draft, we concluded that this 
methodology does not provide a documented approach for performing 
activities in a coherent, consistent, accountable, and repeatable 
manner. Key DOD stakeholders also expressed concerns about the 
methodology. For example, these stakeholders stated that it is an 
"after-the-fact" description of work products rather than a 
prescriptive document that details the approach to be followed to 
develop the architecture.

The descriptions of increments 2 and 3 in DOD's comment letter are 
different than the descriptions the program manager provided to us on 
April 27, 2004, as reflected in this report. Because of the 
inconsistencies in the responses, it remains unclear what the 
department's focus is for increments 2 and 3. DOD's response also 
stated that the executive committee approved all three increments; 
however, DOD did not provide evidence of this approval.

As stated in our report, DOD's objective for achieving an unqualified 
audit opinion is not supported by a DOD-wide plan of action nor are the 
individual component plans linked to program activities.

As stated in our report, DOD does not expect to complete the next 
version of its transition plan until August 2004. In addition, as we 
previously reported,[Footnote 41] its initial transition plan was 
basically a plan to develop a transition plan and did not possess the 
attributes needed to guide its transformation efforts.

Attachment 11 does not list the deliverables the verification and 
validation contractor has reviewed since April 2003.

The matrix does not show the anticipated disposition (e.g., date and 
architecture version) of the verification and validation contractor's 
comments.

This information was provided after fieldwork was completed.

These briefings were presentations given by each domain describing its 
approach for conducting portfolio management reviews. The steering 
committee did not review actual systems investments nor was there 
detailed discussion about the domains' approaches.

The scope of our review did not include an evaluation of the 
realignment of funds made by the logistics domain to determine an 
effective demonstration of portfolio management.

DOD's characterization of its current process for oversight and control 
of modernization investments is inaccurate. We along with the DOD 
Inspector General continue to report on significant weaknesses in the 
department's requirements, acquisition, and budgeting processes for 
business systems modernization projects. For example, we 
reported[Footnote 42] that estimated costs for the Defense Procurement 
Payment System had increased by $274 million and the schedule had 
slipped by almost 4 years after having already invested 7 years and 
over $126 million. DOD Comptroller's noted that the project was being 
terminated due to poor program performance and increased costs.

We look forward to receiving DOD's forthcoming guidance to contracting 
and program officials that describes the process by which these 
officials obtain DOD Comptroller's statutorily required determination 
before making obligations exceeding $1 million for a financial system 
improvement. However, given the obligation data provided to us by DOD 
and included in our report, we are concerned that DOD plans no action 
to review previous obligations for system improvements to ensure that 
there have been no such violations since the enactment of the 
limitation.

We continue to believe that contractor tasks have not been clearly 
defined and that, because DOD is acquiring services on the basis of 
time and materials, it is essential that the department use efficient 
methods and effective cost controls to measure the work being 
performed. While we did not take issue with DOD on its use of a time 
and materials contract, we did state that DOD's failure to execute the 
required Determination and Finding to evaluate the cost and performance 
risks assumed by the government illustrates an overall lack of clear 
architecture development plans.

According to a program official, DOD did not begin requiring acceptance 
criteria prior to the contractor beginning work until August 2003. 
Further, DOD has paid the contractor under the time and materials 
contract even when the contractor did not meet all of the criteria for 
the deliverable. For example, based on reports provided by DOD, the 
department paid the contractor for delivery of version 2.0 of the 
architecture even though this version did not meet 50 percent of the 
acceptance criteria.

Enclosure V:

GAO Contacts and Staff Acknowledgments:

GAO Contacts Jenniffer Wilson, (202) 512-9192:

Cynthia Jackson, (202) 512-5086:

Acknowledgments In addition to the individuals named above, key 
contributors to this report included Beatrice Alff, Francine 
DelVecchio, Abe Dymond, Joanne Fiorino, Neelaxi Lakhmani, J. 
Christopher Martin, Mai Nguyen, Michael Peacock, David Plocher, 
Katherine Schirano, Darby Smith, and Bernard Trescavage.

(192114):

GAO-03-465.

FOOTNOTES

[1] U.S. General Accounting Office, Information Technology: 
Architecture Needed to Guide Modernization of DOD's Financial 
Operations, GAO-01-525 (Washington, D.C.: May 17, 2001).

[2] The Business Management Modernization Program is the department's 
business transformation initiative; it encompasses defense policies, 
processes, people, and systems that guide, perform, or support all 
aspects of business management--including development and 
implementation of the business enterprise architecture. The Under 
Secretary of Defense (Comptroller) established a DOD-wide program 
management office called Business Modernization and Systems Integration 
(BMSI), to oversee and manage the program.

[3] Business systems include financial and nonfinancial systems, such 
as civilian personnel, finance, health, logistics, military personnel, 
procurement, and transportation, with the common element being the 
generation or use of financial data to support DOD’s business 
operations.

[4] Bob Stump National Defense Authorization Act for Fiscal Year 2003, 
Pub. L. No. 107-314, § 1004, 116 Stat. 2458, 2629 (Dec. 2, 2002).

[5] In May 2003, the DOD Comptroller changed the architecture name from 
the Financial Management Enterprise Architecture to the Business 
Enterprise Architecture to reflect the transformation of departmentwide 
business operations and supporting systems, including accounting and 
finance, budget formulation, acquisition, inventory management, 
logistics, personnel, and property management systems.

[6] U.S. General Accounting Office, DOD Business Systems Modernization: 
Summary of GAO's Assessment of the Department of Defense's Initial 
Business Enterprise Architecture, GAO-03-877R (Washington, D.C.: July 
7, 2003).

[7] U.S. General Accounting Office, DOD Business Systems Modernization: 
Important Progress Made to Develop Business Enterprise Architecture, 
but Much Work Remains, GAO-03-1018 (Washington, D.C.: Sept. 19, 2003).

[8] GAO-03-1018.

[9] The steering committee, made up of senior leaders from across the 
department, is advisory in nature and is not accountable for directing, 
overseeing, and approving the architecture.

[10] DOD has six departmental domains, which are (1) accounting and 
finance, (2) acquisition, (3) human resources management, (4) 
installations and environment, (5) logistics, and (6) strategic 
planning and budgeting. It also has one enterprise information 
environment mission area. The domains and the mission area, comprised 
of the Under Secretaries of Defense and the Assistant Secretary of 
Defense for Networks and Information Integration/DOD Chief Information 
Officer, have authority, responsibility, and accountability within 
their domains for business transformation, implementation of the 
architecture, development and execution of the transition plan, 
portfolio management, and establishment of a structure to ensure 
representation of the DOD components and the appropriate federal 
agencies.

[11] U.S. General Accounting Office, Information Technology: A 
Framework for Assessing and Improving Enterprise Architecture 
Management (Version 1.1), GAO-03-584G (Washington, D.C.: April 2003).

[12] GAO-03-1018.

[13] GAO-01-525.

[14] U.S. General Accounting Office, DOD Business Systems 
Modernization: Improvements to Enterprise Architecture Development and 
Implementation Efforts Needed, GAO-03-458 (Washington, D.C.: Feb. 28, 
2003).

[15] GAO-03-1018.

[16] GAO-01-525; U.S. General Accounting Office, DOD Financial 
Management: Important Steps Underway But Reform Will Require a Long-
term Commitment, GAO-02-784T (Washington, D.C.: June 4, 2002); and U.S. 
General Accounting Office, Department of Defense: Further Actions 
Needed to Establish and Implement a Framework for Successful Business 
Transformation, GAO-04-626T (Washington, D.C.: Mar. 31, 2004).

[17] GAO-01-525.

[18] GAO-03-458.

[19] U.S. General Accounting Office, Information Technology: 
Observations on Department of Defense's Draft Enterprise Architecture, 
GAO-03-571R (Washington, D.C.: Mar. 28, 2003).

[20] GAO-03-877R and GAO-03-1018.

[21] GAO-03-1018.

[22] GAO-01-525.

[23] GAO-03-458.

[24] GAO-03-584G.

[25] Best practices recommend that the verification and validation 
function be independent of the architecture program and report directly 
to the steering committee.

[26] JFMIP requirements arise from various public laws, regulations, 
bulletins, circulars, federal accounting standards, and leading 
practices and are applicable governmentwide.

[27] The focus of this review was limited to the "As Is" and "To Be" 
architecture products, because the transition plan has not been 
updated.

[28] We also found that in issuing task orders to International 
Business Machines (IBM) pursuant to the General Services Administration 
contract, DOD did not execute a Determination and Finding to support 
its use of Time-and-Materials task orders, as required by Federal 
Acquisition Regulation 16.601, 48 C.F.R. § 16.601 (2003). This 
Determination and Finding requirement guides agencies in evaluating the 
cost and performance risks assumed by the government in awarding Time-
and-Materials contracts compared to other types of contracts that may 
provide increased incentives for the contractor to control costs and 
efficiency.

[29] Subsection 1004(d) of the Bob Stump National Defense Authorization 
Act for Fiscal Year 2003, Pub. L. No. 107-314, 116 Stat. 2630, (Dec. 2, 
2002), provides that any amount in excess of $1 million may be 
obligated for Defense financial system improvements before approval of 
its enterprise architecture and a supporting transition plan only if 
the DOD Comptroller makes a determination that the improvement is 
necessary for (1) critical national security capability or critical 
safety and security requirements or (2) prevention of significant 
adverse effect on a project that is needed to achieve an essential 
capability. The act further provides that after the architecture and 
transition plan are approved, the DOD Comptroller must determine, 
before making obligations that exceed $1 million for system 
improvements, that such improvements are consistent with the enterprise 
architecture and the transition plan.

[30] GAO-01-525 and GAO-03-458.

[31] We requested the obligational data for fiscal year 2003 for the 
period December 2, 2002, the date of enactment of the act, through 
September 2003.

[32] GAO-04-626T.

[33] U.S. General Accounting Office, DOD Business Systems 
Modernization: Continued Investment in Key Accounting Systems Needs to 
be Justified, GAO-03-465 (Washington, D.C.: Mar. 28, 2003).

[34] Wide Area Workflow is a secure web-based system for electronic 
invoicing, receipt, and acceptance. 

[35] GAO-03-465.

[36] GAO-04-626T.

[37] U.S. General Accounting Office, Department of Defense: Further 
Actions Needed to Establish and Implement a Framework for Successful 
Financial and Business Management Transformation, GAO-04-551T 
(Washington, D.C.: Mar. 23, 2004) and GAO-04-626T.

[38] GAO-03-458.

[39] GAO-04-551T.

[40] U.S. General Accounting Office, Information Technology: A 
Framework for Assessing and Improving Enterprise Architecture 
Management (Version 1.1), GAO-03-584G (Washington, D.C.: April 2003).

[41] External requirements are those that are obtained from 
authoritative sources, such as the Joint Financial Management 
Improvement Program, and constrain various aspects of the architecture.

[42] GAO-03-1018.

[43] GAO-03-465.