This is the accessible text file for GAO report number GAO-10-663 entitled 'Business Systems Modernization: Scope and Content of DOD's Congressional Report and Executive Oversight of Investments Need to Improve' which was released on May 24, 2010. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: United States Government Accountability Office: GAO: May 2010: Business Systems Modernization: Scope and Content of DOD's Congressional Report and Executive Oversight of Investments Need to Improve: GAO-10-663: GAO Highlights: Highlights of GAO-10-663, a report to congressional committees. Why GAO Did This Study: Since 1995, GAO has designated the Department of Defense’s (DOD) multibillion dollar business systems modernization program as high risk, and it continues to do so today. To assist in addressing DOD’s modernization challenges, the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 (the act) requires the department to, among other things, report specific information about business system investments, including (1) milestones and actual performance against specified measures and any revisions and (2) actions taken to certify that a modernization investment involving more than $1 million meets defined conditions before obligating funds. The act also directs GAO to review each report. As agreed, GAO focused on the fiscal year 2010 report’s compliance with, among other things, these provisions of the act. To do so, GAO compared DOD’s report to the act’s reporting requirements, interviewed DOD officials, analyzed relevant documentation, and leveraged prior GAO reports. What GAO Found: DOD’s fiscal year 2010 report to Congress on its business systems modernization program complies with key provisions in the act, but its scope and content are nevertheless limited. Specifically, * The report includes milestones, performance against milestones, and milestone revisions for specific investments. However, other important performance measures, such as measures of progress against program cost, capability, and benefit commitments are not included in the report. DOD officials attributed the missing performance-related information to various factors, including that most of the investments addressed in the report have not progressed far enough in their life cycles to measure cost, capability, and benefit performance. However, the report also cites a number of investments that have produced business improvements and cost savings, and thus it follows that performance-related information about investment costs incurred, capabilities delivered, and benefits realized is available and can be reported relative to program expectations. Moreover, programs that have not yet delivered capabilities or realized benefits have nevertheless incurred costs, which DOD can report relative to expected costs. * The report identifies certification actions associated with 116 business system modernization investments. However, the report omits certification actions for 40 other investments. According to DOD officials, the omitted actions are not new certifications, but rather are recertifications that were intentionally excluded from the report. However, certification memoranda show this is not the case for four of the actions and DOD guidance defines a recertification as a type of certification action. Further, the underlying bases for a number of reported actions are limited because program weaknesses that GAO’s prior work has raised, such as the reliability of the systems’ economic analyses and the sufficiency of business enterprise architecture compliance determinations, are not reflected in the reported certification actions. DOD’s guidance does not require that certification submissions disclose program weaknesses that GAO has raised, and DOD officials stated that reviews are limited to the information that is submitted. As a result, DOD’s annual report does not provide the full range of information that is needed to permit meaningful and informed congressional oversight of the department’s business systems modernization efforts. Moreover, the bases for some certification actions exclude relevant information about known investment weaknesses, and thus these actions may not be sufficiently justified. What GAO Recommends: GAO is recommending that future annual reports include additional information about investment performance measures and certification actions and that DOD guidance be revised to ensure that certification submissions disclose unresolved GAO findings and recommendations. DOD agreed with GAO’s recommendations. View [hyperlink, http://www.gao.gov/products/GAO-10-663] or key components. For more information, contact Randolph C. Hite at (202) 512-3439 or hiter@gao.gov. [End of section] Contents: Letter: Background: DOD's Annual Report Discusses Programs' Performance Against Milestones, but Does Not Address Performance Against Other Program Commitments: DOD's Report Discusses Business Systems Modernization Programs' Improvements in Business Operations and Cost Savings: DOD's Annual Report Does Not Describe Certification Actions for All Systems, and Justification of Reported Certification Actions Is Limited: DOD's Report States That Certification Waivers Were Not Granted: Conclusions: Recommendations for Executive Action: Agency Comments: Appendix I: Objectives, Scope, and Methodology: Appendix II: Comments from the Department of Defense: Appendix III: GAO Contact and Staff Acknowledgments: Tables: Table 1: DOD Business Systems Modernization Governance Entities' Roles, Responsibilities, and Composition: Table 2: DOD Investment Tiers: Table 3: Summary of Systems Certified, Recertified, and Decertified: Figures: Figure 1: Simplified View of DOD Organizational Structure: Figure 2: Key Milestone Status: Abbreviations: ASD(NII)/DOD CIO: Assistant Secretary of Defense (Networks and Information Integration)/Department of Defense Chief Information Officer: BEA: business enterprise architecture: BTA: Business Transformation Agency: CIO: chief information officer: CMO: chief management officer: DBSMC: Defense Business Systems Management Committee: DCMO: Deputy Chief Management Officer: DITPR: Defense Information Technology Portfolio Repository: DOD: Department of Defense: DRRS: Defense Readiness Reporting System: ERP: enterprise resource planning: ETP: enterprise transition plan: GCSS-MC: Global Combat Support System-Marine Corps: IRB: Investment Review Board: IT: information technology: NDAA: National Defense Authorization Act: [End of section] United States Government Accountability Office: Washington, DC 20548: May 24, 2010: Congressional Committees: For decades, the Department of Defense (DOD) has been challenged in modernizing its timeworn business systems.[Footnote 1] In 1995, we designated DOD's Business Systems Modernization Program as high risk, and we continue to designate it as such today.[Footnote 2] In light of the department's business systems modernization management challenges, Congress included provisions in the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005[Footnote 3] (the act) for the department to report annually on its modernization progress. More specifically, the act requires the department's annual report to, among other things, (1) describe milestones and actual performance against specified measures and any revisions, (2) discuss specific improvements in business operations and cost savings resulting from successful business system modernization efforts, (3) describe specific actions on each business system modernization submitted for certification,[Footnote 4] and (4) identify any business system modernization with an obligation in excess of $1 million that was not certified during the preceding fiscal year and reasons for the waiver. Additionally, the act directs us to submit to the cognizant congressional committees--within 60 days of DOD's report submission-- an assessment of the department's actions to comply with these requirements. As agreed with your offices, the objective of our review was to assess the actions taken by DOD to comply with key aspects of section 332 of the act.[Footnote 5] To accomplish this objective, we focused on the extent to which DOD's annual report addressed the four requirements described above. In doing so, we compared the nature and scope of the information contained in the department's report to Congress, which was submitted on March 25, 2010, as well as the supporting information that DOD used in preparing the report, to four requirements in the act. In addition, we leveraged our recent reports related to DOD's business system investments to evaluate the content of key aspects of the report.[Footnote 6] We also met with cognizant DOD officials to discuss actions taken or planned. We conducted this performance audit at DOD offices in Arlington, Virginia, from January to May 2010, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Details on our objective, scope, and methodology are contained in appendix I. Background: DOD is a massive and complex organization entrusted with more taxpayer dollars than any other federal department or agency.[Footnote 7] Organizationally, the department includes the Office of the Secretary of Defense, the Joint Chiefs of Staff, the military departments, numerous defense agencies and field activities, and various unified combatant commands that are responsible for either specific geographic regions or specific functions. (See figure 1 for a simplified depiction of DOD's organizational structure.) Figure 1: Simplified View of DOD Organizational Structure: [Refer PDF for image: organizational structure] Top level: Secretary of Defense; * Deputy Secretary of Defense[A]. Second level, reporting to Secretary of Defense: * Department of the Army; * Department of the Navy; * Department of the Air Force; * Office of the Secretary of Defense; - DOD field activities; - Defense agencies; * Inspector General; * Joint Chiefs of Staff - Combatant commands[B]. Source: GAO based on DOD documentation. [A] The Deputy Secretary of Defense serves as the Chief Management Officer, who is intended to provide focused and sustained leadership over DOD's business transformation efforts. [B] The Chairman of the Joint Chiefs of Staff serves as the spokesman for the commanders of the combatant commands, especially on the administrative requirements of the commands. [End of figure] In support of its military operations, the department performs an assortment of interrelated and interdependent business functions, including logistics management, procurement, health care management, and financial management. As we have previously reported, the DOD systems environment that supports these business functions is overly complex and error prone, and is characterized by (1) little standardization across the department, (2) multiple systems performing the same tasks, (3) the same data stored in multiple systems, and (4) the need for data to be entered manually into multiple systems. [Footnote 8] Moreover, the department's nonintegrated and duplicative systems impair its ability to combat fraud, waste, and abuse.[Footnote 9] The department recently reported that this systems environment is composed of approximately 2,480 separate business systems. For fiscal year 2010, DOD requested about $15.5 billion in funds to operate, maintain, and modernize its business systems and associated information technology (IT) infrastructure. DOD currently bears responsibility, in whole or in part, for 15 of the 31 programs across the federal government that we have designated as high risk because they are highly susceptible to fraud, waste, abuse, and mismanagement.[Footnote 10] Eight of these areas are specific to the department,[Footnote 11] and 7 other high-risk areas are shared with other federal agencies.[Footnote 12] Collectively, these high- risk areas relate to DOD's major business operations that are inextricably linked to the department's ability to perform its overall mission, directly affect the readiness and capabilities of U.S. military forces, and can affect the success of a mission. DOD's business systems modernization is one of the high-risk areas, and it is an essential enabler to addressing many of the department's other high-risk areas. For example, modernized business systems are integral to the department's efforts to address its financial, supply chain, and information security management high-risk areas. DOD's Institutional Approach to Business Systems Modernization: The National Defense Authorization Act (NDAA) for Fiscal Year 2008 designates the Deputy Secretary of Defense as the Chief Management Officer (CMO) for DOD and created a Deputy CMO position.[Footnote 13] The CMO's responsibilities include developing and maintaining a departmentwide strategic plan for business reform and establishing performance goals and measures for improving and evaluating overall economy, efficiency, and effectiveness and monitoring and measuring the progress of the department. The Deputy CMO's responsibilities include recommending to the CMO methodologies and measurement criteria to better synchronize, integrate, and coordinate the business operations to ensure alignment in support of the warfighting mission. The Business Transformation Agency (BTA) supports the Deputy CMO in leading and coordinating business transformation efforts across the department. This includes maintaining and updating the department's enterprise architecture[Footnote 14] for its business mission area. [Footnote 15] The CMO and Deputy CMO are to interact with several entities to guide the direction, oversight, and execution of DOD's business transformation efforts, which include business systems modernization. These entities include the Defense Business Systems Management Committee (DBSMC), which serves as the highest-ranking investment review and decision-making body for business systems modernization activities and is chaired by the Deputy Secretary of Defense; the principal staff assistants, who serve as the certification[Footnote 16] authorities for business system modernizations in their respective core business missions; the investment review boards (IRB),[Footnote 17] which are chaired by the certifying authorities and form the review and decision-making bodies for business system investments in their respective areas of responsibility; and the BTA, which supports IRBs and leads and coordinates business transformation efforts across the department. (Table 1 lists these entities and provides greater detail on their roles, responsibilities, and composition.) Table 1: DOD Business Systems Modernization Governance Entities' Roles, Responsibilities, and Composition: Entity: DBSMC; Roles and responsibilities: * Provides strategic direction and plans for the business mission area in coordination with the warfighting and enterprise information environment mission areas; * Recommends policies and procedures required to integrate DOD business transformation and attain cross-department, end-to-end interoperability of business systems and processes; * Serves as approving authority for business system modernizations greater than $1 million; * Establishes policies and approves the business mission area strategic plan, the enterprise transition plan for implementation of business systems modernization, the transformation program baseline, and the business enterprise architecture (BEA); Composition: Chaired by the Deputy Secretary of Defense/CMO; the Vice Chair is the Deputy Chief Management Officer (DCMO). Includes senior leadership in the Office of the Secretary of Defense such as the Under Secretaries of Defense for Acquisition, Technology, and Logistics; Comptroller; Personnel and Readiness; and Assistant Secretary of Defense (Networks and Information Integration)/Department of Defense Chief Information Officer (ASD(NII)/DOD CIO). Also includes the Military Department Chief Management Officers, the heads of select Defense Agencies, and other senior participation by the Joint Chiefs of Staff and the U.S. Transportation Command. Entity: Principal Staff Assistants/Certification Authorities; Roles and responsibilities: * Support the DBSMC's management of enterprise business IT investments; * Serve as the certification authorities accountable for the obligation of funds for respective business system modernizations within designated core business missions[A]; * Provide the DBSMC with recommendations for system investment approval; Composition: Under Secretaries of Defense for Acquisition, Technology, and Logistics; Comptroller; and Personnel and Readiness; ASD(NII)/DOD CIO; and the Deputy Secretary of Defense. Entity: IRBs; Roles and responsibilities: * Serve as the oversight and investment decision-making bodies for those business capabilities that support activities under their designated areas of responsibility; * Recommend certification for all business systems modernization investments costing more than $1 million that are integrated and compliant with the BEA; Composition: Includes the principal staff assistants; Joint Staff; ASD(NII)/DOD CIO; core business mission area representatives; military departments; defense agencies; and combatant commands. Entity: Component Precertification Authority; Roles and responsibilities: * Ensures component-level investment review processes integrate with the investment management system; * Identifies those component systems that require IRB certification and prepare, review, approve, validate, and transfer investment documentation as required; * Assesses and precertifies architecture compliance of component systems submitted for certification and annual review; * Acts as the component's principal point of contact for communication with the IRBs; Composition: Includes the chief management officers from the Air Force, Army, and Navy and other designated representatives from other defense agencies. Entity: BTA; Roles and responsibilities: * Operates under the authority of the DCMO; * Maintains and updates the department's BEA and enterprise transition plan; * Ensures that functional priorities and requirements of various defense components, such as the Army and the Defense Logistics Agency, are reflected in the architecture; * Ensures adoption of DOD-wide information and process standards as defined in the architecture; * Serves as the day-to-day management entity of the business transformation effort at the DOD enterprise level; * Provides support to IRBs; Composition: Composed of eight directorates (Chief of Staff, Defense Business Systems Acquisition Executive, Enterprise Integration, Enterprise Planning and Investment, Transformation Priorities and Requirements Financial Management, Transformation Priorities and Requirements Human Resource Management, Transformation Priorities and Requirements Supply Chain Management, and Warfighter Requirements). Source: GAO based on DOD documentation. [A] DOD has five core business missions: Human Resources Management, Weapon System Lifecycle Management, Materiel Supply and Service Management, Real Property and Installations Lifecycle Management, and Financial Management. [End of table] Overview of DOD's Tiered Accountability for Business Systems Modernization: Since 2005, DOD has employed a "tiered accountability" approach to business systems modernization. Under this approach, responsibility and accountability for business architectures and systems investment management are assigned to different levels in the organization. For example, the BTA is responsible for developing the corporate BEA (i.e., the thin layer of DOD-wide policies, capabilities, standards, and rules) and the associated enterprise transition plan (ETP). Each component is responsible for defining a component-level architecture and transition plan associated with its own tiers of responsibility and for doing so in a manner that is aligned with (i.e., does not violate) the corporate BEA. Similarly, program managers are responsible for developing program-level architectures and plans and for ensuring alignment with the architectures and transition plans above them. This concept is to allow for autonomy while also ensuring linkages and alignment from the program level through the component level to the corporate level. (Table 2 describes the four investment tiers and identifies the associated reviewing and approving entities.) Table 2: DOD Investment Tiers: Tier 1; Tier description: Major automated information system[A] or major defense acquisition program[B]; Reviewing/approving entities: IRB and DBSMC. Tier 2; Tier description: Exceeding $10 million in total development/ modernization costs, but not designated as a Major Automated Information System or Major Defense Acquisition Program; Reviewing/approving entities: IRB and DBSMC. Tier 3; Tier description: Exceeding $1 million and up to $10 million in total development/modernization costs; Reviewing/approving entities: IRB and DBSMC. Tier 4; Tier description: Investment funding required up to $1 million; Reviewing/approving entities: Component-level review only (unless the system or line of business it supports is designated as an interest program by the IRB chair). Source: GAO based on DOD documentation. [A] A Major Automated Information System is a program or initiative that is so designated by the ASD(NII)/DOD CIO or that is estimated to require program costs in any single year in excess of $32 million, total program costs in excess of $126 million, or total life cycle costs in excess of $378 million in fiscal year 2000 constant dollars. [B] A Major Defense Acquisition Program is an acquisition program that is so designated or estimated by the Under Secretary of Defense for Acquisition, Technology, and Logistics to require an eventual total expenditure for research, development, and test and evaluation of more than $365 million or, for procurement, of more than $2.190 billion in fiscal year 2000 constant dollars. [End of table] Consistent with the tiered accountability approach, the NDAA for Fiscal Year 2008 required the secretaries of the military departments to designate the department under secretaries as CMOs with primary responsibility for business operations.[Footnote 18] Moreover, the Duncan Hunter NDAA for Fiscal Year 2009 required the military departments to establish business transformation offices to assist their CMOs.[Footnote 19] Summary of Fiscal Year 2005 NDAA Requirements: Congress included provisions in the NDAA for Fiscal Year 2005 that are aimed at ensuring DOD's establishment and implementation of effective investment management structures and processes.[Footnote 20] According to the act, DOD is required to develop a BEA; develop an ETP for implementing the architecture; identify each business system proposed for funding in DOD's fiscal year budget submissions; delegate the responsibility for business systems to designated approval authorities within the Office of the Secretary of Defense; require each approval authority to establish investment review structures and processes; and, effective October 1, 2005, not obligate appropriated funds for a defense business system modernization with a total cost of more than $1 million unless the approval authority certifies that the business system modernization meets several conditions. [Footnote 21] The NDAA for Fiscal Year 2005 also requires that the Secretary of Defense submit to congressional defense committees an annual report on the department's compliance with the act's provisions. This report is to: 1. describe actions taken and planned for meeting the act's requirements, including: a) specific milestones and actual performance against specified performance measures and any revision of such milestones and performance measures and: b) specific actions on the defense business system modernizations submitted for certification under such subsection; 2. discuss specific improvements in business operations and cost savings resulting from successful defense business systems modernization efforts; 3. identify the number of defense business system modernizations certified; and: 4. identify any defense business system modernization with an obligation in excess of $1 million during the preceding fiscal year that was not certified as required, and the reasons for the waiver. Summary of Recent GAO Reviews of DOD's Business Systems Modernization and Business Transformation Efforts: Between 2005 and 2008, we reported that DOD had taken increasing steps to comply with key requirements of the NDAA for Fiscal Year 2005 relative to architecture development, transition plan development, budgetary disclosure, and investment review and to satisfy relevant systems modernization management guidance, but that much remained to be accomplished relative to the act's requirements and relevant guidance.[Footnote 22] Nevertheless, we concluded that the department had made important progress in defining and beginning to implement institutional management controls (i.e., processes, structures, and tools). Notwithstanding this progress, in May 2009, we reported that the pace of DOD's efforts in defining and implementing key institutional modernization management controls had slowed compared with progress made in each of the last 4 years, leaving much to be accomplished to fully implement the act's requirements and related guidance.[Footnote 23] For example: * The corporate BEA had yet to be extended (i.e., federated) to the entire family of business mission area architectures, including using an independent verification and validation agent to assess the components' subsidiary architectures and federation efforts. * The fiscal year 2009 budget submission included some, but omitted other key information about business system investments, in part because of the lack of a reliable, comprehensive inventory of all defense business systems. * IT investment management policies and procedures at the corporate and component levels were not fully defined. * The business system information used to support the development of the transition plan and DOD's budget requests, as well as certification and annual reviews, was of questionable reliability. * Business system investments costing more than $1 million continue to be certified and approved, but these decisions were not always based on complete information. Accordingly, we reiterated existing recommendations to address each of these areas and further recommended that DOD, among other things, improve the quality of investment-related information. DOD partially agreed with our recommendations and described actions being planned or under way to address them. DOD is currently in the process of addressing these recommendations. DOD's Annual Report Discusses Programs' Performance Against Milestones, but Does Not Address Performance Against Other Program Commitments: The act requires that DOD's report describe milestones for business system modernization programs and actual performance against performance measures. In addition, the act requires that the report specify any revisions to milestones and performance measures. To its credit, DOD's annual report includes milestones, performance against milestones, and milestone revisions for 76 programs. However, other important performances measures, which typically include measures associated with determining progress against program cost, capability, and benefit commitments, are not included in the report. BTA officials cited various reasons for the scope and content of the information provided and not provided, but these reasons are at odds with other aspects of its report. Without including information on program performance against, and revisions to, such key measures as cost, capability, and benefit commitments, DOD is not providing Congress with the information needed to inform its oversight of business system modernization programs. DOD's Report Includes Program Milestones as Well as Revisions to and Performance Against These Milestones: Consistent with the act's requirement that DOD report on specific milestones and revisions to the milestones, DOD's March 2010 report includes a summary of the status of milestones that were to be completed during fiscal 2009, the revisions associated with these milestones (e.g. delayed or deleted), and the reason for the revision. [Footnote 24] Specifically, the report lists three categories of milestones: * Standard acquisition milestones: key events and dates[Footnote 25] that are provided for under DOD's system acquisition process.[Footnote 26] * BEA compliance milestones: time frames for addressing specific IRB certification conditions related to ensuring BEA compliance. * Interim milestones: key events and dates to supplement DOD's system acquisition process milestones (e.g., implementing specific system capabilities or upgrading infrastructure by a given date). DOD's report includes a total of 224 milestones[Footnote 27] that collectively span 76 programs.[Footnote 28] Of these 224 milestones, 35 are standard acquisition milestones, 22 are BEA compliance milestones, and 167 are interim milestones. The report also discusses performance against these milestones. Specifically, of the 224 milestones, 56 percent are reported to have been met, while 21 and 23 percent are reported to have been deleted (i.e., determined to be unnecessary) or not met, respectively. (See fig 2.) With respect to acquisition and compliance milestones, the percentage of milestones that were reported as not being met was 66 and 50 percent, respectively. Figure 2: Key Milestone Status: [Refer to PDF for image: stacked vertical bar graph] Milestone category: Acquisition; Total number of milestones: Delayed: 14; Total number of milestones: Deleted: 9; Total number of milestones: Met: 12; Total number of milestones: 35. Milestone category: Compliance; Total number of milestones: Delayed: 5; Total number of milestones: Deleted: 6; Total number of milestones: Met: 11; Total number of milestones: 22. Milestone category: Interim; Total number of milestones: Delayed: 33; Total number of milestones: Deleted: 31; Total number of milestones: Met: 103; Total number of milestones: 167. Milestone category: Total; Total number of milestones: Delayed: 52; Total number of milestones: Deleted: 46; Total number of milestones: Met: 126; Total number of milestones: 224. Source: GAO analysis of DOD annual report. [End of figure] DOD's Report Does Not Address Performance Against Other Program Commitments: Beyond milestones, the act requires DOD's annual report to address actual performance against performance measures and any revision of these performance measures. As we have previously reported, meaningful information about program performance is typically measured in terms of program cost, capability, and benefit commitments, in addition to schedule commitments. Through such a range of performance measures, valuable insight into the health and success of a business system investment can be gained.[Footnote 29] As we previously discussed in this report, DOD's annual report does report schedule commitments (i.e., milestones) for its modernizing programs. While DOD's annual report also includes examples of business improvements and costs savings, this program performance information is not reported against performance measure baselines. Further, the report remains silent with respect to other important performance measures such as progress made against cost and capability commitments, which would allow congressional decision makers to understand the extent to which programs are meeting cost, capability, and benefit commitments. BTA officials stated they focused the annual report on programs that had planned milestones during fiscal year 2009. Further, they said they focused on program milestones because most of the investments covered by the report have not progressed far enough in their life cycles to measure cost, capability, and benefit performance. In addition, the annual report states that DOD does not include performance measures in its annual report for any system that has reached initial or full operational capability and is no longer modernizing. Nevertheless, the report includes descriptions of a number of programs that have progressed to the point where DOD reports on actual operational efficiencies and dollar savings that have accrued, which, in turn, means that these programs have progressed to the point that DOD can report on progress against defined performance commitments, such as the costs that have been incurred, the capabilities that have been delivered, and benefits that have been realized. Moreover, the programs that have not yet delivered capabilities or realized benefits have incurred costs, which DOD can report relative to expected costs. Establishing performance measures and monitoring actual-versus- expected performance using the full range of measures are essential to understanding the health of any IT investment. By not including information on each program's performance against defined cost, capability, and benefit commitments, DOD is not providing Congress with important information for informing its oversight of business system modernization investments. DOD's Report Discusses Business Systems Modernization Programs' Improvements in Business Operations and Cost Savings: The act requires that DOD's annual report discuss specific improvements in business operations and cost savings resulting from successful business system modernization programs. DOD met this requirement by including 18 "case in point" examples in the report. Among other things, each narrative generally describes the program and provides high-level information on system capabilities delivered and benefits achieved to date. Specific examples include: * The Air Force Recruiting Information Support System is to be the primary Web-based recruiting system for the Air Force. According to the annual report, the Air Force's legacy recruiting information system has been slow and, at times, unavailable to users. Modernization of the system (e.g., new hardware and software upgrades) is reported to have improved recruiters' productivity by reducing the wait time for processing recruiter requests. Other reported improvements include allowing recruiters to build applicant files off line and upload them at a later time and reducing the recruiter's dependence on Internet connectivity. Future plans include merging the Air Force Recruiting Information Support System with the Air Force Reserve Recruiting System to increase functionality and decrease system response time. * The Army Learning Management System[Footnote 30] is a Web-based system for training, scheduling, and career planning for soldiers. According to the annual report, in 2009 the system contributed to an 88 percent increase in the number of student accounts, 111 percent increase in the number of courses offered, and 157 percent increase in the number of course completions. * Wide Area Work Flow is a Web-based system to centralize and automate DOD's largely manual business payment process. The annual report states that the system has thus far allowed the cost of processing a payment to decrease from between $22 and $30 to between $6 and $12. Other cited benefits include allowing suppliers to have a single point of interface with DOD for payment invoicing, receipt, and acceptance. * The Defense Agencies Initiative is an enterprise resource planning (ERP) system[Footnote 31] to standardize and integrate enterprise data to support financial decisions in real time. According to the annual report, the system resulted in a reduction in the time it takes to post financial obligations from 60 days to less than 2 days, and a reduction in the time to close out monthly financial reports from 4 days to less than 1 day. Further, the report states that financial information is now available to BTA on a real-time basis and thereby enables proactive management of agency finances. DOD's Annual Report Does Not Describe Certification Actions for All Systems, and Justification of Reported Certification Actions Is Limited: Among other things, the act requires DOD's annual report to describe specific actions the department has taken on each business system modernization investment submitted for certification. More specifically, the act states that such investments involving more than $1 million in obligations must be certified by a designated approval authority[Footnote 32] as meeting specific criteria, such as demonstrating compliance with DOD's BEA.[Footnote 33] Further, the act requires the DBSMC to approve each of these certifications. To its credit, DOD's annual report identifies IRB certification actions associated with 116 business system investments. However, certification actions associated with 40 other investments are not included. Further, the bases for several of the fiscal year 2009 system certification actions and subsequent approvals are limited because program weaknesses and issues that our prior work has raised about, for example, the systems' economic analyses and BEA compliance determinations, are not reflected in the reported certification actions. According to BTA officials, only new certifications are included in the report, even though DOD guidance states that recertifications are also certification actions. Moreover, this guidance does not require programs to disclose program weaknesses and issues raised by us or others. By not fully identifying in its annual report the certification actions taken on all business system modernization investments, DOD is not fully informing congressional oversight. Further, by not ensuring that all certifications reflect known program weaknesses, business system modernization program certification and approval decisions are not being fully informed and thus may not be adequately justified. DOD Has Established an Approach to Certifying Business System Investments: DOD has established what it describes as a "tiered accountability" approach to meeting the act's requirements for certifying business system investments. Under this approach, investment review begins within the military departments and defense agencies and advances through a hierarchy of review and decision-making authorities, depending on the size, nature, and significance of the investment. For those investments that meet the act's dollar threshold, namely those with planned modernizations in excess of $1 million, this sequence of review and decision making includes component precertification, IRB certification, and DBMSC approval.[Footnote 34] For those investments that do not meet the dollar threshold, investment decision-making authority remains with the responsible component. According to the department's approach, reviews for modernization investments of more than $1 million focus on program alignment with the BEA; alignment with the department's strategic mission, goals, and objectives; and oversight commensurate with the program's cost, scope, and complexity. The approach further requires that these reviews be completed before a component can obligate modernization funds. At the component level, program managers are responsible for the information about their respective programs in a central repository known as the Defense IT Portfolio Repository system (DITPR).[Footnote 35] The component precertification authority is responsible for precertifying that a given system investment is compliant with the BEA, reviewing the system funding requests, and ensuring that the IRB responsible for the investment receives complete, current, accurate, and timely information. The precertification authority is also responsible for "asserting" the status and validity of the investment information by submitting a component precertification letter to the responsible IRB.[Footnote 36] At the corporate level, an IRB reviews the precertification letter and related material and makes a recommendation for a specific certification action for each of its investments. After the IRB makes its recommendation, it prepares a certification memorandum that documents the IRB's decisions and any related conditions. The memorandum is forwarded to the DBSMC, which either approves or disapproves the IRB's decision and issues a memorandum containing its decision. If the DBSMC disapproves a system investment's certification, it is up to the component's precertification authority to decide whether or not to resubmit the investment after it has resolved the reasons for the disapproval. Under DOD's approach, there are four types of certification actions: * Certify: An IRB certifies the modernization as fully meeting criteria defined in the act and IRB investment review guidance, such as compliance with the BEA and the extent to which the investment is consistent with component and department IT investment portfolios, which are asserted by the component precertification authority. * Certify with conditions: An IRB certifies the modernization with the understanding that it will address specific IRB-imposed conditions. For example, the Army's Real Estate Management Information System was certified with a condition to provide a plan for how data elements would comply with certain business rules in DOD's BEA. * Recertify: An IRB certifies the obligation of additional modernization funds for a previously certified modernization investment. For example, the Air Force's Cargo Movement Operations System was recertified in April 2009 for $6.3 million to be spent in fiscal years 2009 through 2012. This recertification was in addition to the $21.1 million previously certified in fiscal year 2007. In addition, a program must request IRB recertification if the program plans to redistribute previously approved modernization funds among multiple fiscal years and this redistribution will result in the funding for any given fiscal year exceeding the previously approved amount by 10 percent or more. * Decertify: An IRB reduces the amount of modernization funds available to an investment when the entire amount of funding is not to be obligated as previously certified. For example, the Defense Financial Accounting Service's Standard Disbursing Initiative had about $5.5 million decertified because the funds were no longer needed. An IRB may also decertify a modernization after it has been completed. For example, DOD reported that $213,000 for the Air National Guard's Reserve Order Writing System was decertified at the time the system was completed because the funds were no longer needed. DOD's Report Excludes a Number of Certification Actions Taken on Business System Modernizations: The act requires that DOD's annual report describe specific actions taken for each business system investment submitted for certification. However, the department's annual report discusses fiscal year 2009 certification actions on only 116 of the 156 systems on which certification actions were taken. More specifically, the report states that during fiscal year 2009, 92 business system modernizations were certified--32 with and 60 without conditions. For the 32 systems, 58 conditions were collectively reported. Examples of conditions cited in the report are the need for a system to comply with the Standard Financial Information Structure[Footnote 37] and the need to develop a plan for complying with the data standards of DOD's Item Unique Identifier Registry.[Footnote 38] The report also identifies 24 decertifications. For example, the Air Force's Enhanced Technical Information Management System had about $13.9 million in funding decertified (i.e., reduced), and the Defense Financial Accounting Service's Standard Disbursing Initiative had about $5.5 million decertified. However, fiscal year 2009 IRB and DBSMC decision memoranda and meeting minutes show that 40 systems had additional certification actions that were not included in DOD's annual report.[Footnote 39] Of these 40 systems, 2 were certified without conditions, 2 were certified with conditions, and 36 were recertified. Collectively, DOD's annual report omits about 26 percent of its certification actions. (See table 3 for a summary of actual, reported, and unreported certification actions.) Table 3: Summary of Systems Certified, Recertified, and Decertified: Action: Certify without conditions; Actual: 62; Reported: 60; Not reported: 2. Action: Certify with conditions; Actual: 34; Reported: 32; Not reported: 2. Action: Recertify; Actual: 36; Reported: 0; Not reported: 36. Action: Decertify; Actual: 24; Reported: 24; Not reported: 0. Action: Total; Actual: 156; Reported: 116; Not reported: 40. Source: GAO analysis of DOD documentation. [End of table] According to BTA officials, the excluded certification actions are all recertifications, which they said are intentionally not reported because they are not new certifications. They also told us that the four new certifications were, in fact, recertifications. However, DBSMC and IRB memoranda and meeting minutes identify these four certification actions as new certifications.[Footnote 40] Moreover, DOD guidance defines a recertification as a type of certification action, thus making it subject to the act's reporting requirements. Without complete reporting of its certification actions, DOD is not in full compliance with its guidance, and DOD is limiting congressional visibility into the full scope of its business systems modernization efforts. IRB Certification Actions Do Not Reflect and Disclose Known Limitations in Programs' BEA Alignment, Economic Justification, and Management: According to DOD guidance, IRB certification addresses, among other things, a program's alignment with the BEA and its management relative to factors such as system cost, scope, and complexity. To make a certification decision, IRBs rely on documentation submitted by the component precertification authority, including a certification dashboard, which includes cost and schedule status information; an economic viability analysis, which addresses the investment's cost and benefits or cost effectiveness; and regulatory and standards compliance determinations. DOD guidance also gives IRBs broad authority in their certification reviews and actions, thus allowing each board to review and consider whatever investment-related information that it deems appropriate. Moreover, BTA and IRB officials told us that an IRB is not limited in the conditions it can place on a program. The IRB certification actions described in DOD's latest annual report are limited because they do not reflect significant limitations in the department's basis for determining an investment's alignment to the BEA. Specifically, we recently reported that key DOD BEA compliance assessments did not include all relevant architecture products, such as products that specify the technical standards needed to promote interoperability among related systems or examine overlaps with other business systems. In addition, we reported that these compliance assessments were not validated by DOD certification and approval entities.[Footnote 41] Despite these limitations, business systems modernization programs were certified as compliant with the BEA even though they did not adequately demonstrate such compliance. Accordingly, we recommended that DOD revise its BEA compliance guidance, tool, and IRB verification requirements to address these shortfalls. To date, DOD has yet to implement these recommendations, and thus the compliance determination weaknesses remain. Despite this, DOD's latest annual report does not disclose these limitations on any of the 116 investment certification actions that it describes. In addition, the fiscal year 2009 IRB certification actions described in the latest annual report are further limited in that they do not reflect weaknesses we have recently reported with the economic justification for and management of certain programs.[Footnote 42] For example, * We reported in September 2009 that the Defense Readiness Reporting System (DRRS) program[Footnote 43] was not being effectively managed and made recommendations to address a number of acquisition management weaknesses including the absence of a reliable integrated master schedule and well defined and managed requirements, and adequate testing.[Footnote 44] As stated in our report, we briefed the DRRS program office on the results of our work prior to its DOD Human Resources Management IRB certification review. However, these results were not disclosed to the IRB. Rather, the certification package that the precertification authority submitted to the IRB stated that DRRS was on track for meeting its cost, schedule, and performance measures and highlighted no program risks. Based on this submission, DRRS was certified by the IRB and approved by the DBSMC to obligate $24.625 million in fiscal years 2009 and 2010. According to the chair of the IRB, the board did not validate the information in the submissions it received, and the results of our review were not disclosed to the IRB. * We reported in July 2008 that the Global Combat Support System- Marine Corps (GCSS-MC) program[Footnote 45] had not been economically justified on the basis of reliable estimates of both benefits and costs and that key program management controls, such as the use of earned value management and risk management had not been properly implemented.[Footnote 46] Accordingly, we made recommendations to address these weaknesses. GCSS-MC was certified with conditions and recertified during fiscal year 2009. Neither the weaknesses that we previously reported nor the status of our recommendations to address them were evident in the conditions accompanying the certification, even though our recommendations had yet to be implemented at the time of these certification actions. * We reported in September 2008[Footnote 47] that the Navy ERP program did not use important cost estimating practices when economically justifying the program, did not implement key aspects of earned value management, and did not have risk mitigation strategies in place to address the risks described in our report, including risks associated with these issues. Accordingly, we made recommendations to address these weaknesses. However, when Navy ERP was recertified during fiscal year 2009, conditions relative to any of these weaknesses did not accompany the recertification, although our recommendations had yet to be implemented at the time of recertification. Officials representing each of the IRBs stated that the boards depend on the component precertification authorities to provide them with complete and reliable information about each system investment. Among other things, IRB officials stated that such information should include the results of reviews by us and others. However, DOD guidance does not state that GAO-related information, such as open recommendations or the focus and results of our ongoing reviews, is to be included in the certification packages provided to the IRBs. Further, the Special Assistant to the Deputy Chief Management Officer told us it is each program's milestone decision authority[Footnote 48] that is ultimately responsible for addressing known program management issues, including those raised by GAO. By not having and considering relevant information about the state of each system modernization investment certified and approved, such as the results of our reviews and the status of actions to implement our recommendations that pertain to the investment, DOD's certification and approval decisions are based on limited information, and thus may not be justified. DOD's Report States That Certification Waivers Were Not Granted: The act requires DOD to identify in its annual report any defense business system modernization with an obligation in excess of $1 million during the preceding fiscal year that was not certified and approved according to the act's provisions, along with any reasons for these requirements being waived. According to DOD's latest annual report, system investments were certified according to the act's requirements during fiscal year 2009, and no systems were granted a certification waiver. Similarly, each of DOD's annual reports since March of 2006 has stated that no systems were approved on the basis of a certification waiver. According to officials representing each of the IRBs, while program officials sometimes seek to be certified on the basis of a waiver, their practice is to ensure that the program office addresses any issues underlying a waiver request before the investment is placed on an IRB's certification review agenda. As a result, they stated that a system is unlikely to go before an IRB for certification until it can be certified with conditions. Conclusions: DOD's latest annual report on its business systems modernization program complies with statutory requirements pertaining to the report's content, but the scope and completeness of key information that is provided in the report is otherwise limited. In particular, the report omits information on numerous business system investment certification actions taken during fiscal year 2009. In addition, while it includes schedule-focused performance measures and performance against these measures for the modernization investments discussed, as required by statute, it does not include similar information for other performance measures, such as cost, capability, and benefit commitments and performance against these commitments. Collectively, this means that DOD's annual report does not provide congressional committees with the full range of information necessary to permit meaningful and informed oversight of DOD's business systems modernization program. Beyond the scope and content of DOD's annual report, the basis for the IRB certifications have been limited because DOD guidance does not provide for disclosure of our findings concerning investments being considered. In particular, investments have been certified and approved without conditions even though our prior reports have identified program weaknesses that were unresolved at the time of certification and approval. As a result, these certification and approval decisions may not be sufficiently justified. Recommendations for Executive Action: To facilitate congressional oversight and promote departmental accountability, we recommend that the Secretary of Defense direct the Deputy Secretary of Defense, as the chair of the DBSMC, to ensure that the scope and content of future DOD annual reports to Congress on compliance with section 332 of the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, as amended, be expanded to include: * Cost, capability, and benefit performance measures for each business system modernization investment and actual performance against these measures. * All certification actions, as defined in DOD guidance, which were taken in the previous year by the department on its business system modernization investments. To ensure that IRB certification actions are better informed and justified, we further recommend that the Secretary direct the Deputy Secretary to ensure that DOD guidance be revised to include provisions that require IRB certification submissions disclose program weaknesses raised by GAO and the status of actions to address our recommendations to correct the weaknesses. Agency Comments: In written comments on a draft of this report, signed by the Assistant Deputy Chief Management Officer and reprinted in appendix II, the department agreed with our recommendations. We are sending copies of this report to interested congressional committees; the Director, Office of Management and Budget; and the Secretary of Defense. This report will also be available at no charge on our Web site at [hyperlink, http://www.gao.gov]. If you or your staffs have any questions on matters discussed in this report, please contact me at (202) 512-3439 or hiter@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. Signed by: Randolph C. Hite: Director: Information Technology Architecture and Systems Issues: List of Committees: The Honorable Carl Levin: Chairman: The Honorable John McCain: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Daniel Inouye: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee of Defense: Committee on Appropriations: United States Senate: The Honorable Ike Skelton: Chairman: The Honorable Howard P. McKeon: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable Norm Dicks: Chairman: The Honorable C.W. Bill Young: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Appendix I: Objectives, Scope, and Methodology: As agreed with congressional defense committees, our objective was to assess the actions by the Department of Defense (DOD) to comply with the requirements of key aspects of section 332 of the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 (the act). [Footnote 49] To address this, we focused on the extent to which DOD's annual report to Congress addressed the following provisions of the act: (1) describe milestones and actual performance against specified measures and any revisions, (2) discuss specific improvements in business operations and cost savings resulting from successful business system modernization efforts, (3) describe specific actions on each business system investment submitted for certification, and (4) identify any business system investment with an obligation in excess of $1 million that was not certified during the preceding fiscal year and reasons for the waiver. Our methodology relative to each of the four requirements is as follows: * To determine whether the DOD annual report described milestones and actual performance against specified measures and any revisions, we compared information contained in the annual report to what the act required. Further, we compared the types of measures included in the annual report to those commonly associated with program performance, such as those described in prior GAO work related to performance measures.[Footnote 50] In addition, we interviewed officials from the Business Transformation Agency (BTA) and each of DOD's investment review boards (IRB) to understand the process used to identify and track milestones and other performance measures. We did not independently validate the accuracy of the milestone dates included in the report. * To determine the extent to which DOD's annual report discussed specific improvements in business operations and cost savings, we reviewed each of the 18 case-in-point narratives included in the annual report that described examples of business improvements and other benefits. We compared this information with the act's reporting requirements to identify any variances. We did not validate the accuracy of the improvements or benefits discussed in the case-in- point narratives. * To determine the extent to which DOD's annual report identified specific actions on each business systems investment submitted for certification, we reviewed and analyzed all Defense Business Systems Management Committee (DBSMC) certification approval memoranda as well as IRB certification memoranda and IRB meeting minutes issued prior to the DBSMC's final approval decisions for fiscal year 2009 and compared the results to those certification actions described in the annual report to identify differences. We also reviewed DOD IRB guidance to understand the types of actions related to certification of business system modernizations. For certification actions included in the DBSMC and IRB memoranda but not described in the annual report, we interviewed officials from the BTA, IRBs, the Office of the Assistant Secretary of Defense (Networks and Information Integration)/DOD Chief Information Officer (ASD(NII)/ DOD CIO), and the Office of the DOD Deputy Chief Management Officer (DCMO) as to the reason for the differences. For certification actions included in the report and described in fiscal year 2009 DBSMC and IRB memoranda, we compared information about specific DOD programs from recent GAO reports to the conditions associated with certification actions described in the annual report and the DBSMC and IRB memoranda to determine whether IRBs placed certification conditions related to program weaknesses identified by GAO and whether those conditions addressed those weaknesses.[Footnote 51] In addition, we interviewed DCMO, BTA, and IRB staff to discuss conditions that were reported as part of certification actions and what is submitted to the IRBs when individual systems request certification. * To determine if DOD's annual report identified any business system investment with an obligation in excess of $1 million that was not certified during the preceding fiscal year and the reasons for any waivers granted, we reviewed DBSMC and IRB certification memoranda and compared actions taken during fiscal year 2009 to the actions described in DOD's annual report. We also interviewed DCMO and BTA officials, as well as IRB support staff, to determine if any waivers were issued during fiscal year 2009. Finally, we reviewed DOD's annual reports from 2005 to present to determine the extent to which these reports identify any waivers issued prior to fiscal year 2009. We conducted this performance audit at DOD offices in Arlington, Virginia, from January 2010 to May 2010, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Comments from the Department of Defense: Department of Defense: Office Of The Deputy Chief Management Officer: 9010 Defense Pentagon: Washington, DC 20301-9010: May 14 2010: Mr. Randolph C. Hite: Director, Information Technology Architecture and Systems Issues: U.S. Government Accountability Office: 441 G Street, N.W. Washington, DC 20548: Dear Mr. Hite: This is the Department of Defense (DOD) response to the GAO draft report GAO-10-663, "Business Systems Modernization: Scope and Content of DoD's Congressional Report and Executive Oversight of Investments Need to Improve" dated May 4, 2010 (GAO Code 310682). Sincerely, Signed by: Elizabeth A. McGrath: Assistant Deputy Chief Management Officer: [End of letter] GAO Draft Report — Dated May 4, 2010: GAO Code 310682/GA0-10-663: "Business System Modernization: Scope and Content of DoD's Congressional Report and Executive Oversight of Investments Need to Improve" Department Of Defense Comments To The Recommendations: Recommendation 1: The GAO recommends that the Secretary of Defense direct the Deputy Secretary of Defense, as the chair of the Defense Business Systems Management Committee (DBSMC), to ensure that the scope and content of future DoD annual reports to Congress on compliance with section 332 of the National Defense Authorization Act (NDAA) of 2005, as amended, be expanded to include: * Cost, capability, and benefit performance measures for each business system modernization investment and actual performance against these measures. * All certification actions, as defined in DoD guidance, which were taken in the previous year by the Department on its business system modernization investments. DOD Response: Concur. Consistent with the legislative requirements for the Strategic Management Plan (SMP), the Department agrees that capturing information on program performance against cost, capability, and benefit measures would better inform Congress in its oversight of business system modernization programs. The Department is committed to capturing this information and is actively considering whether the most appropriate forum is the Enterprise Transition Plan (ETP) or the annual Congressional report on defense business operations. Once the Department has made this determination. DoD will incorporate the information incrementally, beginning with the Tier I business systems, and will subsequently include Tier II and Tier III business systems. The Department also concurs with the GAO's recommendation to include all certification actions as defined in its guidance, and the Department has broadened the information provided in its Congressional Report since DoD issued its inaugural report in March 2006. As required by statute, Congressional Reports have only counted the total number of new certifications and would not include re-certifications unless there was a change to milestones and performance measures. This year's report added an additional level of detail regarding certifications and included decertification information. DoD did not include recertification in this year's report, but will include them in future reports. Recommendation 2: The GAO recommends that the Secretary of Defense direct the Deputy Secretary of Defense to ensure that DoD guidance be revised to include provisions that require Investment Review Board (ERB) certification submissions disclose program weaknesses raised by GAO and the status of actions to address our recommendations to correct the weaknesses. DOD Response: Concur. DoD agrees with the GAO recommendation to require Investment Review Board (IRB) Certification submissions to disclose program weaknesses raised by GAO and the status of actions to address GAO's recommendations to correct the weaknesses, if applicable or available. At this time, an update to the January 2009 version of the DoD Information Technology (IT) Defense Business Systems Investment Review Process Guidance (IRB Guidance) is in progress, and the Department will include this requirement in the revision. However, it should be noted that the current IRB Guidance does not preclude a program from submitting additional information such as program weaknesses raised by GAO. Current guidance provides: "Pre- Certification Authorities (PCAs) are responsible for internal Certification and review of system modernization funding requests by Program Managers (PMs), in addition to ensuring that requests are submitted to the IRB with complete, current and accurate documentation and within the prescribed deadlines." PCA memoranda are also intended to acknowledge additional program issues that cannot be addressed within the confines of the DoD Information Technology Portfolio Repository (DITPR) dashboard. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Randolph Hite, (202) 512-3439 or hiter@gao.gov: Staff Acknowledgments: In addition to the contact person named above, key contributors to this report were Carl Barden, Justin Booth, Nancy Glover, Michael Holland, Neelaxi Lakhmani (Assistant Director), Kate Nielsen, Constantine Papanastasiou, Christine San, Sylvia Shanks, Jennifer Stavros-Turner, and Adam Vodraska. [End of section] Footnotes: [1] Business systems support DOD's business operations such as civilian personnel, finance, health, logistics, military personnel, procurement, and transportation. [2] GAO, High-Risk Series: An Update, [hyperlink, http://www.gao.gov/products/GAO-09-271] (Washington, D.C.: Jan. 22, 2009). [3] Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct. 28, 2004), as amended (10 U.S.C. § 2222). [4] The act requires designated approval authorities to certify that a defense business system modernization is (1) in compliance with the enterprise architecture, (2) necessary to achieve critical national security capability or address a critical requirement in an area such as safety or security, or (3) necessary to prevent a significant adverse effect on a project that is needed to achieve an essential capability, taking into consideration the alternative solutions for preventing such an adverse effect. [5] 10 U.S.C. § 2222. [6] See for example, GAO, Military Readiness: DOD Needs to Strengthen Management and Oversight of the Defense Readiness Reporting System, [hyperlink, http://www.gao.gov/products/GAO-09-518] (Washington, D.C.: Sept. 25, 2009) and DOD Business Systems Modernization: Recent Slowdown in Institutionalizing Key Management Controls Needs to Be Addressed, [hyperlink, http://www.gao.gov/products/GAO-09-586] (Washington, D.C.: May 18, 2009). [7] Congress provided DOD with about $661 billion in appropriations for fiscal year 2010. [8] GAO, Business Systems Modernization: DOD Continues to Improve Institutional Approach, but Further Steps Needed, [hyperlink, http://www.gao.gov/products/GAO-06-658] (Washington, D.C.: May 15, 2006). [9] See, for example, GAO, DOD Business Systems Modernization: Planned Investment in Navy Program to Create Cashless Shipboard Environment Needs to Be Justified and Better Managed, [hyperlink, http://www.gao.gov/products/GAO-08-922] (Washington, D.C.: Sept. 8, 2008); DOD Travel Cards: Control Weaknesses Resulted in Millions of Dollars of Improper Payments, [hyperlink, http://www.gao.gov/products/GAO-04-576] (Washington, D.C.: June 9, 2004); Military Pay: Army National Guard Personnel Mobilized to Active Duty Experienced Significant Pay Problems, [hyperlink, http://www.gao.gov/products/GAO-04-89] (Washington, D.C.: Nov. 13, 2003); and Defense Inventory: Opportunities Exist to Improve Spare Parts Support Aboard Deployed Navy Ships, [hyperlink, http://www.gao.gov/products/GAO-03-887] (Washington, D.C.: Aug. 29, 2003). [10] [hyperlink, http://www.gao.gov/products/GAO-09-271]. [11] These eight high-risk areas include DOD's overall approach to business transformation, business systems modernization, financial management, the personnel security clearance program, supply chain management, support infrastructure management, weapon systems acquisition, and contract management. [12] The seven governmentwide high-risk areas include disability programs, ensuring the effective protection of technologies critical to U.S. national security interests, interagency contracting, information systems and critical infrastructure, information sharing for homeland security, human capital, and real property. [13] Pub. L. No. 110-181, §904, 122 Stat. 3, 273 (Jan. 28, 2008). [14] An enterprise architecture, or modernization blueprint, provides a clear and comprehensive picture of an entity, whether it is an organization (e.g., federal department or agency) or a functional or mission area that cuts across more than one organization (e.g., financial management). This picture consists of snapshots of the enterprise's current or "as is" operational and technological environment and its target or "to be" environment, and contains a capital investment road map for transitioning from the current to the target environment. These snapshots consist of "views," which are basically one or more architecture products that provide conceptual or logical representations of the enterprise. [15] According to DOD, the business mission area is responsible for ensuring that capabilities, resources, and materiel are reliably delivered to the warfighter. Specifically, the business mission area addresses areas such as real property and human resources management. [16] The act requires designated approval authorities to certify that a defense business system modernization is (1) in compliance with the enterprise architecture, (2) necessary to achieve critical national security capability or address a critical requirement in an area such as safety or security, or (3) necessary to prevent a significant adverse effect on a project that is needed to achieve an essential capability, taking into consideration the alternative solutions for preventing such an adverse effect. [17] These IRBs are for Financial Management, Weapon Systems Lifecycle Management and Materiel Supply and Services Management, Real Property and Installations Lifecycle Management, and Human Resources Management. In August 2009, DOD's Enterprise Guidance Board was chartered as the DOD CIO IRB. [18] Pub. L. No. 110-181, §904. [19] Pub. L. No. 110-417, §908, 122 Stat. 4356, 4569 (Oct. 14, 2008). [20] Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct. 28, 2004), as amended (10 U.S.C. § 2222). [21] The act requires designated approval authorities to certify that a defense business system modernization is (1) in compliance with the enterprise architecture, (2) necessary to achieve critical national security capability or address a critical requirement in an area such as safety or security, or (3) necessary to prevent a significant adverse effect on a project that is needed to achieve an essential capability, taking into consideration the alternative solutions for preventing such an adverse effect. [22] GAO, DOD Business Systems Modernization: Progress in Establishing Corporate Management Controls Needs to Be Replicated Within Military Departments, [hyperlink, http://www.gao.gov/products/GAO-08-705] (Washington, D.C.: May 15, 2008); DOD Business Systems Modernization: Progress Continues to Be Made in Establishing Corporate Management Controls, but Further Steps Are Needed, [hyperlink, http://www.gao.gov/products/GAO-07-733] (Washington, D.C.: May 14, 2007); Business Systems Modernization: DOD Continues to Improve Institutional Approach, but Further Steps Needed, [hyperlink, http://www.gao.gov/products/GAO-06-658] (Washington, D.C.: May 15, 2006); and DOD Business Systems Modernization: Important Progress Made in Establishing Foundational Architecture Products and Investment Management Practices, but Much Work Remains, [hyperlink, http://www.gao.gov/products/GAO-06-219] (Washington, D.C.: Nov. 23, 2005). [23] [hyperlink, http://www.gao.gov/products/GAO-09-586]. [24] According to the department, fiscal year 2009 planned milestones are described in DOD's September 2008 Enterprise Transition Plan. [25] Key milestones in the DOD acquisition process are Milestone A, Milestone B, Milestone C, Initial Operational Capability, and Full Operational Capability. These are decision points to determine whether to initiate, continue, advance, change direction in, or terminate a project or program work effort, which result in advancement to or restriction from entering the next major acquisition process phase. [26] According to DOD's annual report, DOD does not consider programs at or beyond the initial operating capability phase and that are no longer modernizing to be modernization programs and does not report related milestones and performance measures in its fiscal year 2009 enterprise transition plan or March 2010 congressional report. [27] DOD includes more than one fiscal year 2009 milestone for some business systems, while other business systems did not have any planned milestones during fiscal year 2009. [28] The vast majority of business systems either do not involve modernizations or are small investments (i.e., less than $1 million) that are not subject to the IRB certification and DBSMC approval processes. For purposes of reporting milestones in its fiscal year 2010 annual report, DOD only includes programs that have been approved by the DBSMC and that had planned milestones for fiscal year 2009. [29] GAO, Defense Acquisitions: Measuring the Value of DOD's Weapon Programs Requires Starting with Realistic Baselines, [hyperlink, http://www.gao.gov/products/GAO-09-543T] (Washington, D.C.: Apr. 1, 2009); DOD Business Systems Modernization: Planned Investment in Navy Program to Create Cashless Shipboard Environment Needs to Be Justified and Better Managed, [hyperlink, http://www.gao.gov/products/GAO-08-922] (Washington, D.C.: Sept. 8, 2008); Managing for Results: Enhancing Agency Use of Performance Information for Management Decision Making, [hyperlink, http://www.gao.gov/products/GAO-05-927] (Washington, D.C.: Sept. 9, 2005); Information Technology: Customs Automated Commercial Environment Program Progressing, but Need for Management Improvements Continues, [hyperlink, http://www.gao.gov/products/GAO-05-267] (Washington, D.C.: Mar. 14, 2005); Information Technology: DOD's Acquisition Policies and Guidance Need to Incorporate Additional Best Practices and Controls, [hyperlink, http://www.gao.gov/products/GAO-04-722] (Washington, D.C.: July 30, 2004); Executive Guide Improving Mission Performance Through Strategic Information Management and Technology, [hyperlink, http://www.gao.gov/products/GAO/AIMD-94-115] (Washington, D.C.: May 2004). [30] According to DOD, the Army Learning Management System is a subset of the Army Distributed Learning System, which supports IT infrastructure for delivering distributed learning content for training. [31] An ERP solution is an automated system using commercial off-the- shelf software consisting of multiple, integrated functional modules that perform a variety of business-related tasks such as payroll, general ledger accounting, and supply chain management. [32] The approval authorities, as discussed earlier in this report, include the Under Secretary of Defense for Acquisition, Technology, and Logistics; the Under Secretary of Defense (Comptroller); the Under Secretary of Defense for Personnel and Readiness; the ASD(NII)/DOD CIO; and the Deputy Secretary of Defense. They are responsible for the review, approval, and oversight of business systems and must establish investment review processes for systems under their cognizance. [33] The act requires certification by designated approval authorities that the defense business system modernization is (1) in compliance with the enterprise architecture, (2) necessary to achieve critical national security capability or address a critical requirement in an area such as safety or security, or (3) necessary to prevent a significant adverse effect on a project that is needed to achieve an essential capability, taking into consideration the alternative solutions for preventing such an adverse effect. [34] Investments that are below the $1 million threshold but have been designated as an interest program by an IRB are also subject to IRB review and DBSMC approval. [35] Among other things, DITPR, which is the department's authoritative business systems inventory, contains information about certifications, including any certification conditions placed on the system. [36] The NDAA for Fiscal Year 2010 adds the requirement that a component's chief management officer assert that new investments have undertaken sufficient business process reengineering efforts. Since DOD's March 2010 report focuses on fiscal year 2009, our report does not include information about this new requirement. [37] The Standard Financial Information Structure is intended to provide a standardized DOD-wide financial information structure to improve cost accounting, analysis, and reporting. [38] The Item Unique Identification Registry is a relational database that is intended to store acquisition and logistics information to track, catalog, and inventory items, such as equipment and spare parts, via machine-readable item identifiers. [39] DOD's reported numbers combine multiple certification actions that took place on individual systems within specific categories of certification actions. For example, if a system is certified twice, both times without conditions, within fiscal year 2009, this system is counted only once under the category of certification without conditions. For the purposes of comparing our analysis to DOD reported data, we also combined multiple certification actions that took place on individual systems within specific categories of certification actions. [40] Although DOD provided additional documentation indicating the four system certifications might be recertifications, the information was not consistent with the DBSMC and IRB decision memoranda and meeting minutes. [41] [hyperlink, http://www.gao.gov/products/GAO-08-972]. [42] As described previously in this report, DOD does not report on the results of recertifications in its annual report. [43] DRRS is intended to provide timely, objective, and accurate data about DOD force readiness. [44] [hyperlink, http://www.gao.gov/products/GAO-09-518]. [45] GCSS-MC is intended to modernize the Marine Corps logistics systems and to provide the Corps with timely and complete logistics information to support the warfighter. [46] GAO, DOD Business Systems Modernization: Key Marine Corps System Acquisition Needs to be Better Justified, Defined, and Managed, [hyperlink, http://www.gao.gov/products/GAO-08-822] (Washington, D.C.: July 28, 2008). [47] GAO, DOD Business Systems Modernization: Important Management Controls Being Implemented on Major Navy Program, but Improvements Needed in Key Areas, [hyperlink, http://www.gao.gov/products/GAO-08-896] (Washington, D.C.: Sept. 8, 2008). [48] According to DOD, the milestone decision authority is the designated individual who has overall responsibility for an investment. This person has the authority to approve an investment's progression in the acquisition process and is responsible for reporting cost, schedule, and performance results. For example, the milestone decision authority for a Major Automated Information System is the ASD(NII)/DOD CIO or a designee. [49] Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct. 28, 2004), as amended (10 U.S.C. § 2222). [50] GAO, Managing for Results: Enhancing Agency Use of Performance Information for Management Decision Making, [hyperlink, http://www.gao.gov/products/GAO-05-927] (Washington, D.C.: Sept. 9, 2005); Information Technology: Customs Automated Commercial Environment Program Progressing, but Need for Management Improvements Continues, [hyperlink, http://www.gao.gov/products/GAO-05-267] (Washington, D.C.: Mar. 14, 2005); Information Technology: DOD's Acquisition Policies and Guidance Need to Incorporate Additional Best Practices and Controls, [hyperlink, http://www.gao.gov/products/GAO-04-722] (Washington, D.C.: July 30, 2004); and Executive Guide Improving Mission Performance Through Strategic Information Management and Technology, [hyperlink, http://www.gao.gov/products/GAO/AIMD-94-115] (Washington, D.C.: May 2004). [51] GAO, DOD Business Systems Modernization: Planned Investment in Navy Program to Create Cashless Shipboard Environment Needs to Be Justified and Better Managed, [hyperlink, http://www.gao.gov/products/GAO-08-922] (Washington, D.C.: Sept. 8, 2008); DOD Business Systems Modernization: Important Management Controls Being Implemented on Major Navy Program, but Improvements Needed In Key Areas, [hyperlink, http://www.gao.gov/products/GAO-08-896] (Washington, D.C.: Sept. 8, 2008); DOD Business Systems Modernization: Key Navy Programs' Compliance with DOD's Federated Business Enterprise Architecture Needs to Be Adequately Demonstrated, [hyperlink, http://www.gao.gov/products/GAO-08-972] (Washington, D.C.: Aug. 7, 2008); and DOD Business Systems Modernization: Lack of an Integrated Strategy Puts the Army's Asset Visibility System Investments at Risk, [hyperlink, http://www.gao.gov/products/GAO-07-860] (Washington, D.C.: July 27, 2007). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: