This is the accessible text file for GAO report number GAO-14-486 entitled 'Defense Business Systems: Further Refinements Needed to Guide the Investment Management Process' which was released on May 12, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Committees: May 2014: Defense Business Systems: Further Refinements Needed to Guide the Investment Management Process: GAO-14-486: GAO Highlights: Highlights of GAO-14-486, a report to congressional committees. Why GAO Did This Study: GAO designated DOD's multibillion-dollar business systems modernization program as high risk in 1995, and since then has provided a series of recommendations aimed at strengthening DOD's institutional approach to modernizing its business systems investments. Section 332 of the Fiscal Year 2005 NDAA requires the department to take specific actions relative to its modernization efforts and GAO to assess actions taken by DOD to comply with section 332 of the act. In evaluating DOD's compliance, GAO analyzed, among other things, investment management policies and procedures, certification actions for business system investments, and the latest versions of the department's business enterprise architecture and enterprise transition plan. What GAO Found: The Department of Defense (DOD) has developed a portfolio-based investment management process for its defense business systems, certified and approved a majority of its defense business systems, made key improvements to the data used to manage its business investments, and updated its transition plan to assist its efforts in complying with key provisions of section 332 of the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 (NDAA or “the act”), as amended (10 U.S.C. § 2222). However, the department continues to face challenges in fully complying with the act's requirements, modernizing its business systems environment, and addressing GAO's prior recommendations (see table). Table: Status of DOD's Actions Related to the Act's Requirements: Act's requirements: Establish an investment approval and accountability structure, along with an investment review process; Status of related DOD actions: DOD has developed a defense business system investment management framework that reflects key aspects of capital planning and investment control best practices. However, this process is not currently aligned with DOD's budget process and does not call for the department-wide investment review board to focus its attention on more costly, complex, or risky systems, while delegating detailed reviews of other systems to lower organizational levels. Act's requirements: Certify that any business system program costing more than $1 million complies with the business enterprise architecture and has undertaken appropriate business process reengineering; Status of related DOD actions: DOD certified about $6.4 billion in fiscal year 2014 expenditures for 1,173 defense business systems and has improved the information it collects about business system certifications. However, while most systems were certified as complying with the act's requirements, reviews supporting these certifications need to be improved, as GAO has previously recommended. In addition, reviewing systems at differing organizational levels based on cost, complexity, risk, or other specified criteria may help address these issues. Act's requirements: Develop a business enterprise architecture; Status of related DOD actions: DOD has efforts under way to improve the content of its business enterprise architecture. However, the department's guidance does not require components to proactively use the information collected as part of this process to reduce duplication and overlap of business systems. Act's requirements: Develop an enterprise transition plan; Status of related DOD actions: DOD's enterprise transition plan and related documentation include most of information required by the act, such as milestone information, fiscal year 2014 resource needs, and termination dates for legacy systems. Continued effort to address GAO' s prior recommendations in this area will help to better inform oversight of the department's business systems investments. Source: GAO analysis of DOD data. [End of table] Department officials cited various reasons for the shortfalls noted above. For example, they stated that aligning the investment review process with the budgeting process takes time to coordinate. They also noted that different systems are reviewed with different levels of scrutiny based on various factors, but those factors are not documented in existing guidance. Continued progress in improving its investment management approach will allow DOD to more effectively manage the billions of dollars the department invests annually in modernizing its business systems. What GAO Recommends: GAO is making three recommendations to help improve the department's business system investment management process and business enterprise architecture. DOD concurred with two recommendations and partially concurred with the third—to define criteria for reviewing business systems at appropriate levels in the department. In particular, DOD noted that systems are reviewed within components before being reviewed by the executive-level investment review board. However, until DOD ensures that investments are reviewed at an appropriate level based on defined criteria such as cost, scope, complexity, and risk, the department is at an increased risk of failing to identify and address important issues associated with large-scale and costly systems. View GAO-14-486. For more information, contact Carol R. Cha at (202) 512-4456 or chac@gao.gov. [End of section] Contents: Letter: Background: DOD Has Addressed Most Provisions of the Act, but Needs to Continue Improving Its Investment Management Approach: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objective, Scope, and Methodology: Appendix II: Comments from the Department of Defense: Appendix III: GAO Contact and Staff Acknowledgments: Tables: Table 1: DOD Business Systems Modernization Governance Entities' Selected Roles, Responsibilities, and Composition: Table 2: Directorates and Key Responsibilities of the Office of the DCMO: Table 3: Assessment of DOD's Guidance against Key Practices for Instituting an IRB: Table 4: Certification Summary for Fiscal Year 2014 (as of February 2014; dollars in millions): Table 5: Description of Key Data Sources: Figure: Figure 1: DOD's Defense Business Systems by Functional Area (as of November 5, 2013): Abbreviations: BEA: business enterprise architecture: CIO: Chief Information Officer: DBC: Defense Business Council: DCMO: Deputy Chief Management Officer: DITIP: Defense Information Technology Investment Portal: DITPR: Defense Information Technology Portfolio Repository: DOD: Department of Defense: ETP: enterprise transition plan: IRB: investment review board: IT: information technology: NDAA: National Defense Authorization Act: SNAP-IT: Select and Native Programming Data Input System for Information Technology: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: May 12, 2014: Congressional Committees: For decades, the Department of Defense (DOD) has been challenged in modernizing its business systems. In 1995, we designated the department's business systems modernization program as high risk because of its vulnerability to fraud, waste, abuse, and mismanagement, and because of missed opportunities to achieve greater efficiencies; and we continue to designate it as such today.[Footnote 1] In addition, we have reported[Footnote 2] that significant potential exists for identifying and avoiding costs associated with duplicative functionality across DOD business system investments, which include approximately 2,329 systems.[Footnote 3] DOD's business systems include those for personnel, financial management, healthcare, and logistics, and cost the department billions of dollars each year. Since designating this area as high risk in 1995, we have made a series of recommendations aimed at strengthening DOD's institutional approach to modernization and reducing the risk associated with key investments.[Footnote 4] Further, Congress included provisions in the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 (NDAA),[Footnote 5] as amended, that were consistent with our recommendations. Specifically, section 332 of the act, as amended, requires the department to, among other things, (1) establish a systems investment approval and accountability structure along with an investment review process, (2) certify and approve any business system program costing in excess of $1 million, (3) develop a business enterprise architecture (BEA) to cover all defense business systems, (4) develop a transition plan for implementing the architecture, and (5) identify systems information in its annual budget submission. The act directs the Secretary of Defense to submit an annual report to the congressional defense committees on DOD's compliance with certain requirements of the NDAA not later than March 15 of each year, through 2016. The act also directed us to report to these congressional committees-- within 60 days of DOD's report submission--an assessment of the department's actions to comply with the requirements of the act. Accordingly, our objective was to assess the actions taken by DOD to comply with section 332 of the act. To accomplish the objective, we reviewed and analyzed the latest version of DOD's investment management policies and procedures, BEA, and enterprise transition plan (ETP), using our prior reports as a baseline.[Footnote 6] To address the investment management provisions of the act, we reviewed the department's report to Congress, which was submitted on March 14, 2014, analyzed the most recent investment management guidance, and compared this information against the law and our information technology (IT) investment management guidance. To determine the extent to which DOD has certified and approved business systems, we reviewed certification and approval memoranda, related documentation, and associated data, and compared them to the act's requirements and our previous recommendations. To assess DOD's progress in developing the BEA and ETP, we reviewed the most recent versions of the BEA and plan, and compared this information against the act's requirements and our prior recommendations. To assess the system information in the budget submission, we reviewed summary budget data prepared for the fiscal year 2015 submission, and compared them to the requirements in the act. We conducted this performance audit from October 2013 to May 2014, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. Additional details on our objective, scope, and methodology are contained in appendix I. Background: DOD is one of the largest and most complex organizations in the world. In support of its military operations, the department performs an assortment of interrelated and interdependent business functions, such as logistics management, procurement, health care management, and financial management. Yet, we have previously reported that the DOD systems environment that supports these business functions is overly complex and error prone, and is characterized by (1) little standardization across the department, (2) multiple systems performing the same tasks, (3) the same data stored in multiple systems, and (4) the need for data to be entered manually into multiple systems. [Footnote 7] For fiscal year 2015, the department requested about $10.038 billion for its business system investments. According to the department's authoritative source for certification data,[Footnote 8] its environment is composed of approximately 2,329 business systems, including 228 for acquisition, 14 for defense security enterprise, 28 for enterprise IT infrastructure, 286 for financial management, 730 for human resources management, 293 for installations and environment, 702 for logistics and materiel readiness, and 8 for security cooperation.[Footnote 9] (See figure 1.) Of these 2,329 business systems, 1,180 are covered by the act's certification and approval requirements. Figure 1: DOD's Defense Business Systems by Functional Area (as of November 5, 2013): [Refer to PDF for image: pie-chart] Human resources management: 31%; Logistics and materiel readiness: 30%; Installation and environment: 13%; Financial management: 12%; Acquisition: 10%; Other: 2%; Enterprise IT infrastructure: 1%; Defense security enterprise: 1%; Security cooperation: less than 1%. Source: GAO analysis of DOD data. [End of figure] DOD currently bears responsibility, in whole or in part, for 15 of the 30 programs across the federal government that we have designated as high risk.[Footnote 10] Seven of these areas are specific to the department,[Footnote 11] and eight other high-risk areas are shared with other federal agencies.[Footnote 12] Collectively, these high- risk areas relate to DOD's major business operations that are inextricably linked to the department's ability to perform its overall mission and directly affect the readiness and capabilities of U.S. military forces and can affect the success of a mission. In particular, the department's nonintegrated and duplicative systems impair its ability to combat fraud, waste, and abuse.[Footnote 13] Consequently, DOD's business systems modernization is one of the department's specific high-risk areas and is essential for addressing many of the department's other high-risk areas. For example, modernized business systems are integral to the department's efforts to address its financial, supply chain, and information security management high-risk areas. Summary of NDAA Requirements: Congress included provisions in the NDAA, as amended, that are aimed at ensuring DOD's development of a well-defined BEA and associated ETP, as well as the establishment and implementation of effective investment management structures and processes.[Footnote 14] The act requires DOD to, among other things, * establish an investment approval and accountability structure along with an investment review process, * not obligate funds for a defense business system program with a total cost in excess of $1 million unless the approval authority certifies that the business system program meets specified conditions, [Footnote 15] * develop a BEA that covers all defense business systems, * develop an ETP for implementing the architecture, and: * identify systems information in DOD's annual budget submissions. The act also requires that the Secretary of Defense annually submit to the congressional defense committees a report on the department's compliance with these provisions. DOD submitted its annual report to Congress in March 2014, describing steps taken, under way, and planned to address the act's requirements. DOD's Approach to Business Systems Modernization: As of April 2014, the department's approach to modernizing its business systems environment, which is part of DOD's overall effort to transform its business operations, included improving business systems investment management, reengineering the business processes supported by its defense business systems, and improving and using the BEA and associated ETP. These efforts are to be guided by DOD's Chief Management Officer and Deputy Chief Management Officer (DCMO). [Footnote 16] Specifically, the Chief Management Officer's responsibilities include developing and maintaining a department-wide strategic plan for business reform; establishing performance goals and measures for improving and evaluating overall economy, efficiency, and effectiveness; and monitoring and measuring the progress of the department. The DCMO's responsibilities include recommending to the Chief Management Officer methodologies and measurement criteria to better synchronize, integrate, and coordinate the business operations to ensure alignment in support of the warfighting mission and developing and maintaining the department's enterprise architecture for its business mission area.[Footnote 17] DOD has assigned roles and responsibilities to various governance entities and positions related to business systems modernization. For example, the Deputy's Management Action Group is a senior-level forum that meets several times a month to discuss department-wide management issues, including business-related topics. This group is to convene as the Defense Business Systems Management Committee when it reviews defense business system portfolios.[Footnote 18] Under this committee, the Defense Business Council (DBC) acts as a corporate-level investment review board (IRB) by overseeing the approach and guidance for selecting and controlling the investment portfolio and making recommendations on funds certifications. Table 1 describes selected roles and responsibilities and composition of key governance entities and positions related to business systems modernization. Table 1: DOD Business Systems Modernization Governance Entities' Selected Roles, Responsibilities, and Composition: Entity: Deputy's Management Action Group/Defense Business Systems Management Committee[A]; Roles and responsibilities: * Provide strategic direction and plans for the business mission area; * Recommend policies and procedures required to integrate DOD business transformation and attain cross-department, end-to-end interoperability of business systems and processes; * Review defense business system portfolios; * Serve as approval authority for business system investment certifications greater than $1 million; * Establish policies and approve the business mission area strategic plan, the ETP for implementation of business systems modernization, and the BEA; Composition: Meets at the discretion of the Deputy Secretary of Defense and is co-chaired by the Vice Chairman of the Joint Chiefs of Staff. Includes senior leadership in the Office of the Secretary of Defense, as appropriate, such as the DCMO and the DOD Chief Information Officer. Also includes the military department chief management officers, the heads of select defense agencies, and participation by other senior management, including from the Joint Chiefs of Staff and the U.S. Transportation Command. Entity: DBC/IRB; Roles and responsibilities: * Oversee DOD's investment management process and conduct portfolio analysis in support of the review and certification of covered defense business system programs[B]; * Review functional strategies developed by the principal staff assistants and assess component organizational execution plans; * Recommend funds certification to the Deputy's Management Action Group/Defense Business Systems Management Committee; * Prioritize and approve changes for inclusion in the BEA; * Support the development and implementation of the department's end- to-end framework; Composition: Chaired by the DOD DCMO. Includes, for example, the assistant secretaries of defense (Acquisition, Logistics and Materiel Readiness), Office of the Joint Chiefs of Staff, DOD Chief Information Officer, Deputy Chief Financial Officer and Deputy Comptroller (Programs/Budgets), and military departments. Entity: Principal Staff Assistants; Roles and responsibilities: * Senior advisors to the Secretary of Defense that assist in policy development, planning, resource management, fiscal, and program evaluation responsibilities; * Develop functional strategies that are to describe business functions, outcomes, measures, and standards for their respective business areas[C]; * Responsible and accountable for the content of their portions of the BEA; Composition: Composed of the Under Secretaries of Defense for defined functional areas (e.g., Comptroller, Acquisition, Technology, and Logistics, Intelligence, Policy, and Personnel and Readiness) and the DOD Chief Information Officer. Entity: Precertification Authority; Roles and responsibilities: * Ensure component-level investment review processes integrate with the investment management system; * Identify those component systems that require IRB certification and prepare, review, approve, validate, and transfer investment documentation as required; * Assess and determine if appropriate business process reengineering and architecture compliance have been undertaken for component systems submitted for certification and annual review; * Ensure that the components' portfolios of defense business systems are aligned to the business outcomes in the department's functional strategies and Strategic Management Plan; Composition: Chief Management Officer from the Air Force, Army, and Navy; the director or the equivalent from the defense agencies; and those designated by the DOD DCMO as appropriate for programs that support the business processes of more than one military department or defense agency. Entity: Office of the DCMO; Roles and responsibilities: * Update and facilitate the use of the defense business system investment review process; * Conduct and execute investment portfolio reviews; * Develop guidance for integrating the end-to-end business processes into the Integrated Business Framework; * Build and deliver the department's BEA, Strategic Management Plan, and ETP; * Conduct organizational assessments and develop guidance to set forth the priority performance outcomes for the department; Composition: Composed of five directorates (Investment and Acquisition Management, Enterprise Business Integration, Technology and Innovation, Planning and Performance Management, and Operations). Source: GAO analysis of DOD information. [A] The Deputy's Management Action Group is the forum for determining management actions on topics chosen by the Deputy Secretary of Defense; it meets as the Defense Business System Management Committee when considering business system topics. When meeting as the Defense Business Systems Management Committee, the group is chaired by the Deputy Secretary of Defense with the Deputy Chief Management Officer serving as the Vice Chair. [B] A covered defense business system is any defense business system program that is expected to have a total cost in excess of $1 million over the period of the current Future-Years Defense Program, which is the department's financial plan over a 6-year period. [C] DOD has eight functional areas: Acquisition, Defense Security Enterprise, Enterprise IT Infrastructure, Financial Management, Human Resources Management and Health Management, Installation and Environment, Logistics and Materiel Readiness, and Security Cooperation. [End of table] Overview of DOD's Integrated Business Framework: In order to manage and oversee the department's business operations and approximately 1,180 covered defense business systems, the Office of the DCMO developed its Integrated Business Framework.[Footnote 19] According to the Office of the DCMO, this framework is used to align the department's strategic objectives--laid out in the National Security Strategy,[Footnote 20] Quadrennial Defense Review,[Footnote 21] and Strategic Management Plan[Footnote 22]--to its defense business system investments. Using the overarching goals of the Strategic Management Plan, principle staff assistants develop six functional strategies that cover eight functional areas. These functional strategies define business outcomes, priorities, measures, and standards for a given functional area within DOD. The functional areas are: * Acquisition, * Defense Security Enterprise, * Enterprise IT Infrastructure, * Financial Management, * Human Resources Management and Health Management, * Installations and Environment, * Logistics and Materiel Readiness, and: * Security Cooperation. The business objectives and compliance requirements laid out in each functional strategy are to be integrated into the BEA. The precertification authorities in the Air Force, Navy, Army, and other departmental organizations use the functional strategies to guide the development of organizational execution plans, which are to summarize each component's business strategy for each functional area. Each plan includes a description of how the component's goals and objectives align with those in the functional strategies and the Strategic Management Plan. In addition, each organizational execution plan includes a portfolio of defense business system investments organized by functional area. The components submit each of these portfolios of systems to the DBC for certification on an annual basis. According to DOD's investment management guidance, the DBC reviews the organizational execution plans and associated portfolios based on four investment criteria--compliance, strategic alignment, utility, and cost--to determine whether or not to recommend the portfolio for certification of funding. The Vice Chairman of the Defense Business Systems Management Committee approves certification decisions and then documents those decisions in investment decision memoranda. These memoranda state whether an individual organizational execution plan has been certified, conditionally certified (i.e., obligation of funds has been certified and approved but may be subject to conditions that restrict the use of funds, a time line for obligation of funds, or mandatory changes to the portfolio of business systems), or not certified (i.e., certification is not approved due to misalignment with strategic direction, mission needs, or other deficiencies). The Office of the DCMO Currently Manages Business Systems, but a Planned Reorganization May Change These Responsibilities: The Office of the DCMO manages the day-to-day aspects of department- level business system oversight. It is currently composed of five directorates with various responsibilities related to business systems modernization. Table 2 shows the organizational components of the Office of the DCMO and key responsibilities. Table 2: Directorates and Key Responsibilities of the Office of the DCMO: Directorate: Investment and Acquisition Management Directorate; Key responsibilities: * Report to Congress on progress and improvements made in the DOD Business Mission Area; * Develop the ETP; * Conduct and execute investment portfolio reviews. Directorate: Enterprise Business Integration Directorate; Key responsibilities: * Review and integrate the department's functional strategies, identifying relevant cross-functional challenges, risks, and/or interdependencies; * Review and coordinate problem statement submissions; * Develop and facilitate end-to-end guidance and methodology integration into the Integrated Business Framework. Directorate: Technology and Innovation Directorate; Key responsibilities: * Build and deliver the BEA; * Develop and promote enterprise standards for business architecture; * Manage and execute BEA statutory requirements; * Build and maintain analytic capabilities in support of the DCMO and the investment management process. Directorate: Planning and Performance Management Directorate; Key responsibilities: * Develop the Strategic Management Plan; * Improve performance management infrastructure and leverage continuous process improvement methodologies; * Conduct organizational assessments and develop organizational guidance to set forth the priority performance outcomes for the department. Directorate: Operations Directorate; Key responsibilities: * Manage the day-to-day operations of the Office of the Deputy Chief Management Officer (e.g., human resources and budgeting). Source: GAO analysis based on DOD documentation. [End of table] These responsibilities may shift as a result of planned organizational changes. More specifically, on December 4, 2013, the Secretary of Defense issued a memorandum that outlined the results of an organizational review completed by the Office of the Secretary of Defense. Among other things, the Secretary called for strengthening the office of the DOD Chief Information Officer (CIO) by moving the oversight of business systems from the DCMO to the CIO, and for strengthening the Office of the DCMO to better coordinate the department's business affairs. According to the Director of Investment and Acquisition Management in the Office of the DCMO, final decisions about how this directive will be implemented have not yet been made. The memorandum calls for all of the changes to occur by January 2015. Overview of DOD's BEA: DOD's BEA is intended to serve as a blueprint for the department's business transformation efforts.[Footnote 23] In particular, the architecture is to guide and constrain implementation of interoperable defense business systems by, among other things, documenting the department's business functions and activities; the information needed to execute its functions and activities; the business outcomes from using the BEA; and the list of business rules, laws, regulations, and policies associated with its business functions and activities. According to DOD, its architecture is being developed using an incremental approach, where each new version of the architecture addresses business mission area gaps or weaknesses based on priorities identified by the department. The department's BEA focuses on documenting information associated with its end-to-end business process areas (e.g., hire-to-retire and procure-to-pay).[Footnote 24] The department considers its current approach to developing the BEA both a "top down" and "bottom-up" approach. Specifically, according to DOD, the architecture focuses on developing content to support investment management and strategic decision making and oversight ("top down") while also responding to department needs associated with supporting system implementation, system integration, and software development ("bottom up"). The department's approach to developing its BEA involves the development of a federated enterprise architecture, where member architectures (e.g., Air Force, Army, and Navy) conform to an overarching corporate or parent architecture and use a common vocabulary. This approach is to provide governance across all business systems, functions, and activities within the department and improve visibility across DOD's efforts. Prior GAO Reviews of DOD's Business Systems Modernization: Between 2005 and 2008, we reported that DOD had taken steps to comply with key requirements of the NDAA relative to architecture development, transition plan development, budgetary disclosure, and investment review, and to satisfy relevant systems modernization management guidance. However, each report also concluded that much remained to be accomplished relative to the act's requirements and relevant guidance.[Footnote 25] We made recommendations to address each of the areas, and DOD largely agreed with our recommendations. However, in May 2009, we reported that the pace of DOD's efforts in defining and implementing key institutional modernization management controls had slowed compared with progress made in each of the previous 4 years, leaving much to be accomplished to fully implement the act's requirements and related guidance.[Footnote 26] In addition, between 2009 and 2012, we found that progress had been made, but long- standing challenges we had previously identified remained to be addressed.[Footnote 27] For example, the department's budget submission for fiscal year 2013 did not include all systems because of the lack of a reliable, comprehensive inventory of all defense business systems. We concluded that DOD's progress in addressing the act's requirements, its vision for a federated architecture, and our related recommendations was limited, in part, by continued uncertainty surrounding the department's governance mechanisms, such as roles and responsibilities of key organizations and senior leadership positions. Accordingly, we made recommendations to address the issues identified. DOD partially agreed with our recommendations. In 2013, we reported that DOD had continued to make improvements, particularly by taking steps to establish a portfolio-based approach to reviewing and certifying its defense business systems.[Footnote 28] However, among other things, we also found that the BEA and ETP were still missing important content and BEA and business process reengineering certifications were not validated. We made additional recommendations in these areas, and the department partially agreed with our recommendations. In addition, our most recent high-risk report noted that, while DOD's capability and performance relative to business systems modernization had improved, significant challenges remained.[Footnote 29] For example, the department had not fully defined and established a family of management controls, such as corporate and component business architectures and business system investment management processes. These management controls are vital to ensuring that DOD can effectively and efficiently manage an undertaking with the size, complexity, and significance of its business systems modernization, and minimize the associated risks. Furthermore, we also recently reported that, in order to better identify and address potential duplication, DOD needed to develop supporting component architectures, align them with its corporate architecture to complete the federated BEA, and leverage its federated architecture to avoid investments that provide similar, but duplicative, functionality in support of common DOD activities.[Footnote 30] DOD Has Addressed Most Provisions of the Act, but Needs to Continue Improving Its Investment Management Approach: DOD has made progress in addressing the provisions of the NDAA and recommendations that we have made in recent years. Specifically, the department issued its March 2014 Congressional Report on Defense Business Operations, which describes updates and next steps for its investment review process; improved the data used to manage its business systems certification and approval process; issued an updated ETP in December 2013; and issued its fiscal year 2015 funding request for defense business systems. However, the department still faces challenges in complying with key parts of the act and managing its business systems. For example, the department needs to further refine its approach for reviewing system certifications so that the reviews occur at an appropriate level based on factors such as cost and complexity. Without continued progress in improving its investment management approach, DOD will be challenged in its ability to manage the billions of dollars invested annually in modernizing its business system investments. An Integrated Business Framework Guides DOD's Business System Investments, but Further Refinements Are Needed to Improve the Investment Review Process: The act, as amended by the NDAA for Fiscal Year 2012, included significant changes to the requirements for investment review and certification of defense business systems. Specifically, it required DOD to establish a department-wide IRB, chaired by the DCMO, and an investment management process. The act also called for the use of threshold criteria to ensure an appropriate level of review within the department of, and accountability for, defense business system programs depending on scope, complexity, and cost. In addition, IT investment management best practices[Footnote 31] describe key practices for instituting an IRB, including (1) grouping investments into portfolios, (2) coordinating the investment management process with other internal management controls, and (3) ensuring that systems receive an appropriate level of review by developing tiered review boards. Since we reported in May 2013,[Footnote 32] DOD has continued its efforts to establish the IRB and further define and implement its defense business system governance framework--called the Integrated Business Framework--as discussed earlier in this report. This framework is used to manage the department's business operations and investments, and includes six portfolios that align to functional areas.[Footnote 33] Of the three key practices for establishing an IRB described above, DOD's Integrated Business Framework fully addresses one practice, partially addresses one practice, and does not address one practice (see table 3). Table 3: Assessment of DOD's Guidance against Key Practices for Instituting an IRB: Practice category: Instituting investment board operations; Specific practice: The organization should group its investments into portfolios by categories. Categorization involves grouping investments and proposals into predefined logical categories such as business functions or services. Once this is accomplished, investments and proposals can be compared to one another within and across the portfolio categories; Assessment: Fully addressed; Summary of evidence: As described previously, the Integrated Business Framework includes functional strategies and organizational execution plans that are portfolio-based and centered around eight functional areas. These functional areas, such as financial management or human resources management, represent the business functions that cut across DOD's components. Specifically, the department develops functional strategies for each of these functional areas that include business outcomes and priorities. The organizational execution plans go a step further, and define the portfolios of systems that will meet the business outcomes within each component for each functional area. By categorizing its investments into portfolios, the department is better positioned to compare investments based on relative cost, schedule, benefits, and risk. Specific practice: Investment management guidance should specify the manner in which IT investment-related processes will be coordinated with other organizational plans, processes, and documents--including, at a minimum, the strategic plan, budget, and enterprise architecture processes; Assessment: Partially addressed; Summary of evidence: Practice category: DOD's investment management guidance notes that the Integrated Business Framework uses the overarching goals of DOD's Strategic Management Plan to guide the development of the functional strategies, and organizational execution plans must demonstrate how each portfolio aligns to those goals. In addition, the investment management guidance notes that functional strategies should be used to drive content changes in DOD's BEA; However, while the Office of the DCMO has taken initial steps to align the investment management process with the budgeting process, such as briefing the DBC on its vision for aligning the processes, the department does not have documented plans for following through on this vision. This means that, due to the timing of the department's budget process, requested funds are allocated to individual systems in the department's budget for a given fiscal year before precertification authorities and the DBC review certification and approval requests associated with those systems. In addition, according to DOD's March 2014 report on Defense Business Operations, the current lack of alignment has resulted in precertification authorities having limited ability to consider strategic business guidance in their investment management activities. Specific practice: Investment management guidance should ensure that systems receive the appropriate level of review by developing IRBs at multiple levels within an organization. For example, an enterprise- wide investment board may exist to define and oversee the investment management framework, while lower-level boards may exist within business or operational units. Criteria should be developed to minimize overlaps or gaps of responsibility and authority among the various boards. These criteria can be based on costs, benefits, schedule, risk, the life-cycle phase of an investment (for example, development or operations and maintenance), or the function of a business unit (for example, CIO, human resources, or a component office such as the Army); Assessment: Not addressed; Summary of evidence: The DBC currently reviews each portfolio of systems using the same set of guidance and criteria. This guidance does not ensure that systems receive the appropriate level of review by calling for the development of IRBs at multiple levels within the department and ensuring that criteria are developed to minimize overlaps or gaps of responsibility and authority. For example, existing guidance does not call for the DBC to review systems that are relatively costly--such as the Army's Logistics Modernization Program, which was certified and approved for $226.7 million in fiscal year 2014--while delegating responsibility for conducting detailed reviews for systems with significantly lower costs--such as the Air Force's Command 202 system, which was certified and approved to spend $288,000 in the same fiscal year--to lower-level IRBs. Source: GAO analysis of DOD data. [End of table] Officials from the Office of the DCMO provided several reasons why two of the key practices for instituting an IRB were not fully addressed. With respect to aligning the Integrated Business Framework with the budget process, officials stated that specific plans take time and coordination to develop and implement. These officials noted that these efforts are affected by the ongoing shift of responsibilities for business systems from the Office of the DCMO to the DOD CIO. Notwithstanding the uncertainty associated with this transition, until the department takes further steps to align its business system investment review and budgeting processes, it risks allocating resources to business system investments that might be used more efficiently or effectively elsewhere across the department. In addition, aligning the business system investment management process with the budget process would improve the ability of components to guide their investments from a strategic perspective and reduce pressure on the DBC to approve the obligation of funds that have already been budgeted for a given system. Regarding the development of multiple review boards, officials from the Office of the DCMO said that the DBC guidance does not call for developing IRBs at multiple levels within the department because it is important for the DBC to certify and approve all covered business systems, and not just those that are more costly. These officials added that some systems may be important for the business environment regardless of their relatively small dollar value. In addition, these officials stated that the Integrated Business Framework calls for reviewing systems as part of system portfolios, not as individual systems. We agree that the DBC should certify and approve all systems as called for by the act, that it may want to occasionally focus greater attention on smaller systems, and that the department should review systems as part of larger system portfolios. However, by not using a tiered review board approach, the department may hamper the quality of the information submitted and the associated reviews. This is evidenced by the results of the Office of the DCMO's validations of certifications discussed later in this report. Systems-- including systems submitted as part of portfolios--that are larger in scope, complexity, cost, or risk may benefit from additional scrutiny by the DBC, while those that are less complex or lower cost could be reviewed at the component level. Our IT investment management guidance notes that enterprise-wide IT investment boards should be responsible for systems that have high cost, high risk, or significant scope or duration, while retaining responsibility and visibility into other system review activities. However, lower-level boards could be chartered within business units to oversee the investment management process for other systems that are smaller or less costly, risky, or complex. While officials from the Office of the DCMO stated that different systems are reviewed with different levels of scrutiny based on various factors, those factors are not documented in existing guidance. Until DOD ensures that investments are reviewed at an appropriate level based on defined criteria, the DBC is at an increased risk of failing to identify and address important issues associated with large-scale and costly systems. DOD Has Made Improvements in Data Used to Support Its Certification and Approval of Defense Business Systems, but Reviews Supporting These Decisions Can Be Improved: The NDAA requires that, prior to obligating funds, DOD certify that investments meet certain conditions set out in the act. For fiscal year 2014, the department certified and approved most of its business systems investments. In addition, it has made significant improvements to the data used to manage information about these investments. However, while DOD continued to assert the compliance of its investments with the department's BEA and business process reengineering, the quality of the reviews supporting these assertions can be improved. DOD Certified and Approved Most Business System Investments for Fiscal Year 2014: According to DOD's March 2014 Congressional Report on Defense Business Operations, the DBC reviewed certification requests totaling $6.996 billion for 1,180 defense business systems across eight functional areas.[Footnote 34] Of those requests, according to the March report, the council approved $6.379 billion for 1,173 defense business systems and did not approve $617 million associated with 40 business systems (see table 4).[Footnote 35] Table 4: Certification Summary for Fiscal Year 2014 (as of February 2014: Functional portfolio: Acquisition; Approved: $289 million; Not approved: $20 million. Functional portfolio: Defense Security Enterprise; Approved: $39 million; Not approved: $3 million. Functional portfolio: Enterprise IT Infrastructure; Approved: $4 million; Not approved: $0. Functional portfolio: Financial Management; Approved: $718 million; Not approved: $67 million. Functional portfolio: Human Resources Management; Approved: $2.505 billion; Not approved: $414 million. Functional portfolio: Installations & Environment; Approved: $307 million; Not approved: $19 million. Functional portfolio: Logistics & Materiel Readiness; Approved: $2.509 billion; Not approved: $96 million. Functional portfolio: Security Cooperation; Approved: $6 million; Not approved: $0. Functional portfolio: Total; Approved: $6.379 billion[A]; Not approved: $617 million[A]. Source: GAO summary of DOD's March 2014 Congressional Report on Defense Business Operations. [A] Totals do not add due to rounding. [End of table] These certifications included about $563 million in funds for fiscal year 2014 that the DBC conditionally certified (i.e., certified on the condition that additional steps were taken). According to DOD's March 2014 report, the primary reason that systems were denied certification or conditionally certified was that they were lacking a problem statement (or "business need").[Footnote 36] The following are examples of systems that were denied certification or were conditionally certified in fiscal year 2014: * The Defense Logistics Agency's Standard Procurement System[Footnote 37] requested--but was denied--$3.6 million in development funds because a problem statement had not been submitted.[Footnote 38] As of March 2014, no problem statement had been submitted, and the item remained open. * The Defense Health Agency's Armed Forces Health Longitudinal Technology Application[Footnote 39] requested--and was conditionally approved for--about $133.1 million for fiscal year 2014, with the condition that the component submit a problem statement. As of April 2014, the problem statement had been submitted and approved. * The Defense Logistics Agency's Defense Retired and Annuitant Pay System 2[Footnote 40] requested a total of $12.6 million, but was only approved for $2.2 million, with the condition that, among other things, the component submit an updated cost estimate. As of March 2014, the cost estimate had been submitted to the Office of the DCMO, but had not yet been approved. DOD Has Made Important Improvements to the Data Used to Manage Its Business System Investments: Our IT investment management framework states that to make good IT investment decisions, an organization must be able to acquire pertinent information about each investment and store that information in a retrievable format, to be used in future investment decisions. [Footnote 41] As part of this process, an organization should identify its IT assets and create a comprehensive repository of investment information. The information in the repository should identify each IT investment and its associated components and should be accessible where it is of the most value to those making decisions about IT investments. DOD's ability to address the act's certification and approval requirements depends in large part on its ability to capture and use data about business systems and the department's business system environment. To that end, DOD maintains three key data sources or data repositories and requires in its investment management guidance that components include updated data within each source. Table 5 provides additional information about each data source. Table 5: Description of Key Data Sources: Authoritative data source: Defense Information Technology Investment Portal (DITIP); Description: DITIP is DOD's sole authoritative data source for the business system certification process. It is also intended to align system information from its Defense Information Technology Portfolio Repository with budget information from its Select and Native Programming Data Input System for Information Technology, which provides a linkage between investment certification and associated budget information. Authoritative data source: Defense Information Technology Portfolio Repository (DITPR); Description: DITPR is DOD's authoritative repository for system information used to meet internal and external reporting requirements. Among other things, DITPR provides a common central repository for IT system information, such as system start and end dates, to support the certification process. DITPR also supports component-level IT portfolio management. Authoritative data source: Select and Native Programming Data Input Systems for Information Technology (SNAP-IT); Description: SNAP-IT documents DOD's IT budget request, including the obligation authority assigned to each business system over the period of the Future-Years Defense Program. Source: GAO analysis of DOD data. [End of table] Since our May 2013 report, DOD has demonstrated important improvements to the data it uses to manage its business systems. In May 2013, we reported that we were unable to verify the number of certifications that occurred during the fiscal year 2013 certification and approval process.[Footnote 42] At the time, DOD officials noted that certain data fields in the original data sources were incomplete, incorrect, or duplicative when certification requests were submitted. They also stated that extensive manual cleanup efforts had been ongoing since the beginning of the investment review process. Since then, the department has established and used its DITIP system, which it developed in part as a response to our recommendations associated with improving data about business system investments.[Footnote 43] DITIP contains information on certification decisions, including the amounts requested for certification and subsequently certified (or not certified). According to officials from the Office of the DCMO and DOD business system investment review guidance, fiscal year 2014 certification requests and approval decisions were all entered and maintained directly in this system. As a result, information about investment certification requests and approval decisions was consistently documented. While this new approach has allowed DOD to improve certain information about its business system certification and approvals, other information needs continued improvement. Specifically, we have previously reported that information about investment certifications and approvals was not always reliable, and we have made recommendations aimed at improving the department's data about its business systems.[Footnote 44] For example, according to officials from the Office of the DCMO, the numbers of business systems reflected by DITIP and DITPR are not always consistent. These officials stated that data issues persist in DITPR due to the amount of data stored by the systems as well as the tight time frames associated with implementing improvements. DOD intends to make further updates to DITPR, including an update in the fourth quarter of fiscal year 2014, which will include a data validation process intended to align and reconcile the data common to DITPR and SNAP-IT via DITIP. BEA and Business Process Reengineering Compliance Continues to Be Asserted for the Majority of Systems: Under the act, funds may not be obligated for covered business systems unless the precertification authority asserts that, among other things, the system is in compliance with the BEA and has undertaken appropriate business process reengineering efforts. For fiscal year 2014 certification reviews, investments were to follow the Office of the DCMO's March 2013 BEA compliance guidance and September 2012 guidance on business process reengineering. The BEA guidance describes an incremental approach to BEA compliance requirements, including key elements needed for asserting architecture compliance. The business process reengineering guidance calls for specific levels of business process reengineering for systems in different stages of development. For example, prior to investing in development and modernization funds for a defense business system, the components are required to submit a problem statement[Footnote 45] to document and validate the system's business need. DOD's guidance required submission of a form that includes questions in areas related to various aspects of business process reengineering, such as change management and target process improvements. BEA Certifications: For fiscal year 2014, DOD officials reported that 89 percent (1,053 of 1,180 systems) of defense business systems asserted compliance with the BEA.[Footnote 46] Of the 44 systems that asserted non-compliance, 35 were within one component of DOD--the Defense Finance and Accounting Service. According to DOD officials, this component now has plans to comply with BEA requirements and plans to update its assertions. DOD's business system investment review guidance calls for the department to validate selected business system BEA assessments that support the component's assertions of compliance. We have previously reported on the need to improve the quality of BEA compliance assertions, and in 2013 we recommended that DOD implement and use the BEA compliance assessments more effectively to support organizational transformation efforts by, among other things, establishing milestones by which selected validations of BEA compliance assertions are to be completed.[Footnote 47] DOD partially agreed with this recommendation. DOD selected 15 defense business systems for review, as called for in its guidance. DCMO officials stated that of the 15 selected, all had asserted that they complied with version 10.0 of the BEA. However, DCMO officials found that 13 of the 15 selected had issues with their BEA compliance assertions. Specifically, DCMO officials stated that they found that 9 of the 13 had inconsistencies in the organizational execution plan or had significantly incomplete compliance assertions in DOD's BEA Compliance System, and 4 of the 15 were assessed as having other opportunities for improvements. According to DOD officials, some of these concerns were addressed prior to system certification. They further stated that no systems were denied funding certification due to incomplete BEA compliance assertions because the Office of the DCMO worked with the components to address the issues to the satisfaction of the DBC prior to certification. Where additional action was needed, action items were created and DCMO is tracking the progress of these systems' efforts to become compliant. The Office of the DCMO's validations of selected assessments are important for helping to ensure that BEA assessments are accurate. However, as demonstrated by these results, DOD needs to continue working to ensure the quality of BEA assessments, as we have previously recommended. Continued improvement in the BEA assessments will help ensure that programs are being defined and implemented in a way that facilitates interoperability and avoids duplication and overlap, which are both goals of the BEA and the related investment management approach. Business Process Reengineering Certifications: According to DOD officials, in fiscal year 2014, nearly 100 percent of certified and approved defense business systems asserted that business process reengineering had been completed. Further, the Office of the DCMO also validated a sample of assessments associated with these systems. According to DOD's March 2014 report, this validation effort showed that assertions were incomplete. Specifically, the DCMO selected 12 investments for validation, and the results of this effort were as follows: * Four systems received a "positive" rating, indicating that appropriate business process reengineering was conducted. * Four systems received a "positive with recommendations" rating, indicating that sufficient business process reengineering was conducted but more could be done to improve the effort. DOD recommended that the precertification authorities for these systems ensure that appropriate business process reengineering compliance is again assessed at the next milestone. For example, one system, the Learning Management System,[Footnote 48] received a recommendation to consider creating expected business outcomes, such as providing career roadmaps to guide financial management employees in planning and progressing in a DOD career. Further, another system, the Next Generation Resource Management System[Footnote 49] did not provide evidence of a change management plan, or documentation of the resolution of the root causes identified in its "to-be" process models. * Four systems received a "could not validate" rating, indicating that the business process reengineering assessment team was not able to determine that a sufficient business process reengineering assessment had been conducted. In these cases, DOD recommended that the precertification authority ensure business process reengineering assessment compliance is assessed at the next acquisition milestone. For example, according to DOD officials, the systems could not be validated because insufficient objective evidence had been provided to substantiate their business process reengineering assertions. According to the Office of the DCMO, three of these four systems were in operations and maintenance, and it is not often cost-effective to conduct business process reengineering compliance reviews on investments in operations and maintenance. The need for improvement demonstrated by some of DCMO's selected validations is consistent with our previous findings. Specifically, in May 2013 we reported on business process reengineering assertions of four case study systems. All four assertions generally followed DOD's business process reengineering guidance; however, we also found that limited documentation of root cause analyses was provided for all four. We have previously made recommendations to improve DOD's business process reengineering-related efforts, including recommendations aimed at the four systems we assessed in 2013. [Footnote 50] The Office of the DCMO's validations of selected systems' business process reengineering assessments are important for helping to ensure that sufficient business process reengineering is conducted. However, without further validation of the business process reengineering assertions--as we have previously recommended--DOD is at risk of not being able to accurately determine whether appropriate business process reengineering has been undertaken, including ensuring that the business processes supported by defense business systems have been streamlined and have eliminated or reduced unique requirements to the maximum practical extent. DCMO Has Yet to Demonstrate That Its BEA Is Producing Business Value or Resulting in Elimination of Duplicative Systems: The act requires DOD to develop a BEA that covers all defense business systems and their related functions and activities and includes key information. For example, the act requires the BEA to include specific information about data standards, policies and procedures, and performance measures. In addition, the department seeks to use the BEA to help guide, constrain, and enable interoperable business systems. Moreover, we have previously emphasized that the BEA can be an important tool for reducing potential overlap and duplication among DOD business systems.[Footnote 51] As we reported in May 2013, the current version of the BEA--Version 10.0[Footnote 52]--includes information aimed at meeting the act's requirements, such as data standards; business rules to help ensure compliance with the laws, regulations, and policies incorporated in the BEA; and performance measures associated with the department's Strategic Management Plan.[Footnote 53] However, as of April 2014, this version was still missing other important content associated with achieving the department's goal of using the BEA to guide, constrain, and enable interoperable business systems. For example, the BEA was missing information about business systems associated with each of the architecture's business activities, and its business activities have not been defined at a level that allows for more effective identification of potential duplication and overlap. Adding such content is important for various reasons, including improving the department's ability to use the architecture as a tool for managing its business systems investments by more effectively identifying areas of potential duplication and overlap. In addition, DOD has not yet demonstrated that the BEA has produced business value for the department. According to the Director of Investment Acquisition and Management and the Chief Architect, previous versions of the BEA were designed to meet the requirements of the act, but were not clearly focused on achieving business outcomes. However, the act defines specific business outcomes that the BEA is intended to support, such as integrating budget, accounting, and program information and systems, and the department has defined additional BEA goals, such as using the BEA to guide, constrain, and enable interoperable business systems. These officials stated that in the future they plan to use the BEA as a tool to better support interoperability, data sharing, and reducing duplication. To help achieve this goal, according to officials from the Office of the DCMO, DOD has established a process for defining changes to the BEA that are intended to better support the department's ability to achieve business outcomes, and they first used this process for defining updates to BEA 10.0. More specifically, the Office of the DCMO established a BEA configuration control board--composed of representatives from the military departments and other departmental offices--to make recommendations to the DBC on BEA requirements and related content changes. Such a process may help the department as it refines the architecture and seeks to achieve business outcomes. Further, while DOD has established a tool that can assist in identifying potential duplication and overlap among business systems even with the current gaps in BEA content, the department has not demonstrated that it has used this information to reduce duplication and overlap. The April 2013 investment management guidance and a January 2013 memorandum issued by the DCMO established a single BEA compliance tool--the BEA Compliance System--to document system-level BEA compliance assertions and required all business systems submitted for certification and approval for fiscal year 2014 to use this tool. Officials from the Office of the DCMO stated that the results of BEA compliance assessments conducted as part of the business system certification and review process are available to staff across DOD to review in making determinations about potential overlap and duplication. For example, information provided by the department showed that, as of February 2014, 120 systems performed activities associated with maintaining asset information, 110 systems performed activities associated with managing military health services, and 73 systems performed activities associated with receiving and accepting a purchase request.[Footnote 54] Officials from the Army and the Air Force stated that in fiscal year 2014 they have begun to identify systems using BEA data that were potentially duplicative. According to these officials, both the Army and the Air Force have taken initial steps to phase out potentially duplicative systems. However, the department has yet to demonstrate that it has terminated these or other potentially duplicative programs that were identified by using the BEA. Officials from the Office of the DCMO offered several reasons why the BEA compliance process has yet to result in the elimination of potentially duplicative systems. For example, officials stated that it is difficult for the DBC to make such determinations because information presented to the council is high level and only points to potential instances of overlap and duplication, which would require additional research. Instead, these officials said, precertification authorities at the component level are in the best position to determine whether their systems are duplicative of other systems. However, these precertification authorities are not required to identify and assess potential overlap and duplication before asserting compliance with the BEA. In addition, the BEA continues to lack important content, such as additional information about business activities, that would further assist in identifying potential duplication and overlap among business systems. Continued improvements to DOD's BEA--as we have previously recommended[Footnote 55]--and calling for more proactive identification of potential overlap and duplication using information available in existing tools can help the department use its BEA to identify and address potential overlap and duplication among its billions of dollars in annual defense business system investments. Enterprise Transition Plan Content Has Been Improved and Is Consistent with Portfolio-Based Investment Management Approach: The act calls for the development of an ETP that implements the BEA and covers all defense business systems and includes a listing of the: (1) new systems that are expected to be needed to complete the target defense business systems computing environment,[Footnote 56] along with each system's: * time-phased milestones, * performance measures, * financial resource needs, and: * risks or challenges to integration into the BEA; (2) legacy systems that will be phased out of the defense business systems computing environment within 3 years, together with the schedule for terminating those legacy systems; and: (3) existing systems that are part of the target defense business systems computing environment, as well as a strategy for making the modifications to those systems that will be needed to ensure that such systems comply with the defense BEA, including time-phased milestones, performance measures, and financial resource needs. The department's fiscal year 2014 ETP, which was issued in December 2013 and uses data as of October 1, 2013, includes information about 1,179 covered defense business systems,[Footnote 57] including 79 legacy systems, or those that will be terminated within 36 months.[Footnote 58] The ETP also includes links to related documentation, including the functional strategies, organizational execution plans, and investment decision memoranda. DOD has taken steps to improve its ETP by including key content, but has not included other information, as described below. * Milestones: The plan includes acquisition milestone information for covered defense business systems (both legacy and core) through its links to other systems, such as the Defense Acquisition Management Information Retrieval system,[Footnote 59] DITIP,[Footnote 60] DITPR, [Footnote 61] and Office of Management and Budget exhibit 300s. Specifically, according to DOD's April 2013 investment management guidance,[Footnote 62] system owners are required to include start and end dates for life-cycle phases of each system within DITPR.[Footnote 63] * Performance measures: The plan includes portfolio-level performance measures in the organizational execution plans. For example, the Air Force Human Resources Management organizational execution plan includes performance measures associated with improving the accuracy and timeliness of pay for airmen. However, these measures are not linked to specific business systems. * Financial resource needs: The plan includes information about fiscal year 2014 funding that was requested and ultimately approved under the department's business system investment review process for each business system (both legacy and core). * Risks or challenges for BEA integration: The plan and related documentation do not discuss each system's risks or challenges to integration into the BEA. Instead, the organizational execution plans include portfolio-level risks and challenges, with only some related to the BEA. For example, the Air Force Defense Security Enterprise organizational execution plan identifies a challenge that the BEA and business process reengineering process do not fit well with the initiatives in this specific portfolio, and that many BEA and business process reengineering compliance requirements are not applicable. * Termination dates: The ETP data also includes termination dates for all 79 of the covered legacy systems and a listing of both new and existing systems that will be part of the target defense business systems computing environment. Defining when and how the ETP will include the missing information--as we have previously recommended[Footnote 64]--will help to ensure that DOD more fully complies with the requirements of the act and help to inform oversight of its business systems investments. DOD's Fiscal Year 2015 Budget Submission Includes Key Information on Business Systems: Another requirement of the NDAA for Fiscal Year 2005, as amended, is that DOD's annual IT budget submission must include key information on each business system for which funding is being requested, such as the system's precertification authority and designated senior official, the appropriation type and amount of funds associated with modernization and current services (i.e., operation and maintenance), and the associated Defense Business Systems Management Committee approval decisions. The department's fiscal year 2015 budget submission includes a range of information for business system investments requesting funding,[Footnote 65] such as the system's (1) name, (2) precertification authority, (3) designated senior official, (4) approved funding for fiscal year 2014, and (5) requested funding for fiscal year 2015. The submission also identifies the amount of the fiscal year 2015 request that is for modernization versus current services. Conclusions: DOD has made important progress developing a framework to manage the department's business systems and more fully comply with the act, yet additional and sustained efforts are necessary to fully achieve the department's modernization goals. Specifically: * The Office of the DCMO developed a portfolio-based investment management framework--the Integrated Business Framework. However, establishing specific plans for aligning the budgeting and investment management processes would help ensure that systems included in the budget also meet the department's mission needs and requirements, and may result in less money being budgeted for systems that are duplicative or do not meet the needs of the department. In addition, allowing less complex, costly, and risky systems to be reviewed at the component level will give the DBC more time to review more complex, costly, and risky systems. * The department also released a new system that should help to address data reliability issues and certified most systems as meeting the BEA and business process reengineering requirements. Implementing our prior recommendations in this area will help DOD improve the certification process. * The Office of the DCMO has taken steps to improve its ETP by including most of the key content required by the act. Fully implementing our prior recommendations will help to further inform oversight of the department's business systems investments. * The department formed a configuration control board for the BEA to help manage future changes, which should provide decision makers with better information on which to base their certification and approval decisions. However, a more proactive use of the BEA--by requiring precertification authorities to ensure that their systems do not duplicate existing systems--will save time, money, and resources by not investing in redundant systems. Accordingly, further refinements to DOD's business system investment management process and more proactive use of the BEA to inform decisions would assist the department in leveraging its limited resources to more efficiently and effectively manage its business systems. Recommendations for Executive Action: We are recommending that the Secretary of Defense direct the appropriate DOD management entity to take the following three actions to help ensure that the department's business systems modernization program is fully compliant with the act and is more effectively implemented: * Define by when and how the department plans to align its business system certification and approval process with its Planning, Programming, Budgeting, and Execution process. * Define criteria for reviewing defense business systems at an appropriate level in the department based on factors such as complexity, scope, cost, and risk, in support of the certification and approval process. * Develop guidance requiring military departments and other defense organizations to use existing BEA content to more proactively identify potential duplication and overlap. Agency Comments and Our Evaluation: We received written comments on a draft of this report from DOD, reprinted in appendix II. In its comments, the department concurred with two recommendations and partially concurred with one recommendation. DOD concurred with our first recommendation, to define by when and how the department plans to align its business system certification and approval process with its Planning, Programming, Budgeting, and Execution process, and noted that further alignment of these processes will continue in fiscal year 2015. The department partially concurred with our second recommendation, to define criteria for reviewing defense business systems at an appropriate level in the department, in support of the certification and approval process. In particular, DOD stated that it will continue to mature a process that incorporates a variety of factors, including business process complexity, risk, and portfolio costs. However, the department also stated that our recommendation implies that it has not ensured that reviews occur at an appropriate level. DOD noted that pre- certification authorities are required to review the entire portfolio of investments before they are presented to the DBC. The department added that these reviews enable decision making, to include resourcing decisions at an organizational level, which allows the DBC to focus on enterprise-level business issues facing DOD. DOD also stated that, for fiscal year 2015, pre-certification authorities will report the results of their review to the DBC, rather than simply presenting their portfolio for certification. We acknowledge that DOD's guidance calls for reviews of investment portfolios at multiple organizational levels, and our recommendation is not intended to imply otherwise. Rather, its intent is to help ensure that DOD establishes guidance and criteria for ensuring that detailed reviews of individual systems occur at the appropriate levels within the department. As described in this report, such an approach may involve establishing investment review boards at multiple organizational levels. Such a tiered review board approach would allow the DBC--acting at the executive-level IRB for the department--to focus its attention on more costly, complex, or risky systems, while delegating detailed reviews of other systems to lower organizational levels. By not using such an approach, the department may not be able to ensure that the quality of the information submitted for review is sufficient, and the associated investment reviews may not be adequate for identifying and addressing important issues associated with large- scale and costly systems. This is evidenced by the results of the Office of the DCMO's validations of BEA and business process reengineering certifications discussed in this report. While DOD's planned actions reflect continued improvement in the department's oversight of defense business systems, these actions do not fully address the recommendation. Accordingly, we believe that full implementation of our recommendation is needed. Finally, the department concurred with our recommendation to develop guidance requiring military departments and other defense organizations to use existing BEA content to more proactively identify potential duplication and overlap. DOD added that the department plans to improve its investment review process in fiscal year 2015 with a method that uses BEA data to identify candidate systems for further review and, if appropriate, retirement. We are sending copies of this report to the appropriate congressional committees; the Director, Office of Management and Budget; the Secretary of Defense; and other interested parties. This report also is available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff members have any questions on matters discussed in this report, please contact me at (202) 512-4456 or chac@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. Signed by: Carol R. Cha: Director: Information Technology Acquisition Management Issues: List of Committees: The Honorable Carl Levin: Chairman: The Honorable James M. Inhofe: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Richard J. Durbin: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Howard P. "Buck" McKeon: Chairman: The Honorable Adam Smith: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable Rodney Frelinghuysen: Chairman: The Honorable Pete Visclosky: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Appendix I: Objective, Scope, and Methodology: Our objective was to assess the actions taken by the Department of Defense (DOD) to comply with section 332 of the National Defense Authorization Act for Fiscal Year 2005 as amended.[Footnote 66] The specific elements we assessed were (1) establishing a system investment approval and accountability structure along with an investment review process, (2) certifying and approving any business system program costing in excess of $1 million, (3) developing a business enterprise architecture (BEA) to cover all defense business systems, (4) developing a transition plan for implementing the architecture, and (5) identifying systems information in its annual budget submission. To address the act's provision for establishing an investment approval and accountability review structure, along with an investment review process, we reviewed the department's current policies, procedures, and guidance relative to the act's requirements and criteria documented in our information technology (IT) investment management framework.[Footnote 67] Specifically, to determine whether the foundation for DOD's new investment management process had been fully established, we reviewed and analyzed the department's investment management guidance--published in April 2013--and the six functional strategies prepared for the fiscal year 2014 certification process. [Footnote 68] We also reviewed DOD documentation on future plans, including a draft version of its updated investment management guidance and the March 2014 Congressional Report on Defense Business Operations. We reviewed and analyzed DOD's investment management guidance to determine whether criteria and procedures had been fully defined for making portfolio-based investment decisions and the extent to which they aligned to our IT investment management framework. Further we reviewed prior GAO reports, DOD business system investment management guidance, and DOD budget process policy to determine how the budget process aligned with the business system investment review process. Finally, we interviewed cognizant officials in the Office of the Deputy Chief Management Officer (DCMO) to gain a better understanding of the investment management process, its outputs, and planned improvements for fiscal year 2015 defense business system certifications. To determine whether the department was certifying and approving business system programs costing in excess of $1 million in accordance with the act's provisions, we reviewed DOD's March 2014 Congressional Report on Defense Business Operations, and all functional strategies and organizational execution plans, as well as their precertification request memoranda, which were submitted as part of the fiscal year 2014 investment review process. We also reviewed investment decision memoranda that documented the decisions made by the Defense Business Council. To determine whether DOD had made progress in improving the data used to manage its business system investments, we reviewed data from the department's three authoritative data sources--the Defense Information Technology Investment Portal (DITIP), the Defense Information Technology Portfolio Repository (DITPR), and the Select and Native Programming Data Input System for Information Technology (SNAP-IT). We compared our results with the information reported in DOD's annual report to Congress to identify any discrepancies and discussed these with cognizant officials from the Office of the DCMO. Because DOD uses one system--DITIP--as the authoritative source for information about defense business system certification and approvals, we determined that the data were sufficiently reliable for our purposes. As part of our evaluation under the certification requirement, we also identified the processes associated with asserting BEA and business process reengineering compliance for covered defense business systems and the documentation used to support those assertions. To accomplish this, we reviewed DOD guidance and related documentation that applied to the fiscal year 2014 investment portfolio reviews and relevant updates to that guidance issued during our audit. Further, we reviewed action items, the status of those actions, selected business process reengineering and BEA assertions, and data from DOD's BEA compliance assertion system--the BEA Compliance System--to understand the extent to which defense business systems asserted compliance with BEA requirements. In addition, we interviewed officials from the Office of the DCMO about steps completed, under way, or planned relative to BEA and business process reengineering compliance and validation activities. We also reviewed DOD's annual report to Congress to determine if business process reengineering outcomes had been reported. To address the provision associated with developing the BEA, we reviewed our previous findings[Footnote 69] associated with version 10.0 of the BEA, released in February 2013, relative to the act's architectural requirements.[Footnote 70] We also identified recommendations from previous GAO reports related to the act and measuring results and outcomes, and gathered documentation to support the current status of those recommendations. Specifically, we reviewed documentation related to steps completed, under way, or planned to address the act's requirements and previously reported weaknesses, including slides presented to the Defense Business Council regarding BEA requirements criteria and updates, and the March 2014 Congressional Report on Defense Business Operations. In addition, we reviewed documentation related to efforts to achieve business value from the BEA, including meeting minutes and charters from the BEA configuration control board and sub-groups under that forum. To determine whether it is used to identify and reduce potential overlap and duplication, we reviewed data in DOD's BEA Compliance System to determine how systems are mapping to the department's business functions, and reviewed DOD's guidance on BEA compliance. Finally, we interviewed officials from the Office of the DCMO about planned updates and improvements to the BEA's content, and reasons that it has not yet been used to identify duplication and overlap. To address the provision associated with developing a transition plan, we analyzed DOD's enterprise transition plan, released in December 2013, and associated guidance in DOD's April 2013 investment management guidance, relative to the act's transition plan requirements and related findings documented in previous GAO reports. Specifically, we reviewed enterprise transition plan documentation, including functional strategies, organizational execution plans, Office of Management and Budget exhibit 300s, and data from the DITIP and DITPR systems, to identify whether that documentation met the requirements of the act. We also interviewed officials from the Office of the DCMO to determine the steps completed, under way, or planned to address the act's requirements and previously reported weaknesses. To assess the system information in DOD's fiscal year 2015 information technology budget submission, we analyzed information contained in the department's fiscal year 2015 budget request, including the department's Section 332 Report and information contained in the systems that are used to prepare its budget submission--SNAP-IT, DITPR, and DITIP--against the act's requirements in this area. We also interviewed officials from the Office of the DCMO and the Office of the Chief Information Officer to discuss the accuracy and comprehensiveness of information contained in the systems, and reasons for any discrepancies in the information they contained. We conducted this performance audit from October 2013 to May 2014, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. [End of section] Appendix II: Comments from the Department of Defense: Deputy Chief Management Officer: 9010 Defense Pentagon: Washington, DC 20301-9010: April 28, 2014: Ms. Carol Cha: Director, Information Management and Technology Resources Issues: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Ms. Cha: This is the Department of Defense response to the Government Accountability Office draft report GAO-14-486, "Defense Business Systems: Further Refinements Needed to Guide the Investment Management Process," dated April 21, 2014, (Government Accountability Office Code 311605). The Department concurs or partially concurs with all recommendations. The recommendation for establishment of criteria that ensures that review occurs at an appropriate level implies that the Department has not done so. Statutorily defined pre-certification authorities operate as portfolio managers to review investments and report the results of their review. The review includes alignment to architecture and business strategy and business process reengineering in addition to progress toward reduction of operating costs, system duplication and overlap. In Fiscal Year 2014, the Defense Business Council will mature the use of the Integrated Business Framework and deliver the business operations improvements and outcomes required by the warfighter. The Department appreciates the opportunity to respond to your draft report. We look forward to your continued cooperation and dialog toward our common goal of improving business systems modernization throughout the Department of Defense. Should you have any questions, please contact Ms. Sherri Malace, 571-372-3097, Sherri.R.Malace.civ@mail.mil. Sincerely, Signed by: Kevin J. Sheid: Acting Deputy Chief Management Officer: Enclosures: As stated. GAO DRAFT REPORT Dated April 21, 2014: GAO-14-486 (GAO Code 311605): "Defense Business Systems: Further Refinements Needed To Guide The Investment Management Process" Department Of Defense Comments To The GAO Recommendations: The Government Accountability Office (GAO) is recommending that the Secretary of Defense direct the appropriate DOD management entity to take the following actions to help ensure that the Department's business systems modernization program is fully compliant with the Act and is more effectively implemented. Recommendation 1: Define by when and how the Department plans to align its business system certification and approval process with its Planning, Programming, Budgeting, and Execution [PPBE] process. DoD Response: Concur. In FY 2014, the investment review process used information from the PPBE process as part of the annual review of the Defense business system programs in the portfolio. Further alignment will continue in FY 2015 where the review process incorporates examination of the alignment between functional strategy initiatives and investments within a portfolio that may include more than just systems. The process will consider certification and approval of investments prior to obligation/execution in FY 2015 in addition to the nomination of budget issues into the over-arching PPBE process where appropriate. Recommendation 2: Define criteria for reviewing defense business systems at an appropriate level in the department based on factors such as complexity, scope, cost, and risk, in support of the certification and approval process. DoD Response: Partially Concur. The Department will continue to mature a process that incorporates a variety of factors, including business process complexity, risk and portfolio costs. The Precertification Authorities (PCAs) are currently required to review the entire portfolio of investments before they are presented to the Defense Business Council (DBC) for certification. PCA reviews enable decision making, to include resourcing decisions at an organizational level, allowing the Defense Business Council to focus on enterprise level business issues facing DoD. As part of the FY2015 review PCAs will report the results of their review to the DBC vice simply presenting their portfolio for certification. The Department will also leverage the Defense Acquisition System for focused reviews of individual Major Automated Information Systems in accordance with budget thresholds established by statute and policy. Recommendation 3: Develop guidance requiring military departments and other defense organizations to use existing BEA content to more proactively identify potential duplication and overlap. DoD Response: Concur. The Department's Strategic Management Plan goal of a simplified business environment acknowledges that efficiencies can be gained through economies of scale and elimination of duplicative services and products. Improvements are planned for the FY 2015 investment review process using a method that reviews BEA operational activities mappings to systems in order to identify candidates for further review and if appropriate, retirement. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Carol R. Cha, (202) 512-4456 or ChaC@gao.gov: Staff Acknowledgments: In addition to the contact above, individuals making contributions to this report include Michael Holland (Assistant Director), Camille Chaires, Debra Conner, Kate Feild, Angel Ip, Anh Le, Lee McCracken, and Christine San. [End of section] Footnotes: [1] GAO, High-Risk Series: An Update, [hyperlink, http://www.gao.gov/products/GAO-13-283] (Washington, D.C.: February 2013). [2] GAO, Opportunities to Reduce Potential Duplication in Government Programs, Save Tax Dollars, and Enhance Revenue, [hyperlink, http://www.gao.gov/products/GAO-11-318SP] (Washington, D.C.: Mar. 1, 2011); Follow-up on 2011 Report: Status of Actions Taken to Reduce Duplication, Overlap, and Fragmentation, Save Tax Dollars, and Enhance Revenue, [hyperlink, http://www.gao.gov/products/GAO-12-453SP] (Washington, D.C.: Feb. 28, 2012); and [hyperlink, http://www.gao.gov/products/GAO-13-283]. [3] Defense Information Technology Investment Portal, dated November 5, 2013. [4] See, for example, GAO, DOD Business Systems Modernization: Long- standing Weaknesses in Enterprise Architecture Development Need to Be Addressed, [hyperlink, http://www.gao.gov/products/GAO-05-702] (Washington, D.C.: July 22, 2005); DOD Business Systems Modernization: Billions Being Invested without Adequate Oversight, [hyperlink, http://www.gao.gov/products/GAO-05-381] (Washington, D.C.: Apr. 29, 2005); DOD Business Systems Modernization: Limited Progress in Development of Business Enterprise Architecture and Oversight of Information Technology Investments, [hyperlink, http://www.gao.gov/products/GAO-04-731R] (Washington, D.C.: May 17, 2004); DOD Business Systems Modernization: Important Progress Made to Develop Business Enterprise Architecture, but Much Work Remains, [hyperlink, http://www.gao.gov/products/GAO-03-1018] (Washington, D.C.: Sept. 19, 2003); Business Systems Modernization: Summary of GAO's Assessment of the Department of Defense's Initial Business Enterprise Architecture, [hyperlink, http://www.gao.gov/products/GAO-03-877R] (Washington, D.C.: July 7, 2003); Information Technology: Observations on Department of Defense's Draft Enterprise Architecture, [hyperlink, http://www.gao.gov/products/GAO-03-571R] (Washington, D.C.: Mar. 28, 2003); DOD Business Systems Modernization: Improvements to Enterprise Architecture Development and Implementation Efforts Needed, [hyperlink, http://www.gao.gov/products/GAO-03-458] (Washington, D.C.: Feb. 28, 2003); Information Technology: Architecture Needed to Guide Modernization of DOD's Financial Operations, [hyperlink, http://www.gao.gov/products/GAO-01-525] (Washington, D.C.: May 17, 2001); and High-Risk Series: An Overview, [hyperlink, http://www.gao.gov/products/GAO/HR-95-1] (Washington, D.C.: February 1995). [5] Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (2004), as amended by the NDAA for Fiscal Year 2014, Pub. L. No. 113-66, § 901, 127 Stat. 672, 815 (2013), the NDAA for Fiscal Year 2013, Pub. L. No. 112-239, 126 Stat. 1632 (2012), the NDAA for Fiscal Year 2012, Pub. L. No. 112-81, 125 Stat. 1298 (2011), and the NDAA for Fiscal Year 2010, Pub. L. No. 111-84, 123 Stat. 2190 (2009) (codified in part at 10 U.S.C. § 2222. Hereafter, we refer to the provisions of 10 U.S.C. § 2222, including its amendments, as the NDAA or "the act."). [6] GAO, DOD Business Systems Modernization: Further Actions Needed to Address Challenges and Improve Accountability, [hyperlink, http://www.gao.gov/products/GAO-13-557] (Washington, D.C.: May 17, 2013); DOD Business Systems Modernization: Governance Mechanisms for Implementing Management Controls Need to be Implemented, [hyperlink, http://www.gao.gov/products/GAO-12-685] (Washington, D.C.: June 1, 2012); Department of Defense: Further Actions Needed to Institutionalize Key Business System Modernization Management Controls, [hyperlink, http://www.gao.gov/products/GAO-11-684] (Washington, D.C.: June 29, 2011); Business Systems Modernization: Scope and Content of DOD's Congressional Report and Executive Oversight of Investments Need to Improve, [hyperlink, http://www.gao.gov/products/GAO-10-663] (Washington, D.C.: May 24, 2010); DOD Business Systems Modernization: Recent Slowdown in Institutionalizing Key Management Controls Needs to Be Addressed, [hyperlink, http://www.gao.gov/products/GAO-09-586] (Washington, D.C.: May 18, 2009); DOD Business Systems Modernization: Military Departments Need to Strengthen Management of Enterprise Architecture Programs, [hyperlink, http://www.gao.gov/products/GAO-08-519] (Washington, D.C.: May 12, 2008); Business Systems Modernization: Department of the Navy Needs to Establish Management Structure and Fully Define Policies and Procedures for Institutionally Managing Investments, [hyperlink, http://www.gao.gov/products/GAO-08-53] (Washington, D.C.: Oct. 31, 2007); and Business Systems Modernization: Air Force Needs to Fully Define Policies and Procedures for Institutionally Managing Investments, [hyperlink, http://www.gao.gov/products/GAO-08-52] (Washington, D.C.: Oct. 31, 2007). [7] GAO, DOD Financial Management: Implementation Weaknesses in Army and Air Force Business Systems Could Jeopardize DOD's Auditability Goals, [hyperlink, http://www.gao.gov/products/GAO-12-134] (Washington, D.C.: Feb. 28, 2012). [8] Defense Information Technology Investment Portal, dated November 5, 2013. [9] Forty systems are listed as "not assigned," indicating that they are aligned to an "other" business function. [10] [hyperlink, http://www.gao.gov/products/GAO-13-283]. [11] These seven high-risk areas are DOD's overall approach to business transformation, business systems modernization, contract management, financial management, supply chain management, support infrastructure management, and weapon systems acquisition. [12] The eight government-wide high-risk areas are better managing climate change risks, disability programs, ensuring the effective protection of technologies critical to U.S. national security interests, information systems and critical infrastructure, information sharing for homeland security, human capital, mitigating gaps in weather satellite data, and real property. [13] GAO, DOD Business Systems Modernization: Planned Investment in Navy Program to Create Cashless Shipboard Environment Needs to Be Justified and Better Managed, [hyperlink, http://www.gao.gov/products/GAO-08-922] (Washington, D.C.: Sept. 8, 2008); DOD Travel Cards: Control Weaknesses Resulted in Millions of Dollars of Improper Payments, [hyperlink, http://www.gao.gov/products/GAO-04-576] (Washington, D.C.: June 9, 2004); Military Pay: Army National Guard Personnel Mobilized to Active Duty Experienced Significant Pay Problems, [hyperlink, http://www.gao.gov/products/GAO-04-89] (Washington, D.C.: Nov. 13, 2003); and Defense Inventory: Opportunities Exist to Improve Spare Parts Support Aboard Deployed Navy Ships, [hyperlink, http://www.gao.gov/products/GAO-03-887] (Washington, D.C.: Aug. 29, 2003). [14] 10 U.S.C. § 2222. [15] The act (10 U.S.C. § 2222(a)) requires the appropriate precertification authority to determine that a defense business system modernization program (1) (a) is in compliance with the enterprise architecture and (b) has undertaken appropriate business process reengineering efforts; (2) is necessary to achieve a critical national security capability or address a critical requirement in an area such as safety or security; or (3) is necessary to prevent a significant adverse effect on a project that is needed to achieve an essential capability, taking into consideration the alternative solutions for preventing such an adverse effect. The NDAA for fiscal year 2012 requires that the certification and approval requirements apply to all business systems programs that are expected to cost more than $1 million over the period of the current Future-Years Defense Program, which is the department's financial plan over a 6-year period. Previously, the certification requirement only applied to business system modernizations with a total cost in excess of $1 million. [16] The Deputy Secretary of Defense serves as the department's Chief Management Officer. [17] According to DOD officials, the business mission area is responsible for ensuring that capabilities, resources, and materiel are reliably delivered to the warfighter. Specifically, the business mission area addresses functional areas such as real property and human resources management. [18] The Defense Business Systems Management Committee was established under 10 U.S.C. §186, which requires the department to set up a committee to review and approve major updates of the defense BEA and to ensure that the obligation of funds for defense systems modernization is consistent with the criteria set out in 10 U.S.C. § 2222. [19] These 1,180 defense business systems--out of the total 2,329 defense business systems--are the systems covered by the act's certification and approval requirements. That is, the systems are expected to cost more than $1 million over the period of the current Future-Years Defense Program. [20] The National Security Strategy outlines the core national security interests of the United States and calls for a range of actions to implement the strategy. [21] The Quadrennial Defense Review is the strategic plan for DOD and sets forth priority objectives for DOD and major actions to be taken to accomplish these objectives. The most recent review was issued in March 2014. This strategic plan is derived from the core interests listed in the National Security Strategy. [22] The Strategic Management Plan established specific management goals that directly support the strategic goals of the Quadrennial Defense Review. DOD has issued five updates to the Strategic Management Plan since 2008, with the most recent being issued in July 2013 and covering fiscal years 2014 and 2015. [23] An enterprise architecture is a modernization blueprint that provides snapshots of both the current IT environment and its target environment. [24] The department's BEA focuses on documenting information associated with its end-to-end business process areas. For example, the hire-to-retire business process includes steps to plan for, hire, classify, develop, assign, track, account for, compensate, retain, and separate the persons needed to accomplish aspects of the DOD mission. [25] GAO, DOD Business Systems Modernization: Progress in Establishing Corporate Management Controls Needs to Be Replicated Within Military Departments, [hyperlink, http://www.gao.gov/products/GAO-08-705] (Washington, D.C.: May 15, 2008); DOD Business Systems Modernization: Progress Continues to Be Made in Establishing Corporate Management Controls, but Further Steps Are Needed, [hyperlink, http://www.gao.gov/products/GAO-07-733] (Washington, D.C.: May 14, 2007); Business Systems Modernization: DOD Continues to Improve Institutional Approach, but Further Steps Needed, [hyperlink, http://www.gao.gov/products/GAO-06-658] (Washington, D.C.: May 15, 2006); and DOD Business Systems Modernization: Important Progress Made in Establishing Foundational Architecture Products and Investment Management Practices, but Much Work Remains, [hyperlink, http://www.gao.gov/products/GAO-06-219] (Washington, D.C.: Nov. 23, 2005). [26] [hyperlink, http://www.gao.gov/products/GAO-09-586]. [27] [hyperlink, http://www.gao.gov/products/GAO-11-684], [hyperlink, http://www.gao.gov/products/GAO-10-663], and [hyperlink, http://www.gao.gov/products/GAO-09-586]. [28] [hyperlink, http://www.gao.gov/products/GAO-13-557]. [29] [hyperlink, http://www.gao.gov/products/GAO-13-283]. [30] http: //www.gao.gov/duplication/action_tracker/1712]. [31] [hyperlink, http://www.gao.gov/products/GAO-04-394G]. [32] [hyperlink, http://www.gao.gov/products/GAO-13-557]. [33] The eight functional areas are Acquisition, Defense Security Enterprise, Enterprise IT Infrastructure, Financial Management, Human Resources Management and Health Management, Installation and Environment, Logistics and Materiel Readiness, and Security Cooperation. [34] These 1,180 defense business systems--out of the total 2,329 defense business systems--are the systems covered by the act's certification and approval requirements. [35] According to the March Congressional Report, certification results were reported as of October 1, 2013. [36] A problem statement documents the results of the analysis of a perceived business problem, capability gap, or opportunity. [37] The Standard Procurement System is to automate the contracting process for the Defense Logistics Agency, including contract placement, procurement, and contract administration. [38] The Standard Procurement System requested about $40 million in current services funds and was certified for that amount. [39] The Armed Forces Health Longitudinal Technology Application is to serve as a clinical information system providing access to patient's medical records. [40] The Defense Retired and Annuitant Pay System 2 is an update to an enterprise system intended to pay military retirees, former spouses, and survivor beneficiaries. [41] [hyperlink, http://www.gao.gov/products/GAO-04-394G]. [42] [hyperlink, http://www.gao.gov/products/GAO-13-557]. [43] [hyperlink, http://www.gao.gov/products/GAO-12-685] and [hyperlink, http://www.gao.gov/products/GAO-09-586]. [44] [hyperlink, http://www.gao.gov/products/GAO-09-586], [hyperlink, http://www.gao.gov/products/GAO-12-865], and [hyperlink, http://www.gao.gov/products/GAO-13-557]. [45] A problem statement documents the results of the analysis of a perceived business problem, capability gap, or opportunity ("business need"). [46] In addition to the 1,053 that asserted compliance, 44 asserted non-compliance, 16 did not complete the assertion, 1 had an unknown status, and 65 are legacy systems that are not required to assert compliance. [47] [hyperlink, http://www.gao.gov/products/GAO-13-557]. [48] The Learning Management System provides access to online and classroom-based training to DOD. [49] The Next Generation Resource Management System is to modernize antiquated legacy systems and processes used to formulate, justify, present, and defend the entire DOD budget. [50] [hyperlink, http://www.gao.gov/products/GAO-13-557]. [51] [hyperlink, http://www.gao.gov/products/GAO-11-318SP] and [hyperlink, http://www.gao.gov/products/GAO-12-453SP]. [52] BEA Version 11 was not issued in time for consideration as part of this audit. [53] [hyperlink, http://www.gao.gov/products/GAO-13-557]. [54] Maintaining asset information is a process that ensures that individual asset records are fully consistent with the actual status of assets, including their physical, legal, and financial status. Managing military health services involves providing direction, resources, healthcare providers, eligibility, and enrollment information necessary to promote the health of the beneficiary population. Receiving and accepting a purchase request consists of ensuring that a completed and usable procurement requisition for goods and/or services is delivered to an authorized procurement activity. [55] See, for example, [hyperlink, http://www.gao.gov/products/GAO-13-557]. [56] DOD has defined its target defense business systems computing environment as the "to be" environment consisting of (1) the core covered defense business system programs and related resources which DOD will use to conduct its major business processes and (2) the supporting enterprise IT infrastructure and related resources, such as networks, communications, enterprise shared services, and enterprise information assurance, in the enterprise information environment and other mission areas. [57] According to DOD officials, the ETP--using information from October 1, 2013--is missing information about one system. This issue has subsequently been resolved in DITIP. [58] Consistent with the act, the department's investment management guidance defines legacy systems as those that are to be terminated within 36 months of the date of their certification approval; all other systems are considered core systems. [59] The Defense Acquisition Management Information Retrieval system is a DOD initiative that provides enterprise visibility to acquisition program information. [60] DITIP is the authoritative data source for business system certifications; it pulls data from the DITPR and SNAP-IT systems, and also contains other data such as certification request and approval amounts. [61] DITPR is the authoritative data source for system information and primarily contains defense business systems attributes, such as a system description and life-cycle dates. [62] DOD issued an updated version of its investment management guidance in April 2014; however, it was issued too late to be included in this report. [63] These life-cycle phases include material solution analysis, technology development, engineering and manufacturing development, production and deployment, and operations and support. [64] [hyperlink, http://www.gao.gov/products/GAO-13-557]. [65] DOD's annual IT budget submission states that it includes defense business system investments that have been certified by the Defense Business Systems Management Committee. [66] Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (2004), as amended by the NDAA for Fiscal Year 2014, Pub. L. No. 113-66, § 901, 127 Stat. 672, 815 (2013); the NDAA for Fiscal Year 2013, Pub. L. No. 112-239, 126 Stat. 1632 (2012); the NDAA for Fiscal Year 2012, Pub. L. No. 112-81, 125 Stat. 1298 (2011); and the NDAA for Fiscal Year 2010, Pub. L. No. 111-84, 123 Stat. 2190 (2009) (codified in part at 10 U.S.C. § 2222). [67] GAO's IT investment management framework provides a method for evaluating and assessing how well an agency is selecting and managing its IT resources. The framework, which describes five progressive stages of maturity that an agency can achieve in its investment management capabilities, was developed on the basis of our research into the IT investment management practices of leading private-and public-sector organizations. See GAO, Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity, [hyperlink, http://www.gao.gov/products/GAO-04-394G] (Washington, D.C.: March 2004). [68] DOD issued an updated version of its investment management guidance in April 2014; however, it was issued too late to be included in this report. [69] See, for example, [hyperlink, http://www.gao.gov/products/GAO-13-557]. [70] The new version of DOD's BEA was not issued in time to be assessed as part of this report. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]