DOD Business Systems Modernization:

Further Actions Needed to Address Challenges and Improve Accountability

GAO-13-557: Published: May 17, 2013. Publicly Released: May 17, 2013.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6304
melvinv@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Department of Defense (DOD) continues efforts to establish a business enterprise architecture (a modernization blueprint) and transition plan and modernize its business systems and processes, in compliance with key provisions of the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 and amendments. Nonetheless, long-standing challenges remain. The following reflects the status of DOD’s actions to fulfill selected requirements of the act.

  • Develop a business enterprise architecture
    DOD continues to develop content for its business enterprise architecture, such as business rules, and is proceeding with efforts to extend the architecture to its components. However, even though DOD has spent more than 10 years and at least $379 million on its business enterprise architecture, its ability to use the architecture to guide and constrain investments has been limited by, among other things, the lack of a detailed plan.
  • Develop an enterprise transition plan
    The department’s latest version of its transition plan included data on more than 1,200 covered defense business systems; however, important content, such as time-phased milestones and performance measures, is still needed to address the act’s requirements.
  • Establish an investment approval and accountability structure along with an investment review process
    DOD has taken steps to establish a portfolio-based approach to certifying defense business systems, including the establishment of a corporate-level board to oversee the approach and guidance for selecting, controlling, and evaluating the investment portfolio. However, it has yet to fully establish the foundation for its new portfolio-level investment management process or the criteria and procedures for making portfolio-based investment decisions.
  • Certify any business system program costing in excess of $1 million as compliant with the business enterprise architecture and as having undertaken appropriate business process reengineering
    DOD’s portfolio-based investment approach included reviewing and certifying more than 1,200 business systems for fiscal year 2013, totaling about $6.8 billion in funding. However, while DOD continues to perform compliance assertions, it has not ensured the accuracy of business enterprise architecture alignment through validation of individual investments. Further, appropriate business process reengineering assertions were not completed and the associated results and outcomes have yet to be reported.

In addition, the Office of the Deputy Chief Management Officer has yet to determine and follow a strategic approach to managing its human capital needs, thus limiting its ability to, among other things, effectively address the act’s requirements. Collectively, these limitations put the billions of dollars spent annually on approximately 2,100 business system investments that support DOD functions at risk. GAO’s previous recommendations to the department have been aimed at accomplishing these and other activities related to the business systems modernization. However, to date, the department has not implemented 29 of the 63 recommendations that GAO has made in these areas.

According to DOD officials, recent turnover, changes to the act’s requirements significantly expanding the number of systems subject to certification, and the short time frame for implementing the new investment review process contributed to the aforementioned weaknesses. Until DOD implements GAO recommendations and addresses the weaknesses described in this report, it will be challenged in its ability to manage the billions of dollars invested annually in modernizing its business system investments.

Why GAO Did This Study

GAO designated DOD’s multibillion dollar business systems modernization program as high risk in 1995, and, since then, has provided a series of recommendations aimed at strengthening DOD’s institutional approach to modernization and reducing the risk associated with key investments. The act requires the department to report on actions taken relative to its business systems modernization efforts and GAO to assess DOD’s actions to comply with the act. In evaluating DOD’s compliance, GAO analyzed, for example, the latest version of the business enterprise architecture and enterprise transition plan, investment management policies and procedures, and certification actions for its business system investments.

What GAO Recommends

GAO is making recommendations to help ensure that the department’s modernization program is fully compliant with provisions of the act and to improve the department’s architecture, transition plan, and business system investment management and human capital management within the Office of the Deputy Chief Management Officer. DOD concurred with two recommendations, partially concurred with three, and did not concur with three. GAO continues to believe its recommendations are warranted given the department’s need to more effectively manage its billions of dollars of business system investments and minimize or eliminate system overlap and duplication as appropriate.

For more information, contact Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: DOD has taken steps to strengthen the management of the business enterprise architecture (BEA) including establishing a BEA configuration control board, comprised of representatives from the military departments and other departmental offices, to make recommendations to the Defense Business Council on BEA requirements and related content changes. However, as of August 2014, DOD has not demonstrated that it has developed a plan that defines by when and how it will develop an architecture that contains important content, such as additional information about business activities, that would further assist in identifying potential duplication and overlap among business systems.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to define by when and how the department plans to develop an architecture that would extend to all defense components and include, among other things, (a) information about the specific business systems that support business enterprise architecture (BEA) business activities and related system functions; (b) business capabilities for the Hire-to-Retire and Procure-to-Pay business processes; and (c) sufficient information about business activities to allow for more effective identification of potential overlap and duplication.

    Agency Affected: Department of Defense

  2. Status: Open

    Comments: DOD has taken steps to improve its enterprise transition plan by including key content. However, additional information is still needed and a plan for developing such content does not exist. For example, the transition plan includes acquisition milestone information for covered defense business systems through its links to other systems, such as the Defense Acquisition Management Information Retrieval, the Defense Information Technology Investment Portal, the Defense Information Technology Portal Repository, and Office of Management and Budget exhibit 300s. The plan also includes portfolio-level performance measures in the organizational execution plans, however, the measures are not linked to specific business systems. Additionally, the plan includes information about fiscal year 2014 funding that was requested and ultimately approved under the business system investment review process for each business system. Further, the plan also includes termination dates for all 79 of the covered legacy systems and a listing of both new and existing systems that will be part of the target defense business systems computing environment. Despite the progress in some areas, the enterprise transition plan is still missing information that will help to ensure that DOD more fully complies with the requirements of the act and the department has yet to define by when and how it will include missing content in the enterprise transition plan.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to define by when and how the enterprise transition plan will include, among other things, (a) milestones, performance measures, and funding plans for all business systems expected to be part of the target architecture and each system's risks or challenges to integration; (b) time-phased end dates associated with terminating legacy systems in phases; (c) a listing of all other defense business systems (including systems that are considered to be core systems) that will be a part of the target defense business systems computing environment and a strategy for making modifications to those systems that will be needed to ensure that they comply with the defense BEA, including time-phased milestones, performance measures, and financial resource needs; and (d) information about how systems are to be sequenced according to, among other things, dependencies among investments.

    Agency Affected: Department of Defense

  3. Status: Open

    Comments: DOD officials stated that the fiscal year 2014 functional strategies included elements called for in both their fiscal year 2014 investment management guidance and DOD strategic plans. Our review of the fiscal year 2014 strategies showed five of the six functional strategies had fully documented performance measures and all six identified one or more expected outcomes for each functional area goal. Nevertheless, DOD had not established performance measures in which all key attributes identified in its guidance were present. Specifically, the strategies lacked the following five attributes: 1) quantitative data (e.g. cost, production, defect, and time), 2) data that is tracked incrementally over a specified period, 3) a baseline for each performance measure, 4) a target against the baseline, and 5) a rationale for the identified target. In addition, our review of selected fiscal year 2015 functional strategies showed that these more recent strategies also lacked such attributes. More recently, the Office of the Deputy Chief Management Officer (DCMO) stated that the office is revising its investment management guidance to include requirements for including these missing key attributes.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to ensure that the functional strategies include all of the critical elements identified in DOD investment management guidance, including performance measures to determine progress toward achieving the goals that incorporate all of the attributes called for in the department's guidance.

    Agency Affected: Department of Defense

  4. Status: Open

    Comments: In April 2014, the department issued investment management guidance that identifies four criteria and specifies the associated assessments that are to be conducted when reviewing and evaluating component-level organizational execution plans in order to make a portfolio-based investment decision. However, the guidance does not specify a process for conducting an assessment or call for the use of actual versus expected performance data and predetermined thresholds. In addition, the guidance does not specify that the investment portfolios will be evaluated based on factors such as benefits attained; current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated, or accepted to date.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to select and control its mix of investments in a manner that best supports mission needs by (a) documenting a process for evaluating portfolio performance that includes the use of actual versus expected performance data and predetermined thresholds; (b) ensuring that portfolio assessments are conducted in key areas identified in our IT investment management framework: benefits attained; current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated, or accepted to date; and (c) ensuring that the documents provided to the Defense Business Council as part of the investment management process include critical information for conducting all assessments.

    Agency Affected: Department of Defense

  5. Status: Open

    Comments: The March 2014 annual report to Congress on Defense Business Systems contained some information about why some systems were conditionally or not certified due to weaknesses in their Business Process Reengineering (BPR) documentation. More specifically, the report stated that the Defense Business Council (DBC) did not certify $188 million in requests for development/modernization funds and that it conditionally certified $563 million in development/modernization requests pending DBC review and approval of documentation associated with analyzing business problems, capability gaps, or opportunities. The annual report also stated that DBC conducted quality checks and identified more than 300 systems at risk of not being certified for fiscal year 2014 due to insufficient Business Enterprise Architecture (BEA) compliance assessments. According to the report, additional assistance was provided to ensure that the compliance assessment quality improved and subsequently all identified risks among systems were resolved. In addition, the office of the Deputy Chief Management Officer (DCMO) validated a sample of BEA assertions and identified concerns associated with 13 of the 15 that it validated. For example, according to the office of the DCMO, 9 of the 13 had inconsistencies in the organizational execution plan or had significantly incomplete compliance assertions in the BEA Compliance System. According to DOD officials, some of these concerns were addressed prior to system certification. They further stated that no systems were denied funding certification due to incomplete BEA compliance assertions because the Office of the DCMO worked with the components to address the issues to the satisfaction of the DBC prior to certification. Where additional action was needed, action items were created and DCMO is tracking the progress of these systems efforts to become compliant. However, information about these detailed assessments and actions taken to resolve them was not reported. In addition to including some information about systems that were conditionally or not certified due to weaknesses in their BPR documentation, the report also included summary information about known BPR weaknesses based on a DBC-level effort to validate a sample of BPR assertions for systems that were submitted for certification. For example, the annual report stated that 12 systems were evaluated for BPR compliance, and these evaluations concluded that more needed to be done to improve program execution of BPR compliance requirements. More specifically, with respect to ensuring that appropriate BPR assertions have been completed on all investments submitted for certification, the department reported that its BPR assessment team was unable to determine that sufficient BPR was conducted on 4 of the 12 systems reviewed. Accordingly, the team recommended that the Pre-Certification Authorities for each system ensure BPR compliance is assessed at the next milestone. However, though information on BPR compliance validations was provided at high-level, detailed information regarding all known weaknesses was not published in the March report to Congress. For example, the report did not disclose the names of the four systems for which the DBC review was unable to determine if appropriate BPR had been conducted.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to implement and use the BEA and business process reengineering compliance assessments more effectively to support organizational transformation efforts by (a) disclosing relevant information about known weaknesses, such as BEA and business process reengineering compliance weaknesses for systems that were not certified or certified with qualifications in annual reports to Congress; (b) establishing milestones by which selected validations of BEA compliance assertions are to be completed; and (c) ensuring that appropriate business process reengineering assertions have been completed on all investments submitted for the fiscal year 2014 certification reviews prior to the certification of funds.

    Agency Affected: Department of Defense

  6. Status: Open

    Comments: According to the Office of Deputy Chief Management Officer, the department has no plans in place to develop a skills inventory, needs assessment, gap analysis, and address identified gaps as part of a strategic approach to human capital planning. Therefore, the recommendation remains open.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to develop a skills inventory, needs assessment, gap analysis, and plan to address identified gaps as part of a strategic approach to human capital planning for the Office of the Deputy Chief Management Officer.

    Agency Affected: Department of Defense

  7. Status: Open

    Comments: DOD has yet to demonstrate that complete documentation, such as root cause analyses, assessments of existing interfaces for reuse opportunities, and performance metrics related to the reengineering efforts, is provided as part of the certification and approval process for the Integrated Personnel and Pay System - Army (IPPS-A), Integrated Personnel and Pay System - Navy (IPPS-N), Air Force Integrated Personnel and Pay System (AF-IPPS), and Integrated Electronic Health Record (iEHR) investments. In particular, according to the Office of the Deputy Chief Management Officer, as of January 2014, IPPS-A, IPPS-N, AF-IPPS, and iEHR have yet to be reevaluated to determine whether BPR compliance documentation is complete.

    Recommendation: The Secretary of Defense should direct the appropriate authority to ensure that complete documentation, such as root cause analyses, assessments of existing interfaces for reuse opportunities, and performance metrics related to the reengineering efforts, is provided as part of the fiscal year 2014 certification and approval process for the Integrated Personnel and Pay System - Army (IPPS-A), Integrated Personnel and Pay System - Navy (IPPS-N), Air Force Integrated Personnel and Pay System (AF-IPPS), and Integrated Electronic Health Record (iEHR) investments.

    Agency Affected: Department of Defense

  8. Status: Open

    Comments: DOD did not concur with this recommendation and has yet to demonstrate that it has taken steps to determine whether funds were properly obligated under 10 U.S.C. 2222(a)-(b) for systems for which appropriate business process reengineering assertions were not completed.

    Recommendation: The Secretary of Defense should direct the appropriate authority to determine whether funds were properly obligated under 10 U.S.C. 2222(a)-(b) for systems for which appropriate business process reengineering assertions were not completed.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Sep 19, 2014

Sep 18, 2014

Sep 10, 2014

Sep 9, 2014

Sep 8, 2014

Jul 31, 2014

Looking for more? Browse all our products here