DOD Business Systems Modernization:

Further Actions Needed to Address Challenges and Improve Accountability

GAO-13-557: Published: May 17, 2013. Publicly Released: May 17, 2013.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6304
melvinv@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Department of Defense (DOD) continues efforts to establish a business enterprise architecture (a modernization blueprint) and transition plan and modernize its business systems and processes, in compliance with key provisions of the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 and amendments. Nonetheless, long-standing challenges remain. The following reflects the status of DOD’s actions to fulfill selected requirements of the act.

  • Develop a business enterprise architecture
    DOD continues to develop content for its business enterprise architecture, such as business rules, and is proceeding with efforts to extend the architecture to its components. However, even though DOD has spent more than 10 years and at least $379 million on its business enterprise architecture, its ability to use the architecture to guide and constrain investments has been limited by, among other things, the lack of a detailed plan.
  • Develop an enterprise transition plan
    The department’s latest version of its transition plan included data on more than 1,200 covered defense business systems; however, important content, such as time-phased milestones and performance measures, is still needed to address the act’s requirements.
  • Establish an investment approval and accountability structure along with an investment review process
    DOD has taken steps to establish a portfolio-based approach to certifying defense business systems, including the establishment of a corporate-level board to oversee the approach and guidance for selecting, controlling, and evaluating the investment portfolio. However, it has yet to fully establish the foundation for its new portfolio-level investment management process or the criteria and procedures for making portfolio-based investment decisions.
  • Certify any business system program costing in excess of $1 million as compliant with the business enterprise architecture and as having undertaken appropriate business process reengineering
    DOD’s portfolio-based investment approach included reviewing and certifying more than 1,200 business systems for fiscal year 2013, totaling about $6.8 billion in funding. However, while DOD continues to perform compliance assertions, it has not ensured the accuracy of business enterprise architecture alignment through validation of individual investments. Further, appropriate business process reengineering assertions were not completed and the associated results and outcomes have yet to be reported.

In addition, the Office of the Deputy Chief Management Officer has yet to determine and follow a strategic approach to managing its human capital needs, thus limiting its ability to, among other things, effectively address the act’s requirements. Collectively, these limitations put the billions of dollars spent annually on approximately 2,100 business system investments that support DOD functions at risk. GAO’s previous recommendations to the department have been aimed at accomplishing these and other activities related to the business systems modernization. However, to date, the department has not implemented 29 of the 63 recommendations that GAO has made in these areas.

According to DOD officials, recent turnover, changes to the act’s requirements significantly expanding the number of systems subject to certification, and the short time frame for implementing the new investment review process contributed to the aforementioned weaknesses. Until DOD implements GAO recommendations and addresses the weaknesses described in this report, it will be challenged in its ability to manage the billions of dollars invested annually in modernizing its business system investments.

Why GAO Did This Study

GAO designated DOD’s multibillion dollar business systems modernization program as high risk in 1995, and, since then, has provided a series of recommendations aimed at strengthening DOD’s institutional approach to modernization and reducing the risk associated with key investments. The act requires the department to report on actions taken relative to its business systems modernization efforts and GAO to assess DOD’s actions to comply with the act. In evaluating DOD’s compliance, GAO analyzed, for example, the latest version of the business enterprise architecture and enterprise transition plan, investment management policies and procedures, and certification actions for its business system investments.

What GAO Recommends

GAO is making recommendations to help ensure that the department’s modernization program is fully compliant with provisions of the act and to improve the department’s architecture, transition plan, and business system investment management and human capital management within the Office of the Deputy Chief Management Officer. DOD concurred with two recommendations, partially concurred with three, and did not concur with three. GAO continues to believe its recommendations are warranted given the department’s need to more effectively manage its billions of dollars of business system investments and minimize or eliminate system overlap and duplication as appropriate.

For more information, contact Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: As of September 2018, the Department of Defense had made progress addressing the recommendation; however, more remained to be done to meet its intent. In July 2015, we reported that the department had taken steps to improve the integration of business enterprise architecture information with other existing information, which allows DOD to identify information such as mapping of existing business systems to system functions. More recently, in September 2017, the department awarded a contract to improve its business enterprise architecture. According to the department, the objective of the contract is to improve business and system optimization by providing mechanisms to ingest and discover enterprise architecture content from all department components and allow for cross-domain portfolio reviews to include duplication analysis. More specifically, the contract includes developing three major capabilities, including the ability to conduct process and system reviews within and across domains. According to officials from the Office of the Chief Management Officer, the three major capabilities have been decomposed into 14 functional requirements that are expected to be delivered incrementally, However, as of September 2018, the functional requirements that were expected to be delivered in June 2018 had yet to be delivered. According to the department, the delay is due to issues with the system that is to provide the technical environment to host and analyze the business enterprise architecture. As of September 2018, the department expected the system to be available in November 2018, and to fully deploy the 14 functional requirements by June 2019. With regard to including business capabilities for the Hire-to-Retire and Process-to-Pay business processes in the business enterprise architecture, the department stated that the new architecture will identify the business capabilities and processes associated with Lines of Business, which will be defined as a decomposition of the products and processes the business enterprise delivers to the department's components. The department did not indicate when capabilities associated with Lines of Business will be defined and their associated capabilities and processes identified. We will continue to monitor the department's efforts to fully implement the recommendation.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to define by when and how the department plans to develop an architecture that would extend to all defense components and include, among other things, (a) information about the specific business systems that support business enterprise architecture (BEA) business activities and related system functions; (b) business capabilities for the Hire-to-Retire and Procure-to-Pay business processes; and (c) sufficient information about business activities to allow for more effective identification of potential overlap and duplication.

    Agency Affected: Department of Defense

  2. Status: Closed - Implemented

    Comments: As of August 2017, the Department of Defense (DOD) had met the intent of this recommendation. According to the department, its transition plan is comprised of information from functional strategies, component organizational execution plans, and data in the DOD Information Technology Portfolio Repository (DITPR), Select and Native Programming Data Input System for Information Technology (SNAP-IT), Integrated Business Framework Data Alignment Portal (IBF-DAP), and the DOD Information Technology Investment Portal (DITIP). In August 2017, the department provided current examples of enterprise transition plan information that met the intent of our recommendation. For example, the department documented that system milestones are maintained in its DITPR system. The department also provided documentation that its system-specific problem statements identify performance measures and that funding information is contained in its SNAP-IT system. In addition, DOD provided examples of risks and challenges to integration associated with a specific system and documented in its IBF-DAP system. Moreover, the department provided examples of time-phased end dates for terminating legacy systems and documentation of business systems that will be part of the target environment from its DITPR system and an example functional strategy. In addition, DOD documented that is maintains information about how systems are to be sequenced in its functional strategies and its business intelligence tool.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to define by when and how the enterprise transition plan will include, among other things, (a) milestones, performance measures, and funding plans for all business systems expected to be part of the target architecture and each system's risks or challenges to integration; (b) time-phased end dates associated with terminating legacy systems in phases; (c) a listing of all other defense business systems (including systems that are considered to be core systems) that will be a part of the target defense business systems computing environment and a strategy for making modifications to those systems that will be needed to ensure that they comply with the defense BEA, including time-phased milestones, performance measures, and financial resource needs; and (d) information about how systems are to be sequenced according to, among other things, dependencies among investments.

    Agency Affected: Department of Defense

  3. Status: Open

    Comments: As of September 2018, the Department of Defense (DOD) had not addressed this recommendation. In May 2013, we reported that for the fiscal year 2013 certification of business systems, functional strategies included many, but not all, of the critical elements required by DOD's guidance. Specifically, not all demonstrated linkages to business goals in DOD's strategic management plan, and not all included expected outcomes for all functional area goals. In addition, some functional strategies, but not all, had performance measures in place for assessing progress toward achieving stated goals. However, none included performance measures that reflected all of the key attributes identified in DOD's guidance. We also reported that for the fiscal year 2014 certification cycle, the functional strategies had been improved. However, not all of them had performance measures that included all key attributes called for in the guidance. Specifically, all performance measures did not include baseline and target measures, and provide a rationale for the identified targets. In June 2018, DOD revised its defense business system investment management guidance. The guidance still requires that functional strategies include business outcomes that link to goals in DOD's strategic management plan. In addition, while the guidance no longer calls for the key performance measure attributes that we assessed in our 2013 report, the updated guidance requires that business outcomes include measurable targets. However, none of the department's fiscal year 2018 functional management strategies fully addressed most of the elements required to be included In functional strategies. For example, none demonstrated that business outcomes are linked to the department's strategic management plan goals or that business outcomes include measurable targets. Furthermore, none assessed progress in achieving business outcomes, as required by the June 2018 investment management guidance. We will continue to monitor the progress of this recommendation as part of our periodic assessments of the department's efforts to mange its defense business systems.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to ensure that the functional strategies include all of the critical elements identified in DOD investment management guidance, including performance measures to determine progress toward achieving the goals that incorporate all of the attributes called for in the department's guidance.

    Agency Affected: Department of Defense

  4. Status: Open

    Comments: As of September 2018, the Department of Defense had not addressed the recommendation. We reported in 2013 that the department's investment management guidance did not specify a process for conducting an assessment or call for the use of actual versus expected performance data and predetermined thresholds for evaluating portfolio performance. In addition, the department did not call for assessments to be conducted in four key areas: benefits attained; current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated, or accepted to date. We also reported in 2013 that the department's investment guidance identified four criteria and specified the associated assessments that were to be conducted when reviewing and evaluating components' organization execution plans in order to make a portfolio-based investment decision; however guidance did not call for the department's organization execution plans to include critical information for conducting assessments associated with strategic alignment (i.e., information on alignment with the capital planning and investment control practices and Better Buying Power guidance), utility (i.e., interoperability among systems and system scalability to support additional users) and total cost (i.e., cost in relationship to return on investment). In response to our request for an update on the status of addressing the recommendation, in September 2018, the department described activities related to improving and validating mission performance and activities related to program management and oversight; however, the department had not addressed the recommendation, which is intended to improve its ability to select and control its mix of investments through portfolio assessments.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to select and control its mix of investments in a manner that best supports mission needs by (a) documenting a process for evaluating portfolio performance that includes the use of actual versus expected performance data and predetermined thresholds; (b) ensuring that portfolio assessments are conducted in key areas identified in our IT investment management framework: benefits attained; current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated, or accepted to date; and (c) ensuring that the documents provided to the Defense Business Council as part of the investment management process include critical information for conducting all assessments.

    Agency Affected: Department of Defense

  5. Status: Open

    Comments: As of September 2018, the Department of Defense (DOD) had taken steps to address the intent of the recommendation; however, more remained to be done. For example, the 2015 Congressional Report on Defense Business Operations included information consistent with the first element of our recommendation. However, the department has not demonstrated that it has implemented the elements of the recommendation related to validating business enterprise architecture (BEA) and business process reengineering assertions. In May 2013, we reported that the department had asserted BEA compliance for many business systems; however, it had not validated these assertions. We also reported that the Office of the Deputy Chief Management Officer had selected 35 defense business systems to inspect and validate business process reengineering assertions, but at the time of our report, results were not expected to be reported to the Defense Business Council until June 2013. We also reported that the department needed to ensure that its efforts inform the business process reengineering to be performed on all investments for the following fiscal year reviews. In September 2018, the Office of the Chief Management Officer (CMO) said that compliance determinations are to be made by component level leadership before approval by the appropriate officials, which include the DOD CMO and the military department CMOs, and that CMO staff then review the submitted documentation and authoritative source information. The department also provided documentation related to a request for an increase in funding for a financial management system. However, the documentation did not demonstrate that BEA and business process reengineering assertions for the system had been validated. We will continue to monitor the department's efforts to ensure that BEA and business process reengineering assertions are validated.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to implement and use the BEA and business process reengineering compliance assessments more effectively to support organizational transformation efforts by (a) disclosing relevant information about known weaknesses, such as BEA and business process reengineering compliance weaknesses for systems that were not certified or certified with qualifications in annual reports to Congress; (b) establishing milestones by which selected validations of BEA compliance assertions are to be completed; and (c) ensuring that appropriate business process reengineering assertions have been completed on all investments submitted for the fiscal year 2014 certification reviews prior to the certification of funds.

    Agency Affected: Department of Defense

  6. Status: Open

    Comments: The Department of Defense (DOD) has not implemented this recommendation. In September 2018, the department stated that the Chief Management Officer (CMO) has responsibility for reforming business operations and that the department formed the Reform Management Group to guide its business reform efforts. According to the department, the Reform Management Group leads dedicated teams, structured based on a strategic approach to human capital planning that used an existing skills inventory to identify staff. The department also said that reform teams effectively obtain needed skills from across the department and by hiring subject matter experts based on needs assessments and gap analyses. However, the department did not provide evidence of a skills inventory, needs assessment, or gap analysis associated with its Reform Management Group teams. Moreover, the department did not demonstrated that it has developed a skills inventory, needs assessment, gap analysis, and plan to address identified gaps within the Office of the Chief Management Officer in order to support more effective implementation of key components of DOD's business systems modernization program.

    Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to develop a skills inventory, needs assessment, gap analysis, and plan to address identified gaps as part of a strategic approach to human capital planning for the Office of the Deputy Chief Management Officer.

    Agency Affected: Department of Defense

  7. Status: Closed - Implemented

    Comments: As of July 2018, the Department of Defense (DOD) has addressed the intent of this recommendation, and other aspects of the recommendation have been overcome by events. In July 2015, we reported that the department demonstrated that it had completed documentation, such as root cause analyses, assessments of existing interfaces for reuse opportunities, and performance metrics related to the reengineering efforts, and that the documentation was provided as part of the certification and approval process for the Air Force Integrated Personnel and Pay System investment. However, since we made the recommendation, the department has changed its approach to evaluating business process reengineering for its defense business systems. As a result of this change, the department requires different documentation than the documentation required by the process we examined in our report. The department requires business process reengineering to be documented in a problem statement. The department's December 2014 problem statement guidance requires a description of and validation that a thorough review of the business process reengineering was conducted, and no longer specifically requires root cause analyses, assessments of existing interfaces for re-use opportunities, or performance metrics related to reengineering efforts. Regarding the Integrated Personnel and Pay System - Army, in September 2017, the department demonstrated that it had completed a March 2016 description of its business process reengineering efforts and supporting documentation as part of its review and certification process. Further, in July 2018, the department demonstrated that complete documentation related to business process reengineering efforts for the Integrated Personnel and Pay System - Navy (IPPS-N) investment had been approved in February 2018 as part of its certification and approval process . Specifically, the IPPS-N problem statement contains a description of and validation that a review of business process reengineering was conducted. Regarding the Integrated Electronic Health Record investment, the Office of the Deputy Chief Management Officer stated that the department does not plan to conduct business process reengineering because the investment is in sustainment, and the department does not require business process reengineering for systems in sustainment.

    Recommendation: The Secretary of Defense should direct the appropriate authority to ensure that complete documentation, such as root cause analyses, assessments of existing interfaces for reuse opportunities, and performance metrics related to the reengineering efforts, is provided as part of the fiscal year 2014 certification and approval process for the Integrated Personnel and Pay System - Army (IPPS-A), Integrated Personnel and Pay System - Navy (IPPS-N), Air Force Integrated Personnel and Pay System (AF-IPPS), and Integrated Electronic Health Record (iEHR) investments.

    Agency Affected: Department of Defense

  8. Status: Closed - Implemented

    Comments: As we reported in July 2015, officials from the Office of the Deputy Chief Management Officer demonstrated that the department has addressed the intent of this recommendation. Specifically, while the department did not concur with the recommendation and did not make the recommended determination, it has taken mitigating steps to help ensure compliance with business process reengineering requirements. For example, officials stated that, as part of the fiscal year 2013 certification and approval process, conditions were imposed by the investment review board requiring all components to submit a plan on how core defense business systems would become compliant with the act's business process reengineering requirement. These officials also provided documentation showing that the department tracked these conditions. In addition, the department has reported much higher levels of compliance with the act's business process reengineering requirements in subsequent annual review cycles. For example, in May 2013, we reported that, according to DOD, appropriate business process reengineering had been undertaken on only about 41 percent of the approximately 1,200 systems for the fiscal year 2013 certification reviews. In contrast, officials from the Office of the Deputy Chief Management Officer stated that only 2 systems were certified and approved during the fiscal year 2014 certification and approval cycle and 6 systems were certified and approved during the fiscal year 2015 certification and approval cycle that did not have complete business process reengineering assertions. Moreover, these officials provided justifications for why each of these systems did not have complete business process reengineering assertions.

    Recommendation: The Secretary of Defense should direct the appropriate authority to determine whether funds were properly obligated under 10 U.S.C. 2222(a)-(b) for systems for which appropriate business process reengineering assertions were not completed.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Dec 14, 2018

Dec 13, 2018

Dec 12, 2018

Dec 4, 2018

Nov 30, 2018

Nov 29, 2018

Nov 19, 2018

Nov 15, 2018

Nov 14, 2018

Looking for more? Browse all our products here