Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to define by when and how the department plans to develop an architecture that would extend to all defense components and include, among other things, (a) information about the specific business systems that support business enterprise architecture (BEA) business activities and related system functions; (b) business capabilities for the Hire-to-Retire and Procure-to-Pay business processes; and (c) sufficient information about business activities to allow for more effective identification of potential overlap and duplication.
Agency: Department of Defense
Status: Open
Comments: DOD has made significant progress addressing the recommendation; however, as of November 2019, more remained to be done. In particular, in 2015, we reported that the department had taken steps to improve the integration of business enterprise architecture information with other existing information, which allows DOD to identify information such as mapping of existing business systems to system functions. More recently, in 2017, the department awarded a contract to improve its business enterprise architecture. According to the department, the objective of the contract was to improve business and system optimization by providing mechanisms to ingest and discover enterprise architecture content from all department components and allow for cross-domain portfolio reviews to include duplication analysis. More specifically, the contract called for developing three major capabilities, including the ability to conduct process and system reviews within and across domains. In October 2019, the Office of the Chief Management Officer (CMO) demonstrated that it had completed development of the three planned capabilities and the office said it was working to host the capabilities in a government-approved cloud environment. With regard to including business capabilities for the Hire-to-Retire and Procure-to-Pay business processes in the business enterprise architecture, the department stated that the new architecture is to identify the business capabilities and processes associated with Lines of Business, which will be defined as a decomposition of the products and services that the business enterprise delivers to the department's components. In September 2019, officials from the Office of the CMO stated that the department plans to review end-to-end processes that comprise the current business enterprise architecture for currency and relevancy. However, the officials did not indicate when they expect to complete this review.
Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to ensure that the functional strategies include all of the critical elements identified in DOD investment management guidance, including performance measures to determine progress toward achieving the goals that incorporate all of the attributes called for in the department's guidance.
Agency: Department of Defense
Status: Open
Comments: As of November 2019, the Department of Defense (DOD) had not addressed the recommendation. In May 2013, we reported that, for the fiscal year 2013 certification of business systems, functional strategies included many, but not all, of the critical elements required by DOD's guidance. Specifically, not all functional strategies demonstrated linkages to business goals in DOD's strategic management plan, and not all included expected outcomes for all functional area goals. In addition, some, but not all, had performance measures in place for assessing progress toward achieving stated goals. However, none of the functional strategies included performance measures that reflected all of the key attributes identified in DOD's guidance. We also reported that for the 2014 certification cycle, the functional strategies had been improved. However, not all of them had performance measures that included all key attributes called for in the guidance. Specifically, all performance measures did not include baseline and target measures, and provide a rationale for the identified targets. In June 2018, DOD revised the required functional strategy elements in its defense business system investment management guidance. However, as of November 2019, the department had not ensured that its functional strategies include all of the elements identified in the guidance. The guidance still requires that functional strategies include business outcomes that link to goals in DOD's strategic management plan. In addition, while the guidance no longer calls for the key performance measure attributes that we assessed in our 2013 report (i.e., baseline and target measures and a rationale for identified targets), the new guidance requires that business outcomes include measurable targets. However, none of the fiscal year 2019 functional strategies fully addressed most of the required elements. For example, none of the functional strategies demonstrated that business outcomes were clearly linked to the department's strategic management plan goals, as required by the 2018 investment management guidance. In addition, none of the strategies included measurable targets, An official from the office of the Chief Management Officer (CMO) demonstrated that the department's Integrated Business Framework-Data Alignment Portal, which is used to record functional strategies, includes business outcomes that are aligned to goals and objectives in the National Defense Business Operations Plan (i.e., the agency strategic plan). The official also demonstrated that most functional strategies link to at least one performance measure from the National Defense Business Operations Plan. However, the official agreed that the published functional strategies did not clearly link outcomes to the department's strategic management plan. Further, officials from the office of the CMO stated in September 2019 that the functional strategies for fiscal year 2019 were not revised for fiscal year 2020.
Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to select and control its mix of investments in a manner that best supports mission needs by (a) documenting a process for evaluating portfolio performance that includes the use of actual versus expected performance data and predetermined thresholds; (b) ensuring that portfolio assessments are conducted in key areas identified in our IT investment management framework: benefits attained; current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated, or accepted to date; and (c) ensuring that the documents provided to the Defense Business Council as part of the investment management process include critical information for conducting all assessments.
Agency: Department of Defense
Status: Open
Comments: As of November 2019, DOD had not addressed the recommendation. In 2013, we reported that the department's investment management guidance did not specify a process for conducting an assessment or call for the use of actual versus expected performance data and predetermined thresholds for evaluating portfolio performance. In addition, the department did not call for assessments to be conducted in four key areas-benefits attained, current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated or accepted to date. We also reported in 2013, that the department's investment management guidance identified four criteria and specified the associated assessments that were to be conducted when reviewing and evaluating components' organization execution plans in order to make a portfolio-based investment decision. However, the guidance did not call for the department's organizational execution plans to include critical information for conducting assessments associated with three of the four criteria. Specifically, it did not include information for conducting assessments associated with strategic alignment (i.e., information on alignment with the capital planning and investment control practices and Better Buying Power guidance), utility (i.e., interoperability among systems and system scalability to support additional users) and total cost (i.e., cost in relationship to return on investment). In September 2019, the department stated that the Office of the Chief Management Officer's investment management guidance, investment management training materials, and organizational execution plan, addressed elements of the recommendation. However, the documents did not specify a process for evaluating portfolio performance that includes the use of actual versus expected performance data and predetermined thresholds. Regarding ensuring that portfolio assessments are conducted in key areas identified in our IT investment management framework: benefits attained; current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated, or accepted to date, the June 2018 investment management guidance requires organization execution plans to include risks and risk mitigation strategies. In addition, the investment management guidance requires the plans to include information about benefits attained. Specifically, the plans are to include progress against targets for business goals documented in functional strategies. However, the guidance does not address the remaining key areas identified in our IT investment management framework: current schedule; accuracy of project reporting; and risks that have been eliminated or accepted to date. In addition, the guidance does not call for this information to be used as part of portfolio assessments. Regarding ensuring that the documents provided to the Defense Business Council as part of the investment management process include critical information for conducting assessments, in September 2019, the department stated in a written response that business system certification decisions are made in accordance with criteria established in 10 U.S. Code Section 2222. However, as of November 2109, the department had not demonstrated that it established guidance that calls for documents to include critical information on alignment with the capital planning and investment control practices and Better Buying Power guidance), utility (i.e., interoperability among systems and system scalability to support additional users) and total cost (i.e., cost in relationship to return on investment), which are criteria it established in its investment management guidance for making certification decisions.
