Recommendation: The Secretary of the Army should direct the Army Futures Command, in collaboration with the Army Contracting Command and the Assistant Secretary of the Army for Acquisition, Logistics, and Technology, to regularly analyze information on the use of prototype other transactions for Army modernization. (Recommendation 1)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Army should direct the Army Futures Command, in collaboration with the Army Contracting Command, to regularly analyze information on the use of grants, cooperative agreements, research other transactions, technology investment agreements, partnership intermediary agreements, and cooperative research and development agreements for Army modernization. (Recommendation 2)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Army should ensure the Commanding General of the Army Contracting Command establishes consistent practices for its headquarters to collect, archive, and share lessons learned for research and prototype other transactions, grants, cooperative agreements, technology investment agreements, and partnership intermediary agreements. (Recommendation 3)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Army should ensure the Commanding General of the Army Contracting Command has the command's contracting centers work with its headquarters to establish consistent, coordinated practices for the contracting centers to collect, analyze, validate, archive, and share lessons learned for research and prototype other transactions, grants, cooperative agreements, technology investment agreements, and partnership intermediary agreements. (Recommendation 4)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Army should ensure the Commanding General of the Combat Capabilities Development Command establishes consistent, coordinated practices for the command to collect, analyze, validate, archive, and share lessons learned for cooperative research and development agreements. (Recommendation 5)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of the Army should ensure an archive is established to store and share lessons learned information related to the Army's alternative approaches for engaging industry and academia. (Recommendation 6)

Agency: Department of Defense: Department of the Army
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Commissioner of the Social Security Administration should ensure the Office of Acquisition and Grants, in consultation with the Chief Information Officer, revises relevant guidance to identify the role of contracting officials in the Agile software development process.

Agency: Social Security Administration
Status: Open

Comments: The Social Security Administration agreed with the recommendation but has not yet taken actions to implement it.

Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops a program-wide process for measuring, collecting, and tracking information on how ALIS is affecting the performance of the F-35 fleet to include, but not be limited to, its effects on mission capability rates. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops and implements a strategy for the re-design of ALIS. The strategy should be detailed enough to clearly identify and assess the goals, key risks or uncertainties, and costs of re-designing the system. (Recommendation 2)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Agriculture should direct Departmental Administration to work with the mission areas to develop department-level outcome-oriented performance goals and related measures for the business centers, and use them to assess the effectiveness and impact of the business center reforms. (Recommendation 1)

Agency: Department of Agriculture
Status: Open

Comments: In January 2020, USDA officials agreed with our recommendation and stated that the department is evaluating options for the development of performance metrics and inclusion of these metrics and related information as part of the regular and recurring reviews by the department's Deputy Secretary who is identified as the Chief Operating Officer.

Recommendation: The Chairman of FCC should develop specific, measurable goals and performance measures for its efforts to monitor the performance of new WEA capabilities, such as enhanced geo-targeting and expanded alert message length. (Recommendation 1)

Agency: Federal Communications Commission
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Director of the IPAWS program should document how it plans to address key actions needed to educate alerting authorities in their use of IPAWS and implement a mechanism that will allow FEMA to regularly and systematically obtain and analyze feedback on alerting authorities' educational needs. (Recommendation 2)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Director of the IPAWS program should establish procedures to prioritize pending IPAWS applications and to follow up with applicants to address these applications. (Recommendation 3)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Associate Director of OPM's Retirement Services, working in coordination with the Chief Information Officer, should develop, document, and implement a Retirement Services IT modernization plan for initial project phases that is consistent with key aspects of IT project management, such as determining objectives, costs, and time frames for each initial phase. (Recommendation 1)

Agency: Office of Personnel Management
Status: Open
Priority recommendation

Comments: OPM partially concurred with the recommendation. OPM stated that it has established initial high-level funding estimates for each of its five key IT initiatives. OPM cited that its ability to implement the modernization plan depends on the availability of funding and coordination with the agency's top leadership. In December 2019, OPM stated that it was evaluating what corrective actions it plans to take. In April 2020, GAO contacted OPM about the status of this recommendation and we are awaiting a response.

Recommendation: The Associate Director of OPM's Retirement Services should adopt management practices to enhance the use of performance information on processing timeliness to inform how OPM manages operations, identifies problem areas, and allocates resources. For example, OPM could enhance use of performance measures at the operational level or establish a timeliness performance goal for reviewing disability retirement eligibility. (Recommendation 2)

Agency: Office of Personnel Management
Status: Open

Comments: OPM partially concurred with the recommendation and responded that it measures overtime spending, reviews daily work level in each work unit, and assesses employee productivity in these units. Collecting and reviewing such operational-level data contributes to monitoring efforts; however the recommendation emphasizes the importance of using performance information to better manage operations to align with organizational goals. In April 2020, GAO contacted OPM about the status of this recommendation and we are awaiting a response.

Recommendation: The Associate Director of OPM's Retirement Services should provide explanatory information, such as the range of processing times and the exclusion of disability retirement eligibility determinations, as part of the performance measure on processing timeliness. (Recommendation 3)

Agency: Office of Personnel Management
Status: Open

Comments: OPM partially concurred with the recommendation. OPM agreed to add an explanation about disability retirement eligibility determinations to its public reports. OPM disagreed that reporting data on the range of processing times would be beneficial because, according to OPM, it provides processing information through other means, such as through applicants' online accounts and agency benefit officers. OPM also acknowledged that it already collects and shares such data. In April 2020, GAO contacted OPM about the status of this recommendation and we are awaiting a response.

Recommendation: The Associate Director of OPM's Retirement Services should develop and implement policies and procedures for assessing strategies intended to improve processing times, including collecting and improving data needed to support those strategies, such as collecting better productivity data or staffing data and linking them to processing outcomes. (Recommendation 4)

Agency: Office of Personnel Management
Status: Open

Comments: OPM partially concurred with the recommendation. OPM stated that a new case management system could provide better productivity and staffing data with which to assess effectiveness, but is dependent on funding and IT support. In April 2020, GAO contacted OPM about the status of this recommendation and we are awaiting a response.

Recommendation: The Associate Director of OPM's Retirement Services should examine its process for assessing its assistance to agencies on retirement applications. For example, OPM could incorporate into its assessment process more agency feedback or documentation of assessment results, which could improve its partnership with agencies to strengthen the assistance provided. (Recommendation 5)

Agency: Office of Personnel Management
Status: Open

Comments: OPM concurred with the recommendation. In April 2020, GAO contacted OPM about the status of this recommendation and we are awaiting a response.

Recommendation: The Associate Director of OPM's Retirement Services should work with agencies to determine if there are cost-effective ways to make the retirement application error report that it sends to agencies more user-friendly. For example, explore whether there are cost-effective ways to provide the error report in a format that could be manipulated (e.g., Excel spreadsheet), or to include additional information, such as incorporating disability retirement applications or providing clearer descriptions of errors or trend data, some of which OPM already collects. (Recommendation 6)

Agency: Office of Personnel Management
Status: Open

Comments: OPM partially concurred with the recommendation. OPM stated that it will explore using MS Excel spreadsheets and incorporating clearer descriptions of errors and data trends. OPM also stated that collecting disability application error information is not an inexpensive or simple process change. In April 2020, GAO contacted OPM about the status of this recommendation and we are awaiting a response.

Recommendation: The FEMA Administrator should ensure that the GMM program management office finalizes the organizational change management plan and time frames for implementing change management actions. (Recommendation 1)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the GMM program management office updates the program schedule to address the leading practices for a reliable schedule identified in this report. (Recommendation 4)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the FEMA Office of the Chief Information Officer (OCIO) defines sufficiently detailed planned evaluation methods and actual evaluation methods for assessing security controls. (Recommendation 5)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the FEMA OCIO approves a security assessment plan before security assessment reviews are conducted. (Recommendation 6)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the GMM program management office follows DHS guidance on preparing corrective action plans for all security vulnerabilities. (Recommendation 7)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the GMM program management office fully tests all of its security controls for the system. (Recommendation 8)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: OMB's Shared Services Policy Officer should work with GSA to finalize a plan for monitoring the implementation of NewPay. The plan should include:
• implementation goals, a timeline, and milestones for agencies to transition from one provider to another;
• transparent reporting mechanisms on key milestones; and
• a process for capturing and communicating lessons learned.
(Recommendation 1)

Agency: Executive Office of the President: Office of Management and Budget
Status: Open

Comments: OMB staff did not agree or disagree with GAO's March 2019 recommendation. As of January 2020, OMB had not addressed this recommendation. According to OMB staff, they are still deliberating goals and milestones for NewPay. In addition, OMB is working on a knowledge library to capture lessons learned for shared services initiatives. However, OMB staff did not provide a timeline for when they will complete these efforts. Without a detailed monitoring plan that includes goals, transparent reporting mechanisms on key milestones, and a process for capturing and communicating lessons learned, it will be more difficult for OMB and GSA to provide oversight of the transition and its effects on providers and customers, including whether there are interruptions to delivery of services. Additionally, this information could help OMB and GSA avoid gaps in service or costly delays as agencies transition to the new model for obtaining payroll and work management services.

Recommendation: OMB's Shared Services Policy Officer should work with GSA to document key roles and responsibilities, including which agency will be the NewPay Service Management Offices (SMO), who will be assigned to the NewPay Task Order Review Board, how the SMO, the Review Board, and other key stakeholders will work together, and which agency will be responsible for interpreting payroll rules and regulations. (Recommendation 2)

Agency: Executive Office of the President: Office of Management and Budget
Status: Open

Comments: OMB staff did not agree or disagree with GAO's March 2019 recommendation. As of January 2020, OMB had taken some steps to address this recommendation. OMB designated GSA as the Quality Service Management Office (QSMO) for NewPay. As QSMO, GSA will be responsible for managing the payroll marketplace. However, OMB has not documented how GSA and other key stakeholders will work together. For example, OMB has not documented which agency will review and approve task orders. Until OMB and GSA clearly identify, communicate, and document key roles and responsibilities, they run the risk of not achieving their objectives. They also risk repeating past problems associated with the delivery of shared services, such as the inconsistent implementation and interpretation of standards and migrations that encounter costly delays should agencies not follow available guidance.

Recommendation: OMB's Shared Services Policy Officer should work with GSA to update provider information on services offered, pricing, and performance and share that information with prospective customers. (Recommendation 3)

Agency: Executive Office of the President: Office of Management and Budget
Status: Open

Comments: OMB staff did not agree or disagree with GAO's March 2019 recommendation. As of January 2020, OMB had not addressed this recommendation. Without up-to-date information on providers -- such as the services OMB and GSA plan to offer, their level of performance, and their costs -- it will be time- consuming and difficult for potential customers to compare providers. This lack of information could slow the rate of shared services adoption.

Recommendation: OMB's Shared Services Policy Officer should work with GSA to implement a process for collecting and tracking cost-savings data that would allow them to assess progress toward the shared services cost-savings goal of an estimated $2 billion over 10 years. (Recommendation 4)

Agency: Executive Office of the President: Office of Management and Budget
Status: Open

Comments: OMB staff did not agree or disagree with GAO's March 2019 recommendation. As of January 2020, OMB had not addressed this recommendation. According to OMB officials, OMB will collect cost-savings data via the integrated data collection process, which requires agencies to publicly post their cost savings and avoidance data. However, OMB has not demonstrated how it plans to use data from the IT Integrated Data Collection Instrument to track cost savings specifically related to shared services overall and for individual projects. Until OMB and GSA finalize a plan for collecting the needed data and evidence to effectively measure cost-savings goals, it will be difficult to demonstrate progress -- a recurring challenge associated with previous shared services efforts.

Recommendation: The Secretary of Veterans Affairs should ensure that the role and responsibilities of the Interagency Program Office are clearly defined within the governance plans for acquisition of the department's new electronic health record system. (Recommendation 1)

Agency: Department of Veterans Affairs
Status: Open
Priority recommendation

Comments: The Department of Veterans Affairs (VA) concurred with our recommendation to ensure that the role and responsibilities of the Interagency Program Office (IPO) were clearly defined within the governance plans for acquisition of the department's new electronic health record system. As of December 2019, VA and the Department of Defense (DOD) have replaced the IPO with a new joint governance body. Specifically, the Federal Electronic Health Record Modernization (FEHRM) program office has been established to serve as the single point of accountability in the delivery of a common health record between the departments and the advancement of interoperability with the private sector. In its charter, the FEHRM was described as a single decision-making authority to manage issues in support of the departments' integrated electronic health record objectives and its leadership is responsible for, among other things, working to formulate, oversee, de-conflict, and ensure adherence to electronic health record-related VA and DOD policies. However, the corresponding Implementation Plan that is intended to document how the FEHRM executes its full responsibilities has yet to be issued. To fully implement this recommendation, VA needs to document the role and responsibilities of the FEHRM with respect to VA's acquisition of its new electronic health record system, explaining the role, if any, the FEHRM will have in the governance process. We will continue to monitor the departments' incorporation of the FEHRM into the plans for the ongoing acquisition.

Recommendation: The Commissioner of the IRS should ensure the operational analysis for IMF fully addresses greater utilization of technology or consolidation of investments to better meet organizational goals. (Recommendation 1)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In August 2019, IRS provided its fiscal year 2018 Operational Analysis Results report, dated June 24, 2019. The report demonstrated that IRS, in response to our recommendation, had ensured that the operational analysis for IMF fully addressed greater utilization of technology or consolidation of investments to better meet organizational goals. However, the operational analysis did not reflect IRS's progress to date in modernizing IMF and the associated challenges. As we reported, this omission is concerning given the risk exposure from the agency's continued use of the legacy assembly language code. In order to close the recommendation, IRS needs to update the operational analysis to reflect its progress modernizing IMF.

Recommendation: The Commissioner of the IRS should ensure the operational analysis for Telecommunications Systems and Support (TSS) addresses the extent to which the investments support customer processes as designed, and how well the investments are delivering the goods or services they were designed to deliver. (Recommendation 3)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In August 2019, IRS provided its fiscal year (FY) 2018 Operational Analysis Results report, dated June 24, 2019. While the report included a summary of the FY 2018 operational analysis for TSS, it did not identify the metrics used to determine whether TSS supported customer processes or delivered the goods and services that it is intended to deliver. To close this recommendation, IRS will need to provide the detailed operational analysis for TSS incorporating these metrics. As of December 2019, IRS has not provided the full TSS operational analysis to GAO. Upon receiving the document, we will review it to determine if IRS has implemented the recommendation.

Recommendation: The Commissioner of the IRS should ensure the operational analysis for TSS includes a comparison of current performance with a pre-established cost baseline. (Recommendation 4)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In August 2019, IRS provided GAO its fiscal year (FY) 2018 Operational Analysis Results report. While the report included a summary of the FY 2018 operational analysis for the Telecommunications Systems and Support (TSS) investment , including planned and actual cost figures for FY2018, the report did not indicate whether the planned cost figure for FY2018 accounted for reimbursable costs and user fees, as we reported. To address this recommendation, IRS will need to provide a full operational analysis for TSS, as well as documentation showing whether reimbursable costs and user fees are included in the planned cost figure. As of December 2019, IRS has not provided a full TSS operational analysis to GAO. Upon receiving the document, we will review it to determine if IRS has implemented the recommendation.

Recommendation: The Commissioner of the IRS should ensure the operational analysis for End User Systems and Services includes a comparison of current performance with a pre-established cost baseline. (Recommendation 5)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In August 2019, IRS provided its fiscal year (FY) 2018 Operational Analysis Results report, dated June 24, 2019. While the report included a summary of the FY 2018 operational analysis for End User Systems and Services (EUSS) investment, including planned and actual cost figures for FY2018, it did not specify whether the planned cost figure accounted for multi-year funding and user fees, as we reported. To address this recommendation, IRS will need to provide a full operational analysis for EUSS, as well as documentation showing whether multi-year funding and user fees are included in the planned cost figure. As of December 2019, IRS has not provided the full EUSS operational analysis to GAO. Upon receiving it, we will review it to determine if IRS has implemented the recommendation.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with preparing for risk management for the IMF investment. (Recommendation 7)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by November 2019. As of December 2019, IRS has not provided any updates on the status of the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with analyzing risk for the IMF investment. (Recommendation 8)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by November 2019. As of December 2019, IRS has not provided any updates on the status of the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice for prioritizing risk for the IMF investment. (Recommendation 9)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by November 2019. As of December 2019, IRS has not provided any updates indicating whether the agency has implemented the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with mitigating risk for the IMF investment. (Recommendation 10)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by November 2019. As of December 2019, IRS has not provided any updates on the status of the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with monitoring, reporting, and controlling risk for the IMF investment. (Recommendation 11)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by November 2019. As of December 2019, IRS has not provided any updates on the status of the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with preparing for risk management for the IDRS investment. (Recommendation 12)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by November 2019. As of December 2019, IRS has not provided any updates indicating whether the agency has implemented the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with analyzing risk for the IDRS investment. (Recommendation 13)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by November 2019. As of December 2019, IRS has not provided any updates on the status of the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with mitigating risk for the IDRS investment. (Recommendation 14)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by November 2019. As of December 2019, IRS has not provided any updates indicating whether the agency has implemented the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with monitoring, reporting, and controlling risk for the IDRS investment. (Recommendation 15)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by November 2019. As of December 2019, IRS has not provided any updates indicating whether the agency has implemented the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with preparing for risk management for the MSSS investment. (Recommendation 16)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In November 2019, IRS provided its IT Enterprise Operations Mainframe and Servers Services and Support (MSSS) Risk Management Plan, dated October 7, 2019. While the plan addressed most of the activities associated with the preparing for risk management key practice, it did not identify risk constraints, risk assumptions, or risk tolerance for the MSSS investment. Upon receiving further information, we will review it to determine if IRS has fully addressed this recommendation.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with analyzing risk for the MSSS investment. (Recommendation 18)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by October 2019. In November 2019, IRS provided its IT Enterprise Operations Mainframe and Servers Services and Support (MSSS) Risk Management Plan, dated October 7, 2019, along with several other documents associated with the agency's IT risk management process. However, the documents do not demonstrate that IRS has implemented the activities associated with the Analyze Risk key practice. Specifically, while the plan describes a risk analysis process in which risks are classified as high, medium, or low risk, neither the plan nor any of the other documents describes criteria for evaluating and quantifying risk likelihood and severity (impact) levels. Additionally, the Risk Management Plan does not indicate whether analysis of MSSS risks includes both inherent and residual risks. Upon receiving additional information indicating that IRS has addressed these activities, we will review it to determine if IRS has implemented the recommendation.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with mitigating risk for the MSSS investment. (Recommendation 19)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by October 2019. In November 2019, IRS provided its IT Enterprise Operations Mainframe and Servers Services and Support (MSSS) Risk Management Plan, dated October 7, 2019, along with several other documents associated with the agency's IT risk management process. However, the documents do not demonstrate that IRS has established threshold values for MSSS risk categories or alternative courses of action for critical risks. Upon receiving additional information indicating that it has addressed these activities. we will review it to determine if IRS has implemented the recommendation.

Recommendation: The Commissioner of the IRS should fully implement the risk management key practice associated with monitoring, reporting, and controlling risk for the MSSS investment. (Recommendation 20)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it would implement the recommendation by October 2019. In November 2019, IRS provided its IT Enterprise Operations Mainframes and Servers Services and Support (MSSS) Risk Management Plan, dated October 7, 2019, along with several other documents associated with the agency's IT risk management process. However, the documents do not demonstrate that IRS has fully implemented all of the activities associated with the monitoring, reporting, and controlling key practice. Specifically, our review of the documents shows that IRS has not established threshold values for MSSS risk categories, and as a result is unable to compare the status of risks to acceptability thresholds to determine the need for implementing a risk mitigation plan. In addition, although the MSSS Risk Management Plan was updated in October 2019, its previous revision occurred in October 2017, indicating that IRS has not yet reviewed all aspects of the risk management program at least once a year. Upon receiving additional information that IRS has addressed these activities, we will review it to determine if IRS has implemented the recommendation.

Recommendation: The Commissioner of the IRS should fully implement IT workforce planning practices, including the following actions (1) setting the strategic direction for workforce planning; (2) analyzing the workforce to identify skill gaps; (3) developing strategies and implementing activities to address skill gaps; and (4) monitoring and reporting on progress in addressing skill gaps. (Recommendation 21)

Agency: Department of the Treasury: Internal Revenue Service
Status: Open

Comments: In September 2018, IRS told GAO it had initiated efforts to address workforce planning agency-wide. The agency stated that the Human Capital Office in coordination with the Information Technology organization prioritizes critical skills gaps to develop gap mitigation strategies, which are implemented through IT annual training plans and succession planning efforts. IRS also stated that the mitigation plans will be monitored in the current Project and Portfolio Management System and that the Human Capital and Information Technology organizations will monitor resource capacity, skills, assigned work effort, and staff availability. In addition, IRS stated that it would utilize special hiring authorities as a competency and staffing mitigation strategy. The agency noted that the special authorities are subject to the availability of resources and agency approval. Further, IRS stated that, due to the diversion of IT resources to the Tax Cuts and Jobs implementation, development of a plan for scaling and expansion of workforce planning efforts will commence after the opening of Filing Season 2020. IRS stated that, due to those constraints, it could not provide a date for fully implementing the recommendation. As of December 2019, IRS has not provided any updates indicating whether it has implemented the recommendation. When we confirm what actions IRS has taken, we will provide updated information.

Recommendation: The Secretary of Defense should ensure that the DOD Chief Information Officer (CIO) develops an IT enterprise architecture which includes a transition plan that provides a road map for improving the department's IT and computing infrastructure, including for each of its business processes. (Recommendation 5)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: In October 2019, the DOD CIO developed a report on the first increment of version 3 of the department's information enterprise architecture (IEA). The report includes high-level descriptions of the current and target architectures, and high-level plans and schedules for transitioning from the current to the target architecture. The report states that because of the incremental approach to developing the architecture, the plans and schedules are notional and depend on several factors over which the DOD CIO has limited or no control, such as funding and changing world events, priorities, and technology. The report also describes plans to integrate the IEA with the department's business enterprise architecture. However, the report did not define a specific time frame for integrating the architectures. According to the report, for the next increment of the architecture, the department plans to develop compliance criteria and plans for developing an ontology, database, and tool suite. The department did not provide a time frame for completing the next increment. We will continue to monitor the department's efforts to implement the recommendation.

Recommendation: The Secretary of Defense should ensure that the DOD CIO and Chief Management Officer work together to define a specific time frame for when the department plans to integrate its business and IT architectures and ensure that the architectures are integrated. (Recommendation 6)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: In October 2019, the DOD CIO developed a report on the first increment of version 3 of its information enterprise architecture (IEA). The report described planned efforts related to integrating the IEA and the business enterprise architecture. However, the report did not define a specific time frame for when the department plans to integrate the architectures.

Recommendation: The Chief Executive Officer should direct the Chief Information Officer to take steps needed to ensure that system requirements are defined to align with the business needs of CNCS's future risk-based grants monitoring process (Recommendation 1).

Agency: Corporation for National and Community Service
Status: Open

Comments: In November 2018, CNCS officials stated that the agency made the decision to terminate the development of the Grants and Member Management (GMM) system. They subsequently awarded a contract to assess the state of development for the GMM system and to provide recommendations on the actions CNCS needed to take in order to implement a commercial off-the-shelf (COTS) application for core grants management functions. According to CNCS officials, based on the findings from that assessment, further investments in developing customized applications (even an implementation of a COTS application) were not likely to be successful. As of September 2019, CNCS officials stated that they were pursuing the option of a federal shared service as a solution to grants management. As of November 2019, according to CNCS officials, the agency had not yet defined requirements for the grant monitoring system project because the decision to pursue the federal shared services as a solution for grants management is very recent. CNCS officials agreed to provide GAO with an update as further progress is made on this recommendation.

Recommendation: The Chief Executive Officer should direct the Chief Information Officer to ensure that the system development project schedule identifies in the baseline both planned and actual dates for completing all project-level activities, and can be used to monitor and measure progress of the grant monitoring system project (Recommendation 2).

Agency: Corporation for National and Community Service
Status: Open

Comments: In November 2018, CNCS officials stated that the agency made the decision to terminate the development of the GMM system. They subsequently awarded a contract to assess the state of development for the GMM system and to provide recommendations on the actions CNCS needed to take in order to implement a COTS application for core grants management functions. According to CNCS officials, based on the findings from that assessment, further investments in developing customized applications (even an implementation of a COTS application) were not likely to be successful. As of September 2019, CNCS officials stated that they were pursuing the option of a federal shared service as a solution to grants management. As of November 2019, according to CNCS officials, the agency had not yet established a project schedule for completing the grant monitoring system project because the decision to pursue the federal shared services as a solution for grants management is very recent. CNCS officials agreed to provide GAO with an update as further progress is made on this recommendation.

Recommendation: The Chief Executive Officer should direct the Chief Information Officer to ensure that test plans are defined and implemented to include the second version of the grant monitoring system in all stages of testing during development, and results of initial stages are approved before conducting subsequent test stages (Recommendation 3).

Agency: Corporation for National and Community Service
Status: Open

Comments: In November 2018, CNCS officials stated that the agency made the decision to terminate the development of the GMM system. They subsequently awarded a contract to assess the state of development for the GMM system and to provide recommendations on the actions CNCS needed to take in order to implement a COTS application for core grants management functions. According to CNCS officials, based on the findings from that assessment, further investments in developing customized applications (even an implementation of a COTS application) were not likely to be successful. As of September 2019, CNCS officials stated that they were pursuing the option of a federal shared service as a solution to grants management. As of November 2019, according to CNCS officials, the agency had not yet established a timeframe to define test plans for the selected solution for the grant monitoring system project because the decision to pursue the federal shared services as a solution for grants management is very recent. CNCS officials agreed to provide GAO with an update as further progress is made on this recommendation.

Recommendation: To increase the likelihood that its IT investments develop reliable cost estimates, the Secretary of HUD should finalize, and ensure the implementation of, guidance that incorporates the best practices called for in the GAO Cost Estimating and Assessment Guide.

Agency: Department of Housing and Urban Development
Status: Open

Comments: In April 2017, HUD reported that the department concurred with the recommendation and noted that the Office of the Chief Information Officer (OCIO) intended to establish cost estimation guidance for IT projects within its IT Management Framework Guide, incorporating appropriate best practices from the GAO Cost Estimating and Assessment Guide. In March 2019, HUD reported that, with contractor assistance, the department had begun to develop a standard methodology for investment lifecycle cost estimation; however, the methodology had not been fully institutionalized across all investments, and a policy for cost estimation had not been developed. Lacking an updated IT Management Framework and cost estimation policy, OCIO took additional interim action in the most recent budget cycle to reduce cost estimation risk by having the Chief Technology Officer standardize the cost estimates for IT investments. HUD continues to take action intended to address this recommendation; however, OCIO has not yet finalized a cost estimation methodology or the associated policy for IT investments or established a timeframe for implementing cost estimation practices departmentwide.

Recommendation: In order to more fully implement key practices and meet requirements and to better institutionalize standards and practices, the Secretary of HUD should establish a process and schedule for regularly reviewing, revising, and updating HUD's human capital strategic plan, strategic workforce plan, and succession plan.

Agency: Department of Housing and Urban Development
Status: Open

Comments: In response to our recommendation, HUD developed an internal management calendar and associated standard operating procedures. The purpose of the management calendar is to document recurring processes of program offices across the agency, assist in planning and managing the agency's deliverables to ensure that critical deadlines are met, and provide information on ongoing reporting requirements occurring across the agency. We will determine whether HUD has fully implemented our recommendation when the agency provides documentation showing how the management calendar is used for updating human capital, workforce, and succession plans.

Recommendation: In order to more fully implement key practices and meet requirements and to better institutionalize standards and practices, the Secretary of HUD should establish a process and schedule for reviewing and updating policies and procedures to help ensure that policies and procedures for key management functions remain current and complete.

Agency: Department of Housing and Urban Development
Status: Open

Comments: In response to our recommendation, HUD developed an internal management calendar and associated standard operating procedures. The purpose of the management calendar is to document recurring processes of program offices across the agency, assist in planning and managing the agency's deliverables to ensure that critical deadlines are met, and provide information on ongoing reporting requirements occurring across the agency. We will determine whether HUD has fully implemented our recommendation when the agency provides documentation showing how the management calendar is used for updating policies and procedures for key management functions.

Recommendation: The Director of OMB should identify and publish a specific goal associated with its non-provisioned O&M spending measure.

Agency: Executive Office of the President: Office of Management and Budget
Status: Open

Comments: The agency agreed with the recommendation. However, in July 2020, OMB stated that the implementation of this recommendation would be counter to the Administration's focus of prioritizing modernization activities specifically for High Value Assets and, as a result, it does not intend on implementing this recommendation. We disagree and believe that identifying and publishing a specific goal aimed at reducing non-provisioned spending (i.e., spending associated with systems that are not cloud or shared service-based) aligns with the Administration's Cloud Smart strategy to accelerate agency adoption of cloud-based solutions. We will continue to monitor the implementation of this recommendation.

Recommendation: The Director of OMB should commit to a firm date by which its draft guidance on legacy systems will be issued, and subsequently direct agencies to identify legacy systems and/or investments needing to be modernized or replaced.

Agency: Executive Office of the President: Office of Management and Budget
Status: Open

Comments: The agency agreed with the recommendation. In July 2020, OMB stated that agencies were directed to manage the risk to High Value Assets associated with legacy systems in OMB's December 2018 guidance. While OMB's guidance does direct agencies to identify, report, assess, and remediate issues associated with High Value Assets, it does not require agencies to do so for all legacy systems. We will continue to monitor the implementation of this recommendation.

Recommendation: To monitor whether existing investments are meeting the needs of their agencies, the Secretaries of Commerce and the Treasury should direct the respective agency CIO to ensure that required analyses are performed on investments in the operations and maintenance phase.

Agency: Department of the Treasury
Status: Open

Comments: The agency had no comment on the recommendation. In June 2017, Treasury provided an update on the IRS's efforts to ensure that operational analyses are performed on investments in the operations and maintenance phase. However, the recommendation is intended to address issues at the department level and not just at the IRS. In 2017, Treasury declined to provide an update at the department level. As of April 2020, Treasury has not responded to requests for updates. We will continue to monitor the implementation of this recommendation.

Recommendation: To address obsolete IT investments in need of modernization or replacement, the Secretaries of Agriculture, Commerce, Defense, Energy, Health and Human Services, Homeland Security, State, the Treasury, Transportation, and Veterans Affairs; the Attorney General; and the Commissioner of Social Security should direct their respective agency CIOs to identify and plan to modernize or replace legacy systems as needed and consistent with OMB's draft guidance, including time frames, activities to be performed, and functions to be replaced or enhanced.

Agency: Department of Agriculture
Status: Open

Comments: The agency agreed with the recommendation. In May 2019, the agency stated that it had conducted an assessment of its legacy system environment and identified 106 legacy IT assets across 18 components. In a March 2020 update, the agency stated that it is in the process of developing a policy to govern all legacy systems, to include modernization and decommissioning plans. The agency plans to publish this policy by March 2021. We will continue to monitor the implementation of this recommendation.

Recommendation: To address obsolete IT investments in need of modernization or replacement, the Secretaries of Agriculture, Commerce, Defense, Energy, Health and Human Services, Homeland Security, State, the Treasury, Transportation, and Veterans Affairs; the Attorney General; and the Commissioner of Social Security should direct their respective agency CIOs to identify and plan to modernize or replace legacy systems as needed and consistent with OMB's draft guidance, including time frames, activities to be performed, and functions to be replaced or enhanced.

Agency: Department of the Treasury
Status: Open

Comments: The agency had no comment on the recommendation. In June 2017, Treasury provided an update on the IRS's efforts to modernize the IRS's legacy systems. However, the recommendation is intended to address issues at the department level and not just at the IRS. In 2017, Treasury declined to provide an update at the department level. As of April 2020, Treasury has not responded to requests for updates. We will continue to monitor the implementation of this recommendation.

Recommendation: To help ensure that the department can better achieve business process reengineering and enterprise architecture outcomes and benefits, the Secretary of Defense should utilize the results of our portfolio manager survey to determine additional actions that can improve the department's management of its business process reengineering and enterprise architecture activities.

Agency: Department of Defense
Status: Open

Comments: DOD has made progress implementing the recommendation. Specifically, in January 2017, the department issued a business enterprise architecture improvement plan. The plan was intended to address business enterprise architecture usability and deficiencies in information supporting the investment management process. As part of its planning efforts, the department identified opportunities to address the results of our survey. For example, according to the plan, our survey results were used to identify opportunities for improving management and integration of existing enterprise business processes and investments; assessing duplication early in the analysis phase and finding process and capability reuse across the department; and providing a federated business enterprise architecture information environment and capabilities to discover and exchange information from other sources. The plan included delivering three major capabilities. In October 2019, the office of the Chief Management Officer (CMO) demonstrated its new capabilities to GAO. Further, in October 2019, staff within the office of the CMO were working to move the capabilities to a government-approved host environment, although the office had not yet finalized its plan to do so. As of November 2019, the department had not yet deployed the capabilities.

Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to define by when and how the department plans to develop an architecture that would extend to all defense components and include, among other things, (a) information about the specific business systems that support business enterprise architecture (BEA) business activities and related system functions; (b) business capabilities for the Hire-to-Retire and Procure-to-Pay business processes; and (c) sufficient information about business activities to allow for more effective identification of potential overlap and duplication.

Agency: Department of Defense
Status: Open

Comments: DOD has made significant progress addressing the recommendation; however, as of November 2019, more remained to be done. In particular, in 2015, we reported that the department had taken steps to improve the integration of business enterprise architecture information with other existing information, which allows DOD to identify information such as mapping of existing business systems to system functions. More recently, in 2017, the department awarded a contract to improve its business enterprise architecture. According to the department, the objective of the contract was to improve business and system optimization by providing mechanisms to ingest and discover enterprise architecture content from all department components and allow for cross-domain portfolio reviews to include duplication analysis. More specifically, the contract called for developing three major capabilities, including the ability to conduct process and system reviews within and across domains. In October 2019, the Office of the Chief Management Officer (CMO) demonstrated that it had completed development of the three planned capabilities and the office said it was working to host the capabilities in a government-approved cloud environment. With regard to including business capabilities for the Hire-to-Retire and Procure-to-Pay business processes in the business enterprise architecture, the department stated that the new architecture is to identify the business capabilities and processes associated with Lines of Business, which will be defined as a decomposition of the products and services that the business enterprise delivers to the department's components. In September 2019, officials from the Office of the CMO stated that the department plans to review end-to-end processes that comprise the current business enterprise architecture for currency and relevancy. However, the officials did not indicate when they expect to complete this review.

Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to ensure that the functional strategies include all of the critical elements identified in DOD investment management guidance, including performance measures to determine progress toward achieving the goals that incorporate all of the attributes called for in the department's guidance.

Agency: Department of Defense
Status: Open

Comments: As of November 2019, the Department of Defense (DOD) had not addressed the recommendation. In May 2013, we reported that, for the fiscal year 2013 certification of business systems, functional strategies included many, but not all, of the critical elements required by DOD's guidance. Specifically, not all functional strategies demonstrated linkages to business goals in DOD's strategic management plan, and not all included expected outcomes for all functional area goals. In addition, some, but not all, had performance measures in place for assessing progress toward achieving stated goals. However, none of the functional strategies included performance measures that reflected all of the key attributes identified in DOD's guidance. We also reported that for the 2014 certification cycle, the functional strategies had been improved. However, not all of them had performance measures that included all key attributes called for in the guidance. Specifically, all performance measures did not include baseline and target measures, and provide a rationale for the identified targets. In June 2018, DOD revised the required functional strategy elements in its defense business system investment management guidance. However, as of November 2019, the department had not ensured that its functional strategies include all of the elements identified in the guidance. The guidance still requires that functional strategies include business outcomes that link to goals in DOD's strategic management plan. In addition, while the guidance no longer calls for the key performance measure attributes that we assessed in our 2013 report (i.e., baseline and target measures and a rationale for identified targets), the new guidance requires that business outcomes include measurable targets. However, none of the fiscal year 2019 functional strategies fully addressed most of the required elements. For example, none of the functional strategies demonstrated that business outcomes were clearly linked to the department's strategic management plan goals, as required by the 2018 investment management guidance. In addition, none of the strategies included measurable targets, An official from the office of the Chief Management Officer (CMO) demonstrated that the department's Integrated Business Framework-Data Alignment Portal, which is used to record functional strategies, includes business outcomes that are aligned to goals and objectives in the National Defense Business Operations Plan (i.e., the agency strategic plan). The official also demonstrated that most functional strategies link to at least one performance measure from the National Defense Business Operations Plan. However, the official agreed that the published functional strategies did not clearly link outcomes to the department's strategic management plan. Further, officials from the office of the CMO stated in September 2019 that the functional strategies for fiscal year 2019 were not revised for fiscal year 2020.

Recommendation: To effectively implement key components of DOD's business systems modernization program, the Secretary of Defense should direct the Deputy Chief Management Officer to select and control its mix of investments in a manner that best supports mission needs by (a) documenting a process for evaluating portfolio performance that includes the use of actual versus expected performance data and predetermined thresholds; (b) ensuring that portfolio assessments are conducted in key areas identified in our IT investment management framework: benefits attained; current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated, or accepted to date; and (c) ensuring that the documents provided to the Defense Business Council as part of the investment management process include critical information for conducting all assessments.

Agency: Department of Defense
Status: Open

Comments: As of November 2019, DOD had not addressed the recommendation. In 2013, we reported that the department's investment management guidance did not specify a process for conducting an assessment or call for the use of actual versus expected performance data and predetermined thresholds for evaluating portfolio performance. In addition, the department did not call for assessments to be conducted in four key areas-benefits attained, current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated or accepted to date. We also reported in 2013, that the department's investment management guidance identified four criteria and specified the associated assessments that were to be conducted when reviewing and evaluating components' organization execution plans in order to make a portfolio-based investment decision. However, the guidance did not call for the department's organizational execution plans to include critical information for conducting assessments associated with three of the four criteria. Specifically, it did not include information for conducting assessments associated with strategic alignment (i.e., information on alignment with the capital planning and investment control practices and Better Buying Power guidance), utility (i.e., interoperability among systems and system scalability to support additional users) and total cost (i.e., cost in relationship to return on investment). In September 2019, the department stated that the Office of the Chief Management Officer's investment management guidance, investment management training materials, and organizational execution plan, addressed elements of the recommendation. However, the documents did not specify a process for evaluating portfolio performance that includes the use of actual versus expected performance data and predetermined thresholds. Regarding ensuring that portfolio assessments are conducted in key areas identified in our IT investment management framework: benefits attained; current schedule; accuracy of project reporting; and risks that have been mitigated, eliminated, or accepted to date, the June 2018 investment management guidance requires organization execution plans to include risks and risk mitigation strategies. In addition, the investment management guidance requires the plans to include information about benefits attained. Specifically, the plans are to include progress against targets for business goals documented in functional strategies. However, the guidance does not address the remaining key areas identified in our IT investment management framework: current schedule; accuracy of project reporting; and risks that have been eliminated or accepted to date. In addition, the guidance does not call for this information to be used as part of portfolio assessments. Regarding ensuring that the documents provided to the Defense Business Council as part of the investment management process include critical information for conducting assessments, in September 2019, the department stated in a written response that business system certification decisions are made in accordance with criteria established in 10 U.S. Code Section 2222. However, as of November 2109, the department had not demonstrated that it established guidance that calls for documents to include critical information on alignment with the capital planning and investment control practices and Better Buying Power guidance), utility (i.e., interoperability among systems and system scalability to support additional users) and total cost (i.e., cost in relationship to return on investment), which are criteria it established in its investment management guidance for making certification decisions.

Recommendation: To ensure that DOD continues to implement the full range of institutional management controls needed to address its business systems modernization high-risk area, the Secretary of Defense should ensure that the Deputy Secretary of Defense, as the department's Chief Management Officer, establish a policy that clarifies the roles, responsibilities, and relationships among the Chief Management Officer, Deputy Chief Management Officer, DOD and military department Chief Information Officers, Principal Staff Assistants, military department Chief Management Officers, and the heads of the military departments and defense agencies, associated with the development of a federated business enterprise architecture (BEA). Among other things, the policy should address the development and implementation of an overarching taxonomy and associated ontologies to help ensure that each of the respective portions of the architecture will be properly linked and aligned. In addition, the policy should address alignment and coordination of business process areas, military department and defense agency activities associated with developing and implementing each of the various components of the BEA, and relationships among these entities.

Agency: Department of Defense
Status: Open

Comments: While the Department of Defense (DOD) had taken steps to improve its business enterprise architecture, it had not implemented the recommendation as of November 2019. In August 2013, the department established the Business Enterprise Architecture Configuration Control Board, which is chaired by the business enterprise architecture chief architect (Office of the DOD CMO) and includes representatives from the Defense Business Council member organizations. These organizations include, among others, DOD's CIO and the military department CMOs. According to its charter, the Business Enterprise Architecture Configuration Control Board is the principal body for managing the disposition of proposed architecture requirements and change requests. However, the charter does not discuss roles and responsibilities associated with the development of the business enterprise architecture. Specifically, it does not address alignment and coordination of business process areas or military department and defense agency activities associated with developing and implementing each of the various components of the business enterprise architecture, and the relationships among these entities. In addition, in September 2018, the department stated that it was drafting a business enterprise architecture concept of operations that was to outline roles and responsibilities associated with the development of the architecture. However, as of November 2019, the department had not completed the concept of operations or otherwise demonstrated that it had established roles and responsibilities for the development of the architecture. In October 2018, an official from the Office of the CMO described the department's new approach to developing its business enterprise architecture. In addition, the department demonstrated that it had developed a taxonomy for the architecture and was in the process of developing an ontology to help ensure that each of the respective portions of the architecture would be appropriately linked and aligned. In November 2019, the official stated that the ontology had been implemented in the department's new business enterprise architecture tool; however, the department did not demonstrate that it had finished developing the ontology. Specifically, the department's October 2019 ontology document identifies basic concepts, such as "Goal", "Objective", and "LOB" (i.e., line of business) as classes, and the properties and attributes of, and relationships among, classes. However, the document does not include annotations such as for the "description" attribute for an LOB, which would provide information needed to create a specific instance of a class applicability; and had not demonstrated that it had developed ontologies for its business domains, such as acquisition, human resource management, and financial management. Also, the document does not demonstrate if allowed values have been defined for some attributes, such as the options allowed in an "option list" for "status" attributes. Further, the department had not documented general information about the ontology, such as its scope and intended applicability; and had not demonstrated that it had developed ontologies for its business domains, such as acquisition, human resource management, and financial management.