Information Technology:

DHS Needs to Fully Define and Implement Policies and Procedures for Effectively Managing Investments

GAO-07-424: Published: Apr 27, 2007. Publicly Released: Apr 27, 2007.

Additional Materials:

Contact:

David A. Powner
(202) 512-6256
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Homeland Security (DHS) relies extensively on information technology (IT) to carry out its mission. For fiscal year 2008, DHS requested about $4 billion--the third largest planned IT expenditure among federal departments. Given the size and significance of DHS's IT investments, GAO's objectives were to determine whether DHS (1) has established the management structure and associated policies and procedures needed to effectively manage these investments and (2) is implementing key practices needed to effectively control them. GAO used its IT Investment Management (ITIM) framework and associated methodology to address these objectives, focusing on the framework's stages related to the investment management provisions of the Clinger-Cohen Act.

DHS has established the management structure to effectively manage its investments. However, the department has yet to fully define 8 of the 11 related policies and procedures that GAO's ITIM framework defines. Specifically, while DHS has documented the policies and related procedures for project-level management, some of these procedures do not include key elements. For example, procedures for selecting investments do not cite either the specific criteria or steps for prioritizing and selecting new IT proposals. In addition, the department has yet to define most of the policies associated with managing its IT projects as investment portfolios. Officials attributed the absence of policies and procedures at the portfolio level to other investment management priorities. Until DHS fully defines and documents policies and procedures for investment management, it risks selecting investments that will not meet mission needs in the most cost-effective manner. DHS has also not fully implemented the key practices needed to actually control investments--either at the project level or at the portfolio level. For example, according to DHS officials and the department's control review schedule, DHS investment boards have not conducted regular investment reviews. Further, while GAO found that control activities are sometimes performed, they are not performed consistently across projects. In addition, because the policies and procedures for portfolio management have yet to be defined, control of the department's investment portfolios is ad hoc, according to DHS officials. Officials told GAO that they have recently hired a portfolio manager and are recruiting another one to strengthen IT investment management. Until DHS fully implements processes to control its investments, both at the project and portfolio levels, it increases the risk of not meeting cost, schedule, benefit, and risk expectations.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. It should also include portfolio-level practices such as executing adjustments to the IT investment portfolios in response to actual portfolio performance.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: DHS has begun performing portfolio reviews in which the department makes recommendations to adjust the portfolios based on the results of its performance. We obtained two examples of cases in which adjustments had been made as a result of portfolio reviews.

    Recommendation: To strengthen DHS's investment management capability and address the weaknesses discussed in this report, the Secretary of Homeland Security should direct the Undersecretary for Management, in collaboration with the CFO and CIO, to devote the appropriate attention to development and implementation of effective investment management processes. At a minimum, this should include fully defining and documenting project-and portfolio-level policies and procedures that address conducting post-implementation reviews of IT investments, including defining roles and responsibilities for doing so, and specifying how conclusions, lesson learned, and recommended management actions are to be shared with executives and others.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: The August 2010 update of DHS's Capital Planning and Investment Control Guide includes procedures that address conducting post-implementation reviews of IT investments, including defining roles and responsibilities for doing so, and specifying how conclusions, lesson learned, and recommended management actions are to be shared with executives and others.

    Recommendation: To strengthen DHS's investment management capability and address the weaknesses discussed in this report, the Secretary of Homeland Security should direct the Undersecretary for Management, in collaboration with the CFO and CIO, to devote the appropriate attention to development and implementation of effective investment management processes. At a minimum, this should include fully defining and documenting project-and portfolio-level policies and procedures that address assessing portfolio performance at regular intervals to reflect current performance expectations.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: DHS has developed high-level procedures, including a template, for the Office of the Chief Information Officer's (OCIO) annual reviews of the department's portfolios. According to the OCIO's Executive Director for the Enterprise Business Management Office, more specific procedures are to be documented in a concept of operations for portfolio management which is expected to be finalized by the end of 2011.

    Recommendation: To strengthen DHS's investment management capability and address the weaknesses discussed in this report, the Secretary of Homeland Security should direct the Undersecretary for Management, in collaboration with the CFO and CIO, to devote the appropriate attention to development and implementation of effective investment management processes. At a minimum, this should include fully defining and documenting project-and portfolio-level policies and procedures that address analyzing, selecting, and maintaining the investment portfolios.

    Agency Affected: Department of Homeland Security

    Status: Closed - Not Implemented

    Comments: In September 2011, DHS officials stated that analyzing, selecting, maintaining portfolios is primarily a function of the enterprise architecture group. We followed up with this group to obtain the related policies and procedures, but as of September 30, 2011, had not received the requested documentation. Given the amount of time elapsed since we made this recommendation, we are closing it as not implemented.

    Recommendation: To strengthen DHS's investment management capability and address the weaknesses discussed in this report, the Secretary of Homeland Security should direct the Undersecretary for Management, in collaboration with the CFO and CIO, to devote the appropriate attention to development and implementation of effective investment management processes. At a minimum, this should include fully defining and documenting project-and portfolio-level policies and procedures that address creating and modifying IT portfolio selection criteria.

    Agency Affected: Department of Homeland Security

    Status: Closed - Not Implemented

    Comments: In September 2011, DHS officials stated that the creation of portfolio selection criteria is primarily a function of the enterprise architecture group. We followed up with this group to obtain the related policies and procedures, but as of September 30, 2011, had not received the requested documentation. Given the amount of time elapsed since we made this recommendation, we are closing it as not implemented.

    Recommendation: To strengthen DHS's investment management capability and address the weaknesses discussed in this report, the Secretary of Homeland Security should direct the Undersecretary for Management, in collaboration with the CFO and CIO, to devote the appropriate attention to development and implementation of effective investment management processes. At a minimum, this should include fully defining and documenting project-and portfolio-level policies and procedures that address identifying and collecting information about investments, including assigning responsibility for the process and ownership of the information and defining the locations for information storage.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: In October 2008 DHS updated the manual for its Next Generation Periodic Reporting System (nPRS) to, among other things, provide guidance for collecting information on investments, including assigning responsibility for the process and ownership of the information, and defining the location for storing the information. The manual also specifies the type of information that should be collected and the frequency of data collection. In September 2011, DHS officials also noted that the department has several efforts underway to improve the quality of the data needed to inform investment management decisions, including the creation of a decision support tool.

    Recommendation: To strengthen DHS's investment management capability and address the weaknesses discussed in this report, the Secretary of Homeland Security should direct the Undersecretary for Management, in collaboration with the CFO and CIO, to devote the appropriate attention to development and implementation of effective investment management processes. At a minimum, this should include fully defining and documenting project-and portfolio-level policies and procedures that address overseeing (i.e., controlling) IT projects and systems, including specifying the procedural rules for the investment boards' operations and decision making during project oversight.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: The January 2010 revision of DHS's Acquisition Management Directive (MD 102-01) and the August 2010 update of the Capital Planning and Investment Control Guide include policies and procedures for the Acquisition Review Board's operations and decision-making during project oversight.

    Recommendation: To strengthen DHS's investment management capability and address the weaknesses discussed in this report, the Secretary of Homeland Security should direct the Undersecretary for Management, in collaboration with the CFO and CIO, to devote the appropriate attention to development and implementation of effective investment management processes. At a minimum, this should include fully defining and documenting project-and portfolio-level policies and procedures that address reselecting ongoing IT investments, including specifying the criteria and steps for prioritizing and reselecting these investments.

    Agency Affected: Department of Homeland Security

    Status: Closed - Not Implemented

    Comments: In September 2011, DHS officials stated that the reselection of ongoing projects is a function of the Office of the Chief Financial Officer's Program Analysis and Evaluation group and directed us to the department's Financial Management Policy Manual for related procedures, including the criteria and steps for prioritizing and reselecting proposals. We reviewed the manual; however it did not include these procedures. Given the amount of time elapsed since we made this recommendation, we are closing it as not implemented.

    Recommendation: To strengthen DHS's investment management capability and address the weaknesses discussed in this report, the Secretary of Homeland Security should direct the Undersecretary for Management, in collaboration with the Chief Financial Officer (CFO) and Chief Information Officer (CIO), to devote the appropriate attention to development and implementation of effective investment management processes. At a minimum, this should include fully defining and documenting project-and portfolio-level policies and procedures that address selecting new investments, including specifying the criteria and steps for prioritizing and selecting these proposals.

    Agency Affected: Department of Homeland Security

    Status: Closed - Not Implemented

    Comments: In September 2011, DHS officials stated that the selection of new project proposals is a function of the Office of the Chief Financial Officer's Program Analysis and Evaluation group and directed us to the department's Financial Management Policy Manual for related procedures, including the criteria and steps for prioritizing and selecting proposals. We reviewed the manual; however it did not include these procedures. Given the amount of time elapsed since we made this recommendation, we are closing it as not implemented.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. At a minimum, this should include providing adequate resources, including people, funding, and tools, for IT project oversight.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: According to DHS, the Acquisition Program Management Division and the Enterprise Business Management Office, who have primarily responsibility for supporting the review of DHS's projects, the department has adequate resources (both in numbers and skills) to perform their oversight activities. For example, within the Enterprise Business Management Office there are currently 31 federal employees and approximately 20 contractors. This contrasts with the six staff (with only one of them being full-time) that we reported the department had to support departmentwide investment management activities in 2007. According to the Chief Information Officer, the department expects to continue to augment its information technology staff, including those for investment management functions, through the end of fiscal year 2012.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. At a minimum, this should include having IT projects and systems, including those in steady state (operations and maintenance), maintain approved project management plans that include expected cost and schedule milestones and measurable benefit and risk expectations.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: All major programs are required to develop and maintain an acquisition program baseline which, among other things, include expected cost and schedule milestones and measurable benefit and risk expectations. The Acquisition Review Board has been reviewing and approving these program baselines on a more consistent basis than in the past. DHS officials also stated that the development of a new decision support tool and other tools is expected improve the department's ability to monitor the status and review of acquisition program baselines.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. At a minimum, this should include providing data on actual performance (including cost, schedule, benefit, and risk performance) to the appropriate IT investment board.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: Data on actual project performance (including cost, schedule, benefit, and risk performance) are provided to the Acquisition Review Board for its review. According to officials, the department also has efforts underway, including the development of a new decision support tool, to improve the quality of the data.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. It should also include portfolio-level practices such as defining and collecting IT portfolio performance measurement data that are consistent with portfolio performance criteria.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: DHS collects data on the performance of its portfolios that are consistent with the assessment criteria that have been defined.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. It should also include portfolio-level practices such as developing, reviewing, and modifying criteria for assessing portfolio performance at regular intervals to reflect current performance expectations.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: DHS has developed criteria to assess the performance of its portfolios to reflect current performance expectations. They include a combination of mission-specific, acquisition health, and cross-cutting criteria which are examined to determine recommendations for next steps.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. It should also include portfolio-level practices such as providing results of relevant Providing Investment Oversight reviews from Stage 2 to the investment boards.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: The Acquisition Review Board reviews investments at key milestones, which is the focus of the stage 2 providing investment oversight critical process. The Acquisition Review Board is also informed of, and in some cases, participates in--other relevant oversight reviews, including portfolio reviews and Techstat reviews.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. It should also include portfolio-level practices such as making board members familiar with the process for evaluating and improving the portfolio's performance.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: While, according to DHS officials, including the Executive Director for the Enterprise Business Management Office, there is no formal mechanism for making the Acquisition Review Board members familiar with the process for evaluating and improving the portfolio's performance, these members recently received training in the use the decision support tool that will soon be implemented to support investment management decisions.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. It should also include portfolio-level practices such as providing adequate resources, including people, funding, and tools, for reviewing the investment portfolios and their projects.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: According to DHS, the Acquisition Program Management Division and the Enterprise Business Management Office, who have primarily responsibility for supporting the review of DHS's investment portfolios, the department has adequate resources (both in numbers and skills) to perform their oversight activities. For example, within the Enterprise Business Management Office there are currently 31 federal employees and approximately 20 contractors. This contrasts with the six staff (with only one of them being full-time) that we reported the department had to support departmentwide investment management activities in 2007. According to the Chief Information Officer, the department expects to continue to augment its information technology staff, including those for investment management functions, through the end of fiscal year 2012.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. At a minimum, this should include having the investment board regularly track the implementation of corrective actions for each underperforming project until the actions are completed.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: The Acquisition Program Management Division is responsible for tracking the status of corrective actions for underperforming projects until completion.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. At a minimum, this should include taking appropriate actions to correct or terminate each underperforming IT project or system in accordance with defined criteria and the documented policies and procedures for management oversight.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: The results of Acquisition Review Board project review are summarized in Acquisition Decision Memos which capture all action items. The Acquisition Program Management Division tracks the status of these action items until completion in accordance with the Acquisition Review Board's documented procedures.

    Recommendation: In addition, the Department of Homeland Security should implement key investment control processes. At a minimum, this should include having each investment board use verified data to regularly review the performance of IT projects and systems against stated expectations.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: The Acquisition Review Board reviews the performance of IT projects and systems against expectations at key milestone decision points and is also informed of the results of portfolio reviews and the Techstat reviews of projects needing special attention. While, according to officials, the cost and schedule data are verified by the Program Analysis and Evaluation group, the quality of the data used for the board reviews is expected to improve as a result of efforts to create a new decision support tool and other tools.

    Apr 2, 2014

    Feb 26, 2014

    Feb 12, 2014

    Jan 13, 2014

    Nov 13, 2013

    Nov 6, 2013

    Sep 12, 2013

    Sep 11, 2013

    Looking for more? Browse all our products here