Business Systems Modernization:

Air Force Needs to Fully Define Policies and Procedures for Institutionally Managing Investments

GAO-08-52: Published: Oct 31, 2007. Publicly Released: Oct 31, 2007.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6304
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

In 1995, GAO first designated the Department of Defense's (DOD) business systems modernization program as "high-risk" and continues to do so today. In 2004, Congress passed legislation reflecting prior GAO recommendations that DOD adopt a corporate approach to information technology (IT) business systems investment management including tiered accountability for business systems at the department and component levels. To support GAO's legislative mandate to review DOD's efforts, GAO assessed whether the investment management approach of one of DOD's components--the Department of the Air Force (Air Force)--is consistent with leading investment management best practices. In doing so, GAO applied its IT Investment Management (ITIM) framework and associated methodology, focusing on the stages related to the investment management provisions of the Clinger-Cohen Act of 1996.

The Air Force has established the basic management structures needed to effectively manage its IT projects as investments, but has not fully implemented many of the related policies and procedures outlined in GAO's ITIM framework. Air Force has fully implemented three of the nine key practices that call for project-level management structures, policies, and procedures, and has not implemented any of the five practices that call for portfolio-level policies and procedures. Regarding project-level practices, it has established an IT investment board that is responsible for defining and implementing the department's business systems investment governance process, has developed procedures for identifying and collecting information about its business systems to support investment selection and control, and has assigned responsibility for ensuring that the information collected during project identification meets the needs of the investment management process. However, Air Force has not fully documented business systems investment policies and procedures for directing investment board operations, selecting new investments, reselecting ongoing investments, or integrating the investment funding and investment selection processes. In addition, it has not implemented any of the policies and procedures for developing and maintaining a complete business system investment portfolio. Air Force officials stated that they are aware of the absence of documented policies and procedures in certain areas of project-level and portfolio-level management and that they are currently working on guidance to address these areas. For example, officials stated that they had begun drafting portfolio-level policies and procedures. According to Air Force officials, the policies and procedures are expected to be completed and approved by December 2007. Until Air Force fully defines policies and procedures for both individual projects and portfolios of projects, it risks not being able to select and control these business system investments in a way that is consistent and complete, which in turn increases the chances that these investments will not meet mission needs in the most effective manner.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address specifying how systems that are in operations and maintenance will be reviewed and specifying how Air Force's business investment management system is coordinated with Joint Capabilities Integration and Development Syste, planning, programming, budgeting, and execution, and Defense Acquisition System.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: Although Air Force still needs to specify how its business investment management system is to be coordinated with the requirements, budgeting, and acquisition systems, the department has taken actions to develop and issue the policies and procedures that address how these business systems will be reviewed by its investment review board. For example, in November 2008, Air Force updated its IT investment review guidance to require an annual review of all business systems, including those in operations and maintenance, and it defined the process by which its investment review board would review these systems. Further, the guidance specified how business investments, including those in operations and maintenance, were to be prioritized, using factors such as mission, strategic value, and risk. Collectively, these actions should help strengthen Air Force's investment management process by assuring that systems throughout the investment life cycle, including those in operations and maintenance, have adequate oversight and align to business needs.

    Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address ensuring that systems in operations and maintenance are aligned with ongoing and future business needs.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department has taken steps to address this recommendation. For example, Air Force's 2008 IT investment review guidance calls for the review of all business systems, to include those in operations and maintenance, and specifies how these business investments are to be prioritized using factors such as mission and strategic value and risk.

    Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address selecting investments, including specifying how factors, such as cost, schedule, and performance data are to be used in making certification decisions.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department has taken sufficient steps to meet the intent of the recommendation. For example, Air Force has developed IT investment management guidance that includes predefined criteria for adherence to cost, schedule, and performance milestones, and requires that all investments be scored against this criteria as part of the selection process. Air Force officials stated that the department plans update its IT investment guidance in November 2012.

    Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address reselecting ongoing investments, including specifying how factors, such as cost, schedule, and performance data are to be used in making reselection decisions during the annual review process and providing for the reselection of investments that are in operations and maintenance.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department has taken steps to meet the intent of the recommendation. For example, Air Force has developed IT investment management guidance that includes predefined criteria for adherence to cost, schedule, and performance milestones, and requires that all investments be scored against this criteria as part of the reselection process. In addition, the Air Force IT investment review guidance provides for the annual review of all business systems investments, to include those in operations and maintenance. Further, Air Force has stated its intent to update its IT investment guidance in November 2012.

    Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address integrating funding with the process of selecting an investment, including specifying how the precertification authority is using funding information in carrying out decisions on system certification and approvals.

    Agency Affected: Department of Defense

    Status: Closed - Not Implemented

    Comments: In 2011, Air Force officials identified activities currently underway to inform the Investment Review Boards and precertification authorities about funding during certification reviews. However, the department has yet to update its policies and procedures to specify how investment funding and selection are to be integrated and how the precertification authority is to use funding information to make certification or approval decisions. Air Force had stated its intent to replace its IT portfolio management and IT investment guidance in November 2012.

    Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address overseeing IT projects and systems, including specifying policies and procedures that guide the implementation of corrective actions when program expectations are not met.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department has taken steps to meet the intent of the recommendation. For example, Air Force has developed IT investment management guidance that includes predefined criteria and checkpoints for meeting cost, schedule, and performance milestones, and requires the development of corrective actions when a project deviates from milestones. In addition, while the department's IT investment management policies and procedures do not address the implementation of corrective actions for all systems, they do so for the larger business systems. Collectively, these actions should help strengthen Air Force's investment management process by assuring that systems have adequate oversight.

    Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address creating and modifying IT portfolio selection criteria for business system investments.

    Agency Affected: Department of Defense

    Status: Closed - Not Implemented

    Comments: While Air Force has assigned the Major Command and Functional Investment Review Boards responsibility for creating and modifying portfolio criteria (e.g., prioritization and investment trade-offs) for business system investments, it has not yet finalized draft policies, procedures, or criteria for conducting IT portfolio selection. The Air Force has described its current efforts to change its IT investment management organization and form a new branch to lay the foundation for integrated, efficient IT investment management processes, but this office is not yet established. Further, Air Force had stated its intent to replace its IT portfolio management and IT investment guidance, but this update is not expected until November 2012.

    Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address defining the roles and responsibilities for the development and modification of the IT portfolio selection criteria.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department has taken steps to address this recommendation. For example, Air Force's investment management guidance assigns responsibility for the development and modification of the IT portfolio selection criteria to an integrated working team consisting of Major Command representatives, Functional Portfolio Managers, and the Certification Process Manager; and assigns approval of the criteria to a Senior Working Group.

    Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address analyzing, selecting, and maintaining business system investment portfolios.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department has taken steps to meet the intent of the recommendation. For example, Air Force's IT Investment Review Guide describes an overview of four continuous integrated investment activities, including analysis, selection, control, and evaluation, to manage its investment portfolios. Further, Air Force has stated its intent to replace its IT portfolio management and IT investment guidance in November 2012.

    Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address reviewing, evaluating, and improving the performance of the portfolio(s) by using project indicators, such as cost, schedule, and risk.

    Agency Affected: Department of Defense

    Status: Closed - Not Implemented

    Comments: While the Major Command and Functional Investment Review Board draft charters state that the boards are responsible for reviewing indicators associated with portfolio management, such as architecture alignment, capability delivery, and risk, there are no policies and procedures indicating how they are to use these and other project indicators--such as cost, schedule, and risk--to review, evaluate, and improve their portfolios. The Air Force has described its current efforts to change its IT investment management organization and form a new branch to lay the foundation for integrated, efficient IT investment management processes, but this office is not yet established. Further, Air Force had stated its intent to replace its IT portfolio management and IT investment guidance, but this update is not expected until November 2012.

    Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address conducting post-implementation reviews for all investment tiers and directing the investment boards to consider the information gathered to develop lessons learned from these reviews.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department has taken steps to address the recommendation. For example, Air Force has developed policies and procedures for conducting post-implementation reviews for all defense business system investments that cost more than $1 million. Further, Air Force's acquisition guidance directs program managers to document in post-implementation reviews any identified shortfalls between actual program results and established performance goals and incorporate the results in lifecycle management planning documentation.

    Apr 2, 2014

    Feb 26, 2014

    Feb 12, 2014

    Jan 13, 2014

    Nov 13, 2013

    Nov 6, 2013

    Sep 12, 2013

    Sep 11, 2013

    Looking for more? Browse all our products here