Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address reviewing, evaluating, and improving the performance of the portfolio(s) by using project indicators, such as cost, schedule, and risk.

Agency Affected: Department of Defense

Status: Open

Comments: The department has not yet addressed this recommendation. Specifically, Air Force has not yet developed and implemented policies and procedures that address reviewing, evaluating, and improving the performance of the business system investment portfolios, using project indicators such as cost, schedule, and risk. According to the Air Force, such key practices will be addressed in future revisions to its IT investment review guidance.

Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address analyzing, selecting, and maintaining business system investment portfolios.

Agency Affected: Department of Defense

Status: Open

Comments: The department has not yet addressed this recommendation. Specifically, Air Force has not yet developed policies and procedures that address analyzing, selecting, and maintaining its business system investment portfolios. According to the Air Force, such key practices will be addressed in future revisions to its IT investment review guidance.

Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address defining the roles and responsibilities for the development and modification of the IT portfolio selection criteria.

Agency Affected: Department of Defense

Status: Closed - Implemented

Comments: The department has taken steps to address this recommendation. For example, Air Force's investment management guidance assigns responsibility for the development and modification of the IT portfolio selection criteria to an integrated working team consisting of Major Command representatives, Functional Portfolio Managers and the Certification Process Manager, and approval of the criteria to a Senior Working Group.

Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address creating and modifying IT portfolio selection criteria for business system investments.

Agency Affected: Department of Defense

Status: Open

Comments: The department has not yet addressed this recommendation. Specifically, while Air Force has assigned the Major Command and Functional Investment Review Boards responsibility for creating and modifying portfolio criteria (e.g., prioritization and investment trade-offs) for business system investments, it has not yet finalized draft policies, procedures, or criteria for conducting IT portfolio selection. According to the Air Force, such key practices will be addressed in future revisions to its IT investment review guidance.

Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address overseeing IT projects and systems, including specifying policies and procedures that guide the implementation of corrective actions when program expectations are not met.

Agency Affected: Department of Defense

Status: Open

Comments: Although the department has recently taken steps to update its policies and procedures to address key aspects of this recommendation, it has not been fully implemented. For example, Air Force has predefined criteria and checkpoints for meeting cost, schedule, and performance milestones, and requires the development of corrective actions when a project deviates from milestones. Although it has policies and procedures that guide the implementation of corrective actions for Tier 1 systems, it does not have such policies and procedures for the remaining tiers. Further, Air Force may certify a system and impose conditions that must be met in order to obligate system funding. Although the responsibility for tracking conditions has been assigned to the Certification Process Manager, Air Force does not have policies and procedures that define the process for tracking conditions until they are resolved. Department officials stated that these actions will be addressed in future iterations of its IT investment review guidance.

Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address integrating funding with the process of selecting an investment, including specifying how the precertification authority is using funding information in carrying out decisions on system certification and approvals.

Agency Affected: Department of Defense

Status: Open

Comments: Although the department has recently taken steps to update its policies and procedures to address key aspects of this recommendation, it has not been fully implemented. For example, department officials identified activities currently underway including funding information, such as program cost savings, net present value, and a benefits-to-cost ratio, that is used by the Investment Review Board and precertification authority to make certification and approval decisions. However, Air Force policies and procedures do not specify how investment funding and selection are to be integrated nor how the precertification authority is to use funding information to make certification and approval decisions. Department officials stated that these actions will be addressed in future iterations of its IT investment review guidance.

Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address reselecting ongoing investments, including specifying how factors, such as cost, schedule, and performance data are to be used in making reselection decisions during the annual review process and providing for the reselection of investments that are in operations and maintenance.

Agency Affected: Department of Defense

Status: Open

Comments: Although the department has recently taken steps to update its policies and procedures to address key aspects of this recommendation, it has not been fully implemented. For example, Air Force's IT investment management guidance includes predefined criteria for adherence to cost, schedule, and performance milestones, and requires that all investments be scored against this criteria as part of the reselection process. In addition, the Air Force IT investment review guidance provides for the annual review of all business systems investments, to include those in operations and maintenance. However, the guidance does specify how criteria, such as cost, schedule, and performance data, are to be used in making reselection decisions decisions. Air Force plans to address these actions in guidance that will be finalized and approved by October 2011.

Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address selecting investments, including specifying how factors, such as cost, schedule, and performance data are to be used in making certification decisions.

Agency Affected: Department of Defense

Status: Open

Comments: Although the department has recently taken steps to update its policies and procedures to address key aspects of this recommendation, it has not been fully implemented. For example, Air Force's IT investment management guidance includes predefined criteria for adherence to cost, schedule, and performance milestones, and requires that all investments be scored against this criteria as part of the selection process. However, the guidance does not specify how criteria, such as cost, schedule, and performance data, are to be used in making precertification decisions. Air Force plans to address these actions in guidance that will be finalized and approved by October 2011.

Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address ensuring that systems in operations and maintenance are aligned with ongoing and future business needs.

Agency Affected: Department of Defense

Status: Closed - Implemented

Comments: The department has taken steps to address this recommendation. For example, Air Force's 2008 IT investment review guidance calls for the review of all business systems, to include those in operations and maintenance, and specifies how these business investments are to be prioritized using factors such as mission and strategic value and risk.

Recommendation: To strengthen Air Force's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Secretary of the Air Force to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address specifying how systems that are in operations and maintenance will be reviewed and specifying how Air Force's business investment management system is coordinated with Joint Capabilities Integration and Development Syste, planning, programming, budgeting, and execution, and Defense Acquisition System.

Agency Affected: Department of Defense

Status: Open

Comments: Although the department has recently taken steps to update its policies and procedures to address key aspects of this recommendation, it has not been fully implemented. For example, Air Force recently developed IT investment review guidance that calls for the review of all business systems, to include those in operations and maintenance, and it defines the process by which its Investment Review Board will review those systems. However, the guidance does not yet specify how business investment management activities are coordinated with other DOD management systems, such as the Joint Capabilities Integration and Development System (JCIDS), the Defense Acquisition System (DAS), and the Planning, Programming, Budgeting, and Execution (PPBE) process. Air Force plans to revise its investment process to be consistent with DOD's business capability lifecycle policy and guidance, which outlines aspects of how the business system review processes are to be coordinated with JCIDS and DAS, by October 2011.

Recommendation: The Secretary of Defense should ensure these well-defined and disciplined business system investment management policies and procedures also include portfolio-level management policies and procedures that address conducting post-implementation reviews for all investment tiers and directing the investment boards to consider the information gathered to develop lessons learned from these reviews.

Agency Affected: Department of Defense

Status: Open

Comments: Although the department has recently taken steps to update its policies and procedures to address key aspects of this recommendation, it has not been fully implemented. For example, Air Force has developed policies and procedures for conducting post-implementation reviews for Tier 1 systems and draft policies and procedures for Tier 2 and Tier 3 systems. However, it has not developed such policies or procedures for Tier 4 systems (i.e., defense business system investments that cost less than or equal to $1 million). Air Force's acquisition guidance directs program managers to document in post-implementation reviews any identified shortfalls between actual program results and established performance goals and incorporate the results in lifecycle management planning documentation. However, there are no policies and procedures directing the Air Force Senior Working Group, which is accountable for Air Force business system investments, to consider information gathered and develop lessons learned from the post-implementation reviews. According to the Air Force, such key practices will be addressed in future revisions to its IT investment review guidance.