Information Technology Management: Governmentwide Strategic Planning, Performance Measurement, and Investment Management Can Be Further Improved

The Department of Agriculture (USDA) generally agreed with our recommendation. In USDA's Information Technology Strategic Plan, dated September 2006, the department documented its IT strategic planning process in Appendix A, defining the inputs, strategies, analysis, and outputs of the IT planning approach it follows. In addition, key inputs, such as funding and human capital resources, are identified as well as time lines for preparing the IT strategic plan, its scope, and how it links to other departmental strategic and business plans.

As part of its FY 2007 Summary of Budget and Performance, the Department of Agriculture (USDA) reported on the resources and time periods required to ensure the protection and safety of USDA information resources. Additionally, in its FY 2006 Performance and Accountability Report, USDA also reported on actions being taken to address information technology internal control weakness areas identified as part of the department's efforts to comply with FISMA and FMFIA.

In its FY 2007-2011 IT Strategic Plan, completed in September 2006, the Department of Agriculture (USDA) describes IT goals and objectives in four categories that includes (1) IT Strategy and Business Alignment; (2) IT Organization and Skill; (3) IT Management and Governance; and (4) Technology and Architecture. In the Office of Chief Information Officer's (OCIO) IT Strategic Plan, last updated July 2007, responsibilities associated with each goal area are described along with specific initiatives intended to help OCIO achieve desired results. For example, under Goal 2: IT Organization and Skill, OCIO is charged with working with the Office of Human Capital Management and the agencies and staff officers to develop and manage an IT Workforce Plan that anticipates the future workforce needs for the department. As the lead for this critical function, OCIO participates in many Federal activities to ensure that USDA's IT workforce is well planned and well trained.

The Department of Agriculture (USDA) agreed with our recommendation and has taken action to address it. Specifically, in August 2007, the Acting Associate Chief Information Officer (CIO) for Cyber Security reported on a number of steps taken to address software piracy across the department, such as establishing Departmental guidance, which includes directives (DM3140-001); emphasizing copyright violations in the annually required security awareness training; and increasing follow-up activities to address Peer-to-Peer software in the Department--one common way people download copyrighted materials. Moreover, since 2006, the department's computer security reviews have also included reviews of software protection at the policy, technical, and operational levels. For the reviews in fiscal year 2008, a baseline has been established for testing compliance. The metrics are 1) 100 percent of USDA's agencies have software compliance policies in place; 2) piracy incident reporting is reduced by at least 10 percent over the next three years; and 3) security reviews identify no software piracy issues.

In the Department of Agriculture's (USDA) IT Strategic Plan, completed September 2006, USDA maps its IT strategic goals to the strategic goals identified in USDA's Strategic Plan. In its 2007 update of the IT strategic plan, the Office of the Chief Information Officer (OCIO) maps ongoing work on initiatives to the IT strategic goals and identifies performance metrics associated with each of these initiatives, which are tracked by OCIO reports and by assigned teams working on the initiatives. For example, under OCIO's Initiative 3--Increase the efficiency and performance of USDA IT investments through implementation of department-wide and governmentwide approaches--OCIO is tracking progress on actual versus expected metrics for USDA's ongoing work to implement AgLearn, a new USDA learning management system that promotes department-wide e-learning activities.

USDA generally agreed with our recommendation and established an ongoing process for conducting benchmarking studies as stated in its Capital Planning and Investment Control (CPIC) guidance. According to the department's 2006 CPIC guide, USDA compares its CPIC process to the IT capital planning processes in other federal organizations in an ongoing attempt to improve its IT capital planning process. Formally known as benchmarking, the purpose of this comparison is to learn from others so as to improve the USDA capital planning process. An opportunity arose in 2005 to conduct a benchmarking assessment relative to another highly-regarded federal department as part of USDA's post-pilot review of WorkLenz. In the future, USDA will benchmark its IT capital planning process relative to other federal organizations at least once every three years.

The Department of Agriculture (USDA) generally agreed with GAO's recommendation. In USDA's revised Capital Planning and Investment Control (CPIC), dated April 2005, USDA described the relationship between the IT investment management process and the department's enterprise architecture as GAO recommended. In its Information and Technology Transformation regulation, DR 3600-00, issued November 2, 2004, USDA established a policy requiring that IT investments be in compliance with the agency's enterprise architecture.

USDA generally agreed with GAO's recommendation. In April 2004, USDA revised its Information Technology Capital Planning and Investment Control Guide to describe the alignment and coordination of responsibilities of the department's various IT investment management boards for decision making related to IT investments, including cross-cutting investments.

USDA generally agreed with GAO's recommendation. In April 2004, USDA revised the Pre-select Phase of its Information Technology Capital Planning and Investment Control Guide to require that proposed IT investments support work processes that have been simplified or redesigned to reduce costs and improve effectiveness, and that such investments make maximum use of commercial-off-the-shelf software.

USDA generally agreed with GAO's recommendation. In April 2004, USDA revised the Select Phase of its Information Technology Capital Planning and Investment Control Guide to require modularized IT investments.

In the Department of Commerce's IT Investment Performance Measurement and Performance Reporting Policy (May 2006), major IT investments must use Earned Value Management (EVM) to measure and report the developmental, modernization, and enhancement (DME) portion of the investment and must conduct Operational Analysis on the operational portion of the investment. EVM relates project and resource planning to actual cost and schedule achievements and is required to be assessed monthly and reported to the Office of the CIO quarterly. Operational Analysis measures the current performance of the operational portion of an investment against established performance parameters and is required to be reported to the Office of the CIO annually. Both of these reports are then used in the control and evaluation phases of Commerce's Capital Planning and Investment Control Process, specifically to identify an investment that deviates from cost, schedule, or performance goals by more than 10%.

Commerce has implemented a policy requiring post-implementation reviews. The Commerce investment management board's expectations document requires implemented investments to assess lessons learned, including how and why implementation resources and activities differed from planned. Implemented investments are also required to address full life cycle operations and maintenance. The investment management board's review schedule for 2004 also specifies when projects are required to present their post-implementation reviews. A post-implementation review was conducted on a completed Commerce investment, in accordance with this schedule.

In Commerce's fiscal year 2007 Annual Performance Plan, the department identifies the resources supporting implementation of the department's IT Security Program. Specifically, under the Departmental Management Section of the APP, "Measure 1f. Improve the management of information technology" describes the resources supporting the IT Security Program, including the number of personnel assigned to the program and the amount of funding required to support the various components of the program, such as salaries, training, and operating unit IT security functions. In addition, in the Departmental Management section, Commerce provided milestones associated with implementing its IT security program maturity.

At the operating unit (OU) level, Commerce requires OU Chief Information Officers (CIOs) to implement a standard process to manage the selection, control, and evaluation of IT investments. As part of this, each OU should establish an IT Review Board. CIOs are further required to implement a standard process and establish IT investment scoring and ranking criteria for the operating unit's review board to use to determine which IT investments are best suited to meet operating unit needs. At the departmental level, Commerce has established and chartered a Commerce Information Technology Review Board (CITRB), documented a description of the CITRB process, and established circumstances under which an operating unit is to present an investment to the CITRB. For example, the department has established investment characteristics that automatically require submission to the CITRB, such as any system with a life cycle cost of over $25 million. Further, operating units are instructed to assume that the CITRB will want to consider certain types of investments such as something politically sensitive or mission critical, department-wide or inter-operating unit, or investments that deviate from cost, schedule, or performance goals. The CITRB charter also notes that such characteristics require review by the CITRB.

In the Department of Commerce's Strategic Information Technology Plan (2005-2009), the department has established performance measures related to its IT goals. For example, the department established an IT goal of improving the department's CPIC process, by in part measuring the CPIC maturity of the department's operating units. The department had a goal of having 80% of the units at level 3 (on a scale of 1 to 5) and 25% at level 4 by the end of fiscal year 2005. Actual FY 2005 performance was measured at 86% of units at level 3 and 28% at level 4. Another departmental IT goal was to utilize the DOC Enterprise Architecture and operating unit-specific architectures to identify and support key management decisions. One measure of this goal was to track progress in the use of the department's EA through the application of the Departmental EA Maturity Model and an assessment against the OMB EA Assessment Framework. The department had a goal for fiscal year 2005 of achieving a solid 3.0 (i.e. "effective) as measured against OMB's Framework and having 80% of the operating units at a level 3 or higher, with selected operating units (25% or more) achieving level 4. Commerce assessed its operating units as 86% at level 3 or higher and 28% at level 4 or higher.

To evaluate the performance of its IT management processes, Commerce routinely and regularly benchmarks its IT management practices against those of leading organizations. Some means by which Commerce compares its IT functions to those of similar organizations are described in Commerce's Information Technology Capital Planning and Investment Control Process. The department's Fiscal Year 2004--Fiscal Year 2009 Strategic Plan also describes the practice of evaluating all department activities through the use of reviews and reports generated by the department's OIG; OMB, GAO, other congressional organizations; governmentwide task forces; and other objective sources. For example, a case study of a project within Commerce's National Oceanic & Atmospheric Administration was published by the Gartner Group as an example of applying earned value management to small IT projects.

Commerce has established the requirement at the Operating Unit level for a process for selecting, controlling, and evaluating IT investments. Required activities are established as responsibilities of the Commerce Operating Unit Chief Information Officers. Commerce has also established and chartered a Commerce Information Technology Review Board (CITRB), including a description of the CITRB process, and established circumstances under which an operating unit is to present an investment to the CITRB. Commerce has also established investment characteristics that automatically require submission to the CITRB, such as any department-wide system or any system with a life cycle cost of over $25 million.

Commerce has amended several of its capital planning documents to address this requirement. First, Commerce's Exhibit 300 instructions now specify that all IT investments should use COTS products whenever possible and detailed justification is required if such products are not used. Second, the Commerce IT Review Board expectations document specifies that when an Operating Unit appears before the board, that it demonstrate how the proposed IT investment supports work processes that have been simplified or redesigned to reduce costs and improve effectiveness and requires the Operating Unit to encourage maximum use of COTS software. Finally, the December 2003 call for Strategic IT Plan updates specifies that plans should include strategies to ensure that proposed IT investments support work processes that have been simplified or redesigned, and that proposed IT investments make maximum use of COTS software.

In response to GAO's review, Commerce amended its IT investment evaluation criteria (used by the Commerce IT Review Board) to include a consideration of risk-adjusted return-on-investment and net risks.

Commerce has amended several capital planning documents to address this requirement. First, Commerce's Exhibit 300 instructions require the listing of modular design strategies to be used and explain how a modular approach can help mitigate project risk. Second, the Commerce IT Review Board (CITRB) Expectations specify that an Operating Unit (when appearing before the CITRB) demonstrate how the proposed IT investment is taking a modular approach to system design and development. Finally, the December 2003 call for Strategic IT Plan updates specifies that plans should take a modular approach to system development.

Commerce has revised its Commerce IT Review Board Expectations to require projects in the control phase to report, among other things, progress against the baseline schedule, budget, and functional criteria and to describe the role of the earned value management system used in managing the IT investment. Projects are also required to discuss a transition plan if the investment is replacing a legacy system and to address any former IT Review Board findings, corrective actions, and recommendations. Board members then evaluate this control information using Commerce's IT Review Board Evaluation Criteria, which covers such categories as Project Management, Risk Management, and Architectural Compliance. The project's rating in each of these categories is assigned as Green, Yellow, Red, or N/A and recorded on an Evaluation Criteria template. The Board then uses this information to make recommendations on the continuation or termination of projects in the control phase, including when projects fail to meet performance, cost, or schedule criteria.

In the May 2006 Department of Commerce IT Investment Performance Measurement and Performance Reporting Policy, the department requires the submission of quarterly Earned Value performance reports and annual Operational Analysis reports. The Office of the CIO reviews these reports and reports to the Commerce IT Review Board and the Deputy Secretary on investments that deviate from cost, schedule, or performance goals by more than 10%.

To improve the department's IT strategic planning/performance measurement processes, the Department of Defense provided an annual report to the President and the Congress, which included its fiscal year 2004 performance plan. The report included the resources and time periods required to implement the information security program plan required by FISMA, aligned its performance measures with the goals in the plan, and included a description of major IT acquisitions contained in its capital asset plan that bore significantly on its performance goals.

The Secretary of Defense has established a documented process for measuring progress against the Department of Defense (DoD) IT goals. This process is described in the June 2004 DoD Chief Information Officer (CIO) Strategic Plan for Information Resources Management (IRM). This process involves the use of a Balanced Scorecard (BSC) approach which allows the DoD CIO and other Principal Staff Assistants to establish, within their sphere of responsibility, specific outcome goals, performance measures and initiatives that align with overarching strategies such as the Quadrennial Defense Review Report (i.e., the DoD Strategic Plan) and the President's Management Agenda (PMA). Governance forums bring in the people, oversight, control and evaluation dimension; for example, the Defense Resources Board (DRB), Defense Acquisition Board (DAB), Joint Capabilities Integration and Development System (JCIDS), and the DoD CIO Executive Board. The net effect is that net-centric initiatives with specified goals, objectives, outcomes, and metrics are aligned with the CIO goals, DoD outcome goals, risk factors, and the new defense strategy, as well as the PMA. CIO goals are cascaded down to the DoD Components and sub-components, which will be expected to develop supporting initiatives. Work groups, comprised of the DCIO staff and Components, have been formed to address the strategic emphasis areas and initiatives. These groups periodically report to the DCIO and the DoD CIO Executive Board on their progress. The entire process aligns each organization's initiatives with the next higher or lower organization and ensures all organizations performance objectives are related to indicators of the Department's performance at the highest levels. As a result of this process, DoD has begun to develop and track actual-versus-expected performance of IT performance measures related to its IT goals.

To improve the department's IT strategic planning/performance measurement processes, the DoD has developed and is tracking actual-versus-expected performance of IT performance measures related to its IT goals in, for example, its annual Performance and Accountability Report. DoD's IT performance metrics efforts are currently focused on two specific goals intended to transition the Department into a more efficient and effective user of 21st Century IT capabilities. The goals are (1) make information available on a network that people depend on and trust, and (2) populate the network with new, dynamic sources of information to defeat the enemy. In support of these goals, the Department developed metrics to track performance - particularly in the areas of two key efforts: Information Assurance and the Global Information Grid (GIG). Specific metrics supporting our various IT functions are active in the DoD Balanced Scorecard, but are subject to revision as our processes continue to evolve and mature. For example, DoD is developing outcome and output metrics to measure progress toward achieving the strategic goals of the ASD (NII)/DoD CIO.

In October 2005, DoD issued Directive 8115.01, Information Technology Portfolio Management, which establishes policy and assigns responsibilities for the management of DoD information technology (IT) investments as portfolios that focus on improving DoD capabilities and mission outcomes consistent with public laws, OMB guidance, and other DoD policies. The Directive is comprehensive; it applies to all organizational entities comprising DoD ("DoD components"), the Department's Warfighting, Business, Intelligence, and Enterprise Information Environment mission areas, and all current and planning IT investments, including National Security Systems. In October 2006, DOD issued Instruction 8115.02, Information Technology Portfolio Management Implementation, to implement Instruction 81115.01 and describes procedures for managing DoD IT investments as portfolios. This Instruction shows how the investment management process relates to other DoD processes and its integrated architectures, and identifies external and environmental factors that influence the process, such as legislation and federal guidance.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, the Department of Education reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

The Department of Education's FY 2005 IRM Strategic Plan documents a process for measuring progress against the department's IT goals and broadly assigns roles and responsibilities for achieving these goals. Specifically, the plan links the IRM goals to the department's mission goals and identifies a strategy for attaining each goal. Roles and responsibilities for achieving IRM goals are assigned to the Office of Chief Information Officer and its appropriate subcomponents, working in cooperation with the mission area offices.

The 2006 Department of Education IRM Strategic Plan for FY 2007-2011 describes the three areas of primary IT focus (IT Strategic Goals) for the Department; describes objectives for each of the three goals; and identifies how IT contributes to program productivity and the effectiveness and efficiency of agency operations. The plan also depicts the linkage (alignment) between the Department's lines of business; IT delivery organizations; current, planned, and future new IT investments; Presidential E-Gov strategies; current and future IT shared services; and IT and IM processes, such as enterprise architecture, Information Assurance, Lifecycle Management, etc. The Department developed performance measures related to how IT contributes to program productivity, the effectiveness and efficiency of agency operations, and other goals and enterprise-wide IT initiatives in its IRM Strategic plan, and discussed them in its Enterprise Transition Strategy Plan issued in 2008. With respect to performance measures for Computer Software Piracy, in September 2004 the Department CIO issued a Handbook for Software Management and Acquisition Policy to effectively control and monitor compliance with applicable standards that sets forth steps the Department takes in preventing and checking for software piracy, including making annual reviews and assessments to evaluate the effectiveness of its Software Management and Acquisition (SMA) Policy using automated software tools. In addition, the Department also reported that their Standard Security Training, which is required to be taken by all employees and contractors each year and is tracked in the annual FISMA report.

The 2006 Department of Education IRM Strategic Plan for FY 2007-2011 outlined the future vision for each of the Department's enterprise-wide IT initiatives. The progress toward achieving these enterprise-wide IT initiatives' future visions is measured and reported through the Department's accompanying Enterprise Transition Strategy Plan. The first such Plan was issued in February 2008 and contains quantifiable and results-oriented implementation milestones that define major events in the IT initiative's lifecycle and performance milestones that define performance improvements resulting from each implementation milestone. The Department also reported the status on each of the milestones in its February 2008 Enterprise Transition Strategy Plan and informed us that all milestones are regularly measured (in terms of actual versus expected) and that such results are reported to OMB through the Department's annual Enterprise Architecture assessment submission and Education Exhibit 300 business cases (redacted versions of which are accessible online). Further, the Department reported that investments not aligning to the Department's IRM Strategic Plan or failing to meet defined milestones are reported through the Department's IT Governance Process for mitigation according to the Department.

According to the Department of Education's June 2004 Investment Review Board Charter, the department only has one IT investment management board, called the Investment Review Board (IRB) for decision making related to IT investments. According to the charter, the Planning and Investment Review Working Group (PIRWG) makes recommendations based on the IT business case for consideration by the board. The Data and Information Review Working Group provides added advice to the CIO regarding the management, consolidation, and integration of the department's data collection activities. The charter also sets forth the roles and responsibilities of the IRB (chaired by the Deputy Secretary and vice chaired by the CIO), as well as those of the PIRWG.

The Department of Education's September 2004 IT Investment Management Guide's FY2005-2006 Select Phase Review Plan requires that all IT investments satisfy fundamental characteristics necessary to effectively maintain the department's IT investment portfolio. This includes demonstrating that each proposed IT investment supports work processes that have been simplified or redesigned before being approved for scoring.

The Department of Education revised its project selection criteria to include net risks and risk-adjusted return-on-investment as part of its scoring methodology for its fiscal year 2006 information technology investments.

The Department's Planning and Investment Review Working Group (PIRWG) conducts Independent IV & V reviews as part of the ITIM Control Phases to assess the validity of cost and schedule as well as performance measure information provided. The PIRWG also conducts quarterly IV & V reviews of all high risk IT investments. In addition, the Department has recently completed development of a detailed Post Implementation Review (PIR) Guide.

Department of Education Control Phase reviews for significant IT initiatives occur quarterly, or at the completion of the life cycle phases. Results from the Control Phase reviews are used to determine initiative status, to measure variances, to review progress in meeting improvement items, to adjust funding levels, and other items. The Planning and Investment Review Working Group (PIRWG) conducts in-depth initiative assessments if certain circumstances exist, such as the initiative shows a variance of minus 10% or less in either cost or schedule, the initiative is not meeting established performance measures, or the initiative is determined to be high risk. Minutes of PIRWG meetings provided documentary evidence to support that the department had implemented Control Phase actions to track resolution of corrective actions for underperforming projects.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, the Department of Energy reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

The Department of Energy has established two strategic goals in its FY 2006-2008 Information Resources Management Strategic Plan. One goal is to simplify access to DOE information and products. Within this goal, Energy has identified an objective of supporting the department's e-Government activities and an outcome of realizing cost savings through streamlined information technology activities. A second goal is to institute a robust information technology governance program within DOE. This goal has, among several, an objective of maintaining a complete, mature enterprise architecture which will identify and implement common solutions and eliminate redundant systems. This goal also aims to ensure effective information technology project performance in order to adhere to cost, schedule, and performance targets.

Consistent with our recommendation to identify metrics for assessing effectiveness in controls over software piracy, the Department of Energy has taken steps to implement the Department-wide software enterprise licensing agreement program to consolidate Commercial Off the Shelf (COTS) software contracts. By integrating and building on existing capabilities within the department, this program allows Energy to develop and enforce policies and procedures supporting the identification, acquisition, oversight and compliance of enterprise software agreements and supports Clinger-Cohen, Paperwork Reduction Act, and other legislative and DOE policies. Energy performance indicator involves defining an enterprise license inventory process to enabling the department to track software licenses and optimize license utilization across the enterprise.

As we recommended, the Department of Energy's Information Resources Management (IRM) Strategic Plan for FY 2008-2010 links to the department's Strategic Plan by relating the IT goals of the department to Energy's overall strategic themes. The IRM Strategic Plan also provides a detailed breakdown of outcomes, objectives, and performance measures associated with each IT goal. These goals are tracked by way of Energy's annual performance reports. For example, objectives to expand E-Government and improve information security are tracked in Energy's Annual Performance Report for FY2007. In addition, annual performance reports provide actual versus expected tracking of performance measures associated with Energy's goal for using information technology to improve mission accomplishment, such as tracking the percentage of Web site users over time.

Consistent with our recommendation, the Department of Energy included, as part of its September 2007 Guide to IT Capital Planning and Investment Control (CPIC), a step for benchmarking its IT management practices against other sources such as GAO's Information Technology Investment Management Framework. Specifically, the guide calls for benchmarking performance to assess lessons learned and to make continued improvements to the department's processes.

To improve the Department of Energy's IT investment management processes, the department is implementing the Continuous Asset Monitoring System (CAMS) capability that includes establishing a mandatory infrastructure for the collection and reporting of information technology assets: including desktops; workstations; servers; network components, such as routers; security devices, such as firewalls, access controls, and user authentication; printers; wireless access systems; mid-range computing resources; mainframe computers; process control systems containing computers; and handheld devices in the department. Energy has established an IT inventory as one component of an overall IT asset monitoring initiative. The Office of the CIO acquired asset auditing tools to support a process for inventorying and tracking IT assets agency-wide. Sites are providing asset inventory information.

The Department of Energy's Guide to IT Capital Planning and Investment Control (CPIC) identifies OMB policy guidelines that require agencies to ensure that IT investments are put into effect in phased, successive, chunks that are short-term and narrow in scope and independently solve part of an overall mission. The Guide to IT CPIC further states that the Energy Office of the CIO then reviews the Exhibit 300 submissions for proposed investments to ensure that the investments are consistent with criteria established and promulgated by OMB and that Exhibit 300s that fail these criteria are sent back to the programmatic offices for review and correction.

In the September 2005 Guide to IT Capital Planning and Investment Control, the Department of Energy has documented the role, responsibility, and authority of the stakeholders involved in the IT investment management process. These stakeholders are the OCIO IT Planning Division, the A Team, the Corporate Review Budget Board, and the DOE Management Council. The Guide to IT CPIC clearly distinguishes between all these organizations, establishes a hierarchy among them, and discusses the various decision making responsibilities of each. Further, the Guide also documents the Control and Evaluate phases of its CPIC process, including requirements for monitoring the performance of ongoing investments and the requirement for the conduct of a post-implementation review of completed investments.

In its FY 2006-2008 Information Resources Management Strategic Plan, the Department of Energy has established a strategic goal to institute a robust information technology governance program within DOE. As part of this goal, Energy has established an objective of recruiting, developing, and retaining a qualified, professional IT workforce. According to the IRM Strategic Plan, achieving this objective will align workforce skills with DOE missions and priorities. Further, in the department's Target Enterprise Architecture, Energy has mapped how this objective is linked to DOE business functions, including Human Resource Management, Information and Technology Management, and Planning and Resource Allocation.

HHS has implemented three of the four factors listed in this recommendation. In December 2005, the Department of Health and Human Services published revised investment management policies and procedures that govern its Capital Planning and Control Process (CPIC). The revised policies address (1) how CPIC process relates to other agency processes, (2) external and environmental factors that may impact the process, and (3) a description of the relationship between the process and the department's enterprise architecture. HHS stated that they have not yet conducted an IV&V of its CPIC process.

In December 2005, the Department of Health and Human Services (HHS) published investment management policy and procedures governing its Capital Planning and Control Process (CPIC). The CPIC policy sets forth procedures for its department-level Information Technology Investment Review Board (ITIRB) to document and review IT investments. For example, the CPIC Policy calls for the ITIRB to ensure that each IT investment requiring Department-level review complies with CPIC and other management policies. The ITIRB is also responsible for establishing a prioritized IT investment portfolio during each annual budget cycle to support HHS budget formulation. Finally, other committees and councils serve advisory roles to the ITIRB. Specifically, the HHS CIO Council reviews the technical approach of individual IT investments coming before the ITIRB and makes recommendations to the ITIRB regarding technical aspects of affordability, soundness of design, risk, and compliance with architectural and security standards.

In December 2005, the Department of Health and Human Services (HHS) published investment management policy and procedures governing its Capital Planning and Control Process (CPIC). In the CPIC policy, the department documented the alignment and coordination of responsibilities of its IT investment management boards for decision making related to IT investments. The policy outlines an IT governance structure that includes Information Technology Investment Review Boards (ITIRB) at the department level and operating-division level. The HHS operating division (OPDIV) ITIRBs are tasked with reviewing IT investments unless it meets one of the seven Department-level CPIC review thresholds; in such cases a review is made by the Department-level ITIRB. For example, the Department-level ITIRB reviews cross-cutting IT investments for which a single Department-wide solution may be possible, a Department-wide team has been established or planned, or that impacts more than one HHS OPDIV.

The Department of Health and Human Services (HHS) implemented its portfolio management tool in May 2004 which serves as the primary repository for identifying and collecting information about both department and component agency IT projects and systems. The department's portfolio management tool is easily accessible to decision makers at both the department and component level and the Office of the CIO has provided decision makers with various training manuals and guidance memoranda. The portfolio management tool also contains criteria for analyzing, prioritizing, selecting and reselecting new and ongoing IT investments.

In December 2005, the Department of Health and Human Services (HHS) published investment management policy and procedures governing its Capital Planning and Control Process (CPIC). As part of the CPIC policy, the department established a policy requiring that IT investments should support simplified or otherwise redesigned work processes to minimize cost and improve effectiveness. This policy is contained in section 4.8.4 of the December 2005 Health and Human Services OCIO Policy for IT Capital Planning and Investment Control (CPIC).

The Department of Health and Human Services (HHS) has established a two-component IT investment selection process: (1) investment screening for new investment proposals and (2) investment scoring and screening for ongoing investments. The process includes specifying the roles of key people involved as well as identifying significant events and decision points. In addition, the process explains how the IT investment management process is coordinated with other organizational entities and their plans and processes.

In December 2005, the Department of Health and Human Services (HHS) published investment management policy and procedures governing its Capital Planning and Control Process (CPIC). In the related CPIC procedures, decision-making rules are documented that help guide the departmental investment review board's oversight of IT investments during the HHS CPIC control phase. According to the CPIC procedures, criteria used during the Control Phase to guide decision-making help ensure that an IT investment's projected benefits are being realized; cost, schedule and performance goals are being met; risks are minimized and managed; and the investment continues to meet strategic needs. Based on the results of these periodic control reviews, the HHS ITIRB will determine if a project is continued, modified, or terminated.

The Health and Human Services' (HHS) IT Capital Planning Investment Control Policy, published in December 2005, includes requirements for IT investment milestone reviews. Specifically, under this policy, milestone reviews are conducted to monitor the progress or performance of on-going IT investments against projected cost, schedule, performance, and delivered benefits.

The Department of Health and Human Services (HHS) published Capital Planning and Investment Control policies that provide for tracking the resolution of corrective actions for under-performing projects. These policies, documented in HHS' Procedures for Information Technology Capital Planning and Investment Control, require creation of a Corrective Action Plan (CAP) and the reporting of results to the HHS investment review board when an investment is over budget and cost or behind schedule by more than 10% for more than two quarters. According to HHS documentation, the department tracks the performance of underperforming projects, including tracking completion of a Corrective Action Plan and the date of each investment's approval by the HHS investment review board for cost and schedule re-baselining.

In December 2005, the Department of Health and Human Services (HHS) published investment management policy and procedures governing its Capital Planning and Control Process (CPIC). According to the CPIC policy, post implementation reviews (PIR) are required to validate benefits and costs following each IT investment or useful segment's implementation. PIRs are also required for any IT investment cancelled before going into operation.

The Department of Health and Human Services' (HHS) Information Resources Management Strategic Plan 2007-2012, completed in February 2007, documents how the department's strategic planning, tactical planning, capital planning and investment control (CPIC), and performance management planning processes have been integrated. For example, in the plan, HHS describes the planning process consisting of four key steps: understand the business, develop the strategy, implement the strategy, and evaluate performance. The department also describes how it held segment workshops focused on a particular domain of the HHS mission, and called upon subject matter experts to identify desired IRM support both for specific business needs and for a robust and flexible general infrastructure. HHS' plan also documents how the critical partners refer data-justified IRM needs back ultimately to the decision-makers at the Secretary's Budget Council. Through its process, as described, HHS helps to promote collaboration across the department and build upon previous strategic planning efforts while integrating the HHS Segment Architecture into the IRM strategic planning.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, HHS reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA. Business cases for major IT acquisitions identify associated performance goals.

The Department of Health and Human Services (HHS) documented six strategic goals in its Information Resources Management Strategic Plan 2007-2012, completed in February 2007. Consistent with our recommendation, as documented in the plan, HHS developed performance measures to assess progress against these goals. HHS also identified the purpose of each performance measure, the methodology for collecting data, and the frequency with which each performance measure will be assessed.

The Department of Health and Human Services has developed a performance measure related to the effectiveness of controls to prevent software piracy. As a next step, the Department is developing baseline values and performance targets related to this measure.

The Department of Health and Human Services has developed a strategic-level IT performance management program. The program includes the development of performance measures, establishment of targets for those measures, collection of data for each measure, analysis of that data, and reporting on the performance measure.

On January 28, 2005, the Department of Housing and Urban Development (HUD) reported that it had established policy requiring that proposed IT investments support work processes that have been simplified or redesigned to reduce costs and improve effectiveness and that makes maximum use of COTS software. HUD provided documentation of the policy that is stated in its IRM handbook dated November 2004 and in its Information Technology Investment Management Process Guide dated September 2004.

On April 12, 2004, the Department of Housing and Urban Development (HUD) reported that it had established net risk and risk-adjusted return on investment in HUD's fiscal year (FY) 2006 selection criteria. On March 24, 2004, the Technology Investment Board Executive Committee approved the FY 2006 selection criteria. Information on net risk and risk-adjusted return on investment is available on the Department's intranet. HUD provided a copy of the approved criteria.

On January 28, 2005, the Department of Housing and Urban Development (HUD) reported that it had established a policy requiring modularized IT investments. Specifically, HUD revised its IRM Handbook 2400.1, Select User Guide and the Information Technology Investment Management Process Guide to include a policy that requires IT investments to be modularized (e.g., managed and procured in well-defined, useful segments) to the maximum extent achievable. HUD included the requirement for modularized investments in its IRM handbook dated November 2004 and in its Information Technology Investment Management Process Guide dated September 2004.

On January 28, 2005, the Department of Housing and Urban Development (HUD) reported that it had revised its Portfolio Management Review Board (PMRB) Charter to require that IT projects report on deviations in system capability and monitor IT projects at key milestones. Specifically, as required by the revised October 2004 charter, reviews are to be conducted on IT investments determined to be "unhealthy" or to have reached key milestones in order to assess project and project manager performance. HUD PMRB reviews are also required for analyzing deviations from planned system capabilities and from planned business performance goals.

On January 28, 2005, the Department of Housing and Urban Development (HUD) reported that it had developed a process to use independent verification and validation (IV&V) reviews, when appropriate. Specifically, in September 2004, HUD revised its Information Technology Investment Management Process guide to incorporate IV&V reviews into its project monitoring process, and in October 2004, HUD revised the Portfolio Management Review Board charter to require that the board will perform IV&V reviews.

In response to our recommendation, HUD updated its ITSP Process Guide on April 26, 2005, to identify key stakeholders such as HUD ITSP Staff, ITSP Executive Steering Committee, and the CIO, among others, and their specific roles, including the unique perspectives they provide. As a result, the HUD is better able to articulate its strategic direction and to establish linkages among planning elements such as goals, objectives, and strategies. In addition, HUD's Human Capital Strategies and Organizational Policy describes how its Implementation Action Plan addresses the need for uniform planning, including the impact of human resources and policy requirements on IT Security and E-Government strategies.

According to the Department of Housing and Urban Development (HUD), the HUD IT Security Program was formally established in July 2005. The Department Annual Performance Plan for FY 2007 includes a performance indicator describing how HUD will continue to meet specified IT-related security requirements and controls in compliance with FISMA. In the description of this indicator, HUD cites the budget amount requested for FY 2007 and the current personnel level allocated to support the HUD IT Security Program. In addition, the section discusses a number of goals related to three requirements to be implemented during FY 2007 to enable the Department to continue to reduce risks and vulnerabilities and protect its information systems and resources from unauthorized access, use, and modification.

In January 2004, GAO recommended that the Department of Housing and Urban Development (HUD) develop a documented process to develop IT goals in support of agency needs, measure progress against these goals, and assign roles and responsibilities for achieving these goals. In response, HUD updated its IT Strategic Planning (ITSP) Process Guide, dated April 2005, to document the IT Strategy Development Cycle, which, among other things, guides the Department in identifying when to develop a new plan, how to develop IT strategies, what stakeholders to involve in the process, and how and when to maintain and assess progress toward the plan. HUD also documents workflows used to assess progress against the IT Strategic Plan and Implementation Action Plan. Further, HUD has identified and assigned roles and responsibilities of key stakeholders in order to effectively measure progress against their IT Goals.

The Department of Housing and Urban Development (HUD) IT Strategic Plan, FY 2005- FY2010, states that IT is an enabler of HUD's mission, directly supporting HUD's business by supporting HUD's intermediate customer, industry business partners, and the ultimate customer, the citizen. The Plan includes a goal that focuses on improving the impact of IT on HUD's core lines of business and functions, and as a result, improving HUD's service delivery and customer service. The plan includes a number of performances measures related to how IT contributes to mission impact, as well as improved efficiencies, effectiveness, and service delivery. The HUD IT Security Policy handbook incorporates the software piracy policy through identification of configuration management and copyrighted software policies and procedures. All infrastructure computing platforms (Desktops, Notes, File Servers, Internet, Intranet, Mainframes) have Department documented standards and procedures for tracking and monitoring installed software, and access controls and configuration management procedures are in place. Performance measurements are implemented through annual assessments of each information technology system (general support system and major application) in which controls to assure protection against copyright infringement are required to be assessed; use of tracking software within the HUD processing environment by HUD's infrastructure contractors to identify rogue programs; restriction of administrative rights by HUD contractors during workstation set-up to prevent the loading of unauthorized software; and through mechanisms designed within licensed software to restrict the number of licenses that can be in use. The effectiveness of these controls is addressed in part through monthly vulnerability scanning, quarterly Plan of Action and Milestones (POA&M) reviews, annual penetration testing, and re-certification of systems controls at a minimum of every three years. In addition, to gather and assess data on the performance of the HUD IT Security Program, four Performance Measures and ten IT Strategy Implementation Action Plans for IT Security have been established and are tracked in the HUD IT Strategy Quarterly Assessment process.

The Department of Housing and Urban Development (HUD) IT Strategic Plan, FY 2005- FY2010, established four broad goals in its IT Strategic Plan. The HUD Strategic Plan also included a number of specific Objectives, Outcomes, Performance Measures, and IT Strategy Implementation Action Plans for each goal. In order to gather and assess data on HUD's performance against its IT goals, HUD implemented in 2005 an IT Strategy Quarterly Assessment process in which it tracks and compares actual performance to the performance baselines and performance established in the prior quarterly assessment.

On January 28, 2005, the Department of Housing and Urban Development (HUD) reported that it had developed a mechanism for benchmarking the Department's IT processes, when appropriate. Specifically, HUD's September 2004 Information Technology Investment Management Process Guide requires routine and systematic benchmarking of the department's IT management processes against models of excellence in the public and private sectors and sets forth procedures for determining processes to benchmark and for identifying entities with similar complexity and structure. HUD expects to complete at least one benchmarking effort each fiscal year.

In Justice's October 2004 DOJ IT Strategic Management Framework, the department documented its IT strategic management processes. Specifically, Justice's framework outlines roles and responsibilities, including the relationship between the CFO and CIO within these processes. In addition, the framework describes the method by which Justice defines its program information needs and the processes followed in developing strategies, systems and capabilities to meet the strategic and business priorities of the department.

In Justice's October 2004 IT Strategic Management Framework, the department documented how its IT management operations and decisions are integrated with other departmental processes, including human capital management. Specifically, the framework documents Justice's IT strategic management phases--IT planning, IT Funding and Architecture, and IT Investment Oversight--with continuous integrated processes that include Human Capital, Portfolio Management, Enterprise Architecture, IT Security, E-Gov and others. In addition, Justice's framework describes the human capital planning and organizational transitions needed to staff and manage the IT investment portfolios and projects in the pipeline for approved funding.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, Justice reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

In Justice's DOJ IT Strategic Planning Framework document, issued in October 2004, the department outlines its process followed to develop IT goals in support of agency needs. Specifically, the document outlines the general approach Justice follows to identify IT strategic and business priorities, information use and management requirements, and technology integration as well as other issues in developing a workable strategy for migrating the current IT architecture to the target architecture. Various other sub-processes and their linkages to the IT strategic planning process are also described. The framework further identifies the process by which Justice measures performance against established goals and objectives. Roles and responsibilities for the development and achievement of goals are listed in the framework as well.

Justice's IT Annual Progress Report, dated October 2004, identifies performance measures associated with the IT goals the department established in its IT Strategic Plan (updated in January 2005) and tracks progress achieving these goals. Specifically, the report provides qualitative results achieved since July 2002 by tracking progress made on each targeted objective under the department's IT strategic goal areas. In addition to these performance measures, the report describes results achieved on critical success factors identified in Justice's IT Strategic Plan and gives a description of progress implementing the department's major IT initiatives.

Through the development of the DOJ IT Investment Planning Guide and the Department Investment Review Board (DIRB), Justice has developed work processes and procedures for the department's investment management boards whereby the boards are aligned and coordinate IT investment decisions among its various boards and at the Department level.

In 2005, the Department updated its IRM Order 2880.1B requiring that the Department and its component IT investments be aligned with the Department's enterprise architecture. Specifically, the order requires that the EA program ensure that department and component IT investments are aligned with the department architecture, and that those investments are delivering the expected technical and functional performance results.

Department of Justice guidance and instructions describe methods used by the department for establishing priorities for information technology decision making. For example, Justice components establish project/program priority using standard criteria as input to the governance board's budget decision making process. In addition, according to Justice officials, the Chief Information Officer (CIO) consolidates component prioritized budget requests into the department's information technology (IT) portfolio that is prioritized based on the department's investment priorities that is reviewed by the Department Investment Review Board. Justice provided the fiscal year 2008 Spring IT Budget that shows the prioritization of investments across department components based on a 10 point scale.

Justice established responsibility within the Department Investment Review Board?s charter for the board to recommend corrective action to continue, modify, or terminate a project. In addition, the Department established an automated system to track the progress made for high cost, high risk, or high visibility projects. Program managers are also required to report the status of all open actions monthly and must review the status of all open actions at the beginning of each Department Investment Review Board project review. For example, on December 3 2007, the Department Investment Review Board met with component officials regarding the United Financial Management System. According to the documentation, the board directed two action items and the next meeting was scheduled for March 2008. Also, on January 18, 2008, the board met with component officials to discuss the Sentinel program. According to the documentation, the board direct the component and the department Chief Information Officer to take action and scheduled the next review meeting for April 2008.

In the FY 2005 Department of Labor's IT Capital Planning Guide, the department has described the relationship between the IT investment management process and the department's enterprise architecture. The department has defined enterprise architecture as one of 4 phases in the IT investment management process and the Office of the CIO is required to recommend EA subject matter experts to serve on the integrated project teams for individual investments. Additionally, Labor's Capital Planning Guide requires a thorough discussion of enterprise architecture considerations as one of 10 sections of the Exhibit 300 Business Case for a proposed investment. Finally, the Office of the CIO will analyze all IT initiatives to determine, among other things, whether the proposed initiative is aligned with the department's enterprise architecture.

The Department of Labor's internal assessment for proposed fiscal year 2008 information technology initiatives considered whether a proposed IT initiative accounts for net risks and risk-adjusted return on investment in the proposal's cost-benefit analysis.

The Department of Labor's FY 2005 System Development Life Cycle Manual (SDLCM) requires that projects should follow a phased and modular incremental approach to system development as recommended.

The Department of Labor developed decision-making rules. During the Control phase of its Capital Planning process, for example, Labor follows decision-making rules that help guide the oversight of IT investments by the department's investment management function. Specifically, during quarterly reviews, project managers are required to submit documentation on project status that includes performance measures, goals, targets, and explanations for any cost and schedule variances from established baselines. Such information as this, and other data, is used to determine whether the project should proceed, be modified, replaced, or retired. Project performance is reviewed against risk factors such as schedule or budget variance greater than 10% and major milestones missed. At the conclusion of the quarterly review, a decision will be made to either proceed with the project, modify the project, or terminate the project. The results of quarterly reviews are then reviewed by the Technical Review Board.

The Department of Labor implemented an Earned Value Management System (EVMS) for major IT projects to address this recommendation. The EVMS is designed to serve as an integrated set of processes for monitoring cost, schedule, and performance of major IT investments over time. While an investment is in the Control phase, the project manager is responsible for generating and submitting, among other things, reports that demonstrate implementation of EVM and EVM performance measures. These reports are submitted to a Quarterly Review in which project managers are required to, among other things, explain any cost and schedule variances provide performance measures, goals, and targets. The Quarterly Review results are used to determine whether the project should proceed, be modified, replaced, or retired. Project performance is reviewed against schedule for a budget variance greater than 10% and any missed major milestones or deadlines. The department's Technical Review Board conducts operational reviews of all ongoing IT investments and recommends IT portfolios to the Management Review Board, which provides strategic review of all ongoing IT investments.

Department of Labor has published the resources and time periods required to implement the information security program plan required by FISMA as an appendix to the 7/6/07 version of the department's Computer Security Program Plan. The Director of the Department of Labor Office of the Chief Information Officer (DOL OCIO) stated that this program plan was developed to ensure compliance with the FISMA and that the plan of action and milestones (POA&M) in the appendix was a living document and updated on a periodic basis, most recently May 30, 2007. The POA&M identifies specific weaknesses and for each weakness, provides the type of resources required to address the weakness (including dollar amounts), the anticipated completion date, and various milestone task descriptions (and completion dates) within the weakness.

The Department of Labor's FY 2005-2009 IT Strategic Plan includes a description of the IT strategic planning process and how goals are established. The Strategic Plan also includes five IT strategic goals. Labor has mapped these IT goals to the department's E-Government Scorecard Criteria and documented how the criteria support the accomplishment of those goals. Projects' performances against these criteria are assessed either quarterly or on semi-annual basis. Individual initiatives are responsible for meeting the E-Government criteria that support the E-Government areas that in turn support the accomplishment of IT strategic goals.

The Department of Labor established five IT strategic goals, several of which relate to how IT contributes to program productivity. Labor's E-Government goal is designed to ensure that IT initiatives and investments are customer-focused, results-oriented, market-based, and cost-effective. Additionally, the department's IT Management and Governance goal is meant to promote cost-effective IT solutions by sharing and implementing best practices, collaborating on projects and initiatives, and ensuring interoperability where appropriate.

The Department of Labor developed several performance measures related to how IT contributes to program productivity and efficiency. For example, Labor measures average IT help desk response time and % of IT projects delivered on time. Labor also measures the percentage of Labor major information systems operating securely. All 21 performance measures identified by Labor as contributing to productivity and efficiency are tracked in terms of actual-versus-expected performance. For example, in 2006, 82% of IT projects were delivered on time, exceeding the goal of 60%. Also in 2006, 100% of major information systems operated securely, meeting the department's goal. Labor reported that the department has been directed to take the necessary steps to monitor and ensure compliance with Executive Order 13103, "Computer Software Piracy."

In their joint FY 2006 Performance Summary, the Department of State and U.S. Agency for International Development (USAID) identified the resources and time periods necessary to implement an information security management program required by the Federal Information Security Management Act (FISMA). Specifically, under Goal 2--MODERNIZED, SECURE, AND HIGH QUALITY INFORMATION TECHNOLOGY MANAGEMENT AND INFRASTRUCTURE THAT MEET CRITICAL BUSINESS REQUIREMENTS--the performance summary lists FY 2006 and FY 2005 targets and specific implementation activities associated with these targets to complete such as expanding training and certification of employees and enforcing requirements for annual information security self-assessments.

In January 2004, the Department of State developed a plan for benchmarking the department's information technology (IT) management processes (the plan was updated in October 2004). State's plan establishes a process for benchmarking to assess and improve the department's IT management processes. Two self-assessments were also conducted by State. These assessments baselined the department's investment management processes and were used to identify potential benchmarking metrics. Further work also included identifying potential benchmarking partners.

In its revised Capital Planning and Investment Control (CPIC) Guide, dated February 2005, State established, as a guiding principle, that IT investments will support work processes that have been simplified or redesigned to reduce costs, improve effectiveness, and make maximum use of commercial, off-the-shelf (COTS) technologies. In addition, State set forth specific criteria to be used in the pre-select and select scoring process of both major and non-major IT initiatives. Also, the department noted that it is reviewing the Foreign Affairs Manual (FAM) for IT Project Management to incorporate similar provisions.

In its revised Capital Planning and Investment Control (CPIC) Guide, updated February 2005, State established, as a guiding principle, that IT investments will be structured in useful segments or modules with a narrow scope and brief duration and will have an approved cost, schedule, and performance baseline. In addition, State set forth specific criteria to be used in the pre-select and select scoring process of both major and non-major IT initiatives. Also, the department noted that it is reviewing the Foreign Affairs Manual (FAM) for IT Project Management to incorporate similar provisions.

In its revised Capital Planning and Investment Control (CPIC) Guide, updated February 2005, State included risk-adjusted return-on-investment in its IT project selection criteria. According to the guide, this criteria is used to score IT investments.

In the Department of State's revised Capital Planning and Investment Control (CPIC) Program Guide, updated February 2005, State's Control and Evaluation process includes a requirement for quarterly reviews of IT investments. According to the guide, after IT investments are selected and receive funding, their progress is monitored in the Control Phase. Actual versus forecasted cost, schedule, and performance information on IT investments is reviewed quarterly to determine project health and whether the investment is proceeding in a timely manner towards agreed upon milestones. Corrective actions are identified and tracked, if necessary. In addition, a baseline change request process is included to review changes that may be required as part of the department's capital planning process. Results of the control reviews are reported to the State's E-Gov Public Board to ensure that IT initiatives perform according to cost, schedule, and performance goals.

The Department of State's revised Capital Planning and Investment Control (CPIC) Program Guide, updated February 2004, describes in detail the CPIC Control Review Process. In addition, a CPIC Program Control Review User Guide is developed each quarter to provide added guidance on the control review process for that quarter. In accordance with these CPIC instructions, State E-Gov Public Board oversees funded investments to ensure that IT initiatives perform according to cost, schedule, and performance goals. Specifically, control reviews determine whether the investment continues to fulfill ongoing and anticipated mission requirements and is proceeding in a timely manner towards agreed upon milestones. If the project is under-performing, the reviews determine corrective actions that are required to bring the project back in line with cost and schedule baselines. The CPIC guide also requires the use of ANSI/EIA 748 compliant earned value management (EVM) techniques as a mechanism for the continual monitoring of IT projects and in-depth independent reviews, if necessary.

The Air Force Information Technology (IT) Investment Review Guide outlines the agency's investment certification and annual IT investment management review processes. It identifies the roles/responsibilities, including the Senior Working Group (SWG), the designated IT investment management board for reviewing and approving Air Force IT initiatives prior to submission to DoD. Air Force conducts IT investment management oversight via the certification review process, whereby it recommends the initiation, continuation, modification or termination of systems. For example, the certification of the IDECS II system included an assessment of cost, schedule, and performance, as well as expected outcomes and return on investment. In addition, the annual review of the Enterprise Business System identified changes in cost, schedule, and performance, plus reported corrective action where warranted. Air Force's IT Investment Review Guide also describes procedures for tracking and resolving correction actions identified through the certification process. Air Force maintains a tracking list that identifies by individual system the status of conditions reported. DoD Instruction 5000.2 requires post-deployment performance reviews for major defense acquisition programs and major automated information systems programs to validate expected benefits and costs, and to document and disseminate lessons learned. For example, the Air Force reported on the completion of a pilot prior to deployment for its AFWAY system, as directed by the SWG, using the lessons learned from the pilot to formulate an appropriate deployment strategy.

To improve the department's IT strategic planning/performance measurement processes, the Department of the Air Force established a documented process for measuring progress against the department's IT goals and assigned roles and responsibilities for achieving these goals. The August 2004 Air Force IT/IM Strategic Plan (Air Force Information Resource Flight Plan) addresses this recommendation. The Plan includes nine Air Force Information Strategy (AFIS) goals and performance metrics for each of these goals. The plan also assigns responsibilities for achieving and measuring the performance of each goal. Air Force portfolio management and capital investment reporting activities are included under the AFIS goal to "Ensure responsible stewardship of Air Force financial resources spent on information management and related information technology." The IT Strategic Plan was developed in concert with the Air Force Portfolio Management process. The Air Force reported that a structured Capital Planning and Investment Control (CPIC) process was implemented by means of the Air Force IT Portfolio Management process and that this process ensures IT investments are synchronized with the Air Force vision and priorities.

To improve the department's IT strategic planning/performance measurement processes, the Department of the Air Force has developed IT performance measures related to the IT goals in the department's information strategy and is tracking actual-versus-expected performance. The August 2004 Air Force IT/IM Strategic Plan (Air Force Information Resource Flight Plan) addresses this recommendation. The Plan includes nine Air Force Information Strategy (AFIS) Goals and performance metrics for each goal. There are ten IT performance measures that are currently being tracked. The Department expects to expand the scope of its IT performance tracking in the near future; next steps include combining four Air Force component organization flight plans into an Integrated Roadmap which will contain metrics from all four sources.

The Department of the Air Force issued an Information Technology Budget Reporting and Capital Investment Report Preparation Guide in December 2005 as an aid in developing Air Force (AF) Information Technology (IT) budget report submissions for FY 07. The 2005 guide superseded earlier AF preparation guides developed for previous budget reporting cycles, and complements policy memoranda and other forms of guidance provided by the AF CIO. The Guide includes a description of the relationship between the IT investment management process and the department's enterprise architecture, and an identification of external and environmental factors to be considered in its portfolio development process.

The Department of the Air Force issued an Information Technology Budget Reporting and Capital Investment Report Preparation Guide in December 2005 as an aid in developing Air Force (AF) Information Technology (IT) budget report submissions for FY 07. The 2005 guide superseded earlier AF preparation guides developed for previous budget reporting cycles, and complements policy memoranda and other forms of guidance provided by the AF CIO. The Guide provides that proposed IT investments shall be selected on the basis of evaluation criteria including costs, benefits schedule, and risk elements as well as measures such as net benefits, net risks, and risk-adjusted return-on-investment.

To improve the department's IT investment management processes, the Department of the Air Force has implemented a scoring model and develop a prioritized list of IT investments as part of its project selection process. The updated scoring model is in the Air Force Information Technology Capital Investment Report and Selected Capital Investment Report Preparation Guide dated June 2005. According to the Department, this scoring approach provides the Air Force with an organized way to evaluate investments and results in a high-level prioritization or grouping for all IT investments. The results of the scoring efforts serve as a tool for the Air Force Corporate Structure to use as it develops and balances the overall budget of the Department. Criteria are updated annually to align with updated OMB guidance (Circular A-11 and new DOD Exhibit 300 scoring criteria).

To improve the department's IT strategic planning/performance measurement processes, the Department of the Army has developed IT performance measures related to the Army's enterprisewide IT goals and tracks actual-versus-expected performance. In May 2005, the Department of Army CIO updated and integrated the Army information management and IT strategy (vision, mission, goals, and objectives), developed related performance measures, and began performance reviews. As a next step, the Army CIO is in the process of moving the strategic management system to the Army's automated system and will link and align to Army's updated strategy.

To improve the department's IT investment management processes, the Department of the Army policy describes the relationship between the IT investment management process and the department's enterprise architecture. The Department of the Army Capital Planning and Investment Management process was instituted in the fall of 2004, during which all IT Investments were evaluated for contribution to the Enterprise in line with Army strategic objectives. Following the initial evaluation, a group of subject matter experts met to rationalize the analytical scores in line with emerging requirements or senior level guidance. The finalized list was provided to participants in the Army funding process as a coordinated Army IT funding strategy, identifying where risk could be taken in line with level of funding for an investment, and more importantly, where any decrement in funding would have severe impact across the enterprise. The Army issued an IT Portfolio Governance policy in July 2005 that describes the relationship between the IT investment management process and the department's enterprise architecture.

To improve the department's IT investment management processes, the Secretary of the Army has documented the alignment and coordination of responsibilities of its various IT investment management boards for decision making related to IT investments. The Army CIO has conducted and documented an IT investment strategy process for several POM cycles. The process is documented in the form of a CIO executive board charter linking stakeholders across the Army, including an example of enterprise-wide investment strategy guidance across the Army program areas. The Charter for the IT Capital Planning and Investment Management process was approved by the Army CIO Executive Board in the fall of 2004. The process is further described and institutionalized in the Army capstone regulation on Army Information Management, with additional direction and guidance for Army Major Commands being included in another policy document. Also, recent requirements were established for reviews to be made by the Office of Secretary of Defense (OSD/DOD) Investment Review Boards and the Defense Business System Management Council of all IT systems with a total cost in excess of $1 million. The Army CIO will continue to maintain and clarify appropriate linkages, within the Army and across DOD, as part of the governance for the IT Capital Planning and Investment Management process.

In July 2005, the Secretary of the Army and Chief of Staff of the Army issued guidance establishing Capabilities-Based Information Technology Portfolio Governance in the Army. In implementing this guidance, the Army is implementing the Army Portfolio Management Solution (AMPS) to aid in the management of its IT investments. AMPS resides on an interactive database, and addresses specific IT Portfolio Management needs (e.g., Registry and Reporting; Alignment and Domain Certification; IT Investment Prioritization; and IT Budget Reporting which supports OMB300 development). The Capital Planning and Investment Management (CPIM) and Budget Reporting (CPIC IT) modules of APMS support the evaluation and prioritization of IT Investments in support of the Army's Portfolio Selection, Planning, Programming, and Budgeting. There are seven evaluative criteria areas for scoring each proposed investment, along with three other evaluation criteria areas for further analysis calculations. The evaluation criteria and metrics encompass costs, benefits, schedule, and risk elements as well as measures such as net benefits, net risks, and risk-adjusted return-on-investment. AMPS development began in May 2005 and the training/implementation on the application began in August 2005. As with any new application, there is a maturing process during which data validity and processes are refined and this is being supported within the Army through a quarterly Configuration Control Board which addresses emerging/refined requirements.

To improve the department's IT investment management processes, the Secretary of the Army has involved the department's IT investment management boards in controlling and evaluating IT investments, including the development and documentation of oversight processes for the review of ongoing IT systems. The overarching Army CIO Executive Board is informed by IT investment control and management practices conducted by IT program and acquisition managers. Required oversight and management practices are contained in DOD and Army acquisition regulations such as the DOD Instructions for the Operation of the Defense Acquisition System for Mission-Critical/Mission-Essential Information Systems. Also, recent requirements were established for reviews to be made by the Office of Secretary of Defense (OSD/DOD) Investment Review Boards and the Defense Business System Management Council of all IT systems with a total cost in excess of $1 million.

Interior's IT strategic management process has been documented in updated departmental plans. Specifically, Interior's revised fiscal year (FY) 2003-FY 2008 strategic plan outlines Interior's IT management processes (EA, IT security, CPIC, records management, etc.), and highlights process improvements ongoing and planned in these key IT management areas. In addition, Interior's E-Government Strategy Governance Framework, dated December 2003, (1) describes current IT management processes--including a graphic flowchart illustrating the current management decision-making and strategic planning processes, (2) critiques and gives an assessment of the current processes in line with the department's E-Government vision, and (3) defines further improvements and capabilities needed to achieve that vision. Also, by linking these plans and by describing the current and planned management processes, the department has also documented how its IT planning is integrated with organizational planning, budget, financial management, and human resource management.

To improve the department's IT strategic planning/performance measurement processes, the Department of Interior (DOI) FY 2006 budget request/annual performance plan includes information on the resources and time periods required to implement the information security program plan required by FISMA and major IT acquisitions contained in its capital asset plan that bear significantly on its performance goals. The 2006 budget annual performance plan includes a request of $883 million for overall IT spending and $12.8 million to support information security activities of the department. The budget plan notes that, in 2004, DOI accelerated its timeframes for completing the initial information security certification and accreditation of systems using government-wide standard processes, improved the content and delivery of IT security training, and put in place consistent IT security program reviews and testing programs.

Interior's E-Government Strategy and related E-Government Strategy Governance Framework, dated December 2003, have documented its process for establishing IT goals in support of the department's mission needs by providing a detailed description of the E-Government Governance model. The E-Government Strategy also identifies performance measures linked to the department's strategic plan and missions. Furthermore, in describing roles related to this process, the E-Government Strategy Governance Framework assigns responsibility and accountability for evaluating progress made against the established performance measures.

The Department of Interior (DOI) agreed with our recommendation and has taken action to address it. Specifically, in February 2006, the department CIO issued a directive providing guidance for preventing software piracy, and in June 2006, issued supplemental instructions on software piracy performance metrics. The instruction identified performance metrics associated with software piracy prevention controls; suggested that performance be documented through audits for compliance on DOI computers; and directed DOI Bureaus and offices to report the results of spot audits in their annual reports to the CIO. The DOI CIO provided examples of selected Bureau software piracy perfromance measures and reports prepared to fulfill these DOI requirements.

To improve the Department of Interior's IT strategic planning/performance measurement processes, Interior tracks actual-versus-expected performance for the department's enterprisewide IT performance measures in its IRM plan. DOI has been actively implementing Earned Value Management (EVM) and executive oversight of its IT investments. The DOI IT Strategic Plan and E-Gov Strategy, dated December 2003, identifies IT performance measures that map back to the department's revised fiscal year (FY) 2003-FY 2008 Strategic Plan and to OMB's Performance Reference Model. DOI has also included specific reporting requirements for performance measures in its CPIC Guide v2.0. This Guide requires DOI IT Project Managers to complete the DOI IT Quarterly Scorecard that provides project status on performance (cost and schedule), collaboration, and risks by assigning green, yellow, or red "stoplight" scores.

To improve the Department of Interior's IT strategic planning/performance measurement processes, the Interior has established an Enterprise Architecture Methodology for Business Transformation, which incorporates requirements for benchmarking. DOI has also begun executing a plan for implementing improvements in its IT investment management practices which involve benchmarking the department's IT management processes. To improve its investment management practices, DOI has entered into an agreement with the SAIC Center for Capital Planning and Investment Control to benchmark the department's IT management processes. The SAIC Center has been working with the department to provide it with independent benchmarking analyses of the department's progress in achieving the IT management processes delineated in the GAO Information Technology Investment Management (ITIM) capability-maturity model.

To improve the department's IT investment management processes, the Department of the Interior has established a policy requiring that proposed IT investments support work processes that have been simplified or redesigned to reduce costs and improve effectiveness. This policy is included in the IT Capital Planning and Investment Control Guide, Version 2.0 (February 2005). In addition, the policy is incorporated in DOI Enterprise Architecture principles for Interior modernization and business process transformation. Requirements for Business Process Reengineering are incorporated in DOI Enterprise Architecture documents, including the Conceptual Architecture and the Methodology for Business Transformation, is help the department improve mission performance through business transformation.

Interior has implemented this recommendation. Specifically, according to Interior's IT Investment Rating and Ranking Criteria, dated August 9, 2004, the department included cost and schedule in its project selection and prioritization criteria. These criteria, used to rate and rank IT investments in support of investment management decisionmaking, were included in the department's February 2005 Capital Planning and Investment Control guide.

To improve the Department of Interior IT investment management processes, Interior has established and is implementing a policy requiring modularized IT investments. This policy is included in the DOI IT Capital Planning and Investment Control (CPIC) Guide, Version 2.0 (February 2005) and the DOI Enterprise Architecture. The CPIC Guide requires that the capability to modularize an investment be assessed in defining the benefits for IT systems and comparing alternatives. IT investment decision criteria in the CPIC Guide also require the assessment of project implementation and scheduling risks based on whether IT projects adopt a modular approach that combines controlled systems development with rapid prototyping techniques. The DOI Enterprise Architecture includes a basic principle of "Modular and Adaptive Development." According to the department, this principle expands on the Clinger-Cohen Act requirement to build modular systems whose components can deliver usable business modules in a phased approach.

To improve the department's IT investment management processes, the Department of Interior has instituted and is implementing a requirement that corrective actions be undertaken, tracked, and reported to DOI IT investment management board for under-performing projects. Quarterly control reports are produced for each DOI major IT investment. These reports are reviewed by the DOI IT Investment Review Board and decisions regarding the disposition of these investments are made. Under-performing projects that are performing outside the allowable performance variance are put on monthly reporting and are required to complete a Corrective Actions Report. Specifically, the DOI IT Capital Planning and Investment Control (CPIC) Guide, Version 2.0 (February 2005) requires that if an IT project has a "yellow" (between -6 and -9 percent), "red" (less than 10 percent), or "blue" (greater than 10 percent) variance, or the variance category's status has changed (deteriorated) since the previous quarter, a Corrective Actions Report must accompany the Quarterly Scorecard.

To improve the department's IT investment management processes, the Department of Interior has implemented an evaluation process for IT investments. The DOI IT Capital Planning and Investment Control (CPIC) Guide, Version 2.0 (February 2005) requires that post-implementation reviews be conducted to validate expected benefits and costs and to document and disseminate lessons learned and the department conducts such reviews, usually either after a system has been in operation for about six months or immediately following investment termination. Interior also provided documentation to demonstrate that post-implementation reviews have been performed. To supplement the CPIC Guide, DOI has also issued a Best Management Practices for Conducting a Post Implementation Review, Version 1.2 (December 30, 2004).

The Department of the Navy (DON) has developed a documented process to measure progress against the department's enterprisewide IM/IT goals and assign roles and responsibilities for achieving these goals. The DON IM/IT Strategic Plan for FY 2006-2007 contains the latest version of the Navy's IM/IT vision, mission, governing principles, and goals that span the timeframe from FY 2006 through FY 2013; the plan specifies the strategies that will be implemented during the FY 2006-2007 period for each goal. The strategic plan also states that the Department will define plans of action and milestones and performance metrics for the goals and strategies in the plan, so the progress can be clearly measured. Navy officially launched its initiative to establish the DON IM and IT Performance Measurement Program in April 2006. Navy also provided a copy of its documented process for implementing its Performance Measurement Program to measure progress against the enterprisewide IM/IT goals of the Department.

To improve the department's IT strategic planning/performance measurement processes, the Department of the Navy developed an IT goal related to service delivery to the public. This goal was included in the FY 2005 Update to the DON IM/IT Strategic Plan.

As we recommended, Navy developed IT performance measures/indicators for each of the goals in its FY 2008-2009 Information Management & Information Technology (IM/IT) Strategic Plan to bridge the gap between strategic planning and results. These performance indicators provide a means for Navy to determine how well IT contributes to the effectiveness and efficiency of the department's operations in meeting its mission. For example, under its infrastructure management goal, Navy reports on a performance indicator for tracking its progress in reducing the number of legacy networks that are not part of its IT enterprise solution. Similarly, for its information management transformation goal, Navy developed an indicator for tracking the reduction of applications in its overall portfolio. With respect to addressing electronic government initiatives, Navy provides quarterly status updates to the Department of Defense (DoD) as part of DoD?s process for measuring and reporting progress towards e-gov objectives. Although Navy?s acquisition of commercially available software must be consistent with federal policy on computer software piracy, it was unable to provide information on performance measures developed, if any, to address agency compliance with such policy.

To improve the department's IT investment management processes, the Department of the Navy has included net risks and risk-adjusted return-on-investment in the department's IT project selection criteria. In accordance with Section 5122(c)(3) of the Clinger-Cohen Act (Public Law 104-106, August 8, 1996), the Department's requirement/capability, Planning/Programming/Budgeting/Evaluation, and acquisition processes require a risk analysis, risk mitigation plan and the calculation of ROI. A Clinger-Cohen Act compliance package is also required at each milestone and includes the requirement for an ROI calculation. In addition, the DON CIO issued guidance on the use of projected net risk-adjusted ROI for IT investments in February 2001.

To improve the department's IT investment management processes, the Department of the Navy has implemented a structured IT selection process that (1) uses selection criteria; (2) identifies and addresses possible IT investments and proposals that are conflicting, overlapping, strategically unlinked, or redundant; (3) prioritizes proposals; and (4) is integrated with budget, financial, and program management decisions. The structured IT selection process is described in the DON IT Capital Planning Guide and DON policy guidance. The DOD Defense Acquisition System and the Department of the Navy IT capital planning, acquisition, Planning/Programming/Budgeting/Evaluation (PPBE), and Functional Area Manager processes provide the foundation for the DON Capital Planning and Investment Control Process that complies with each of the elements of this recommendation. Also, the DON CIO has issued IT policy guidance on IT selection criteria, on fiscal year expenditures, and on DON Business IT System Pre-Certification and Workflow. This CIO guidance institutes processes to identify and address possible IT investments and proposals that are conflicting, overlapping, strategically unlinked or redundant. The Navy IT Capital Planning Guide and the DON CIO policy guidance also address prioritizing investments, and the policy guidance, the DON acquisition process, and the Business Management Modernization Program process described in the DON CIO Pre-Certification and Workflow Guidance integrate the IT selection process with budget, financial, and program management decisions.

To improve the department's IT investment management processes, the Department of the Navy involves all elements of the department's IT investment management board governance process in selecting, controlling, and evaluating IT investments. This governance process is described in the Navy IT Capital Planning Guide and CIO IT policy guidance. In addition, the Department of the Navy CIO issued in August 2005 DON Business IT System Pre-Certification and Workflow Guidance providing that the Office of the Secretary of Defense (DOD) Investment Review Board must review and certify all Department of the Navy activities acquiring, developing, or modifying business information technology systems that have a total cost in excess of $1 million. In addition, the August 2005 DON CIO Guidance provides that the DOD Defense Business System Modernization Committee must review and approve all Navy activities to acquire, develop, or modify such business information technology systems. These reviews, certifications, and approvals of IT systems must be completed prior to the obligation of funds for any IT system.

To improve the department's IT investment management processes, the Department of the Navy has documented the role, responsibility, and authority of its IT investment management boards, including work processes, alignment, and coordination of decision making among its various boards, and documented the processes for controlling and evaluating IT investments. The roles, responsibilities, authorities, and criteria for controlling and evaluating IT investments by the investment management review boards are described and accomplished by the processes detailed in the DON (1) Capital Planning Guide, (2) CIO IT policy guidance on expenditures and portfolio management, (3) CIO Guidance on DON Business IT System Pre-Certification and Workflow, (4) (and DOD) acquisition process, and (5) Functional Area Manager process.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, Treasury reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

To improve Treasury's IT strategic planning/performance measurement processes, the department implemented a mechanism for benchmarking its IT management processes, when appropriate. Specifically, the Department benchmarks its processes against other agencies using key measures that include cost, schedule, risk, and strategic alignment.

Treasury has implemented this recommendation by developing a Capital Planning and Investment Control Policy (CPIC) guide that includes elements such as identifying roles and responsibilities, including the role of the CIO, and identifying external or environmental factors that should be considered in the investment process.

The Department of the Treasury has implemented this recommendation by developing work processes and procedures specified in its Capital Planning and Investment Control Policy (CPIC) guide for its Executive Board and Technical Investment Review Board. Treasury's CPIC guide, for example, delineates who the key business executives are and how often funding decisions are made, and establishes that the Executive Board is the enterprise-level board with final decision-making responsibility.

The Department of the Treasury issued its Information Technology (IT) Capital Planning and Investment Control (CPIC) Policy Guide in October 2006. According to the guide, all Treasury IT investments must comply with the guide. In defining the CPIC process, the guide cites the need to follow Treasury's Information System Lifecycle (ISLC) which defines required functional technical requirements for IT investments and specifies use of COTS technology or supported reengineered work processes. By issuing its CPIC guide and specifying the use of Treasury's ISLC, the department has met the intent of our recommendation for establishing a policy requiring that proposed IT investments support work processes that have been simplified or redesigned and that makes maximum use of COTS software.

The Department of the Treasury issued its Information Technology (IT) Capital Planning and Investment Control (CPIC) Policy Guide in October 2006. The guide identifies the processes and activities necessary to ensure that Treasury's IT investments are well thought out, cost effective, and support missions and business goals of the organization. At the highest level, the CPIC process is a circular flow of Treasury's IT investments through the following four sequential phases including Pre-Select, and Select processes. Within the Select Process elements of practices 2.12 and 2.13 have been included. For example, Treasury's project selection criteria is aimed at providing a selection of technically and financially sound investments that are best aligned with the President's Management Agenda (PMA) and Treasury and Bureau business priorities. For example, Treasury's FY2008 selection process scored investments on how they support department and Bureau strategic objectives and are meeting cost, schedule, and performance goals. In addition, Treasury uses the process to help ensure there are no duplicative investments.

The Department of the Treasury issued its Information Technology (IT) Capital Planning and Investment Control (CPIC) Policy Guide in October 2006. According to the guide, all Treasury IT investments must comply with the guide. In defining the CPIC process, the guide cites the need to follow Treasury's Information System Lifecycle (ISLC) which defines a lifecycle model may be tailored to the unique needs of an information system project. In particular, the ISLC specifies that IT investment project managers need to consider the size, complexity, and scope of the information project when preparing the project plan. According to the ISLC, tasks and work products can sometimes be omitted as long as the resulting approach provides for the delivery of a quality system. By issuing its CPIC guide and specifying the use of Treasury's ISLC, the department has met the intent of our recommendation for establishing a policy on modularized IT investments.

As we recommended, Treasury issued its Information Technology (IT) Capital Planning and Investment Control (CPIC) Policy Guide in March 2006, which defines the control and evaluation processes and activities necessary to ensure that Treasury's IT investments are cost effective and support missions and business goals of the organization. The guide calls for quarterly Control Reviews which focus on ensuring that 1) projected benefits are being realized; 2) cost, schedule and performance goals are being met; 3) risks are minimized and managed; and 4) the investment continues to meet strategic needs. As defined, the guide addresses the Control process elements of practice 2.15, 2.16 2.17. In addition to establishing written policies and procedures for the investment board's management oversight of IT projects during the quarterly Control reviews, Treasury has developed instructions and tools to help it collect information required for the quarterly reviews. While Treasury's CPIC Policy Guide provides direction, the department is in the initial stages of implementing its control and evaluation processes and is working to define actions and measures necessary to institute the full use of these processes.

As we recommended, Treasury issued its Information Technology (IT) Capital Planning and Investment Control (CPIC) Policy Guide in March 2006, which defines the control and evaluation processes and activities necessary to ensure that Treasury's IT investments are cost effective and support missions and business goals of the organization. The guide calls for an annual evaluation phase, conducted in the first quarter of the fiscal year, to examine major investments that are in operation. Project Manager initiate a Post Implementation Review, in accordance with the Treasury CPIC calendar, to measure user satisfaction and the achievement of strategic goals, system performance, risk, cost, schedule, and performance. Lessons learned for both the investment and the CPIC process are also identified to assess and improve the CPIC process, as warranted. While Treasury's CPIC Policy Guide provides direction, the department is in the initial stages of implementing its control and evaluation processes and is working to define actions and measures necessary to institute the full use of these processes.

In the Department of Transportation's September 2004 CIO Capital Planning and Investment Control Implementation (CPIC) Guide, the department documented decision-making rules to help guide the departmental investment review board's oversight of IT investments during the control phase. The guide provides that after IT investments are selected, budgeted, and receive funding, their progress will be monitored in the CPIC Control phase. The Control phase helps ensure that IT investments are performing as expected. The process includes on-going efforts by DOT to first identify potential at risk programs, and then monitor, review, and correct, as necessary, their performance. This is accomplished by conducting quarterly control reviews. Departmental IRB control review objectives emphasize measurement of investment health in terms of performance, schedule, cost, risk and security; identify investments that are performing below expectations; and define and enforce corrective actions. The CPIC Guide identifies key stakeholders and participants in Transportation's IT CPIC process, and their salient roles and responsibilities during the Control Phase. The guide also describes decision rules during Pre-control review Activities, Step 1: Departmental CIO Portfolio and Trend Analysis, and Step 2: Determination of "at risk" programs. In addition, specific guidance for protocols and decision-rules is provided in the CPIC guide for each investment board.

The Department of Transportation (DOT) established an Earned Value Management policy in January 2008 to provide greater visibility into technical, cost, and schedule perferomance of its investments. In accordance with this policy, information necesary to provide an objective, quantifiable measurement of performance at defined periods is collected and reviewed to enhance risk mitigation. For example, quarterly reports are used by the department to track and monitor cost and schedule performance trends for investments and, based on these reports, conduct independent reviews and the follow-up necessary to help ensure that investment variances are caught, explained, and addressed.

The Department of Transportation has documented its IT strategic planning process in its FY 2004-2009 Information Resources Management Plan. For example, through its plan, Transportation has established a comprehensive set of IT management frameworks and processes to help ensure that the department achieves its strategies. These frameworks, which are described in the plan along with associated performance metrics, include IT Governance, IT Program Management, IT Capital Planning and Investment Control, Enterprise Architecture, and Security and Privacy. For example, through Transportation's IT governance structure, the department can better ensure IT management processes implemented by the department are conducted in a comprehensive and integrated manner. The plan provides an overview of this integrated framework and outlines approaches to facilitate better coordination, management, and visibility of IT investments across the department.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, Transportation reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

The Department of Transportation has established strategic goals in its FY 2004-2009 Information Resources Management Plan that address how IT can contribute to its program productivity. For example, Transportation includes the goals to (1) improve services to citizens by leveraging the Federal Enterprise Architecture and the Department's EA; (2) support improved mission performance by enhancing the contribution of information technologies to each DOT strategic goal; and (3) improve customer relationships by implementing a Department-wide, citizen-centered E-government strategy. Each goal lists associated activities and performance outcomes. For example, supporting the improvement of mission performance by enhancing the contribution of information technology to each DOT strategic goal includes activities such as developing IT portfolios for each DOT strategic goal and linking strategic goal portfolio analysis with DOT EA target business architecture.

In the Department of Transporation's (DOT) Information Resources Management (IRM) Strategic Plan (FY2007-FY2012), DOT established goals and measures that link to the Department's overall strategic plan. For example, DOT's Strategic Plan, dated October 2006, identifies the need to institutionalize and operationalize the Enterprise Architecture (EA) processes throughout the department to improve operational efficiency, information sharing and utilization of information resources in support of all DOT outcomes. In its IRM strategic plan, DOT identifies strategies, outcomes, and performance indicators for establishing EA as the authoritative decision tool for all DOT information technology investments. These included (1) the number of FY 2008 duplicate systems identified in the transition strategy and (2) the extent to which DOT received improved scores on external assessments.

In the department's September 2004 CIO Capital Planning and Investment Control (CPIC) Implementation Guide, Transportation documented the alignment and coordination of responsibilities of its IT investment management boards for decision making related to IT investments. The guide identifies the roles and responsibilities for the Departmental Investment Review Board (IRB), Departmental Architecture Review Board (ARB), and Operating Administration IRBs. For example, the guide provides that the Departmental IRB makes decisions regarding cross-cutting investments common to two or more DOT organizations, the ARB reviews all major IT investments for compliance with OMB Circular A-11 prior to Departmental Investment Review Board (IRB) review and approval, and the Operating Administration IRBs make IT investment decisions that are not under the purview of the Departmental IRB. Additionally, the CIO CPIC guide includes the charters of these IT investment management boards, which further outlines their alignment and coordination.

The Department of Transportation's Capital Planning and Investment Control (CPIC) Implementation Guide, dated September 2004, describes procedures designed to ensure that DOT's portfolio of IT investments adequately addresses it's overall mission and strategic objectives, Office of Management and Budget requirements, and is managed to achieve expected benefits in accordance with cost, schedule, technical, security, privacy, risk, enterprise architecture standards, and performance parameters. Consistent with our recommendation, DOT's CPIC guide requires that each IT investment has a documented rationale supporting a business need and that alternatives to pursuing investments are considered, including changing a business process in lieu of new/modified technology. In assessing the business case, the guide describes the process used to ensure that COTS products are considered as the project is designed and implemented in accordance with requirements outlined in OMB Circular A-11 and DOT?s IT Budget Guidance and use of .

In the Department of Transportation's September 2004 CIO Capital Planning and Investment Control Implementation (CPIC) Guide, the department established a policy requiring modularized IT investments. Specifically, as required in the CPIC policy guide, each IT investment must have a documented rationale supporting a business need. As part of this documented rationale, a key requirement is that modularity of the investment has been maximized to the extent practicable.

In Veterans Affairs'(VA) FY2007 Budget Submission/Annual Performance Plan, the department reported that the combined Office of Cyber and Information Security management, operational, and technical activities continue to function as an effective nucleus for the department-wide IT Security Program. Within VA's Budget Summary, the full amount for the department's Enterprise Cyber Security Program is cited in the Table depicting the proposed FY 2007 major IT initiatives of the Department, which includes salaries, training and travel. Also, the Budget Summary lists a performance goal assessed over time (FY2005-FY2007) focusing on the percentage of operational IT systems that have effective security controls. Current assessments are being done using a modified version of VA's Security Management and Reporting Tool (SMART), which is also used to collect the results of the department's annual FISMA security self-assessment survey. The enhanced tool will allow testing and evaluation of the effectiveness of deployed security controls.

The Department of Veterans' Affairs (VA) has documented three IT strategic goals and seven associated IT priorities in its Information & Technology Strategic Plan FY 2006-2011. VA also identified initiatives that are aligned to each priority and developed performance measures for these initiatives to measure their progress against the strategic goals. Further, VA identified baseline and target performance metrics for each performance measure. The department's Information & Technology Strategic Plan FY 2006-2011 states that VA's Office of Information & Technology managers and executives will be responsible for executing each of these initiatives and assessing VA's progress in achieving the target performance metrics.

The Department of Veterans Affairs (VA) identified strategic goals and related IT priorities in its FY 2006-2011 Information Technology Strategic Plan. For each IT priority, the plan aligns initiatives and associated performance measures to the priorities, and identifies performance targets for FY 2008 and FY 2011. VA also developed a performance accountability database to store and track these measures for each initiative. VA provided a summary of the VA's Department of IT Enterprise Strategy, Policy, Plans & Programs FY 2008 performance for its 24 key measures.

To improve the Department of Veterans Affairs IT investment management processes, the department has documented the alignment and coordination of responsibilities of the department's various IT investment management boards for decision making related to IT investments, including cross-cutting investments. The VA IT investment management and governance process and the boards/groups and individuals with roles and responsibilities in this process are described in the VA IT Portfolio Management Guide, which was issued September 16, 2004. The key IT investment management board for decision making is the VA Enterprise Information Board (EIB). The EIB is the (1) forum for deliberation and decisions about department IT priorities and expenditures needed to achieve mission and business requirements and (2) executive decision-making body for the department's IT portfolio management and capital planning and investment control process that provides for the selection, management, and evaluation of IT investments. The EIB implements the VA IT Portfolio Management Goals to (1) manage IT projects from an enterprise perspective investments, encouraging partnerships, eliminating duplicative and stovepipe projects, and balancing benefits against costs and risks; (2) undertake IT projects with the greatest return on investment; and (3) manage projects to achieve the desired results. The EIB, through its capacity to decide whether to recommend continuance, modification, or termination of projects, resolves conflicting, overlapping, redundant, strategically unlinked or non-aligned cross-cutting investment proposals.

To improve the Department of Veterans Affairs' IT investment management processes, the VA IT investment selection process includes mechanisms to identify possible conflicting, overlapping, strategically unlinked, or redundant proposals and prioritize the department's IT investments. These mechanisms center on the VA Enterprise Information Board (EIB), the executive decision-making body for the department's IT capital planning and investment control process. The EIB sets priorities among IT investments and determines IT investment expenditure levels. The EIB, through its capacity to decide whether to recommend continuance, modification, or termination of projects, resolves conflicting, overlapping, redundant, strategically unlinked or non-aligned cross-cutting investment proposals. Other mechanisms include the Department's annual budget formulation processes prior to EIB review, in which VA teams examine the entire VA IT Portfolio, and prepare and review Capital Asset Plans and Business Cases for each of the Department's major IT investments.

Consistent with our recommendation and to improve EPA's IT investment management processes, the Administrator of the EPA established a System Life Cycle Management policy in July 2005 that requires phased systems development. Management review and approval is required at each phase, and must be approved before a system moves into a new phase and/or receives incremental funding of each stage.

To improve the Environmental Protection Agency's (EPA) IT investment management processes, the Administrator of the Environmental Protection Agency (EPA) has implemented practices within the IT investment management control phase. For example, EPA?s System Lifecycle Management Procedure document, approved in June 2007, delineates the documentation, milestones, and reviews that support planning and management practices for IT systems, including those for the control phase. The lifecycle includes periodic, documented senior management-level reviews that determine whether systems continue within the lifecycle or are terminated. These reviews are used to ensure that projects are meeting stated objectives and are within cost and schedule. In addition, EPA?s CPIC Procedures provide guidance for updating business cases annually to determine whether the strengths, weaknesses, opportunities, and threats for the agency have changed; decide whether the project will continue to satisfy business requirements and customer needs; and analyze whether the investment is meeting cost and schedule targets and project milestones. EPA employs EVM techniques and reporting to ensure to support early awareness of potential cost and schedule risks, and reviews data that has been validated by independent sources to determine if investments are performing to expectations. Finally, according to EPA?s Earned Value Management Procedures (an addendum to the CPIC Procedures), major investments that have cost and schedule variances of 10% or more are required to develop a corrective action plan. These underperforming projects are reviewed at quarterly meetings of the information investment subcommittee. This board is presented with the status and corrective actions for the projects, and it decides whether or not to continue the investment based on this information. For example, a quarterly meeting summary of the EPA Quality Information Council Information Investment Subcommittee, documented corrective actions for investments being monitored and tracked for the desired outcomes over three quarters.

The Environmental Protection Agency (EPA) took action to address this recommendation. EPA's Enterprise Architecture Policy, dated November 2005, states specifically that all EPA information management and technology development, modernization, enhancement, and acquisitions shall conform with the Enterprise Architecture and comply with applicable Enterprise Architecture requirements of the Capital Planning and Investment Control (CPIC) and Agency budget process, as published in periodic procedures, technical standards, and guidelines. In FY2005, the EPA further aligned the Enterprise Architecture with the strategic planning process through integration with budgeting and acquisitions. For example, EPA mapped budget codes to the BRM sub-functions as line items in the FY2005 operating plan as executed in the Budget Automation System, which is the central agency system used to integrate strategic planning, annual planning, budgeting, and financial management. In addition, the BAS contains resources (such as dollars and full-time equivalents (FTE)), planning and performance data.

In the Environmental Protection Agency's (EPA) budget summary/performance plan for FY2007 under the IT/Data Management/Security program area, the agency identified the resources and the time periods required to implement the information security program plan required by FISMA. For example, the agency identified key milestones for implementing the program and related FTE increases required for meeting these objectives.

The Environmental Protection Agency's (EPA) 2006-2011 Strategic Plan identifies the goals and performance measures to be achieved by the agency, and the Strategic Information Plan, dated Sept 2007, is linked to the agency?s strategic plan. For example, EPA's Strategic Information Plan describes a number of goals focused on the Agency's strategic goal of making environmental information more accessible. A related goal described in the Strategic Information Plan, for example, entitled "invest in our people/human capital," identifies the need to recruit and retain employees with appropriate skills. Consistent with this goal, EPA's FY2008 Annual Performance Plan tracks actual versus expected percentage increases in targeted skill levels of employees as well as the average time (in days) required to hire new staff into the Agency.

To improve the Environmental Protection Agency's (EPA) IT strategic planning/performance measurement processes, the Environmental Protection Agency established an agencywide management process for driving higher levels of IT performance through the use of benchmarking. Benchmarking is the process of measuring and improving operational performance over time. According to EPA's 2006 Operational Analysis Guidance, IT investments select a process to be measured using both internal and external best practices and compare the actual performance to the best practices in terms of operating performance, customer satisfaction, quality, time, service, cost and other performance indicators.

To improve the Environmental Protection Agency's (EPA) IT investment management processes, the Environmental Protection Agency (EPA) developed Capital Planning and Investment Control (CPIC) procedures that include costs, benefits, risks, risk-adjusted return-on-investment, and qualitative criteria in the agency's project selection criteria. Specifically, EPA's July 2007 exhibit 300 scoring guidance for the 2009 budget year notes that an alternatives analysis should be performed that conforms to EPA's CPIC procedures. During EPA's select phase in the CPIC process, a business case is developed that includes a cost/benefit analysis and an alternatives analysis. Criteria for evaluating each alternative include costs, quantitative and qualitative benefits, risks, and risk-adjusted return on investment.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, GSA reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

In the CIO portion of GSA's Performance Management Tool (PMT), the agency developed a performance metric related to the effectiveness of controls to prevent software piracy. Specifically, GSA's PMT included an action item to annually monitor compliance with its software management policy and address reports and incidents of alleged violations of the policy, with emphasis on violation of the software piracy provisions. According to the PMT, based on this developed metric, GSA began monitoring that status of software piracy incidents in September 2005.

GSA uses the Performance Measurement Tool (PMT) to track performance against the measures associated with goals in the agency's IT strategic plan. Specifically, IT strategic goals in the IRM plan and their corresponding PMT goals, along with relevant performance measures are currently being tracked and briefed to the appropriate officials. Specifically, GSA officials reported that measures are reported monthly within the Office of the Chief Information Officer and are reviewed quarterly by the Administrator of GSA.

In accordance with the Office of Management and Budget's Information Technology Line of Business initiative, GSA is required to develop a five year information technology infrastructure optimization plan and provide annual agency progress reports. According to OMB guidance, the progress reports are being used to benchmark agency performance against private industry metrics. In addition, GSA's directive (CIO 2135.2A) states that the CIO Council proposes and monitors implementation of policies, programs, standards, performance measures, benchmarks, and strategies to ensure their consistency throughout the agency. GSA has also worked with an industry benchmarking organization to analyze and assess its current IT management and delivery processes. GSA provided documentation detailing their efforts in developing an enterprise IT performance management program comprised of several strategic and organizational objectives.

GSA included in the revised Information Technology Capital Planning and Investment Control Order CIO 2135.2 requirements that its IT investment boards track resolution of corrective actions for under-performing projects. The order directs the head of the Services or Staff Offices (SSO) of the GSA and to take corrective actions if variances exceeding negative ten percent are reported. The order directs project managers to report project management information on a monthly basis, to perform a monthly IT investment performance status reports for the CIO, and for DME projects to perform monthly analysis of earned value data and provides reports to the SSO Technical Review Board (TRB) of any negative variances. In cases of negative variance exceeding ten percent, project managers are to develop and implement corrective action plan and report results to the SSO Information Technology Resources Board (ITRB) and its CIO. Additionally, the project manager must provide monthly IT performance status reports for the CIO.

To improve the GSA's IT investment management processes, the Administrator of the GSA established a policy requiring modularized IT investments in the revised Information Technology Capital Planning and Investment Control Order CIO 2135.2. The Order instructs to agency to structure major acquisitions into useful segments with a narrow scope and brief duration.

GSA included in the revised Information Technology Capital Planning and Investment Control Order CIO 2135.2 clear decision-making rules for its IT investment boards, requirements that IT projects report on variances, and requirements that corrective actions are planned and taken if cost and the schedule of milestones show negative variances exceeding ten percent. The Information Technology Council is responsible for assessing the IT portfolio by reviewing information provided to the OCIO including cost and schedule variances and performance information. Additionally, the Information Technology Resources Board is responsible for making corrective action recommendations if the costs and schedule of milestones show negative variances exceeding ten percent. The Technical Review Board is responsible for making recommendations to the Service and Staff Office Information Technology Resources Board if negative variances occur in technical reviews that monitor IT investments using earned value management methods.

In FY 2008, GSA's Office of Chief Information issued new guidance on the agency's quarterly control review process that provides a mechanism for documenting corrective actions for the agency's major IT investments. The control guidance includes a set of criteria for corrective actions based upon factors, such as cost variance, schedule variance and performance measurement, security assessment, risk assessment and operational analysis. GSA's FY 2008 quarterly control review documented that the process is being used to track and report the resolution of corrective actions for under-performing IT projects. Further, GSA's Information Technology Council, chaired by GSA's CIO, has responsibility for making recommendations for corrective actions for major IT initiatives and reports to GSA's Business Systems Council, a subset of GSA's Executive Committee, that oversees impact of IT on achieving business objectives and business process changes.

NASA has documented its IT strategic management process in a March 2004 document titled, "Information Resources Management (IRM) Strategic Plan Process." This document provides references to NASA documents pertinent to the strategic management process, provides a graphic flowchart illustrating the IRM Strategic Plan process, and describes the actions required for each step of the flowchart.

In its September 2004 Information Resources Management Strategic Plan, NASA has established a strategic goal of maintaining a strong IT workforce through effective human capital management. Some of these goal's objectives are to recruit and retain a talented, diverse workforce and to develop and maintain competency in IT project management. Some of the strategies to meet this strategic goal include working with stakeholders to identify issues and provide guidance and direction related to human capital management of the IT workforce and to provide training and/or other developmental opportunities to ensure technical and project management skills are developed and maintained. Further, NASA has established two performance measures and targets related to this goal: (1) 100% of project managers validated according to Federal CIO Council guidelines and (2) one IT professional per quarter completing developmental assignments in the office of the NASA CIO.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, NASA reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

NASA has largely documented its process for developing information technology (IT) goals in support of agency needs. NASA?s September 2004 Procedural Requirements (NPR) 2800.1 specifies that NASA?s Senior Management Council conducts an annual review of, among other things, NASA?s mission, goals, objectives, and strategies. The NASA CIO participates in that review. The CIO is then responsible for taking the context of the NASA Strategic Plan and identifying IT strategic goals and objectives to support planned NASA missions and to promote cross-enterprise efficiencies. These IT goals and objectives form the foundation for the IT budget guidance issued as part of the annual budget planning process. The goals themselves are documented in the September 2006 NASA Information Resources Management (IRM) Strategic Plan and are mapped against elements from the NASA Strategic Plan. The IRM Strategic Plan also provides, for each IT strategic goal, a summary of performance for each against the previous IRM Strategic Plan. Finally, the March 2008 NASA Policy Directive 2800.1B assigns the NASA Chief Information Officer the responsibility for: (1) ensuring that IT enables NASA?s missions, goals, and objectives; and (2) developing, maintaining, and implementing the IRM Strategic Plan.

NASA has largely met the intent behind this recommendation by restricting users? access to software applications through several mechanisms. In August 2008, NASA officials told us that the agency had taken several actions. First, NASA has mandated the use of standard software loads for desktop and laptop machines provided by the agency?s Outsourcing Desktop Initiative for NASA (ODIN) initiative. Second, additional software needed above the standard load is readily provided by ODIN or by an authorized NASA software vendor, removing the incentive for users to install pirated software. Additionally, NASA is implementing the Federal Desktop Core Configuration (FDCC) as mandated by the Office of Management and Budget. The FDCC removes administrative rights for NASA users, thereby removing the users? ability to load software, including pirated software. Finally, NASA Centers and Headquarters scan user machines, and flag for investigation, software that is not authorized.

NASA's September 2004 Information Resources Management Plan tracks actual versus expected results for 10 performance measures, falling under 5 goals. These measures include a target of conducting initial application/service reviews at all NASA Centers by the 4th quarter of FY 2004 (all reviews were completed in the 3rd quarter of that fiscal year) and a goal of meeting 90% of milestones for corrective actions for protecting and securing NASA's information assets.

In November 2007, NASA published an IT Management Benchmarking Program Plan. This plan provides for the identification of target areas for improvement by either the Chief Information Officer (CIO), Deputy CIO, the Office of the CIO (OCIO) Strategy and Investment Board, or a project manager. Any area identified by a project manager must be approved by one of the other three positions, as appropriate. The owner of the target area must plan and implement an approved performance measurement process, including setting performance goals, identifying best practices, and creating a strategy and implementation plan to enact those best practices. Finally, progress towards performance goals shall be monitored by either the CIO, Deputy CIO, the OCIO Program Management Board, or the OCIO Operations Board. Finally, the reports that are produced by the benchmarking process will be maintained in a repository in order to support continuous organizational learning.

NASA has revised its information technology (IT) investment management process to explicitly link these processes to the agency?s enterprise architecture (EA). The March 2008 NASA Policy Directive (NPD) 2800.1B directly links NASA?s EA with IT planning and states that the EA and Information Resources Management Strategic Plan should be developed, maintained, and implemented together. Further, the three levels of NASA IT review boards all incorporate the EA in their processes, as defined in their charters. For example, at the highest level, the February 2008 charter for the Office of the CIO Strategy and Investment Board (SIB) uses updates to the EA as inputs and produces an annual review of NASA?s EA Executive Summary. Below the SIB, the January 2008 charter for the IT Program Management Board uses results of EA reviews and EA standards as inputs to its decisions. Finally, the March 2008 charter for the IT Management Board uses the results of EA reviews as an input and includes as members the Associate Chief Information Officer for Architecture & Infrastructure and the NASA Enterprise Architect.

As part of NASA's investment selection process, proposed investments are evaluated as to whether the initiative complies with the NASA enterprise architecture and whether the investment has ensured that improvements to existing information systems and the development of planned information systems do not unnecessarily duplicate IT capabilities within the same agency, from other agencies, or from the private sector. NASA has also developed an investment scoring model that assesses investments against a uniform set of evaluation criteria and thresholds. The model systematically scores investments using objective criteria and ranks and compares the investment against other investments. The criteria used are related to mission benefits, compliance with enterprise architecture, costs, and risks. NASA FY 2007 documentation demonstrates that investments are both assigned a final weighted score, components of which include: (1) Linkage to Strategic Plan; and (2) Mission Effectiveness; and (3) given a priority relative to the other proposed investments.

NASA has established and documented a hierarchy of three information technology (IT) management boards that control and evaluate agency IT investments. These boards oversee NASA's IT investment control and and evaluation processes. In February 2008, NASA chartered the Office of the Chief Information Officer Strategy and Investment Board (SIB) to make decisions regarding IT strategy and resultant policies, prioritization and approval of significant IT investments, and the NASA enterprise architecture. The charter for the SIB states that its governance structure applies to significant investments in IT infrastructure services (exceeding $500,000 per year) and applications (exceeding $1,000,000 per year). The SIB ensures alignment of NASA IT strategy, policy, and investments with the NASA mission; reviews investment proposals from the IT Program Management Board (PMB) and the IT Management Board (ITMB); and produces investment recommendations and priorities. The PMB, which was chartered in January 2008, provides executive oversight/decisions for application/infrastructure projects to ensure that investments approved by the SIB stay on track. In addition, the PMB ensures NASA-wide IT programs and projects are integrated across stakeholders and aligned with priorities; ensures that program/project outcomes achieve goals and objectives; reviews and approves IT programs/projects based on cost, schedule, risk management, and requirements; reviews IT program and project status reports; and makes recommendations for proram/project improvements. The ITMB, which was chartered in March 2008, makes decisions regarding performance, integration, and other issues pertaining to operational systems. According to its charter, it reviews operations metrics and other key performance indicators; reviews and recommends high-level scope and requirements, and associated changes; reviews IT operations and maintenance status reports; and reviews results of enterprise architecture reviews. The role of each board is further summarized in the March 2008 NASA Policy Directive (NPD) 2800.1B, which describes NASA policy on managing information technology, and which also specifies that the PMB and ITMB are subordinate to the SIB. NPD 2800.1B and all three board charters also identify the NASA policies this governance structure is to enforce, such as those policies pertaining to IT investment planning and control and NASA?s enterprise architecture.

National Science Foundations (NSF) IT strategic management processes are documented in an integrated suite of planning documents and through a variety of review and oversight activities. The suite of documents is intended to provide a clear "line of sight" from the enterprise architecture to the Transition Strategy to the NSF Capital Planning and Investment Control (CPIC) process and IT investment portfolio. The key documents outlining these processes include the NSF: (1) the Strategic Plan; (2) the Information Resource Plan; and (3) Technology Governance Framework. Linkages are also provided to other NSF strategic management processes, such as financial, human capital, and business process reengineering.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, The National Science Foundation reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

The NSF Strategic Plan and Performance and Accountability Report outline how information technology is linked to agency strategic objectives, and the NSF Technology Governance Framework establishes and defines the structures, roles, responsibilities, and schedules of various agency committees, working groups, and teams for realizing NSF IT goals.

The National Science Foundation (NSF) reported that it has implemented additional IT security and privacy polices and purchased technical solutions, including software scanning tools, to ensure the security of agency IT infrastructure. According to NSF, the enhanced scanning program includes performance measures to test configuration requirements, detect malware, and to enhance intrusion controls. NSF reported to OMB in June 2008 that it was 100% compliant with the Federal Desktop Core Configuration (FDCC)--FDCC removes administrative rights for users, which prevents them from installing all but a few limited types of software. As evidence, the agency produced a tracking sheet that it used to track its progress progress toward compliance with FDCC.

As we recommended, the National Science Foundation (NSF) utilizes benchmarking practices to assess and improve its IT strategic planning and management processes. For example, NSF reported that its FY06 agency-wide Business Analysis activity compared key NSF activities (including IT management) to best practices in comparable public and private sector organizations. In addition, NSF's new Capital Planning and Investment Control (CPIC) Policy and related CPIC guidance, published in May 2005 and revised in February 2007, were based in part on other agencies' best practices in capital planning and investment management. Further, NSF's Technology Governance Framework, dated May 22, 2005, sets forth a process for using the Software Engineering Institute's Capability Maturity Model and a method for assessing NSF's change management process. NSF has also benchmarked the agency's IT strategic planning, performance measurement, and IT management processes using federal improvement initiatives such as the President's Management Agenda (PMA).

The National Science Foundation (NSF) Technology Governance Framework, dated May 2005, includes agency Capital Planning and Investment Control (CPIC) and Enterprise Architecture (EA) guidance. In 2007, NSF issued its CPIC guide as a separate document. As we recommended, NSF's framework describes the relationship between the CPIC process and other organizational plans and processes, such as EA (EA management and planning processes); IT management structures (organizational committees and working groups structures and processes); and performance and risk (organizational processes for managing and mitigating risks). The framework also identifies external factors that influence NSF's CPIC processes. For example, the framework describes how NSF tracks statutory and regulatory mandates to manage and respond to such mandates that are related to information technology (IT) and NSF's IT investments.

The National Science Foundation (NSF) defined and documented its post implementation review (PIR) processes as part of the Capital Planning and Investment Control (CPIC) guidance in the agency's Technology Governance Framenwork, dated May 2005. Consistent with our recommendation, NSF's guidance defined the time periods and methodology for conducting PIRs to determine benefits and costs, such as the impact on the customer, the ability to deliver IT performance measures, and the ability to meet baseline goals. In addition, NSF's guidance describes the use of the PIRs to leverage lessons learned for improving agency CPIC processes.

NRC generally agreed with our recommendation. In its NRC Management Directive 2.8 -"Program Management Methodology", issued June 2007, NRC documented its process for integrating its requirements for CPIC, enterprise architecture, and system development life cycle methodology into a single directive. The directive also calls for a plan to ensure that the budget implications for IT investments, such as reprioritizing resources or requesting additional funding, are considered in the budget planning process. In addition, NRC's IT/IM Strategic Plan for FY 2008 to 2012 defines strategies and performance measures for supporting NRC's IT Human Capital Goal for increasing staff awareness, proficiency, and innovation in applying IT/IM tools and services to strengthen individual and organizational performance. NRC's executive investment management charters for its Information Technology Senior Advisory Council and Information Technology Business Council outline the processes, roles and responsibilities for the justification and approval of IT investments by the executive councils.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, the NRC reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

NRC generally agreed with our recommendation. In its IT/IM Strategic Plan for Fiscal Years 2008 through 2012, NRC documented its enterprise-wide IT goals. In addition, its new management directive 2.8 "Program Management Methodology" NRC has documented a single reference that provides a process assigning roles and responsibilities for the approval of IT investments (programs and projects). This directive is further supported by two executive investment review board charters. The charters describe the processes, roles and responsibilities for the two boards (Information Technology Senior Advisory Council (ITSAC) and Information Technology Business Council(ITBC)) responsible for justifying and approving NRC's IT investments. More specifically, the charter for ITSAC states that NRC's executive review body is accountable for NRC's IT/IM strategic direction, which is stated as strategies and goals in NRC's IT/IM strategic plan. The board is chaired by the CIO and voting members must be office directors or above.

The Nuclear Regulatory Commission (NRC) has documented software piracy performance measures in its FY 2008 Operating Plan on Internal Controls - FY08 1st Quarter. The Office of Information Services (OIS) is responsible for tracking the performance measures related to the effectiveness of controls to prevent software piracy. The data for the plan is captured and generated by NRC's internal tracking system "Sharepoint", which capture performance metrics. Specifically, starting in the 1st quarter of 2008, NRC started tracking the following three software piracy performance measures: (1) user guidance on compliance with Executive Order (E.O) 13103, (2) user signatures acknowledging their compliance with E.O 13103, and (3) procedures for monitoring compliance with E.O. 13103.

NRC published its initial set IT/IM performance measures in its FY 2007 Green Book. In a summary report - Fiscal Year 2007 Information Management Performance Review - issued by the CIO's office in December 21, 2007, NRC assessed and discussed the IT performance FY 2007 Green Book performance measures. The review summarizes the actual versus expected (targeted) performance for NRC's IT strategic goals and discusses future performance recommendations for FY 2008-2010. NRC also defined the enterprisewide strategic goals that it would use to help guide its IT/IM activities. The strategic goals are: (1)information, (2)IT applications, (3) IT security and (4) IT infrastructure. The plan defines the set of strategies, as well as a set of related performance measures that it is using to assess the agency's progress.

NRC?s Management Directive (MD) 2.8 - Program Management Methodology (PMM) - was approved on June 19, 2007. The MD 2.8 directive integrates several stove-pipe directives into one. In its MD 2.8 directive and IT/IM Strategic Plan, NRC described the relationship between the investment management process, its other organizational plans, the department?s enterprise architecture and the external factors that influence its CPIC process as GAO recommended. For example, the PMM states the MD 2.8 will serve as a single reference for addressing all life cycle aspects of IT investment management, including CPIC and enterprise architecture (EA) activities, system development, use of the NRC production operating environment (POE), operation support, and eventual retirement of systems. The directive also addresses the need to ensure that NRC?s planning and budgeting process for IT is integrated with NRC?s overall planning, budgeting and performance management process.

The Nuclear Regulatory Commission (NRC) agreed with our recommendation. NRC has instituted an IT governance framework (process) for its investment boards. The framework is documented in a June 2007 Management Directive. The directive lays out the initial processes and procedures needed by NRC's executive boards to plan, select, manage and evaluate NRC's IT investments in accordance with Federal statutes and regulations. In addition, the agency recently issued governance charters for its two executive investment boards The charters describe the processes, roles and investment management responsibilities for the two boards - Information Technology Senior Advisory Council (ITSAC) and Information Technology Business Council.(ITBC).

NRC?s Office of Information Services (OIS) developed an information repository called the NRC System Inventory Control Database (NSICD). NSICD captures information on all of NRC?s Information Technology (IT) investments and consolidates information that previously resided on various other NRC databases. The NSICD system captures IT investment information including information on enterprise architecture, privacy, security, records management and other general information in addition to information related to the CPIC process. In addition, OIS issued a policy (OIS-9000B-0002 - Entering New System Inventory Data in the NSICD) that documents the procedures for entering information into NSICD.

The Nuclear Regulatory Commission (NRC) updated its Management Directive 2.2 on Capital Planning and Investment Control (CPIC) in January 2004, establishing a structured IT investment management selection process that includes project selection criteria based on a three-tier model. In addition, to improve the department?s IT investment management selection processes, NRC implemented a scoring model and develops a prioritized list of IT investments as part of its selection process. The updated scoring model NRC uses is the ?Decision Lens Tool? which scores and prioritizes investments. Alongside the 2008-2012 IT /IM Strategic Plan, the tool helps guide the selection of IT investments. The IT/IM plan provides guidance and direction to the CPIC review bodies involved in the selection and prioritization of new major IT investments in the IT Capital Plan. These review bodies can also use it as a basis for approving the continuation of funding for existing major investments in the IT Capital Plan through the control and evaluation phases in the lifecycle of these investments.

As we recommended, the Nuclear Regulatory Commission (NRC) issued its Management Directive 2.8-Project Management Methodology (PMM) in June 2007 that defines the process to facilitate the control and evaluation of IT investments throughout the life cycle. Treasury's Information Technology Business Council is responsible for monitoring these IT investments, which includes periodic reviews to track IT projects to ensure they meet stated objectives and are within cost and schedule. Use of Earned Value techniques provide visibility into the actual progress of IT projects, which supports early awareness of potential risks. IT investments are further required to undergo independent project reviews. While the PMM provides direction, NRC documentation indicated that the Commission is currently in the initial stages of implementing the control and evaluation processes and is working to define actions and measures necessary to institute the full use of these processes.

Consistent with this recommendation, the Office of E-Gov and OIRA have taken numerous steps through which agencies have been informed of key IRM strategic planning requirements, covering areas such as collection, dissemination, security, privacy, and systems management coupled with goals, strategies and performance measures. For example, the CIO Council, working in cooperation with OMB, developed a Strategic Plan that established a vision and governmentwide goals for addressing e-government, security, and IT skills and resources, as well as strategies for reaching the goals. In addition, OMB's March 2005 Federal Enterprise Architecture Action (FEA) Plan prepared by OMB's Office of E-Gov consists of a set of interrelated reference models for facilitating cross-agency analysis and the identification of duplicative IT investments, gaps and opportunities for collaboration within and across agencies on a range of IRM activities. The FEA also includes a Records Management Profile that provides a strategy for agencies to establish appropriate management controls.

During the past year, OPM established policy requiring that IT initiatives and systems be implemented in a phased or modular manner with scheduled reviews of performance measures to assess investment effectiveness. This policy is documented in the new Charter for the OPM IT Investment Review Board and in OPM's web-based Systems Development Life Cycle software requirement that the agency use, to the maximum extent possible, modular (incremental) contracting for the acquisition of any major information technology system.

OPM has established requirements that corrective actions be undertaken, tracked, and reported to the investment management board for under-performing information technology projects. OPM mechanisms for information technology portfolio monitoring and control are specified in the OPM IT Strategic Plan and in the OPM Earned Value Management Policy.

The Office of Personnel Management (OPM), in its Congressional Budget Justification and Annual Performance Plan, reported $15.2 million as the expected full cost in FY 2005 of providing a reliable and secure computing environment for OPM. To achieve a secure computing environment, OPM's Annual Performance Plan states that the office provides information services, network support, infrastructure, and enterprise-wide IT systems and has computer security performance indicators and targets, including one requiring that computer security awareness training be completed annually by 100 percent of OPM staff and contractors with access to the local area network. The OPM Performance and Accountability Report for 2004 also states that OPM enforces a 90-day complex password replacement rule for all system users to ensure information security. The IT Strategic Plan for FY 2004-FY 2009 reported that OPM fully implemented and institutionalized its agency-wide information security program at an approximate cost of $20 million.

OPM has instituted a Software Piracy Prevention Program and has developed performance measures to track the agency's effectiveness in preventing software piracy in both its mainframe and network environments. OPM's piracy prevention effectiveness is measured on a quarterly basis. The quarterly performance measurement process began in February 2005, following an assessment completed in the fall of 2004 of OPM's software piracy prevention policies and procedures.

OPM included in its May 2005 IT Strategic Plan a Balanced Scorecard of performance measures for tracking actual-vs.-expected performance for the agency's enterprise-wide IT objectives. It also included tables reporting actual results on the agency's key IT performance indicators for FY 2003 and FY 2004 and key accomplishments on its planned IT objectives for FY 2004.

OPM has defined a benchmarking process in its IT Strategic Plan and has established a goal to conduct at least one major IT benchmarking effort each fiscal year. Areas for benchmarking include general/administration organization performance, desktop/helpdesk, telecom/network, and investment management/enterprise architecture. Most of the benchmarks specified in the Strategic Plan for these areas involve working with OMB to obtain similar information for other agencies and/or reviewing independent research for private sector figures. During the past year, OPM participated in an academic Federal Government IT Benchmarking Survey that benchmarked OPM IT relative to the IT organizations at the Fortune 1000 and other Federal agencies that completed the survey. In addition, OPM acquired and has been using a key benchmarking report related to helpdesk and network support for government and private industry.

OPM has developed work processes and procedures for the agency's investment management board, including establishing criteria for defining major systems and documenting a process for handling cross-functional investments. The work processes and procedures are documented in the May 2005 OPM Information Technology (IT) Strategic Plan for FY 2004-FY 2009. In FY 2005, OPM took steps to strengthen its investment review process by defining a more structured and active role for its Investment Review Board. A formal charter was developed for the Board, which reflects responsibilities in selecting, controlling and evaluating IT investments to help ensure that major IT investments are closely tied to the mission objectives of the agency, cost-effective, appropriately integrated, and delivered according to schedule and within budget. The Charter and Strategic Plan document the criteria for defining major systems, and the Board's purpose, roles and responsibilities, procedures, and processes for assuring that the major agency systems are appropriated integrated and consolidated.

In response to GAO's recommendation, OPM now requires that proposed IT investments support work processes that have been simplified or redesigned to reduce costs and improve effectiveness and that make maximum use of COTS software. This requirement is documented in the office's new Charter for the OPM IT Investment Review Board, the IT Strategic Plan Consolidation Questionnaire Template for Each Associate Director Office for Use in Preparing for the Investment Review Process, and OPM's web-based Systems Development Life Cycle software requirements. Specifically, OPM requires that (1) IT project requirement analyses include a review of work processes for simplification or re-engineering/redesign and (2) COTS software be used as the first choice of software development efforts.

As we recommended, the Small Business Administration (SBA) documented its IT strategic management process in the agency's FY 2005-2009 IT Strategic Plan, IT Investment Management Overview, Business Technology Investment Council Charter, and Technology Review Board Charter. Specifically, the Strategic Plan and these other dcouments describe SBA's IT strategic planning process by defining the agency's (1) IT Strategy and Business Alignment; (2) IT Organization and Skills; (3) IT Management and Governance; and (4) Technology and Architecture.

In its annual FISMA report to the Congress, in accordance with OMB M-07-19, the agency reported on both the resources and time periods associated with actions that are necessary to implement the information security program plan required by FISMA.

To improve the agency's IT strategic planning/performance measurement processes, the Small Business Administration (SBA) documented its process for developing IT goals in support of agency needs as we recommended. As reported in its 2007 to 2011 IT strategic plan, for example, SBA described a process that consisted of OCIO facilitated sessions with 13 SBA program offices. The results of these sessions were incorporated into an IT Strategic Roadmap report, which was used to develop the IT Strategic Plan. Each of the goals laid out in the IT Strategic Plan is linked to agency initiatives documented in SBA?s agencywide strategic plan. The IT Strategic Plan also describes objectives with which to measure progress toward the IT goals. Based on the goals and objectives, the OCIO is responsible for developing an operating plan with performance indicators, milestones, and key dates, and a corresponding scorecard to report and monitor progress. According to SBA, IT initiatives are assigned to owners of logical SBA Lines of Business (LOB) as they are identified in the conceptual phase for implementation.

As we recommended, the Small Business Administration (SBA) has developed performance measures for information technology (IT) goals, including how IT contributes to program productivity, efficiency, effectiveness, and the overall performance of its IT programs, and has tracked performance for these measures. For example, one of SBA's long-term objectives is to simplify the interaction between small businesses and the Federal Government through the use of the Internet and IT. Key measures identified were hours saved and customer satisfaction. In SBA's 2007 performance report, for hours saved, the agency reported actual performance of 3.25 million against its goal of 4.37 million. For customer satisfaction, SBA reported actual performance of 70% against its goal of 75%. Another of SBA's long-term objective is to manage information and related technology effectively and securely through SBA leveraging data and systems to support program execution and promote cost efficiency. Goals included continuing the enterprise architecture by aligning programs and systems; developing standard IT portfolio selection, control, and evaluation processes and performance metrics and using this information to gauge the progress of investments and their contribution to program outcomes by FY 2008; and achieving efficiencies of business processes and cost reductions by leveraging common E-Government solutions and technologies. SBA's Performance and Accountability Report tracks and reports progress toward these goals. SBA also developed and monitored quantitative measures for IT systems availability and unauthorized network or data breaches.

To improve the agency's information technology (IT) strategic planning/performance measurement processes, the Small Business Administration (SBA) developed a mechanism for benchmarking the agency's IT management processes as we recommended. For example, as described in the agency's FY 2009 Performance Budget and FY 2007 Performance Plan, SBA recognized that key processes, such as IT governance, continuously evolve offering opportunities to improve and mature related policies and practices in ways that meet both the agency's program management needs and federal standards established by statute and OMB policies. Therefore, SBA developed a scorecard for use in improving enterprise architecture, capital planning, program management, and IT strategic planning. The scorecard was based on mandates within applicable laws, including the Clinger-Cohen Act, E-Government Act, and Paperwork Reduction Act, and related OMB guidance, including circulars A-11, A-130, and A-94. SBA then analyzed its existing IT management processes against the scorecard measurements in both 2005 and 2007, stated goals for the measurements for 2008, and developed a "future state" of IT management based on the scorecard results.

According to the Small Business Administration's (SBA) Entrprise Architecture Program Policies and Procedures, SBA's Business Technology Investment Council is responsible for approving the SBA Enterprise Architecture, reviewing proposed IT investments, and making final investment funding recommendations to the Administrator. SBA's Business Technology Investment Committee (BTIC) accepts program and project proposals assessed by the Technology Review Board (TRB) and the Business Technology Investment Advisory Council (BTIAC). The BTIC receives recommendations from both the TRB and BTIAC before making the final determination concerning program/project fit within the overall budgetary and funding goals for the enterprise.

The Small Business Administration (SBA) EA Blueprint document, dated February 2007, provides management policies regarding IT system development design and the need to emphasize simplification and process re-engineering principles in line with our recommendation. For example, the document states that business process redesign should drive the process of decomposing a business process into well-defined functional units that map to responsible personnel at particular geographic locations.

SSA generally agreed with GAO's recommendation. In its revised FY 2005 Annual Performance Plan, SSA detailed its information security program, including information about the resources and time periods required for implementation as required by FISMA.

According to the the The Social Security Agency's (SSA) Capital Planning and Investment Control Guide, dated September 2006, benchmarking is performed, when appropriate, where comparable processes in the public or private sectors exist to identify best practices and compare the performance of Agency processes with those of comparable organizations. According to the agency's Associate Chief Information Officer, the agency looks outside to provide state-of-the-art best practices when internal project teams may not have the needed experience. SSA's Associate Chief Information Officer provided examples of Gartner, Forrester, and the Corporate Executive Board studies it uses to provide best practices benchmarks. In addition, SSA participates in communities of practice related to information technology investment management more broadly.

SSA generally agreed with GAO's recommendation. In October 2004, SSA revised its Information Technology Capital Planning and Investment Control (CPIC) guide to require modularized IT investments. Specifically, according to the revised CPIC requirements, "IT investments will be implemented in a modularized manner."

The Social Security Administration (SSA) generally agreed with GAO's recommendations. In its September 2006 CPIC guide and also in its December 2007 Information Technology (IT) Planning Training Package, SSA has documented the role, responsibility and authority of the stakeholders involved in the IT investment management process. These stakeholders are the Information Technology Advisory Board (ITAB), Deputy Commissioners for each of the SSA portfolios, Deputy Commissioner for Systems(DCS), portfolio team support staff (PTSS) and IT Planning Executives (ITPE). This guidance clearly distinguishes between all these organizations, establishes a hierarchy among them, and discusses the various decision-making responsibilities of each. Further, the guidance also documents the Control and Evaluation phases of its CPIC process, including requirements for monitoring the performance of ongoing investments and the requirement for conducting post-implementation reviews of IT investments that have been completed.

SSA generally agreed with GAO's recommendation. In its Capital Planning and Investment Control (CPIC) Process Guide, dated September 11, 2006, the Social Security Administration requires the documentation of a corrective action plan for IT investments that undergo significant cost, schedule, or performance problems. If an IT investment experiences significant problems, the CIO can require an in-process review to determine the reason for the problems, explore potential corrective actions, and provide input for decision making on whether to continue, modify, or cancel the project. As a part of the in-process review, the project manager must provide documentation of planned corrective actions to remedy the problems encountered. Additionally, IT investment profiles are used by management to review investments and can contain corrective action information.

In its revised May 2005 Capital Planning and Investment Control policy, USAID expanded its policy to identify and describe specific work processes and procedures for its IT investment board. For example, with respect to IT selection and control activities, the policy describes the process by which (1) IT new investments are selected and (2) ongoing investments are reviewed on a quarterly basis.

USAID's May 2005 Capital Planning and Investment Control policy requires that IT investments must be aligned with the Federal Enterprise Architecture and must be consistent with the joint State/USAID Enterprise Architecture as soon as it becomes available.

In May 2005, USAID revised its Capital Planning and Investment Control policy to require that investments support work processes that have been simplified or otherwise redesigned to reduce costs, improve effectiveness, and make maximum use of commercial, off-the-shelf (COTS) technology.

In May 2005, USAID revised its Capital Planning and Investment Control decision criteria for the selection of IT investments to include calculating Return-On-Investment (ROI), which reflects risk factors such as the project's technical complexity, agency management complexity, the likelihood of cost overruns, and the consequences of under- or non-performance. Qualitative criteria that can be used to compare and prioritize investments is also included such as identifying measurable and sustainable efficiency gains.

The Agency for International Development's (USAID) Information Technology Capital Planning and Investment Control directive (ADS 577) dated 6/6/2005, sets forth steps and decision criteria to identify whether proposed IT investments are linked to the agency's missions and goals, and are integrated with other systems supporting the workflow of the business function. By doing so, USAID can identify possible conflicting, overlapping, strategically unlinked, or redundant IT investment proposals within its Capital Planning and Investment Control process.

In May 2005, USAID revised its Capital Planning and Investment Control policy to address this recommendation. Specifically, the policy requires that IT investments must be implemented in phased, successive chunks as narrow in scope and brief in duration as practicable, each of which solves a specific part of an overall mission problem and delivers a measurable net benefit independent of future chunks.

In May 2005, USAID established procedures to guide decisions on IT investments as part of the required control phase quarterly review process. Specifically, the guidelines set forth decision-rules and steps for tracking and reporting progress of investments at major milestones or at points where deviations may have occurred. These procedures help USAID decide when to continue, modify, or cancel investments.

In its revised May 2005 Capital Planning and Investment Control policy, the Agency for International Development (USAID) included processes and steps for identifying investments that exceed the baseline investment cost and schedule by 10 percent or more. In August 2006, USAID issued the Program-Funded Information Technology (IT) Reviews Directive (ADS 548) which provides the authority, policy directives, and required procedures for the mandated Independent Verification and Validation (IV&V) review of USAID's program-funded IT investments with a threshold of $100,000 or more over the full IT project life cycle up to five (5) years. Full IT life cycle costs include all direct and indirect costs for planning, procuring, operating and maintaining, and disposing of the IT components. When performed in parallel with the IT project life cycle, IV&V provides for the early detection and identification of risk elements within a program. The Project Manager is then able to take action to mitigate risks early in the project life cycle.

In May 2005, USAID revised its Capital Planning and Investment Control policy to require the tracking and reporting of corrective actions for under-performing projects. Specifically, USAID's policy sets forth procedures for the preparation of quarterly reports that track the status of under-performing investments and identify problems contributing to their under-performance, give recommended corrective actions, and provide a summary of the implications to the project's success and business objectives. Doing so gives USAID executives the information they need to more quickly intervene and decide whether to continue, modify, or cancel an investment.

USAID and the Department of State identified in their FY 2006 Performance Summary the resources and time periods necessary to implement an information security management program required by the Federal Information Security Management Act (FISMA). Specifically, under Goal 2--MODERNIZED, SECURE, AND HIGH QUALITY INFORMATION TECHNOLOGY MANAGEMENT AND INFRASTRUCTURE THAT MEET CRITICAL BUSINESS REQUIREMENTS, the performance summary lists FY 2006 and FY 2005 targets and specific implementation activities associated with these targets to complete, such as expanding training and certification of employees and enforcing requirements for annual information security self-assessments.

In its revised May 2005 Capital Planning and Investment Control policy, USAID requires, where comparable processes and organizations in the public and private sectors exist, the use of quantitative benchmarking of agency process performance against such processes in terms of cost, speed, productivity, and quality of outputs and outcomes. To meet its policy requirement, USAID has implemented the use of the Control Objectives for Information and related Technology (COBIT) methodology.