Information Technology:

Agencies Need to Improve the Accuracy and Reliability of Investment Information

GAO-06-250: Published: Jan 12, 2006. Publicly Released: Feb 13, 2006.

Additional Materials:

Contact:

David A. Powner
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Each year, agencies submit to the Office of Management and Budget (OMB) a Capital Asset Plan and Business Case--the exhibit 300--to justify each request for a major information technology (IT) investment. The exhibit's content should reflect controls that agencies have established to ensure good project management, as well as showing that they have defined cost, schedule, and performance goals. It is thus a tool to help OMB and agencies identify and correct poorly planned or performing investments. In its budget and oversight role, OMB relies on the accuracy and completeness of this information. GAO was asked to determine the extent to which selected agencies have underlying support for the information in their fiscal year 2006 exhibit 300s. From five major departments having over $1 billion in IT expenditures in that year, GAO chose for analysis 29 exhibits for projects that supported a cross section of federal activities.

Underlying support was often inadequate for information provided in the exhibit 300s reviewed. Three general types of weaknesses were evident. All exhibit 300s had documentation weaknesses. Documentation either did not exist or did not fully agree with specific areas of the exhibit 300. For example, both these problems occurred in relation to calculations of financial benefits for most investments. In addition, for 23 of the 29 investments, information on performance goals and measures was not supported by explanations of how agencies had initially measured their baseline levels of performance (from which they determine progress) or how they determined the actual progress reported in the exhibit 300. Agencies did not always demonstrate that they complied with federal or departmental requirements or policies with regard to management and reporting processes. For example, 21 investments were required to use a specific management system as the basis for the cost, schedule, and performance information in the exhibit 300, but only 6 did so following OMB-required standards. Also, none had cost analyses that fully complied with OMB requirements for cost-benefit and cost-effectiveness analyses. In contrast, most investments did demonstrate compliance with information security planning and training requirements. In sections that required actual cost data, these data were unreliable because they were not derived from cost-accounting systems with adequate controls. In the absence of such systems, agencies generally derived cost information from ad hoc processes. Officials from the five agencies (the Departments of Agriculture, Commerce, Energy, Transportation, and the Treasury) attributed these shortcomings in support to lack of understanding of a requirement or how to respond to it. Agency officials mentioned in particular insufficient guidance or training, as well as lack of familiarity with particular requirements. The weaknesses in the 29 exhibit 300s raise questions regarding the sufficiency of the business cases for these major investments and the quality of the projects' management. Without adequate support in key areas, OMB and agency executives may be depending on unreliable information to make critical decisions on IT projects, thus putting at risk millions of dollars. Further, although the 29 examples cannot be directly projected to the over one thousand business cases developed each year across the federal government, the results suggest that the underlying causes for the weaknesses identified need attention. These weaknesses and their causes are also consistent with problems in project and investment management that are pervasive governmentwide, including at such agencies as the Departments of Defense, Health and Human Services, and Homeland Security, as documented in reports by GAO and others.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In January 2006, we reported that underlying support was often inadequate for information in agency IT budget request business cases (exhibit 300s) provided to the Office of Management and Budget (OMB). As a result, we recommended that OMB develop more explicit guidance for sections of the exhibit 300 that cause confusion, including addressing the weaknesses we identified. Consistent with our recommendation, in June 2006, OMB modified exhibit 300 requirements and provided more explicit guidance for key sections. On key areas that we addressed as having weaknesses, OMB clarified guidance on earned value management reporting, and added guidance on operational analysis, among other things. Establishing a new exhibit 300 with clearer guidance should help to improve the quality of the information reported in agency business cases.

    Recommendation: To help ensure that agency personnel completing exhibit 300s better understand their responsibilities, the Director of OMB should provide for training of agency personnel responsible for completing exhibit 300s. This training should go beyond a description of changes from prior year's guidance and include working through examples for a variety of investments. In developing the training, OMB should consult with agencies to identify deficiencies that the training should address.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  2. Status: Closed - Implemented

    Comments: In January 2006, we reported that underlying support was often inadequate for information provided in the business cases (exhibit 300s) reviewed. As a result, we recommended that the Director of the Office of Management and Budget (OMB) develop more explicit guidance for sections of the exhibit 300 that cause confusion, including addressing weaknesses we identified. Consistent with our recommendation, in June 2006, OMB modified exhibit 300 requirements and provided more explicit guidance for key sections. On key areas that we addressed as having weaknesses, OMB removed the risk inventory requirement, clarified guidance on earned value management reporting, and added guidance on operational analysis, among other things. Establishing a new exhibit 300 with clearer guidance will improve the quality of the information reported in the business case.

    Recommendation: To help ensure that agency personnel completing exhibit 300s better understand their responsibilities, the Director of OMB should, in advance of OMB's next issuance of the Circular A-11 update, develop and promulgate clearer and more explicit guidance for sections of the exhibit 300 business case that cause confusion, including addressing weaknesses identified in this report (as indicated below) and consulting with agency personnel having responsibility for completing exhibit 300s across government to identify other areas of confusion. At a minimum, the guidance should do the following: (1) provide a more detailed description of the requirements for completing an operational analysis, as referred to in the supplement to Part 7 of Circular A-11, the Capital Programming Guide; and (2) address or clarify possible flexibilities and alternative approaches available to agencies in completing their exhibit 300s: for example, whether the analysis of alternatives section of the exhibit 300 needs to be updated every year for steady state investments and whether all risk areas are relevant for all investments.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  3. Status: Closed - Implemented

    Comments: In response to our recommendation, in December 2009, OMB issued guidance that directed federal agencies to ensure that they are complying with OMB guidance on information quality. This guidance also required each agency to designate a senior official to be accountable for the accuracy and reliability of investment information publicly disseminated on websites such as the IT Dashboard, which is where agencies must now submit their exhibit 300s. Further, the Federal Chief Information Officer (CIO) met with cabinet agency CIOs to emphasize the importance of ensuring their agencies are reporting reliable information to the IT Dashboard. As a result, OMB officials should have access to more accurate investment information, which will help them to make better-informed budgetary decisions.

    Recommendation: Because decision makers should be aware of any weaknesses in the processes used to develop the information in the exhibit 300s, the Director of OMB should direct agencies to determine the extent to which the information contained in each exhibit 300 is accurate and reliable. Where weaknesses in accuracy and reliability are identified, the agency should be required to disclose them and explain the agency's approach to mitigating them.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  4. Status: Closed - Implemented

    Comments: In response to our recommendation, in June 2010, the Office of Management and Budget (OMB) invited Chief Information Officer (CIO) Council members to a training session to review changes to the guidance for exhibit 300 submissions. OMB also requested that CIO Council members share this training information with the appropriate staff in their agencies. As a result of OMB officials' efforts to work with the CIO Council, they are helping to ensure that the exhibit 300 guidance and training include best practices that will be implemented governmentwide.

    Recommendation: In implementing these recommendations, OMB should work with the Chief Information Officers Council to develop the necessary guidance and implement an effective training program to ensure governmentwide acceptance of these changes.

    Agency Affected: Executive Office of the President: Office of Management and Budget

 

Explore the full database of GAO's Open Recommendations »

Sep 25, 2014

Sep 23, 2014

Jun 10, 2014

May 22, 2014

May 12, 2014

May 8, 2014

May 7, 2014

Apr 2, 2014

Looking for more? Browse all our products here