Agencies Are Managing Investment Risk, but Related Ratings Need to Be More Accurate and Available
GAO-14-64: Published: Dec 12, 2013. Publicly Released: Jan 13, 2014.
What GAO Found
As of August 2013, the Chief Information Officers (CIO) at the eight selected agencies rated 198 of their 244 major information technology (IT) investments listed on the Federal IT Dashboard (Dashboard) as low risk or moderately low risk, 41 as medium risk, and 5 as high risk or moderately high risk. However, the total number of investments reported by these agencies has varied over time, which impacts the number of investments receiving CIO ratings. For example, Energy reclassified several of its supercomputer investments from IT to facilities and Commerce decided to reclassify its satellite ground system investments. Both decisions resulted in the removal of the investments from the Dashboard, even though the investments were clearly IT. In addition, the Office of Management and Budget (OMB) does not update the public version of the Dashboard as the President's budget request is being created. As a result, the public version of the Dashboard was not updated for 15 of the past 24 months, and so was not available as a tool for investment oversight and decision making.
Of the 80 investments reviewed, 53 of the CIO ratings were consistent with the investment risk, 20 were partially consistent, and 7 were inconsistent. While two agencies' CIO ratings were entirely consistent, other agencies' ratings were inconsistent for a variety of reasons, including delays in updating the Dashboard and how investment performance was tracked. For example, the Department of Justice downgraded an investment in July 2012, but the Dashboard was not updated to reflect this until April 2013. Further, the Social Security Administration resets investment cost and schedule performance baselines annually, an approach that increases the risk of undetected cost or schedule variances that will impact investment success.
Of the eight investments that were at highest risk in 2012, seven were reviewed by their agencies using tools such as TechStat sessions--evidence-based reviews intended to improve investment performance and other high-level reviews. Each of these resulted in action items intended to improve performance. The final investment was scheduled to have a TechStat, but instead, according to department officials, a decision was made to modify its program cost and schedule commitments to better reflect the investment's actual performance.
Why GAO Did This Study
OMB launched the Dashboard in June 2009 as a public Web site that reports performance for major IT investments--on which the federal government plans to invest over $38 billion in fiscal year 2014. The Dashboard is to provide transparency for these investments and to facilitate public monitoring of them. After its launch, OMB began using it to identify at-risk investments.
This report (1) characterizes the CIO ratings for selected federal agencies' IT investments as reported on the Dashboard over time, (2) determines the extent to which selected agencies' CIO ratings are consistent with investment risk, and (3) determines the extent to which selected agencies are addressing at-risk investments. GAO selected the eight agencies with the most reported major IT spending in fiscal year 2012 (excluding those GAO recently reviewed) and selected 10 investments at each agency. GAO reviewed the investments' documentation, compared it to the CIO ratings, and reviewed processes used for the highest-risk investments. GAO also interviewed appropriate officials.
What GAO Recommends
GAO recommends that OMB make Dashboard information available independent of the budget process, and that agencies appropriately categorize IT investments and address identified weaknesses. OMB neither agreed nor disagreed. Six agencies generally agreed with the report or had no comments and two others did not agree, believing their categorizations were appropriate. GAO continues to believe its recommendations remain valid, as discussed.
For more information, contact David A. Powner at (202) 512-9286 or email@example.com.
Recommendations for Executive Action
Comments: Although the Federal CIO did not agree or disagree with our recommendation, OMB has taken initial steps to implement it. Specifically, OMB recently updated the Dashboard with a number of changes, and OMB officials stated in 2015 that they intended for the Dashboard to be able to show updates throughout the year. That said, OMB has yet to implement this recommendation. Specifically, OMB did not publish updates to the public version of the Dashboard during the fiscal year 2018 budget formulation process, starting at the end of August 2016. We will continue to monitor the Dashboard to determine if portions of the public version of the Dashboard (such as CIO ratings) are available throughout the year. Maintaining the availability of these data is important for increasing the utility of the Dashboard as a tool for greater IT investment oversight and transparency.
Recommendation: To better ensure that the Dashboard provides meaningful ratings and reliable investment data, the Director of OMB should direct the Federal CIO to make accessible regularly updated portions of the public version of the Dashboard (such as CIO ratings) independent of the annual budget process.
Agency Affected: Executive Office of the President: Office of Management and Budget
Comments: The Department of Commerce disagreed with this recommendation. In written correspondence, the Department noted that, although it is no longer reporting three of the 10 investments reviewed for this engagement on the IT Dashboard, it is maintaining oversight through monthly Dashboard-like assessments. As of July 28, 2016, the Department stated that it did not have plans to re-categorize these three particular investments as IT and report the data on the IT Dashboard. We continue to believe that this recommendation has merit and will monitor the Department's efforts to maintain oversight for these investments.
Recommendation: To better ensure that the Dashboard provides accurate ratings, the Secretary of Commerce should direct the department CIO to ensure that the department's investments are appropriately categorized in accordance with existing statutes and that major IT investments are included on the Dashboard.
Agency Affected: Department of Commerce
Comments: While the Department of Energy had agreed with this recommendation, in subsequent written correspondence, it explained that five of the eight investments noted by GAO as being IT were no longer being reported in the IT Portfolio on the Dashboard. Instead, the Department was reporting these data to OMB via an alternative reporting mechanism specific to high performance computing. In addition, the Department noted that the remaining three investments were deconsolidated or downgraded into non-major investments, or eliminated by funding and, as such, these investments will not be included on the Dashboard. However, we continue to believe that this recommendation has merit and that the remaining investments are more properly classified as IT. We will continue to monitor the Department's efforts to maintain oversight for these investments.
Recommendation: To better ensure that the Dashboard provides accurate ratings, the Secretary of Energy should direct the department CIO to ensure that the department's investments are appropriately categorized in accordance with existing statutes and that major IT investments are included on the Dashboard.
Agency Affected: Department of Energy
Status: Closed - Implemented
Comments: The Social Security Administration (SSA) agreed with this recommendation and subsequently revised its baselining policies. For example, in its Performance Measurement Baseline Policy, SSA states that rebaselining should not occur on a regular basis and that it is not permitted solely for improving program performance. Rather, the policy explains that a rebaseline is appropriate when the performance measurement baseline is no longer adequate to provide valid cost, schedule, and/or operational performance measurement information relative to the remaining program work. This new policy is consistent with the baselining best practices identified in our Cost Estimating and Assessment Guide. This guidance states that baseline changes should be infrequent and that a rebaselining effort should, among other things, ensure that the new baseline is adequate and meaningful for future performance measurement. By implementing our recommendation, SSA will be able to better analyze cost and schedule variances, monitor performance, and estimate the life-cycle costs of its IT investments.
Recommendation: To better ensure that the Dashboard provides accurate ratings, the Commissioner of the Social Security Administration should direct the CIO to revise the agency's investment management processes to ensure that they are consistent with the baselining best practices identified in our published guidance on cost estimating and assessment.
Agency Affected: Social Security Administration