Information Technology Dashboard: Opportunities Exist to Improve Transparency and Oversight of Investment Risk at Select Agencies
Highlights
What GAO Found
Chief Information Officers (CIO) at six federal agencies rated the majority of their information technology (IT) investments as low risk, and many ratings remained constant over time. Specifically, CIOs at the selected agencies rated a majority of investments listed on the federal IT Dashboard as low risk or moderately low risk from June 2009 through March 2012; at five of these agencies, these risk levels accounted for at least 66 percent of investments. These agencies also rated no more than 12 percent of their investments as high or moderately high risk, and two agencies (Department of Defense (DOD) and the National Science Foundation (NSF)) rated no investments at these risk levels. Over time, about 47 percent of the agencies' Dashboard investments received the same rating in every rating period. For ratings that changed, the Department of Homeland Security (DHS) and Office of Personnel Management (OPM) reported more investments with reduced risk when initial ratings were compared with those in March 2012; the other four agencies reported more investments with increased risk. In the past, the Office of Management and Budget (OMB) reported trends for risky IT investments needing management attention as part of its annual budget submission, but discontinued this reporting in fiscal year 2010.
Agencies generally followed OMB's instructions for assigning CIO ratings, which included considering stakeholder input, updating ratings when new data become available, and applying OMB's six evaluation factors. DOD's ratings were unique in reflecting additional considerations, such as the likelihood of OMB review, and consequently DOD did not rate any of its investments as high risk. However, in selected cases, these ratings did not appropriately reflect significant cost, schedule, and performance issues reported by GAO and others. Moreover, DOD did not apply its own risk management guidance to the ratings, which reduces their value for investment management and oversight.
Various benefits were associated with producing and reporting CIO ratings. Most agencies reported (1) increased quality of their performance data, (2) greater transparency and visibility of investments, and (3) increased focus on project management practices. Agencies also noted challenges, such as (1) the effort required to gather, validate, and gain internal approval for CIO ratings; and (2) obtaining information from OMB to execute required changes to the Dashboard. OMB has taken steps to improve its communications with agencies.
Why GAO Did This Study
In June 2009, OMB launched the federal IT Dashboard, a public website that reports performance data for over 700 major IT investments that represent about $40 billion of the estimated $80 billion budgeted for IT in fiscal year 2012. The Dashboard is to provide transparency for these investments to aid public monitoring of government operations. It does so by reporting, among other things, how agency CIOs rate investment risk. GAO was asked to (1) characterize the CIO ratings for selected federal agencies' IT investments as reported over time on the Dashboard, (2) determine how agencies' approaches for assigning and updating CIO ratings vary, and (3) describe the benefits and challenges associated with agencies' approaches to the CIO rating.
To do so, GAO selected six agencies spanning a range of 2011 IT spending levels and analyzed data reported for each of their investments on the Dashboard. GAO also interviewed agency officials and analyzed related documentation and written responses to questions about ratings and evaluation approaches, as well as agency views on the benefits and challenges related to the CIO rating.
Recommendations
GAO is recommending that OMB analyze agencies' investment risk over time as reflected in the Dashboard's CIO ratings and present its analysis with the President's annual budget submission, and that DOD ensure that its CIO ratings reflect available investment performance assessments and its risk management guidance. Both OMB and DOD concurred with our recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget | To ensure that OMB's preparation of the President's budget submission accurately reflects the risks associated with all major IT investments, the Federal CIO should analyze agency trends reflected in Dashboard CIO ratings, and present the results of this analysis with the President's annual budget submission. |
Closed – Implemented
In response to our recommendation, OMB included analyses of the IT Dashboard's CIO risk ratings in its Analytical Perspectives on the Budget of the U.S. Government for fiscal years 2014, 2016, and 2017. In the fiscal year 2017 document, OMB noted that the increased proportion of investments rated as low risk or moderately low risk on the IT Dashboard showed continued improvement in the general health of IT investments across government. As evidence, OMB stated that the percentage of investments rated as low risk or moderately low risk comprised 77 percent of all rated investments in January 2016 compared to 69 percent in 2012. By integrating these analyses into the President's annual budget submissions, OMB communicates an increased awareness of how the CIO risk ratings are changing over time. Knowledge of these changes helps ensure that investment risk is assessed accurately and that patterns warranting special management attention are observed, identified, and addressed.
|
| Department of Defense | To ensure that DOD's CIO evaluations of investment risk for its major IT Dashboard investments reflect all available performance assessments and are consistent with the department's own guidance for managing risk, the Secretary of Defense should direct the department's CIO to reassess the department's considerations for assigning CIO risk levels for Dashboard investments, including assessments of investment performance and risk from outside the programs, and apply the appropriate elements of the department's risk management guidance to OMB's evaluation factors in determining CIO ratings. |
Closed – Implemented
In response to our recommendation, DOD reassessed its process for assigning CIO ratings (risk levels) for its Dashboard investments. Specifically, in March 2014, DOD revised its CIO ratings process to take into account additional information about the risk of its investments, such as investment complexity, execution issues, and external risk assessments (such as GAO reports). By updating its CIO rating process, DOD has increased the transparency of the performance of the agency's investments and, in doing so, has allowed users of the IT Dashboard to better hold DOD accountable for investment results and progress.
|