Information Security:

Concerted Effort Needed to Improve Federal Performance Measures

GAO-09-617: Published: Sep 14, 2009. Publicly Released: Oct 29, 2009.

Additional Materials:

Contact:

Gregory C. Wilshusen
(202) 512-6244
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Information security is a critical consideration for federal agencies, which depend on information systems to carry out their missions. Increases in reports of security incidents demonstrate the urgency of adequately protecting the federal government's data and information systems. Agencies are required to report to the Office of Management and Budget (OMB) on their information security programs, and OMB is to report results to Congress. Agencies have reported progress in carrying out their activities and have used a variety of measures as the basis of that reporting. GAO was asked to (1) describe key types and attributes of performance measures, (2) identify practices of leading organizations for developing and using measures to guide and monitor information security activities, (3) identify the measures used by federal agencies and how they are developed, and (4) assess the federal government's practices for informing Congress on the effectiveness of information security programs. To do this, GAO met with leading organizations, consulted with experts, and reviewed major federal agencies' policies and practices.

Experts and leading organizations (nationally known organizations, academic institutions, and state agencies with enterprisewide information security measurement programs) have identified key types and attributes of successful information security measures. These measures fell into three major types: (1) compliance with policies, standards, or legal and regulatory requirements; (2) effectiveness of information security controls; and (3) overall impact of an organization's information security program. Experts and leading organizations also identified four key attributes of successful measures. Specifically, measures should be quantifiable, meaningful (i.e., have targets for tracking progress, be clearly defined, and be linked to organizational priorities), repeatable and consistent, and actionable (i.e., be able to be used to make decisions). Practices of leading organizations for developing measures emphasized the importance of focusing on the risks facing the organization, involving stakeholders from the beginning of the development process, assigning accountability for results, and linking information security programs to overall business goals. Key practices for using the resulting measurements include tailoring information to specific audiences (e.g., senior executives or unit managers); correlating measures to better assess outcomes; and reporting on the progress, trends, and weaknesses revealed by the collected data. Federal agencies have tended to rely on compliance measures for evaluating their information security controls and programs. The measures developed by agencies have not always exhibited the key attributes identified by leading organizations, and agencies have not always followed key practices in developing their measures, such as focusing on risks. To the extent that agencies do not measure the effectiveness and impact of their information security activities, they may be unable to determine whether their information security programs are meeting their goals. OMB's process for collecting and reporting on agency information security programs employs key practices identified by leading organizations and experts but is lacking in some areas. Specifically, many of the measures that OMB requires have key attributes such as being quantifiable, having targets, and being repeatable and consistent, but others do not. Further, OMB's process for collecting information from agencies relies on measures that do not demonstrate the effectiveness of control activities or the impact of information security programs. In addition, OMB does not adequately tailor its reporting for its congressional audience, correlate the data it collects, or discuss trends and weaknesses in information security controls and programs. Until OMB collects measures of the effectiveness of information security programs and appropriately reports the results, Congress will be hindered in its assessment of federal agencies' information security programs.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In fiscal year 2010, we verified that OMB revised the annual reporting guidance to agencies to require (1) reporting on a more balanced set of measures, including measures that focus on the effectiveness of control activities and program impact, and (2) inclusion of all key attributes in the development of measures. (8/13/2013)

    Recommendation: To improve OMB's process for collecting measures and reporting to Congress on the status of information security programs, the Director of OMB should revise annual reporting guidance to agencies to require (1) reporting on a balanced set of measures, including measures that focus on the effectiveness of control activities and program impact, and (2) inclusion of all key attributes in the development of measures.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  2. Status: Closed - Implemented

    Comments: In fiscal year 2012, we verified that OMB took steps to direct agency chief information officers to employ key practices identified by leading organizations in developing their measures (i.e., focusing on risk, involving key stakeholders in development, assigning accountability, and linking measures to business goals). (8/13/2013)

    Recommendation: To assist federal agencies in developing and using measures that better address the effectiveness of their information security programs, the Director of the OMB should direct agency CIOs to employ key practices identified by leading organizations in developing their measures (i.e., focusing on risk, involving key stakeholders in development, assigning accountability, and linking measures to business goals).

    Agency Affected: Executive Office of the President: Office of Management and Budget

  3. Status: Closed - Implemented

    Comments: In fiscal year 2011, we verified that OMB took steps to direct agency chief information officers to ensure that their measures exhibited the four key attributes (i.e., that it be measurable, meaningful, repeatable and consistent, and actionable). (8/13/2013)

    Recommendation: To assist federal agencies in developing and using measures that better address the effectiveness of their information security programs, the Director of the OMB should direct agency CIOs to ensure that all of their measures exhibit the four key attributes of a measure (i.e., that it be measurable, meaningful, repeatable and consistent, and actionable).

    Agency Affected: Executive Office of the President: Office of Management and Budget

  4. Status: Closed - Implemented

    Comments: In fiscal year 2012, we verified that OMB took steps to issue revised information security guidance to agency chief information officers to reinforce the requirement that agencies follow NIST guidance in developing measures and clarifying the need to develop and use a balanced set of measures that includes compliance, control effectiveness, and program impact measures. (8/13/2013)

    Recommendation: To assist federal agencies in developing and using measures that better address the effectiveness of their information security programs, the Director of the OMB should issue revised information security guidance to agency chief information officers (CIO) reinforcing the existing requirement that agencies follow National Institute of Standards and Technology (NIST) guidance (which correlates with key practices) in developing measures and clarifying the need to develop and use a balanced set of measures that includes compliance, control effectiveness, and program impact measures.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  5. Status: Closed - Implemented

    Comments: In fiscal year 2012, we verified that OMB took steps to revise its annual report to Congress to provide better status information, including information on the effectiveness of agency information security programs, the extent to which major risks are being addressed, and progress that has been made in improving the security posture of the federal government. (8/17/2013)

    Recommendation: To improve OMB's process for collecting measures and reporting to Congress on the status of information security programs, the Director of OMB should revise the annual report to Congress to provide better status information, including information on the effectiveness of agency information security programs, the extent to which major risks are being addressed, and progress that has been made in improving the security posture of the federal government.

    Agency Affected: Executive Office of the President: Office of Management and Budget

 

Explore the full database of GAO's Open Recommendations »

Nov 18, 2014

Nov 17, 2014

Sep 18, 2014

Sep 16, 2014

Sep 8, 2014

Jul 17, 2014

Jun 25, 2014

May 30, 2014

Apr 17, 2014

Apr 2, 2014

Looking for more? Browse all our products here