Concerted Effort Needed to Improve Federal Performance Measures
GAO-10-159T: Published: Oct 29, 2009. Publicly Released: Oct 29, 2009.
Cyber security is a critical consideration for any organization that depends on information systems and computer networks to carry out its mission or business. Organizations are faced with a variety of information security threats, such as fraudulent activity from cyber criminals, unauthorized access by disgruntled or dishonest employees, and denial-of-service attacks and other disruptions. The recent dramatic increase in reports of security incidents, the wide availability of hacking tools, and steady advances in the sophistication and effectiveness of attack technology all contribute to the urgency of ensuring that adequate steps are taken to protect the federal government's information and the systems that contain and process it. The Federal Information Security Management Act (FISMA), which was enacted in 2002, sets forth a comprehensive framework for ensuring the effectiveness of security controls over information resources that support federal operations and assets. The act assigns specific responsibilities to federal agencies, the Office of Management and Budget (OMB), and the National Institute of Standards and Technology (NIST). It also requires agencies and OMB to annually report on the adequacy and effectiveness of agency information security programs and compliance with the provisions of the act. To help meet these requirements, OMB established a uniform set of information security measures that all federal agencies report on annually.
Leading organizations and experts have identified different types of measures that are useful in helping to achieve information security goals. While officials categorized these types using varying terminology, we concluded that they generally fell into three types: (1) compliance, (2) control effectiveness, and (3) program impact. These types are consistent with those laid out by NIST in its information security performance measurement guide. In addition, while information security measures can be grouped into these three major types, organizations and experts reported that all such measures generally have certain key characteristics, or attributes. These attributes include being (1) measurable, (2) meaningful, (3) repeatable and consistent, and (4) actionable. Further, these organizations and experts indicated that the successful development of information security measures depends on adherence to a number of key practices, including focusing on risks, involving stakeholders, assigning accountability, and linking to business goals. Additional practices are critical to ensuring that the measures are useful in effectively conveying information to operational managers, executives, and oversight officials. These include tailoring measures to the audience; correlating data; and capturing progress, trends, and weaknesses. We determined that federal agencies have not always followed key practices identified by leading organizations for developing information security performance measures. While agencies have developed measures that fall into each of the three major types (i.e. compliance, control effectiveness, and program impact), on balance they have relied primarily on compliance measures, which have a limited ability to gauge program effectiveness. Agencies stated that, for the most part, they predominantly collected measures of compliance because they were focused on measures associated with OMB's FISMA reporting requirements. In addition, while most agencies have developed some measures that include the four key attributes identified by leading organizations and experts, these attributes were not always present in all agency measures. Further, agencies have not always followed key practices in developing measures, such as focusing on risks.