This is the accessible text file for GAO report number GAO-05-1
entitled 'Informing Our Nation: Improving How to Understand and Assess
the USA's Position and Progress' which was released on November 10,
2004.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Subcommittee on Science, Technology, and Space,
Committee on Commerce, Science, and Transportation, U.S. Senate:
November 2004:
INFORMING OUR NATION:
Improving How to Understand and Assess the USA's Position and Progress:
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-1]:
GAO Highlights:
Highlights of GAO-05-1, a report to the Chairman, Subcommittee on
Science, Technology, and Space, Committee on Commerce, Science, and
Transportation, U.S. Senate:
Why GAO Did This Study:
There has been growing activity and interest in developing a system of
key national indicators that would provide an independent, trusted,
reliable, widely available, and usable source of information. Such a
system would facilitate fact-based assessments of the position and
progress of the United States, on both an absolute and relative basis.
This interest emerges from the following perspectives.
* The nation’s complex challenges and decisions require more
sophisticated information resources than are now available.
* Large investments have been made in indicators on a variety of topics
ranging from health and education to the economy and the environment
that could be aggregated and disseminated in ways to better inform the
nation.
* The United States does not have a national system that assembles key
information on economic, environmental, and social and cultural issues.
Congressional and other leaders recognized that they could benefit from
the experiences of others who have already developed and implemented
such key indicator systems. GAO was asked to conduct a study on: (1)
The state of the practice in these systems in the United States and
around the world, (2) Lessons learned and implications for the nation,
and (3) Observations, options, and next steps to be considered if
further action is taken.
What GAO Found:
GAO studied a diverse set of key indicator systems that provide
economic, environmental, social and cultural information for local,
state, or regional jurisdictions covering about 25 percent of the U.S.
population—as well as several systems outside of the United States. GAO
found opportunities to improve how our nation understands and assesses
its position and progress.
Citizens in diverse locations and at all levels of society have key
indicator systems. Building on a wide array of topical bodies of
knowledge in areas such as the economy, education, health, and the
environment, GAO found that individuals and institutions across the
United States, other nations, and international organizations have key
indicator systems to better inform themselves. These systems focus on
providing a public good: a single, freely available source for key
indicators of a jurisdiction’s position and progress that is
disseminated to broad audiences. A broad consortium of public and
private leaders has begun to develop such a system for our nation as a
whole.
These systems are a noteworthy development with potentially broad
applicability. Although indicator systems are diverse, GAO identified
important similarities. For example, they faced common challenges in
areas such as agreeing on the types and number of indicators to include
and securing and maintaining adequate funding. Further, they showed
evidence of positive effects, such as enhancing collaboration to
address public issues, and helping to inform decision making and
improve research. Because these systems exist throughout the United
States, in other nations, and at the supranational level, the potential
for broad applicability exists, although the extent of applicability
has yet to be determined.
Congress and the nation have options to consider for further action.
GAO identified nine key design features to help guide the development
and implementation of an indicator system. For instance, these features
include establishing a clear purpose, defining target audiences and
their needs, and ensuring independence and accountability. Customized
factors will be crucial in adapting such features to any particular
level of society or location. Also, there are several alternative
options for a lead entity to initiate and sustain an indicator system:
publicly led, privately led, or a public-private partnership in either
a new or existing organization.
Observations, Options, and Next Steps:
Key indicator systems merit serious discussion at all levels of
society, including the national level, and clear implementation options
exist from which to choose. Hence, Congress and the nation should
consider how to
* improve awareness of these systems and their implications for the
nation,
* support and pursue further research,
* help to catalyze discussion on further activity at subnational
levels, and
* begin a broader dialogue on the potential for a U.S. key indicator
system.
www.gao.gov/cgi-bin/getrpt?GAO-05-1.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Christopher Hoenig at
(202) 512-6779 or hoenigc@gao.gov.
[End of section]
Contents:
Letter:
Summary:
Purpose:
Background:
Scope and Methodology:
Results in Brief:
Chapter 1: Introduction:
Indicators and Indicator Systems:
An Illustrative History of National Efforts in the United States:
Current Activities to Inform the Nation through Comprehensive Key
Indicator Systems:
Detailed Scope and Methodology:
Chapter 2: Citizens in Diverse Locations and at All Levels of Society
Have Indicator Systems:
Topical Indicator Systems in the United States Form a Vital Foundation
for Comprehensive Key Indicator Systems:
The Practice of Developing Comprehensive Key Indicator Systems Is
Active and Diverse:
Chapter 3: Comprehensive Key Indicator Systems Are a Noteworthy
Development with Potentially Broad Applicability:
A Diverse Set of Systems Faced Similar Challenges:
Comprehensive Key Indicator Systems Show Evidence of Positive Effects:
Comprehensive Key Indicator Systems Have Potentially Broad
Applicability:
Chapter 4: Congress and the Nation Have Options to Consider in Taking
Further Action:
Certain Design Features Should Guide the Development of Any System,
Including a U.S. National System:
Congress Could Choose from a Range of Organizational Options as
Starting Points for a U.S. National System:
Others Considering Comprehensive Key Indicator Systems Have Similar
Options:
Chapter 5: Observations and Next Steps:
Observations:
Next Steps:
Appendixes:
Appendix I: U.S. National Topical Indicator Systems Included in This
Study:
Appendix II: Overview of Social and Cultural Indicators:
Appendix III: Comprehensive Key Indicator Systems Included in This
Study:
Appendix IV: Timeline and Evolution of the Boston Indicators Project:
Appendix V: Timeline and Evolution of the Oregon Benchmarks:
Appendix VI: The Role of Indicators in the European Union:
Appendix VII: Selected Bibliography on Indicator Systems:
Appendix VIII: GAO Contact and Contributors:
GAO Contact:
Major Contributors:
Other Contributors:
Tables:
Table 1: Comprehensive Key Indicator Systems Selected for GAO's Study:
Table 2: Selected Topical Areas Covered by Federal Statistical
Programs:
Table 3: Selected Highlights of Indicator Traditions in the United
States:
Table 4: Comprehensive Key Indicator Systems Reviewed for This Study,
by Level of Jurisdiction:
Table 5: European Structural Indicators--Headline Indicators:
Table 6: Characteristics, Advantages, and Disadvantages of the Public
Organizational Option:
Table 7: Characteristics, Advantages, and Disadvantages of the Private
Organizational Option:
Table 8: Characteristics, Advantages, and Disadvantages of the Public-
Private Organizational Option:
Table 9: Advantages and Disadvantages of a New Versus an Existing
Organization:
Table 10: Organizational Types of the Systems Studied for Our Review:
Figures:
Figure 1: Possible Topics for a Comprehensive Key Indicator System:
Figure 2: An Economic Indicator Showing World Exports of Goods and
Services as a Percentage of World GDP, 1970-2002:
Figure 3: A Social and Cultural Indicator Showing the Percentage of
Persons Ages 16-24 Who Were Neither Enrolled in School Nor Working, by
Race/Ethnicity (Selected Years 1986-2003):
Figure 4: An Environmental Indicator Showing the Number and Percentage
of Days with an Air Quality Index (AQI) Greater Than 100, 1988-2001:
Figure 5: GPI Per Capita for Burlington, Vermont; Chittenden County,
Vermont; the State of Vermont; and the United States, 1950-2000:
Figure 6: Revisions in the Leading Index of the Business Cycle
Indicators, 1984-1997:
Figure 7: Coronary Heart Disease and Stroke Deaths, by Year, in the
United States, 1979-1998:
Figure 8: Reported Sources of Pollution That Resulted in Beach Closings
or Advisories, 2001:
Figure 9: Percentage of Children Ages 6 to 18 Who Are Overweight, by
Gender, Race, and Mexican-American Origin, Selected Years 1976-1980,
1988-1994, 1999-2000:
Figure 10: Percentage of Medicare Beneficiaries Age 65 or Older Who
Reported Having Had Problems with Access to Health Care, 1992-1996:
Figure 11: Relative Longevity of Selected Comprehensive Key Indicator
Systems in the United States and Abroad:
Figure 12: Boston's Data Items by Source:
Figure 13: Neighborhood Facts Database Sample, Denver:
Figure 14: SAVI Web Site Sample, Indianapolis:
Figure 15: The Boston Indicators Project's Interactive Web Site:
Figure 16: Number of Publicly Traded Gazelle Firms in the Silicon
Valley:
Figure 17: Students Carrying Weapons--Percentage of Students Who Carry
Weapons in Oregon:
Figure 18: Percentage of Working-Age People Who Are Currently Employed
in the United Kingdom by Region for 2000 and 2003:
Figure 19: Long-term Unemployment Rates for Men, 1999-2002:
Figure 20: Median Number of Days It Takes for Homes to Sell in a
Particular Area of Baltimore:
Figure 21: Different Indicators Used to Measure the Success of Public
Schools in Jacksonville, Florida:
Figure 22: SAVI Interactive Tools:
Figure 23: Traffic Congestion in Chicago--Actual 1996 and Projected
2030:
Figure 24: Travel Trends Placing Stress on the Chicago Regional Traffic
System:
Figure 25: Percentage of 9th Graders Reporting Use of Alcohol in the
Last 30 Days:
Figure 26: Oregon State Agencies Whose Programs Are Linked to Child
Abuse or Neglect:
Figure 27: Population Coverage of Select Comprehensive Key Indicator
Systems in the United States:
Letter November 10, 2004:
The Honorable Sam Brownback:
Chairman:
Subcommittee on Science, Technology, and Space:
Committee on Commerce, Science, and Transportation:
United States Senate:
Dear Mr. Chairman:
Since the founding of our republic, the importance of informing the
nation has been an essential component of a healthy democracy. In our
country, power resides with the people and their duly elected
representatives, and knowledge serves to both inform and constrain the
use of power. This idea is embodied in forms ranging from the decennial
census to the notion of annually reporting on the state of the union,
with its history of providing a broad, general picture of the nation's
position and progress, along with the President's agenda for the coming
year.
Our founding fathers recognized that this critical issue needed ongoing
attention. President George Washington, in his first annual message to
Congress on January 8, 1790, said, "Knowledge is in every country the
surest basis of public happiness. In one in which the measures of
government receive their impressions so immediately from the sense of
the community as in ours it is proportionably [sic] essential." Since
that time, there has been a long history--checkered by both success and
failure--of attempts to create ever more advanced ways to inform our
public dialogues and generate a context for civic choices and
democratic governance.
This bedrock principle of informing our nation and its citizens has
maintained its simple, common sense relevance for centuries. Yet, it
has also evolved and adapted over time to encompass new national and
global challenges.
At the time of our nation's founding, collecting and disseminating
information was achieved primarily through word of mouth and the
printing press, drawn from few institutional sources, and traveled at
speeds of 10 to 20 miles per hour. The availability of information was
primarily limited to elite groups, and broad general perspectives were
difficult to develop because of a dearth of factual information.
Today, information is collected and disseminated at the speed of light,
is generated in massive amounts from an array of sources, and is
available throughout the world to almost anyone. It is so diverse and
rich that general perspectives are difficult to develop because of a
surfeit of information.
Yet it is just those perspectives we now need in order to work through
the short-and long-term challenges facing our nation, particularly
when, at the federal level, the gap between public expectations and
available resources is expected to widen. There is no substitute for
being able to understand the whole (e.g., the position and progress of
the nation) in order to better assess and act on the parts (e.g., the
various key issues that we face).
The opportunity before us is to build sophisticated information
resources and comprehensive key indicator systems that aggregate vital
information across sectors, levels of societies, and institutions.
These would be available to any person or institution, anywhere at any
time, and for any purpose.
They would add a key dimension to how we inform ourselves. We now have
many diverse and extensive bodies of information on issues of limited
focus (e.g., health care). But we could use comprehensive key indicator
systems on a broader array of critical issues to help generate a
broader perspective, clarify problems and opportunities, identify gaps
in what we know, set priorities, test effective solutions, and track
progress towards achieving results. For instance, across the federal
government, such systems could inform a much needed re-examination of
the base of existing programs, policies, functions, and activities.
To be a leading democracy in the information age may very well mean
producing unique public sources of objective, independent,
scientifically grounded, and widely shared quality information so that
we know where the United States stands now and how we are trending, on
both an absolute and relative basis--including comparisons with other
nations. By ensuring that the best facts are made more accessible and
usable by the many different members of our society, we increase the
probability of well-framed problems, good decisions, and effective
solutions.
The stakes are high, including considerations regarding allocations of
scarce public resources, strengthing the economy, creating jobs,
stimulating future industries, enhancing security, promoting safety,
strengthening our competitive edge, sustaining the environment,
preserving our culture, and promoting quality of life. As a result,
Congress has a crucial interest in the evolution of comprehensive key
indicator systems throughout our nation and the world.
Given the variety of activity and interest we observed at all levels of
U.S. society on this issue, this report can benefit not only those
seeking to develop a national key indicator system, but also the local
and state communities who would like to learn more, develop new
systems, or refine their existing efforts. We look forward to working
with you and other leaders in joining the effort to develop new
approaches to informing our nation that will be of truly lasting value
to the American people.
Copies of this report are being sent to appropriate congressional
committees and other interested parties in the United States and around
the world. We will also make copies available to others upon request.
This report will also be available at no charge on the GAO Web site at
[Hyperlink, http://www.gao.gov]. If you or your staff has any questions
about matters discussed in this report, please contact me at (202)
512-5500 or Christopher Hoenig, Managing Director, Strategic Issues, at
(202) 512-6779 or [Hyperlink, hoenigc@gao.gov]. Key contributors are
listed in appendix VIII.
Sincerely yours,
Signed by:
David M. Walker:
Comptroller General of the United States:
[End of section]
Summary:
Purpose:
A substantial amount of activity is taking place throughout the United
States and around the world to develop comprehensive key indicator
systems for communities, cities, states, and nations that include
essential economic, environmental, and social and cultural indicators.
These systems help people and organizations answer vital questions,
such as: How is their community, state and/or nation as a whole doing
in fact? How does it compare to others or to prior conditions? And how
does that information help them make better choices? Such systems can
become an essential part of civic dialogue and decision making.
Many in the United States believe that comprehensive key indicator
systems represent a significant and evolving opportunity to improve how
individuals, groups, and institutions inform themselves. This is
because they can enable assessment of the position and progress not
just of a wide range of jurisdictions throughout the country, but also
of the nation as a whole. Figure 1 illustrates the variety of topics
that might be included in such a system.
Figure 1: Possible Topics for a Comprehensive Key Indicator System:
[See PDF for image]
[End of figure]
To begin the process of considering whether or how to develop such a
system at the national level in the United States, congressional and
other leaders have an interest in better understanding the experiences
of those who have already designed and implemented comprehensive key
indicators systems. GAO was not asked to develop a set of national
indicators or conduct an assessment of the position and progress of the
United States, but rather to address the following three questions.
1. What is the state of the practice in developing and implementing
comprehensive key indicator systems in the United States and around the
world?
2. What are the lessons learned from these systems and future
implications?
3. What are some options for Congress to consider in identifying an
organization to develop and implement a national system?
Background:
An indicator is a quantitative measure that describes an economic,
environmental, social or cultural condition over time. The unemployment
rate, infant mortality rates, and air quality indexes are a few
examples.
An indicator system is an organized effort to assemble and disseminate
a group of indicators that together tell a story about the position and
progress of a jurisdiction or jurisdictions, such as the City of
Boston, the State of Oregon, or the United States of America. Indicator
systems collect information from suppliers (e.g., individuals who
respond to surveys or institutions that provide data they have
collected), which providers (e.g., the Census Bureau) then package into
products and services for the benefit of users (e.g., leaders,
researchers, planners, and citizens).
Topical indicator systems involve specific or related sets of issues,
such as health, education, public safety, employment, or
transportation. They also form the foundation of information resources
for the general public, the media, professionals, researchers,
institutions, leaders, and policymakers.
Comprehensive key indicator systems pull together only the most
essential indicators on a range of economic, environmental, and social
and cultural issues, as opposed to a group of indicators on one topic.
Comprehensive systems are only as good as the topical systems they draw
from.
Both comprehensive and topical indicator systems use indicators from
public and private sources, and often disseminate this information to
diverse audiences, such as in a report or on a Web site. Ultimately,
however, comprehensive key indicator systems attempt to address
questions that topical systems (which focus on a specific issue) or
current statistical databases (which are detailed and highly technical)
cannot answer for wide and diverse audiences.
Comprehensive key indicator systems can help to identify a
jurisdiction's significant challenges and opportunities, highlight
their importance and urgency, inform choices regarding the allocation
of scarce public resources, assess whether solutions are working, and
make comparisons to other jurisdictions. They exist in a number of
countries, including Australia, Canada, Germany, and the United
Kingdom, as well as supranational entities like the European Union
(EU).[Footnote 1]
There is a long history of considering the need for a national
comprehensive key indicator system in the United States going back at
least to the 1930s. Currently, although a number of cities, states, and
regions in the United States have comprehensive key indicator systems,
there is no such system for the United States as a whole. The federal
government has, however, invested billions of dollars in a rich variety
of topical information that could underpin a national system.[Footnote
2] It also supports various efforts to enhance the availability of that
information, such as Fedstats and The Statistical Abstract of the
U.S.[Footnote 3]
Currently, a consortium of not-for-profit, private, and public sector
efforts is collaborating to create a comprehensive key indicator system
for the United States.[Footnote 4] This initiative, known as the Key
National Indicators Initiative (KNII), emerged after GAO--in
cooperation with the National Academies--convened a forum in February
2003.[Footnote 5] At this forum, a cross-section of leaders provided
their views on whether and how to develop such a national system and
believed that it was an important idea that should be explored
further.[Footnote 6] They also suggested that it should build on
lessons learned from other efforts both around the country and
worldwide.
The KNII has grown to include a diverse group of over 200 leaders from
government, business, research, and the nonprofit sector. This group
consists of experts as well as representatives from broad-based
institutions throughout the nation. The National Academies currently
houses a secretariat to incubate this effort. It has recently begun to
organize more formally and received initial operational funding. One of
its goals is to create and test a prototype "State of the USA" Web
site.
Scope and Methodology:
This report is a first step in examining how existing comprehensive key
indicator systems are working and their implications for the nation. It
presents information obtained from a select, but not necessarily
representative, group of 29 comprehensive key indicator systems at all
levels of society and diverse geographic locations, as shown on table
1. GAO interviewed representatives from each of the selected indicator
systems, as well as a range of experts in the field. In addition, GAO
conducted in-depth reviews--including interviews with officials,
stakeholders, and users--of 5 of these 29 systems: Boston, Oregon,
Germany, the United Kingdom, and the EU. GAO also studied U.S. topical
indicator systems in five areas: the business cycle, science and
engineering, health, children and families, and aging. To explore
options for Congress, GAO drew upon its professional judgment,
historical and legal analysis, fieldwork, and expert interviews.
Table 1: Comprehensive Key Indicator Systems Selected for GAO's Study:
Name of system: U.S local/regional level: State of the Region (Southern
California);
Approximate population: 17,123,000;
Approximate duration: (in years): 7.
Name of system: U.S local/regional level: Chicago Metropolis 2020;
Approximate population: 8,090,000;
Approximate duration: (in years): 8.
Name of system: U.S local/regional level: New York City Social
Indicators;
Approximate population: 8,080,000;
Approximate duration: (in years): 15.
Name of system: U.S local/regional level: Index of Silicon Valley
(California);
Approximate population: 2,300,000;
Approximate duration: (in years): 12.
Name of system: U.S local/regional level: King County Benchmarks
(Washington);
Approximate population: 1,760,000;
Approximate duration: (in years): 14.
Name of system: U.S local/regional level: Social Assets and
Vulnerabilities Indicators (Indianapolis);
Approximate population: 1,600,000;
Approximate duration: (in years): 11.
Name of system: U.S local/regional level: Indicators for Progress
(Jacksonville, Fla.);
Approximate population: 1,200,000;
Approximate duration: (in years): 19.
Name of system: U.S local/regional level: Hennepin County Community
Indicators (Minneapolis);
Approximate population: 1,120,000;
Approximate duration: (in years): 9.
Name of system: U.S local/regional level: Community Atlas (Tampa area,
Fla.);
Approximate population: 1,070,000;
Approximate duration: (in years): 7.
Name of system: U.S local/regional level: Compass Index of
Sustainability (Orange County, Fla.);
: Approximate population: 965,000;
Approximate duration: (in years): 12.
Name of system: U.S local/regional level: Portland Multnomah
Benchmarks;
Approximate population: 678,000;
Approximate duration: (in years): 11.
Name of system: U.S local/regional level: Baltimore's Vital Signs;
Approximate population: 640,000;
Approximate duration: (in years): 4.
Name of system: U.S local/regional level: Boston Indicators Project;
Approximate population: 590,000;
Approximate duration: (in years): 7.
Name of system: U.S local/regional level: Milwaukee Neighborhood Data
Center;
Approximate population: 590,000;
Approximate duration: (in years): 13.
Name of system: U.S local/regional level: Sustainable Seattle;
Approximate population: 570,000;
Approximate duration: (in years): 12.
Name of system: U.S local/regional level: Denver Neighborhood Facts;
Approximate population: 560,000;
Approximate duration: (in years): 10.
Name of system: U.S local/regional level: Santa Cruz County Community
Assessment Project;
Approximate population: 250,000;
Approximate duration: (in years): 11.
Name of system: U.S local/regional level: Benchmarking Municipal and
Neighborhood Services in Worcester (Massachusetts);
Approximate population: 175,000;
Approximate duration: (in years): 6.
Name of system: U.S local/regional level: Santa Monica Sustainable City
(California);
Approximate population: 84,000;
Approximate duration: (in years): 10.
Name of system: U.S local/regional level: Burlington Legacy Project
(Vermont);
Approximate population: 39,000;
Approximate duration: (in years): 5.
Name of system: U.S state level: North Carolina 20/20;
Approximate population: 8,407,000;
Approximate duration: (in years): 9.
Name of system: U.S state level: Minnesota Milestones[A];
Approximate population: 5,059,000;
Approximate duration: (in years): 13.
Name of system: U.S state level: Oregon Benchmarks;
Approximate population: 3,560,000;
Approximate duration: (in years): 15.
Name of system: U.S state level: Results Iowa;
Approximate population: 2,944,000;
Approximate duration: (in years): 5.
Name of system: U.S state level: Maine's Measures of Growth;
Approximate population: 1,306,000;
Approximate duration: (in years): 11.
Name of system: U.S state level: Social Well-Being of Vermonters;
Approximate population: 619,000;
Approximate duration: (in years): 11.
Name of system: National level outside the United States: German System
of Social Indicators;
Approximate population: 83,000,000;
Approximate duration: (in years): 30.
Name of system: National level outside the United States: United
Kingdom Sustainable Development Indicators;
Approximate population: 60,000,000;
Approximate duration: (in years): 5.
Name of system: Supranational level: European Structural Indicators;
Approximate population: 450,000,000;
Approximate duration: (in years): 4.
Source: GAO.
Note: for more information on each of these systems, see app. III of
this report. The Web links for each of these systems can be found at
[Hyperlink, http://www.keyindicators.org].
[A] Since GAO conducted its interviews in fall 2003, Minnesota
Milestones ceased to be an active system. State officials told us that
the Web site will be maintained but there are no plans to update the
data in the near future.
[End of table]
GAO selected comprehensive key indicator systems that were recognized
by experts and others as being useful and accessible; and had been in
existence for more than 2 years. Also, GAO asked national associations
representing state and local governments to validate the selections.
The European examples were selected after consultation with OECD,
several European national statistical offices, and other experts. GAO
selected one system in each of the topical areas it reviewed on the
basis of experts' recommendations. GAO also conducted a literature
review. Importantly, GAO has not defined explicit, objective criteria
for the success or failure of a comprehensive key indicator system.
More research is needed in this area because so many situational,
evaluative, and contextual factors influence the determination of such
criteria.
Most of the graphics presented in this report from the indicator
systems GAO studied are only to illustrate the types of information and
the variety of ways it is presented in the reports or on the Web sites
of these systems. The examples are not intended to highlight or frame
discussions of the substantive issues conveyed by them.
GAO did not, nor was it asked to, catalogue the full universe of the
potentially large number of topical or comprehensive key indicator
systems. Moreover, indicators are only one part of the complex
knowledge base required to inform a nation. For instance, comprehensive
key indicator systems must be supported by detailed databases for those
who want or need to conduct more extensive research or analysis. A
review of these databases and other elements that contribute to an
informed society are beyond the scope of this report.
Given the relatively small number of systems GAO studied in-depth, this
report's findings and conclusions may not be universally applicable.
GAO did not review the entire body of knowledge associated with
indicator systems in either private enterprises or government agencies
and did not perform a formal cost and benefit analysis of the systems
reviewed. Nor did GAO evaluate the federal statistical system and its
related agencies. Most of the indicator system efforts GAO studied are
not necessarily comparable in size and political-economic structure to
the United States, which potentially limits the validity of
generalizations to the U.S. national context.
To gain additional insights, GAO solicited and received comments on a
draft copy of the report from over 60 experts who possess knowledge and
experience in this field, including leaders from the statistical and
scientific communities. Sections of the report were also reviewed by
the systems GAO studied to confirm facts and figures. GAO incorporated
comments where appropriate in this final version. GAO's work was
conducted from July 2003 through September 2004 in accordance with
generally accepted government auditing standards.
Results in Brief:
GAO found that comprehensive key indicator systems are active, diverse,
and evolving. Individuals and institutions from local, state, and
regional levels across the United States--as well as some other nations
and the EU--have comprehensive key indicator systems to better inform
themselves. GAO found enough similarities in the challenges they
encountered and the positive effects they have had to view them as a
coherent, noteworthy development in governance. They also have
potentially broad applicability. Accordingly, GAO has identified key
design features and defined a set of options for Congress and the
nation to consider regarding the further development of comprehensive
key indicator systems at all levels of society, including the U.S.
national level.
State of the Practice: Citizens and Institutions in Diverse Locations
and at All Levels of Society Have Comprehensive Key Indicator Systems:
Jurisdictions throughout this country and around the world are
operating comprehensive key indicator systems and have been for years.
Many recognize that these systems could represent a significant tool to
better inform public and private debate and decision making.
Topical Systems Provide the Foundation for Comprehensive Key Indicator
Systems:
The United States has a wide variety of topical indicator systems at
the national level that provide a resource for comprehensive key
indicator systems to draw upon. The interrelationship between topical
and comprehensive key indicator systems is complementary. Topical
systems form the essential underpinning for aggregating information
into comprehensive key indicator systems. Comprehensive key indicator
systems create a broad picture for users that illuminates the relative
coverage, depth, and sophistication of topical systems. The broader
perspective that comprehensive key indicator systems provide can also
help identify new areas where topical indicators are needed.
One of the U.S. national topical indicator systems is Healthy People (a
federal effort led by the Department of Health and Human Services).
This system provides a set of national health objectives, along with
indicators to measure progress, which are revisited every 10 years. It
also highlights 10 leading health indicators, such as physical
activity, overweight and obesity, tobacco use, and substance abuse.
Since it was established in 1979, Healthy People has engaged a diverse
group of stakeholders throughout the country, including a Healthy
People Consortium. The Healthy People Consortium is a group of public
and private organizations that is dedicated to taking action to achieve
the Healthy People agenda. Further, most states have their own Healthy
People plans.
Comprehensive Key Indicator Systems Are Active, Diverse, and Evolving:
The comprehensive key indicator systems GAO studied each bring together
diverse sources of information to provide an easily accessible and
useful tool for a broad variety of audiences and uses. The Boston
Indicators Project, for example, brings together a set of indicators
from sources such as the U.S. decennial census, state and city
agencies, nonprofit organizations, and universities. It groups the
indicators into categories and established goals in these
areas.[Footnote 7]
These systems are oriented toward both public and private choices. They
incorporate individual and institutional perspectives and address a
wide range of audiences, including business, nonprofit, government, and
media users, as well as the general public. A small business owner of a
company that provides health care services, for example, might use
information from an indicator system to investigate market
opportunities in a particular geographic area or demographic group. A
foundation or nonprofit could use indicators regarding the status of
children's education, health, and family environment to inform
decisions to fund certain grant applications. Information from
comprehensive key indicator systems could be used to help government
leaders establish priorities and allocate scarce public resources. They
can also help individuals understand more about issues that affect
their life choices, such as how progress in community development,
public safety, and education could affect where they might want to
live.
Comprehensive Key Indicator Systems Are Oriented Primarily toward
Learning or Outcomes:
GAO found that comprehensive key indicator systems are primarily, but
not exclusively, either learning-oriented or outcome-
oriented.[Footnote 8]
Some systems are oriented more toward learning and information
exchange. The indicators in these systems are primarily selected based
upon the information needs of their target audiences and are grouped
into categories without specific links to outcomes or goals.
Information is often presented on Web sites with limited commentary or
analysis of results. The Social Assets and Vulnerabilities Indicators
(SAVI) system in Indianapolis is an example of a learning-oriented
system. It collects, organizes, and presents information on "community
assets," such as schools, libraries, hospitals, and community centers.
It also includes indicators in areas like health, education, and
criminal justice that highlight "vulnerabilities," such as
neighborhoods with high crime or unemployment. Learning-oriented
systems enable citizens, researchers, and leaders to learn more about
and monitor conditions in their jurisdictions and may help inform
decision making.[Footnote 9]
Other comprehensive key indicator systems encompass an outcome-oriented
focus on societal aspirations or goals. These indicator systems are
used to monitor and encourage progress toward a vision for the future-
-or in some cases a specific set of goals--which have been established
by the people and institutions within a jurisdiction. Most of the
systems GAO studied were outcome oriented. One of these, the Oregon
Benchmarks system, measures progress toward a strategic vision and
related goals for the state, known as Oregon Shines. It is organized
around three broad goals (1) quality jobs; (2) safe, caring, and
engaged communities; and (3) healthy and sustainable surroundings; each
of which has specific objectives. Under the goal for safe, caring, and
engaged communities, for example, Oregon has a specific objective to
decrease the number of students carrying weapons, measured by the
percentage of students who report carrying them (based on a state wide
survey).
Attention to Relevant Issues, Aspirations, and Questions Is Important
in the Development and Evolution of Comprehensive Key Indicator
Systems:
GAO's work showed that an orientation toward outcomes--whether outcomes
were formative and implicit or advanced and explicit--had an important
influence on focusing and facilitating the development of the system.
Audiences are more likely to use information if they see how it is
relevant to their aspirations or interests. Therefore, outcome-oriented
systems can help create focused information for their audiences that
may enhance the use of and continuing support for these systems.
Moreover, broad discussions about strategic issues and opportunities
can help to reframe existing problems in new ways or identify important
gaps in knowledge about certain issues or populations. The notion of
progress assumes some agreement on the most important questions,
issues, or opportunities facing a jurisdiction. The civic dialogue and
processes used to reach common ground in the systems GAO studied were
often extensive, complex, and time-intensive. Such processes are a pre-
requisite for initiating, and are critical in sustaining, any
comprehensive key indicator system.
Lessons Learned and Implications: Comprehensive Key Indicator Systems
Are a Noteworthy Development with Potentially Broad Applicability:
Comprehensive key indicator systems add a dimension of information
about society that is currently not available to most people. The 29
systems GAO studied showed evidence of positive effects, such as
improving decision making, enhancing collaboration on issues, and
increasing the availability of knowledge. These systems, although very
diverse, encountered similar challenges and applied many of the same
design features. Because GAO found systems at all levels of society,
including other nations, this demonstrates the potential for
transferability--meaning that approaches used in other jurisdictions
may be adapted and used elsewhere. Thus, the development and use of
comprehensive key indicator systems has the potential for broad
applicability throughout the United States at the subnational and
national levels.
Comprehensive Key Indicator Systems Showed Some Evidence of Positive
Effects:
GAO found that comprehensive key indicator systems showed evidence of
positive effects in four areas. They enhanced collaboration to address
public issues, provided tools to encourage progress, helped inform
decision making and improve research, and increased public knowledge
about key economic, environmental, and social and cultural issues.
These positive effects are a function of how different stakeholders use
indicators (along with other resources and information) within the
context of various political, economic, and other factors. Individuals,
the media, businesses, non-profits, interest groups, professionals, and
governments, among others, all may play a role in influencing ideas,
choices, and actions. Thus, it is difficult to attribute actions
directly to an indicator system. In several cases, these systems
generated information that appeared to spur action and produce positive
effects in the short term. It can take years, however, for an indicator
system to become a widely used and effective tool.
Enhanced Collaboration to Address Public Issues:
By revealing significant public policy problems or raising the profile
of new, divisive, or poorly understood issues, comprehensive key
indicator systems can help spur or facilitate collaboration. Focusing
attention on a particular condition may bring increased pressure to
bear on diverse parties in the public and private sectors to
collaborate on strategies for change. Providing a common source of
information also facilitates a shared understanding of existing
conditions.
The Chicago Metropolis 2020 indicator report, for example, highlighted
the region's severe traffic congestion and its effects. This report was
a key factor leading to the formation of a task force of public and
private leaders, supported by the state's governor and legislature, to
deal with transportation problems in the Chicago metropolitan region.
The task force recommended actions intended to transform transportation
and planning agencies into a more coherent regional system, which are
under consideration.
Provided Tools to Encourage Progress:
Users of comprehensive key indicator systems found that they provide an
effective tool for monitoring and encouraging progress toward a shared
vision or goals. Some jurisdictions used information from these systems
to assess the extent to which various parties, including government
agencies, not-for-profit organizations, and businesses, contributed to
achieving results.
For instance, the European Structural Indicators system helps officials
determine how well countries in the EU are meeting agreed-upon policy
goals that are spelled out in the Lisbon Strategy. Spotlighting each
country's progress, or lack thereof, in an annual, publicly released
report encourages each country to improve its performance, which could
then raise the overall position of the EU.[Footnote 10] When the EU
determines, based on a review of the related indicators, that a member
country has not made sufficient progress toward a particular goal, it
can recommend specific actions to help further that country's progress.
Some countries have changed their policies in response to EU
recommendations, such as Spain, which has agreed to take steps to raise
its employment rate among women.
Helped Inform Decision Making and Improved Research:
Bringing relevant information together in a single resource helps
leaders, researchers, and citizens to easily access and use it.
Therefore, comprehensive key indicator systems--if they are viewed as
credible, relevant, and legitimate--provide the capacity for many to
work from, and make choices based upon, the same source of reliable
information. This also enhances efficiency by eliminating the need for
individuals or institutions to expend additional time and resources
looking for or compiling information from disparate sources.
Researchers, for example, could more easily determine what knowledge
exists to help identify existing or new areas meriting further study.
In Indianapolis, officials from the Social Assets and Vulnerabilities
Indicators system (SAVI) provided input, based upon the system's
economic, public safety, demographic, and program indicators, on where
to locate a new Young Men's Christian Association (YMCA) facility for
the city. SAVI used its indicators to map areas of need and found that
numerous parts of the city were equally in need of better recreation
and educational facilities. That is, no one part of the city was a
clear-cut choice based on analysis of the indicators. As a result, the
YMCA made a decision to not construct a single new building. Instead it
created a "YMCA Without Walls" program offering a variety of new
services throughout the city in existing facilities, such as churches,
schools, and community centers.
Increased Knowledge about Key Economic, Environmental, and Social and
Cultural Issues:
Comprehensive key indicator systems allow users to better understand
the interrelationships between issues that may not have been apparent
when viewed separately. New insights may also result from looking at
economic, environmental, and social and cultural information from
crosscutting perspectives (e.g., opportunity, equity).
Further, comprehensive key indicator systems helped expose information
or knowledge gaps about significant issues. These gaps may result from
(1) the absence of information; (2) inadequate knowledge about the
interrelationships among various indicators (e.g., the impact of
economic development on crime rates); or (3) a poor understanding about
the conditions of certain population groups. As a result, indicator
system providers and users can help spur new data collection efforts or
redirect existing efforts to reduce gaps and increase knowledge.
For example, when developing the Compass Index of Sustainability (in
Orange County, Florida), gaps were identified in knowledge about the
county's aging population. Neither government agencies nor other
organizations were collecting adequate data on the health and well-
being of aging residents. The system's report commented on these gaps,
leading county commissioners to appoint a task force. The task force
reviewed existing data collection efforts and recommended improvements
that are now underway, thereby increasing knowledge about a major
segment of the population.
System Costs Are Difficult to Quantify:
Most of the systems GAO studied are located in larger organizations or
agencies and the reported costs dedicated to developing, implementing,
and sustaining them are difficult to quantify. Because the system
managers were able to borrow or leverage staff and resources from their
parent organizations, the full costs of the time and effort to develop,
implement, and sustain these systems were not fully captured. In most
cases, one to three persons worked on the project full-time. For
example, one person (in the city's Department of Public Works) manages
Santa Monica's Sustainable City indicator system. Further, because
these systems rely primarily on indicators or data collected by others,
the costs incurred by others to collect data generally are not
reflected as part of an indicator system's costs.
According to officials from the systems GAO reviewed, systems'
significant cost items included acquiring and managing technology,
paying staff and consultants, and printing and distributing reports.
For example, representatives of the Southern California Association of
Governments' State of the Region system said that they dedicated
approximately $200,000 for their system's 2002 annual indicators
report. Of this amount, approximately $25,000 went to printing the
reports, which were distributed to various officials, academia,
businesses, and nonprofit organizations in southern California. The
rest of the funding was for two staff members and related costs to
draft and process the report. This cost structure was for the most part
consistent with the other systems GAO studied. However, any variation
in costs in relation to the size of the population covered by the
system has not yet been determined.
Certain Design Features Are Needed to Overcome a Range of Key
Challenges:
GAO identified a number of challenges experienced by the 29
comprehensive key indicator systems it reviewed and identified nine
common design features they exhibited. The nature of these challenges,
as well as the ways in which the design features were applied, varied
based on factors such as the system's size, purpose, target audiences,
and the jurisdiction's political and economic structures.
The primary challenges that systems experienced included (a) gaining
and sustaining stakeholders' support, (b) securing and maintaining
adequate funding, (c) agreeing on the types and numbers of indicators
to include, (d) obtaining indicators or data for the system, and (e)
effectively leveraging information technology. Many of these challenges
are continuous and interrelated. For example, challenges in obtaining
indicators or data for the system are exacerbated when systems have
difficulty maintaining adequate funding.
To address these challenges up front and help ensure a lasting, well-
used system, GAO's work in the United States and around the world
strongly suggests that the development of a comprehensive key indicator
system at any geographic level--including a U.S. national system--would
benefit from considering and applying these nine design features. At
the outset, establishing a clear purpose and defining a target audience
and its needs are most crucial. Decisions about how to incorporate
other important features into the system's design should follow
decisions about purpose and target audience.
1. Establish a Clear Purpose and Define Target Audiences and Their
Needs:
Deciding whether the system will focus primarily on allowing users to
learn more about the conditions of their jurisdiction, or whether it
would also measure progress toward specific outcomes, is a first step
in designing a comprehensive key indicator system. Another important
factor is whether to design the system for a specifically targeted
audience, such as government policymakers, or for a wider audience,
including business leaders, researchers, not-for-profit organizations,
the media, and citizens. The media are an especially critical audience
because of the role they often play in conveying the information
presented in indicator systems to the general public.
2. Ensure Independence and Accountability:
It is important to insulate comprehensive key indicator systems from
political pressures and other sources of potential bias as much as
possible. When indicator systems are perceived as biased toward a
particular ideological or partisan perspective, the indicators are less
likely to have credibility and may lose support from a broad group of
users. Mechanisms for helping to ensure transparency and accountability
to stakeholders include demonstrating that the system's managers are
achieving the indicator system's stated aims, using scarce resources
effectively, remaining independent from political processes, and
emphasizing problem areas or opportunities for improvement.
3. Create a Broad-Based Governing Structure and Actively Involve
Stakeholders:
A comprehensive key indicator system should be governed by a structure
that includes a blend of public and private officials and represents
views from various communities.[Footnote 11] The system's governing
officials typically make decisions about how to apply and implement the
design features and set the policies for the system's staff to follow,
including what products and services will be provided. The challenge of
gaining and sustaining support is continuous, even among systems with
champions or large user bases. A governing structure representing
various interests can help ensure that the system maintains a balanced
perspective to meet diverse needs and avoid "capture" by one party or
particular interest group.
4. Secure Stable and Diversified Funding Sources:
Securing adequate funding to initiate the system and sustain it over
time is a constant challenge. One way to help ensure that funding
remains stable over time--and an important aspect of maintaining
independence of the system--is to diversify the number and types of
funding sources. GAO found that a lack of diversified funding sources
made indicator systems more vulnerable to fiscal constraints. Systems
that relied on multiple funding sources, such as government, corporate,
and non-profit foundations, could make up for reductions from one
source by turning to others.
5. Design Effective Development and Implementation Processes:
It is critical to have transparent, collaborative, and repeatable
processes in place to effectively carry out basic functions of a
comprehensive key indicator system, including, but not limited to:
* developing and modifying an organizing framework for the indicators,
* selecting and revising the indicators on an ongoing basis,
* acquiring indicators or data to compute indicators as needed,
* engaging data providers,
* assessing the quality and reliability of the indicators or data, and:
* seeking and maintaining funding.
For example, many of the indicator systems GAO reviewed established
criteria for facilitating the process of selecting indicators, such as
relevance, comparability, and reliability. Selecting indicators is
particularly challenging because it involves making subjective
judgments about, and reaching agreement on, the relative importance of
issues to a jurisdiction.
6. Identify and Obtain Needed Indicators or Data:
Comprehensive key indicator systems often report on indicators or use
data that are originally collected by others. Identifying and gaining
access to indicators or data that are controlled by other organizations
is critical to these systems. Some systems have established formal
processes that specify how they will use the data and when and in what
form they will receive the data from providers. In addition to having
legal authority to access the information, the system should have
responsibility, including legal responsibility, for protecting the
privacy of the information when necessary.
7. Attract and Retain Staff with Appropriate Skills:
Systems cannot operate effectively on a day-to-day basis if they do not
have staff with appropriate skills and abilities. The ability to
collaborate with diverse stakeholders is a fundamental requirement.
Systems also need to involve people with a wide variety of skills and
knowledge in areas including statistics, information technology
management, and marketing. Working knowledge and experience with key
economic, environmental, and social and cultural issues are also
important.
8. Implement Marketing and Communications Strategies for Target
Audiences:
Reaching diverse audiences, including the print and electronic media,
requires multifaceted marketing and communications strategies. These
strategies spread the word about the existence and features of the
system; disseminate information on what the indicator trends are
showing; help to encourage a broader base of individuals and
organizations to make use of the system; and provide training and
assistance to users.
9. Acquire and Leverage Information Technologies:
The development of advanced information technologies (e.g., the World
Wide Web) has transformed the tools available for comprehensive key
indicator systems, although the extent to which systems have leveraged
these technologies varied in the systems GAO reviewed. According to
many of the system managers, effectively using technology, including
the Internet, has made it possible to transfer data quickly,
disseminate it economically, and make it more widely available.
However, gaining access to new technologies can be costly and requires
staff or users to have technical expertise.
Comprehensive Key Indicator Systems Have Potentially Broad
Applicability:
Comprehensive key indicator systems exist across all levels of society,
and GAO's review of selected systems indicates that these systems have
potentially broad applicability. They exhibit similar features that can
be transferred and adapted by other systems, and have years of
experience from which to draw. Further, existing mainstream information
technologies have lowered costs of distribution and increased the
methods available to make information more accessible and usable. Other
developed nations already have comprehensive key indicator systems.
Several specific factors demonstrate the feasibility for a U.S.
national system.
Strong Foundations. Since comprehensive key indicator systems for the
most part aggregate existing indicators to enhance dissemination and
usage, a U.S. system has a large body of indicators from which to
select. An array of existing topical indicator systems are continually
evolving and developing broader conceptions of how to understand and
assess a society's position and progress.
Demonstrated Scalability and Comparability. GAO has found working
systems at all levels of society in the United States and abroad,
including neighborhoods, communities, cities, regions, states,
nations, and supranational entities. They range from small population
scales in the millions to the largest system GAO studied, the EU, at
over 450 million.[Footnote 12] Hence, a system for the U.S. population
of over 290 million is potentially feasible.
Evidence of Transferability. Elements from existing systems are being
adapted by new entities to meet specific needs and interact with one
another, especially at the local levels in the United States. For
example, the Boston Foundation has developed technology and processes
that could be used by other cities, and a group of organizations in
Dallas has developed a comprehensive key indicator system (Dallas
Indicators) that is, in part, based on the Boston Indicators Project.
Hence, there is abundant knowledge and expertise at varying scales that
could be applied, with recognition of unique factors, to a U.S.
national system.
Credible Activity. There is a significant amount of activity and
interest across the United States in further developing and sharing
information on comprehensive key indicator systems that could
contribute to and complement a national system. Moreover, the Key
National Indicators Initiative is currently in the process of planning
a national comprehensive key indicator system for the United States.
Observations and Next Steps: Congress and the Nation Have Options to
Consider in Taking Further Action:
The United States confronts profound challenges resulting from a
variety of factors, including changing security threats, dramatic
shifts in demographic patterns, increasing globalization, and the
accelerating pace of technological change. Addressing these challenges
will likely depend on information resources that better portray a broad
picture of society and its interrelationships.
However, in light of the United States's large supply of topical
indicators, a natural question is: If the nation has so much
information on so many issues, why does it need a comprehensive key
indicator system? One answer to this question is that having
information on all the parts--while important and necessary--is not a
substitute for looking at the whole, whether in life, business,
science, or governance and politics.
A National Indicator System for the United States Merits Serious
Discussion:
It appears feasible to create a comprehensive key indicator system for
the nation that provides independent, objective, and usable information
on the nation's position and progress. If designed and executed well, a
national comprehensive key indicator system could have wide impact--
that is, if American citizens, leaders, and institutions pay attention
to it, access it, and use it to inform their personal and professional
choices. Alternatively, if it is poorly planned and implemented, the
effort could absorb scarce time and resources, fail to meet
expectations, and might even make it more difficult to create such a
system in the future.
The potential positive benefits of a U.S. comprehensive key indicator
system could include the ability to:
* highlight areas in which progress has been made in improving people's
living conditions as well as areas needing new or higher levels of
public attention;
* connect debates about the relative merits of competing demands with
reliable indicators to help make choices among competing priorities and
direct resources where they have the most impact;
* provide information about the possible impact of particular
interventions and policies, thereby providing greater accountability
and learning;
* facilitate comparisons within the United States or of the nation as a
whole with other countries;
* accelerate the identification of important gaps in the nation's
knowledge about important issues and populations;
* enhance fact-based consensus on issues and aspirations, thereby
devoting more time, energy, and resources to discussing priorities and
effective solutions;
* provide more people and institutions with an accessible "window" into
the nation's critical sources of information, thereby increasing the
return on the large investments that have already been made to collect
it; and:
* at the federal level, inform a much-needed re-examination of the base
of existing programs, politics, functions, and activities as well as
the mandated creation of a governmentwide performance plan.
However, there are some pitfalls that a key national indicator system
would need to avoid. First, because there are some areas where
indicators or data may not exist (e.g., certain aspects of the
environment) or are difficult to measure (e.g., certain aspects of
culture), a key U.S. indicator set could have an implicit bias towards
areas with existing measures. It will be important for the nation to
focus on what it needs to measure, not just on what it currently
measures. Second, poor indicator selection or lack of attention to data
quality, in the context of such a highly visible system, raises the
risk in terms of possible misinformation or unintended consequences
arising from use of the system. Finally, exploring a broad number of
creative solutions to the problem of how to better inform the nation--
including the possibility of competing efforts--may help to encourage
faster or more robust development. A single system, if not designed to
be open and innovative and implemented in such a fashion, could
restrain innovation.
Comprehensive Key Indicator Systems Could Help Better Inform the Nation
at Many Levels:
One distinguishing characteristic of the United States is unity built
out of diversity. This diversity finds its expressions in the multiple
levels and branches of government, the different sectors of economic
and social activity, the varied geographic regions, and the widely
ranging racial, ethnic, professional, cultural, and other communities
of interest. Accordingly, questions about a national system from a
local, state, or regional perspective might include the following: Can
it provide specific or contextual information, at an appropriate level
of disaggregation (e.g., geographic areas or population subgroups) that
helps localities, states, and regions become better informed?
Alternatively, how could a U.S. national comprehensive key indicator
system help subnational jurisdictions better understand themselves in a
national context?
A comprehensive key indicator system for the entire United States could
be designed in different ways. It could express only national-level
indicators (e.g., the average national unemployment rate) and
coordinate with subnational levels and others as they develop their own
comprehensive key indicator systems with more localized information.
Experts GAO talked with made it clear that this is an achievable aim
and would add value.
Alternatively, a national system could also include some capability for
users to get not only national-level information but also information
for geographic areas and demographic subgroups (e.g., unemployment
rates for metropolitan areas or school achievement levels for certain
population groups). Experts said that, due to availability and
comparability issues, limited progress toward such capabilities would
be possible in the short term. Much more work must be done to determine
how much flexibility in comparison and disaggregation could be built
into a single national system over time, versus what would be available
in separately managed databases.
Congress Could Choose from a Range of Organizational Options as
Starting Points for a U.S. National System:
The basic issue for Congress, or any other entity or jurisdiction
considering a comprehensive key indicator system, concerns who is to
develop, implement, and manage the system. It is important to note that
the specific organizational option Congress or any other decision maker
chooses as a starting point may be less important than ensuring that it
incorporates the nine key design features presented in this report.
GAO identified three basic organizational options for a U.S.
comprehensive key indicator system. Each option would allow for
incorporation of all or most of the nine design features but to varying
degrees: (A) a public organization, (B) a private organization, or (C)
a combination public-private organization. There are advantages and
disadvantages to each option.
Regardless of which option is chosen, the organization would need to
involve public and private individuals and institutions. Assessing the
position and progress of a market-oriented democracy like the United
States would benefit from aggregating both publicly and privately
produced information for two reasons. First, private sector providers
produce much useful information (e.g., attitudinal data on consumer
confidence). Second, much of the information collected by federal
agencies is tied directly to functional or programmatic purposes and,
therefore, is generally focused on areas where the government has
traditionally played a role. As a result, the federal government's
statistical programs could be supplemented with information collected
by others as the nation evolves and attempts to meet emerging
challenges in new ways. In addition, public and private institutions,
individuals, and a wide variety of groups have an interest in being
engaged in a national comprehensive key indicator system so that it
will meet their needs. Finally, public sector institutions that
currently provide indicators rely heavily on data collected from
private individuals or institutions. All of them have an interest in
seeing more available and accessible information in return for their
time, expense, and energy.
Option A: A Public Organization:
A national comprehensive key indicator system could be led by a federal
agency or a component of a larger agency or department. This option
would entail operating as either (1) a new organization within an
existing agency, (2) a completely new agency, or (3) an added
responsibility in the mission and activities of an existing agency. In
terms of advantages, a public organization could build upon the vast
institutional capacity and skills within the federal government.
Difficulties involved in mixing official and unofficial statistical
information would be a disadvantage for a public organization. It could
also be constrained by federal management and human capital policies.
The U.S. Census Bureau illustrates some of the main features of a
publicly led option. It is one of the main federal statistical
agencies, with an extensive statistical infrastructure and skill base.
As such, it provides an example of a potentially viable option for
housing a national system in an existing agency.
[End of table]
Option B: A Private Organization:
Another option would be to identify or charter a private organization
to develop and implement a national system. A private, non-profit
organization would be better suited than a for-profit organization to
develop a widely accessible, independent system. A common type of
congressionally chartered organization that would be an appropriate
venue for a national system is the federal Title 36 corporation. It
provides some degree of prestige and indirect financial benefits in
that it can receive federal funding, along with private gifts and
bequests. Federal supervision of such organizations is very limited as
these organizations are set apart from the executive and legislative
branches. In terms of advantages, a private organization would be more
adaptable and have flexibility in soliciting donations from a range of
sources and developing its management and human capital policies. A
disadvantage is that a private organization would be disconnected from
political appropriations and authorization processes, possibly making
it more difficult to encourage policymakers to accept and use the
indicator system. The National Academy of Sciences (NAS) is an example
of a Title 36 organization chartered by Congress. NAS is noted for its
reputation of providing independent, scientific information to the
nation, and provides an example of a potentially viable option to house
a national system in a private organization.
[End of table]
Option C: A Public-Private Organization:
Under the third option of a public-private organization, Congress would
have a great deal of flexibility in designing a unique organization and
selecting from a range of possible features. Congress would need to
decide which existing laws, such as the Privacy Act, should apply.
Advantages would include the opportunity to build on the capabilities
of the federal government while retaining the ability to more easily
adapt to changing circumstances. The mix of public and private
interests could also help balance the critical need for independence
with important connections to the political process. Of course, public-
private organizations are not immune to political pressures and would
need to build institutional processes and a culture focused on quality
and independence. Further, some risks that the organization would
overlap or compete with existing federal functions are possible even if
the organization is carefully structured. In designing a public-private
organization, various entities serve as possible models, including the
Smithsonian Institution (although it is not a viable option to house
such a system). The Smithsonian Institution is a hybrid organization
that is publicly supported and privately endowed, illustrating the
degree of flexibility Congress would have in establishing a public-
private partnership to house a national system.
[End of table]
Choosing a New or Existing Organization Carries Certain Advantages and
Disadvantages:
Unlike existing organizations, the most significant disadvantage for a
new organization is the difficulty of incubating it--that is, getting
it off to a successful start. The challenges of funding, establishing
networks internally and with key external communities, new operating
policies and procedures, and human capital issues are all more
difficult in a start-up situation. In addition, it is more difficult to
build awareness, trust, and credibility. However, a new organization
also provides the opportunity to make a fresh start and design an
organization that suits the key design features and enhances the
likelihood that it will become a long-lasting, well-used indicator
system.
A New Public-Private Organization Could Offer Greater Flexibility to
Apply Design Features:
A new public-private organization could facilitate collaboration among
a variety of communities and combine the best features of federal
support and engagement. Congress could incorporate flexibilities by
selectively determining which federal management and human capital
policies would apply to the organization. A public-private organization
could solicit both public and private funds, or it could be designed to
coordinate the separate actions of a few leading public and private
institutions. Most of the experts GAO interviewed believed that a
public-private partnership would probably be the best venue for a
national system. However, comprehensive key indicator systems could
begin by being housed in any of the three organizational options
discussed in this report. GAO found no significant reason why any
option should be ruled out, especially as a starting point.
From a broader national perspective, other jurisdictions throughout the
United States that are considering development of a comprehensive key
indicator system have similar options from which to choose. Unique
aspects and applications of local, state, and national laws, culture,
economic conditions, and considerations about existing organizations
and operations will affect which organizational option is best suited
for a particular jurisdiction. GAO's work revealed that lasting
comprehensive key indicator systems existed in a range of
organizational formats in jurisdictions throughout the United States,
from strictly public systems, such as the Oregon Benchmarks, to those
housed in private, nonprofit organizations, such as Chicago 2020.
Next Steps for Congress and the Nation:
In addition to Congress and the executive branch at the federal level,
there are many providers and users of information in thousands of
jurisdictions who could benefit from the findings in this report.
Accordingly, GAO's suggested next steps are addressed to a broad
audience around the nation.
Encourage Awareness and Education:
Expanding efforts to make leaders, professionals, and the public more
aware of comprehensive key indicator systems and their implications
could enhance discussions and enrich considerations about their
significance and potential application. Specific actions could include
conducting briefings, workshops, or media events; convening forums or
conferences; or holding congressional hearings.
Pursue Additional Research:
As it is becoming more feasible for jurisdictions to create such
systems, more research should be encouraged. Research conducted thus
far on these systems has shown that many questions remain, such as how
much time, money, and effort are required to create them and are they
worth it? A common research agenda, developed among interested parties,
would be of value. Learning more about large-scale systems, such as
those in other nations, would help inform the development of a possible
U.S. national comprehensive key indicator system.
Support Further Development of Comprehensive Key Indicator Systems:
A high degree of innovation is taking place at local levels, which can
help in building the nation's body of experience and inform
considerations at the state and national levels. One way to enhance the
improvement of existing systems and increase the probability of
successful new ones would be to institutionalize a national network of
practitioners and experts. The regular exchange of knowledge in such a
community of practice could reduce risks, expand opportunities, and
avoid reinventing solutions by leveraging accumulated expertise.
Widen the Dialogue on Options for a U.S. National System:
It is important to initiate a broader dialogue on the possible
development of a national comprehensive key indicator system that would
include Congress, the administration, other levels of government, and
different sectors of society. Such a dialogue should explore potential
benefits, costs, risks, and opportunities involved. Engaging interested
parties across the nation would help ensure collaboration across
boundaries, leverage existing information assets, build on existing
knowledge and experience, and position the nation to make choices about
whether and how to develop a national comprehensive key indicator
system for the United States.
[End of section]
Chapter 1: Introduction:
Difficult decisions related to societal aims, such as improving health
care, enhancing security, or sustaining the environment require
reliable, unbiased, and useful indicators that are readily accessible
to citizens, the media, advocates, businesses, policymakers, nonprofit
leaders, researchers, and other audiences. While in many ways such
information about the world is more available today than ever before,
too often it is in diverse formats and locations that may make it
difficult to locate and use effectively and to provide a general
picture of a jurisdiction's position and progress. In addition, it is
not easy to ensure that the most relevant and important information is
accessible, recognized, and used by a wide variety of people and
institutions. As a result, public and private decision making about
issues and solutions may be based on information that is limited,
fragmented, and incomplete.
One example where progress has been made is a single entry point for
federal statistical data ([Hyperlink, http://www.fedstats.gov]), which
gives access to statistics from over 100 federal agencies, available by
both state and topical area. It is a valuable resource for
professionals and those who need information on a specific topic.
However, the site does not provide access to a limited number of
indicators that have been agreed upon as important for understanding
and assessing the position and progress of the United States. Further,
it is not designed to allow a user to easily assemble indicators in
multiple topical areas at the same time, navigate easily through
different areas, or interact with the system for different purposes
(e.g., producing a report). Because the site links directly to agency
Web sites, a wide variety of formats exist and users must also navigate
within each agency's site to find desired information.
The nation's challenges at all levels demand new and more cross-sector,
cross-border responses involving many different individual and
institutional participants in U.S. society. These responses, in turn,
depend on more integrated information resources to support informed
public debate and decisions within and between different levels of
government and society. For example, individuals and institutions play
multiple roles in life (i.e., resident in a particular neighborhood and
borough in New York City, resident of the city itself, resident of the
State of New York, and citizen of the United States), illustrating one
reason why the interrelationships between indicator systems are
important.
Looking at the parts of a society is no substitute for viewing the
whole. Along these lines, there are examples of citizens, institutions,
and leaders, in both private and public roles and settings, that have
comprehensive key indicator systems. Such systems bring together a
select set of indicators that provides information conveniently in one
place on a broad range of topical areas, such as economic development
and employment, air and water quality, and public health and education.
We use the term comprehensive to denote systems that include indicators
from each of the three following domains: economic, environmental, and
social and cultural.[Footnote 13]
Organizers and users of comprehensive key indicator systems attempt to
address questions such as: What are our most significant challenges and
opportunities? What are their relative importance and urgency? Are we
making optimal choices to allocate scarce public resources, create
jobs, stimulate future industries, maintain a global competitive edge,
enhance security, sustain environmental health, and promote quality of
life considerations? Are our solutions working and compared to what?
How do we really know if they are working?
Importantly, indicator systems are oriented toward both public and
private choices; individual and institutional perspectives; business,
nonprofit, government, and media points of view; and leaders, voters,
and employees. Their intent is to improve the availability of quality
information for better decision making and problem solving. For
example, a small business owner could use such a system to investigate
market opportunities in particular geographic areas or among certain
demographic groups. A foundation might use the information on the
status of children's education, health, and family environment to make
decisions about competing grant applications. Policymakers in
government might use such information to inform priorities and allocate
scarce public resources.
Indicators and Indicator Systems:
An indicator is a quantitative measure that describes an economic,
environmental, or social and cultural condition. There are many widely
known indicators, such as the unemployment rate. Yet, there are many
more indicators that are less widely understood but of comparable
importance. For example, the number of patent applications or patents
granted in a particular industry or jurisdiction[Footnote 14] is
sometimes used to measure the degree of "inventiveness." Such an
indicator can be useful to businesses seeking to locate in places with
highly educated and creative potential employees. An indicator such as
this one could also be useful for assessing relative competitive
advantage in research and development.
The indicators related to unemployment and patent applications
illustrate another difference between indicators--direct vs. indirect
or "proxy" indicators. Experts in the field of statistics emphasize
this distinction because it highlights things that are difficult to
measure. A direct indicator measures exactly what it says it does--in
this case the unemployment rate. In contrast, an indirect indicator,
such as the number of patents, cannot directly measure inventiveness.
In fact, it may be impossible to measure such a concept directly and it
is possible that it could only be approximated through a variety of
quantitative proxy measures.
In this report, we define "indicator systems" as systematic efforts to
institutionalize the provision of indicators through various products
and services to satisfy the needs of targeted audiences. Indicator
systems measure many things, including attributes of people,
institutions, industries, and the physical environment, among others.
In terms of management and ownership, many topical indicator systems in
the United States are primarily public in character, such as the
National Income and Product Accounts maintained by the Bureau of
Economic Analysis. Others are privately led, such as the Institute for
Survey Research at the University of Michigan, which produces consumer
confidence indicators.
Indicators are based on data collected from suppliers (e.g.,
individuals and institutions that fill out surveys or census forms),
which can then be designed and packaged into products and services by
providers (e.g., the Bureau of Labor Statistics or the Conference
Board) for the benefit of various users (e.g., leaders, researchers,
planners, or voters). Audiences can use the information packaged in an
indicator system for a variety of reasons: to stimulate awareness,
increase understanding, frame points of view on issues, plan
strategically, assess progress, or make choices.
Indicator systems also vary to the degree that they focus on (1)
detailed account structures (e.g., the U.S. National Income and Product
Accounts); (2) portfolios of individual indicators; (3) single
composite indices that are constructed out of many individual
indicators (e.g., the U.S. Index of Leading Economic Indicators); or
(4) some combination of the above.
Further, indicators are only one part of the base of knowledge and
information necessary to inform a nation. They are important for
summarizing, highlighting, and synthesizing what can sometimes be
complex and bewildering information for many audiences. However, they
must be supported by more extensive databases to support analysts who
want to probe into a deeper understanding of the reasons for movements
in certain indicators.
Topical and Comprehensive Key Indicator Systems:
It is useful to distinguish between two types of indicator systems:
topical and comprehensive. "Topical indicator systems" consist of
indicators pertaining to a related set of issues, such as health, water
quality, education, science, technology, or transportation. For
example, a topical system in health might have related indicators like
the prevalence of certain diseases, such as cancer or heart disease;
levels of certain risk behaviors, such as cigarette smoking or drug
use; the number of citizens with access to health insurance; and the
number of doctors or hospitals available for use by citizens in a
particular jurisdiction. Topical indicator systems exist at different
geographical levels, including local, state, regional, national, and
supranational. They are a major source of information for the media,
professionals, researchers, citizens, and policymakers.
In contrast with topical systems, comprehensive key indicator systems
aggregate key economic, environmental, and social and cultural
indicators into a single system that disseminates information products
and services. Comprehensive key indicator systems are built selectively
by members of a jurisdiction from the foundation of many existing
topical indicators. Indicator systems have an institutional foundation
to sustain and improve them over time. Comprehensive key indicator
systems can make it easier to see a more complete, general picture of
the position and progress of a particular jurisdiction without
requiring the review of exhaustive detail. These comprehensive systems
also facilitate analysis and our understanding of how changes in one
domain can affect other domains. For example, public health (which
would be included in the social and cultural domain) may also be
affected by both economic and environmental factors.
Selecting the key aspects or activities of a society that are most
important to measure is a challenge for comprehensive key indicator
systems. Citizens of any jurisdiction view the world differently based
on their culture, geography, aspirations, values, and beliefs, among
other factors. Diverse perspectives and value judgments significantly
affect indicator choices and definitions, which are inherently
subjective. For example, poverty is a characteristic of society that is
frequently monitored, and it can be defined and measured in a number of
ways. The proportion of the population that is low income can be
selected as one indicator of poverty, which frames it in financial
terms. However, other possible indicators, based on nonfinancial
factors like physical, psychological and spiritual well-being and
education levels, also could be considered as broader indicators of
poverty.
Focus of U.S. National Topical Systems on Specific Issues:
The United States has national-level indicator systems in a variety of
topical areas, most of which are supported by the federal statistical
system. Because of the natural interrelationship between topical and
comprehensive systems, GAO included five U.S. national topical systems
in our study to provide context, including (1) the Conference Board's
Business Cycle Indicators,[Footnote 15] (2) the National Science
Foundation's Science and Engineering Indicators, (3) the Department of
Health and Human Services' Healthy People, (4) the Federal Interagency
Forum on Child and Family Statistics' America's Children: Key National
Indicators of Well-being, and (5) the Federal Interagency Forum on
Aging-Related Statistics' Older Americans: Key Indicators of Well-
being. (See app. I for further details on these systems.) These systems
and others provide a foundation for a national comprehensive key
indicator system as well as lessons learned that would be useful in
developing it. Accordingly, it is important to note the common elements
exhibited as part of the development and implementation of these
topical indicator systems.[Footnote 16] These systems have:
* originated in response to certain national challenges or concerns,
* evolved over time by expanding their scope and refocusing their
activities,
* been used in a variety of ways by the public and private sectors,
* relied heavily upon indicators from the federal statistical system,
* spurred the development of new or different indicators, and:
* enhanced approaches for collecting data.
Economic, Environmental, and Social and Cultural Domains:
The topical indicator systems we examined fell into either the
economic, environmental, or social and cultural domain. For example, at
the national level in the United States, the annual Economic Report of
the President covers several topical areas within the economic domain,
such as business, markets, finance, and employment.[Footnote 17] The
environmental domain includes areas such as natural resources and
ecosystems. The social and cultural domain includes topical areas such
as education and health care.
The following three figures illustrate some indicators that fall under
each domain. First, to illustrate the economic domain, one measure of
growing worldwide interdependence is the total share of world goods and
services that is traded. As shown in figure 2, from 1970 through 2002,
world exports increased from about 12 percent to 24 percent of world
gross domestic product (GDP). Hence, all over the world, people are
depending more and more on other nations to consume the goods they
produce and to produce the goods they, in turn, consume.
Figure 2: An Economic Indicator Showing World Exports of Goods and
Services as a Percentage of World GDP, 1970-2002:
[See PDF for image]
Note: Calculated from International Monetary Fund data.
[End of figure]
To illustrate an indicator in the social and cultural domain, one
indicator of the status of youth in the United States is a measure of
the percentage of persons ages 16 to 24 who are neither enrolled in
school nor working, as shown in figure 3. This indicator provides
information on a transition period for youth when most are finishing
their education and joining the workforce, a critical period for young
people as they are achieving their educational goals and choosing their
career paths. A breakdown of the data by race and ethnic group shows
that the percentage of youth that fall into this category of neither
being in school nor working has been consistently higher for American
Indian, Black, and Hispanic youths than for White and Asian/Pacific
Islander youths since 1986.
Figure 3: A Social and Cultural Indicator Showing the Percentage of
Persons Ages 16-24 Who Were Neither Enrolled in School Nor Working, by
Race/Ethnicity (Selected Years 1986-2003):
[See PDF for image]
Note: Data from Current Population Survey, March Supplement, selected
years 1986-2003, previously unpublished tabulation December 2003.
[End of figure]
As an example from the environmental domain, in 2003 the Environmental
Protection Agency (EPA) published a Draft Report on the Environment
2003 that covered topical areas in this domain, such as air, land, and
water.[Footnote 18] The air quality index, for example, is used for
daily reporting of air quality as related to ozone, particulate matter,
carbon monoxide, nitrogen dioxide, and sulfur dioxide. The higher the
index, the poorer the air quality. When air quality index values are
higher than 100, the air quality is deemed unhealthy for certain
sensitive groups of people. Based on EPA's air quality index data, the
percentage of days across the country on which air quality exceeded 100
dropped from almost 10 percent in 1988 to 3 percent in 2001, as shown
in figure 4.
Figure 4: An Environmental Indicator Showing the Number and Percentage
of Days with an Air Quality Index (AQI) Greater Than 100, 1988-2001:
[See PDF for image]
Source: EPA.
Note: Data used to create graphic are drawn from EPA, Office of Air
Quality Planning and Standards. National Air Quality and Emissions
Trends Report, 1997. Table A-15. December 1998; EPA, Office of Air
Quality Planning and Standards; Air Trends: Metropolitan areas trends,
Table A-17, 2001; (February 25, 2003;
[Hyperlink, http://www.epa.gov/airtrends/metro.html]).
[End of figure]
Significant national-level research has been conducted on topical
systems. For example, the National Academies, which brings together
committees of experts in areas of scientific and technological endeavor
to address critical national issues and advise the federal government
and the public, has conducted extensive research on indicator systems
in the United States and around the world. Specifically, the Academies
has done work in response to several requests from federal agencies
over the past 15 years to develop, evaluate, or propose statistics or
select indicators in fields such as the economy, health, education,
families, the environment, transportation, science, and technology.
Some indicators, however, can be considered under more than one of the
three domains. The number of housing starts, for example, could be
considered under the economic domain, but housing availability also
affects the social and cultural domain, which includes aspects of
quality of life. The health effects resulting from various
environmental conditions provide another example where the distinction
between different domains blurs. A wider perspective is also crucial in
the area of health care, which involves economic as well as social and
cultural indicators. For example, participants in a recent GAO forum on
health care observed that, although a nation's wealth is the principal
driver of health care spending, that wealth alone does not explain the
high level of spending in the United States.[Footnote 19] These
interrelationships point to one of the strengths of comprehensive key
indicator systems--they provide a tool to bring information together
more easily on an ongoing basis. This means they are especially
suitable for assessing increasingly complex, crosscutting issues that
are affected by a wide range of factors.
Comprehensive Systems' Broad Focus on Position and Progress across All
Three Domains:
A comprehensive key indicator system can be defined more specifically
as shown below.
* Comprehensive--Contains information from the three main domains:
economic, environmental, and social and cultural (note that
crosscutting categories such as sustainability do not fit neatly into
one domain). It is comprehensive in the sense that it provides broad
coverage across the three domains.
* Key--A core set of information that a group of citizens has selected
from a much larger range of possibilities. There is no "right" number
of key indicators. How jurisdictions strike the balance between
simplicity and effective coverage can differ widely. An indicator set
can include a few to hundreds of indicators, but it is not intended to
be exhaustive. Because these are a select set, they cannot provide a
full description of the position and progress of a jurisdiction but
rather focus on providing a generally accurate picture of the whole.
* Indicator--Description of an economic, environmental, or social and
cultural condition over time. These indicators can be but are not
necessarily tied directly to goals or formulated as objectives, or have
specific performance targets associated with them.
* System--The products, services, people, processes, and technologies
involved in an organizational form to sustain and adapt the set of
indicators. This refers to a larger set of civic, scientific,
technical, and other processes that involve suppliers (of data),
providers (of indicators), or users (of information).
Although comprehensive key indicator systems are functioning in the
United States at the community, local, state, and regional levels,
limited research appears to have been conducted with comprehensive key
indicator systems themselves as the focus of analysis. Appendix VII
provides a bibliography of some of the existing literature related to
topical and comprehensive key indicator systems.
Figure 1 shown earlier in the summary section of this report
illustrates how a comprehensive key indicator system might integrate
information from the three domains into a single conceptual framework.
Note that this framework also allows for crosscutting indicators that
do not easily fit into one of the three domains. Some comprehensive key
indicator systems are based primarily on broad, crosscutting conceptual
areas, such as quality of life or sustainable development. An example
of an indicator system that is tracking quality of life is the
Burlington Legacy Project of Burlington, Vermont. The Burlington Legacy
Project has calculated a single index of quality of life--referred to
as the genuine progress indicator (GPI) index, which is a composite of
26 economic, environmental, and social and cultural indicators. Figure
5 shows the GPI calculated for Burlington, Vermont; Chittenden County,
Vermont; the State of Vermont; and the United States.
Figure 5: GPI Per Capita for Burlington, Vermont; Chittenden County,
Vermont; the State of Vermont; and the United States, 1950-2000:
[See PDF for image]
Note: See also Costanza, et al., "Estimates of the Genuine Progress
Indicator (GPI) for Vermont, Chittenden County, and Burlington, from
1950 to 2000," Ecological Economics.
[End of figure]
Nations with Comprehensive Key Indicator Systems:
A number of countries, including Australia, Canada, and the United
Kingdom, have comprehensive key indicator systems at the national
level. Some exist at the supranational level, such as the European
Union's (EU) European Structural Indicators system.[Footnote 20]
Although we did not study the Canadian and Australian systems as part
of this review, they nonetheless illustrate how national comprehensive
key indicator systems can be organized.
Canada's Treasury Board maintains an annually updated comprehensive key
indicator system consisting of 20 indicators intended to reflect a
balance of economic, environmental, and social and cultural
conditions.[Footnote 21] This system provides a snapshot of where
Canada stands in comparison with other countries. The Treasury Board's
indicator system complements government departmental reports by giving
Canadians a broad perspective on national performance, providing a
context for assessing the performance of government programs, and
reporting on basic information to support dialogue among Canadians
about future directions in public policy. The Board grouped indicators
into the following four themes.
* Economic opportunities and innovation--real gross domestic product
per capita, real disposable income per capita, innovation, employment,
literacy and educational attainment.
* Health--life expectancy, self-rated health status, infant mortality
and healthy lifestyles.
* Environment--climate change, air quality, water quality,
biodiversity, and toxic substances and the environment.
* Strength and safety of communities--volunteerism, attitudes toward
diversity, cultural participation, political participation, and safety
and security.
Australia's comprehensive system--Measures of Australia's Progress--is
organized around four dimensions of progress with associated topical
areas. System organizers selected a variety of indicators to measure
progress in each of the topical areas. The dimensions and associated
topical areas for the 2004 report are as follows.[Footnote 22]
* Individuals--health, education and training, and work.
* Economy and economic resources--national income, financial hardship,
national wealth, housing, and productivity.
* Environment--the natural landscape, the human environment, oceans and
estuaries, and international environmental concerns.
* Living together--family, community, and social cohesion; crime; and
democracy, governance, and citizenship.
An Illustrative History of National Efforts in the United States:
A consistent message from the many experts and practitioners engaged in
this field has been to look at indicator systems from a historical
perspective. This is not only because such systems typically have
evolved over long periods, but also because some understanding of the
evolution of how U.S. citizens and organizations inform themselves
provides a basic foundation for describing comprehensive key indicator
systems. This history is intended to emphasize a few critical ideas.
First, our substantial information assets have evolved as the nation
confronted great problems or questions and needed to know more. Second,
the topical areas that resulted are the essential foundation for how
the nation informs itself. Third, since early in the 20th century, many
observers have recognized the potential value of a more comprehensive,
objective view of the United States. But it is only now, for a variety
of reasons, becoming potentially feasible to plan, design, and
implement such a resource.
National Challenges and Concerns Led to the Creation of Topical Area
Indicator Systems, Which Have Evolved Over Time:
The indicators required to inform our nation have developed over time
in response to important issues and opportunities. As national-level
indicators developed in the economic, environmental, and social and
cultural domains, each evolved with its own history and traditions. The
call for economic indicators grew out of the nation's experiences
during the Great Depression. Social upheavals after World War II and
the Great Society in the 1960s helped spark a desire for social and
cultural information. Scientific studies that raised concerns about
society's impact on the environment pointed to a need for more
information on environmental conditions. Substantial information
assets now exist in these topical areas--providing a foundation
consisting of thousands of indicators--on which we all depend for
decision making.
The U.S. federal statistical system includes indicators on many
specific topics and consists of numerous agencies and programs, each
established separately in response to different needs. The Office of
Management and Budget (OMB) has identified 70 federal agencies that
each spends at least $500,000 annually on statistical
activities.[Footnote 23] The U.S. federal statistical system is looked
to as a worldwide leader in terms of the sheer volume, scope, and
experience in developing and refining information sets in particular
domains and topical areas. Together, the output of these agencies
constitutes the federal statistical system. Ten of these agencies are
considered by OMB to be the principal statistical agencies because they
collect, produce, and disseminate statistical information as their
primary missions, while the other agencies that produce and disseminate
statistical data do so as an ancillary part of their missions. Table 2
provides a list of topical areas selected to illustrate the variety of
subjects covered by the federal statistical system.[Footnote 24]
Table 2: Selected Topical Areas Covered by Federal Statistical
Programs:
* Agriculture;
* Food and nutrition;
* Natural resources;
* Education;
* Health;
* International trade;
* Patents and trademarks;
* Energy;
* Occupational safety and health;
* Aging;
* Children and families;
* Homeland security;
* Housing;
* Crime and Justice;
* Employment;
* Job training;
* Transportation;
* Science and technology;
* Small business;
* Urban development.
Source: Office of Management and Budget.
[End of table]
Table 3 provides selected highlights of indicator traditions in the
economic, environmental, and social and cultural domains. These
highlights demonstrate three recognizable traditions in the development
of the United States' indicator systems that continue today but are now
being complemented by the development and evolution of comprehensive
systems. These national topical area indicator systems have evolved in
response to needs for new or different types of information, new
challenges, and shifting issues and priorities. They reflect an
investment of billions of dollars to create, maintain, and revise.
Table 3: Selected Highlights of Indicator Traditions in the United
States:
Tradition/domain: Economic indicators;
Illustrative examples: National Income and Product Accounts were
initially formulated to account for the flow of commodities and
services during World War II. They provide a base for key economic
indicators such as gross domestic product.
Tradition/domain: Economic indicators;
Illustrative examples: Business Cycle Indicators were created in the
1930s by the National Bureau of Economic Research and have been
compiled by the Conference Board since 1995. They were first compiled
by the U.S. Census Bureau for government agency use from 1961 to 1968
and then for public use from 1968 to 1972; the Bureau of Economic
Analysis compiled them from 1972 to 1995. The Conference Board
determines the specific data series included in the composite leading,
coincident, and lagging indicators, such as stock prices, employment,
and change in consumer prices for services respectively.
Tradition/domain: Economic indicators;
Illustrative examples: The Employment Act of 1946[A] committed the
federal government to the goals of full employment and economic
stability. The act created the Council of Economic Advisors, which
released the first Economic Report of the President in 1947. The
Council continues to publish it to this day.
Tradition/domain: Social and cultural indicators;
Illustrative examples: The Department of Labor, Children's Bureau's
Handbook of Federal Statistics on Children,[B] published in 1913,
attempted to bring together "scattered" federal data and other
information on children's welfare. The handbook was an early effort to
develop indicators for consistent monitoring of children and health.
Tradition/domain: Social and cultural indicators;
Illustrative examples: A proposed bill called the Full Opportunity and
Social Accounting Act[C] was first introduced in 1967. Although the
bill was never passed, it called for an annual social report from the
President to Congress and helped focus a national dialogue on social
indicators.
Tradition/domain: Social and cultural indicators;
Illustrative examples: In 1969, the Department of Health, Education and
Welfare published a report on social and cultural indicators called
Toward a Social Report.[D] The report was prepared at the direction of
President Johnson who sought "ways to improve the nation's ability to
chart its social progress." In 1973, federal statistical agencies
published a report on social indicators. Subsequent reports on social
indicators were published in 1976 and 1980.
Tradition/domain: Environmental indicators;
Illustrative examples: The National Environmental Policy Act (NEPA),
[E] signed into law on January 1, 1970, requires federal agencies to
assess the impacts of their decisions on the natural environment. While
NEPA did not establish any specific indicators, it does require that
federal agencies assess major federal actions significantly affecting
the environment. NEPA also established the Council on Environmental
Quality to advise the President on environmental matters.
Tradition/domain: Environmental indicators;
Illustrative examples: During the same year, EPA was created as an
independent agency to establish and enforce federal air standards and
water pollution control laws and to monitor the environment. The Clean
Air Act of 1970[F] also was passed. These initiatives focused national
attention on indicators of environmental quality.
Tradition/domain: Environmental indicators;
Illustrative examples: The Endangered Species Act of 1973[G] suggests
indicators of species viability, such as size and geographical
distribution of species' populations and their habitats. These
indicators can be used as the basis for avoiding the extinction of
species.
Source: GAO.
[A] Pub. L. No. 79-304, 60 Stat. 23 (1946).
[B] Department of Labor, Children's Bureau, Handbook of Federal
Statistics on Children (Washington, D.C.: Government Printing Office,
1913).
[C] 90th Congress, S-843.
[D] Department of Health, Education, and Welfare, Toward a Social
Report (Washington, D.C.: 1969).
[E] 42 U.S.C. §§ 4321-4370f.
[F] 42 U.S.C. §§ 7401-7671q.
[G] 16 U.S.C. §§ 1531-1544.
[End of table]
Economic Indicator Systems:
As the Great Depression deepened in the 1930s, the United States
established mechanisms to improve the collection of indicators on
particular economic and social and cultural conditions, including
national surveys on labor and health issues. During the 1940s and early
1950s, efforts increasingly focused on economic monitoring and
reporting. Key economic indicators, such as the National Income and
Product Accounts, became regularly reported and widely referenced by
policymakers, the business community, researchers, and the
public.[Footnote 25] The United States has been refining these
indicators since the 1930s, and work continues to this day. For
example, our 1997 report on the consumer price index (CPI) identified
more frequent updating of market basket expenditures weights as a way
to significantly improve the accuracy of the index and have a positive
impact on the federal budget deficit.[Footnote 26] Based on this and
other reports, the Bureau of Labor Statistics has made important
improvements in the CPI methodology, including more frequent updating
of the market basket.
An example of a specific topical area within the economic domain is the
Business Cycle Indicators system that is currently maintained by the
Conference Board. It consists of three sets of composite leading,
coincident, and lagging indexes--and is a well-known tool for
forecasting economic activity.[Footnote 27] The continuity of the
system has been critical for achieving a high level of attention from
national and business leaders.
Like most other U.S. economic indicators, the Business Cycle Indicators
system had its impetus in the dramatic economic transformations of the
Great Depression of the 1930s and World War II and its aftermath.
During the Great Depression, leaders were not able to adequately track
or forecast changes in the business cycle due to significant gaps in
our knowledge of the U.S. economy.
The Business Cycle Indicators system has been developed and refined
through public-private interactions over time. Business cycle indexes
have been published continuously since 1968, albeit with numerous
revisions and substitutions in response to factors like structural
changes in the economy due to, for example, increased globalization,
and new understandings of how the business cycle unfolds. Initially,
work on researching what would become the Business Cycle Indicators
came not from the government but from the private sector. Specifically,
this work began during the late 1930s at the private, nonprofit
National Bureau of Economic Research (NBER). NBER initially helped to
identify the most important business issues to measure and the types of
indicators needed. By the 1960s, NBER had refined the Business Cycle
Indicators and, in 1961 the U.S. Census Bureau began to regularly
publish reports based upon the indicators for government agency use. In
1968, the U.S. Census Bureau began publishing a report on the Business
Cycle Indicators not just for government agency use, but also for
public use and did so through 1972. The Bureau of Economic Analysis
then published the indicators from 1972 to 1995, although the program
was scaled back over time. The reports also included a sizeable
chartbook containing underlying economic data, which was eventually
eliminated. By 1995, the Business Cycle Indicators had become well
established, and the federal government granted the Conference Board
exclusive rights to produce the Business Cycle Indicators, which it has
done ever since.
Figure 6 illustrates how an indicator system may change over time. This
illustration shows how two different versions of the leading index--the
old leading index (or "current leading index" in the figure) and the
"new leading index" that replaced it in late 1996--predicted different
patterns for the U.S. economy. Specifically, figure 6 compares two sets
of trends: one based on the original ("current") leading index and the
other based on recalculations using a new, revised index. For example,
the old ("current") leading index provided a "false signal" of an
oncoming recession in 1984, whereas the revised leading index ("new")
provided a much more muted signal.
Figure 6: Revisions in the Leading Index of the Business Cycle
Indicators, 1984-1997:
[See PDF for image]
Note: Data from Business Cycle Indicators, vol.1, no. 11, December
1996.
[End of figure]
Social and Cultural Indicator Systems:
The apparent success of economic indicators in contributing to
discussions and decisions about managing economic policy helped spark
interest in producing indicators on the social and cultural well-being
of the nation and increased institutional support for enhancing the
availability of information to support planning and policy making. In
the 1960s, some believed that economic indicators alone were not
adequate to monitor the dramatic social changes taking place. A
heightened focus and debate on social and cultural indicators led
certain observers to label this effort as a "social indicators
movement"--even though some attempts were made to focus on
environmental indicators as well. (See app. II for more information on
the social and cultural domain.)
There were some attempts during the 1960s to unite economic indicators
with improved social and cultural and environmental indicators in order
to provide a comprehensive view of the position and progress of the
nation. A first step to enhance social and cultural indicators and
report more comprehensively on the position of the nation as a whole
occurred in 1962 when the National Aeronautics and Space Administration
commissioned the American Academy of Arts and Sciences to explore the
potential side effects of space exploration on U.S. society. The
resulting Social Indicators report, published in 1966, found that
adequate information for assessing American life was not as widely
available as economic information was. It called for increased
collection of social and cultural statistics and recommended the
development of a system of national social accounts to help guide
policy decisions.[Footnote 28]
In 1967, several senators proposed legislation calling for the creation
of a national system of social accounting and a Council of Social
Advisers that was to have been comparable to the Council of Economic
Advisers. Hearings were conducted on a proposed bill that would have
established an annual social report similar to the Economic Report of
the President, although the bill did not pass.
In 1969, the Department of Health, Education, and Welfare--now the
Department of Health and Human Services--produced an influential
publication entitled Toward a Social Report. This report was
commissioned by presidential directive to "develop the necessary social
statistics and indicators to supplement those prepared by the Bureau of
Labor Statistics and the Council of Economic Advisers." The report
dealt with various environmental and social and cultural concerns of
American society, such as health and illness; social mobility; the
physical environment; income and poverty; public order and safety;
learning, science, and art; citizen participation; and the perceived
alienation of certain groups of citizens. The report assessed
prevailing conditions on each of these topics, concluded that
indicators on social and cultural conditions were lacking, and
recommended that the executive branch prepare a comprehensive social
report for the nation with emphasis on indicators to measure social
change that could be used in setting policy and goals.[Footnote 29]
There were several other developments in the area of social and
cultural indicators during the 1970s and 1980s. In 1972, the Social
Science Research Council--a non-profit organization--established the
Center for Coordination of Research on Social Indicators.
In 1973, 1977, and 1980, the federal government published three
reference volumes , entitled Social Indicators.[Footnote 30] These
reports presented information on important aspects of the country's
social condition along with underlying historical trends and
developments. Subject areas included population; the family; health and
nutrition; housing; the environment; transportation; public safety;
education and training; work; social security and welfare; income and
productivity; social mobility and participation; and culture, leisure,
and use of time. However, the U.S. government discontinued the Social
Indicators series after the 1980 volume. Moreover, the Center for
Coordination of Research on Social Indicators also closed. Although the
absence of these consolidated efforts creates the appearance that the
production of literature on social and cultural indicators declined,
this is difficult to substantiate. An equally plausible possibility is
that it simply dispersed and continued to develop in respective topical
areas in academic, governmental, and non-profit settings.
Other developments during the 1970s and 1980s included publication of a
number of works on social indicators and the launch of several periodic
sample population surveys, such as the General Social Survey and the
National Crime Victimization Survey.[Footnote 31] Research on social
and cultural indicators was also under way in other countries and
involved some international organizations. For example, building on the
work completed in the United States, researchers in Germany continued
to develop social indicators. Their work formed the basis for the
German System of Social Indicators, which has been in place for 30
years. Additionally, the Organisation for Economic Co-operation and
Development (OECD) launched a social indicators program in 1970. This
program, with the help of an international network of researchers and
national statisticians, developed a model survey and a list of social
indicators intended to provide systematic indicators for national and
comparative use. OECD's first Programme of Work on Social Indicators
was cancelled after the publication of the first (and only) edition of
the report, Living Conditions in OECD Countries in 1986.[Footnote 32]
OECD began work on its current social indicators project in 1998, which
led to the publication of a 2002 report.[Footnote 33]
Observers have proposed a number of explanations as to why national
attempts to create more integrated social and cultural reporting appear
to have declined. One factor cited was that western industrial
societies experienced an economic crisis in the early 1980s that
continued to focus attention on economic problems. Further, the large
government budget deficits that accumulated during the 1980s reduced
the funding available for social research--along with many other
domestic policy priorities. Others believe that initial expectations
about what social and cultural indicators could accomplish may have
been "oversold." These observers argued that the usefulness of the
existing social and cultural indicators had not been demonstrated to
leaders and that, therefore, the indicators were not directly used in
policy making. Further, social processes were proving to be more
complex and less clearly understood than economic ones, and there was
no theoretical framework comparable to economic theory. An additional
factor may have been that the extensive cost of and effort associated
with collecting and analyzing social data were significant due to the
limited technology available at that time; and benefits were unclear.
In fact, the diversity of the ways in which social and cultural
indicators can be conceptualized continues to be a challenge. Many
topical areas that appear to reside clearly within that domain (e.g.,
social equity), upon further investigation, turned out to be
crosscutting and could only be examined in the context of
interrelationships with the other two domains. The difficulty of work
in the social and cultural domain is accentuated by the fact that it
covers many sensitive moral, racial, or religious issues, among others.
Healthy People, led by the Department of Health and Human Services, is
a specific example of a topical indicator system currently operating in
the social and cultural domain at the U.S. national level.[Footnote 34]
Healthy People originated in the late 1970s during a movement in the
medical, scientific, and public health communities to enhance health
promotion, health protection, and disease prevention in the nation.
Specifically, its purpose is to provide a consensus set of national
objectives related to various health concerns--such as the prevalence
of cigarette smoking and related illnesses among Americans--that the
health community could agree to, obtain data on, and monitor over time.
Healthy People was envisioned as a tool for progress, with a number of
objectives established to provide consistent guidance to the process.
The Healthy People system has increasingly engaged stakeholders at the
subnational levels to assist in progress toward national health goals
and objectives. In 1987 the Healthy People Consortium--an alliance that
now consists of more than 350 organizations and 250 state and local
agencies--was created to forge a coalition that is dedicated to taking
action to achieve the Healthy People objectives, such as reducing
obesity. It facilitates broad participation in the process of
developing the national prevention agenda and engages local chapters
and their members in the provision of community and neighborhood
leadership. The National Medical Association, Wellness Councils of
America, American Hospital Association, and American Medical
Association are examples of Consortium members that use their
expertise, contacts and resources to adopt, promote, and achieve the
Healthy People agenda. The Consortium also seeks to coordinate Healthy
People with state, local, and community level initiatives. Further, 41
states and the District of Columbia have their own Healthy People
plans.
Since 1980, Healthy People has evolved into a series of 10-year
efforts. For each upcoming decade, Healthy People has established new
sets of goal statements, focus areas, and objectives that build upon
the work of the prior decades' efforts. Healthy People 2010:
Understanding and Improving Health, was issued in 2000 and continues
the tradition by setting forth two overarching goals: (1) increasing
the quality and years of healthy life and (2) eliminating health
disparities. These goals are detailed in 28 focus areas that include
467 specific objectives, along with indicators to be used in monitoring
progress.[Footnote 35]
Figure 7 provides an example of current Healthy People indicators that
measure the objective of improving cardiovascular health and quality of
life through prevention, detection, and treatment of risk factors;
identifying and treating heart attacks and strokes; and preventing
recurrences--rates of coronary heart disease and stroke deaths (per
100,000 people). It shows that the age-adjusted death rate for heart
disease (per 100,000 people) declined throughout the 1980s and 1990s to
208 in 1998, while the rate of deaths due to strokes declined to 60.
Figure 7: Coronary Heart Disease and Stroke Deaths, by Year, in the
United States, 1979-1998:
[See PDF for image]
Notes: Data from National Vital Statistics Systems, 1979-98. The rates
are age adjusted by the year 2000 standard population to compensate for
the relative increase in the number of older people in the United
States, who have higher rates of death from coronary heart disease and
strokes.
* Age adjusted to the year 2000 standard population:
[End of figure]
Another innovation that emerged in the Healthy People 2010 report is
the identification of a smaller set of 10 "Leading Health Indicators,"
which provides a succinct, user-friendly measure of the health of the
U.S. population. These indicators are intended to increase general
public awareness and motivate action at the federal, state, and local
levels. The leading indicators include measures of:
* physical activity,
* overweight and obesity,
* tobacco use,
* substance abuse,
* responsible sexual behavior,
* mental health,
* injury and violence,
* environmental quality,
* immunization, and:
* access to health care.
Environmental Indicator Systems:
Public concerns about the quality of the environment date back to
around the turn of the 20thcentury but began to reach a critical mass
in the 1960s. Initially, many of these concerns centered on the effects
of pollution. In 1962, Rachel Carson published Silent Spring,
chronicling the effects of bioaccumulation.[Footnote 36] Several
reports raised similar concerns regarding the quality of the nation's
rivers, lakes, and estuaries. For example, the Potomac River was
heavily polluted, beach closures and warnings regarding shellfish
contamination were common events, and the Cuyahoga River in Ohio caught
fire. By the 1970s, the political momentum to protect the environment
and the public from the hazards of pollution led to a number of laws
and initiatives, including creating the EPA, establishing national
standards for drinking water, legislating protections for endangered
species, and enacting air and water pollution control laws.
For example, water quality is one area in which various efforts have
been undertaken to develop and implement environmental policies and
related indicators. Among these actions was the passage of the Federal
Water Pollution Control Act Amendments of 1972, which, as amended, is
commonly known as the Clean Water Act.[Footnote 37] The primary
objective of the act is to "restore and maintain the chemical,
physical, and biological integrity of the Nation's waters." Under the
act, states have primary responsibility for implementing programs to
manage water quality. In particular, state responsibilities include
establishing water quality standards to achieve designated uses (the
purposes for which a given body of water is intended to serve),
assessing whether the quality of their waters meets state water quality
standards, and developing and implementing cleanup plans for waters
that do not meet standards.
Monitoring information on water quality--for example, the presence of
chemicals such as chlorine, physical characteristics such as
temperature, and biological characteristics such as the health or
abundance of fish--is the linchpin that allows states to perform their
responsibilities. States generally monitor water quality directly, but
often supplement their efforts with information collected by federal
agencies, volunteer groups, and other entities. For example, many
states use data collected by the U.S. Geological Survey (USGS), which
has a large program for monitoring water quality.
While the use of water quality data is critical to meeting the
objectives of the Clean Water Act, other organizations use water
quality data for a variety of other purposes. Federal land management
agencies (including the Department of the Interior's Fish and Wildlife
Servi