An Evaluation Culture and Collaborative Partnerships Help Build Agency Capacity
GAO-03-454, May 2, 2003
Agencies are increasingly asked to demonstrate results, but many programs lack credible performance information and the capacity to rigorously evaluate program results. To assist agency efforts to provide credible information, GAO examined the experiences of five agencies that demonstrated evaluation capacity in their performance reports: the Administration for Children and Families (ACF), the Coast Guard, the Department of Housing and Urban Development (HUD), the National Highway Traffic Safety Administration (NHTSA), and the National Science Foundation (NSF).
In the five agencies GAO reviewed, the key elements of evaluation capacity were an evaluation culture--a commitment to self-examination, data quality, analytic expertise, and collaborative partnerships. ACF, NHTSA, and NSF initiated evaluations regularly, through a formal process, while HUD and the Coast Guard conducted them as specific questions arose. Access to credible, reliable, and consistent data was critical to ensure findings were trustworthy. These agencies needed access to expertise in both research methods and subject matter to produce rigorous and objective assessments. Collaborative partnerships leveraged resources and expertise. ACF, HUD, and NHTSA primarily partnered with state and local agencies; the Coast Guard partnered primarily with federal agencies and the private sector. The five agencies used various strategies to develop and improve evaluation: Commitment to learning from evaluation developed to support policy debates and demands for accountability. Some agencies improved administrative systems to improve data quality. Others turned to specialized data collection. All five agencies typically contracted with experts for specialized analyses. Some agencies provided their state partners with technical assistance. These five agencies used creative strategies to leverage resources and obtain useful evaluations. Other agencies could adopt these strategies--with leadership commitment--to develop evaluation capacity, despite possible impediments: constraints on spending, local control over flexible programs, and restrictions on federal information collection. The agencies agreed with our descriptions of their programs and evaluations.