Program Evaluation: Annual Agency-Wide Plans Could Enhance Leadership Support for Program Evaluations
Highlights
What GAO Found
In a 2017 government-wide survey, GAO found that most federal managers lack recent evaluations of their programs. Forty percent reported that an evaluation had been completed within the past 5 years of any program, operation, or project they were involved in. Another 39 percent of managers reported that they did not know if an evaluation had been completed, and 18 percent reported having none. Managers who reported having evaluations also reported that those evaluations contributed to a great or very great extent to improving program management or performance (54 percent) and to assessing program effectiveness or value (48 percent). These figures are not statistically different from the results of GAO's 2013 survey.
Of the 40 percent of managers who reported having evaluations, the factor most often rated as having hindered use to a great or very great extent, as in 2013, was lack of resources to implement the evaluation findings (29 percent). Managers reported limited knowledge of congressional support for using their results; 35 percent were not able to judge whether lack of support was a barrier.
Federal managers who reported having evaluations most frequently reported that agency leadership support for evaluation, staff involvement, and an evaluation's relevance to decision makers facilitated evaluation use. GAO previously reported that involving agency staff in planning and conducting evaluations helps to ensure they are relevant, credible, and used in decision making. The Office of Management and Budget (OMB) encouraged agencies to use the annual strategic reviews the GPRA Modernization Act of 2010 (GPRAMA) requires to assess evidence gaps and inform their strategic decisions and budget making.
GAO and OMB have noted the importance of developing an evaluation plan or agenda to ensure that an agency's scarce research and evaluation resources are targeted to its most important issues. While 28 percent of managers with evaluations rated consultation with stakeholders high for facilitating use, another 22 percent reported having no basis to judge. GAO previously noted limited knowledge of agency consultation with the Congress. While 23 percent of managers with evaluations reported congressional requests or mandates facilitated evaluation use, more (31 percent) reported having no basis to judge.
GAO concludes that
- Agencies' continued lack of evaluations may be the greatest barrier to their informing managers and policy makers and constitutes a lost opportunity to improve the efficiency and effectiveness of limited government resources.
- Although only some agencies have developed agency-wide evaluation plans, evaluators who have them found that obtaining stakeholder input helped ensure evaluation relevance and facilitate use of their results.
- Congressional consultation on agency evaluation plans could increase the studies' credibility with those whose support is needed to implement program reforms.
- An agency's annual strategic review provides a good opportunity to help target its evaluation agenda to its management, budget, and policy priorities.
Why GAO Did This Study
GPRAMA aims to ensure that agencies use performance information in decision making to achieve results and improve government performance. GPRAMA requires GAO to evaluate the act's implementation; this report is one in a series on its implementation. GAO examined the extent of agencies' use of program evaluations—a particular form of performance information—and factors that may hinder or facilitate their use in program management and policy making.
GAO surveyed a stratified random sample of 4,395 federal civilian managers and supervisors to obtain their perspectives on several results-oriented management topics, including the extent of and factors influencing evaluation use. GAO compared the results to those of a similar GAO survey of federal managers in 2013 and a GAO survey of Performance Improvement Officers in 2014. GAO also interviewed OMB staff and reviewed guidance on using evaluation in decision making.
Recommendations
To help ensure that agencies obtain the evidence needed to address important questions to improve program implementation and performance, GAO recommends that the Director of OMB direct federal agencies to prepare an annual agency-wide evaluation plan that describes the congressional and other stakeholders that were consulted.
OMB staff stated that agencies should be encouraged, rather than directed, to create an annual evaluation plan. Because OMB has already been encouraging evaluation, GAO believes a more directive approach is needed.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget | To help ensure that federal agencies obtain the evidence needed to address the most important questions to improve program implementation and performance, the Director of the Office of Management and Budget should direct each of the 24 Chief Financial Officer Act agencies to prepare an annual agency-wide evaluation plan that describes the (1) key questions for each significant evaluation study that the agency plans to begin in the next fiscal year, and (2) congressional committees; federal, state and local program partners; researchers; and other stakeholders that were consulted in preparing their plan. (Recommendation 1) |
Closed – Implemented
On July 10, 2019 the Office of Management and Budget issued Memorandum M-19-23 which directs designated agency Evaluation Officers to lead, coordinate, develop, and implement Annual Evaluation Plans. As we recommended, the plans are to explain how the evaluation activities help answer the priority evaluation questions identified in the evidence-building plans that are required by the Foundations for Evidence-Base Policymaking Act of 2018.
|