Trends in Evaluation

An evaluator faces a formidable quantity of prior evaluation reports. Under conditions of curtailed evaluation budgets and complex policy questions, there are strong incentives to effectively sort, categorize, and use relevant prior research and evaluation as part of the current evaluation database. Also, rapid growth in the quantity of completed work increases the potential useful information available for evaluations. Evaluators are developing methodological approaches for combining the results of past evaluation and research. Reanalysis usually focuses on a single salient study or a small number of studies to obtain the data used for the original work and to reanalyze them with what the evaluator believes might be better statistical methods. Continuing secondary analysis of salient studies may bring new insights and a higher confidence in the validity of the data. Meta-evaluation is an appropriate method for evaluators concerned with the findings of a large number of studies. It is most likely to stop with a statistical analysis of the reported results. One shortcoming of this method is that poorly designed or poorly executed studies may be weighted equally with the most valid studies. Content analysis offers evaluators the potential for systematically analyzing written material. Depending on the time and resources available, various combinations of these methods could be used. As these methods become more widely tested and accepted, each new evaluation may be seen more as a component of a vast body of organized knowledge rather than an independent piece of information.