Preliminary Results From National Evaluation Are Not Conclusive
HEHS-98-167: Published: Jun 15, 1998. Publicly Released: Jul 15, 1998.
- Full Report:
Pursuant to a congressional request, GAO reviewed the preliminary results of a national evaluation of the Healthy Start program, focusing on: (1) the plan for national evaluation; (2) what Mathematica Policy Research, Inc.'s (MPR) preliminary evaluation results indicate; and (3) what is expected from the final evaluation.
GAO noted that: (1) MPR's preliminary reports from the national evaluation of Healthy Start do not provide a complete assessment of the program and, therefore, should not be used to judge the program's success; (2) even the final report will not contain all the data expected to be analyzed for the national evaluation; (3) if the evaluation plan were expanded to include data from the sixth and final year of the demonstration, conclusions about whether the program has met its goals of reducing infant mortality could be strengthened; (4) the national evaluation of the Healthy Start program had two major components: (a) an impact evaluation, to determine whether the infant mortality rates in Healthy Start communities have declined; and (b) a process evaluation to describe how the program actually operates; (5) once these evaluations are completed, MPR plans to link outcomes with processes in its final report to determine why Healthy Start has or has not succeeded and what would be required for a similar intervention elsewhere; (6) while MPR's draft report on its impact evaluation suggests that Healthy Start has little effect in reducing infant morality in targeted communities, drawing such a conclusion at this time would be premature for several reasons; (7) the process evaluation is also incomplete; (8) only some of the reports that it comprises are available; (9) eventually, MPR plans to cover program implementation at all sites, the characteristics of program participants, and details about some of the most important strategies used by the program; (10) with these two major components of the evaluation in preliminary stages or incomplete, MPR cannot yet relate process to impact; (11) the final evaluation is expected to include an analysis of infant morality data from the original 5 years of the demonstration for all 15 sites; (12) however, the final report on the evaluation, now planned for early 1999, will include data from only the first 4 years; and (13) further, because implementation of the program was slower than anticipated, and the program was mature for fewer years of the original demonstration period than planned for in the evaluation, even results from the final report are likely to be inconclusive and should be considered preliminary.
Recommendation for Executive Action
Status: Closed - Implemented
Comments: The agency agrees with the recommendation; it so indicated in its comments.
Recommendation: To increase the value of the investment in the national evaluation of Healthy Start, the Health Resources and Services Administration should contract with MPR to expand the evaluation to include impact data from the sixth year.
Agency Affected: Department of Health and Human Services: Public Health Service: Health Resources and Services Administration