Drug Abuse Prevention:

Federal Efforts to Identify Exemplary Programs Need Stronger Design

PEMD-91-15: Published: Aug 22, 1991. Publicly Released: Sep 24, 1991.

Additional Materials:

Contact:

Robert L. York
(202) 275-5885
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Pursuant to a congressional request, GAO examined the policies and methods used by the Department of Education and the Department of Health and Human Services (HHS) to identify exemplary drug abuse prevention programs.

GAO found that: (1) Education and HHS unnecessarily limited the search for successful drug abuse prevention programs by considering only those programs with a no-use approach; (2) nomination procedures for Education's Drug-Free School Recognition Program were not sufficiently systematic and comprehensive to include all interested and eligible programs and Education did not provide guidelines for the initial screening of applications; (3) Education did not clearly define evaluation criteria; (4) the procedures used to obtain data on those criteria were adequate, although sufficient data were not obtained; (5) HHS and Education review teams lacked individuals with skills in research and evaluation or in any type of scientific data analysis to interpret the applications' evaluation information; (6) the recommendations made by knowledgeable Education reviewers were subject to change by a steering committee with no additional data or formal guidance; and (7) the HHS Exemplary Program Study did not conduct site visits to validate and supplement the evidence provided in written applications, provide sufficient time to read applications, or clearly define application criteria.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: The agency disagrees strongly with GAO and no change is likely in the near future.

    Recommendation: The Secretaries of Education and Health and Human Services should require that the policies of the recognition efforts be revised to allow consideration of the full variety of prevention strategies commonly used in schools and other agencies. This may involve consideration of programs that do not adhere to a strict no-use approach towards alcohol or tobacco for adults and youths over age 15. An intermediate step may be to conduct evaluations on the relative merits of no-use and responsible-use approaches, which could provide evidence that supports the current restriction. In addition, application materials should be revised to encourage the submission of applications by programs featuring a broader range of those prevention strategies that the literature suggests hold promise of success.

    Agency Affected: Department of Education

  2. Status: Closed - Not Implemented

    Comments: The agency disagrees strongly with GAO and no change is likely in the near future.

    Recommendation: The Secretaries of Education and Health and Human Services should require that the policies of the recognition efforts be revised to allow consideration of the full variety of prevention strategies commonly used in schools and other agencies. This may involve consideration of programs that do not adhere to a strict no-use approach towards alcohol or tobacco for adults and youths over age 15. An intermediate step may be to conduct evaluations on the relative merits of no-use and responsible-use approaches, which could provide evidence that supports the current restriction. In addition, application materials should be revised to encourage the submission of applications by programs featuring a broader range of those prevention strategies that the literature suggests hold promise of success.

    Agency Affected: Department of Health and Human Services

  3. Status: Closed - Implemented

    Comments: Education expanded its methods of soliciting nominations.

    Recommendation: The Secretaries of Education and Health and Human Services should require that the recognition efforts be open to all interested applicants through a more comprehensive and systematic call for applications. The nominations procedures need not be abandoned altogether; rather, they should be restructured to ensure equal opportunity for nomination to all eligible programs.

    Agency Affected: Department of Education

  4. Status: Closed - Implemented

    Comments: HHS expanded its methods for soliciting nominations.

    Recommendation: The Secretaries of Education and Health and Human Services should require that the recognition efforts be open to all interested applicants through a more comprehensive and systematic call for applications. The nominations procedures need not be abandoned altogether; rather, they should be restructured to ensure equal opportunity for nomination to all eligible programs.

    Agency Affected: Department of Health and Human Services

  5. Status: Closed - Implemented

    Comments: Additional reviewers are being used with the recommended expertise.

    Recommendation: The Secretaries of Education and Health and Human Services should require that the Drug-Free School Recognition Program add to each review team, and the Exemplary Program Study add to the review panel, at least one and two members, respectively, who have backgrounds in social science evaluation or research methodology that enable them to conduct skillful analyses of effectiveness evidence.

    Agency Affected: Department of Education

  6. Status: Closed - Implemented

    Comments: Review panel membership was expanded to include recommended expertise.

    Recommendation: The Secretaries of Education and Health and Human Services should require that the Drug-Free School Recognition Program add to each review team, and the Exemplary Program Study add to the review panel, at least one and two members, respectively, who have backgrounds in social science evaluation or research methodology that enable them to conduct skillful analyses of effectiveness evidence.

    Agency Affected: Department of Health and Human Services

  7. Status: Closed - Implemented

    Comments: Education has revised the review criteria.

    Recommendation: The Secretaries of Education and Health and Human Services should require that application criteria be revised to improve their clarity, and standards of evidence should be established for each.

    Agency Affected: Department of Education

  8. Status: Closed - Implemented

    Comments: HHS has revised the recognition review criteria.

    Recommendation: The Secretaries of Education and Health and Human Services should require that application criteria be revised to improve their clarity, and standards of evidence should be established for each.

    Agency Affected: Department of Health and Human Services

  9. Status: Closed - Implemented

    Comments: Education is requiring evidence of effectiveness as part of the application.

    Recommendation: The Secretaries of Education and Health and Human Services should require that applicants be required to submit data demonstrating program effectiveness in order to be considered for recognition. To emphasize to applicants the importance of effectiveness evidence, the application materials should include a separate section that requires applicants to provide specific (preferably quantitative) evidence of effectiveness.

    Agency Affected: Department of Education

  10. Status: Closed - Implemented

    Comments: HHS is requiring evidence of effectiveness from finalists in the competition.

    Recommendation: The Secretaries of Education and Health and Human Services should require that applicants be required to submit data demonstrating program effectiveness in order to be considered for recognition. To emphasize to applicants the importance of effectiveness evidence, the application materials should include a separate section that requires applicants to provide specific (preferably quantitative) evidence of effectiveness.

    Agency Affected: Department of Health and Human Services

  11. Status: Closed - Implemented

    Comments: Education has dropped the steering committee altogether.

    Recommendation: The Secretary of Education should eliminate the current veto power held by the Department of Education Steering Committee and only permit that group to send back recommendations for further review, perhaps with the addition of specific questions to be addressed by the knowledgeable reviewers.

    Agency Affected: Department of Education

  12. Status: Closed - Not Implemented

    Comments: HHS agrees with the recommendation that site visits would be beneficial but does not have adequate funds to carry out the visits.

    Recommendation: The Secretary of Health and Human Services should expand the Exemplary Program Study data collection design to include site visits.

    Agency Affected: Department of Health and Human Services

  13. Status: Closed - Implemented

    Comments: HHS has changed the procedures to permit adequate advance review.

    Recommendation: The Secretary of Health and Human Services should expand the review panel schedule to include a period of training and more time for reading and panel discussion of applications. One way to allow panel members to consider more data would be to subdivide the work among several smaller teams, as is done in the Drug-Free School Recognition Program. A smaller work load would permit all members of a team to read the full set of applications to be evaluated. Teams would then have sufficient time to review the evidence under each application criterion, rather than simply select points.

    Agency Affected: Department of Health and Human Services

 

Explore the full database of GAO's Open Recommendations »

Sep 28, 2016

Sep 15, 2016

Sep 14, 2016

Sep 12, 2016

Sep 9, 2016

Sep 6, 2016

Aug 31, 2016

Looking for more? Browse all our products here