Runaway and Homeless Youth Grants:

Improvements Needed in the Grant Award Process

GAO-10-335: Published: May 10, 2010. Publicly Released: May 10, 2010.

Additional Materials:

Contact:

Kay E. Brown
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Health and Human Services (HHS) awards grants to provide shelter and services to runaway and homeless youth through the Basic Center, Transitional Living and Street Outreach Programs. In response to a mandate for a review of the grant award process for these programs in the Reconnecting Homeless Youth Act of 2008 (Pub. L. No. 110-378), GAO examined (1) grant announcements and application requirements, (2) technical assistance for grant applicants, (3) how grant award decisions are made, and (4) notification of grant award decisions. GAO reviewed requirements, documents, and records associated with this process for fiscal years 2007 and 2008, observed the grant evaluation portion of this process, and interviewed applicants, peer reviewers, and agency officials.

Based on GAO's review of past grant announcements for these programs, GAO found that the criteria upon which grant applications were evaluated were not clearly identified or presented in a single location in the announcement. Rather, GAO found that criteria were scattered throughout various sections of the announcement, had multiple labels, and were not presented in an orderly manner. As a result, applications that did not address the criteria from all sections were likely to receive lower evaluation scores, decreasing their chances of receiving a grant. HHS provides technical assistance to potential applicants for runaway and homeless youth grants, as required by statute. Of the 20 applicants GAO interviewed who sought technical assistance, 17 were satisfied with the help they received. Grant award decisions are primarily based on the results of the peer review process, and internal controls in place to ensure that applications are evaluated consistently were not always adequate. GAO found weaknesses in four out of the six procedures the agency relies on to ensure consistent evaluation of applications. For example, although HHS policy requires peer reviewers to be experts in the field of runaway and homeless youth programs, about one- quarter of the reviewers who evaluated applications for 2009 Street Outreach grants had little or no experience in this area. With regard to notification of grant award decisions, GAO found that they have not always been communicated to applicants in a timely manner, which can delay the start of new programs and present planning challenges for existing ones. GAO also found that the information in notification letters to applicants who were not awarded grants was not always clear or complete.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To enhance transparency and fairness in the grant award process, and improve grantees ability to plan for services, the Secretary of Health and Human Services should direct the Assistant Secretary for the Administration for Children and Families to clearly identify in grant announcements all the criteria that peer reviewers will use to evaluate and score applications, and ensure that peer reviewers use only those criteria during the peer review process.

    Agency Affected: Department of Health and Human Services

    Status: Closed - Implemented

    Comments: In FY12, HHS reported that its grant reviewers are trained and instructed to review and score each application independently based on the published criteria. In FY13, the agency reported that ACF streamlined the grants process and standardized all training across the agency. Internal guidance was provided to program offices regarding the grant review process and program offices must ensure that reviewers do not use any templates other than the published funding opportunity announcement (FOA) for review criteria. In addition, as part of the grant review process, each panel is overseen by a federal staff member to ensure that comments and scores reflect the criteria in the published FOA. HHS reported in May 2014 that ACF continues to provide extensive training to program offices on how to draft FOA's, specifically where information is to be placed within the FOA. Peer reviewers are instructed to review all applications using the evaluation criteria located in Section V.1 of the FOA. Evaluation criteria are not included in any other sections of the FOA. ACF policy states that offices must ensure that reviewers do not use any templates other than the published FOA for review criteria as described in Section V.1 only. In addition, ACF uses the ARM system, a web-based tool used for managing, reviewing, and monitoring the review process for applications for each published FOA. Comments and or scores are not finalized until reviewed and approved by the federal staff member.

    Recommendation: To enhance transparency and fairness in the grant award process, and improve grantees ability to plan for services, the Secretary of Health and Human Services should direct the Assistant Secretary for the Administration for Children and Families to select peer reviewers with expertise in the programmatic area for which they are evaluating grant applications.

    Agency Affected: Department of Health and Human Services

    Status: Closed - Implemented

    Comments: In 2010, HHS commented that the agency elected to accept reviewers who are knowledgeable of the risk factors faced by runaway and homeless youth, and that many professional disciplines often intersect with runaway and homeless youth. The agency plans to take steps to ensure that all reviewers possess the knowledge and expertise in the particular program for which they are reviewing grant applications. In the event of a shortage of reviewers, they will staff panels with at least one peer reviewer with extensive relevant knowledge, which would continue to differ from the current policy that grants must receive an objective, advisory review by a minimum of "three" unbiased reviewers with expertise in the programmatic area for which applications are submitted. GAO notes that because professionals in varied disciplines could have sufficient expertise to serve as reviewers and it may be difficult for the agency to find enough reviewers with expertise in a particular program, this recommendation includes those that have expertise in the programmatic area for which they are evaluating grant applications, and not a specific program. In FY12, HHS responded that it actively recruits reviewers from the Runaway and Homeless Youth (RHY) field. When that's not possible, they make a concerted effort to find individuals from professional disciplines who have expertise and a working knowledge of the field. Since the last update in FY 12, ACF has implemented and used the ARM system for reviews. The ARM system is a web-based tool used for managing, reviewing, and monitoring the review process for grant applications. The Application Review Module (ARM) greatly increases the number of potential reviewers than can be available for reviews since it gives program offices the flexibility to access and hold reviews with individuals across the country. The grant review logistics contractor ensures that cumulative professional experiences meet a baseline of knowledge required to effectively evaluate RHY grant applications. In addition, ACF continues to take steps to ensure that all reviewers possess the knowledge and expertise through orientation and continuing education of the subject matter prior to selection as a reviewer.

    Recommendation: To enhance transparency and fairness in the grant award process, and improve grantees ability to plan for services, the Secretary of Health and Human Services should direct the Assistant Secretary for the Administration for Children and Families to make all meetings for peer reviewers, including those for new reviewers and chairs, mandatory.

    Agency Affected: Department of Health and Human Services

    Status: Closed - Implemented

    Comments: In 2010, HHS indicated that all meetings for peer reviewers and chairs are already mandatory but due to unforeseen factors, it is not always possible for all reviewers to attend. GAO observed that attendance at these meetings was not explicitly mandatory, but highly encouraged. Furthermore, attendance was not enforced and attendance records were not maintained. HHS plans to officially notify all reviewers and chairpersons participating in future reviews that all training is mandatory. In the event some reviewers and chairpersons are not able to attend the mandatory training sessions due to unforeseen circumstances, the agency intends to offer "make up" sessions. In FY12, HHS reported that all grant review meetings are mandatory for all review participants.

    Recommendation: To enhance transparency and fairness in the grant award process, and improve grantees ability to plan for services, the Secretary of Health and Human Services should direct the Assistant Secretary for the Administration for Children and Families to document and maintain records of Administration for Children and Families (ACF) comments to peer review panels during the review process.

    Agency Affected: Department of Health and Human Services

    Status: Closed - Implemented

    Comments: In FY12, HHS reported that it now uses the Application Review Module (ARM), a secure web-based system, to track the comments flowing between federal staff and the panels. ARM captures comments of federal staff on each application provided as part of the back and forth in their roles as panel managers, and electronically captures and maintains the reviewer comments and scores associated with a panel review.

    Recommendation: To enhance transparency and fairness in the grant award process, and improve grantees ability to plan for services, the Secretary of Health and Human Services should direct the Assistant Secretary for the Administration for Children and Families to document the specific reasons for denying grants to high-scoring applicants in favor of other applicants for the agency record.

    Agency Affected: Department of Health and Human Services

    Status: Closed - Implemented

    Comments: In 2010, HHS reported its plans to include more details concerning geographic distribution in the letters to applicants who are denied grants for this reason. In FY12 and FY13, HHS reported that it documents the reasons for not funding high scoring applications as a part of the funding decision memo, which is maintained in the GrantSolutions system, ACF's official record keeping system. The reasons for not funding an applicant are documented in accordance with the HHS Awarding Agency Grants Administration Manual, which requires that "if the application's position in the list of applications approved for funding is different from its position in the ranking list, a statement of the reasons for the difference that influenced the judgment of the approval. This should include a justification for funding of the particular application." HHS reported in FY13 that it is beginning to note in the letters to unsuccessful grant applicants when a geographic exception resulted in their application being unfunded.

    Recommendation: To enhance transparency and fairness in the grant award process, and improve grantees ability to plan for services, the Secretary of Health and Human Services should direct the Assistant Secretary for the Administration for Children and Families to provide clear information to applicants about the specific reasons their applications were not approved.

    Agency Affected: Department of Health and Human Services

    Status: Closed - Implemented

    Comments: HHS reported in FY12 that it took steps to provide clear information to applicants. Applicants that are disqualified are screened and notified before the grant review; these applications are not reviewed and scored. ACF is standardizing the screen out letters to these applicants so they are clear that they did not meet the requirements. Starting in FY 2013, every applicant - whether funded or unfunded - receives the aggregated score of their application along with the compilation of reviewer comments outlining the strengths and weaknesses on all criteria as determined by the review. For those applicants who may not have been funded due to low scores, this will be helpful in clarifying how the applicants failed to meet the evaluation criteria. When the pages in an application are decreased due to formatting requirements (e.g., page limitations, single spacing, number of files uploaded), the applicant will receive a letter from ACF notifying them why their application has been reduced. In addition, staff are made available by phone and/or email to answer questions from concerned applicants.

    Recommendation: To enhance transparency and fairness in the grant award process, and improve grantees ability to plan for services, the Secretary of Health and Human Services should direct the Assistant Secretary for the Administration for Children and Families to notify applicants about the outcome of their applications as soon as grant award decisions are made.

    Agency Affected: Department of Health and Human Services

    Status: Closed - Implemented

    Comments: In 2010, HHS commented that, as part of the grant application process, the agency plans to explain to applicants that final grant decisions depend on the results of the grant award negotiations between ACF and the prospective grantees. In FY12, HHS reported that immediately following the finalization of funding decisions, organizations whose applications were not selected for funding are notified by letter. In FY13, HHS reported that ACF continues to refine the grant negotiation process to provide timely notification to successful and unsuccessful applicants. Successful applicants will be notified via the GrantSolutions system, informing them how to access their Notice of Award and terms and conditions. Unsuccessful applicants will be notified either by email or hard copy mail. These applicants will also receive the average score and a compilation of the reviewers strengths and weakness statements on all criteria. If the application was reduced, the applicant will also receive a letter explaining the reason for the reduction.

    Jul 16, 2014

    May 29, 2014

    May 22, 2014

    Apr 24, 2014

    Apr 9, 2014

    Jan 30, 2014

    Sep 13, 2013

    May 15, 2013

    Mar 25, 2013

    Looking for more? Browse all our products here