Public Transportation: FTA's Triennial Review Program Has Improved, But Assessments of Grantees' Performance Could Be Enhanced

GAO reported in June 2009 on the Federal Transit Administration's (FTA) Triennial Review Program--an evaluation of grantees' adherence to federal requirements at least once every 3 years. Our analysis of FTA's oversight tracking data found that a few grantees consistently met nearly all federal requirements while all grantees reviewed generally faced problems in 5 requirement areas, such as procurement and maintenance requirements. We reported that while FTA analyzed some data to track and monitor oversight activities, it did not fully analyze oversight tracking data to identify, for example, high performing grantees along with management practices (i.e., best practices) that contributed to their high performance. FTA agreed that sharing best practices among grantees contributes to improved performance since grantees learn from other grantees, including high performing grantees. FTA also did not analyze oversight tracking data to identify problem areas and their underlying causes. Identifying the problem areas and their common underlying causes could provide FTA with valuable information to help many grantees find solutions to correct their findings and improve grantees' long-term performance. Therefore, we recommended that FTA further analyze and use oversight tracking data on the triennial review to, for example, help grantees address common problem areas and sustain their performance by applying best practices learned from high performing grantees. Starting in 2010, FTA began analyzing oversight data to identify the overall top 10 problem areas and the causes for each of these problem areas-information that is now available on its public website and has been discussing these problem areas with grantees during triennial review workshops each year. FTA has also begun sharing best practices among all grantees by publishing these best practices in the fiscal year 2011 and 2012 triennial review preparation workbooks. As a result, these actions will provide FTA with valuable opportunities to increase the triennial review program's effectiveness in helping grantees improve their performance.

In June 2009, we reported that the Federal Transit Administration's (FTA) Triennial Review Program could be strengthened. Federal law requires FTA to review and evaluate completely grantees' adherence to federal requirements at least once every 3 years. However, guidance to contractors conducting the reviews provided flexibility to not review some areas in certain circumstances and documentation did not clearly show that grantees were receiving a complete review once every 3 years. For example, 10 triennial review reports for 2008 showed that the drug and alcohol program area was "not reviewed" because these grantees had received special drug and alcohol reviews in fiscal year 2006. Because this area was not reviewed in 2008, these grantees may not be reviewed in this area for 5 years (this area was last reviewed during the 2006 special oversight review and will not be reviewed again until the 2011 triennial review). Therefore, we recommended that the FTA Administrator help ensure that a complete performance review and evaluation of each grantee is conducted at least once every three years, in accordance with the statutory requirement, by revising the contractors' guide to remove the "not reviewed" category, requiring an updated evaluation of areas reviewed in the prior two years of the triennial review, and describing how contractors should document the triennial review reports with updates to other reviews, such as special oversight reviews conducted in the prior two years. In response, FTA revised its 2010 triennial review guidance, which states for all areas with special oversight reviews that even if such a special oversight review is scheduled for the current fiscal year or has been recently conducted, all questions are still to be reviewed. Proper evaluation and documentation provides future reviewers with a more transparent audit trail of findings and gives FTA managers the confidence that grantees were completely reviewed and evaluated in all areas at least once every three years.

In June 2009, we reported that the Federal Transit Administration's (FTA) Triennial Review Program, which evaluates grantees' adherence to federal requirements at least once every 3 years, could be strengthened. FTA Order 5400.1 established an oversight review council and gave the council the responsibility to develop and coordinate oversight review schedules of various oversight reviews to avoid unnecessary duplication and overlap. However, the council was not fulfilling its responsibility as oversight coordinator. In a limited number of cases, for example, the council did not coordinate the schedules of FTA's special oversight reviews with the triennial review, which may have placed an undue burden on a few grantees receiving multiple oversight reviews in the same fiscal year. Therefore, we recommended that the FTA Administrator help ensure that the council, or another body, carries out the responsibilities of the oversight review coordinator, as specified in the FTA order. FTA responded to our recommendation by creating a publicly-available, oversight master calendar that allows FTA staff, contractors, and others to know when special oversight reviews, such as procurement, civil rights, or financial management reviews, have occurred and coordinate the scheduling of future oversight reviews. Furthermore, the current calendar shows special oversight reviews in the upcoming 6-months as well as reviews within in the past year or more. As a result, FTA could increase its oversight coverage of grantees and minimize the undue burden of uncoordinated multiple oversight reviews on some grantees.

The Federal Transit Administration (FTA) oversees grants to transit agencies serving urban areas (grantee) in part through its triennial review program. The reviews provide FTA with an opportunity to evaluate urban area grantees' use of federal funds in areas such as procurement, financial management, and drug and alcohol programs. FTA's reviews can result in findings of deficiency- findings that represent areas for which the grantee is not adhering to federal requirements. In June 2009, GAO reported that within the triennial review program its "Close Findings Timely" performance measure links throughout the organization, had measurable targets, were clearly stated and did not overlap. However, the Close Findings measure did not meet the objectivity and reliability attribute. Specifically, GAO's analysis of the Close Findings measure found that its objectivity was limited because decisions to close findings were done largely at the discretion of regional staff, in consultation with grantees, and, when appropriate, headquarters officials. GAO found regional staff may use different criteria for when to close a deficient finding. The lack of criteria reduces the transparency of FTA's performance in closing findings. For example, an FTA official stated that the Closing Findings Timely performance measure could result in the unintended consequence of prematurely closing findings. Given these findings, the triennial review program could be improved to minimize the possibility for manipulation and increase the reliability of closed finding data. Therefore, GAO recommended that FTA should improve the objectivity and reliability with which FTA ensured deficient findings were satisfactorily closed. This would help reduce the incidents of non-compliance. In 2020, GAO confirmed that FTA developed a Quality Assurance Program for its reviews of grantees. Among other things, this improved review program is designed to promote consistency in implementation of the program. As part of quality monitoring, FTA will determine the percent of findings closed only after sufficient evidence of resolution has been provided. The program will also ensure that interpretation of statutes, regulations, guidance, issuance of findings, and application of corrective actions is consistent. This approach will help FTA ensure that grantees have satisfactorily resolved findings before closing them. Thus, reducing incidents of continuing or repeated non-compliance FTA is better positioned to determine the validity of reported results and make appropriate decisions.

FTA oversees grants to transit agencies serving urban areas (grantee) in part through its triennial review program. The reviews provide FTA with an opportunity to evaluate urban area grantees' use of federal funds in areas such as procurement, financial management, and drug and alcohol programs. In June 2009, GAO reported that FTA lacked the performance measures for evaluating the outcome and quality of the triennial review program. The performance measures assess the output of the triennial review program. Outcome performance measures-such as grantees' performance over time-can help evaluate the results of the triennial review programs. Without such a measure, FTA was limited in the extent to which it could effectively ascertain and document the extent to which the triennial review program was improving grantees' performance in meeting federal requirements and FTA managers had limited information on program performance, making the possibility of achieving program goals less likely. Therefore, GAO recommended that FTA should develop a method for evaluating improvements in grantee performance in meeting more federal requirements over time. In 2020, GAO confirmed that FTA developed a Quality Assurance Program for its reviews of grantees. Among other things, this improved review program is designed to promote grantee compliance. For example, as part of quality monitoring, FTA will determine the percent of grantees that demonstrate an increase or decrease in compliance with federal requirements from the prior review. Further, FTA will look at the percent of recipients that have repeat findings from the previous review. The program will also use its oversight reviews to promote recipient understanding of federal requirements and thus long-term compliance. As a result of this program, FTA is better positioned to provide decision makers the essential information they need determine areas for improvement and increase the likelihood of achieving program goals, such as increasing the number of grantees that meet federal requirements.

FTA oversees grants to transit agencies serving urban areas (grantee) in part through its triennial review program. The reviews provide FTA with an opportunity to evaluate urban area grantees' use of federal funds in areas such as procurement, financial management, and drug and alcohol programs. In June 2009, GAO reported that FTA's triennial review program performance measures address only one of three governmentwide priorities and, therefore, are not balanced-an attribute that applies to the overall suite of measures, rather than the measures individually. Balanced performance measures allow managers to understand what other triennial review aspects are working well and why. Adding outcome and quality performance measures will help balance the existing timeliness measures and could allow managers to better set triennial review priorities by not overemphasizing one or two priorities at the expense of others. FTA officials recognize current performance measures did not assess some aspects of the triennial review program and were undertaking efforts to enhance them. Therefore, GAO recommended that FTA should ensure additional outcome and quality performance measures are balanced with the existing measures for assessing the timeliness of the triennial review program so that one priority is not emphasized at the expense of others. In 2021, GAO confirmed that FTA had taken sufficient action to address the intent of this recommendation. Specifically, FTA established the Quality Assurance Program (QA) to enhance the quality and outcomes of the Triennial Review Program. The QA program created measures of recipient compliance that assess outcomes of the Triennial Review Program. These include the percent of recipients that demonstrate an increase or decrease in recipient compliance with federal requirements from the prior review as well as the percent of recipients that have repeat findings from the prior review, with a focus on findings that have a material impact on the riding public and/or FTA's financial interest. Through this action, FTA has enhanced its performance measures for evaluating the outcome and quality of the triennial review program, which will enable managers to better set triennial review priorities without overemphasizing one priority at the expense of others.