Airport Cooperative Research Program Addresses Many Needs but Could Enhance Transparency and Clarify Scope of Research Role
GAO-10-729: Published: Jul 15, 2010. Publicly Released: Jul 15, 2010.
Airports are a vital part of the nation's air transportation system and face many similar challenges. In 2003, the Airport Cooperative Research Program (ACRP) was authorized to conduct applied research to help airport operators solve shared challenges that are not addressed by other federal research. As requested, this report addresses (1) the extent to which ACRP's processes reflect criteria for conducting a high-quality research program and (2) ACRP's results to date and their usefulness for the aviation community. GAO reviewed ACRP documentation and compared ACRP processes to criteria previously developed by GAO that can be applied to research programs. These criteria identify three phases of the applied research process and steps to help produce high-quality results. GAO also reviewed ACRP projects and publications and interviewed ACRP stakeholders and airport officials.
In each of the three phases of applied research that GAO has identified, ACRP conducts its research with processes that align with many of GAO's criteria for producing high-quality research, but some gaps exist. (1) Selecting projects: ACRP has established a governing board, the ACRP Oversight Committee (AOC), which is composed of airport executives and other key industry stakeholders, and processes to determine the research needs of users and to select specific projects for funding. However, one organization that participates on the board--the Airport Consultants Council--and the consensus approach used to make project selection decisions are not included in the program's documented operating procedures. ACRP stakeholders commended the council's participation and the consensus approach, but their omission from documentation potentially diminishes program transparency. (2)Implementing projects: ACRP's processes for establishing a project panel to manage research projects, selecting a researcher, and overseeing projects are well documented and include quality control steps. However, product dissemination efforts may miss some potential users, particularly staff at smaller airports and mid-level staff. The AOC has initiated a project to improve research dissemination to better serve these groups, although the project's scope and time frame is still being determined. (3) Evaluating projects and the program overall: ACRP maintains considerable information on ongoing and completed projects that are used by program managers and the AOC to review project progress. The program, however, does not currently have a systematic process for evaluating the impact of individual projects or implementing continuous improvements to the program's overall performance. Two initiatives--the dissemination project and a project initiated to review ACRP processes--could address current gaps in project and program evaluation, though the scope and time frames of these projects are still being determined. Through 2009, ACRP approved 169 projects, about half of which have been completed, and published 66 products on topics such as environmental impacts, policy and planning, and administration. Airport operators and other ACRP stakeholders consistently told GAO that the program provides the industry with useful and unique research that individual airports, particularly smaller airports, have neither the time nor budget to conduct. However, ACRP's role in conducting security research is unclear. ACRP materials, such as its annual solicitation of project ideas, include security as a potential topic within the scope of the program. However, the AOC has not recently funded security projects, in part because of differing views about whether ACRP should do this research. The Federal Aviation Administration, as a member of the AOC, indicated that the Department of Homeland Security is a better venue for such research. Conversely, other AOC members told GAO that ACRP could address some unmet security research needs. The AOC has the authority to determine what role, if any, is appropriate for ACRP in this area. By not doing so, over time, user satisfaction with the program could decline. GAO recommends that the Secretary of Transportation (1) ensure ACRP documentation reflects all participants and governance practices and (2) clarify ACRP's role in conducting security research. The Department of Transportation generally agreed with the report, provided technical comments, and is considering the recommendations. The Department of Homeland Security and the Transportation Research Board did not provide any comments on the draft report.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: In 2010, we reported that the Air Cooperative Research Program (ACRP) had implemented many practices and procedures that help to assure the production of high-quality applied research, but gaps existed in some areas of its research processes. The program's documented operating procedures did not reflect (1) the involvement of the Airport Consultants Council (ACC) on the ACRP Oversight Committee (AOC) (2) the enhanced role of ex-officio members in project selection decisions, and (3) the AOC's current consensus-driven approach to selecting projects. This incomplete documentation of certain program practices potentially diminished the transparency of some aspects of the AOC's operations. Therefore, we recommended that the Department of Transportation, and other relevant entities, revise key program documentation to align with actual board membership and that ACRP use its project selection rules. In 2010, ACC was formally appointed to ACRP's governing board and program documentation--such as annual reports--was updated accordingly. Additionally, the governing board holds a formal vote--after the board discusses proposed projects--to make project selections. In 2013, we confirmed that ACC's appointment is reflected in updated documentation and continued adherence to formal project selection practices provide greater program transparency and stability, as intended by our recommendation, and substantively address the gaps we identified in our report. As a result, ACRP operations are more transparent to program stakeholders.
Recommendation: To better align key program documentation with ACRP program practices as implemented and to increase the transparency and stability of the program over time, the Secretary of Transportation should take steps to revise the Memorandum of Agreement (MOA) between the Department of Transportation and the National Academy of Sciences, and other appropriate documentation, such that (1) all organizations, including ex-officio members, that are involved in ACRP Oversight Committee (AOC) proceedings are included in program documentation, and (2) project selection procedures documented in the MOA are supplemented to include, as an option, a consensus-based approach in addition to voting procedures, and a more explicitly defined role for ex-officio members in project selection.
Agency Affected: Department of Transportation
Status: Closed - Implemented
Comments: In 2010, we reported that the Air Cooperative Research Program (ACRP) had produced a variety of high-quality and useful results for the airport industry, according to the officials we interviewed. However, the program's role in conducting security research was unclear. ACRP materials, such as its annual solicitation of project ideas, included security as a potential topic within the scope of the program. However, the ACRP Oversight Committee (AOC) had not generally funded security projects, in part because of differing views about whether ACRP should do this research. Airport security was certainly a concern shared by airport operators and AOC members. Given the significant amount of aviation security research that the Department of Homeland Security (DHS) had conducted, ACRP may or may not have been the appropriate venue to address unmet needs of airports in this area. Although FAA had stated that ACRP should not conduct security research, it was the responsibility of the AOC as a whole, of which FAA was a part, to determine what role, if any, the program had in this area. The AOC had discussed the program's role in addressing security, but it was not clear what actions, if any, it planned to take to resolve the lack of clarity. This lack of agreement among AOC members about the program's role in addressing security research had left the program, in effect, not addressing security research, but holding out the possibility that it could do so. Therefore, we recommended that the Department of Transportation, with the involvement of other relevant entities, clearly articulate ACRP's role, if any, in security research and update program documentation accordingly. Between 2010 and 2013, FAA, the Transportation Research Board (TRB), and the Transportation Security Administration (TSA) conferred about each entity's role in airport security research. A memorandum of agreement between FAA and TSA, signed in 2011, explains that ACRP will continue to accept and review security research proposals, but will forward these proposals to the TSA for evaluation and possible inclusion in a research program it manages. The memorandum of agreement also requires a process for soliciting security research proposal from the airport community and the wide distribution of research reports to the airport community. Finally, the solicitation for fiscal year 2015 projects disseminated to the airport community does not include security as a project area. As a result, AOC members--who represent airports and other industry stakeholders--have a better understanding of what research can be expected from ACRP.
Recommendation: To clarify the role of ACRP in conducting security research, the Secretary of Transportation should take steps to encourage the AOC--in collaboration with other key federal agencies and stakeholders--to clearly articulate ACRP's role, if any, in conducting security research and, subsequently, to ensure that ACRP's program documentation clearly and accurately reflects this role, such that airport operators and others can readily understand what to expect of the program in this area.
Agency Affected: Department of Transportation