Pipeline Safety:

Systematic Process Needed to Evaluate Outcomes of Research and Development Program

GAO-03-746: Published: Jun 30, 2003. Publicly Released: Jun 30, 2003.

Additional Materials:

Contact:

Jayetta Hecker
(202) 512-8984
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

From 1998 through 2002, a total of 1,770 pipeline accidents occurred, resulting in 100 fatalities and $621 million in property damage. The Office of Pipeline Safety (OPS) within the Department of Transportation operates a research and development (R&D) program aimed at advancing the most promising technologies for ensuring the safe operation of pipelines. In fiscal year 2003, OPS received $8.7 million for its R&D program, a sevenfold increase since fiscal year 1998. In response to a directive from the House Committee on Appropriations, GAO (1) assessed OPS's distribution of funding among various areas of R&D and the alignment of this funding with its mission and goals, (2) surveyed experts to obtain their views on R&D priorities, and (3) determined how OPS evaluates R&D outcomes.

OPS distributes its R&D budget among four main areas. For example, in fiscal year 2003, the office plans to allocate its $8.7 million budget as follows: 46 percent ($4.0 million) to developing new technologies to prevent damage to pipelines and prevent leaks; 21 percent ($1.9 million) to improving technologies for operating, controlling, and monitoring the condition of pipelines; 19 percent ($1.7 million) to improved pipeline materials, such as materials that are resistant to damage and defects; and 14 percent ($1.2 million) to efforts to improve data on the location and safety performance of pipelines. On the basis of our work, we believe that OPS's R&D funding is generally aligned with its mission and pipeline safety goals. OPS has taken a number of steps to ensure this alignment. For example, it obtained the views of a variety of experts and stakeholders in deciding on its R&D priorities and has described in various plans how its R&D efforts can lead to new and improved technologies that can help achieve its safety performance goals, such as reducing the impacts of pipeline accidents. The pipeline safety R&D priorities of the experts we surveyed are generally consistent with OPS's R&D priorities. For example, most assigned a high priority to the two areas of R&D that receive the highest amount of funding from OPS. OPS's efforts to evaluate the outcomes of its R&D have been limited. The agency has taken some preliminary steps toward developing an evaluation process for its R&D program, such as identifying possible measures of program results. Leading research organizations, the Office of Management and Budget, and GAO have identified a number of best practices for systematically evaluating the outcomes of federal R&D programs, such as setting clear R&D goals, measuring progress toward goals, and reporting periodically on evaluation results. These best practices can help OPS to determine the effectiveness of its R&D program in achieving desired outcomes, such as the development and use of new and improved technologies that can enhance pipeline safety.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In response to this recommendation, OPS has developed a process for systematically evaluating the outcomes of its R&D program. This process includes the following components: (1) clear goals, (2) measures of progress toward achieving these goals, (3) expert review to evaluate the quality of research outcomes, and (4) periodic reporting on evaluation results. In its report, GAO had cited each of these as best practices for evaluating R&D that had been identified by leading research organizations, OMB, and GAO.

    Recommendation: To improve OPS's ability to demonstrate the effectiveness of its R&D program and make the most effective use of program funds, the Secretary of Transportation should direct OPS to develop a systematic process for evaluating the outcomes of its R&D program that incorporates identified best practices.

    Agency Affected: Department of Transportation

  2. Status: Closed - Implemented

    Comments: In response to GAO's recommendation, OPS included performance measures for its R&D program in the interagency pipeline R&D report to Congress for fiscal year 2004. This report--which was prepared by OPS, DOE, and the National Institute of Standards and Technology--was provided to Congress in June 2005. OPS plans to continue to report performance measures for its R&D program in future annual reports to Congress on interagency pipeline R&D efforts.

    Recommendation: To improve OPS's ability to demonstrate the effectiveness of its R&D program and make the most effective use of program funds, the Secretary of Transportation should include in the annual reports to Congress, which are required by the Pipeline Safety Improvement Act, information on the results of R&D evaluations.

    Agency Affected: Department of Transportation

 

Explore the full database of GAO's Open Recommendations »

Sep 8, 2020

Aug 20, 2020

Aug 18, 2020

Aug 10, 2020

Jul 27, 2020

Jul 16, 2020

Jun 23, 2020

Apr 30, 2020

Apr 29, 2020

Looking for more? Browse all our products here