Army Corps of Engineers: Peer Review Process for Civil Works Project Studies Can Be Improved
Highlights
What GAO Found
Since enactment of the Water Resources Development Act of 2007, 49 project studies have undergone peer review but it is unclear how many were performed in response to section 2034 requirements because the Army Corps of Engineers (Corps) does not make specific determinations or track if a peer review is being conducted under section 2034. In February 2011, in response to section 2034, the Corps submitted its initial report to Congress summarizing its implementation of the peer review process. In its report, however, the Corps did not distinguish which studies had been selected for peer review in accordance with section 2034 and therefore, did not provide Congress information that would help decision makers evaluate the requirements of section 2034 at the end of the trial period.
The 49 peer reviews resulted in both direct and indirect costs. Specifically, these peer reviews resulted in direct costs of over $9 million in contract costs and fees. In addition, Corps staff resources were used to manage the reviews, although these costs are not fully quantifiable. Furthermore, the addition of peer review to the Corps study process has resulted in indirect costs by altering project study schedules to allow for time needed to complete peer reviews. In some cases where a peer review was not planned during the early stages of the study process, significant delays to project studies occurred while funds were sought to pay for the peer review. In contrast, according to some Corps officials, when project managers have built in time and identified funding for peer reviews early, the process has had less of an impact on project study schedules.
The Corps’ process for determining whether a project study is subject to peer review is more expansive than section 2034 requirements because it uses broader criteria, resulting in peer reviews of studies outside the scope of section 2034. In addition, the process the Corps uses does not include the flexibility provided in section 2034, which allows for the exclusion of certain project studies from peer review. Moreover, some studies are undergoing peer reviews that do not warrant it, according to some Corps officials GAO spoke with.
The Corps has a process to review general information on contractors’ conflicts of interest and independence when selecting them to establish peer review panels, but it does not have a process for reviewing project-level information on conflicts of interest and independence. As a result, it cannot be assured that contractors do not have conflicts at the project-level. In contrast, the Corps’ contractors do have a process for reviewing information related to conflicts of interest and the independence of experts selected for each peer review panel.
The Corps has adopted and incorporated into its project study reports most of the peer review recommendations it has received. Doing so has resulted in some technical improvements to study reports but generally has not changed the Corps’ decisions about project alternatives, in part because the peer review process occurs too late in the project study process to affect decision making, according to some Corps officials GAO spoke with. As a result, some recommendations about alternatives may not have been implemented because the decision on the preferred design had already been made.
Why GAO Did This Study
Section 2034 of the Water Resources Development Act of 2007 requires that certain U.S. Army Corps of Engineers (Corps) civil works project studies undergo independent external peer review to assess the adequacy and acceptability of the methods, models, and analyses used. In the act, Congress established a 7-year trial period for this requirement and also required the Corps to submit two reports on its experiences with the peer review process.
GAO was asked to examine (1) the number of Corps project studies that have undergone independent peer review in response to section 2034, (2) the cost of these peer reviews, (3) the extent to which the Corps’ process for determining if a project study is subject to peer review is consistent with section 2034, (4) the process the Corps uses to ensure that the contractors it hires and the experts the contractors select to review project studies are independent and free from conflicts of interest, and (5) the extent to which peer review recommendations have been incorporated into project studies. GAO reviewed relevant laws, agency guidance, and documents and interviewed Corps officials and contractors.
Recommendations
GAO recommends that the Department of Defense direct the Corps to, among other actions, better track peer review studies, revise the criteria for determining which studies undergo peer review and the timing of these reviews, and improve its process for ensuring contractor independence. The department generally concurred with these recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | To facilitate congressional evaluation of the 7-year trial period outlined in section 2034, the Secretary of Defense should direct the Chief of Engineers and Commanding General of the U.S. Army Corps of Engineers to identify for each past and future peer review the specific statutory authority under which the peer review was conducted and the criteria triggering peer review under the Corps' civil works review policy. |
Closed – Implemented
In a November 2013 report to Congress, the Corps provided a list of project studies that identified the criteria triggering peer review for the studies. According to the Corps, it developed and implemented a method to list the specific authorities under which each peer review was conducted and the criteria triggering the peer review under Corps policy, a list that the Corps plans to update annually. In June 2014, Congress passed the Water Resources Reform and Development Act of 2014 (WRRDA) which changed the criteria for project studies required to undergo peer review. Specifically, WRRDA increased the project cost threshold requiring peer review from projects that have an estimated total cost of more than $45 million to those with estimated total costs over $200 million.
|
| Department of Defense | To better reflect section 2034 and provide more effective stewardship of public resources and ensure efficient and effective operations, the Secretary of Defense should direct the Chief of Engineers and Commanding General of the U.S. Army Corps of Engineers to revise the criteria in the Corps' process for conducting peer review to focus on larger, more complex, and controversial projects; to encourage peer review to occur earlier in the study process; and to include exclusions to peer review that align with section 2034. |
Closed – Not Implemented
In December 2012, the Corps updated its processes for implementing product review requirements for civil works projects outlined in Engineering Circular 1165-2-214 which includes its peer review policy. However, the Corps did not revise the criteria for conducting peer review to align with section 2034 of the Water Resources Development Act of 2007. In addition, the Water Resources Reform and Development Act of 2014 increased the threshold for requiring peer review from projects with an estimated total cost of $45 million to $200 million, consistent with our conclusions that the Corps should focus on larger, more complex, controversial projects. The Corps has not updated its policy guidance since the change in legislation.
|
| Department of Defense | To better reflect section 2034 and provide more effective stewardship of public resources and ensure efficient and effective operations, the Secretary of Defense should direct the Chief of Engineers and Commanding General of the U.S. Army Corps of Engineers to develop a documented process to ensure that contractors are independent and free from conflicts of interest on a project-specific basis. |
Closed – Implemented
Based on our recommendation, the Corps established a process to obtain information from contractors about their independence and potential conflicts of interest on a project-specific basis. In November 2012, the Corps provided us with a questionnaire it developed for completion by potential contractors as part of each project-specific solicitation. According to the Assistant Secretary of the Army, any potential conflicts of interest indicated on the form will be further investigated by the contracting officer to determine the appropriate agency response to any circumstances identified.
|