This is the accessible text file for GAO report number GAO-02-798 
entitled 'Contract Reform: DOE Has Made Progress, but Actions Needed to 
Ensure Initiatives Have Improved Results' which was released on 
September 13, 2002.



This text file was formatted by the U.S. General Accounting Office 

(GAO) to be accessible to users with visual impairments, as part of a 

longer term project to improve GAO products’ accessibility. Every 

attempt has been made to maintain the structural and data integrity of 

the original printed product. Accessibility features, such as text 

descriptions of tables, consecutively numbered footnotes placed at the 

end of the file, and the text of agency comment letters, are provided 

but may not exactly duplicate the presentation or format of the printed 

version. The portable document format (PDF) file is an exact electronic 

replica of the printed version. We welcome your feedback. Please E-mail 

your comments regarding the contents or accessibility features of this 

document to Webmaster@gao.gov.



Report to the Secretary of Energy:



United States General Accounting Office:



September 2002:



Contract Reform:



DOE Has Made Progress, but Actions Needed to Ensure Initiatives Have 

Improved Results:



DOE Contract Reform:



GAO-02-798:



Contents:



Letter:



Results in Brief:



Background:



DOE Has Made Progress in Implementing Contract Reforms, but Challenges 

Remain:



Contract Reforms May Not Have Improved Contractor Performance:



DOE’s Approach to Contract Reform Was Not Consistent with Best 

Practices:



Conclusions:



Recommendation for Executive Action:



Agency Comments:



Appendix I: DOE’s Decisions to either Compete or Extend 

Non-Competitively Its Major Site Contracts:



Appendix II: Cost and Schedule Performance on DOE’s Major 

Projects as of December 2001:



Appendix III: Comments from the Department of Energy:



Appendix IV: Scope and Methodology:



Appendix V: GAO Staff and Acknowledgments:



GAO Contacts:



Staff Acknowledgments:



Tables:



Table 1: Extent to which DOE Competitively Awarded Its Major Site 

Contracts, 1991-2001:



Table 2: Ten DOE Federally Funded Research and Development Centers That 

Do Not Have Competitively Awarded Contracts:



Table 3: Comparison of Cost Overruns and Schedule Delays for Ongoing 

Projects in 2001 with Ongoing Projects in 1996:



Table 4: DOE Management Improvement Initiatives That Could Enhance 

Contract Reform:



Table 5: Original and Current Cost Estimates and Schedule for DOE 

Projects with Estimated Costs Greater Than $200 Million as of December 

2001:



Figure:



Figure 1: Total Aggregate Fee Available to Contractors on Major Site 

Contracts, by Type of Fee, Fiscal Years 1996 and 2001:



Abbreviations:



DEAR: Department of Energy Acquisition Regulation

DOE: Department of Energy

OIG: Office of Inspector General:



United States General Accounting Office:



Washington, DC 20548:



September 13, 2002:



The Honorable Spencer Abraham

Secretary of Energy:



Dear Mr. Secretary:



The Department of Energy (DOE), the largest civilian-contracting agency 

in the federal government, relies primarily on contractors to operate 

its sites and carry out its diverse missions, such as maintaining the 

nuclear weapons stockpile, cleaning up radioactive and hazardous 

wastes, and performing research. To carry out these missions, DOE often 

contracts for designing, constructing, and operating multimillion-

dollar, one-of-a-kind facilities. For fiscal year 2001, DOE spent about 

90 percent of its total annual budget, or $18.2 billion, on contracts. 

Of that amount, DOE spent about $16.2 billion on contracts to manage or 

operate 28 DOE sites.



DOE’s contracting activities are governed by federal law and 

regulations, including the Federal Acquisition Regulation and the 

Department of Energy Acquisition Regulation (DEAR). Although federal 

law generally requires federal agencies to use competition in selecting 

a contractor, until the mid-1990’s, DOE contracts for the management 

and operation of its sites generally fit within an exception that 

allowed for the use of noncompetitive procedures. Furthermore, those 

contracts were subject to rules in the DEAR that established 

noncompetitive extensions of contracts with incumbent contractors as 

the norm and permitted competition only when it appeared likely that 

the competition would result in improved cost or contractor performance 

and would not be contrary to the government’s best interests. DOE has 

traditionally used a cost-plus-fee type of contract, in which DOE 

reimburses a contractor for all allowable costs under the contract and 

also pays a fee over and above the contractor’s costs. In some 

situations, however, DOE has used a fixed-price contract, in which a 

contractor accepts responsibility for completing a specified amount of 

work for a fixed price. In that case, the contractor earns a profit if 

its total costs are less than the contract price and loses money if its 

total costs exceed the contract price.



For over a decade, DOE has been criticized by GAO, DOE’s Office of 

Inspector General, and others for its contracting practices, 

particularly inadequate management and oversight, and for failure to 

hold its contractors accountable for results. Under its long-standing 

approach to its site contracts, DOE developed a broadly defined 

statement of work, provided considerable direction to the contractor, 

and reimbursed virtually all costs. This approach placed limited 

emphasis on cost control or accountability for results. Furthermore, 

the poor performance of DOE’s contractors led to schedule delays and 

cost increases on many of the department’s major projects. Since 1990, 

such problems have led us to designate DOE contract management--defined 

broadly to include both contract administration and management of the 

projects--as a high-risk area for fraud, waste, abuse, and 

mismanagement.



In 1994, DOE began its contract reform initiative to improve 

contractors’ performance. This initiative recommended numerous 

changes, including making greater use of alternatives to traditional 

site management and operating contracts, such as the use of more fixed-

price contracts; increasing the use of competition as a basis for 

selecting contractors to perform the work; and developing and using 

performance-based contracting tools to reward good performance and 

penalize poor performance. In an earlier review of DOE’s contract 

reform initiative, we reported, among other things, that DOE had issued 

revised policy and guidance to implement the reforms but that it still 

was not using competition for some of its major contracts and that its 

early efforts at performance-based contracting were not linking a 

contract’s objectives to DOE’s overall strategic goals.[Footnote 1]



In addition to its contract reform initiative, DOE has begun several 

other initiatives that could enhance contract reform efforts. These 

include DOE’s efforts to strengthen its management of projects (project 

management initiative); develop and use information systems for 

oversight and control (management information systems initiative); and 

improve training and expertise of the DOE staff overseeing contractor 

activities (human capital initiative). None of those initiatives had 

been fully implemented at the time of our review.



As part of our ongoing review of the high-risk area of DOE contracting 

and project management, we (1) assessed the progress that DOE has made 

since 1996 in implementing contract reform initiatives in key areas; 

(2) determined the extent to which these initiatives have resulted in 

improved contractor performance; and (3) identified the challenges, if 

any, that DOE faces in ensuring the effectiveness of its contract 

reform initiatives.



Results in Brief:



Since 1996, DOE has made progress toward implementing contract reform 

initiatives in three key areas--developing alternative contracting 

approaches, increasing competition, and using performance-based 

contracts. However, DOE continues to encounter challenges in 

implementing these initiatives. Regarding alternative contracting 

approaches, DOE tested several different types of contracts aimed at 

controlling costs while achieving specific results, but it did not 

systematically determine an appropriate contracting approach for a 

given project or activity. For example, the department used 

competitively awarded, fixed-price contracts for both small, relatively 

simple projects, such as laundry services, as well as for large, 

complex cleanup projects. Under a variation of this approach, known as 

“privatization,” the contractor finances, designs, constructs, owns, 

and operates treatment facilities for waste or other material and 

receives payments per unit of treated material. DOE’s experiences 

demonstrated that privatization was not successful in controlling costs 

on large, complex cleanup projects. But privatization worked 

effectively on smaller, less complex projects that posed fewer risks. 

To strengthen the process of selecting a contracting approach, DOE is 

developing and implementing a formal acquisition strategy that 

systematically evaluates contract and financing alternatives and the 

risks associated with various approaches. In the second reform area--

increasing competition--the department changed its contracting rules to 

set competition as the standard approach to awarding contracts. As of 

December 2001, DOE had awarded competitively 14 of 25 contracts (56 

percent) for its major sites, an increase from the 38 percent awarded 

competitively as of 1996. All but one of the contracts that had not 

been awarded competitively as of December 2001 were for research and 

development centers, which, according to federal law, are exempt from 

mandatory competition. Although the department now evaluates whether to 

extend or compete these contracts, thus far it has decided on 

extensions. Finally, DOE now requires performance-based contracts at 

all of its major sites. Such contracts incorporate results-oriented 

statements of work and set the performance objectives and measures that 

DOE will use to evaluate the contractor’s performance. In addition, DOE 

has increased over time the proportion of contractors’ fees tied to 

achieving the performance objectives. Nevertheless, numerous studies 

and reports have criticized DOE’s performance-based contracts for 

ineffective performance measures. DOE continues to modify and test its 

performance measures, for example by improving the linkage between 

performance measures and the department’s strategic goals.



Although DOE has made strides in implementing contract reform 

initiatives, it is difficult to determine whether contractors’ 

performance has improved because objective performance information is 

scarce. Over the past 8 years, DOE has primarily gauged progress by 

measuring its implementation of the reforms, such as the number of 

contracts competed each year, and by reviewing individual contract 

performance incentives. While this information is important, it does 

not help the department determine if, for example, competing more 

contracts resulted in more favorable contract terms or better 

performing contractors, or if performance-based contracts resulted in 

shorter project schedules, reduced costs, or other improvements. 

Nevertheless, DOE program managers and procurement officials at DOE 

headquarters and several sites believe that contract reforms have made 

a difference. These officials offered anecdotes as the primary evidence 

for their view. Officials at DOE’s Albuquerque operations office 

pointed out, for example, that after awarding competitively the 

contract at the Pantex site in Texas, the current contractor met 

required production levels that were not achieved by the previous 

contractor. However, anecdotal examples of poor performance are also 

easily identified. For example, we reported in August 2000 that 

management and oversight failures had caused major cost overruns and 

schedule delays on the National Ignition Facility at the Lawrence 

Livermore National Laboratory. These problems occurred despite 

incorporating performance measures into the overall contract with the 

University of California, which operates the laboratory and manages the 

construction project. Furthermore, according to DOE’s February 2002 

review of its Environmental Management program, the program needed to 

improve significantly its management of performance-based contracts and 

focus on accomplishing measurable results. The Assistant Secretary for 

Environmental Management reported that the program indicators “measured 

process, not progress, opinions, not results.” As a potential indicator 

of improved performance, we evaluated changes in cost and schedule for 

16 of DOE’s current major projects and compared those changes to 

similar information we developed on DOE’s major projects in 1996. We 

found no indication of improved performance--in both groups of projects 

over half of the ongoing projects were experiencing significant cost 

increases, schedule delays, or both. Although the comparison provides 

only a limited view of contractor performance, it does raise questions 

about the overall impact of DOE’s contract reform efforts.



DOE faces a fundamental challenge to ensuring the effectiveness of its 

contract reform initiatives--developing an approach to managing its 

initiatives and sustaining improvements that would incorporate the best 

management practices of high-performing organizations. These practices 

include four key elements: (1) clearly defined goals; (2) an 

implementation strategy that sets milestones and establishes 

responsibility; (3) results-oriented outcome measures, established 

early in the process; and (4) systematic use of results-oriented data 

to evaluate the effectiveness of the initiative and make additional 

changes where warranted. DOE has an implementation strategy for its 

contract reform initiatives, but it largely omitted the other three key 

elements from its reform efforts. For example, by not setting specific 

goals for its contract reform initiatives, DOE emphasized contract 

reform itself as a goal rather than improved results. Similar 

weaknesses in other management improvement initiatives, such as the 

lack of results-oriented outcome measures to gauge progress in DOE’s 

project management initiative, could also limit DOE’s ability to obtain 

better performance from its contractors. However, if successful, the 

project management initiative may help to improve contractor 

performance by providing early indications of problems with cost and 

schedule on projects. We are recommending that to ensure the 

effectiveness of its contract reform initiative, as well as other 

management improvement initiatives, DOE take steps to align its 

management of the initiatives with current best practices.



We provided a copy of our draft report to DOE for review and comment. 

DOE agreed with our recommendation that it adopt an approach to 

implementing management improvement initiatives that is more consistent 

with the best practices of high-performing organizations. However, DOE 

also said that characterizing its contract reform initiative as a 

fundamental management challenge does not fully capture the real issues 

facing the department and creates a false sense that the procurement 

system is capable of solving all of DOE’s problems. Although we agree 

that DOE faces many other challenges, we also believe that, within the 

context of those challenges, strengthening contract management is an 

important need. DOE also said that its contract reform initiative was 

managed in a systematic manner and that our report implies otherwise. 

We revised our report to clarify our concern that DOE’s approach to 

contract reform was not consistent with the best practices found in 

high-performing organizations. Finally, DOE said that evidence of its 

troubled projects has less to do with contract management issues and 

more to do with program and project management issues. We believe that 

the extended discussion in our report fairly describes the relationship 

between contract reform and these other factors. DOE’s comments are 

presented in appendix III.



Background:



DOE’s missions include developing, maintaining, and securing the 

nation’s nuclear weapons capability; cleaning up the environmental 

legacy resulting from over 50 years of producing nuclear weapons; and 

conducting basic energy and science research and development. The 

department carries out these diverse missions at over 50 major 

installations in 35 states. With a DOE workforce of about 16,000 

employees and over 100,000 contractor staff, the department relies on 

its contractors to manage and operate its facilities and accomplish its 

missions. DOE manages these functions through its program offices at 

DOE headquarters and its field offices. The three largest program 

offices--Environmental Management, Defense Programs, and Science--

accounted for over 70 percent of DOE’s budget for fiscal year 2001.



DOE’s reliance on contractors to carry out its missions and the 

department’s history of both inadequate management and oversight and 

failure to hold its contractors accountable for results led us to 

designate DOE contract management as a high-risk area vulnerable to 

fraud, waste, abuse, and mismanagement. In response to these and other 

criticisms, DOE began evaluating its contracting practices and, in 

February 1994, issued a report--Making Contracting Work Better and Cost 

Less--that contained 48 recommendations. The recommendations included 

three key areas: selecting alternatives to traditional contracting 

arrangements used for management and operation of its sites, increasing 

competition to improve performance, and developing and using 

performance-based contracting tools. To facilitate and oversee the 

implementation of the contract reform recommendations, in June 1994, 

DOE established the Contract Reform Project Office, which became the 

Office of Contract Reform and Privatization in 1997. This office, which 

monitored and assessed the progress of DOE’s contract reform 

initiative, was disbanded in late 2001 as part of the department’s 

reorganization of its support offices. DOE’s Office of Management, 

Budget, and Evaluation/Chief Financial Officer is now responsible for 

oversight of DOE’s contract reform efforts.



DOE Has Made Progress in Implementing Contract Reforms, but Challenges 

Remain:



Since 1996, the department has made progress in implementing three key 

contract reform initiatives--developing alternative contracting 

approaches, increasing competition, and converting to performance-

based contracts, although DOE continues to address challenges in 

implementing these initiatives. Concerning alternative contracting 

approaches, DOE encouraged the use of different types of contracts 

aimed at improving contractor performance and results. However, DOE did 

not use a systematic approach to determine the best contract type for a 

given situation and experienced problems with implementation. To become 

more systematic in making this contract selection decision, DOE has 

been developing a formal strategy to evaluate contract and financing 

alternatives and the risks associated with various approaches. In the 

second reform area--increasing competition--DOE changed its 

contracting rules to set competition as the standard approach to 

awarding contracts. Under these rules, the percentage of major site 

contracts awarded competitively (competed) increased to 56 percent as 

of 2001, up from 38 percent as of 1996. All but one of the 11 contracts 

that had not been competed were for managing research and development 

centers exempted by statute from mandatory competition. The department 

evaluates these contracts to determine whether they should be extended 

or competed. DOE has thus far decided on non-competitive extensions for 

these contracts, including some for contractors that have experienced 

performance problems. DOE opted to address these performance problems 

with specific contract provisions, but it remains to be seen whether 

this approach will succeed. Finally, all of DOE’s major site contracts 

are now performance-based, incorporating results-oriented statements 

of work and the performance objectives and measures used to evaluate 

contractor performance. To further emphasize the importance of the 

performance-based approach, DOE has increased the proportion of 

contractor fees tied to achieving the performance objectives to 70 

percent in fiscal year 2001 from 34 percent in fiscal year 1996. 

However, development of good performance measures has continued to be a 

challenge, and DOE acknowledges that it must make further progress in 

this area.



DOE Implemented Alternative Contract Approaches, but Did Not Establish 

an Acquisition Strategy:



One of the major focuses of DOE’s contract reform initiative has been 

developing alternatives to the traditional contracts used for the 

management and operation of its major sites and facilities. Under these 

“management and operating” contracts, one primary contractor performed 

almost all of the work at a site, the contractor had broadly defined 

statements of work, and DOE reimbursed the contractor for virtually all 

costs. As a result, work under these contracts focused more on annual 

work plans and budgets rather than on specific schedule and cost 

targets for accomplishing work.



In implementing alternatives to its traditional contracting 

arrangements, DOE’s intent was to use the best contracting alternative 

given the required work and the objectives and risks associated with 

that work. DOE implemented four main actions as alternatives to these 

management and operating contracts, but has experienced problems with 

implementation, in part due to difficulties in determining the most 

appropriate approach for a given situation, as follows:



* Reducing the number of large, cost-reimbursement contracts that cover 

virtually all of the activities at a DOE site. DOE has modified a total 

of 20 site contracts since 1994, so that no single contractor manages 

and operates those sites. Some of these management and operating 

contracts were divided into smaller service contracts, such as for 

guard services. Other management and operating contracts were changed 

to integration contracts (commonly called management and integration 

contracts). According to DOE officials, integration contracts were used 

to better reflect the changing mission of the site and to better tailor 

the contract scope to the program requirements. Under a management and 

integration contract, one contractor is responsible for integrating the 

work of a variety of subcontractors that carry out most of the actual 

work at the sites. The integrating contractor is responsible for 

selecting “best-in-class” subcontractors for specific work activities, 

overseeing the work done by the subcontractors, and ensuring that 

activities at the site are effectively coordinated. DOE has used this 

integration contract approach at sites such as Oak Ridge in Tennessee 

for environmental restoration work. However, DOE’s Office of Inspector 

General reported in March 2001 that the integrating contractor at Oak 

Ridge has subcontracted out a third less work than originally proposed, 

resulting in less cost savings to the government.[Footnote 2]



* Implementing a more disciplined approach to “make-or-buy” decisions 

by site contractors. DOE revised its regulations in 1997 to require 

that its major site contractors develop make-or-buy plans instead of 

having most of the work at a site performed by the primary contractor. 

Under these plans, the primary contractor must identify work functions 

that could be performed at less cost or more effectively through 

subcontracts. Although all of its major contractors have approved make-

or-buy plans, DOE acknowledges that it does not routinely gather 

information on how much work is done by subcontractors, making it 

difficult to determine the extent to which this approach was 

implemented. In addition, DOE’s Office of Inspector General reported in 

February 2000 that three of the four contractors that it reviewed had 

either not included all functions in their make-or-buy plans or had not 

done the required cost-benefit analysis on work functions that could 

have been subcontracted.[Footnote 3]



* Implementing an alternative contracting and financing approach called 

privatization. DOE started its “privatization initiative” in 1995 as a 

way to reduce the cost and speed the cleanup of its contaminated sites. 

This initiative was primarily an alternative contracting and financing 

strategy to foster open competition for fixed-price contracts; to 

require the contractor to design, finance, build, own, and operate the 

facilities necessary to meet waste treatment requirements; and to pay 

the contractor for units of successfully treated waste. DOE’s 

experiences with this approach showed that privatization could achieve 

cost savings on projects with a well-defined scope of work and few 

uncertainties, such as laundry facilities for contaminated uniforms and 

other items at the Hanford site. However, on complex cleanup projects 

such as the effort at Idaho Falls to clean up Pit 9,[Footnote 4] 

privatization had little success in achieving cost savings, keeping the 

project moving forward on schedule, or getting improved contractor 

performance.



* Establishing “closure contracts” that tie performance incentives to 

contract completion, not to annual activities. DOE has used closure 

contracts at several sites that are scheduled for cleanup and closure, 

including the Rocky Flats site in Colorado and the Fernald site in 

Ohio. These contracts emphasize completing all work at a site or a 

portion of a site by a target date and at a target cost. Most of the 

fee or profit to be earned by the contractor depends upon meeting the 

schedule and cost targets. If the contractor can complete all work on 

time or sooner and below the target cost, then the contractor can earn 

additional fee. For example, under the Rocky Flats closure contract, 

the amount of incentive fee that the contractor can earn ranges from 

$130 million to $460 million, depending on cost and schedule 

performance against the targets. Since the target closure date for this 

contract is December 2006, it remains to be seen whether this approach 

will be effective in completing the work on time and at lower costs to 

the government.



These problems reflected the lack of a systematic approach to deciding 

which contract type was best for a given situation. For example, we 

reported in May of 1998 that DOE’s use of fixed-price contracting was 

appropriate when projects were well-defined, when uncertainties could 

be allocated between DOE and the contractor, and when either adequate 

cost information or multiple competing bidders were available to 

determine a fair and reasonable price for the work.[Footnote 5] 

However, when these conditions did not exist, cost overruns and 

schedule delays could occur on these fixed-price contracts.



DOE has begun to develop a more systematic approach to determining the 

best contract type for a given situation. For example, in October 2000, 

DOE issued new policy and guidance for the acquisition of capital 

assets such as waste treatment facilities. The guidance includes 

developing an acquisition plan that considers the financial, technical, 

and performance risks associated with a new project. This policy is 

consistent with DOE’s overall goal of tailoring the contract type to 

the work to be performed and the business and technical risks 

associated with that work. In addition, to strengthen oversight of 

major acquisitions, in November 2001 DOE issued additional guidance 

that requires approval of acquisition plans for projects of $5 million 

and above at the assistant secretary level or higher. Despite these 

initial steps, DOE is still developing and implementing its formal 

acquisition strategy, and it is too soon to tell whether this new 

strategy will help DOE make better decisions about how to acquire 

capital assets.



Use of Competitively Awarded Contracts Increased, and Allowable 

Exceptions Continue:



DOE has increased the proportion of major site contracts awarded 

competitively, but still extends a number of these site contracts non-

competitively, as allowed by procurement law, including contracts for 

some sites that have experienced contractor performance problems. DOE 

competed 56 percent of its major site contracts that were up for award 

or renewal from 1997 through 2001, a significant increase over the 38 

percent it had competed from 1991 through 1996 (see table 1).



Table 1: Extent to which DOE Competitively Awarded Its Major Site 

Contracts, 

1991-2001:



Major site contract awards: Contracts awarded through competition[A]; 

1991-1996: 11; 1991-1996: (38%); 1997-2001: 14; 1997-2001: (56%).



Major site contract awards: Contracts extended or awarded without 

competition; 1991-1996: ; 18; 1991-1996: ; (62%); 1997-2001: ; 11; 

1997-2001: ; (44%).



Major site contract awards: Totals; 1991-1996: 29; 1991-1996: (100%); 

1997-2001: 25; 1997-2001: (100%).



[A] To be classified as a contract awarded through competition, DOE 

must have issued a request for proposals and a public announcement 

inviting proposals. In addition to the above contracting actions, in 

several cases DOE exercised an option to extend a competitively awarded 

contract for a second period of up to 5 years. From 1991 to 1996, DOE 

exercised an option to extend one such contract non-competitively. From 

1997 to 2001, DOE exercised options to extend seven such contracts non-

competitively. We did not include those non-competitive extensions in 

the table calculations.



Source: GAO analysis of DOE data.



[End of table]



During the 1997 through 2001 period, DOE selected new contractors for 

10 of the 14 competitively awarded contracts, compared to 9 new 

contractors for the 11 competitive awards from 1991 through 1996. 

(Appendix I contains a listing of DOE’s major site contracts in 2001 

and the extent to which they have been competed). The growth in 

competition at major DOE sites is largely a result of new regulations 

the department issued under contract reform. The new rules generally 

require competition for major site contracts and allow a contract 

period consisting of an initial term of up to 5 years with options to 

extend the contract provided that the total contract period does not 

exceed 10 years.



Many of the contracts that DOE did not compete have been for its 

federally funded research and development centers for which DOE may 

extend contracts non-competitively under the Competition in Contracting 

Act of 1984.[Footnote 6] By 2001, all but one of the 11 contracts 

extended without competition fell under this exemption for research and 

development centers. The exception was the major site contract for the 

management of DOE’s West Valley Demonstration Project in New York. DOE 

extended the contract in 1998 and recently announced plans for another 

extension. According to DOE procurement officials, this recent 

extension was because of the limited amount of cleanup work remaining 

at the site and the lack of interest by other contractors to compete 

for the work.



As part of its overall effort to increase competition for site 

contracts, DOE also reassessed which sites it should continue to 

designate as federally funded research and development centers. As a 

result of the reassessment, DOE has removed six of 22 sites from the 

federally funded research and development center designation. The 

department subsequently competed the contracts for two of these, the 

Knolls and Bettis Atomic Power Laboratories in New York and 

Pennsylvania. The department restructured the other four contracts and 

no longer regards them as major site contracts. In six other instances, 

although DOE has thus far decided the sites should remain designated as 

federally funded research and development centers, the department has 

competed the contracts even though federal law and regulations allow 

DOE to extend the contracts non-competitively. These six competed 

contracts included those for the Oak Ridge National Laboratory in 

Tennessee and the Idaho National Engineering and Environmental 

Laboratory.



In addition to its reassessment effort, in 1996 the department issued 

guidance that it must follow to support any recommendation for a non-

competitive extension of any major site contract. Among other things, 

the guidance called for DOE to provide:



* a certification that competition is not in the best interest of the 

department,



* a description of the incumbent contractor’s past performance,



* an outline of the principal issues and/or significant changes to be 

negotiated in the contract extension, and:



* in the case of a federally funded research and development center, a 

showing of the continued need for the research and development center.



Based on such documentation, the agency head can authorize a contract 

extension of up to 5 years. Table 2 lists the ten federally funded 

research and development centers for which DOE has awarded contracts 

non-competitively since this guidance was issued.



Table 2: Ten DOE Federally Funded Research and Development Centers That 

Do Not Have Competitively Awarded Contracts:



Site name: Ames National Laboratory; Site contractor: Iowa State 

University; Year contractor began operating the site: 1943.



Site name: Argonne National Laboratory; Site contractor: University of 

Chicago; Year contractor began operating the site: 1946.



Site name: Fermi National Laboratory; Site contractor: Universities 

Research Association; Year contractor began operating the site: 1967.



Site name: Jefferson Lab; Site contractor: Southeastern Universities 

Research Association; Year contractor began operating the site: 1984.



Site name: Lawrence Berkeley National Laboratory; Site contractor: 

University of California; Year contractor began operating the site: 
1947.



Site name: Lawrence Livermore National Laboratory; Site contractor: 

University of California; Year contractor began operating the site: 
1952.



Site name: Los Alamos National Laboratory; Site contractor: University 

of California; Year contractor began operating the site: 1943.



Site name: Pacific Northwest National Laboratory; Site contractor: 

Battelle Memorial Institute; Year contractor began operating the site: 
1964.



Site name: Princeton Plasma Physics Laboratory; Site contractor: 

Princeton University; Year contractor began operating the site: 1975.



Site name: Stanford Linear Accelerator Facility; Site contractor: 

Stanford University; Year contractor began operating the site: 1976.



Source: GAO analysis of DOE data.



[End of table]



DOE’s decision not to compete some of the federally funded research and 

development center contracts has not been without controversy. For 

example, in 2001, DOE extended the management and operating contracts 

with the University of California for the Los Alamos and Lawrence 

Livermore National Laboratories. The University of California has 

operated these sites for 50 years or more and is the only contractor 

ever to have operated them. In recent years, we and other organizations 

have documented significant problems with laboratory operations and 

management at these two laboratories--particularly in the areas of 

safeguards, security, and project management.[Footnote 7] 

Congressional committees and others have called for DOE to compete 

these contracts. Even with these problems and concerns, however, DOE 

chose not to compete these contracts. This decision was made at the 

highest levels in the department and was based on national security 

considerations. Rather than compete these contracts, DOE intends to 

address these performance problems using contract mechanisms.



In the 2001 contract extension, DOE required the university to focus on 

strengthening management performance in five areas, including 

initiatives for safety and project management. For the first 2 years of 

the 5-year contract period, the University of California must meet 

specific requirements before it can earn any of the $17 million in 

incentive fees available under the contract. DOE is to assess the 

university’s performance on these specific requirements on a pass/fail 

basis. After the first 2 years of the contract, performance in these 5 

areas will be assessed as part of the regular performance measures in 

the contract. The department’s first (2001) annual assessment found 

that the contractor was meeting the required milestones for all of the 

improvement initiatives. However, many of the milestones in the first 

year involved evaluating existing systems or developing action plans. 

For other objectives that focus on results, such as demonstrating 

improved performance in nuclear facility operations, the final outcomes 

will not be known for several years. Therefore, it remains to be seen 

whether DOE will be successful in improving the University of 

California’s performance using these contracting tools. If the 

University of California does not make significant improvements in its 

performance, DOE may need to reconsider its decision not to compete the 

contracts.



Contracts Are Performance-Based, but DOE Has Difficulty Developing 

Effective Performance Objectives and Measures:



DOE has reported that all of its major site contracts incorporate 

performance-based techniques to define requirements and measure 

results. Before DOE initiated its contract reforms, major site 

contracts generally had broad statements of work that focused more on 

annual budgets and work plans rather than specific results to be 

achieved. Fees[Footnote 8] under these contracts usually consisted of a 

base fee that was guaranteed (fixed) plus an award fee that was paid if 

the contractor met general performance expectations. In the mid-1990s, 

DOE began restructuring its major site contracts to use results-

oriented statements of work and, for most of the major site contracts, 

to incorporate performance incentive fees that were designed to reward 

the contractor if it met or exceeded specific performance expectations 

in priority areas. These fees may be tied to either subjective or 

objective performance measures, but DOE regulations suggest the use of 

specific and quantifiable measures whenever possible. In 1999, DOE 

issued additional regulations that limited the use of base fee and 

established a clear preference for contracts where all of the fee was 

based on a contractor’s performance.



Since DOE changed its policy in favor of using incentive fees, there 

has been a substantial shift in the type of fees available on DOE 

contracts. As shown in figure 1, between fiscal years 1996 and 2001, 

DOE decreased the total aggregate amount of base and award fee 

available to its contractors and substantially increased the amount of 

fee that is based on performance incentives. For individual contracts, 

the percentage of each fee type varied widely. For example, in fiscal 

year 2001, the Sandia National Laboratories contract had 100 percent 

base fee, and the Oak Ridge National Laboratory contract had 100 

percent performance incentive fee.



Figure 1: Total Aggregate Fee Available to Contractors on Major Site 

Contracts, by Type of Fee, Fiscal Years 1996 and 2001:



[See PDF for image]



Note: The 2001 fee data excludes two site contracts--Fernald and Rocky 

Flats. These contracts included multi-year incentive fees that will not 

be computed until site closure occurs. The contracts call for site 

closure in 2010 and 2006, respectively.



Source: GAO analysis of DOE Data.



[End of figure]



In addition to shifting most of the fee available to incentive fee, in 

1999, DOE also established a new contract clause making payment of fee 

conditional on meeting certain safety requirements and other minimum 

requirements in the contract. According to language in this clause, in 

order to receive all of the earned fee, the contractor must meet, among 

other requirements, minimum environment, safety, and health 

requirements and avoid any “catastrophic” events such as a fatality or 

serious workplace-related injury. Since 1999, DOE has withheld over $5 

million in fees from six contractors under this conditional payment of 

fee clause. The largest fee withheld--$2 million--was from CH2M Hill 

Hanford Group, Inc., for “failures to meet the contractually imposed 

minimum environment, safety, and health performance requirements” as 

defined by the contractor’s integrated safety management system.



Although these changes reflect a marked shift in DOE’s approach, the 

lack of good performance measures blunted their effect. Since 1997, 

numerous studies and reports--both internal and external to the 

department--criticized DOE’s performance-based contracts for 

ineffective performance measures. Examples include the following:



* DOE’s Office of Inspector General has issued 11 reports since 1997 

that found multiple problems with DOE’s performance measures. In 2001, 

the Inspector General reported, after reviewing the Office of River 

Protection Tank Farm Management, Oak Ridge Y-12 Plant, and Kansas City 

Plant contracts, that DOE was not focusing on high priority outcomes, 

was loosening performance requirements over time without adequate 

justification, and was failing to match appropriately challenging 

contract requirements with fee amounts. The department disagreed with 

this report, stating that it was not appropriate to evaluate the 

overall success of performance-based contracts by looking at individual 

performance measures.



* In 1999, reporting on a self-assessment of its performance-based 

contracting practices, DOE concluded that while significant 

improvements had been made in the management of performance-based 

contracts, several issues had arisen. These issues included 

difficulties with measuring the results of basic science activities, 

establishing performance measures that were consistent with project 

baselines, determining the appropriate use of incentive fees for non-

profit contractors, and balancing incentives that both challenge the 

contractor and continue to reward performance that has been sustained 

at an excellent level.



* In its 1999 review of project management at DOE, the National 

Research Council found that DOE did not always take advantage of the 

performance-based incentive approach and did not have standard methods 

for measuring project performance. The council’s 2001 follow-up 

assessment stressed the importance of using methods such as 

performance-based contracting to focus contractors on achieving desired 

results. The council added that success would be determined by how well 

these methods are followed and recommended that DOE strengthen its 

performance-based contracting guidance and practices.



In response to these and other criticisms of its performance-based 

incentives, DOE has taken several actions that include issuing criteria 

for a performance incentive development process at the field office 

level and focusing on developing performance incentives more directly 

linked to a site’s strategic objectives. For example, DOE officials 

said that multi-year incentives in the Hanford contract and multi-site 

incentives that tie together activities at four production sites--

Kansas City in Missouri, Savannah River in South Carolina, Pantex in 

Texas, and Y-12 at Oak Ridge, Tennessee--strive to establish the 

strategic focus that was absent from performance incentives in earlier 

contracts. DOE officials pointed out that, with these new incentives, 

greater progress was being made. For example, the Hanford site had 

reached its cleanup goals for fiscal year 2001. However, it remains to 

be seen if contractors will meet milestones throughout the contracts’ 

full length and, if they do not, if DOE will require contractors to 

forfeit the provisional fee payments as allowed under the contracts.



Contract Reforms May Not Have Improved Contractor Performance:



Although DOE has made strides in implementing its contract reform 

initiatives and has reviewed the performance measures in many of its 

contracts, the department has developed little objective information to 

demonstrate whether the reforms have resulted in improved contractor 

performance. In the early years of contract reform, DOE measured 

progress in terms of developing and issuing new contracting policies 

and guidance. As new policies were established, the department also 

focused on assessing its progress in implementing these policies in key 

areas of competition and performance-based contracting. More recently, 

DOE has reviewed many of its site contracts to determine, among other 

things, whether the performance incentives are working properly. While 

these steps are useful, this information does not help DOE determine 

outcomes--whether, for example, competing more contracts resulted in 

more favorable contract terms for the government or better performance 

from its contractors. DOE program managers and procurement officials at 

DOE headquarters and several sites believe that contract reforms have 

resulted in improved contractor performance, and they cite a number of 

examples where they believe contractor performance has improved. 

However, there are also numerous examples of contractors who performed 

poorly. Furthermore, DOE’s February 2002 review of its Environmental 

Management program observed that significant progress in cleanup and 

risk reduction had not been achieved despite the performance-based 

contracting approach. Since DOE does not have measures to determine 

whether the contract reform initiatives had resulted in improved 

performance, we examined the extent of cost overruns and schedule 

delays on a number of DOE’s major projects as a partial indicator of 

success. For these projects, cost and schedule data showed no 

improvement when compared to similar data in 1996. While this 

performance information provides only a limited view of department-wide 

contractor performance, it does raise questions regarding the overall 

effectiveness of the reform initiatives.



DOE’s Measurement Focused on Implementation, Not Outcomes:



At the outset of contract reform, DOE established specific action steps 

and related time frames for changing its contracting practices. For 

example, DOE set a goal of developing guidance by August 1994 for 

increasing competition in awarding contracts. Subsequently, DOE 

proposed new regulations concerning contract reforms in the areas of 

competition, performance-based contracting and fee policies. As the 

department’s contract reform activities shifted from issuing guidance 

to restructuring actual contracts, officials began to monitor the 

extent to which its contracting organizations adopted DOE’s contracting 

policy changes in key reform areas. Because the contract cycle for the 

large site contracts was so long--typically contracts were renewed 

about every 5 years--DOE encouraged early incorporation of contract 

reform principles as each contract came up for renewal.



Over the 8 years since the contract reform initiative was introduced, 

DOE has primarily gauged its progress by monitoring implementation of 

the reforms and reviewing individual contracts rather than by 

developing objective measures to determine whether the reforms have 

resulted in improved contractor performance. In addition to tracking 

the number of contracts that incorporated the new requirements to use 

competition and performance-based features, the department reviewed the 

implementation of performance-based contracting for many of its major 

contracts. Some examples of DOE’s monitoring activities include:



* DOE’s annual performance reports required under the Government 

Performance and Results Act contained measures for both competing major 

site contracts and converting them to performance-based contracts. In 

1999, DOE reported that it exceeded the goal of awarding at least 50 

percent of the major site contracts using competitive procedures. In 

the reports for the years 1999, 2000, and 2001, DOE met its performance 

goals to convert all major site contracts awarded in each year to 

performance-based contracts.



* DOE’s Office of Procurement and Assistance Management monitored the 

contracts awarded at major sites. For the years 1997 through 2000, the 

office reported that DOE met its annual goal of awarding contracts that 

were performance-based at all of the major sites.



* DOE maintains a Web site that provides information on the status of 

its procurement goals. These goals include increasing the use of 

competition in awarding contracts and of performance-based concepts in 

those contracts. DOE’s Web site reports that as of 2001, 26 of its 

major site and facility contracts were competed and that 100 percent of 

these major contracts are performance-based.



* In 1997, the department’s self-assessment of contract reform 

determined that progress had been made in implementing contract reforms 

across the complex. However, the report noted difficulties in 

identifying and quantifying contract reform data and recommended on-

going analysis of key reform areas such as the effectiveness of fixed-

price contracting.



* In both 1997 and 1999, the department reported on its use of 

performance-based incentives in major site contracts. The department 

documented considerable progress in developing guidance and in 

incorporating performance-based incentives but also found that early 

incorporation of performance-based concepts had resulted in some poorly 

structured incentives. For example, performance incentives were 

sometimes overly focused on process milestones rather than outcomes. 

The 1997 report recommended issuing guidance on how to restructure 

performance objectives, but not on how to assess the effectiveness of 

the restructured incentives. The 1999 report concluded that the quality 

of contractor performance incentives had improved and that the 

performance incentives were incorporated into contracts in a more 

timely manner. The report further stated that the best measure of the 

effectiveness of the incentives was improvement in contractor 

performance. The report discussed specific contracts but did not 

present overall data on contractor performance.



Procurement and program officials in headquarters continue to be 

actively involved in developing and reviewing performance measures in 

major site contracts. DOE officials said this oversight is improving 

the quality of performance incentives and providing valuable 

information on lessons learned. They acknowledged, however, that DOE 

has not developed objective information on the outcomes associated with 

the reforms. Such results-oriented information is important to 

determine the extent to which the contract changes have resulted in 

improved contractor performance.



Anecdotal Evidence Provides No Overall Measure of Improved Performance:



Although objective performance information focusing on results is not 

available, DOE program managers and procurement officials at both DOE 

headquarters and field operations offices believe that contract reforms 

have made a difference. In support of this view, DOE officials 

generally provide examples that they believe demonstrate improved 

contractor performance. For example, officials at DOE’s Albuquerque 

operations office pointed out that after competing the contract for the 

Pantex site, the new contractor met required production levels that 

were not achieved by the previous contractor. These officials also 

mentioned that the poor performance by the previous contractor was one 

of the deciding factors in competing the contract for the Pantex site.



In addition to the examples of improved performance provided by DOE 

officials, DOE’s 1999 review of its performance-based contracting 

practices reported that “anecdotal evidence supports that the proper 

use of well-structured performance-based incentives is leading to 

improvements in performance at some DOE sites.”[Footnote 9] One of the 

examples cited in this internal review was improved performance at the 

Rocky Flats site under a performance-based contract established in 

1995. Under the previous contract with a broad statement of work, the 

contractor was primarily safeguarding and maintaining facilities at the 

site, and no buildings had been decontaminated, demolished, and 

removed. When DOE competed the contract in 1995 and selected a new 

contractor, DOE also incorporated performance measures into the 

contract. Consistent with these measures, the new contractor 

decontaminated, demolished, and removed six buildings during fiscal 

year 1996 and 12 during fiscal year 1998.



Other examples demonstrate, however, that the instances DOE cites are 

not necessarily representative of the overall performance of DOE’s 

contractors. Examples of poor performance by DOE’s contractors include 

the following:



* DOE has experienced major cost overruns and schedule delays on the 

National Ignition Facility at Lawrence Livermore National Laboratory in 

California. This facility, the size of a football stadium, is designed 

to produce intense pressures and temperatures to simulate in a 

laboratory the thermonuclear conditions created in nuclear explosions. 

DOE considers the facility to be an essential component of the program 

to ensure the safety and reliability of the nuclear weapons stockpile 

in the absence of nuclear testing. Although DOE had incorporated 

performance-based measures and incentives into the overall contract 

with the University of California, which operates the laboratory and 

manages the construction project, performance problems still occurred. 

We reported in August 2000 that the estimated cost of this facility had 

increased from $2.1 billion to $3.3 billion and that the scheduled 

completion date had been extended by 6 years to 2008. We attributed 

these major cost and schedule changes to inadequate management by the 

contractor and DOE oversight failures. [Footnote 10] We also found that 

the performance-based contract placed little emphasis on the National 

Ignition Facility project even though it dominated the laboratory’s 

budget and mission. DOE withheld $2 million of the fiscal year 1999 

performance fee in recognition of the “significant mission disruption” 

caused by problems with this project. DOE officials said that the 

department has since modified the performance-based contract to 

increase the emphasis on this project and has taken additional steps to 

improve both contractor management and DOE oversight.



* DOE has had problems with cost and schedule performance on its 

contract for the Mound site in Ohio. In August 1997, DOE awarded a 

cost-plus-award fee performance-based contract for the accelerated 

cleanup of the Mound site. This contract called for cleaning up the 

site and transferring facilities to the local community by no later 

than September 2005 at a total estimated cost of $427 million. In May 

2001, DOE’s Office of Inspector General reported that the department 

and the contractor had committed to that schedule without knowing 

whether the date was achievable and that the cost and schedule had been 

established with limited knowledge of the soil and building 

contamination.[Footnote 11] The report added that completion of this 

work was estimated for December 2009 at a cost of over $1 billion.



DOE is becoming aware of the problems with relying heavily on anecdotal 

information when trying to assess outcomes. Officials in one of DOE’s 

largest program offices--Environmental Management, representing almost 

a third of the department’s overall budget--recently reported 

fundamental problems with their program, and with the department’s 

ability to manage for results. In a February 2002 review, the office 

stated that although the Environmental Management program had spent 

over $60 billion since 1989, little progress had been made toward 

cleaning up radioactive and hazardous wastes resulting from over 50 

years of producing nuclear weapons, or toward reducing risks to the 

public and the environment.[Footnote 12] During fiscal years 2000 and 

2001, however, most of the contractors at Environmental Management 

sites had earned more than 90 percent of their available performance 

incentive fee, indicating that the contractors were successfully 

achieving the performance goals established in their contracts. The 

Assistant Secretary for Environmental Management reported that if such 

“successes” can take place without significant progress in cleanup and 

risk reduction, the program has been using the wrong set of indicators 

to measure success. She added that Environmental Management program 

indicators “measured process, not progress, opinions, not results.” 

Among the conclusions in the report was that the Environmental 

Management program needed to significantly improve its management of 

performance-based contracts, focus on accomplishing measurable 

results, and align contractors’ performance fees with end points rather 

than intermediate milestones.



Status of Major DOE Projects Does Not Indicate Improved Performance:



Based on our review of the performance of selected projects, it does 

not appear that DOE’s contractors have significantly improved their 

performance since 1996. Because we could not determine whether DOE’s 

contract reform initiatives had resulted in improved performance using 

the department’s measures, we reviewed DOE’s ongoing projects to assess 

whether they were experiencing cost overruns or schedule delays. We 

compared current ongoing DOE projects with estimated total costs 

exceeding $200 million with similar information we developed in 1996 on 

projects[Footnote 13] with estimated total costs exceeding $100 

million.[Footnote 14]



In both 1996 and 2001, over half of the projects we reviewed had both 

schedule delays and cost increases. Furthermore, as shown in table 3, 

the proportion of projects experiencing cost increases of more than 

double the initial cost estimates or schedule delays of 5 years or more 

increased during the 6-year period. For example, the initial cost 

estimate in 1998 for the spent nuclear fuels dry storage project at 

Idaho Falls, Idaho, was $123.8 million with a completion date of 2001. 

Currently, the cost estimate for this project is $273 million with a 

completion date of 2006. Appendix II contains additional information on 

DOE’s ongoing major projects as of December 2001.



Table 3: Comparison of Cost Overruns and Schedule Delays for Ongoing 

Projects in 2001 with Ongoing Projects in 1996:



Number of projects reviewed; Number of projects: 1996: 25[A]; Number 

of projects: 2001: 16[B];



Projects with a cost estimate of more than double the initial cost 

estimate; Number of projects: 1996: 7; Number of projects: 1996: (28%); 

Number of projects: 2001: 6; Number of projects: 2001: (38%).



Projects with schedule delays of 5 years or more; Number of projects: 

1996: 8; Number of projects: 1996: (32%); Number of projects: 2001: 6; 

Number of projects: 2001: (38%).



[A] We evaluated 34 projects in 1996 with estimated costs greater than 

$100 million. However, nine of the projects were environmental 

restoration projects, and DOE’s original and/or current cost estimates 

did not estimate costs through project completion. In 1998, DOE divided 

these environmental restoration projects into multiple projects at each 

site. Therefore, we excluded these projects from our current analysis.



[B] There are 10 additional projects with total project costs greater 

than $200 million, but those projects have either recently started or 

have been suspended.



Source: GAO analysis of DOE data.



[End of table]



The projects we reviewed--with estimated costs ranging from $270 

million to $8.4 billion--may not be representative of all DOE 

projects.[Footnote 15] Although this comparison provides only a limited 

measure of contractor performance, it does raise questions about the 

overall impact of DOE’s contract reform initiative on improving 

contractor performance.



DOE’s Approach to Contract Reform Was Not Consistent with Best 

Practices:



The problems with DOE’s ability to track the results of contract reform 

reflect a broader need to develop an approach to managing its 

initiatives that is more consistent with best practices. As part of our 

review, we looked at best practices for managing improvement 

initiatives. We found that high-performing organizations use a 

systematic results-oriented management approach that includes defining 

goals for the initiative and gauging progress towards those goals. They 

also use information on results to continuously adjust the 

implementation of the initiative and sustain improvements. DOE’s 

approach to contract reform did not incorporate these best practices, 

and its emphasis on measuring progress in terms of implementation 

indicated a focus primarily on contract reform itself as a goal rather 

than improved performance. Furthermore, DOE faces the same fundamental 

challenge--lack of a results-oriented approach--in several other 

management improvement initiatives that, if successful, could enhance 

its contract reform efforts.



DOE Has Not Followed Best Practices in Implementing Its Contract Reform 

Initiative:



DOE’s approach to implementing its contract reform initiatives has not 

followed best management practices. In our review of authoritative 

literature[Footnote 16] we found that leading organizations were able 

to sustain such management improvement initiatives by using a 

systematic, results-oriented approach that incorporated a rigorous 

measurement of progress. Such an approach typically included the 

following steps: (1) define clear goals for the initiative, (2) develop 

an implementation strategy that sets milestones and establishes 

responsibility, (3) establish results-oriented outcome measures to 

gauge progress toward the goals, and (4) use results-oriented data to 

evaluate the effectiveness of the initiative and make additional 

changes where warranted. While DOE followed an implementation strategy 

for its contract reform initiatives, it implemented those initiatives 

largely without clearly defining goals, gauging progress toward those 

goals with results-oriented measures, or using results-oriented data to 

evaluate the effectiveness of its reforms.



Although DOE had set general, overarching goals for its contract reform 

efforts, the department did not further define those goals. As stated 

in the 1994 report of the Contract Reform Team, the overall goal of 

contract reform was to make the department’s contracting process “…work 

better and cost less.” The secretary’s preface to the report presented 

the fundamental problem: “DOE is not adequately in control of its 

contractors. As a result, the contractors are not sufficiently 

accountable to the department, and we are not in a position to ensure 

prudent expenditure of taxpayer dollars in pursuit of our principle 

missions.” However, DOE did not further align those broad goals in 

relation to the specific contract reform efforts. For example, the 

department did not frame its contract reform initiatives to increase 

competition in terms of improved contractor accountability, better 

performance, or reduced costs. While increasing the number of 

competitively awarded contracts is a positive development, it does not 

by itself indicate that the department’s contracting processes work 

better or cost less.



DOE was effective at establishing an implementation strategy that set 

milestones and assigned responsibility for carrying it out. For 

example, DOE’s February 1994 report by its contract reform team 

contained 48 specific reform actions, each containing a required 

action, establishing a deadline, and assigning a specific DOE office 

with responsibility for developing the reform action. These reform 

actions, for the most part, involved developing policies, procedures, 

guidance, and plans to implement reforms such as competitive 

procurements and performance incentives. Our 1996 assessment[Footnote 

17] of DOE progress toward implementing those goals found that DOE had 

completed 47 of 48 reform actions. Since that time, DOE has continued 

to set milestones and assign responsibility for its reform initiatives. 

For example, following an internal review in 1997, the department 

developed another series of actions to improve its implementation of 

reform initiatives pertaining to performance-based incentives. Those 

actions also had milestones for completion and assigned responsibility 

for carrying them out.



DOE did not establish results-oriented outcome measures for its 

contract reform initiatives. Instead, as discussed earlier, DOE 

generally focused on measuring the progress of implementing its reform 

initiatives and reviewing individual contracts, but did not develop 

ways to gauge progress towards its overarching reform goals of making 

contracting work better and cost less. A shortcoming of goals defined 

so generally is the lack of objective ways in which to measure progress 

in meeting those goals. Translating the general goal of “working 

better” into a more specific objective, such as having contractors 

complete a greater number of their projects on time and within budget, 

would have helped the department to identify ways it could measure 

results and, therefore, gauge progress towards the goals of contract 

reform.



Finally, DOE does not have the results-oriented data to evaluate the 

effectiveness of its contract reform initiatives. Because the 

department did not develop clear goals and results-oriented measures, 

it does not have the results-oriented data necessary to systematically 

review progress, take corrective action, and reinforce success. 

Although DOE has received feedback on its reform efforts from internal 

reviews such as self-assessment reports and external reports by the DOE 

Inspector General, GAO, and others, these outside reviews are not a 

substitute for a systematic feedback process.



Despite not following best practices for reform initiatives, DOE has 

taken steps to strengthen the management and oversight of its 

activities. For example, DOE has recently taken steps to integrate 

contract, project, and financial management functions under a single 

office--the Office of Management, Budget, and Evaluation/Chief 

Financial Officer. DOE officials believe that this action will improve 

the coordination, oversight, and control of these important activities.



Other Management Improvement Initiatives That Could Enhance Contract 

Reform Have Similar Problems:



Although DOE’s contract reform initiative has focused on increasing 

competition and holding contractors more accountable for results, DOE 

recognizes that contract reform by itself is not enough to ensure that 

improved contractor performance actually occurs. DOE has begun several 

other initiatives that, if successfully implemented, could enhance its 

contract reform efforts. These initiatives include efforts to 

strengthen its management of projects, develop and use information 

systems for oversight and control, and improve the training and 

expertise of the DOE staff overseeing contractor activities. We 

conducted only a limited review of these initiatives and did not fully 

assess DOE’s implementation against all four steps in a “best 

practices” approach. Nevertheless, we identified instances where, as 

with the contract reform initiative, DOE’s management of the initiative 

fell short of best management practices in one or more areas. Table 4 

below outlines these initiatives, how they could enhance the contract 

reform efforts, and the potential management weakness that could limit 

their effectiveness.



Table 4: DOE Management Improvement Initiatives That Could Enhance 

Contract Reform:



Improvement initiative: Project management; Description/status: Began 

in June 1999 to implement recommendations in National Research Council 

review[A] on how to improve cost and schedule performance on major 

projects; In 2000, DOE issued new policy, order, and guidance on 

managing and controlling projects; In 2001, DOE established a 

project tracking system and required monthly status reporting on all 

projects with total costs over $5 million; In 2002, DOE established 

a goal that 85 percent of major projects have less than a 10 percent 

variance in cost and schedule; How the initiative could enhance 

contract reform efforts: Provide early indications of problems with 

projects’ cost and schedule; Provide DOE managers with information 

necessary to hold contractors accountable for results; Potential 

problem: National Research Council follow-up report[B] found that DOE 

had not clearly defined its goals for the initiative nor developed the 

results-oriented outcome measures to gauge progress.



Improvement initiative: Management information systems; Description/

status: In 2001, DOE began developing a unified planning, programming, 

budgeting and evaluation process to integrate budget and program 

results information; System expected to be available for fiscal year 

2004 budget cycle; How the initiative could enhance contract reform 

efforts: Provide timely feedback on results to DOE managers; Provide 

better management information for both budget requests to the Congress 

and internal management use; Potential problem: DOE has not established 

performance measures and indicators to evaluate progress towards 
meeting 

program goals.



Improvement initiative: Human capital; Description/status: In 1998, DOE 

began developing a training and certification program for acquisition 

management; In 2001, DOE began developing a training and certification 

program for federal project management; In 2001, DOE began efforts to 

address an aging workforce, the need for succession planning, and skill 

gaps; DOE’s September 2001 Five-Year Workforce Restructuring Plan 

included strategies to address acquisition and project management skill 

gaps; How the initiative could enhance contract reform efforts: Provide 

DOE staff with technical and managerial skills necessary for effective 

oversight of contractor activities; Provide DOE staff with knowledge 
and 

experience to assess technical and performance risks and determine 

optimum contracting approach; Potential problem: DOE has not developed 

performance measures to indicate whether the human capital initiatives 

will result in improved performance.



[A] National Research Council, Improving Project Management in the 

Department of Energy (Washington, D.C.: June 1999).



[B] National Research Council, Progress in Improving Project Management 

at the Department of Energy--2001 Assessment (Washington, D.C.: Nov. 

2001).



Source: GAO analysis of DOE and National Research Council data.



[End of table]



Although none of these initiatives have been fully implemented, their 

effectiveness may be limited by the same lack of a results-oriented 

approach to managing the initiative and sustaining improvement as does 

the department’s contract reform efforts.



Conclusions:



Poor performance by DOE contractors and inadequate DOE management and 

oversight of those contractors led us to conclude in 1990 that DOE’s 

contracting practices were at high risk for fraud, waste, abuse, and 

mismanagement. Subsequently, DOE began its contract reform initiative 

to improve the performance and accountability of its contractors. 

Although DOE has undertaken a number of reforms over the years and has 

monitored its progress in implementing those reforms, it has no good 

measure of the results of the reforms. Aside from individual examples 

of good or poor performance on specific projects, DOE cannot tell, for 

example, if the contract reforms have resulted in better performance by 

its contractors or more favorable contract terms for the government. 

Limited evidence we developed suggests that contractors managing DOE’s 

major projects are performing no better in 2001 than on similar 

projects in 1996.



DOE faces a fundamental challenge to ensuring the effectiveness of its 

contract reform initiative--developing an approach to managing the 

initiative that is more consistent with the best practices of high-

performing organizations. DOE’s practices in managing its contract 

reform initiative, as well as its other initiatives such as project 

management, that could also help to improve contractor performance, 

fall short of the best practices followed by high-performing 

organizations. Unless DOE strengthens the way in which it manages 

initiatives such as contract reform, DOE may not be able to fully 

realize the benefits of these initiatives and ensure that its programs 

are adequately protected from fraud, waste, abuse, and mismanagement.



Recommendation for Executive Action:



To improve the effectiveness of DOE’s contract reform initiative, as 

well as other management improvement initiatives, we recommend that the 

department develop an approach to implementing its initiatives that 

incorporates best practices including the key elements of (1) clearly 

defined goals, (2) an implementation strategy that sets milestones and 

establishes responsibility, (3) results-oriented outcome measures, and 

(4) a mechanism that uses results-oriented data to evaluate the 

effectiveness of the department’s initiatives and to take corrective 

actions as needed.



Agency Comments:



We provided a draft of this report to the Department of Energy for its 

review and comment. DOE’s Director, Office of Management, Budget, and 

Evaluation/Chief Financial Officer responded that DOE had three main 

concerns about our report but agreed with our recommendation that DOE 

develop an approach to its management improvement initiatives, such as 

contract reform, that is more consistent with the practices of high-

performing organizations. DOE’s first concern was that the report 

characterizes contract reform as DOE’s fundamental management challenge 

but the report also discusses program and project management issues. 

DOE believes this creates the misperception that the procurement system 

can be used to address the myriad of issues facing the department. We 

believe that our report fairly and accurately describes the context of 

contract management in DOE. Our report identifies contract management 

as a major management challenge for DOE, and one that we have reported 

on for over 10 years. The report does not suggest that contract 

management is DOE’s primary or most fundamental management challenge. 

In fact, we have issued other reports such as our December 2001 report 

on DOE’s major mission, structure, and accountability problems[Footnote 

18] that discuss more fundamental management issues. However, within 

the context of those more fundamental management challenges, DOE can 

and should strive to effectively manage its contracts. Our report does 

not imply that effective contract management will solve the other 

problems facing the department. In fact, the report discusses 

initiatives other than contract reform that are under way at DOE, 

including the project management initiative, because those initiatives 

could also have an impact on the results of the contract reform 

initiative.



DOE’s second concern was that our report concluded that its contract 

reform initiative was not managed in a systematic manner. DOE said that 

its 1994 contract reform initiative was managed systematically and 

included top management oversight, a matrixed implementation team, 

clearly defined goals and objectives, an implementation strategy, and 

identified outcomes. DOE also said it used internal assessments of the 

effectiveness of specific reform initiatives. Our analysis involved 

comparing DOE’s approach to contract reform with the best practices for 

managing improvement initiatives followed by high-performing 

organizations. That comparison showed that DOE’s approach to contract 

reform, and to several other management improvement initiatives, was 

not consistent with those best practices, particularly in the areas of 

defining measurable goals, establishing results-oriented outcome 

measures, or developing results-oriented data with which to measure the 

effectiveness of the initiatives. We revised our report to clarify this 

point. DOE also questioned how we could criticize its approach to 

contract reform when we had recommended in earlier reports that it 

pursue contract reform. Our report does not question the need for 

contract reform in DOE or the components of DOE’s reform initiative, 

such as increasing competition and the use of performance-based 

contracts. Rather, our report assesses what progress DOE has made in 

implementing the initiatives, whether the initiatives have resulted in 

improved contractor performance, and any challenges DOE faces in 

ensuring that its contract reform initiatives are effective.



DOE’s third concern was that the report identifies a limited number of 

projects to support a conclusion that DOE’s contract management system 

is in trouble. DOE believes the problems are more likely due to program 

and project management issues and the risks generally associated with 

unique, technically complex projects and DOE’s funding and political 

environment. We believe that our report fairly characterizes DOE’s 

contract management system. Our report clearly states that DOE has 

developed little objective information to demonstrate whether its 

contract reforms have improved contractor performance. We pointed out 

that anecdotal examples can be used to illustrate both improved 

contractor performance and continued poor contractor performance. And 

we identify other evidence to suggest that contractor performance may 

not have improved. We also acknowledged that other factors, such as 

DOE’s approach to managing projects, could also affect the outcome of 

DOE’s contract reform efforts.



Regarding our recommendation that DOE develop an approach to 

implementing its management improvement initiatives that includes the 

key elements found in the best practices of high-performing 

organizations, DOE agreed with the recommendation and said that it 

would incorporate our observations and recommendation into its future 

improvement efforts.



DOE also provided technical corrections, which we incorporated as 

appropriate. DOE’s written comments on our draft report are included in 

appendix III.



We conducted our review from October 2001 through August 2002, in 

accordance with generally accepted government auditing standards. 

Appendix IV provides details on our scope and methodology.



This report contains a recommendation to you. As you know, 31 U.S.C. 

720 requires the head of a federal agency to submit a written statement 

of the actions taken on our recommendations to the Senate Committee on 

Governmental Affairs and to the House Committee on Government Reform 

not later than 60 days from the date of this letter and to the House 

and Senate Committees on Appropriations with the agency’s first request 

for appropriations made more than 60 days after the date of this 

letter.



Copies of this report are available on request. In addition, the report 

will be available at no charge on the GAO Web site at http://

www.gao.gov. If you or your staff has any questions on this report, 

please call me at (202) 512-3841. Key contributors to this report are 

listed in appendix V.



Sincerely yours,



(Ms.) Gary L. Jones

Director, Natural Resources and Environment:



[Signed by Gary L. Jones]



[End of section]



Appendix I: DOE’s Decisions to either Compete or Extend Non-

Competitively Its Major Site Contracts:



[End of section]



Major site or facility: Ames Laboratory; Contractor in 2001: Iowa State 

University; Year of decision to compete or extend: 1999; Contract 

action[A]: Extended; Budget estimate for fiscal year 2001: $24.8.



Major site or facility: Argonne National Laboratory; Contractor in 

2001: University of Chicago; Year of decision to compete or extend: 

1999; Contract action[A]: Extended; Budget estimate for fiscal year 

2001: 498.3.



Major site or facility: Bettis Laboratory; Contractor in 2001: Bechtel 

Bettis Inc; Year of decision to compete or extend: 1998; Contract 

action[A]: Competed; Budget estimate for fiscal year 2001: 342.0.



Major site or facility: Brookhaven National Laboratory; Contractor in 

2001: Brookhaven Science Associates; Year of decision to compete or 

extend: 1997; Contract action[A]: Competed; Budget estimate for fiscal 

year 2001: 453.2.



Major site or facility: Fermi National Accelerator Laboratory; 

Contractor in 2001: Universities Research Association; Year of decision 

to compete or extend: 2000; Contract action[A]: Extended; Budget 

estimate for fiscal year 2001: 308.0.



Major site or facility: Fernald Environmental Management Project; 

Contractor in 2001: Fluor Fernald, Inc; Year of decision to compete or 

extend: 2000; Contract action[A]: Competed; Budget estimate for fiscal 

year 2001: 290.0.



Major site or facility: Hanford Environmental Restoration; Contractor 

in 2001: Bechtel Hanford Inc; Year of decision to compete or extend: 

1998; Contract action[A]: Extended[B]; Budget estimate for fiscal year 

2001: 177.0.



Major site or facility: Hanford Site; Contractor in 2001: Fluor Hanford 

Inc; Year of decision to compete or extend: 2000; Contract action[A]: 

Extended[B]; Budget estimate for fiscal year 2001: 665.6.



Major site or facility: Idaho National Engineering and Environmental 

Laboratory; Contractor in 2001: Bechtel BWXT Idaho, LLC; Year of 

decision to compete or extend: 1999; Contract action[A]: Competed; 

Budget estimate for fiscal year 2001: 683.0.



Major site or facility: Jefferson Lab; Contractor in 2001: Southeastern 

Universities Research Association; Year of decision to compete or 

extend: 1999; Contract action[A]: Extended; Budget estimate for fiscal 

year 2001: 76.0.



Major site or facility: Kansas City Plant; Contractor in 2001: 

Honeywell Federal Manufacturing and Technologies; Year of decision to 

compete or extend: 2000; Contract action[A]: Competed; Budget estimate 

for fiscal year 2001: 383.5.



Major site or facility: Knolls Atomic Power Laboratory; Contractor in 

2001: KAPL, Inc.; Year of decision to compete or extend: 2000; Contract 

action[A]: Competed; Budget estimate for fiscal year 2001: 269.0.



Major site or facility: Lawrence Berkeley National Laboratory; 

Contractor in 2001: University of California; Year of decision to 

compete or extend: 1997; Contract action[A]: Extended; Budget estimate 

for fiscal year 2001: 320.0.



Major site or facility: Lawrence Livermore National Laboratory; 

Contractor in 2001: University of California; Year of decision to 

compete or extend: 2000; Contract action[A]: Extended; Budget estimate 

for fiscal year 2001: 1,389.1.



Major site or facility: Los Alamos National Laboratory; Contractor in 

2001: University of California; Year of decision to compete or extend: 

2000; Contract action[A]: Extended; Budget estimate for fiscal year 

2001: 2,000.0.



Major site or facility: Mound; Contractor in 2001: BWXT of Ohio; Year 

of decision to compete or extend: 1997; Contract action[A]: Competed; 

Budget estimate for fiscal year 2001: 98.9.



Major site or facility: National Renewable Energy Laboratory; 

Contractor in 2001: Midwest Research Institute; Year of decision to 

compete or extend: 1998; Contract action[A]: Competed; Budget estimate 

for fiscal year 2001: 218.1.



Major site or facility: Nevada Test Site; Contractor in 2001: Bechtel 

Nevada Corp.; Year of decision to compete or extend: 2000; Contract 

action[A]: Extended[B]; Budget estimate for fiscal year 2001: 320.0.



Major site or facility: Oak Ridge Environmental Management; Contractor 

in 2001: Bechtel Jacobs Company, LLC; Year of decision to compete or 

extend: 1997; Contract action[A]: Competed; Budget estimate for fiscal 

year 2001: 546.5.



Major site or facility: Oak Ridge National Laboratory; Contractor in 

2001: UT-Battelle, LLC; Year of decision to compete or extend: 1999; 

Contract action[A]: Competed; Budget estimate for fiscal year 2001: 

769.4.



Major site or facility: Pacific Northwest National Laboratory; 

Contractor in 2001: Battelle Memorial Institute; Year of decision to 

compete or extend: 1997; Contract action[A]: Extended; Budget estimate 

for fiscal year 2001: 457.0.



Major site or facility: Pantex Plant; Contractor in 2001: BWXT Pantex, 

LLC; Year of decision to compete or extend: 2000; Contract action[A]: 

Competed; Budget estimate for fiscal year 2001: 354.7.



Major site or facility: Princeton Plasma Physics Laboratory; Contractor 

in 2001: Princeton University; Year of decision to compete or extend: 

2001; Contract action[A]: Extended; Budget estimate for fiscal year 

2001: 74.1.



Major site or facility: River Protection Project Tank Farm Management; 

Contractor in 2001: CH2M Hill Hanford Group; Year of decision to 

compete or extend: 2001; Contract action[A]: Extended[B]; Budget 

estimate for fiscal year 2001: 402.7.



Major site or facility: Rocky Flats Environmental Technology Site; 

Contractor in 2001: Kaiser-Hill Co. LLC; Year of decision to compete or 

extend: 2000; Contract action[A]: Extended[C]; Budget estimate for 

fiscal year 2001: 661.0.



Major site or facility: Sandia National Laboratories; Contractor in 

2001: Sandia Corporation; Year of decision to compete or extend: 1998; 

Contract action[A]: Extended[B]; Budget estimate for fiscal year 2001: 

1,596.5.



Major site or facility: Savannah River Site; Contractor in 2001: 

Westinghouse Savannah 

River Co.; Year of decision to compete or extend: 2001; Contract 

action[A]: Extended[B]; Budget estimate for fiscal year 2001: 1,431.0.



Major site or facility: Stanford Linear Accelerator Center; Contractor 

in 2001: Stanford University; Year of decision to compete or extend: 

1998; Contract action[A]: Extended; Budget estimate for fiscal year 

2001: 190.0.



Major site or facility: Strategic Petroleum Reserve; Contractor in 

2001: Dyn McDermott Petroleum Operations Company; Year of decision to 

compete or extend: 1998; Contract action[A]: Extended[B]; Budget 

estimate for fiscal year 2001: 101.8.



Major site or facility: Waste Isolation Pilot Plant; Contractor in 

2001: Westinghouse TRU Solutions LLC; Year of decision to compete or 

extend: 2000; Contract action[A]: Competed; Budget estimate for fiscal 

year 2001: 101.3.



Major site or facility: West Valley Demonstration Project; Contractor 

in 2001: West Valley Nuclear Services; Year of decision to compete or 

extend: 1998/2002; Contract action[A]: Extended; Budget estimate for 

fiscal year 2001: 107.4.



Major site or facility: Y-12 National Security Complex; Contractor in 

2001: BWXT Y-12, LLC; Year of decision to compete or extend: 2000; 

Contract action[A]: Competed; Budget estimate for fiscal year 2001: 

567.4.



Major site or facility: Yucca Mountain Site; Contractor in 2001: 

Bechtel SAIC; Year of decision to compete or extend: 2000; Contract 

action[A]: Competed; Budget estimate for fiscal year 2001: 294.5.



[A] To be classified as a competitively awarded contract, DOE must

have issued a request for proposals and a public announcement inviting 

proposals.



[B] DOE competitively awarded the first 5-year contract and 

subsequently exercised its option to extend the contracts for up to 5 

more years, in accordance with its regulations.



[C] The 1995 contract for the Rocky Flats site did not include an 

option for an extension. When the closure of the site was accelerated 

to 2006, DOE decided to extend the contract and renegotiate it as a 

closure contract.



Source: GAO presentation of DOE data.



[End of table]



[End of section]



Appendix II: Cost and Schedule Performance on DOE’s Major Projects as 
of 

December 2001:



The following table shows the original and current cost estimates and 

completion dates for ongoing DOE projects with estimated costs greater 

than $200 million. The table does not include 10 additional DOE 

projects with estimated costs greater than $200 million because the 

projects were suspended or only recently started as of December 2001.



Table 5: Original and Current Cost Estimates and Schedule for DOE 

Projects with Estimated Costs Greater than $200 Million as of December 

2001:



Dollars in millions.



Advanced Mixed Waste Treatment Project (97-PVT-2)[C]; Cost: Original 

cost estimate[B]: $1,078.9; Cost: Current cost estimate: $1,087.7[D]; 

Cost: [Empty]; Schedule: Original completion date: December 2002; 

Schedule: Current completion date: December 2002.



Civilian Radioactive Waste Management Program[E]; Cost: Original cost 

estimate[B]: 6,300.0[F]; Cost: Current cost estimate: 8,394.6; Cost: 

[Empty]; Schedule: Original completion date: October 2001[F]; Schedule: 

Current completion date: December 2004.



Dual-Axis Radiographic Hydrodynamic Test Facility (97-D-102)[G]; Cost: 

Original cost estimate[B]: 30.0[H]; Cost: Current cost estimate: 269.7; 

Cost: [Empty]; Schedule: Original completion date: September 1990; 

Schedule: Current completion date: December 2002.



East Tennessee Technology Park Three-Building Decontamination and 

Decommissioning and Recycle Project (OR-493); Cost: Original cost 

estimate[B]: 283.9; Cost: Current cost estimate: 348.1; Cost: [Empty]; 

Schedule: Original completion date: December 2003; Schedule: Current 

completion date: March 2004.



Facilities Capability Assurance Program (88-D-122)[I]; Cost: Original 

cost estimate[B]: N/A[J]; Cost: Current cost estimate: 445.6; Cost: 

[Empty]; Schedule: Original completion date: N/A[J]; Schedule: Current 

completion date: June 2000.



Hanford Tank Waste Treatment and Immobilization Plant (01-D-416); Cost: 

Original cost estimate[B]: 12,488.0[K]; Cost: Current cost estimate: 

4,350.0; Cost: [Empty]; Schedule: Original completion date: 2007; 

Schedule: Current completion date: 2007.



High-Level Waste Removal from Filled Waste Tanks (93-D-187)[L]; Cost: 

Original cost estimate[B]: 88.6[M]; Cost: Current cost estimate: 

1,550.5; Cost: [Empty]; Schedule: Original completion date: September 

1999[M]; Schedule: Current completion date: September 2028.



Initial Tank Retrieval Systems (94-D-407); Cost: Original cost 

estimate[B]: 245.0[N]; Cost: Current cost estimate: 274.9; Cost: 

[Empty]; Schedule: Original completion date: March 2000[N]; Schedule: 

Current completion date: December 2015.



National Ignition Facility (96-D-111); Cost: Original cost estimate[B]: 

1,073.6; Cost: Current cost estimate: 2,248.1; Cost: [Empty]; Schedule: 

Original completion date: June 2002; Schedule: Current completion date: 

September 2008.



Silos; Cost: Original cost estimate[B]: N/A; Cost: Current cost 

estimate: 338.1; Cost: [Empty]; Schedule: Original completion date: N/

A; Schedule: Current completion date: December 2006.



Spallation Neutron Source (99-E-334); Cost: Original cost estimate[B]: 

1,332.8; Cost: Current cost estimate: 1,411.7; Cost: [Empty]; Schedule: 

Original completion date: September 2005; Schedule: Current completion 

date: June 2006.



Spent Nuclear Fuel Dry Storage (98-PVT-2)[O]; Cost: Original cost 

estimate[B]: 123.8; Cost: Current cost estimate: 273.0; Cost: [Empty]; 

Schedule: Original completion date: June 2001; Schedule: Current 

completion date: December 2005.



Hanford Spent Nuclear Fuels; Cost: Original cost estimate[B]: 714.8; 

Cost: Current cost estimate: 1,600.0; Cost: [Empty]; Schedule: Original 

completion date: 2001; Schedule: Current completion date: September 

2006.



Tank Farm Restoration and Safe Operations (97-D-402); Cost: Original 

cost estimate[B]: 289.2; Cost: Current cost estimate: 285.3; Cost: 

[Empty]; Schedule: Original completion date: June 2005; Schedule: 

Current completion date: June 2005.



Tritium Extraction Facility (98-D-125)[P]; Cost: Original cost 

estimate[B]: 390.7; Cost: Current cost estimate: 401.0; Cost: [Empty]; 

Schedule: Original completion date: June 2005; Schedule: Current 

completion date: March 2006.



Weldon Springs Site Remedial Action Project; Cost: Original cost 

estimate[B]: 357.7[Q]; Cost: Current cost estimate: 905.2; Cost: 

[Empty]; Schedule: Original completion date: September 1995[Q]; 

Schedule: Current completion date: September 2002.



[A] Projects that are not funded as construction line items do not have 

project numbers. All costs, unless otherwise specified, are “total 

project costs.” The cost data were obtained from DOE Congressional 

budget requests and other DOE-provided data. The term N/A means cost or 

schedule not available or not yet developed.



[B] For consistency we used, when available, preliminary budget 

estimates submitted to Congress as the basis for original cost 

estimates.



[C] Total project cost for construction projects typically includes 

only the design, construction, and startup costs that precede 

production operations. Total project cost for this project also 

includes estimated costs for over 10 years of production operations and 

other associated costs. The current completion date refers to 

completion of the construction phase.



[D] The contractor has submitted a “Request for Equitable Adjustment” 

of over $48 million due to a six-month schedule slip the project 

experienced as a result of a delay in the issuance of environmental 

permits. Because the Request for Equitable Adjustment is still under 

review, the $48 million is not included in the current cost estimate.



[E] The original baseline for this program included construction of the 

exploratory studies facility and, if suitable, a site recommendation 

and a license application. The current scope of the program was 

broadened in 1997 to include all elements of the Civilian Radioactive 

Waste Management Program, which now includes development of license 

application, design and construction of Yucca Mountain Repository, 

licensing interactions with the Nuclear Regulatory Commission, and 

development of a transportation system. The current completion date is 

only for the license application.



[F] We reported in 1996 that the current cost and completion date for 

the Yucca Mountain Site Characterization Project were $4,300 million 

and March 2002, respectively. In 1997, DOE expanded the project to 

include the entire Civilian Radioactive Waste Management Program.



[G] The original scope of this project at initial authorization in 1988 

included two buildings and two single pulse flash x-ray machines. The 

project has since undergone several changes in scope, which now 

includes three buildings, a containment vessel to reduce emissions to 

the environment, a single pulse machine, and a multiphase machine.



[H] This amount is a total estimated cost from the fiscal year 1988 

Budget Request, which does not include other project costs. Other 

project costs include supporting research and development and plant 

support costs during construction, activation, and startup. There was 

no requirement for a total project cost estimate in 1988.



[I] This project has a few subprojects completing closeout activities 

and two still underway. DOE anticipates additional funding needs and a 

schedule extension to complete the final two subprojects.



[J] We reported in 1996 that the current cost for the Facilities 

Capability Assurance Program was $447 million and the completion date 

was not available. No cost estimate was available when the project was 

originally proposed.



[K] This original cost estimate from the fiscal year 2001 Budget 

Request was based upon the privatization concept and included plant 

operations through fiscal year 2018.



[L] DOE expanded the original scope of this project in fiscal year 1994 

to incorporate three ongoing projects, which increased the total 

project cost from $88.6 million to $828 million and the project 

completion date from 1999 to 2008 in the fiscal year 1996 budget. The 

cost and schedule were revised again in fiscal year 2000 to include, 

among other projects, the equipment and infrastructure required to 

remove the high level waste inventory from nine additional tanks.



[M] We reported in 1996 that the current cost and completion date for 

the High Level Waste Removal project were $828.2 million and September 

2008, respectively. DOE expanded the scope of this project in 1994.



[N] We reported in 1996 that the current cost and completion date for 

the Initial Tank Retrieval System project were $358.2 million and March 

2010, respectively.



[End of table]



[O] The original and current estimated costs include design, 

construction, startup, and operating costs. The current completion date 

refers to completion of the construction and startup phase.



[PI] n June 2002 DOE’s Office of Inspector General reported that the 

total project cost for the Tritium Extraction Facility could increase 

to as much as $500 million and that the facility may not be completed 

until December 2006.



[Q] We reported in 1996 that the current cost and completion date for 

the Weldon Springs Remedial Action Project were $865.0 million and 

2001, respectively.



[End of section]



Source: GAO analysis of DOE and National Research Council data.



[End of section]



Appendix III: Comments from the Department of Energy:



Department of Energy Washington, DC 20585:



AUG 27 2002:



Ms. Gary L. Jones:



Director, Natural Resources and Environment, U.S. General Accounting 

Office:



441 G Street, NW Room 2964 Washington, DC 20548:



Dear Ms. Jones:



We have reviewed the draft General Accounting Office (GAO) Report 

entitled “Contract Reform: DOE Has Made Progress, but Actions Needed to 

Ensure Initiatives Have Improved Results” (GAO-02-798). Specific 

technical comments to the draft report are attached. We appreciate the 

opportunity to discuss these matters with you and are pleased that the 

report recognizes the Department of Energy’s (DOE) positive 

achievements in a number of areas related to contract and project 

management, particularly in GAO’s recognition of DOE’s progress in 

implementing its 1994 Contract Reform initiative. That early initiative 

was an important first step in what is an important and continuing area 

of management attention in DOE. Your past advice has assisted us in 

defining parallel, as well as, subsequent improvement initiatives. The 

report also identifies DOE’s progress in such areas as: use of 

competition for its major site and facility management contracts; the 

use of performance-based contracting approaches, including new contract 

types that shift more performance and financial risk to the contractor; 

and improvements in project management.



We continue to have a basic concern with GAO’s formally stated opinion 

that contract reform is DOE’s fundamental management challenge. We note 

that the original audit focus was contract reform, but this focus was 

broadened to include program and project management. Having broadened 

its scope of inquiry, however, the GAO’s report title and overall 

summation remains the same. That is, the report continues to underscore 

the misperception that the terms and conditions of the contract, and 

procurement regulations and guidance, can fully address the myriad of 

issues facing DOE.



We also disagree with the report’s apparent conclusion that the 1994 

Contract Reform initiative was not managed in a systematic manner. To 

the contrary, the initiative had the clear support of the Secretary, a 

top-level board of directors, a project manager, a matrixed 

implementation team, clearly defined goals and objectives, an 

implementation strategy and schedule, identified roles and 

responsibilities, identified outcomes and tracking mechanisms. 

Furthermore, DOE used a number of mechanisms, most notably internal 

assessments of the effectiveness of specific contract reform 

initiatives, to determine whether the reforms were accomplished and to

identify their impact. Where problems were identified, corrective 

measures have been taken; where changes did not result in anticipated 

outcomes, adjustments have been made. Parallel and follow-on reform 

initiatives not discussed in this report have been integrated into the 

Department’s strategic planning and performance measurement systems. 

Although GAO has conducted numerous past assessments of the 1994 

contract reform effort and was very familiar with the Department’s 

methodology, it has not critiqued DOE’s management approach. 

Furthermore, we note that GAO questions certain of the original 

recommendations/objectives of contract reform which were in direct 

response to earlier criticisms and recommendations made by the GAO and 

Congressional Committees.



Regarding the issues related to performance-based contracting, 

measures, and incentives, we agree that the concept of performance-

based contracts presents a difficult challenge and that progress is 

evolutionary. In this regard, a recent study by outside consultants 

compared DOE facility management/remediation contracts to similar 

contracts awarded by other federal agencies. The study found that DOE 

major site and facility contracts, when compared to other agencies’ 

contracts, generally evidenced significantly higher risk factors (for 

the contractor) and technical complexity and somewhat lower available 

fees. The study team also found that the other-agency contracts, while 

characterized as performance-based, relied on largely subjective 

performance objectives and measures not unlike the cost-plus-award fee 

contracts used by DOE prior to 1994. On the other hand, DOE has 

aggressively pursued the use of performance-based contracting 

approaches and is one of the few Federal agencies where performance 

management has been reflected in its contracts.



We are also concerned that the report anecdotally identifies a limited 

number of troubled projects to support its contention that DOE’s 

contract management system is in trouble. A review of the projects at 

issue would indicate the difficulties have less to do with contract 

management issues than with programmatic and project management issues 

and the significant risks generally associated with one-of-a-kind 

projects involving advanced technologies with extensive complexity, 

uncertain funding, and difficult political environments.



Notwithstanding the foregoing, we agree with the report’s conceptional 

recommendation that DOE adopt systematic approaches to developing, 

implementing and managing various management initiatives. Finally, we 

note that the GAO report presents a historical snapshot of the 

Department’s implementation of performance-based contracts. The 

Department continues to strive toward putting effective and efficient 

contracts in place that will result in achieving or exceeding the 

Department’s objectives at its various sites. Commencing in 2001, the 

Department put in place numerous contracts with either multi-year 

incentives or which were awarded on a cost-plus-incentive-fee 

completion basis. These contracts not only provide incentives to the 

contractor to achieve the Department’s objectives, but if possible, 

exceed them.



Even with the placement of these contracts, the Department continues to 

review its approach to performance-based contracting as evidenced by 

the numerous internal reviews/reports examining the Department’s 

implementation of these contracts. The Department continues to seek 

ways to expand the implementation of what works and correct what does 
not. 

However, this is a learning process, and requires time to fully 

implement. We appreciate the GAO’s observations and recommendations and 

will incorporate them into our future improvement efforts.



Bruce M. Carnes, Director, Office of Management, Budget and Evaluation/
Chief 

Financial Officer:



[Signed by Bruce M. Carnes]



Attachment:



[End of section]



Appendix IV: Scope and Methodology:



To assess the progress that DOE has made since 1996 in implementing 

contract reform initiatives in the key areas of developing alternative 

contracting approaches, increasing competition, and using performance-

based contracts, we reviewed DOE’s three self-assessment reports on 

contract reform efforts and GAO and DOE Office of Inspector General 

reports on DOE contract and project management since 1996. We also 

interviewed officials from DOE’s Offices of Contract Management and 

Procurement and Assistance Policy, and procurement officials with the 

National Nuclear Security Administration. The National Nuclear Security 

Administration, a semi-autonomous agency within DOE, has its own 

procurement organization. However, since both entities follow the same 

policies, regulations, and guidance, we have not made a distinction in 

this report between contracts and projects of the two organizations. To 

assess the extent to which DOE had incorporated the key contract 

reforms into its major facility contracts, we obtained information on 

33 contracts that DOE’s headquarters procurement office identified as 

site or facility management contracts. We reviewed the contract award 

history of these major facility contracts, to determine which contracts 

had been competed as of 1996 and as of 2001. To qualify as a 

competitively awarded contract, DOE must have issued a request for 

proposals and a public announcement inviting proposals. We also 

obtained data on annual budgets and fees available and earned for these 

same contractors for fiscal years 1996 through 2001. We did not attempt 

to validate this information provided by DOE. In addition, we reviewed 

documentation for major facility contracts obtained from DOE’s 

Albuquerque Operations Office, Richland Operations Office, and the 

Office of River Protection.



To determine the extent to which these initiatives have resulted in 

improved contractor performance, we interviewed DOE officials from the 

Office of Contract Management and the three largest program offices--

Environmental Management, Defense Programs, and Science. In addition, 

we interviewed procurement and program office officials at DOE’s 

Albuquerque Operations Office, Richland Operations Office, and the 

Office of River Protection. We reviewed documents they provided, 

including the procurement organization’s balanced scorecard. In 

addition, we reviewed DOE’s February 2002 review of the Environmental 

Management program, and numerous GAO and Inspector General reports. 

Because DOE did not have objective results-oriented measures of 

contractor performance, as a potential indicator of that performance, 

we developed information as of December 2001 on the cost and schedule 

performance of DOE’s ongoing projects and compared that information 

with similar information we developed in 1996 on DOE major system 

acquisitions. In 1996, DOE categorized a “major system acquisition” as 

a project with a total project cost greater than $100 million. When we 

began our review in January 2002, we learned that DOE had since raised 

the threshold of “major project” to $400 million. Since our compilation 

of DOE reported data revealed only 19 ongoing projects that meet the 

current $400 million threshold (nine of which had recently started or 

were on hold), we expanded our scope to projects with total project 

costs greater than $200 million, in order to compare results on a 

similar number of projects. Those projects were under the management 

and oversight of DOE’s site contractors or under privatization projects 

under DOE’s oversight. There may be other projects with total project 

costs greater than $200 million, but they were not identified by DOE 

during our review. Because DOE does not maintain centralized data on 

its projects, we obtained information from project management offices 

within DOE and its National Nuclear Security Administration. We did not 

verify the data obtained from DOE, but we did examine the 

reasonableness of these data based on information in prior GAO reports 

and audits. For consistency, we used, when available, preliminary 

budget estimates submitted to the Congress as the basis for original 

cost estimates and completion dates, comparing those to current cost 

estimates and completions dates as of December 2001. For this report, 

we used, wherever possible, the projects’ “total project cost,” which 

includes construction and operating funds. Where these costs are not 

available, we used the “total estimated cost,” which includes 

construction costs. We have footnoted the latter. (See appendix II.):



To identify the challenges, if any, that DOE faces in ensuring the 

effectiveness of its contract reform initiatives, we reviewed the 

reports of the National Research Council on improving DOE project 

management. In addition, we reviewed reports and other documentation 

from the National Academy of Public Administration, the Project 

Management Institute, and prior GAO work to develop best practices 

criteria for managing improvement initiatives. We compared DOE’s 

implementation of its contract reform initiative to these best 

practices criteria to determine areas of concern. To identify the other 

management improvement initiatives that could impact contract reform, 

we reviewed the reports of the National Research Council, GAO and 

Inspector General; the President’s Management Agenda for fiscal year 

2002; and DOE’s 5-year workforce restructuring plans. We also 

interviewed DOE officials in the Office of Engineering and Construction 

Management and the Office of Program Analysis and Evaluation.



We conducted our review from October 2001 through August 2002 in 

accordance with generally accepted government auditing standards.



[End of section]



Appendix V: GAO Staff and Acknowledgments:



GAO Contacts:



William Swick (206) 287-4800:



Staff Acknowledgments:



In addition to those named above, Carole Blackwell, Robert Crystal, 

Doreen Feldman, Molly Laster, Patricia Rennie, Carol Shulman, Stan 

Stenersen, and Arvin Wu made key contributions to this report.



GAO’s Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAO’s commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAO’s Web site (www.gao.gov) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select “Subscribe to daily E-mail alert for newly 

released products” under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548:



To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537

Fax: (202) 512-6061



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470



Public Affairs:



Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548:



FOOTNOTES



[1] U.S. General Accounting Office, Department of Energy: Contract 

Reform Is Progressing, but Full Implementation Will Take Years, 

GAO-RCED-97-18 (Washington, D.C.: Dec. 10, 1996).



[2] U.S. Department of Energy, Bechtel Jacobs Company LLC’s Management 

and Integration Contract at Oak Ridge, DOE/IG-0498 (Washington, D.C.: 

Mar. 21, 2001).



[3] U.S. Department of Energy, The Department’s Management and 

Operating Contactor Make-or-Buy Program, DOE/IG-0460 (Washington, 

D.C.: Feb. 17, 2000).



[4] The Pit 9 project involved an effort to clean up 250,000 cubic feet 

of buried radioactive and hazardous wastes. The waste treatment 

operations were to start in August 1996 and be completed by February 

1999. At the time the contract was terminated because of problems with 

contractor performance, the project was at least 26 months behind 

schedule, had been assessed $940,000 by state regulators for failing to 

meet deadlines, and the contractor had requested an additional $257 

million over the contract price.



[5] U.S. General Accounting Office, Department of Energy: Alternative 

Financing and Contracting Strategies for Cleanup Projects, 

GAO-RCED-98-169 (Washington, D.C.: 

May 29, 1998).



[6] Under the Competition in Contracting Act of 1984, an agency may 

award new contracts or extend existing ones with educational or 

nonprofit institutions or federally funded research and development 

centers non-competitively in order to maintain an essential research 

and development capability. See 41 U.S.C. 253 (c)(3).



[7] For example, see U.S. General Accounting Office, Department of 

Energy: Key Factors Underlying Security Problems at DOE Facilities, 

GAO/T-RCED-99-159 (Washington, D.C.: Apr. 20, 1999); U.S. General 

Accounting Office, Nuclear Security: Improvements Needed in DOE’s 

Safeguards and Security Oversight, GAO/RCED-00-62 (Washington, D.C.: 

Feb. 24, 2000); and A Special Investigative Panel, President’s Foreign 

Intelligence Advisory Board, Science at its Best, Security at its 

Worst: A Report on Security Problems of the U.S. Department of Energy 

(Washington, D.C.: June 1999).



[8] The contract fee, or profit, is the amount DOE pays to the 

contractor over the allowable costs.



[9] U.S. Department of Energy, Follow-up Assessment of the 

Effectiveness of Actions Taken to Improve Performance-Based Incentives 

in Performance-Based Management and Management and Integration 

Contracts (Washington, D.C.: Mar. 31, 1999).



[10] U.S. General Accounting Office, National Ignition Facility: 

Management and Oversight Failures Caused Major Cost Overruns and 

Schedule Delays, GAO/RCED-00-271 (Washington, D.C.: Aug. 8, 2000).



[11] U.S. Department of Energy, Remediation and Closure of the 

Miamisburg Environmental Management Project, DOE/IG-0501 (Washington, 

D.C.: May 2, 2001). 



[12] U.S. Department of Energy, A Review of the Environmental 

Management Program, (Washington, D.C.: Feb. 4, 2002).



[13] U.S. General Accounting Office, Department of Energy: Opportunity 

to Improve Management of Major System Acquisitions, GAO/RCED-97-17 

(Washington, D.C.: Nov. 26, 1996).



[14] We selected projects with estimated total costs exceeding $200 

million in 2001 to ensure a roughly equivalent sample size of projects 

compared to the number of projects we reviewed in 1996, while limiting 

the sample size to a manageable number of projects.



[15] As of January 2002, DOE records indicated at least 42 ongoing 

projects with estimated costs greater than $100 million. We did not 

review all of DOE’s capital projects with costs over $100 million 

because of the level of effort that would have been required, since DOE 

does not maintain centralized information on those projects. 

Furthermore, five of the ongoing projects we reviewed in 2001 began 

before the advent of DOE’s contract reform initiatives. 



[16] This literature included publications from the National Academy of 

Public Administration, the Project Management Institute, the Six Sigma 

model for process improvement, and past GAO studies on implementation 

of the Government Performance Results Act and other management 

initiatives.



[17] Department of Energy, Contract Reform Is Progressing, but Full 

Implementation Will Take Years, RCED-97-18 (Washington, D.C.: Dec. 10, 

1996).



[18] U.S. General Accounting Office, Department of Energy: Fundamental 

Reassessment Needed to Address Major Mission, Structure, and 

Accountability Problems, GAO-02-51 (Washington, D.C.: Dec. 21, 2001).



GAO’s Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAO’s commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAO’s Web site ( www.gao.gov ) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select “Subscribe to daily E-mail alert for newly 

released products” under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office



441 G Street NW,



Room LM Washington,



D.C. 20548:



To order by Phone: 	



Voice: (202) 512-6000:

TDD: (202) 512-2537:

Fax: (202) 512-6061:



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov



Automated answering system: (800) 424-5454 or (202) 512-7470:



Public Affairs:



Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.



General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.



20548: