Defense Logistics:

Improved Analysis and Cost Data Needed to Evaluate the Cost-effectiveness of Performance Based Logistics

GAO-09-41: Published: Dec 19, 2008. Publicly Released: Dec 19, 2008.

Additional Materials:

Contact:

William M. Solis
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

In 2001, the Department of Defense (DOD) identified performance based logistics (PBL) as the preferred weapon system support strategy. Within DOD, PBL is the purchase of performance outcomes, such as system availability, rather than the purchase of individual elements of logistics support--such as parts, repairs, and engineering support. Although PBL initially arose from efforts to reduce support costs, questions have arisen about whether PBL has reduced support costs as originally intended. GAO was asked to evaluate the extent to which DOD has used business case analyses to guide decisions related to PBL arrangements and the impact PBL arrangements have had on weapon system support costs. In conducting the review, GAO analyzed the implementation of PBL arrangements for 29 weapon system programs. GAO also looked at the use and characteristics of performance-based contracting in the United Kingdom's Ministry of Defence.

Although DOD's guidance recommends that business case analyses be used to guide decision making regarding the implementation of PBL to provide weapon system support, the services are not consistent in their use of such analyses. About half of the DOD program offices responsible for the 29 PBL arrangements GAO reviewed either did not use a business case analysis or could not provide documentation for significant parts of their analyses. Almost all of the remaining analyses were missing one or more of the recommended elements in DOD's instruction for economic analysis. Finally, business case analyses were often not updated in accordance with service policies and guidance. Program office use of these analyses is inconsistent because DOD only recommends, but does not require, that they be prepared and because DOD's guidance on preparing a business case analysis is not comprehensive and does not adequately specify the criteria to be included. Also, most of the services have not established effective internal controls to ensure that the analyses are prepared or that they provide a consistent and comprehensive assessment. As a result, DOD has implemented PBL arrangements without the benefit of sound analyses that ensure that the chosen approach will provide the most cost-effective support option. While one of DOD's goals in moving toward the use of PBL arrangements was to reduce weapon system support costs, the ability of these arrangements to reduce costs remains unclear 7 years after DOD first identified PBL as the preferred weapon system support strategy. Many DOD program offices that implemented PBL arrangements have limited cost data, and various other factors--such as the lack of business case analyses--further limit an evaluation of the costs of this support strategy. Available data from the programs GAO reviewed indicated mixed results. Although a few programs in GAO's sample provided evidence of some cost reductions, GAO's analysis of the only two systems in its sample that are managed using both a PBL arrangement and a more traditional, non-PBL arrangement indicated that in both cases the PBL arrangement had higher costs. Also, GAO found that certain characteristics of DOD's PBL arrangements--contract length, funding stability, ownership of inventory, and the lack of cost metrics and effective incentives--could limit the ability of and incentive for contractors to reduce support costs. Neither DOD nor the services require detailed cost reporting for PBL arrangements and the lack of detailed cost data hinders DOD's ability to determine whether PBL has reduced support costs as intended. GAO describes the use of performance-based arrangements for weapon system support in the United Kingdom's Ministry of Defence, which the ministry refers to as contracting for availability. The Ministry of Defence began awarding availability contracts as an approach to reduce weapon system support costs, and officials believe that support cost reductions have been achieved as a result of using availability contracts. In general, the availability contracts used are significantly longer than those used by DOD, and the ministry uses an "open book accounting" arrangement to gain visibility into the contractors' costs to provide support.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: In its comments, DOD concurred with our recommendation and noted that the Army had implemented internal controls, including a review and approval process, necessary to ensure that business case analysis (BCA) are conducted. DOD planned to review the Army's internal control processes as part of its Product Support Assessment Team (PSAT) for potential expansion into DOD-wide policy. However, internal controls were not addressed in the final PSAT report, issued in November 2009, and a DOD-wide BCA internal control requirement has not been issued.

    Recommendation: To ensure that PBL arrangements are the most cost-effective option for weapon system support, the Secretary of Defense should direct the Under Secretary of Defense (Acquisition, Technology and Logistics) to require that each service revise guidance to implement internal controls to ensure that program offices prepare and update business case analyses that are comprehensive and sound.

    Agency Affected: Department of Defense

  2. Status: Closed - Implemented

    Comments: DOD issued a Product Support Business Case Analysis Guidebook in April 2011. According to the guidebook, weapon system program offices should conduct a revalidation of the previous business case analysis every five years or prior to a change in support strategy. The revalidation should examine the actual results versus the planned or estimated results and include four primary categories of information: operations, cost, performance, and funding.

    Recommendation: To ensure that PBL arrangements are the most cost-effective option for weapon system support, the Secretary of Defense should direct the Under Secretary of Defense (Acquisition, Technology and Logistics) to revise PBL business case analysis guidance to more clearly define when business case analyses should be updated during the weapon system life cycle.

    Agency Affected: Department of Defense

  3. Status: Closed - Implemented

    Comments: DOD issued a Product Support Business Case Analysis Guidebook in April 2011 to standardize business case analyses. Among other information, the guidebook provides a business case analysis outline and a checklist. The guidebook also references DOD's economic analysis instruction (DODI 7041.3) and Office of Management and Budget Circular A-94, Benefit Cost Analysis of Federal Programs, both of which provide criteria for analyzing the costs and benefits of DOD's weapon system support arrangements.

    Recommendation: To ensure that PBL arrangements are the most cost-effective option for weapon system support, the Secretary of Defense should direct the Under Secretary of Defense (Acquisition, Technology and Logistics) to revise PBL business case analysis guidance to more clearly define what should be included in a business case analysis and to establish specific criteria and methods for evaluating PBL support arrangements, including evaluation at the subsystem and component levels.

    Agency Affected: Department of Defense

  4. Status: Closed - Not Implemented

    Comments: DOD concurred with this recommendation and stated that the next iteration of its acquisition policy would require that the use of business case analyses be mandatory. In October 2010, DOD issued an interim directive, recently updated to be valid until August 2013, which requires the periodic revalidation of business case analyses for major defense acquisition programs and major weapon system programs. According to officials, DOD is in the process of revising its acquisition policy. However, DOD has not yet completed the revision.

    Recommendation: To ensure that PBL arrangements are the most cost-effective option for weapon system support, the Secretary of Defense should direct the Under Secretary of Defense (Acquisition, Technology and Logistics) to revise DOD's Acquisition Directive to require development of a business case analysis to support the decision-making process regarding weapon system support alternatives, including PBL.

    Agency Affected: Department of Defense

  5. Status: Closed - Not Implemented

    Comments: DOD partially concurred with our recommendation and has required the collection and reporting of support cost data for certain contractor performance-based logistics arrangements. However, it has not yet clarified the detailed operating and support (O&S) cost elements that should be collected and reported. As noted in DOD's written comments, DOD Instruction 5000.02 was issued in December 2008 and included language stating that sustainment contracts for acquisition category I programs shall provide tailored cost reporting that can facilitate future cost estimating and price analysis. However, the instruction does not specify a consistent reporting format or the O&S cost data to be collected. Additionally, in November 2010, DOD added a clause in the Defense Federal Acquisition Regulation Supplement (DFARS) that requires cost reporting for specific contracts. This clause requires detailed cost reporting for contracts for major defense acquisition programs and major automated information system programs above $50 million. (The DFARS further allows managers for these types of programs with contracts valued between $20 million and $50 million to direct the use of this clause with the approval of the Deputy Director for Cost Assessment.) However, this clause also does not specify the format or details of the O&S cost data to be collected. In 2012, the Director, Cost Assessment and Program Evaluation (CAPE), identified specific O&S cost elements and developed a Contractor Sustainment Report form for the consistent collection and reporting of this detailed O&S cost data. According to DOD officials, CAPE is revising its cost-estimating guidance to require use of this new form. DOD officials expect to issue this cost-estimating guidance by December 2013.

    Recommendation: To ensure that PBL arrangements are the most cost-effective option for weapon system support, the Secretary of Defense should direct the Under Secretary of Defense (Acquisition, Technology and Logistics) to require program offices to collect and report cost data for PBL arrangements in a consistent, standardized format with sufficient detail to support traditional cost analysis and effective program management.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Nov 20, 2014

Nov 19, 2014

Nov 18, 2014

Oct 30, 2014

Oct 16, 2014

Oct 10, 2014

Oct 8, 2014

Sep 30, 2014

Looking for more? Browse all our products here