This is the accessible text file for GAO report number GAO-10-284 
entitled 'Defense Acquisitions: Status of DOD's Implementation of 
Independent Management Reviews for Services Acquisitions' which was 
released on January 28, 2010. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

United States Government Accountability Office: 
GAO: 

January 2010: 

Defense Acquisitions: 

Status of DOD's Implementation of Independent Management Reviews for 
Services Acquisitions: 

GAO-10-284: 

GAO Highlights: 

Highlights of GAO-10-284, a report to congressional committees. 

Why GAO Did This Study: 

The Department of Defense (DOD) is the federal governmentís largest 
purchaser of contractor-provided services, obligating more than $207 
billion on services contracts in fiscal year 2009. DOD contract 
management has been on GAOís high-risk list since 1992, in part 
because of continued weaknesses in DODís management and oversight of 
contracts for services. 

The National Defense Authorization Act for Fiscal Year 2008 directed 
DOD to issue guidance providing for independent management reviews for 
services acquisitions. The Act required that the guidance provide a 
means to evaluate specific contracting issues and to address other 
issues, including identifying procedures for tracking recommendations 
and disseminating lessons learned. 

The Act also directed GAO to report on DODís implementation of its 
guidance. GAO (1) assessed the extent to which DODís guidance 
addressed the Actís requirements and how the guidance was implemented 
and (2) determined the status of actions taken by the military 
departments pursuant to DODís guidance. GAO compared DODís guidance 
with the Actís requirements; obtained data on the number of reviews 
conducted as of September 2009; and analyzed memoranda of 29 
acquisitions valued at over $1 billion. 

In its written comments, DOD noted it planned to refine its processes 
to better share the lessons learned and best practices identified 
during peer reviews. 

What GAO Found: 

To meet the legislative requirement regarding independent management 
reviews, DOD issued guidance in September 2008 and February 2009 
providing for a peer review process for services acquisitions. DODís 
guidance generally addresses requirements in the Act to issue guidance 
designed to evaluate specified contracting issues, but according to 
officials, DOD has not yet determined how it plans to disseminate 
lessons learned or track recommendations that result from the newly 
instituted reviews. Under this guidance, the Office of Defense 
Procurement and Acquisition Policy (DPAP) is responsible for 
conducting pre- and post-award peer reviews for services acquisitions 
with an estimated value of over $1 billion. Peer review teams include 
senior contracting officials from the military departments and defense 
agencies as well as legal advisors. As of September 30, 2009, DPAP had 
conducted 29 reviews of 18 services acquisitions, including 3 post-
award reviews. DOD has also conducted peer reviews on two task orders 
but has not yet determined if it will do so on individual task orders 
in the future. The peer review teams made a number of recommendations 
and identified some best practices. DOD officials expect to refine 
their processes, including developing a more formal means for 
disseminating lessons learned and tracking recommendations, as DOD 
assesses its initial experiences with peer reviews. 

Each of the military departments has issued guidance establishing peer 
review processes for services acquisitions valued at less than $1 
billion although the guidance is still evolving. The departmentsí 
guidance identifies the offices or commands tasked with conducting 
peer reviews based on various dollar thresholds. The military 
departments reported conducting hundreds of peer reviews for services 
acquisitions as of September 30, 2009, but could not provide exact 
numbers because of the lack of comprehensive reporting processes. 
Further, as peer review processes evolve, the military departments are 
considering ways to disseminate lessons learned and track 
recommendations. 

Figure: Comparison of Peer Review Processes Established by DPAP and 
the Military Departments: 

[Refer to PDF for image: illustrated table] 

Department: DPAP; 
Review cycle: Pre-award; 
Milestone: Issue solicitation: review is to occur; 
Milestone: Request final proposal: review is to occur; 
Milestone: Award contract: review is to occur; 
Review cycle: Post-award:
Milestone: End contract or exercise option: review is to occur; 
Threshold for review: Over $1 billion. 

Department: Air Force; 
Review cycle: Pre-award; 
Milestone: Issue solicitation: 2 reviews to occur; 
Milestone: Request final proposal: 2 reviews to occur; 
Milestone: Award contract: review is to occur; 
Review cycle: Post-award: 
Milestone: End contract or exercise option: review is to occur; 
Threshold for review: Under $1 billion. 

Department: Army; 
Review cycle: Pre-award; 
Milestone: Issue solicitation: review is to occur; 
Milestone: Request final proposal; 
Milestone: Award contract: review is to occur; 
Review cycle: Post-award: 
Milestone: End contract or exercise option; Threshold for review: 
Under $1 billion. 

Department: Navy; 
Review cycle: Pre-award; 
Milestone: Issue solicitation: review is to occur; 
Milestone: Request final proposal: review is to occur; 
Milestone: Award contract: review is to occur; 
Review cycle: Post-award: 
Milestone: End contract or exercise option: review is to occur; 
Threshold for review: Under $1 billion. 

Source: GAO analysis of DOD guidance. 

[End of figure] 

View [hyperlink, http://www.gao.gov/products/GAO-10-284] or key 
components. For more information, contact John Hutton at (202) 512-
4841 or huttonj@gao.gov. 

[End of section] 

Contents: 

Letter: 

DOD Has Implemented Guidance That Generally Addresses Legislative 
Requirements: 

Concluding Observations: 

Agency Comments: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Defense: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Figures: 

Figure 1: Comparison of DPAP's Peer Reviews and Management Reviews for 
Services Acquisitions Estimated to Be Valued at $1 Billion or More: 

Figure 2: Peer Reviews of Major Services Acquisitions Conducted by 
DPAP as of September 30, 2009: 

Figure 3: Comparison of Peer Review Processes Established by DPAP and 
the Military Departments: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

January 28, 2010: 

[Refer to PDF for image] 

[End of figure] 

Congressional Committees: 

The Department of Defense (DOD) is the federal government's largest 
purchaser of contractor-provided services, including professional and 
management support, information technology support, weapon system and 
base operations support, and intelligence support. DOD obligated more 
than $207 billion on services contracts in fiscal year 2009, more than 
double the amount it obligated in fiscal year 2001, when measured in 
real terms.[Footnote 1] DOD's reliance on contracted services makes 
effective management and oversight of these contracts essential; 
however, DOD contract management has been on our high-risk list since 
1992.[Footnote 2] 

Our recent work continues to identify weaknesses in DOD's management 
and oversight of services contracts. In 2006, we found that DOD's 
approach to managing services acquisitions did not allow the 
department to determine whether its investments in services achieved 
the desired outcomes. For example, we found that the department often 
focused its efforts on awarding contracts without determining the 
areas of risk that needed greater attention or capturing the knowledge 
to enable more informed strategic decisions when contracting for such 
services.[Footnote 3] In November 2009, we reported that DOD needed to 
take further actions to improve its management of professional and 
management support services, particularly in assessing the risks 
associated with contractors performing tasks that closely support 
inherently governmental functions and in providing guidance for 
enhancing oversight when contracting for such services.[Footnote 4] 

To improve DOD's services acquisition process, Section 808 of the 
National Defense Authorization Act for Fiscal Year 2008 (the Act) 
directed the Secretary of Defense to issue guidance and implementing 
instructions providing for periodic independent management reviews of 
contracts for services.[Footnote 5] The Act specified that the 
guidance and instructions were to be designed to evaluate certain 
issues (hereafter referred to as contracting issues), including: 

* contract performance in terms of cost, schedule, and requirements; 

* the use of contracting mechanisms, including the use of competition, 
the contract structure and type, the definition of contract 
requirements, cost or pricing methods, the award and negotiation of 
task orders,[Footnote 6] and management and oversight mechanisms; 

* the contractor's use, management, and oversight of subcontractors; 

* the staffing of contract management and oversight functions; and: 

* the extent of any pass-through or excessive pass-through charges by 
the contractor.[Footnote 7] 

Further, when one contractor provides oversight for services performed 
by other contractors, the Act required the DOD guidance to provide 
procedures for the periodic review of such contracts to include the 
evaluation of: 

* the extent of DOD's reliance on the contractor performing 
acquisition functions closely associated with inherently governmental 
functions and: 

* the financial interest of any prime contractor performing 
acquisition functions closely associated with inherently governmental 
functions in any contract or subcontract in which the prime contractor 
provided advice or recommendations to the agency. 

In addition to the contracting issues above, the Act required the DOD 
guidance to address several elements related to the review process. 
These elements include the contracts subject to review, the frequency 
of reviews, procedures for tracking the implementation of 
recommendations made during reviews, and procedures for identifying 
and disseminating lessons learned from reviews. 

The Act also directed that we report on DOD's implementation of the 
guidance and instructions for independent management reviews. 
Specifically, we (1) assessed the extent to which DOD's guidance 
addressed the Act's requirements at the department level and how the 
guidance was implemented and (2) determined the status of actions 
taken by the military departments pursuant to DOD's guidance. To do 
so, we reviewed memoranda issued by DOD in September 2008 and February 
2009 and compared these documents to the requirements of the Act. We 
obtained information on the number of reviews that the Office of the 
Under Secretary of Defense for Acquisition, Technology and Logistics' 
Office of Defense Procurement and Acquisition Policy (DPAP) and the 
military departments conducted as of September 30, 2009. We reviewed 
summary memoranda for DPAP-led reviews of services contracts and 
compared the topics discussed in the summary memoranda to DOD policy 
and guidance, focusing on the contracting issues specified in the Act. 
We also interviewed officials from DPAP and the military departments 
to discuss how the guidance and instructions were developed and 
implemented for the review process. A more detailed description of our 
scope and methodology is included in appendix I. 

We conducted this performance audit from October 2009 through January 
2010 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

DOD Has Implemented Guidance That Generally Addresses Legislative 
Requirements: 

To meet the legislative requirements regarding independent management 
reviews, DOD issued guidance and instructions providing for a peer 
review process for services acquisitions. DOD's guidance generally 
addresses requirements prescribed in the Act to develop a process to 
evaluate the specified contracting issues, but according to DOD 
officials, DOD has not yet determined how the department plans to 
disseminate lessons learned or track recommendations that result from 
the newly instituted reviews. DOD officials expect to further refine 
their processes, including developing a more formal means for 
disseminating lessons learned and tracking recommendations as DOD 
assesses its initial experiences with peer reviews. Through the first 
year of implementation, DPAP, which is responsible for conducting 
reviews of acquisitions over $1 billion, had conducted 29 peer reviews 
on 18 services acquisitions. Similarly, the military departments, 
which are responsible for conducting reviews of their acquisitions 
under $1 billion, issued guidance that provides for peer reviews at 
various levels within the departments based on dollar values. The 
military departments could not, however, determine the exact number of 
peer reviews conducted because of the absence of comprehensive 
reporting processes. Further, as peer review processes evolve, the 
military departments are considering ways to disseminate lessons 
learned and track recommendations. 

DPAP Has Established a Peer Review Process for Services Acquisitions: 

DPAP issued a memorandum in September 2008 establishing a peer review 
process to fulfill the requirement for an independent management 
review of contracts for services.[Footnote 8] The requirement for a 
peer review process was subsequently incorporated into DOD Instruction 
5000.02, Operation of the Defense Acquisition System, in December 
2008. The guidance states that these reviews are intended to ensure 
consistent and appropriate implementation of policy and regulations, 
improve the quality of contracting processes, and facilitate sharing 
best practices and lessons learned. According to DOD officials, peer 
reviews by design are a means of improving individual acquisitions and 
not necessarily a tool for strategically managing DOD's services 
portfolio. 

Under DPAP's guidance, peer reviews supplement its existing process to 
review and approve services acquisitions. Pursuant to congressional 
direction, DOD had previously established a management review process 
that was intended to ensure that DOD services acquisitions are based 
on clear, performance-based requirements with measurable outcomes and 
that acquisitions are planned and administered to achieve intended 
results.[Footnote 9] In these management reviews, DPAP assesses and 
approves the acquisition strategies submitted by the military 
departments or defense agencies for obtaining contractor-provided 
services estimated to be valued at $1 billion or more. Once the 
acquisition strategies are approved, DOD contracting offices may 
continue the acquisition process, including soliciting bids for 
proposed work and subsequently awarding contracts. DOD may award 
different contract types to acquire products and services, or issue 
task orders under existing contracts. In November 2009, we reported 
that the number of contracts and task orders issued after the 
acquisition strategies were approved was significant. For example, we 
reported that nearly 1,900 task orders were issued under the seven 
professional and management support services acquisitions we 
reviewed.[Footnote 10] 

DOD generally conducts peer reviews at three key points in the 
acquisition process prior to contract award--prior to issuance of the 
solicitation (phase 1), prior to request for final proposal revisions 
(phase 2), and prior to contract award (phase 3)--and is to conduct 
periodic post-award reviews (phase 4) (see figure 1).[Footnote 11] 

Figure 1: Comparison of DPAP's Peer Reviews and Management Reviews for 
Services Acquisitions Estimated to Be Valued at $1 Billion or More: 

[Refer to PDF for image: illustration] 

Peer review: 

Phase 1: Prior to issuance of the solicitation: Documents typically 
reviewed: performance work statement, quality assurance surveillance 
plan, request for proposal, and source selection plan; 
Milestones for services acquisition: Development of acquisition 
strategy and supporting documentation begins; 
Management review: Acquisition strategy review and approval: Documents 
typically reviewed: draft acquisition strategy, instructions for 
proposals, and proposal evaluation criteria. 

Phase 2: Prior to request for final proposal revisions: Documents 
typically reviewed: instructions for proposals and proposal evaluation 
criteria, source selection evaluation guide, source selection plan, 
and evaluations of contractor proposals; 
Milestones for services acquisition: Solicitation issued. 

Phase 3: Prior to contract award: Documents typically reviewed: 
proposal analysis report and selection decision document; 
Milestones for services acquisition: Final request for proposals 
issued. 

Phase 4: Post-award review: Documents typically reviewed: any 
documentation related to the program, such as task orders, award fee 
plan, and performance assessments; 
Milestones for services acquisition: Contract awarded; Midpoint of 
performance period or exercise of option. 

Source: GAO analysis of DOD policy and guidance. 

[End of figure] 

In February 2009, DOD issued guidance that clarified the relationship 
between the management reviews and the peer reviews. For example, the 
guidance identifies specific issues to assess and the criteria for the 
reviewers to use during the management reviews or pre-award peer 
reviews. According to the guidance, some contracting issues identified 
in the Act, such as contract type and competition, are to be assessed 
during the management reviews. Conversely, other contracting issues 
identified in the Act, including requirements definition and the 
extent of the agency's reliance on contractors to perform functions 
closely associated with inherently governmental functions, are to be 
assessed during pre-award peer reviews. The pre-award peer reviews 
also are to evaluate several elements of the source selection process 
that are not specified in the Act, such as the clarity and consistency 
of the documentation. Further, the guidance established review 
criteria for post-award reviews that address each of the contracting 
issues identified in the Act. For example, during post-award reviews, 
reviewers are to assess the extent to which the contracting office was 
able to achieve competition for orders and whether it was using 
appropriate contract types, well-defined requirements, and appropriate 
cost/pricing methods. 

According to DOD officials, in conducting these reviews, DPAP convenes 
a peer review team consisting of three to five members. Officials said 
that the teams are generally chaired by a deputy director within DPAP 
and include participation from senior contracting officials from the 
military departments and defense agencies as well as legal advisors 
from the Office of the Secretary of Defense's General Counsel. The 
teams review acquisition documents prior to an on-site review and hold 
discussions with contracting officers over multiple days. Upon 
completion of the on-site review, peer review teams develop summary 
memoranda that include observations and recommendations. 

The February 2009 guidance indicated that DPAP is to review services 
acquisitions with an expected value of over $1 billion. In addition, 
DPAP may review acquisitions under that threshold that it has 
designated as special interest because of the nature or sensitivity of 
the services to be acquired. According to DOD officials, DPAP does not 
have a capability to independently identify acquisitions that will 
require its review, but rather relies on the military departments and 
defense agencies to notify DPAP of acquisitions that will exceed the 
threshold. DPAP officials noted that some reviews were not conducted 
because the military departments did not notify DPAP that a peer 
review was necessary. 

DPAP officials stated that they are currently focusing on the pre-
award peer reviews and are phasing in post-award peer reviews. As of 
September 30, 2009, DPAP had conducted 29 peer reviews for 18 services 
acquisitions.[Footnote 12] Because the peer review process was only 
implemented in September 2008, no single acquisition has been subject 
to all phases of the peer review process and no acquisition has been 
peer reviewed in both the pre-and post-award phases. While most of the 
reviews have focused on proposed acquisitions for which the initial 
contract had not yet been awarded, DPAP has also conducted two phase 3 
peer reviews for proposed task orders valued at over $1 billion that 
were to be issued under an existing contract that had previously been 
reviewed. DPAP has not yet determined if it will establish a policy 
for conducting peer reviews for all individual task orders over this 
amount in the future. For the 29 peer reviews of services acquisitions 
that DPAP conducted, figure 2 shows when each review occurred and the 
corresponding milestone. For example, DPAP conducted a phase 1 peer 
review prior to the issuance of the solicitation for 12 of the 18 
services acquisitions. 

Figure 2: Peer Reviews of Major Services Acquisitions Conducted by 
DPAP as of September 30, 2009: 

[Refer to PDF for image: illustrated timeline] 

Pre-award review, conducted prior to issuance of solicitation: 
November 2008; 
Pre-award review, conducted prior to request for final proposals: 
April 2009; 
Pre-award review, conducted prior to contract award: May 2009. 

Pre-award review, conducted prior to issuance of solicitation: 
December 2008; 
Pre-award review, conducted prior to request for final proposals: June 
2009; 
Pre-award review, conducted prior to contract award: September 2009. 

Pre-award review, conducted prior to issuance of solicitation: 
November 2008; 
Pre-award review, conducted prior to request for final proposals: 
August 2009. 

Pre-award review, conducted prior to issuance of solicitation: 
December 2008; 
Pre-award review, conducted prior to request for final proposals: May 
2009; 
Pre-award review, conducted prior to contract award: August 2009. 

Pre-award review, conducted prior to issuance of solicitation: January 
2009; 
Pre-award review, conducted prior to request for final proposals: 
April 2009; 
Pre-award review, conducted prior to contract award: May 2009. 

Pre-award review, conducted prior to issuance of solicitation: January 
2009; 
Pre-award review, conducted prior to request for final proposals: 
August 2009. 

Post-award review, Review conducted midway through performance period 
or prior to exercising an option: March 2009; 
Pre-award review, conducted prior to contract award: June 2009; 
Pre-award review, conducted prior to contract award: July 2009. 

Post-award review, Review conducted midway through performance period 
or prior to exercising an option: March 2009. 

Pre-award review, conducted prior to request for final proposals: 
March 2009; 
Pre-award review, conducted prior to contract award: May 2009. 

Pre-award review, conducted prior to contract award: May 2009. 

Pre-award review, conducted prior to request for final proposals: 
April 2009. 

Pre-award review, conducted prior to issuance of solicitation: April 
2009. 

Pre-award review, conducted prior to issuance of solicitation: May 
2009. 

Post-award review, Review conducted midway through performance period 
or prior to exercising an option: July 2009. 

Pre-award review, conducted prior to issuance of solicitation: August 
2009. 

Pre-award review, conducted prior to issuance of solicitation: 
September 2009. 

Pre-award review, conducted prior to issuance of solicitation: 
September 2009. 

Source: GAO. 

[End of figure] 

Our review of the summary memoranda of the pre-award peer reviews that 
DPAP conducted as of September 30, 2009, found that review teams 
generally documented the evaluation of the use of contracting 
mechanisms and, to a lesser extent, the use, management, and oversight 
of subcontractors. DPAP officials noted that other contracting issues 
may have been discussed during pre-award site visits and not included 
in the summary memorandum because the peer review team did not 
identify any concerns that warranted inclusion. Further, we found that 
review teams made several related recommendations, as illustrated in 
the following examples: 

* One pre-award peer review team recommended that the contracting 
office reconsider the number of contracts that it had proposed be 
awarded under an acquisition. In this case, the contracting office had 
proposed limiting the number of contracts to three prior to knowing 
what proposals and business arrangements would be submitted by 
industry. The peer review team noted that this may unduly restrict 
flexibility of the military department. Further, the team was unsure 
if documentation to support the limitation on contract number would be 
sufficient to withstand a bid protest from an unsuccessful offeror. 

* Another pre-award peer review recommended that the contracting 
office increase its use of subcontractors and encourage the prime 
contractors to establish mentor-protťgť relationships with their 
subcontractors to bring more qualified contractors into an industry. 

Our review of the summary memoranda for the three post-award peer 
reviews conducted by DPAP found that consistent with guidance, the 
review teams evaluated all the contracting issues identified in the 
Act. All three summary memoranda listed the required contracting 
issues and then reported the peer review teams' observations and 
recommendations for the contracting offices to consider for the 
acquisition, as illustrated by the following examples: 

* One post-award peer review team recommended that the contracting 
officer modify the contract to include provisions requiring the 
contractor to provide information on pass-through charges for all 
future task orders issued. At the time of the peer review, the 
contract did not contain a clause requiring the contractor to provide 
such information, and therefore the government was unable to determine 
the extent of pass-through charges and whether they were excessive. 

* Another post-award team recommended that the contracting office 
reduce the use of time-and-materials task orders. In this case, the 
acquisition strategy envisioned that most of the work would be 
performed through fixed-priced task orders; however, time-and- 
materials task orders accounted for 62 percent of the value of orders 
issued under the contract in the first 2 years of performance. 

While DPAP's guidance noted that the recommendations made during peer 
reviews are advisory in nature, it also states that contracting 
offices are to document in the contract file the disposition of all 
pre-award peer review recommendations prior to contract award. The 
guidance does not address recommendations made during post-award 
reviews. According to DOD officials, contracting offices generally 
accept recommendations provided by the peer review teams. DPAP 
officials said that if the contracting office decides not to accept a 
peer review team's recommendation, the contracting officer is expected 
to document the reason in the contract file and provide a copy to DPAP. 

In addition to providing recommendations to address potential issues 
in proposed acquisitions, the peer review teams have also identified 
some best practices. For example, in one summary memorandum the team 
called attention to the contracting office's post-award performance 
plan for the acquisition, which specified how the office intended to 
evaluate and assess contract performance to maintain effective 
contract surveillance procedures. The team noted that the plan allowed 
real-time access to detailed cost performance data when combined with 
regular surveillance. According to officials, DOD, however, has not 
yet issued guidance establishing procedures to systematically track 
the recommendations made by peer review teams or disseminate best 
practices as required by the Act. DOD officials noted that to date, 
sharing lessons learned from peer reviews has largely occurred through 
word of mouth or through conferences. For example, at a December 2009 
conference for senior DOD contracting officials, DPAP presented an 
update on its peer review process that included a discussion of 
lessons learned. To identify methods to better disseminate trends, 
lessons learned, and best practices identified during peer reviews, in 
August 2009 DPAP established a subcommittee within the Panel on 
Contracting Integrity.[Footnote 13] DPAP officials expect that the 
subcommittee will report on its findings in 2010. Further, an official 
stated that DPAP plans to consider ways to track the implementation of 
recommendations made during peer reviews. 

Military Departments' Peer Review Processes Are Evolving: 

The September 2008 DPAP guidance required the military departments to 
establish their own procedures for conducting pre-and post-award peer 
reviews on acquisitions under $1 billion, but provided the flexibility 
to the services to tailor the process to best meet their needs. In 
response, the Air Force issued its guidance in January 2009,[Footnote 
14] the Navy in March 2009,[Footnote 15] and the Army in April 
2009.[Footnote 16] The military departments' policies varied in such 
areas as the frequency and timing of the reviews and the 
organizational levels delegated responsibility for conducting the 
reviews. For example, the Air Force conducts up to five pre-award pre-
award peer reviews whereas the Army conducts two (see figure 3). The 
military departments plan to refine their policies as they gain 
experience with the peer review process. 

Figure 3: Comparison of Peer Review Processes Established by DPAP and 
the Military Departments: 

[Refer to PDF for image: illustrated table] 

Department: DPAP; 
Review cycle: Pre-award; 
Milestone: Issue solicitation: review is to occur; 
Milestone: Request final proposal: review is to occur; 
Milestone: Award contract: review is to occur; 
Review cycle: Post-award:
Milestone: End contract or exercise option: review is to occur; 
Threshold for review: Over $1 billion. 

Department: Air Force; 
Review cycle: Pre-award; 
Milestone: Issue solicitation: 2 reviews to occur; 
Milestone: Request final proposal: 2 reviews to occur; 
Milestone: Award contract: review is to occur; 
Review cycle: Post-award: 
Milestone: End contract or exercise option: review is to occur; 
Threshold for review: Under $1 billion. 

Department: Army; 
Review cycle: Pre-award; 
Milestone: Issue solicitation: review is to occur; 
Milestone: Request final proposal; 
Milestone: Award contract: review is to occur; 
Review cycle: Post-award: 
Milestone: End contract or exercise option; Threshold for review: 
Under $1 billion. 

Department: Navy; 
Review cycle: Pre-award; 
Milestone: Issue solicitation: review is to occur; 
Milestone: Request final proposal: review is to occur; 
Milestone: Award contract: review is to occur; 
Review cycle: Post-award: 
Milestone: End contract or exercise option: review is to occur; 
Threshold for review: Under $1 billion. 

Source: GAO analysis of DOD guidance. 

Note: While DPAP is to conduct peer reviews for acquisitions over $1 
billion, Air Force and Navy officials noted that they also review 
these acquisitions prior to submitting them to DPAP. 

[End of figure] 

According to officials, both the Air Force and Army modified existing 
pre-award reviews to incorporate the peer review requirements. The 
existing reviews were mandatory steps in each department's contract 
award process and, as such, focused on the proposed acquisition's 
contracting approach, source selection process, and readiness to issue 
a contract solicitation. Air Force officials stated that the 
department previously had a post-award review process that focused on 
cost, schedule, and performance metrics, which was revised to 
incorporate peer review requirements. Army officials noted that the 
Army has focused its attention on implementing pre-award peer reviews, 
but has not yet established a post-award peer review process. These 
officials noted that the Army plans to issue guidance on conducting 
post-award reviews in 2010. 

In contrast, the Navy developed a new process, modeled on DPAP's 
process, to review proposed services acquisitions. Navy officials are 
considering making some refinements to this process. For example, at 
the time of our review the Navy had not yet determined the optimal 
timing of its post-award peer reviews. The department was trying to 
determine a point at which there had been enough contract performance 
to evaluate the contractor while still allowing the contracting 
officers sufficient time to implement any peer review team 
recommendations prior to exercising an option year. 

While DPAP was not required to approve the military departments' 
guidance, DPAP officials reported that the guidance issued by the 
military departments was consistent with the intent of the September 
2008 guidance. There are differences, however, in how the military 
departments addressed certain issues. For example, each of the 
military departments delegated responsibility for conducting peer 
reviews to commands and organizational units within their departments 
based on expected acquisition value. In that regard: 

* The Air Force delegated responsibility for conducting peer reviews 
to its major commands for proposed services acquisitions valued from 
$50 million to $1 billion. 

* The Army delegated responsibility to the head of the contracting 
activity within each of its commands for conducting peer reviews for 
services acquisitions valued from $250 million to $1 billion. 
Similarly, it identified the principal assistant responsible for 
contracting as being responsible for conducting peer reviews for 
acquisitions from $50 million to $250 million. 

* The Navy delegated responsibility to the Deputy Assistant Secretary 
of the Navy - Acquisition and Logistics Management (DASN-A&LM) for 
conducting peer reviews for acquisitions valued from $250 million to 
$1 billion, while individual commands are responsible for conducting 
reviews of acquisitions valued from $50 million to $250 million. 

Further, the Air Force does not require peer reviews on noncompetitive 
acquisitions--in other words, on contracts awarded using other than 
full-and-open competition. Air Force officials explained that such 
contracts are already reviewed under a separate process and therefore 
believed that an additional peer review would be unnecessary. 
Similarly, both the Air Force and Army allow the offices responsible 
for conducting reviews to waive peer reviews under certain 
circumstances, whereas the Navy does not provide for a waiver process. 
Air Force guidance allows peer reviews to be waived based on 
acquisition/source selection history, such as for recurring 
acquisitions and where there is no history of bid protests. The Army 
also allows peer reviews to be waived but did not specify in its 
guidance which acquisitions could be waived. 

As of September 2009, the military departments reported conducting 
hundreds of peer reviews for services acquisitions, but the 
departments do not have comprehensive processes for determining the 
exact number of reviews conducted. Specifically: 

* The Navy reported that it had conducted 257 peer reviews for 
services acquisitions, including 5 post-award reviews. The Navy could 
not identify how many of the reviews conducted by the commands 
occurred by September 30, 2009. DASN-A&LM conducted its first 4 peer 
reviews on September 22, 2009. 

* Though the Air Force did not know the specific number of peer 
reviews conducted, officials noted that it had conducted up to five 
pre-award reviews on approximately 85 services acquisitions as of 
September 30, 2009. 

* Army officials stated that though commands had conducted pre-award 
peer reviews, an exact number of reviews could not be identified 
because the Army does not have a reporting process. The Army also 
acknowledged that it did not conduct any post-award reviews because it 
has not yet established a post-award peer review process. 

As peer review processes evolve, the military departments are 
considering ways to disseminate lessons learned and track 
recommendations. For example, Navy officials said the department is 
waiting to see the results of initial reviews and will then develop 
additional guidance to address lessons learned made during peer 
reviews. Army officials stated that the department plans to address 
recommendations and lessons learned in 2010 when it issues guidance on 
post-award reviews. Finally, Air Force policy requires commands to 
submit annual reports to the Secretary of the Air Force - Acquisition 
and Contracting Policy that are to include major issues identified 
during pre-award peer reviews and the resolutions taken. 

Concluding Observations: 

DOD's guidance implementing a peer review process for major services 
acquisitions at the departmental level generally addresses the 
requirements prescribed by the Act. While DOD has derived benefits 
from these initial reviews, it has also recognized that there are 
issues that still need to be addressed, such as how to track 
recommendations and disseminate lessons learned. Further, at this 
stage, DOD's focus has been on evaluating acquisition strategies and 
proposed contracts at the pre-award stage. DOD has conducted 
relatively few post-award reviews, in which DOD assesses how well it 
is managing the contractor's actual performance. A key issue is 
whether and how to apply the peer review process to task orders 
through which DOD obtains much of its contractor-provided services. 
Few of these are large enough to reach the $1 billion DOD review 
threshold, but below the threshold they could be so numerous as to 
overtax the departments' peer review processes. Addressing these 
issues, as well as those at the military department level, is 
important if DOD is to achieve its stated objectives for peer reviews--
ensuring consistent and appropriate implementation of policy and 
regulations, improving the quality of contracting processes, and 
facilitating sharing best practices and lessons learned--on a more 
strategic or enterprisewide basis rather than limiting the peer 
reviews' benefits to the individual acquisitions being reviewed. 
Although we are not making any recommendations because DOD plans to 
address these issues, resolving these concerns in a timely manner is 
essential if DOD is to maximize the benefits of the peer review 
process. 

Agency Comments: 

DOD provided written comments on a draft of this report. In its 
comments, DOD stated that peer reviews had improved the quality of its 
significant business arrangements. DOD indicated that it will continue 
to refine its peer review process to better disseminate trends, 
lessons learned, and best practices that are identified during peer 
reviews. DOD provided a technical comment, which was incorporated into 
the report. DOD's comments are reprinted in appendix II. 

We are sending copies of this report to the Secretary of Defense; the 
Secretaries of the Air Force, Army, and Navy; and interested 
congressional committees. The report also is available at no charge on 
GAO's Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-4841. Contact points for our Offices of 
Congressional Relations and Public Affairs may be found on the last 
page of this report. GAO staff who made major contributions to this 
report are listed in appendix III. 

Signed by: 

John P. Hutton: 
Director: 
Acquisition and Sourcing Management: 

List of Committees: 

The Honorable Carl Levin: 
Chairman: 
The Honorable John McCain: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

The Honorable Daniel K. Inouye: 
Chairman: 
The Honorable Thad Cochran: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
United States Senate: 

The Honorable Ike Skelton: 
Chairman: 
The Honorable Howard P. McKeon: 
Ranking Member: 
Committee on Armed Services: 
House of Representatives: 

The Honorable John Murtha: 
Chairman: 
The Honorable C.W. Bill Young: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
House of Representatives: 

[End of section] 

Appendix I: Scope and Methodology: 

Section 808 of the National Defense Authorization Act for Fiscal Year 
2008 (the Act) directs GAO to report on the Department of Defense's 
(DOD) implementation of its guidance and implementing instructions 
providing for periodic management reviews of contracts for services. 
In response to this mandate, we (1) assessed the extent to which DOD's 
guidance addressed the Act's requirements at the department level and 
how the guidance was implemented and (2) determined the status of 
actions taken by the military departments pursuant to DOD's guidance. 

To do so, we reviewed DOD's September 2008 and February 2009 guidance 
issued by the Under Secretary of Defense for Acquisition, Technology 
and Logistics' Office of Defense Procurement and Acquisition Policy 
(DPAP). We compared the guidance and instructions to the requirements 
stipulated in Section 808 of the Act. The September 2008 guidance 
indicated that peer reviews were to be conducted for both supplies and 
services. As the Act's requirements were specific to services 
acquisitions, we limited our analysis to services. We also obtained 
guidance and implementing instructions issued by the Departments of 
the Air Force, Army, and Navy. We interviewed officials from DPAP and 
the Departments of the Army, Navy, and Air Force to gain further 
insight into how each organization developed its guidance and 
instructions. DOD's September 2008 memorandum also indicated that 
defense agencies were required to develop their own guidance. While 
these were outside the scope of our review, DPAP officials indicated 
that 13 of 17 defense agencies that DPAP believed would be required to 
develop guidance had done so at the time of this review. 

We obtained information on the number of peer reviews on services 
acquisitions that DPAP and the military departments reported they had 
conducted as of September 30, 2009. DPAP was able to identify the 
number of reviews that it had conducted. We determined this 
information to be sufficiently reliable for the purposes of our 
review. The Air Force provided an approximate number of acquisitions 
that had been reviewed but could not identify the number of individual 
peer reviews conducted. The Army did not provide any information on 
the specific number of reviews conducted. The Navy provided 
information on the number of reviews it had conducted but could not 
specify how many had been conducted as of September 30, 2009. We could 
not independently verify the information provided by the military 
departments because of the lack of available documentation. 

To determine the nature of the discussions and the issues addressed 
during peer reviews, we obtained the summary memoranda from each of 
the 29 peer reviews conducted by DPAP as of September 30, 2009. These 
29 memoranda represented 18 unique acquisitions, as DPAP had reviewed 
some acquisitions more than once. Twenty-six of the memoranda were for 
pre-award peer reviews and 3 were for post-award reviews. We analyzed 
summary memoranda from each of the 29 peer reviews to determine the 
topics discussed in the memoranda, focusing specifically on the 
contracting issues identified in the Act. We also interviewed DPAP 
officials who chaired or participated in these reviews to obtain their 
views on the peer review process. 

We conducted this performance audit from October 2009 through January 
2010 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Defense: 

Office Of The Under Secretary Of Defense: 
Acquisition, Technology And Logistics: 
3000 Defense Pentagon: 
Washington, DC 20301-3000: 

January 21, 2010: 

Mr. John Hutton: 
Director, Acquisition and Sourcing Management: 
U.S. Government Accountability Office: 
441 G Street, N.W. 
Washington, DC 20548: 

Dear Mr. Hutton: 

This is the Department of Defense ODD) response to the GAO Draft 
Report, GAO-10-284, "Defense Contracting: Status of Implementation of 
Independent Management Reviews for Services Acquisitions," dated 
December 23, 2009 (GAO Code 120858). 

We appreciate GAO's assessment of the Department's peer review 
initiative and our approach to implement the statutory requirements 
for Independent Management Reviews. Looking back at our first year of 
execution, there is no question the peer review initiative has yielded 
innumerable benefits to improve the quality of the Department's 
significant business arrangements. This year, as indicated in the 
report, we will evolve the process to implement methods to better 
disseminate trends, lessons learned, and best practices that are 
identified during peer reviews. To this end, the Peer Review 
subcommittee of the Department's Panel on Contracting Integrity will 
guide our efforts as we seek to adopt any refinements that are 
necessary to make this initiative even more effective. 

Sincerely, 

[Signed by] 

Shay D. Assad: 
Director, Defense Procurement and Acquisition Policy: 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

John P. Hutton, (202) 512-4841 or huttonj@gao.gov: 

Acknowledgments: 

In addition to the contact named above, Timothy DiNapoli, Assistant 
Director; E. Brandon Booth; Morgan Delaney Ramaker; Christopher 
Mulkins; Thomas Twambly; and Alyssa Weir made key contributions to 
this report. 

[End of section] 

Footnotes: 

[1] According to the Federal Procurement Data System-Next Generation, 
DOD's total obligations for services in fiscal year 2009 were about 
$193 billion. However, this figure reflects an approximately $13.9 
billion downward adjustment made by DOD to correct an administrative 
error made in fiscal year 2008. As this adjustment significantly 
affected DOD's reported obligations in fiscal year 2009, the $207 
billion figure we report reflects what DOD's total obligations for 
services would have been had the error not occurred. 

[2] GAO, High-Risk Series: An Update, [hyperlink, 
http://www.gao.gov/products/GAO-09-271] (Washington, D.C.: January 
2009). 

[3] GAO, Defense Acquisitions: Tailored Approach Needed to Improve 
Service Acquisition Outcomes, [hyperlink, 
http://www.gao.gov/products/GAO-07-20] (Washington, D.C.: Nov. 9, 
2006). 

[4] GAO, Defense Acquisitions: Further Actions Needed to Address 
Weaknesses in DOD's Management of Professional and Management Support 
Contracts, [hyperlink, http://www.gao.gov/products/GAO-10-39] 
(Washington, D.C.: Nov. 20, 2009). 

[5] Pub. L. No. 110-181, ß 808 (2008). 

[6] The Federal Acquisition Regulation defines a task order as an 
order for services placed against an established contract or 
government sources. 

[7] Pass-through charges are contractor charges for the costs 
associated with subcontracting work. Section 852 of the John Warner 
National Defense Authorization Act for Fiscal Year 2007, Pub. L. No. 
109-364, states that an "excessive pass-through charge" with respect 
to a contractor or subcontractor that adds no, or negligible, value to 
a contract or subcontract, means a charge to the government by the 
contractor or subcontractor that is for overhead or profit on work 
performed by a lower-tier contractor or subcontractor (other than 
charges for the direct costs of managing lower-tier contracts and 
subcontracts and overhead and profit based on such direct costs). 

[8] Office of the Under Secretary of Defense for Acquisition, 
Technology and Logistics, Peer Reviews of Contracts for Supplies and 
Services, September 29, 2008. 

[9] We use the term management review to collectively refer to the 
procedures established by DOD for the review and approval of 
acquisition strategies. For additional information on DOD's management 
review processes, see [hyperlink, 
http://www.gao.gov/products/GAO-10-39]. 

[10] [hyperlink, http://www.gao.gov/products/GAO-10-39]. 

[11] According to DPAP guidance, acquisitions using other than full- 
and-open competition are to have two pre-award peer reviews instead of 
three. 

[12] While the Act only requires DOD to issue guidance on services 
contracts, DOD's policy is to review both services and products, 
including weapon systems. In addition to 29 reviews for services 
acquisitions, DPAP also conducted 22 pre-award peer reviews for 
acquisitions of supplies and weapon systems as of September 30, 2009. 

[13] DOD established the Panel on Contracting Integrity in response to 
Section 813 of the John Warner National Defense Authorization Act for 
Fiscal Year 2007, Pub. L. No. 109-364 (2006). This act directed DOD to 
establish a panel consisting of senior leaders representing a cross- 
section of the department to conduct reviews of progress made to 
eliminate areas of vulnerability within the defense contracting system 
that allow fraud, waste, and abuse to occur and recommend changes in 
law, regulations, and policy that it determined necessary to eliminate 
such areas of vulnerability. The Under Secretary of Defense for 
Acquisition, Technology and Logistics subsequently tasked the panel 
with taking a holistic view of all ongoing efforts and initiatives to 
improve performance in identified areas of weakness. 

[14] Office of the Assistant Secretary of the Air Force, Peer Reviews 
of Contracts for Supplies and Services (Jan. 5, 2009). The Air Force 
refers to its pre-award reviews as multifunctional independent review 
teams. It refers to its post-award reviews as annual execution reviews. 

[15] Office of the Assistant Secretary of the Department of Navy, 
Department of the Navy Peer Review Program (Mar. 26, 2009). 

[16] Office of the Assistant Secretary of the Army, Peer Reviews of 
Contracts for Supplies and Services (Apr. 23, 2009). The Army refers 
to its pre-award reviews as solicitation review boards and contract 
review boards. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAOís actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAOís Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: