Matter of: BAE Systems File: B-287189; B-287189.2 Date: May 14, 2001

Matter of: BAE Systems File: B-287189; B-287189.2 Date: May 14, 2001

DIGEST

Attorneys

DECISION

BAE Systems protests the decision of the Department of the Army under request for proposals (RFP) No. DAPC50-98-R-0012 to retain in-house (rather than contract-out) performance of logistics support and services for the U.S. Army Garrison in Hawaii. The decision to retain the services in-house was as a result of a cost comparison pursuant to Office of Management and Budget (OMB) Circular No. A-76, which compared BAE's proposal to perform the work against the government's in-house offer. /1/ BAE challenges the decisions of the agency's administrative appeals board (AAB) that reversed the agency's initial decision to contract out this work to BAE. BAE contends that the government's in-house offer did not satisfy the RFP's minimum performance requirements or offer the same level of performance and quality as offered by BAE.

We sustain the protest.

BACKGROUND

1. The Performance Work Statement and the Solicitation

2. The RFP was issued by the Army's Directorate of Contracting (DOC), Fort Shafter, Hawaii, on December 28, 1999, and provided for the award of a cost-plus-award-fee contract to provide all resources and management necessary to perform logistics support and services in accordance with the solicitation's performance work statement (PWS) for the U.S. Army Garrison in Hawaii. Offerors were informed that the RFP was issued as part of a government cost comparison to determine whether accomplishing the specified work under contract or by government performance was more economical. If government performance was determined to be more economical, then no award under the RFP would be made and the solicitation would be canceled. /2/

The PWS identified the services to be performed and stated performance standards. Generally, the required services were in four functional areas: supply and services, transportation, maintenance operations, and Island of Hawaii satellite operations. PWS Sec. C.5. Among other things, the PWS, as amended, required, as part of the transportation operations, the provision of personal property shipment services at Schofield Barracks and Fort Shafter on the Island of Oahu, and at Hilo on the Island of Hawaii. PWS Sec. C.5.2.1. These services included counseling and preparation of paperwork for movement of household goods and baggage and for shipment of privately owned vehicles. With respect to personal property counseling, the PWS provided:

The Contractor shall provide pre-counseling for appointments and walk-in customers. The Contractor shall service customers with an appointment within 10 minutes of the appointment time and walk-in customers within 30 minutes of arrival. The Contractor shall provide customer counseling on shipment and/or storage of personal property household goods (HHG) and unaccompanied baggage (UB). The Contractor shall inform customers of entitlements and responsibilities during counseling sessions . . . .

PWS Sec. C.5.2.6.1.

The PWS identified a number of other performance requirements. For example, the PWS identified a project manager, assistant project manager, and environmental protection specialist as key personnel. With respect to the project manager and assistant project manager, the PWS, as amended, required that these personnel "have a minimum of five years experience in the management of a similar or related multi-function operation within the last 8 years" and that resumes for key personnel be provided as part of a required management operations plan. PWS Sec. C.1.6.2.

The PWS also required offerors to submit a quality control plan with their proposals. The PWS, as amended, provided:

The Contractor [quality control plan] shall be independent of the on-site Contractor's organization. Additionally, the Quality control Administration shall report directly to the Contractor's Corporate Headquarters. The [quality control plan] shall address overall project management and provide a comprehensive plan to deliver quality services. The [quality control plan] shall describe an acceptable method of identifying deficiencies in quality of service performed under this contract. The Contractor shall address and initiate processes for corrective actions without dependence upon Government direction.

PWS Sec. C.1.13.1. With respect to quality control, the PWS required the establishment of an inspection system covering all required services and required that "[t]his plan shall specify areas to be inspected on both a scheduled or unscheduled basis and the title of the individual who will do the inspection." The PWS also contained requirements for a customer complaint feedback system and for reports of quality control inspections and corrective actions. PWS Sec. C.1.13.2.

The RFP provided that the successful private-sector offeror could be selected on the basis of a cost/technical tradeoff. The following evaluation factors were identified: technical, management, past performance, small business program support, and cost. Offerors were informed that the management factor (which included evaluation of the management operations plan, quality control plan, and other planning and control documents) would be evaluated on a go/no-go basis. Of the remaining factors, the technical factor was stated to be more important than the past performance and small business program support factors, and these three factors were together equally important to cost. RFP Secs. L.7, M.2.

Detailed proposal preparation instructions were provided, requiring the submission of separate technical, management, past performance, small business program support, and cost volumes. RFP Sec. L.6. In addition, the RFP identified information that must be provided for each of the evaluation factors. For example, under the management evaluation factor, offerors were directed to provide a quality control plan addressing "a complete and comprehensive quality control system to support performance of the contract." RFP Sec. L.7.2.b(3). The RFP also required the submission of resumes for key personnel (such as the project manager and assistant project manager) demonstrating compliance with the PWS requirements. RFP Sec. L.7.2.a.(4); PWS Sec. C.1.6.2. Finally, the RFP encouraged new and innovative approaches in the proposals, and required offerors to identify where the proposed level of performance was above the acceptable level in the PWS. RFP Sec. L.7.1.a.

3. Preparation and Review of the In-House Plan

4. Prior to the closing date for receipt of proposals under the RFP, the agency's Directorate of Logistics (DOL) Study Team prepared a "most efficient organization" (MEO), technical performance plan (TPP), and in-house cost estimate for performing the work in-house. These documents were reviewed by the Army Audit Agency, which, as the independent review officer (IRO), certified that the agency's MEO/TPP and in-house cost estimate were in accordance with OMB Circular No. A-76 requirements and reasonably established the agency's ability to perform the PWS requirements. Hearing Transcript (Tr.) at 652; /3/ Agency Report, Tab 24, Certification of the In-House Cost Estimate, at 13,229. After the Army Audit Agency's review, the MEO and TPP were sealed on February 3. Tr. at 653.

On February 14, 2000, the RFP was amended, changing a number of PWS requirements, including, as pertinent here, the requirements concerning personal property services and quality control. RFP amend. 3. This amendment was provided to the DOL Study Team, but not to the Army Audit Agency (the IRO). Tr. at 621, 641-42. The agency's MEO/TPP and in-house cost estimate were not revised as a result of the solicitation amendment. Tr. at 737. On February 24, the sealed MEO, TPP, and in-house cost estimate were delivered to the contracting officer. Contracting Officer's Statement at 2.

5. Evaluation of BAE's Offer

6. On February 25, BAE submitted the only private-sector offer in response to the RFP. BAE's proposal was evaluated by the agency's SSEB as follows:

Management Plan Go

Technical Acceptable

Past Performance Excellent

Small Business Program Support

Small Disadvantaged Excellent Business Participation

Small Business Concern Acceptable participation

Agency Report, Tab 8, SSEB Summary of Findings for Contracting Officer, at 3,927-41. Some individual evaluators noted strengths in BAE's proposal under the technical evaluation factor -- notably, that BAE had proposed to service walk-in customers within the personal property offices within 15 minutes as opposed to the 30 minutes required by the RFP. The evaluators' consensus judgment, however, was that BAE's proposal was only acceptable overall, with no strengths or weaknesses. /4/ Tr. at 13, 125.

Cost discussions were conducted with BAE, and revised cost proposals received. /5/ On August 1, the SSEB provided its evaluation report and briefed the SSA on BAE's proposal. Agency Report, Tab 16A, SSEB Briefing to SSA (Aug. 1, 2000), at 8,773-99, SSEB Report of Findings for the SSA, at 8,801-21. The SSA was informed that BAE's proposal was acceptable and satisfied the PWS requirements. Agency Report, Tab 16A, SSEB Briefing to SSA (Aug. 1, 2000), at 8,783. The SSA selected BAE's proposal to compete against the government's in-house offer, finding that

BAE Systems was the only contractor to submit a proposal for this requirement. Since there was only one proposal, I could not make a best value decision or make any trade-offs.

The SSA concluded that BAE's offer met "but does not exceed the level of performance and quality required by the [PWS] in any of the factors or sub-factors evaluated." Agency Report, Tab 40A, SSA Decision Memorandum (Aug. 3, 2000), at 20,179.

7. The SSEB's Evaluation of the In-House Offer

8. On August 1, after the SSA's briefing, the in-house TPP was unsealed and provided to the SSEB (the technical evaluators). /6/ Tr. at 21-22. The evaluators found that the TPP was not prepared in accordance with the RFP proposal preparation instructions, and advised the DOL Study Team that it failed to provide sufficient data to demonstrate that the technical approach and staffing were sufficient to satisfy the PWS requirements. Agency Report, Tab 16B, Contracting Officer's Memorandum to DOL Study Team, at 8,823-77. Moreover, the SSEB noted a number of other problems, including that the TPP did not show that the MEO project manager and assistant project manager would satisfy the PWS experience requirements. Id. at 8,825. The SSEB's comments, as well as detailed instructions for writing an "acceptable" TPP, were sent to the DOL Study Team. Id. at 8,855.

On August 21, the SSEB received a revised TPP. Tr. at 32. Whereas the original TPP was approximately 60 pages in length, the revised TPP exceeded 200 pages.

The SSEB then evaluated the supplemented TPP. At the hearing, the SSEB Chair stressed that the benchmark, against which the evaluators compared the TPP to ascertain its acceptability, and whether adjustments should be made, was BAE's proposal and not the PWS. Tr. at 39, 122, 161. As a result of its evaluation, the SSEB prepared a written request for clarification, which was delivered to the DOL Study Team on August 24. Agency Report, Tab 16C, Contracting Officer's Request for Clarification of the TPP from the DOL Study Team (Aug. 24, 2000), at 8,879-85. Among other things, the SSEB requested resumes for key personnel and the identification of quality control personnel referenced in the TPP. Id. at 8,881, 8,883. With respect to personal property services, the SSEB asked the DOL Study Team how the PWS requirements were going to be performed with the staffing proposed and for an explanation of the revised TPP's apparent limitation of walk-in customers to "emergencies." Id. at 8,883, see Revised TPP, vol. I, Technical Proposal, at 113.

On August 30, the evaluators received and evaluated the DOL Study Team's responses to the SSEB's questions and revised TPP pages. The SSEB found that the DOL Study Team had answered only approximately half of the clarification questions posed. Tr. at 40; Agency Report, Tab 16D, SSEB Evaluation of DOL Study Team Responses, at 8,887-905. For example, the DOL Study Team did not provide resumes for key personnel, asserting that resumes for specific personnel could not be provided prior to the determination to retain performance of the services in-house. Id. at 8,887. Also, the response to the question concerning the personal property services function indicated that the in-house offer provided for operating a personal property shipment office only at Schofield Barracks and not at Fort Shafter, as required by the revised PWS. /7/ Id. at 8,897. In response to the SSEB's inquiry regarding the limitation of walk-in customers to "emergencies," the DOL Study Team stated that "emergencies" meant "rush orders, deaths, confinement, hardship, medical, and DUI/Chapter 16." Id. at 8,899. The response to the question concerning the identity of quality control personnel indicated that these personnel were the MEO key personnel and supervisors (not a separate or independent quality control group). Id. at 8,903.

Subsequently, in response to further queries by the SSEB, the DOL Study Team provided two sets of resumes for the project manager and assistant project manager positions. The SSEB concluded that neither set of resumes satisfied the PWS's 5-year experience requirement. The SSEB did not resolve this concern because it was informed that it was not the SSEB's responsibility to evaluate the TPP against the go/no-go management evaluation factor. Tr. at 61-62, 133-34.

9. The SSA's Evaluation of, and Changes Directed in, the In-House Offer

On September 8, the results of the SSEB's evaluation of the in-house offer were provided to the SSA. /8/ Agency Report, Tab 16E, SSEB Briefing to SSA (Sept. 8, 2000), at 9,051-121; SSEB Comparison of Offeror to Government MEO, at 9,123-37; SSEB Consensus Evaluation of TPP, at 9,139-53. The SSA was informed by the SSEB that he needed to compare the TPP to BAE's proposal "to ensure that the TPP will provide the same level of performance and quality as offered by the contractor." Agency Report, Tab 16E, SSEB Briefing to SSA (Sept. 8, 2000), at 9,095. The SSA was also informed by the SSEB that there were a number of areas in which the staffing offered by the TPP needed to be increased to provide the level of performance and quality offered by BAE's proposal. Tr. at 304-05.

More specifically, the SSEB informed the SSA that the in-house offer's staffing to perform personal property services must be increased by nine full-time equivalents (FTE) to the level offered by BAE's proposal to provide for seeing walk-in customers within 30 minutes and for the operation of personal property shipment offices at Schofield Barracks and Fort Shafter at the level offered by BAE. /9/ Agency Report, Tab 16E, SSEB Briefing to SSA (Sept. 8, 2000), at 9,099, 9,107. The SSA was also informed in this regard that the in-house offer's approach of consolidating the Fort Shafter and Schofield Barracks operations appeared to conflict with the PWS requirements requiring personal property shipment offices at both locations. Agency Report, Tab 16E, SSEB Comparison of Offeror to Government MEO, at 9,127, 9,129; SSEB Consensus Evaluation of TPP, at 9,143. In addition, the SSA was informed that the TPP appeared to limit walk-in customers to emergencies, which was not consistent with the PWS or BAE's proposal. Agency Report, Tab 16E, SSEB Briefing to SSA (Sept. 8, 2000), at 9,107; SSEB Consensus Evaluation of TPP, at 9,143.

Similarly, the SSA was informed that, whereas BAE had proposed an independent quality control section staffed with three FTEs, the in-house offer did not propose an independent quality control function. Agency Report, Tab 16E, SSEB Briefing to SSA (Sept. 8, 2000), at 9,113. The SSEB noted that the revised PWS required a quality control plan "independent of the on-site Contractor's organization." /10/ Agency Report, Tab 16E, SSEB Comparison of Offeror to Government MEO, at 9,135. The SSEB recommended, based on BAE's proposal, that the in-house offer be revised to require an independent quality control section consisting of three FTEs. Tr. at 54. In the SSEB's view, BAE's proposal satisfied only the minimum PWS requirements in this respect, and the recommended changes were needed to raise the in-house offer to the minimum PWS level. See Tr. at 52.

The SSA issued a memorandum to the DOL Study Team on September 8, directing the Study Team to "make the following changes to their bid to bring the in-house bid up to the same level of performance and quality offered by the contractor's proposal." Agency Report, Tab 22C, Memorandum of SSA to DOL Study Team (Sept. 8, 2000), at 10,935-37; see Tr. at 306. The SSA directed nine changes to the in-house offer, including the following ones relevant here:

You will increase the number of non-supervisory FTEs currently provided to perform Personal Property (PPTY) Services by 9 FTEs.

1. You will staff and operate a Personal Property (PPTY) Services Office at Fort Shafter.

2. You will distribute the hours on the following basis. Fort Shafter 39 percent & Schofield 61 percent.

3. You will see walk-in customers at both locations [within] 30 minutes of arrival.

4. You may not restrict walk-in customers to "emergencies."

Agency Report, Tab 22C, Memorandum of SSA to DOL Study Team (Sept. 8, 2000), at 10,935.

Also, the SSA directed:

You will create a Quality Control section staffed with a minimum of three FTEs. This section will report directly to [the Directorate of Resource Management].

Id.

On October 2, the DOL Study Team submitted revised pages to the TPP. The DOL Study Team initially added no additional staffing to the MEO in response to the SSA's September 8 direction, but only shifted staffing already proposed. Tr. at 718-19. After reviewing the revised TPP pages, the DOL Study Team was directed to make additional changes to the TPP. Specifically, the DOL Study Team was directed to return the two FTEs the team had taken from the MEO's supply support function to supplement the staff of the personal property services function and to add two new FTEs to the personal property services function. Also, the team was directed to return the three FTEs it had moved from the project management function to staff the quality control function and to add three new FTEs to create a quality control function. Agency Report, Tab 39C, Memorandum of the Deputy Garrison Commander to DOL Study Team, (Oct. 5, 2000), at 19,865.

On October 19, the SSEB provided a final briefing and its report, including supporting documentation, to the SSA concerning the in-house offer. /11/ Agency Report, Tab 16G, SSEB Briefing to the SSA (Oct. 19, 2000), at 9,163-91. The SSA was informed that the DOL Study Team had complied with the SSA's September 8 directive, and that the in-house offer was now at the same level of performance and quality as BAE's proposal. /12/ Id. at 9,169. Specifically, the SSEB informed the SSA that the MEO now offered 218 FTEs, representing 389,688 productive hours, and BAE offered 209 FTEs, representing 391,457 productive hours. Id. at 9,191. The SSA accepted the SSEB's recommendation to proceed to the cost comparison. Agency Report, Tab 40C, Memorandum of SSA Directing Cost Comparison, (Oct. 23, 2000), at 20,185-89.

1. Initial Cost Comparison

2. The agency determined that, even after appropriate adjustments were made, performance by BAE would cost less than performance in-house, as shown by the following table:

In-House Cost Estimate BAE

Total proposed costs $60,426,010 $49,650,712

Contract 0 1,963,495 administration

One-time conversion 0 2,390,520 costs

Federal income tax 0

Total adjusted costs 60,426,010 53,706,823 Minimum conversion 0 5,312,253 differential

Final adjusted costs $60,426,010 $59,019,076

See Contracting Officer's Statement at 4; Agency Report, Tab 24, Cost Comparison Form (Oct. 24, 2000), at 12,765.

The cost comparison was forwarded to the Army Audit Agency to certify, as the IRO. The Army Audit Agency did not agree with the SSA's directed changes to the in-house offer, believing that it was inappropriate to direct an increase in the staffing level of the in-house offer where there was no showing that BAE had offered a level of quality exceeding the PWS requirements. Tr. at 660-61. The Army Audit Agency issued a qualified certification that certified that the costs were properly calculated, but asserted that the changes directed by the SSA were not in accordance with OMB Circular No. A-76 requirements. /13/ Tr. at 661-62; Agency Report, Tab 24, Memorandum of the Army Audit Agency to Garrison Commander (Oct. 20, 2000), at 13,211-15.

3. Appeals of Cost Comparison Results

4. Public announcement of the cost comparison results was made on October 25, and a conditional award made to BAE. Following the public review period, the agency received six appeals from affected employees and employee unions. Among other things, a number of the appeals challenged the addition of nine FTEs to the in-house offer to perform the personal property services function and of three FTEs to perform the quality control function. A number of appeals argued that relocation and retraining costs (which were added to BAE's cost proposal as a one-time conversion cost) were understated and that the number of contract administrators required to oversee BAE's contract was also understated. /14/ Agency Report, Tabs 27A, C, E, G, I, and K, Appeals. The protester did not file its own appeal or intervene in the employees' and unions' appeals.

A three-member AAB was appointed by the agency. Agency Record, Tab 38D, Memorandum of Commander, U.S. Army, Pacific, (Nov. 8, 2000), at 18,097-99; Tr. at 183. As an initial matter, the AAB requested and received general information from the installation (the U.S. Army Garrison) addressing the appeals. Tr. at 187. The installation appointed an "installation appeals team" to provide information and the installation's position to the AAB; this team consisted of representatives from the SSEB, the DOL Study Team, the Directorate of Resource Management, the DOC, and the Staff Judge Advocate's Office. Tr. at 607, 609. The AAB also requested and received specific information from, and conducted interviews with, representatives of the SSEB, the Army Audit Agency, the DOL Study Team, the contracting officer, and the staff judge advocate responding to the specific appeal issues. See Agency Report, Tab 38D, AAB Chair Notes, at 18,067-69; Tr. at 187-88.

In reviewing the personal property services and quality control issues, the AAB was "puzzled" by an apparent "disconnect" in the record; specifically, the AAB noted that BAE's proposal was found to meet, but not exceed, the levels of performance and quality required by the PWS, but the SSA had directed changes to the in-house offer to bring the government's in-house offer up to the same level of performance offered by BAE. Tr. at 193-94, 332-33. In addition, the Army Audit Agency informed the AAB that, as the IRO, it had certified that the original in-house offer satisfied the PWS requirements. Tr. at 194. Nevertheless, the AAB agreed with the SSA and SSEB that the in-house offer, as revised during discussions with the SSEB, did not provide the PWS-required level of performance in areas such as maintaining property services offices at both Schofield Barracks and Fort Shafter and having an independent quality control process. The AAB disagreed, however, with the SSA's (and SSEB's) analysis as to the amount of staffing required to perform these functions. Tr. at 278.

Specifically, with respect to the personal property services area, a number of appeals challenged the SSA's direction to add nine FTEs of staffing to the in-house offer to bring the government's in-house offer up to the level of quality and performance offered by BAE. See, e.g., Agency Report, Tab 27A, Appeal No. 1, at 13,965; Tab 27C, Appeal No. 2, at 14,123; and Tab 27E, Appeal No. 3, at 14,215. The AAB found that the PWS required the staffing of personal property services offices at both Schofield Barracks and Fort Shafter, but that the TPP had proposed closing the Fort Shafter office, which was not consistent with the amended PWS requirements. See, e.g., Agency Report, Tab 27B, Appeal No. 1 Decision, at 14,089.

The question addressed by the AAB was what amount of staffing was required to provide services at both Fort Shafter and Schofield Barracks. One appellant (Appeal No. 1) asserted that an additional two FTEs were sufficient to perform all the PWS requirements, while another (Appeal No. 2) asserted that an additional four FTEs would be required. The AAB asked the SSEB for its analysis supporting the addition of nine FTEs to perform the PWS requirements; the SSEB informed the AAB that the additional nine FTEs was based only upon BAE's proposed staffing. Tr. at 336, 381. In addition, the AAB asked the SSEB for its analysis supporting its staffing allocation of 61 to 39 percent at Schofield Barracks and Fort Shafter, in response to which the SSEB informed the AAB that this "was an educated guess." Tr. at 336. Although, as indicated above, the SSEB chair had done additional analysis supporting its staffing and allocation calculations, the SSEB chair did not inform the AAB of this analysis. Tr. at 73-74, 100. The AAB concluded that the SSA's directed addition of nine FTEs was not based upon "appropriate rigor or substance" and that the directed allocation of staffing at 61 percent at Schofield Barracks and 39 percent at Fort Shafter was not an auditable estimate and was without credibility. See, e.g., Agency Report, Tab 27B, Appeal No. 1 Decision, at 14,089.

The AAB asked others for their analysis of the number of FTEs that would be required to perform the PWS requirements, including operating an office at Fort Shafter. The Army Audit Agency informed the AAB that no additional FTEs would be required to perform the PWS requirements, but that if the AAB found that operating an office at Fort Shafter was required, only one additional FTE would be necessary. The Director of the DOL informed the AAB that an additional two FTEs could perform the PWS requirements.

The AAB concluded that only an additional two FTEs were needed to satisfy the requirement to staff personal property offices at both Fort Shafter and Schofield Barracks. This judgment was essentially based upon the opinions provided by the Army Audit Agency and the Director of the DOL. /15/ Specifically, the AAB accepted the opinion of the Army Audit Agency because the Army Audit Agency was the IRO and "had timed how long things take." Tr. at 458, 503-04. The board accepted the opinion of the DOL Director because he was disinterested and "was an objective source of information." Tr. at 335, 459-60. The AAB also noted that although the PWS had been amended to require a staffed office at Fort Shafter, the PWS workload data for the personal property services function were not adjusted, which suggests that few additional FTEs would be required. Tr. at 202. Also, one AAB member noted that the personal property services had most recently been performed at Schofield Barracks and Fort Shafter with only seven FTEs and that his family had personal experience with using the personal property services office and had been satisfied. Tr. at 458-59. This board member also noted that prior to closing the personal property services office at Fort Shafter the installation had staffing of two FTEs at that office and had to "farm work down [there] to keep those people occupied." Tr. at 460. In addition, the AAB noted that the Army's standard installation plan indicated that an allocation of staffing at Schofield Barracks and Fort Shafter should be approximately 73/27 percent (as opposed to the 61/39 percentage split estimated by the SSEB); applying this percentage allocation to the nine FTEs the TPP proposed for Schofield Barracks, the AAB members stated that it confirmed that the addition of two FTEs to staff Fort Shafter was reasonable. Tr. at 337, 457.

With respect to the quality control area, a number of appeals challenged the SSA's direction to add three FTEs of staffing to the TPP to bring the government's in-house offer up to the level of quality and performance offered by BAE. See, e.g., Agency Report, Tab 27A, Appeal No. 1, at 13,965; Tab 27E, Appeal No. 3, at 14,215; Tab 27G, Appeal No. 4, at 14,267. Here, too, the AAB agreed with the SSA that the in-house offer did not satisfy the requirement to have an independent quality control function; the question was what amount of staffing would satisfy the RFP requirements for quality control. See, e.g., Agency Report, Tab 27B, Appeal No. 1 Decision, at 14,097; Tr. at 278, 341-42, 473. The AAB concluded that the SSA's decision to rely upon BAE's proposed level of staffing to determine the level of staffing the government must provide to satisfy the quality control requirements was inappropriate, finding [a]lthough expedient, this action lacks the requisite link with associated workload data and does not consider in-house [quality control plan] efficiencies which rely upon embedded installation management processes.

Agency Report, Tab 27B, Appeal No. 1 Decision, at 14,097. The AAB thus decided to rely upon its own experience to determine the amount of staffing required to perform the quality control function, Tr. at 215-17, 474-75, because the board concluded that there was no workload data available regarding the provision of quality control services. Tr. at 216, 343. From this experience, the AAB was aware that the Army had a variety of quality and performance processes embedded in management systems, such as the Army performance improvement criteria, to ensure quality, /16/ and included within these embedded processes was the generation of reports concerning quality. Tr. at 344-45; Agency's Post-Hearing Comments at 8. In the board's judgment, the independent quality control envisioned by the RFP provided for a person who would be "nothing more than a giant collector of data." Tr. at 217. Thus, the board concluded, based upon a "technical estimate of the required workload," that a staffing level of one FTE would satisfy the required quality control function. /17/ Agency Report, Tab 27B, Appeal No. 1 Decision, at 14,097.

Several appeals argued that the number of contract administrators estimated to administer BAE's contract was understated. /18/ Specifically, the in-house cost estimate provided for only five FTEs for the contract administration function, and the appeals asserted that the RSH authorized eight FTEs to perform contract administration where the MEO is of the size envisioned here. /19/ See Agency Report, Tab 27C, Appeal No. 2, at 14,117; Tab 27G, Appeal No. 4, at 14,257-59; Tab 27H, Appeal No. 5, at 14,416; see also RSH, Part II, ch. 3, table 3-1, at 26. The AAB found that the RSH required eight FTEs to perform contract administration where the MEO staffing was estimated to be between 201 and 250 FTEs, as here. See, e.g., Agency Report, Tab 27D, Appeal No. 2 Decision, at 14,199. Although the RSH states the contract administration figure as a ceiling, RSH, part II, ch. 3, Sec. C, the board found that the guidance provided by the RSH was "mandatory" and "should be adhered to." Tr. at 246, 359-60, 485.

In a number of appeals, it was argued that the costs for retraining and relocation of affected employees, which are included as a one-time conversion cost added to the cost of BAE's offer for the purposes of the cost comparison, were understated. /20/ Agency Report, Tab 27C, Appeal No. 2, at 14,119; Tab 27G, Appeal No. 4, at 14,261; Tab 27I, Appeal No. 5, at 14,417. From its review, the AAB concluded that the amount applied for retraining costs was a "fairly conservative number" and that relocation costs were probably understated. Tr. at 357. Nevertheless, the AAB concluded that it would not require increasing these costs because of "evolving departmental policy in [the one-term conversion costs] area to keep those costs down." Tr. at 348. In this regard, the interim version of the DOD A-76 Costing Manual, Paras. C10.2.2 and C10.2.3, provided that retraining and relocation costs should be included in a general severance pay calculation (which the manual stated would be 4 percent of the annual basic pay of all government civilian positions included in the MEO in the first full period of performance). See DOD A-76 Costing Manual, Interim Guidance (Mar. 14, 2001) at 80-81 (found at ). Although the AAB concluded that this interim guidance was not binding, the board found persuasive the stated policy to contain those cost adjustments. See, e.g., Agency Report, Tab 27H, Appeal No. 4 Decision, at 14,388-90.

5. Revised Cost Comparison

As noted, the AAB's decisions on the six appeals sustained some of the appeal issues and denied others. As a result of the board's decisions, a new cost comparison was performed and certified by the Army Audit Agency. Performance in-house was determined to cost less than performance by BAE, as shown by the following table (revisions are in bold):

In-House Cost Estimate BAE

Total proposed costs $58,606,530 $49,650,712

Contract 0 3,073,056 administration

One-time conversion 0 2,287,710 costs

Federal income tax 0

Total adjusted costs 58,606,530 54,713,574

Minimum conversion 0 5,167,955 differential

Final adjusted costs $58,606,530 $59,881,529

See Agency Report, Tab 38A, Revised Cost Comparison Form (Jan. 17, 2001), at 16,707. The revised cost comparison reflected the following changes: the in-house cost estimate's total proposed costs were reduced by $1.8 million, primarily reflecting the reduction in staffing of the personal property shipment offices and the quality control function from those directed by the SSA; contract administration costs (added to BAE's offer) were increased by $1.1 million; one-time conversion costs (added to BAE's offer) were decreased by approximately $100,000 to reflect a decrease in the MEO's overall staffing. As a result of this cost comparison, the cost of in-house performance (after adjustments) was determined to be $1,274,999 lower than performance by BAE.

BAE was notified of the results of the revised cost comparison and requested and received a debriefing. /21/ This protest followed.

THRESHOLD ISSUE: EXHAUSTION OF REMEDIES

As an initial matter, the Army requests that we dismiss BAE's protest because the protester failed to exhaust its administrative remedies by filing an appeal with the agency. Army Motion to Dismiss (Feb. 12, 2001) at 4-10. In this regard, the Army argues that the RSH, as revised by Transmittal Memorandum No. 22, 65 Fed. Reg. 54,568 (Sept. 8, 2000), requires all interested parties to review the tentative cost comparison decision and bring potential errors to the attention of the AAB.

With respect to challenges to cost comparisons under Circular A-76 procedures, we have adopted a policy, for the sake of comity and efficiency, of requiring protesters to exhaust the available administrative appeal process. Thus, we have held that where there is a relatively speedy appeal process for the review of an agency's cost comparison decision, we will not consider objections to the cost comparison that were not appealed to the agency. See Professional Servs. Unified, Inc., B-257360.2, July 21, 1994, 94-2 CPD Para. 39 at 3; Direct Delivery Sys., B-198361, May 16, 1980, 80-1 CPD Para. 343 at 2. Nevertheless, there is no statutory or regulatory requirement that an offeror exhaust available agency-level remedies before protesting to our Office, and it is our view that we retain discretion to waive the policy requiring the exhaustion of the Circular A-76 appeals process where good cause is shown -- for example, where we find that application of the policy in a particular case would not serve its intended purpose of promoting the efficient review of challenges to the cost comparison. Here, we need not decide whether this is an appropriate case for waiving our policy because, as explained below, the applicable agency A-76 appeals procedures did not require BAE to file an appeal.

We reach that conclusion because Transmittal Memorandum No. 22, which is the basis for the Army's dismissal request, did not apply to this cost comparison. /22/ That memorandum states that it is applicable only to cost comparisons "where the in-house offer remains sealed as of the date of this publication" (here, September 8). 65 Fed. Reg. at 54,570. As of September 8, the TPP and MEO had already been unsealed and provided to the SSEB. Tr. at 21-22. The Army argues that the in-house cost estimate remained sealed until October 24. See Army Supplement to Dismissal Request (Feb. 15, 2000); Tr. at 593. In our view, however, OMB's reference to the "in-house offer" in the transmittal memorandum did not mean the in-house cost estimate alone; rather, the RSH makes clear that the "in-house offer" refers to the agency's management plan, that is, the TPP and the MEO. /23/

The RSH, prior to the effective date of Transmittal Memorandum No. 22, did not require BAE to file its own appeal. Rather, the applicable RSH section provided that appeals of the cost comparison to the AAB "must" be filed by an eligible appellant and "[d]emonstrate that the items appealed, individually or in the aggregate, would reverse the tentative decision." /24/ RSH, part I, ch. 3, Para. K.1.e. DOD's implementation of Circular No. A-76 and the RSH also provide that for an appeal to be "eligible for review under the DOD component appeals procedures, [it] shall: . . . (iii) [d]emonstrate that the result of the appeal may change the decision." 32 C.F.R. Sec. 169a.18(a)(5)(iii). Under that rule, BAE was not required, and may not be permitted, to file a "defensive" appeal with the AAB to preserve its right to protest in the event that the agency revised the cost comparison as a result of the appeals on behalf of the MEO team.

BAE'S PROTEST GROUNDS

BAE challenges numerous aspects of the agency's conduct of the cost comparison and of the AAB's decisions, including the government's in-house offer's failure to satisfy the PWS's minimum experience requirements for certain key personnel; the AAB's judgment regarding the amount of staffing required by the in-house offer to perform the personal property services and quality control functions in accordance with the PWS requirements; the Army's failure to consider the evaluated strength in BAE's offer to see walk-in customers in the personal property shipment office within 15 minutes; and the calculations regarding contract administration and retraining/relocation costs made in the cost comparison.

ANALYSIS

1. Summary of Agency's Errors in Conducting Cost Comparison

2. Where, as here, an agency has conducted a cost comparison under OMB Circular No. A-76, thus using the procurement system to determine whether to contract out or to perform work in-house, our Office will consider a protest alleging that the agency has not complied with the applicable procedures in its selection process or has conducted an evaluation that is inconsistent with the solicitation criteria or is otherwise unreasonable. See Trajen, Inc., B-284310, B-284310.2, Mar. 28, 2000, 2000 CPD Para. 61 at 3. To succeed in its protest, the protester must demonstrate not only that the agency failed to follow established procedures, but also that its failure could have materially affected the outcome of the cost comparison. Aberdeen Tech. Servs., B-283727.2, Feb. 22, 2000, 2000 CPD Para. 46 at 5.

Here, we find that the record reflects numerous errors and misunderstandings of the requirements governing cost comparisons under OMB Circular No. A-76 procedures. We address several key flaws in this procurement before turning to the specific grounds on which we sustain the protest.

First, to preserve the integrity of the A-76 cost comparison, private-sector offerors and the government must compete on the basis of the same scope of work. See RSH, part I, ch. 3, Para. H.3.e; see also Aberdeen Tech. Servs., supra, at 8. In the first instance, the RSH requires that both the in-house offer and the private-sector proposals must comply with the minimum PWS requirements. RSH, part II, ch. 2, Para. A.1.b. This determination must be made before there is any consideration as to whether the successful private-sector proposal offers quality and performance exceeding the PWS requirements, such that the in-house offer must be brought up to the private-sector proposal's level of performance and quality. RSH, part I, ch. 3, Para. H.3.d.

It is the IRO's responsibility prior to sealing the government's in-house offer to ensure that the in-house offer satisfies the minimum PWS requirements and that the adjustments necessary to satisfy the PWS requirements are made. See RSH, part I, ch. 3, Paras. H, I, J. Here, the record indicates that the IRO failed to properly carry out its responsibility.

Secondly, the PWS was significantly revised after the in-house offer was certified by the IRO and sealed. The IRO did not consider whether the in-house offer complied with the revised PWS, although that was the basis on which BAE's proposal was prepared and evaluated. See, e.g., Tr. at 672, 678, 697-98. Because of the PWS revisions, the TPP should have been opened prior to the receipt of private-sector offers, examined against the revised requirements and adjusted, as required, and certified anew as satisfying the revised PWS requirements. This was not done here.

Thirdly, the agency apparently believed that no revisions could be made to the in-house offer once it was initially sealed, except to the extent necessary to bring it up to the level of the private-sector offeror's proposal. However, even after completion of the private-sector competition, the agency must ensure the compliance of the in-house offer with the PWS requirements (unless these requirements are also waived for the private-sector offeror). Yet here, even though they found the in-house offer was noncompliant with the PWS requirements, the SSEB and SSA apparently believed that it was inappropriate for them to compare the in-house offer to the PWS requirements to determine what was needed to make the in-house offer compliant. /25/ See, e.g., Tr. at 39, 122, 161, 320-22. In our view, there was no reasonable basis for this belief; once the SSEB or the SSA determined that the in-house offer did not satisfy the PWS requirements, that deficiency needed to be resolved before the agency could proceed to the public/private cost comparison.

Fourthly, the failure to focus on the in-house offer's compliance with the PWS requirements led to a further deficiency. In our view, the SSEB and SSA erred in simply adopting the private-sector offeror's proposed staffing levels to determine the amount of staffing required by the in-house offer to comply with the PWS requirements. Just as two competing private-sector offerors may reasonably propose different levels of staffing, depending on each offeror's technical approach and proposed efficiencies, so, too, the in-house offer may be based on a level of staffing different from that offered by the private-sector proposal. Neither the SSEB nor the SSA should impose the private-sector proposal's staffing level on the in-house team.

Finally, the agency unreasonably failed to determine the in-house offer's compliance with the PWS key personnel experience requirements in the face of evidence indicating noncompliance.

Turning now to the bases on which we sustain BAE's protest, we agree with BAE that the record does not demonstrate that the in-house offer satisfied the minimum PWS requirements. That is, the record does not establish that the in-house offer complies with the PWS key personnel requirements or that the TPP staffing (as adjusted by the AAB) could reasonably satisfy all of the PWS requirements for personal property services. We also find that the record does not establish that the agency considered whether BAE's offer to service walk-in customers to the personal property services offices within 15 minutes offered a level of performance that the in-house offer should have been required to meet. Given the number and extent of the errors made in this procurement and the closeness of the competition, we find that the protester has established that the failure to comply with OMB Circular No. A-76 requirements materially affected the outcome of the cost comparison.

3. In-House Offer's Failure to Meet Key Personnel Experience Requirements

4. As noted, BAE argues that the in-house offer did not satisfy the PWS requirements concerning key personnel. Supplemental Protest at 11-13. Specifically, BAE states that the PWS required, as a part of the proposal, the submission of resumes for key personnel, such as the proposed project manager and assistant project manager, who were stated to be "essential for successful accomplishment of the services to be performed under this contract." The project manager and assistant project manager were required to "have a minimum of five years experience in the management of a similar or related multi-function operation within the last 8 years." PWS Sec. C.1.6.2. The TPP initially did not provide resumes for the project manager and assistant project manager. In response to the SSEB's repeated requests, resumes for these positions were twice provided, but these resumes (including those of the current Director and the Deputy Director of the DOL) did not show the requisite 5 years of experience. Tr. at 61-62. Although the SSEB concluded that the resumes did not demonstrate compliance with this PWS requirement, the SSEB was informed that they should "not worry about them, that that wasn't our -- it wasn't our responsibility to evaluate those." Tr. at 133. As BAE notes, it was required to propose highly experienced (and therefore higher paid) personnel to satisfy this mandatory solicitation provision, which raised the cost of its proposal. Protester's Post-Hearing Comments at 20.

The Army does not contend that either of the two sets of resumes for the project manager and assistant project manager positions submitted by the DOL Study Team satisfied the minimum experience requirements stated in the PWS. Rather, the Army argues that federal personnel law does not permit the agency to offer specific individuals for positions within the MEO (and that the agency could not know exactly who would occupy spaces within the MEO) until after selection of the in-house offer as most economical, and therefore the in-house offer could not contain resumes for key personnel. /26/ Army's Post-Hearing Comments at 17-18.

We need not decide whether the Army was permitted to designate specific individuals for inclusion in the MEO because answering this question does not resolve BAE's fundamental complaint, that is, that BAE and the MEO were not competing on the same basis. Although BAE was required to propose a project manager and assistant project manager satisfying the PWS experience requirements, the government's in-house team provided no evidence that it could provide a project manager and assistant project manager who will satisfy these requirements; to the contrary, the record indicates that the government cannot comply with these requirements. This is not a question of designating a specific individual for inclusion within the MEO, but establishing the in-house team's capability of providing key personnel with the requisite experience. See Aberdeen Tech. Servs., supra, at 9-10 (the in-house offer, like the private-sector offer, must comply with personnel requirements stated in the RFP).

Here, the original TPP did not provide resumes for key personnel, nor state that the project manager and assistant project manager to be ultimately provided would satisfy the PWS requirements. The revised TPP also did not include resumes, but stated that the project manager would satisfy the PWS 5-year experience requirement (the revised TPP did not state that the assistant project manager would satisfy the PWS requirements). Also, none of the four resumes provided in response to the SSEB's requests show that any of these four individuals satisfy the experience requirements. Significantly, the second set of resumes submitted are those of the DOL's Director (as project manager) and Deputy Director (as assistant project manager); although these individuals are responsible for directing and managing all of the installation's logistics functions, these resumes do not show the requisite 5 years of experience. Although the Army now states that, when these positions are filled, individuals meeting the PWS requirements will be provided, the agency does not provide any evidence establishing its capability to do so.

BAE argues that, because the in-house offer did not satisfy this minimum PWS requirement, even after repeated clarifications from the SSEB, the in-house offer should have been rejected as unacceptable. /27/ Protester's Post-Hearing Comments at 15. We disagree. In a cost comparison under Circular No. A-76, the government's in-house offer cannot simply be rejected as unacceptable, even where it does not satisfy the PWS requirements. Rather, unless it concludes that a private/public cost comparison is not appropriate, it is the agency's obligation either to ensure that the in-house offer is adjusted to satisfy the minimum PWS requirements, see Symvionics, Inc., B-281199.2, Mar. 4, 1999, 99-1 CPD Para. 48 at 10-11, or, if the minimum requirements are relaxed or waived, to revise the PWS requirements and allow the private-sector offeror an opportunity to meet the relaxed requirements. See Aberdeen Tech. Servs., supra, at 8 (to preserve the integrity of the cost comparison, the government and private-sector offeror must compete on the same scope of work). Here, the record shows that the PWS's key personnel experience requirements were essentially relaxed or waived for the in-house offer, and BAE was prejudiced by not being given an opportunity to address these less stringent requirements.

5. AAB's Authority to Review SSA's Decision

6. BAE also complains that, as adjusted by the AAB, the in-house offer's proposed staffing is insufficient to satisfy the minimum PWS requirements for the personal property services and quality control functions. In this regard, BAE contends that the AAB improperly usurped the SSA's authority by substituting the board's judgment for that of the SSA. In BAE's view, the AAB could disagree with the SSA's judgment only if the SSA's determinations were found to be unreasonable. Protester's Post-Hearing Comments at 44-46. BAE also asserts that the SSA's determinations as to the amount of staffing required to perform the personal property services and quality control function are management decisions to which the appeals process does not apply. See RSH, part I, ch. 3, Para. K.6.c ("Agency A-76 Administrative Appeal procedures do not apply to questions concerning: . . . (c) Government management decisions involving the Government's certified in-house MEO.")

We disagree with BAE that the AAB was barred by the RSH from reviewing these appeals issues and making its own judgments as to the amount of staffing required for the in-house offer to satisfy the minimum PWS requirements. The very purpose of the appeal process is to provide affected parties with an opportunity for a higher-level administrative review of the agency's cost comparison decision prior to that decision becoming final. See, e.g., 32 C.F.R. Sec. 169a.18(a) (DOD Administrative Appeal Procedures); see also Diebold v. United States, 947 F.2d 787, 806 (6th Cir. 1991) (A-76 administrative appeals process provides for final agency action). Given this purpose, we do not think that the restriction in the RSH cited by BAE was intended to apply to the decisions made by the SSA that were the subject of the appeals here. In this context, the SSA is not making management decisions but is part of the evaluation process, and therefore we see no basis why the SSA's decisions, like any other evaluation determination, cannot be reviewed by the AAB.

Our standard for reviewing the decisions of the AAB with respect to the issues raised by BAE is the same as that employed in reviewing other challenges to an agency's procurement decisions. That is, we will disturb the AAB's decision only where it is shown to be unreasonable, inconsistent with applicable law or regulations, or not in accord with the solicitation criteria. See Trajen, Inc., supra, at 3.

7. In-House Offer's Failure to Meet Personal Property Services Requirement

8. With respect to the personal property services area, as indicated above, the AAB agreed with the SSA that the in-house offer did not satisfy all of the PWS requirements. That is, the AAB concluded that the in-house offer did not offer to maintain personal property services offices at both Schofield Barracks and Fort Shafter, as required by the PWS. Instead of the nine FTEs required by the SSA, the AAB determined that an additional two FTEs of staffing to be based at Fort Shafter would satisfy all of the PWS requirements. In making this determination, however, the AAB did not adequately consider the in-house offer's compliance with the PWS requirement to service walk-in customers within 30 minutes.

As noted by the SSA and SSEB, the in-house offer did not comply with the revised PWS requirements for these services at either Schofield Barracks or Fort Shafter, since the in-house offer did not commit to servicing walk-in customers within 30 minutes as required by the PWS, but limited walk-in customers to emergencies. See Agency Report, Tab 16E, SSA Briefing (Sept. 8, 2000), at 9,105, 9,107. Notwithstanding this determination, the AAB did not perform a reasoned analysis as to what personal property services were required by the PWS and whether the staffing proposed by the in-house offer could reasonably satisfy these minimum requirements. Instead, the AAB accepted the initial staffing included in the in-house offer for performance of personal property services at Schofield Barracks and sought to determine only what additional staffing would be necessary to provide personal property services at Fort Shafter. Such an analysis does not account for satisfying the 30-minute customer walk-in service requirement at either Schofield Barracks or Fort Shafter, as required by the PWS. /28/

In their hearing testimony, the AAB members expressed confidence that the in-house offer could meet the 30-minute walk-in customer service requirement with its proposed staffing, but indicated no understanding of what was actually required. Rather, the AAB members believed that the requirement to service walk-in customers within 30 minutes could be limited to emergencies or that the requirement would be met by seeing a walk-in customer within 30 minutes for the purpose of scheduling a later appointment for that customer. /29/ See Tr. at 425, 518-19.

The PWS did not, however, restrict servicing walk-in customers to emergencies, and we do not think that the PWS can be reasonably read to provide that "servicing" walk-in customers means merely scheduling later appointments for the customers to return. The PWS provides in the same sentence that the "Contractor shall service customers with an appointment within 10 minutes of the appointment time and walk-in customers within 30 minutes of arrival." See PWS Sec. C.5.2.6.1. The next sentence of this PWS section states the services to be provided, that is, customers will be counseled as to their rights and responsibilities concerning the shipment and storage of personal property. In light of that language in the PWS, and because the PWS refers to servicing customers who already have appointments, "servicing" (whether for walk-in customers or others) cannot reasonably be read to mean merely scheduling appointments.

The AAB attempted to support its flawed analysis with opinions from the Army Audit Agency and the Director of the DOL, to little avail, in our view. /30/ The AAB believed that the Army Audit Agency had done its own analysis as to the time required to perform the personal property services transactions and determined how many FTEs would be required. One board member testified that the Army Audit Agency, as the IRO, had "timed how long things take. They had, based upon that workload data, they had done an audit of the particular organization. So they had good data there." Tr. at 458. In fact, the Army Audit Agency had not done any time studies of the historical personal property services function, as the AAB apparently believed, but instead had simply verified the DOL Study Team's mathematical calculations; the Army Audit Agency did not evaluate or verify the DOL Study Team time estimates at all. /31/ See Tr. at 672, 678-80, 702-03, 706. Indeed, the Army Audit Agency representative testified that it was not until the appeals were filed that the Army Audit Agency "realized that there were time standards associated with the workload for personal property." Tr. at 678. The Army Audit Agency's judgment as to the amount of FTEs required to perform the PWS requirements was apparently based upon its determination that the in-house offer's original staffing estimate could handle the expected work at Schofield Barracks and that any additional workload required to service an office at Fort Shafter could be handled on an "on-call basis." Tr. at 677.

The AAB also gave great weight to the opinion of the DOL Director because he was seen as an unbiased, knowledgeable source of information concerning these functions. /32/ Tr. at 198, 335, 365, 459-60. There is no documentation in the record of the DOL Director's opinion, which was provided orally to the AAB. Nor is there any evidence as to whether this opinion is based upon meeting all of the PWS requirements (such as the requirement to service walk-in customers within 30 minutes) or as to the DOL Director's knowledge of the PWS requirements. Given the lack of support for the Army Audit Agency's and the DOL Director's opinions of the amount of staff required for the personal property shipment offices, we think the AAB's reliance upon these opinions was unreasonable.

9. Agency's Failure to Consider Whether BAE's Offer Exceeded the PWS Requirements

10. We are also concerned that neither the AAB nor the SSA/SSEB gave any real consideration to BAE's offer to service walk-in customers within 15 minutes.

Where an agency identifies strengths in a proposal ultimately selected under an A-76 best value procurement where award is based upon a cost/technical tradeoff, or if it identifies areas in which that proposal exceed the PWS requirements, the agency must consider those strengths in comparing that proposal with the in-house offer. RSH, part I, ch. 3, Paras. H.3.d, e; The Jones/Hill Joint Venture-Costs, B-286194.3, Mar. 27, 2001, 2001 CPD Para. __ at 10. An agency's determination that certain identified strengths are not important or of no value must have a reasonable basis. The Jones/Hill Joint Venture-Costs, supra, at 10. This must be done even where, as here, only one offer is received in response to a best value solicitation. This "leveling of the playing field" is necessary because a best value solicitation invites submission of proposals that exceed the RFP requirements, together with the higher costs or prices that often accompany a technically superior approach. Id.

Here, BAE proposed to make

the Customer base aware of group counseling opportunities through unit bulletins, recorded phone messages, an Internet based home page, and during direct Customer inquiry. The use of group counseling frees Section personnel to more efficiently handle walk-in Customers and enables us to define a waiting time goal of no more than 15 minutes.

Agency Report, Tab 7, BAE Technical Proposal, at 1,639. This appears to exceed the PWS requirement to service walk-in customers within 30 minutes. This was noted as a real proposal strength by one SSEB member, who observed that BAE had a "[h]igh probability of success" in meeting this goal. Agency Report, Tab 8K, Consensus Subfactor Rating, Technical Team, at 4,203; Tr. at 126. The fact that BAE offered to service walk-in customers within 15 minutes was not presented to the SSA, who received only summary evaluation findings that did not mention this point. There is no other contemporaneous documentation in the record indicating the SSEB's or SSA's consideration of BAE's offer to service walk-in customers within 15 minutes or why this was not important or of no value to the agency.

The Army now argues, however, that BAE's offer was not a real strength because it was stated to be only a goal. /33/ Contracting Officer's Statement at 6; Tr. at 171. This explanation does not appear in the contemporaneous evaluation record and appears inconsistent with the evaluator's statement that this "goal" was highly achievable. The Army's current position fails to address whether BAE's proposal exceeds the PWS requirements. /34/ While we recognize that the Army may ultimately conclude that BAE's offer to service walk-in customers within 15 minutes does not exceed the PWS requirements in a meaningful way, the record does not indicate that the agency considered this issue. If BAE's proposal does exceed the PWS requirements in this regard, in order to level the playing field, the Army was required either to reasonably determine why this was of no value to the government or to ensure that the in-house offer provided a comparable level of quality and performance. See Rice Servs., Ltd., B-284997, June 29, 2000, 2000 CPD Para. 113 at 7; Aberdeen Tech. Servs., supra, at 14-15.

11. Conclusion

In sum, we find that the record does not establish that the in-house offer satisfied the minimum PWS requirements regarding key personnel and personal property services. /35/ We also find that the record does not indicate that the agency considered whether BAE's offer to service walk-in customers to the personal property services offices within 15 minutes exceeded the PWS requirements and offered a level of performance that the in-house offer should have been required to meet. We also conclude that BAE was prejudiced by these errors, and accordingly we sustain the protest.

OTHER ISSUES

BAE raised a number of other protest arguments that we need not specifically address. For example, BAE asserts that the in-house offer's staffing to perform quality control, after the AAB's decisions, is insufficient to perform all the PWS requirements. Although we do not resolve this issue, we note that the parties' arguments reflect a basic disagreement as to what was required. That is, BAE, as well as the SSA and SSEB, clearly believed, not without some justification, that the requirement for an independent quality plan required an independent quality control staff, while the AAB and the Army believe that this independence can be obtained by having oversight (over an internal quality control process) by an independent person. In any case, we think that the PWS requirements concerning quality control are far from a model of clarity and suggest that, given our protest recommendation, the Army review this matter to ensure that the PWS clearly states the government's actual requirements. /36/

BAE also challenged the agency's calculation of one-time conversion (specifically, relocation and retraining) costs and contract administration costs. Although we do not resolve these issues, we note that, with respect to relocation and retraining costs, the record reflects a number of conflicting calculations. Tr. at 243 . Given the DOD's interim guidance to restrict the amount of relocation and retraining costs added to the cost of the private-sector proposal to ensure that these costs are not overstated, this is a matter that the agency may again wish to review. /37/ With respect to contract administration, while the additional contract administrators required by the AAB's decisions may have been warranted, the record shows that the AAB misread the RSH as requiring eight FTEs of staffing for contract administration for an MEO of the staffing size presented here, see Tr. at 246, 359-60, 485, when in fact the RSH presented this number as a ceiling and provided for the agency to perform a reasoned analysis of the amount of contract administration staffing required. RSH, part II, ch. 3, table 3-1, at 26. The agency may also wish to review this matter.

We sustain the protest.

RECOMMENDATION

Since it appears that the PWS does not accurately and unambiguously state the Army's actual requirements and this caused the competition to be on an unequal basis, we recommend that the agency review its needs and revise the PWS accordingly. After revising the PWS, a new in-house offer should be prepared, a revised proposal solicited from BAE, and a new evaluation and cost comparison performed. Given the length of time that this procurement has already taken, we recommend that the agency implement this as expeditiously as possible. We also recommend that the protester be reimbursed the reasonable costs of filing and pursuing the protest, including reasonable attorneys' fees. 4 C.F.R. Sec. 21.8(d)(1) (2001). The protester's certified claim for costs, detailing the time spent and costs incurred, must be submitted to the agency within 60 days of receiving this decision. 4 C.F.R. Sec. 21.8(f)(1).

Anthony H. Gamboa

General Counsel

1. For the sake of convenience and consistency with the record, we use the term "in-house offer" to refer to the government's management plan, as does the OMB Circular No. A-76 Revised Supplemental Handbook (RSH), even though the government's plan to perform the work in-house is not, in fact, an offer. American Fed'n of Gov't Employees, AFL-CIO et al., B-282904.2, June 7, 2000, 2000 CPD Para. 87 at 3-4.

2. The procedures for determining whether the government should perform an activity in-house, or have the activity performed by a contractor, are set forth in OMB Circular No. A-76 and the RSH, which have been made expressly applicable to the Department of Defense (DOD) and its military departments and agencies. See 32 C.F.R. Sec. 169a.15(d) (2000). The process set out in the Circular and the RSH broadly encompasses the following steps in the conduct of a public-private competition. First, after the PWS has been drafted, the agency ensures that the government's in-house offer has been prepared based on the PWS. RSH, part I, ch. 3, Para. I. Second, there is a competition among private-sector offerors, which is conducted much as any competed federal procurement is conducted. Third, if that competition is done on the basis of a cost/technical tradeoff, the government's in-house offer is compared with the winning private-sector offer to assess whether or not the same level of performance and performance quality will be achieved -- and if it will not, to make all changes necessary to meet the performance standards of the private-sector proposal. Id., Paras. H.3.d, e. Finally, once the playing field is thus leveled, there is a cost comparison between the private-sector offer and the in-house offer. Id. Paras. H, J.

3. At the hearing that our Office conducted in connection with this protest, testimony was elicited from the source selection authority (SSA), the source selection evaluation board (SSEB) Chair, the three AAB members, the Army Audit Agency Audit Manager, the DOL Study Team leader, the contracting officer, and a personnel officer.

4. The contemporaneous evaluation record does not further discuss BAE's offer to see walk-in customers within 15 minutes, and the SSA was not informed of this feature of BAE's proposal.

5. No technical discussions were conducted with BAE. Tr. at 20, 642.

6. The TPP consisted of two volumes: volume one was the technical proposal and volume two was the management operations plan.

7. The initial PWS, upon which the in-house offer was based, provided for a personal property shipment office only at Schofield Barracks. Fort Shafter was added as another personal property shipment office location by amendment No. 3 to the RFP.

8. The original SSA left the command after the selection of BAE's proposal to compete against the public offer. The new SSA was appointed just before the September 8 briefing. Tr. at 43, 303.

9. BAE offered to staff the personal property offices at Fort Shafter and Schofield Barracks with 18 FTEs, all of which were "workers." The TPP offered to staff a personal property office at Schofield Barracks with 1 "nonworking" supervisor and 9 "actual workers," and proposed closing the existing Fort Shafter office. Tr. at 46, 48.

10. The requirement that the quality control function be independent was one of the requirements that, as explained above, had been added by amendment No. 3 to the RFP after the in-house offer had been certified and sealed.

11. After the September 8 briefing to the SSA, the SSEB Chair had prepared analyses to confirm the SSEB's judgment to add nine FTEs for personal property services, three FTEs for quality control, and 5,328 hours for maintenance to the MEO. Agency Report, Tab 16G, SSEB Analyses, 9,303-57, 9,379-419; Tr. at 71-73, 76. For reasons not clear in the record, the information concerning the quality control and personal property services were not presented to, or seen by, the SSA or the AAB. Tr. at 72-74, 100. The SSEB Chair's methodology supporting the additional maintenance hours was provided to the AAB, which (presumably relying on that analysis) denied the appeals challenging the addition of 5,328 hours for maintenance.

12. The SSEB Chair testified that the SSEB's consensus judgment was that after the in-house offer was revised it satisfied all of the "requirements." Tr. at 60.

13. The Army Audit Agency was not aware until after the appeals were filed that the PWS had been amended after the in-house offer had been certified and sealed. Tr. at 666, 676.

14. The six appeals raised a variety of other issues that are not relevant to the issues raised in BAE's protest.

15. Although the Army asserts that the AAB independently determined that two additional FTEs would satisfy the PWS requirements, the record establishes that the AAB did not perform its own analysis of what was required to perform the PWS requirements, but rather assessed the reasonableness of the opinions of the various groups presenting information to the board. See, e.g., Tr. at 337-38, 461-62.

16. The Army performance improvement criteria system was described as being patterned after the Malcolm Baldrige Criteria for Performance Excellence. Tr. at 345. This system is described by the Army's web site as providing a framework for in-depth organizational assessment and measurement of the continuous improvement efforts that are the hallmark of "Total Army Quality." It is also said to guide Army managers through seven categories, which examine all aspects of the Army's organization and determine how well it is meeting its goals. See >.

17. This "technical estimate" was not documented but consisted of the board's "estimate of the scope of responsibility of the logistics mission defined in the PWS." Tr. at 344.

18. The government is permitted to add to the proposed price of the proposal of the most advantageous commercial offeror the costs that would be incurred in administering the contract. RSH, part II, ch. 3, Para. C.1.

19. It is not clear why the RSH bases the number of contract administrators necessary to administer the private-sector contract upon the size of the staffing of the MEO, rather than the proposed staffing level of the private-sector offer.

20. The government is also permitted to add to the proposed price of the proposal of the most advantageous commercial offeror certain "one-time conversion" costs that would be incurred "as a result of the conversion" from in-house to contractor performance. One-time conversion costs may include such things as relocation and retraining costs. RSH, part II, ch. 3, Paras. E.1, E.3.

21. After performing this new cost comparison, the agency determined that BAE's revised cost proposal had not addressed the hazardous pay differential and requested that BAE address this in a response to be submitted by January 26, 2001. Agency Report, Tab 30, Letter from Contracting Officer to BAE (Jan. 19, 2001), at 14,779. BAE responded with a revised cost proposal that changed more than that part dealing with the hazardous pay differential; the Army rejected that proposal and requested that BAE submit changes for the hazardous pay differential only. The Army is currently reviewing that submission, but the agency apparently believes this would not affect the results of the revised cost comparison. Tr. at 603-04.

22. We do not decide whether Transmittal Memorandum No. 22, if applicable, would have required or permitted BAE to appeal.

23. RSH, part I, ch. 3, Para. H.3.d, provides:

With the selection of the competitive offer, the contracting officer submits to the [SSA] the Government's in-house Management Plan, which must comply with the technical proposal requirements of the solicitation. The [SSA] evaluates the in-house offer and assesses whether or not the same level of performance and performance quality will be achieved. The [SSA] should not review or have access to the in-house cost estimate. [Emphasis added.]

24. Transmittal Memorandum No. 22 rescinded this paragraph of the RSH. 65 Fed. Reg. 54,568 (2000).

25. Although the SSA and SSEB Chair recognized that the in-house offer did not fully satisfy minimum PWS requirements, they insisted that they only compared (and could only compare) the in-house offer to the level of quality and performance of BAE's proposal. Tr. at 39, 122, 161, 320-22.

26. The Army also argues that this protest allegation (which was first raised in BAE's supplemental protest) is untimely because the original and revised TPP were available during the public review period after the tentative selection of BAE's offer and these documents do not contain resumes for key personnel. We find the protest allegation timely. Although the Army characterizes BAE's protest as merely asserting that the TPP did not contain key personnel resumes, BAE's actual complaint is that the belatedly provided resumes do not satisfy the minimum experience requirements. BAE did not discover the basis of this complaint until it received the agency's report on its initial protest; BAE timely raised this new complaint within 10 days of receiving the agency's report. 4 C.F.R. Sec. 21.2(a)(2) (2001).

27. BAE also argues in its Supplemental Protest (at 13-16) that the TPP did not comply with the PWS requirement to address, at a minimum, in its management operations plan "the Contractor's project office autonomy, i.e., show how the Contractor plans to provide on-site decision authority and independence from the Contractor's corporate headquarters commensurate with project office responsibility," see PWS Sec. C.1.7.1.1, and that the in-house offer should therefore have been rejected. BAE does not show that the revised TPP failed to demonstrate on-site decision authority and independence commensurate with project office autonomy. From our review of the revised TPP, we also find no basis to question the in-house offer's compliance with this requirement and deny this aspect of the protest.

28. The AAB used an allocation formula for services that indicated that 73 percent of the services would be performed at Schofield Barracks and 27 percent at Fort Shafter. The AAB applied this formula to the in-house offer's original staffing for Schofield Barracks, and concluded that approximately two FTEs of additional staffing would be required to perform the personal property services at Fort Shafter. Tr. at 201-02. This allocation formula fails to account for the fact that the staffing at Schofield Barracks was not based upon satisfying the PWS requirements regarding walk-in customers.

29. The DOL Study Team Leader also testified that the walk-in customer requirement could be satisfied by making an appointment for that person to return later. Tr. at 763-64.

30. We also find of little probative value the one board member's anecdotal observation that his family had been happy with services rendered by one of the personal property shipment offices when that function was being staffed with as few as seven FTEs, as this does not address the PWS requirements, and there is no evidence in the record that the PWS requirements, such as servicing walk-in customers within 30 minutes, were historically provided by the personal property shipment offices.

31. At the time the Army Audit Agency reviewed the original in-house offer for compliance with the PWS requirements, prior to the in-house offer being sealed, it did not focus its attention on the personal property services area because this was not a "high risk area." Tr. at 672.

32. The DOL Director holds what the Army refers to as a "government in nature" position within the function under study here.

33. The Army also argues that this protest argument is untimely because BAE was informed in a written debriefing letter, dated December 20, 2000, after BAE was selected as the tentative winner of the competition and before the AAB had decided the appeals, that its offer of this "strength" did not contribute to a rating higher than acceptable, and BAE did not protest within 10 days of that date. Agency Legal Memorandum at 4. BAE timely protested this issue, however, when there was adverse agency action, that is, when BAE learned that performance of the work would stay in-house.

34. The SSEB Chair also testified that BAE's offer was not a real strength because it only offered to do group counseling within 15 minutes. Tr. at 167. This is a misreading of BAE's proposal, which, as indicated above, proposed group counseling as a means of freeing its staff to service walk-in customers.

35. Testimony elicited at the hearing, as well as the in-house offer's response to the revised PWS requirements, indicates that the Army may well not need to staff a personal property shipment office at Fort Shafter nor require that walk-in customers to the personal property shipment offices be seen within 30 minutes. Given our recommendation to revise the PWS and obtain revised offers from BAE and the in-house team, the Army should consider whether these, in fact, represent its actual needs. There is also a suggestion that the AAB may have reviewed data more current than was available in the PWS. In this regard, the AAB noted in a number of its decisions on the appeals that Army Pamphlet 5-20 Para. 4-12d provides that "[c]ommanders will ensure that the solicitation and MEO are based on the most current workload data." See, e.g., Agency Report, Tab 27B, Appeal No. 1 Decision, at 14,085. To the extent that there is more current workload data available, it would be appropriate to make this data available to BAE and the preparers of the in-house offer.

36. This would appear to be particularly advisable given that the "embedded management processes" for quality control, upon which the AAB relied, would have the effect of reducing the amount of staff necessary to perform this function and that these processes would also be available to a contractor in its performance of the contract. Tr. at 382-83, 433.

37. "It is DoD policy that relocation costs (for civilians) are included in the 4 percent severance factor since current Department-wide statistics indicate that relocation costs paid to civilians are minimal and do not support additive costs in a cost comparison." DOD A-76 Costing Manual, Interim Guidance Para. C10.2.3.