Matter of: Booz, Allen & Hamilton, Inc. File: B-249236.4; B-249236.5 Date: March 5, 1993

Matter of: Booz, Allen & Hamilton, Inc. File: B-249236.4; B-249236.5 Date: March 5, 1993

PROCUREMENT Competitive Negotiation Discussion Adequacy Criteria The requirement for meaningful discussions did not obligate agency to advise protester of proposal weakness, which impacted performance risk assessment of otherwise acceptable proposal, that was not deemed significant during evaluation, even though it ultimately became a determinative factor in selecting awardee's closely-ranked proposal. PROCUREMENT Competitive Negotiation Contract awards Administrative discretion Cost/technical tradeoffs Technical superiority PROCUREMENT Competitive Negotiation Offers Evaluation Adjectival ratings Where solicitation provided that technical and management proposals would be rated on a color/adjectival and a risk basis and both technical and management were of more importance than cost, selection of a higher priced, lower risk proposal over lower priced, higher risk proposal was proper since the agency reasonably found the protester's otherwise acceptable proposal involved greater risk of accomplishing the solicitation's requirements.

Attorneys

DECISION Booz, Allen & Hamilton, Inc. protests the award of a contract to Battelle Memorial Institute by the Department of the Air Force, Wright- Patterson Air Force Base, under request for proposals (RFP) No. F33657-91- R-0039. The protester alleges that the evaluation of proposals was improper.

We deny the protest.

The RFP solicited proposals for the establishment and operation of a Supportability Investment Decision Analysis Center (SIDAC), which is designed to gather, analyze, format, and present information regarding the supportability of certain weapons systems so that the Air Force can make informed investment decisions regarding such systems. The solicitation contemplated award of a 5-year, multi-year indefinite quantity contract, and identified the following categories of work to be performed under the contract: core operations services (firm-fixed-price), extended products and services (cost-plus-fixed-fee), and special projects (indefinite quantity time and materials).[1]

Offerors were informed that the evaluation of proposals would be conducted under the source-selection procedures of Air Force Regulation 70 -30, and that award would be made to the offeror whose proposal, conforming to the solicitation, was deemed most advantageous to the government. The RFP reserved the right of the government to make award to other than the low cost offeror. The solicitation stated that the source selection would include an assessment of each offeror's ability to satisfy the requirements of the solicitation, which would consist of an evaluation of both general considerations (such as past performance, proposed contractual terms and conditions, and preaward surveys), and technical and price proposals. The evaluation factors, in descending order of importance, were technical, management, and cost/price.

Under the technical factor were listed the following six subfactors: (1) technical capability, (2) computer models and data, (3) consultation and analysis, (4) connectivity to external data bases, (5) facilities and infrastructure, and (6) bibliographic data base. Under the management factor were three subfactors: (1) management capability, (2) operations plan, and (3) user service and reimbursement. The RFP advised that the technical and management subfactors would be evaluated with color/adjectival ratings (blue/exceptional, green/acceptable, yellow/marginal, and red/unacceptable) and risk ratings (high, moderate and low). The color rating depicted how well the offeror's proposal met the evaluation standards and solicitation requirements and the risk rating assessed the risks associated with the offeror's proposed effort in accomplishing the solicitation's requirements. The RFP also provided that the government would conduct a performance risk assessment based on each offeror's past and present performance as it relates to the probability of successfully accomplishing the proposed effort. Performance risk was to be equal in importance to the color rating and proposal risk assessment. Finally, the RFP advised that proposed total costs were to be evaluated for reasonableness, realism, and completeness.

Four firms responded to the RFP, each proposing teaming arrangements with themselves as prime contractors and other companies as subcontractors. After initial evaluation by the Source Selection Evaluation Team (SSET), all four initial proposals were included in the competitive range. Clarification requests and deficiency reports were sent to the offerors, their responses were received and the agency conducted final evaluations. Under the technical and management factors, both Battelle's and Booz Allen's proposals received green color ratings for all technical and management subfactors. However, Battelle's proposal received a low risk rating under each of the subfactors, while Booz Allen's proposal received moderate risk ratings for two subfactors, technical capability and facilities/infrastructure, and a high risk rating under management capability. Both Booz Allen and Battelle received low risk ratings for past performance. Booz Allen offered the low estimated total price for the requirement, $36,046,606, and Battelle was second low at $43,487,183. Both offerors' proposed costs were determined to be reasonable, realistic, and complete. However, aspects of Booz Allen's proposal were determined to pose a moderate risk of increased cost, while Battelle's proposal was determined to pose a low risk of increased cost.

The Source Selection Authority (SSA) determined that Battelle's proposal offered the best overall value. The difference in Battelle's higher estimated program cost was considered to be offset by the "excellent" management approach offered by the firm and the low risk its approach offered in satisfying the requirements. Further, the SSA determined that, although Battelle's total estimated program cost was not low, in all likelihood it offered the lowest most probable program cost due to its low proposal and cost risks. In addition, the SSA considered that Battelle offered the lowest evaluated cost for core operations and extended products and services, both of which are mandatory for the day-to-day operations of SIDAC and represent the annual funding requirements for the program. Based on these considerations, the SSA made award to Battelle on June 2, 1992.

Booz Allen challenges numerous aspects of the evaluation. The evaluation of technical proposals is the function of the contracting agency; our review of an allegedly improper evaluation is limited to determining whether the evaluation was reasonable and consistent with the stated criteria. Computer Based Sys., Inc., 70 Comp.Gen. 172 (1991), 91-1 CPD Para. 14; CORVAC, Inc., B-244766, Nov. 13, 1991, 91-2 CPD Para. 454. Based on our review of the record, we conclude that the evaluation was reasonable. We discuss several of Booz Allen's arguments below.

TECHNICAL CAPABILITY SUBFACTOR

Personnel Qualifications

Under the technical capability subfactor, the RFP provided that "the training and experience of the offeror's employees and those of its subcontractors" would be assessed with attention to "the offeror's personnel qualifications relevant to establishing and maintaining the SIDAC services and products." While under this subfactor the evaluators rated Booz Allen's proposed core operations staff green/acceptable, they nevertheless found some weaknesses in qualifications and experience. The evaluators determined that Booz Allen's proposed librarian had "marginal" qualifications (high school diploma and 5 years experience), and that the firm's proposed computer scientist had "minimal" experience (bachelor's degree in computer engineering and 4 years experience). In addition, the evaluators determined that instead of proposing a technical writer available for core operations, Booz Allen proposed a non-degreed publications specialist, with no technical writing experience, to develop numerous core publications. According to the evaluators, "[these] shortcomings in the core technical staff pose the potential of degraded performance of the core operations as well as less-than-satisfactory quality of the resulting products." It was this determination of possible degraded performance on which the evaluators based Booz Allen's moderate risk assessment for technical capability.

Booz Allen contends that its proposal improperly was assessed higher risk than Battelle's under this subfactor. The protester points out that while the individuals proposed by Battelle for the positions of librarian (bachelor's degree in chemistry and 25 years experience) and computer scientist (bachelor's degree in computer science and 8 years experience) "may appear to have an advantage" over Booz Allen's, the agency failed to consider that Booz Allen proposed double the labor hours for core operations proposed by Battelle. The protester believes this additional level-of-effort should have overcome any evaluated risk concerning the firm's proposed librarian and computer scientist. Further, concerning the technical writer, Booz Allen complains that the agency failed to consider that the firm's approach "anticipated that . . . more qualified technical personnel [other than the publications specialist would] provide the limited writing capabilities required for core operations," and that, in any event, Battelle's failure to propose a technical writer for core operations was not similarly assessed higher risk.[2]

We find nothing unreasonable in the evaluation under this subfactor. While Booz Allen proposed more core operations hours than Battelle, we see no reason why the agency would have been required to look to offset the risk associated with relatively underqualified personnel with these additional hours. There certainly was nothing in the RFP indicating that the number of core operations hours would be applied in this manner in the evaluation.[3]

As for Booz Allen's failure to specifically propose a technical writer available for core operations, the RFP advised offerors that "as a minimum, the contractor shall make available for SIDAC use, personnel necessary to fulfill all requirements for core operations." The statement of work listed numerous required core operations publications, including an informational brochure for current and potential users; a monthly bulletin which would include articles, critiques and summaries of technical breakthroughs, applications, and developments; and quarterly newsletters which would detail current research and development topics. Thus, contrary to Booz Allen's characterization of the need for a technical writer as limited, it appears the on-going technical writing required could be rather extensive. This being the case, we see nothing unreasonable in the agency's conclusion that a failure to propose a technical writer available for core operations introduced some degree of risk into Booz Allen's performance.

Booz Allen contends that in accomplishing the core operations it would have access to other experts, its proposal stated that, when necessary, core staff would be able to "draw on a much larger pool of experts." Also, Booz Allen listed technical writers and their qualifications in the extended products and services/special projects portion of its proposal. However, it was not evident from these aspects of Booz Allen's proposal alone that the firm would have a technical writer available for core operations. Additionally, the agency noted in this regard that Booz Allen failed to provide any hours for a core operations technical writer in the firm's operations plan. We conclude that Booze Allen's proposal gave no clear indication that a technical writer would be available for core operations, and thus see no basis to question the agency's evaluation in this regard.

In contrast, while Battelle also listed technical writers and their qualifications in the extended products and services/special projects portion of its proposal, we believe that the agency reasonably interpreted the firm's proposal as indicating that the writers would be available for core operations. In its proposal, Battelle described its listed core personnel as "key," indicating that Battelle employees in addition to those listed as key may be used for core operations. Further, Battelle stated in its proposal that the SIDAC program manager "can call upon any individual on the Battelle staff or any team members to respond to support requirements;" the technical writers were then listed under the title "Survey of Battelle Team Capabilities." Moreover, Battelle's operations plan, unlike Booz Allen's, included hours for a core operations technical writer. Based on these circumstances, we conclude that the evaluation in this area was reasonable.

Discussions

Booz Allen argues that to the extent that its staff qualifications or experience posed any weakness, the agency improperly failed to discuss the matter with the firm. We disagree. In general, agencies are required to hold discussions with all offerors in the competitive range and this mandate is satisfied only when discussions are meaningful. However, agencies are not obligated to afford offerors all-encompassing discussions. The content and extent of meaningful discussions in a given case is a matter of judgement primarily for the determination by the agency involved and not subject to question by our Office unless clearly arbitrary or without a reasonable basis. Where a proposal is considered to be acceptable and in the competitive range, the agency is under no obligation to discuss every aspect of the proposal receiving less than the maximum ranking. See Associated Chem. and Envtl. Servs. et al., 67 Comp. Gen. 314 (1988), 88-1 CPD Para. 248; Fairchild Space and Defense Corp., B-243716; B-243716.2, Aug. 23, 1991, 91-2 CPD Para. 190.

Here, while the experience and qualifications of certain Booz Allen personnel led the agency to assess the firm's proposal with a higher level of risk, this was a relatively minor concern to agency evaluators compared to the firm's proposed assignment of SIDAC core staff to another Air Force contract. This aspect of the initial proposal caused the evaluators to assess the firm's proposal with a high risk due to the probability of degraded performance and was the subject of a clarification request. Booz Allen's response, which included plans for reassignment and replacement of staff on the other Air Force contract, satisfied the agency and the firm's risk rating was reduced to moderate. Because Booz Allen's experience and qualifications weakness was not viewed as significant compared to the staff assignment problem, and the proposal had been rated green in this area in the technical evaluation, the Air Force did not consider it necessary to raise this matter during discussions.

We find that the Air Force's judgement was reasonable under the circumstances. While Booz Allen's moderate risk rating in this area ultimately was a factor in the selection of Battelle, we think the agency is correct that this aspect of the risk assessment did not, by itself, appear sufficiently significant during the evaluation that it would become one of the factors (along with the other higher risk areas) in the award decision. Rather, it was not until the agency performed a final comparative evaluation of the two closely-rated proposals that the difference in core staffing qualifications and experience became a determinative factor. Under the circumstances, we do not believe that the Air Force was required to reopen discussions after this became apparent. Training and Mgmt. Resources, Inc., B-220965, Mar. 12, 1986, 86-1 CPD Para. 244; Prison Health Servs., Inc., B-215613.2, Dec. 10, 1984, 84-2 CPD Para. 643.

FACILITIES AND INFRASTRUCTURE

Support Equipment

Under the facilities and infrastructure subfactor, the RFP provided for an assessment of both the proposed facilities to house the SIDAC and the support equipment that would be required to permit the staff to fulfill the contract requirements. In evaluating Booz Allen's proposed equipment, the SSET considered the firm's approach of using computer hardware equipment currently owned by the firm. The firm proposed its own Digital Equipment Corporation MICRO VAX II as its core processor, networked to other Booz Allen-owned computers and peripherals. The evaluators determined that this approach posed a risk to the government in the transition of the SIDAC system to a successor contractor at expiration of the contract here because there would be no leases on equipment which could be transferred to the existing operating system. This, according to the agency, would pose the problem of either obtaining matching hardware equipment, or rehosting, i.e., installing software and databases created under the contract here on new hardware, if the hardware obtained was not exactly the same as the original system. The evaluators were concerned that the risk of rehosting was high because the MICRO VAX II is no longer in production, thus making it highly unlikely that identical equipment would be readily available (or desirable) as the basis for a successor contractor's system 5 years hence.

Based on past experience, the evaluators determined that rehosting could cause significant problems for the government in the form of schedule risk to the continuity of SIDAC services and cost risk to the successor contractor due to the uncertain workload required to successfully rehost the software on other equipment. Based on these considerations, the agency determined that Booz Allen's approach of using currently-owned ADP hardware equipment was a less favorable approach than using leased equipment and thus assessed this approach a moderate risk.[4]

Booz Allen argues that the evaluation of its hardware approach was improperly based on an unstated requirement for the delivery of hardware at contract termination. Additionally, the protester argues that to the extent its approach posed any risk, an equivalent risk should have been assessed Battelle since its proposed equipment is in the same family of equipment as Booz Allen's.

We find no basis to question the agency's assessment of moderate risk due to Booz Allen's approach of using its own out-of-production hardware first manufactured in 1985. This did not constitute the imposition of an unstated requirement; rather, it merely took into account, in evaluating the support equipment as stated under this subfactor, the relative desirability of the approach. The Air Force's conclusion that out-of- production equipment could present rehosting problems in the future appears logical, as does its concern that such old equipment may not be desirable even if it were available at the end of the contract 5 years hence. The fact (asserted by Booz Allen) that the proposed hardware may be currently commercially available and in use by the contracting agency does not address the agency's concern about future requirements. We conclude that the assessment of moderate risk in this area was reasonable.

We find no merit to the protester's contention that the agency should have assessed moderate risk to Battelle based on its proposed equipment. According to the agency, Battelle's proposed hardware, the MICRO VAX 4000, is state-of-the-art equipment, 16 to 32 times more powerful than that proposed by Booz Allen. Booz Allen does not challenge this assessment. Consequently, while Battelle's equipment may be manufactured by the same company it is not the same equipment. While the leased equipment Battelle proposed may also be unavailable or out-of-production at contract expiration, as the protester argues, we believe that the agency reasonably viewed the leases as alleviating any risk of nonavailability, since leases on the equipment could be transferred to a new contractor at the expiration of the contract.

SIDAC Remote Access

Contributing to the moderate risk assessed Booz Allen's proposal under the facilities and infrastructure subfactor was the firm's proposed approach for remote user access to the SIDAC system. The RFP stated that "at least five users must be able to simultaneously, electronically access the data bases and software tools on the SIDAC system, both from within the SIDAC and from remote facilities via a modem interface using a commercial telephone number." In this regard, Booz Allen stated in its proposal:

"We have configured every computer in SIDAC with hardware and software capability to communicate with electronic databases. Furthermore, each analyst's microcomputer has its own dedicated telephone line to provide direct access when required. In addition, the Micro VAX II, with which the microcomputers are linked via a LAN has five dedicated telephone lines that may be used for both incoming and outgoing calls. Our high-speed Telebridge modem, which has another separate telephone line dedicated to communications, provides additional backup capability. This robust connectivity configuration has been designed especially to ensure SIDAC analysts can access data quickly and without delay whenever the actual database resides." (Emphasis added.)

The Air Force determined that Booz Allen's proposed approach of five modem-connected telephone lines met the requirement for access to the SIDAC by five users simultaneously, but nevertheless assessed risk to the firm's approach. The evaluators considered the bi-directional nature of Booz Allen's five offered lines, i.e., the lines could be used for both outgoing and incoming connections, to pose a potential access problem; if outgoing connections were made, incoming connections would be limited. The evaluators concluded that because Booz Allen's approach could result in limited incoming connections, and it thus was possible that at any given time fewer than five remote users would be able to access the SIDAC simultaneously as required, Booz Allen's approach was determined to pose moderate risk of degraded performance.

Booz Allen argues that it fully complied with the requirement, but that the agency misunderstood its approach. Specifically, the protester maintains that its proposed system was configured in such a way (with each terminal within the SIDAC having its own internal modem, telephone line, and software) "so that outgoing calls from users inside the SIDAC would not involve the five lines which could be used for access by remote users." Further, the protester argues that its approach is at least as good as, if not superior to, Battelle's, which was not assessed with similar higher risk.

We find nothing improper in the evaluation in this area. Booz Allen acknowledges that its five proposed telephone lines could be used for both incoming and outgoing communications, and we agree with the agency's assessment that there was nothing else in the proposal that clearly eliminated the obvious resulting remote user accessibility problem. In this regard, even though each microcomputer would have its own dedicated telephone line to the SIDAC database, users still would have to use the five lines to access external databases. This clearly could result in fewer than five remote users having access simultaneously.

In contrast, Battelle proposed to use eight modems, three more than the minimum, for access to the SIDAC by five simultaneous remote users, and further proposed a separate routing system to connect internal users to external databases, and thereby control outgoing communications. The protester contends that Battelle's external users could end up competing with internal users on the proposed eight modems under the same theory the agency applied to its five modems. Even if Booz Allen is correct, the protester does not address the effect of the added features of Battelle's system, we see nothing unreasonable in the agency's view that providing eight lines for remote users, along with a separate system to route outgoing communications to external databases, sufficiently reduced the risk of interference with remote user access so that a higher risk assessment was not warranted.

MANAGEMENT CAPABILITY

In the management area, under the management capability subfactor, the RFP provided that the evaluators would assess "how well suited the offeror's proposed management structure, methods and staffing are for providing the services and products specified in the [statement of work] in a rapid, efficient, and high-quality manner," along with the risks associated with the proposed effort. In this area, Booz Allen's proposal was rated high risk, while Battelle's was rated low risk. Booz Allen's proposed approach for managing subcontractors under portions of its proposal would involve personnel from one company managing personnel from another company on a given task. Under this approach, for a hypothetical labor mix given by the RFP, Booz Allen planned to retain 43 percent of the labor hours for a delivery order task and have the remaining hours assigned to personnel from different subcontractors in different labor categories. When more than one company possessed the same needed skill, Booz Allen would choose the company that offered a "cost advantage," i.e., the lowest rate for a given labor category from among all of the firm's subcontractors. Booz Allen's proposal indicated that program manager hours were to be distributed among different subcontractors in percentages approximately equal to each company's total hours. This meant that Booz Allen would retain 43 percent of the management hours and 5 subcontractors would provide varying percentages (from 17 to 5 percent) of the remainder of the hours.

The evaluators believed that the firm's approach to managing personnel across company lines would be difficult to execute because of conflicting priorities among competing subcontractors and multiple management chains, which would result in slower communications and increased response times. The evaluators concluded that there was a high risk of degraded performance in Booz Allen's management approach. Additionally, the evaluators determined that there was a related moderate risk of increased cost because overlapping management could involve the expenditure of more hours.[5]

The agency's concern with this inter-company matrix management was the subject of a clarification request submitted to the protester as follows:

"The methodology used to allocate subcontractor and prime contractor effort among labor categories for the [special projects] T&M effort is unclear; furthermore, it appears to be managerially difficult, particularly related to subcontractor effort. Please explain the methodology and its managerial impact."

The protester responded that it did not expect every subcontractor to be involved with each special project; based on past experience, it expected that most special projects would require no more than one or two subcontractors and four to seven technical labor categories. The response further provided that this approach would simplify the individual task management process, and that it would keep managerial risk "to a minimum" by having its program manager meet with the task managers monthly to ensure that all task obligations are being met.

Booz Allen argues that the high risk rating its proposal received under management capability was unwarranted, since Battelle proposed a similar approach of assigning personnel to programs managed by other companies, but Battelle's proposal was not assessed a comparable higher risk.

Our review of the record indicates no reason to question the agency's assessment of risk in this area. Contrary to Booz Allen's argument, Battelle did not offer the exact same management approach as Booz Allen. Specifically, Battelle proposed to supply 67 percent of the special projects manhours, with the balance supplied by the firm's subcontractors, and, most significantly, proposed to retain 85 percent of the program management effort. Thus, while Battelle's approach, like Booz Allen's, could result in personnel working on programs managed by other companies, the agency viewed the firm's retention of significantly larger portions of the management responsibility as eliminating the risk found in Booz Allen's approach. Since additional hours likely would not be required under Battelle's management approach, the approach was found to introduce no performance or cost risk.

We deny the protest.

1. The core operations include core facilities, a source library, a computerized data base, links to Air Force computer data bases, dial-in access to SIDAC computerized resources, newsletter and promotional materials, and bibliographic information. The extended products and services include bibliographic services, technical expertise/consultation in analysis, training and seminars, distribution of handbooks/standards, and tailoring of models and simulations. Special projects include long- term studies/analyses and conferences/symposia.

2. Booze Allen also initially argued that the agency's criticism of the firm's proposed librarian and computer scientist was unreasonable because its proposed staff met all requirements. This argument is without merit. The agency's criticism notwithstanding, Booz Allen's proposal was rated green/acceptable in this area, indicating that it was found to satisfy the RFP requirements.

3. While the solicitation instructed offerors to provide core operations manhours by labor category, this was in connection with the operations plan of the management portion of the proposal. The RFP provided that the operations plan was to be used to assess an offeror's plan for quickly and efficiently implementing the products and services in the SOW, and would be incorporated into the contract.

4. Transition risk also had an impact on technical subfactors 4 and 6, connectivity to external data bases and bibliographic data base, where the RFP provided for the evaluation of offerors' approaches for transition at contract expiration of certain SIDAC computerized data bases. However, rather than double count transition risk for Booze Allen, the evaluators counted it only once under technical subfactor 5, facilities and infrastructure, which was most closely associated with the ADP system approach.

5. Additionally, under cost, because the team mix from different subcontractors for the composite rate is not contractually binding, the evaluators perceived a moderate risk of increased cost if Booz Allen deviated from the team mix by using one of the higher priced subcontractors or elected to perform more of the effort itself, since Booz Allen's average composite rate is 13 percent higher than the firm's overall team average composite rate. Booz Allen's protest that the cost risk assessed to its proposal was arbitrary is untimely filed and will not be considered. Our Bid Protest Regulations require that a protest be filed within 10 working days after the basis of the protest is known or should have been known. 4 C.F.R. Sec. 21.2(a)(2). Each new protest ground must independently satisfy the timeliness requirement of our regulations, which do not contemplate the piecemeal presentation or development of protest issues, with the possible resulting disruption of the procurement of goods and services indefinitely. RRRS Enters., Inc., B-241512 et al., Feb. 12, 1991, 91-1 CPD Para. 152. The basis for the cost risk protest was the debriefing held on June 12, 1992. Both the debriefing script and charts show that the cost risk assessment was disclosed to the protester during the debriefing. Consequently, in order to be timely, Booz Allen was required to file the protest no later than June 26. Because the protest was not filed until August 17, it will not be considered.