Skip to Highlights
Highlights

Alleged deficiencies in agency source selection plan do not themselves provide a basis for questioning the validity of an award selection since source selection plans are internal agency instructions and as such do not give outside parties any rights. Under the project the Air Force is reassigning certain intelligence data handling system responsibilities from Headquarters. QSI contends that the evaluation process was tainted by an apparent conflict of interest since one member of the evaluation panel and an employee of a Booz-Allen subcontractor are close personal friends. That the Air Force's evaluation of the QSI proposal was unreasonable and that the agency failed to hold meaningful discussions with QSI.

View Decision

B-235344, B-235344.2, Aug 31, 1989, 89-2 CPD 197

PROCUREMENT - Competitive Negotiation - Technical evaluation boards - Bias allegation - Allegation substantiation - Evidence sufficiency PROCUREMENT - Bid Protests - Non-prejudicial allegation - GAO review DIGEST: 1. Protester fails to show that improper conduct occurred resulting in a conflict of interest where most of the social contacts which the protester describes between evaluation panel member and employee of awardee's consultant occurred long before the start of procurement and there has been no opportunity for information to be improperly disclosed by the evaluator or for the evaluator and the consultant to improperly discuss the procurement and the record contains no evidence of bias or preferential treatment toward awardee. PROCUREMENT - Competitive Negotiation - Contract awards - Propriety Evaluation errors - Materiality 2. Although agency improperly downgraded proposal based on evaluators' erroneous conclusion that key employee would not be available to the extent proposed and agency failed to raise in discussions evaluators' concern with inflexibility of software proposed, these deficiencies in the procurement had no impact on the selection of the awardee's higher rated technical and significantly lower cost proposal. PROCUREMENT - Competitive Negotiation - Contract awards - Propriety 3. Alleged deficiencies in agency source selection plan do not themselves provide a basis for questioning the validity of an award selection since source selection plans are internal agency instructions and as such do not give outside parties any rights.

Quality Systems, Inc.:

Quality Systems, Inc. (QSI), protests the award of a cost- plus award-fee contract to Booz-Allen Hamilton under request for proposals (RFP) No. F30602-89-R-0008, issued by the Air Force for support services for the agency's intelligence data handling systems project. Under the project the Air Force is reassigning certain intelligence data handling system responsibilities from Headquarters, Air Force in Washington, D.C. to the Air Force's Electronic Systems Division and to the Rome Air Development Center at Griffiss Air Force Base, New York. Among other allegations, QSI contends that the evaluation process was tainted by an apparent conflict of interest since one member of the evaluation panel and an employee of a Booz-Allen subcontractor are close personal friends, that the Air Force's evaluation of the QSI proposal was unreasonable and that the agency failed to hold meaningful discussions with QSI. We deny the protest.

The solicitation, issued on November 23, 1988, indicated that award would be made to the responsible offeror whose proposal contained the combination of capabilities offering the best overall value to the government. According to the solicitation, the agency was more concerned with obtaining superior technical and management features than with making award at the lowest cost although the solicitation indicated that the government would not make award at a significantly higher cost to achieve only a slight technical or management advantage.

The technical evaluation criteria, listed in descending order of importance, were integration, technical support and technical organization. Those three factors were each to be evaluated against the following assessment criteria, all of equal importance: understanding of the effort, soundness of approach, novel approach and management capability. The cost evaluation was to consider reasonableness, realism, completeness and credibility and was to include an evaluation of the cost risk inherent in each offeror's proposal based on a technical risk assessment of each proposal.

Four firms submitted initial proposals. After a technical evaluation and a determination to include all proposals in the competitive range, the agency held oral discussions with all four offerors and requested best and final offers (BAFOs).

The evaluation panel gave each of the four BAFOs one of the following color-coded ratings on each of the three noncost evaluation factors: blue (exceptional), green (acceptable), yellow (marginal) and red (unacceptable). Also, in accordance with the solicitation evaluation scheme, each proposal was rated for risk. The initial and final evaluation results for QSI and Booz-Allen were as follows:

Technical Technical BAFO

Integration Support Organization Cost

Booz-Allen

initial green green yellow risk low moderate high

BAFO greenblue green $7,100,599

risk low moderate moderate

QSI

initial green green yellow

risk low moderate high

BAFO green green yellow $8,454,195

risklow moderate moderate

The technical evaluation panel ranked the Booz-Allen and QSI proposals as first and second. The source selection authority concluded that Booz- Allen proposed the best overall value to the government. The Air Force therefore awarded the contract to Booz-Allen on April 19, 1989. After a debriefing on April 25, QSI protested on April 28 and on June 21. The Air Force suspended performance of the contract pending our decision on the protest.

CONFLICT OF INTEREST

QSI first contends that a member of the Air Force technical evaluation panel has a close personal relationship with an employee of a Booz-Allen subcontractor on this project and that this relationship caused a conflict of interest that has tainted the award decision. Specifically, QSI contends that the Air Force evaluator and the employee frequently have played golf together, that the evaluator was a guest at the employee's vacation condominium and that the employee was an overnight guest at the home of the evaluator on several occasions. The protester notes that the conflict of interest provision of the Air Force's source selection procedures requires persons involved in the source selection process to inform the source selection team chairman if their participation would cause a conflict of interest and requires the disqualification of such persons from participation in the evaluation process. QSI argues that, as a result of his relationship with this employee, the evaluator in question here should have informed the chairman of the apparent conflict and should have been excluded from participation on the evaluation panel. Although QSI alleges no impropriety beyond the evaluator's failure to make known his relationship with the subcontractor employee and the agency's failure to exclude the evaluator, the protester argues that the evaluator's relationship with the employee left him in a position to influence the selection decision and tainted the procurement so that the award to Booz- Allen should be canceled.

The Air Force disagrees. In affidavits submitted with the agency's report on the protest, the evaluation panel member and the employee admit that they have known each other for approximately 7 years and that they became acquainted while the subcontractor employee was employed by the protester. According to the affidavits, they played golf together on several occasions including two golf tournaments sponsored by the Air Force Communications and Electronics Association and on three or four other occasions with several other government and QSI employees. On at least three of those golf outings, the evaluation panel member and others stayed in condominiums in North Carolina, one of which was partially owned by the then employee of the protester. The affidavits emphasize that each participant in those outings paid a share of the rent and otherwise paid his own expenses. According to the affidavits, these golf outings all occurred prior to the evaluation panel member's involvement in this procurement.

The affidavits also explain that in August 1988, before the evaluation panel member became involved in this procurement, the subcontractor employee and his wife were guests at the home of the evaluation panel member for 2 days. According to the evaluator's affidavit, since he had no knowledge of this procurement at that time, he could not have discussed the procurement with the subcontractor employee.

The Air Force states that the two individuals have had no social contact during the course of the procurement and, according to the agency, the Air Force conflict of interest regulation does not envision disqualification of evaluators under the circumstance presented here. The agency also says that there is no evidence of any improper influence in the evaluation or the source selection and, in fact, the contribution of the evaluator in question was consistent with that of the other four panel members, so that even if his results were not counted, the selection decision would be the same.

In addressing allegations of bias on the part of an evaluation official, we focus on whether the individual involved in the alleged improper conduct exerted improper influence in the procurement on behalf of the awardee. ITECH, Inc., et al., B-231693 et al., Sept. 20, 1988, 88-2 CPD Para. 268. We examine the circumstances of the procurement and the involvement of the agency personnel who allegedly engaged in the improper conduct to determine if the award decision was improperly influenced. Id.

Here, we do not find that improper conduct occurred. First, the golf outings in which the subcontractor employee and the evaluator participated occurred long before the start of this procurement and involved others, including agency employees and the protester's employees. Further, although the subcontractor employee and the evaluator admit playing golf together and otherwise socializing over a 7-year period and admit that the employee and his wife were guests at the evaluator's home, since the record shows the two have had no social contact during the procurement, there has been no opportunity for information to be improperly disclosed by the evaluator or for the two to improperly discuss this procurement. The protester has been able to produce no evidence which contradicts the affidavits. Since the record contains no evidence of bias or preferential treatment toward Booz Allen by the evaluator, and because the protester's mere suspicion regarding potential conflicts of interest does not justify overturning the award to Booz-Allen, the protest is denied on this basis. Mariah Assocs., Inc., B-231710, Oct. 17, 1988, 88-2 CPD Para. 357.

AWARD SELECTION

QSI makes a number of allegations concerning the Air Force's evaluation of the proposals and its failure to conduct meaningful discussions. The most significant of these concern the Air Force evaluators' determination that one of QSI's key employees would not be available full time and their view that QSI's proposed software tools lack flexibility. In connection with the QSI key employee, the protester complains that the Air Force improperly downgraded its proposal based on the evaluators' erroneous belief that the employee QSI had proposed to manage the transition effort would not be available to work the hours proposed on the transition plan. As far as software tool flexibility is concerned, the protester complains that its proposal was improperly downgraded under two evaluation factors for the same alleged weakness and argues that the matter was not raised during discussions. While we agree with the protester that to some extent these matter constituted deficiencies in the evaluation process, we do not believe, for the reasons that follow, that they would have impacted the selection of the awardee's technically higher rated, significantly lower cost proposal. Also, as will be explained below, we disagree with the protester's numerous other arguments concerning the evaluation process.

Key employee

During the evaluation of the initial proposals, the evaluators questioned whether the key individual proposed by QSI to manage the transition would actually be available in view of the statement in the protester's proposal that the employee "will be made available at 50 percent for this effort." Because of this, during discussions, the Air Force asked QSI to clarify this individual's status. In its BAFO, QSI indicated that this individual "will participate throughout the life of the contract at approximately a 50 percent level" and that "for the anticipated 6-month transition period, however, his participation will approach 100 percent." Finally, the BAFO indicated that this employee "will be on-site ... each week, Tuesday through Friday, and will be available during normal working hours."

The evaluation report on QSI's BAFO under the technical organization factor states that QSI's transition plan "still contains technical and schedule risk," and commented that, although the key employee is proposed for 100 percent of the start-up, he is also proposed for a "related AFIA follow-on in the same time period, so his actual utilization on our effort carries some risk." The evaluation report also states that "there is a potential for conflict in the key employee's contract start-up support."

QSI argues that the evaluators' conclusion with respect to the availability of this individual was in error since the other solicitation with which the agency was concerned had not even been released at that time. Moreover, QSI argues that in speculating beyond the written BAFO as to the availability of this individual, the Air Force applied criteria beyond those stated in the solicitation. QSI notes that its rating on the technical organization factor was its lowest and argues that the agency's failure to accept QSI's written commitment to use this individual violated the terms of the RFP and deprived the firm of a chance to improve its rating.

The Air Force says that its evaluators were concerned that the key QSI employee had been or would be proposed for another contract and thus would not be available for this contract to the extent proposed. Also, according to the agency, QSI's proposal itself was unclear as to what extent this individual would actually work on the contract.

It is not the function of our Office to evaluate proposals or to resolve disputes over their rating. Rather, we will examine an agency's evaluation to insure that it was reasonable and consistent with the stated evaluation criteria. The determination of the relative merits of proposals is primarily a matter of administrative discretion which we will not disturb unless we find it to be arbi-trary. William B. Hackett Assocs., Inc., B-232799, Jan. 18, 1989, 89-1 CPD Para. 46. Further, although an agency is not required to discuss every element of a technically acceptable competitive range proposal that has received less than the maximum possible rating, it generally must lead offerors into the areas of their proposals which require amplification. S.T. Research Corp., B-233115, Feb. 15, 1989, 89-1 CPD Para. 159. In our view, although agency officials were legitimately concerned about the commitment of the individual in question, the evaluators should not have looked beyond QSI's written proposal to resolve this issue without discussing the matter with the protester. To the extent they had a concern about that particular employee's availability because of other contract work, they should have informed QSI that the employee's commitment to another contract was the basis of their concern. Further, we do not agree with the Air Force's argument raised in the protest report that QSI's BAFO was unclear as to the commitment of this key individual to the contract. In response to the Air Force question during discussions, QSI indicated in its BAFO that this individual would participate at a 50 percent level over the life of the contract but that during the anticipated 6-month transition period his participation would approach 100 percent. We think that represented a reasonably clear commitment to have that employee available for the transition effort. In fact, the only doubt reflected in the evaluation report concerned that employee's alleged obligation to perform under another contract, a concern which the protester says has no basis in fact or in its BAFO. We agree with the protester that, under the circumstances here, its BAFO should not have been downgraded because of this matter.

Software flexibility

QSI also contends that the agency evaluators misapplied the RFP evaluation criteria when they considered the flexibility of software tools proposed by QSI. QSI notes that the initial technical evaluation report cites the inflexibility of QSI software tools as a weakness under both the technical support factor and the technical organization factor although software tools only should have been considered under technical support. QSI also says that the agency failed in its duty to conduct meaningful discussions because it did not point out to the firm the evaluators' concern with the inflexibility of the software tools proposed by QSI even though both the initial and BAFO evaluation summaries indicate that the evaluation panel was concerned with this issue.

In response, the Air Force explains that the technical evaluation report on QSI's initial proposal contained a word processing error in which a paragraph in the risk discussion under the technical support evaluation factor was included also in the risk discussion under the technical organization factor. Thus, under both factors, the technical evaluation of QSI's initial proposal included criticism of "the status and operational availability of several of the proposed tools" and specifically stated, by way of example, "MML and DIS are not very flexible tools." The agency explains that inflexibility of software tools should have been listed as a risk factor under the technical support criterion but not under the technical organization criterion.

The agency also says that the issue concerning QSI's software tools was not addressed in the discussions because it was not a major weakness that affected the firm's color-coded proposal rating. In an affidavit submitted by the Air Force, the head of the agency's technical evaluation team explains that software inflexibility was not a significant technical issue since the technical evaluators knew that the same or similar software was operating under another contract. According to the Air Force, this issue only affected the technical support risk assessment and not the color-coded rating under the technical support criterion.

QSI, nonetheless, argues that in the initial and BAFO evaluation reports there were five references to inflexibility of software tools and that only one of these references is attributable to the word processing error. Under the circumstances, QSI maintains that it is unreasonable to argue that inflexibility of software tools was not a significant technical issue that counted heavily against QSI in the final evaluation and in the selection decision. Thus, QSI maintains that software inflexibility should have been an issue covered in discussions.

In our view, the fact that software tools erroneously was listed as a risk factor under technical organization in the initial evaluation report resulted in no prejudice to QSI since QSI was included in the competitive range, participated in discussions and was given an opportunity to submit a BAFO. Moreover, although in the BAFO evaluation inflexibility of software tools is still listed as a risk factor under the technical support criterion, the BAFO evaluation report does not include any criticism of software tools under the technical organization factor.

We, nonetheless, agree with QSI that the software inflexibility issue should have been raised in discussions with the firm. As we indicated earlier, an agency is required to conduct meaningful discussions with those offerors within the competitive range. In order to fulfill this obligation, the agency generally must lead offerors into areas of their proposals which are deficient. See S.T. Research Corp., B-233115, supra. Here, both the initial and BAFO evaluation summaries include references to this matter. The Air Force says that software inflexibility related only to risk and did not affect QSI's color-coded technical rating. Nevertheless, the frequency with which this issue was mentioned in the initial and BAFO evaluations indicates that this was a proposal deficiency of some significance which should have been raised with QSI during discussions to give the firm an opportunity to improve its moderate risk rating under the technical support factor. It is clear from the evaluation reports and the source selection memorandum that both technical and risk factors were considered in making the selection.

Effect of deficiencies on selection

Although, we agree with QSI that the Air Force should have discussed with QSI the availability of QSI's key employee and the software inflexibility concerns, we nonetheless think that the evaluation record here shows that these errors did not affect the selection of Booz Allen's lower cost proposal. First, even if QSI was given full credit under the technical organization factor for the availability of the key employee on the transition, it is not likely that its color-coded rating of yellow (marginal) would have been raised to green (acceptable) since the evaluators found a number of other more significant deficiencies under that factor. For instance, aside from the availability of the key employee, the BAFO evaluation report criticized QSI's plan for the transition of certain functions from Washington, D.C. to Griffiss Air Force Base, N.Y. /1/ Thus, although the agency's error regarding the availability of the key employee may have contributed to QSI's marginal rating on the technical organization factor, there is no indication that but for this error QSI would have received a higher rating on this factor. Moreover, even if QSI's rating of yellow on the technical organization factor is changed to green, making the firm equal to Booz- Allen on this factor, the two proposals would still not be technically equal. Similarly, Booz Allen's "blue" rating on the technical support factor is higher than QSI's "green" rating on that factor. Although, QSI may have been able to increase its risk rating under the technical support factor if it had been informed during discussions of the evaluators' concern with software inflexibility, since this was not a concern under the color coded ratings, there is no reason to credit QSI with a higher color coded rating on the technical support factor. Rather, we believe that, given the opportunity to address the software inflexibility issue in its BAFO, at best, QSI could have improved its risk rating under the technical organization factor from "moderate" to "low." Thus, even if we gave QSI maximum credit for the two errors in the procurement relating to the key employee on the transition and software inflexibility, we believe that Booz-Allen would still have a higher overall technical rating because of its rating of blue (exceptional) with moderate risk compared to QSI's green (acceptable) even with its risk rating improved to low under the technical support factor.

More important, even if the QSI and Booz-Allen proposals were to be considered technically equal, the choice of Booz-Allen for the award was further supported by the fact that Booz-Allen proposed to perform the effort for approximately $1.3 million less than QSI. In this respect, consistent with the evaluation scheme set out in the RFP, the Air Force source selection decision based the award on the determination that Booz- Allen would provide the "best overall value to satisfy Air Force needs," indicating the significance of cost in the final selection decision. Thus, even if the elimination of these errors would have resulted in a technical rating equal to that of Booz Allen, QSI in our view still would not have received the award. See B.K. Dynamics, Inc., 67 Comp.Gen. 45 (1987), 87-2 CPD Para. 429; B.K. Dynamics, Inc.-- Reconsideration, 67 Comp.Gen. 264 (1988), 88-1 CPD Para. 165.

OTHER ALLEGATIONS

QSI makes numerous other allegations about the evaluation and the agency's conduct of discussions. For the reasons that follow, we disagree with these contentions. For instance, as we explained earlier, QSI argues that the Air Force failed to adequately inform the firm during discussions, of the evaluators' concern with the firm's transition plan. The protester says that during discussions, in response to agency concerns with the QSI transition plan for the configuration management information system (CMIS), QSI presented its transition plan and the evaluators indicated that the plan would be acceptable. QSI says that although it relied on the government's representations in formulating its BAFO, the agency determined that the transition plan was weak with respect to initial staffing and the firm's remote CMIS solution, the very areas which according to the protester, met with government approval during negotiations.

In response, the agency points out that during discussions, contracting officials asked QSI to "provide proposed concepts, staffing strategies and activities for the AFIA/AFSC transition." According to the agency, it told QSI during discussions that it was particularly interested in the firm's plan to transition the CMIS, which is a vital automated hardware/software support element of the configuration management task of the RFP. The agency says that during discussions, QSI described an interim configuration in which CMIS hardware continued to be located in the Washington, D.C. area with remote access provided by terminals in Rome, New York until the host system could be installed and made operational in Rome.

In an affidavit, the head of the technical evaluation panel says that during negotiations, contracting officials approved of QSI's approach as an interim measure only and did not indicate to QSI that the agency's concerns regarding the transition issue as a whole had been satisfied. According to the Air Force, it should have been clear to QSI that the firm would need to provide more information on transition in its BAFO.

The record indicates that QSI was asked during discussions for information on its proposed approach to the transition and the protester does not dispute that the transition issue was discussed. Although QSI says that during discussions the agency approved QSI's remote access transition approach, QSI does not dispute the assertion by the head of the agency's technical evaluation team that approval went only to the remote approach as an interim measure. Moreover, it should have been clear to QSI that since the agency wanted information on the transition, the firm would need to submit information on the complete transition from Washington, D.C. to Rome, New York and not just on a partial or interim solution. In this respect, an overall purpose of the contract is to consolidate intelligence data handling functions in Rome, New York and QSI, as the incumbent contractor supporting certain of these functions in the Washington, D.C. area, was well aware of the agency's transition needs. Under the circumstances, we believe that QSI was adequately apprised during discussions of the agency's concern regarding the transition. QSI also alleges that its proposal was downgraded because its employees did not have security clearances. The initial evaluation stated that QSI's "ability to provide and commit 25 full time and cleared personnel must be confirmed in BAFO." According to the protester, the solicitation only required that a limited number of key personnel be cleared or clearable upon award of the contract.

During discussions, QSI was asked to provide a staffing chart including names of proposed employees, clearance status and other information. According to the BAFO evaluation summary, the chart submitted by QSI in its BAFO included all information requested and satisfied the concerns of the evaluators. Thus, this issue had no effect on the final evaluation or on QSI's final rating.

QSI also argues that the evaluation panel failed to comply with Air Force Regulation 70-30, Streamlined Source Selection procedures, which, according to the protester, required the panel to establish evaluation standards in the source selection plan and apply those standards in the evaluation. According to QSI, the failure to use evaluation standards, which indicate the minimum performance or compliance levels which contractors are required to meet and against which proposals are to be evaluated, resulted in an unequal application of the evaluation criteria. The alleged deficiencies in the agency's application of the source selection plan do not themselves provide a basis for questioning the validity of the award selection. Source selection plans are internal agency instructions and as such do not give outside parties any rights. Burnside-Ott Aviation Training Center., Inc., et al., B-233113 et al., Feb. 15, 1989, 89-1 CPD Para. 158. /2/ Nonetheless, the agency is required to follow the evaluation scheme set forth in the RFP and to conduct its evaluation in a manner so as to reach a rational result. Id. Here, the agency followed the RFP evaluation scheme and we see no legal basis to question the evaluation based on these arguments. /3/

Finally, QSI's initial protest submission raised a number of other issues regarding the evaluation of proposals including whether it was given sufficient credit for its past performance, whether Booz-Allen had available sufficient personnel for the contract and whether Booz Allen had an office in Rome, New York prior to the award of the contract. The Air Force responded to these allegations in its report explaining that QSI's past performance and capability were evaluated as required by the RFP evaluation criteria, that Booz-Allen's proposal included commitment letters from all key employees and that Booz-Allen had an acceptable Rome office at a subcontractor's facility. QSI offered no further argument or evidence in support of these contentions and, thus, appears to have abandoned these issues. See The Big Picture Co., Inc., B-220859.2, Mar. 4, 1986, 86-1 CPD Para. 218. In any event, we have reviewed the evaluation record in the context of these contentions as well as the protester's general argument that Booz-Allen should have received less credit than QSI under the technical organization factor because Booz- Allen, as a new firm to the project, represented a greater risk than QSI since it did not offer a full complement of security cleared staff already "on board." We have no basis in this regard to question the evaluators' judgment in arriving at their rating of Booz-Allen.

The protest is denied.

/1/ QSI also argues that the procurement was deficient because the agency did not adequately apprise QSI of the evaluators' concern with its transition plan and, in fact, approved of that plan during oral negotiations. For reasons which we explain later, we do not agree with the protester and we conclude that the agency adequately discussed this issue and that the agency was justified in not improving QSI's BAFO rating because of concern with the firm's transition plan.

/2/ Further, the regulation cited by QSI concerns an internal Air Force procedure and compliance with such a regulation in the circumstances here is a matter of executive branch policy which we normally regard as an internal matter to be resolved by the Air Force rather than through the bid protest process. See True Mach. Co., B-215885, Jan. 4, 1985, 85-1 CPD Para. 18.

/3/ Although QSI also argues that the four evaluation assessment criteria listed in the RFP were not applied by the evaluation panel, both the initial and BAFO evaluation summaries include evaluation matrices which indicated that the assessment criteria were applied.

GAO Contacts