Matter of: Lovelace Scientific Resources, Inc. File: B-256315 Date: June 9, 1994

Matter of: Lovelace Scientific Resources, Inc. File: B-256315 Date: June 9, 1994

DIGEST

Attorneys

DECISION

We deny the protest.

The solicitation sought proposals for the establishment of clinical centers to conduct multiyear studies of three major diseases--coronary heart disease, cancer, and osteoporosis--in post-menopausal women. The goal of the studies is to establish risk factors for the development of the conditions and to evaluate the effect of various interventions (adherence to a low-fat diet, hormone replacement therapy, and calcium and Vitamin D supplementation) in preventing these diseases. Each clinical center is to recruit approximately 1,400 women for the clinical trials and approximately 2,200 for an observational study. Award of approximately 29 contracts is anticipated.

The solicitation advised offerors that only technical proposals of high quality would be considered for award and that estimated cost would be used to distinguish among high quality proposals. The RFP also provided for consideration in the selection process of government goals for minority representation and for geographic balance in the study population. To ensure that minority women were adequately represented in the study, offerors were advised that proposals focusing on study populations comprising more that 60 percent minority women would be evaluated separately from proposals focusing on study populations with less than 60 percent minority representation, and that the number of awards from each group would be determined with the objective of achieving an overall study population in which minority women would be proportionately represented.

The RFP advised potential offerors that technical evaluation would be based on the following factors and subfactors:

Factor Weight

I. Technical Approach 30 points

a. Availability and accessibility of target population and proposed approach to recruitment

b. Feasibility and adequacy of plans to screen, enroll, and provide intervention to women in Clinical Trial and to collect data for women in both Clinical Trial and Observational Study. Adequacy of plans for referral for management of disease conditions discovered during study.

c. Feasibility and adequacy of plans to ensure protocol compliance, adherence to nutritional and drug interventions, and long-term participation of women

d. Management plan

e. Adequacy of protection of rights and privacy of participants

II. Qualifications and Availability of Personnel 30 points

a. Qualifications of principal investigator and key personnel to establish/operate a clinical center for a study involving women's health

b. Experience of proposed project team in planning/conducting large collaborative clinical trials

c. Expertise/experience of personnel responsible for recruitment, retention, follow-up of participants

d. Expertise/experience of personnel responsible for delivery of nutrition intervention program and dietary data collection

e. Expertise/experience of personnel responsible for counseling and monitoring women receiving hormone replacement therapy

f. Expertise/experience of personnel responsible for biomedical activities

III. Facilities and Equipment 20 points

IV. Organizational Experience and Performance 20 points

The solicitation provided that for purposes of initial technical evaluation, proposals might be divided into two groups (East and West). Twenty-five proposals were received from the West region. An additional 38 proposals were received from offerors in the East, resulting in a total pool of 63 proposals for 29 awards. Eight of the West region proposals, including Lovelace's, focused on study populations comprising more than 60 percent minority women, and were placed in Pool I.

The proposals were evaluated first by the special emphasis panel (SEP), a group of peer reviewers from outside the government, which analyzed the strengths and weaknesses of each proposal, assigned each a score,[1] and rated each as either acceptable or unacceptable. Eighteen of the 25 West region proposals, including 6 from Pool I, were found to be technically acceptable. Lovelace's proposal was included within this group although 5 of the 10 evaluators had rated it as unacceptable.

A second panel of in-house evaluators, the source evaluation group (SEG), then performed an additional review of the proposals that the SEP had rated as technically acceptable. The SEG noted its concurrence (or disagreement) with the findings of the SEP, identified additional strengths and weaknesses in the proposals, and recommended proposals for inclusion in or exclusion from the competitive range. The SEG recommended that Lovelace be excluded from the competitive range. The contracting officer concurred in this recommendation, and on January 14, 1994, determined that the proposal would not be considered further. Fourteen other West region proposals, including four from Pool I, were included within the competitive range.

Lovelace protests the exclusion of its proposal from the competitive range, arguing that the proposal could have been improved enough through discussions to stand a reasonable chance of being selected for award, especially given that 29 proposals were to be selected.

The evaluation of proposals and the resulting determination as to whether an offer is in the competitive range are matters within the discretion of the contracting activity, since it is responsible for defining its needs and the best method of accommodating them. LRL Sciences, Inc., B-251903, May 3, 1993, 93-1 CPD Para. 357. In reviewing protests against competitive range determinations, our Office will not reevaluate the proposals for the purpose of substituting our judgment for that of the agency; instead, we examine the agency's evaluation to ensure that it was reasonable and in accord with the evaluation criteria. Id. A protester's mere disagreement with the agency does not render the evaluation unreasonable. Madison Servs., Inc., B-236776, Nov. 17, 1989, 89-2 CPD Para. 475.

Based on the record here, we conclude that the agency's evaluation of Lovelace's proposal was reasonable and in accord with the evaluation criteria, and that the agency properly excluded the proposal from the competitive range.

The SEP evaluators assigned Lovelace's proposal an average score of 4 (of 10) under the evaluation criterion governing Technical Approach, an average score of 4.5 under the Qualifications and Availability of Personnel criterion, an average score of 5 for its Facilities and Equipment, and an average score of 5 for its Organizational Experience and Performance. Overall, the proposal received a score of 451 (of a possible 1,000). The evaluators viewed the diversity of Lovelace's target population (the population of the state of New Mexico), which includes substantial percentages of Hispanic, African-American, and Native American women, as a strength, but considered the relatively limited overall number of age-eligible women in the state as a weakness. The evaluators noted as additional weaknesses relating to the protester's proposed approach to recruitment its lack of experience with minority populations in New Mexico, its failure to discuss specific strategies for recruiting and retaining Hispanic and African-American women, and its failure to document that sufficient numbers of Native American women could be recruited. The evaluators noted as weaknesses relating to other aspects of Lovelace's technical approach its failure to set forth detailed procedures for enrolling, screening, and retaining participants; for administering the various interventions; for obtaining follow-up information; for ensuring compliance with the protocol; and for protecting the rights of the participants.

With regard to the qualifications of Lovelace's proposed personnel, the evaluators found its principal investigator had experience with both women's and minority health issues, but that she had no experience in conducting large-scale clinical trials. Similarly, the experience of the personnel responsible for recruitment, retention, and follow-up was limited to small-scale clinical trials. Further, Lovelace's proposed personnel had no experience in conducting dietary trials and interventions, in furnishing nutritional counseling, in conducting hormonal replacement studies, or in conducting studies focusing on women.

With regard to the remaining two evaluation criteria, the evaluators found that the offeror's Albuquerque facilities were adequate, but that it had not furnished sufficient detail concerning outreach locations in the Native American community, and that the offeror had limited experience in recruiting and retaining women for large-scale prevention projects.

The protester takes issue with a number of the evaluators' findings. Lovelace complains first that the evaluators found the New Mexico population to be marginally sufficient to meet the RFP requirements for participation, but that they included the proposal of the University of New Mexico, which targeted the same population, in the competitive range.

Lovelace's proposal was not excluded from the competitive range simply because the evaluators viewed its target population as marginally sufficient; the limited number of women in the target population was merely one of many weaknesses, which, when considered together, convinced the evaluators that the proposal did not stand a reasonable chance of being selected for award. Further, the evaluators did not view the target population as necessarily insufficient; they simply noted that due to the relatively small size of the recruitment pool, particularly effective recruitment strategies would be required to ensure an adequate number of participants, and that Lovelace had not articulated such strategies in its proposal. With regard to the protester's argument that the limited number of age-eligible women in New Mexico should also have been viewed as a weakness in the University of New Mexico's proposal, our review of the evaluation record reveals that it was in fact so viewed. The University of New Mexico's proposal was stronger than Lovelace's in other areas, however, and thus was included within the competitive range.

The protester argues next that the evaluators should not have found that it lacked relevant experience or questioned its ability to recruit an adequate number of participants given its previous experience with two large-scale studies, one involving 5,402 Vietnam veterans and the other involving 668 factory workers, and, in addition, its extensive experience in recruiting patients into projects sponsored by the industrial clinical research community. Lovelace also contends that it was unfair for the agency to cite as a weakness in its proposal its lack of experience in recruiting and retaining women for large-scale prevention projects given that few large-scale studies have previously been conducted on women and thus few offerors can claim such experience.

The evaluators recognized that Lovelace had participated in two large-scale studies; neither was a clinical trial involving interventions, however. In addition, neither study focused on women or minority group members; thus, neither demonstrated that the protester had established access to these groups. The evaluators also recognized that Lovelace had been involved in a large number of clinical trials for pharmaceutical companies, but these trials involved only a fraction of the number of participants sought for the Women's Health Initiative trials.[2] Given the protester's failure to demonstrate in its proposal that it had previously successfully recruited large numbers of women or minority group members for clinical trials or to demonstrate that it had established avenues of access to these populations, we think that it was clearly reasonable for the evaluators to view Lovelace as lacking in relevant experience and to question its capability to recruit the number of subjects required for the studies. With regard to the protester's assertion that few other offerors could possess previous experience in conducting large-scale trials on women, we fail to see why the fact that other offerors may also have lacked highly similar experience should have precluded the agency from viewing Lovelace's lack thereof as a weakness.

The protester also takes issue with the evaluators' finding that it failed to furnish sufficient detail regarding its plans for administering the nutritional and hormone replacement therapies. Lovelace contends that it would have been inconsistent with good clinical trial operating procedures for it to have developed full plans for implementation of these interventions without permitting its site personnel to review the protocol.

The RFP advised offerors that among the subfactors to be considered in the evaluation of technical proposals were the feasibility and adequacy of the offerors' plans to provide, and to ensure adherence to, the interventions. Thus, offerors were clearly on notice that they should explain in detail their proposed approach to administering the interventions. Moreover, since the protocol was included in the RFP, we see no reason why Lovelace's site personnel could not have reviewed it and provided feedback to the proposal drafters at the time Lovelace was formulating its proposal.

The protester also objects to a number of the evaluators' findings regarding its proposed personnel. Lovelace complains that its principal investigator was criticized as lacking experience with large-scale clinical trials of women, but that many of the principal investigators proposed by other offerors must also have lacked such experience.

This is the same complaint that the protester raised with regard to its lack of organizational experience in conducting large-scale clinical trials on women and our response is the same: the fact that other offerors may have shared the same weakness does not preclude the agency from viewing it as a weakness.

The protester complains about the evaluators' findings regarding its proposed personnel's lack of experience in conducting nutrition and hormone replacement therapy interventions. Lovelace does not assert that its proposed personnel had such experience; it simply states that the experience its personnel do have is adequate to meet the RFP's minimum requirements. Such an assertion is insufficient to convince us that the evaluators' findings regarding the protester's lack of experience in these areas were unreasonable.

Lovelace argues that the agency unfairly downgraded its proposal on the grounds that its clinical dietician did not have a Ph.D. degree when the RFP did not require such a credential. The protester misunderstands the evaluators' criticism of its proposal under this subfactor. The evaluators did not find that the protester's dietician lacked the requisite academic credentials; they found that she did not have a great deal of experience subsequent to receiving her degree.

In our view, Lovelace has not demonstrated that any of the evaluators' findings regarding its technical approach or qualifications was without foundation. Nor, given the fundamental nature of a number of the weaknesses--i.e., the marginal size of the target population pool, the protester's lack of experience in recruiting members of the targeted minority groups into clinical trials, and the lack of experience of both the protester and its principal investigator in conducting both large-scale clinical trials and large-scale studies focusing on women--do we think that it was unreasonable for the contracting officer to conclude that the proposal could not be improved enough through discussions to stand a reasonable chance of being selected for award. See American Sys. Corp., B-247923.3, Sept. 8, 1992, 92-2 CDP Para. 158.

In addition to protesting the evaluation of its proposal, Lovelace complains that the agency applied grant review, rather than competitive procurement, procedures in evaluating its proposal. In this regard, the protester alleges that the evaluators unfairly considered their own personal knowledge (or lack of knowledge) of a particular principal investigator's qualifications, rather than the qualifications of the organization as presented in the proposal, in evaluating the proposals.

The record does not substantiate the protester's allegations. Although proposals were evaluated by two separate panels of reviewers, we are aware of no restriction which would preclude such a two-tiered review. It was in no way improper for the evaluators to consider the qualifications and experience of Lovelace's principal investigator in evaluating its proposal, since the RFP clearly provided for consideration of these matters. Furthermore, there is no evidence in the record that the reviewers considered anything other than the information presented in the proposals in evaluating the principal investigator's qualifications.

The protester also argues that the evaluators were biased against it as a for-profit entity, and in favor of universities and medical schools. Again, we find no substantiation in the record for Lovelace's allegation. An independent research institute, rather than a university or medical school, in fact received the highest West region technical score.

The protest is denied.

1. Proposals were rated on a scale of 1 through 10 under each evaluation factor; each rating was then multiplied by the particular factor's weight and the resulting products totaled.

2. The recruitment goal for the largest of the studies summarized by Lovelace in its proposal was 120.