Matter of: Bristol-Myers Squibb Company File: B-275277 Date: February 5, 1997

Matter of: Bristol-Myers Squibb Company File: B-275277 Date: February 5, 1997

Price evaluation methodology, as reflected in the estimates set forth in a solicitation for drugs used in the treatment of hypercholesterolemia, is not objectionable where the protester fails to show that the methodology and estimates, which were in part the result of a statistical analysis, are not reasonably accurate, or are less accurate than the estimates would have been if they had been determined solely from the review of the agency's historical data. Protest of the relative importance of certain evaluation factors set forth in a solicitation for drugs used in the treatment of hypercholesterolemia is dismissed where the relative importance of the particular evaluation factors is a reflection of the agency's medical policy.

Attorneys

DECISION

Bristol-Myers Squibb Company protests the terms of request for proposals (RFP) No. M5-Q6-96, issued by the Department of Veterans Affairs (VA), for hydroxymethylglutaryl-coenzyme A (HMG-CoA) reductase inhibitors. [1] Bristol-Myers protests the RFP's price evaluation methodology as reflected in the estimates for the HMG-CoA reductase inhibitors and the relative importance of the RFP's evaluation factors.

We deny the protest.

This procurement is a part of VA's effort to standardize medical and pharmaceutical items to achieve concentrated buying power. To assist in this effort, and to better manage its procurement of $1 billion of pharmaceuticals per year, the VA created a Pharmacy Benefits Management (PBM) section, which is staffed by Doctors of Pharmacy. The PBM is tasked with developing "evidence-based pharmacologist management guidelines for improving quality and providing best-value patient care." The PBM section is guided in part by a Medical Advisory Panel (MAP), which is comprised of nine practicing VA physicians. The pharmacologic treatment guidelines developed by the MAP provide guidance for standardized research-based care at all VA medical centers.

There are currently four HMG-CoA reductase inhibitors (also known as "statins") available in the United States for prescription. They are pravastatin, produced by Bristol-Myers; lovastatin and simvastatin, produced by Merck & Company; and fluvastatin, produced by Sandoz. Because of their differing chemical compositions, the four statins differ as to efficacy in lowering low-density-lipoprotein (LDL) cholesterol. The VA currently purchases varying quantities of each of the four statins for prescription by VA physicians to their patients.

The MAP determined, based upon its review of the relevant medical literature and the needs of VA physicians, that two statins should be procured for use at VA medical centers, with one serving as "a first-line statin . . . designated as the preferred formulary agent," and a second statin as an alternate "for those [patients] who do not tolerate the preferred agent." The MAP contemplates that once the primary and secondary statins are selected under this RFP, the agency will issue guidelines for converting all VA patients currently prescribed a statin (other than the statin selected as the primary statin) to the primary statin, and, for those patients who cannot be prescribed the primary statin, to the secondary statin. For example, if a patient is currently prescribed a particular dosage of simvastatin, and pravastatin is selected as the primary drug for the treatment of hypercholesterolemia, that patient would be converted to pravastatin in accordance with MAP guidelines.

The RFP provides for the award of two fixed-price requirements contracts, for a base period of 1 year with four 1-year options, for the supply of the two statins. Consistent with the MAP's recommendations, one award will be made for the supply of one statin as the primary drug to serve an estimated 60 to 80 percent of VA's 80,000 patients currently being treated with statins, and another award will be made for the supply of a secondary drug for treatment of the remaining 20 to 40 percent of patients who cannot use the primary drug.

The RFP provides that offerors may submit offers to supply any of the four statins, and that award will be made to the offeror(s) whose proposal(s) represent(s) the best overall value to the government. The RFP states that technical merit is more important than price, and sets forth the following technical evaluation factors: Safety, Efficacy, Outcome, Compliance, VA Patients, and Pharmacy. The RFP states that the safety and efficacy evaluation factors are of equal importance, and each is more important than the outcome factor, and that the compliance and VA patients factors are of equal importance, and each is less important than the outcome but more important than the pharmacy factor. The RFP provides detailed explanations for each of the evaluation factors; for example, the RFP explains that under the efficacy factor the agency will consider "the efficacy or reduction in LDL cholesterol levels which use of any drug yields," and under the outcome factor the agency will consider such matters as positive "reduction[s] in cardiovascular event rate[s] or mortalit[ies]" from the use of the drug.

The RFP requires the completion of a schedule of supplies and prices which differs depending on which of the four statins is offered. For example, for pravastatin, per dose and extended prices are requested for an estimated 11 million 10 milligram (mg.) tablets, 21 million 20 mg. tablets, and 18 million 40 mg. tablets; for simvastatin, prices are requested for an estimated 30 million 5 mg. tablets, 16 million 10 mg. tablets, 3 million 20 mg. tablets, and 1 million 40 mg. tablets. [2] The total evaluated price for each offer is the total of the extended prices.

The RFP explains that the estimated quantities set forth in the schedule are "based on a scenario of converting all veteran patients currently taking [an] HMG-CoA [reductase inhibitor] to one of the competing agents." For example, the estimated quantities set forth in the RFP for pravastatin or simvastatin are based upon VA's estimated requirements should it prescribe either pravastatin or simvastatin to all of its patients currently being treated with any of the four statins. The process through which the agency determined its estimated quantities is detailed in the RFP. With regard to the estimated quantities set forth in the RFP, the MAP found that it could not project, based upon its past prescribing patterns for the four statins, its future requirements with any degree of reliability for the environment contemplated by this RFP--where on a national basis one statin will serve as the primary drug for treatment of patients with hypercholesterolemia, and another statin will serve as the secondary drug. The VA explains that currently there is no standardized national formulary for the supply of statins, nor standardized protocol for the treatment of hypercholesterolemia with statins, and that because of this, its medical centers have different formularies and protocols for how statins are to be administered to VA patients.

For example, the VA states that many of its medical centers use fluvastatin as the primary drug for treatment of hypercholesterolemia because it is currently the least expensive of the statins, and if a patient's cholesterol is not controlled by fluvastatin, the patient is administered the more potent lovastatin or simvastatin. The agency adds here that because simvastatin has been prescribed by many of its medical centers primarily when treatment with a less potent statin has failed to produce the desired result of lowering LDL cholesterol, the prescribing patterns are skewed towards higher dosages of simvastatin. To illustrate this example, the chairman of the MAP points to the protocol of the VA's West Los Angeles medical center, which, depending on the desired cholesterol level, first treats patients with 10 mg. or 20 mg. doses of pravastatin. If the desired level is not achieved, the patient is prescribed 40 mg. doses of pravastatin, and if that treatment fails to produce the desired level, the patient is prescribed 20 mg. doses of the more potent simvastatin, or 40 mg. doses of simvastatin, depending on the success of the 20 mg. simvastatin treatment. The result of this protocol, among other things, is that 5 mg. doses of simvastatin are "very rarely prescribed at [the] West Los Angeles" medical center. The agency adds that each medical center's prescribing practices and the resulting prescribing patterns are also influenced by, among other things, the effectiveness of the various pharmaceutical manufacturers in marketing their products.

The MAP thus determined that it could best estimate its needs in an environment where one statin would be procured as the primary drug for treatment of hypercholesterolemia nationwide, and another statin as the secondary drug for such treatment, by considering, in conjunction with its current prescribing patterns, the relative potency of each of the statins. With the assistance of a consultant, VA determined the potency of each of the four statins in the various dose amounts, in part through the performance of a statistical analysis of certain data as to the statins' relative efficaciousness. [3] In performing its statistical analysis, the consultant analyzed 11 clinical studies provided by the agency, [4] setting forth a total of 47 dose-effect data points. [5] From these 47 dose-effect data points, the consultant calculated three dose-effect data points each for fluvastatin and lovastatin, five dose-effect data points for pravastatin, and six dose-effect data points for simvastatin. The consultant plotted these dose-effect data points on a graph where one axis represented the dose in mg. of statins and the other axis represented the effect of the dose in lowering LDL cholesterol by percentage. The consultant then fit, with computer assistance, four different model curves for each statin, and determined for each statin which curve best fit the plotted dose-effect data points, considering the relative weight that each dose-effect data point should have on the analysis based upon the calculated precision of the data from which each dose-effect data point was derived. From the selected logarithmic curves, dose-effect relationships were established for each of the statins. For example, while there were differences between the curves depending on the dosages, simvastatin and lovastatin were determined to be, on average, 3 times and 1.32 times more potent, respectively, than pravastatin.

This information was then converted by the agency, using a mathematical formula, to a comparative analysis of the commercially available doses for each of the statins with the presumption that all patients will be taking only one statin. For example, the agency determined, based upon the foregoing statistical analysis of the clinical trials, that 6 mg. of simvastatin would have the same efficacy in lowering a patient's LDL cholesterol as 20 mg. of pravastatin. Because simvastatin is only commercially available in 5 mg., 10 mg., 20 mg., and 40 mg. doses (not 6 mg. doses), the agency, for price evaluation purposes, calculated that 80 percent of the patients currently taking 20 mg. of pravastatin would require 5 mg. of simvastatin, and 20 percent would require 10 mg. of simvastatin. The agency then determined its requirements for each of the statins in the differing dose amounts, based upon the estimated dose equivalency ratios and the actual doses of the four statins dispensed by the VA to its patients in the first 6 months of 1996. These estimates were included in the RFP schedule for pricing purposes.

Bristol-Myers protests that the RFP's price evaluation scheme, as reflected in the RFP's estimated quantities, is flawed to the prejudice of its product, pravastatin. Bristol Myers attacks in some detail the statistical analysis performed by the agency to determine dose-effect relationships for each of the four statins, arguing that it "is so fundamentally flawed, both in concept and in execution, that it should not be used for any purpose." The protester insists that because of this, the RFP's price evaluation methodology, and the resultant estimated quantities for each of the statins, must "be based on actual prescribing patterns, adjusted only on the basis of concrete empirical data on any likely changes in those patterns during the term of the contract." [6]

Where estimates are provided in a solicitation, there is no requirement that they be absolutely correct; rather, they must be based on the best information available and present a reasonably accurate representation of the agency's anticipated needs. Lederle-Praxis Biologicals Div., Am. Cyanamid Corp., B-257104 et al., Aug. 22, 1994, 94-2 CPD Para. 205. In this case, what we must therefore determine is whether the protester has shown that the agency's statistical analysis was flawed and that the data derived from the analysis is thus not reasonably accurate or is inaccurate to such an extent that the consideration of only VA's past prescribing patterns would have yielded more accurate estimates of the agency's needs.

The protester first points out that the analysis does not indicate "the selection criteria that were used to identify the studies on which the analysis was based," and concludes that from its review of the studies used that "the VA's selection criteria were invalid." In this regard, the protester questions why the VA considered in its analysis only "multi- center" studies (studies where the same study protocol is followed at more than one research center), but not "single center" studies.

While it is not disputed that the selection criteria by which various clinical studies were included or excluded in a statistical analysis is important, the protester has not identified any specific studies that were wrongfully included or not considered in the statistical analysis. More importantly, the protester has not stated how or even if the data derived from the analysis would have been materially different if some of the studies used had been excluded and others included. Thus, the agency's failure to specify its selection criteria does not provide a basis for our Office to find the analysis unreasonable.

The protester also contends that the consultant's statistical analysis is "useless" because it does not differentiate between comparative, within- study results, and non-comparative, across-study results. That is, the consultant considered in his statistical analysis dose-effect data points for pravastatin and simvastatin which were derived from studies of the efficacy of pravastatin and simvastatin on a comparative basis as well as from studies which did not involve the comparison of the efficacy of the same two statins (e.g., a study of the efficacy of simvastatin and lovastatin). Although we agree that it is generally accepted that, in the performance of an analysis like that done here, data derived from "within- study" comparisons of the relative efficacy of statins is preferable to that derived from "across-study" comparisons, the protester fails to demonstrate that because of this aspect of the consultant's methodology, the results of the analysis were not reasonably accurate. Instead, the protester merely asserts that the methodology is inconsistent with VA's statement that comparative trials between statins should be the basis for the estimates included in the RFP, and that the methodology employed by the consultant renders the analysis "statistically invalid."

The protester disagrees with a number of other aspects of the statistical analysis (e.g., the derivation of the dose-effect curves, and the inversion of the dose-effect curves to obtain dosage equivalents), and points out that the analysis itself cautions that "no error analysis" has been done on the results and that estimating dose-effect relationships from a curve derived from only three known dose-effect data points--as was the case with both fluvastatin and lovastatin--can be unreliable. Again, while the statistical analysis sets forth a number of cautionary statements with regard to its results, the protester simply has not shown why these statements render the analysis "useless" or that the results of the analysis are not reasonably accurate. Simply put, the protester, while pointing out that the conduct of the analysis was not an ideal "textbook" analysis, does not at any time provide any data indicating that had the analysis been performed in any other manner it would have yielded different results. [7]

The protester next argues that regardless of the methodology followed by the agency in performing the analysis, the results of the analysis must be in error because a number of published articles provide that simvastatin and lovastatin are, at most, two times and equally as potent, respectively, in lowering LDL cholesterol as pravastatin. The agency responds that the articles cited by the protester do not demonstrate that the results of its statistical analysis are invalid and cites other articles that it asserts validates its analysis. The agency points out, for example, that one of the articles cited by the protester does not describe the methodology or studies on which the article's conclusion, that 15 mg. of simvastatin is as potent in lowering LDL cholesterol as 20 mg. of pravastatin, is based. The agency adds that this article was published in December 1993, and a number of clinical studies completed since then establish that simvastatin is considerably more potent than pravastatin in lowering LDL cholesterol than the articles reported 1.33 to 1 ratio. Merck notes that some of the statements from the articles cited by the protester are out of context or are not based on valid clinical studies, and that none of these reports demonstrates that the agency's analysis is in error. In our view, the articles referenced by the protester, agency, and intervenor evidence that there is considerable disagreement in the medical community as to the relative potency of the various statins, and, based on our review, we cannot conclude that these articles either invalidate the results of the agency's statistical analysis or validate the protester's view that simvastatin and lovastatin are at most two times as potent and equally potent, respectively, as pravastatin. [8]

The protester also argues that the estimates set forth in the RFP, and thus the results of the VA's statistical analysis, must be invalid because the estimates differ (in some instances greatly) from the VA's past prescribing patterns. For example, the protester points out that 60 percent of the RFP's estimated requirements for simvastatin are in the amount of 5 mg. doses, yet current data show that of all simvastatin doses dispensed by VA physicians, only 10 percent are in the amount of 5 mg. The protester adds that on average, the RFP's estimates provide for an average simvastatin dose of 8.2 mg., in contrast with the average dose indicated by VA's past prescribing data of 16.2 mg., and an average dose of 25 mg. of pravastatin, as opposed to the past average pravastatin dose of 19.9 mg.

We think the differences in the average doses are to be expected, given that simvastatin is, according to the data and analysis provided by the agency and the protester, more potent than pravastatin, and given the agency's explanation of the protocols in prescribing statins followed by VA's various medical centers, which was affected by considerations other than the relative potency of the statins (e.g., cost, local marketing, or VA doctor or center preferences). For example, the RFP contemplates, and the estimated schedule quantities are based upon, the presumption that if simvastatin is selected as the first line statin, the VA physicians, who are currently prescribing the less potent fluvastatin, lovastatin, or pravastatin to their patients, will convert those patients to the more potent simvastatin, rather than converting only those patients to simvastatin when the other statins fail to sufficiently lower the patients' LDL cholesterol as is reportedly being done under existing protocols. The logical result of this practice would be to increase the number of doses of simvastatin in the lower dose amounts of 5 and 10 mg., and accordingly, the average dose of simvastatin prescribed by VA would be lower than that evident from past prescribing patterns.

With regard to the protester's position that the RFP's estimates should be based solely upon the agency's historical data regarding its use of the statins, we agree with the agency that basing the estimates solely on such data would be inappropriate given the circumstances here. We simply fail to see how historical data derived from VA's past prescribing patterns-- given the VA's current policy of purchasing various quantities of each of all four of the statins and the differing treatment protocols of the varying medical centers--would, considered alone, result in accurate estimates of the agency's requirements in the environment contemplated by this RFP where there will be a standardized treatment protocol and only two statins available, with one designated as the primary drug for prescription. In short, we agree with the agency that the differing potencies of the statins is an appropriate consideration in determining its estimated requirements.

Although the protester obviously disagrees with the agency's position as to the appropriate price evaluation and estimating methodology for this procurement, it has not shown the agency's determination in this regard to be unreasonable. Specifically, the protester has not demonstrated that the statistical analysis did not yield reasonably accurate results or that a reliance on historical data alone would have produced more accurate estimates under the circumstances of this procurement. As such, we cannot find that the resulting estimates in the RFP schedule were not based on the best information available or were not a reasonably accurate representation of the agency's anticipated needs. [9] Id.

Bristol-Myers also protests the relative importance of the efficacy and outcome evaluation factors, arguing that a statin's ability to produce the desired outcomes of a reduction in cardiovascular event or mortality should be considered more important than a statin's efficacy in lowering LDL cholesterol levels, particularly since potency is being considered as part of the price evaluation. Bristol-Myers argues in the alternative that the consideration of a statin's ability to produce positive outcomes should be considered under the efficacy evaluation factor, and not as a separate, less important, evaluation factor.

The agency explains that the MAP believes as a matter of medical policy that the safety associated with the use of the particular statins, and their relative efficacy in reducing LDL cholesterol levels, are more important in the treatment of hypercholesterolemia than the statins' abilities to produce certain other desired outcomes. The agency's determination here is a reflection of its medical policies and judgments, which we will not consider under our bid protest function. IVAC Corp., 67 Comp.Gen. 531 (1988), 88-2 CPD Para. 75; Travenol Labs., Inc., B-215739; B-216961, Jan. 29, 1985, 85-1 CPD Para. 114.

The protest is denied.

Comptroller General of the United States

1. HMG-CoA reductase inhibitors are used in the treatment of hypercholesterolemia (i.e., high cholesterol).

2. Prices are also solicited for various size packages of each of these statins.

3. The RFP schedule, as initially issued, was not based on this analysis. It was developed after a Bristol-Myers's agency-level protest.

4. The consultant initially considered 12 studies, but rejected one of the studies for inclusion in the statistical analysis because it lacked sufficient data to assess the relative precision of the study's results.

5. Each dose-effect data point represented the average percentage reduction in LDL cholesterol in patients who were administered a particular dosage of a particular statin.

6. It is unclear from the protester's submissions as to what "concrete empirical data on any likely changes" in the VA's prescribing patterns that it believes exists.

7. The parties have submitted various comments on whether the analysis performed by the consultant was a proper or formal "meta-analysis." Since the protester has not shown that the analysis actually performed did not yield reasonably accurate results, there is no need to determine whether it was a properly conducted or formal "meta-analysis."

8. While Bristol-Myers asserts that the principal article cited by the agency should be discounted because of its authors' ties with Merck, the protester has not shown that this article is either biased or clearly erroneous.

9. While the protester correctly notes that the RFP estimates are based upon 100-percent usage of the first line statin, even though the RFP contemplates only 60- to 80-percent usage and that a second line statin would also be selected, there is no suggestion that the relative percentages of the designated estimates for each dosage of each statin would be different; thus, this provides no basis to object to the evaluation scheme.