Skip to main content

4-D Neuroimaging, B-286155.2; B-286155.3, October 10, 2001

B-286155.2,B-286155.3 Oct 10, 2001
Jump To:
Skip to Highlights

Highlights

Technical evaluation identifying various features of awardee's proposal as advantages was unobjectionable where agency followed solicitation's evaluation criteria and record establishes that evaluated advantages have a reasonable basis. 2. Agency's waiver of testing requirement included in solicitation by mistake did not prejudice protester where cost of the testing is de minimis. Reducing protester's price by cost of the testing would not have affected the price-technical tradeoff. 3. Agency was not required to conduct discussions regarding what protester characterizes as perceived weaknesses in its proposal where proposal was rated acceptable under all factors and "weaknesses" were simply areas where awardee's higher-rated proposal had relative technical advantages. 4.

View Decision

4-D Neuroimaging, B-286155.2; B-286155.3, October 10, 2001

DIGEST

Attorneys

DECISION

4-D Neuroimaging protests the award of a contract to the Canadian Commercial Corporation (CCC), on behalf of CTF Systems, Inc., under request for proposals (RFP) No. SP0200-99-R-8031, issued by the Defense Logistics Agency (DLA) for a magnetoencephalography (MEG) and electroencephalography (EEG) whole-head scanner system, for use by the National Institute of Mental Health (NIMH). 4-D principally challenges the agency's determination that CTF's proposal was technically superior to its own, and that these technical advantages warranted paying CTF's higher price.

We deny the protest.

BACKGROUND

The MEG/EEG

The whole-head MEG/EEG system is intended for an advanced neuroimaging research facility at NIMH. The research involves detecting and recording signals of brain activity during cognitive performance in human subjects. Two key features are necessary in an instrument designed to detect this activity: sensitivity (the ability to detect very small signals in a background of noise) and spatial selectivity (the ability to determine where in the brain the signals are being generated, and to discriminate among different spatial patterns of brain activity). Agency Report (AR), Tab 3, attach. A, Para. 1. The two methods of recording this brain activity are the EEG (electrodes attached to the subject's scalp to read the brain's electrical signals) and MEG (a large array of superconducting sensor coils, cooled by liquid helium and arranged in a helmet that fits over the subject's head to detect the magnetic field generated by the brain's electrical activity). Id. Para. 2.

The sensors--which are either magnetometers or gradiometers /1/--gather analog signals (flux) from the brain, which are subsequently digitized to allow the computer to record and analyze the data. Id. Paras. 4, 9. "Primary" sensors are placed close to the scalp to detect the brain's flux. Because the primary sensors detect some background magnetic activity (noise), the MEG also employs "reference" sensors, which are placed far enough away from the head so as to record only background noise. As part of the noise cancellation system, the MEG subtracts the reference sensor information from the primary sensor information to arrive at a more accurate measurement of the signals. Because the brain's magnetic field is millions of times smaller than the earth's magnetic field and that of other environmental sources (e.g., passing vehicles, power lines, and nearby elevators), the MEG is usually housed in a magnetically shielded room (MSR), and employs other methods of noise cancellation to obtain more accurate data. Id. Para. 2.

The Solicitation

NIMH, through its intramural research program (IRP), plans and administers a comprehensive, long-term, multidisciplinary brain and behavioral research program dealing with the causes, diagnosis, treatment, and prevention of mental disorders, as well as the biological and the psychosocial factors to determine normal and pathological human behavior. The procurement here was designed to expand the IRP's neuroimaging research capability to include a "state of the art electrophysiological brain mapping system . . . capable of recording over the entire head the magnetic and electrical signals generated by brain electrical activity." RFP Sec. 1.5. The solicited system was to include an MEG/EEG, turnkey installation, training, data, and extended warranty, to be furnished on a fixed-price basis. As amended (in response to an earlier protest by 4-D /2/, the RFP advised offerors that the agency understood that the MEG/EEG scanner system is specialized commercial equipment modified (customized) to suit the intended use and physical configuration of the purchasing institution; that the MEG/EEG scanner systems offered should be "designed with the most up-to-date technology and built with the most up-to-date materials"; and that, because the agency intended to purchase "the most advanced, state-of-the-art MEG/EEG system," offerors should "use their best technological abilities in their design of their offered system," and should review their original proposals for the purpose of enhancing their designs. RFP, amend. 0006, Sec. 6(b), (c).

Proposals were to be evaluated on the basis of price and three technical factors listed in descending order of importance: technical approach; maintenance, product and customer support; and past performance. The technical approach factor included the following equally weighted subfactors: MEG/EEG system (overall functional performance, hardware design, and software design) and turnkey/extended installation. RFP at 69-70. These non-price evaluation factors were considered significantly more important than price, with price becoming more important as proposals became more equal technically. Award was to be made to the offeror whose proposal was most advantageous to the government based on the listed factors.

In response to the amended solicitation, CTF submitted its original proposal, and 4-D submitted four alternate proposals. The proposals were evaluated without reliance on the prior evaluations or source selection documents. Contracting Officer's Statement Para. 13. Three of 4-D's proposals were evaluated as acceptable and one was evaluated as unacceptable and eliminated from the competitive range. After the agency conducted discussions with both offerors, the final evaluation results were as follows:

Offeror (System) CCC/CTF 4-D (Magnes 4-D (Magnes 4-D (Magnes /3/ (Omega) WH WH WH

([deleted]) ([deleted])) ([deleted]))

1. Technical HA A A A Approach /4/

(a 1) Functional HA A A A Perf.

(a 2) Hardware HA (s) A (s,w) A (s,w,w) A (s,w,w) Design

(a 3) Software A A A A Design

(b) A A A A Turnkey/extended installation

2. Maint., A A A A Product, & Support

3. Past A A A A Performance

Overall Rating HA A A A

Price $2,745,000 $2,173,366 $2,173,366 $2,173,366

In making his award determination, the source selection authority (SSA) considered that the proposals were rated equal in the areas of software design, maintenance, product, and customer support, and past performance, Source Selection Decision (SSD) Paras. 18-24, but found that CTF's proposal was technically superior, specifically noting the following advantages to its hardware design as compared to 4-D's: greater number of sensors and reference channels; use of gradiometer-type sensors; higher front-end sampling rate; greater dynamic range; ability to recalculate raw data; data integration; and superior noise cancellation scheme. He concluded that CTF's proposal was "deservedly the most highly technically rated," SSD Para. 6, and further observed:

[4-D]'s Acceptable proposals are not so technically close to CTF's Highly Acceptable Omega system that the lower price would increase in importance relative to technical merit. The overall functional performance of the CTF Omega system is clearly the most suited for NIMH's research mission. The Omega system's superior technical proposal warrants the additional cost [26 percent higher] and therefore, I have determined that the award shall be made to the Canadian Commercial Corporation/CTF.

SSD Para. 6. DLA awarded the contract to CCC/CTF on June 20, 2001. After a debriefing, 4-D filed this protest challenging the technical evaluation, the conduct of discussions, and the price-technical tradeoff. After reviewing the agency report, 4-D also challenged CTF's compliance with an RFP requirement for testing of the MSR. /5/

ANALYSIS

Technical Features

4-D primarily objects to the agency's finding that CTF's proposal offered several features that were technically superior to 4-D's.

In reviewing a protest against a procuring agency's proposal evaluation, our role is limited to ensuring that the evaluation was reasonable and consistent with the terms of the solicitation and applicable statutes and regulations. National Toxicology Labs., Inc., B-281074.2, Jan. 11, 1999, 99-1 CPD Para. 5 at 3. The protester and agency have provided extensive competing technical arguments in support of their positions. Ultimately, these submissions and hearing testimony /6/ demonstrate that 4-D disagrees with the agency's judgment, but do not establish that it was unreasonable for the agency to find that certain features of the CTF system provide meaningful advantages over the 4-D systems. See Recon-Optical, Inc., B-286529, Jan. 18, 2001, 2001 CPD Para. 14 at 6. We discuss some of 4-D's principal arguments below.

In three areas--number of sensors, front-end sampling, and dynamic range--the protester (through the testimony and written submissions of its technical expert) does not disagree that the CTF system provided a technical advantage over 4-D's; rather, 4-D considers any advantage not worth the price premium paid by the agency, Hearing Transcript (Tr.) at 13-14, 78, asserting that both offerors' systems are "adequate" to perform as required. Id. at 78. However, this mere disagreement with the agency's tradeoff decision (discussed below) does not establish that the evaluation itself was unreasonable. In this regard, CTF's system offers 275 primary sensors, while 4-D's systems offer only 248, and the TEP chair testified that the additional sensors proposed by CTF represent an incremental advantage over 4-D's systems. Id. at 11-12. Contrary to the protester's speculation that CTF's additional sensors would be arrayed in places producing little usable data, the TEP chair testified that all proposed systems' sensor arrays cover approximately the same area; CTF simply packs more sensors into that area. Id. at 9-10, 12. Since the protester's technical expert conceded (prior to the hearing, where CTF's sensor array was discussed) that "[o]nly an increased density of the sensors would produce better scientific results . . . ," there is no basis to question the agency's conclusion that the extra sensors provide an advantage over 4-D's systems. 4-D Expert's Supplemental Declaration (Suppl. Decl.), Para. 10. Similarly, the hearing testimony clarified the validity of CTF's more advantageous front-end sampling rate. The TEP chair testified that CTF's use of digital signal processing, utilizing a digital flux loop, provided greater flexibility to the researcher. Tr. at 161-62. The protester's expert conceded that he understood why the agency considered this a strong point. Id. at 166-67.

As for dynamic range, the agency found that CTF's 32-bit dynamic range provided an advantage over 4-D's [deleted]bit range because it would more fully record the brain signal and provided a margin of error or safety to keep track of the signal during transition states and when transient, unexpected noise events occur. /7/ SSD, Para. 16; Tr. at 34, 36, 43, 87-88. As an example, the TEP chair noted that, in a hospital setting like NIMH, moving elevators, a passing metal gurney, or a passing oxygen bottle, would produce a significant magnetic signal that would make a particular portion of the data bad. Tr. at 74. The protester's expert conceded that data would be lost due to transient noise events (id. at 102-03), but asserted his belief that these events were rare and could be handled with fewer bits given the shielding available in the MSR and the inability of any system to record signals below the sensors' own noise level (4-D Expert's Suppl. Decl., Para. 13; Tr. at 35-36, 63-64). However, the TEP chair testified in response that "[i]t's not such a rare event . . . particularly if you're doing cognitive neuroscience experiments, which last for a long time in patients which are very difficult to get calmed down and get into the scanner and into the situation[;] you don't want a transient event to mess up your experiment." Tr. at 36. Given that the agency is in a better position to judge the rarity of transient noise events, and that most of the research to be done with the MEG involves very low frequency signals (Id. at 74-75, 126), which is where most of the noise problems are found (Tr. at 60), we are persuaded that even relatively rare events may interfere with the agency's research. We conclude that the agency reasonably determined that the margin of safety offered by CTF's larger dynamic range represented a distinct advantage over 4-D's systems.

4-D also challenges the agency's conclusion that two elements of CTF's systems' noise cancellation scheme--its greater number of reference channels, and its use of third order gradiometers--were superior to 4-D's system. /8/ Regarding reference channels, the agency found that CTF's 29 reference channels, as compared to 4-D's [deleted]channels /9/, were advantageous because they would provide a more accurate determination of the ambient environmental noise field, which would result in improved noise cancellation. TEP Chair Suppl. Decl., Para. 6; SSD Para. 13. At the hearing, CTF's expert testified that only [deleted]of its 29 reference channels were necessary for the noise cancellation system, and that the other [deleted]were for redundancy. Tr. at 120-21. This testimony led 4-D to assert that this means that the agency's evaluation was flawed because it should have compared this lower number ([deleted]) to 4-D's [deleted]channels. This argument is without merit. CTF's expert went on to testify that its system's [deleted]redundant channels were available for additional noise cancellation, id. at 121, and 4-D has not asserted or shown otherwise. Since CTF's additional channels thus are not merely redundant, we see nothing unreasonable in the agency's basing its comparison on all 29 channels. In any case, even if the agency had based the comparison only on CTF's [deleted]"necessary" channels, this still was [deleted]more reference channels than 4-D offered. Moreover, 4-D's president testified that some unidentified number of 4-D's [deleted]channels were for redundancy, id. at 142, so this difference would be greater. We conclude that the identified advantage was valid.

With regard to CTF's use of third order gradiometers and the agency's view that they were superior to 4-D's system (SSD Paras. 13, 17), 4-D asserts that this terminology is merely a "marketing gimmick" (Suppl. Protest at 3), /10/ and that the combination of the MSR and its own noise cancellation methodology, similar to CTF's, reaches the noise level of the sensors, and thus eliminates any advantage. 4-D Expert's Suppl. Decl., Para. 15.

We find no basis for disturbing the agency's evaluation conclusion. CTF's expert testified at length about the operation of the firm's proprietary system, and explained both its greater flexibility and its ability to lower the onset of low frequency noise. Tr. at 110-13, 118, 122-25. As explained by CTF's expert (undisputed by 4-D), lowering the onset of low frequency noise provides the researcher with a broader range where the noise is "really clean." Id. at 126. The TEP chair agreed that the ability of the third order gradiometer to reduce the onset of low frequency noise below 1 hertz (Hz) represented an advantage because the low frequencies are one of the major interest areas for NIMH research. Id. Even though it was not clear when the advantage would come into play, the TEP chair testified that CTF's third order gradiometers, when combined with the additional noise cancellation features of its MEG, provided a distinct "edge" and "incremental benefit" over 4-D's. Id. at 133, 151-52, 154.

As evidence of CTF's capability, its expert relied on actual performance data of its third order gradiometer as compared with a first order gradiometer--included in CTF's proposal--which showed that CTF's system effectively lowered the level at which low frequency noise is detected by the sensors (i.e., the onset of low frequency noise) to .5 Hz. Tr. at 109-10, 125; CTF Proposal, Vol. II, Figure B.11. While the protester at one point asserted that its system could reduce the onset of low frequency noise at the .1 Hz level (Tr. at 147), its president later retracted that assertion, admitting that he was "not sure" of 4-D's specification in this area (Tr. at 149). Indeed, though 4-D's president represented that 4-D's specification was contained in its proposal, he was unable to provide a proposal reference at the hearing, and has not subsequently identified it in 4-D's post-hearing comments. Likewise, 4-D's expert was unable to provide any definitive information demonstrating that 4-D's system would perform at the same level as CTF's. For example, while he testified that he knew that a 4-D system had reached the 1 Hz level, he could not recall if it had reached the .1 Hz level. Tr. at 133. In sum, while 4-D's expert disagreed that there was any edge to CTF's system, id. at 153-55, he was unable to provide any persuasive support for his view. We conclude that 4-D has failed to establish--beyond its unsupported opinion and disagreement with the agency's conclusion--that its noise cancellation approach performs at the same level as CTF's third order gradiometers. Accordingly, we have no basis to question the agency's evaluation conclusion that CTF's system provides a technical advantage in this area.

Software

4-D challenges the agency's conclusion that the software packages proposed by 4-D ([deleted]) and CTF ([deleted]) were both acceptable since, it claims, the [deleted]software package is "demonstrably superior" to CTF's. /11/ Suppl. Protest at 5. Specifically, 4-D's expert opines that 4-D's [deleted]package "has many more options, exhibits better graphics and is more compatible with other types of data," and that an unidentified purchaser of a CTF system purchased the [deleted]software at its own expense because of its superiority. 4-D Expert's Decl., Para. 13; 4-D Expert's Suppl. Decl., Para. 21.

This argument is without merit. The agency explains that the two competing software packages represented only one among a large complement of packages proposed by each offeror. Each of the two packages, viewed in isolation, has both strong and weak points compared to the other, but when each proposal's total complement of software was evaluated, the TEP judged that both offerors' software would work equally well to support the offerors' unique technological approaches. TEP Chair's Suppl. Decl., Para. 13. Apart from generally stating that the [deleted]software is superior to the [deleted]software, 4-D has provided no specific explanation of how its software is superior to CTF's software. Nor has it explained how the remaining packages in its software complement are superior to those in CTF's. We conclude that 4-D has not established that the evaluation was unreasonable.

Waiver

4-D asserts that CTF took exception to a material solicitation requirement concerning performance testing of the MSR. Specifically, CTF's proposal suggested an alternative based on whether its MEG met the proposed total system noise specification, and specifically stated that the cost of the specified testing was not included in its price. DLA essentially concedes that CTF took exception to the requirement, but explains that the requirement was included in the RFP by mistake, and that DLA never intended for the successful offeror to perform the MSR testing.

DLA clearly should have amended the RFP or otherwise apprised offerors that it had effectively waived this requirement. However, our Office will not sustain a protest unless the protester demonstrates a reasonable possibility that it was prejudiced by the agency's actions; that is that, but for the agency's actions, it would have had a substantial chance of receiving the award. McDonald-Bradley, B-270126, Feb. 8, 1996, 96-1 CPD Para. 54 at 3; see Statistica, Inc., v. Christopher, 102 F. 3d 1577, 1581 (Fed. Cir. 1996). Unfair competitive prejudice from a waiver or relaxation of the terms and conditions of the RFP for one offeror exists only where the protester would have been able to alter its proposal to its competitive advantage, were it given a similar opportunity. RGII Tech., Inc.--Recon. and Protest, B-278352.2, B-278352.3, Apr. 14, 1998, 98-1 CPD Para. 130 at 8. Here, there is no basis for finding that, had 4-D known of the waiver, it would have changed its proposal to improve its competitive position. DLA states that the cost of the testing, if required, would be between $5,000 and $11,000, and a document submitted by 4-D estimates the testing cost as $5,000 plus travel expenses. Protester's Suppl. Comments at 20; Letter from Testing Organization at 2. Since the agency already has determined that the technical advantages of CTF's proposal were worth its $2.74 million, a premium of 26 percent above 4-D's price, there is no reason to believe that providing 4-D an opportunity to reduce its price by this de minimis amount would have affected the award decision.

Discussions

4-D asserts that the agency failed to provide it with meaningful discussions with regard to three identified weaknesses in its proposed system: recovery of all raw sensor data; use of mixed [deleted]technology; and failure to integrate the MEG and EEG data. According to 4-D, had the agency advised it of these weaknesses, it could have substituted different sensors and clarified its proposals to eliminate the weaknesses. This argument is without merit. The points identified by 4-D were not problem areas of its proposal but, rather, are drawn from the TEP's and SSA's discussion of the advantages present in the CTF system as compared to the 4-D system. While agencies' discussions with offerors must be meaningful, agencies are not required to conduct discussions regarding relative disadvantages of a proposal that is otherwise acceptable. Avtec, Inc., B-238824, June 22, 1990, 90-1 CPD Para. 581 at 4.

Price-Technical Tradeoff

As noted above, much of 4-D's protest concerns its disagreement with the value of any technical advantages in CTF's proposal. 4-D asserts that the agency's price-technical tradeoff was flawed because the advantages identified by the agency were not sufficient to justify paying a 26-percent price premium.

Source selection officials have broad discretion in determining the manner and extent to which they will make use of the technical and cost evaluation results, and their judgments are governed only by the tests of rationality and consistency with the stated evaluation criteria. Chemical Demilitarization Assocs., B-277700, Nov. 13, 1997, 98-1 CPD Para. 171 at 6. Where, as here, the RFP allows for a price-technical tradeoff, the selection official retains discretion to select a higher-priced but also technically higher-rated submission, if doing so is in the government's best interest and is consistent with the solicitation's stated evaluation and source selection scheme. University of Kansas Med. Ctr., B-278400, Jan. 26, 1998, 98-1 CPD Para. 120 at 6.

Technical approach was the most important evaluation factor, and the non-price factors combined were significantly more important than price. RFP at 69-70. In making his source selection, the SSA prepared a detailed decision document comparing CTF's and 4-D's evaluated proposals and explaining the advantages he found in the CTF system; as discussed above, we find that the identified advantages are supported by the record. The SSA then specifically found that, "based on the technological superiority of the CTF system and the primary mission of NIMH, the overall cost difference is well worth the expense." SSD Para. 26. In other words, the SSA was aware of the technical advantages of CTF's proposal, and specifically determined that those advantages were worth CTF's higher cost. This is all that is required for a proper tradeoff, and the fact that the protester believes the price premium is too great is not sufficient to establish that the SSA's determination was unreasonable. General Servs. Eng'g, Inc., B-245458, Jan. 9, 1992, 92-1 CPD Para. 44 at 11 (award to offeror whose technically superior proposal was 125 percent higher in cost than lower technically rated proposal was proper).

The protest is denied.

Anthony H. Gamboa General Counsel

1. A magnetometer measures the absolute value of the magnetic flux at a sole detector coil. A gradiometer measures the change over space (gradient) of the magnetic flux over two coils. AR, Tab 3, attach. A, Paras. 4-5.

2. Award originally was made to CCC/CTF. 4-D protested, challenging the technical evaluation and price-technical tradeoff (B-286155). In response, DLA took corrective action by amending the RFP to clarify the relative weights of the evaluation criteria and to clarify its desire for the most advanced, state-of-the-art MEG/EEG system. We dismissed that protest as academic on September 25, 2000.

3. The 4-D systems are distinguished primarily by the type of sensors used. [deleted]

4. The adjectival ratings have been abbreviated as follows: Highly Acceptable (HA); Acceptable (A); strong point (s); and weak point (w).

5. After reviewing the agency report, 4-D withdrew earlier-raised protest grounds concerning CTF's compliance with Food and Drug Administration requirements and the past performance evaluation.

6. A hearing was convened as a technical roundtable to hear from the protester's expert, the chairman of the TEP, the president of 4-D, and an expert from CTF.

7. As explained at the hearing, the dynamic range is important because it represents the maximum signal the system can tolerate and the resolution of the signal. Tr. at 47.

8. Because the brain's magnetic signals are relatively weak, it is difficult to measure them amid the background magnetic "noise." Noise cancellation, on a very basic level, involves the MEG system's subtraction of reference sensor information (noise) from the primary sensor information (brain signal plus noise) to obtain a clearer record of the brain signal. Noise is canceled by a combination of system features, including reference channels and software.

9. In addition to the [deleted] channels, 4-D's system provided an additional [deleted] channels, but these [deleted] were only derived, rather than independent, channels. As conceded by 4-D, the [deleted] independent channels are the only ones relevant to the evaluation. 4-D's Post-Hearing Comments at 6.

10. CTF's third order gradiometer is a software-created, virtual gradiometer that takes the primary sensor signal, in conjunction with the reference channel, to compute a third spatial difference, resulting in additional noise cancellation. AR, Tab 3, attach. A, Para. 7. 4-D's assertion that this is merely a marketing gimmick appears to be based on its lack of familiarity with the proprietary details of the system; tellingly, 4-D's expert, who had access to CTF's proposal and heard all testimony at the hearing, did not dispute the existence of CTF's third order gradiometer.

11. 4-D also challenges the agency's evaluation of both offerors' proposed system maintenance and product/customer support as "acceptable," arguing that its maintenance and training were "palpably more extensive and superior." Suppl. Protest at 5. In response, DLA fully explained its rationale for determining that both offerors' proposals were adequate in this area. Because 4-D's comments did not contain any rebuttal to the agency's rationale, we consider this allegation abandoned. Analex Space Sys. Inc.; PAI Corp., B-259024, B-259024.2, Feb. 21, 1995, 95-1 CPD Para. 106 at 9.

GAO Contacts

Office of Public Affairs