RTI International

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: RTI International

File: B-411268

Date: June 26, 2015

DIGEST

1. Protest challenging the agency’s technical evaluation on the basis that the sample filters furnished by the agency for performance testing were defective is denied where the government-furnished sample filters were appropriate for the performance testing anticipated by the solicitation.

2. Protest asserting that the agency relied upon an unstated evaluation criterion by failing to disclose material information regarding the archiving and storage of government-furnished sample filters is denied where the agency’s process was reasonable and in accordance with the terms of the solicitation.

RTI International, of Research Triangle Park, North Carolina, protests the elimination of its proposal from the competitive range under request for proposals (RFP) No. SOL-NC-13-00020, which was issued by the Environmental Protection Agency (EPA), for filter handling and analytical support services in support of air quality monitoring. The protester’s proposal was disqualified for failing to achieve a passing score on a pre-award performance testing analysis. RTI argues that the agency’s technical evaluation was flawed because EPA unreasonably provided offerors with unreliable sample filters for performance testing, or, alternatively, failed to apprise offerors of material information regarding the archiving and storage of the sample filters.

We deny the protest.

BACKGROUND

The RFP, which was issued on September 15, 2014, sought proposals for analytical services in support of EPA’s obligations to monitor the National Ambient Air Quality Standards (NAAQS) as mandated by the Clean Air Act. Agency Report (AR) at 1. The RFP anticipated the award of a fixed‑price requirements type contract, with a 5‑year ordering period, for filter handling and analytical support services in connection with EPA’s Chemical Speciation Network (CSN).[1] RFP at B-1, F-1.[2] EPA established the CSN to conduct routine speciation monitoring of certain fine air particulates following the agency’s PM2.5 NAAQS review, which was completed in 1997. RFP, Statement of Work, at 1.[3] The CSN is comprised of 50 Speciation Trends Network sites and approximately 150 State and Local Air Monitoring Station supplemental sites located throughout the United States. Id. Aerosol samples are collected on filters at these sites and are analyzed for PM2.5 mass, a number of trace elements, major ions (e.g., sulfate, nitrate, and ammonium), and organic and elemental carbon. Id. States use the monitoring data analyzed by the CSN support contractor to determine if they are in attainment or nonattainment of mandated EPA air quality parameters. Contracting Officer’s Statement of Facts (COSF) (Apr. 22, 2015), at 2.

For purposes of award, the RFP provided that EPA would evaluate proposals under the following four factors: (1) technical approach; (2) past performance; (3) quality assurance; and (4) small-disadvantaged business participation. RFP at M-1.[4] Offerors determined to be in the competitive range were to participate in performance testing (PT) on a pass/fail basis. Id. at L-13, M-2. Specifically, the RFP stated that:

As part of the proposal evaluation, each bidder . . . that meets the minimum technical requirements will be given the opportunity to analyze and report results for a set of performance testing (PT) samples. The PT samples will be provided by the EPA which provides Quality Assurance (QA) oversight for the [CSN] analytical support laboratories. . . . The pre-award PT results will be evaluated on a pass/fail basis by comparing the reported results to EPA results as stated below. . . . To pass this portion of the evaluation the offeror shall successfully determine the mass of a specified analyte. . . . The acceptance criteria have been determined using the past eight years of performance evaluation results from the four to five laboratories that routinely provide analytical services for the CSN. The acceptance criteria have been established as 3-sigma confidence limits after examining the difference in results reported by the test labs and EPA’s reference lab, as shown in the tables below.

 

If offeror fails any of the specified acceptance criteria, this will result in a total failure of this evaluation.

Id. at M-2 (emphasis in original).

In response to RTI’s protest, EPA submitted a declaration from a physical scientist employed by the EPA’s Office of Radiation and Indoor Air regarding the agency’s process for preparing filter samples for PT. The first step of the process, as explained by the agency’s physical scientist, is that the EPA’s National Analytical Radiation Environmental Laboratory (NAREL) creates sample filters by using multiple collocated speciation air samplers, which are programmed to start and conclude sampling at the same date and time so that each sample filter collects particulate matter from the ambient air at the same rate. AR, Tab 15, Decl. of EPA Physical Scientist (Apr. 21, 2015) ¶ 3. NAREL has the capacity to create up to 16 sample filter replicates during each sampling event. Id. At the conclusion of a sampling event, NAREL removes all of the replicates from the air samplers and places them into clean, labeled Petri slides. Id. ¶ 4. All of the replicates from the same sampling event are then placed into a single plastic bag for storage. Id. PT filter samples are then stored in standard, unrefrigerated laboratory cabinets. Id. ¶ 5. Blank filters are collocated with the PT samples, and are routinely analyzed to test for possible contamination of the cabinet. Id.

The EPA physical scientist further explained that when PT is conducted, NAREL assembles PT sample sets and ships them in small coolers to participating laboratories on the same day using overnight express mail. Id. ¶ 8. Participating laboratories then conduct their analyses of the replicate filters, and the results are compared against the results of the filters from the same sampling event that are retained and analyzed by NAREL. Id. ¶ 6. The above procedures were utilized in this procurement, as well as in annual inter-laboratory comparison studies for the PM2.5 chemical speciation program. Id. ¶ 11. The inter-laboratory comparison studies, which EPA has conducted since 2005, involve between five and seven laboratories per year, and RTI, the incumbent CSN support contractor, has participated in all nine of the studies. Id. ¶¶ 9-10.[5]

For this procurement, offerors invited to participate in PT were to conduct four different types of laboratory analyses on three different types of filters. RFP at M‑2‑M‑4. Relevant to the issues in this protest is the required ion chromatography analysis of nylon filters. RTI’s consultant retained for the purpose of this protest, a university professor specializing in atmospheric science, submitted a declaration that included a detailed discussion of the ion chromatography process. RTI’s consultant explained that ion chromatography is a laboratory technique used to detect and quantify the presence of ions in a sample. Decl. of RTI’s Consultant (May 4, 2015) ¶ 8.[6] To conduct ion chromatography analysis for aerosol particles collected on an air filter, the filter is placed into deionized water or another liquid in order to extract soluable particulates from the filter. Id. ¶ 9; see also AR, Tab 16A, EPA Air Filter Analysis Report, at 2. A portion of the extracted sample is then pumped through a column packed with ion exchange resin. Decl. of RTI’s Consultant (May 4, 2015) ¶ 9. Interactions between sample ions and the charged resin retard the movement of the sample ions through the column. Id. Different ions exit (or elute) from the column at different rates, and the rates are quantified by an increase in measured charge as the solution passes through a conductivity measurement cell. Id. Substances are identified based upon the rate they move through the column (the “retention time”) in comparison to retention times established through instrument calibration using standard solutions of known ions. Id. Thus, an analyst can determine the amount and type of ions in a sample by analyzing the retention times recorded. As relevant to the protester’s contentions regarding the testing results here, however, if two compounds elute at the same rate (or co-elute), the result may be an overestimation of the amount of the targeted compound. Id. ¶ 11.

The RFP provided, in relevant part, the following instructions and acceptance criteria for the ion chromatography analysis:

EPA will provide six (6) pre-loaded 47-mm Nylon filters to each laboratory. The filters must be extracted and analyzed for anions and cations using proposed laboratory procedures.

 

Category	Acceptance Criteria*
Anions
Nitrate (NO3-)	+13 ≥ D ≥ -14 [micrograms (µg)]/filter
Sulfate (SO4=)	+35 ≥ D ≥ -27 µg/filter
Cations
Potassium (K+)	+2 ≥ D ≥ -3 µg/filter
Ammonium (NH4+)	+35 ≥ D ≥ -23 µg/filter
Sodium (Na+)	+4 ≥ D ≥ -3 µg/filter
*D = (Test Lab Result) – (EPA Result)

Successful performance will be if no more than two (2) of the thirty (30) individual ion analysis results fall outside of the acceptance criteria.

 

Results must be reported to EPA within 14 calendar days of sample receipt. . . . Please include the current laboratory method detection limit (MDL) and other appropriate expressions of uncertainty along with the PT results. Procedures and raw data for determination of the MDL and components of uncertainty used in the calculation must be identified either with your results or in the proposed technical approach or [standard operating procedure].

RFP at M-3.[7]

To establish the acceptance criteria, the EPA’s physical scientist explained that the agency examined its historical database of analytical results from the annual inter-laboratory comparison studies dating back to 2005. AR, Tab 15, Decl. of EPA Physical Scientist (Apr. 21, 2015) ¶ 18. Each record in the historical database contains three fields: (1) the test lab result; (2) the EPA result; and (3) the difference (D). Id. Based on the historical results, the agency calculated the historical mean and standard deviations for each anion and cation to be analyzed. Id. As a result of that analysis, EPA elected to set what it characterized as wide acceptance parameters (i.e., three standard deviations) for the PT results to accommodate random fluctuations such as variability in filter replicates and instrument performance. Id. For example, based on the 132 historical results for sulfate analysis, EPA found that under a three standard deviation acceptance parameter, 130 of the 132 historical results (or approximately 98.5 percent) would meet the acceptance parameter. Id.

EPA evaluated RTI’s proposal as meeting the RFP’s minimum technical requirements and invited RTI to participate in PT. On December 10, 2014, EPA shipped six PT filter samples from four sampling events to RTI and the other offerors in the competitive range, respectively. Id. ¶ 14. For each of the four sampling events, NAREL analyzed three filters from the set of replicates created during each event. COSF (Apr. 22, 2015), at 5.[8]

RTI received the sample filters on or around December 11, and timely submitted its test results to the EPA on December 23. Protest (Mar. 23, 2015) at 5. On March 2, 2015, the agency notified RTI that it had failed five of the ion chromatography analyses due to sulfate readings in excess of the permissible upper limits. RTI’s proposal was eliminated from the competition for the laboratory analysis (CLINs 0006-0008) portion of the procurement because the five failed analyses exceeded the RFP’s acceptance criteria of no more than two failed analyses. AR, Tab 7, Notification of Elimination from the Competitive Range (Mar. 2, 2015), at 1. After a debriefing and additional exchanges between the parties, this timely protest followed.

DECISION

RTI challenges its exclusion from the competition on two primary grounds. First, the protester contends that EPA’s technical evaluation was unreasonable because EPA provided RTI with defective sample filters for PT. Alternatively, RTI alleges that even if the sample filters provided by EPA were not defective, the agency improperly relied on, in essence, an unstated evaluation criterion by failing to disclose material information regarding the archiving and storage of the sample filters.

In reviewing a protest challenging an agency’s evaluation, our Office will not reevaluate proposals, nor substitute our judgment for that of the agency, as the evaluation of proposals is a matter within the agency’s discretion. Computer World Servs. Corp., B-410513, B-410513.2, Dec. 31, 2014, 2015 CPD ¶ 21 at 6. Rather, we will review the record only to determine whether the agency’s evaluation was reasonable and consistent with the stated evaluation criteria and with applicable procurement statutes and regulations. Id. Based on the record, we find that neither of RTI’s objections provides a basis on which to sustain the protest.

Defective Sample Filters

RTI alleges that EPA’s PT was unreasonable because the government-furnished sample filters were defective. The protester contends that the agency handled and stored the sample filters in an unanticipated, unreliable, and unreasonable way, which in turn led to degraded filters that skewed RTI’s PT results. For the reasons that follow, we deny this aspect of the protest.

As discussed above, EPA used archived sample filters created between 2012 and 2014 that had been stored in standard, unrefrigerated laboratory cabinets. RTI both challenges the procedures used to store and handle the sample filters provided to offerors for the PT, and specifically asserts that the filters provided to RTI were defective. The protester first argues that it was unreasonable for the agency to provide offerors with archived sample filters for PT that were subject to long-term storage and unsound handling procedures. Specifically, RTI asserts that the composition of filters degrades over time, especially where the sample filters are not refrigerated/frozen or stored in dark and secure conditions. See Protester’s Comments (May 4, 2015) at 7-8; Decl. of RTI’s Consultant (May 4, 2015) ¶¶ 18, 20‑22. In this regard, the protester argues that the PT samples were prepared, handled, and stored in a materially different manner than the air filter samples that the CSN contractor will have to handle and analyze under the resulting contract. In contrast to the PT procedures used here, RTI contends that under established CSN procedures air filter samples are collected for 24 hours at the CSN network sites, stored in cold, dark, and secure conditions, and tested within 20 to 30 days of collection. See Protest (Mar. 23, 2015) at 6; Protester’s Comments (May 4, 2015) at 3.

RTI contends that the alleged mishandling and unreasonable archiving of the sample filters resulted in testing interferences in the form of organic acids that developed on the filters that caused inaccurate, elevated sulfate level readings in RTI’s test results. See Protest (Mar. 23, 2015) at 7. The protester asserts that its subsequent testing of the government-furnished PT filters, which were provided by the agency after the exclusion of RTI from the competitive range, identified the presence of malonate and succinate, organic acids with generally low and difficult to detect concentrations in air sampling filters. See Decl. of RTI Research Chemist (May 4, 2015) ¶ 15; Decl. of RTI’s Consultant (May 4, 2015) ¶¶ 24, 26. RTI argues that EPA’s improper storage of the filters was the likely cause of the presence of these inorganic acids, as improper storage can result in concentrations of these compounds growing due to chemical oxidation of less oxidized forms of particulate organic carbon collected on the sample filters. Decl. of RTI’s Consultant (May 4, 2015) ¶ 26. The protester submits that the presence of these inorganic acids skewed the identified sulfate rates higher because the two organic acids co-elute at similar rates to sulfate. Id. ¶ 25. In addition to the presence of the two identified organic acids, RTI’s consultant also suggested that sulfur dioxide gas may have collected on the filter samples during collection, and then reacted during archival storage or filter extraction to form sulfate, which in turn would further skew the sulfate results identified during PT. Id. ¶ 30. The protester also argues that sample degradation impacts laboratories differently because each laboratory uses different procedures and equipment (e.g., eluent solutions, ion columns). See id. ¶ 27.

EPA responds that its technical evaluation was reasonable and in accordance with established PT procedures. EPA argues that its methodology and approach to preparing, shipping, and analyzing PT samples for this procurement conformed to the agency’s well-established standard PT procedures used in its annual inter-laboratory studies for the CSN program. AR at 4-5.[9] The agency argues that RTI’s challenges are premised on a misconception regarding the purpose of PT testing versus routine air sample testing that will be performed by the CSN support contractor. EPA argues that the objective of the testing under the CSN contract is to determine the composition of the air at a certain time and place. AR at 4; Supp. Decl. of EPA Physical Scientist (May 11, 2015) ¶ 2. Therefore, the samples are not subject to long-term archiving, and preventative measures, such as refrigeration, are taken to prevent routine air samples from changing. Supp. Decl. of EPA Physical Scientist (May 11, 2015) ¶ 2.

In contrast, the purpose of PT, according to EPA, is to test a laboratory’s performance, which is achieved by comparing the testing laboratory’s results against the results obtained by the agency. Id. Therefore, so long as the PT sample filters provided to the test laboratories are good replicates of the samples retained by EPA for testing, the same concerns regarding maintaining the integrity of the routine air samples to capture conditions at a fixed point in time are not applicable to the PT samples. Id. In essence, the agency argues that any damage due to storage and handling conditions for the PT sample filters is not relevant because PT is meant to check a laboratory’s ability to detect whatever is on the filter, not to precisely determine the air conditions at a specific location at a fixed point in time. In this regard, EPA argues that its testing results and those of the other two offerors in the competitive range were very similar. AR at 5‑6. Additionally, the agency argues that the RFP’s broad acceptance criteria reasonably accounted for potential variability in the sample filters and/or each laboratory’s procedures and equipment. Id. at 6-7. Based on the record, we find that the agency’s PT procedures here were reasonable.

First, EPA has advanced a reasonable explanation regarding the different uses for PT samples and routine air samples, and why its handling of the PT filters here was reasonable. The purpose of the PT was to evaluate a laboratory’s ability to accurately measure the presence of certain target analytes as measured against EPA’s contemporaneous and historical analytical results. In contrast, the purpose of testing routine air samples under the CSN contract will be to accurately measure conditions of air quality at a specific location and a specific point in time. Thus, the agency has reasonably explained why the additional measures taken to ensure the integrity of routine air samples are not necessary for PT samples because the purpose of PT is to measure a laboratory’s ability to analyze the PT samples in their current state, not to analyze air conditions at a specific location at a fixed point in time. We also find relevant EPA’s argument that the PT procedures utilized in this procurement were consistent with the procedures utilized over the past decade in inter-laboratory comparison studies of laboratories supporting the CSN program.

Second, the record reflects that very similar results were obtained by EPA and the other offerors in the competitive range utilizing sample filters that were simultaneously prepared, and handled and stored, under the same conditions. As addressed above, EPA explained that all of the replicate filters from the same sampling event were simultaneously prepared and co-stored in the same plastic bag under the same conditions at NAREL’s lab. See AR, Tab 15, Decl. of EPA Physical Scientist (Apr. 21, 2015) ¶¶ 4, 13. As reflected in the ion chromatography results below, the agency and other offerors in the competitive range obtained materially similar results for sulfates for all filters across all of the sample sets, while RTI was near or exceeded the permissible upper variance limits in all cases:

Comparison of PT Results for Sulfate
Events	EPA Test Results	Offeror A Test Results	Offeror B Test Results	RTI Test Results	Upper Acceptance Limit	Lower Acceptance Limit
140-hour	87.97 / 88.64 / 95.61	95.93	99.45	119.30	125.52	63.70
168-hour	215.44 / 215.81 / 221.21	223.96	217.17	277.65	252.26	190.45
170-hour	136.24 / 138.79 / 168.04	172.38	171.84	214.51	182.47	120.66
260-hour	185.32	189.08	190.75	238.34	220.42	158.61
260-hour	186.41	189.71	188.19	231.55	220.42	158.61
260-hour	185.21	190.23	189.35	232.88	220.42	158.61

AR, Tab 16C, Ion Chromatography Results, at 1.[10]

The very similar results obtained by EPA and the other offerors in the competitive range support the agency’s position that RTI’s PT procedures were responsible for its unacceptable test results. In this regard, we note the assertion of RTI’s consultant that, notwithstanding that the filters from each sampling event were simultaneously created in the same manner and co-stored in the same plastic bags in the same NAREL laboratory cabinets, the amount of organic acid formation could vary from sample to sample depending on various factors, including the amount of precursor species on the filters. See Supp. Decl. of RTI’s Consultant (May 13, 2015) ¶ 19. While filter degradation issues could perhaps explain one or two of RTI’s PT failures, we do not find plausible the protester’s contention that all six of the filters from four different sampling events provided to RTI--and none of the at least 18 filters retained by EPA that were simultaneously created and co-stored with the filters provided to RTI or the 12 filters provided to the other offerors in the competitive range--were adversely affected by degradation issues. Rather, we find more plausible EPA’s assertion that, to the extent there was any degradation resulting from the archiving and storage of the replicates, all of the co-stored filters from the same sampling event would have been generally affected in the same manner. See ­­­Supp. AR at 3.

RTI counters that degradation issues could uniquely impact each laboratory due to the different procedures and equipment utilized. See Protester’s Supp. Comments (May 14, 2015) at 3-4. We find this phenomenon, however, to be the result of independent professional judgment on the part of each laboratory regarding its selection of the appropriate procedures and equipment to employ, and not an inherent fault in EPA’s PT procedures. See id. at 3-4 (“If the laboratories had truly used the same testing equipment / procedures, the retention times of each ion would be identical. . . . [T]he organic acids which impacted RTI’s sulfate results may not have impacted the sulfate results of the EPA or any other laboratory.”). In this regard, RTI has represented that through alternative testing procedures and equipment employed in analyses after its elimination from the competitive range, it was able to separate out the organic acids that co-eluted with the sulfate. See Decl. of RTI Research Chemist (May 4, 2015) ¶¶ 12-15. In contrast, EPA and the other offerors in the competitive range obtained similar sulfate results in their initial analyses apparently without the need for alternative methodologies or equipment. Under these circumstances, we do not find that EPA employed unreasonable PT procedures or that RTI has conclusively demonstrated that the government-furnished sample filters were not appropriate for PT.

Moreover, even assuming RTI’s argument that potential degradation of the filter samples could affect each laboratory differently based on the laboratory’s unique procedures and equipment, and thus could explain some variability in the results, EPA allowed for very broad acceptance parameters--three standard deviations. RFP at M-2. The agency explained that approximately 98.5 percent (130/132) of historical ion chromatography analysis results for sulfate would have been rated acceptable under the broad acceptance range adopted for this procurement. AR, Tab 15, Decl. of EPA Physical Scientist (Apr. 21, 2015) ¶ 18. In light of the broad acceptance range afforded to offerors, we cannot conclude that the agency’s evaluation here was unreasonable.

In sum, in the absence of any definitive evidence that the sample filters provided to RTI were not appropriate for PT, we find no basis to sustain the protest on this ground.

Unstated Evaluation Criterion

RTI next argues that, even assuming that the use of archival sample filters was not per se unreasonable, EPA’s failure to disclose to offerors that archival filters would be used instead of routine air samples, as will be handled and analyzed under the resulting contract, constituted an unstated evaluation criterion. We find no basis to sustain RTI’s protest on this basis.

It is axiomatic that in a negotiated procurement an agency must evaluate proposals based only on the solicitation’s enumerated evaluation factors. Wood Cuts, B‑403960.3, May 19, 2011, 2011 CPD ¶ 105 at 4; Federal Acquisition Regulation § 15.305(a) (“An agency shall evaluate competitive proposals and then assess their relative qualities solely on the factors and subfactors specified in the solicitation.”). RTI asserts that the RFP’s direction that offerors were to conduct their PT “using [their] proposed laboratory procedures” reasonably led offerors to believe that the agency would provide routine air samples prepared and stored in accordance with applicable CSN procedures. See Protest (Mar. 23, 2015) at 11. The protester alleges that the agency deviated from this implied representation by requiring offerors to pass PT using samples that were not handled or stored in the same manner as the filters that will be provided on the contract. See Protester’s Comments (May 4, 2015) at 8.

EPA responds that the RFP does not include either an implicit or explicit representation regarding the handling and storage of the sample filters provided for PT, and that the agency reasonably evaluated proposals in accordance with the RFP’s express evaluation criteria. See AR at 8. The agency also argues that the PT procedures utilized in this procurement were identical to those utilized in previous inter-laboratory comparison studies, which RTI was familiar with based on its participation in the studies. Id. Based on the record, we do not find that the agency relied upon an unstated evaluation criterion.

The record shows that EPA conducted PT in the manner disclosed in the solicitation. Specifically, the agency provided the appropriate number of filters to each offeror and analyzed the target analytes in accordance with the RFP’s enumerated acceptance parameters. See AR, Tab 15, Decl. of EPA Physical Scientist (Apr. 21, 2015) ¶¶ 13-14, 16-17; Tab 16C, Ion Chromatography Results. The RFP does not include any representation regarding the storage and handling of the sample filters that would be furnished to offerors for PT. RFP at L‑13, M-2. Additionally, EPA demonstrated, without rebuttal from RTI, that it utilized PT procedures that have been in use in connection with PT on the CSN program for a decade. AR, Tab 15, Decl. of EPA Physical Scientist (Apr. 21, 2015) ¶¶ 9-11. RTI has not demonstrated why it was reasonable to assume that the agency would provide PT samples that were stored under the same conditions as those that would be provided during contract performance. As discussed above, the purpose of PT was to test the accuracy of the offeror’s ability to test a sample, not to determine the air conditions at a fixed location at a fixed point in time.

Additionally, we also again note that, notwithstanding potential variability in laboratory procedures and equipment and the significantly broad acceptance criteria, EPA and the other two offerors in the competitive range obtained very similar test results for all of the sampling events. As the majority of the laboratories were in material agreement, we cannot conclude that the agency imposed an unstated evaluation criterion that materially affected the validity of the PT.

The protest is denied.

Susan A. Poling
General Counsel