Atlantic Research Marketing Systems, Inc., B-292743, December 1, 2003
B-292743: Dec 1, 2003
It was inconsistent with the terms of the solicitation for the agency to remove the protester's proposal from consideration for award for allegedly failing to offer certain desired improvements. The Navy determined that each of ARMS's RIS configurations was unacceptable due to operational unsuitability. The special operations peculiar modifications (SOPMOD) system is designed and intended to provide special operations force (SOF) members. The system's core capability is accommodated by a rail interface system. The goal of the miniature day/night sight (MDNS) development effort is to improve on current SOPMOD capabilities through miniaturization. Among the SOPMOD subsystems for which an improved version is sought is the rail interface surface. /1/ The solicitation specified both minimum or threshold (T) requirements and desired or objective (O) requirements for the RIS and other MDNS subsystems.
Atlantic Research Marketing Systems, Inc., B-292743, December 1, 2003
Atlantic Research Marketing Systems, Inc. (ARMS) protests the rejection of its proposal for four configurations of a rail interface system (RIS) under request for proposals (RFP) No. N00164-02-R-0014, issued by the Department of the Navy for miniature day/night sight development for the special operations peculiar modifications system. The Navy determined that each of ARMS's RIS configurations was unacceptable due to operational unsuitability.
We sustain the protest.
The special operations peculiar modifications (SOPMOD) system is designed and intended to provide special operations force (SOF) members, who operate in a wide range of extreme conditions, with the ability to reconfigure their weapons for various mission scenarios rapidly and reliably. The system's core capability is accommodated by a rail interface system, which attaches to the M4A1 carbine and provides attachment points for accessories such as sights and lasers. The goal of the miniature day/night sight (MDNS) development effort is to improve on current SOPMOD capabilities through miniaturization, ruggedization, combination, or other enhancements to existing SOPMOD subsystems. Among the SOPMOD subsystems for which an improved version is sought is the rail interface surface. /1/
The solicitation specified both minimum or threshold (T) requirements and desired or objective (O) requirements for the RIS and other MDNS subsystems. In addition, it identified Key Performance Parameters (KPP) and Additional Performance Parameters (APP). The solicitation described KPPs as "must-pass testing events" and provided that "[a]ny offering failing the (T) value of any of the KPPs will be removed from further testing and will not be considered for contract award." RFP, amend. 5, Sec. 3.1.2. The solicitation described APPs as "tradable parameters . . . used to measure effectiveness and performance," and provided that [f]ailure to meet either (T) or (O) requirement values specified in an APP does not remove a submission from further testing or from consideration for contract award. APPs are evaluated to provide information leading to a best value award determination.
Id. Sec. 3.1.3. The solicitation further noted at Sec. 3.1.4 that "KPP (O) values and APP (T) and (O) values are the equivalent of research and development goals." In other words, KPP (T) values represented mandatory features of the solicited subsystems, while KPP (O) values and APP (T) and (O) values represented desired improvements to existing subsystems.
The solicitation identified KPP (T) values applying to the RIS as waterproofing (subsection 126.96.36.199) and interoperability (subsection 188.8.131.52). /2/ RFP, addend. 3, at 4. It also identified a list of APPs that applied to the RIS as well as other subsystems; these parameters, which represented desired general improvements to current core capabilities, included improved operational test results (Sec. 3.2.1), improved portability (Sec. 3.2.2), improved corrosion resistance and ease of cleaning and maintenance (Sec. 3.2.3), improved endurance (Sec. 3.2.4), improved reliability (Sec. 3.2.5, Sec. 3.2.20), and improved safety features (Sec. 3.2.6). In addition to the foregoing general APPs, the solicitation identified APPs that represented desired improvements to the RIS specifically. These included a rigid, free-floating barrel design concept (Sec. 184.108.40.206.1.2.1), a method to attach future 6:00 systems /3/ as close as possible to the carbine barrel without touching it or interfering with the natural barrel vibrations during firing (Sec. 220.127.116.11.1.2.2), and, with particular significance for this protest, a method of attaching a grenade launcher with the existing mount or of mounting it free of the carbine barrel through application of "a simple modification plan/modification" (Sec. 18.104.22.168.1.2.3).
The RFP allowed for the award of one or more indefinite-delivery/ indefinite-quantity, fixed-price contracts for developmental test prototypes, operational test prototypes, limited user test items, and production quantities for the RIS and each of the seven other subsystems. The solicitation provided for a three-phase evaluation process: phase 1 was to result in the selection of the proposal(s) that would move on to phase 2; phase 2 was to consist of developmental testing; and phase 3 was to consist of operational testing, followed by final source selection. Phase 1, step 1 was to consist of a preliminary review of proposals to determine whether go/no go criteria (i.e., KPP thresholds) had been satisfied; phase 1, step 2, to be conducted after an oral presentation/demonstration by each offeror, was to consist of an evaluation of proposals on the basis of the following factors and subfactors, listed in descending order of importance:
1. Technical/User Assessment
a. Go/No Go (KPP Thresholds)
b. APPs and KPP Objectives
2. Contracting and Management
a. Past performance
RFP, addend. 3, at 1-2.
ARMS submitted a proposal for four configurations of its Selective Integrated Rail (SIR) System: models 45, 46, 50, and 58. The agency evaluators determined that each model met the KPP threshold values. After the protester's oral presentation/demonstration, however, the operational evaluation team found that each of the ARMS models was operationally unsuitable and therefore unacceptable. On the basis of this finding, the source selection authority determined that ARMS's proposal for all four models should be rejected. By letter dated July 29, 2003, the contracting officer notified ARMS that its proposal had been determined unacceptable "due to operational unsuitability for each of the four (4) configurations proposed." ARMS immediately requested a pre-award debriefing, which the Navy furnished by letter of August 11. ARMS asked the contracting officer to reconsider his position, and, upon receipt of his letter confirming his earlier decision, filed a protest with our Office on August 21.
In his debriefing letter, the contracting officer explained that ARMS's four SIR models had been determined unsuitable on two bases: all four models raised the operator's line of sight, and none adequately provided for the mounting of the M203 grenade launcher free of the carbine barrel. With regard to the first point, the contracting officer noted that, per section 3.1 of the specification, MDNS subsystems were to "allow SOF operators to better and more rapidly acquire, identify, and accurately fire on enemy targets in combat," but that "[b]y raising the optical line of sight, . . . the SIR decreased the SOF operators' ability to better and more rapidly acquire, identify, and accurately fire on enemy targets because the standard cheek weld positions for currently fielded SOPMOD equipment would not be compatible with this configuration." Debriefing Letter at 1. With regard to the second point, the contracting officer noted that:
SIR Model 45 is designed such that the M203 grenade launcher is to be mounted to the M4A1 barrel. This is in violation of Performance Specification Paragraph 22.214.171.124.1 which lists independence of the M4A1 barrel as a core requirement.
SIR Models 46, 50, and 58 provide a design that prohibits the mounting of the M203 grenade launcher without permanently and irreversibly modifying the fundamental M203 receiver. This mounting scheme is not compatible with currently fielded M203s. This design does not conform to paragraphs 126.96.36.199 and 188.8.131.52.1.2.3 of the Performance Specification.
The protester challenges both aspects of the agency's technical evaluation. In reviewing such a challenge, we will not reevaluate proposals, but will instead review the record to determine whether the evaluation was reasonable and consistent with the terms of the solicitation. Rolf Jensen & Assocs., Inc., B-289475.2, B-289734.3, July 1, 2002, 2002 CPD Para. 110 at 4. As explained below, we conclude that it was unreasonable and inconsistent with the solicitation for the agency to reject the protester's proposal from further consideration for award based on the evaluation findings it made.
In response to the contracting officer's first point, ARMS argues that its proposed SIRs comply with the KPPs and APPs set forth in the RFP, and that, therefore, they should not have been rejected as unacceptable. The protester further argues that its SIRs do not decrease the SOF operators' ability to acquire, identify, and accurately fire on enemy targets, as demonstrated by testing data that it included in its proposal.
First, to the extent that by noting that "the SIR decreased the SOF operators' ability to better and more rapidly acquire, identify, and accurately fire on enemy targets," the contracting officer meant that the SIR configurations failed to improve the operators' ability to sight and fire on enemy targets, improvement in firing accuracy was an APP (i.e., an objective or desired improvement), as opposed to a KPP (i.e., a requirement). See Subsection 3.2.1. /4/ Accordingly, to the extent that the evaluators found ARMS's SIRs unsuitable on the basis that they failed to improve operators' ability to acquire, identify, and fire on enemy targets, the evaluators in effect found the SIRs unsuitable for failing to meet the values specified in an APP. In our view, rejection on such a basis was contrary to the terms of the RFP, which, as previously noted, provided that "[f}ailure to meet either (T) or (O) specified in an APP does not remove a submission from further testing or from consideration for contract award."
Second, to the extent that the contracting officer instead meant that the SIRs not only failed to improve firing accuracy, but in fact decreased it, any such conclusion was not based on a reasonable review of the record before the agency. Specifically, the protester submitted with its proposal an evaluation report, completed by the Assistant Commandant of the 75th Ranger Regiment, a Special Operations Command, that showed that the SIR system improved shooters' accuracy over the existing system. The report noted that to determine the accuracy of the SIR system, shooters fired 50 rounds at different targets placed at varying ranges from 50 to 300 meters and that 49 of 50 hits scored. Protester's Proposal, app. 1C at 2. The evaluation concluded as follows:
The SIR system is overall an extremely good candidate to replace the current SOPMOD RAS system. It is extremely rugged and truly modular. All shooters that used the system were impressed with its performance and versatility. Shooters were especially impressed with the increased reliability of the weapon. All of the shooters stated that even though the SIR system was slightly heavier than the RAS system, the weight tradeoff was more than worth it. This was due to the fact that the SIR system gave them an increased advantage in 3 areas: 1) weapons reliability, 2) weapons accuracy (both are factors of increased survivability) and 3) modularity.
Id. at 2.
The agency argues that it justifiably gave the foregoing report no weight in its evaluation because rather than a valid operational test, it was merely the personal assessment of an individual user.
We fail to see a reasonable basis for the Navy's having summarily dismissed the test data contained in the user assessment from consideration, particularly given that it had no testing data of its own (i.e., it was dismissing the user assessment in favor of no, as opposed to better, data of its own). Consistent with the protester's explanation, the data appears to be not simply a biased endorsement of the ARMS product, but rather a user's assessment of the strengths and weaknesses of the product undertaken to evaluate it for possible future use and to provide feedback to the manufacturer so that it would be able to refine its product further to suit the military's needs. While not the equivalent of--and thus reasonably entitled to lesser weight than--the results of operational testing, we nonetheless think that the user assessment was sufficient to overcome the evaluators' assumption of decreased accuracy, which was based on no testing data at all. In sum, assuming that the contracting officer's conclusion, quoted above, was meant to express the agency's view that the protester's SIR system decreased firing accuracy, we cannot conclude that any such determination has a reasonable basis, in view of the agency's failure to consider the protester's testing data, and the lack of any testing data (or other support) for the agency's conclusion.
With regard to the agency's second basis for determining the protester's SIRs unsuitable, i.e., that they did not adequately provide for the mounting of the M203 grenade launcher free of the carbine barrel, the protester argues the subsection pertaining to mounting of the grenade launcher, 184.108.40.206.1.2.3, was an APP, rather than a KPP, and thus noncompliance with it should not have resulted in rejection of the SIRs. The protester further argues that at its oral presentation, it proposed a plan, applicable to all four of its models, for mounting the M203 grenade launcher free of the carbine barrel.
The solicitation provided as follows at Sec. 220.127.116.11.1:
Supplemental Specifications for the Rail Interface System, M4A1 Carbine and Other Weapons. Notes: The SOPMOD program is not seeking alternate sources for systems that duplicate the currently fielded RIS/RAS. The SOPMOD Program instead is seeking alternative mounting subsystems with improved operational performance characteristics. The core requirements for the future RIS are rigidity (no loss of zero due to rough handling), independence of the M4A1 Barrel (no interference with the natural harmonic vibrations of the barrel during firing), and provision for mounting future 6:00 subsystems more closely to the gun barrel (decrease in offset between the 6:00 subsystems and the axis of the bore.)
. * * * * *
In addition to the requirements and desired improvements listed in 3.1 and 3.2 above, the RIS shall demonstrate the following minimum performance characteristics (Threshold):
18.104.22.168.1.1 Additional RIS KPPs:
No supplemental KPPs are required over and above those in 3.1. above.
22.214.171.124.1.2 Additional RIS APPs:
126.96.36.199.1.2.1 The future RIS should incorporate design features which improve overall system performance. This may include a rigid, floating barrel design concept. (APP)
188.8.131.52.1.2.2 The future RIS should provide a method to attach future 6:00 systems as close as possible to the carbine barrel without touching it or interfering with the natural barrel vibrations during firing. This attachment method should provide repeatability of the zero of 6:00 systems when attached and detached. The future RIS should contain design features which facilitate cleaning/maintenance of the area between the carbine barrel and the body of the future RIS. (APP)
184.108.40.206.1.2.3 Proposals for the Future RIS should either allow for the attachment of the M203/M203E1 with existing SOPMOD Grenade Launcher Mount, . . . (T) or provide a simple modification plan/mechanism for the mounting of the M203 free of the carbine barrel (T), or both (O). (APP)
First, regarding the agency's rejection of the SIR model 45 on the grounds that mounting it to the M4A1 barrel violates subsection 220.127.116.11.1, which establishes independence of the barrel as a core requirement, the referenced language from subsection 18.104.22.168.1 was, like the introductory paragraph under 3.1 discussed above, a summary of the agency's objectives, which were then spelled out in other subsections as KPPs and/or APPs. The very fact that the improvements referred to as "core requirements" in the introductory paragraph, including the "requirement" for independence of the barrel, were further along in the specification identified as APPs, rather than KPPs, indicates that they were desired improvements, as opposed to mandatory requirements. Moreover, subsection 22.214.171.124.1.2.3 designated as a (T) value attachment of the M203 with the existing grenade launcher mount, which presumably means attachment to the barrel, since, according to the subsection, a modification plan/mechanism is required for mounting of the M203 free of the barrel. Thus, we find no basis for the agency's argument that the model 45's mounting of the M203 grenade launcher to the M4A1 barrel violated a mandatory requirement.
Regarding the other models, again we note that subsection 126.96.36.199.1.2.3 pertaining to attachment of the grenade launcher was an APP, as opposed to a KPP, and thus noncompliance with it did not furnish a basis for removal of ARMS's proposal from further consideration for award. We also note that the protester has alleged that during its oral presentation, in addition to the plan to modify the M203 receiver that the agency views as unacceptable, it "offered a simple plan to change the receiver of the M203, as it is easily interchangeable without modification, to provide a free floating M203 grenade launcher," which would obviate concerns about interoperability. Protester's Comments, Oct. 2, 2003, at 20. The agency responded to this allegation by noting that ARMS had failed to furnish a sample modified receiver at the oral presentation/demonstration. Given that subsection 188.8.131.52.1.2.3 stated that proposals should provide either a modification plan or a mechanism for the mounting of the M203 free of the carbine barrel, we do not see how the agency could reasonably have determined the protester's furnishing of only a plan unacceptable.
In our view, while there is no reason that the agency could not have considered the impact of raising the operators' line of sight on their speed and accuracy in firing and/or the feasibility of the protester's plans for mounting the grenade launcher free of the carbine barrel in a best value determination, it was unreasonable and contrary to the terms of the solicitation for it to reject the protester's models as unacceptable for the reasons cited. Accordingly, we sustain the protest.
We recommend that the agency include the models proposed by ARMS among those under consideration for award. We also recommend that the protester be reimbursed for the costs of filing and pursuing its protest. Bid Protest Regulations, 4 C.F.R. Sec. 21.8(d)(1) (2003). In accordance with our regulations, ARMS's certified claim for such costs, detailing the time expended and the costs incurred, must be submitted directly to the agency within 60 days after receipt of the decision.
The protest is sustained.
Anthony H. Gamboa
1. Other subsystems for which this solicitation seeks improved versions are the enhanced combat optical scope, miniature night vision sight, clip-on night vision device, back-up iron sight, visible bright light, carbine visible laser, and infrared pointing, illuminating, and aiming laser.
2. Subsection 184.108.40.206 provided with regard to waterproofing: No MDNS subsystem prototype will exhibit design features that preclude waterproofing to a depth of 66 feet. All MDNS subsystems shall be waterproof to a depth of 66 feet for a minimum of two hours (KPP). Subsection 220.127.116.11 provided with regard to interoperability: MDNS subsystem designs shall not cause unsafe conditions or interfere with the functioning of the M4A1 Carbine (T). MDNS subsystems will interoperate with existing SOPMOD subsystems, or substitute them singly or in combination (O).
3. As we understand it, a 6:00 system is one mounted on the bottom part of the carbine.
4. Subsection 3.2.1 provided as follows: Improved Operational Test Results. Compatibility. SOF operators while wearing various uniforms, equipment, and possibly eyeglasses appropriate for each individual mission shall achieve proper cheekweld, sight picture, and eye relief. The shooting position shall be the same as used for current fielded M4A1 Carbines and other SOF Small Arms. MDNS subsystems will exhibit improved Operation Suitability and Operational Effectiveness. This includes improved performance/hit scores in semi-automatic and full automatic fire, improved performance at both shorter and longer ranges, and improved performance under low-illumination and other adverse conditions. The overall objective (O) is increased hit scores at all ranges from 2 meters to 800 meters by SOF operators during day, night, rain, mist, smoke, vegetation, fog, dust, and extreme low light conditions (APP). Subsection 3.1, from which the contracting officer excerpted the language concerning better and more rapid target acquisition, is merely an introductory paragraph that summarizes the overall goals of MDNS development; subsystem requirements and developmental objectives are set forth in subsequent subsections as KPPs and APPs.