Optical Systems Technology, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Matter of: Optical Systems Technology, Inc.

File: B-296516.2; B-296516.3

Date: March 17, 2006

DIGEST

Protest that rejection of proposal was based on unreasonable testing of protester's sample night sights for .50 caliber rifles is denied where, contrary to protester's assertion, record indicates that agency undertook reasonable efforts to ensure that sights were securely mounted and properly adjusted during testing; even if these efforts were not entirely successful (and there is no basis in the record for reaching such a conclusion), the agency could reasonably conclude that the susceptibility of protester's sights to significant damage and degraded performance, notwithstanding reasonable efforts to mount the sights correctly, rendered the sights technically unacceptable.

Optical Systems Technology, Inc. (OSTI) protests the award of a contract to Knight's Armament Company (KAC) under request for proposals (RFP) No. H92222-05-R-0007, issued by the United States Special Operations Command for non'developmental Visual Augmentation System (VAS) In-Line Clip-on Night Sights. OSTI asserts that the agency's rejection of its proposal was based on unreasonable testing of its sample items. OSTI also challenges the evaluation of KAC's proposal.

We deny the protest.

The RFP contemplated award of an indefinite-delivery/indefinite-quantity, fixed-unit-price contract for up to 3,000 VAS night sights over a 5-year period. The RFP's performance specification provided that the night sight shall be an in-line, clip-on image intensification sight, utilizing a GEN III/OMNI IV image intensifier tube as a minimum, mainly for use on Army XM107 and Navy M88PIP (Mk15) .50 caliber sniper rifles during nighttime operations. The night sight clips onto a mounting rail along the top of the weapon, directly in front of the existing dayscope, providing a quick attach/detach capability for nighttime operation while maintaining the dayscope boresight. Performance Specification sect. 3.2; VAS Night Vision Devices Sample Test Report sect. 1. The performance specification required use of an adjustable, locking single-throw lever-type mounting system, KAC Knightscope base assembly part No. 22097 or equivalent, allowing for single-hand operation and attachment/mounting on a Military Standard (MIL-STD) 1913 mounting rail. Performance Specification sect. 3.4.3.

Among the several performance requirements set forth in the performance specification were requirements relating to accuracy and resistance to weapons shock. Regarding accuracy, the specification provided that the sight –shall allow a trained sniper to maintain his current level of accuracy as a (threshhold), and deliver precise fire within one minute of angle (1 MOA) (objective).— Id. sections 3.5.4, 4.5.4.[1] However, the specification further stated that –[a]ny sight placed on the weapon shall not degrade the shooters current level of accuracy—; according to the specification, –[i]f a weapon is accurate to 1 MOA accuracy, then with all other factors, environment, shooter, ammunition, etc., factored in, the shooter shall be able to maintain that level of accuracy or whatever accuracy he can attain with his current scope.— Id.As for weapons shock, the performance specification provided as follows:

The Sight in its operational configuration, shall not be damaged nor exhibit any degradation in performance when subjected to five groups of five rounds each. The Sight in its operational configuration, shall not be damaged nor exhibit any degradation in performance when subjected to a total of 300 rounds of equivalent shock on the .50 caliber sniper rifles. Equivalent shock is equal to [an] average peak acceleration height of 4000gs for a mean duration of 1 millisecond half sine wave.

Id. sections 3.5.15, 4.5.10.

The solicitation required offerors to submit two sample sights –representative of production ready systems,— and provided that –[t]he Government will test the samples requested and evaluate them for compliance with the Performance Specifications and Specification Matrix.— RFP at 17. Award was to be made to the responsible offeror whose proposal was determined to represent the –best value— to the government based on three evaluation factors: (1) technical, including technical approach and management approach; (2) past performance; and (3) price. The technical approach subfactor included consideration not only of the extent to which the product sample met the performance specifications, but also of the extent to which the overall proposal demonstrated that the proposed night sight enhances the effectiveness of military units under a spectrum of operational conditions. RFP at 22. The technical evaluation factor was significantly more important than past performance, which was significantly more important than price.

KAC and OSTI submitted proposals by the initial closing time. The agency then conducted discussions with the offerors and requested revised proposals. When award subsequently was made to KAC, on May 17, OSTI protested to our Office, alleging (among other things) that the agency had failed to test its sample sights in accordance with the requirements of the solicitation. In response, SOC undertook corrective action, opening discussions with KAC and OSTI, and requesting revised proposals. In its revised proposal, OSTI proposed its MUNS 911M night sight, and also proposed its MUNS 911XR sight; KAC proposed its UNS LR-LP sight. The prior product samples having been returned, the offerors submitted new product samples.

During the Naval Surface Warfare Center (NSWC) Crane's August testing of OSTI's sample MUNS 911M night sights, after the firing of 10 rounds by an XM107 .50 caliber rifle and 10 rounds by an Mk15 .50 caliber rifle, one of OSTI's two sights (serial number (S/N) 0060) sustained damage in the form of a crack at a notch at the bottom of the objective lens at the front of the sight. When the agency then resumed testing with the other OSTI MUNS 911M sample (S/N 0061), that sight sustained damage in the form of a shattered image intensification tube after a total of 89 .50 caliber rounds were fired by the XM107 and Mk15 rifles. Since neither sample sight satisfied the performance specification requirement that the sight not suffer any damage when subjected to the firing of 300 rounds by a .50 caliber rifle, the MUNS 911M was rated unacceptable under the technical factor. Likewise, during testing of one of OSTI's sample MUNS 911XR sights, the sight introduced an approximately 2.7 MOA shift with the Mk15 rifle and up to a 4.1 MOA shift with the XM107 rifle between the groups of rounds fired with the dayscope and the groups fired when the dayscope and night sight were used in combination, thereby failing to meet the performance specification requirement that the current level of accuracy not be degraded by addition of the night scope. As a result, the MUNS 911XR also was determined to be unacceptable under the technical factor. In contrast, KAC's UNS LR-LP sight was determined to be technically acceptable. Inasmuch as KAC's proposal was rated low risk under the past performance factor, and its price was evaluated as fair and reasonable, KAC's proposal, the only acceptable proposal, was determined to offer the best value to the government. Upon learning of the resulting award to KAC, and after being debriefed, OSTI filed this protest with our Office.

SAMPLE TESTING

OSTI challenges the evaluation of its sample items as unacceptable on the basis that the agency's testing was conducted improperly in that the testers failed to mount the sights properly. In this regard, OSTI furnished its sights to the agency mounted atop a KAC mount that had been modified by OSTI. The KAC mount was to be clipped onto the mounting rail on top of the rifle using a single mounting lever, followed by adjustment using a pair of adjustment screws to ensure a tight (but not too tight) fit, and tightening of a pair of locking (jam) screws to ensure that the adjustment screws would not come loose or back out under the significant recoil forces experienced during the firing of the .50 caliber rifle. Specifically, according to the laminated directions sheet furnished with the sights, in order –to install and lock— the sights onto the mounting rail atop the rifle, the shooter was to –push lever flat against mount base until it clicks,— and then –adjust hex head screws to make sure the base is securely seated and tightened down onto the rail. (See adjustment instructions for base).— The referenced adjustment instructions panel on the laminated sheet read as follows:

Adjusting MountBase. (Mount should be adjusted for each weapon it is placed on.) To tighten the mount first loosen the two button head Allen [adjustment] screws. Turn the set [locking] screws on the opposite side of the mount clockwise slightly (making sure to turn in the set screws an equal amount). Retighten the two button head Allen screws. To loosen mount loosen the two button head Allen screws. Turn the set screws on the opposite side of the mount slightly counter-clockwise (making sure to turn out the set screws an equal amount). Retighten the two button head screws. Test mount on the rail to make sure adjustment is correct. MOUNT SHOULD BE VERY TIGHT ON RAIL.

OSTI Laminated Directions Sheet; see Hearing Transcript (Tr.) I-149 to I-154.

According to the testimony of OSTI's vice president for technology (and co'owner) at the hearing conducted by our Office in this matter, and as confirmed by the agency's night vision sight technical expert, in the event the mount was loose on the rail, the mount and sight could undergo a pitching or rocking motion during firing, with the front of the mount and sight rocking forward and down towards the mounting rail and then rocking back and upward. Tr. at I-153, II-307, II-345. The protester's vice president noted in this regard that the sight could appear to be tight on the rail if tested by attempting to move the sight back and forth, but still not be tight enough to be securely mounted. As support for the possibility that the sights were not securely mounted on the rail, the vice president testified that a locking set screw was missing on the tested MUNS 911XR sample sight returned at the conclusion of the procurement and that there did not appear to be the amount of wear on the heads of the mount screws on the returned sights that would be expected had the screws been repeatedly adjusted. The vice president concluded that the most likely cause of the up to 4.1 MOA shift in OSTI's MUNS 911XR sample sight was either the sight striking the mounting rail, or simply severe whiplash from the rocking motion during firing, causing an internal element of the sight to move. Similarly, according to the vice president, the most likely cause of the cracking of the objective lens on one of the sample MUNS 911M night sights and the shattering of the image intensifier tube on the other was either a rail strike or severe whiplash. Tr. at II-307 to II'310, II-321, II-336 to II'359. Since, according to the vice president, the night sight will not come loose during firing if properly mounted, he concluded that the mounting screws were improperly adjusted. Id.Specifically, as OSTI stated in its post-hearing comments,

[d]espite the instructions provided by OSTI regarding the need for adjustments each time the nightsight is mounted onto a rail for the first time, the record demonstrates that the adjustments were not made as required by OSTI's instructions. As a result, OSTI's nightsights suffered physical damage and a degradation of performance, which caused the Army to exclude OSTI's proposal from final consideration for award.

OSTI Comments, Feb. 17, 2006, at 35.

Our Office will review an allegedly improper technical evaluation of product samples to determine whether the evaluation was fair, reasonable, and consistent with the evaluation criteria. We will not make an independent determination of the merits of an offeror's proposal; rather, we will review the evaluation record, including the results of any test demonstration, to ensure that the agency's technical judgment has a rational basis and is consistent with the stated evaluation criteria. USIA Underwater Equip. Sales Corp., B'292827.2, Jan. 30, 2004, 2004 CPD para. 32 at 3; Sun Chem. Corp., B-288466 et al., Oct. 17, 2001, 2001 CPD para. 185 at 7.

We find no basis for concluding that the agency unreasonably failed to ensure that the night sights were securely mounted during testing. The XM107 and Mk15 .50 caliber sniper rifles used for testing were equipped with MIL-STD 1913 mounting rails, as specified in the performance specification, and the tests were conducted by experienced weapons testers and/or snipers, including (1) an engineer who served as the agency's technical expert for night vision weapons sights for special operations forces, (2) an experienced weapons test engineer, and (3) a retired Navy SEAL sniper. The agency's night vision sights expert testified that he initially mounted the sights on the rifles and adjusted them; he followed OSTI's written directions on the laminated sheet when mounting OSTI's sights and instructed the other two testers in how to mount the sights in accordance with OSTI's directions; the testers had OSTI's laminated directions card available next to the rifles and consulted it; he personally observed the other testers most of the time and saw them making adjustments consistent with OSTI's directions during the daytime shooting; and the screws on the mounts were adjusted when switching the sights to a new weapon or when the screws appeared loose. Although this expert stated he was unable to directly observe the adjustments made by the tester during the night accuracy shooting, he testified that this did not occur until the third day of testing, at which point the designated night shooter, the retired Navy SEAL sniper, was comfortable with the process. Tr. at I-156 to I-165, I-185 to I-200, I-248 to I-250. In any case, the record indicates that the damage to OSTI's MUNS 911M sights occurred during the preceding daytime shooting, not during the nighttime shooting on the third day of testing. Declaration of Agency Night Vision Weapons Sights Expert, Mar. 8, 2006; see VAS Night Vision Devices Sample Test Report sect. 3.3.4.2, MUNS Test Data Sheets. Thus, contrary to OSTI's position, eyewitness testimony supports the view that the agency's testers, all of whom were experienced, reasonably attempted to perform the adjustments in accordance with OSTI's written directions.

The agency asserts that the damage to OSTI's sights and degredation in their performance more likely was caused by the unique characteristics and design of the sights, rather than by any testing errors. The record supports this view. In this regard, we note that the contemporanous records of the testing, as well as the declarations and testimony of the agency's night vision sights expert, indicate that the screws on the mounts of KAC's sights came loose as frequently as the screws on OSTI's mounts. Tr. at I-217. However, damage and degraded performance were experienced by OSTI's sights but not by KAC's sights. This appears especially significant in light of the fact that the mounts on KAC's sights were adjusted less frequently than those on OSTI's sights; KAC's mounts were adjusted only when they appeared loose, not also when the sights were transferred between rifles, as was necessary with OSTI's sights. Tr. at I-218, I-254 to I-257, I-281 to I'282.

Further, by its own statements OSTI has essentially conceded that its design may have been responsible for damage to the MUNS 911M sights. Specifically, the MUNS 911M sight, which has a larger diameter objective lens than either OSTI's MUNS 911XR sight or the KAC sight, has a notch cut into the base of the lens that is contoured to allow the sight to fit closely around the MIL-STD 1913 mounting rail. The XM107 rifle, meanwhile, has a pop-up iron sight that, when raised, is directly in front of the objective lens. The agency's night vision sights expert testified that he believed that the objective lens on the MUNS 911M sight (S/N 0060) became cracked at the notch when the lens struck the iron sight, which was folded down during firing. Tr. at I-230, I-284 to I'287; Declaration of Agency Night Vision Sights Technical Expert, Feb. 24, 2006. [2] In a January 9, 2004 memorandum to agency personnel, the subject of which was –MUNS Prototype Testing,— OSTI's vice president recognized the risk posed by the front sight on the XM107, noting that it was necessary –to be careful that the units do not hit the front sight on the [XM107]. This can occur if the scope is mounted too far forward. It does not seem to be a problem when the sight is up.— This memorandum lends support to the agency's expert's opinion regarding the cause of the damage to the MUNS 911M sight.

OSTI asserts that the agency's decision to test the MUNS 911M with the sight folded down was unreasonable, stating its position as follows:

Nothing in the Solicitation mentioned the fact that the offerors should be prepared for additional accessories to be located between the rail and the proposed sight. Based on these representations, OSTI offered the MUNS 911M, which allowed enough space between the notched lens and the rail to permit limited pitching of the nightsight without striking the rail. OSTI did not expect or plan for enough space for additional equipment to be introduced between the nightsight and the rail.

OSTI Comments, Feb. 17, 2006, at 46-47. As noted by the agency, however, front and rear pop-up/fold-down iron sights are a permanent part of the XM107 rifle, as specified in the official configuration for that weapon. Further, the agency tested the sights with the iron sight folded down, since it is positioned directly in front of the night sight's objective lens, and thus would obscure part of the lens if left in the raised position, thereby reducing the amount of incoming light and the resulting image quality. Tr. at I-289 to I-290, Declaration of Agency Night Vision Sights Technical Expert, Feb. 24, 2006. We see nothing unreasonable in the agency's testing methodology with regard to the iron sight.[3]

We conclude that the agency undertook reasonable efforts to ensure that OSTI's night sights were securely mounted and properly adjusted during testing. Even if these efforts were not entirely successful (again, there is no basis in the record for reaching such a conclusion), we think the agency could reasonably conclude that the susceptibility of OSTI's sights to significant damage and degraded performance, notwithstanding reasonable efforts to mount the sights correctly, rendered the sights technically unacceptable. Certainly, we see nothing unreasonable in the agency's determining that the sights did not meet the agency's stated need for a rugged, reliable and accurate night vision sight that could be mounted on a special forces sniper's rifle at night and without detectable noise and light emissions. Performance Specification sections 3.5.3, 4.5.4, 4.5.10, 4.6.1, 4.6.2.[4]

MATERIAL MISREPRESENTATION

OSTI asserts that, during discussions, KAC made material misrepresentations in responding to the agency's expressed concerns regarding reported past performance problems. In this regard, in a July 21, 2005 discussion letter to KAC, the agency noted that there had been performance problems under two prior (NSWC Crane) KAC contracts--N00164-02-D-8506 (8506) (night vision sights) and N00164-02-D-8512 (8512) (night vision sights plus dayscopes). Specifically, the agency noted that there had been intellectual property disputes with subcontractors under 8506 (including OSTI, its primary subcontractor for the night vision sights), delivery problems due both to these disputes and to a lack of image intensifier tubes (manufactured by another company), rejection of some units on account of quality or configuration concerns, and late delivery of logistics documentation and status reports. In KAC's proposal as revised, KAC explained the measures it had taken to ensure that there would be no problems under the contemplated contract, including ensuring a supply of image intensifier tubes and lens, reducing dependence on subcontractors, ensuring that KAC has ownership of the required intellectual property rights, negotiating agreements with proposed subcontractors to preclude the reoccurrence of the issues encountered with the subcontractors under the prior contracts, organizing a night vision division with oversight by a newly hired KAC vice president with significant experience with rifle-mounted night vision and thermal products, hiring additional personnel to manage logistics support and reporting, and obtaining a recommendation for approval for ISO 9001:2000 certification. KAC Revised Proposal, Discussions Response at 6-14, App. A, E, I, AF.

However, as noted by the protester, in addition to explaining that changed circumstances made a reccurrence of the prior problems unlikely, KAC also responded in its August submission as follows:

Most issues for nonperformance of the contract on 8512 and 8506 lie squarely at the feet of the subcontractor [OSTI]. In an attempt to sell the product direct to the government, the subcontractor initially refused to honor its verbal agreements with KAC resulting in problems on 8512. The subcontractor subsequently raised the price to KAC causing great financial loss to KAC as KAC continued to buy the product and resell it to the government at the contract price.

KAC Discussions Response, Aug. 3, 2005, at 10. OSTI asserts that this statement included several misrepresentations: OSTI could not be blamed for most of KAC's performance problems, OSTI did not act inconsistently with its oral agreement with KAC, and OSTI's price increase to KAC was not unreasonable.

An offeror's material misrepresentation in its proposal can provide a basis for disqualification of the proposal and cancellation of a contract award based upon the proposal. A misrepresentation is material where the agency relied upon it and it likely had a significant impact on the evaluation. Greenleaf Constr. Co., Inc., B'293105.18, B-293105.19, Jan. 17, 2006, 2006 CPD para. __ at 4; Integration Techs. Group, Inc., B'291657, Feb. 13, 2003, 2003 CPD para. 55 at 2-3.

We need not consider whether the disputed statements constitute misrepresentations, since we agree with the agency that there is no basis in the record for finding that the agency relied upon the statements such that they had a significant impact on the evaluation. As noted by the agency, the record indicates that contracting officials carefully researched KAC's past performance, including conducting interviews with contracting officers, program managers and logisticians familiar with KAC's prior contracts, and reviewing the Contractor Performance Assessment Reports (CPAR) for the contracts. While the CPARs documented the problems encountered under the contracts, they also documented KAC's correction of many of these problems. The final CPAR for 8506 indicated that the assessing official –definitely would award— to KAC again, and the initial CPAR for 8512 indicated that the assessing official –probably would award— to KAC again. In addition, in evaluating KAC's proposal as low risk, the agency took into account the further measures, as discussed in KAC's proposal, that KAC undertook to preclude a reccurrence of the problems encountered under the prior contracts. Source Selection Decision at 1; Source Selection Evaluation Final Report at 10; Tr. at I-21 to I-37, I-48, I-54, I-77 to I-80, I-83. We conclude that there is no basis for finding that the statements in question had a significant impact upon the evaluation of KAC's proposal.

OSTI asserts that KAC also misrepresented the role to be played in contract performance by a proposed subcontractor, Optics 1. In this regard, in response to the agency's notice during discussions that KAC's proposal failed to state what roles the contractor and subcontractors would serve in the production of the sights, KAC responded on March 8 that its proposed subcontractor –Optics 1 will be tasked with assembly and test of the optical subassemblies.— KAC Discussions Response, Mar. 8, 2005. However, KACs vice president in charge of its weapons sight program testified during the hearing in this matter that Optics 1 –is a good optics facility that I use on various programs,— but that it is one that he nevertheless –do[es] not currently intend to use— on this program. Tr. at I-356 to I'357. OSTI asserts that the testimony of KAC's vice president establishes that KAC misrepresented in its proposal Optics 1's expected role in contract performance.

OSTI's argument is without merit. KAC's vice president testified that KAC intended as early as May 2004 to develop an in-house capability for optics assembly, but that it was not able to equip the necessary space and hire and train the necessary personnel to manufacture its sight in-house until the fall of 2005. Thus, when KAC submitted its proposal in January 2005, it fully intended to use Optics 1 for optics assembly, consistent with its proposal. Tr. at II-425 to II-438. In any case, KAC's final revised proposal, submitted in August 2005, advised the agency of KAC's intentions in this regard. Specifically, the proposal stated that

it must be noted that KAC is only minimally dependent on subcontractors for successfully meeting the requirements of this solicitation. As previously stated, the mechanical design was done at KAC, the optical design was purchased, the parts are manufactured at KAC and currently assembled and tested at KAC. Tubes and lenses are currently 7 months ahead of the delivery schedule required by this solicitation if the order was placed today.

KAC Revised Proposal, Aug. 3, 2005, at 13. While it appears that the agency may not have fully understood KAC's plans with respect to Optics 1, Tr. at I-372, the testimony of the SSEB chairman indicates that the agency did generally understand that –KAC was taking on a bigger role in the manufacturing of this sight.— Tr. at I-375. At any rate, given the statement in its August 2005 final proposal that –the parts are

manufactured at KAC and currently assembled and tested at KAC,— there is no basis for concluding that KAC misrepresented its intention to assemble the parts in-house.[5]

The protest is denied.

Anthony H. Gamboa

General Counsel