IBM Corporation, U.S. Federal
B-406934,B-406934.2,B-406934.3: Oct 3, 2012
- Full Report:
IBM Corporation, U.S. Federal, protests the award of a contract to HP Enterprise Services, LLC under request for proposals (RFP) No. VA118A-12-RP-0118, issued by the Department of Veterans Affairs (VA) for a real time location system (RTLS). IBM maintains that the agency misevaluated proposals and made an irrational source selection decision.
We sustain the protest.
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.
Matter of: IBM Corporation, U.S. Federal
File: B-406934; B-406934.2; B-406934.3
Date: October 3, 2012
Protest that agency misevaluated proposals and made an unreasonable source selection decision is sustained where record shows that agencys evaluation and source selection decision--which relied principally on two broad technical discriminators--were unreasonable and based on erroneous evaluation results.
IBM Corporation, U.S. Federal, protests the award of a contract to HP Enterprise Services, LLC under request for proposals (RFP) No. VA118A-12-RP-0118, issued by the Department of Veterans Affairs (VA) for a real time location system (RTLS). IBM maintains that the agency misevaluated proposals and made an irrational source selection decision.
We sustain the protest.
The RFP contemplates the award of a fixed-price, indefinite-delivery/indefinite-quantity contract for a 5-year ordering period with a minimum guaranteed value of $20,000 and a maximum value of $543 million. RFP at 7, 12. The successful contractor will be responsible for furnishing all hardware, software, documentation and incidental services necessary to provide the VA with a complete solution to the agencys requirement for an RTLS.
Broadly speaking, an RTLS is an integrated system of hardware and software products for tracking assets, personnel and patients throughout VA facilities in real time. The system is comprised of: (1) badges, tags and labels affixed to assets, personnel and patients; (2) reader/locator technologies (including WiFi technology, alternative wireless technologies, barcode reader technology, as well as ultrasound and infrared reader/detection technology); (3) data collection and aggregation hardware (principally data server and storage components); and (4) software components that enable and integrate the various hardware components of the system. The systems software also is required to integrate the RTLS system with various legacy and third-party systems currently used by the agency for data collection and dissemination.
The overall objective of the RTLS acquisition is to enable VA to track assets, personnel and patients on a real time basis throughout the VA system of medical facilities in order to provide the VA with total asset visibility. The RTLS also will provide the VA with information relating to medical and business processes that will help the VA improve the efficiency and efficacy of those processes.
The RFP advised offerors that the agency would evaluate proposals and make award on a best value basis, considering price and several non-price factors. RFP at 50. The four non-price factors were: technical, past performance, veterans involvement, and small business participation commitment. Id. Technical was deemed significantly more important than past performance; past performance was deemed slightly more important than veterans involvement; veterans involvement was deemed slightly more important than small business participation commitment; and small business participation commitment was deemed slightly more important than price. Id. Firms were advised that the non-price factors were, collectively, significantly more important than price. Id.
The technical factor was further divided among three subfactors, which were listed in descending order of importance; each subfactor was described as slightly more important than the one beneath it. The subfactors were: (1) task order execution (which included two equally-weighted task orders, the veterans integrated service network (VISN) 23 task order, and the enterprise system engineering (ESE) task order); (2) project execution; and, (3) sample task (which included two sample tasks listed in descending order of importance, medical center A and national data repository). RFP at 51. All of the issues in this protest concern the propriety of the agencys evaluation of proposals under the technical factor.
In response to the solicitation, the agency received six proposals, three of which--the awardees, the protesters and a third offerors--were included in the competitive range. The agency engaged in several rounds of discussions with the competitive range offerors and each firm was invited to demonstrate its product. At the conclusion of these activities, all three firms submitted final proposal revisions. The agency evaluated the final proposals and arrived at the following evaluation results for HP and IBM:
A. Task Order Execution
B. Project Execution
C. Sample Tasks
Small Business Participation
AR, exh. 3, at 2. On the basis of these evaluation results, the agency made award to HP, finding principally that two particular features of its proposed RTLS solution merited the cost premium associated with its proposal. Specifically, the agency valued HPs offer of something the agency termed single active tag technology, and its offer of an open system architecture that is infrastructure and tag agnostic. (We discuss both of these findings in detail below.) After learning of the agencys award decision, and requesting and receiving a debriefing from the agency, IBM filed this protest.
IBM has raised a large number of allegations (many of which were withdrawn during the course of the protest), but its arguments, in the main, focus on the two areas where the agency perceived greater value in HPs proposal--HPs use of single active tag technology and its use of an open system architecture that is infrastructure and tag agnostic. In these two areas, IBM maintains that the agency misevaluated the proposals because it concluded that only HP offered these two features when, in fact, the IBM proposal offered these two features as well.
IBM argues secondarily that the agency erred in evaluating the offerors respective autoclavable active tags and that the agencys selection decision contained fundamental errors that overstated the number of strengths found in the HP proposal and the number of weaknesses found in the IBM proposal.
We have carefully reviewed each of IBMs contentions. We sustain IBMs protest based on the issues discussed below, and deny the remainder of IBMs assertions. Our discussion focuses, first, on the two principal discriminators identified by the agency (single active tag technology and open system architecture that is infrastructure and tag agnostic) as meriting award to HP at a price higher than that offered by IBM. Secondly, we discuss the propriety of the agencys evaluation of the offerors respective autoclavable active tags and also the inaccuracies in the agencys selection decision.
We note at the outset that, in protests where there is a challenge to the propriety of the agencys evaluation of proposals, we do not independently evaluate proposals; rather, we review the agencys evaluation to ensure that it was reasonable and consistent with the solicitations evaluation scheme and applicable statutes and regulations. SOS Intl, Ltd., B-402558.3, B-402558.9, June 3, 2010, 2010 CPD ¶ 131 at 2. For the reasons discussed below, we conclude that the agency misevaluated IBMs proposal and, by extension, relied on unreasonable evaluation conclusions in its best value decision.
I. Single Active Tag Technology
IBM challenges the agencys evaluation of the proposals because the agency assigned a significant strength to the HP proposal for offering something the agency termed single active tag technology. IBM contends that it also offered this feature and the agency improperly did not also assign its proposal a significant strength.
The record shows that one of two principal discriminators between the HP and IBM proposals was HPs use of single active tag technology. As noted, the agencys evaluators assigned the HP proposal a significant strength for offering a single active tag technology solution, but did not identify this as a strength in the IBM proposal. The source selection authority (SSA) relied on this significant strength as a major discriminator between the two proposals.
As noted, the RFP calls for the RTLS to use a variety of tags, badges and labels. Some of these tags and badges are characterized as active because they dynamically interact with the RTLS system, either through the emission of a broadcast beacon signal, which, in turn, is detected by the facilitys WiFi network, or through association with a facilitys WiFi network. AR, exh. 11a, at 9. Some of the active tags also are required to interact with the facility using a supplemental technology--for example, ultrasound or infrared emissions detected by appropriate sensors--for purposes of increasing the accuracy of the tag from room-level accuracy to bed-level or bay-level accuracy. Id.
In addition to the WiFi and supplemental technology requirement, the RFP also specified that the contractor was required to provide an alternate infrastructure technology solution where WiFi was not available. The agency currently is in the process of installing WiFi networks at its facilities, but it has not yet completed this task; in other instances, there will be facilities where the agency does not intend to install a WiFi network (for example, in small facilities where the agency is located in leased space and does not want to invest heavily in permanent infrastructure). As a consequence, there are (or will be) some VA facilities that have a WiFi network installed, and others that do not. Because VA ultimately intends the RTLS to operate on an enterprise-wide basis, the RFP requires an alternate infrastructure technology solution. AR, exh. 11a, at 9, 32.
HP proposed a [deleted], and [deleted]. AR, exh. 18, at 19-21. According to the HP proposal, [deleted]. Id. at 19. The HP proposal also represents that its [deleted]. Id. In addition to this capability, the HP proposal represents that, [deleted]. Id. at 21.
The capability of the [deleted] is what the agency was referring to when it used the term single active tag technology, because the device can [deleted] using a single tag as opposed to using multiple types of tags, depending on the environment. As noted, the agency assigned the HP proposal a significant strength based on its offer of a single active tag technology solution, AR, exh. 19, at 9, and this strength ultimately was identified by the SSA as a major discriminator between the HP and IBM proposals. AR, exh. 3, at 4. IBM asserts that there was no basis to distinguish between the two proposals on this basis because it also proposed a single active tag technology solution that could operate [deleted]. IBM therefore maintains that it was unreasonable for the agency to discriminate between the proposals on this basis.
The agency maintains that IBMs proposal does not offer a complete non-WiFi solution. According to the agency, a review of the IBM proposal shows that IBM offered an array of tags manufactured by [deleted]. The agency contends that IBMs solution was incomplete because not all of the [deleted] include the [deleted].
The record supports IBMs position. In its proposal, IBM describes its solution for situations where there is an available WiFi network. IBMs proposal explains that its proposed [deleted] function in WiFi environments, and also achieve the enhanced accuracy required by the RFP by [deleted] (IBMs supplemental technology). AR, exh. 14a, at 21-23. IBMs proposal explains its solution for situations involving non-WiFi environments elsewhere. Specifically, IBMs proposal provides that its solution uses [deleted] where WiFi is not available in a facility. AR, exh. 14a, at 104. The IBM proposal provides as follows:
Id. The record therefore shows that, like HP, IBM offered a single active tag solution, and that its active tags will function in both WiFi and Non-WiFi (alternate infrastructure) environments.
IBM points out that, to the extent that the agency alleges that IBMs proposed solution is incomplete because not all of its proposed active tags have [deleted] capability, this assertion reflects a misunderstanding on the part of the agency regarding what was being offered by IBM. We agree.
The record shows that the use of [deleted] was proposed by IBM as the supplemental technology required by the RFP for enhanced (bed-level or bay-level) accuracy. AR, exh. 14a, at 21-23. As noted, however, this enhanced accuracy is not necessary for every active tag application, but only for certain applications. AR, exh. 11a, at 9. Thus, the record shows that IBM proposed that some--but not all--of its active tags would include the supplemental [deleted] technology. AR, exh. 14c i, at 1.
In contrast, the record shows that the [deleted] were proposed to satisfy the RFPs requirement for an alternate infrastructure (as opposed to supplemental) technology where WiFi is not available. AR, exh. 14a, at 104. IBMs proposal specifically represents that all of its tags (as well as other CCX compatible tags) will function in the [deleted] environment. Id. Accordingly, based on this record, we agree with IBM that its proposal--like HPs--includes a single active tag technology solution.
The agency maintains secondarily that the [deleted] tags were considered better by the evaluators because they offered more alternative technologies [deleted] compared to the [deleted] tags [deleted]. Again, the record does not support the agencys assertion. The evaluation record is silent on the number of technologies included in the HP tags. AR, exh. 19. Moreover, there is nothing in the record to show that the number of technologies included in the HP tags was compared in any way to the number of technologies included in the IBM proposed tags, or that the SSA even was aware of this difference between the two tags.
In light of these considerations, we sustain this aspect of IBMs protest.
II. Open Architecture and Agnosticism
As noted, the record shows that the other principal discriminator used by the agency to justify award of the contract to HP was the agencys perception that HP proposed a solution that involved open architecture that was infrastructure agnostic and tag agnostic, (open architecture, infrastructure agnostic and tag agnostic are defined and discussed in detail below). In contrast, the agency found that IBM offered a proprietary active tag and software solution. AR, exh. 3, at 4.
IBM asserts generally that the agency also erred in assigning the HP proposal a significant strength for offering what the agency termed an open architecture that was infrastructure and tag agnostic. As with the single active tag technology discriminator discussed above, IBM maintains that it proposed a solution that also offered an open architecture that was infrastructure and tag agnostic; IBM maintains that its proposed solution was in many respects similar to the solution offered by HP.
The agency maintains that it reasonably awarded the HP proposal a significant strength for this feature while not also assigning the IBM proposal a significant strength for the same reason. According to the agency, the open and agnostic nature of the proposed HP solution will allow the agency to evolve its RTLS through technology insertion at later points in time. In contrast, according to the agency, IBMs proposed solution will be limited in its ability to allow technology insertion at a later time because it employs what the agency describes as a proprietary  active tag and software solution.
We find that the record supports IBMs assertion, and that the agencys assignment of a significant strength only to the HP proposal for offering an open architecture system that was tag and infrastructure agnostic was unreasonable. We provide several brief definitions, and thereafter discuss: (1) the infrastructure agnosticism of the firms respective proposed solutions; and (2) the tag agnosticism of their respective solutions.
Although the RFP did not specifically define the term open architecture, all parties have made reference to section 5.6.2 of the basic RTLS performance work statement. That section provides: The Contractor shall provide Application Program Interfaces (APIs) to facilitate integration of the RTLS solution with other VA applications and third-party applications through open standards. AR, exh. 11a, at 39. The parties have focused on the concept of open standards, which are to be used to create APIs. Essentially, if open standards are used to create the APIs, new programs or applications can be integrated into the existing RTLS at a later time, simply by creating an API for the new program using the same open standards. Correspondingly, the RTLS can be integrated with all of the VAs existing legacy and third-party applications through the creation of APIs using open standards.
A related concept is agnosticism, which refers to the proposed RTLS solutions capability to interoperate with--or accommodate--different manufacturers tags or software products, and also refers to the RTLS solutions ability to be incorporated into the VAs existing infrastructure; the more agnostic or tolerant a solution, the more readily other products and legacy systems can be incorporated with it. See generally, http://whatis.techtarget.com/definition/agnostic.
B. Open Architecture Software Solution--Infrastructure Agnosticism
IBM maintains that the agency unreasonably assigned HPs proposal--but not IBMs proposal--a significant strength for offering an open architecture solution that is infrastructure agnostic. According to the protester, it also proposed an open architecture system that is as flexible as the system proposed by HP, and its proposed solution also will easily integrate into the existing VA legacy and third-party systems and allow for technology insertion in the future; the protester therefore maintains that its proposed solution is infrastructure agnostic.
As set forth below, the record supports IBMs assertion that it offered an open architecture platform that meets the RFPs requirement for integration with existing VA and third-party applications and, therefore, is infrastructure agnostic to use the agency evaluators phrase. Moreover, the record also shows that, as maintained by IBM, although both firms proposed different software solutions, their respective proposed system architectures appear comparatively similar. As discussed in detail below, we find that: (1) both firms proposed to comply with VAs development standards and protocols; (2) both firms proposed to use commercial off-the-shelf software platforms with an integrator software component to achieve interoperability; and (3) both firms used a standards-based common data model, along with proposing the use of existing VA data translation assets, to achieve intercommunication of the RTLS system with VAs legacy and third-party applications.
1. Compliance with VA Development Standards and Protocols
First, consistent with IBMs position, the record shows that both offerors proposed to develop their respective interfaces in a manner consistent with VA established standards and protocols. In the case of IBM, the firm proposed as follows:
AR, exh. 14a, at 45.
Similarly, HP proposed as follows:
AR, exh. 18, at 45.
Based on a reading of the two proposals, the record therefore shows that both firms proposed to comply with VA standards and protocols for the development of their respective solutions.
2. Use of Commercial-off-the-Shelf Software Product with an Integrator Software Element
Also consistent with IBMs position, the record shows that both firms offered a commercial-off-the-shelf (COTS) software solution for the RTLS and both firms proposed the development of interfaces that would be included in an intermediary software element. In the case of IBM, its proposed RTLS software solution is a COTS product known as [deleted]. AR, exh. 14a, at 8-9. [deleted]. As stated in the IBM proposal:
AR, exh. 14a, at 45.
Similar to IBM, the record shows that HP proposed an RTLS COTS product known as [deleted]. HPs proposal states:
AR, exh. 18, at 37.
The record therefore shows that both firms proposed a solution that involved use of an integrator or middleware software element to facilitate the interoperability of the RTLS system with existing VA legacy systems and third-party applications.
3. Standards Based Common Data Model and Use of Existing VA Data Translation Assets
Finally, consistent with IBMs position, the record shows that both firms also proposed to use a standards based, common data model strategy to translate data from different systems (VA legacy systems and third-party applications, along with the RTLS data) to allow for enterprise-wide interoperability, and also proposed to utilize preexisting VA data translation systems where appropriate. In the case of IBM, it proposed as follows:
AR, exh. 14a, at 45. IBM also proposed to use [deleted] where appropriate:
AR, exh. 14a, at 48.
In comparison, the HP proposal offers a similar strategy, combining a [deleted] where appropriate:
AR, exh. 18, at 36. The HP proposal goes on to describe how the [deleted] component uses the common data model:
Ar, exh. 18, at 37. In addition, like IBM, HP also proposed to use [deleted] where appropriate:
AR, exh. 18, at 38.
Our review of the record confirms that, consistent with IBMs contention, both firms proposed to use a standards-based, common data model strategy to translate data from different systems, and thereby achieve enterprise-wide interoperability.
4. Conclusion--Infrastructure Agnosticism
We conclude from a review of the record that both of the offerors proposed similar solutions--development of a common data model that functions with a middleware or integrator component in their proposed solution, along with leveraging existing data interchange assets where appropriate--to achieve interoperability among the RTLS and the various VA legacy and third-party systems. We therefore agree with IBMs contentions that both proposed solutions offer what may be described as an open architecture (able to mediate and accommodate language and protocol differences among disparate VA legacy and third-party applications) that is infrastructure agnostic (capable of being inserted into existing disparate VA legacy and third-party applications, and capable of accommodating future technology insertions).
We therefore conclude that the agencys evaluation finding--that the HP software solution proposed open architecture that is infrastructure agnostic while the IBM software solution did not offer these benefits--is not supported by the record.
C. Open Architecture Software Solution--Tag Agnosticism
As noted earlier, the VA also concluded that the HP software solution or platform was more advantageous than IBMs solution because it was tag agnostic. On this issue, IBM maintains that its solution is as tag agnostic as the HP solution, and that [deleted] solution does not demonstrate that its proposed solution is proprietary, or that it limits the agencys insertion of new technology, as noted by the SSA. We conclude that the record supports the protesters position.
1. Software-Agnostic-Tags Versus Tag-Agnostic-Systems
Before discussing this question in detail, we note that both the agency and the intervenor have used two distinct concepts interchangeably during the course of this protest, namely, the concept of software-agnostic-tags on the one hand, and the concept of tag-agnostic-systems (software platforms) on the other. The term software-agnostic-tags refers to the ability of a tag to function with various software platforms. The term tag-agnostic-systems refers to the ability of a software platform to function using different tags. This distinction is important because of the agencys evaluation and source selection findings. Specifically, the agency evaluators assigned the following significant strength to the HP proposal:
[deleted]. This increases flexibility as RTLS tag technology matures and reduces risk associated with interfacing different types of tags with proprietary systems. This increases the probability of successful integration [of] future use cases.
AR, exh. 19, at 2. The SSA, in turn, found as follows:
Offeror B's [HPs] open architecture and tag agnostic based RTLS solution is a major benefit to VA due to its ability to support growth and insertion of new hardware as technology changes and new RTLS use cases are identified and implemented over time.
AR, exh. 3, at 4. We conclude from these two quotations that both the evaluators and the SSA were focused on the benefits of tag-agnostic-systems (software platforms) rather than software-agnostic-tags; the object of the evaluators remarks was the [deleted], and the object of the SSAs observations is the RTLS solution. The SSA also focused on the fact that the benefit allows the insertion of new hardware; inasmuch as the [deleted] and the SSAs observation clearly is focused on the insertion of new hardware (e.g. tags) into the system, her focus clearly was on the benefits of HPs tag-agnostic-system, and not on any benefit that may arise because if its allegedly software-agnostic-tags.
2. Tag-Agnostic-Systems in Relation to Alternative Tags
The agencys defense on this issue essentially has been to argue that the HP proposal is superior because it offers software-agnostic-tags. In this regard, every reference cited by the agency (as well as the intervenor) in support of its position relates to HPs allegedly software-agnostic-tags as opposed to its tag-agnostic-system or software platform. Agency Supp. Report, Aug.23, 2012, at 20-21; see also HPs Comments on the Supp. Agency Report, Aug. 30, 2012, at 13.
We find the agencys defense on this issue largely inapposite. This is so because, as discussed above, the evaluators and SSA were focused on HPs tag-agnostic-system, rather than its allegedly software-agnostic-tags; any showing on the part of the agency relating to the agnosticism of the proposed HP tags is simply irrelevant to the evaluators and the SSAs conclusions.
Beyond the inapposite citations to the HP proposal referenced above, the agency has not identified any aspect of the HP proposal that distinguishes it from IBMs on the question of the relative benefit of its proposed tag-agnostic-system. In addition, the agency has not directed our attention to any portion of the evaluation and source selection record that explains or otherwise elaborates on the conclusory statements quoted above relating to the alleged superiority of HPs tag-agnostic-system. We therefore agree with IBMs assertion that the agencys evaluation conclusion--that the HP-proposed tag-agnostic-system is superior to that offered by IBM--appears to be unsupported by the record.
3. Tag Agnosticism and Total Solution
IBM asserts--and the record appears to show--that, although there may be no identified difference in the relative agnosticism of the offerors respective software platforms standing alone, there may be differences among the offerors proposed total solutions in terms of accommodating alternative tag technology.
As discussed above, the HP active tag solution relies principally on [deleted]. The HP solution relies on installation of the [deleted] in areas where no WiFi is present, and HPs proposed tags function essentially seamlessly between these two infrastructure environments [deleted].
The IBM solution relies on [deleted] in order to function in areas where there is no WiFi infrastructure. Where the [deleted] technology is installed, IBMs proposed tags also will function seamlessly [deleted].
Significantly, and as pointed out by IBM, there does appear to be a difference between the functionality of third-party tags in the two offerors proposed alternate infrastructure technology environments. In this regard, the IBM proposal specifically represents that, in its [deleted] environment, other [deleted]. AR, exh. 14a, at 104. The proposal goes on to represent that [deleted] in both WiFi and non-WiFi [deleted] environments. It therefore appears that, where a [deleted] is being used, it will function seamlessly in the IBM solution. In contrast, the HP proposal contains no similar representation respecting the functionality of other tags in its [deleted] environment.
4. Conclusion--Tag Agnosticism
We conclude that the record supports IBMs position that, overall, there is no basis to distinguish the HP and IBM proposed software platforms in terms of their comparative tag agnosticism, and that IBMs proposed [deleted] does not, standing alone, demonstrate that its solution is proprietary. Moreover, our review of IBMs arguments and the proposals leads us to agree that IBMs solution may, in fact, offer greater flexibility in terms of introducing alternative tags into its proposed total solution (hardware, software and alternate infrastructure solution).
In light of these considerations, we sustain this aspect of IBMs protest.
III. Additional Evaluation and Source Selection Decision Errors
A. Autoclavable Active Tags
IBM also maintains that the agencys evaluation of the offerors respective autoclavable active tags is based on factual inaccuracies relating to the rate of replacement for those tags, as well as the need to return these tags to the manufacturer for battery replacement. In this regard, the evaluators assigned a weakness to the IBM proposal for offering autoclavable active tags that, among other things, required replacement after only a limited number of cleaning cycles, and also did not allow for replacement of the batteries in the field, requiring instead that the tags be returned to the manufacturer for battery replacement. AR, exh. 5, at 3. The SSA, in turn, noted these considerations and contrasted IBMs proposed tags with HPs proposed tags, which she found had an advantage because they all had field replaceable batteries. AR, exh. 3, at 4. The SSA concluded her comparison by finding that the HP tags were superior because they would require replacement at a lower rate and could be serviced (have their batteries changed) in the field.
IBM maintains that both its autoclavable active tags and those offered by HP require return of the tags to the manufacturer for battery replacement. IBM also asserts that its autoclavable tags can withstand a greater number of sterilization cycles compared to those offered by HP. IBM therefore maintains that the agency erred in its conclusions relating to the comparative benefits or deficiencies associated with each firms proposed autoclavable active tags.
The record supports IBMs assertion that this issue may not provide a meaningful basis to distinguish the proposals. While IBM acknowledges that its proposed autoclavable active tags require that they be returned to the manufacturer for battery replacement, IBMs Third Supp. Protest, July 19, 2012, at 11, the agency concedes that the HP autoclavable tags also must be returned to the manufacturer for battery replacement. Supplemental Agency Report, Aug. 1, 2012, at 12 n.8. In addition, the record appears to support IBMs assertion that its autoclavable tags may have a lower replacement frequency than HPs autoclavable tags. In this connection, the record shows that IBMs proposed tags will withstand a greater number of steam sterilization cycles, as compared to HPs proposed tags; IBMs tags are able to withstand [deleted] steam sterilization cycles, while HPs tags are able to withstand only [deleted] steam sterilization cycles. Compare, AR, exh. 8, at 12 with AR, exh. 21, at 213.
In light of these considerations, we conclude that the agency may have assigned the IBM proposal a weakness relating to its autoclavable active tags without identifying a similar weakness in the HP proposal.
B. Number of Strengths and Weaknesses Enumerated in the Selection Decision Document
As noted in the initial part of this decision, the record includes two selection decision documents, one prepared at the time of the agencys source selection and another prepared by the agency after IBM filed its protest. IBM maintains that the original selection decision improperly relied on an inaccurate and exaggerated characterization of the comparative number of strengths and weaknesses in the two proposals. IBM also contends that the new selection decision was prepared in the heat of litigation and cannot be relied on by the agency to repair problems with the initial selection document. The agency, on the other hand, maintains that neither the original nor the revised selection decision relied in any way on the quantitative difference between the number of strengths and weaknesses identified in the proposals, but, rather, the selection decision was based on the qualitative difference between the strengths and weaknesses identified in the proposals.
The record shows that the agencys original selection decision included an inaccurate qualitative characterization of the number of strengths and weaknesses for the IBM proposal as compared to the HP proposal. Specifically, the original selection decision provided that HP had a superior number of strengths as compared to IBM, and also had significantly fewer weaknesses as compared to IBM. AR, exh. 3, at 4. In fact, the evaluation record shows that the IBM proposal had more strengths than HPs proposal (IBM had 7 significant strengths and 6 strengths compared to HPs 5 significant strengths and 28 strengths) and that the two firms had a comparable number of weaknesses (IBM was assigned 9 weaknesses and 5 significant weaknesses, while HP was assigned 7 weaknesses and 5 significant weaknesses). AR, exhs, 4, 5, 19.
At its debriefing, IBM was provided information about the evaluation of its proposal and also a redacted copy of the agencys selection decision. One of IBMs original protest grounds was that the selection decision erroneously failed to give its proposal credit for having more strengths than HPs, and also erroneously identified its proposal as having significantly more weaknesses than HPs when, in fact, the two firms had a comparable number of weaknesses. Shortly after IBM filed its original protest, the agency prepared a new selection decision that changed the agencys characterization of the strengths and weaknesses assigned to the two firms proposals. The new selection decision provided: Although strengths were similar in number, Offeror B [HP] had qualitatively superior strengths as compared to Offeror C [IBM], particularly in the most important Technical Factor subfactor--Subfactor A, Task Order Execution--and one less weakness. AR, exh. 13, at 4.
We have long held that we accord greater weight to contemporaneous source selection materials rather than judgments, such as those embodied in the agencys second selection decision here, made in response to protest arguments. Boeing Sikorsky Aircraft Support, B-277263.2, B-277263.3, Sept. 29, 1997, 97-2 CPD ¶ 91 at 15.
Here, the agencys original selection decision relied on a factually inaccurate characterization of the relative number of strengths and weaknesses identified in the offerors respective proposals, and made no mention of the qualitative differences between the strengths in the HP proposal versus the strengths in the IBM proposal. Although the agency argues that the original, inaccurate quantitative characterization of the comparative strengths and weaknesses was not considered--or relied on--by the SSA in arriving at her selection decision, the first observation in the document relating to HP and IBM compares the number of strengths and weaknesses found in the two firms proposals. In addition, the document clearly identifies the different number of strengths and weaknesses as a basis for distinguishing between the offers. In contrast, the revised selection decisions reference to the qualitatively superior strengths of HPs versus IBMs proposal was never considered during the agencys original selection decision and was advanced by the agency only during the heat of litigation. As a result, our Office will give greater weight to the selection decision that was prepared contemporaneously with the selection decision. Boeing Sikorsky Aircraft Support, supra. We therefore sustain this aspect of IBMs protest.
In light of the foregoing discussion, we sustain IBMs protest. We recommend that the agency reevaluate proposals, engage in discussions should they prove necessary or advisable, and make a new source selection on the basis of the agencys reevaluation of current or revised proposals. We also recommend that the agency reimburse IBM the costs associated with filing and pursuing its protest, including reasonable attorneys fees. 4 C.F.R. § 21.8(d)(1) (2012). The protesters certified claim for costs, detailing the time expanded and costs incurred, must be submitted to the agency within 60 days after receipt of this decision. 4 C.F.R. § 21.8(f)(1).
The protest is sustained.
Lynn H. Gibson
 For the technical and small business participation commitment factors, the agency assigned adjectival ratings to the proposals of outstanding, good, acceptable, susceptible of being made acceptable, or unacceptable; for the past performance factor, the agency assigned adjectival ratings of high risk, moderate risk, low risk, or unknown risk; and for the veterans involvement factor, the agency assigned adjectival ratings of full credit, partial credit, some consideration, or no credit. Agency Report (AR), exh. 12, at 25-27.
 The RFP also contemplated something referred to as a structured product demonstration. Firms were advised that, if their proposal was deemed to be within the initial competitive range, they would be asked to demonstrate their product. The results of the demonstration were to be considered under the project execution subfactor. RFP at 52.
 Exhibit 3 is the agencys selection decision document . Subsequent to IBMs protest being filed, the agency prepared a second selection decision (AR, exh. 13) to correct inaccuracies in the first selection decision. We discuss the two documents below.
 In its report, the agency represents that it gave significant importance to this aspect of the HP solution, asserting that it is one of the primary reasons the SSA selected the HP proposal and that: It is difficult to overstate the importance of this Single Active Tag Technology. Supplemental Agency Report, Aug. 23, 2012, at 7.
 According to the HP proposal, the [deleted]. AR, exh. 18, at 147. There is nothing in the evaluation record relating to this [deleted] and it does not appear to have factored into the agencys assignment of a significant strength to the HP proposal for its single active tag technology.
 The agency maintains that this aspect of IBMs protest is untimely because IBM learned of it from the unredacted selection decision that was provided to IBMs outside counsel on July 10. We disagree. The phrase single active tag technology is not defined by the RFP and is not otherwise defined in the selection decision in a manner that would have apprised IBM of what the agency meant by the phrase. While the unredacted selection decision did make reference to HPs single active tag solution (and compare it to IBMs proposed solution that allegedly required a second active tag), it is clear that the VA did not effectively convey what it meant by the phrase single active tag technology in the unredacted selection decision; IBM only understood what the agency meant by the phrase single active tag technology after its outside counsel received and reviewed both the HP proposal and the agencys evaluation materials for the HP proposal. As evidence that IBM has been vigilant in pursuit of its protest rights, we note that IBM filed a supplemental protest within 10 days of its receipt of the unredacted selection decision alleging that HPs proposal of a single universal tag for all applications--as opposed to the multiple sizes and forms of tags required by the RFP--could not have met the RFPs requirements. Once IBMs outside counsel received the HP proposal and related evaluation materials, it became evident to IBM that its earlier protest was based on an incorrect factual predicate, and it timely filed a supplemental protest alleging that it, too, had offered a single active tag technology solution.
 The SSA found that the HP proposed solution was tag agnostic, AR, exh. 3, at 4, while the agency evaluators found, somewhat more expansively, that the HP proposed solution was tag and infrastructure agnostic and awarded the proposal a significant strength based on that conclusion. AR, exh. 19, at 2.
 As a preliminary matter, part of the SSAs conclusion about the difference in the agnosticism of HPs and IBMs approach was the SSAs view that IBM was offering a proprietary approach. Specifically, the SSA found that: In contrast to Offeror B [HP], Offeror C's [IBMs] solution includes a proprietary [deleted] solution which limits the flexibility of the system architecture to evolve through technology insertion. AR, exh. 3, at 4.
For the record, we note the agencys selection decision contains the only contemporaneous characterization of the IBM proposed solution as proprietary. While source selection officials properly may disagree with the findings and recommendations of the agencys evaluators, their conclusions must be reasonable, consistent with the solicitations evaluation scheme and adequately documented. Clark/Foulger-Pratt JV, B-406627, B-406627.2, July 23, 2012, 2012 CPD ¶ 213 at 13-14. Here, there is nothing in the agencys underlying evaluation record or other materials presented to the SSA discussing or supporting her conclusion, and the SSA provides no independent details or explanation for her finding. While it is true that IBM proposed [deleted], AR, exh. 14a, at 15-17, that fact, without more, does not demonstrate that the IBM solution is any more or less proprietary than the solution offered by HP, or that the IBM solution is, comparatively, less open or agnostic than the HP solution. We therefore have no basis on the record before us to find reasonable the SSAs conclusion that the IBM proposed solution is, as compared to the HP solution, proprietary.
 The agency also contends that this aspect of IBMs protest is untimely because IBM did not allege that it, too, proposed an open system architecture that was tag agnostic within 10 days of receiving the unredacted selection decision, and instead waited until it reviewed the HP proposal (along with the agencys evaluation materials relating to the HP proposal) before making this assertion. The record shows, however, that, within 10 days of receiving the unredacted selection decision, IBM alleged that the agency had used unstated evaluation criteria (because neither open architecture nor tag agnosticism were mentioned as requirements in the RFP--an allegation IBM subsequently withdrew). IBM also alleged that its proposed solution, in fact, also offered open architecture and could be used with different tags (IBM maintained that its proposal did not include an elaborate discussion of these points because of the RFPs page limitations, and because these were not express RFP requirements). IBM Supp. Protest, July 19, 2012, at 3-4. Given these contentions in IBMs July 19 protest letter, we conclude that this aspect of IBMs protest was timely.
 The solicitation included five different performance work statements, one for the RTLS, one for the VISN 23 task order, one for the ESE task order, one for the national data repository sample task, and one for the medical center A sample task. AR, exhs. 11a-11e.
 It follows that, the more agnostic a solution, the more catholic its adoption likely will be.
 To the extent that one of the proposals can be distinguished from the other, the IBM proposal appears to offer greater detail than the HP proposal because it specifically identifies the intermediate data language the firm will use, whereas the HP proposal merely refers generically to an unspecified canonical data model.
 This conclusion is reinforced by the fact that both HPs and IBMs RTLS solutions select a software platform [deleted], and there is nothing in the record to suggest that the agency intends to select a different RTLS software platform once it completes the current acquisition. Thus, the fact that HPs proposed tags may be software-agnostic-tags that could function with other RTLS software platforms is immaterial to the underlying reason the agency assigned a significant strength to the HP proposal.
 The citations to the HP proposal relied on by the agency in support of its position on this issue are to portions of HPs proposal that describe its structured product demonstration. In contrast to its defense on this issue, however, the agency has repeatedly argued that it was precluded from considering the structured product demonstration portion of the IBM proposal under technical subfactor A, task order execution. AR, July 25, 2012, at 57-61; Agency Supp. Report, Aug. 8, 2012, at 3-7. According to the agency, the RFP required that the results of the structured product demonstration be considered only in the agencys evaluation of proposals under technical subfactor B, project execution. RFP at 52. Here, the agency is using the proposal contents of the HP structured product demonstration (albeit ineffectively) to show why it assigned a significant strength to the HP proposal under technical subfactor A. (The agency also made repeated citations to the HP structured product demonstration portion of its proposal to demonstrate that its proposed system offered an open architecture solution. Agency Supp. Report, Aug. 23, 2012, at 19-20.) To the extent that the agency used the contents of the HP proposal in its evaluation in a manner that was inconsistent with the terms of the RFP, while not also affording IBM the same consideration, such action amounts to unequal treatment and provides an independent basis to sustain the protest. See The Clay Group, LLC, B-406647, B-406647.2, July 30, 2012, 2012 CPD ¶ 214 at 10.
 IBM also contends that, in evaluating the offerors proposed tags, the agency improperly used a life cycle cost evaluation. IBM asserts that a life cycle cost evaluation was never called for under the terms of the RFP and, therefore, the agency was required, at a minimum, to have raised this issue during discussions. While the record confirms IBMs contention in the limited sense that the selection decision used the phrase life cycle cost in relation to the offerors proposed tags, there is nothing in the record to show that the agency actually performed a life cycle cost evaluation or used the results of such an evaluation in its source selection. Rather, the SSA was simply using this phrase as a shorthand to highlight what she perceived as a potential technical consideration that could impact the cost of IBMs proposed solution. Consequently, the agency was under no obligation to discuss life cycle cost evaluation issues with the firms during the acquisition.
 There were other considerations that went into the agencys overall evaluation of the quality of the offerors proposed tags, for example ease of maintenance and overall battery life, and these considerations have not been shown by the protester to be incorrect. Nonetheless, to the extent that the agencys evaluation conclusions distinguished between the offerors tags based on whether they were required to be returned to the manufacturer for battery replacement, and the frequency of their replacement, the agencys evaluation conclusions appear to be inconsistent with the contents of the proposals.
 Two of IBMs significant strengths and seven of its strengths were assigned under the small business participation commitment factor. AR, exh. 4, at 53.
 IBM was assigned one of its weaknesses under the past performance factor. AR, exh. 4, at 40.