Matter of: Cincom Systems, Inc. File: B-275055; B-275055.2 Date: January 21, 1997 * Redacted Decicion

Matter of: Cincom Systems, Inc. File: B-275055; B-275055.2 Date: January 21, 1997 * Redacted Decicion

Protest that agency improperly conducted a cost realism analysis under solicitation providing for fixed-price contract is denied where solicitation provided for price realism analysis and cost of implementation analysis as part of the best value determination. Agency reasonably added costs of additional software necessary to utilize offeror's proposal based on agency's evaluation of information in offeror's proposal. In making its award determination, agency reasonably concluded that awardee's proposal, though technically equivalent to protester's proposal, provided more benefits and was less expensive than the protesters under appropriate cost of implementation analysis.

Attorneys

DECISION

Cincom Systems, Inc. protests the award of a contract to Western Data Systems (WDS) under request for proposals (RFP) No. M67004-96-R-0006, issued by the U.S. Marine Corps for commercial off-the-shelf (COTS) software for use in reparables management at Department of Defense (DOD) maintenance depots. Cincom challenges the agency's technical and price realism evaluations and contends that under a proper evaluation it is entitled to the award.

We deny the protest.

Department of Defense depots, operated by the Army, Air Force, Navy, and Marine Corps, are responsible for repair of the operating forces' tactical equipment and weapons. The COTS software, known as Manufacturing Resource Planning (MRP II) software, is to be a component system of the depot maintenance program. The depot maintenance program is comprised of automated information systems supporting depot unique business functions. The software will perform planning, scheduling and tracking functions for repair of depot maintenance reparable items. Under the terms of the RFP, the COTS system offered was to have been developed and tested by the vendor and capable of operating in the depot environment upon acquisition and installation. The RFP contemplated award of a firm, fixed-price, indefinite quantity contract including certain time and materials line items for a base year with 5 option years.

Proposals were evaluated on the basis of technical and price considerations. Proposals were first evaluated on the basis of certain screening factors which included features which the offered software was required to possess in order to be considered further in the evaluation. Only proposals meeting the screening factors were to be included in the detailed evaluation for technical merit. This detailed evaluation was based on six factors: functional capabilities (38 percent); technical capabilities (18 percent); past performance (18 percent); interface capabilities (15 percent); life cycle support (9 percent); and subcontracting plan goals (2 percent). Offerors whose proposals were in the competitive range were allowed to provide an operational capability demonstration (OCD) which was approximately one-half as important as the detailed evaluation. Proposals were scored numerically with corresponding colors in descending order of merit: "Blue," "Green," "Yellow," and "Red." Price was evaluated on the basis of all line items in both the basic and option periods. Offers also were evaluated for price realism to determine whether the goods and/or services could be furnished at the proposed prices.

Award was to be made to the offeror whose proposal offered the best value to the government determined by comparing the differences in the value of technical merit features with the associated cost to the government. In making the comparison, the government was more concerned with obtaining superior technical performance than with making an award at the lowest overall cost. To this end, the RFP provided for the conduct of a cost- benefit tradeoff analysis which would take into consideration seven listed factors including government implementation costs, ease of use, and additional functional capabilities.

Cincom, WDS, and a third offeror submitted proposals by the May 24, 1996, closing date. The source selection evaluation team (SSET) evaluated the offerors' written proposals and observed the OCDs for each offeror, conducted written discussions, obtained written responses, and evaluated the offerors' best and final offers (BAFO) resulting in the following relevant final ratings:

Offeror WDS Cincom

Functional (score/risk) [deleted] [deleted]

Technical [deleted] [deleted]

Past Performance [deleted] [deleted]

Interfaceability [deleted] [deleted]

Life Cycle Support [deleted] [deleted]

Subcontracting Plan [deleted] [deleted]

Technical Total [deleted] [deleted]

OCD [deleted] [deleted]

Overall [1] [deleted] [deleted]

BAFO Price $29,459,496 [deleted]

In addition to the final color ratings, the SSET compiled lists of each offeror's strengths, weaknesses, and risks, specifically identifying those denominated as key discriminators for use in the best value analysis. In making the best value determination, the SSET first considered the results of the final technical evaluation. The SSET then conducted a cost realism analysis to ensure that all line items had been evaluated and to determine and consider the total cost to the government for the offerors' proposals. The SSET determined that approximately [deleted] million for additional software should be added to WDS' price and approximately [deleted] million should be added to Cincom's price to reflect the necessary costs of implementing their respective proposed solutions.

The best value analysis then addressed the potential benefits of the competing offers based on the strengths and weaknesses identified as discriminators impacting the value of the proposed packages when implemented in the DOD repair environment. The SSET found that WDS' proposal represented three positive benefits and two negative benefits for an overall positive rating, while Cincom's proposal represented only one positive benefit and four negative benefits for an overall negative rating. Given the two offerors' comparable technical scores, WDS' lower total cost as adjusted for realism ($2.33 million less), and its overall positive benefit rating, the SSET recommended award to WDS as representing the best value. The source selection authority concurred, and the agency awarded the contract to WDS on September 27. After receiving a debriefing, Cincom filed this protest challenging the agency's realism analysis which effectively added costs to its fixed-price proposal.

Generally, cost realism (a measurement of the likely cost of performance in a cost reimbursement contract) is not a factor in the evaluation of proposals when a fixed-price contract is to be awarded since the government's liability is fixed, and the risk of cost escalation is borne by the contractor. PHP Healthcare Corp.; Sisters of Charity of the Incarnate Word, B-251799 et al., May 4, 1993, 93-1 CPD Para. 366. However, even where a contract is to be awarded on a fixed-price basis, a contracting agency may properly examine proposed prices for realism by performing a "should cost" analysis. OAO Corp., B-211803, July 17, 1984, 84-2 CPD Para. 54; Ocean Data Equip. Div. of Data Instruments, Inc. B-209776, Sept. 29, 1983, 83-2 CPD Para. 387. The depth of an agency's should cost or price realism analysis is a matter within the sound exercise of the agency's discretion. See Family Realty, B-247772, July 6, 1992, 92-2 CPD Para. 6.

Cincom first argues that nothing in the RFP allowed for the type of realism analysis which the agency conducted. In this regard, Cincom notes that the RFP provided simply that the total estimated prices for all line items would be added and that the realism analysis would simply compare the prices of products and services "proposed by the offerors" with other prices proposed in connection with this and prior procurements and sales lists. However, Cincom's analysis fails to acknowledge all of the cost realism provisions in the RFP.

Section M-4(e) provides that "Price realism may be determined by a comparison of proposed prices either to apparent costs, sales information, competitive prices, or a combination of any of the above." [Emphasis added.] Further, section M-2.F specifically provides for the conduct of a "cost-benefit trade off analysis." The first factor to be considered in that analysis is "government implementation costs estimated as a result of the offeror's proposed approach including costs associated with site preparation, data conversion, data loading, interface development and training." In practice, the SSET assessed what additional costs, if any, were necessary to implement the software proposed by the offerors, including the cost of third party software which was not included in the offers. The SSET based its determination on its evaluation of the proposals including each offeror's own advice as to what hardware and software the government would need to implement the proposed software.

In our view, the RFP plainly disclosed the agency's intent to conduct this type of "should cost" analysis. Calculating the cost of software needed to implement the proposed solutions is plainly within the parameters set out both in the price realism and cost-benefit analysis provisions. At a minimum, necessary software is relevant to the issues of site preparation and interface development, the cost of which the RFP clearly advised offerors would be considered in the cost-benefit analysis. Likewise, the cost of necessary additional software is reasonably included in an analysis of "apparent costs" compared to proposed prices. Because the RFP clearly disclosed the agency's intention to perform the analysis in question, Cincom's objection to the conduct of this analysis is without merit.

Cincom next argues that the agency improperly added costs for certain ORACLE software identified in its proposal. Cincom states that the software products, ORACLE "Distributed Database" and "Parallel Query," were clearly denominated only as "required for each concurrent user needing access to data across multiple servers." Cincom argues that since its solution is designed to operate on a single server, there is no need for access across multiple servers, thus eliminating any need for this software. We disagree.

The RFP required the MRP II software to meet certain performance specifications including support of 250 concurrent users with a minimum of 1,000 transactions per minute with a 1- to 2-second response time, while executing normal daytime batch run processes, if any, in background. The agency's evaluation identified weaknesses in Cincom's solution including complex and data intensive system screens and slow response time during the OCD. The evaluation also found higher risk of performance issues due to the proposed hardware configuration's placement of a greater workload on the server. Further, one of Cincom's customers had experienced significant run-time problems using the proposed software in a single server/single processor environment at a much lower performance level than that required by the RFP. In addition, the proposed MRP II software was required to operate on both Hewlett-Packard (HP) 9000 and Sun Sparc 2000 server platforms. Cincom's proposal included an HP9000 sizing guide "to assist the user in selecting the most appropriately sized hardware configuration for use with Cincom's [software] products." According to the guide, the largest server available uses twelve processors but only supports slightly more than half the daily transactions required by the RFP. [2] Thus, the agency concluded that at least two servers would be required to meet the minimum requirements.

To arrive at a realistic cost for each site, the agency added the cost of obtaining a single distributed data base software set for 250 concurrent users and 250 copies of the parallel query software. Since a cost realism analysis is a judgment function on the part of the contracting agency, our review is limited to a determination of whether an agency's cost evaluation was reasonably based and not arbitrary. General Research Corp., 70 Comp.Gen. 279 (1991), 91-1 CPD Para. 183; Science Applications Int'l Corp., B-238136.2, June 1, 1990, 90-1 CPD Para. 517. While the protester argues that it proposed use of multiple servers only as an option, we believe the agency's realism analysis was reasonably based. The technical evaluation, coupled with Cincom's own proposal sizing guide, make clear that multiple servers are required for it to meet the performance requirements. In this regard, we also note that the RFP indicates that the software solution must be compliant with the Defense Information Systems Agency's Technical Architecture Framework for Information Management (TAFIM). This means implementing the Defense Information Infrastructure/Command Operating Environment for multiple services with interoperability across multiple client/server platforms. The TAFIM requires the ability to have real access over multiple servers on a client-server basis. Since each server must be a client and each client a server, the use of multiple servers is implicit in the requirement. Once the agency determined that multiple servers are necessary, Cincom's own option language mandated that the agency have the software identified by Cincom as required "for each concurrent user needing access across multiple servers." Since the cost of that additional software was not included in Cincom's proposal, it was reasonable for the agency to consider that cost in determining the best value solution. Cincom's simple disagreement with the agency's judgment does not render this evaluation unreasonable. Medland Controls, Inc., B-255204; B-255204.3, Feb. 17, 1994, 94-1 CPD Para. 260. [3]

We reach the same conclusion regarding the agency's addition of software costs associated with Cincom's proposed Planner Toolkit product (Toolkit). During the OCD, Cincom demonstrated Toolkit, a graphical user interface (GUI). Cincom argues that this GUI is only in a beta version and was demonstrated solely to show the agency its capabilities. Cincom claims that it never intended to propose the Toolkit product and avers that it clearly explained this during the OCD. However, prior to the submission of BAFOs, the agency specifically asked about software needed to operate the Toolkit GUI. Cincom's written response stated that "both" the Toolkit and PowerUser GUIs "are included in the proposed application." [4] Based on this representation, the agency evaluated Cincom's proposal as including the Toolkit. In the same response, Cincom advised that the Toolkit would not run under the Windows programs currently in use at the depots but would require Windows 95 or Windows NT. Since the depots did not have sufficient copies of either software product, the agency added the cost of Windows 95, the less expensive alternative, for 1,000 workstations at each site. Since the protester itself clearly advised the agency that the Toolkit was part of its proposal, and acknowledged that it would not operate without appropriate Windows software, the agency reasonably included the cost of that software as part of the best value evaluation. [5]

Cincom also takes issue with the manner in which the agency calculated the cost of the necessary Windows software. In Cincom's view, the agency could have purchased upgrade software at a fraction of the cost, would not require as many copies of the software as it estimated, and failed to consider the cost benefits of using Windows 95 software. The agency explains that an upgrade is inadequate since the depots must continue to use current applications which use an earlier Windows program and will not run under Windows 95. Thus, the depots will have to use both programs. It requires an estimated 1,000 per site based on its need to have depot users able to run "what-if" simulations at workstations all across the depots. Further, since multiple Windows versions would be in operation, the potential benefits of the faster Windows 95 software would be negated and could degrade overall client performance due to the processing load involved. Under these circumstances, we find nothing objectionable in the agency's methodology.

Cincom finally challenges the best value determination, arguing that as the offeror with the highest technical score and lowest price, it was entitled to the award. Cincom is incorrect for two reasons. First, as explained above, the agency reasonably conducted a "should cost" realism analysis of the offerors' proposals as part of its best value determination. Under that analysis, the cost of additional software raised the cost associated with Cincom's proposal more than $2 million above that associated with WDS' proposal. Thus, for purposes of the best value analysis, Cincom's proposal did not offer the lowest price. Second, Cincom's proposal was not the highest technically rated. Section M makes clear that the evaluation would include technical factors, the OCD, and the best value analysis which would take into account additional factors as discriminators. The agency evaluated both proposals as technically equivalent prior to the best value analysis. In this regard, while Cincom's proposal was scored slightly higher on the technical factors evaluation, it was scored lower under the OCD. When the scores for both were combined, the proposals were determined to be equivalent with ratings of Green with Yellow (moderate) risk. The agency's determination that the proposals were equivalent is also supported by the fact that the total numerical scores were less than one-fifth of a point apart.

In the best value analysis the agency concluded WDS' proposal was superior in that it represented greater benefits. For example, while Cincom's proposal offered the best functional solution, its [deleted] and it was not yet available on the Sun Sparc platform. WDS' proposal was identified as "extremely" user friendly, the easiest to use and interface, and provided the best accounting of actual cost for repair. Its only discriminating weaknesses concerned the [deleted]. [6] In view of the equivalent total technical ratings, WDS' lower-evaluated "real" cost, and WDS' greater number of beneficial discriminators, we find nothing objectionable in the agency's conclusion that WDS' proposal represented the best value notwithstanding its slightly higher BAFO price. [7]

The protest is denied.

Comptroller General of the United States

1. In the final evaluation Cincom scored 4.36 on the technical evaluation and 1.97 on the OCD for a combined score of 6.33. WDS scored 3.82 on the technical evaluation and 2.32 on the OCD for a combined score of 6.14.

2. The RFP required 1000 transactions per minute. In an 8-hour day this would equal 480,000 transactions. The largest server could only do 254,016 transactions in 8 hours. Cincom claims that this transaction figure represented "control" transactions which could be made up of 30 or more individual queries. Thus, the sizing guide did not directly compare with the agency's performance requirement. We find this argument unpersuasive. Cincom's software is called "Control: Open" and the column from which the transaction figures were taken is entitled "Estimated # of Control: Open Trans. [8] hrs." Thus, it was reasonable for the agency to conclude that the transactions per 8 hours figures represented transactions comparable to those identified in the performance specification. In this regard, nowhere in the sizing guide does Cincom represent that the "Control: Open" transactions include some sort of bundled transactions.

3. In related arguments, Cincom contends that the agency should have added additional cost to WDS' proposal to cover the cost of missing, but necessary data extraction tools and the same ORACLE software associated with Cincom's proposal. With regard to the ORACLE software, the agency explains that it was not necessary to do so because the WDS system does not require the same software to meet the RFP's performance requirement. Further, to the extent additional ORACLE software would be needed for WDS' proposal, WDS' product is native to ORACLE meaning that all necessary ORACLE functions are built-in. With regard to the data extraction tools, an evaluator noted that there was no evidence of the tools in the proposal. However, the SSET chairman's final evaluation explains that further research, discussion responses, and the OCD found evidence to support WDS' ability to support the requirement. Accordingly, there was no need to add cost for this software.

4. In fact, Cincom concedes in its protest comments that its written response during discussions did not make clear its intention not to offer the Toolkit, and now notes that it would furnish this application at no cost to the agency.

5. The agency adds that Cincom's technical evaluation was enhanced by the presence of the Toolkit. Thus, if it were not offered, Cincom's technical score would decrease.

6. Cincom argues that these matters do not fall within the discriminators identified in the RFP. Although the discriminating factors identified by the agency do not necessarily quote the language of the discriminators, they clearly are reasonably included within them. For example, ease of use and ease of interface clearly fall within the discriminator "Ease of use including menu navigation, data entry . . . and response time.

7. We point out that there is significant doubt on this record as to whether Cincom's proposal is technically acceptable. Under the terms of the RFP, an offeror's proposed COTS software was required to operate on the Sun Sparc Center 2000 server platform. Offerors unable to meet this requirement, as well as others, were not to be evaluated further. At the time of its proposal and BAFO, Cincom's software would not operate on the Sun platform, but Cincom promised that it could do so by September 30, the projected award date. The agency allowed Cincom to remain in the competition notwithstanding its failure to meet the requirement. (WDS' initial proposal also was lacking in a particular screening area, which was corrected prior to award.) While Cincom now claims that it meets the Sun requirement, it never notified the agency that it had met this requirement. Further, although Cincom states that it has completed testing of its software on this platform, the agency never had an opportunity to subject it to an OCD on the Sun platform as it did the other offerors' software. In addition, a representative of Sun Microsystems, Inc. advises our Office that his company is unaware of any commercially available Cincom product available for use on Sun platforms. We note that at present even Cincom's own Internet homepage does not specifically identify Sun as one of the platforms on which its software will operate, nor is there any evidence that it has publicly announced this capability. Thus, it is not clear that the software meets the agency's requirements for COTS software.

DOCUMENT FOR PUBLIC RELEASE A protected decision was issued on the date below and was subject to a GAO Protective Order. This version has been redacted or approved by the parties involved for public release.