Matter of: Management Technology, Inc. File: B-257269.2 Date: November 8, 1994 *REDACTED VERSION[*]

Matter of: Management Technology, Inc. File: B-257269.2 Date: November 8, 1994 *REDACTED VERSION[*]

Protest that agency improperly relied on undisclosed criteria in technical evaluation of proposals is denied where matters considered in evaluation were reasonably related to the stated evaluation subfactors. Agency adequately documented evaluation where record contained detailed scoring; summary statements of evaluated strengths, weakness and risks; explanations of changes in best and final offer scoring; and post-protest amplification of areas of significant evaluated difference between proposals under each technical factor. New and independent grounds of protest concerning discussions are dismissed as untimely where the later-raised issues did not independently satisfy the timeliness requirements of the General Accounting Office's Bid Protest Regulations; extension of time for filing comments on agency report does not waive the timeliness requirements for filing bid protest. Protest that price/technical tradeoff was inadequately documented is denied where solicitation contained numerically weighted 70/30 technical/price formula, and thus, in effect, notified offerors that agency had predetermined tradeoff between technical and price factors; under these circumstances, since award decision was consistent with evaluation methodology set forth in solicitation, separate determination justifying payment of price premium was unnecessary.

Attorneys

DECISION

Management Technology, Inc. (MTI) protests the award of a contract to Pinkerton Computer Consultants, Inc. under request for proposals (RFP) No. S-FBOAD-93-R-0065, issued by the Department of State, Office of Foreign Buildings Operation (FBO) for services to manage a computer facility. MTI contends that the agency relied on undisclosed evaluation criteria, the evaluation was inadequately documented, discussions were inadequate, and the award decision was based on an inadequate price/technical tradeoff.

We deny the protest.

The RFP solicited fixed price (labor hours) and technical proposals for computer facilities management and operations, user support, and systems development for a base and 4 option years. The following technical evaluation factors, their available points, and weights were set forth in the RFP:

(1) Personnel Qualifications--600 points (40 percent); [1] (2) Understanding of Requirements and Proposed Approach--525 points (35 percent); (3) Offeror's Experience--300 points (20 percent); (4) Offeror's Commitment--75 points (5 percent).

Additionally, under each factor, the RFP listed subfactors and their available points.

Award was to be made to the responsible contractor whose proposal offered the greatest value to the government from both a cost and technical standpoint. In the case of substantially equal technical proposals, the determining factor for award was to be the proposed price. Award to other than the lowest acceptable offeror could be made, according to the RFP, if another proposal offered "significant technical advantages"; in such a situation, a determination was to be made as to whether the price premium was warranted by the technical advantages. In making this determination, the technical evaluation would be weighted approximately 70 percent, and price approximately 30 percent. The RFP provided for the use of certain calculations to "normalize" scores, whereby the technical proposal receiving the highest technical score would be awarded the maximum technical weight of 70 and the acceptable proposal receiving the lowest evaluated cost would be awarded the maximum cost weight of 30, while other acceptable proposals would receive proportionally less technical and cost weight. [2] The total evaluation score would be the sum of the technical weight and the cost weight.

Offers were received from five firms, including MTI and Pinkerton. After initial evaluation, all five proposals were included in the competitive range; written discussions were held and best and final offers (BAFO) were requested. After BAFO evaluation, Pinkerton's technical proposal was ranked first with a raw score of 1,379 (out of a total 1,500) available points, and MTI's was ranked second with a score of 1,236. [3] Pinkerton offered a BAFO price of $12,124,090 and MTI offered $9,993,493. Based on a determination of significant technical differences between the two proposals, the contracting officer applied the RFP's 70/30 technical/cost formula and calculations. Pinkerton's "normalized" total score was 94.60 (70 technical + 24.60 price) and MTI's was 93.00 (63 technical + 30 price). [4] On the basis of Pinkerton's high total score, award was made to the firm on March 10, 1994.

UNANNOUNCED CRITERIA

MTI complains that the evaluators improperly gave higher scores to proposals based on preferences not disclosed to offerors in the RFP evaluation criteria or otherwise discussed. At issue is the evaluation of two Personnel Qualifications subfactors--Local Area Networks (LANs) and Systems Development/Data Base Management Systems (DBMS)--where the firm's proposal was downgraded. MTI received 95 out of 125 available points (76 percent) under LANs and 81 out of 100 available points (81 percent) under Systems Development/DBMS. According to MTI, in the evaluation of these two subfactors, an undisclosed premium was placed on Banyan Vines LANs (operating system) experience under the first subfactor, and ORACLE version 7 (applications software) experience under the second subfactor. [5]

The agency acknowledges that Banyan Vines LAN operating system experience and the latest version of ORACLE, i.e., ORACLE 7, applications software experience were rated at higher levels than other LANs and DBMS experience. (In the case of ORACLE 7, it states that there was only a "modest" increase in scores for this latest version of ORACLE.) In this regard, the agency maintains that the eventual progression of FBO to Banyan Vines LANs and ORACLE were sufficiently disclosed in the RFP for offerors to reasonably expect their evaluation at a higher level.

A procuring agency may properly evaluate proposals based on considerations not specifically shown in the RFP to be evaluation criteria where the considerations are reasonably related to or encompassed by the stated criteria. Loral Western Dev. Labs., B-256066, May 5, 1994, 94-1 CPD Para. 295.

Here, the RFP clearly put offerors on notice of the potential evaluation benefits available from proposing Banyan Vines LAN and ORACLE experience. Under the Personnel Qualifications factor, qualifications of proposed personnel were to be evaluated to assess the knowledge, skills and abilities of the proposed staff in the areas required for performance under the statement of work (SOW). Concerning the LANs subfactor, the RFP provided that "specific points" would be given for the following:

"LANs (Banyan VINES LANs or other LANs, LAN system administration, LAN installations, LAN gateways, LAN e-mail, LAN networks, LAN applications development, support of LAN-developed applications)."

The SOW further notified offerors concerning LANs that "each LAN server is currently configured with the following software: Banyan VINES 4.11 . . ., Banyan Mail, MailMan (Windows front-end to Banyan Mail . . ." and that FBO "will migrate from a Wang VS-based network architecture to a LAN-based architecture, standardizing on the Banyan VINES LAN operating system." SOW Sec. C.1.2.1.3. In this regard, the SOW notified offerors that "the [above-described] migration began in the fall of 1992 and is expected to continue through 1995." SOW Sec. C.1.2.1.3.

Concerning the System Development/DBMS subfactor, the RFP notified offerors that "specific points" would be given for the following:

"System Development/DBMS (ORACLE, PACE, COBOL, NETRON/CAP, ResponseR, large-scale LAN-based DBMS, other minicomputer or client/server DBMS, Wang glossaries)."

Also in this area, and concerning applications software, SOW Sec. C.1.2.4. generally provided that:

"FBO operates and maintains a number of application systems that were developed using COBOL, BASIC, and Netron, Inc.'s CAP (Computer Aided Program), which is a COBOL generator; [t]he relational database management systems (RDBMS) used for development are Wang's relational database management system PACE (Professional Application Creation Environment) and ResponseR and its related tools and utilities. . . ."

The SOW further specifically provided that:

"FBO has recently installed the ORACLE RDBMS and related tools on one of the VS minicomputers for applications development; [o]ver the next few years, existing VS applications will be redeveloped in ORACLE, and when all divisions within FBO have LAN access, the applications will be ported from the VSs to LAN servers (retaining the ORACLE implementation.)" SOW Sec. C.1.2.4.

Although the RFP did not specifically state that premium points would be awarded for these types of experience over the others listed, we believe that when the subfactors at issue are read in conjunction with the SOW description of these areas, specifically concerning FBO's migration to Banyan Vines LAN and its planned redevelopment of existing DBMS VS applications to ORACLE, sufficient information was imparted to offerors to put them on notice of the importance of these areas to FBO operations and that as a result they could be more highly rated than other areas of listed experience under the two subfactors at issue. On the evaluation of ORACLE, we further believe that it was only reasonable for offerors to assume that the latest version of this applications software could be more highly rated than an older version facing replacement. Based on these considerations, we believe that offerors were put on sufficient notice of the areas of evaluation. The evaluation in this area therefore was unobjectionable.

EVALUATION DOCUMENTATION

MTI complains that the written evaluation narratives for both the individual and consensus ratings are either nonexistent or so cursory that they fail to provide a reasonable basis for the evaluation and further prevented it from intelligently forming its arguments. In this regard, the protester maintains that even after consideration is given to the agency's post-protest explanations of the scoring, the lack of sufficient narrative comments delineating the rationale for the ratings fails to comply with the agency's own "Procurement Officer's Contracting Handbook," which provides, under the heading "Narrative Comments," that "[e]ach evaluator must provide narrative comments supporting the rating, for each evaluation criterion, on each proposal." [6]

While judgments concerning the evaluation of proposals are by their nature often subjective, they must be reasonable; such judgments must bear a rational relationship to the announced criteria upon which competing offers are evaluated. See Wadell Eng'g Corp., 60 Comp.Gen. 11 (1980), 80-2 CPD Para. 269. Implicit in the foregoing is that the rationale for these judgments must be documented in sufficient detail to show that they are not arbitrary and that there was a reasonable basis for the selection decision. See, e.g., TRW, Inc., 68 Comp.Gen. 511 (1989), 89-1 CPD Para. 584. Federal Acquisition Regulation (FAR) Sec. 15.608 requires documentation to support the evaluation of proposals, including the basis for evaluation and an analysis of the technically acceptable and unacceptable proposals, an assessment of each offeror's ability to accomplish the technical requirement, and a summary of findings. FAR Sec. 15.612(d)(2) requires supporting documentation for the source selection decision, stating the basis and reasons for the decision and showing the relative differences among proposals and their strengths, weaknesses, and risks in terms of the evaluation criteria. Where there is inadequate supporting rationale in the record for the source selection decision, we cannot conclude that the agency had a reasonable basis for the selection decision. See American President Lines, Ltd., B-236834.3, July 20, 1990, 90-2 CPD Para. 53.

We find that, considering both the contemporaneous evaluation record and post-protest amplifications, the agency adequately documented the evaluation here. While the protester is correct that generally the individual evaluator scoring sheets and consensus scoring sheets do not include narrative explanations in support of the scoring under each specific factor, subfactor, and element, this alone does not render the record inadequate for us to test the reasonableness of the agency's judgment. See Hydraudyne Sys. and Eng'g B.V., B-241236; B-241236.2, Jan. 30, 1991, 91-1 CPD Para. 88. The adequacy of the record is to be determined from all of the information provided, including the arguments of the parties. Id. Further, in this regard, the fact that the agency did not follow its own internal handbook is not dispositive, since an agency's internal policies and regulations provide guidance for agency personnel and do not in themselves provide outside parties with any legal rights. See Harvard Interiors Mfg. Co., B-247400, May 1, 1992, 92-1 CPD Para. 413; Loral Fairchild Corp.--Recon., B-242957.3, Dec. 9, 1991, 91-2 CPD Para. 524.

The contemporaneous initial documentation includes score sheets and narrative summaries. The evaluators individually scored each proposal using score sheets with detailed statements of the factors, subfactors, and subfactor elements which had been set forth in the RFP. [7] Under Personnel Qualifications, the most heavily weighted subfactor, the evaluators scored 28 labor categories of proposed personnel under the applicable 9 subfactors for each category. Each of the evaluators also made written narrative summary statements explaining or listing the strengths, weakness, and risks of each proposal. Additionally, some evaluators made handwritten notes either on the score sheets or on separate pages with further evaluation comments and questions, such as areas where a proposal was deficient and questions for discussion. The individual evaluator score sheets and narrative comments were consolidated into consensus score sheets and narrative summary statements of strengths, weaknesses, and risks for each proposal.

In the evaluation of BAFOs, changes in scoring were indicated on the individual evaluator score sheets by either a crossing out of the initial score and writing of the new score beside it, or by writing the additional points scored beside the initial score. Additionally, two of the evaluators made handwritten notes explaining the BAFO changes in scoring-- one evaluator made brief narrative explanations and the other made a handwritten matrix indicating the plus or minus BAFO score change under the personnel qualifications subfactors for the proposed individuals. After the individual BAFO scoring of proposals, the record indicates, the evaluators met and agreed to a "consolidated" technical score for each offeror's proposal. The record also includes BAFO consensus summary comments on each offeror's proposal explaining what newly submitted BAFO material was considered and its effect on scoring.

In addition to the contemporaneous evaluation documentation, the record includes a post-protest amplification of the scoring. In this regard, the technical evaluation panel (TEP) was reconvened to "further consolidat[e] the documentation relative to the significant technical advantages of [Pinkerton's proposal] over [MTI's proposal]." The TEP chairman has submitted a memorandum which details the differences in the scoring of the two proposals and Pinkerton's technical advantages considered significant under each of the four technical factors. [8] For example, under Personnel Qualifications, which constituted 40 percent of the technical evaluation, it is explained that with only two minor exceptions all three evaluators scored MTI's proposed staff lower than Pinkerton's in all nine of the evaluated subfactors. The TEP chairman notes that the key personnel proposed by Pinkerton were scored as much as 20 percent higher than those proposed by MTI. Specifically, of the eight key personnel proposed by the two offerors, particularly under the subfactors LANs and software development experience, Pinkerton's proposed staff showed significantly more experience than MTI's: Pinkerton's total years of LAN experience was 54 and MTI's was 32; in software development Pinkerton's total years of experience was 78 compared to MTI's 40.

The difference in personnel qualifications reportedly was most pronounced for the facility manager, the most important key employee. While both offerors' proposed facility managers met the minimum requirements in the RFP, Pinkerton's facility manager had more than twice as many years of ADP experience as MTI's. Additionally, the TEP chairman reports that it was noted that Pinkerton's facility manager is a company vice president with "the capability of providing company resources as necessary to quickly resolve problems where the Facility Manager proposed by MTI had four years with the company and was not a company principal." (In this regard, we note that one evaluator's evaluation narrative includes the comment "good strong line of command.") The TEP chairman reports that all of these differences were significant in the opinion of the TEP.

We conclude that the information in the record--contemporaneous evaluation scoring, summary statements of evaluated strengths, weakness and risks, explanations of changes in BAFO scoring, and post-protest amplifications-- provided a sufficiently clear basis for the award decision. The evaluation scoring was broken down into quite detailed items and the agency's post-protest explanation of the evaluation includes specific examples of areas of significant evaluated difference between proposals under each technical factor. Certainly, there was enough information available to understand the agency's rationale for the award decision and, thus, to permit MTI to determine whether it agreed with the scores or conclusions regarding specific evaluation areas. This aspect of the protest therefore is without merit.

DISCUSSIONS

MTI argues that the written discussions held with it were inadequate. It cites four areas of alleged evaluated weakness and risk in its proposal-- proposed subcontractor use, staff commitment, number of users in past contracts, and attachments--and one subfactor under Personnel Qualifications--Facility Management/Computer Operations--which, although downgraded, were not discussed with the firm. The protester believes that if discussions had been adequate it could have raised its technical score by a sufficient amount to overtake Pinkerton.

We dismiss this argument as untimely filed. Under our Bid Protest Regulations, a protest not based on an apparent solicitation impropriety must be filed within 10 working days after the basis of the protest is known or should have been known, whichever is earlier. 4 C.F.R. Sec. 21.2(a)(2) (1994). Where a protester initially files a timely protest and later supplements it with new and independent grounds of protest, the new allegations must independently satisfy our timeliness requirements. Palomar Grading and Paving, Inc., B-255382, Feb. 7, 1994, 94-1 CPD Para. 85.

MTI's argument challenging the sufficiency of discussions is based on the evaluation materials concerning MTI's own proposal. These materials were included in the agency's administrative report, which MTI acknowledges it received on June 16, 1994, but the protester raised this argument for the first time in its July 11 comments on the agency report. If MTI believed that these areas of alleged deficiency should have been the subject of discussions, it had only until June 30--10 working days later--to raise the argument. Because the argument was not raised until July 11, it is untimely and will not be considered. While the protester received an extension of time for filing its comments on the agency report, such an extension does not have the effect of waiving the timeliness requirements for filing bid protests; notwithstanding the comment due date, any protest arguments based on information in the agency report had to be filed no later than 10 working days after MTI received the report. Keci Corp.-- Recon., B-255193.2, May 25, 1994, 94-1 CPD Para. 323.

PRICE/TECHNICAL TRADEOFF--APPLICATION OF FORMULA

Finally, MTI argues that the agency failed to conduct an adequate price/technical tradeoff. The protester believes that the agency mechanically applied the numerical evaluation formula and that the formula by itself was an insufficient basis upon which to make award. According to the protester, the agency was obligated to make a separate determination justifying the payment of the $2.13 million price premium for the technical benefits of Pinkerton's proposal, represented by 1.60 normalized technical points.

The agency maintains that a formal price/technical tradeoff determination was unnecessary once it applied the numerical evaluation formula in the RFP since that formula essentially incorporated the tradeoff in the 70/30 technical price formula.

Where an agency chooses between a higher-cost, higher-rated proposal and a lower-cost, lower-rated one, our review is limited to a determination of whether the cost/technical tradeoff that the agency performed was reasonable and consistent with the solicitation evaluation criteria. Central Texas College, 71 Comp.Gen. 164 (1992), 92-1 CPD Para. 121. We agree with the agency that its approach here satisfied this standard.

As previously discussed, the solicitation advised offerors that if there were significant technical differences between competing proposals, the agency would make an award on the basis of a numerically weighted formula under which points had already been assigned, and explained that technical merit was more important than price. Thus, in effect, the solicitation notified offerors that the agency had predetermined the tradeoff between technical merit and price. See Eaton-Kenway, B-212575.2, June 20, 1984, 84-1 CPD Para. 649; Harrison Sys., Ltd., 63 Comp.Gen. 379 (1984), 84-1 CPD Para. 572. The formula applied here because the agency specifically determined that there was a significant difference in the technical merit of the proposals. The contracting officer affirms this in a post-protest declaration. Additionally, the record contains a post-protest memorandum by the chairman of the TEP which details the specific advantages under each evaluation factor. In addition to Pinkerton's significant advantage in key personnel experience (discussed above), Pinkerton's commitment not to move or change any personnel before or during the contract without ample notice and concurrence of the government was deemed a significant advantage; MTI did not make this commitment. (As MTI has not challenged any of the technical advantages detailed by the TEP chairman, we have no basis to question the agency's determination that Pinkerton's proposal reflected significant technical advantages.)

Under these circumstances, the agency properly adhered to the results of the evaluation formula. The agency was not required to further justify choosing the higher-priced offeror by making a separate documented determination, since the difference in cost was factored into the evaluation formula; that is, the tradeoff determination was made when the formula was devised. Stone & Webster Eng'g Corp., B-255286.2, Apr. 12, 1994, 94-1 CPD Para. 306.

The protest is denied.

* The decision issued on November 8, 1994, contained proprietary information and was subject to a General Accounting Office protective order. It was released to the parties admitted to the protective order. The parties have agreed that this decision should be released in its entirety.

1. The qualifications of proposed personnel were first to be evaluated to ensure that the RFP requirements concerning minimum education and years of experience were met; if so, the proposed personnel then would be point scored under nine subfactors.

2. The specific formulas were as follows:

technical weight = lower rating x 70 highest rating of all offerors

cost weight = lowest evaluated cost x 30 higher evaluated cost

3. It is evident from the record that the final scores were arrived at by taking the mean (i.e., middle) of the three evaluators' scores. The basis for use of the mean rather than the average (as the protester advocates) score in determining the consolidated score is not clear from the record. However, even if the average total scores of 1,380 for Pinkerton and 1,243 for MTI were used in the normalized score calculations, there would be no change in the score for each offeror; that is, Pinkerton's normalized technical score would remain unchanged at 70 and MTI's at 63.

4. The price negotiation memorandum indicates the initial consensus technical scores and prices were also normalized. Pinkerton's initial aggregate score was 95.20 (70 technical + 25.20 price) and MTI's was 91.11 (61.11 technical + 30 price).

5. According to the protester, a gain in a portion of the 49 points it lost under the two subfactors at issue would have provided sufficient points to overtake Pinkerton. Specifically, the protester maintains that an increase in its technical score of only 33 points would have resulted in a total normalized score of 94.77 for the firm which would have overtaken Pinkerton's total normalized score of 94.60.

6. Also in this area, the protester complains that no minutes exist of the consensus evaluation meetings.

7. The score sheets set forth the maximum available points for the factors, subfactors, and subfactor elements and provided spaces for the actual scoring. While the RFP had set forth the maximum available points for just the factors and subfactors, it nevertheless listed the subfactor elements that would be evaluated.

8. Contrary to the protester's suggestion, it is clear from the record that the agency's post-protest amplification of scoring relates to BAFO and not initial scoring since the amplification includes discussion of MTI's improved BAFO score.