Matter of: MetaMetrics, Inc. File: B-248603.2 Date: October 30, 1992

Matter of: MetaMetrics, Inc. File: B-248603.2 Date: October 30, 1992

Protest against award on the basis of awardee's low cost, technically superior proposal is denied where evaluation of technical proposals was reasonable and consistent with stated evaluation criteria; agency properly considered whether the identification of the protester's proposed program coordinator with prior economic reform program that led to riots and overthrow of prior government would hinder accomplishment of the statement of work requirement to assist in implementing the next phase of economic reforms.

Attorneys

DECISION

MetaMetrics, Inc. protests the Agency for International Development's (AID) award of a contract to Abt Associates Inc., under request for proposals (RFP) No. 92-001, to assist the Republic of Mali in implementing the Policy Reform for Economic Development (PRED) project. MetaMetrics challenges both the technical evaluation and the adequacy of discussions with respect to perceived weaknesses in its proposal.

We deny the protest.

The solicitation requested proposals for a 4-year cost-plus-fixed-fee contract to furnish 4 long-term advisors--including a coordinator (48 months of effort), private sector advisor (42 months), public finance advisor (24 months), and a management information system (MIS) specialist (24 Months)--60 person-months of effort by short-term advisors, and other training support and procurement assistance to Mali to assist in implementing and monitoring the impact of the PRED project. The PRED project is a continuation of a prior structural adjustment program, for which MetaMetrics provided assistance under contract with AID, under which measures were introduced to reduce public sector size, operating deficits and debt, make more efficient the administration of public finances, reduce government regulation and increase market and trade liberalization, and encourage business formation and export growth. According to the solicitation, although the prior economic reform program was blamed for a declining standard of living, and the resulting protests of the declining economic conditions led to rioting which brought about the overthrow of the prior government, the new government has pledged itself to continue with a structural adjustment of the economy. Accordingly, as set forth in the statement of work (SOW), the goal of the PRED program is "to support private-sector-led sustained economic growth, by reducing tax burden on, and government interference with, the private sector, while increasing the positive role of government as in affects this sector."

The solicitation provided for award to be made to the offeror whose proposal offered the best overall value to the government, considering both technical and cost factors. The RFP listed three technical evaluation factors: qualifications/experience of personnel (50 of 100 technical evaluation points), quality and responsiveness of proposal (30 points), and qualifications/capability of the institution (20 points). The most important subfactor under the personnel factor was that for long- term personnel (35 points), of whom the proposed coordinator was most heavily weighted (12 points). The initial point scores in each category were to be normalized, with the highest-scored offer receiving the maximum final points and lesser-scored offers receiving proportionately fewer final points. The solicitation provided for the technical score to be worth 85 percent and cost to be worth 15 percent in the overall evaluation.

AID received three proposals in response to the solicitation; the agency included MetaMetrics' and Abt's proposals in the competitive range. AID then conducted written discussions with the offerors and requested best and final offers (BAFO).

A panel of three evaluators scored the BAFOs; a combined score was calculated by mathematically averaging their individual scores. Since Abt's resulting BAFO technical score (86.3 points) exceeded MetaMetrics' (78.7 points), and the estimated cost of its proposal ($4,920,580) was less than the cost ($5,091,802) of MetaMetrics' proposal, Abt received the maximum score under both categories when scores were normalized as provided for in the solicitation. As a result, Abt received an overall evaluation score of 100, 8.5 points more than MetaMetrics' score of 91.5 points.

AID specifically determined that the quality of analysis and presentation in Abt's proposal were superior in many respects; according to the agency, Abt's proposal included "clear, concrete and well though out" proposals for action and was the only proposal to give proper attention to monitoring and evaluation. AID also noted that Abt was a large firm whose "extensive corporate capacity and past performance in similar projects" demonstrated its ability to perform the contract. In addition, the agency specifically found Abt's proposed coordinator, a former minister of finance in Ethiopia with 17 years of experience at the African Development Bank, to be "an extremely well qualified" individual with extensive West African experience.

AID also found that MetaMetrics' proposal demonstrated MetaMetrics' knowledge of the subject matter. According to the agency, MetaMetrics' program description and discussion clearly reflected "an intimate knowledge of the Malian economic reform program" and that this was the proposal's greatest strength. The agency evaluation also noted that MetaMetrics' proposal of its incumbent coordinator for the prior program as coordinator for the new PRED project was viewed by some evaluators as a strength because it would help ensure continuity of assistance.

AID, however, questioned several aspects of MetaMetrics' proposal. Notwithstanding any advantages offered by MetaMetrics' proposal of its incumbent program coordinator, evaluators questioned the choice on the basis that the individual possessed little private sector experience, having instead focused in his career on policy reform, and that this lack was especially significant in view of the increased emphasis under the PRED program on the role of the private sector. Moreover, at least one of the three evaluators viewed MetaMetrics' selection of the incumbent coordinator as a "handicap" because of his identification with the prior program. (As noted in the solicitation, the economic reform program had been blamed for the declining standard of living and the resulting protests has led to riots and the overthrow of the prior government.) In addition, agency evaluators questioned the public finance advisor's knowledge of computers and data management and the extent of his experience with public finance, the extent of the private sector advisor's private sector experience and experience with computer hardware possessed by MetaMetrics' primary candidate for MIS specialist. Furthermore, the agency found that MetaMetrics had failed to adequately described the respective roles and responsibilities in the relationship between the proposed advisors and their Malian counterparts. Although MetaMetrics had attempted to address the agency's concern by proposing in its BAFO to form a "Program Review Committee," composed of the Malian Minister of the Economy and Finance (or his representative), the head of the Malian unit responsible for implementing the PRED project, the AID project officer, and MetaMetrics' program coordinator, the agency "did not find the high- level Program Review Committee, proposed to meet approximately every quarter, to be a workable or acceptable solution" to the problem of defining and maintaining counterpart relationships.

Based on the evaluation of BAFOs, AID determined Abt's proposal to be the technically superior, low cost proposal offering the best overall value to the government. Upon learning of the resulting award to the Abt, MetaMetrics filed this protest with our Office.

MetaMetrics contends that several aspects of the evaluation of its proposal were unreasonable and based upon undisclosed evaluation criteria. MetaMetrics questions why Abt's proposed coordinator received a higher score (10.2 technical evaluation points) than did MetaMetrics" (8 points) when MetaMetrics' candidate possessed experience with the Malian government while Abt's did not. The protester, referring to the private sector experience cited in its BAFO in response to the agency's discussion question in this regard, maintains that its candidate in fact possessed extensive private sector experience. MetaMetrics' also argues that the agency's concern with his identification with the prior reform program reflected the application of an undisclosed evaluation criterion. Likewise, MetaMetrics argues that the agency's concern with the lack of computer knowledge of its public finance advisor also reflected the application of an undisclosed evaluation criterion. In addition, MetaMetrics challenges the conclusion that it had failed to adequately discuss counterpart responsibilities and roles; according to the protester, this ignores the discussion of this area in its BAFO and, to the extent it reflects the agency's rejection of its proposal of a high- level oversight committee, again involves the application of an undisclosed evaluation criterion.

In reviewing protests against an agency's evaluation of proposals, it is not the function of our Office to independently evaluate those proposals. Signal Corp., B-241849 et al., Feb. 26, 1991, 91-1 CPD Para. 218. Rather, we will question the agency's technical evaluation only where the record shows that the evaluation lacks a reasonable basis or is inconsistent with the evaluation criteria listed in the RFP. Id. In this regard, contracting agencies are required to include in solicitations all significant evaluation factors and their relative importance. 41 U.S.C. Sec. 253a(b)(1) (1988). Solicitations must disclose any significant subfactors to be considered in the source selection decision and their relative importance. Federal Acquisition Regulation Sec. 15.605(e). However, all matters taken into account under these factor need not be specifically identified in the RFP, provided they are reasonable encompassed within the stated evaluation criteria. See Management Sys. Designers, Inc.; et al., B-244383.4 et al., Dec 6, 1991, 91-2 CPD Para. 518; Moran Assocs., B-240564.2, Nov. 27, 1991, 91-2 CPD Para. 495.

MetaMetrics has not challenged the validity of AID'S concern with its proposal as coordinator for the PRED project of an individual who, as incumbent coordinator, was identified with the prior reform program; as noted in the SOW, the prior program was blamed for a declining standard of living and the resulting protests led to rioting and the overthrow of the prior government. Rather, while MetaMetrics maintains that the successful performance of the incumbent coordinator under the prior contract was an appropriate matter for consideration in the evaluation, it argues that the agency was precluded from considering that the incumbent coordinator's experience under that contract might also prove to be a liability in accomplishing the goals set forth under the SOW for the next phase of the economic reform program.

We find MetaMetrics' position to be without merit. The SOW described the "essential" role of the project coordinator in implementing the PRED; according to the solicitation, he would be the principal authoritative source of contact "with both AID and the Malian Ministry of the Economy and Finance (MEF)," and the "source of policy and operational advice to the MEF on the conduct of the reform program." In addition, the statement of evaluation criteria specifically provided for evaluation of the qualifications and experience of the individuals proposed for the four long-term advisor positions, including the coordinator. In view of the critical, high-visibility role of the coordinator and the specific solicitation provision for evaluation of his experience, we believe the agency could reasonably consider not only whether and how well he had performed similar duties in the past, but also whether his prior activities might hinder his effectiveness in implementing the next phase of the reform program and the successful accomplishment of the PRED project. This matter was reasonably encompassed within the personnel qualifications and experience evaluation factor. See generally Management Servs., Inc., 55 Comp.Gen. 715 (1976), 71-1 CPD Para. 74, where we viewed consideration of the possibility of labor unrest as a result of what or who was proposed to be appropriate under broadly stated evaluation criteria.

We also find no basis in the record for questioning AID's evaluation of the private sector experience of MetaMetrics' proposed coordinator. Apart from his work as a consultant, his private sector experience appears to have been primarily limited to positions held in Vietnam, where he also served in various public positions, including Finance Minister, before the fall of the Republic of Vietnam in 1975. In contrast, Abt's proposed coordinator, who most recently served as Finance Minister in Ethiopia, worked from 1973 to 1990 at the African Development Bank, where he reportedly was involved in lending programs and conducting negotiations with other financial institutions. In our view, AID could reasonably conclude that the extensive, more recent private sector experience of Abt's proposed coordinator was superior to the private sector experience of MetaMetrics' coordinator. We therefore find no basis on which to question the agency's evaluation of proposed coordinators.

As for the agency's concern with the lack of computer knowledge of MetaMetrics' proposed public finance advisor, we find that this was reasonably encompassed within, and therefore properly taken into account, under the evaluation factor for the public finance advisor's qualification and experience. The SOW listed the duties of the public finance advisor as including the requirement to: (1) "provide the lead in analytical . . . aspects of public finance"; (2) "lead the process of gathering, organizing and effectively utilizing relevant data and analyses needed in the reform efforts"; and (3) "determine what equipment and material are to be provided under the project in support of the above tasks and assist in the development of specifications for such materials and equipment." Clearly, the successful performance of these responsibilities may require the acquisition and use of automated data processing software and hardware. The public finance advisor's familiarity with computers therefore was relevant to his ability to satisfactorily perform under the SOW.

We also find that AID COULD reasonably conclude that MetaMetrics' discussion of counterpart responsibilities and roles was less detailed than desired. The SOW required the advisors to cooperate and work with various Malian offices and officials in implementing the PRED project. The solicitation specifically required offerors to discuss in their proposals what steps would be taken to guarantee "good communication and coordination" among the advisors and the Malian government and the AID mission, and to ensure the "effective transfer of professional knowledge and skills to [Malian] counterpart staff." In response to the agency's discussion question in this regard, MetaMetrics included in its BAFO a brief, general overview of the relevant Malian institutions; it specifically proposed the high-level Program Review Committee--whose membership included only the program coordinator from among the long-term advisors--and simply stated that the "long-term advisors will work closely with assigned counterpart officials." In contrast, Abt, which was also questioned by AID in this regard, included in its BAFO a detailed discussion of the relationship of each of the long-term advisors, not merely the program coordinator, with their Malian counterparts. In our view, AID reasonably concluded that MetaMetrics' proposal of a high-level committee that would meet once a quarter was no substitute for the more detailed discussion of specific, counterpart relationships at all levels of the PRED project which had been requested by AID (and which was furnished by Abt). Furthermore, in view of the specific solicitation requirement to discuss the offeror's approach to assuring the coordination and transfer of information required by the SOW, we believe that the utility of MetaMetrics' high-level committee in satisfying the solicitation requirements was a matter reasonably encompassed within the stated evaluation criterion for quality and responsiveness of the offeror's technical approach and proposal.

MetaMetrics also questions the evaluation of the French language capability of its proposed short-term personnel, which was one of the listed subcriteria under the evaluation factor for qualifications and experience of personnel. Two of the three evaluators downgraded MetaMetrics' proposal in this area on the basis that it was difficult to determine from the proposal the language capability of proposed short-term personnel.

Our review of MetaMetrics' proposal confirms the conclusion reached by the third evaluator, who found with respect to the French language capability of the short-term personnel that it "appears all have the French capability." Specifically, the biographical information submitted by MetaMetrics in its proposal indicated that the short-term personnel are native French speakers, speak fluent French, or are certified as French speaking by means of a standardized testing process. However, although it is not evident from the record or from the administrative report submitted in response to this protest why MetaMetrics' proposal was down-graded in this area, it is clear that MetaMetrics did not thereby suffer competitive prejudice. As noted above, MetaMetrics' final technical score was 7.6 points lower than Abt's; had it received full credit under the subcriterion for French language capability, its final technical score would have increased by only 1.3 points.

MetaMetrics also contends that the discussions conducted by AID with offerors in the competitive range did not satisfy the statutory and regulatory requirement that the discussions be meaningful--that is, that the agency point out weaknesses, excesses or deficiencies in proposals unless doing so would result in technical transfusion or technical leveling--because the agency did not advise MetaMetrics of all of the perceived weaknesses in the experience and qualifications of its proposed personnel. See Columbia Research Corp., B-247631, June 22, 1992, 92-1 CPD Para. 539. MetaMetrics contends that AID was required to advise it that one of the three evaluators questioned the extent of the proposed public finance advisor's knowledge of computers and data management, and that another evaluator questioned his public finance experience. Likewise, MetaMetrics maintains that AID was required to advise it that its proposed private sector advisor was viewed by one of the evaluators as possessing only "limited private sector experience" and by another evaluator as lacking experience in the area of institutional reform. In addition, AID did not advise MetaMetrics that the identification of its proposed program coordinator with the prior reform program would hinder the successful accomplishment of the new program. According to the protester, had it been fully advised of the agency's concerns, "it would have attempted to resolve the concerns."

We agree with MetaMetrics that AID's failure to advise it during discussion of significant perceived weaknesses in the experience and qualifications of its proposed long-term advisors was inconsistent with the agency's obligation to conduct meaningful discussions. However, competitive prejudice is an essential element of a viable protest; where no prejudice is shown or is otherwise evident, our Office will not sustain a protest, even if a deficiency in the procurement is evident. See United Int'l Enq'g, Inc. et al., 71 Comp.Gen. 177 (1992), 92-1 CPD Para. 122; Latins Am., Inc., B-247674, June 15, 1992, 92-1 CPD Para. 519. We find no basis from our review of the record for concluding that MetaMetrics was prejudiced by AID'S failure to discuss these matters. MetaMetrics does not explain how its proposed long-term advisors possessed the desired experience and qualifications. Nor does MetaMetrics state that it would have replaced personnel whose experience and qualifications were considered not as strong as desired if it had been advised of the agency's concerns in this regard. See Specialized Technical Servs., Inc., supra. When AID did question MetaMetrics concerning the private sector experience of its proposed program coordinator, noting that "some committee members recommended that the firm propose an alternate candidate," the firm declined to propose a new coordinator. Nothing in the record indicates that also been advised of the agency's additional concern regarding his identification with the prior program.

Further, although MetaMetrics included in its initial proposal alternate candidates for the public finance and private sector advisor positions, the description of their experience in the proposal reflects limited private sector experience and no experience with computers or data management. Finally, even if MetaMetrics had proposed a new program coordinator, public finance advisor and private sector advisor, it does not appear that the results of the technical evaluation would have changed. If MetaMetrics' proposal had not been downgraded for the weaknesses not discussed with the firm, and if the firm had received the same score in these areas as Abt's proposal (which was not deficient in these areas), MetaMetrics' overall technical score could not have increased sufficiently to eliminate Abt's evaluated technical superiority. MetaMetrics, with its higher cost proposal, thus would not have moved into line for award.

The protest is denied.