Kalman & Company, Inc., B-287442.2, March 21, 2002

Kalman & Company, Inc., B-287442.2, March 21, 2002

DIGEST

Attorneys

DECISION

Kalman & Company, Inc. protests the award of a contract to Analytical Services & Materials, Inc. (AS&M) under request for proposals (RFP) No. RFP4-00017, issued by the National Aeronautics and Space Administration (NASA) to obtain engineering and technical services (ETS) in support of flight research and development programs for NASA's Dryden Flight Research Center, located at Edwards Air Force Base, California. Kalman argues that the agency improperly evaluated technical, past performance, and cost proposals.

We deny the protest.

The ETS contract is the primary vehicle by which the Dryden research, projects, management operations, chief engineer's office, and public affairs/education/commercialization organizations obtain support for flight research projects, internal research and development, and various technical support functions. RFP Statement of Work (SOW) at 3. As a result, the SOW divides the technical requirements into four sections: research engineering support; management system support; chief engineer's office support; and public affairs, commercialization, education outreach (PACE) and history office support. Id. This acquisition combines the current ETS contract with two other Dryden contracts. The solicitation, issued as a small business set-aside, contemplated award of an indefinite-delivery/indefinite-quantity contract with cost-plus-award-fee and cost-reimbursable-no-fee contract line items. Work was to be performed pursuant to specific task orders over a base period of 2 years, with one 3-year option period.

Award was to be made to the firm whose proposal was most advantageous to the government, considering mission suitability, past performance, and cost factors. The combined importance of the mission suitability and past performance factors was significantly more important than the cost factor. As individual factors, the mission suitability and past performance factors were approximately equal in importance, and both were individually significantly more important than cost. RFP Sec. M.2.

The mission suitability factor, which was to be point-scored and adjectivally rated, was comprised of two subfactors. The first subfactor, "understanding the requirements" (UR), was worth 600 of the 1,000 points available for the mission suitability factor and was comprised of two sub-subfactors, "overall understanding and technical approach" and "representative task orders" (RTO). The second subfactor, "management and staffing" (MS), was worth 400 of the 1,000 points available and was comprised of six sub-subfactors. RFP Sec. M.3.

The past performance evaluation was to assess the extent of the offeror's relevant experience associated with each area of the SOW, as evidenced within the contract(s) referenced in the proposal or otherwise known to the government. In addition to providing for the specific consideration of technical performance, schedule performance, and cost performance, NASA planned to evaluate the complexity and magnitude of the offeror's previous and current work and its relation to the work required for this procurement, with the quality and quantity of specific technical and management experience directly applicable to this requirement being especially important. Current and relevant experience was to have greater impact than less recent and less relevant experience. The term "relevant" was defined as meaning "contracts having comparable size (in excess of $4 [million] annually) or scope, and significant technical or cost similarity, association or other relationship with the effort and/or contract type described in this RFP." RFP Sec. M.4(c). Citing section 15.305(a)(2) of the Federal Acquisition Regulation (FAR), the RFP added that "[o]fferors without a record of relevant past performance or for whom information on past performance is not available, will not be evaluated favorably or unfavorably on past performance, but will receive a 'Neutral' rating." RFP Sec. M.4.

The agency planned to conduct a cost realism analysis of each cost proposal to determine the probable cost of performance for each offeror. This analysis would determine whether the estimated proposed cost elements were realistic for the work to be performed; reflected a clear understanding of the requirements; and were consistent with the unique methods of performance and materials described in the offeror's technical proposal. The probable cost would be used for purposes of evaluation. RFP Sec. M.5.

NASA received proposals from six offerors. The source evaluation committee (SEC) presented its initial evaluation results to the source selection authority (SSA), who decided that discussions were necessary. A list of discussion items and a request for a final proposal revision (FPR) were sent to each of the four offerors in the competitive range, including Kalman and AS&M, the incumbent ETS contractor. The SEC evaluated the FPRs and arrived at the following final evaluation results:

. Mission Suitability Past Cost . Performance (in Millions)

AS&M 850 Very Good Excellent $43.12 Proposed

. - UR 520/Very Low Risk $43.26 Probable . Good/Low Risk

. - MS 330/Very . Good/Low Risk

Offeror A 745 Good-Very Good Good $41.95 Proposed

. - UR 420/Good/Moderate Moderate Risk $42.30 Probable . Risk

. - MS 325/Good-Very . Good/Low Risk

Offeror B 605 Good Marginal $48.14 Proposed

. - UR 390/Good/Low Risk Moderate Risk $46.86 Probable

. - MS 215/Good/Moderate . Risk

Kalman 590 Good Neutral $46.17 Proposed

. - UR 405/Good/Moderate High Risk $48.23 Probable . Risk

. - MS 185/Fair/Moderate . Risk

The SSA determined that AS&M's proposal represented the best value to the government. AS&M received the highest score under the mission suitability factor due, in part, to having the most significant strengths, including a demonstrated understanding of the SOW requirements and of the contractor's role in performing the work. The SSA concluded that when these strengths were combined with AS&M's other major strengths, the higher rating was justified. While other offerors received significant strengths in specific areas, none demonstrated as deep or comprehensive an understanding of all functional areas as AS&M. Under the past performance factor, the SSA stated that AS&M's excellent rating reflected mostly excellent responses in its past performance questionnaires, and its past performance also reflected relevant experience on the SOW requirements. Finally, with respect to cost, the SSA noted that Kalman and Offeror B had significantly higher proposed and probable costs than AS&M and Offeror A. While AS&M had a higher probable cost than Offeror A, the SSA stated that the difference was outweighed by the significant strengths in AS&M's proposal and its excellent past performance. On December 17, the unsuccessful offerors in the competitive range were notified that NASA planned to make award to AS&M. Kalman filed this protest after its debriefing, raising various challenges to the reasonableness of the agency's evaluation of its mission suitability and past performance proposals, as well as to the reasonableness of the agency's evaluation of both AS&M's and its own cost proposals. Award has been suspended pending the outcome of the protest.

Kalman's allegation that NASA unreasonably evaluated its proposal under the mission suitability factor is focused on the two significant weaknesses NASA found with respect to the "understanding the requirement" subfactor, and two of the eight weaknesses NASA found with respect to the "management and staffing" subfactor.

The evaluation of technical proposals is primarily the responsibility of the contracting agency, and our Office does not make an independent evaluation of their merits. Rather, we examine the agency's evaluation to ensure that it is reasonable and consistent with the stated evaluation criteria. The protester bears the burden of showing that the evaluation is unreasonable, and the fact that it disagrees with the agency does not render the evaluation unreasonable. ESCO, Inc., B-225565, Apr. 29, 1987, 87-1 CPD Para. 450 at 7. Kalman has not met that burden.

In reviewing proposals under the "understanding the requirement" subfactor's "overall understanding and technical approach" sub-subfactor, NASA was to evaluate the offeror's understanding of the work to be performed as related to the functions in the SOW; the approach described by the offeror to performing the work; the methodology used to identify, report, and resolve problems encountered in the performance of the requirements; how the offeror identified and mitigated the risks to fulfilling the requirements; and how the narrative reflected the offeror's understanding of the role of the contractor in the Dryden environment.

RFP Sec. M.3(a)(1).

Kalman's proposal was evaluated as having a significant weakness because NASA believed it demonstrated a lack of understanding of the complexity and organizational structure of the PACE office and associated tasks. The caucus summary report on Kalman's initial proposal shows that the SEC had a number of detailed reasons to support this finding: (1) the proposal referred to "PACE and History Office Support" but the History Office is in the PACE organization and not a separate entity; (2) the response to SOW Para. 4.2, "PACE Office," simply stated the requirements and identified problems that were not representative of typical issues under this task; (3) the description of the firm's understanding under SOW Para. 4.2 did not provide a comprehensive response to the task; (4) the response to SOW Para. 4.3, "Public Affairs," included little response to the public affairs tasks but, instead, a reiteration of current policies and practices which did not describe how the firm would perform the task; (5) the response to SOW Para. 4.4, "Communicate Knowledge," contained no information about how the firm would interface with the PACE organization and did not recognize established priorities within this task element;

(6) the response to SOW Para. 4.5, "Education," did not state how the firm would perform the tasks but basically restated the SOW requirements; and (7) the response to SOW Para. 4.7, "History," did not state how the firm was going to perform the tasks, listed problems that were not within the scope of the task, and did not mention the level of expertise and skill required for the task.

Kalman's FPR included a response to the SEC's request to revisit its discussion of this section of the SOW. The SEC evaluated this response and concluded that it was inadequate because the only significant information presented was the listing of the Kalman team that would support the effort. The SEC also remarked that there was still a lack of understanding of the PACE Office since the revised response contained references to the "PACE and History Offices," in the plural, indicating Kalman was not aware that the History Office was part of the PACE organization and not a separate entity. The SEC's final caucus report did not delete any of the specific bases for evaluating this aspect of the proposal as having a significant weakness.

Kalman challenges only one of the specific bases underpinning the SEC's finding, the comment about the firm's references to the PACE and History Offices, in the plural. Kalman argues that NASA has extrapolated that the use of a particular title demonstrates a lack of understanding and that NASA ignored all of the specifics in its proposal. Kalman's argument is plainly contradicted by the detailed findings made by the SEC which show that NASA did note the specifics of Kalman's proposal but found them lacking. We also agree with NASA that the references to the PACE and History Offices, in the plural, give rise to the reasonable concern whether Kalman was aware that they were one organization. Kalman further contends that NASA's finding is unreasonable because the parts of the proposal pertaining to the PACE office were prepared by a former Dryden employee with intimate knowledge of the office, and because Kalman had contingent hire letters from numerous current key staff members and prior employees. This argument does not address the specific problems found by NASA during its evaluation of Kalman's proposal; moreover, this subfactor was not designed to evaluate the abilities of the firm's proposed key personnel /1/ but, rather, whether the firm could demonstrate a full understanding of the nature of the work to be performed. Kalman has given us no reason to question NASA's conclusion that the firm demonstrated a lack of understanding of the PACE office and its associated tasks or that this failure represented a significant weakness.

In reviewing proposals under the "understanding the requirement" subfactor's "RTO" sub-subfactor, NASA was to evaluate the depth of discussion of how the task would be performed, including technical approach and schedule; the realism of the proposed staffing plan, including skill mix and level of staffing needed to accomplish the objectives; the realism of additional resources necessary to successfully complete the task; and the completeness and accuracy in the identification of critical issues, including risk identification and mitigation. RFP Sec. M.3(a)(2). Kalman's proposal was evaluated as having a significant weakness because NASA found it reflected a significant lack of understanding of the requirements associated with the Controlled Impact Demonstration Anniversary Symposium RTO. The caucus summary report on Kalman's initial proposal shows NASA reached this conclusion because Kalman's proposal deleted the requirement for paper copies, stating that there would be no sensitive material; did not include a risk assessment or mitigation; did not include resources and staffing estimates; and because the major portion of Kalman's response was a detailed schedule of events and week-of activities.

Kalman's FPR included a response to the SEC's request to revisit its discussion of this RTO. The SEC evaluated the response and found it inadequate because, among other things, it did not discuss attendees' classification and processing; did not discuss how symposium papers would be distributed if they contained sensitive information or information that might not be readily available to the public; and because Kalman relied on web/electronic distribution of symposium material and did not provide an approach for supporting the print and distribution requirements. The SEC's final caucus report did not delete any of the above specific bases for evaluating this aspect of Kalman's proposal as having a significant weakness.

Kalman argues that the SEC's criticism that the major portion of its response was a detailed schedule of events and activities is in error since the SEC was required to evaluate "[the] [d]epth of [the] discussion of how the task would be performed, including technical approach and schedule." RFP Sec. M.3(a)(2). Kalman has misread the record. The SEC's concern was not that Kalman included a detailed schedule of events and activities, but that this information constituted the major portion of its response at the expense of other information to be evaluated. Our review of the record gives us no basis to question the SEC's criticism.

Kalman also objects that there was no requirement for paper copies and that it should have been given extra credit for its plans to use electronic publishing methods. Kalman is incorrect. One subtask to be performed in this RTO was to publish and distribute all symposium papers in one document. A Dryden publication governing how technical reports and papers are to be published was posted as part of the solicitation on the agency's web site. That publication states that the Dryden technical publications office will print the document and references a publications manual detailing the procedures. The final step in these procedures is to print the report. The contracting officer states that Kalman's proposal was downgraded because it did not even recognize the documented procedures in place for publication; had it done so in addition to proposing electronic publication as an alternative, it might have received additional credit. We cannot find the agency's evaluation unreasonable. Kalman next contests two of the eight weaknesses the SEC identified in its proposal under the "management and staffing" subfactor.

The first weakness stems from the SEC's finding that the benefit packages among Kalman's team members were not uniform and differed substantially. The SEC was concerned that these disparities--such as widely varying numbers of paid holidays and sick days--could affect employee morale. The SEC was also concerned that the proposed salaries varied greatly between the team members for the same position, which could potentially harm employee recruitment and morale. During discussions, Kalman was asked to explain how it planned to mitigate the differences that appeared in the various compensation plans such that employee recruitment, retention and morale would be maintained. In response, Kalman stated that, "Ideally, [it] would like each of its team members to offer the same competitive benefit package" and that each team member was revisiting its compensation plan and would be encouraged to share information to ensure that benefits would be comparable. Kalman FPR at 28. The SEC found this response to be generic, noted that the FPR had greater salary discrepancies than did the initial proposal, and did not change its view that this was a weakness. Kalman's contention that "it is not for the government to argue for consistency if it leads to mediocrity," Protester's Comments, Feb. 21, 2002, at 4, both fails to address the agency's concerns about employee morale and is inconsistent with its FPR statement that it wanted to encourage its team members to offer the same competitive benefit package. /2/

The second weakness results from the SEC's concern that Kalman's proposal failed to disclose the duration of time its instrumentation engineering key person would be available to work on the effort. Kalman asserts that the same problem existed in AS&M's proposal but that it, unlike Kalman, had an opportunity to correct the problem during discussions. Kalman's assertion that the agency treated the offerors disparately again reflects a misreading of the record.

The SEC evaluated Kalman's initial proposal as having a weakness because it failed to identify the key person in instrumentation engineering. During discussions, Kalman was asked to identify this key person and to provide his resume and letter of intent. Kalman's FPR did so, and the SEC deleted the original weakness it had assigned, but added a new weakness because neither the newly-identified person's resume nor his letter of intent made it clear how long this person would be available to work on the effort. There was no disparate treatment because both AS&M and Kalman were given the opportunity to address their original weaknesses. The weakness at issue here was first introduced by Kalman in its FPR, when it first submitted this person's resume and letter of intent, and the agency was not required to reopen discussions to permit the firm to address this weakness. The Communities Group, B-283147, Oct. 12, 1999, 99-2 CPD Para. 101 at 4-5.

Kalman has thus given us no basis to find that NASA unreasonably evaluated its proposal under the mission suitability factor. Moreover, despite having access to the entire evaluation record and AS&M's proposal, Kalman has failed to make any specific argument in support of its protest allegation that, on information and belief, NASA's evaluation of AS&M's proposal relied not squarely on its proposal, but on its awareness of the firm's capabilities based on its incumbency. As a result, this allegation lacks a valid basis. Siebe Envtl. Controls, B-275999.2, Feb. 12, 1997, 97-1 CPD Para. 70 at 2.

Kalman next alleges that NASA's evaluation of its past performance as neutral--a rating required for offerors without a record of relevant past performance--was unreasonable and inconsistent with NASA's finding that the Kalman team as a whole had relevant experience for the majority of SOW requirements. Our review of the record shows that the apparent inconsistency merely reflects two different meanings of the term "relevant."

The solicitation provided for a specific evaluation of an offeror's technical performance, schedule performance, and cost performance. The solicitation also provided for a more general assessment of the extent of an offeror's relevant experience associated with each of the SOW areas, as evidenced within contracts referenced in the proposal, or otherwise known to the government. As noted earlier, current and relevant experience was to have greater impact in the government's assessment than less recent and less relevant experience. The RFP defined the term "relevant" as meaning "contracts having comparable size (in excess of $4 [million] annually) or scope, and significant technical or cost similarity, association or relationship with the effort and/or contract type described in this RFP."

RFP Sec. M.4(c). Citing FAR Sec. 15.305(a)(2), the RFP stated that offerors without a record of relevant past performance or for whom information on past performance was not available would not be evaluated favorably or unfavorably on past performance, but would receive a neutral rating. RFP Sec. M.4(b).

Kalman's proposal did not identify any "relevant" contracts, i.e., any "contracts having comparable size (in excess of $4 [million] annually) or scope, and significant technical or cost similarity, association or other relationship with the effort and/or contract type described in this RFP." RFP Sec. M.4(c). Since Kalman had no record of "relevant" past performance as defined by the solicitation, NASA assigned its proposal a neutral rating in accordance with the solicitation's terms. See also FAR Sec. 15.305(a)(2). NASA's finding that the Kalman team as a whole had "relevant technical experience" refers to the fact that the team's technical performance experience was relevant to the SOW tasks, not that the proposal met the solicitation's definition of the term "relevant." Under the circumstances, and given the express language of the solicitation, we cannot find the evaluation unreasonable. /3/ See Boland Well Sys., Inc., B-287030, Mar. 7, 2001, 2001 CPD Para. 51 at 3.

In its protest, Kalman also argued that NASA's evaluation was unreasonable because NASA claimed not to have received, and evaluated, past performance questionnaires on one of its subcontractors. Kalman asserted that the questionnaires were, in fact, submitted by the respondents. In its report, NASA denied that it had received these questionnaires but explained that Kalman was not prejudiced thereby because the subcontractor represented such a small percentage of the total proposed workforce here. Kalman's comments on the agency report do not rebut NASA's claim that it did not receive the questionnaires, and we have no basis to find otherwise. In any event, there is no legal requirement that all past performance references be included in a valid review of past performance. Advanced Data Concepts, Inc., B-277801.4, June 1, 1998, 98-1 CPD Para. 145 at 10. For our Office to sustain a protest challenging the failure to obtain a reference's assessment of past performance, a protester must show unusual factual circumstances that convert the failure to a significant inequity for the protester. MCS of Tampa, Inc., B-288271.5, Feb. 8, 2002, 2002 CPD Para. _ at 5. Kalman has made no such showing and, given the basis for the neutral rating, we find none.

Kalman finally argues that NASA improperly failed to upwardly adjust AS&M's proposed direct labor rates in conducting its cost realism analysis. Kalman asserts that, in various cases, AS&M's proposed direct labor rates are below the rates found in some market surveys and that NASA failed to note the substantial performance risk presented by these allegedly unreasonably low rates.

Where an agency evaluates proposals for award of a cost-reimbursement contract, an offeror's proposed estimated costs of contract performance and proposed fees are not considered controlling, since the offeror's estimated costs may not provide valid indications of the final actual costs that the government is required, within certain limits, to pay. Advanced Communication Sys., Inc., B-283650 et al., Dec. 16, 1999, 99-2 CPD Para. 3 at 5. Accordingly, a cost realism analysis must be performed when a cost-reimbursement contact is contemplated. FAR Sec. 15.404-1(d)(2). A cost realism analysis is the process of independently reviewing and evaluating specific elements of each offeror's proposed cost estimate to determine whether the estimated proposed cost elements are realistic for the work to be performed, reflect a clear understanding of the requirements, and are consistent with the unique methods of performance and materials described in the offeror's technical proposal. FAR Sec. 15.404-1(d)(1). Because the agency is in the best position to make this cost realism determination, our review is limited to determining whether its cost evaluation was reasonably based and not arbitrary. NV Servs., B-284119.2, Feb. 25, 2000, 2000 CPD Para. 64 at 7.

The RFP advised offerors that, in establishing the probable cost, the SEC might make use of any available source, including data provided by the offerors, other government agencies, commercial sources, in-house sources, and any other sources, as appropriate. RFP Sec. M.5. Section M.3(b)(4) of the RFP stated that proposed compensation would be reviewed for a clear understanding of the work to be performed and an indication of the capability of the proposed compensation structure to obtain and keep suitably qualified personnel to meet mission objectives. . . . [P]roposals envisioning compensation levels lower than those of predecessor contractors for the same work will be evaluated on the basis of maintaining program continuity, uninterrupted high quality of work, and availability of required competent professional service employees.

In response to Kalman's complaint that several of the direct labor rates proposed by AS&M are below the rates found in some market surveys, NASA explains that the RFP did not emphasize survey data to the exclusion of all other data, and that its evaluation was not so limited. Specifically, the agency reviewed both the survey data provided in AS&M's proposal and historical data in the form of the wage data for the current ETS and other NASA contracts to evaluate the realism of AS&M's proposed direct labor rates. AS&M is the incumbent ETS contractor, and based its proposed direct labor rates on those it was paying under the current contract. Since AS&M was currently and successfully performing the contract, NASA states that it had no basis to find that the firm's wages were not realistic or consistent, no basis to find a negative impact on recruiting and retention, and no basis to adjust AS&M's proposed costs upward to accommodate higher wage rates than those proposed. Notwithstanding its conclusion that the rates were realistic, NASA did downgrade AS&M's proposal as Kalman suggests by assigning it a weakness under the management and staffing subfactor because certain of its proposed direct labor rates were at the low end of the scale. Under the circumstances, we cannot conclude that NASA's evaluation was unreasonable.

In any event, we need not give further consideration to Kalman's allegations about AS&M's cost proposal or to its allegation that NASA unreasonably adjusted its own cost proposal. Under the Competition in Contracting Act of 1984, 31 U.S.C. Sec. 3553(a) (1994), and our Bid Protest Regulations, 4 C.F.R. Sec. 21.1(a) (2001), a protester must be an "interested party" before we will consider its protest. A protester is not an interested party if it would not be in line for award if its protest were sustained. Avondale Tech. Servs., Inc., B-243330, July 18, 1991, 91-2 CPD Para. 72 at 2.

We have already found Kalman's allegations regarding the evaluation of its own proposal under the non-cost factors to be without basis. In addition, there is at least one offeror aside from AS&M who received higher non-cost ratings and whose proposed and probable costs are lower than those of Kalman; Kalman does not challenge the evaluation results with respect to any offeror save AS&M. Thus, even if we found that AS&M's proposed costs should have been upwardly adjusted to a substantial degree and found to represent high performance risk, and even if we found that NASA should not have made any upward adjustments at all to Kalman's proposed costs, at least one offeror, not Kalman, would be next in line for award. Accordingly, Kalman lacks the requisite direct and substantial interest with regard to the award to be considered an interested party to protest the evaluation.

RS Information Sys., Inc., B-287185.2, B-287185.3, May 16, 2001, 2001 CPD Para. 98 at 5.

The protest is denied.

Anthony H. Gamboa General Counsel

1. The record shows that Kalman was given credit for the strength of its key personnel under the management and staffing subfactor.

2. Our review of the record shows that Kalman's contention that AS&M's team also proposed significantly disparate benefit packages is without basis. See, e.g., AS&M FPR at 5.

3. Kalman does not challenge the reasonableness of the underlying past performance evaluation, which identified one significant weakness, three weaknesses, and various reasons for the firm's high risk rating.