Skip to main content

B-246837, Apr 1, 1992

B-246837 Apr 01, 1992
Jump To:
Skip to Highlights

Highlights

Protest of exclusion of firm's proposal from the competitive range is denied where the agency reasonably concluded in accordance with the evaluation criteria in the solicitation that the protester did not have the qualifications and relevant experience necessary to perform the contract. Protester properly found technically unacceptable is not an interested party to challenge the agency's evaluation of the awardee's proposal and the award to that firm since the protester would not be in line for award if its protest were sustained. The protester contends that its lower priced proposal was improperly evaluated as technically unacceptable. The USHMM is a storytelling museum containing artifacts.

View Decision

B-246837, Apr 1, 1992

DIGEST: 1. Protest of exclusion of firm's proposal from the competitive range is denied where the agency reasonably concluded in accordance with the evaluation criteria in the solicitation that the protester did not have the qualifications and relevant experience necessary to perform the contract, did not demonstrate its understanding of the purpose and subject matter of the procurement, and did not propose a level-of-effort which reflected its understanding of the scope of work. 2. Protester properly found technically unacceptable is not an interested party to challenge the agency's evaluation of the awardee's proposal and the award to that firm since the protester would not be in line for award if its protest were sustained.

Attorneys

Dick Young Productions Ltd.:

Dick Young Productions Ltd. protests the exclusion of its proposal from the competitive range and the award of a contract to Wentworth Films, Inc. under request for proposals (RFP) No. CX-1100-RFP-1020, issued by the Department of the Interior for the production of a motion picture for the United States Holocaust Memorial Museum (USHMM). The protester contends that its lower priced proposal was improperly evaluated as technically unacceptable. The protester also challenges the evaluation of the awardee's proposal.

We deny the protest in part and dismiss it in part.

The USHMM is a storytelling museum containing artifacts, documents, and displays that narrate the history of the Holocaust. The solicitation, a total small business set-aside issued on June 29, 1991, contemplated the award of a firm, fixed-price contract for the production of a motion picture, entitled "Testimony," containing newly shot interviews with survivors of the Holocaust and focusing on the themes of defiance, rescue, and resistance. The solicitation stated that the objective of the film is to give visitors to the USHMM "the opportunity to see and hear survivors' remembering, to identify with an individual's experience, to begin to glimpse a world full of contradiction, confusion, and choices."

The solicitation contained the following technical evaluation factors and the weight factor assigned to each evaluation factor: (A) qualifications and relevant experience of proposed production team members (30); (B) technical quality of sample production (25); (C) understanding of the production's purpose and subject matter (25); and (D) creativity, as demonstrated in sample production (20). The solicitation advised offerors to submit a comprehensive and complete technical proposal to support the items an offeror believed represented its capabilities and experience as outlined and required by the statement of work. The solicitation also advised that an offeror's price proposal would be evaluated to determine if the offeror demonstrated an understanding of the scope of work by committing appropriate resources in the required areas. The solicitation stated that failure of an offeror to demonstrate such an understanding could result in rejection of its proposal. The solicitation stated that technical quality would be considered more important than price and that the award would be made to the responsible offeror whose proposal conformed to the requirements of the solicitation and was most advantageous to the government. The government reserved the right to make the award to other than the lowest priced offeror.

Sixteen firms, including the protester and Wentworth, submitted initial technical proposals by the closing date of July 19. The agency's five- member technical evaluation panel (TEP) individually scored each offeror's proposal by assigning points for each evaluation factor. The individual scores for each evaluation factor were supported by narratives listing the strengths and weaknesses of each offeror's proposal. While the protester received almost perfect scores for evaluation factors (B) and (D) which respectively involved the technical quality and creativity of its sample production, it received 17.6 out of 30 points for evaluation factor (A)-- qualifications and relevant experience of proposed production team members -- and 10.6 out of 25 points for evaluation factor (C)-- understanding of the production's purpose and subject matter. These factors represented 55 percent of the total points available. The individual evaluators' scores for each evaluation factor were averaged to determine an overall consensus score. The initial consensus scores ranged from 31 to 95.8 out of 100 points. The protester's proposal received 65.2 points and was ranked sixth out of the 16 technical proposals received. The contracting officer then reviewed the TEP's narratives listing the strengths and weaknesses of each offeror's proposal, the corresponding point scores, and each offeror's price. In making her competitive range determination, the contracting officer compared the strengths and weaknesses of the proposals and ranked the proposals according to technical excellence. The contracting officer found the protester's proposal unacceptable in two technical areas and concluded that its proposed level-of-effort was inadequate. The contracting officer also determined that while 4 of the 16 firms which submitted initial technical proposals were technically acceptable, only the proposals of 2 of these firms-- Wentworth, which received 95.8 points, and Gardner Films, which received 91.8 points-- were clearly technically superior and would be included in the competitive range.

Following discussions with the two competitive range offerors, which submitted significantly higher prices than the protester, and the submission of best and final offers, the contracting officer, on September 18, awarded a contract to Wentworth, a higher technically rated, higher priced offeror. By letter of September 18, the contracting officer notified the protester of the award. The protester received the agency's written debriefing on October 7. By letter dated October 18, the protester filed an agency-level protest challenging the evaluation of its proposal and the award to Wentworth. By letter dated November 22, the contracting officer denied the protester's agency level protest. December 2, the protester filed this protest.

The protester argues that the agency improperly evaluated its proposal as technically unacceptable based on evaluation factors (A) and (C) and improperly excluded its proposal from the competitive range. The protester contends that since its price was less than the prices of the offerors whose proposals were in the competitive range, its proposal should have been included also. The protester further challenges the agency's evaluation of Wentworth's proposal and the award to Wentworth.

The law and implementing regulations require that written or oral discussions be held with all offerors whose proposals are within the competitive range; competitive range proposals are those that have a reasonable chance of being selected for award, cost or price and other factors as stated in the solicitation having been considered. 41 U.S.C. Sec. 253b(d)(2) (1988); Federal Acquisition Regulation (FAR) Sec. 15.609. In reviewing protests concerning the reasonableness of the evaluation of a technical proposal, and the resulting determination of whether the proposal is within the competitive range, we determine whether the agency's evaluation was reasonable and otherwise free from violations of procurement laws and regulations. CKJ Realty/Bayview Group, B-244492, Oct. 21, 1991, 91-2 CPD Para. 349. Here, the solicitation clearly stated that the technical quality of an offeror's proposal was more important than an offeror's price in determining the most advantageous offer to the government. As discussed below, we find that the agency reasonably found the protester's proposal unacceptable based on its qualifications and relevant experience and its understanding of the production's purpose and subject matter as reflected in its proposal.

With respect to the protester's qualifications and relevant experience, the agency determined, based on resumes and other information submitted, that although the protester's three-member production team had a wide range of production experience over a 10 year period, none of the three production team members had any background in Holocaust studies, historical training, and relevant interviewing experience. The agency recognized that while the protester's historical expert hired for this project had an impressive educational background in Holocaust studies and had published numerous articles and books and had lectured extensively on the Holocaust, this individual had no film production experience and, in the agency's opinion, realistically could not be available during all phases of production, including the researching, interviewing, and editing phases, as suggested by the protester, to provide the needed historical context for the individual interviews and the completed film.

The protester argues that since a background in Holocaust studies, historical training, and interviewing experience were not listed as evaluation criteria in the solicitation, it was improperly evaluated and downgraded for not having these qualifications and experience. The protester also disagrees with the agency's assessment of its proposed use of its historical expert.

While proposals are to be evaluated in accordance with the evaluation criteria set forth in the solicitation, the evaluation criteria by their nature are used to measure the extent to which and how well proposals satisfy agency requirements; they are not a statement of the requirements themselves, which are set forth in other parts of the solicitation. JSA Healthcare Corp., B-242313; B-242313.2, Apr. 19, 1991, 91-1 CPD Para. 388.

Here, the solicitation stated that the film produced for the USHMM would be entitled "Testimony" and would contain newly shot interviews with Holocaust survivors, eyewitnesses who would describe their diverse experiences during the Holocaust without any input from historians or commentators. In our view, while evaluation factor (A) did not expressly state that an offeror would be evaluated for its production team's background in Holocaust studies, historical training, and interviewing experience, the evaluation factor did require the production team members to be qualified and have "relevant experience." In this case, we believe it was clear from the solicitation that since the film was to consist exclusively of interviews with Holocaust survivors, the agency could properly consider whether an offeror had any prior experience in conducting interviews and whether an offeror had any background or familiarity with the historical period of the Holocaust. Thus, under the evaluation criteria in the solicitation, the agency could properly evaluate the extent to which the protester possessed interviewing skills which were the key to achieving meaningful and substantive film content and the protester's historical background which was necessary for providing the framework for the film, and by doing so, the agency did not rely on unstated evaluation criteria. Id.

The agency was not persuaded by the proposal that the protester's experience and background were relevant to the contract requirements or that the protester's capability to do a film based on interviewing Holocaust survivors was established. We therefore find reasonable the agency's conclusions that the protester's qualifications and relevant experience for the work to be accomplished were unacceptable.

Finally, the protester's disagreement with the agency's evaluation of the feasibility of its proposed approach for the use of its historical expert is not enough to show that the evaluation was unreasonable. Horizon Trading Co., Inc.; Drexel Heritage Furnishings, Inc., B-231177; B-231177.2, July 26, 1988, 88-2 CPD Para. 86. With respect to the protester's understanding of the production's purpose and subject matter, the solicitation required an offeror to submit a brief narrative describing its understanding of the purpose and subject matter of the film production, with special emphasis on the offeror's "dedication to historical authenticity and accuracy and its ability to synthesize history and interpret human experience in a way that transcends the materials to be used." In its narrative, the protester stated it "understood ... the purpose of the proposed production as survivors telling in their own words of how they and others they knew defied or resisted the Nazi death machine." Concerning its dedication to historical authenticity and accuracy, the protester stated that "where questions arise regarding objective, factual information, it would rely on its historical expert and the USHMM's staff to reconcile any areas in doubt or such information would not be used in the film." Concerning its ability to synthesize history and interpret human experience, it generally referenced documentaries it had produced on various addictions and the addicted individuals' accounts of "the agonies" they faced and it referenced a documentary involving the history of "the Wild West." The protester also stated that it "has filmed" at a concentration camp in Germany and at the Holocaust memorial in Israel. The protester stated that these experiences were "profoundly moving and gave an insight into the horrors of Holocaust events."

Based on the narrative, the agency expressed concern that the protester had no more that a general understanding of the Holocaust, and did not comprehend the complexity and depth of the subject matter to be covered. The agency found that the protester's narrative lacked depth or subtlety of understanding, and was concerned that the superficiality of the narrative was indicative of the quality of the film which the protester would produce. Referencing the protester's statement on historical authenticity and accuracy, the agency criticized the protester's intention to make the USHMM staff responsible to any degree for the film's historical content. The agency found that the protester's references to subject matters of previous productions were not relevant to reflecting its understanding of the requirements for this project. Finally, the agency questioned why the protester did not fully describe the context and purpose of its filming experiences at the German concentration camp and the Holocaust memorial in Israel. The agency stated that the protester did not link these experiences to its proposal for satisfying the film production requirements for this project.

The protester disagrees with the agency's evaluation of its understanding of the production's purpose and subject matter based on the statements it made in its narrative. The evaluation of a personal "narrative," just like the evaluation of any other piece of personally created "artwork," by its nature is an extremely subjective exercise, and the fact that a protester may disagree with the agency's judgment does not invalidate it. See ImageMatrix, Inc., B-243367, July 16, 1991, 91-2 CPD Para. 61; Centurion Films, Inc., B-205570, Mar. 25, 1982, 82-1 CPD Para. 285. Here, we find reasonable the agency's judgment that the protester, in its narrative, makes general, cursory statements concerning the historical period of the Holocaust which reflected the protester's lack of understanding of the solicitation requirements. For example, the protester paraphrases language from the solicitation itself in attempting to convey its understanding. While the protester now states it did not intend that the staff of the USHMM be responsible for the historical content of the film, limiting the role of the museum staff to approving interviewees not registered in the museum's archives as provided by the solicitation, the protester did not articulate this approach in its narrative submitted as part of its proposal. The protester also states in its narrative only that it "has filmed" at a concentration camp and another Holocaust memorial without providing details explaining the objectives and themes of these filmings and without relating the impact of these personal experiences on its understanding of the requirements of this solicitation. Given the subjective nature of the evaluation of a narrative, in our view, the agency could reasonably conclude that the protester's understanding of the production's purpose and subject matter was unacceptable.

The record also shows that the agency prepared a level-of-effort analysis for each offeror based primarily on an offeror's price proposal in order to ascertain each firm's understanding of the scope of work. The contracting officer, in her competitive range determination, reviewed these analyses and found that only two offerors allocated the time and resources which the agency believed were necessary to perform the contract requirements. For example, interviewing Holocaust survivors was considered an extremely labor intensive effort. The selection of the right persons to tell their stories involved research and screening. The interviewing process would most likely be stressful and lengthy, and the editing process, required to produce a cohesive film, was considered to involve significant time. Here, the protester's level-of-effort for researching, shooting, and editing was significantly lower than the levels -of-effort proposed by the two competitive range offerors. Thus, the protester's argument that based on its lower price, its proposal should have been included in the competitive range is without merit since the agency, we think reasonably, viewed that lower price as actually reflecting the protester's inadequate understanding of the contract requirements.

We find the agency reasonably evaluated the protester's technical and price proposals as unacceptable. Further, we have no basis to question the contracting officer's conclusion that the deficiencies in the protester's proposal concerning its qualifications, relevant experience, and its understanding of the contract requirements and scope of work were so material that major revisions of the protester's proposal would be necessary. Accordingly, we find that the contracting officer properly excluded the protester's proposal from the competitive range. National Contract Management Servs., B-240564, Dec. 3, 1990, 90-2 CPD Para. 446.

Finally, although the protester challenges the agency's evaluation of Wentworth's proposal and the award to that firm, under our Bid Protest Regulations, a party is not interested to maintain a protest if it would not be in line for award if the protest were sustained. 4 C.F.R. Sec. 21.0(a) and 21.1(a) (1991). Since the agency properly found the protester technically unacceptable, and since there was another acceptable proposal other than Wentworth's in the competitive range, the protester is not an interested party for purposes of challenging the award to Wentworth. Hughes Technical Servs. Co., supra.

Accordingly, the protest is denied in part and dismissed in part.

GAO Contacts

Office of Public Affairs