B-234379, Jun 9, 1989

B-234379, Jun 9, 1989

PROCUREMENT - Competitive Negotiation - Competitive advantage - Non prejudicial allegation DIGEST: Protest that review and selection process for supplier of basal reading materials for Department of Defense Dependent's Schools was tainted by awardee's allegedly improper submission of certain information to reviewers is denied. While it is possible that allegedly improper information had some impact on the review process, it is extremely doubtful that the impact could have been more than a few points out of 1,500 possible points. Moreover, given the fact that agency's price analysis shows that awardee's program will cost less than protester's, there is no legal basis upon which to object to the selection.

Harcourt Brace Jovanovich, Inc.:

Harcourt Brace Jovanovich, Inc., protests the selection of the Macmillan Publishing Company, by the Department of Defense Office of Dependents' Schools (DODDS) as its source for basal reading materials. /1/ Harcourt contends that the review and selection process conducted by DODDS was tainted by Macmillan's submission of certain information to the reviewers that DODDS had directed all participating publishers not to submit. deny the protest.

BACKGROUND

DODDS is responsible for operating schools for the minor dependents of Department of Defense military and civilian employees assigned overseas. Defense Dependent's Education Act of 1978, 20 U.S.C. Secs. 921-932 (1982). To provide instructional materials, such as textbooks, for its entire school system, DODDS employs a review and selection process of materials submitted by publishers, referred to as an "adoption." Materials selected in the review process are then acquired for DODDS by the Defense Logistics Agency (DLA) as the contracting activity.

The record shows that the materials review process is an integral part of DODDS education program development plan, a 7-year cycle of selection, use and evaluation of instructional materials. Under that plan, which is described in DODDS Manual 2000.5, Sept. 30, 1987, DODDS establishes curriculum, scope and sequence objectives and then reviews and selects for use in its schools those instructional materials which best satisfy those objectives. After DODDS selects the education materials of a particular publisher, those materials are ordered from that publisher under an existing requirements contract which the agency maintains with each publisher. Typically the materials are used over the 7-year cycle, with certain consumable materials such as individual student workbooks ordered annually.

Here, DODDS conducted a two-step review process for textbooks and related materials for basal reading in kindergarten through grade eight. letter dated October 28, 1987, DODDS informed publishers of basal readers of the initial phase of the review process, which was to be conducted in DODDS' Germany region. Based on the initial review process, DODDS explains that out of eight publishers that submitted materials, its reviewers selected the basal reading materials of Harcourt, Macmillan, Scott-Foresman and Company and Houghton-Mifflin Company.

In a letter dated April 26, 1988, DODDS notified the four publishers that they had been selected for the final review which would be conducted in all five of DODDS' regions during the first semester of the 1988 - 1989 school year. The April 26 letter instructed the publishers to send to each of 20 review sites a set of the following basal reading materials:

1. Student Texts

2. Student Workbook/Practice Book

3. Teacher's Edition Of Text

4. Teacher's Edition Of Workbook

5. Placement Tests

6. Unit Tests

7. End of Book Tests

8. Student Record Card/Sheet

9. Instructional Charts

10. Teacher's Resource Book

11. Publisher's Scope & Sequence

12. First Grade Word Cards

13. Kindergarten/First Grade Big Books

14. Kindergarten Kit

15. Program Specific Computer Software (Instructional and Management)

16. Video Tape Program Overview If Available (3/4 Inch; not To Exceed 20 Minutes)

With respect to the video tape, item number 16, the memorandum also stated:

"Item number 16 above asks that you include if available, a 3/4" video tape overview of your basal reading system to each review site. If you do send an overview tape, please make sure it does not exceed 20 minutes in length."

Among other matters, the April 26 letter also cautioned that to ensure the integrity of the competitive process the publishers were expected to comply with an attached DODDS policy statement concerning contacts by sales representatives with school-level staff. That statement says that DODDS employees (1) are asked not to respond to surveys, requests for program comparisons or to provide information relative to specific materials and, (2) will not invite or receive publishers/supplier personnel on school grounds during the school day unless the visit is approved by the DODDS regional director.

The April 26 letter also included a series of questions and instructed the publishers to respond in writing to DODDS' program review committee, rather than the review sites. Publishers were asked whether they would agree to sell all components of the adopted materials at the same price for the remainder of the 7-year adoption period, whether they would provide teacher's editions and other supplementary materials at no cost and at what level and cost to DODDS would the publisher agree to provide inservice training. /2/

All four publishers submitted materials and responded to the questions in the April 26 letter. According to DODDS, all four agreed to provide free inservice training, teachers' editions and other ancillary materials and the free materials offered by each of the four were equivalent in value.

The four publishers also submitted video tapes with their textbooks and other materials which were to be sent to the review sites. When they viewed the video tapes before the final review process, DODDS officials noticed that Macmillan's tape discussed free materials and inservice training. Since the tapes were already distributed and the review process was about to begin, DODDS explains that it decided to allow the Macmillan tape to be played but states that all the reviewers were to be instructed by reading coordinators at all the review sites to disregard any mention of free materials or inservice training since all publishers offer the same package. DODDS also decided that all reviewers would be reminded to disregard "puffery" and marketing techniques in the tapes because the sole responsibility of the reviewers was to evaluate the quality of the educational materials. According to DODDS, its reading coordinators gave these instructions at all review sites before they played the video tapes.

DODDS conducted its final review of the basal reading materials in October and November 1988. Approximately 500 reviewers, consisting of DODDS teachers, reading specialists administrators and a few parents and students congregated at 20 sites in five regions. According to DODDS, after playing all four video tapes, and instructing reviewers to ignore the discussion of free materials and inservice training, each reviewer read a particular publisher's reading materials for a specific grade level and then completed a 12-page evaluation instrument for that publisher. The evaluation instrument allowed reviewers to rate each publisher's materials for readability, organization, instructional methods, testing materials, and other criteria and allowed reviewers to make written comments. Then, each evaluator proceeded to review materials of each of the other publishers and complete evaluation instruments in the same manner. Thus, each of the approximately 500 reviewers completed a 12-page evaluation instrument on each of the 4 publishers.

After each reviewer completed his or her four evaluation instruments, they each transferred the individual scores onto a ballot, recording the scores in rank order from the highest score to the lowest. The ballots were forwarded to DODDS headquarters. The totals of those rankings resulted in the following final scores: /3/ Macmillan, 1028; Harcourt, 1032; Scott-Foresman, 1238; and Houghton Miffin, 1252. Since the scores were based on the total of the rankings, "1" through "4," the "best" score was the lowest, in this case Macmillan's.

DODDS explains that during the final review, it performed a price analysis of the publishers' instructional materials and found that the average cost of the four publishers' materials was between $48 and $53 per student. Based on the results of this analysis, DODDS determined that the cost of each of the four publishers' programs was "comparable" and within DODDS' budget estimate. In other words, according to DODDS, each of the four programs would have been acceptable to DODDS from a price standpoint, although the purchase "of those at the higher end of the range may have required ordering less materials."

By letter of November 23, 1988, DODDS informed the four publishers that it had selected Macmillan's basal reading program in the final review. According to that letter, DODDS anticipated that Macmillan's materials would be ordered so that they will be available for use at the beginning of the 1989-90 school year but that the decision on ordering would not be made until the fiscal year 1989 budget was approved. After requesting and receiving a debriefing, Harcourt protested the selection of Macmillan to this Office.

On April 26, after the protest was filed, DLA determined pursuant to 31 U.S.C. Sec. 3553(c)(2)(A) (Supp. IV 1986) that urgent and compelling circumstances significantly affecting the interests of the United States did not permit withholding award and delaying performance of the contract pending our decision on the protest. DLA modified an existing Macmillan requirements contract, No. DLA420-89-D-004, for educational materials, to purchase the basal reading materials.

ISSUES RAISED

Harcourt's principal contention is that Macmillan had an unfair competitive advantage because its video tape discussed free supplementary materials and inservice training and therefore DODDS' evaluation of the educational materials was flawed. The protester argues that, based on DODDS' past practice and under the ground rules of this selection process, the publishers were not permitted to attempt to influence the selection process by informing the evaluators of their free supplementary materials or inservice training. In this respect, Harcourt says that on three different occasions, DODDS' headquarters reading coordinator told Harcourt that each publisher's submissions, of which the video tape was a significant part, should address the content and organization of the reading program and that the reading coordinator specifically told Harcourt not to provide any information to the review sites regarding free materials or the amount or extent of inservice training. Further, Harcourt says that the reading coordinator indicated that DODDS officials gave the same instructions to each of the other three publishers involved in the final review.

According to Harcourt, as a result of the April 26 letter and other agency communications, all four publishers were also aware that the video tape was not to exceed 20 minutes in length and that they were not to send any promotional literature to DODDS school-level personnel after August 1.

Harcourt says that, in spite of these ground rules, Macmillan's video tape ran approximately 27 minutes, and included repeated references to free supplementary materials and inservice training and that Macmillan sent materials to DODDS' reviewers after the August 1 cut-off date. Further, Harcourt maintains that even though DODDS officials were aware that Macmillan's tape was too long and discussed free materials and inservice training, DODDS officials did not take appropriate corrective action.

In response, DODDS says that it did not specifically instruct publishers to omit references to free supplementary materials and inservice training from their video tapes but rather, in phone calls to all of the publishers, DODDS' headquarters reading coordinator instructed the publishers to focus on the content of their basal reading programs. According to DODDS, when Harcourt asked if its tape could refer to free materials, the reading coordinator simply said that it should focus on content and gave no directive not to mention free materials. DODDS also says that no other publisher asked about free materials and that there was no past practice relating to video tapes since they had never been used before in a textbook review.

With respect to the August 1 cut-off date, DODDS explains that in accordance with its usual practice, it requested that publishers voluntarily cease sending promotional literature to schools after that date. According to DODDS, this was an oral request, not a prohibition, that it typically makes before textbook reviews simply to avoid overloading DODDS' mail and distribution system with promotional literature. Further, DODDS says that it checked with its schools and it is unaware of any publishers' promotional literature mailed to the schools after August 1, although some may have been received after that date.

DODDS also argues that, even if Macmillan's video tape violated the selection process ground rules, the effect of that violation was neutralized by the instructions which the regional reading coordinators gave to all reviewers at the review sites. In this respect, DODDS explains that the coordinators reminded the reviewers of the industry practice of providing free materials and inservice training and that all four publishers in the final review offered virtually identical packages of free teacher's editions, free ancillary materials and free inservice training. On the other hand, the protester maintains that reviewers who saw all four video tapes could only have been confused and influenced by the fact that only one publisher mentioned free materials or inservice training on its video tape.

Harcourt also argues that the extreme closeness of its score to Macmillan's demonstrates the prejudice to Harcourt in DODDS allowing Macmillan to violate the ground rules. In this respect, since the two firms were separated by only four points, Harcourt argues that a shift of only five points from Macmillan to Harcourt would have altered the outcome of the competition and a change in the scoring of only a few evaluators out of approximately 500 could have caused that shift.

According to Harcourt, DODDS' instructions to the evaluators to disregard discussion of free materials and inservice training on the Macmillan video tape were ineffective. In this respect, Harcourt argues that comments by a number of the evaluators on the evaluation instruments and on questionnaires completed by some evaluators to assess the selection process indicate that some evaluators reacted strongly to the video tapes and other evaluators expressed concerns with the cost of supplementary materials of some publishers and with the necessity of extensive inservice training with some publishers' materials.

As a consequence of the defective evaluation procedure and the prejudice to the other publishers, the protester requests that we recommend that the selection of Macmillan be rescinded and a new final review process conducted.

DODDS responds that the record indicates that Macmillan's video tape had no significant effect on the review and selection process. First, DODDS says that in affidavits prepared in response to the protest, a number of the reviewers indicated that they were not influenced by Macmillan's video tape because they were already aware that all publishers provide essentially the same free supplementary materials and training and because they had been instructed to focus on the content of the reading programs, not marketing ploys, price and offers of free materials. DODDS also says that the reviewer evaluation instruments and the final review questionnaires filled out by some reviewers do not indicate any prejudicial effect as a result of Macmillan's video tape. For example, DODDS points out that there are only two references to video tapes in the evaluation forms and those particular reviewers made negative comments that Macmillan's tape included too much "hardcore" selling. Also, DODDS points out that only 11 of the approximately 220 questionnaires mentioned video tapes and that those comments do not indicate that the tapes biased the outcome of the review.

ANALYSIS

Harcourt's protest focuses on the fact that its final review score and that of Macmillan were separated by only four out of 1,500 possible points. /4/ Given the closeness of the scoring, Harcourt maintains that there is more than a reasonable possibility that it was displaced by Macmillan's alleged unfair competitive advantage.

After reviewing the extensive record relating to the selection procedure, including reviewer affidavits, a number of final review evaluation forms and evaluator questionnaires, we cannot conclude that information concerning the free materials and inservice training had no impact at all on the review. We nevertheless do not believe that the impact of those materials affected the selection.

First, it is not clear that Macmillan violated the rules of the selection process by discussing free supplementary materials and inservice training on its video tape. The written instructions which DODDS gave the publishers did not address this issue, and, although the protester understood otherwise, according to DODDS, the oral instructions it gave the publishers simply asked that they focus on the content of their programs in the video tapes and did not specifically prohibit the mention of free materials or inservice training.

Nonetheless, if we assume that the ground rules of the review were as Harcourt describes them, we think the record falls short of establishing the likelihood that the selection decision would have been different. do note that a number of reviewer comments on the evaluation forms and the questionnaires lend some support to the protester's contention that Macmillan's video tape had some influence. For example, a few reviewers stressed the importance and cost of supplementary materials, raising the possibility that those reviewers could have been influenced by the knowledge of what free materials one publisher, Macmillan, would provide. Also, a few reviewers mentioned that some or all of the publishers' programs would need inservice training, possibly indicating that those reviewers would favor a program, such as Macmillan's, if they were specifically told in the video tape that inservice training would be provided.

We are persuaded by the record before us, however, that the reviewers, who were mostly professional educators and administrators, were able to focus on their assigned task-- evaluating the merits of the publishers' programs-- and were not significantly sidetracked by the video tapes. There is nothing in the record to indicate that any significant number of reviewers ignored DODDS' corrective instructions to the reviewers and the explanation that all publishers provide similar free supplemental materials and inservice training. /5/ Moreover, out of the approximately 500 evaluators, the protester points to none who specifically indicated on an evaluation form or a questionnaire that he or she had favored or opposed a particular publisher in the scoring as a result of free materials or training. Thus, while it is possible that a small number of reviewers were influenced by the Macmillan video tape, based on the record before us, and the lack of any direct evidence that the tape had any significant influence, we think it is extremely doubtful that the impact of Macmillan's video tape could have been more than a few points.

That being so, we think it is clear that a downward adjustment of Macmillan's total score by a few points would not change the selection decision since, from a technical standpoint, the competing proposals would have been regarded as essentially equal. When proposals are viewed as essentially equal technically, price properly becomes the determining factor in the selection of an awardee. See Inlingua Schools of Languages, B-229784, Apr. 5, 1988, 88-1 CPD Para. 340. Here, Macmillan's program was less costly than the protester's by more than $400,000. Thus, while the agency viewed the costs of each publisher's program as "comparable," under applicable law it could not ignore the cost savings associated with Macmillan's program. Accordingly, we think the selection of Macmillan was consistent with the rules for source selection in these circumstances.

The protest is denied.

/1/ Basal reading materials are used in kindergarten through grade eight for reading instruction.

/2/ Supplementary materials, some of which are provided at no charge, include items ancillary to textbooks such as workbooks, activity books, video tapes and test booklets. Inservice training generally consists of workshops conducted over the life of a textbook adoption by the publishers for teachers and administrators. The workshops focus on teaching methods and management of a particular textbook program.

/3/ A "1" ranking was assigned for the highest score on each ballot, a "2" ranking for the second highest score on each ballot, and so on for each of the approximately 500 ballots.

/4/ The range was 1,500 points because the best possible score was 500 and the worst possible score was 2,000.

/5/ Further, although the protester argues that Macmillan's tape was 7 minutes longer than allowed, we do not see how the length of the tape was inherently prejudicial. With respect to the August 1 cut-off date for mailings to school staff, there is no evidence in the record that Macmillan mailed any promotional literature after that date.