Skip to main content

B-242767, B-242767.2, Jun 5, 1991, 91-1 CPD ***

B-242767,B-242767.2 Jun 05, 1991
Jump To:
Skip to Highlights

Highlights

Protest that an agency improperly conducted procurement by using alternative source selection procedures is dismissed as untimely under the General Accounting Office Bid Protest Regulations. Agency properly evaluated proposals where the agency's scoring of the proposals was reasonable and related to the solicitation's stated evaluation criteria. Agency conducted meaningful discussions where the agency's questions were sufficient to direct the protester to the primary area of concern about its proposal. Protest that agency improperly included protester's proposal in the competitive range is denied where all offerors had significant omissions in their initial proposals and the protester could have become substantially more competitive as a result of discussions.

View Decision

B-242767, B-242767.2, Jun 5, 1991, 91-1 CPD ***

PROCUREMENT - Bid Protests - GAO procedures - Protest timeliness - 10 day rule DIGEST: 1. Protest that an agency improperly conducted procurement by using alternative source selection procedures is dismissed as untimely under the General Accounting Office Bid Protest Regulations, where the protester learned the basis of its protest from an agency letter that opened discussions and did not protest the agency's use of the procedures until several months after agency had both closed discussions and received best and final offers. PROCUREMENT - Competitive Negotiation - Offers - Evaluation errors - Evaluation criteria - Application 2. Agency properly evaluated proposals where the agency's scoring of the proposals was reasonable and related to the solicitation's stated evaluation criteria. PROCUREMENT - Competitive Negotiation - Discussion - Adequacy - Criteria 3. Agency conducted meaningful discussions where the agency's questions were sufficient to direct the protester to the primary area of concern about its proposal. PROCUREMENT - Competitive Negotiation - Offers - Competitive ranges Inclusion - Administrative discretion PROCUREMENT - Competitive Negotiation - Discussion - Adequacy - Criteria 4. Protest that agency improperly included protester's proposal in the competitive range is denied where all offerors had significant omissions in their initial proposals and the protester could have become substantially more competitive as a result of discussions.

Attorneys

Caldwell Consulting Associates:

Caldwell Consulting Associates protests the National Aeronautics and Space Administration's (NASA) selection of Kalman Company, Inc. for the award of a time and materials contract under request for proposals (RFP) No. 2-33319- (BXC), issued by the NASA Ames Research Center for contracting officer technical representative (COTR) training services (course development, presentation and maintenance) for a base year and 4 option years. /1/ Caldwell also protests NASA's choice of procurement procedures, evaluation of proposals, conduct of discussions, and decision to include Caldwell in the competitive range.

We dismiss in part and deny in part the protests.

BACKGROUND

The RFP called for the submission of cost and technical proposals. Cost proposals were to contain detailed cost data, substantiation, /2/ and a complete cost breakout of the offeror's "mock proposal." /3/ Technical proposals were to demonstrate the offerors' understanding of NASA's requirements, proposed approach, resources and proposed key personnel.

The RFP provided for award to the offeror whose conforming proposal would be most advantageous to the government, cost or price and other factors considered. Federal Acquisition Regulation (FAR) Sec. 52.215 16. The RFP identified four principal evaluation factors: (1) mission suitability; /4/ (2) cost; /5/ (3) experience and past performance; /6/ and (4) other factors. /7/ Only the mission suitability factor was numerically scored; NASA evaluated the rest of the factors using adjectival ratings. The RFP stated that all four factors were generally of equal importance-- in the sense that NASA could reject a proposal for reasons attributable to any single factor-- and that the award evaluation would rest on an integrated assessment of all four factors.

On June 30, 1988, seven offerors submitted proposals. NASA's initial evaluation only covered two factors (mission suitability and cost) because all offerors failed to furnish information necessary to evaluate the rest of the factors. Under the mission suitability factor, NASA ranked Kalman fifth and Caldwell last. /8/ Under the cost factor, NASA determined that Kalman had the second lowest probable cost, and that Caldwell's proposal could not be evaluated. /9/ The agency decided that all proposals should be included in the competitive range since there were significant uncertainties in all proposals received (e. g., two out of four factors had not even been evaluated).

More than 2 years later, on August 23, 1990, NASA amended the RFP to clarify both the cost proposal requirements and the evaluation of probable cost, and called for written responses to agency questions concerning the firms' proposals by September 7, 1990. NASA addressed 14 questions and requests to Caldwell concerning its proposal. On September 14, NASA invited proposal revisions-- in effect, best and final offers (BAFO)-- from all offerors by October 9. During this period, one offeror withdrew from the competition leaving a competitive range of six.

NASA's final evaluation considered the offerors' answers to the questions and BAFOs. Under the mission suitability factor, little changed from the initial evaluation-- Kalman's ranking advanced to fourth place and Caldwell's proposal remained in last place. Also, Kalman and Caldwell received the same good/excellent ratings for experience and past performance, while Kalman received a good rating and Caldwell a fair rating for "other factors." For "validity of proposed costs," Kalman received a fair rating /10/ while Caldwell received a poor rating. /11/ Kalman proposed the lowest cost and was evaluated as having the lowest probable cost. Caldwell's proposed cost was second low, but its evaluated cost was the highest of the six offerors.

NASA concluded that Kalman's BAFO offered the most advantageous proposal meeting the RFP's requirements. In making its cost/technical tradeoff, NASA considered (1) Kalman's substantial advantage over the other offerors in probable cost (Kalman's BAFO had lowered its initial prices); (2) Kalman's minor disadvantages in both mission suitability and experience; and (3) the lack of meaningful differences among the proposals in the area of "other factors."

UNTIMELY ISSUES

Caldwell first protests NASA's choice of procurement procedures, specifically NASA's use of its alternative source selection procedures, 48 C.F.R. Sec. 1815.613 (1990), and the alternative procedures' sanctioning of the practice of selecting a single firm for negotiations after evaluating BAFOs.

We find these arguments untimely. Caldwell received, and on September 5 responded to, NASA's August 23 letter transmitting an RFP amendment and 14 questions to Caldwell. The letter stated that "written and Oral Discussions will now be opened in accordance with NASA FAR Supplement (NFS) 18-15.613-71(b)(5)(ii)" (i. e., NASA's alternative source selection procedures). In our view, Caldwell knew or should have known from this letter that NASA was proceeding under the protested alternative procedures. See ABC Bldg. Servs., B-220320, Jan. 27, 1986, 86-1 CPD Para. 91, aff'd, B-220320.2, May 29, 1986, 86-1 CPD Para. 501.

Under our Bid Protest Regulations, 4 C.F.R. Secs. 21.2(a)(1), (2) (1991), protests based on improprieties that are incorporated into a solicitation after the receipt of initial proposals must be filed not later than the next closing date for receipt of proposals; in other cases, protests must be filed no later than 10 days after the basis of protest is known or should have been known, whichever is earlier. Caldwell's protests, filed respectively on January 28, 1991 and March 4, 1991, more than 4 months after NASA closed discussions and more than 3 months after the October 9 closing date for BAFOs, are therefore untimely. See Loral Def. Sys.-- Arizona, B-240537, Nov. 16, 1990, 90-2 CPD Para. 399.

To the extent that Caldwell's protests challenge the RFP's specification of a time and materials contract /12/ as opposed to a fixed-price contract, these allegations are untimely since the contract type was apparent on the face of the solicitation. Our Bid Protest Regulations require that protests, based upon alleged improprieties in a solicitation that are apparent prior to the closing date for receipt of initial proposals, must be filed by the closing date. 4 C.F.R. 21.2(a)(1); Englehard Corp., B-237824, Mar. 23, 1990, 90-1 CPD Para. 324.

Finally, Caldwell's complaints about the timeliness of this procurement are untimely since they were not protested prior to the BAFO due date. C.F.R. Sec. 21.2(a)(1).

FAIRNESS OF EVALUATION

Caldwell contends that NASA's evaluation was unfair because (1) NASA did not apply the stated evaluation criteria to the contending proposals in an even-handed manner, and (2) NASA did not accord the stated evaluation factors their announced weights during the evaluation in that it placed inordinate emphasis on probable cost.

The evaluation of technical proposals is a matter within the discretion of the contracting agency since the agency is responsible for defining its needs and the best method of accommodating them. Science Sys. and Applications, Inc., B-240311; B-240-311.2, Nov. 9, 1990, 90-2 CPD Para. 381. In reviewing an agency's technical evaluation, we will not reevaluate the proposal, but instead will examine the agency's evaluation to ensure that it was reasonable and not in violation of the procurement laws and regulations. Information Sys. Networks Corp., 69 Comp.Gen. 239 (1990), 90-1 CPD Para. 203. We also bear in mind that the offeror has the burden of submitting adequately written proposals and proposal revisions for the agency to evaluate, Complere, Inc. B-22782, Sept. 15, 1987, 87-2 CPD Para. 254, and that an offeror's disagreement with the agency's judgment is not sufficient to establish that the agency acted unreasonably. United HealthServ Inc., B-232640 et al., Jan. 18, 1989, 89-1 CPD Para. 43.

We have reviewed Caldwell's arguments, its proposal, Kalman's proposal, the evaluators' worksheets, the source selection evaluation reports, the agency report and other NASA submissions, and discern no basis for finding the evaluation of either Caldwell's or Kalman's proposals unfair or unreasonable.

The record shows that NASA found Caldwell to be acceptable, but with a number of notable weaknesses. The fundamental shortcoming was Caldwell's apparent insistence on providing its standard course when NASA sought a course custom tailored to address the procurement/contracting problems currently existing at NASA's Ames facilities.

In this regard, Caldwell's BAFO was downgraded both for not appreciating that course development work was necessary before the first class was presented, and for asserting that Caldwell could gather information required to address Ames' acquisition procedures from NASA Headquarters in Washington, D.C. NASA thought it unlikely that its headquarters personnel would know enough about the "unique culture and problems" at the California installations to provide the kind of information required to develop an Ames-specific training course. In contrast, Kalman (and other offerors) proposed a course development plan, as requested by the RFP, involving interaction with Ames personnel in California, which NASA considered acceptable. Caldwell was also downgraded for apparently limiting its course research to classroom feedback and monitoring of current procurement publications. /13/ We find NASA's downgrading of Caldwell's proposal was reasonable since the RFP clearly contemplated a course addressing the situation at Ames, and because course development is a significant aspect of the requirement-- i.e., it is one of only three line items in the RFP.

NASA also downgraded Caldwell's proposal because its proposed course outline was based primarily on pre-award functions. NASA reasonably viewed this with concern since the COTRs' primary functions involve post- award matters.

Caldwell contends that NASA was less than even-handed in assessing the proposals. For example, Caldwell argues that it was unfair for NASA to evaluate one offeror's offer to "use ... generic course material where possible" as a major strength and simultaneously view Caldwell's offer of a standard course as a major weakness. However, the other offerors proposed the development of an Ames-specific course into which it would introduce generic materials where possible, while Caldwell planned to present its standard course with adjustments made before and after each training session. /14/ We see nothing unfair in NASA's evaluation of the two proposals.

Caldwell contends that it was unfair for NASA to designate Caldwell's failure to use on-site interviews with Ames personnel as part of Caldwell's course research effort a "serious deficiency," when NASA considered the same matter to be only a minor weakness in another offeror's proposal. We think NASA could reasonably view the absence of on -site interviews from an offeror's otherwise satisfactory course research effort as a less serious concern than the absence of the same research from a proposal, such as Caldwell's, that NASA read as offering very questionable course research.

Caldwell contends that NASA's rating of Caldwell's key personnel as only a minor instead of a major strength is improper because NASA recognized Caldwell's excellent past performance. Caldwell states that if its past performance is highly regarded, its personnel necessarily must also be so regarded. It does not necessarily follow that Caldwell's excellent rating for past performance is attributable to the efforts of its current personnel. Moreover, the key personnel subfactor consists of more than personnel qualifications and experience; it also weighs the offeror's approach for backup and replacement of key personnel. In short, we see no basis to conclude that the one subfactor is related to the other subfactor such that the ratings under both should be the same.

With regard to the cost evaluation, Caldwell argues that NASA selected Kalman without adjusting Kalman's underestimated travel costs. The record, however, shows that NASA recognized the problem with Kalman's travel costs and adjusted Kalman's probable costs accordingly.

Finally, Caldwell makes a series of allegations concerning alleged inconsistencies in Caldwell's and other offerors' ratings, e.g., NASA's failure to give Caldwell credit for proposing a firm, fixed-price contract, even though this was not called for by the RFP. We have reviewed all these contentions and find them meritless.

Caldwell contends that NASA did not accord the stated evaluation factors their announced weights during the evaluation in that NASA disregarded the RFP advice that the evaluation factors were generally equal in weight and placed an inordinate emphasis on probable cost. We disagree. The RFP provided that "the Source Selection Official will make his/her decision on the basis of an integrated assessment of all four groups of factors." The record shows the agency made an appropriate cost/technical tradeoff and decided that the technical advantages in the higher cost, higher technically rated offers did not offset Kalman's cost advantage. See ABC Bldg. Servs., B-220320, supra. In any case, Caldwell is not prejudiced by the relative weight given cost in the selection, since it was low-rated technically and had a higher evaluated cost than Kalman. In sum, our review of the record shows that NASA reasonably evaluated the Kalman and Caldwell proposals, and that the selection was in accordance with the evaluation criteria. DISCUSSIONS

Caldwell contends that NASA failed to conduct meaningful discussions. The governing provision of the Competition in Contracting Act of 1984, 10 U.S.C. Sec. 2305(b)(4)(B) (1988), requires that written or oral discussions be held with all responsible sources whose proposals are within the competitive range. Price Waterhouse, 65 Comp.Gen. 205 (1986), 86-1 CPD Para. 54, aff'd, B-220049.2, Apr. 7, 1986, 86-1 CPD Para. 333. However, while agencies generally must conduct discussions with all offerors in the competitive range, advising them of deficiencies in their proposals and offering them the opportunity to submit revised proposals, this does not mean that offerors are entitled to all-encompassing discussions. Agencies are only required to lead offerors into areas of their proposals that require amplification. Fairchild Weston Sys., Inc., B-229568.2, Apr. 22, 1988, 88-1 CPD Para. 394. Where a proposal is considered to be acceptable and in the competitive range, an agency is not obligated to discuss every aspect of the proposal that receives less than the maximum possible score. Mech El Inc., B-233092, Feb. 21, 1989, 89-1 CPD Para. 175.

As discussed above, the basic flaw of Caldwell's proposal is that it was based on its standard COTR course rather than one custom tailored for the Ames facilities. Caldwell, in effect, argues that it did not realize that this procurement was not susceptible to its standard course approach and that NASA had a duty during discussions to so alert it and inform it of the unique aspects of the Ames facilities' "mission."

It is true that NASA did not expressly advise Caldwell during discussions that its basic approach was not considered sound, in that its proposed course approach did not appear to be customized to the Ames mission. Nevertheless, we think the following questions and requests addressed by NASA to Caldwell during discussions should have reasonably led Caldwell to the areas of concern about its proposal:

"How do you plan to acquire an adequate knowledge of NASA-Ames acquisition procedures?"

"How will the course research be accomplished?"

"Your explanation of course design lacks detail and does not address specific issues pertinent to Ames Research Center's acquisition procedures. Please expand your discussion to define the techniques and approaches generally used and identify those that would be used on the sample courses in RFP Appendix 7 mock proposal."

"How will the video tapes enhance and apply specifically to the Ames experiences and necessary COTR knowledge."

These discussion questions and requests reflect NASA's concern for a COTR course tailor-made to the Ames mission. For example, several of the questions/requests concern how Caldwell will develop and research courses that are expressly addressed to the Ames needs, e.g., specific issues related to the Ames acquisition procedures. The question as to how course research will be accomplished strongly implies that something other than an off-the-shelf course was what was required. Indeed, the statement of work expressly states the course should be tailored to each locale. The query regarding how the standard video tapes offered by Caldwell apply to Ames also indicates a concern with Caldwell's standard approach. We think the foregoing should have led an experienced firm to realize that the COTR course must be tailored to meet Ames' specific needs, and that a standard off-the-shelf course may not be acceptable. Consequently, we find the discussions with Caldwell were meaningful. /15/

INCLUSION IN COMPETITIVE RANGE

Caldwell contends that if NASA's initial evaluation disclosed all the flaws that NASA states were present in Caldwell's proposal, NASA should not have included Caldwell in the competitive range. We disagree. agency should not automatically reject a nonconforming proposal in the same manner that it would reject a nonresponsive bid. It is a fundamental purpose of negotiated procurements to determine whether deficient proposals are reasonably susceptible of being made acceptable through discussions. Fairchild Weston Sys., Inc., B-229568. 2, supra. Under NASA's alternative source selection procedures NASA is to include doubtful proposal in the competitive range. 48 C.F.R. Sec. 1815.613- 71(b)(4)((ii). NASA was also required, in projecting Caldwell's potential for proposal improvement during discussions, to base its projection upon the assumption that meaningful discussions with Caldwell could open the door to a significant improvement in its proposal. Joule Eng'g Corp.-- Recon., 64 Comp.Gen. 540 (1985), 85-1 CPD Para. 589.

Here, NASA entertained considerable uncertainty about all the offers. The lack of required information in all of the proposals-- the discrepancy that barred NASA's initial evaluation from ranking all of the evaluation factors-- made it difficult for NASA to predict the fallout of discussions and BAFOs with regard to the ordering of proposals. In our view, NASA reasonably included Caldwell in the competitive range since Caldwell's proposal could have been substantially improved following discussions.

The protests are dismissed in part and denied in part. with this approach. Thus, we find that NASA's failure to address more specific questions in these ancillary areas of concern did not render the discussions not less than meaningful. In any case, as indicated above, NASA was not obligated to mention during discussions all weaknesses found in Caldwell's proposal.

/1/ The RFP called for proposals to address three line items:

(1) course development, which stated:

"The courses shall be developed for scientists, technical personnel, and administrative personnel. The courses shall be up to date with current procurement laws, regulations, policies and procedures for each locale (Ames Moffett and Ames Dryden).";

(2) course presentation at two NASA installations in California (Ames Moffett and Ames Dryden); and

(3) course revision, to keep the courses current and up-to-date.

/2/ The RFP, as amended, required offerors to substantiate the pricing of any items that they claimed were priced using catalog or commercial pricing. The substantiation had to be furnished on a Standard Form 1412 (Claim for Exemption From Submission of Certified Cost or Pricing Data) along with required supporting data.

/3/ Offerors' mock proposals were to price both development and presentation aspects of a 1-day financial management course to be taught at Ames Moffett and a 3-day research and development contract course to be presented at Ames Dryden.

/4/ Subfactors of mission suitability include: (1) key personnel (qualifications and experience, and approach for backup and replacement of key personnel); (2) excellence of approach (course research, course design, and course presentation); (3) understanding the requirement (technical understanding, and work management); and (4) resources (resources proposed, and resource availability).

/5/ Subfactors of cost include: (1) validity of proposed cost; (2) probable cost; and (3) probable cost differences.

/6/ Subfactors of experience and past performance include: (1) experience in performing comparable or relevant work; and (2) past performance (resiliency, resourcefulness, and management determination) on similar contracts.

/7/Subfactors of other factors include: (1) financial condition and capability; and (2) "History of how well costs have been estimated and controlled on similar requirements."

/8/ Caldwell's poor showing resulted from NASA's perception that Caldwell was offering a standard course-- that it would repeat each time-- and did not intend to develop courses specifically addressing the needs of the two NASA Ames facilities.

/9/ Caldwell's was the only proposal lacking sufficient information for evaluation of its probable cost. This resulted from Caldwell's offer of fixed prices for the courses without providing either cost data or substantiation showing that its course pricing was properly based on established catalog or commercial prices. Caldwell also neglected to provide the "mock proposal" cost breakout.

/10/ Kalman's validity of proposed cost was only considered fair because Kalman had substantially underestimated the costs it would incur traveling between Virginia and California.

/11/ Caldwell's cost validity is rated poor because Caldwell did not provide sufficient information to evaluate its costs.

/12/ For example, Caldwell objects to the RFP requirements that offerors substantiate their pricing, provide cost data, and furnish cost breakouts of their "mock proposals."

/13/ In response to NASA's question: "How will the course research be accomplished?" Caldwell replied: "Course 'research' is something we do each day as part of our role as trainers." Caldwell added that it would monitor various procurement publications and finished its answer with the observation that:

"In summary, we do not have to 'create' the material which Ames requires. Instead, we will bring the most current developments in the field to the program, incorporating any special needs you may require into our program. You will find this to be cost effective, since it eliminates 'rediscovery of the wheel.'"

/14/ Caldwell's proposal offered training in the form of 1-day, 2 day and 3-day programs. Caldwell's initial technical proposal (p.4) states that: "Accordingly, Caldwell ... proposes to have the instructor undertake planning meetings with key NASA personnel prior to program presentation, to insure that the materials and the course plans conform to these specific needs. "Furthermore, upon completion of a program, a similar meeting will be held with NASA personnel to review results and determine the extent (if any) to which revisions may be desired."

/15/ Caldwell has submitted a list of questions that it asserts NASA should have asked to accomplish meaningful discussions. However, Caldwell did not modify its proposed approach of basically offering its standard COTR course, after the discussions that should have reasonably apprised it of the perceived concerns

GAO Contacts

Office of Public Affairs