B-182558, MAR 24, 1975

B-182558, MAR 24, 1975

PROTEST ALLEGING IMPROPER SOURCE SELECTION BY NATIONAL SCIENCE FOUNDATION (NSF) IS DENIED, SINCE TECHNICAL EVALUATION INDICATED SUPERIORITY OF SELECTED PROPOSAL NOTWITHSTANDING ITS HIGHER ESTIMATED COSTS AND RECORD DOES NOT SUPPORT CLAIM OF BIAS AGAINST LOW OFFEROR. ALTHOUGH NSF NUMERICALLY SCORED INITIAL PROPOSALS, IT WAS NOT REQUIRED TO DO SO AND ITS FAILURE TO SO SCORE REVISED PROPOSALS IS NOT SUBJECT TO OBJECTION. MOREOVER, NSF PREFERENCE FOR PARTICULAR APPROACH FOR DEVELOPING ASSESSMENT CAPABILITY AND AWARENESS OF ENERGY PROJECTIONS WAS NOT CONTRARY TO RFP PROVISIONS.

DECISION SCIENCES CORPORATION:

THIS PROTEST CHALLENGES THE PROPRIETY OF AN EVALUATION MADE BY THE NATIONAL SCIENCE FOUNDATION (NSF) OF BEST AND FINAL OFFERS SUBMITTED PURSUANT TO A SOLICITATION CALLING FOR OFFERS TO DEVELOP AN ASSESSMENT CAPABILITY AND STATE OF THE ART AWARENESS IN THE AREA OF ENERGY PROJECTIONS AND MODELING. FOR THE REASONS STATED BELOW, THE PROTEST IS DENIED.

REQUEST FOR PROPOSALS (RFP) 75-107 WAS ISSUED ON AUGUST 7, 1974, AND WAS SENT TO 17 ORGANIZATIONS WITH WHICH NSF HAD A BASIC ORDERING AGREEMENT. SEVEN PROPOSALS WERE RECEIVED, AND AFTER EVALUATION THREE, INCLUDING THE PROPOSAL OF DECISION SCIENCES CORPORATION (DSC), WERE DEEMED TO BE IN THE COMPETITIVE RANGE. AFTER NEGOTIATIONS WERE CONDUCTED AND BEST AND FINAL OFFERS RECEIVED, NSF SELECTED THE OFFER OF THE STANFORD RESEARCH INSTITUTE (SRI) AS TECHNICALLY SUPERIOR TO THE OTHERS AND PROPOSED TO MAKE AN AWARD TO THAT FIRM. FOLLOWING A DEBRIEFING, DSC PROTESTED THE SOURCE SELECTION TO THIS OFFICE, CLAIMING THAT THE TECHNICAL EVALUATION WAS ARBITRARY AND NOT CONSISTENT WITH THE EVALUATION FACTORS SET FORTH IN THE RFP, THAT COST WAS NOT PROPERLY CONSIDERED, THAT MEANINGFUL NEGOTIATIONS WERE NOT CONDUCTED WITH IT, AND THAT NSF WAS BIASED AGAINST PROFIT-MAKING FIRMS IN GENERAL AND DSC IN PARTICULAR.

THE SOLICITATION SET FORTH THE FOLLOWING CRITERIA FOR EVALUATION AND SELECTION:

"1. COMPREHENSION OF STUDY REQUIREMENTS - UNDERSTANDING OF THE PROJECT OBJECTIVES, IDENTIFICATION OF PROBLEMS INVOLVED IN THE AREA OF THE STUDY, FAMILIARITY WITH PRESENT AND PAST REVIEWS AND EVALUATIONS OF ENERGY PROJECTIONS.

"2. METHODOLOGY AND RESEARCH DESIGN - GENERAL PLAN OF WORK, FRAMEWORK FOR CLASSIFICATION, COMPETENCE IN MODELING.

"3. ORGANIZATION AND MANAGEMENT - PROJECT MANAGEMENT PLAN, PERSONNEL SKILL MIX AND QUALIFICATIONS ESPECIALLY - IN AREAS OF ENERGY AND MODELING, ORGANIZATION RESOURCES, COMMITMENT AND EXPERIENCE.

"4. PRACTICALITY OF FRAMEWORK FOR COMPARING ALTERNATIVE ENERGY PROJECTIONS AS INPUTS TO POLICY ANALYSIS.

"5. ADEQUACY OF CONTRACTEE IN SATISFYING PROGRAM'S DELIVERY REQUIREMENTS."

THE SOLICITATION STATED THAT APPROXIMATELY EQUAL EMPHASIS WOULD BE PLACED ON EACH OF THE ABOVE FACTORS.

THE INITIAL PROPOSALS WERE EVALUATED BY FOUR NSF TECHNICAL EXPERTS IN THE FIELD OF ENERGY STUDIES. EACH MEMBER OF THIS SELECTION PANEL DEVELOPED WRITTEN COMMENTS ON AND GRADED EACH PROPOSAL IN ACCORDANCE WITH THE ABOVE CRITERIA. ALTHOUGH IT WAS FELT THAT NONE OF THE PROPOSALS WAS SATISFACTORY FOR AWARD PURPOSES - ALL WERE REGARDED AS DEFICIENT IN ONE OR MORE KEY AREAS - THE PANEL BELIEVED THAT THREE PROPOSALS SEEMED CONSIDERABLY CLOSER TO MEETING THE GENERAL OBJECTIVES OF THE PROPOSED STUDY. THE COMPOSITE SCORES AND INITIAL PROPOSED COSTS PLUS FEE OF THESE THREE OFFERORS THUS CONSIDERED TO BE IN THE COMPETITIVE RANGE WERE:

ARTHUR D. LITTLE (ADL) 177 $140,398

DSC 148 80,357

SRI 127 96,379

FOLLOWING NEGOTIATING SESSIONS, THE THREE OFFERORS SUBMITTED REVISED PROPOSALS WITH THE FOLLOWING PROPOSED COSTS PLUS FEE:

ADL $133,000 (APPROXIMATELY)

DSC 74,488

SRI 93,870

DSC ALSO PROPOSED TO PERFORM THE WORK FOR A FIXED PRICE OF $74,000. THESE REVISED PROPOSALS WERE REVIEWED BY THREE MEMBERS OF THE NSF NEGOTIATING TEAM (TWO OF WHOM HAD SERVED ON THE ORIGINAL SELECTION PANEL) AND ANOTHER TECHNICAL EXPERT FROM NSF'S OFFICE OF ENERGY RESEARCH AND DEVELOPMENT POLICY (OEP). THIS GROUP DID NOT NUMERICALLY EVALUATE THE REVISED OFFERS, BUT FELT THAT THE SRI PROPOSAL WAS "SIGNIFICANTLY SUPERIOR TO THE OTHERS." IT REGARDED THE ADL PROPOSAL AS TOO COSTLY AND THE DSC PROPOSAL AS ONE WHICH PROMISED "LITTLE SOPHISTICATED QUALITY ASSESSMENT," WHILE THE SRI PROPOSAL WAS SEEN AS "PARTICULARLY STRONG" IN RECOGNIZING THAT "THE KEY OBJECTIVE IS AN UNDERSTANDING OF THE STATE OF THE ART, ITS LEVEL OF CAPABILITY, THE LEADING GROUPS, AND STRENGTHS AND WEAKNESSES OF THE ALTERNATIVE APPROACHES."

ACCORDING TO NSF, DSC'S INITIAL PROPOSAL "WAS DEEMED TO BE AN ATTEMPT TO CATALOG ALL MODELLING AND PROJECTION EFFORTS EXTANT," AND INDICATED THAT DSC "WAS GOING TO EXPEND MUCH OF ITS EFFORT ON RELATIVELY TRIVIAL AND SINGLE ISSUE SPECIFIC MODELS OR PROJECTIONS THAT WERE CLEARLY NOT MAJOR ***." THE NSF REVIEWERS ALSO BELIEVED THAT DSC MIGHT HAVE BEEN OFFERING LITTLE MORE THAN AN UPDATED VERSION OF A SURVEY DSC HAD PREVIOUSLY DONE FOR THE COUNCIL ON ENVIRONMENTAL QUALITY, A SURVEY WHICH WOULD NOT SATISFY NSF'S REQUIREMENTS. THEREFORE, NSF REPORTS, NEGOTIATIONS WITH DSC WERE UTILIZED TO EXPRESS CONCERN "ABOUT THE NUMBER OF 'MAJOR' MODELS AND THE NATURE AND QUALITY OF THE IN-DEPTH ANALYSIS THAT WOULD GET DOWN TO THE HEART OF THE ISSUE OF THE STATE-OF THE-ART." ACCORDING TO NSF, DSC RESISTED THE IDEA OF FOCUSING ON SELECTED MAJOR MODELS AND INSTEAD INSISTED THAT IT WAS UNIQUELY QUALIFIED TO ANALYZE THE ENTIRE MODELLING UNIVERSE. ALSO, ACCORDING TO NSF, DSC'S REVISED PROPOSAL REMAINED RELATIVELY UNCHANGED IN THE AREAS OF EXPLANATION OF STUDY REQUIREMENTS, SCHEDULE, PROJECT ORGANIZATION, AND FINAL REPORT OUTLINE. FURTHERMORE, ALTHOUGH DSC DID MODIFY ITS PROPOSAL BY INDICATING THE PLANNED USE OF AN ADVISORY COUNCIL TO ANALYZE MODELS, NSF REGARDED THIS AS AN INADEQUATE FRAMEWORK FOR CRITICAL IN-DEPTH ASSESSMENT.

ON THE OTHER HAND, DSC DISAGREES WITH THE EVALUATION OF BOTH ITS INITIAL AND REVISED PROPOSALS. IT CLAIMS THAT IT OFFERED, AT A MORE ADVANTAGEOUS PRICE, EVERYTHING THAT SRI DID, AND IT HAS SUBMITTED DETAILED, LENGTHY ARGUMENTS PURPORTING TO REBUT NSF'S EVALUATION. DSC FURTHER CHALLENGES THE EVALUATION OF BEST AND FINAL OFFERS IN THAT, UNLIKE THE EVALUATION OF INITIAL PROPOSALS, THE EVALUATION OF REVISED OFFERS WAS TOTALLY SUBJECTIVE AND UNSUPPORTED BY ANY OBJECTIVE OR QUANTITATIVE DATA. DSC ALSO CLAIMS THAT ITS BEST AND FINAL OFFER WOULD HAVE BEEN MORE TO NSF'S LIKING HAD NEGOTIATIONS BEEN ADEQUATE, AND THAT NSF'S PREFERENCE FOR FOCUS ON A LIMITED NUMBER OF MODELS WAS IMPROPER BECAUSE IT WAS CONTRARY TO THE STATEMENT OF WORK SET FORTH IN THE RFP.

WE DO NOT AGREE WITH DSC'S ASSERTIONS REGARDING THE CONSIDERATION THAT SHOULD HAVE BEEN GIVEN TO PROPOSED COST. WE NOTE THAT THE RFP DID NOT SET FORTH COST AS AN EVALUATION FACTOR, ALTHOUGH IT WARNED OFFERORS TO SUBMIT INITIAL OFFERS "ON THE MOST FAVORABLE TERMS FROM A PRICE AND TECHNICAL STANDPOINT" SINCE THE GOVERNMENT RESERVED THE RIGHT TO MAKE AWARD ON THE BASIS OF INITIAL OFFERS WITHOUT ANY DISCUSSION. HOWEVER, THE RFP CLEARLY CONTEMPLATED A COST-TYPE CONTRACT, WITH RESPECT TO WHICH FPR 1-3.805-2 (1964 ED.) STATES THAT "ESTIMATED COSTS OF CONTRACT PERFORMANCE AND PROPOSED FEES SHOULD NOT BE CONTROLLING" IN CONTRACTOR SELECTION. SEE 50 COMP. GEN. 739, 744 (1971). FURTHERMORE, WHILE THE APPLICABLE REGULATIONS DO NOT PRECLUDE CONSIDERATION OF COST IN EVALUATING PROPOSALS, 52 COMP. GEN. 686 (1973); 50 ID. 390, 407 (1970), A TECHNICALLY UNACCEPTABLE OR INFERIOR OFFER MAY BE REJECTED NOTWITHSTANDING THE REJECTED OFFEROR'S PROPOSED LOWER COSTS. MATTER OF AUSTIN ELECTRONICS, 54 COMP. GEN. 60 (1974). THIS IS TRUE WHETHER THE CONTRACT IS TO BE AWARDED ON A COST- REIMBURSEMENT BASIS, MATTER OF AUSTIN ELECTRONICS, SUPRA, OR ON A FIXED PRICE BASIS AS ALSO PROPOSED BY DSC. SEE, E.G., 50 COMP. GEN. 110, 113 (1970); 52 ID. 382 (1972). HERE IT APPEARS THAT NSF, CONSISTENT WITH THESE PRINCIPLES, REGARDED ADL'S SIGNIFICANTLY HIGHER PROPOSED COSTS AS NOT ADVANTAGEOUS TO THE GOVERNMENT WITH RESPECT TO THE WORK REQUIRED BY THE RFP, WHILE IT REGARDED SRI'S PROPOSAL AS MORE ADVANTAGEOUS THAN DSC'S BECAUSE OF ITS TECHNICAL SUPERIORITY, NOTWITHSTANDING DSC'S LOWER PROPOSED COSTS.

WE ALSO DO NOT AGREE THAT THE EVALUATION OF REVISED PROPOSALS IS SUBJECT TO CHALLENGE BECAUSE A NUMERICAL OR OTHER OBJECTIVE SCORING SCHEME WAS NOT UTILIZED. WE HAVE HELD THAT ASSIGNMENT OF NUMERICAL RATINGS TO PROPOSALS IS NEITHER A REQUIREMENT OF STATUTE NOR OF SOUND PROCUREMENT PRACTICE, THAT SUCH RATINGS ARE SIMPLY AN ATTEMPT TO QUANTIFY WHAT IS ESSENTIALLY A SUBJECTIVE JUDGMENT FOR THE PURPOSES OF REALISTIC AND FAIR PROPOSAL EVALUATION, AND THAT AN AGENCY'S FAILURE TO NUMERICALLY SCORE REVISED PROPOSALS IS THEREFORE NOT SUBJECT TO OBJECTION. 52 COMP. GEN. 198, 209 (1972); SEE ALSO B-174799, JUNE 10, 1972.

WITH RESPECT TO THE ADEQUACY OF NEGOTIATIONS, DSC CLAIMS THAT THE NEGOTIATING SESSION CONDUCTED WITH IT WAS TOO SHORT AND RESEMBLED AN "INQUISITION PROCESS IN WHICH (AN NSF OFFICIAL) CONTINUED TO MAKE A SERIES OF NEGATIVE STATEMENTS" ABOUT DSC. DSC FURTHER CLAIMS THAT NSF NEVER MADE CLEAR TO IT DURING THIS SESSION, AS IT APPARENTLY DID TO SRI, THAT FOCUSING ON SELECTED MAJOR MODELS WAS DESIRED. ON THE OTHER HAND, AS NOTED ABOVE, NSF STATES THAT IT ATTEMPTED TO MAKE ITS DESIRES CLEAR TO DSC DURING THAT SESSION BUT THAT DSC "RESISTED THE IDEA OF UNDERTAKING A CRITICAL EVALUATION OF THE MODELS." WE, OF COURSE, CANNOT DETERMINE PRECISELY WHAT TRANSPIRED DURING THE NEGOTIATIONS. WE HAVE OFTEN STATED THAT NEGOTIATIONS, WHEN THEY ARE CONDUCTED, MUST BE MEANINGFUL, 47 COMP. GEN. 19 (1967); 51 ID. 431 (1972), BUT THAT THE EXTENT AND CONTENT OF MEANINGFUL DISCUSSIONS MUST DEPEND ON THE CIRCUMSTANCES OF EACH CASE AND ARE NOT SUBJECT TO ANY FIXED, INFLEXIBLE RULE. 51 COMP. GEN. 621 (1972). "THE DURATION PER SE OF A NEGOTIATION SESSION IS BY NO MEANS DETERMINATIVE OF WHETHER MEANINGFUL DISCUSSIONS HAVE BEEN HELD." 52 COMP. GEN. 161, 164 (1972). FURTHERMORE, THE EXTENT AND CONTENT OF THOSE DISCUSSIONS "IS A MATTER OF JUDGMENT PRIMARILY FOR DETERMINATION BY THE PROCURING AGENCY *** AND *** SUCH DETERMINATION IS NOT SUBJECT TO QUESTION BY OUR OFFICE UNLESS CLEARLY ARBITRARY OR WITHOUT A REASONABLE BASIS." B-173677, MARCH 31, 1972 (SUMMARIZED AT 51 COMP. GEN. 621, SUPRA). ALTHOUGH NSF AND DSC DISAGREE ON PRECISELY WHAT OCCURRED DURING THE NEGOTIATING SESSION, AND WHILE IT MAY BE THAT DSC WOULD HAVE BENEFITED FROM MORE EXTENSIVE DISCUSSIONS, THE RECORD DOES NOT PERMIT US TO CONCLUDE THAT THE DISCUSSIONS THAT WERE CONDUCTED WERE AN ABUSE OF DISCRETION ON THE PART OF NSF OFFICIALS.

WE ALSO DO NOT AGREE WITH DSC THAT NSF'S EMPHASIS ON MAJOR MODEL FOCUSING FOR IN-DEPTH ANALYSIS WAS CONTRARY TO THE RFP. THE SCOPE OF WORK SECTION OF THE RFP STATED THAT THE PROJECT "WILL INVOLVE IDENTIFYING AND CONTACTING ALL MAJOR GROUPS ***. SUMMARIES OF ACTIVITIES, METHODOLOGIES, CONCLUSIONS, AND PROPOSED WORK ARE TO BE OBTAINED ***. THE REPORTS WILL INCLUDE: A) DESCRIPTIONS OF MODEL AND TECHNIQUES ***. NSF IS PARTICULARLY CONCERNED WITH HAVING AVAILABLE *** COMPLETE INFORMATION ON *** ENERGY DEMAND AND SUPPLY PROJECTIONS BY FUEL TYPES AS DEVELOPED BY ALL MAJOR GROUPS ***." WE AGREE THAT THE THRUST OF THESE STATEMENTS IS BROAD AND ALL-INCLUSIVE. HOWEVER, WE SEE NOTHING INCONSISTENT BETWEEN THESE STATEMENTS AND THE EVALUATION CRITERIA ON THE ONE HAND AND THE DESIRE FOR FOCUSING ON MAJOR MODELS ON THE OTHER, SINCE SUCH FOCUSING WOULD NOT PRECLUDE APPROPRIATE ANALYSIS OF OTHER MAJOR MODELS, WHICH IS PRECISELY WHAT SRI PROPOSED TO DO. MOREOVER, IT REASONABLY APPEARS THAT NSF MADE ITS PREFERRED APPROACH KNOWN DURING NEGOTIATIONS AND PRIOR TO SUBMISSION OF BEST AND FINAL OFFERS.

THE MAIN THRUST OF DSC'S PROTEST CONCERNS THE OVERALL TECHNICAL EVALUATION THAT RESULTED IN SELECTION OF SRI. IT IS CLEAR THAT STRONG DISAGREEMENT EXISTS BETWEEN NSF AND DSC AS TO THE VALIDITY OF THE EVALUATION. HOWEVER, IT IS NOT THE FUNCTION OF THIS OFFICE TO EVALUATE PROPOSALS OR TO RESOLVE DISPUTES OF THIS NATURE. B-180795, SEPTEMBER 16, 1974; 52 COMP. GEN. 382, SUPRA. THE DETERMINATION OF THE NEEDS OF THE GOVERNMENT AND THE METHOD OF ACCOMMODATING SUCH NEEDS IS PRIMARILY THE RESPONSIBILITY OF THE PROCURING AGENCY, 46 COMP. GEN. 606 (1967), WHICH THEREFORE IS RESPONSIBLE FOR THE OVERALL DETERMINATION OF THE RELATIVE DESIRABILITY OF PROPOSALS. IN MAKING SUCH DETERMINATIONS, CONTRACTING OFFICERS ENJOY "A REASONABLE RANGE OF DISCRETION" IN DETERMINING WHICH OFFER SHOULD BE ACCEPTED FOR AWARD, AND THEIR DETERMINATIONS WILL NOT BE QUESTIONED BY OUR OFFICE UNLESS THERE IS "A CLEAR SHOWING OF UNREASONABLENESS, AN ARBITRARY ABUSE OF DISCRETION, OR A VIOLATION OF THE PROCUREMENT STATUTES AND REGULATIONS." MATTER OF METIS CORP., B-181387, JANUARY 24, 1975, 54 COMP. GEN. . DSC ARGUES THAT NSF WAS ARBITRARY IN THAT ITS PROPOSAL COULD NOT PROPERLY BE REGARDED AS INFERIOR TO SRI'S OR AS NOT MEETING THE NEEDS OF NSF BECAUSE IT WAS INCLUDED IN THE "ZONE OF TECHNICAL COMPETENCE" AND BECAUSE DSC WAS PRE-QUALIFIED AS A BASIC ORDERING AGREEMENT (BOA) FIRM. HOWEVER, THE ISSUANCE OF A BOA TO A FIRM PROVIDES NO GUARANTEE THAT THE FIRM WILL OFFER SUPPLIES OR SERVICES WHICH ARE MOST ADVANTAGEOUS TO THE GOVERNMENT. FURTHERMORE, IT IS CLEAR HERE THAT DSC'S INITIAL PROPOSAL WAS NEVER REGARDED AS COMPLETELY ACCEPTABLE, BUT ONLY AS HAVING A REASONABLE CHANCE OF BEING MADE ACCEPTABLE FOR AWARD. IN THE JUDGMENT OF NSF, DSC'S REVISED PROPOSAL DID NOT INVOLVE ANY SIGNIFICANT IMPROVEMENT OVER ITS INITIAL PROPOSAL. WE ARE UNABLE TO DISAGREE WITH THAT JUDGMENT. FINALLY, DSC ARGUES THAT NSF IS BIASED AGAINST INDUSTRIAL PROFIT-MAKING ORGANIZATIONS, PARTICULARLY SMALL BUSINESS FIRMS, AND AGAINST DSC. VARIOUS STATISTICS ARE CITED BY DSC TO DEMONSTRATE THAT NSF AWARDS MOST OF ITS CONTRACTS TO UNIVERSITY AND EDUCATIONAL AGENCIES OR LARGE INDUSTRIAL ORGANIZATIONS AND WHICH PURPORTEDLY SHOW THAT NSF HAS A BUILT-IN BIAS AGAINST DSC AND SIMILAR FIRMS. NSF DENIES THE EXISTENCE OF ANY SUCH BIAS, BUT POINTS OUT THAT "THE OVERWHELMING MAJORITY OF RESEARCH PROJECTS AND STUDIES THAT ARE FUNDED BY NSF ARE PERFORMED BY ACADEMIC INSTITUTIONS AND NON-PROFIT ORGANIZATIONS, MOSTLY THROUGH THE GRANTING MECHANISM." ACCORDING TO NSF, IT HAS BEEN SOLICITING PRIVATE INDUSTRY, PURSUANT TO PRESIDENTIAL DIRECTIVE, SINCE 1972, WITH COMMERCIAL FIRMS FARING "EXTREMELY WELL IN THOSE INSTANCES WHERE THEY WERE INVITED TO COMPETE FOR AWARDS." IN ANY EVENT, THE RECORD DOES NOT SUPPORT THE ALLEGATION THAT THE SELECTION OF SRI WAS THE RESULT OF ANY BIAS IN FAVOR OF NONPROFIT ORGANIZATIONS OR AGAINST DSC IN PARTICULAR.

FOR THE FOREGOING REASONS, THE PROTEST IS DENIED.