B-184654, FEB 18, 1976

B-184654, FEB 18, 1976

1. ALLEGATION BY PROTESTER THAT TECHNICAL QUESTIONS, SENT TO FOUR PROPOSERS DETERMINED TO BE WITHIN COMPETITIVE RANGE, TRANSFUSED TO OTHER PROPOSERS UNIQUE AND INNOVATIVE CONCEPTS FROM PROTESTER'S INITIAL PROPOSAL SUBMITTED IN RESPONSE TO RFP IS WITHOUT MERIT SINCE EVALUATION PANEL WHO DRAFTED QUESTIONS DID NOT RELY ON ANY PARTICULAR PROPOSAL. MOREOVER, EVIDENCE OF RECORD DID NOT INDICATE THAT THERE WAS, IN FACT, ANY SUCH TRANSFUSION. 2. FACT THAT MEMBERS OF EVALUATION PANEL WHO WERE EMPLOYED BY USING ACTIVITY GAVE PROTESTER LOWER TECHNICAL SCORE THAN DID THOSE MEMBERS WHO WERE NOT EMPLOYED BY USING ACTIVITY DID NOT INDICATE BIAS AGAINST PROTESTER. RECORD INDICATES THAT MEMBERS EMPLOYED BY USING ACTIVITY HAD MORE EXPERIENCE IN OPERATIONS AND SKILLS CALLED FOR UNDER RFP AND, THEREFORE, WERE MORE CRITICAL IN THEIR EVALUATION OF PROTESTER'S PROPOSAL, AS WELL AS OTHER PROPOSERS. ADDITIONALLY, RECORD INDICATES THAT PROPOSAL SUBMITTED BY SUCCESSFUL PROPOSER WAS TECHNICALLY SUPERIOR TO PROTESTER'S PROPOSAL WHICH WOULD ACCOUNT FOR PROTESTER'S LOWER SCORE. 3. AWARD OF NEGOTIATED COST-TYPE CONTRACT TO PROPOSER RECEIVING HIGHEST TECHNICAL RATING EVEN THOUGH IT WAS NOT LOW PROPOSER WAS PROPER EXERCISE OF ADMINISTRATIVE DISCRETION AS POINT SPREAD INDICATED THAT PROPOSAL WAS SUPERIOR TO PROPOSAL SUBMITTED BY PROTESTER WHO RECEIVED SECOND HIGHEST TECHNICAL RATING. THEREFORE, COST WAS NOT DETERMINATIVE FACTOR SINCE PROPOSALS WERE NOT SUBSTANTIALLY EQUAL TECHNICALLY AND FPR SEC. 1-3.805-2 DOES NOT REQUIRE AWARD TO OFFEROR WITH LOWEST COST ESTIMATE. 4. FAILURE TO INDICATE RELATIVE IMPORTANCE OF PRICE VIS-A-VIS OTHER FACTORS WHICH WERE LISTED IS CONTRARY TO GAO VIEW THAT INTELLIGENT COMPETITION REQUIRES THAT OFFERORS BE ADVISED OF EVALUATION FACTORS TO BE USED AND IMPORTANCE OF THESE FACTORS. EACH OFFEROR IS ENTITLED TO KNOW WHETHER PROCUREMENT IS INTENDED TO ACHIEVE MINIMUM STANDARD AT LOWEST COST OR WHETHER COST IS SECONDARY TO QUALITY. HOWEVER, AWARD NOT FOUND TO BE ILLEGAL SINCE AWARD WAS BASED ON TECHNICAL SUPERIORITY OF SUCCESSFUL OFFEROR'S PROPOSAL RATHER THAN FROM NEGATIVE FINDINGS WITH RESPECT TO PROTESTER'S COST ESTIMATES.

APPLIED MANAGEMENT SCIENCES, INC.:

BY TELEX MESSAGE OF AUGUST 4, 1975, AS SUPPLEMENTED BY LETTER DATED AUGUST 11, 1975, APPLIED MANAGEMENT SCIENCES, INC. (AMS), PROTESTED AGAINST THE AWARD OF A CONTRACT UNDER REQUEST FOR PROPOSALS (RFP) 0E-75 58 ISSUED BY THE OFFICE OF EDUCATION, DEPARTMENT OF HEALTH, EDUCATION, AND WELFARE (HEW).

RFP 0E-75-58, ISSUED ON MARCH 26, 1975, SOLICITED PROPOSALS FOR FURNISHING ALL PERSONNEL, SERVICES, AND FACILITIES NECESSARY FOR THE DESIGN AND OPERATION OF A FAST RESPONSE SURVEY SYSTEM (FRSS) IN ACCORDANCE WITH THE INCORPORATED SCOPE OF WORK. THE FRSS IS TO BE A DATA COLLECTION AND DISSEMINATION SYSTEM WHICH WILL ENABLE THE NATIONAL CENTER FOR EDUCATION STATISTICS (NCES) TO ACCUMULATE AND TABULATE EDUCATIONAL STATISTICS WITHIN A SHORT TIME FRAME IN RESPONSE TO REQUESTS FOR STATISTICAL INFORMATION UNAVAILABLE FROM ROUTINE SURVEYS CONDUCTED BY NCES. THE RFP CALLED FOR A NEGOTIATED, COST-TYPE CONTRACT AND REQUESTED THAT THE PROPOSERS SUBMIT SEPARATE TECHNICAL AND COST PROPOSALS. TEN FIRMS RESPONDED TO THE RFP PRIOR TO THE DUE DATE OF MAY 7, 1975. THE PROPOSALS WERE EVALUATED BY A NINE-MEMBER EVALUATION PANEL AND EACH FIRM WAS GIVEN A NUMERICAL RATING. THE TOP FOUR FIRMS WERE CONSIDERED TO BE WITHIN THE COMPETITIVE RANGE. THE SCORES OF THESE FOUR FIRMS WERE AS FOLLOWS:

WESTAT 79.1 AMS 72.8 PACIFIC-SIERRA 70.7 NATIONAL ANALYSTS 68.6

SUBSEQUENTLY, THE EVALUATION PANEL FORMULATED SETS OF WRITTEN TECHNICAL QUESTIONS TO BE SUBMITTED TO EACH OF THE FIRMS WITHIN THE COMPETITIVE RANGE. ACCORDING TO AMS, THE EVALUATION PANEL FORWARDED A SET OF 25 QUESTIONS TO AMS AND THE FIRST SEVEN QUESTIONS WERE ALSO DIRECTED TO THE OTHER THREE FIRMS.

THE CONTRACTING OFFICER REQUESTED THAT BEST AND FINAL OFFERS BE SUBMITTED BY JUNE 25, 1975.

ON JUNE 27, THE EVALUATION PANEL COMPLETED ITS FINAL REVIEW AND NUMERICAL SCORING OF THE PROPOSALS. THE PANEL RECOMMENDED THAT AWARD BE MADE TO WESTAT. THE FINAL AVERAGE SCORES AND NUMERICAL RANKING OF THE FOUR OFFERORS WERE AS FOLLOWS:

WESTAT 83.2 AMS 72.3 PACIFIC-SIERRA 70.0 NATIONAL ANALYSTS 68.1

ON JULY 25, 1975, THE CONTRACT WAS AWARDED TO WESTAT.

IT IS CONTENDED BY AMS THAT NCES ABUSED ITS DISCRETION AND UTILIZED IMPROPER PROCUREMENT PROCEDURES BY AWARDING THE CONTRACT TO WESTAT. SPECIFICALLY, IT IS ALLEGED BY AMS THAT: (1) AT LEAST FOUR OF THE QUESTIONS SUBMITTED TO THE FIRMS IN THE COMPETITIVE RANGE REFLECTED UNIQUE AND INNOVATIVE CONCEPTS EXCLUSIVELY DEVELOPED IN THE AMS TECHNICAL PROPOSAL SUBMITTED IN RESPONSE TO THE RFP'S EXTREMELY GENERAL DESCRIPTION OF THE FRSS, AND THUS, REPRESENTED AN UNFAIR TRANSFUSION OF AMS' IDEAS TO THE OTHER OFFERORS; (2) THE NUMERICAL SCORES OF THE FIVE NCES MEMBERS OF THE EVALUATION PANEL REFLECT A PRECONCEIVED BIAS AGAINST AMS; AND (3) NCES DID NOT CONSIDER WESTAT'S TECHNICAL PROPOSAL TO BE SIGNIFICANTLY SUPERIOR TO THE AMS PROPOSAL AND, THEREFORE, AWARD SHOULD HAVE BEEN MADE TO AMS SINCE IT HAD A LOWER COST ESTIMATE.

IN REGARD TO THE CONTENTION BY AMS THAT THE EVALUATION PANEL THROUGH GENERAL TECHNICAL QUESTIONS TRANSFUSED UNIQUE AND INNOVATIVE CONCEPTS EXCLUSIVELY DEVELOPED IN ITS PROPOSAL, AMS CITES FOUR INSTANCES WHERE THIS TRANSFUSION OCCURRED. THE FIRST INSTANCE IS IN CONNECTION WITH QUESTION 1 WHICH ASKED, IN PART, WHETHER THE OFFERORS HAD CONSIDERED WITHIN THEIR PROPOSED SYSTEM DESIGN A MECHANISM FOR ALLOWING REPRESENTATIVES OF EDUCATIONAL INSTITUTIONS TO DETERMINE THE PROCEDURES FOR CONDUCTING FAST RESPONSE SURVEYS WITHIN THEIR STATE JURISDICTIONS. ACCORDING TO AMS, THIS REPRESENTS A SIGNIFICANT EXPANSION OVER THE RFP WHICH MERELY REQUIRED THE CONTRACTOR TO "MAINTAIN ITS (NCES') GOOD RELATIONSHIP WITH RESPONDENTS" AND REPRESENTS UNFAIR TRANSFUSION OF THE DETAILED AMS CONCEPT TO DESIGN A SYSTEM OF OBTAINING RESPONSES BASED ON THE RIGHT OF REPRESENTATIVE ORGANIZATIONS TO DETERMINE HOW INDIVIDUAL SURVEYS WOULD BE HANDLED WITHIN THEIR JURISDICTIONS.

QUESTION NO. 1 DEALS WITH THE NECESSITY OF COOPERATION AND COORDINATION OF DATA COLLECTION WITH THE COUNCIL OF CHIEF STATE SCHOOL OFFICERS (CCSSO). BOTH AMS (ON PAGE 2.42-2.44) AND WESTAT (IN SECTION 2.1.3) DEALT WITH THIS MATTER IN THEIR INITIAL PROPOSALS. WHILE WESTAT DID NOT SPECIFICALLY OFFER TO DESIGN A SYSTEM OF OBTAINING RESPONSES BASED ON THE RIGHT OF REPRESENTATIVE ORGANIZATIONS TO DETERMINE HOW SURVEYS WOULD BE HANDLED WITHIN THEIR JURISDICTIONS, IT DOES NOT APPEAR THAT AMS SPECIFICALLY OFFERED TO DESIGN SUCH A SYSTEM EITHER. ALTHOUGH IT APPEARS TO BE IMPLIED IN BOTH AMS' AND WESTAT'S INITIAL PROPOSALS THAT REPRESENTATIVES OF EDUCATIONAL SYSTEMS WOULD HAVE THE RIGHT TO DETERMINE HOW INDIVIDUAL SURVEYS WOULD BE HANDLED WITHIN THEIR JURISDICTIONS, THIS UNDOUBTEDLY ACCOUNTS, IN PART, FOR THE EMPHASIS BY BOTH AMS AND WESTAT ON COOPERATION WITH REPRESENTATIVES OF EDUCATIONAL SYSTEMS. IN THIS CONNECTION, AS PREVIOUSLY MENTIONED, THE RFP REQUIRED THAT THE CONTRACTOR MAINTAINS NCES' GOOD RELATIONSHIP WITH THE RESPONDENTS (BY RESPONDENT WE MEAN THOSE EDUCATIONAL INSTITUTIONS RESPONDING TO NCES' SURVEY QUESTIONS). QUESTION NO. 1 ALSO DEALT WITH THE VARYING ROLES IN THE DATA COLLECTION THAT MAY BE DESIRED TO BE ASSUMED BY EACH STATE, EVEN TO THE POINT OF PERHAPS COLLECTING THE DATA FOR NCES FROM THE LOCAL ORGANIZATION AND SCHOOLS. IN REGARD TO THIS MATTER WHILE WESTAT STATED THAT IT WOULD "DEVISE APPROPRIATE PROCEDURAL MODIFICATIONS," IT DOES NOT APPEAR THAT AMS ADDRESSED THIS MATTER EITHER IN ITS INITIAL PROPOSAL OR IN ITS REPLY TO THE TECHNICAL QUESTIONS. THUS, THERE DOES NOT APPEAR TO HAVE BEEN ANY TRANSFUSION OF UNIQUE OR INNOVATIVE IDEAS IN THIS INSTANCE.

THE SECOND INSTANCE OF TECHNICAL TRANSFUSION CITED BY AMS INVOLVES QUESTION NO. 2 OF THE TECHNICAL QUESTIONS. THIS QUESTION ASKED WHAT STRATEGIES WOULD BE USED WHEN THE SAMPLING FRAME IS NOT CURRENT AND REQUESTED THE PROPOSER TO PROVIDE FULL COST ESTIMATES AS WELL AS TO INDICATE CHANGES IN ITS TIME SCHEDULE. AMS CONTENDS THAT THIS QUESTION ALERTED OTHER OFFERORS TO THE FACT THAT THESE LISTS, I.E., INFORMATION CONTAINED ON DATA TAPES FROM WHICH THE SAMPLES WOULD BE TAKEN, ARE NOT CURRENT AND THUS, TRANSFUSED INFORMATION FROM AMS' PROPOSAL WHICH NOTED THE AGE OF THE AVAILABLE TAPES AND PROPOSED TECHNIQUES FOR RESOLVING THE PROBLEM BY UPDATING THEM. AMS STATES THAT WESTAT SHOWED NO RECOGNITION, IN ITS INITIAL PROPOSAL, OF THE ISSUE OF OUTDATED LISTS AND THAT WESTAT, IN FACT, STATED THAT "THE NCES FILES TO BE THE LATEST AND MOST COMPLETE SOURCES FOR SAMPLES (WITH NO APPARENT NEED TO EXPLORE OTHER SOURCES)." HOWEVER, WE NOTE THAT AMS STATED IN ITS INITIAL PROPOSAL (ON PAGE 2.4), IN REFERENCE TO NCES' DATA TAPES FOR PUBLIC ELEMENTARY AND SECONDARY SCHOOLS, THAT "THE MOST CURRENT DATA TAPE AVAILABLE FOR ACCESS IS REPRESENTATIVE OF THE 1973-74 SCHOOL YEAR AND IS RELATIVELY COMPLETE AND ACCURATE." ALSO, IN CONNECTION WITH THE NCES' TAPES FOR LOCAL EDUCATION AGENCIES, AMS INDICATED THAT (ON PAGE 2.7) THEY WERE AVAILABLE FOR 1974-75 AND THAT DATA TAPES FOR POST SECONDARY SCHOOLS WERE AVAILABLE FOR THE 1972-73 ACADEMIC YEAR. WE ARE ADVISED THAT THE MOST CURRENT NCES DATA TAPE FOR PUBLIC ELEMENTARY AND SECONDARY SCHOOLS IS FALL 1972, FOR LOCAL EDUCATION AGENCIES IS FALL 1973, AND FALL 1973 IS THE MOST CURRENT FOR THE POST SECONDARY SCHOOLS. THUS, IT APPEARS THAT QUESTION 2 ALERTED AMS TO THE FACT THAT THE LISTS WERE MORE OUTDATED THAN IT (AMS) HAD ORIGINALLY THOUGHT. HOWEVER, WE NOTE THAT WHILE IN ITS INITIAL PROPOSAL AMS DID DISCUSS THE UPDATING OF THESE TAPES, WESTAT ALSO DISCUSSED UPDATING OF THE SAMPLES. THIS INDICATES THAT BOTH AMS AND WESTAT RECOGNIZED THE PROBLEM OF USING SAMPLE FRAMES FROM OUTDATED DATA FILES. THEREFORE, WE ARE NOT CONVINCED THAT TRANSFUSION OF UNIQUE AND INNOVATIVE IDEAS OCCURRED IN THIS INSTANCE.

THE THIRD INSTANCE OF TECHNICAL TRANSFUSION CITED BY AMS IS IN CONNECTION WITH QUESTION 4, WHICH STATES THE FOLLOWING:

"WE WANT THE SYSTEM DOCUMENTATION MANUALS AND TRAINING MANUALS TO BE SUFFICIENTLY COMPREHENSIVE TO ENABLE THE SYSTEM TO BE USEFUL AS AN ON GOING, STAND-ALONE BASIS SO THAT IT CAN BE OPERATED BY SOMEONE OTHER THAN THE CONTRACTOR. ELABORATE ON HOW THIS LEVEL OF DOCUMENTATION WILL BE ACCOMPLISHED. PROVIDE A FULL COST ESTIMATE IF NOT ALREADY INCLUDED IN THE COST PROPOSAL."

AMS CONTENDS THAT THERE WAS A TRANSFUSION OF ITS IDEA FOR THE DESIGN OF COMPREHENSIVE MANUALS FOR INSTRUCTING FUTURE MANAGERS IN THE DETAILS OF THE SYSTEM'S OPERATION. IT APPEARS THAT WESTAT ADDRESSED THIS PROBLEM (IN SECTIONS 2.2.76, 2.2.8 AND 2.2.9) IN ITS INITIAL PROPOSAL AND CLARIFIED ITS PROPOSED DOCUMENTATION IN ITS REPLY TO QUESTION 4. THUS, IT DOES NOT APPEAR THAT THE IDEA OF SUCH DOCUMENTATION WAS UNIQUE TO AMS.

THE FINAL INSTANCE OF TECHNICAL TRANSFUSION CITED BY AMS IS IN CONNECTION WITH QUESTION 5, WHICH ASKS:

"WOULD THE SYSTEM BE FLEXIBLE ENOUGH TO BE USED FOR SURVEYS THAT WOULD REQUIRE COLLECTING DATA NOT ALREADY EXISTING AT THE STATE OR INSTITUTION LEVEL?

"DISCUSSION SHOULD BE PROVIDED ON:

- WHAT THE LIMITS OF THE SYSTEM ARE IF THE DATA DOES EXIST IN READILY ACCESSIBLE FORM;

- WHAT PROVISIONS WOULD BE TAKEN FOR GETTING RECORDS THAT EXIST IN NON- HOMOGENEOUS FORM - REQUIRING UNIT RECORD SEARCH;

- HOW THE FOUR SURVEYS WOULD BE CONDUCTED TO TEST THE LIMITS OF THE FAST RESPONSE SURVEY SYSTEM;

- HOW CONSULTANTS SHOULD INTERACT AT START OF SURVEY TO DECIDE ON FEASIBILITY OF SURVEY, ADDITIONAL COSTS THAT MIGHT BE INCURRED, RECOMMENDATIONS."

ACCORDING TO AMS, THE ABOVE QUESTION AND OUTLINE OF DISCUSSION TRANSFUSED ITS CONCEPT OF DEVELOPING A SYSTEM FLEXIBLE ENOUGH TO MEET THE MYRIAD OF DATA REQUESTS THAT COULD BE EXPECTED TO BE RAISED BY INDIVIDUALS NOT FAMILIAR WITH EDUCATION INFORMATION SYSTEMS AND THEIR LIMITS. AMS ALSO STATES THAT THE QUESTION PICKED UP ON RELATED ISSUES DISCUSSED IN AMS' INITIAL PROPOSAL AND, THUS, REQUESTED OFFERORS TO EXPLAIN HOW THEY WOULD TEST THE LIMITS OF THE SYSTEM AND EVALUATE INDIVIDUAL DATA REQUESTS. AMS APPEARS TO BE REFERRING TO A SYSTEM FLEXIBLE ENOUGH TO RESPOND TO DATA REQUESTS FROM INDIVIDUALS WHO ARE NOT FAMILIAR WITH EDUCATION INFORMATION SYSTEMS. HOWEVER, THE QUESTION APPEARS TO BE ADDRESSED TO THE ISSUE OF WHETHER OR NOT THE SYSTEM IS FLEXIBLE ENOUGH TO BE USED FOR SURVEYS REQUIRING COLLECTION OF DATA NOT ALREADY EXISTING AT THE STATE OR INSTITUTION LEVEL. THUS, THE SECTIONS OF ITS PROPOSAL REFERRED TO BY AMS AS HAVING BEEN TRANSFUSED BY THE QUESTION DO NOT APPEAR TO REFER TO THE SAME MATTER AS COVERED BY THE QUESTION. IT IS UNCLEAR AS TO WHAT THE RELATED ISSUES ARE THAT THE QUESTION PICKS UP ON. THIS BEING THE CASE, THERE COULD BE NO TRANSFUSION OF UNIQUE OR INNOVATIVE IDEAS.

WE AGREE THAT DISCLOSURE TO OTHER PROPOSERS OF ONE PROPOSER'S INNOVATIVE OR UNIQUE SOLUTION TO A PROBLEM IS UNFAIR AND THAT SUCH TRANSFUSION SHOULD BE AVOIDED. SEE 51 COMP.GEN. 621 (1972). HOWEVER, THERE IS NO EVIDENCE OF RECORD TO INDICATE THAT THERE WAS ANY SUCH TRANSFUSION IN THE PRESENT CASE. ACCORDING TO THE NCES PROJECT OFFICER, WHO WAS ALSO A MEMBER OF THE EVALUATION PANEL, AMS' INITIAL PROPOSAL CONTAINED NO INNOVATIVE OR UNIQUE IDEAS. ON THE CONTRARY, IT IS PROBABLY TRUE THAT THE AMS PROPOSAL WAS LARGELY BASED ON THE TRANSFUSION OF NCES' CONCEPTS TO AMS THROUGH ITS PREVIOUS CONTRACTS WITH NCES.

ACCORDING TO THE PROJECT OFFICER, IN DRAFTING THE TECHNICAL QUESTIONS, THE EVALUATION PANEL CAME TO REALIZE THAT CERTAIN AREAS WERE OF SUFFICIENT SIGNIFICANCE AND CONCERN AS TO BE INCLUDED IN THE QUESTIONS FOR ALL THE OFFERORS IN THE COMPETITIVE RANGE. THE PROJECT OFFICER STATED THAT THE PANEL MEMBERS REALIZED THAT THESE GENERAL AREAS HAD BEEN DISCUSSED BY AND LARGE IN THE INITIAL PROPOSALS IN VARIOUS WAYS; HOWEVER, THE QUESTIONS OFTEN PRESENTED A SPECIFIC SLANT OR EMPHASIS FOR WHICH THE PANEL WISHED TO OBTAIN THE PROPOSER'S CLARIFICATIONS. ACCORDING TO THE PROJECT OFFICER THE QUESTIONS WERE DEVELOPED IN THE FOLLOWING MANNER:

1. EACH PANEL MEMBER SUBMITTED DRAFT QUESTIONS;

2. AT AN EVALUATION PANEL MEETING, THESE DRAFT QUESTIONS WERE REVIEWED, REVISED, AND REDRAFTED TO GET THE EXACT MEANING DESIRED. ACCORDING TO THE PROJECT OFFICER, AT NO TIME DID THE PANEL MEMBERS USE ANY PROPOSAL AS A BASIS FOR THE GENERAL QUESTIONS, BUT RATHER EXPRESSED THEIR CONCERN THROUGH ALL THE TECHNICAL QUESTIONS.

ON THE BASIS OF THE RECORD, WE ARE UNABLE TO CONCLUDE THAT THE TECHNICAL QUESTIONS WERE DEVELOPED IN ANY MANNER OTHER THAN THAT DESCRIBED ABOVE.

THE NEXT ISSUE CONCERNS THE ALLEGATION BY AMS THAT THE NUMERICAL SCORES OF THE FIVE NCES MEMBERS OF THE EVALUATION PANEL REFLECT A PRECONCEIVED BIAS AGAINST AMS. WHILE ADMITTEDLY THE NCES MEMBERS DID SCORE AMS LOWER THAN DID THE NON-NCES MEMBERS, THERE IS NO EVIDENCE OF RECORD TO SUPPORT AMS' CONTENTION THAT THE NCES MEMBERS WERE BIASED AGAINST AMS. FOR THAT MATTER, IT IS JUST AS REASONABLE TO ASSUME THAT SINCE THE NCES MEMBERS HAVE EXPERIENCE IN ACTUAL SURVEY OPERATIONS AND KNOW THE TYPE OF WORK REQUIRED, THE TYPES OF SKILLS AND TIME REQUIRED TO PERFORM THESE OPERATIONS, THAT THEY WOULD BE MORE CRITICAL IN THEIR EVALUATION OF PROPOSERS THAN WOULD THE NON-NCES MEMBERS. THIS IS BORNE OUT BY THE FACT THAT THE NCES MEMBERS SCORED THREE OF THE FOUR PROPOSERS LOWER THAN DID THE NON-NCES MEMBERS. MOREOVER, THE RECORD INDICATES THAT THE LOWER SCORE GIVEN TO AMS BY NCES MEMBERS COULD BE BASED ON THE FACT THAT WESTAT'S PROPOSAL WAS TECHNICALLY SUPERIOR. FINALLY, FOR CONSIDERATION IS AMS' CONTENTION THAT SINCE NCES DID NOT CONSIDER WESTAT'S TECHNICAL PROPOSAL TO BE SIGNIFICANTLY SUPERIOR TO THE PROPOSAL SUBMITTED BY AMS, AWARD SHOULD HAVE BEEN MADE TO AMS SINCE IT HAD A LOWER COST ESTIMATE. IN SUPPORT OF THIS CONTENTION AMS CITES ANALYTIC SYSTEMS, INCORPORATED, B-179259, FEBRUARY 14, 1974, 74-1 CPD 71, WHEREIN WE HELD THAT AWARD OF A NEGOTIATED COST-TYPE CONTRACT TO THE LOW OFFEROR WHOSE TECHNICAL RATING WAS 3 POINTS LESS THAN THAT OF THE PROTESTER WAS A PROPER EXERCISE OF ADMINISTRATIVE DISCRETION. WE STATED IN THAT CASE THAT THE POINT SPREAD DID NOT AUTOMATICALLY ESTABLISH THAT THE HIGHER RATED PROPOSAL IS SIGNIFICANTLY SUPERIOR AND THAT WHERE AS IN THAT CASE, TWO OFFERORS ARE ESSENTIALLY EQUAL TECHNICALLY PRICE IS PROPERLY THE DETERMINATIVE ISSUE. HOWEVER, IN THE PRESENT CASE, ON THE TECHNICAL EVALUATION OF THE INITIAL PROPOSALS WESTAT'S TECHNICAL RATING WAS 6 POINTS HIGHER THAN AMS' TECHNICAL RATING AND ON THE FINAL TECHNICAL RATING THERE IS A DIFFERENCE OF ALMOST 11 POINTS. WHILE AMS ATTRIBUTES THIS DIFFERENCE TO THE BIAS AND TRANSFUSION OF ITS UNIQUE AND INNOVATIVE IDEAS, THE RECORD LENDS NO SUPPORT TO THIS CONTENTION. FOR THAT MATTER, THERE IS SOME EVIDENCE OF RECORD TO SUPPORT THE PROCURING ACTIVITY'S CONTENTION THAT WESTAT'S SCORE IMPROVED ON THE FINAL EVALUATION NOT BECAUSE OF THE TRANSFUSION OF AMS' CONCEPTS, BUT BECAUSE ITS RESPONSE TO THE TECHNICAL QUESTIONS ADDRESSED PROBLEMS RAISED BY THE QUESTIONS, WHEREAS AMS' REPLY WAS, ON THE WHOLE, NOT RESPONSIVE TO THE QUESTIONS. ALSO, IT SHOULD BE POINTED OUT THAT THERE IS NO EVIDENCE INDICATING THAT THE PROPOSALS OF AMS AND WESTAT WERE CONSIDERED TO BE EQUAL OR SUBSTANTIALLY EQUAL. ACCORDING TO THE NCES PROJECT OFFICER, WHILE IT WAS NOTED THAT ALL FOUR COMPANIES UNDERSTOOD THE PROBLEMS AND COULD DO THE WORK (WHICH WOULD INCLUDE THEM IN THE COMPETITIVE RANGE), IT SHOULD NOT BE INFERRED THAT ANY OF THE PROPOSALS WERE EQUAL. ON THE CONTRARY, WESTAT'S FINAL PROPOSAL, WHICH WAS EVALUATED ALMOST 11 POINTS HIGHER THAN AMS' PROPOSAL, WAS CONSIDERED TO BE SIGNIFICANTLY SUPERIOR ESPECIALLY IN ITS CAPACITY TO PERFORM. THE PROJECT OFFICER STATED THAT WESTAT'S PROPOSED KEY PROJECT PERSONNEL WERE SUPERIOR TO AMS' KEY PROJECT PERSONNEL. ALSO, ACCORDING TO THE PROJECT OFFICER, AMS' PROPOSAL MANLOADING AND MANHOURS PROPOSED BY TASKS HAD MARKED WEAKNESSES. OUR OFFICE HAS HELD THAT WHETHER A GIVEN POINT SPREAD BETWEEN TWO COMPETING PROPOSALS INDICATES A SIGNIFICANT SUPERIORITY OF ONE PROPOSAL OVER ANOTHER DEPENDS UPON THE FACTS AND CIRCUMSTANCES OF EACH PROCUREMENT AND IS PRIMARILY WITHIN THE DISCRETION OF THE PROCURING AGENCY. 52 COMP.GEN. 686, 690 (1973). IN THE PRESENT CASE, THE RECORD DOES NOT ESTABLISH THAT THE PROCURING ACTIVITY ABUSED ITS DISCRETION IN DETERMINING THAT THE POINT DIFFERENCE BETWEEN WESTAT AND AMS INDICATED THE SUPERIORITY OF WESTAT'S PROPOSAL AND THAT COST WOULD NOT BE THE DETERMINATIVE FACTOR FOR AWARD.

MOREOVER, WE NOTE THAT THE PRESENT CONTRACT IS A COST REIMBURSEMENT TYPE CONTRACT AND IN THIS CONNECTION SECTION 1-3.805-2 (1964 ED. CIRC. 1) OF THE FEDERAL PROCUREMENT REGULATIONS STATES:

"IN SELECTING THE CONTRACTOR FOR A COST REIMBURSEMENT TYPE CONTRACT, ESTIMATED COSTS OF CONTRACT PERFORMANCE AND PROPOSED FEES SHOULD NOT BE CONSIDERED AS CONTROLLING, SINCE IN THIS TYPE OF CONTRACT ADVANCE ESTIMATES OF COST MAY NOT PROVIDE VALID INDICATORS OF FINAL ACTUAL COSTS. THERE IS NO REQUIREMENT THAT COST REIMBURSEMENT TYPE CONTRACTS BE AWARDED ON THE BASIS OF EITHER (A) THE LOWEST PROPOSED COST, (B) THE LOWEST PROPOSED FEE, OR (C) THE LOWEST TOTAL ESTIMATED COST PLUS PROPOSED FEE. THE AWARD OF COST-REIMBURSEMENT TYPE CONTRACTS PRIMARILY ON THE BASIS OF ESTIMATED COSTS MAY ENCOURAGE THE SUBMISSION OF UNREALISTICALLY LOW ESTIMATES AND INCREASE THE LIKELIHOOD OF COST OVERRUNS. THE COST ESTIMATE IS IMPORTANT TO DETERMINE THE PROSPECTIVE CONTRACTOR'S UNDERSTANDING OF THE PROJECT AND ABILITY TO ORGANIZE AND PERFORM THE CONTRACT. THE AGREED FEE MUST BE WITHIN THE LIMITS PRESCRIBED BY LAW AND AGENCY PROCEDURES AND APPROPRIATE TO THE WORK TO BE PERFORMED (SEE SEC. 1-3.808). BEYOND THIS, HOWEVER, THE PRIMARY CONSIDERATION IN DETERMINING TO WHOM THE AWARD SHALL BE MADE IS WHICH CONTRACTOR CAN PERFORM THE CONTRACT IN A MANNER MOST ADVANTAGEOUS TO THE GOVERNMENT."

ACCORDINGLY, OUR OFFICE HAS INTERPOSED NO LEGAL OBJECTION TO THE AWARD OF SUCH CONTRACTS AT HIGHER ESTIMATED COSTS WHERE TECHNICAL SUPERIORITY JUSTIFIES THE COST PREMIUM. B-174756, JUNE 30, 1972; BELLMORE JOHNSON TOOL COMPANY, B-179030, JANUARY 24, 1974, 74-1 CPD 26.

HOWEVER, WE ARE CONCERNED THAT THE INSTANT RFP DID NOT ADVISE OFFERORS OF THE RELATIVE IMPORTANCE OF COST VIS-A-VIS THE EVALUATION FACTORS WHICH WERE LISTED. THIS FAILURE TO SHOW THE RELATIVE IMPORTANCE OF PRICE IS CONTRARY TO THE LONGSTANDING VIEW OF OUR OFFICE THAT INTELLIGENT COMPETITION REQUIRES, AS A MATTER OF SOUND PROCUREMENT POLICY, THAT OFFERORS BE ADVISED OF THE EVALUATION FACTORS TO BE USED AND THE RELATIVE IMPORTANCE OF THOSE FACTORS. 49 COMP.GEN. 229 (1969). THUS, WE STATED IN 52 COMP.GEN. 161, 164 (1972) THAT "EACH OFFEROR HAS A RIGHT TO KNOW WHETHER THE PROCUREMENT IS INTENDED TO ACHIEVE A MINIMUM STANDARD AT THE LOWEST COST OR WHETHER COST IS SECONDARY TO QUALITY." WE ARE UNABLE TO CONCLUDE, HOWEVER, THAT THIS DEFICIENCY IN THE NEGOTIATION PROCEDURE RESULTED IN A PATENTLY ILLEGAL AWARD AS IT IS OUR VIEW THAT EVEN IF OFFERORS HAD BEEN ADVISED OF THE WEIGHT OF PRICE AS AN EVALUATION FACTOR, THIS WOULD NOT HAVE AFFECTED THE RELATIVE RANKING BETWEEN AMS AND WESTAT. WE BELIEVE IT IS CLEAR FROM THE TECHNICAL NARRATIVE ABOVE THAT WESTAT'S SELECTION RESULTED PRIMARILY BECAUSE OF THE SUPERIORITY OF ITS TECHNICAL PROPOSAL, RATHER THAN FROM ANY NEGATIVE FINDING WITH RESPECT TO AMS' COST ESTIMATES.

FOR THE ABOVE REASONS, THE PROTEST OF AMS IS DENIED.