B-167508, DEC. 8, 1969

B-167508: Dec 8, 1969

Additional Materials:

Contact:

Edda Emmanuelli Perez
(202) 512-2853
EmmanuelliPerezE@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

EVALUATION PROPRIETY WHILE PROPOSAL FOR AIR TRAFFIC CONTROL EQUIPMENT UPON EVALUATION BY SOURCE SELECTION BOARD WAS REJECTED AS "TECHNICALLY UNACCEPTABLE" AND AWARD WAS MADE TO OTHER THAN LOW BIDDER. GAO WILL NOT OBJECT TO AWARD BUT IT DOES FEEL PROCUREMENT PROCEDURES EMPLOYED WERE DEFICIENT IN DETERMINATION AND APPLICATION OF EVALUATION WEIGHTS AND THAT EVALUATION OF PROPOSALS FOR PURPOSES OF DETERMINING WHETHER PROPOSAL WAS WITHIN COMPETITIVE RANGE TECHNICALLY WAS NOT IN STRICT ACCORDANCE WITH SOURCE SELECTION INSTRUCTIONS OR PRIOR COMPTROLLER GENERAL DECISIONS. IN FUTURE SOURCE SELECTION ADVISORY COUNCIL PREDETERMINED EVALUATION WEIGHTS SHOULD BE INCLUDED IN REASONABLE DETAIL IN SOLICITATION DOCUMENTS AND CONSIDERED AND APPLIED BEFORE PROPOSALS ARE DETERMINED UNACCEPTABLE AND NOT WITHIN COMPETITIVE RANGE.

B-167508, DEC. 8, 1969

SPECIFICATIONS--CONFORMABILITY OF EQUIPMENT, ETC; OFFERED--TECHNICAL DEFICIENCIES--EVALUATION PROPRIETY WHILE PROPOSAL FOR AIR TRAFFIC CONTROL EQUIPMENT UPON EVALUATION BY SOURCE SELECTION BOARD WAS REJECTED AS "TECHNICALLY UNACCEPTABLE" AND AWARD WAS MADE TO OTHER THAN LOW BIDDER, GAO WILL NOT OBJECT TO AWARD BUT IT DOES FEEL PROCUREMENT PROCEDURES EMPLOYED WERE DEFICIENT IN DETERMINATION AND APPLICATION OF EVALUATION WEIGHTS AND THAT EVALUATION OF PROPOSALS FOR PURPOSES OF DETERMINING WHETHER PROPOSAL WAS WITHIN COMPETITIVE RANGE TECHNICALLY WAS NOT IN STRICT ACCORDANCE WITH SOURCE SELECTION INSTRUCTIONS OR PRIOR COMPTROLLER GENERAL DECISIONS. IN FUTURE SOURCE SELECTION ADVISORY COUNCIL PREDETERMINED EVALUATION WEIGHTS SHOULD BE INCLUDED IN REASONABLE DETAIL IN SOLICITATION DOCUMENTS AND CONSIDERED AND APPLIED BEFORE PROPOSALS ARE DETERMINED UNACCEPTABLE AND NOT WITHIN COMPETITIVE RANGE.

TO MR. SECRETARY:

BY LETTERS DATED SEPTEMBER 18 AND NOVEMBER 4, 1969, WITH ENCLOSURES, THE DEPUTY CHIEF, PROCUREMENT OPERATIONS DIVISION, DIRECTORATE, PROCUREMENT POLICY, DCS/S-AND-L, FURNISHED OUR OFFICE WITH REPORTS ON THE PROTEST OF BURROUGHS CORPORATION UNDER REQUEST FOR QUOTATIONS (RFQ) NO. F33657-69-Q- 0564, ISSUED BY HEADQUARTERS AERONAUTICAL SYSTEMS DIVISION, WRIGHT- PATTERSON AIR FORCE BASE, OHIO.

THE SUBJECT RFQ, ISSUED ON MARCH 24, 1969, CONTEMPLATED A FIRM-FIXED PRICE MULTIYEAR CONTRACT FOR AIR TRAFFIC CONTROL EQUIPMENT DESIGNATED AS AN/TPX-42 INTERROGATOR SETS. THE RFQ STATED IN PARAGRAPH 18 THAT:

"PROPOSALS WILL BE EVALUATED BY A SOURCE SELECTION BOARD REPRESENTING INTERESTED GOVERNMENT AGENCIES IN ACCORDANCE WITH THE SOURCE SELECTION EVALUATION CRITERIA SET FORTH IN ATTACHMENT -3- HERETO AND MADE A PART HEREOF.' ATTACHMENT "3" IS SET OUT BELOW IN ITS ENTIRETY: "I. INTRODUCTION

"THESE CRITERIA IDENTIFY AREAS AND ITEMS OF CONCERN TO THE GOVERNMENT IN DETERMINING THE SOURCE FOR AN INTERROGATOR SET AN/TPX-42 ( ). THE CRITERIA LISTED ARE INTENDED TO SHOW THE SCOPE OF THE EVALUATION TO BE PERFORMED ON PROPOSALS SUBMITTED IN RESPONSE TO THE REQUEST FOR PROPOSAL. EVALUATION OF THE PROPOSALS WILL BE ACCOMPLISHED BY COMPARING PROPOSAL ELEMENTS AGAINST STANDARDS WHICH WILL BE DEVELOPED BY THE GOVERNMENT IN ACCORDANCE WITH THESE CRITERIA. THESE STANDARDS ARE PREPARED PRIOR TO RECEIPT OF PROPOSALS AND ARE CONSISTENT WITH THE REQUIREMENTS CONTAINED IN THE REQUEST FOR PROPOSAL AND STATEMENT OF WORK. THEIR CONTENT IS DIVULGED ONLY TO GOVERNMENT PERSONNEL DIRECTLY INVOLVED IN THE EVALUATION AND THE SELECTION PROCESS.'II. GENERAL CONSIDERATIONS

"A. THE CONTRACTOR'S PAST PERFORMANCE WILL BE EVALUATED TO DETERMINE THE ABILITY TO MEET AIR FORCE TECHNICAL REQUIREMENTS, BUILD A QUALITY PRODUCT, CONTROL COSTS, MAKE TIMELY DELIVERY, PRODUCE WITHOUT UNDUE GOVERNMENT ASSISTANCE AND CORRECT PAST PERFORMANCE DEFICIENCIES. IN ADDITION, EACH CONTRACTOR'S CURRENT MANAGEMENT AND PRODUCTION CAPABILITY WILL BE EVALUATED TO DETERMINE ADEQUACY AND SUITABILITY TO ACCOMPLISH THE REQUIREMENTS OF THE PROGRAM. AS PART OF THIS EVALUATION, CONSIDERATION WILL BE GIVEN TO THE CONTRACTOR'S PRESENT AND PROJECTED WORKLOAD DURING THE TIME PERIOD OF CONTRACT PERFORMANCE. ALSO, DEMONSTRATED COMPETENCE IN HANDLING PROGRAMS OF A SIMILAR NATURE AND COMPLEXITY WILL BE TAKEN INTO ACCOUNT.

"B. RESPONSIVENESS OF THE CONTRACTOR TO THE CONTRACTUAL ASPECTS OF THE PROGRAM WILL BE ASSESSED AND TAKEN INTO ACCOUNT IN THE SOURCE SELECTION DECISION MAKING PROCESS. THIS WILL INCLUDE CONSIDERATIONS OF THE ADEQUACY AND COMPLETENESS OF THE PROPOSAL, TERMS AND CONDITIONS, DELIVERY SCHEDULE, AND PRICE. CONSIDERATION WILL BE GIVEN TO A CONTRACTOR'S CAPABILITY TO PERFORM AS A PRIME SYSTEM CONTRACTOR, PROPER ORGANIZATION, PROPER FACILITIES, AND CAPABILITY TO CONTROL AND UPDATE THE PROGRAM THROUGH SUCCESSFUL COMPLETION.'III. GENERAL CRITERIA

"A. THE SELECTION OF THE SOURCE FOR AWARD OF THE AN/TPX-42 PROGRAM CONTRACT WILL BE MADE ON THE BASIS OF AN INTEGRATED ASSESSMENT OF SEVERAL FACTORS DESIGNED TO DETERMINE WHICH PROPOSAL OFFERS THE BEST PROSPECT FOR SUCCESSFUL ACCOMPLISHMENT FOR THIS PROGRAM IN CONFORMANCE WITH ALL REQUIREMENTS AND AT REASONABLE COST. THIS INTEGRATED ASSESSMENT WILL TAKE INTO ACCOUNT THE FINDINGS RESULTING FROM EVALUATION OF CONTRACTOR'S PROPOSALS AGAINST THE ITEMS SHOWN IN THE SPECIFIC CRITERIA. IT WILL ALSO TAKE INTO ACCOUNT THOSE CONSIDERATIONS SHOWN UNDER PARAGRAPH II OF THESE CRITERIA. THIS WILL INCLUDE PARTICULAR ATTENTION TO THE WILLINGNESS OF THE CONTRACTOR TO COMMIT CONTRACTUALLY TO A PROGRAM WHICH WILL INSURE TIMELY DELIVERY OF A FULLY INTEGRATED SYSTEM WITH ALL NECESSARY LOGISTICS SUPPORT. THE RESULTS OF THIS INTEGRATED ASSESSMENT AND EVALUATION WILL BE CONSIDERED IN THE FOLLOWING ORDER OF IMPORTANCE:

"1. THE ABILITY OF THE INTERROGATOR SUBSYSTEM TO PERFORM THE MISSION SUPPORT TASKS AND TO BE DELIVERED FULLY OPERATIONAL AND SUPPORTABLE WITHIN THE ESTABLISHED ACQUISITION SCHEDULE.

"2. TOTAL COST TO THE GOVERNMENT.

"B. SPECIFIC CRITERIA ARE DIVIDED INTO 5 BASIC AREAS AS FOLLOWS: GENERAL CONSIDERATIONS, TECHNICAL, OPERATIONAL, LOGISTICS, AND COST TO GOVERNMENT.

"C. EACH AREA IS SUBDIVIDED INTO ITEMS WHICH DELINEATE AND DEFINE THE SCOPE AND NATURE OF THE PARTICULAR AREA. IN ADDITION, ALTHOUGH NOT A PART OF THESE CRITERIA, EACH ITEM IN TURN WILL BE SUBDIVIDED INTO FACTORS AS MAY BE NECESSARY FOR PROPER EVALUATION.

"D. CORRECTION POTENTIAL. WHERE AN ELEMENT OF THE CONTRACTOR'S PROPOSAL IS FOUND TO BE DEFICIENT IN ANY AREA, THE RATING GIVEN TO THAT ELEMENT WILL BE DEPENDENT ON THE EFFECT IT HAS ON THE OVERALL PROPOSAL, I.E., THE EXTENT OF REVISION NECESSARY TO CORRECT THE DEFICIENCY AND THE EFFECT OF THE CORRECTION ON OTHER ELEMENTS OF THE PROPOSAL, SUCH AS PERFORMANCE, SCHEDULE, AND COST.'IV. SPECIFIC CRITERIA

"A. TECHNICAL AREA. THIS AREA WILL INVOLVE CONSIDERATION OF ALL ELEMENTS OF THE CONTRACTOR'S BASIC DESIGN INCLUDING PROPOSED MODIFICATIONS FROM THE STANDPOINT OF PERFORMANCE, TECHNICAL EXCELLENCE, SYSTEM RELIABILITY, AND RECOGNITION OF AN ALLOWANCE FOR QUALIFICATION AND TESTING OF ITEMS NEWLY DEVELOPED OR MODIFIED. THE ITEMS TO BE EVALUATED WILL INCLUDE, BUT NOT LIMITED TO: TEST PLANS AND CONCEPTS, FROM A STANDPOINT OF ADEQUACY TO MEET THE OPERATIONAL REQUIREMENTS OF THE SYSTEM. THE ABILITY OF THE BASIC DESIGN TO ACCEPT OPTIONAL FEATURES AND/OR EQUIPMENTS WILL BE CONSIDERED.

"B. OPERATIONAL UTILITY AREA. OPERATIONAL UTILITY EVALUATION WILL CONSIDER THE CAPABILITY OF THE EQUIPMENT TO ACCOMPLISH FUNCTIONS IN ACCORDANCE WITH DOD AIMS SPECIFICATION 65-527B.

"C. LOGISTICS AREA. THE LOGISTICS EVALUATION WITHIN THIS AREA WILL INCLUDE AN EVALUATION OF THE CONTRACTOR'S LOGISTIC SUPPORT, THE SUPPLY SUPPORT PROPOSAL, AND MAINTENANCE/MAINTAINABILITY FORECAST AND AEROSPACE GROUND EQUIPMENT.

"D. COST TO THE GOVERNMENT AREA. REASONABLENESS, REALISM, AND COMPLETENESS OF THE CONTRACTOR'S PRICE PROPOSAL FOR PROTOTYPE AND PRODUCTION OPTIONS WILL BE ASSESSED UNDER THIS ITEM. IT IS CONTEMPLATED THAT THE BASIS OF THIS ASSESSMENT WILL BE PRICE ANALYSIS AS DEFINED IN THE ASPR, AS WELL AS BY COMPARATIVE ANALYSIS, ELEMENT BY ELEMENT OF THE COST QUOTATION SUPPORTING EACH CONTRACTOR'S PROPOSAL. IF DEEMED APPROPRIATE, COST ANALYSIS AND NEGOTIATION MAY BE EMPLOYED BY THE GOVERNMENT IN THE INTEREST OF ESTABLISHING SOUND PRICES FOR PERFORMANCE OF THE PROGRAM.'

A PREPROPOSAL CONFERENCE WAS CONDUCTED ON APRIL 15, 1969, FOR THE BENEFIT OF PROSPECTIVE OFFERORS, AT WHICH TIME PROSPECTIVE OFFERORS WERE REPORTEDLY ADVISED THAT A PARTIAL LIBRARY CONTAINING DATA ACCUMULATED ON THE RADAR EQUIPMENT COVERED BY THE RFQ WOULD BE MADE AVAILABLE TO OFFERORS BY APPOINTMENT. FIVE PROPOSALS WERE SUBMITTED BY JUNE 3, 1969, THE CLOSING DATE FOR RECEIPT OF PROPOSALS. AFTER A TECHNICAL EVALUATION BY A SOURCE SELECTION EVALUATION BOARD (SSEB) CONSTITUTED IN ACCORDANCE WITH AIR FORCE REGULATION 70-15 AND AIR FORCE MANUAL 70-10, REPORTEDLY CONSUMING SOME 2,500 MAN-HOURS, IT WAS DETERMINED THAT THE BURROUGHS PROPOSAL SHOULD BE REJECTED AS "TECHNICALLY UNACCEPTABLE.' ONE OTHER OF THE FIVE TECHNICAL PROPOSALS WAS ALSO REJECTED AS ,TECHNICALLY UNACCEPTABLE.'

BURROUGHS WAS ADVISED BY PHONE ON JULY 3, 1969, AND ALSO BY LETTER OF THE SAME DATE OF THE REJECTION ACTION. THE JULY 3 LETTER ADVISED ALSO THAT FURTHER NEGOTIATIONS WITH BURROUGHS WERE NOT CONTEMPLATED AND THAT PROPOSAL REVISIONS WOULD NOT BE ACCEPTED. APPARENTLY CROSSING IN THE MAIL WITH THE JULY 3 REJECTION NOTICE WAS A MODIFICATION OF THE BURROUGHS TECHNICAL PROPOSAL TO CORRECT A DEFICIENCY DISCOVERED BY BURROUGHS. THIS MODIFICATION WAS CONSIDERED TO BE A LATE MODIFICATION AND THEREFORE WAS REJECTED IN ACCORDANCE WITH THE ARMED SERVICES PROCUREMENT REGULATION (ASPR) 3-506.

BY LETTER DATED JULY 9, 1969, TO THE PROCURING CONTRACTING OFFICER, A COPY OF WHICH WAS FURNISHED TO OUR OFFICE, BURROUGHS PROTESTED THE REJECTION OF ITS PROPOSAL ON THE BASIS THAT THE BURROUGHS PRICE, BOTH FROM THE STANDPOINT OF PROPOSAL PRICES AND LIFE CYCLE COST, WAS SIGNIFICANTLY LOWER THAN THE PRICE SUBMITTED BY ANY OTHER OFFEROR. BURROUGHS ALSO CONTENDED THAT THE TECHNICAL EVALUATION OF ITS PROPOSAL WAS NOT PROPERLY CONDUCTED IN THAT THE CRITERIA STATED IN RFQ ATTACHMENT 3 WERE NOT FOLLOWED; THAT NO OPPORTUNITY FOR ORAL PRESENTATION WAS PROVIDED BURROUGHS; THAT OFFERORS WERE NOT INFORMED OF ALL EVALUATION FACTORS AND OF THE RELATIVE WEIGHTS TO BE APPLIED TO THOSE FACTORS (AS REQUIRED BY RECENT DECISIONS OF OUR OFFICE); AND THAT THE TYPICAL DEFICIENCIES CITED BY CONTRACTING OFFICIALS AS REQUIRING THE REJECTION OF THE BURROUGHS PROPOSAL WERE "MINOR OR INVALID.' ALSO, BURROUGHS COMPLAINED THAT THE DATA MADE AVAILABLE BY THE AIR FORCE FOR THE AID OF OFFERORS WAS "NEGLIGIBLE AND OF LITTLE USE" AND "CONTRIBUTED TO THE DEFICIENCIES CITED AGAINST US.' FINALLY, BURROUGHS CONTENDED THAT ITS JULY 3 PROPOSAL MODIFICATION SHOULD BE CONSIDERED BECAUSE IT WAS SUBMITTED BEFORE RECEIPT OF THE NOTICE THAT PROPOSAL MODIFICATIONS BY BURROUGHS WOULD NOT BE CONSIDERED AND BECAUSE NO FINAL DATE FOR SUBMISSION OF PROPOSAL MODIFICATIONS HAD BEEN SET WITH RESPECT TO THE OTHER OFFERORS AT THE TIME THE BURROUGHS MODIFICATION WAS REJECTED.

AFTER NOTIFICATION TO OUR OFFICE AS REQUIRED BY ASPR 2-407.9 (B) (2), AWARD WAS MADE ON OCTOBER 1, 1969, ON THE BASIS OF URGENCY, TO AIRBORNE INSTRUMENTS LABORATORY (AIL), DIVISION OF CUTLER-HAMMER, INC., AT A CONTRACT PRICE OF $40,444,333.

THE TECHNICAL REASONS ADVANCED FOR THE REJECTION OF THE BURROUGHS PROPOSAL WERE SUMMARIZED IN A REPORT TO OUR OFFICE FROM THE DEPARTMENT AS FOLLOWS:

"THE MOST SERIOUS CRITICISMS OF THE TECHNICAL PROPOSAL RELATE TO:

"A. THE DEGRADATION OF PRIMARY RADARS WHICH COULD RESULT IF THE SWEEP FAILED SINCE THE REQUIRED BACKUP SWEEP GENERATOR WAS NOT COVERED IN THE PROPOSAL.

"B. THE IMPACT OF LOSS OR REMOVAL OF ONE OR MORE INDICATORS AND THE AVAILABILITY OF BRACKET VIDEO UPON ITS DISPLAY PROCESSOR FAILURE WERE NOT PRESENTED.

"C. THE VIDEO COMPRESSION TECHNIQUES PROPOSED WOULD DEFINITELY DEGRADE THE PRIMARY RADAR DISPLAY SINCE THERE WERE INSUFFICIENT DATA CELLS IN THE VIDEO COMPRESSION MEMORY AT RANGES BELOW 30 MILES. THE LONG RANGE VIDEO DISPLAY RESOLUTION WAS ACCEPTABLE; HOWEVER, AT THE SHORT RANGE THE VIDEO WOULD HAVE POOR TO UNUSABLE RESOLUTION.

"D. THE AZIMUTH ERROR INTRODUCED BY THE DESIGN WAS FOUR TIMES THE ERROR ALLOWED IN THE SPECIFICATION.

"E. THE RFQ REQUIRED NEW INDICATORS TO REPLACE ALL ROTATING YOKE INDICATORS. BURROUGHS PROPOSED RETAINING EXISTING SEARCH CENTRAL CONTROL OF THE AN/MPA-9/31 WHICH ARE USED IN MOBILE GCA'S AND RAPCON'S WITH A ROTATING YOKE INDICATOR. THIS DOES NOT SATISFY THE REQUIREMENT ALTHOUGH IT WOULD REDUCE THE COST.

"F. THE CONTROLS BURROUGHS PROPOSED TO BE PLACED ON THE FACE OF NEW INDICATORS ON/OFF, FOCUS AND BRIGHTNESS WOULD NOT PERMIT THE REQUIRED SIX OTHER FUNCTIONS (I.E., MTI, MAX VIDEO, ETC.) TO BE PRESENTED AND OPTIMUM INDICATOR PARAMETERS TO BE SELECTED OR ADJUSTED BY THE OPERATOR.'

THE CONCLUSION OF THE TECHNICAL EVALUATORS WAS THAT "THE BURROUGHS' PROPOSAL WAS UNACCEPTABLE AND WAS NOT REASONABLY SUSCEPTIBLE TO BEING MADE ACCEPTABLE BY ADDITIONAL OR CLARIFYING INFORMATION WITHOUT BASICALLY CHANGING THE PROPOSAL AS SUBMITTED.' IN SUBSTANTIATION OF THIS CONCLUSION, A DETAILED "SUMMARY OF EVALUATION REPORT" WAS SUBMITTED BROKEN DOWN GENERALLY TO THE "ITEM" LEVEL IN ACCORDANCE WITH THE CRITERIA SET OUT IN RFQ ATTACHMENT 3, QUOTED ABOVE, AND CONTAINING NUMERICAL ,ITEM RATINGS" RANGING FROM 0-10. IN ADDITION, A DEBRIEFING WAS CONDUCTED BY THE AIR FORCE FOR BURROUGHS' BENEFIT ON OCTOBER 17, 1969, AT WHICH TIME AIR FORCE PERSONNEL SUMMARIZED THE EVALUATION PROCEDURES EMPLOYED AS WELL AS SPECIFIC AREAS WHEREIN THE BURROUGHS' PROPOSAL WAS DETERMINED TO BE INFERIOR. AT THE MEETING BURROUGHS REPRESENTATIVES IN EFFECT PRESENTED A REBUTTAL OF THE AIR FORCE TECHNICAL ARGUMENTS, CONTENDING GENERALLY THAT WHILE THE BURROUGHS PROPOSAL CONTAINED TECHNICAL DEFICIENCIES, SUCH DEFICIENCIES WERE MINOR IN NATURE AND PROPERLY SHOULD HAVE BEEN CORRECTED BY EXTENDING NEGOTIATION OPPORTUNITY TO BURROUGHS.

CLEARLY, THERE IS STILL STRONG DISAGREEMENT BETWEEN THE AIR FORCE AND BURROUGHS AS TO THE ADEQUACY OF ITS TECHNICAL PROPOSAL. IN THIS REGARD, HOWEVER, WE HAVE STATED MANY TIMES THAT IT IS NOT THE FUNCTION OF OUR OFFICE TO RESOLVE SUCH TECHNICAL DISPUTES. NOT ONLY DOES OUR OFFICE NOT POSSESS THE TECHNICAL EXPERTISE TO RESOLVE SUCH DISPUTES, BUT THE RESOLUTION OF SUCH DISPUTES PROPERLY FALLS WITHIN THE WIDE RANGE OF DISCRETION RESERVED TO THE ADMINISTRATIVE AGENCIES SUBJECT TO QUESTION BY OUR OFFICE ONLY WHERE CLEAR EVIDENCE OF ABUSE OF THAT DISCRETION IS PRESENT.

CONCERNING THE NATURE AND EXTENT OF NEGOTIATIONS TO BE CONDUCTED WITH PROSPECTIVE CONTRACTORS AND THE DETERMINATION AS TO WHETHER INDIVIDUAL PROPOSALS ARE OR ARE NOT WITHIN A "COMPETITIVE GE," AND THEREFORE ENTITLED TO NEGOTIATION OPPORTUNITY, THE FOLLOWING STATEMENT IS CONTAINED IN 48 COMP. GEN. 314 (1968): "WITH REGARD TO THE NATURE AND EXTENT OF NEGOTIATIONS TO BE CONDUCTED WITH OFFERORS, ASPR 3-805.1, IN IMPLEMENTATION OF 10 U.S.C. 2304 (G), REQUIRES THAT -WRITTEN OR ORAL DISCUSSIONS SHALL BE CONDUCTED WITH ALL RESPONSIBLE OFFERORS WHO SUBMIT PROPOSALS WITHIN A COMPETITIVE RANGE, PRICE, AND OTHER FACTORS CONSIDERED. - ALSO, ASPR 3-805.1 (A) (V) REQUIRES THAT -IN ANY CASE WHERE THERE IS UNCERTAINTY AS TO THE PRICING OR TECHNICAL ASPECTS OF ANY PROPOSALS, THE CONTRACTING OFFICER SHALL NOT MAKE AWARD WITHOUT FURTHER EXPLORATION AND DISCUSSION PRIOR TO AWARD.-WHILE OUR OFFICE HAS HELD THAT A PROPOSAL MUST BE CONSIDERED TO BE WITHIN A COMPETITIVE RANGE SO AS TO REQUIRE NEGOTIATIONS UNLESS IT IS SO TECHNICALLY INFERIOR OR OUT OF LINE WITH REGARD TO PRICE THAT MEANINGFUL NEGOTIATIONS ARE PRECLUDED, WE HAVE ALSO RECOGNIZED THAT THE DETERMINATION OF COMPETITIVE RANGE, PARTICULARLY AS REGARDS TECHNICAL CONSIDERATIONS, IS PRIMARILY A MATTER OF ADMINISTRATIVE DISCRETION WHICH WILL NOT BE DISTURBED IN THE ABSENCE OF A CLEAR SHOWING THAT SUCH DETERMINATION WAS AN ARBITRARY ABUSE OF DISCRETION. SEE B- 164313 DATED JULY 5, 1968, 48 COMP. GEN. ---.

"THERE IS VEHEMENT DISAGREEMENT ON THE PART OF THE PROTESTANTS WITH REGARD TO THE TECHNICAL CONCLUSIONS REACHED BY THE EVALUATION TEAM. HOWEVER, IN VIEW OF THE TECHNICAL NATURE OF THE DETERMINATIONS, OUR LACK OF EXPERTISE IN THIS AREA, AND THE WIDE RANGE OF DISCRETION VESTED IN THE CONTRACTING OFFICIALS, WE CANNOT CONCLUDE THAT THE DETERMINATION THAT ONLY ONE PROPOSAL WAS ACCEPTABLE CONSTITUTED A CLEAR ABUSE OF DISCRETION.'

WE THINK THAT THE ABOVE REASONING APPLIES EQUALLY HERE. NOT ONLY DO THE QUALIFICATIONS OF THE EVALUATION PERSONNEL APPEAR TO BE BEYOND ANY QUESTION, NO QUESTION AS TO THEIR GOOD FAITH WOULD SEEM TO EXIST NOR HAS ONE BEEN RAISED. ACCORDINGLY, SINCE WE THINK THAT THE RECORD CLEARLY DEMONSTRATES THAT THE SOURCE SELECTION OFFICIALS ACTED IN GOOD FAITH AND DID NOT ABUSE THE DISCRETION VESTED IN THEM, WE CONCLUDE THAT THE REJECTION OF THE BURROUGHS PROPOSAL AS "TECHNICALLY UNACCEPTABLE" IS NOT SUBJECT TO QUESTION. WE ALSO CONCLUDE THAT SINCE THE AWARD TO AIL WAS MADE IN GOOD FAITH UNDER REPORTED URGENCY CIRCUMSTANCES, THE CONTRACT AWARDED IS NOT SUBJECT TO QUESTION BY OUR OFFICE.

RESPECTING THE REJECTION OF THE BURROUGHS PROPOSAL MODIFICATION, WE LIKEWISE CONCLUDE THAT THE ACTION OF THE CONTRACTING OFFICIALS IN THAT REGARD IS NOT SUBJECT TO QUESTION. IN B-163882, FEBRUARY 13, 1969, AND B- 165837, FEBRUARY 28, 1969, WE HELD THAT A COMMON CUTOFF DATE MUST BE ESTABLISHED FOR THE SUBMISSION OF PROPOSAL MODIFICATIONS BY OFFERORS DETERMINED TO BE WITHIN A COMPETITIVE RANGE. ALSO, ASPR 3-805.1 (A) EXPLICITLY LIMITS REQUIREMENTS FOR NEGOTIATION, THE SETTING OF CUTOFF DATES, ETC., TO THOSE OFFERORS DETERMINED TO BE WITHIN A COMPETITIVE RANGE. INASMUCH AS BURROUGHS' PROPOSAL WAS REJECTED, IN EFFECT, AS NOT BEING WITHIN A COMPETITIVE RANGE, NO REQUIREMENT FOR NEGOTIATION WITH BURROUGHS EXISTED AND, THEREFORE, NO REQUIREMENT EXISTED THAT A CUTOFF DATE FOR PROPOSAL MODIFICATION BE ESTABLISHED WITH RESPECT TO BURROUGHS' PROPOSAL. ADDITIONALLY, THE SEPTEMBER 18 ADMINISTRATIVE REPORT ADVISED THAT "EVEN IF THIS MODIFICATION HAD BEEN ACCEPTED BY THE CONTRACTING OFFICER IT WOULD NOT HAVE RENDERED THE BURROUGHS TECHNICAL PROPOSAL ACCEPTABLE SINCE THE MODIFICATION ADDRESSED ITSELF ONLY TO THE INTERFERENCE BLANKER SUB-UNIT AND NOT THE OVERALL SYSTEM DEFICIENCIES.'

WE ALSO AGREE WITH THE POSITION TAKEN BY THE ADMINISTRATIVE REPORT THAT BURROUGHS' PRICING, WHILE SOME 5.5 MILLION DOLLARS LOWER THAN THE ULTIMATE AWARD PRICE, IS PROPERLY NOT FOR CONSIDERATION INASMUCH AS THE BURROUGHS PROPOSAL, AS SUBMITTED, WAS INADEQUATE AND NO BASIS EXISTED FOR ESTIMATING WHAT BURROUGHS' COSTS WOULD HAVE BEEN HAD ITS PROPOSAL BEEN ACCEPTABLE FROM A TECHNICAL STANDPOINT. IN ADDITION, ATTACHMENT 3 TO THE RFQ STATED THAT COST WAS TO BE A SECONDARY CONSIDERATION IN PROPOSAL EVALUATION.

SIMILARLY, WE AGREE WITH THE ADMINISTRATIVE POSITION THAT THE FACT THAT TECHNICALLY ACCEPTABLE OFFERORS RAISED NO OBJECTION TO THE UTILITY OF THE AIR FORCE REFERENCE DATA SUFFICIENTLY REBUTS THE BURROUGHS CONTENTION THAT SUCH DATA CONTRIBUTED TO ITS PROPOSAL DEFICIENCIES.

WHILE, AS MENTIONED EARLIER, IT IS OUR OPINION THAT THE AWARD IN THIS CASE IS NOT SUBJECT TO LEGAL OBJECTION, WE FEEL THAT THE PROCUREMENT PROCEDURES EMPLOYED WERE SOMEWHAT DEFICIENT IN THE DETERMINATION AND APPLICATION OF EVALUATION WEIGHTS.

OUR OFFICE HAS HELD THAT ASPR 3-501 (B) REQUIRES THAT SOLICITATIONS CONTAIN "THE INFORMATION NECESSARY TO ENABLE A PROSPECTIVE OFFEROR TO PREPARE A PROPOSAL OR QUOTATION PROPERLY" AND THAT ,SOUND PROCUREMENT POLICY DICTATES THAT OFFERORS BE INFORMED OF ALL EVALUATION FACTORS AND OF THE RELATIVE WEIGHTS TO BE ATTACHED TO EACH FACTOR.' 48 COMP. GEN. 314 (1968); B-167175 (1), OCTOBER 13, 1969; 49 COMP. GEN.----; 47 ID. 252 (1967); 44 ID. 439 (1965).

AS NOTED EARLIER, PROPOSAL EVALUATION WAS CONDUCTED IN ACCORDANCE WITH AIR FORCE MANUAL 70-10, JANUARY 22, 1968, IMPLEMENTING DOD DIRECTIVE 4105.62, APRIL 6, 1965, AND AIR FORCE REGULATION 70-15, MAY 17, 1968. THESE INSTRUCTIONS PRESCRIBED PROCEDURES FOR PROPOSAL EVALUATION AND SOURCE SELECTION UNDER WHICH INITIAL EVALUATIONS AND RECOMMENDATIONS ARE MADE BY SSEB TO A SOURCE SELECTION ADVISORY COUNCIL (SSAC), WHICH IN TURN MAKES RECOMMENDATIONS TO A SOURCE SELECTION AUTHORITY (SSA), DESIGNATED BY THE SECRETARY OF THE AIR FORCE. THE ABOVE DIVISION OF RESPONSIBILITY REQUIRES THAT THE SSEB EVALUATE PROPOSALS AGAINST STANDARDS PROVIDED TO IT BY THE SSAC. WEIGHTS TO BE APPLIED TO VARIOUS PROPOSAL ELEMENTS ARE DETERMINED BY THE SSAC AND APPLIED BY THE SSAC TO PROPOSALS AFTER EVALUATION HAS BEEN ACCOMPLISHED BY THE SSEB. IN THE INSTANT CASE, THE NOVEMBER 4, 1969, AIR FORCE REPORT CONTAINS THE FOLLOWING STATEMENT WITH REGARD TO THE EVALUATION OF THE TWO PROPOSALS DETERMINED TO BE UNACCEPTABLE:

"WITH RESPECT TO THE EVALUATION WEIGHTS AND STANDARDS USED IN THIS PROCUREMENT, THE LETTER OF INSTRUCTIONS FROM THE CHAIRMAN, SOURCE SELECTION ADVISORY COUNCIL (SSAC), TO THE CHAIRMAN, SOURCE SELECTION EVALUATION BOARD (SSEB), DIRECTED THAT THE BOARD USE THE SAME SOURCE SELECTION CRITERIA AS WERE INCLUDED IN ATTACHMENT 3 TO THE RFQ. ALL PROPOSALS WERE EVALUATED BY THE SSEB IN ACCORDANCE WITH THESE CRITERIA AND THE OTHER REQUIREMENTS OF THE RFQ. THE BURROUGHS AND THE TECHNICAL PROPOSALS WERE EVALUATED AS UNACCEPTABLE BY THE SSEB AND SO RECOMMENDED TO THE SSAC. THIS RECOMMENDATION WAS SUPPORTED BY THE SSAC AND WAS SUBSEQUENTLY APPROVED BY THE SOURCE SELECTION AUTHORITY (COMMANDER, AERONAUTICAL SYSTEMS DIVISION). APPLICATION OF WEIGHTS TO THE EVALUATIONS MADE BY THE SSEB WAS MADE BY THE SSAC ONLY TO PROPOSALS WHICH WERE EVALUATED ACCEPTABLE. THEREFORE, ALTHOUGH THE BURROUGHS PROPOSAL WAS COMPLETELY EVALUATED AS ALL OTHER PROPOSALS AGAINST THE SAME CRITERIA AND STANDARDS, WEIGHTS WERE NOT APPLIED THERETO.'

IN OUR OPINION, INITIAL EVALUATION OF PROPOSALS IN THE MANNER DESCRIBED ABOVE FOR PURPOSES OF DETERMINING WHETHER THEY CAME WITHIN A COMPETITIVE RANGE TECHNICALLY WAS NOT IN STRICT ACCORDANCE WITH THE ABOVE-CITED SOURCE SELECTION INSTRUCTIONS OR WITH PRIOR DECISIONS OF OUR OFFICE. SPECIFICALLY, SECTION VI, PARAGRAPH "D" OF DOD DIRECTIVE 4105.62 PROVIDES THAT THE SSAC SHALL, IN ADDITION TO ESTABLISHING EVALUATION CRITERIA AND WEIGHTS TO BE APPLIED TO THOSE CRITERIA,"ESTABLISH THE RELATIVE IMPORTANCE OF THE EVALUATION CRITERIA IN A FORM TO BE USED IN PROPOSAL SOLICITATION" AND "REVIEW THE EVALUATION BOARD FINDINGS AND APPLY THE ESTABLISHED WEIGHTS TO THE EVALUATION RESULTS.' SIMILARLY, AIR FORCE MANUAL 70-10 STATES AT PARAGRAPH 29 THAT:

"* * * THE SELECTION PLAN ESTABLISHES THE RELATIVE IMPORTANCE OF THE PROPOSAL ELEMENTS TO BE EVALUATED FOR A PARTICULAR PROCUREMENT. THE SSAC ASSIGNS WEIGHTS TO THE ITEMS TO BE USED IN THE EVALUATION TO SHOW THIS RELATIVE IMPORTANCE. PROPOSALS ARE EVALUATED, AND THE ITEMS ARE PRESENTED IN THE EVALUATION REPORT WITH UNWEIGHTED OR RAW SCORES. SSAC APPLICATION OF THE PREDETERMINED WEIGHTS CREATES BASIC SOURCE SELECTION DECISION DATA. * * *"

WHERE THE WEIGHTS TO BE APPLIED TO SSAC ESTABLISHED EVALUATION STANDARDS ARE DETERMINED IN ADVANCE OF SOLICITATION ISSUANCE, AS THE ABOVE-CITED INSTRUCTIONS WOULD SEEM TO REQUIRE, THOSE WEIGHTS, IN OUR OPINION, SHOULD BE SPELLED OUT IN THE SOLICITATION DOCUMENT WITH AS MUCH PARTICULARITY AS POSSIBLE UNDER THE CIRCUMSTANCES OF THE PROCUREMENT INVOLVED. TO MEASURE THE RELATIVE MERITS OF PROPOSALS BY THE PLACING OF EMPHASIS IN AREAS NOT APPROPRIATELY IDENTIFIED IN THE SOLICITATION CONSTITUTES A DISSERVICE TO OFFERORS WHO EXPENDED CONSIDERABLE TIME AND MONEY IN PROPOSAL PREPARATION. NEITHER DOES SUCH PRACTICE CONFORM TO THE INTENT OF THE DECISIONS OF OUR OFFICE IN THIS AREA. IN OUR OPINION, THE FACT THAT THE PREDETERMINED WEIGHTS ARE NOT ACTUALLY APPLIED TO PROPOSALS UNTIL AFTER THE INITIAL EVALUATION BY THE SSEB DOES NOT EXCUSE THE FAILURE TO INCLUDE SUCH WEIGHTS IN THE RFQ.

WE ALSO QUESTION THE REJECTION OF THE BURROUGHS PROPOSAL WITHOUT THE APPLICATION OF WEIGHTS ESTABLISHED BY THE SSAC. WHILE THERE APPEARS TO BE LITTLE QUESTION THAT HAD THE PREDETERMINED WEIGHTS BEEN APPLIED TO THE BURROUGHS PROPOSAL IT WOULD HAVE BEEN FOUND TECHNICALLY LACKING IN THOSE AREAS IN WHICH GREATER EMPHASIS WAS PLACED, WE THINK THAT FAIRNESS AND LOGIC SHOULD HAVE REQUIRED THAT BEFORE A DETERMINATION WAS MADE THAT A PROPOSAL ADMITTEDLY ACCEPTABLE IN SOME AREAS DID NOT MEASURE UP ON AN OVERALL BASIS, SOME CONSIDERATION SHOULD HAVE BEEN GIVEN TO ITS RESPONSES TO THOSE AREAS OF GREATER EMPHASIS AND WEIGHT.

ACCORDINGLY, IT IS SUGGESTED THAT IN THE FUTURE CARE BE TAKEN TO INCLUDE SSAC PREDETERMINED EVALUATION WEIGHTS IN REASONABLE DETAIL IN SOLICITATION DOCUMENTS AND THAT SUCH WEIGHTS BE CONSIDERED AND APPLIED BEFORE PROPOSALS ARE DETERMINED TO BE UNACCEPTABLE AND HENCE NOT WITHIN A COMPETITIVE RANGE.