Skip to main content

B-183216, JUN 16, 1975

B-183216 Jun 16, 1975
Jump To:
Skip to Highlights

Highlights

APPLICATION IN EVALUATION OF TECHNICAL SUBCRITERION - "CAPABILITY TO CONDUCT OBJECTIVE STUDY" - IS NOT OBJECTIONABLE IN ITSELF. AGENCY CAN MAKE REASONED JUDGMENT THAT ONE OFFEROR IS MORE OR LESS LIKELY THAN ANOTHER TO CARRY OUT PROJECT IN "OBJECTIVE" MANNER. 2. PROPOSAL WHICH PREDETERMINED OPTIMAL METHOD FOR ASBESTOS ANALYSIS WAS PROPERLY DOWNGRADED BY EPA FOR LACK OF OBJECTIVITY. RECORD DOES NOT SUPPORT CONTENTION THAT EVALUATION WAS UNREASONABLE BECAUSE IT WAS PREMISED ON MORE LIMITED PURPOSES THAN THOSE STATED IN RFP. WHILE CONCERNS EXPRESSED BY FOUR OF FIVE OFFERORS IN COMPETITIVE RANGE REGARDING APPLICATION OF RFP'S "OBJECTIVITY" SUBCRITERION ARE UNDERSTOOD. NO BASIS IS SEEN ON RECORD FOR GAO OBJECTION TO AGENCY'S EVALUATION OF COMPLEX TECHNICAL ISSUE.

View Decision

B-183216, JUN 16, 1975

1. WHERE RFP CONTEMPLATES STUDY AND EVALUATION OF SCIENTIFIC PROCEDURES AND EVOLVEMENT OF OPTIMAL METHOD, APPLICATION IN EVALUATION OF TECHNICAL SUBCRITERION - "CAPABILITY TO CONDUCT OBJECTIVE STUDY" - IS NOT OBJECTIONABLE IN ITSELF. AGENCY CAN MAKE REASONED JUDGMENT THAT ONE OFFEROR IS MORE OR LESS LIKELY THAN ANOTHER TO CARRY OUT PROJECT IN "OBJECTIVE" MANNER. 2. PROPOSAL WHICH PREDETERMINED OPTIMAL METHOD FOR ASBESTOS ANALYSIS WAS PROPERLY DOWNGRADED BY EPA FOR LACK OF OBJECTIVITY, AND RECORD DOES NOT SUPPORT CONTENTION THAT EVALUATION WAS UNREASONABLE BECAUSE IT WAS PREMISED ON MORE LIMITED PURPOSES THAN THOSE STATED IN RFP. NEED FOR MAJOR REVISIONS TO PROTESTER'S UNACCEPTABLE PROPOSAL JUSTIFIES EXCLUSION FROM COMPETITIVE RANGE. 3. WHILE CONCERNS EXPRESSED BY FOUR OF FIVE OFFERORS IN COMPETITIVE RANGE REGARDING APPLICATION OF RFP'S "OBJECTIVITY" SUBCRITERION ARE UNDERSTOOD, NO BASIS IS SEEN ON RECORD FOR GAO OBJECTION TO AGENCY'S EVALUATION OF COMPLEX TECHNICAL ISSUE. ALSO, SINCE AGENCY HAS BECOME AWARE OF OFFERORS' COMMENTS IN CONNECTION WITH PROTEST PROCEEDINGS, IT IS BELIEVED THAT FINAL EVALUATION OF OFFERS PREPARATORY TO SOURCE SELECTION HAS BEEN MADE WITH BENEFIT OF BROADER PERSPECTIVE THAN WAS AVAILABLE AT TIME OF INITIAL EVALUATION.

EMVENTIONS INC.:

THE PROTEST OF EMVENTIONS INC. IS DIRECTED AT THE CONDUCT OF THE TECHNICAL EVALUATION AND THE EXCLUSION OF THE EMVENTIONS OFFER FROM THE COMPETITIVE RANGE IN A NEGOTIATED PROCUREMENT CONDUCTED BY THE ENVIRONMENTAL PROTECTION AGENCY (EPA) WHICH SOUGHT OFFERS TO STUDY AND EVALUATE ELECTRON MICROSCOPE METHODS FOR MEASUREMENT OF AIRBORNE ASBESTOS CONCENTRATION AND EVOLVEMENT OF AN OPTIMAL PROCEDURE (REQUEST FOR PROPOSALS (RFP) NO. DU-75-B019).

EMVENTIONS' PRINCIPAL CONTENTION CONCERNS ONE OF THE TECHNICAL EVALUATION SUBCRITERIA, THE "CAPABILITY TO CONDUCT AN OBJECTIVE STUDY."

THIS SUBCRITERION WAS ACCORDED A NUMERICAL WEIGHT OF 10 OUT OF A TOTAL OF 100 TECHNICAL EVALUATION POINTS. THE PROTESTER CONTENDS EPA MISAPPLIED THE SUBCRITERION AND ERRONEOUSLY DOWNGRADED SEVEN OF THE 15 PROPOSALS IN THIS AREA, INCLUDING SEVERAL OF THE FIVE PROPOSALS WITHIN THE COMPETITIVE RANGE. EMVENTIONS BELIEVES THE DOWNGRADING OCCURRED BECAUSE EPA EQUATED EXPERIENCE IN ASBESTOS RESEARCH WITH LACK OF OBJECTIVITY, OR BIAS, AND CONTENDS THAT THIS UNSUPPORTED ASSUMPTION IS DEMEANING TO THE SCIENTIFIC INTEGRITY OF THE INSTITUTIONS CONCERNED. NOTING THAT THREE OFFERORS WITH LITTLE OR NO ASBESTOS EXPERIENCE RECEIVED HIGH SCORES ON THIS SUBCRITERION, EMVENTIONS FURTHER CONTESTS WHAT IT TERMS EPA'S UNSUPPORTED ASSUMPTION THAT LACK OF EXPERIENCE AND OBJECTIVITY ARE EQUIVALENT.

EMVENTIONS' POSITION ON THIS ISSUE HAS RECEIVED SUPPORT IN THE COMMENTS FROM SEVERAL OTHER INTERESTED PARTIES WHOSE OFFERS WERE FOUND TO BE WITHIN THE COMPETITIVE RANGE. ONE OFFEROR WHICH WAS DOWNGRADED ON THE OBJECTIVITY SUBCRITERION COMMENTS, "SINCE THE VERY ESSENCE OF SCIENCE IS OBJECTIVITY, TO SAY THAT WE WOULD BE BIASED IS TO SAY THAT WE CANNOT ACT AS GOOD SCIENTISTS." ANOTHER OFFEROR CONTESTS THE ACCURACY OF A STATEMENT IN THE EPA EVALUATION THAT ITS EXPERIENCE IN ASBESTOS ANALYSIS HAS BEEN MAINLY THROUGH ITS WORK FOR A PARTICULAR COMPANY, JOHNS-MANVILLE, ALLEGING THAT IN FACT LESS THAN 5 PERCENT OF ITS WORK OVER THE PAST 3 YEARS HAS BEEN WITH JOHNS-MANVILLE. ALTOGETHER, FOUR OF THE FIVE OFFERORS IN THE COMPETITIVE RANGE HAVE COMMENTED ADVERSELY CONCERNING THE APPLICATION OF THE OBJECTIVITY SUBCRITERION.

EPA'S POSITION IS BASICALLY THAT THE OBJECTIVITY SUBCRITERION REPRESENTS A VALID CONCERN OF THE CONTRACTING AGENCY IN THE CONTEXT OF THE PRESENT PROCUREMENT, AND THAT IT WAS PROPERLY APPLIED IN THE TECHNICAL EVALUATION OF THE PROPOSALS, AS EVIDENCED BY THE PERTINENT DOCUMENTS OF RECORD.

THE SCOPE OF OUR OFFICE'S REVIEW OF AN AGENCY'S TECHNICAL JUDGMENTS ON COMPLEX SCIENTIFIC ISSUES WAS DESCRIBED AS FOLLOWS IN B-178526, AUGUST 15, 1973, A CASE INVOLVING TECHNICAL QUESTIONS OF COMPETING SCIENTIFIC METHODOLOGIES AND THEIR "OBJECTIVITY":

"OUR OFFICE DOES NOT POSSESS THE RESOURCES TO RENDER AN INDEPENDENT SCIENTIFIC JUDGMENT ON THE MERITS OF A TECHNICAL ISSUE AS THE ONE INVOLVED HERE. IN SUCH SITUATIONS, IN VIEW OF THE WIDE RANGE OF DISCRETION VESTED IN THE CONTRACTING OFFICIALS, WE MUST DEFER TO THEIR JUDGMENT UNLESS EVIDENCE IS PRESENTED WHICH CLEARLY DEMONSTRATES THAT THE TECHNICAL DETERMINATION IS INCORRECT, *** OR WHICH OTHERWISE SHOWS THAT THE CONTRACTING OFFICIALS ACTED ARBITRARILY. *** IN THE PRESENT CASE, RESOLUTION OF THE QUESTION OF THE OPTIMUM SCIENTIFIC METHODOLOGY TO BE EMPLOYED IN SUCH TESTS, AND THE ACCURACY OF THE RESULTS DERIVED THEREFROM, WILL APPARENTLY HAVE TO AWAIT THE RESULTS OF FURTHER RESEARCH. WE DO NOT BELIEVE YOUR ALLEGATIONS CLEARLY DEMONSTRATE THAT THE ARMY'S CONCLUSION IN THIS MATTER WAS INCORRECT. ALSO, THE RECORD ADEQUATELY DEMONSTRATES THAT THE CONTRACTING OFFICIALS ACTED UPON CAREFUL CONSIDERATION OF THE TECHNICAL ISSUES INVOLVED AND NOT IN AN ARBITRARY MANNER. ***"

ALSO, WE HAVE EXPRESSED THE VIEW THAT THE EXISTENCE OF AN "HONEST DIFFERENCE OF TECHNICAL OPINION" IS NOT NECESSARILY SUFFICIENT CAUSE FOR OUR OFFICE TO SUBSTITUTE ITS JUDGMENT FOR THE AGENCY'S. 52 COMP. GEN. 393, 399-400 (1972).

IN ADDITION, IT IS WELL ESTABLISHED THAT THE QUESTION OF WHETHER A PROPOSAL IS TO BE EXCLUDED FROM THE COMPETITIVE RANGE BECAUSE OF TECHNICAL UNACCEPTABILITY IS PRIMARILY FOR RESOLUTION BY THE CONTRACTING AGENCY, WHICH MUST BE ACCORDED A REASONABLE DEGREE OF JUDGMENT AND DISCRETION IN MAKING SUCH DETERMINATIONS. WHERE THE RECORD DOCUMENTS DEFICIENCIES SO SUBSTANTIAL THAT A PROPOSAL COULD NOT BE MADE ACCEPTABLE WITHOUT MAJOR REVISIONS, AND THIS BASIS FOR REJECTION OF THE PROPOSAL IS NOT SHOWN BY THE PROTESTER'S EVIDENCE TO BE WITHOUT A REASONABLE BASIS, OUR OFFICE WILL NOT OBJECT TO THE AGENCY'S ACTION. SEE ESSEX CORPORATION, B-182595, APRIL 23, 1975, AND DECISIONS CITED THEREIN; B-178526, SUPRA.

WE HAVE NO DIFFICULTY WITH THE CONCEPT OF THE OBJECTIVITY SUBCRITERION. THE RFP APPEARS TO CONTEMPLATE A FRESH OVERVIEW OF A PARTICULAR AREA OF SCIENTIFIC STUDY. WE THINK EPA CAN BE LEGITMATELY CONCERNED AS TO WHETHER SOME OFFERORS ARE MORE LIKELY TO SATISFACTORILY ACCOMPLISH THIS TASK THAN OTHERS. IN THIS LIGHT, WE DO NOT BELIEVE A FINDING OF LACK OF OBJECTIVITY OR BIAS IS PROPERLY TO BE TAKEN IN ITS MOST PEJORATIVE SENSE. THAT IS, WE DO NOT BELIEVE THAT TO CONCLUDE AN OFFEROR MAY LACK OBJECTIVITY UNDER THE PARTICULAR CIRCUMSTANCES OF THIS PROCUREMENT IS TO CONCLUDE THAT THE OFFEROR DOES NOT USE GOOD SCIENTIFIC METHODS GENERALLY OR IS NOT STAFFED BY GOOD SCIENTISTS. IN OUR VIEW, IT MERELY MEANS THAT, GIVEN THE NEED FOR A FRESH OVERVIEW BY A CONTRACTOR WHICH CAN FUNCTION BASICALLY UNENCUMBERED BY PRECONCEPTIONS ROOTED IN ITS PRIOR EXPERIENCE, THE PARTICULAR OFFEROR WHICH WAS DOWNGRADED WAS JUDGED LESS CAPABLE IN THIS RESPECT THAN OTHER OFFERORS. THIS WOULD APPEAR TO BE AN AREA CALLING FOR SUBTLE AND COMPLEX TECHNICAL JUDGMENTS BASED UPON THE PROPOSALS SUBMITTED AND THE OFFERORS' PAST EXPERIENCE.

WE AGREE THAT THE SUBCRITERION SHOULD NOT BE APPLIED IN A MECHANICAL, UNTHINKING MANNER, SUCH AS BY EQUATING EXPERIENCE WITH LACK OF OBJECTIVITY, AND VICE VERSA. THE QUESTION, THEN, IS WHETHER THE RECORD ADEQUATELY SHOWS THAT THE SUBCRITERION WAS APPLIED IN A REASONED MANNER AND WHETHER EVIDENCE HAS BEEN PRESENTED TO OVERCOME THAT SHOWING AND DEMONSTRATE THAT THE AGENCY ACTED WITHOUT A REASONABLE BASIS.

WE HAVE REVIEWED THE RECORD OF THE TECHNICAL EVALUATION. AS FOR THE EMVENTIONS PROPOSAL, EPA CONCLUDED THAT THE PROPOSAL SHOWED A PREDETERMINATION OF AN OPTIMAL PROCEDURE AND OFFERED ONLY EFFORTS TO DEVELOP AND TEST IT. EPA CONCLUDED THAT THE LACK OF OBJECTIVITY THUS EVIDENT MADE EMVENTIONS' PROPOSED STUDY INAPPROPRIATE.

EMVENTIONS DOES NOT DENY THAT ITS PROPOSAL PREDETERMINED AN OPTIMAL PROCEDURE. RATHER, IT BELIEVES THAT SINCE ITS PREFERENCE WAS BASED UPON SOUND SCIENTIFIC PRINCIPLES AND OBJECTIVE EXPERIENCE, IT WAS OBJECTIVE IN EVERY SENSE OF THE DEFINITION OF THE WORD.

AS USED IN THE RFP, WE THINK THE MEANING OF "OBJECTIVE" REFERS TO "EXPRESSING OR INVOLVING THE USE OF FACTS WITHOUT DISTORTION BY PERSONAL FEELINGS OR PREJUDICES." WEBSTER'S THIRD NEW INTERNATIONAL DICTIONARY. WE NOTE THAT THE RFP'S WORK EFFORT CONTEMPLATED A PROCESS OF STUDY AND EVALUATION, TO BE FOLLOWED BY EVOLVEMENT OF AN OPTIMAL PROCEDURE. IN THIS LIGHT, WE BELIEVE THAT TO HAVE PREDETERMINED THE OPTIMAL PROCEDURE IN THE PROPOSAL INDICATES A LACK OF OBJECTIVITY, IN THAT THE OFFEROR HAS NOT INDICATED A WILLINGNESS TO BECOME SUFFICIENTLY INVOLVED WITH ALL OF THE FACTS OF THE SITUATION. WE THINK THIS IS TRUE EVEN IF THE PROPOSAL'S OPTIMAL APPROACH IN ITSELF WAS BASED UPON THE SOUNDEST SCIENTIFIC PRINCIPLES.

EMVENTIONS NEXT CONTENDS THAT THE TECHNICAL EVALUATION WAS FOCUSED ON MORE LIMITED PURPOSES THAN THOSE STATED IN THE RFP. EMVENTIONS POINTS OUT THAT THE RFP REQUESTED OFFERORS TO "DEVELOP AND DEFINE" A SINGLE PROCEDURE AS OPTIMAL, WHEREAS A MEMORANDUM WRITTEN BY THE TECHNICAL EVALUATOR INDICATES THAT THE ACTUAL INTENDED PURPOSE WAS TO "SELECT OR DEFINE" A PROCEDURE. EMVENTIONS FURTHER CONTENDS THAT THE MEMORANDUM INDICATES THAT EPA INTENDED TO OBTAIN A PROCEDURE TO BE USED IN ANIMAL TOXICITY STUDIES, A LIMITED PURPOSE NOT STATED IN THE RFP. EMVENTIONS ALSO POINTS OUT THAT OTHER EPA PROCUREMENTS HAVE CALLED FOR ASBESTOS "SCREENING" OR "SURVEY" PROCEDURES, AND THAT THE PRESENT RFP DID NOT SPECIFICALLY EXCLUDE SUCH PURPOSES. EMVENTIONS ALLEGES THAT IT WAS PENALIZED IN THE EVALUATION BECAUSE OF THE SUBSTANTIAL ARGUMENTS IT MADE IN FAVOR OF SCREENING AND SURVEY PROCEDURES.

WE FIRST NOTE THAT THE TECHNICAL EVALUATOR'S MEMORANDUM APPARENTLY REFERS TO ANIMAL TOXICITY AS AN EXAMPLE OF AN AREA IN WHICH ELECTRON MISCROSCOPE TECHNIQUES WOULD BE USEFUL, NOT AS THE SOLE PURPOSE OF THE RFP'S REQUESTED WORK EFFORT. ALSO, THE FACT THAT THE RFP MAY NOT HAVE EXPLICITLY EXCLUDED SCREENING OR SURVEY PROCEDURES IS NOT, IN OUR VIEW, A PERSUASIVE INDICATION THAT A PROPOSAL EMPHASIZING THE DEVELOPMENT OF SUCH PROCEDURES SHOULD HAVE BEEN CONSIDERED ACCEPTABLE. WE DO NOT BELIEVE THAT THE PROTESTER HAS SHOWN ANY BASIS FOR A CONCLUSION THAT EPA'S TECHNICAL JUDGMENT IN THESE AREAS WAS UNREASONABLE. ALSO, WE DO NOT SEE ANY BASIS TO CONCLUDE THAT THE EMVENTIONS PROPOSAL WOULD NOT HAVE REQUIRED MAJOR REVISIONS TO MAKE IT ACCEPTABLE. THEREFORE, WE HAVE NO OBJECTION TO EPA'S DECISION TO EXCLUDE THE PROPOSAL FROM THE COMPETITIVE RANGE.

AT THE CONFERENCE ON THE PROTEST EMVENTIONS COMPLAINED OF THE FACT THAT THE TECHNICAL EVALUATION WAS CONDUCTED BY ONLY ONE PERSON. EMVENTIONS SUGGESTED THAT A MULTIMEMBER TECHNICAL PANEL SHOULD HAVE BEEN USED, WHICH IDEALLY WOULD INCLUDE BOTH EPA PERSONNEL AND PRIVATE CONSULTANTS. COMPOSITE EVALUATION PANELS MAY BE DESIRABLE IN HELPING TO INSURE INDEPENDENT JUDGMENT AND FAIRNESS IN A TECHNICAL EVALUATION. CF. 53 COMP. GEN. 253, 257 (1973). HOWEVER, WE ARE UNAWARE OF A LEGAL REQUIREMENT FOR SUCH PROCEDURES UNDER THE CIRCUMSTANCES OF THE PRESENT CASE.

IN VIEW OF THE FOREGOING, EMVENTIONS' PROTEST IS DENIED. AS FOR THE COMMENTS BY SEVERAL OF THE OFFERORS WITHIN THE COMPETITIVE RANGE CONCERNING THE OBJECTIVITY SUBCRITERION, WE SHARE TO SOME EXTENT THEIR CONCERN OVER ITS APPLICATION BY EPA. IN THIS REGARD, THE NARRATIVE PORTIONS OF THE TECHNICAL EVALUATION ARE SOMEWHAT TERSE AND CONCLUSORY ON THIS POINT. ON THE OTHER HAND, WE DO NOT BELIEVE THE PARTIES' COMMENTS HAVE ESTABLISHED A BASIS FOR OUR OFFICE TO OBJECT TO THE EVALUATION. THIS REGARD, WE UNDERSTAND THAT EPA RAISED QUESTIONS WITH THE OFFERORS CONCERNING OBJECTIVITY DURING THE NEGOTIATIONS, AND THAT THE AGENCY HAS BEEN EVALUATING THE BEST AND FINAL OFFERS DURING THE PENDENCY OF THE PROTEST. ALSO, EPA HAS BECOME AWARE OF THE PARTIES' WRITTEN COMMENTS ON THIS MATTER AND OF THEIR VIEWS EXPRESSED ORALLY AT THE CONFERENCE HELD ON THE PROTEST. THE CONFERENCE WAS ATTENDED BY THE EPA TECHNICAL EVALUATOR AND OTHER EPA PERSONNEL. THUS, WE BELIEVE THAT EPA IN MAKING ITS FINAL EVALUATION PREPARATORY TO SOURCE SELECTION HAS HAD THE BENEFIT OF A BROADER PERSPECTIVE ON THIS ISSUE THAN WAS AVAILABLE AT THE TIME OF THE INITIAL EVALUATION.

GAO Contacts

Office of Public Affairs