B-182742, JUL 9, 1975

B-182742, JUL 9, 1975

1. ADMINISTRATIVE DETERMINATION THAT REVISED TECHNICAL PROPOSAL, INITIALLY FOUND POTENTIALLY ACCEPTABLE BY CONTRACTING OFFICER CONTRARY TO EVALUATION PANEL'S RECOMMENDATION THAT PROPOSAL WAS UNACCEPTABLE, WAS NOT IN COMPETITIVE RANGE WILL NOT BE QUESTIONED SINCE DISCRETION WAS EXERCISED IN REASONABLE MANNER. 2. PROPOSED COST OF TECHNICALLY UNACCEPTABLE PROPOSAL NEED NOT BE CONSIDERED SINCE TECHNICAL ACCEPTABILITY IS PREREQUISITE TO COST CONSIDERATION.

PACIFIC TRAINING & TECHNICAL ASSISTANCE CORPORATION:

PACIFIC TRAINING & TECHNICAL ASSISTANCE CORPORATION (PACIFIC) IS CONTESTING THE DETERMINATION OF THE DEPARTMENT OF HEALTH, EDUCATION, AND WELFARE, OFFICE OF CHILD DEVELOPMENT (OCD), THAT PACIFIC'S REVISED PROPOSAL, SUBMITTED IN RESPONSE TO REQUEST FOR PROPOSALS (RFP) RFP-4-75 HEW-OS, WAS OUTSIDE THE COMPETITIVE RANGE. THE RFP ENVISIONED PERFORMANCE OF A PROCESS EVALUATION OF PROJECT DEVELOPMENT CONTINUITY. THE TERM WAS FOR 1 YEAR WITH TWO 1-YEAR OPTIONS.

SEVEN PROPOSALS WERE RECEIVED PRIOR TO THE CLOSING DATE FOR RECEIPT OF INITIAL PROPOSALS. AN EVALUATION OF PROPOSALS CONDUCTED BY A PANEL OF OCD PERSONNEL RECOMMENDED TO THE CONTRACTING OFFICER THAT ONLY TWO FIRMS HAD SUBMITTED PROPOSALS THAT WERE SUSCEPTIBLE OF BEING MADE ACCEPTABLE BY FURTHER NEGOTIATIONS. THE TWO FIRMS WERE HIGH/SCOPE EDUCATIONAL RESEARCH FOUNDATION (HIGH/SCOPE), TECHNICALLY RATED 89 OUT OF A POTENTIAL 100 POINTS, AND ARTHUR YOUNG & CO., RATED 68 TECHNICALLY. AFTER REVIEWING THE RECOMMENDATION, THE CONTRACTING OFFICER DETERMINED THAT THREE OF THE FIRMS THAT THE OCD PANEL HAD DEEMED TECHNICALLY UNACCEPTABLE WERE ACTUALLY SUSCEPTIBLE OF BEING MADE ACCEPTABLE. THE THREE FIRMS DETERMINED BY THE CONTRACTING OFFICER TO BE IN THE COMPETITIVE RANGE FOR NEGOTIATION PURPOSES WERE: PACIFIC AT 47 POINTS; JWK INTERNATIONAL AT 46 POINTS; AND UNIVERSITY RESEARCH AT 39 POINTS.

DISCUSSIONS WERE CONDUCTED WITH ALL FIRMS IN THE COMPETITIVE RANGE. SPECIFIC WRITTEN QUESTIONS CONCERNING EACH PROPOSAL'S WEAKNESSES AND DEFICIENCIES WERE DISSEMINATED TO EACH FIRM AT THE RESPECTIVE DISCUSSIONS. WHILE ORAL RESPONSES TO SOME OF THE QUESTIONS WERE REQUESTED, THE OFFERORS WERE AFFORDED APPROXIMATELY 5 DAYS TO SUBMIT REVISED PROPOSALS. REVISED PROPOSALS WERE TIMELY RECEIVED FROM ALL FIVE FIRMS. UPON REEXAMINATION OF THE REVISED PROPOSALS, THE OCD PANEL DETERMINED THAT ONLY HIGH/SCOPE HAD SUBMITTED A TECHNICALLY ACCEPTABLE PROPOSAL AND RATED IT 96. THE TECHNICAL SCORES OF THE OTHER FIRMS, IN ORDER, WERE: ARTHUR YOUNG & CO. - 46; PACIFIC - 36; JWK INTERNATIONAL 33; UNIVERSITY RESEARCH - 29. SINCE IT WAS DETERMINED THAT HIGH/SCOPE HAD SUBMITTED THE ONLY TECHNICALLY ACCEPTABLE PROPOSAL, CONTRACT NO. HEW-100- 75-0037 WAS AWARDED TO IT AT $408,740 FOR THE FIRST YEAR; $598,943 FOR THE FIRST OPTION; AND $595,242 FOR THE SECOND OPTION YEAR.

ESSENTIALLY, PACIFIC DISPUTES THE OBJECTIVITY AND COMPETENCE OF THE TECHNICAL EVALUATION PANEL THAT DETERMINED ITS PROPOSAL WAS TECHNICALLY UNACCEPTABLE. IN ITS SUBMISSIONS, PACIFIC HAS OUTLINED THE VARIOUS STEPS THAT OCCURRED FROM SUBMISSION OF THE ORIGINAL PROPOSALS THROUGH CONTRACT AWARD. IN MANY INSTANCES, THE GIST OF PACIFIC'S ARGUMENT PERTAINS TO THE FIRST EVALUATION. HOWEVER, PACIFIC WAS INCLUDED IN THE COMPETITIVE RANGE AND WAS AFFORDED AN OPPORTUNITY TO DISCUSS ITS PROPOSAL. THIS ACTION ON THE PART OF THE CONTRACTING OFFICER WAS REASONABLE AND IN ACCORDANCE WITH FEDERAL PROCUREMENT REGULATIONS SEC. 3-805(A) (1974). IN THIS LIGHT, WE DO NOT THINK A DETAILED DISCUSSION OF THE INITIAL EVALUATION GERMANE TO THE PROTEST. THEREFORE, WE SHALL CONFINE OUR CONSIDERATION TO THAT PART OF THE PROTEST THAT CONCERNS THE DISCUSSIONS AND EVALUATION OF PACIFIC'S BEST AND FINAL OFFER.

INITIALLY, IT MUST BE RECOGNIZED THAT PACIFIC BEARS A HEAVY BURDEN IN THIS TYPE OF CASE. THE GENERAL RULE IS THAT THE DETERMINATION WHETHER A PROPOSAL IS TECHNICALLY ACCEPTABLE AND IN THE COMPETITIVE RANGE IS A MATTER OF ADMINISTRATIVE DISCRETION WHICH WILL NOT BE DISTURBED ABSENT A CLEAR SHOWING THAT THE DETERMINATION WAS ARBITRARY OR UNREASONABLE. RIGGINS AND WILLIAMSON MACHINE COMPANY, INCORPORATED, B-182801, MARCH 21, 1975; MEI-CHARLTON, INC., B-179793, FEBRUARY 26, 1974; MEI CHARLTON, INC., B-179165, FEBRUARY 11, 1974; 52 COMP. GEN. 382 (1972). MOREOVER, WE HAVE UPHELD ADMINISTRATIVE DETERMINATIONS TO EXCLUDE FIRMS INITIALLY DETERMINED TO BE WITHIN THE COMPETITIVE RANGE FROM FURTHER AWARD CONSIDERATION AFTER THEIR REVISED PROPOSALS WERE FOUND TO BE TECHNICALLY UNACCEPTABLE AND NO LONGER WITHIN THE COMPETITIVE RANGE. PIONEER PARACHUTE CO., INC., B- 179263, APRIL 17, 1974; 52 COMP. GEN. 198 (1972).

PACIFIC SEEKS TO DEMONSTRATE THAT THE EVALUATION OF ITS PROPOSAL, VIS A- VIS HIGH/SCOPE'S, EVIDENCES DISPARATE TREATMENT. FURTHER, PACIFIC CONTENDS THAT THE EVALUATORS DID NOT COMPREHEND THE ADVANTAGES OF THE APPROACHES PROPOSED BY PACIFIC, AND IN MANY INSTANCES DISPUTE THE OCD CONCLUSIONS AS TOTALLY WITHOUT FOUNDATION IN FACT. PACIFIC HAS CATEGORIZED THE PROTEST INTO THREE BASIC ISSUES - (1) VALIDITY OF THE PROPOSED ON-SITE EVALUATOR RESEARCH STRATEGY; (2) APPROPRIATE NUMBER OF CORE STAFF; AND (3) POSSESSION OF ESSENTIAL EXPERIENCE AND STAFF BACKGROUND.

PACIFIC VIEWS ITS PROPOSED ON-SITE EVALUATOR RESEARCH STRATEGY AS AN INNOVATIVE APPROACH THAT IS COST EFFECTIVE BY ELIMINATING SUBSTANTIAL TRAVEL COSTS. PACIFIC REQUESTS OUR OFFICE TO REVIEW AND EVALUATE ITS RESPONSE TO JUSTIFY ITS APPROACH, IN TERMS OF ALLOCATION, NUMBER AND QUALIFICATION OF PROPOSED PERSONNEL. FURTHER, PACIFIC REQUESTS THAT WE COMPARE ITS APPROACH WITH THAT OF HIGH/SCOPE TO DETERMINE ITS QUALITY. THIS VEIN, IN REBUTTAL TO OCD'S DETERMINATION THAT PACIFIC'S PROPOSAL EVIDENCED UNDERSTAFFING, PACIFIC COUNTERS THAT HIGH/SCOPE WAS OVERSTAFFED. IN RESPONSE TO OCD'S QUESTION CONCERNING A LACK OF GROUND TRAVEL ALLOWANCES FOR THE PROPOSED ON-SITE EVALUATORS, PACIFIC STATES THAT ADDITIONAL TRAVEL EXPENSE IS UNNECESSARY IN VIEW OF THE SELECTION OF EVALUATORS IN CLOSE PROXIMITY TO THE SITES.

PACIFIC HAS SUBMITTED RESUMES OF ITS PROPOSED KEY PERSONNEL TO DEMONSTRATE THEIR QUALIFICATIONS. ONE OF OCD'S OBJECTIONS IN THIS REGARD CONCERNS THE THREE CORE STAFF INDIVIDUALS AND THE PROJECT DIRECTOR WHOSE EXPERIENCE IS CHARACTERIZED AS "LACKING A BROAD RANGE" WITH NO PRIOR EXPERIENCE IN "HANDLING PROJECTS OF THIS MAGNITUDE." THEIR EXPERIENCE WAS SUMMARIZED AS LACKING IN:

"CAPABILITIES IN MANAGEMENT RESEARCH DESIGN, OR TESTS AND MEASUREMENTS *** THREE PEOPLE CANNOT INSURE QUALITY CONTROL, DO ALL DATA ANALYSIS, AND NECESSARY FIELD WORK. FURTHER, THERE IS NO EVIDENCE THAT ANY OF THE PROPOSED STAFF HAVE DONE LARGE SCALE STUDIES IN EARLY CHILDHOOD."

WE HAVE REVIEWED THE CURRICULUM VITAE OF THE PROPOSED INDIVIDUALS AND WHILE THEIR CREDENTIALS ARE SATISFACTORY, WE CANNOT STATE THAT OCD'S COMMENTS WERE WHOLLY UNREASONABLE. CF. ADAMS ASSOCIATES, B-182469, JANUARY 6, 1975.

THE CRUX OF THE DISAGREEMENT BETWEEN PACIFIC AND OCD IS THE VALIDITY AND DESIRABILITY OF USING ON-SITE EVALUATORS RATHER THAN VISITING FIELD MONITORS. THIS, IN TURN, IMPACTS UPON THE NUMBER OF PERSONNEL AND THEIR QUALIFICATIONS. IT IS THIS PROBLEM THAT PROMPTED OCD'S OBSERVATION THAT THE CORE STAFF IS OVERBURDENED. ONE OF THE QUESTIONS IN THIS REGARD IS THE NEED FOR BILINGUAL-BICULTURAL CORE STAFF. PACIFIC ALLEGES THAT IT PROPOSED THE NECESSARY STAFFING AND ANY CRITICISM IN THIS REGARD COULD ONLY RESULT FROM A FAILURE TO READ ITS PROPOSAL. OCD RESPONDED THAT PACIFIC DID PROPOSE ON-SITE EVALUATORS AND TESTER-OBSERVERS WITH THE REQUISITE BILINGUAL-BICULTURAL QUALITIES. OCD'S RESERVATIONS WERE THAT THE CORE STAFF DID NOT POSSESS THESE CHARACTERISTICS. IT IS THE CORE STAFF THAT WAS RESPONSIBLE FOR THE BASE WORK, OR DESIGN AND INSTRUMENT SELECTION PHASE. HOWEVER, THE EVALUATION REPORT DID RECOGNIZE THAT PACIFIC HAD SATISFACTORILY RESPONDED TO ITS QUESTION ON THIS MATTER NOTWITHSTANDING THE STATEMENT ON THE CONTRACTING OFFICER'S REPORT THAT THE CORE STAFF DID NOT PRESENT THE REQUISITE BILINGUAL-BICULTURAL ETHNIC BACKGROUND.

OCD'S EVALUATION WAS BASED UPON PACIFIC'S RESPONSES TO THE APPROXIMATELY 40 QUESTIONS PRESENTED IT IN CONJUNCTION WITH THE PROPOSAL. OCD'S ASSESSMENT OF 18 RESPONSES WAS THAT THEY WERE UNACCEPTABLE. PACIFIC HAS ALLEGED THESE QUESTIONS AND THE ORAL DISCUSSIONS WERE NOT MEANINGFUL BECAUSE OCD'S DISTASTE FOR THE ON-SITE EVALUATOR APPROACH WAS NEVER CLEARLY COMMUNICATED. THE PURPOSE OF HOLDING DISCUSSIONS WITH AN OFFEROR WHOSE PROPOSAL HAS BEEN RATED TECHNICALLY SUSCEPTIBLE OF BEING MADE ACCEPTABLE IS TO EXPLORE AVENUES OF UPGRADING THE PROPOSAL WITHOUT CAUSING MAJOR REVISIONS OR WHOLESALE REWRITING OF THE PROPOSAL. IN THIS INSTANCE, OCD COULD NOT HAVE EFFECTED A CHANGE IN PACIFIC'S PROPOSED USE OF THE ON- SITE EVALUATOR APPROACH WITHOUT CAUSING A MAJOR REWRITE TO THE PROPOSAL. TO DO SO WOULD HAVE BEEN BEYOND THE SCOPE OF THE PURPOSE OF THE DISCUSSIONS. WE NOTE THAT ONE OF THE MAJOR PRECEPTS OF PACIFIC'S PROTEST IS STILL THAT THE USE OF ON-SITE EVALUATORS SHOULD HAVE BEEN ACCEPTABLE TO OCD. FURTHER, VARIOUS ASPECTS OF THE ON-SITE EVALUATION CONCEPT WERE QUESTIONED. WHILE OCD DID NOT APPEAR TO HAVE STATED DURING THE NEGOTIATIONS THAT THE ON-SITE METHOD WAS UNACCEPTABLE, THE INFERENCE COULD HAVE BEEN DRAWN FROM THE PERIPHERAL AREAS OF STAFFING AND METHODOLOGY QUESTION.

ON THE SUBJECT OF PROPOSED STAFF OCD STATED:

"THE OFFEROR HAS NOT ALTERED THE STAFF ORIGINALLY PROPOSED. IT IS TOO SMALL AND TOO BURDENED. THE REDUCTION OF THE NUMBER OF TESTERS ALREADY JUDGED TOO SMALL WILL CAUSE TESTING TO EXTEND OVER TOO LONG A PERIOD LEADING TO NON-COMPARABLE TEST DATA. PT&TA PROPOSES THAT 8 OF THE ON-SITE EVALUATORS BE CONSULTANTS. PAGE 14, PARA. F OF ATTACHMENT A OF THE RFP SPECIFICALLY PRECLUDES CONSULTANTS FROM ANY STAFF ACTIVITY. CONSULTANTS ARE TO BE ADVISORY ONLY."

ON THE MATTER OF CORE STUDIES, OCD OBSERVED:

"*** THE OFFEROR PROPOSES TO PILOT TEST IN FOUR SITES. SINCE THESE TRIPS WERE NOT ACCOUNTED FOR IN THE BUDGET, THE PILOT TEST WILL BE DONE BY THE ON-SITE EVALUATORS. THIS IS NOT ACCEPTABLE. PILOT TESTING MUST BE DONE BY THE DESIGNERS OF THE INSTRUMENTS FOR RELIABILITY AND VALID FEEDBACK TO MAKE THE NECESSARY CHANGES IN THE INSTRUMENT.

"THE USE OF ON-SITE EVALUATORS, AS PROPOSED BY THE OFFEROR, WITH INADEQUATE STAFF MONITORING IS UNACCEPTABLE. THERE ARE NO BUILT IN CHECKS TO ASSURE THE RELIABILITY. VALIDITY OR COMPARABILITY OF DATA ACROSS SITES. IN DEPTH INTERVIEWS WITH ALL THE ACTORS CANNOT BE CONDUCTED IN TWO DAY VISITS ***

"A SECOND POINT IS THAT THE CORE STAFF WILL NOT HAVE ENOUGH TIME AVAILABLE TO WRITE UP ALL THE CASE STUDIES AND DO ALL THE REQUIRED ANALYSES.

"A THIRD POINT IS THAT THE REVIEW BY THE ON-SITE EVALUATOR OF THE CASE STUDY DRAFT IS NOT SUFFICIENT. THE PROJECT STAFF MUST REVIEW THE DRAFT FOR FACTUAL ERRORS. THIS HAS NOT BEEN PROVIDED FOR."

RELATIVE TO THE INFORMATION SUBMITTED IN THE TYPES OF SKILLS AND TRAINING OF THE ON-SITE EVALUATORS, OCD CONCLUDED:

"THE OFFERORS RESPONSE IS A GENERAL DISCUSSION OF SKILLS AND DOES NOT GET INTO CRITERIA FOR SELECTION OF ON-SITE EVALUATORS. THE NARRATIVE SUPPORTS THE POSSIBILITY OF AN ALTERNATIVE SELECTION TO THE PERSON ALREADY IDENTIFIED, HOWEVER THE BUDGET IDENTIFIES 7 PEOPLE AS PT AND TA STAFF AND 8 AS CONSULTANTS. THIS IDENTIFICATION IN THE BUDGET LOCKS IN A RIGIDITY NOT REFLECTED IN THE NARRATIVE.

IT APPEARS THAT A SERIES OF 'TESTS' WILL TAKE PLACE FOR THE ON-SITE EVALUATORS WITH THE POSSIBILITY OF SOME SHIFTING. ***

"*** NO STAFF JOBS CAN BE FILLED BY CONSULTANTS. THE ON-SITE EVALUATOR IS A CRITICAL STAFF JOB, NOT AN ADVISORY FUNCTION."

FROM OUR REVIEW OF THE OCD EVALUATION, WE CONCLUDE THAT THE EVALUATION REPRESENTED A REASONABLE EXERCISE OF PROCUREMENT JUDGMENT AND DISCRETION.

PACIFIC ALSO STRESSES THE FACT THAT ITS PROPOSAL WAS SUBMITTED ON A LOWER FIRM, FIXED-PRICE BASIS THAN HIGH/SCOPE'S COST-PLUS CONTRACT. HOWEVER, THE QUESTION OF COST MAY NOT BE CONSIDERED IN DETERMINING WHETHER AN OFFER IS WITHIN THE COMPETITIVE RANGE UNLESS THE OFFER IS FIRST DEEMED TECHNICALLY ACCEPTABLE. 52 COMP. GEN. 382 (1972); 49 COMP. GEN. 309 (1969); THE BLK GROUP, INC., B-178887, APRIL 10, 1974.

FINALLY, IN HIS REPORT THE DEPUTY ASSISTANT SECRETARY FOR GRANTS AND PROCUREMENT MANAGEMENT RAISED A DEFICIENCY IN THE EVALUATION PLAN. IT WAS NOTED THAT EACH PROPOSAL WAS NOT EVALUATED BY THE SAME NUMBER OF EVALUATORS. PACIFIC'S INITIAL PROPOSAL WAS EVALUATED BY ALL EIGHT EVALUATORS ASSIGNED THE PROJECT, BUT ONLY SIX EVALUATED ITS BEST AND FINAL OFFER. HIGH/SCOPE WAS INITIALLY EVALUATED BY ALL EIGHT EVALUATORS AND HAD ONLY FOUR EVALUATORS CONSIDER ITS BEST AND FINAL OFFER. BASED UPON OUR REVIEW, WE CANNOT DISPUTE OCD'S REPRESENTATION THAT THE SCORING WAS RELATIVELY CONSISTANT BY ALL EVALUATORS AND THAT FURTHER EVALUATION WOULD NOT HAVE CHANGED THE SCORING. WE NOTE WITH APPROVAL IN THIS REGARD ADVICE FROM THE AGENCY THAT IT WILL APPRISE ITS PROCUREMENT PERSONNEL OF THE NECESSITY OF HAVING EVALUATORS PARTICIPATE IN THE PROPOSAL EVALUATION PROCESSES ON A MORE UNIFORM BASIS.

THEREFORE, THE PROTEST IS DENIED.