B-224892.2, MAR 3, 1987, 87-1 CPD 239

B-224892.2, MAR 3, 1987, 87-1 CPD 239

PROCUREMENT - COMPETITIVE NEGOTIATION - OFFERS - EVALUATION - TECHNICAL ACCEPTABILITY DIGEST: 1. AGENCY'S LOW RATING OF A TECHNICAL PROPOSAL FOR A DEVELOPMENTAL COMPUTER PROGRAM IS REASONABLE WHEN THE PROPOSAL RELIES ON EQUATIONS THAT THE AGENCY CONSIDERS VERY OLD; THE EQUATIONS DO NOT ACCOUNT FOR NUMEROUS VARIABLES OR PRODUCE ALL THE RESULTS REQUIRED BY THE SOLICITATION; AND THE AGENCY CONSIDERS THE RISKS INVOLVED IN PROPOSED MODIFICATIONS TO THE EQUATIONS TO BE UNACCEPTABLE. PROCUREMENT - COMPETITIVE NEGOTIATION - DISCUSSION - MISLEADING INFORMATION - ALLEGATION SUBSTANTIATION 2. AGENCY'S ALLEGEDLY MISLEADING ADVICE THAT PROTESTER SHOULD INCREASE CERTAIN PROPOSED COSTS TO COVER VERIFICATION OF A DEVELOPMENTAL COMPUTER PROGRAM DOES NOT NECESSARILY INDICATE UNEQUAL COMPETITION WHEN THE EXTENT AND TYPE OF VERIFICATION REQUIRED DEPENDS UPON THE PROGRAM OFFERED AND THE TESTING TO WHICH IT HAS PREVIOUSLY BEEN SUBJECTED. PROCUREMENT - BID PROTESTS - NON-PREJUDICIAL ALLEGATION - GAO REVIEW 3. EVEN IF SOLICITATION AND DISCUSSIONS COULD HAVE BEEN MORE SPECIFIC AS TO VERIFICATION REQUIREMENTS FOR A DEVELOPMENTAL COMPUTER PROGRAM, WHEN THE PROTESTER IS THE LOWEST-RANKED OF SIX OFFERORS AND ITS TECHNICAL SCORE IS 48.5 POINTS LESS THAN THE AWARDEE'S, THE PROTESTER IS NOT PREJUDICED BY THE ALLEGED DEFICIENCIES, SINCE IT HAD NO REASONABLE CHANCE FOR AWARD.

JG ENGINEERING RESEARCH ASSOCIATES:

JG ENGINEERING RESEARCH ASSOCIATES PROTESTS THE EVALUATION OF ITS PROPOSAL FOR THE DEVELOPMENT OF A COMPUTER PROGRAM TO BE USED BY THE U.S. ARMY TANK AUTOMOTIVE COMMAND, WARREN, MICHIGAN. THE ACTIVITY ISSUED REQUEST FOR PROPOSALS (RFP) NO. DAAE07-86-R-R038 ON FEBRUARY 26, 1986, AND ON SEPTEMBER 5 AWARDED A $196,934 COST-PLUS-FIXED-FEE CONTRACT TO THE UNIVERSITY OF DENVER'S DENVER RESEARCH INSTITUTE.

THE PROTESTER DISAGREES WITH THE AGENCY'S LOW RATING OF ITS TECHNICAL PROPOSAL AND CONTENDS THAT THE ARMY IMPROPERLY ADVISED IT TO INCREASE CERTAIN ELEMENTS OF ITS COST PROPOSAL TO COVER VERIFICATION AND TESTING OF ITS PROGRAM. THE FIRM CONTENDS THAT COMPETITION WAS UNEQUAL BECAUSE, WITHOUT A FORMAL WRITTEN AMENDMENT TO THE RFP, ITS UNDERSTANDING OF THE TESTING REQUIREMENT DIFFERED FROM THE AGENCY'S. WE DENY THE PROTEST.

BACKGROUND

THE RFP SOUGHT A CONTRACTOR TO DEVISE, DEVELOP, AND ASSEMBLE A "COMPOSITE (MULTI-LAYER) ARMOR MODEL" COMPUTER CODE THAT COULD CALCULATE GRAPHICALLY DISPLAY THE EFFECT OF VARIOUS TYPES OF SMALL ARMS ON DIFFERENT ARMOR SYSTEMS. THE RFP INDICATED THAT THE ARMY'S CURRENT MODELS CAN ANALYZE THE EFFECT OF A LIMITED RANGE OF MATERIALS AND THREATS ON A SINGLE LAYER OF ARMOR. FUTURE VEHICLE CONCEPTS AND DESIGNS, HOWEVER, WILL REQUIRE MULTI- LAYER ARMOR PLATES OF VARIOUS MATERIALS AND THICKNESSES, SEPARATED BY AIR AND "EXPOSED TO A WIDE RANGE OF THREATS." THE MODEL TO BE DEVELOPED BY THE CONTRACTOR WILL BE USED AS A DESIGN AND EVALUATION TOOL FOR SUCH FUTURE VEHICLES.

THE RFP LISTED TWO MAJOR AREAS OF EVALUATION: TECHNICAL AND COST. THE TECHNICAL CRITERIA, WHICH WERE SIGNIFICANTLY MORE IMPORTANT THAN COST, INCLUDED (1) PLAN ELEMENT; (2) USABILITY OF RESULTS; (3) EXPERIENCE; AND (4) PROBLEM STATEMENT. THE COST CRITERIA WERE REALISM AND COMPLETENESS. AWARD WAS TO BE MADE TO THE OFFEROR SUBMITTING THE BEST TECHNICAL PROPOSAL AT AN AFFORDABLE COST.

THE AGENCY RECEIVED SIX PROPOSALS BY THE APRIL 14 OPENING DATE. IN THE INITIAL EVALUATION, IT GAVE DENVER RESEARCH INSTITUTE THE HIGHEST WEIGHTED TECHNICAL SCORE, 95, AND JG THE LOWEST, 39; THE SCORES OF THE REMAINING OFFERORS RANGED FROM 59.5 TO 85. PROPOSED COSTS RANGED FROM JG'S LOW OF $94,500 TO A HIGH OF $327,332. ALL OFFERORS RECEIVED ADEQUATE COST RATINGS EXCEPT JG, WHICH RECEIVED AN INADEQUATE RATING.

DURING DISCUSSIONS, THE ARMY ADVISED JG, AMONG OTHER THINGS, THAT IT WOULD NEED TO ADD TO ITS COST PROPOSAL. SPECIFICALLY, THE AGENCY RECOMMENDED AN OVERALL INCREASE OF 25 PERCENT IN DIRECT LABOR HOURS, PLUS $15,000 FOR MATERIAL AND EQUIPMENT AND 300 HOURS FOR A TESTING PHASE TO VERIFY ITS COMPUTER CODE DEVELOPMENT. IN ITS BEST AND FINAL OFFER, JG OFFERED TO DEVELOP AND MONITOR A BALLISTICS TESTING PROGRAM TO BE CARRIED OUT BY THE ARMY FOR AN ADDITION $65,978, RESULTING IN A TOTAL PROPOSED COST PLUS FEE OF $160,478.

AFTER EVALUATION OF BEST AND FINAL OFFERS, THE AGENCY INCREASED JG'S TECHNICAL SCORE TO 46.5 AND ITS COST RATING TO ADEQUATE. DENVER RESEARCH INSTITUTE'S TECHNICAL SCORE REMAINED 95, AND ITS BEST AND FINAL OFFER AND THE RESULTING CONTRACT PRICE, AS NOTED ABOVE, WAS $196,934.

JG'S TECHNICAL PROPOSAL

JG'S PROTEST CONSISTS FIRST OF RESPONSES TO THE ARMY'S CRITICISMS OF ITS TECHNICAL PROPOSAL, MADE DURING ITS OCTOBER 9 DEBRIEFING. THE PRIMARY AREA OF DISAGREEMENT IS JG'S RELIANCE UPON CERTAIN EQUATIONS, DESIGNATED "THOR." AS DISCUSSED IN JG'S PROPOSAL, THOR EQUATIONS ARE BASED ON EXPERIMENTS (RATHER THAN THEORETICAL ANALYSIS) CONDUCTED FOR THE ARMY OVER A PERIOD OF YEARS. THE EQUATIONS PROVIDE VALUES FOR THE RESIDUAL VELOCITY AND RESIDUAL MASS OF STEEL FRAGMENTS AFTER THEY PERFORATE METALLIC AND NONMETALLIC MATERIALS. IN ITS PROPOSAL, JG RECOGNIZED THAT ONE OF THE DRAWBACKS OF THESE "PURELY EXPERIMENTAL" EQUATIONS IS THAT RESULTS CANNOT BE EXTRAPOLATED FOR MATERIALS AND CONFIGURATIONS THAT HAVE NOT BEEN TESTED. HOWEVER, THE PROPOSAL STATED, IN 1976 JG HAD DEVELOPED A METHOD TO RECTIFY THIS DRAWBACK THAT IT PROPOSED TO USE TO DEVELOP THE COMPOSITE ARMOR MODEL.

IN ITS EVALUATION, THE ARMY CRITICIZED JG'S RELIANCE ON THE THOR EQUATIONS, DESCRIBING THEM AS VERY OLD AND STATING THAT JG'S FAILURE TO REALIZE THAT USE OF THE THOR EQUATIONS ALONE WAS NOT SUFFICIENT WAS EVIDENCE OF ITS FAILURE TO UNDERSTAND THE PROBLEMS PRESENTED BY THE CONTRACT. THE PROTESTER, HOWEVER, MAINTAINS THAT THERE IS NO OTHER "SET OF EXPERIMENTS FOR RESIDUAL MASS WHICH IS AS EXTENSIVE, ACCURATE, AND RELIABLE AS THE THOR EXPERIMENTS."

BASED ON OUR REVIEW OF THE PROPOSAL AND THE EVALUATION RECORD, WE BELIEVE THAT THE AGENCY'S LOW RATING OF JG'S TECHNICAL PROPOSAL WAS REASONABLE. THE AGENCY LACKED CONFIDENCE THAT THE PROTESTER COULD IN FACT ADAPT THE THOR EQUATIONS TO MEET ITS NEEDS AND QUESTIONED WHETHER THIS COULD BE DONE WITHOUT FURTHER TESTING, AS THE PROTESTER INITIALLY PROPOSED. (IN CONTRAST, THE AWARDEE'S APPROACH WAS TO RELY ON A NUMBER OF EXISTING BALLISTICS PENETRATION MODELS AND TO DEVELOP OTHER MODELS DURING THE CONTRACT.)

FOR EXAMPLE, AS NOTED ABOVE, JG STATED IN ITS PROPOSAL THAT THE THOR EQUATIONS PROVIDE VALUES FOR RESIDUAL VELOCITY AND RESIDUAL MASS FOLLOWING PENETRATION OF A SINGLE ARMOR PLATE. AS STATED IN THE RFP, HOWEVER, THE COMPUTER PROGRAM TO BE DEVELOPED FOR TACOM MUST BE APPLICABLE TO MUTLI- LAYER ARMOR PLATES OF DIFFERENT MATERIALS AND THICKNESSES, WITH DIFFERENT AMOUNTS OF SPACE BETWEEN THEM. IN ADDITION, THE PROGRAM MUST ALLOW FOR PROJECTILES OF DIFFERENT SIZES AND TYPES, PERFORATING (OR NOT PERFORATING) AN ARMOR SYSTEM AT DIFFERENT ANGLES AND SPEEDS. THE PROGRAM MUST CALCULATE AND GRAPHICALLY DISPLAY THE PATH OF THE PROJECTILE, AND IT MUST NOT ONLY PROVIDE VALUES FOR RESIDUAL VELOCITY AND RESIDUAL MASS, BUT ALSO SHOW THE ANGLE OF EXIT AND THE ANGLE OF PENETRATION FOR ARMOR LAYERS AFTER THE FIRST. IN THE ARMY'S JUDGMENT, THE THOR EQUATIONS DO NOT ACCOUNT FOR THESE VARIABLES OR PRODUCE ALL THE RESULTS REQUIRED BY THE RFP, AND IT APPARENTLY CONSIDERS THE RISKS INVOLVED IN MODIFYING THE EQUATIONS UNACCEPTABLE.

IN ADDITION, THE ARMY STATES THAT CERTAIN OF JG'S ASSUMPTIONS, AS SET FORTH IN ITS PROPOSAL, ARE FAULTY. FOR EXAMPLE, THE PROPOSAL ASSUMES THAT THERE IS NO CHANGE IN A PROJECTILE'S ANGLE AFTER PENETRATION, I.E., THAT THE ANGLE OF INCIDENCE ON A SECOND LAYER OF ARMOR IS EQUAL TO THE ORIGINAL ANGLE. THE PROPOSAL REFERENCES A DECEMBER 1969 REPORT PREPARED BY THE DENVER RESEARCH INSTITUTE, AND, IN ITS PROTEST, JG ASKS WHETHER THE INSTITUTE HAS CHANGED ITS MIND IN THIS REGARD. IN ITS ADMINISTRATIVE REPORT, THE ARMY RESPONDS THAT THE INSTITUTE AND "ALMOST EVERYBODY ELSE" NOW UNDERSTANDS THAT THERE CAN BE A CHANGE IN A PROJECTILE'S DIRECTION AND EXIT ANGLE. JG DID NOT ATTEMPT TO REBUT THIS ASPECT OF THE ARMY'S EVALUATION, BUT RATHER STATED IN ITS COMMENTS THAT ITS PROTEST WAS BASED PRIMARILY ON THE ARMY'S ALLEGEDLY MISLEADING INSTRUCTIONS REGARDING ITS COST PROPOSAL.

IN VIEW OF THE ABOVE, WE CANNOT CONCLUDE THAT THE ARMY'S EVALUATION OF JG'S TECHNICAL PROPOSAL WAS UNREASONABLE. WE DENY THE PROTEST ON THIS BASIS.

JG'S COST PROPOSAL

REMAINING AT ISSUE IS THE PROTESTER'S ALLEGATION THAT IT WAS NOT ADEQUATELY INFORMED OF THE AGENCY'S REQUIREMENT CONCERNING THE VERIFICATION DURING DISCUSSIONS, RESULTING IN UNEQUAL COMPETITION.

TO THE EXTENT THAT JG IS ARGUING THAT THIS WAS AN ENTIRELY NEW REQUIREMENT THAT SHOULD HAVE BEEN THE SUBJECT OF AN AMENDMENT TO THE RFP, WE FIND THE PROTEST UNTIMELY. JG SHOULD HAVE PROTESTED ON THIS BASIS WITHIN 10 DAYS OF BEING ORALLY ADVISED THAT IT SHOULD INCREASE ITS PROPOSED COSTS FOR VERIFICATION AND TESTING, AND IN ANY EVENT NO LATER THAN THE DUE DATE FOR BEST AND FINAL OFFERS, 4 C.F.R. SEC. 21.2 (1986), RATHER THAN WAITING UNTIL AFTER ITS DEBRIEFING.

TO THE EXTENT THAT JG MAY NOT HAVE BEEN AWARE UNTIL THE DEBRIEFING THAT ITS OWN AND THE AGENCY'S INTERPRETATION OF THE VERIFICATION REQUIREMENT DIFFERED, ALLEGEDLY RESULTING IN UNEQUAL COMPETITION, WE FIND THE PROTEST WITHOUT MERIT. WE NOTE FIRST THAT THE SOLICITATION DID INDICATE THAT SOME SORT OF VERIFICATION WAS REQUIRED. THE LIST OF CONTRACT DELIVERABLES ATTACHED TO THE RFP (DD FORM 1664) INCLUDED "ADEQUATE TEST CASES DEMONSTRATING THAT THE PROGRAM IS OPERATIONAL AND FULLY DEBUGGED."

THE AGENCY MAINTAINS THAT WHILE THE RFP HAD NO OTHER SPECIFIC REQUIREMENT FOR COMPUTER CODE VERIFICATION, IT CONSIDERED THE TASK "INHERENT" IN THE SCOPE OF WORK AND NECESSARY FOR SUCCESSFUL COMPLETION OF THE CONTRACT. NOTE THAT THE AWARDEE PROPOSED BALLISTICS TESTING FOR NUMERICAL VERIFICATION OF ITS PROGRAM'S OUTPUT, WHILE OTHER OFFERORS PROPOSED TO USE DATA THAT HAD BEEN VERIFIED THROUGH PREVIOUSLY CONDUCTED BALLISTICS TESTING.

JG'S INITIAL PROPOSAL, HOWEVER DESCRIBED ONLY THOSE TASKS THAT IT PROPOSED TO UNDERTAKE IN REVISING THE THOR EQUATIONS; IT DID NOT INDICATE WHETHER OR HOW IT PLANNED TO VERIFY THE COMPOSITE ARMOR MODEL. MOREOVER, WHEN THE SUBJECT OF VERIFICATION WAS RAISED DURING DISCUSSIONS, THERE IS NO EVIDENCE THAT THE FIRM SOUGHT SPECIFIC INFORMATION AS TO THE EXTENT OR TYPE OF TESTING THAT THE ARMY EXPECTED TO BE COVERED BY AN INCREASE OF $15,000 FOR MATERIALS AND OF 300 LABOR HOURS.

FOLLOWING, THE DISCUSSIONS, JG ADDED A BALLISTICS TEST PHASE TO ITS BEST AND FINAL OFFER, WHICH IT INCREASED TO $160,478. AS INDICATED ABOVE, IT OFFERED TO DEVELOP AND MONITOR TESTS THAT ACTUALLY WOULD BE CONDUCTED BY THE ARMY. IN ITS REPORT ON THE PROTEST, THE AGENCY CRITICIZES THIS APPROACH BECAUSE THE GOVERNMENT WOULD INCUR ADDITIONAL COSTS IF IT RAN THE VERIFICATION PROGRAM. JG RESPONDS THAT HAD IT OFFERED TO PERFORM BALLISTICS TESTS WITHOUT GOVERNMENT INVOLVEMENT, ITS COSTS WOULD HAVE BEEN HIGHER THAN THE AWARDEE'S AND ITS PROPOSAL WOULD HAVE BEEN NONCOMPETITIVE.

A CONCEPT BASIC TO FEDERAL PROCUREMENT IS THAT ALL OFFERORS COMPETE ON AN EQUAL BASIS, PROPOSING TO THE SAME TERMS, CONDITIONS, AND SPECIFICATIONS. SEE MACRO SYSTEMS, INC., B-208540.2, JAN. 24, 1983, 83-1 CPD PARA. 79. HERE, HOWEVER, IT APPEARS THAT THE EXTENT AND TYPE OF VERIFICATION REQUIRED DEPENDED UPON THE TYPE OF COMPUTER PROGRAM OFFERED AND THE TESTING TO WHICH IT HAD PREVIOUSLY BEEN SUBJECTED. IN SEVERAL CASES, EVALUATORS NOTED THAT NO TESTING WAS REQUIRED. THUS, THE ADVICE TO JG THAT IT SHOULD INCREASE CERTAIN OF ITS PROPOSED COSTS TO ACCOMMODATE A VERIFICATION PROGRAM DOES NOT NECESSARILY INDICATE UNEQUAL TREATMENT.

EVEN IF WE VIEW THE RFP AS DEFICIENT OR THE DISCUSSIONS INADEQUATE IN NOT PROVIDING DETAILS OF THE VERIFICATION REQUIREMENT, WE DO NOT FIND ANY PREJUDICE TO THE PROTESTER. THE RECORD INDICATES THAT JG DID NOT HAVE A REASONABLE CHANCE FOR AWARD WITHOUT SUBSTANTIAL AND BASIC CHANGES TO ITS APPROACH. IT WAS THE LOWEST-RANKED OF SIX OFFERORS; THERE WAS A 48.5 POINT DIFFERENCE BETWEEN ITS SCORE AND THAT OF THE AWARDEE, AND TECHNICAL FACTORS WERE SIGNIFICANTLY MORE IMPORTANT THAN COST. IN VIEW OF THESE CIRCUMSTANCES, WE QUESTION WHETHER THE AGENCY SHOULD EVEN HAVE DISCUSSED COSTS WITH JG OR REQUESTED A BEST AND FINAL OFFER FROM THE FIRM.

THE PROTEST IS DENIED.