B-176278, OCT 25, 1972

B-176278, OCT 25, 1972

BID PROTEST - SPECIFICATIONS - BENCHMARKING DECISION DENYING THE PROTEST OF LAMBDA CORPORATION AGAINST AWARD OF ANY CONTRACT UNDER AN RFP ISSUED BY THE FEDERAL RAILROAD ADMINISTRATION FOR THE DEVELOPMENT AND EVALUATION OF A LARGE CAPACITY, FLEXIBLE, COMPUTER- BASED ANALYTICAL MODEL FOR THE EFFICIENT REALTIME ASSIGNMENT AND CONTROL OF EMPTY RAILROAD FREIGHT CARS. THE RESPONSIBILITY FOR DRAFTING SPECIFICATIONS REFLECTING THE MINIMUM NEEDS OF THE GOVERNMENT AND THE FACTUAL DETERMINATION AS TO COMPLIANCE WITH THOSE SPECIFICATIONS IS THE PREROGATIVE OF THE PROCURING AGENCY AND NOT SUBJECT TO LEGAL OBJECTION BY GAO UNLESS CLEARLY ARBITRARY. 49 COMP. GEN. 156, 160 (1969). BENCHMARKING IS APPROPRIATE IN PROCURING COMPUTER HARDWARE TO DETERMINE THE CAPABILITY OF A POTENTIAL CONTRACTOR AND THE SUITABILITY AND COST EFFECTIVENESS OF HIS EQUIPMENT. 51 COMP. GEN. , B- 175317, SEPTEMBER 7, 1972. EVEN THOUGH COMPUTER SOFTWARE IS THE OBJECT OF THIS PROCUREMENT, THE NEED FOR AND APPROPRIATENESS OF BENCHMARKING HAS BEEN ADEQUATELY JUSTIFIED. FURTHERMORE, THERE IS NO EVIDENCE THAT THE REQUIREMENT WAS AN UNFAIR OR UNDUE RESTRICTION ON COMPETITION, OR THAT THERE WAS ANY ATTEMPT TO FAVOR A PARTICULAR OFFEROR.

TO LAMBDA CORPORATION:

WE REFER TO YOUR LETTER OF JUNE 16, 1972, AND SUBSEQUENT CORRESPONDENCE, PROTESTING AGAINST THE AWARD OF ANY CONTRACT UNDER REQUEST FOR PROPOSALS (RFP) NO. DOT-FR-20062, ISSUED BY THE FEDERAL RAILROAD ADMINISTRATION (FRA), DEPARTMENT OF TRANSPORTATION. THE SOLICITATION WAS ISSUED ON APRIL 17, 1972, WITH TWO AMENDMENTS BEING ADDED ON APRIL 26 AND MAY 8, 1972. THE RFP REQUESTED PROPOSALS FOR THE DEVELOPMENT AND EVALUATION OF A LARGE CAPACITY, FLEXIBLE, COMPUTER BASED ANALYTICAL MODEL FOR THE EFFICIENT REALTIME ASSIGNMENT AND CONTROL OF EMPTY RAILROAD FREIGHT CARS. ALTHOUGH YOUR PROTEST WAS MADE TO OUR OFFICE PRIOR TO THE AWARD OF A CONTRACT, THE FEDERAL RAILROAD ADMINISTRATION HAS MADE A DETERMINATION THAT A PROMPT AWARD WOULD BE ADVANTAGEOUS TO THE GOVERNMENT AND PURSUANT THERETO IS PREPARING TO MAKE AN AWARD TO DECISION SYSTEMS ASSOCIATES.

THE RFP, AS AMENDED, REQUIRED RESPONDENTS TO SUBMIT WITH THEIR PROPOSAL A SOLUTION TO A BENCHMARK PROBLEM WHICH WAS ATTACHED TO THE SOLICITATION. THE STATED PURPOSE OF THIS REQUIREMENT WAS TO PROVIDE A QUANTITATIVE BASIS FOR EVALUATION OF TECHNICAL COMPETENCE AND TO ALLOW PROPOSERS TO DEMONSTRATE THE CAPABILITY OF THEIR EXISTING TECHNOLOGY. THE RFP FURTHER STATED THAT RESPONDENTS MAY CONSIDER MODIFICATIONS TO THE PROBLEM THAT COULD FURTHER DEMONSTRATE THEIR PARTICULAR TECHNOLOGY.

BASICALLY, YOUR PROTEST INVOLVES (1) THE MANDATORY REQUIREMENT FOR SUBMISSION OF THE SOLUTION TO A BENCHMARK PROBLEM WITH THE PROPOSAL AND (2) THE APPROPRIATENESS OF THE PARTICULAR BENCHMARK REQUIRED. WITH REGARD TO THE FIRST POINT, YOU CONTEND THAT THE BENCHMARK REQUIREMENT IS BOTH UNFAIR TO POTENTIAL OFFERORS AND UNNECESSARY TO THE EVALUATION OF PROPOSALS. YOU POINT OUT THAT UNDER THE RFP AS ORIGINALLY ISSUED SUBMISSION OF A BENCHMARK WAS OPTIONAL AND, THEREFORE, YOU HAD CHOSEN TO ESTABLISH YOUR QUALIFICATIONS AND THE CAPABILITY OF YOUR EXISTING PROGRAMS BY OTHER MEANS. YOU CONTEND THAT AS A RESULT OF THE AMENDED REQUIREMENT FOR A MANDATORY BENCHMARK THE CONSIDERABLE EFFORT AND EXPENSE EXPENDED IN THE PREPARATION OF YOUR PROPOSAL WAS WASTED. YOU ALSO OBJECT TO THE FAILURE TO GRANT YOUR REQUEST FOR AN ADDITIONAL DELAY IN THE ONE-WEEK EXTENSION IN THE DUE DATE FOR PROPOSALS. FURTHERMORE, IT IS YOUR POSITION THAT THE REQUIREMENT EXCLUDED ALL FIRMS EXCEPT THOSE WHICH HAD ALREADY TUNED THEIR PROGRAMS TO THE PARTICULAR BENCHMARK AND TENDED TO CONFIRM RUMORS THAT THE PROCUREMENT WAS BEING DIRECTED TO A PARTICULAR COMPANY. IT IS ALSO YOUR POSITION THAT THE BENCHMARK IS NOT NECESSARY TO THE EVALUATION OF PROPOSALS AS IT IS NOT THE END ITEM OF THE PROCUREMENT, BUT RATHER THE CORE OF A COMPUTER PROGRAM WHICH IS TO BE FURTHER DESIGNED AND DEVELOPED. IN THIS CONNECTION, YOU ALSO ARGUE THAT WHILE BENCHMARKING COMPUTER HARDWARE IS JUSTIFIED BECAUSE IT IS FULLY DEVELOPED AT THE TIME OF EVALUATION, THE SAME JUSTIFICATION DOES NOT APPLY TO COMPUTER SOFTWARE WHICH, AS HERE, IS NOT BEING PURCHASED AS A FULLY DEVELOPED PACKAGE.

AS TO THE APPROPRIATENESS OF THE PARTICULAR BENCHMARK REQUIRED, IT IS YOUR BASIC POSITION THAT PROGRAMS WHICH CAN SOLVE THAT BENCHMARK CANNOT SOLVE THE REAL-LIFE PROBLEMS DESCRIBED IN THE SCOPE OF WORK OF THE RFP AS THE LATTER PROBLEMS REQUIRE CONSIDERABLY MORE ADVANCED AND DIFFERENT TECHNOLOGY. IN THIS CONNECTION, YOU HAVE CITED THREE PRIMARY OBJECTIONS AS REPRESENTATIVE OF REALISTIC PROBLEMS THAT THE BENCHMARK DOES NOT TEST. FURTHERMORE, YOU SAY THAT SPECIAL SOLUTION TECHNIQUES THAT EXIST FOR SOLUTION TO THE BENCHMARK NOT ONLY ARE UNSUITABLE FOR SOLUTION OF THE REAL -LIFE PROBLEMS BUT ALSO FAVOR CERTAIN OFFERORS WHOSE PROGRAMS USE THOSE SPECIAL TECHNIQUES. IN THIS CONNECTION, YOU STATE THAT MANY OF THE TERMS AND CONCEPTS OF THE RFP APPEAR IN THE LITERATURE OF ONE OF THE OTHER OFFERORS. YOU ALSO STATE THAT WHILE MOST CANDIDATE PROGRAMS WERE IN A HIGHER LEVEL LANGUAGE PRIOR TO ISSUANCE OF THE RFP, THE PROGRAM OF DECISION SYSTEMS ASSOCIATES WAS IN "MACHINE" LANGUAGE. YOU BELIEVE IT IS MORE THAN COINCIDENCE THAT THE RFP PERMITTED PROPOSALS TO BE SUBMITTED BASED ON "MACHINE" LANGUAGE.

THE FEDERAL RAILROAD ADMINISTRATION REPORTS THAT SINCE IT DESIRED TO PROCURE AN AVAILABLE RESOURCE ALLOCATION TECHNOLOGY PROGRAM TO AVOID EXTENSIVE DEVELOPMENT WORK AND YET NOT EXCLUDE ANY QUALIFIED FIRM, THE RFP ORIGINALLY PROVIDED THAT TECHNICAL COMPETENCE COULD BE EVIDENCED BY PRIOR WORK DESCRIBED IN THE PROPOSAL AND AS DEMONSTRATED BY A BENCHMARK PROBLEM, OR:

"SO AS NOT TO ARBITRARILY EXCLUDE ANY POTENTIAL RESPONDENT WHO DOES NOT WISH TO SUBMIT A BENCHMARK WITH THIS PROPOSAL, SUCH PROPOSALS LACKING BENCHMARKS WILL BE ACCEPTED. HOWEVER, A SPECIAL BURDEN WILL FALL ON SUCH RESPONDENTS TO ESTABLISH THEIR TECHNICAL COMPETENCE BY OTHER MEANS. (EVALUATION CRITERIA; 2. BENCHMARK, P. 13.)"

FURTHER, IT IS POINTED OUT THAT THE ORIGINAL RFP ALSO PROVIDED FOR BENCHMARKING UPON 15 DAYS NOTICE AFTER THE PRELIMINARY EVALUATION IF CONSIDERED NECESSARY.

THE AMENDMENT REQUIRING MANDATORY BENCHMARKING WAS ISSUED REPORTEDLY AFTER IT WAS REALIZED THAT EVALUATION OF ALL OFFERORS ON AN EQUAL BASIS NECESSITATED SOLVING IDENTICAL BENCHMARK PROBLEMS. IT IS ALSO FRA'S POSITION THAT THE BENCHMARK WAS NECESSARY TO AN OBJECTIVE EVALUATION AND TO DEMONSTRATE THAT THE RESPECTIVE OFFERORS POSSESSED THE MINIMUM TECHNICAL CAPABILITY NEEDED TO SOLVE THE ACTUAL CAR ASSIGNMENT PROBLEMS. FRA ALSO ARGUES THAT JUST AS BENCHMARKING HAS BEEN RECOGNIZED AS AN APPROPRIATE REQUIREMENT IN THE PROCUREMENT OF COMPUTER HARDWARE, IT IS EQUALLY APPROPRIATE IN THE PROCUREMENT OF EXPENSIVE OFF THE-SHELF TECHNOLOGY IN THE FORM OF COMPUTER SOFTWARE.

WITH REGARD TO THE APPROPRIATENESS OF THE REQUIRED BENCHMARK, THE FRA ASSERTS THAT THE BENCHMARK WAS DESIGNED TO AID IN ESTABLISHING WHETHER OFFERORS POSSESSED THE MINIMUM TECHNOLOGICAL CAPABILITY TO FULFILL FRA'S NEEDS. FRA POINTS OUT THAT THE BENCHMARK WAS DEVELOPED BY QUALIFIED PERSONNEL FROM THE FRA AND THE NATIONAL BUREAU OF STANDARDS: THE INDIVIDUAL FROM FRA BEING A FORMER TRANSPORTATION ANALYST AND OPERATING OFFICER OF A MAJOR RAILROAD AND THE OTHER FROM NBS BEING A MATHEMATICIAN WELL QUALIFIED IN OPERATIONS RESEARCH WITH SPECIALIZED EXPERIENCE IN RESOURCE ALLOCATION PROBLEMS. IN ADDITION, THE FRA HAS FURNISHED OUR OFFICE A DETAILED TECHNICAL JUSTIFICATION OF THE DESIGN OF THE BENCHMARK FROM WHICH WE QUOTE THE PREAMBLE:

"PRIOR TO DESIGNING THE BENCHMARK, A CAREFUL ANALYSIS WAS MADE OF THE INTERSTATE COMMERCE ACT, ICC REGULATIONS, RELEVENT (RELEVANT) COURT CASES, AAR CAR SERVICE MILES, RAILROAD POLICIES AND OPERATIONS PRACTICES, AND RELEVENT (RELEVANT) RESOURCE ALLOCATION LITERATURE. THE RESULTING BENCHMARK REFLECTS THOSE CHARACTERISTICS OF THE REAL WORLD PROBLEM JUDGED TO BE MOST SIGNIFICANT IN MEASURING THE CAPABILITY OF A RESOURCE ALLOCATION PROGRAM TO BE ADAPTED TO OPERATIONAL RAILROAD USE."

THE FRA POINTS OUT THAT YOUR REQUEST FOR A FURTHER EXTENSION IN THE DUE DATE FOR PROPOSALS WAS NOT MADE UNTIL MAY 19, 1972, THE AMENDED DATE PROPOSALS WERE DUE, EVEN THOUGH THE AMENDMENT ADDING THE MANDATORY BENCHMARK WAS DATED APRIL 26, 1972. FURTHERMORE, IT IS REPORTED THAT FOUR PROPOSALS, ALONG WITH THE SOLUTION TO THE BENCHMARK, WERE TIMELY RECEIVED.

FRA EMPHATICALLY DENIES ANY ATTEMPT TO DIRECT THE PROCUREMENT TO A PARTICULAR OFFEROR. IN THIS CONNECTION, IT IS POINTED OUT THAT THE SELECTION WAS MADE BY AN INDEPENDENT SOURCE EVALUATION BOARD WHICH HAD NO OTHER RESPONSIBILITY OR CONNECTION WITH THE PROCUREMENT. IT IS REPORTED THAT TO AVOID ANY POSSIBILITY OF PREJUDICE THE DEVELOPERS OF THE BENCHMARK WERE PURPOSEFULLY EXCLUDED FROM THE SEB. IT IS ALSO REPORTED THAT IN CONNECTION WITH PREPARING THE RFP AVAILABLE PUBLISHED LITERATURE, INCLUDING THAT OF LAMBDA, ON COMPUTER-BASED RESOURCE ALLOCATION WAS RESEARCHED, WHICH MAY HAVE COINCIDENTALLY RESULTED IN SOME LANGUAGE IN THE RFP BEING SIMILAR TO SOME OF THE SOURCES RESEARCHED.

IT IS A WELL-ESTABLISHED PRINCIPLE THAT THE RESPONSIBILITY FOR DRAFTING SPECIFICATIONS REFLECTING THE MINIMUM NEEDS OF THE GOVERNMENT AND THE FACTUAL DETERMINATION AS TO COMPLIANCE WITH THOSE SPECIFICATIONS IS THE PREROGATIVE OF THE PROCURING AGENCY AND NOT SUBJECT TO LEGAL OBJECTION BY THIS OFFICE UNLESS CLEARLY ARBITRARY. 49 COMP. GEN. 156, 160 (1969). OUR OFFICE HAS RECOGNIZED THE APPROPRIATENESS OF BENCHMARKING IN PROCURING COMPUTER HARDWARE TO DETERMINE THE CAPABILITY OF A POTENTIAL CONTRACTOR AND THE SUITABILITY AND COST EFFECTIVENESS OF HIS EQUIPMENT. 52 COMP. GEN.

, B-175317, SEPTEMBER 8, 1972. EVEN THOUGH COMPUTER SOFTWARE IS THE OBJECT OF THIS PROCUREMENT, WE BELIEVE THE NEED FOR AND APPROPRIATENESS OF BENCHMARKING HAS BEEN ADEQUATELY JUSTIFIED. FURTHERMORE, WE ARE UNABLE TO CONCLUDE FROM THE RECORD BEFORE US THAT THE REQUIREMENT WAS AN UNFAIR OR UNDUE RESTRICTION ON COMPETITION, OR THAT THERE WAS ANY ATTEMPT TO FAVOR A PARTICULAR OFFEROR.

ACCORDINGLY, YOUR PROTEST IS DENIED.