B-212979, SEP 17, 1984, 84-2 CPD 295

B-212979, SEP 17, 1984, 84-2 CPD 295

CONTRACTS - PROTESTS - GENERAL ACCOUNTING OFFICE PROCEDURES - TIMELINESS OF PROTEST - SOLICITATION IMPROPRIETIES - APPARENT PRIOR TO BID OPENING/CLOSING DATE FOR PROPOSALS DIGEST: 1. SINCE ALLEGED AMBIGUITIES INVOLVE LANGUAGE IN THE SOLICITATION ITSELF, THEY ORDINARILY MUST BE PROTESTED IN ACCORDANCE WITH GAO'S BID PROTEST PROCEDURES, 4 C.F.R. SEC. 21.2(B)(1) (1984), PRIOR TO THE SOLICITATION'S CLOSING DATE. THE ONLY EXCEPTION IS WHERE THE PROTESTER IS UNAWARE PRIOR TO THE CLOSING DATE THAT ITS INTERPRETATION IS NOT THE ONLY ONE POSSIBLE. GAO IS UNABLE TO CONCLUDE THAT THE PROTESTER'S INTERPRETATION OF THE ALLEGEDLY AMBIGUOUS SOLICITATION PROVISION WAS THE ONLY ONE POSSIBLE. CONTRACTS - NEGOTIATION - OFFERS OR PROPOSALS - EVALUATION - FACTORS NOT IN SOLICITATION - ORAL DISCLOSURE DURING NEGOTIATIONS 2. WHERE THE OFFEROR IS ORALLY INFORMED DURING NEGOTIATIONS OF THE MEANING OF A SOLICITATION REQUIREMENT AND THE METHODOLOGY BY WHICH THE REQUIREMENT WILL BE APPLIED IN THE BENCHMARKING, THE OFFEROR IS ON NOTICE OF THE REQUIREMENT AND METHOD OF APPLICATION AND GAO WILL DISMISS A PROTEST BASED ON THE AGENCY'S FAILURE TO ADEQUATELY STATE THE REQUIREMENT IN THE SOLICITATION WHEN FILED MORE THAN 10 DAYS AFTER OFFEROR IS INFORMED. CONTRACTS - NEGOTIATION - OFFERS OR PROPOSALS - REJECTION - EXTENT OF GAO REVIEW 3. WHERE THE REASONS ADVANCED BY THE AGENCY JUSTIFYING THE REJECTION OF THE PROTESTER'S OFFER WERE ERRONEOUS AT THE TIME OF REJECTION, A SUBSEQUENT STATEMENT BY THE AGENCY OF DIFFERENT REASONS WHICH WOULD HAVE SUPPORTED THE REJECTION, IF ADVANCED INITIALLY, IS ACCEPTABLE. GAO'S FUNCTION IS TO LOOK TO WHETHER, IN LIGHT OF THE RECORD, THE AGENCY'S ACTION HAS ANY SUPPORTING BASIS, NOT JUST TO EXAMINE THE BASIS WHICH THE AGENCY ADVANCED AT THE TIME IT TOOK THE ACTION. CONTRACTS - NEGOTIATION - OFFERS OR PROPOSALS - DEFICIENT PROPOSALS OPPORTUNITY TO CORRECT 4. GAO FINDS THAT THE AGENCY AFFORDED THE PROTESTER THE OPPORTUNITY TO CORRECT ITS PROPOSAL WHERE IT ASKED QUESTIONS CONCERNING THE PROTESTER'S PROPOSAL AND CONDUCTED TWO BENCHMARKS ON THE PROTESTER'S PROPOSED COMPUTER SOFTWARE.

CENTENNIAL COMPUTER PRODUCTS, INC.:

CENTENNIAL COMPUTER PRODUCTS, INC. (CENTENNIAL), PROTESTS THE REJECTION OF ITS PROPOSAL UNDER REQUEST FOR PROPOSALS (RFP) IRS-83-053 ISSUED BY THE DEPARTMENT OF THE TREASURY, INTERNAL REVENUE SERVICE (IRS). THE RFP WAS FOR THE LEASE OF TAPE, DISK, AND CACHE/DISK SUBSYSTEMS TO ENHANCE THE UNIVAC 1100/80 COMPUTER SYSTEM AT THE IRS'S DETROIT DATA CENTER. AN AWARD PENDING THE PROTEST WAS MADE TO AMPERIF CORPORATION (AMPERIF).

CENTENNIAL CONTENDS THAT THE IRS ERRONEOUSLY DETERMINED THAT ITS PROPOSED CACHE/DISK SUBSYSTEM FAILED TO MEET THE RFP'S REQUIREMENTS. IN ADDITION, CENTENNIAL ASSERTS THAT THE IRS APPLIED A DIFFERENT AND MORE LENIENT STANDARD IN EVALUATING AMPERIF'S PROPOSED CACHE/DISK SUBSYSTEMS.

FOR THE REASONS SET FORTH BELOW, WE DENY CENTENNIAL'S PROTEST.

BACKGROUND

ONLY CENTENNIAL AND AMPERIF SUBMITTED PROPOSALS IN RESPONSE TO THE RFP. THE IRS MET WITH BOTH OFFERORS TO DISCUSS BENCHMARKING REQUIREMENTS AND PROCEDURES. FOLLOWING THESE DISCUSSIONS, THE RFP WAS AMENDED TO ADD THE BENCHMARKING REQUIREMENTS. BOTH CENTENNIAL AND AMPERIF COMPLETED THE BENCHMARK IN JUNE 1983. IN JULY 1983, THE IRS NOTIFIED BOTH OFFERORS THAT THEY DID NOT SUCCESSFULLY COMPLETE THE BENCHMARK, BUT WOULD BE GIVEN ANOTHER OPPORTUNITY. THE RFP WAS AGAIN AMENDED TO INCLUDE REVISED BENCHMARKING REQUIREMENTS.

AFTER CENTENNIAL AND AMPERIF RERAN THE BENCHMARK UNDER THE REVISED REQUIREMENTS, THE IRS FOUND AMPERIF TO BE TECHNICALLY ACCEPTABLE AND CENTENNIAL TO BE TECHNICALLY UNACCEPTABLE. BY LETTER DATED AUGUST 29, 1983, THE IRS INFORMED CENTENNIAL THAT ITS PROPOSAL WAS TECHNICALLY UNACCEPTABLE AND WOULD NOT BE CONSIDERED FURTHER BECAUSE THE BENCHMARKING REQUIREMENTS FOR AN OFFEROR'S CACHE/DISK SUBSYSTEM REQUIRED A "CACHE HIT RATE" OF 85 PERCENT OR LESS AND CENTENNIAL'S RATE DURING THE BENCHMARKING WAS 86.8 PERCENT.

CENTENNIAL, ON SEPTEMBER 9, 1983, PROTESTED TO OUR OFFICE THE AWARD TO ANYONE OTHER THAN CENTENNIAL ON THE GROUNDS THAT CENTENNIAL WAS THE LOWEST COST, MOST TECHNICALLY CAPABLE OFFEROR AND AWARD TO ANOTHER OFFEROR WOULD BE CONTRARY TO THE TERMS OF THE RFP.

AS BACKGROUND, "CACHE" MEMORY IS A FORM OF SOLID STATE MEMORY. THE PURPOSE OF CACHE MEMORY IS TO INCREASE THE SPEED OF A COMPUTER SYSTEM'S OPERATION BY REDUCING THE AMOUNT OF TIME REQUIRED TO SEARCH FOR NEEDED DATA. DATA THAT IS STORED IN SOLID STATE MEMORY CAN BE RETRIEVED MORE QUICKLY THAN DATA THAT IS STORED ON DISKS OR TAPE. IT IS NORMALLY NECESSARY TO HAVE ACCESS TO MORE DATA THAN CAN BE ECONOMICALLY STORED IN SOLID STATE MEMORY. CACHE MEMORY IS A MEANS OF SELECTING THE MOST FREQUENTLY ACCESSED DATA FOR STORAGE IN SOLID STATE FORM, THEREBY MINIMIZING THE NEED TO MAKE TIME-CONSUMING SEARCHES OF DISK OR TAPE MEMORY.

THE RFP REQUIRED THAT AN OFFEROR'S CACHE MEMORY MUST OFFER A 50 PERCENT INCREASE IN SPEED OVER NONCACHE OPERATION UNDER SPECIFIED CRITERIA. WHEN THE COMPUTER SYSTEM NEEDS ACCESS TO PARTICULAR DATA, IT FIRST REQUESTS THAT DATA FROM THE FASTER CACHE MEMORY. IF THE REQUESTED DATA IS STORED IN CACHE MEMORY, THE SYSTEM HAS SCORED A "HIT." THE "HIT RATIO" IS THE RATIO OF HITS TO REQUESTS.

TIMELINESS

THE IRS CONTENDS THAT CENTENNIAL'S PROTEST IS UNTIMELY BECAUSE IT WAS FILED AFTER THE RFP'S CLOSING DATE FOR RECEIPT OF PROPOSALS. IN THE IRS'S VIEW, CENTENNIAL IS OBJECTING TO A SPECIFIC REQUIREMENT IN THE RFP PERTAINING TO THE MAXIMUM ALLOWABLE CAPABILITY FOR ACCESSING CACHE MEMORY. THE IRS POINTS OUT THAT UNDER OUR BID PROTEST PROCEDURES, PROTESTS BASED UPON ALLEGED IMPROPRIETIES IN A SOLICITATION WHICH ARE APPARENT PRIOR TO THE CLOSING DATE FOR RECEIPT OF INITIAL PROPOSALS SHALL BE FILED PRIOR TO THAT DATE. 4 C.F.R. SEC. 21.2(B)(1) (1984).

IN THE ALTERNATIVE, THE IRS ARGUES THAT IF CENTENNIAL'S PROTEST IS FOUND NOT TO BE AGAINST AN ALLEGED APPARENT SOLICITATION IMPROPRIETY, CENTENNIAL IS STILL PRECLUDED FROM RAISING THE MATTER WITH OUR OFFICE. THE IRS STATES THAT BY LETTER DATED AUGUST 29, 1983, IT NOTIFIED CENTENNIAL THAT THE COMPANY WAS DISQUALIFIED FROM THE COMPETITION BASED ON ITS FAILURE TO SUCCESSFULLY COMPLETE THE SECOND BENCHMARK WITH RESPECT TO THE LIMITATION ON ACCESSING CACHE MEMORY. THE IRS ARGUES THAT SINCE OUR BID PROTEST PROCEDURES REQUIRE THAT PROTESTS ALSO BE FILED WITHIN 10 WORKING DAYS AFTER THE BASIS OF PROTEST IS KNOWN OR SHOULD HAVE BEEN KNOWN, CENTENNIAL HAD UNTIL SEPTEMBER 13, 1983, TO PROTEST ITS DISQUALIFICATION FROM FURTHER COMPETITION. THE IRS POINTS OUT THAT CENTENNIAL DID NOT OBJECT TO THE 85- PERCENT LIMITATION ON CACHE HIT RATE UNTIL IT FILED ITS NOVEMBER 15, 1983, COMMENTS ON THE PROTEST REPORT.

IN OUR OPINION, CENTENNIAL IS NOT PROTESTING THE PROPRIETY OF THE RFP'S REQUIREMENT BUT, RATHER, CENTENNIAL IS ESSENTIALLY PROTESTING THAT THE IRS IMPROPERLY DISQUALIFIED IT FROM COMPETITION BASED ON A NONEXISTENT SOLICITATION REQUIREMENT. IN THIS REGARD, PARAGRAPH F.2.3. OF THE RFP PROVIDED AS FOLLOWS:

"... AT A CACHE HIT RATE OF 85% OR LESS, CACHE SHALL PERFORM AT A RATE 50% FASTER THAN UNIVAC 8470 DISK (I.E. ACCESS AND TRANSFER AN 1800 WORD BLOCK SEQUENTIALLY USING THE STANDARD UNIVAC IOW$ WRITE FUNCTION IN 10 MILLISECONDS OR LESS.) WHEN USING A DISK PREP FACTOR OF 112 WORDS PER SECTOR."

CENTENNIAL ARGUES THAT THE ABOVE-QUOTED RFP PROVISION IMPOSES NO LIMITATION ON THE ACCESS TO CACHE MEMORY. ACCORDING TO CENTENNIAL, THE ONLY POSSIBLE INTERPRETATION OF THE LANGUAGE "AT A CACHE HIT RATE OF 85% OR LESS" IS THAT 85 PERCENT WAS MERELY ONE OF THE PARAMETERS OF THE SPEED REQUIREMENT FOR AN OFFEROR'S CACHE/DISK SUBSYSTEM, THAT IS, CACHE MEMORY OPERATION MUST BE 50 PERCENT FASTER THAN NONCACHE MEMORY OPERATION AT A CACHE HIT RATE OF 85 PERCENT.

ORDINARILY, ALLEGED AMBIGUITIES IN THE LANGUAGE OF A SOLICITATION PROVISION MUST BE PROTESTED TO OUR OFFICE PRIOR TO THE SOLICITATION'S CLOSING DATE. SKYTOP PLASTICS, INC., B-207022, OCT. 15, 1982, 82-2 CPD PARA. 340. WE HAVE RECOGNIZED AN EXCEPTION TO THIS RULE WHERE THE PROTESTER WAS UNAWARE, PRIOR TO THE CLOSING DATE, THAT ITS INTERPRETATION OF THE SOLICITATION PROVISION WAS NOT THE ONLY ONE POSSIBLE. THIS IS BECAUSE ABSENT AWARENESS OF A SECOND INTERPRETATION, THE PROTESTER CANNOT BE AWARE OF AN AMBIGUITY. SEE CONRAC CORPORATION, B-205562, APR. 5, 1982, 82-1 CPD PARA. 309. HERE, HOWEVER, WE CANNOT CONCLUDE THAT CENTENNIAL'S INTERPRETATION OF THE RFP'S 85 PERCENT HIT RATE LANGUAGE IS THE ONLY ONE POSSIBLE. IN OUR VIEW, CENTENNIAL'S INTERPRETATION DOES NOT TAKE INTO ACCOUNT THE EFFECT OF THE WORDS "OR LESS" THAT APPEAR WITH THE 85-PERCENT LANGUAGE.

IN ANY EVENT, THE IRS HAS SUBMITTED AFFIDAVITS WHICH SHOW THAT ITS PROCUREMENT OFFICIALS ORALLY ADVISED CENTENNIAL ON SEVERAL OCCASIONS SUBSEQUENT TO THE CLOSING DATE THAT THE SOLICITATION'S 85-PERCENT CACHE HIT RATE LANGUAGE WAS A LIMITATION WHICH COULD NOT BE EXCEEDED. SPECIFICALLY, THESE OFFICIALS STATE THAT IN RESPONSE TO COMMENTS BY A CENTENNIAL OFFICIAL CONCERNING THE LOW CACHE HIT RATE WHILE THE JUNE 1983 BENCHMARK WAS BEING CONDUCTED, IT WAS EXPLAINED THAT 85 PERCENT REPRESENTED THE UPPER LIMIT OF ACCEPTABLE HIT RATES. THE OFFICIALS ALSO STATE THAT AT AN AUGUST 8, 1983, CONFERENCE BEFORE THE RUNNING OF THE SECOND BENCHMARK BEGAN, IT WAS REEMPHASIZED THAT 85 PERCENT WAS THE MAXIMUM LIMITATION. THE IRS ALSO POINTS OUT THAT THE 85-PERCENT LIMITATION WAS INTENDED AS A CHECK TO PREVENT FINE TUNING OF EQUIPMENT THAT WOULD "ARTIFICIALLY" INCREASE EQUIPMENT PERFORMANCE DURING BENCHMARKING. WE HAVE HELD CONSISTENTLY THAT WHEN AN OFFEROR IS INFORMED OF AN AGENCY'S REQUIREMENT DURING NEGOTIATIONS, NOTWITHSTANDING ITS ABSENCE IN THE SOLICITATION, THE OFFEROR IS ON NOTICE OF THE REQUIREMENT. SOUTHLAND ASSOCIATES, B-207350, NOV. 17, 1982, 82-2 CPD PARA. 441. HERE, THERE WAS A WRITTEN SOLICITATION PROVISION TO WHICH THE IRS PROVIDED AN ORAL CLARIFICATION DURING NEGOTIATIONS. NEVERTHELESS, THE RESULT IS THE SAME; CENTENNIAL WAS PUT ON NOTICE DURING NEGOTIATIONS OF WHAT THE IRS'S EXACT REQUIREMENT WAS REGARDING THE ACCESS OF CACHE MEMORY AND THE SUBSEQUENT PROTEST ON THIS ISSUE IS UNTIMELY.

CENTENNIAL DENIES THAT THE IRS EVER ORALLY NOTIFIED IT THAT THE 85 PERCENT RATE WAS A CEILING THAT WAS NOT TO BE EXCEEDED UNDER ANY CIRCUMSTANCES. CENTENNIAL HAS SUBMITTED AFFIDAVITS FROM ITS OWN REPRESENTATIVES WHO WERE INVOLVED IN THE PROCUREMENT STATING THAT AT NO TIME PRIOR TO THE IRS'S REJECTION OF THE COMPANY'S INITIAL PROPOSAL DID ANY IRS OFFICIAL STATE THAT 85 PERCENT WAS THE PEAK EFFICIENCY RATE FOR CACHE. HOWEVER, CENTENNIAL, AS THE PROTESTER, HAS THE BURDEN OF AFFIRMATIVELY PROVING THAT IT WAS NEVER INFORMED OF THE 85-PERCENT LIMITATION. WE WILL NOT CONSIDER THIS BURDEN MET WHERE THE ONLY EVIDENCE IS CONFLICTING STATEMENTS BY CENTENNIAL AND THE IRS. ALCHEMY, INC., B-207954, JAN. 10, 1983, 83-1 CPD PARA. 18.

FINALLY, WE NOTE THAT CENTENNIAL MAINTAINS THAT EVEN THOUGH ITS CACHE HIT RATE MIGHT HAVE EXCEEDED 85 PERCENT DURING THE SECOND BENCHMARKING, CACHE MEMORY WAS STILL OVER 50 PERCENT FASTER DURING BENCHMARKING THAN THE UNIVAC 84070'S NONCACHE MEMORY AT AN 85-PERCENT OR ABOVE HIT RATE. WHILE THIS MAY, INDEED, BE TRUE, THE RFP'S 85 PERCENT CEILING REQUIREMENT ON THE HIT RATE FOR CACHE MEMORY CANNOT BE DISREGARDED. THE RFP IMPOSED TWO INTERRELATED REQUIREMENTS ON CACHE MEMORY: (1) THAT IT BE 50 PERCENT FASTER THAN NONCACHE MEMORY, AND (2) THAT THIS FASTER SPEED BE ACHIEVED AT A CACHE HIT RATE THAT DOES NOT EXCEED 85 PERCENT.

DETERMINATION OF THE ACTUAL CACHE HIT RATE

IN THE ALTERNATIVE, CENTENNIAL ARGUES THAT IF 85 PERCENT WAS A LIMITATION RATE ON ACCESSING CACHE MEMORY CENTENNIAL'S ACTUAL CACHE HIT RATE DURING THE SECOND BENCHMARKING WAS ONLY 65.3 PERCENT, NOT 86.8 PERCENT AS STATED BY THE IRS. CENTENNIAL ALLEGES THAT BY STANDARD INDUSTRY DEFINITION, A "HIT" OCCURS WHEN REQUESTED DATA IS FOUND IN CACHE MEMORY UPON REQUEST, THEREBY AVOIDING THE ADDITIONAL TIME NEEDED FOR A SEARCH OF THE MEMORY STORED ON DISK. CENTENNIAL FURTHER ALLEGES THAT THE TERM "HIT RATIO" ALSO HAS ONLY ONE INDUSTRY DEFINITION WHICH IS THE RATIO OF CACHE HITS TO THE TOTAL NUMBER OF REQUESTS. CENTENNIAL ARGUES THAT THE IRS MISTREATED AS A "CACHE HIT" A FUNCTION PERFORMED BY ITS COMPUTER KNOWN AS "CACHE IT." ACCORDING TO CENTENNIAL, WHEN DATA IS NOT FOUND IN CACHE MEMORY, CERTAIN TESTS ARE THEN APPLIED BY THE INTERNAL LOGIC OF ITS COMPUTER SYSTEM DURING A SEARCH OF DISK MEMORY IN ORDER TO DETERMINE THE LIKELIHOOD OF ANY SECOND USE OF THE REQUESTED DATA. CENTENNIAL ASSERTS THAT IF THE LIKELIHOOD OF A SECOND USE OF SUCH DATA IS SUFFICIENTLY HIGH, THE SYSTEM WILL "CACHE IT" BY ADDING THE DATA TO CACHE MEMORY SO THAT A SECOND REQUEST FOR THE DATA WILL SCORE A CACHE HIT RATHER THAN A SECOND "CACHE" MISS. CENTENNIAL CONTENDS THAT THE INITIAL DATA REQUEST SHOULD RESULT IN A "CACHE MISS" WHEN THE DATA IS NOT IN THE CACHE MEMORY REGARDLESS OF THE FACT THAT THE DATA IS SUBSEQUENTLY BROUGHT INTO CACHE MEMORY THROUGH THE CACHE IT FUNCTION.

THE IRS CONTENDS THAT CENTENNIAL KNEW AND UNDERSTOOD THE METHODOLOGY WHICH THE IRS USED TO CALCULATE CENTENNIAL'S CACHE HIT RATE DURING THE BENCHMARKING AND THAT CENTENNIAL'S CACHE HIT RATE WAS PROPERLY CALCULATED IN ACCORDANCE WITH THIS METHODOLOGY. IN SPECIFIC, THE IRS STATES THAT DURING TECHNICAL MEETINGS CONDUCTED BEFORE THE FIRST BENCHMARK BEGAN, CENTENNIAL AGREED TO DEFINE CENTENNIAL'S CACHE IT FUNCTION AS CACHE HIT BECAUSE IT INVOLVED THE TRANSFER OF REQUESTED DATA "FROM CACHE MEMORY TO THE MAIN COMPUTER AFTER SUCH DATA HAD BEEN LOADED FROM DISK UNITS AT SOME PREVIOUS POINT IN TIME." THE IRS POINTS OUT THAT THIS TREATMENT OF CENTENNIAL'S CACHE IT FUNCTION WAS SIMILAR TO THE DEFINITION OF A CACHE HIT WHICH WAS THE TRANSFER OF DATA FROM CACHE MEMORY TO THE MAIN COMPUTER UPON REQUEST. THEREFORE, THE IRS TAKES THE POSITION THAT CENTENNIAL CANNOT AT THIS LATE DATE OBJECT TO HOW THE CACHE IT FUNCTION WAS TREATED FOR PURPOSES OF DETERMINING THE CACHE HIT RATE WHEN IT WAS PUT ON NOTICE EARLY DURING NEGOTIATIONS THAT IT WOULD BE VIEWED AS A CACHE HIT.

FINALLY, THE IRS ASSERTS THAT, IN ADDITION TO EXCEEDING THE 85 PERCENT CACHE HIT RATE LIMITATION DURING THE SECOND BENCHMARK, CENTENNIAL FINE TUNED ITS COMPUTER EQUIPMENT AND ALSO FAILED TO HAVE A "DATA SAVE" FEATURE ON ITS EQUIPMENT, BOTH OF WHICH WERE VIOLATIONS OF RFP REQUIREMENTS. ACCORDING TO THE IRS, CENTENNIAL DELIBERATELY SLOWED DOWN THE RATE OF TIME IT TOOK TO COMPLETE THE NONCACHE OPERATIONS SO AS TO MAKE THE CACHE OPERATIONS SEEM SIGNIFICANTLY FASTER IN COMPARISON. AS TO THE CHARGE THAT CENTENNIAL DID NOT HAVE A DATA SAVE FEATURE, THE IRS STATES THAT THE RFP CLEARLY SPECIFIED THAT IF A VENDOR'S EQUIPMENT "WRITES" DATA TO CACHE MEMORY, A DATA SAVE FEATURE MUST BE PROVIDED IN ORDER TO ENSURE THAT THE DATA IN CACHE MEMORY WILL BE WRITTEN OUT TO DISK UNITS IN CASE OF A POWER FAILURE.

IN RESPONSE, CENTENNIAL DENIES THAT IT WAS EVER TOLD BY THE IRS PRIOR TO THE REJECTION OF ITS PROPOSAL THAT INSTANCES IN WHICH DATA NOT FOUND IN CACHE MEMORY WHEN REQUESTED WAS TO BE TREATED AS A CACHE HIT BECAUSE IT WAS SUBSEQUENTLY ADDED TO CACHE MEMORY TO SERVICE FURTHER REQUESTS. THIS REGARD, CENTENNIAL HAS SUBMITTED AFFIDAVITS OF DENIAL FROM ITS REPRESENTATIVES WHO WERE INVOLVED IN THE PROCUREMENT. IN ADDITION, CENTENNIAL HAS SUBMITTED A SERIES OF MATHEMATICAL EQUATIONS TO SHOW THAT THE IRS DID NOT USE STANDARD DEFINITIONS IN DETERMINING THE CACHE HIT RATE AND TO SHOW THAT THE IRS COMMITTED "AN ALGEBRAIC ERROR" IN APPLYING THE FORMULA USED TO ARRIVE AT CENTENNIAL'S ACTUAL CACHE HIT RATE FROM THE SECOND BENCHMARKING. FROM OUR REVIEW OF THE RECORD, WE FIND THAT THERE IS CLEARLY AN INDUSTRY WIDE FORMULA FOR DETERMINING THE CACHE HIT RATIO, WHICH IS THE NUMBER OF TIMES REQUESTED DATA IS FOUND IN CACHE MEMORY DIVIDED BY THE TOTAL REQUESTS MADE. ALSO, WE FIND THAT THE TERM "CACHE HIT" HAS A COMMONLY UNDERSTOOD MEANING WHICH ESSENTIALLY IS THE FINDING OF DATA IN CACHE MEMORY WHEN THE DATA IS REQUESTED. WHILE THE RECORD DOES SHOW THAT THE TERM "CACHE IT" IS ONE USED ONLY BY CENTENNIAL AND, THUS, LACKS INDUSTRY WIDE RECOGNITION, WE QUESTION THE IRS'S TREATMENT OF CENTENNIAL'S CACHE IT FUNCTION AS A CACHE HIT IN VIEW OF THE FACT THAT THE PURPOSE OF HAVING A CACHE MEMORY IS TO AVOID THE NEED TO SEARCH THE SLOWER MEMORY STORED ON DISK. DESPITE THE IRS'S STATEMENT THAT ITS TREATMENT OF CENTENNIAL'S CACHE IT FUNCTION CORRESPONDS TO THE DEFINITION OF A CACHE HIT, WE FIND THAT CENTENNIAL'S CACHE IT FUNCTION IS MUCH CLOSER TO THE IRS'S STATED DEFINITION OF "CACHE MISS" FOR BENCHMARKING PURPOSES, WHICH WAS THE TRANSFER OF REQUESTED DATA TO THE MAIN COMPUTER FROM DISK. MOREOVER, WE FAIL TO UNDERSTAND WHY THE IRS INSISTED ON TREATING THE USE OF CENTENNIAL'S CACHE IT FUNCTION AS A HIT WHEN IT IS CLEAR FROM THE RECORD THAT, REGARDLESS OF THE SUBSEQUENT TRANSFER OF THE DATA TO CACHE, THE DATA WAS NOT IN CACHE MEMORY WHEN THE INITIAL INPUT/OUTPUT REQUEST WAS MADE. THEREFORE, CENTENNIAL'S CACHE HIT RATE WAS ACTUALLY BELOW THE 85- PERCENT LIMITATION SINCE THOSE TIMES DURING THE BENCHMARK WHEN CENTENNIAL'S CACHE IT FUNCTION WAS EMPLOYED SHOULD HAVE BEEN SCORED A "MISS" RATHER THAN A "HIT."

OTHER DEFICIENCIES

WHILE WE HAVE CONCLUDED THAT IRS INCORRECTLY CALCULATED CENTENNIAL'S CACHE HIT RATE, WE AGREE WITH THE IRS THAT CENTENNIAL FINE TUNED ITS EQUIPMENT FOR THE SECOND BENCHMARK AND ALSO FAILED TO HAVE A DATA SAVE FEATURE ON ITS EQUIPMENT DURING THE SECOND BENCHMARK. WE RECOGNIZE THAT CENTENNIAL WAS INFORMED AT THE TIME ITS OFFER WAS REJECTED THAT THE ONLY REASON FOR REJECTION WAS CENTENNIAL'S FAILURE TO ADHERE TO THE RFP'S CACHE HIT RATE LIMITATION. HOWEVER, WE HAVE HELD THAT EVEN WHERE THE REASONS ADVANCED BY A CONTRACTING ACTIVITY JUSTIFYING A PARTICULAR ACTION WERE ERRONEOUS AT THE TIME THE ACTION WAS TAKEN, A SUBSEQUENT STATEMENT OF DIFFERENT REASONS WHICH WOULD HAVE SUPPORTED THE ACTION, IF ADVANCED INITIALLY, IS ACCEPTABLE. SEE TOSCO CORPORATION, B-187776, MAY 10, 1977, 77-1 CPD PARA. 329. OUR APPROACH IS TO LOOK TO WHETHER, IN LIGHT OF THE RECORD, THE AGENCY'S ACTION HAS ANY BASIS SUPPORTING IT, NOT SIMPLY TO EXAMINE THE BASIS WHICH THE AGENCY ADVANCED AT THE TIME IT TOOK THE ACTION.

THE IRS ASSERTS THAT CENTENNIAL FINE TUNED ITS EQUIPMENT BY SLOWING DOWN THE SPEED OF THE DISK UNITS FOR NONCACHE OPERATIONS DURING THE SECOND BENCHMARK SO THAT THE CACHE OPERATIONS WOULD MEET THE RFP'S REQUIREMENT FOR 50 PERCENT FASTER OPERATION THAN NONCACHE. THE IRS STATES THAT IT USED THE SAME PROCEDURES FOR BOTH BENCHMARKS TO CREATE DATA FILES. MORE SPECIFICALLY, THE IRS STATES THAT THE SAME FILES WERE PLACED ON FOUR DISK UNITS IN THE SAME MANNER FOR BOTH BENCHMARKS. THE IRS FURTHER STATES THAT IN BOTH BENCHMARKS, THE METHOD USED TO CALCULATE THE TIME TAKEN FOR NONCACHE OPERATIONS WAS TO DETERMINE THE OVERALL AVERAGE TIME REQUIRED TO COMPLETE A REQUEST COMPUTED FOR AT LEAST 100,000 INPUT/OUTPUT'S EXECUTED ON THE FOUR DISK UNITS. THE IRS POINTS OUT THAT THE AVERAGE TIME TAKEN TO COMPLETE A NONCACHE INPUT/OUTPUT OPERATION IN THE SECOND BENCHMARK INCREASED BY 30.6 PERCENT FROM THE FIRST BENCHMARK. IN THE IRS'S VIEW, THIS SHOWS THAT THE FOUR DISK UNITS WERE INTENTIONALLY SLOWED DOWN BY CENTENNIAL SO THAT IT COULD PASS THE SECOND BENCHMARK.

THE RECORD SHOWS THAT THE NUMBER OF NONCACHE INPUT/OUTPUT OPERATIONS DECREASED IN FREQUENCY FROM 108,244 FOR THE FIRST BENCHMARK TO 100,607 FOR THE SECOND BENCHMARK. CONSEQUENTLY, THE NUMBER OF OPERATIONS PER SECOND DECREASED FOR ALL FOUR UNITS. LOGICALLY, THEN, AN OFFEROR'S AVERAGE TIME TO COMPLETE A NONCACHE OPERATION SHOULD ALSO HAVE FALLEN. INSTEAD, THE AVERAGE COMPLETION TIME ROSE SIGNIFICANTLY FOR EACH OF THE FOUR UNITS IN CENTENNIAL'S SECOND BENCHMARK. FOR EXAMPLE, THE NUMBER OF OPERATIONS PER SECOND ON UNIT 1, WHICH CONTAINED ONE FILE, DECREASED FROM 19.6 TO 16.0. YET, CENTENNIAL'S AVERAGE COMPLETION TIME FOR THE SAME FILE INCREASED FROM 34 MILLISECONDS IN THE FIRST BENCHMARK TO 51 MILLISECONDS DURING THE SECOND BENCHMARK. THUS, WE FIND THAT THE RESULTS OF CENTENNIAL'S SECOND BENCHMARK SUPPORT THE IRS'S ASSERTION THAT THE COMPANY FINE TUNED ITS EQUIPMENT BY DELIBERATELY SLOWING THE SPEED OF THE NONCACHE OPERATIONS DURING THE SECOND BENCHMARK, PARTICULARLY WHERE THE RECORD SHOWS THAT CENTENNIAL WOULD NOT HAVE MET THE 50-PERCENT FASTER REQUIREMENT FOR CACHE OPERATION OVER NONCACHE OPERATION IF THE SAME RESULTS HAD BEEN OBTAINED FOR CENTENNIAL'S NONCACHE OPERATION DURING THE SECOND BENCHMARK AS WERE OBTAINED DURING THE FIRST BENCHMARK.

WE NOTE THAT CENTENNIAL ALLEGES THAT IT HAD NO OPPORTUNITY TO FINE TUNE PRIOR TO THE RUNNING OF THE SECOND BENCHMARK BECAUSE IT HAD BEEN TOLD BY THE IRS THAT THE SECOND BENCHMARK WOULD BE "DIFFERENT." THE RECORD, HOWEVER, SHOWS THAT THE SECOND BENCHMARK WAS CONDUCTED OVER A 3 DAY PERIOD. THE RECORD FURTHER SHOWS THAT CENTENNIAL HAD COMPLETE ACCESS TO ITS EQUIPMENT AND ALSO TO THE TEST PROGRAMS AND DATA AT THE END OF THE FIRST AND SECOND DAYS OF BENCHMARKING. IN THIS REGARD, THE IRS POINTS OUT THAT CENTENNIAL COULD HAVE RUN THE TEST PROGRAMS REPEATEDLY UNTIL THE EQUIPMENT CONFIGURATION NECESSARY TO MEET THE RFP'S REQUIREMENTS WAS FOUND. ACCORDING TO THE IRS, THE DISKS CAN BE SLOWED DOWN IN LESS THAN 15 MINUTES ONCE THE "PROPER SETTING" IS KNOWN.

WITH RESPECT TO THE FAILURE TO HAVE A DATA SAVE FEATURE ON ITS EQUIPMENT DURING THE SECOND BENCHMARK, CENTENNIAL ARGUES THAT THERE WAS NO NEED FOR THIS FEATURE BECAUSE NO DATA WAS EVER IN DANGER OF BEING LOST IN A POWER FAILURE. CENTENNIAL ALLEGES THAT ITS SYSTEM NEVER PLACES DATA IN CACHE MEMORY UNLESS SUCH DATA IS ALSO PERMANENTLY PLACED ON DISK. WE FIND THAT THIS ARGUMENT FAILS TO EXPLAIN HOW CENTENNIAL WAS MUCH FASTER IN PERFORMING ITS "WRITE" OPERATIONS TO CACHE DURING THE SECOND BENCHMARK THAN IT WAS DURING THE FIRST BENCHMARK. FOR EXAMPLE, CENTENNIAL'S AVERAGE PERFORMANCE TIME FOR EACH WRITE OPERATION ON FILE 8 IN THE FOURTH DISK UNIT FELL FROM 184 MILLISECONDS DURING THE FIRST BENCHMARK TO 71 MILLISECONDS DURING THE SECOND BENCHMARK, DESPITE THE FACT THE FILE 8 OPERATIONS HAD TO CONTEND FOR TIME WITH THE FILE 9 WRITE OPERATIONS THAT WERE MORE IN NUMBER THAN FILE 9 WRITE OPERATIONS AND FILE 10 READ OPERATIONS COMBINED DURING THE FIRST BENCHMARK. IF NEW OR UPDATED DATA ENTERED ON THE COMPUTER WAS BEING WRITTEN TO BOTH CACHE AND TO DISK AT THE SAME TIME AS ALLEGED BY CENTENNIAL, THEN THE AVERAGE WRITE OPERATION TIME FOR FILE 8 SHOULD HAVE INCREASED DURING THE SECOND BENCHMARK BECAUSE OF INCREASED NUMBER OF FILE 9 WRITE OPERATIONS WHICH PLACED COMPETING DEMANDS FOR ACCESS TO THE DATA ON THE DISK UNIT. IN OUR OPINION, THE ONLY POSSIBLE EXPLANATION FOR THE REDUCED PERFORMANCE TIME IS THAT CENTENNIAL WAS WRITING DATA SOLELY TO CACHE DURING THE SECOND BENCHMARK. THUS, CENTENNIAL SHOULD HAVE HAD A DATA SAVE FEATURE ON ITS EQUIPMENT AS REQUIRED BY THE RFP.

UNEQUAL TREATMENT OF OFFERORS

CENTENNIAL CONTENDS THAT THE IRS USED DIFFERENT DEFINITIONS OF "ON DISK" IN COMPUTING THE HIT RATIOS OF AMPERIF AND CENTENNIAL. SPECIFICALLY, CENTENNIAL ALLEGES THAT IRS DEFINED "ON DISK" FOR THE COMPANY TO EXCLUDE ALL CACHE IT COUNTS WHILE IT DEFINED "ON DISK" FOR AMPERIF TO INCLUDE ALL "CACHE IT" COUNTS. IN SUPPORT OF THIS ALLEGATION, CENTENNIAL POINTS OUT THAT AMPERIF'S EQUIPMENT HAS THE SAME CACHE IT FEATURE AS CENTENNIAL'S AND THAT THE IRS ADMITTED AT THE CONFERENCE ON ITS PROTEST THAT AMPERIF DID NOT PROVIDE A "PROBE POINT" THAT WOULD PERMIT THE CACHE IT COUNTS TO BE SEPARATED FROM THE REMAINDER OF THE "ON DISK" COUNTS. CENTENNIAL ASSERTS THAT IF THE DEFINITION THE IRS APPLIED TO AMPERIF HAD BEEN APPLIED TO CENTENNIAL, ITS CACHE HIT RATIO WOULD HAVE BEEN COMPUTED TO BE 65.3 PERCENT. AS TO AMPERIF, CENTENNIAL ARGUES THE COMPANY'S HIT RATE WOULD HAVE BEEN 100 PERCENT IF THE IRS HAD CALCULATED IT IN THE SAME MANNER AS IT DID FOR CENTENNIAL BECAUSE THE AMPERIF EQUIPMENT "NORMALLY" BRINGS DATA INTO CACHE AFTER A REQUEST.

THE IRS CATEGORICALLY DENIES ANY UNEQUAL TREATMENT OF THE OFFERORS IN DETERMINING THE CACHE HIT RATE. THE IRS STATES THAT CENTENNIAL AND AMPERIF WERE TREATED EQUALLY WITH RESPECT TO ALL ASPECTS OF THE PROCUREMENT AND THAT THE SAME FORMULA WAS APPLIED IN THE SAME MANNER TO BOTH OFFERORS IN CALCULATING THE CACHE HIT RATE.

WE HAVE ALREADY CONCLUDED THAT IRS INCORRECTLY CALCULATED CENTENNIAL'S CACHE HIT RATE BY COUNTING HITS THAT SHOULD HAVE BEEN MISSES. ASSUMING IRS DID COMPUTE AMPERIF'S RATE DIFFERENTLY BY NOT INCLUDING AS HITS THE TIMES AMPERIF'S CACHE IT FUNCTION WAS EMPLOYED, THIS IS EXACTLY WHAT WE FOUND IRS SHOULD HAVE DONE FOR CENTENNIAL. THEREFORE, AMPERIF'S HIT RATIO WOULD REMAIN THE SAME UNDER THE METHOD OF COUNTING THAT SHOULD HAVE BEEN APPLIED TO CENTENNIAL. WHILE CENTENNIAL'S RATE SHOULD HAVE BEEN COMPUTED AT BELOW 85 PERCENT, THE FACT REMAINS CENTENNIAL WAS PROPERLY REJECTED BECAUSE OF FINE TUNING AND THE LACK OF A DATA SAVE FEATURE.

OPPORTUNITY TO CURE DEFICIENCIES

FINALLY, CENTENNIAL ARGUES THAT THE IRS SHOULD NOT HAVE ELIMINATED IT FROM FURTHER COMPETITION WITHOUT INFORMING IT OF THE DEFICIENCIES IN ITS PROPOSAL AND PROVIDING AN OPPORTUNITY TO CURE SUCH DEFICIENCIES. CENTENNIAL ALLEGES THAT THE IRS SIMPLY INFORMED IT THAT ITS EQUIPMENT DID NOT SATISFY AN RFP REQUIREMENT WITHOUT PROVIDING ANY OPPORTUNITY TO CURE THE MATTER. CENTENNIAL POINTS OUT THAT WE HAVE HELD THAT A DECISION TO EXCLUDE AN OFFEROR FROM THE COMPETITIVE RANGE SHOULD BE CAREFULLY SCRUTINIZED WHEN THE RESULT IS THAT ONLY ONE OFFEROR IS IN THE COMPETITIVE RANGE.

OUR DECISIONS THAT CLOSELY SCRUTINIZE AN AGENCY'S DECISION TO ELIMINATE AN OFFEROR FROM THE COMPETITIVE RANGE INVOLVE PROCUREMENTS WHERE AN OFFEROR INITIALLY WAS EXCLUDED WITHOUT CONDUCTING ANY DISCUSSIONS. SEE DYNALECTRON CORPORATION, B-185027, SEPT. 26, 1976, 76-2 CPD PARA. 267. COMTEN-COMRESS, B-183379, JUNE 30, 1975, 75-1 CPD PARA. 400. HERE, NOT ONLY WERE TWO BENCHMARKS CONDUCTED, THE OFFERORS WERE ALSO ASKED PRIOR TO BENCHMARKING SEVERAL QUESTIONS BY THE IRS CONCERNING THEIR PROPOSALS. FURTHER, THE IRS HAD SEVERAL MEETINGS WITH CENTENNIAL CONCERNING HOW THE BENCHMARKS WOULD BE CONDUCTED. CENTENNIAL'S PROPOSAL WAS NOT REJECTED UNTIL AFTER THE COMPANY HAD FAILED THE SECOND BENCHMARK. FURTHER, WHILE OUR DECISIONS DO CAUTION AGAINST PASS/FAIL BENCHMARKS, WE VIEW CENTENNIAL'S SECOND BENCHMARK AS MERELY THE FINAL PART OF THE IRS'S ONGOING EVALUATION AND DISCUSSION PROCESS IN THIS PROCUREMENT. SEE CONTROL DATA CORPORATION, B-209166.2, DEC. 27, 1983, 84-1 CPD PARA. 21. CONSEQUENTLY, WE FIND THAT CENTENNIAL WAS GIVEN THE OPPORTUNITY TO CORRECT ITS PROPOSAL THROUGH DISCUSSIONS.

CONCLUSION

ACCORDINGLY, WE DENY CENTENNIAL'S PROTEST.