B-186372, SEPTEMBER 22, 1976, 55 COMP.GEN. 1450

B-186372, SEPTEMBER 22, 1976, 55 COMP.GEN. 1450

CONTRACTS - NEGOTIATION - OFFERS OR PROPOSALS - BEST AND FINAL - REVISED PROPOSAL SUBMITTED - REOPENING OF NEGOTIATIONS NOT REQUIRED AGENCY WAS NOT REQUIRED TO SEEK FURTHER CLARIFICATION IN NEGOTIATED PROCUREMENT WHERE PROTESTER SUBSTANTIALLY REVISED BUILDING DESIGN IN BEST AND FINAL OFFER AND FAILED TO SUPPORT SUCH CHANGE WITH ADEQUATE DOCUMENTATION. IN SUCH CIRCUMSTANCES CONTRACTING OFFICER NEED NOT REOPEN NEGOTIATIONS BUT MAY LOWER HIS RATING OF FINAL PROPOSAL SUBMITTED. CONTRACTS - NEGOTIATION - EVALUATION FACTORS - PRICE ELEMENTS FOR CONSIDERATION - ANTICIPATED COSTS FACT THAT PROTESTER WOULD HAVE TO ABSORB ALL DIRECT COSTS EXCEEDING ITS CEILING PRICE IN FIXED PRICE INCENTIVE CONTRACT DOES NOT NEGATE EVALUATOR'S LEGITIMATE CONCERN FOR ANTICIPATED COSTS OVER CEILING CONSIDERING PERFORMANCE AND ADMINISTRATION PROBLEMS WHICH REASONABLY CAN BE EXPECTED TO RESULT FROM CONTRACTOR'S LOSS POSITION. CONTRACTS - NEGOTIATION - EVALUATION FACTORS - FACTORS OTHER THAN PRICE - TECHNICAL ACCEPTABILITY SOURCE SELECTION AUTHORITY'S CONCLUSION THAT PROTESTOR'S LOWER TARGET AND CEILING PRICES FOR FIXED PRICE INCENTIVE CONTRACT OFFERED LITTLE IN WAY OF ADVANTAGES TO GOVERNMENT AND WERE NOT SUFFICIENTLY SIGNIFICANT TO OVERCOME SELECTED FIRM'S SUPERIORITY IN TECHNICAL AND OPERATIONS AREA IS SUPPORTED BY RECORD AND IS CONSISTENT WITH EVALUATION CRITERIA WHICH GAVE MORE WEIGHT TO TECHNICAL AND OPERATIONS AREA. PROTESTER'S CONTENTION THAT NEGOTIATIONS WERE NOT CONDUCTED IN COMPLIANCE WITH 10 U.S.C. 2304(G) IS DENIED.

IN THE MATTER OF THE GENERAL ELECTRIC COMPANY, SEPTEMBER 22, 1976:

##HC1451

THE GENERAL ELECTRIC COMPANY (GE) PROTESTS THE AWARD OF A CONTRACT TO THE RAYTHEON COMPANY (RAYTHEON) PURSUANT TO REQUEST FOR PROPOSALS (RFP) NO. F1968-75-R-0014 ISSUED BY THE ELECTRONIC SYSTEMS DIVISION (ESD) OF THE UNITED STATES AIR FORCE. THE SOLICITATION COVERED A PHASED ARRAY WARNING SYSTEM CALLED PAVE PAWS. THE CONTRACTOR IS TO DESIGN, DEVELOP, FABRICATE, INSTALL AND TEST TWO DUAL FACED PHASED ARRAY RADARS, ONE EACH ON THE EAST AND WEST COASTS, FOR THE DETECTION OF SUBMARINE LAUNCHED MISSILES. THE CONTRACTOR ALSO IS REQUIRED TO DESIGN AND CONSTRUCT A BUILDING AT EACH SITE TO HOUSE THE COMPUTER BASED RADARS. THE RFP SPECIFIED A FIXED PRICE INCENTIVE FEE (FPIF) TYPE OF CONTRACT. A SHARING ARRANGEMENT OF 80/20 (GOVERNMENT/CONTRACTOR) FOR COSTS OVER OR UNDER THE TARGET PRICE, A FIRM CEILING PRICE OF 130 PERCENT OF TARGET COSTS AND A FEE OF 10 PERCENT OF TARGET COSTS WERE ALSO SPECIFIED.

THE RFP PROVIDED THAT AWARD WOULD BE MADE TO THE OFFEROR WHOSE PROPOSAL WAS MOST ADVANTAGEOUS TO THE GOVERNMENT, COST AND OTHER FACTORS CONSIDERED. PRIMARY EMPHASIS DURING PROPOSAL EVALUATION WAS TO BE PLACED ON TECHNICAL AND OPERATIONAL EXCELLENCE. COST WAS THE SECOND MOST IMPORTANT SELECTION FACTOR AND MANAGEMENT AND LOGISTICS WERE CONSIDERED THIRD IN THE ORDER OF PRIORITY. IN ADDITION, THE RFP INDICATED THAT AN OFFEROR WOULD BE FAVORABLY EVALUATED FOR THE USE OF PROVEN TECHNICAL PROCESSES AND OFF-THE-SHELF COMPONENTS AND MATERIALS IN ITS HARDWARE AND SOFTWARE DESIGN APPROACH. THE RFP WARNED THAT ANY SIGNIFICANT INCONSISTENCY BETWEEN PROMISED PERFORMANCE AND PRICE, IF UNEXPLAINED, WOULD RAISE AN ISSUE REGARDING THE OFFEROR'S UNDERSTANDING OF THE WORK AND COULD RESULT IN REJECTION OF SUCH PROPOSAL AND THAT THE "BURDEN OF PROOF" AS TO COST CREDIBILITY RESTED WITH THE OFFEROR. THE INSTRUCTIONS ALSO REQUIRED THAT THE PROPOSALS PROVIDE SUFFICIENT DETAIL FOR EVALUATION AND OFFERORS WERE WARNED THAT NONCONFORMANCE WITH THE SPECIFIED CONTENT COULD BE CAUSE FOR REJECTION.

THREE COMPANIES INCLUDING GE AND RAYTHEON SUBMITTED PROPOSALS. ALL OFFERORS WERE DETERMINED TO BE WITHIN THE COMPETITIVE RANGE AND ORAL AND WRITTEN DISCUSSIONS WERE CONDUCTED WITH THEM UNTIL NEGOTIATIONS WERE CLOSED ON FEBRUARY 13, 1976. THESE DISCUSSIONS CONSISTED OF DEFICIENCY REPORTS AND CLARIFICATION REQUESTS FROM THE SOURCE SELECTION EVALUATION BOARD (SSEB) TO THE OFFERORS, THEIR WRITTEN RESPONSES THERETO AND FACE-TO- FACE NEGOTIATIONS. ARMED SERVICES PROCUREMENT REGULATION (ASPR) SEC. 3- 805.3(A) DEFINES A DEFICIENCY AS THAT "PART OF AN OFFEROR'S PROPOSAL WHICH WOULD NOT SATISFY THE GOVERNMENT'S REQUIREMENTS." THE AIR FORCE DEFINES A CLARIFICATION REQUEST AS RELATING EITHER TO AN AREA IN A PROPOSAL WHICH IS UNCLEAR OR TO SOLICITATION REQUIREMENTS WHICH THE OFFEROR MAY HAVE MISUNDERSTOOD.

ALL DEFICIENCY REPORTS CONCERNING GE AND RAYTHEON WERE RESOLVED WHEN THE SSEB CLOSED NEGOTIATIONS AND REQUESTED ON FEBRUARY 13, 1976 THAT BEST AND FINAL OFFERS BE SUBMITTED NO LATER THAN FEBRUARY 27, 1976. THE SSEB PRESENTED ITS FINDINGS TO THE SOURCE SELECTION ADVISORY COUNCIL (SSAC) WHICH, IN TURN, PROVIDED THE SOURCE SELECTION AUTHORITY (SSA) WITH ITS FINDINGS AND A RISK ANALYSIS OF THE THREE PROPOSALS IN THEIR FINAL FORM. ON MARCH 22, 1976, THE MEMBERS OF THE SSAC UNANIMOUSLY RECOMMENDED TO THE SSA THAT AWARD BE MADE TO RAYTHEON. THE SSA DECISION DOCUMENT WAS SIGNED ON APRIL 2, 1976 AND ON APRIL 12, 1976, A CONTRACT FOR ONE RADAR SITE WAS AWARDED TO RAYTHEON.

ON APRIL 14, 1976, GE RECEIVED NOTICE OF AWARD WHICH ADVISED OF THE WEAKNESSES AND RISKS IN ITS PROPOSAL AS FOLLOWS:

A. THE SIZE AND COMPLEXITY OF THE SOFTWARE EFFORT WAS UNDERESTIMATED AND REFLECTED A SERIOUS WEAKNESS IN AN AREA WHERE THE GREATEST PROGRAM RISK IS IDENTIFIED.

B. CHANGES IN THE TECHNICAL FACILITY AS OUTLINED IN THE BAFO (BEST AND FINAL OFFER) DID NOT PROVIDE SUFFICIENT DETAIL FOR GOVERNMENT EVALUATION.

C. THE RISK IN THESE AREAS REFLECTED CONCERN AS TO THE COST REALISM OF (GE'S) BEST AND FINAL OFFER.

THEREAFTER, GE REQUESTED AND OBTAINED A DEBRIEFING. THE SSEB'S MINUTES OF THIS DEBRIEFING INDICATE THAT GE WAS TOLD THAT THE COMPETITION HAD BEEN INTENSE, THE OVERALL PROPOSAL EVALUATION RESULTS WERE CLOSE AND THAT GE'S SOFTWARE DESIGN AND BUILDING DESIGN SIGNIFICANTLY IMPACTED THE EVALUATION OF TECHNICAL RISK. BECAUSE OF MAJOR DESIGN CHANGES PROPOSED BY GE IN THE DATA PROCESSING AREA, THE SSEB HAD REEVALUATED GE'S PROPOSAL AFTER THE ORAL NEGOTIATIONS AND HAD DETERMINED THAT SOME WEAKNESSES IN THE SOFTWARE SUBSYSTEM STILL REMAINED. THE SSEB STATED THAT FURTHER DISCUSSIONS WERE NOT CONDUCTED TO AVOID TECHNICAL LEVELLING WHICH COULD RESULT FROM ANY FURTHER DESIGN CHANGES. THE AIR FORCE INFORMED GE OF ITS BELIEF THAT THE FIRM HAD MADE A MAJOR CHANGE IN SOFTWARE SIZING IN ITS BEST AND FINAL OFFER. SPECIFICALLY, THE AIR FORCE CLAIMED THAT 79,000 EXISTING INSTRUCTIONS WOULD BE TAKEN FROM COBRA TALON AND SITE DEFENSE RADARS AND WOULD MORE THAN DOUBLE THE PREVIOUS SOFTWARE SIZE OF 50,000 MISSION REQUIRED INSTRUCTIONS. THE AIR FORCE STATED THAT THIS CHANGE WAS NOT SUBSTANTIATED WITH ENOUGH DETAIL TO SHOW THAT THE INSTRUCTIONS COULD BE USED IN PAVE PAWS WITHOUT CONVERSION OR MODIFICATION AND ADDITIONAL COSTS.

WITH RESPECT TO THE BUILDING DESIGN CHANGES PROPOSED IN GE'S BEST AND FINAL OFFER, THESE CHANGES WERE VIEWED AS SUBSTANTIAL AND THE AIR FORCE WAS CONCERNED BECAUSE ONLY VERY LIMITED SUPPORTING DETAIL HAD BEEN PROVIDED. THE AIR FORCE ADVISED THAT IT WAS NOT POSSIBLE TO FULLY ASSESS AND SUBSTANTIATE THE ADEQUACY OF THE NEW DESIGN OR TO TRACE TECHNICALLY THE SUBSTANTIAL COST SAVINGS ALLEGED TO RESULT FROM THE NEW FACILITY DESIGN.

GE THEN PROTESTED THE SELECTION OF RAYTHEON TO THIS OFFICE, CONTENDING THAT ITS FINAL PRICE WAS SIGNIFICANTLY LOWER THAN RAYTHEON'S AND THAT GE'S PROPOSAL WAS TECHNICALLY SUPERIOR IN MOST AREAS. GE CONTENDS THAT THE AIR FORCE FAILED TO COMMUNICATE ITS CONCERN OR NEGOTIATE REGARDING THE SIZE AND COMPLEXITY OF GE'S PROPOSED SOFTWARE AND THAT THE AIR FORCE DID NOT TREAT THIS AS A PROPOSAL DEFICIENCY EVEN THOUGH THE AIR FORCE HAD IDENTIFIED THE SIZE AND COMPLEXITY OF SOFTWARE AS A PROGRAM RISK. MOREOVER, GE ARGUES THAT THE CONCERN OF THE AIR FORCE OVER TECHNICAL RISK IN GE'S SOFTWARE INSTRUCTION COUNT WAS UNFOUNDED BECAUSE GE'S RADAR SYSTEM DESIGN WHICH USED PROVEN TECHNIQUES AND PROCESSES REDUCED THE NUMBER OF SOURCE INSTRUCTIONS NEEDED. ACCORDINGLY, GE CONTENDS THAT THE AIR FORCE DID NOT DISCHARGE ITS RESPONSIBILITY UNDER 10 U.S.C. 2304(G) (1970) TO CONDUCT MEANINGFUL DISCUSSIONS CONCERNING THE WEAKNESSES AND TECHNICAL RISKS IDENTIFIED IN THE FIRM'S PROPOSAL.

IN ADDITION, GE ARGUES THAT THE CHANGES IN ITS FACILITY DESIGN AS PROPOSED IN THE FIRM'S BEST AND FINAL OFFER WERE NOT MASSIVE AS STATED BY THE AIR FORCE. THE PROTESTOR BELIEVES THAT ENOUGH INFORMATION WAS CONTAINED IN ITS BEST AND FINAL OFFER TO PERMIT THE GOVERNMENT TO EVALUATE THE CHANGES MADE IN THE FIRM'S FINAL FACILITY DESIGN. BECAUSE THE BUILDING WAS NOT DESIGNATED AS A HIGH RISK AREA, GE DOES NOT BELIEVE THAT ANY LACK OF DETAIL IN THIS REGARD IS A REASON FOR AWARDING THE CONTRACT TO RAYTHEON.

THE GE TECHNICAL FACILITY (BUILDING)

THE INITIAL BUILDING DESIGN PROPOSED BY GE WAS EVALUATED TO DETERMINE ITS STRUCTURAL, ELECTRICAL, MECHANICAL, ARCHITECTURAL AND FACILITY INTERFACE ADEQUACY. DURING NEGOTIATIONS, THE SSEB ISSUED TO GE 6 DEFICIENCY REPORTS AND 26 CLARIFICATION REQUESTS REGARDING THE BUILDING DESIGN. GE'S RESPONSE THERETO WERE EVALUATED AS SATISFACTORY AND THE PROPOSED DESIGN WAS CONSIDERED ADEQUATE TO MEET THE REQUIREMENTS. GE'S INITIAL PRICE FOR THE BUILDING WAS $16.5 MILLION. BEFORE RECEIVING BEST AND FINAL OFFERS THE SSEB PROJECTED THAT THE PRICE FOR GE'S FACILITY, AS NEGOTIATED, WOULD BE $19.4 MILLION BECAUSE OF THE MODIFICATIONS MADE DURING NEGOTIATIONS AND THE STATEMENT OF GE. THERE IS NO QUESTION AS TO THE ADEQUACY OF THE MEANINGFUL DISCUSSIONS REGARDING THE BUILDING DESIGN INITIALLY PROPOSED.

THE CALL FOR BEST AND FINAL OFFERS REQUIRED THAT A RECONCILIATION BY COST ELEMENT BE PERFORMED FOR EACH PROPOSED LINE ITEM AND SUB-LINE ITEM FOR WHICH A DD FORM 633 HAD BEEN SUBMITTED. IT FURTHER REQUIRED THAT A NARRATIVE DESCRIPTIVE DESCRIPTION OF THE SPECIFIC CHANGES OVER $24,999 BE ATTACHED TO THE LINE ITEM AND SUB-LINE ITEM RECONCILIATIONS. IT ADVISED THAT ANY TECHNICAL REVISIONS OR NONCONCURRENCES TO CONTRACT TERMS AND CONDITIONS SUBMITTED IN THE BEST AND FINAL OFFER WOULD NOT BE NEGOTIATED FURTHER. GE'S BEST AND FINAL OFFER STATED THAT BECAUSE OF "KEY CHANGES" IN THE STRUCTURAL DESIGN AND OPERATIONAL LAYOUT MADE BY A NEW ARCHITECTURAL AND ENGINEERING SUBCONTRACTOR AND A NEW CONSTRUCTION SUBCONTRACTOR, IT WAS ABLE TO REDUCE ITS BUILDING PRICE TO $10.7 MILLION. THIS REPRESENTED A REDUCTION OF $5.8 MILLION OR 35 PERCENT FROM ITS ORIGINAL PRICE FOR THIS ITEM.

THE AIR FORCE AND GE DISAGREES AS TO THE ADEQUACY OF THE TECHNICAL DESCRIPTION AND DETAIL SUBMITTED BY GE IN SUPPORT OF ITS CHANGES. THE AIR FORCE ASSERTS THAT THE SUPPORTING TECHNICAL INFORMATION INCLUDED ONLY A HALF PAGE DESCRIPTION, A SKETCH OF THE NEW DESIGN COMPARED TO THE OLD DESIGN, AND COST INFORMATION, ALL OF WHICH WERE INSUFFICIENT TO PERMIT PROPER EVALUATION OF THE PROPOSED CHANGE. THE AIR FORCE CLAIMS IT NEEDED, AMONG OTHER THINGS, DESIGN INFORMATION REGARDING ELECTRICAL LOADS, AIR CONDITIONING, CONSTRUCTION MATERIALS, PLUMBING, FIRE PROTECTION EQUIPMENT, AND LAYOUT OF THE TACTICAL OPERATIONS AND OTHER ROOMS. IN ADDITION, THE CHANGE OF SUBCONTRACTORS WITH THE LARGE PRICE REDUCTION RAISED QUESTIONS AS TO HOW WELL THE NEW FIRMS UNDERSTOOD THE LEVEL OF EFFORT REQUIRED.

GE DENIES THAT THE BUILDING DESIGN CHANGE WAS AS EXTENSIVE AS THE AIR FORCE CLAIMS AND CONTENDS THAT THE NEW DESIGN RETAINED THE SAME SQUARE FOOTAGE OF ROOM AREA AND THE SAME FUNCTIONAL RELATIONSHIPS AS THE OLD DESIGN. GE ASSERTS THAT IT DID NOT SUBMIT INFORMATION REGARDING THE SUPPORT FACILITIES BECAUSE THEY WERE NOT CHANGED AND THAT, BY THE TERMS OF THE CALL FOR BEST AND FINAL OFFERS, SUCH INFORMATION WAS NOT REQUIRED. FURTHER ALLEGES THAT THE BUILDING INVOLVED NO RESEARCH AND DEVELOPMENT EFFORT AND DID NOT REPRESENT A MAJOR RISK. THIS, IT CLAIMS, IS EVIDENCED BY THE FACT THAT THE SSEB, IN EVALUATING GE'S PROPOSED PRICE, ADDED NO FACTOR TO COVER THE FINANCIAL IMPACT OF THE RISK IT CLAIMED TO SEE IN THE NEW BUILDING DESIGN EVEN THOUGH THE RFP STATED THAT THE FINANCIAL IMPACT OF HIGH RISK AREAS WOULD BE ADDED TO THE OFFEROR'S PROPOSAL TO DEVELOP THE MOST PROBABLE COST TO THE GOVERNMENT.

WE HAVE REVIEWED THE RECORD AS IT PERTAINS TO THE BUILDING DESIGN PROPOSED BY GE IN ITS BEST AND FINAL OFFER AND THE INITIAL DESIGN AS IT WAS AFTER ORAL NEGOTIATIONS. OUR CONCLUSION IS THAT THE DETERMINATION OF THE SSEB WAS REASONABLE. WE ARE IN AGREEMENT WITH THE SSEB THAT THE TECHNICAL EXPLANATION AND SUPPORT WAS SO MEAGER THAT THE ADEQUACY OF THE NEW DESIGN COULD NOT HAVE BEEN DETERMINED WITHOUT ADDITIONAL DISCUSSIONS AND DOCUMENTARY SUPPORT. ALTHOUGH MUCH COST INFORMATION WAS SUBMITTED, IT WAS ESSENTIALLY USELESS BECAUSE THE ADEQUACY OF THE BUILDING ITSELF COULD NOT BE ASSESSED. THE SSEB PROJECTED A FIGURE OF $1.25 MILLION IN EVALUATING THE NEW DESIGN, BUT IT NEVER TRANSLATED ITS FULL CONCERN ABOUT THE NEW DESIGN INTO A COMPLETE PRECISE DOLLAR AMOUNT TO BE ADDED TO THE GE PROPOSED BUILDING PRICE. ALTHOUGH IT MAY BE TRUE, AS GE CONTENDS, THAT THERE WAS LITTLE, IF ANY, RESEARCH AND DEVELOPMENT INVOLVED WITH THE BUILDING, THE SSEB WAS PROPERLY CONCERNED ABOUT THE FINANCIAL IMPACT UPON PAVE PAWS RESULTING FROM POSSIBLE CONSTRUCTION COST OVERRUNS AND DELIVERY SLIPPAGES.

BASED ON THE RECORD BEFORE US, WE FIND THE TWO GE DESIGNS SUBSTANTIALLY DIFFERENT AND WE DOUBT THAT THE AIR CONDITIONING, PLUMBING, FIRE PROTECTION EQUIPMENT, SOIL SAMPLES, FOUNDATIONS AND OTHER SUPPORT FACILITIES AND THE ANALYSIS FOR THE OLD BUILDING WOULD BE APPROPRIATE FOR THE NEW DESIGN. THE INITIAL BUILDING DESIGN, AS NEGOTIATED, INVOLVED SUPPORTING FIVE FLOORS ON A LEVEL PLATEAU. THE NEW DESIGN INCORPORATED A SIGNIFICANT CONCEPT CHANGE BY SUPPORTING THREE FLOORS ON THE PLATEAU AND TWO FLOORS ON THE SLOPE OF THE TERRAIN. NO DOCUMENTATION WAS PRESENTED TO SHOW THAT THE SOIL AND DRAINAGE CONDITIONS WERE SUCH THAT THIS CHANGE WOULD BE ACCEPTABLE. ALTHOUGH THE STRUCTURE WAS CONCEPTUALLY SHOWN TO REST ON THE RETAINING WALL FOUNDATION, THERE WAS NO INFORMATION AS TO THE CONNECTION BETWEEN THE SUPERSTRUCTURE AND THE SUBSTRUCTURE. SINCE IT IS TECHNICALLY AND FINANCIALLY IMPRACTICAL TO DESIGN FOR NO SOIL MOVEMENT, THE INTERACTION BETWEEN THE SOIL, FOUNDATION AND STRUCTURE IS A MATTER OF PARAMOUNT IMPORTANCE ESPECIALLY WHERE SOIL WOULD EXIST ABOUT THE LOWER PART OF THE RADAR FACE. THERE WAS NO INDICATION AS TO HOW POSSIBLE SEEPAGE PROBLEMS ON THE LOWER FLOORS COULD BE AVOIDED OR WHETHER THE SOIL WAS AMENABLE FOR SUCH CONVENTIONAL SOLUTIONS AS WATERPROOFING, WEEP HOLES AND DRAINAGE PIPES. IT WOULD BE FUTILE TO TRANSFER THE STRUCTURAL ANALYSIS, DIMENSIONS AND MEMBER SIZES FROM THE NOVEMBER DESIGN TO THAT PROPOSED IN THE BEST AND FINAL OFFER. IN OUR OPINION, THE DOCUMENTATION WAS INSUFFICIENT TO DETERMINE THE ACCEPTABILITY OF THE NEW DESIGN AND THERE WAS VERY LITTLE INFORMATION FROM THE PREVIOUSLY NEGOTIATED DESIGN WHICH COULD BE TRANSFERRED FOR THIS EVALUATION. WE AGREE WITH THE AIR FORCE THAT IN THE CIRCUMSTANCES IT WAS IMPOSSIBLE TO INCLUDE AN AMOUNT IN ITS MOST PROBABLE COST TO THE GOVERNMENT TO COVER THE RISK INHERENT IN GE'S NEW BUILDING DESIGN. THE FAILURE TO DO SO REFLECTS, IN OUR OPINION, ON THE INADEQUACY OF THE PROPOSAL RATHER THAN THE NONEXISTENCE OF RISK. EVEN THOUGH THE BUILDING WOULD NOT REQUIRE RESEARCH AND DEVELOPMENT, OTHER RISKS EXISTED AS INDICATED ABOVE. IN THE CIRCUMSTANCES, THE EVALUATION BOARD WAS NOT REQUIRED TO REOPEN NEGOTIATIONS AFTER RECEIPT OF BEST AND FINAL OFFERS TO OBTAIN EXTENSIVE SUPPORTING INFORMATION WHICH GE SHOULD HAVE SUBMITTED WITH ITS NEW DESIGN. THE BURDEN IS ON THE OFFEROR IN SUBMITTING ITS BEST AND FINAL OFFER TO AFFIRMATIVELY DEMONSTRATE ITS MERITS. WHEN AFTER CLOSE OF NEGOTIATIONS AN OFFEROR SUBMITS A REVISED PROPOSAL WITHOUT SUCH SUBSTANTIATION, THE CONTRACTING OFFICER NEED NOT REOPEN NEGOTIATIONS BUT MAY LOWER HIS RATING OF THE FINAL PROPOSAL SUBMITTED. ELECTRONICS COMMUNICATIONS, INC., 55 COMP.GEN. 636 (1976), 76-1 CPD 15. DECISION SCIENCES CORPORATION, B-184438, AUGUST 3, 1976, 76-2 CPD 114.

TECHNICAL AND OPERATIONS AREA

THE RFP REQUIRED OFFERORS TO PREPARE AND SUBMIT A "COST FORMAT A" WHICH WAS TO SHOW THE SOFTWARE DEVELOPMENT TASK BREAKDOWN. THIS DATA WOULD BE USED TO ASSESS THE REASONABLENESS OF THE PROPOSED COMPUTER PROGRAMMING EFFORT AS COMPARED TO AN INDEPENDENT GOVERNMENT ESTIMATE. THE DETAILED INSTRUCTIONS FOR "COST FORMAT A" STATED THAT THE OFFEROR SHOULD INDICATE "THE ESTIMATED TOTAL NUMBER OF SOURCE LANGUAGE STATEMENT/INSTRUCTIONS," INCLUDING THE SIZE OF EXISTING PROGRAMS WHICH IT ANTICIPATED WOULD BE USED TO FULFILL THE PERFORMANCE REQUIREMENTS. FURTHER, IT REQUIRED THAT THE OFFEROR INDICATE THE NEW SOURCE CODE WHICH WAS TO BE GENERATED, THE ESTIMATED NUMBER OF INSTRUCTIONS REQUIRED TO BE CHANGED OR DELETED AND THE NUMBER OF SOURCE INSTRUCTIONS WHICH WERE TO BE CONVERTED FROM ONE SOURCE LANGUAGE TO ANOTHER OR FROM ONE MACHINE TO ANOTHER. THE OFFERORS WERE INSTRUCTED TO INDICATE THE SOURCE LANGUAGE IN WHICH THE PROGRAM WAS TO BE WRITTEN AND IT WRITTEN IN A MIXTURE OF SOURCE LANGUAGES TO INDICATE THE APPROXIMATE FRACTION FOR EACH LANGUAGE.

THE INITIAL GE PROPOSAL, AS AMENDED, PROVIDED FOR 141,356 INSTRUCTIONS, ALL OF WHICH WOULD BE NEWLY DEVELOPED, MODIFIED OR CONVERTED FOR PAVE PAWS. GE'S INSTRUCTION COUNT, HOWEVER, INCLUDED MACHINE LANGUAGE INSTRUCTIONS CONTRARY TO THE DIRECTIONS ON COST FORMAT A FOR A STATEMENT OF SOURCE LANGUAGE INSTRUCTIONS. BECAUSE THERE CAN BE A RATIO OF UP TO FIVE MACHINE LANGUAGE INSTRUCTIONS FOR EACH SOURCE LANGUAGE INSTRUCTION, THIS INITIALLY MISLED THE SSEB TO BELIEVE THAT GE PROPOSED SUBSTANTIALLY MORE SOURCE LANGUAGE INSTRUCTIONS THAN GE INTENDED, THEREBY DELAYING THE AIR FORCE'S AWARENESS OF THE WEAKNESS ULTIMATELY PERCEIVED IN GE'S SOFTWARE PACKAGE.

AFTER THE EVALUATION OF THE INITIAL PROPOSAL, AS AMENDED, THE SSEB SENT TO GE 28 DEFICIENCY REPORTS AND 120 CLARIFICATION REQUESTS. FOUR OF THESE RELATED TO THE SOFTWARE BUT DID NOT EXPRESSLY CONCERN THE ADEQUACY OF THE SOFTWARE INSTRUCTION COUNT. DEFICIENCY REPORT 14-B STATED THAT THE COMPUTER MEMORY SIZING ANALYSIS WAS NOT SUFFICIENTLY DESCRIBED TO PROVIDE CREDIBILITY TO PERFORM THE MISSION OF PAVE PAWS AND THAT IN COMPARISON TO OTHER PHASED ARRAY SYSTEMS, THE AMOUNT OF MEMORY ALLOCATED TO DO THE FUNCTION WAS "GRAVELY UNDERESTIMATED." GE'S RESPONSE OF SEPTEMBER 11, 1975, TO DEFICIENCY REPORT 14-B CONSISTED OF 25 PAGES AND WAS PREFACED BY THE FOLLOWING:

THE COMPUTER MEMORY ALLOCATION AND MEMORY MANAGEMENT REQUIREMENTS FOR THE PAVE PAWS SYSTEM WERE POORLY DESCRIBED IN THE GE PROPOSAL. TYPOGRAPHICAL ERRORS, OMISSIONS FROM KEY TABLES, AND THE LACK OF A SINGLE LOCALIZED DISCUSSION OF MEMORY HAVE CAUSED CONCERN FOR THE CREDIBILITY OF THE MEMORY ESTIMATES.

THIS RESPONSE WAS EVALUATED BY THE SSEB AS PROVIDING INSUFFICIENT DATA LEAVING SEVERAL POINTS FOR FURTHER NEGOTIATIONS. AFTER THE EVALUATION OF THE SECOND WRITTEN RESPONSE, GE HAD SATISFIED THE SSEB EXCEPT THAT:

COST FORMAT A WAS SUPPOSED TO BE BROKEN DOWN BY SOURCE LANGUAGE, NOT MACHINE LANGUAGE. THIS IS STILL INADEQUATE TO TRACK COST. TOPIC WILL BE ADDRESSED DURING FORMAL (ORAL) NEGOTIATIONS.

THE MINUTES FOR THE ORAL NEGOTIATIONS OF OCTOBER 30, 1975, WITH REGARD TO DEFICIENCY REPORT 14-B STATED THAT "COST FORMAT A" SHOULD BE IN SOURCE LANGUAGE AND THAT THE CONTRACTOR WOULD DISCUSS THIS AT THE NEXT MEETING. THE MINUTES FOR THE ORAL NEGOTIATION OF NOVEMBER 12, 1975 CONTAIN THE FOLLOWING ENTRY: "DR 14-B COST FORMAT A DATA WAS PRESENTED IN SOURCE CODE VS MACHINE LANGUAGE." GE'S LETTER OF NOVEMBER 11, 1975 INFORMED THE SSEB THAT THE COST IMPACT OF THE MODIFICATIONS MADE IN RESPONSE TO THE DEFICIENCY REPORT WOULD BE AN INCREASE OF $2.6 MILLION. (THESE MODIFICATIONS INCLUDED A SUBSTANTIAL INCREASE IN PROPOSED HARDWARE.)

THE AIR FORCE CONTENDS THAT ALTHOUGH ITS OVERALL CONCERN ABOUT THE GE SOFTWARE DESIGN APPROACH WAS THOROUGHLY DISCUSSED THROUGHOUT THE EVALUATION PROCESS, THE INADEQUACY OF THE NUMBER OF SOFTWARE INSTRUCTIONS WAS NEVER POINTED OUT TO GE AS A DEFICIENCY OR WEAKNESS. IT ARGUES THAT INITIALLY IT WAS MISLED INTO BELIEVING THAT A LARGE PORTION OF THE SOFTWARE INSTRUCTIONS WERE APPROXIMATELY FIVE TIMES GREATER THAN THEY ACTUALLY WERE BECAUSE GE HAD SUBMITTED WITHOUT EXPLANATION ITS COUNT IN MACHINE LANGUAGE RATHER THAN SOURCE LANGUAGE AS REQUIRED BY THE RFP. THE AIR FORCE STATES THAT THE SYSTEM SPECIFICATIONS DID NOT REQUIRE A MINIMUM NUMBER OF INSTRUCTIONS. MOREOVER, THE AGENCY'S OVERALL ESTIMATE OF SOFTWARE INSTRUCTIONS HAD BEEN PREPARED PREVIOUSLY ONLY FOR PRICING PURPOSES AND THEREFORE WAS NOT USED BY THE SSEB IN ITS EVALUATION OF THE ADEQUACY OF THE SOFTWARE COUNT. IT APPEARS THAT AFTER THERE NEGOTIATION SESSIONS WITH GE, THE BOARD CONCLUDED THAT GE'S SOFTWARE COUNT WAS LOW ON THE BASIS OF ITS MEMBERS' GENERAL BACKGROUND KNOWLEDGE AND EXPERTISE IN SUCH MATTERS. CONTRARY TO GE'S INSISTENCE, THERE IS NO CLEAR EVIDENCE IN THE RECORD THAT A CREDIBLE ESTIMATE OF INSTRUCTIONS REQUIRED IN EACH FUNCTIONAL AREA EXISTED PRIOR TO THE SSAC'S PRESENTATION OF ITS EVALUATION TO THE SSA AFTER RECEIPT OF BEST AND FINAL OFFERS. THE SSA THEN INSISTED UPON SUCH AN ESTIMATE AS AN ADDITIONAL MEASURE OF CREDIBILITY CONCERNING THE RECOMMENDED REJECTION OF GE.

THE SYSTEM SPECIFICATION WHICH DESCRIBES THE REQUIREMENTS FOR THE VARIOUS SOFTWARE CATEGORIES STATES WITH RESPECT TO SUPPORT SOFTWARE THAT ALL COMMERCIALLY AVAILABLE SUPPORT SOFTWARE SHALL BE PROVIDED WITH THE DELIVERABLE SOFTWARE PACKAGE. THE SSEB DETERMINED THAT GE'S PROPOSAL INCLUDED 48,961 DEVELOPED, MODIFIED OR CONVERTED INSTRUCTIONS FOR THE MISSION SOFTWARE, 79,250 EXISTING INSTRUCTIONS FOR THE SUPPORT SOFTWARE AND 14,000 EXISTING INSTRUCTIONS FOR THE OPERATING SYSTEM SOFTWARE FOR A TOTAL OF 144,211. THE SSEB ASSUMED THAT 79,750 INSTRUCTIONS WERE EXISTING SUPPORT SOFTWARE TO BE SUPPLIED BY THE COMPUTER VENDORS BECAUSE THE INSTRUCTIONS WERE NOT IDENTIFIED AS CONTAINING GE EXISTING SOFTWARE. THE 48,961 FIGURE FOR MISSION SOFTWARE WAS FAR BELOW THAT OFFERED BY GE'S COMPETITORS AND OTHER EXISTING RADAR SYSTEMS.

GE ASSERTS THAT 16,000 INSTRUCTIONS WERE COMPUTER VENDOR SUPPLIED EXISTING INSTRUCTIONS. IT STATES THAT IF THESE 16,000 INSTRUCTIONS ARE SUBTRACTED FROM THE TOTAL INSTRUCTIONS OF 144,211, ITS NOVEMBER PROPOSAL INDICATED THAT A TOTAL OF 128,211 INSTRUCTIONS WOULD BE DELIVERED BY GE. THUS, GE STATES THAT THE 132,576 INSTRUCTIONS PROPOSED IN ITS BEST AND FINAL OFFER REPRESENTED AN INCREASE OF ONLY 4,365 INSTRUCTIONS. GE ALSO ARGUES THAT ITS NOVEMBER PROPOSAL SHOWED THAT TOTAL OF 79,750 SUPPORT INSTRUCTIONS WOULD BE DELIVERED, OF WHICH 500 WOULD BE DEVELOPED AND 79,250 WOULD BE EXISTING INSTRUCTIONS. GE CLAIMS THAT UNLIKE THE "EXECUTIVE" FUNCTION SOFTWARE, THESE EXISTING SOFTWARE INSTRUCTIONS WERE NOT COMPUTER VENDOR SUPPLIED BUT WERE EXISTING GE SOFTWARE WHICH COULD BE USED ON PAVE PAWS. HOWEVER, IN OUR OPINION, THE INFORMATION ON GE'S "COST FORMAT A" PROVIDES NO BASIS ON WHICH THE SSEB COULD HAVE REACHED SUCH A CONCLUSION. WHILE THE ANTICIPATED USE OF EXISTING SOURCE INSTRUCTIONS WHICH WOULD NOT BE DEVELOPED, MODIFIED OR CONVERTED FOR PAVE PAWS COULD BE DETERMINED FROM COST FORMAT A, THERE WAS NO WAY TO DISTINGUISH BETWEEN THOSE EXISTING INSTRUCTIONS THAT WERE SUPPLIED BY THE COMPUTER VENDOR AND THOSE THAT WERE GE INSTRUCTIONS FROM OTHER PROGRAMS.

GE'S RESPONSE TO DEFICIENCY REPORT 14-B REFERRED TO "ADAPTABLE" SOFTWARE MODULES WHICH COULD BE USED ON PAVE PAWS. IT STATED THE DESIRABILITY OF REPROGRAMMING SUCH MODULES TO JOVIAL COMPUTER LANGUAGE AND ALSO STATED THAT THEY COULD BE DEVELOPED WITH LITTLE RISK IN TERMS OF PERFORMANCE, SCHEDULE, COST AND INTERFACE. HOWEVER, GE DID NOT SUPPLY ANY TECHNICAL DOCUMENTATION TO SUPPORT ITS STATEMENT THAT 79,250 EXISTING INSTRUCTIONS COULD BE REPROGRAMMED IN JOVIAL LANGUAGE AND ADAPTED TO PAVE PAWS WITH LITTLE COST OR SCHEDULE IMPACT. THE SSEB, WITHOUT ASKING GE FOR FURTHER INFORMATION, ASSUMED THAT IT COULD NOT BE DONE WITHOUT HIGH RISK. IN OUR VIEW, THE SSEB WAS JUSTIFIED IN ITS CONCERN AS TO HOW INSTRUCTIONS PREPARED FOR ONE COMPUTER COULD BE MOVED TO ANOTHER HAVING A DIFFERENT MISSION, DIFFERENT WORD LENGTHS AND A DIFFERENT OPERATING SYSTEM. SUCH TRANSFERS OFTEN CAN REQUIRE AN INSTRUCTION-BY-INSTRUCTION EXAMINATION AND AN EXTENSIVE REWRITE OF THE EXISTING PROGRAMS.

IN ITS BEST AND FINAL OFFER, GE PRESENTED ITS SOFTWARE INSTRUCTIONS COUNT IN A SERIES OF TABLES. IT EXPLAINED THAT THE PAVE PAWS SOFTWARE IS GROUPED INTO THREE CATEGORIES AS FOLLOWS: (1) SOFTWARE DEVELOPED FOR PAVE PAWS; (2) EXISTING GE SOFTWARE APPLICABLE TO PAVE PAWS; AND (3) EXISTING COMPUTER VENDOR SOFTWARE USED AS THE OPERATING SYSTEM FOR MISSION AND BACK -UP SOFTWARE CONTROL, DIAGNOSTIC SOFTWARE AND SUPPORT SOFTWARE FOR SOFTWARE DEVELOPMENT AND MAINTENANCE. IT DID NOT BREAK DOWN THE GE SOFTWARE INTO THOSE CATEGORIES REQUIRING MODIFICATION OR CONVERSION.

TABLE 2-3 OF GE'S FINAL OFFER TABULATED THE DEVELOPED AND DELIVERABLE SOFTWARE BY COMPUTER PROGRAM CONFIGURATION ITEM AND BY CODE TYPE. THE OTHER TABLES (2-1, 2-4, 2-5 AND A SUMMARY IN THE COST SECTION) SET FORTH THE SAME INFORMATION EXCEPT THAT THE COMPUTER VENDOR SOFTWARE WAS BROKEN DOWN INTO SUPPORT, OPERATING AND DIAGNOSTICS CATEGORIES. TABLES 2-1, 2-5 AND THE SUMMARY IDENTIFIED THE 79,250 INSTRUCTIONS AS EXISTING GE SUPPORT SOFTWARE. TABLE 2-4 ALSO IDENTIFIED THE 79,250 INSTRUCTIONS AS "MISSION REQUIRED" AND INCLUDED THEREIN AN ADDITIONAL 39,546 "DEVELOPED/MODIFIED/CONVERTED" INSTRUCTIONS PLUS 14,780 "DEVELOPED" INSTRUCTIONS FOR A TOTAL OF 132,576 GE INSTRUCTIONS TO BE DELIVERED ALONG WITH THE COMPUTER VENDOR INSTRUCTIONS.

THE AIR FORCE STATES THAT THERE ARE TWO POSSIBLE INTERPRETATIONS OF THE SOFTWARE SUBMISSION IN THE GE BEST AND FINAL OFFER. THE FIRST IS THAT 79,250 INSTRUCTIONS SHOULD BE ADDED TO THE MISSION SOFTWARE WHICH RESULTS IN AN INCREASE OF 143 PERCENT OVER THE PREVIOUS PROPOSAL OF 53,000 MISSION INSTRUCTIONS. THE SECOND INTERPRETATION IS THAT THE 79,250 INSTRUCTIONS WERE IN THE SUPPORT AREA AS THE AIR FORCE CLAIMED WAS DOCUMENTED IN THE GE SUBMISSION DURING NOVEMBER.

EVEN THOUGH THE 79,250 INSTRUCTIONS WERE IDENTIFIED AS SUPPORT SOFTWARE IN TABLES 2-1 AND 2-5 AND THE SUMMARY, THE AIR FORCE CLAIMS TO HAVE ADOPTED THE FIRST INTERPRETATION BECAUSE THE INSTRUCTIONS WERE INCLUDED IN TABLE 2-4 IN A COLUMN UNDER THE HEADING OF "MISSION REQUIRED." THE SSEB BELIEVED THAT SUCH A SUBSTANTIAL INCREASE IN MISSION REQUIRED SOFTWARE WAS TECHNICALLY RISKY ESPECIALLY IN VIEW OF THE LIMITED SUPPORTING TECHNICAL RATIONALE. THE AIR FORCE FURTHER CLAIMS, HOWEVER, THAT THE SECOND INTERPRETATION, WHICH WOULD LEAVE ONLY 53,326 INSTRUCTIONS IN THE MISSION SOFTWARE, WOULD HAVE MAGNIFIED ITS CONCERN BECAUSE OF ITS BELIEF THAT THE AMOUNT OF MISSION SOFTWARE WOULD BE INSUFFICIENT TO DO THE JOB AND WOULD REQUIRE THE PREPARATION OF ADDITIONAL INSTRUCTIONS WITH UNKNOWN IMPACT UPON SCHEDULE AND COSTS.

THE AIR FORCE POST-PROTEST STATEMENTS ASSERT THAT IN THE FINAL EVALUATION AFTER THE RECEIPT OF THE BEST AND FINAL OFFERS, SIGNIFICANT CONCERNS WERE IDENTIFIED IN THE GE PROPOSAL REGARDING THE SOFTWARE DESIGN APPROACH. THE SSEB DOUBTED THAT THE EXISTING INSTRUCTIONS WHICH ACCOUNTED FOR 60 PERCENT OF THE MISSION REQUIRED SOFTWARE COULD BE MOVED BETWEEN COMPUTERS WITHOUT MODIFICATION OR CONVERSION AND IT BELIEVED THAT THE USE OF A LARGE NUMBER OF EXISTING INSTRUCTIONS WOULD ADVERSELY AFFECT THE "TOP-DOWN" /1/ DESIGN REQUIRED BY THE RFP. IT FEARED THAT THE EXCESSIVE USE OF SEVERAL COMPUTER LANGUAGES WOULD COMPLICATE THE SOFTWARE INTERFACE AND MAINTENANCE. ADDITION, THE AIR FORCE STATES THAT THE SSEB BELIEVED THAT THE FREQUENCY AND MAGNITUDE OF THE CHANGES MADE BY GE IN ITS SOFTWARE ESTIMATES THROUGHOUT THE NEGOTIATION PROCESS DEMONSTRATED THAT GE DID NOT FULLY UNDERSTAND THE DATA PROCESSING REQUIREMENTS OR THE COMPLEXITY OF THE FUNCTIONS NECESSARY TO SATISFY SUCH REQUIREMENTS.

GE CONTENDS THAT THESE CONCERNS WERE UNFOUNDED, NEVER DISCUSSED AND IN CONFLICT WITH THE RFP. IN ESSENCE, GE QUESTIONS HOW THE SPECIFIC FACTORS CITED BY THE AIR FORCE FOR THE NONSELECTION OF GE COULD HAVE BEEN IGNORED DURING THE DISCUSSIONS AND YET BE CONSIDERED SO SIGNIFICANT AS TO RESULT IN THE REJECTION OF ITS LOWEST PRICED PROPOSAL. GE FURTHER CONTENDS THAT THE SSEB MISTAKENLY INTERPRETED ITS BEST AND FINAL OFFER AS PROVIDING FOR AN ADDITIONAL 79,250 MISSION REQUIRED INSTRUCTIONS AND THAT THE SSEB'S PRIOR FAILURE TO CONDUCT MEANINGFUL DISCUSSIONS LED TO SUCH ERRONEOUS INTERPRETATION. GE STATES THAT SUPPORT SOFTWARE IS NEVER USED DIRECTLY IN THE OPERATIONAL SYSTEM AND THAT, THEREFORE, IT COULD NOT COMPLICATE THE "TOP-DOWN" DEVELOPMENT REQUIRED BY THE SOLICITATION, AFFECT THE EFFICIENCY OF THE MISSION REQUIRED SOFTWARE OR HAVE ANY IMPACT ON THE TIMING OF THE MISSION REQUIRED SOFTWARE. GE ALSO POINTS OUT THAT THE USE OF MULTIPLE COMPUTER LANGUAGES IS RECOGNIZED IN THE RFP, WHICH PROVIDED THAT IF A MIXTURE OF COMPUTER LANGUAGES IS ANTICIPATED, THE APPROXIMATE FRACTION OF EACH LANGUAGE SHOULD BE INDICATED. FURTHER, GE ASSERTS THAT ITS RADAR DESIGN WAS CHOSEN BECAUSE OF PROVEN TECHNIQUES AND PROCESSES WHICH REDUCE THE NUMBER OF SOFTWARE SOURCE INSTRUCTIONS NEEDED.

THE COMPLETE RECORD FOR THE PERIOD AFTER THE CALL FOR BEST AND FINAL OFFERS INDICATES THAT NEITHER THE SSEB NOR THE SSAC WAS CONFUSED AS TO THE TOTAL NUMBER OF SOURCE INSTRUCTIONS PROPOSED IN GE'S BEST AND FINAL OFFER. THE MAIN SOFTWARE CONCERN OF EACH WAS THE LOW NUMBER OF NEWLY DEVELOPED, MODIFIED OR CONVERTED SOURCE LANGUAGE INSTRUCTIONS (53,326). SERIOUS QUESTION EXISTED AS TO THE FEASIBILITY OF TRANSFERRING THE 79,250 EXISTING SOURCE INS INSTRUCTIONS TO THE PAVE PAWS PROJECT WITHOUT MODIFICATION, CONVERSIONS OR COST AND SCHEDULE IMPACT.

THE SSEB REPORT OF MARCH 4, 1976 TO THE SSAC AND THE SSA INDICATED THAT THE SOFTWARE TO BE DEVELOPED WAS APPROXIMATELY 50,000 SOURCE LANGUAGE INSTRUCTIONS. THE WEAKNESSES WERE IDENTIFIED TO BE THAT THE APPLICATIONS SOFTWARE WAS UNDERESTIMATED, THE PROPOSED APPROACH TO "TOP DOWN" DEVELOPMENT WAS IN FACT A COMBINATION OF "TOP-DOWN/BOTTOM-UP" METHOD AND THE USE OF THE FORTRAN PROGRAMS INCREASED THE REQUIRED SUPPORT SOFTWARE AND COMPLICATED THE SOFTWARE INTERFACE. THE SSEB ALSO STATED THAT GE APPARENTLY UNDERESTIMATED THE INSTRUCTIONS BY 100,000 AND RELATED COSTS BY $600,000.

THE SSAC'S WRITTEN REPORT OF MARCH 9, 1976 TO THE SSA STATED THAT THE SOFTWARE APPEARED TO BE SIGNIFICANTLY UNDERESTIMATED BECAUSE ONLY ABOUT 50,000 OF THE 132,000 SOURCE INSTRUCTIONS PROPOSED WERE NEWLY DEVELOPED, MODIFIED OR CONVERTED AS COMPARED TO 160,000 AND 175,000 FOR THE OTHER COMPETITORS. IT STATED THAT THE GE ESTIMATE OF NEW SOURCE INSTRUCTIONS APPEARED TO BE UNDERSTATED BY APPROXIMATELY 30,000 TO 40,000 WHICH THE SSAC ESTIMATED WOULD HAVE A COST IMPACT OF $1.0 MILLION, ASIDE FROM THE COST OF ANY SCHEDULE SLIP. IT CONSIDERED AS UNREALISTIC THE TRANSFER OF 79,250 EXISTING SOURCE INSTRUCTIONS FROM OTHER, GE SOFTWARE SYSTEMS AND GE HAD PROVIDED NO TECHNICAL DESCRIPTION AS TO HOW IT COULD BE DONE.

THE SLIDE USED BY THE SSAC IN ITS ORAL PRESENTATION TO THE SSA DID NOT ALLOCATE THE 79,250 INSTRUCTIONS TO ANY OF THE SPECIFIC FUNCTIONAL AREAS, SUCH AS THE SUPPORT AREA. INSTEAD, IT ADDED THE 79,250 TO THE 48,961 TOTAL INSTRUCTIONS FOR ALL FUNCTIONAL AREAS. THE PREPARED SCRIPT ACCOMPANYING THE SLIDE INDICATED THAT 79,000 INSTRUCTIONS WOULD BE BORROWED FROM OTHER PROGRAMS BUT THAT NO SUBSTANTIATING DETAILS WERE GIVEN BY GE. IT THEN EXPRESSED THE EXPECTATION THAT ANY BORROWED CODE WOULD FALL INTO THE EXECUTIVE CONTROL AND SUPPORT AREAS. IN THIS CONNECTION, WE NOTE THAT THE AIR FORCE HAS STATED DURING THIS PROTEST THAT SUCH INSTRUCTIONS WERE NOT TREATED AS BEING IN THE SUPPORT AREA. HOWEVER, AS WE SEE THE RECORD, THE SSAC INCLUDED THE BULK OF THE 79,250 INSTRUCTIONS IN THE SUPPORT AREA WHERE GE CONTENDS THEY SHOULD BE.

GE'S SOFTWARE SUBMITTAL IN ITS BEST AND FINAL OFFER WAS PRESENTED IN A SERIES OF TABLES. IT WAS CONFUSING AND NOT EASILY TRACEABLE IN ALL RESPECTS TO GE'S PREVIOUS SUBMITTALS. IT WAS DIFFICULT TO INTERPRET WITH CERTAINTY AND REQUIRED MORE ANALYSIS THAN SHOULD BE EXPECTED CONSIDERING IT IS THE OFFEROR'S BURDEN TO DEMONSTRATE THE MERITS OF ITS PROPOSAL. THIS CONNECTION, WE ALSO NOTE THAT GE'S TELEGRAM OF APRIL 1, 1976, WHICH WAS REJECTED AS A LATE MODIFICATION, STATED THAT ERRORS HAD BEEN MADE IN THE SOFTWARE COUNT WHICH INVOLVED THE ERRONEOUS ADDITION OF MACHINE LANGUAGE INSTRUCTIONS WITH SOURCE LANGUAGE INSTRUCTIONS. BY OUR CALCULATIONS, THIS TELEGRAM WOULD HAVE INCREASED THE SOFTWARE INSTRUCTION COUNT IN SOURCE LANGUAGE FROM 132,576 TO 140,000-145,000 INSTRUCTIONS. SPITE OF THIS, WE SEE NO SIGNIFICANT MISINTERPRETATION BY THE AIR FORCE OF GE'S FINAL SOFTWARE PRESENTATION. IT APPEARS, HOWEVER, THAT THE AIR FORCE IN ITS POST PROTEST STATEMENTS MAY HAVE MISINTERPRETED ITS OWN PRESELECTION ACTIONS PERTAINING TO THE 79,250 INSTRUCTIONS. THIS WOULD NOT, HOWEVER, BE RELEVANT TO THE VALIDITY OF THE SELECTION WHICH HAD BEEN MADE PRIOR TO SUCH INTERPRETATIONS. IN ADDITION, ALTHOUGH GE CONTENDS THAT THE NUMBER OF SOURCE INSTRUCTIONS NEEDED WAS REDUCED BECAUSE OF ITS PROVEN RADAR TECHNIQUES AND PROCESSES, THE RECORD IS NOT CONVINCING IN THIS REGARD.

DURING THE COURSE OF THE NEGOTIATIONS THE SSEB ASSUMED THAT 79,250 GE SUPPORT INSTRUCTIONS WERE COMPUTER-VENDOR SUPPLIED AND THEREFORE DIFFICULT TO ADAPT TO PAVE PAWS. APPARENTLY, AFTER GE SUBMITTED ITS BEST AND FINAL OFFER IT BECAME CLEAR TO THE SSEB THAT THE 79,250 SUPPORT INSTRUCTIONS WERE GE-SUPPLIED BUT THE AIR FORCE EVALUATORS STILL CONSIDERED THESE INSTRUCTIONS DIFFICULT TO ADAPT TO PAVE PAWS. IT SEEMS TO US, HOWEVER, THAT THE SSEB WOULD HAVE BEEN IN A BETTER POSITION TO EVALUATE THE ADEQUACY OF THE GE SOFTWARE APPROACH, BOTH INITIALLY AND FINALLY, IF IT HAD ASKED GE TO PRODUCE THE PRELIMINARY FUNCTIONAL REQUIREMENTS TREES WHICH THE RFP REQUIRED TO BE DEVELOPED BY OFFERORS FOR EACH COMPUTER PROGRAM CONFIGURATION ITEM. IN THE ABSENCE OF THIS INFORMATION AND AMPLIFYING TECHNICAL DOCUMENTATION, WE ARE UNABLE TO DETERMINE WHETHER GE'S EXISTING SOFTWARE WOULD BE READILY ADAPTABLE TO PAVE PAWS. NEVERTHELESS, WE AGREE WITH THE AIR FORCE THAT GE'S SOFTWARE APPROACH AS OUTLINED IN ITS PROPOSAL PRESENTED A RISK IN THIS AREA.

AS THE SSA POINTS OUT, HE DID NOT CONSIDER THE SOFTWARE INSTRUCTION COUNT, OF ITSELF, TO BE AN ISSUE OF PARAMOUNT SIGNIFICANCE BECAUSE GE COULD PROVIDE MORE INSTRUCTIONS WITH A COMPARATIVELY MODEST COST INCREASE. WHAT WAS OF CONCERN TO HIM WAS GE'S APPARENT LACK OF UNDERSTANDING OF THE DATA PROCESSING REQUIREMENT AS INDICATED BY THE VACILLATION PERCEIVED IN ITS DESIGN APPROACH TO BOTH HARDWARE AND SOFTWARE, THE LACK OF SUFFICIENT DETAILED DOCUMENTATION TO ENABLE THE GOVERNMENT TO FULLY UNDERSTAND THE OVERALL DATA PROCESSING SYSTEM IT PROPOSED, AND THE UNCERTAINTIES REFLECTED BY AMBIGUITIES IN ITS PROPOSAL THROUGH BEST AND FINAL OFFERS. IN REACHING HIS FINAL CONCLUSION IN THIS SUB-AREA, THE SSA GAVE GE THE BENEFIT OF DOUBT ON THE JUDGMENT CALLS AND STILL CONCLUDED THAT RAYTHEON CLEARLY OFFERED THE SOUNDEST, MOST REALISTIC AND MOST ADVANTAGEOUS PROPOSAL OF THE TWO. MOREOVER, HE AGREED WITH THE EVALUATORS THAT, ON BALANCE, THE RISKS OF ENCOUNTERING SOFTWARE PROBLEMS THAT WOULD ENTAIL PROGRAM DISRUPTIONS THROUGH SCHEDULE SLIPS AND THE ADDED COSTS THAT SUCH SLIPS ENTAIL WERE MUCH GREATER FOR GE THAN THEY WERE FOR RAYTHEON.

COST AREA

THE AIR FORCE POINTS OUT THAT THE TARGET PRICE IN GE'S BEST AND FINAL OFFER WAS $6 MILLION LOWER THAN THAT OF RAYTHEON FOR ALL ITEMS EVALUATED FOR AWARD, BUT CONTENDS THAT THE REALISM OF GE'S COST ESTIMATE COULD NOT BE SUBSTANTIATED. THE RFP STATED THAT A DETERMINATION OF COST REALISM WOULD "INCLUDE" AN EVALUATION OF THE PROPOSED COSTS WITH THE GOVERNMENT ESTIMATE. THE RECORD INDICATES THAT THIS WAS DONE. THE RFP FURTHER STATED THAT THE FINANCIAL IMPACT OF THE HIGH RISK AREAS WOULD BE ADDED TO THE PROPOSED COSTS IN ORDER TO DEVELOP A MOST PROBABLE COST TO THE GOVERNMENT. GE, IN EFFECT, ARGUES THAT THE SSEB WAS INCONSISTENT IN ADDING TO GE'S FINAL PRICE ONLY $1 MILLION FOR SOFTWARE SHORTAGES WHILE STATING THAT GE WOULD EXCEED CEILING PRICE. HOWEVER, THE AIR FORCE ARGUES THAT IT COULD NOT USE THE TARGET COST FIGURE PROPOSED BY GE AS THE BASE FOR THE ADDITION OF THE RISK FACTOR. RATHER, IT BASED ITS ESTIMATE ON AN ANALYSIS OF THE ENTIRE GE SYSTEM AS IT WAS IN NOVEMBER 1975 AND ALLOCATED TO THE VARIOUS ELEMENTS OF THE GE WORK BREAKDOWN STRUCTURE, COSTS BASED ON ESTIMATES AND COSTS OF SIMILAR COMPONENTS, SYSTEM AND EFFORTS. AFTER THE RECEIPT OF THE BEST AND FINAL OFFERS, THE MOST PROBABLE COST WAS ADJUSTED IN THE LIGHT OF THE TECHNICAL CHANGES AND PRICE REDUCTIONS. THE AIR FORCE STATES THAT AT THIS TIME ITS ESTIMATE, EXCLUSIVE OF RISK DOLLARS, WAS $1 MILLION LESS THAN GE'S CEILING PRICE AND EQUAL THERETO WITH THE ADDITION OF $1 MILLION FOR THE SOFTWARE SHORTAGE. FURTHER, THE SSEB CONSIDERED THAT AN UNQUANTIFIED AMOUNT SHOULD BE ADDED FOR THE SCHEDULE SLIPPAGE WHICH IT ANTICIPATED WOULD BE ENCOUNTERED BY GE AS A RESULT OF SOFTWARE DEVELOPMENT PROBLEMS AND THE UNCERTAINTIES ABOUT THE NEW BUILDING DESIGN. ALTHOUGH THE AIR FORCE ASSERTS THAT IT COULD NOT ESTIMATE A SPECIFIC AMOUNT TO BE ADDED TO THE GE BUILDING PRICE OF $10.7 MILLION, THE SSEB USED, THROUGHOUT ITS PRESENTATION TO THE SSAC, A MOST PROBABLE COST FIGURE OF $12.5 MILLION FOR THE BUILDING AND INCLUDED THIS FIGURE IN ITS ESTIMATE FOR THE TOTAL SYSTEM.

AFTER BEST AND FINAL OFFERS, THE SSEB AND SSAC ESTIMATES FOR THE TWO SITES AND THE PRICES PROPOSED BY GE AND RAYTHEON WERE AS FOLLOWS: (TABLE OMITTED)

THE PRESENTATIONS OF THE SSEB AND THE SSAC MADE IT CLEAR THAT THEIR FIGURES DID NOT INCLUDE THE COST OF THE EXPECTED PROGRAM SLIPPAGE OR A FACTOR FOR THE RISKS BOTH SAW IN THE NEW BUILDING DESIGN, THE PRECISE AMOUNT OF WHICH BOTH FOUND IMPOSSIBLE TO ASSESS. THE FIGURES FOR RAYTHEON INCLUDED FACTORS FOR ALL KNOWN RISKS. THE SSA WAS LEFT WITH A CHOICE BETWEEN THE SUPERIOR RATED RAYTHEON PROPOSAL, THE COST REALISM OF WHICH THE SSA WAS RELATIVELY CONFIDENT, AND THE LOWER RATE GE PROPOSAL, THE COST REALISM OF WHICH COULD NOT BE ESTABLISHED BEYOND THE BELIEF THAT IT WOULD EXCEED CEILING PRICE BY SEVERAL MILLION DOLLARS. IN OUR OPINION, THE SSEB'S ESTIMATE OF THE MOST PROBABLE COST OF GE'S FINAL PROPOSAL, PARTICULARLY THE ADDITION OF A FACTOR FOR DEVELOPING ADDITIONAL SOFTWARE, WAS NOT UNREASONABLE. IT WAS REASONABLE FOR THE SSEB NOT TO FACTOR IN TO ITS MOST PROBABLE COST OR TO GE'S PROPOSED PRICE A PRECISE AMOUNT FOR SCHEDULE SLIPPAGE AND BUILDING UNCERTAINTIES, IN VIEW OF THE INADEQUACY OF GE'S PROPOSAL. THE FACT THAT GE WOULD HAVE TO ABSORB ALL DIRECT COSTS EXCEEDING ITS CEILING PRICE OF $9.3 MILLION DOES NOT NEGATE THE LEGITIMATE CONCERN OF THE SSEB FOR THE COSTS OVER CEILING AND THE PERFORMANCE, SCHEDULE AND ADMINISTRATION PROBLEMS WHICH CAN REASONABLY BE EXPECTED TO RESULT FROM A LOSS POSITION.

IN ARRIVING AT THIS ULTIMATE SELECTION, THE SSA CONSIDERED MUCH MORE THAN THE COSTS PROPOSED BY THE OFFERORS BECAUSE OTHER FACTORS WERE CONSIDERED TO BE MORE SIGNIFICANT. THE SSA DETERMINED, IN ACCORDANCE WITH THE SOLICITATION, WHICH PROPOSED PROVIDED THE MOST ADVANTAGEOUS SYSTEM TO THE GOVERNMENT WITH DUE CONSIDERATION TO THE TECHNICAL AND OPERATIONS ASPECTS OF THE PROPOSALS. WE NOTE THAT, ALTHOUGH GE WAS ADVISED AT THE DEBRIEFING THAT THE COMPETITION WAS "CLOSE," THE RECORD SHOWS THAT IN THE SSA'S OPINION RAYTHEON HAD A CLEAR CUT MARGIN, IN VARYING DEGREES, OVER GE IN ALL THREE SUB-AREAS WITHIN THE TECHNICAL AND OPERATIONS AREA, THAT IS, RADAR SUBSYSTEM, DATA PROCESSING HARDWARE AND SOFTWARE SUBSYSTEM AND TECHNICAL FACILITY. WE NOTE THAT THERE WAS NO DISCUSSION OF THE SSA'S CONCLUSIONS CONCERNING THE RADAR SUBSYSTEM IN THE POST AWARD NOTICE TO GE. NEVERTHELESS, THE RECORD SHOWS THAT WHILE GE'S RADAR SUBSYSTEM HAD THE ADVANTAGE OF SLIGHTLY GREATER POWER AND RANGE, THE SSA CONSIDERED THAT THIS WAS MORE THAN OFFSET BY SEVERAL RADAR OPERATIONAL ADVANTAGES IN THE RAYTHEON PROPOSAL AND THE FACT THAT RAYTHEON WAS EVALUATED AS OFFERING THE MORE RELIABLE RADAR SYSTEM, AN ESSENTIAL CONSIDERATION OF THE OPERATING COMMAND. AMONG THE COST FACTORS CONSIDERED BY THE SSA WERE:

(1) WHAT WAS THE MOST LIKELY EVENTUAL COST TO THE GOVERNMENT OF THE OVERALL SYSTEM PROPOSED BY EACH OFFEROR? (FPIF CONTRACT WITH A 130% CEILING PRICE.)

(2) WAS THE PRICE FOR THE SYSTEM PROPOSED BY EACH OFFEROR A FAIR AND REASONABLE ONE TO THE GOVERNMENT? (EACH SYSTEM, OF COURSE, WAS DIFFERENT AND EACH OFFERED SOME DIFFERENT FEATURES.)

(3) WERE ANY ADDED COSTS INVOLVED IN FEATURES OF ONE OFFEROR'S APPROACH VERSUS THAT OF ANOTHER FULLY JUSTIFIABLE IN TERMS OF ADDED VALUE TO BE RECEIVED BY THE GOVERNMENT?

(4) HOW WOULD THE CONCLUSIONS REACHED ON COSTS AFFECT THE OFFEROR'S RELATIVE STANDINGS IN THE TECHNICAL/OPERATIONS AREA? WOULD THE COST RATINGS, COMING AS SECOND PRIORITY, BE SUFFICIENT TO RE-ORDER THE OVERALL CONTRACTOR RANKING FOR AWARD?

THE SSA TOOK INTO ACCOUNT THE SSAC'S INDEPENDENT COST ESTIMATE WHICH INDICATED THAT THE COSTS FOR EITHER GE'S OR RAYTHEON'S SYSTEM WOULD BE ROUGHLY COMPARABLE BUT WAS INFLUENCED BY THE FACT THAT THE DIFFERENCES BETWEEN GE'S OFFERED PRICES AND THE SSAC'S ESTIMATES WERE MUCH GREATER THAN THE DIFFERENCES BETWEEN RAYTHEON'S OFFERED PRICES AND THE SSAC'S ESTIMATES. ON THE BASIS OF THE SSAC'S ESTIMATES, HE FOUND A DISTINCT PROBABILITY THAT GE WOULD GO BEYOND ITS CEILING PRICE. THE SSA REASONED THAT THE SSAC'S COST ESTIMATES FOR GE'S PROPOSAL WERE ESPECIALLY CREDIBLE BECAUSE OF GE'S SUBMISSION OF SUBSTANTIAL COST REDUCTIONS IN ITS BEST AND FINAL OFFER WITHOUT ADEQUATE EXPLANATORY DOCUMENTATION OR COST TRACEABILITY. ALTHOUGH THERE EXISTED A $6 MILLION DIFFERENCE IN TOTAL TARGET PRICE FOR ALL EVALUATED ITEMS BETWEEN GE AND RAYTHEON, THIS WAS DISCOUNTED BY THE SSA BECAUSE THE DIFFERENCE LARGELY WAS OBTAINED IN AREAS WHERE THE GOVERNEMENT'S CONCERNS OVER THE VALIDITY OF THE GE PRICES WERE THE GREATEST. IT WAS ALSO NOTED THAT $2.4 MILLION OF THIS DIFFERENCE REPRESENTED PROPOSAL SAVINGS IN GE'S OPERATIONS AND MAINTENANCE PROVISIONS BUT THAT GE WAS SUBSTANTIALLY BELOW THE GOVERNMENT'S ESTIMATES IN THIS AREA BECAUSE IT PROPOSED CONSIDERABLY LESS MANPOWER TO CARRY OUT THE OPERATIONS AND MAINTENANCE FUNCTION THAN WAS CONSIDERED NECESSARY BY THE GOVERNMENT. IN THE FINAL ANALYSIS, THE SSA CONCLUDED THAT GE'S LOWER TARGET AND CEILING PRICES OFFERED LITTLE IN THE WAY OF ADVANTAGES TO THE GOVERNMENT AND, IN ANY EVENT, WERE NOT CONSIDERED TO BE OF SUFFICIENT SIGNIFICANCE TO OVERCOME RAYTHEON'S SUPERIORITY IN THE TECHNICAL OPERATIONS AREA. AS TO THE REMAINING TWO EVALUATION AREAS, LOGISTICS AND MANAGEMENT, IT ALSO SHOULD BE NOTED THAT THE RECORD INDICATES THAT THE SSA CONSIDERED GE AND RAYTHEON TO BE RELATIVELY EQUAL IN LOGISTICS AND RAYTHEON TO BE STRONGER THAN GE IN THE MANAGEMENT AREA. IT DOES NOT APPEAR, HOWEVER, THAT EITHER OF THE LATTER TWO AREAS WAS DETERMINATIVE IN HIS SELECTION.

FINALLY, A QUESTION HAS BEEN RAISED AS TO WHETHER THE REPORT AND PRESENTATION OF THE SSEB WAS PROPERLY AND FAIRLY PRESENTED BY THE SSAC TO THE SSA AND WHETHER THE SSA WAS DENIED THE ESSENCE OF THE SSEB'S EVALUATION. OUR REVIEW INDICATES THAT WHILE THE SSEB REPORT AND THE SSAC REPORT DIFFERED IN EMPHASIS AND IN SOME DETAILS, THE ESSENTIAL THRUST OF EACH WAS THE SAME. MOREOVER, IT IS UNFORTUNATE THAT AN ADMINISTRATIVE ERROR CAUSED CONSIDERABLE SUSPICIONS REGARDING THE SOURCE SELECTION PROCESS IN THIS CASE. THE PRICING DIVISION, ESD, IN ACCORDANCE WITH ASPR SEC. 1-308, PREPARED A PRICE NEGOTIATION MEMORANDUM ON GE WHICH WAS DATED MARCH 30, 1976 AND ANOTHER ON RAYTHEON DATED MARCH 31, 1976. BOTH REFERENCED THE SAME CONTRACT NUMBER AND EACH GAVE THE IMPRESSION THAT THE COMPANY HAD BEEN SELECTED FOR AWARD. THESE WERE INTERNAL AIR FORCE DOCUMENTS MARKED "FOR OFFICIAL USE ONLY" AND ARE GENERALLY PREPARED PRIOR TO CONTRACTOR SELECTION AND ALWAYS PRIOR TO CONTRACTOR SELECTION AND ALWAYS PRIOR TO CONTRACT AWARD. THEY ARE USED DURING PROCUREMENT REVIEWS. AFTER AWARD THE WINNER'S PRICE NEGOTIATION MEMORANDUM IS SENT TO THE COGNIZANT DEFENSE CONTRACT AUDIT AGENCY (DCAA) OFFICE SO THAT IT CAN COMPARE THE INFORMATION TO ITS PROPOSAL AUDIT RECOMMENDATIONS. IN THIS INSTANCE, MEMORANDA FOR BOTH GE AND RAYTHEON WERE INADVERTENTLY DISTRIBUTED AND RECEIVED BY THE DCAA OFFICES ON APRIL 2, 1976, WHICH WAS THE SAME DATE THAT THE SOURCE SELECTION DECISION DOCUMENT WAS SIGNED. WHEN THE MISTAKE WAS DISCOVERED, THE MEMORANDA WERE RECALLED BY ESD. WHILE THE MEMORANDUM FOR GE WAS IN THE DCAA OFFICE IN SYRACUSE, GE CONCLUDED THAT IT HAD WON THE CONTRACT. THE RETURN OF THE MEMORANDUM TO ESD MAY HAVE LED IT TO BELIEVE THAT THERE HAD BEEN A CHANGE IN CONTRACTOR SELECTION. OUR REVIEW OF THIS MATTER CONVINCES US THAT CONTRARY TO GE'S BELIEF, NO SELECTION HAD BEEN MADE WHEN THE MEMORANDUM WAS INADVERTENTLY SENT TO DCAA IN SYRACUSE AND THAT GE HAD AT NO TIME BEEN SELECTED FOR AWARD.

FOR THE FOREGOING REASONS, WE HAVE CONCLUDED THAT THE SELECTION OF RAYTHEON WAS REASONABLY CONSISTENT WITH THE EVALUATION CRITERION PROVIDED IN THE SOLICITATION AND THAT THE RECORD SUPPORTS THE SELECTION ON THE BASIS OF RAYTHEON'S SUPERIORITY IN THE TECHNICAL AND OPERATIONS AREA. CANNOT CONCLUDE THAT THE AIR FORCE FAILED TO DISCHARGE ITS RESPONSIBILITIES UNDER 10 U.S.C. 2304(G) (1970) TO CONDUCT MEANINGFUL DISCUSSIONS WITH GE.

ACCORDINGLY, GE'S PROTEST IS DENIED.

/1/ THE "TOP-DOWN" DESIGN APPROACH STARTS BUILDING THE SOFTWARE MODULES IN A NATURAL ORDER STARTING WITH THE BROADER SCOPE REQUIREMENTS. IT THEN EXPANDS DOWNWARD INTO MANY MORE MODULES WHICH CONTAIN ROUTINES OF INCREASING DETAIL. DESIGN IS LESS RESTRICTED WITH THIS METHOD THAN IT IS BY INTERCONNECTING MANY EXISTING DETAILED MODULES WITH EACH OTHER AND OTHERS AT THE HIGHER LEVELS. SUCH AS IN A "BOTTOM-UP" DESIGN.