Skip to main content

Matter of: United International Engineering, Inc. File: B-257607.3 Date: September 12, 1995

B-257607.3 Sep 12, 1995
Jump To:
Skip to Highlights

Highlights

Protest of high performance risk rating in procurement for software engineering support of battlefield automated systems is denied where. Is denied where solicitation requested information with respect to contracts for similar work and expressly provided for consideration of offerors' relevant and recent experience under a number of evaluation factors. Which was described as "most important. " all of which were of equal importance. The evaluation of which is at issue here. This assessment will focus on relevant and recent . . . contractual activity performed since 1 Jan 91. . . .". The solicitation required offerors to describe government contracts "received or on-going since 1 Jan 1991 . . . which are similar to the effort required by this solicitation.

View Decision

Matter of: United International Engineering, Inc. File: B-257607.3 Date: September 12, 1995

Protest of high performance risk rating in procurement for software engineering support of battlefield automated systems is denied where, although protester proposed to subcontract 37 percent of effort to subcontractors with good performance records, the agency received reports of past performance problems and demonstrated inadequate software engineering capabilities on the part of the protester, and reasonably determined that there existed a significant risk that the protester lacked the technical expertise necessary in order to effectively manage the performance of its subcontractors in the software area. Protest that agency improperly considered as an advantage incumbent's superior experience with, and resulting knowledge of, software in weapons systems to be supported under contemplated contract, and as a disadvantage protester's relative lack of experience in this regard, is denied where solicitation requested information with respect to contracts for similar work and expressly provided for consideration of offerors' relevant and recent experience under a number of evaluation factors.

Attorneys

DECISION

United International Engineering, Inc. (UIE) protests the award of a contract to EER Systems Corporation under request for proposals (RFP) No. DAAH01-94-R-R008, issued as a small disadvantaged business (SDB) set-aside by the U.S. Army Materiel Command, Army Missile Command (MICOM), for battlefield automated systems engineering support (BASES). UIE challenges the evaluation of past performance and experience and the overall cost/technical trade-off.

We deny the protest.

The solicitation contemplated award of a cost-plus-award-fee contract for 1 base year with 6 option years for engineering support of the software engineering directorate (SED), research, development and engineering center, MICOM, in providing total life cycle support of battlefield automated systems, including: (1) acquisition and development of mission critical computer resources; (2) software verification and validation; (3) command, control and communications engineering and interoperability testing; (4) requirements definition for, and development of, software tools; (5) software quality assurance; and (6) configuration and data management. The solicitation established 16 required labor categories, setting forth minimum educational and experience requirements and anticipated levels of effort for each.

The solicitation generally provided for award to be made to the responsible offeror whose proposal offered the best value to the government. The RFP listed four specific evaluation factors: (1) technical, including technical approach and resumes, which was described as "most important, and significantly more important than" (2) performance risk, (3) management and (4) "price," all of which were of equal importance.

With respect to performance risk, the evaluation of which is at issue here, the solicitation provided that the agency would

"make a performance risk assessment based upon each offeror's and his subcontractor's current and past records of performance as they relate to the probability of successful completion of the required effort. This assessment will focus on relevant and recent . . . contractual activity performed since 1 Jan 91. . . ."

The solicitation required offerors to describe government contracts "received or on-going since 1 Jan 1991 . . . which are similar to the effort required by this solicitation," including furnishing: (1) the names and telephone numbers of the procuring and administering contracting officers and the contracting activity's technical and other representatives; (2) "a narrative explanation for the Statement of Work, similarities of that work with the work required by this solicitation, objectives achieved, and cost growths or schedule delays encountered"; and (3) an explanation of the reasons for any failure to "meet original requirements with regard to either cost, schedule, or technical performance," and of "any corrective actions taken to avoid recurrence." In addition, the solicitation provided that the technical, management, and price evaluations would include an evaluation of "[p]roposal risk" that would "focus on those aspects of the offeror's proposal which may tend to decrease confidence in the ability to satisfy the contractual requirement in a timely and qualitative manner consistent with the proposed approach."

Seven proposals were received by the closing time. Six--including EER's and UIE's--were included in the competitive range. Following discussions with the offerors, the agency requested best and final offers (BAFO).

Based upon its evaluation of BAFOs, MICOM determined that EER had submitted the proposal offering the best value to the government. Both UIE's and EER's proposals received above average ratings under the technical factor, and UIE's proposal received a higher management rating (above average) than EER's (average) and offered lower proposed ($141,685,273) and evaluated ($146,335,538) prices than did EER's ($162,555,326 proposed and $164,329,203 evaluated). However, MICOM concluded that UIE's apparent advantage under the management and price factors was offset by (1) the fact that EER's proposal was evaluated as superior to UIE's under the technical evaluation elements for command, control, communications and intelligence engineering and interoperability testing, software quality assurance, and special level software engineering, while UIE's proposal was not found to be superior to EER's under any of the technical approach evaluation elements; (2) UIE's proposal of lower indirect rates than it historically had experienced, which MICOM determined "raises some doubt that UIE can effect its proposed technical approach and staffing within its proposed cost" and thereby increased overall proposal risk; and (3) UIE's high performance risk rating, resulting from its past performance problems in the area of software engineering. MICOM concluded that EER's proposal represented the best value to the government, and therefore selected that firm for award.

Upon learning of the selection of EER, UIE and two other offerors protested EER's SDB status. After the Small Business Administration (SBA) initially determined that EER was other than an SDB, MICOM selected Camber Corporation for award. Thereafter, however, SBA determined that EER was in fact an SDB; MICOM thus rescinded the selection of Camber and again selected EER. Upon learning of the resulting award to EER, UIE protested the selection, first to the General Services Administration Board of Contract Appeals (GSBCA), and then, after GSBCA declined to accept jurisdiction, to our Office.

PERFORMANCE RISK

UIE challenges MICOM's evaluation of its proposal as characterized by high performance risk as a result of perceived past performance problems in the area of software engineering. According to the protester, MICOM unduly emphasized a single report of inadequate performance and failed to take into account UIE's overall performance. In this regard, UIE questions why its proposal received an overall high performance risk rating and EER's received an overall low performance risk rating when, in response to the agency's detailed performance questionnaire, MICOM received more ratings of inadequate performance for EER (10 of 679 responses) than for UIE (1 of 437 responses). Further, argues UIE, even if its prior performance with respect to software engineering was reasonably found to be unsatisfactory, this did not support MICOM's high performance risk appraisal since UIE had proposed to subcontract software engineering to firms with past performance records considered satisfactory by the agency.

In reviewing an agency's evaluation of proposals, we consider whether it was in accord with the evaluation criteria listed in the solicitation and whether the actual evaluation was reasonable. John Brown U.S. Servs., Inc., B-258158 et al., Dec. 21, 1994, 95-1 CPD Para. 35; Information Sys. & Networks Corp., 69 Comp.Gen. 284 (1990), 90-1 CPD Para. 203.

The evaluation of UIE's past performance and the resulting high performance risk rating were reasonable. Notwithstanding UIE's claim to the contrary, the record establishes that MICOM based its risk assessment on a number of reports concerning UIE's performance under several contracts, which called into question the probability of UIE's successfully completing the required effort. UIE identified 10 contracts, of which 9 were considered relevant to this contract effort and were reviewed by the agency. MICOM received reports of performance problems under at least four of these contracts; the criticism of UIE's performance on three of these was considered sufficiently serious by MICOM as to raise "serious doubts that the UIE team could successfully perform this effort if managed by UIE and with two-thirds of the technical work proposed to be done by UIE."

On the largest contract referenced by UIE (DAAH07-90-C-0008), and the one involving the most similar work--software engineering support--the contracting officer's technical representative (COTR) reported that:

"[i]n the area of software development, UIE has demonstrated an inability to plan, develop, and implement procedures which result in the delivery of high quality products and services on schedule and within budget. Despite encountering problems in the software development area, UIE has provided virtually no training for their on-site personnel in this area, nor does it appear that resources exist at the corporate level to provide assistance."

The COTR characterized UIE's software as "band aid, glitchy code"; he stated that he knew of "no [software] product they can market." Problems with UIE's performance led the contracting agency to issue a cure notice to UIE in March 1994 requiring correction of unacceptable software, and even then, according to the COTR, it took several attempts by UIE and "lots of help from [a] special government team" to get an acceptable product. Likewise, according to the contracting agency's technical inspector, one of the sources identified by UIE in its response, and described by the protester as being "in the best position to judge the technical competence and performance of UIE," UIE "requires too much government direction, i.e., government has to show contractor how to do job, quality control."

Further, it does not appear that UIE's performance difficulties under contract -0008 were limited to the failure to perform which led to issuance of the cure notice. The contracting officer stated in the cure notice that "the unacceptable product initially produced by UIE . . . is not an isolated incident"; "[i]t appears that this is symptomatic of UIE's inability to plan, develop, and implement procedures which result in the delivery of high quality products on schedule and within budget." Although UIE responded during negotiations to MICOM's expression of concern regarding UIE's performance under this contract by noting that the contract had been extended, both the COTR and the contracting officer advised MICOM that the extension was issued only because all of the contracting agency's similar contracts were being consolidated and the extension of UIE's contract was necessary so that the contracts would end on the same date. Likewise, while UIE enclosed with its BAFO two letters from the contracting agency commending a number of contractor personnel for their efforts, UIE failed to furnish the cover letter to the letters of commendation, which specifically noted that the letters: "recognize individual contributions. This, however, is not an indication that as a company, United Engineering's overall performance was up to the desired level." In summary, reported the COTR, the contracting agency had "not been able to confidently assign support requirements to UIE with the expectation that the requirements will be understood and executed in an effective manner."

While issuance of the cure notice led to replacement of UIE's program manager (PM) under contract -0008, and the new PM's efforts to cure the problems resulted in reports that UIE was "doing better" (according to the COTR) and that there had been no problems in the last 6 months (contracting officer), the COTR reported that UIE "still [has a] substantial learning curve. SW [software] is still weak." MICOM reports that the contracting agency's technical inspector--again, the agency official described by the protester as being "in the best position to judge the technical competence and performance of UIE"--similarly advised that "[o]n software engineering efforts, UIE doesn't currently have anyone to do the job."

MICOM also relied upon reports that UIE had encountered performance problems under other contracts. For example, under contract No. DAAH01- 89D-0139, for technical support, including software modeling, the COTR reported that UIE was "not up on state-of-art" in the software field, resulting in UIE being "[v]ery slow getting started" and having "[s]everal false starts," which led to revision of the delivery schedule. According to the COTR:

"[w]hen [we] finally got [an] acceptable product, it was due to lots of government work. They were supposed to be experts and government was more expert than they were.

"[UIE] [w]anted to do good job but didn't have what it took to do good job."

Although MICOM received a generally favorable report when, at UIE's suggestion during negotiations, it contacted the technical point of contact for one of the task orders under contract -0139, that task did not involve software engineering. Moreover, even that contact cautioned that because UIE was "very penny wise" and lacked initiative, UIE "only gives what [you] ask for." He added that since they are a small company and lack resources, the agency can only "give them things [they are] good at." According to the agency, the contact questioned whether UIE could perform as well on the sizable contract effort contemplated here as it could on a smaller contract effort.

Under another contract (No. DAAH01-92-C-0312), for system engineering and integration support, the engineering supervisor reported that UIE's weakness was in "[i]n-depth technical expertise." According to the engineering supervisor, UIE "[did not] have staff"; as a result, the "[p]rime doesn't do engineering. Sub does it." Although UIE claimed during negotiations that it had provided one-third of the engineers supporting the project office, the supervisor of the engineering supervisor confirmed that the project office looks to the subcontractor for all technical support. The overall supervisor stated that:

"the project office does not look to UIE for technical support and doubts [it] could get it from them. [The subcontractor] provides all leadership and direction to the UIE employees, and [the subcontractor] provides all the technical support to the project office."

MICOM reasonably concluded that these reports of past performance problems and demonstrated inadequate software engineering capabilities raised "serious doubts that the UIE team could successfully perform this effort if managed by UIE and with two-thirds of the technical work proposed to be done by UIE."

The fact that UIE proposed to subcontract approximately 37 percent of the contract effort to subcontractors that received good to superior software engineering experience ratings does not change our conclusion. This is because, while the subcontractors in question were to perform approximately 37 percent of the overall contract effort, UIE's proposal indicated that UIE would take the lead in significant areas of contract performance, including contract technical management, program management, task requirements, acquisition/development engineering support, systems and management analysis and assistance, maintaining state-of-the-art practice, operational and acceptance testing, data collection, reduction and analysis, and software verification and validation. MICOM reports that UIE's prior performance problems are related to these areas. UIE does not adequately explain, nor is it otherwise evident, why its performance problems and demonstrated weaknesses with respect to software engineering capability are not relevant to the significantly larger portion of the contract that UIE would perform. In addition, given the reported weaknesses in UIE's software engineering capabilities, we think the agency reasonably was concerned that there was a significant risk that UIE lacked the technical expertise necessary to effectively manage the performance of its subcontractors in the software area.

The fact that MICOM received more ratings of inadequate performance for EER (10 of 679 responses) than for UIE (1 of 437 responses) does not demonstrate that the performance risk ratings for UIE (high) and EER (low) were unreasonable. The difference in risk ratings essentially reflected the fact that MICOM had received reports of significant problems on the part of UIE, but not EER, in meeting performance and delivery requirements related to software engineering. As noted by MICOM, while UIE received an inadequate performance rating in the technical area, and a total of five adequate/inadequate ratings in the technical and delivery areas, EER received no rating lower than good in the technical area or lower than adequate in the delivery area. Rather, seven of EER's inadequate ratings were in the area of cost (with one in the personnel area and two in the subcontractor area). MICOM found that EER's prior cost difficulties resulted from a number of factors, including the failure to anticipate the complexity of the required work and delays in obtaining clearances for proposed staff. Based on EER's proposed approach to cost control under the contemplated contract, and the fact that EER was the incumbent contractor and possessed an already cleared staff (and, presumably, knowledge of the contract requirements), MICOM determined that EER's prior cost difficulties were unlikely to reoccur here. Having evaluated EER as demonstrating outstanding past performance with respect to schedule, management and quality, MICOM assigned EER a low performance risk rating; there is no basis for finding this determination unreasonable. Nor, as discussed above, has the protester furnished any basis for questioning the agency's determination that the reports of UIE's past performance problems in the technical and delivery areas and demonstrated inadequate software engineering capabilities raised serious doubts as to whether UIE could successfully perform the contemplated contract effort. We conclude that the agency reasonably determined that UIE's past technical and delivery problems in the software engineering area were more indicative of high risk in performance of a software engineering contract than were EER's past cost problems. UIE's proposal thus warranted a higher performance risk rating.

RELEVANT EXPERIENCE

UIE claims that the source selection authority (SSA) improperly took into consideration EER's experience as the incumbent contractor and the fact that, as a result of that experience, EER possessed more SED and MICOM- related experience. For example, the SSA stated in this regard that:

"EER is the incumbent contractor for the current BASES effort. Their response to the RFP SOW was thorough. . . . They have a good knowledge of the software and computer resources in the tactical weapons systems at SED, and current working knowledge of the tools and procedures needed to perform the RFP SOW [statement of work] requirements. The proposal presented a very good description of their experience with the ADA programming language and with other languages used in tactical weapons systems. EER also presented their in-depth knowledge and experience of software development in accordance with [Department of Defense Standard No. 2167A (DOD-STD-2167A)]. This extensive background experience and capability made the proposal risk low. The claims made by EER were substantiated by numerous examples of support to SED projects."

In addition, UIE argues that MICOM improperly took into consideration UIE's relative lack of experience with MICOM systems and the fact that UIE's inadequate rating was received for the contract (-0008) involving "the most MICOM experience."

We find no improper consideration of offerors' experience. The solicitation expressly provided for consideration of relevant experience under a number of evaluation factors. The RFP stated that the agency would make a performance risk assessment based upon each offeror's "relevant and recent" experience as it related "to the probability of successful completion of the required effort." For the cited contracts, offerors were required to explain the "similarities of that work with the work required by this solicitation." Further, the solicitation provided that the required proposal risk evaluation would "focus on those aspects of the offeror's proposal which may tend to increase confidence in the ability to satisfy the contractual requirements in a timely and qualitative manner," including such factors as an offeror's "lack of corporate experience directly related to this requirement." In addition, under the technical evaluation factor, the solicitation provided for consideration of (1) "the offeror's experience with the development and evaluation of software in Ada, higher order languages, in assembly languages and the languages of [weapons] systems shown in Appendix B of the SOW"; and (2) "the offeror's experience and knowledge with DOD-STD- 2167A." In view of these solicitation provisions providing for consideration of relevant experience, and given the fact that the contemplated contract called for furnishing software engineering support to SED/MICOM in performance of its mission to support battlefield automated systems, MICOM properly considered as an advantage EER's superior experience with, and resulting knowledge of, SED/MICOM-supported systems, and as a disadvantage UIE's relative lack of experience in this regard. Likewise, the agency could properly take into account, in determining "the probability of successful completion of the required effort," the fact that UIE encountered performance problems on a contract for related services.

PROPOSAL RISK

UIE challenges MICOM's assessment of an increased proposal risk as a result of UIE's proposal of inadequate indirect rates. In this regard, as noted above, the solicitation generally provided that an evaluation of proposal risk would be undertaken as part of the technical, management and price evaluations, and that this evaluation would "focus on those aspects of the offeror's proposal which may tend to decrease confidence in the ability to satisfy the contractual requirement in a timely and qualitative manner consistent with the proposed approach."

The Defense Contract Audit Agency (DCAA), in examining the realism of UIE's initially proposed costs, noted that UIE had proposed indirect rates which in many instances were below its actual, incurred rates for fiscal year 1993; this discrepancy contributed to DCAA's overall finding of a potential underestimation of costs by UIE. UIE had proposed a newly created division for performance of this contract, and DCAA reportedly believed that it lacked the data to recommend specific indirect rates; instead, DCAA generally recommended that MICOM consider negotiating indirect rate ceilings with UIE. Although, after being advised during negotiations of the agency's concern in this regard, UIE slightly increased most of the proposed general and administrative rates in its BAFO, the protester lowered its proposed labor overhead rates still further, resulting in an overall net decrease in its proposed indirect rates. MICOM reports that, as a result, UIE's "Most Probable Price [- -i.e., evaluated cost--] . . . incorporated the initially proposed rates," that is, was adjusted upward by approximately $2.4 million.

Given this upward adjustment in evaluated cost, UIE challenges MICOM's concurrent determination that UIE's proposal of lower indirect rates than it historically incurred "raises some doubt that UIE can effect its proposed technical approach and staffing within its proposed cost" and thereby increased overall proposal risk. UIE argues that it was improper for the agency to assign proposal risk as a result of the inadequate indirect rates when it was also adjusting UIE's evaluated cost upward to account for them.

UIE's position incorrectly assumes that the upward adjustment in its evaluated cost accounted for and corrected its inadequate indirect rates. In fact, reports MICOM, "the adjustments did not eliminate the concern that UIE could well incur higher indirect rates than those proposed." In other words, adjusting the BAFO indirect rates upward to the levels proposed in the initial proposal did not, in the agency's view, fully account for DCAA's concern that the initial rates themselves were lower than what UIE historically had incurred and were inadequate. The overall proposal risk therefore was increased to reflect the agency's concern that the actual cost of performance could be even higher than the evaluated cost used in the cost/technical tradeoff. We find no basis for questioning the assessment of an increase in proposal risk on this basis.

COST/TECHNICAL TRADEOFF

UIE challenges the overall cost/technical tradeoff on the basis that MICOM has failed to identify and quantify the value of any technical benefits that would warrant award to the higher-cost offeror.

In a negotiated procurement, there is no requirement that award be made on the basis of lowest cost unless the RFP so specifies. Henry H. Hackett & Sons, B-237181, Feb. 1, 1990, 90-1 CPD Para. 136. Cost/technical tradeoffs may be made in deciding between competing proposals; the propriety of such a tradeoff turns not on the difference in technical scores or ratings per se, but on whether the agency's judgment concerning the significance of that difference was reasonable and adequately justified in light of the RFP evaluation scheme. Brunswick Defense, B-255764, Mar. 30, 1994, 94-1 CPD Para. 225. Federal Acquisition Regulation Sec. 15.612(d)(2) requires that documentation supporting the selection decision show the relative differences among proposals; their strengths, weaknesses and risks; and the basis and reasons for the decision. There is no requirement, however, that selection of a higher- cost proposal be justified through an exact quantification of the dollar value to the agency of the proposal's technical superiority. Picker Int'l, Inc., B-249699.3, Mar. 30, 1993, 93-1 CPD Para. 275. Further, even where a selection official does not specifically discuss the cost/technical tradeoff in the selection decision document, we will not object to the tradeoff if it is clearly supported by the record. Maytag Aircraft Corp., B-237068.3, Apr. 26, 1990, 90-1 CPD Para. 430.

The record supports the agency's cost/technical tradeoff. Although EER's proposal received the same overall rating as UIE's under the technical evaluation factor, which was significantly more important than the remaining evaluation factors, the SSA determined that as a result of superiority with respect to command, control, communications and intelligence engineering and interoperability testing, software quality assurance, and special level software engineering, EER's proposal in fact was superior to UIE's under the overall technical factor. At the same time, UIE's past performance record was viewed as casting serious doubt on whether it could successfully perform the required contract effort. Further, although the evaluated cost of UIE's proposal was somewhat lower than EER's, the agency determined that there was some risk that UIE's actual cost of performance would exceed its evaluated cost, thereby diminishing UIE's advantage in this regard. In light of these considerations, we think the agency reasonably determined that, notwithstanding the higher management rating and lower evaluated cost of UIE's proposal, EER's proposal was most advantageous under the stated evaluation criteria.

The protest is denied.

GAO Contacts

Office of Public Affairs