B-400500.8; B-400500.9, Velos, Inc., December 14, 2009
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.
1. Source selection decision, which concluded that the awardee's proposal for providing software was technically superior to the protester's proposal because of one aspect of the software, was unreasonable where the agency's technical evaluation panel had determined that this aspect of the software did not change the panel's determination that the proposals were technically equal and the source selection decision was substantially based on the evaluation of a consultant (not a panel member), who only considered the offerors' responses to a discussion question on this point and did not consider existing relevant information included in the panel's evaluation report and the offerors' proposals.
2. Performance risk assigned to the protester's proposal for failing to provide a more current Dun & Bradstreet (D&B) report was unreasonable where the contracting officer did not question the financial viability of the protester or the accuracy of the information in the report and only relied on the currency of the date on the D&B report to assign the rating.
3. Contracting officer's decision not to evaluate the protester's alternate technical and price proposals contained in its final proposal revisions (FPR) was reasonable where the agency expressly advised offerors in the request for FPR that it would not consider any revisions to the technical proposals or price, except for matters relating to software license and escrow provision.
Velos, Inc. of Fremont, California protests the award of a contract to Medidata Solutions, Inc. of New York, New York under request for proposals (RFP) No. 1406-04-08-RP-20201, issued by the Department of the Interior on behalf of the National Cancer Institute (NCI), for data capture and management system software. Velos, among other things, challenges the propriety of the source-selection decision.
The NCI sponsors and conducts clinical trials in the areas of cancer therapy, prevention, diagnosis, and epidemiology. These studies are conducted at the NCI and at NCI designated cancer centers, which collectively are referred to as the NCI Clinical Research Enterprise.
The RFP, issued on October 30, 2007, sought to acquire commercial-off-the-shelf data capture and management systems software, a perpetual use license to the software, and related installation, support, and maintenance services needed to support clinical and related human subjects cancer research within the NCI Clinical Research Enterprise. The contractor was required to deliver commercial clinical management software that met the RFP requirements, including updated versions of the software and technical support. The RFP contemplated the award of a fixed-price contract for a 6‑month base period with 9 option years.
Award was to be made to the offeror whose proposal represented the best-value considering the following evaluation factors: (1) technical approach, (2) business experience/history, (3) past performance, (4) small disadvantaged business participation, (5) small business participation, and (6) price. The technical approach factor was the most important technical factor, followed by the equally weighted business experience/history and past performance factors, followed by the least and equally weighted small disadvantaged business participation and small business participation factors. The combined weight of the technical factors was more important than price. RFP sect. M.1. The RFP further stated that "[t]he government will evaluate whether it shall be granted a perpetual use license in accordance with the requirements [stated in the RFP]. Contractors unable to comply with this requirement will not be considered for award." Amended RFP sect. C.1.
Eleven offerors submitted proposals by the December 10 closing date. A technical evaluation panel (TEP) evaluated proposals under the five technical factors utilizing an adjectival rating scale. Four proposals, including Velos's (rated very good overall) and Medidata's (rated very good overall), were included in the competitive range. Following oral presentations, discussions, and the receipt of final proposal revisions (FPR) on March 11, 2008, it appeared to the contracting officer that Velos was in line for award, based on Velos's proposal's significant price advantage and very good overall technical rating.
Thus, the contracting officer decided to conduct exclusive negotiations with Velos to resolve issues related to its application program interface (API)  and commercial software license. Velos revised the technical and price sections of its proposal to address the API requirement and to increase its price to [DELETED]. Meanwhile, extensive negotiations were conducted with Velos in an attempt to reach agreement upon the terms of Velos's software license, including a software escrow provision dealing with source code. Due to "numerous unresolved issues" between the agency and Velos regarding the terms of the software license, the agency terminated these negotiations and requested another round of FPRs from all the competitive range offerors, who were requested to address the API requirements and provide an acceptable commercial software license. All offerors' prices remained the same, except for Medidata, which significantly reduced its price to [DELETED]; the technical ratings of the proposals were unchanged.
Based on his review of the FPRs, the contracting officer found that Medidata's proposal represented the overall best value considering its very good technical rating, second lowest price, and acceptable license agreement. Velos's proposal, which was also rated very good and was lowest in price, was eliminated from further consideration solely because its license agreement was determined to be unacceptable. Medidata was awarded the contract on August 14. A number of protests of this award were filed by Velos and the other two unsuccessful offerors that had been included in the competitive range. These protests concerned the evaluation and rejection of the protesters' proposals, the evaluation of Medidata's proposal, and the conduct of discussions.
On November 28, we sustained Velos's protest finding that Velos was prejudicially misled in discussions concerning its license agreement. Velos, Inc; OmniComm Sys. Inc.; PercipEnz Techs., Inc., B-400500 et al., Nov. 28, 2008. We recommended that the agency consider amending the RFP to change or clarify its license requirements to reflect the agency's actual needs, conduct meaningful discussions with the offerors determined to be in the competitive range, request FPRs, and make a new source selection. Id. at 13-14. We denied the protests of the other two competitive range offerors and denied the remaining protest grounds raised by Velos, which primarily concerned the evaluation of its and Medidata's proposals. Id. at 11-12. In so doing, we found the evaluation documentation adequately supported the agency's rationale for the evaluations, including detailing the strengths and weaknesses of the various proposals that formed the basis for the agency's evaluation ratings and the selection decision. Id. at 12.
In implementing our recommendation, the agency first determined that only the proposals of Velos and Medidata would be included in the competitive range. AR, Tab 27, Award Summary (Aug. 7, 2009). On December 22, the contracting officer began exchanges with the two offerors by amending and clarifying the software license requirements included in the RFP and requesting a new license be submitted in compliance with those requirements. The letter also advised that further discussions and FPRs would be requested. The letter stated that "no additional technical proposal information other than the license and escrow agreement is permitted or will be considered by the Government when it requests FPRs" and "that the clarifications described in this letter should have no impact on the offerors' previously evaluated pricing." AR, Tab 2C, Discussions Letter (Dec. 22, 2008). In response to this letter, Velos requested the agency to permit pricing changes in order to take into account changes in business and economic conditions. AR, Tab 4, Velos's Letter (Dec. 28, 2008). On January 15, 2009, the agency further advised the offerors:
It is the Government's position that changes in pricing will be permitted only if they are directly related to the changes in the amended RFP in response to the recommendation of the GAO to clarify the license requirement. The Government will not consider pricing changes for any other reason. Offerors will have to demonstrate that any pricing changes are related to the clarification of the license requirement. Reasonable and realistic pricing has already been established through a lengthy, time consuming, highly competitive process which already allowed for price revisions.
AR, Tab 5D, Vendor Q & A (Jan. 15, 2009), at 1. A similar admonition limiting/prohibiting pricing changes appeared in each subsequent request for revised proposals, even though Velos made various requests to be able to submit a revised price proposal. E.g., AR, Tab 8C, Discussion Letter (Jan. 28, 2009); Tab 7, Velos Letter to Agency (Jan. 20, 2009).
The contracting officer noted that the "very good" ratings assigned Medidata's and Velos's proposals by the technical evaluation panel (TEP), which was the basis for the prior source selection decisions, were documented with various strengths and weaknesses, but did not provide a basis for determining which of the two proposals was technically superior. Hearing Transcript (Tr.) at 167, 197, 200, 245-46. Consequently, in December 2008 and January 2009, the contracting officer requested the contracting officer's technical representative (COTR), who was not a member of the TEP, to review the previous consensus technical evaluations of Medidata's and Velos's proposals to determine if there were any significant issues for consideration by the contracting officer for a best-value determination. The COTR confirmed the reasonableness of the overall technical ratings of very good assigned to each offeror, but was concerned about a weakness concerning the Velos product identified by the TEP that "a considerable amount of business logic is stored in the database in the form of packages, stored procedures, functions, and triggers, etc.". The COTR found that a lack of separation between the business and persistence tiers of the software would be inconsistent with software engineering best practices and could negatively impact the government's objective of implementing the offeror's product widely throughout the NCI Clinical Research Enterprise. AR, COTR Technical Assessment Memorandum (July 15, 2009), at 1. As explained below, the COTR believed that this lack of separation in software tiers did not represent the best practices of software engineering because it would cause the software to have less flexibility in meeting the RFP requirements for integration, easy upgrades and scalability. Id. at 8-10.
On January 28, the agency issued a request for FPRs, with amendment No. 0005, which incorporated a revised RFP. AR, Tab 8C, Request for FPRs (Jan. 28, 2009). The amendment clarified the license requirement and changed verbiage in the statement of work. Specifically, while maintaining the basic requirement for commercial-off-the-shelf data capture and management system software, RFP amend. 5, at 7, the amendment otherwise replaced the term "commercial software" with the term "Software" elsewhere in the statement of work. RFP, Amend. 5. While the record is not entirely clear as to why this language was changed, the contracting officer testified that no substantive change to the RFP requirements was intended and that this was a matter of "drafting convenience." Tr. at 97-99.
After further exchanges with offerors, the agency revised its request for new FPRs to ask offerors to provide the following:
1) A license agreement (and escrow agreement if appropriate) for evaluation by the Government as directed by the RFP.
2) An Excel spreadsheet with your pricing that conforms to Section B. Document all assumptions on the spreadsheet. You are permitted to change your pricing only as it directly relates to the changes in the license requirements as indicated in updates . . . in the amended RFP which were undertaken at the recommendation of the GAO to clarify the license requirement. You must document the reasons for any changes in prices.
3) Financial information to show your current financial status and viability to support this contract throughout the 9.5 year term of the contract. Include your most recently available D&B [Dun and Bradstreet] report.
4) Elaborate on the separation between the persistence tier (RDBMS) and the business tier for your product. Please address the following issues in you answer.Separation between persistence and business tiers consistent with best practices for designing information systems.Business logic stored in the persistence layer in the form of packages, stored procedures, functions and triggers, etc.The extent to which your system can support multiple RDBMS systems.Your product's reliance on a particular RDBMS system can be mitigated so as to support alternative RDBMS systems throughout the multiple sites of the NCI Clinical Research Enterprise.
5) Respond to adverse past performance information that has already been provided to you.
You are advised that no other technical or price proposal changes or additional submissions are permitted.
AR, Tab 12C, Request for FPRs (Feb. 3, 2009), at 1-2.
FPRs from both offerors were submitted on February 4. Medidata confirmed its previous price proposal of [DELETED]. Velos submitted three alternative price proposals: Proposal A was the same as its previous proposal of [DELETED]; proposal B, priced at [DELETED], was based on [DELETED] to cover only those features in its software that Velos believed was required by the revised definition of Software in amendment 5 of the RFP; and Proposal C, priced at [DELETED], was based on [DELETED] that were not required by what it believed was the revised definition of Software. Velos advised the agency that revisions to its technical proposal, as reflected in its alternate proposals were justified because of the deletion of the word commercial from the definition of software appearing in the RFP's statement of work. Velos explained
This change is significant to a company like Velos for whom the term "Software" as defined in the RFP, requires [DELETED] Velos's standard commercial software. For Velos, NCI's substitution of the term "Software" for "commercial software" this specifies software requirements that are narrower than that which Velos's "commercial software" satisfies.
AR, Tab 14B, Velos FPR (Feb. 3, 2009), at 1-3. The contracting officer did not consider alternative proposals B and C because he found that they did not comply with the limitations/prohibitions on price and technical revisions set forth above. AR, Tab 27, Award Summary (Aug. 7, 2009), at 19-20.
Both offerors provided further information regarding the separation between the persistence tier and the business tier and related information as requested in item 4 of the request for FPRs (quoted above). Based on their evaluations of the FPRs, the TEP members unanimously concluded that no change was warranted to the technical ratings previously assigned to the proposals and that there was no TEP consensus for finding that one proposal was technically superior overall. AR, Tab 27, Award Summary, at 15; Tr. at 29.
While the overall ratings were not changed, each of the TEP members evaluated FPRs as to item 4 of the FPR request and documented their individual evaluations in some depth. AR, Tab 16, TEP's Evaluation Sheets on Medidata's FPR; Tab 17, TEP's Evaluation Sheets on Velos's FPR. In so doing, some TEP members' comments made positive remarks about Velos's FPR and Medidata's FPR with regard to item 4 and some expressed concerns about these offerors' responses. For example, one TEP member, upon reviewing the FPRs, essentially found this aspect of Velos's technical proposal superior to Medidata's and provided supporting analyses for his evaluation. Specifically, this member found that Velos's response to this question was "clear and informative"; that "[t]hey describe the language used . . . and the nature and purpose of the tiers in their product"; and that "[t]hey further explain specific instances such as performance issues with large complex queries, the ability to easily adapt to different installations, and the facilitation of substantial audit log for regulatory purposes." He also found that Velos's "actual experience in support of many cancer centers (the intended target for this solicitation) has allowed them to develop clinical trial database best practices. Their response reflects a remarkable understanding of how software configuration issues can and do effect performance." AR, Tab 17, Evaluator No. 6 Evaluation Sheet (May 11, 2009), at 1-2. In contrast, he found that "the response by Medidata confirms our previous assessment of the software design," that "[t]he approach is supported by industry best practices and therefore is adequate." With regard to multiple RDBMS support by Medidata, he found that "[a]lthough the current version does not have that capability, the vendor expects that their product will include this capability in the next major release," but "not addressed is the level of difficulty that this development will involve" and "[t]he vendor does not provide any timeframe of when the new release (that has this capability) will be completed, tested, and installed." This TEP member concluded that "[Medidata's] response reiterates the separation of the persistence and business tiers and the locations of many components within the system," but "[u]unfortunately the response did not offer much reassurance as to the support of multiple RDBMS or alternative RDBMS." See AR, Tab 16, TEP Evaluator No. 6 Evaluation Sheet, at 1‑3.
Because the TEP did not determine which proposal was technically superior, the contracting officer again sought the input of the COTR regarding the offeror's responses to FPR item 4 concerning the separation of the business and persistence tiers to determine if there was any issue for consideration in the best value determination. In so doing, the contracting officer specifically requested the COTR to review the evaluations of the TEP and the responses of the offerors to FPR item 4. AR, Tab 23, COTR's Technical Assessment Memorandum (July 15, 2009), at 2; Tab 25, Award Summary, at 25.
In a detailed memorandum, the COTR concluded that Medidata's software was superior to Velos's software with regard to adhering to software engineering best practices because his review of the FPRs suggested that more business logic was being stored in the persistence tier of Velos's software than in Medidata's software. The COTR further found that because Medidata proposed to support more than one RDBMS in the future, Medidata's software was more likely to be deployed across the NCI Clinical Research Enterprise over a long period of time and that a cleaner separation of tiers would reduce the costs of supporting multiple databases in the future, which will allow NCI organizations to utilize the software more cost effectively, whereas the lesser separation in Velos's product meant a loss of flexibility and scalability of the software. The COTR found that Medidata's software was the best value since the yearly cost to overcome the lack of separation could be approximately [DELETED], which exceeded the difference in price between the proposals. AR, Tab 23, COTR Technical Assessment Memorandum.
The contracting officer adopted the views of the COTR in his source selection document finding that Medidata's proposal was superior to Velos's under the technical approach factor. Repeating much of the COTR's analysis, he found that Medidata more closely adhered to software engineering best practices because Medidata's software had less business logic embedded in the persistence tier than Velos, and that Medidata had identified the amount of business logic embedded in the persistence tier, while Velos did not; that Medidata had a more sound approach for meeting the RFP objective for wide implementation of the offeror's product throughout the NCI Clinical Research Enterprise because it proposed support for multiple RDBMSs, rather than one RDBMS as proposed by Velos; and that Medidata's adherence to software engineering best practices provided the government with more flexibility and a sounder approach in meeting the RFP requirements for integration, easy upgrades and scalability. He also noted the cost implications associated with Velos's approach that were reported by the COTR. See AR, Tab 27, Award Summary, at 26.
The contracting officer also identified another discriminator in the source selection decision based on the offerors' FPR responses to item 3 of the request for FPRs (quoted above) that requested financial information to show current financial status and viability to support this contract throughout the 9.5-year term. After reviewing the FPRs, the contracting officer found, under the business experience/history factor, that "[t]here is less performance risk associated with Medidata's overall viability in performing for the term of the contract, as a result of Medidata providing a current D&B credit report from February 2009," as compared to Velos's providing an "out of date" D&B report from August 2008, which the contracting officer concluded was "minimally responsive to the Government's request for current information" and resulted in "less visibility into Velos's current financial status during the worst economic climate in decades." Id. at 28.
Focusing on these two discriminators, the contracting officer determined that Medidata's higher priced proposal represented the overall best value to the government. Id. at 27-29. Award was made to Medidata on August 7. After a debriefing, Velos's protests of the source selection decision and the failure to consider Velos's alternate price proposals followed.
Velos challenges the propriety of the contracting officer's source selection decision. Velos argues that the contracting officer's and COTR's view, that the degree of separation between the persistence and business tiers in Velos's software made its software less susceptible to integration, easy upgrades, and scalability than Medidata's software, was unreasonable and inconsistent with the RFP, and failed to reasonably consider the TEP's determination (confirmed during the corrective action) that this did not constitute a reason to find Medidata's proposal technically superior to Velos's proposal. Velos also challenges the reasonableness of the discriminator under the business experience/history factor based on its failure to provide a more current D&B report because the agency has not otherwise questioned Velos's financial viability.
In reviewing a protest of an agency's evaluation and source selection decision, we will not re-evaluate proposals, but will review the record to determine whether the evaluation and selection decision are reasonable and consistent with the stated evaluation criteria, and with applicable procurement laws and regulations. Spherix, Inc., B-294572, B-294572.2, Dec. 1, 2004, 2005 CPD para. 3 at 8.
As noted above, the TEP rated the technical proposals of Medidata and Velos very good overall and provided no basis for determining either proposal to be considered technically superior; this was the evaluation upon which Velos's proposal had been previously selected for award based upon its lower price. The contracting officer explained that the strengths and weaknesses of the two proposals "were more or less roughly a wash" and that was why the COTR was requested to evaluate the TEP's report, which formed the basis for the prior source selection, to review the report to ascertain whether there was a "discerning factor" between the proposals. Tr. at 66‑67. The COTR identified a weakness in Velos's proposal that pertained to the separation of tiers, even though that had been only regarded as a minor weakness by the TEP. See Tr. at 272. More information relating to this issue was requested from the offerors in the FPRs, but the TEP analysis of the FPR responses still found no reason to find that either proposal had further distinguished itself so as to be found technically superior. However, the contracting officer testified that, after reviewing the FPR responses to item 4 and the TEP's evaluation, he was not satisfied with the TEP's evaluation because the TEP did not identify which of the two proposals was technically superior. This led him to again seek the advice of the COTR, who was an expert in the field (as compared to the contracting officer who admittedly is not) to independently evaluate the offerors' responses to the FPRs, to ascertain whether either of proposals was technically superior. See Tr. at 126-128, 148, 159.
The contracting officer ultimately decided, based upon an evaluation of the FPRs, that Medidata's proposed software was technically superior to Velos's in terms of integration, upgrades, and scalability because of the relative amount of business logic stored in the persistence tier. Tr. at 126-28. As indicated, these judgments were substantially based upon the advice of the COTR, particularly given the contracting officer's admitted lack of expertise in this technical area. Tr. at 124-28. However, the record evidences that the COTR did not review information that was relevant to this conclusion existing in the evaluation record and that this failure undermines the reasonableness of his (and thus the contracting officer's) judgment.
Specifically, the COTR admitted at the hearing that despite the contracting officer's request that he consider the TEP's evaluation of the offerors' FPR responses to item 4, he chose not to consider this information. See Tr. at 399-402. Both the COTR and contracting officer testified that the contracting officer was not aware that the COTR had not reviewed the TEP's evaluation of the FPR responses before the contracting officer made his source selection decision. Tr. at 258, 407. The record shows that each of the TEP members provided a detailed evaluation of the FPR responses. While there were some variant opinions among the TEP regarding the FPR responses, at least one TEP member expressed the view that Velos's actual experience in support of many cancer centers has allowed them to develop clinical trial database best practices, that its [DELETED] has been scaled for use at many NCI Clinical research Enterprise facilities, and that its database was the industry standard. AR, Tab 17, Evaluator No. 6 Evaluation Sheet, at 1-2; see Hearing exh. No. 5 at 2-4 (Velos's original proposal identifying presence in the NCI Clinical Research Enterprise). At the hearing, the COTR indicated that he was not aware of the number of cancer research programs and clinical research programs where Velos's software was actually in use. See Tr. at 410-18.
Moreover, the COTR admitted that he formed his opinion as to the degree of integration, upgrades, and scalability of each offerors' software solely on the basis of the offerors' responses to the FPR, and that he did not review either Medidata's or Velos's earlier proposals to discern whether or not there was other evidence in the proposals that was relevant to the question of integration, upgrades, and scalability. See Tr. at 330, 406-07. Medidata's and Velos's technical proposals are obviously very relevant to this evaluation. Indeed, the record evidences that Velos's initial proposal described in detail its architecture, including the layers or tiers, which provided information on the questions of flexibility for purposes of scalability, integration and upgrades, while Medidata's initial proposal did not include the same level of details. See Hearing exh. 5, Velos's Initial Proposal, at 59-64; Medidata's Initial Proposal at 5‑89. Under cross examination, the COTR admitted that certain information in Velos's initial proposal of which he was unaware, which addressed the architecture of Velos's software, including the separation of tiers, was evidence that the software may have been less affected by the amount of business logic in the persistence tier. Tr. at 397-98, 442-43, 496-97. Moreover, the COTR also testified, during cross examination, that neither Medidata nor Velos responded to item 4 in sufficient detail for him to determine precisely the amount of business logic that resided in the persistence tier versus the business tier and how this would affect scalability. See Tr. at 358-61, 373, 445.
Thus, it appears that the TEP's FPR evaluation and the offerors' technical proposals included relevant information, which could have affected the final evaluation judgment. Because the COTR did not consider this information, we find his evaluation was not reasonably based upon the entire evaluation record. See Carson Helicopter Servs., Inc., B-299720, B-299720.2, July 30, 2007, 2007 CPD para. 142 at 8-9. While the COTR testified during the hearing that nothing in Velos's proposal that was brought to his attention during the hearing would have changed his opinion that Medidata's software was superior, Tr. at 443‑44, we accord little weight to this testimony given in the heat of litigation. Boeing Sikorsky Aircraft Support, B‑277263.2, B-277263.3, Sept. 29, 1997, 97-2 CPD para. 91 at 15. Since the contracting officer relied on the COTR's incomplete analysis to find that Medidata's proposal was technically superior and represented the best value, we find that the source selection decision was not reasonable. See Metro Mach. Corp., B‑295744, B‑295744.2, Apr. 21, 2005, 2005 CPD para. 112 at 11; Computer Info. Specialist, Inc., B‑293049, B‑293049.2, Jan. 23, 2004, 2004 CPD para. 1 at 3; OneSource Energy Servs., Inc., B‑283445, Nov. 19, 1999, 2000 CPD para. 109 at 10.
The agency nevertheless argues that to the extent that Velos failed to adequately respond to the agency's discussion question and FPR request on this matter, Velos assumed the risk that its proposal would be downgraded. However, given Velos's inclusion of a detailed discussion of its architecture in its original proposal, it was not unreasonable for Velos to have addressed item 4 of the FPR request more generally, given a reasonable expectation that the evaluators would be familiar with what Velos had already included in its proposal, as opposed to Medidata, whose initial proposal had not addressed its system architecture at the same level of detail. See Hearing exh. 5, Velos's Initial Proposal, at 59-64; Medidata's Initial Proposal at 5‑89. Moreover, despite the COTR's concern and focus on this issue as the possible discriminator between the technical proposals, the agency did not frame the discussion question in such a way that would have caused Velos to necessarily fully address the agency's concern that the amount of business logic in its persistence tier might negatively impact its software's susceptibility for integration, upgrades, and scalability.
Velos also protests the reasonableness of the contracting officer assigning a performance risk to its proposal under business experience/history for not providing a more current D&B report than did Medidata.
This factor included evaluating "the overall viability of the Offeror in performing for the term of the contract including financial viability and organizational stability." RFP sect. M.1. Item 3 of the agency's FPR request was for "financial information to show current financial status" to include "your most recently available D&B report." AR, Tab 12C, Request for FPRs (Feb. 3, 2009), at 1-2. In response to item 3, Velos provided its last D&B report dated August 2008; updated financial information from that previously submitted, including an updated income statement and balance sheet; and additional explanations addressing its financial viability. AR, Tab 14B, Velos FPR (Feb. 3, 2009) at 3-4, attachs.
The contracting officer has not expressed, nor does the record establish, that the agency had a specific basis to challenge the actual financial viability of Velos. Indeed, the contracting officer explained that the only concern was based on the date of the D&B report. Tr. at 79-83. Velos states that it furnished the most current financial reports on its company in its FPR and that the D&B report dated August 2008 provided by Velos was the most recent report in its possession. In the absence of any evidence that the contracting officer had any specific concerns about the financial viability of Velos, it was not reasonable to assign a performance risk solely on the basis of the date on Velos's D&B report, particularly since the agency did not request that Velos have its D&B report updated. Thus, this discriminator between the proposals also lacks a reasonable basis.
Finally, Velos contends that the contracting officer erred in not considering its alternate proposals submitted with its FPR. Velos argues that the RFP permitted alternate proposals and that its proposals were submitted because the revised RFP changed the definition from "commercial software" to "Software" in the RFP's statement of work. We find that Velos's interpretation of the guidelines that the contracting officer set for the submission of FPRs is not reasonable. As noted above, the contracting officer in December 2008 specifically advised the protester that no additional technical proposal information other than that pertaining to the license and escrow agreement would be permitted or considered by the government and advised that no additional price revisions would be accepted, unless related to the license and escrow agreement. These instructions were reiterated in subsequent FPR requests. Thus, the agency's decision to not consider Velos's alternate proposals was reasonable, given the ground rules established for the submission of FPRs. In this regard, Velos's alternate proposals clearly involved revisions to its previously submitted technical proposal and new pricing for issues not related to the license and escrow agreement. Thus, we think that the contracting officer properly determined that these alternate proposals could not be considered and this protest basis is denied.
For the reasons discussed above, we conclude that the source selection decision was flawed and sustain the protest.
We recommend that the agency reopen discussions if
appropriate, request final proposal revisions, and/or otherwise reevaluate
proposals, and make a new source selection.
If as a result of the reevaluation Medidata is not selected for award,
the agency should terminate Medidata's contract and make award to Velos. We also recommend that Velos be reimbursed
the reasonable costs of filing and pursuing the protests, including attorney's
fees. 4 C.F.R. sect. 21.8(d)(1). The protester should submit its certified
claim for such costs incurred, directly to the contracting agency within 60
days after receipt of this decision.
Lynn H. Gibson
Acting General Counsel
 The Department of the Interior, National Business Center, and Acquisition Services Directorate conducted the procurement on behalf of NCI.
 Under technical approach, the RFP stated that "the Government will evaluate the thoroughness, clarity, creativity, and soundness of the Offeror's understanding and approach in meeting each of the requirements and objectives in this solicitation," including that "[t]he contractor must demonstrate the use of a formal software development methodology and best practices in software engineering." RFP sect. M.1.
 Under business experience/history, the RFP stated that "[t]he Government will evaluate the nature of the Offeror's business experience and organization in meeting the requirements in the solicitation, as well as the overall viability of the Offeror in performing for the term of the contract including financial viability and organizational stability" to include "[e]valuating organizational stability through the provision of financial and customer base information." RFP sect. M.1.
 The adjectival ratings were excellent, very good, satisfactory, poor, and unacceptable.
 In this regard, Velos proposal, priced at [DELETED], was rated very good and Medidata's proposal, priced at [DELETED], was also rated very good and was considered to be next in line for award.
 An API is an interface that a software program implements in order to allow other software to interact with it.
 The TEP was composed of six members, who, according to the contracting officer, were experts in their respective field. Tr. at 55, 58.
 Our Office conducted a hearing to address the issues concerning the price/technical trade off and the source selection decision.
 The COTR is the Deputy Director and Acting Chief Operating Officer of the NCI's Center for Biomedical Informatics and Information Technology and an expert in the development and deployment of biomedical information systems. AR, Tab 23, COTR Technical Assessment Memorandum, at 1.
 The data capture management system software was anticipated to be based on a layered application; the way in which the software performs in usual practice is separated by at least three tiers. The bottom layer is the persistence tier, which is where the actual data resides and is known as the relational database management system (RDBMS). The next layer up is known as the business tier, which is where the work of the software is accomplished, such as querying the persistence tier as to where information is stored. The top layer is the presentation tier, which is what the user would see on the computer screen. See Tr. at 301-04.
 While it is not directly pertinent to this protest, we note that another request for FPRs was subsequently made because of concerns related to limitations placed on technical support of the software by each offeror and the agency requested offerors to consider revising their respective approaches to technical support of the software; both offerors affirmatively responded to this request. AR, Tabs 18-21, Requests for FPRs (June 15, 2009) and Velos's and Medidata's Responses (June 17, 2009).
 At least one TEP member expressed some concerns about Velos's software's use of the [DELETED]. E.g., AR, Tab 17, Evaluator No. 2 Worksheet, at 1.
 As a preliminary matter, the agency requests dismissal of the protest regarding the source selection, asserting that this protest is untimely because it was filed on September 3, more than 10 days after the protester was notified of the award on August 10. Velos's protest was filed at our Office on September 3 within 10 days after receiving its debriefing on August 24. However, the agency explains, without contradiction, that this was not a "required debriefing" because it was not requested within 3 days of being notified of the award but was requested on August 18. Agency Request for Dismissal at 5. Under our Bid Protest Regulations, protests other than those involving alleged solicitation improprieties, such as this, shall be filed not later than 10 days after the basis of protest is known or should have been known (whichever is earlier). 4 C.F.R. sect. 21.2(a)(2) (2009). Here, the record clearly evidences that Velos's protest of the source selection decision is based on the information it diligently obtained from the agency's debriefing and not information it should have known on August 10, given that the agency's award notice contained little information regarding the basis for the award decision. See AR, Tab 29, Award Notice (Aug. 10, 2009). In this regard, we note that even where a disappointed offeror does not secure a required debriefing, it continues to retain its right to file a protest within 10 days after its learns, or should have learned, the basis for protest, provided it has diligently pursued the matter, and this includes the right to file a timely protest based on information obtained during a debriefing that was not required. See Raith Eng'g & Mfg. Co., W.L.L., B-298333.3, Jan. 9, 2007, 2007 CPD para. 9 at 3. We find that Velos diligently pursued the information that it learned at the debriefing and its protest based on the information learned at the debriefing is timely filed.
In contrast, Velos's protest that Medidata's proposal was unacceptable for failing to meet several RFP requirements, also filed on September 3, is untimely filed because it was not based upon information learned at a required debriefing and was apparently based on information learned during the course of the prior protests; thus, Velos was required to protest these matters within 10 days of learning that Medidata was selected for award. See The MIL Corp., B-297508, B-297508.2, Jan. 26, 2006, 2006 CPD para. 34 at 5.
 There is no basis to object to the contracting officer's consultation with the COTR per se, even though he was not part of any formally constituted evaluation panel. See Litton Sys., Inc.; Varian Assocs., Inc., B-229921 et al., May 10, 1988, 88-1 CPD para. 448 at 8-9.
 While under the evaluation scheme the contracting officer had a duty to determine the best value between the two competing proposals, it appears that the contracting officer here believed that he was obligated to find one of the proposals to be technically superior as part of the best value determination. This is not the case. It is well settled that where proposals are technically equal, an award must be made based on price. See HMR Tech, LLC, B-295968, B-295968.2, May 19, 2005, 2005 CPD para. 101 at 8.
 This evaluator also was skeptical that Medidata's promise of an updated software release and stated his view that Medidata's response did not offer much reassurance as to the support of multiple RDBMS or alternative RDBMS. AR, Tab 16, TEP Evaluator No. 6 Evaluation Sheet, at 1‑3.
 On this point, at the hearing, the COTR admitted that his analysis was based on an assumption of a three-level tiered architecture, instead of the [DELETED] tiered system in Velos's software, and that Velos's [DELETED]-tiered architecture could allow for more flexibility and scalability in the software. See Tr. at 436-37, 442-43, 446-47.
 The record evidences that neither offeror identified precisely the exact amount or nature of business logic that its software contained in the persistence tier and business tier. See AR, Tab 13E, Medidata's FPR at 1; Tab 14B, Velos FPR at 5-6.
 The contracting officer explained that there was no requirement in the RFP for separation of the business and persistence tiers, that offerors were not required to include this information in their proposals to be acceptable, and that the evaluation criteria did not include a definition of software engineering best practices. Tr. at 67‑68. Thus, the difference in detail included in the proposals regarding system architecture is understandable.
 For example, offerors were not requested in Item 4 of the FPR request to quantify or describe the business logic stored in the persistence tier.
 In any case, the agency has explained why the wording changes in amendment No. 0005 did not change, but only clarified, the RFP's software requirements.