This is the accessible text file for GAO report number GAO-11-549R entitled 'Improvements Needed to Help Ensure Reliability of SBA's Performance Data on Procurement Center Representatives' which was released on June 15, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-11-549R: United States Government Accountability Office: Washington, DC 20548: June 15, 2011: The Honorable Mary L. Landrieu: Chair: The Honorable Olympia J. Snowe: Ranking Member: Committee on Small Business and Entrepreneurship: United States Senate: The Honorable Sam Graves: Chairman: The Honorable Nydia M. Velázquez: Ranking Member: Committee on Small Business: House of Representatives: Subject: Improvements Needed to Help Ensure Reliability of SBA's Performance Data on Procurement Center Representatives: This letter formally transmits the attached briefing slides (see enclosure I) in response to the mandate contained in Section 1312(c) of the Small Business Jobs Act of 2010, for GAO to conduct a study of the Small Business Administration's (SBA) Procurement Center Representatives (PCR) and Commercial Market Representatives (CMR), including ways to improve their effectiveness.[Footnote 1] To fulfill this mandate, we provided your staff a briefing on the results of this work in meetings with them on March 22, 23, and 24, 2011. Each year, the federal government awards hundreds of billions of dollars in contracts for goods and services--more than $500 billion in fiscal year 2010 alone. It uses this buying power to maximize procurement opportunities for small businesses through long-standing policies such as set-asides and requiring large contractors to set goals for using small business subcontractors. SBA's PCRs and CMRs play an important role in helping ensure that small businesses gain access to contracting and subcontracting opportunities. In particular, a PCR's key responsibilities include reviewing proposed agency contract events--such as potentially bundled or consolidated contracts--and making set-aside recommendations to agency contracting officers (through informal and formal means), reviewing agency small business programs (surveillance reviews), and counseling small businesses.[Footnote 2] In addition, PCRs can appeal a contracting officer's rejection of their formal set-aside recommendation.[Footnote 3] PCRs also review the proposed subcontracting plans of large prime contractors and provide advice and recommendations on them to contracting officers.[Footnote 4] A CMR's key responsibilities include counseling small businesses on obtaining subcontracts and performing "matchmaking" activities to link large prime contractors with small businesses. CMRs also conduct reviews of large prime contractors with subcontracting plans, including Subcontracting Orientation and Assistance Reviews, performance reviews, and compliance reviews. [Footnote 5] We and SBA's Office of the Inspector General (OIG) previously reported on resource constraints that limited the ability of PCRs and CMRs to effectively perform their functions. In May 2005, SBA's OIG reported that SBA had not reviewed the majority of potentially bundled contracts reported by procurement agencies because of limited staff and a weak tracking system.[Footnote 6] In September 2007, the SBA OIG reported that CMRs monitored less than half of the 2,200 large prime contractors in fiscal year 2006.[Footnote 7] In November 2008, we reported that years of SBA downsizing and budget reductions reduced staff resources and resulted in most PCRs covering multiple agencies and "buying activities" within agencies.[Footnote 8] We also reported that CMRs with whom we spoke had large portfolios (ranging from approximately 90 to 200 prime contractors), which diminished their ability to monitor prime contractors through compliance reviews. We recommended that SBA assess the resources allocated to PCRs and CMRs and develop a plan to better ensure that these staff could carry out their responsibilities.[Footnote 9] SBA agreed with this recommendation and stated it was assessing statutory requirements and resources for PCRs and CMRs to develop such a plan. In September 2010, SBA contracted with a consulting firm to conduct a study to define the optimal environment in which a PCR's efforts would have the maximum impact on directing contracts to qualified and capable small businesses. As of the date of this letter, SBA officials told us that they were reviewing the study results. In accordance with the mandate and discussions with committee offices, we identified (1) measures SBA uses to determine the effectiveness of PCRs and CMRs in carrying out their responsibilities; (2) key challenges PCRs and CMRs cited related to carrying out their responsibilities; and (3) options PCRs, CMRs, and other key stakeholders cite to increase the effectiveness of PCRs and CMRs, including advantages and disadvantages. To help address the objectives, we reviewed SBA data on PCR and CMR performance as reported in the agency's monthly Government Contracting Area Report (GCAR) for fiscal years 2006 to 2007 and 2009 to 2010. [Footnote 10] We talked with agency officials and staff about data quality control procedures and reviewed relevant documentation. We reviewed and analyzed relevant laws and regulations, SBA position descriptions, staff guidance for SBA's Prime Contracts and Subcontracting Assistance Programs, and SBA's Office of Government Contracting staffing directories from August 2008 and August 2010. We interviewed (1) SBA officials who manage PCRs and CMRs, including five of six area directors; (2) a random sample of 27 of 77 SBA staff with PCR or CMR responsibilities (stratified to represent each SBA area office); (3) contracting staff from six major buying activities at three federal agencies with assigned PCRs--the Department of Defense (DOD), General Services Administration (GSA), and Department of Veterans Affairs (VA); and (4) five national small business stakeholders and a regional group representing large prime contractors with active subcontracting plans. As agreed with your offices, we focused on the policies, procedures, and performance measures SBA has for PCRs and CMRs but did not independently evaluate the effectiveness of PCRs and CMRs. In our March 2011 briefing, we reported information on measures SBA uses to assess PCR and CMR effectiveness. We also stated that we would conduct additional work to further assess the reliability of the data SBA reports for these measures. Following our March 2011 briefing, we compared the data that three SBA area offices reported on the GCAR in July and August 2010 with documentation maintained by PCRs and CMRs for five measures--formal Form 70s issued to initiate small business set-asides, informal Form 70s issued to initiate small business set- asides, surveillance reviews conducted, subcontracting reviews, and training for federal agencies. We identified several instances where the GCAR data did not match the documentation or could not be verified based on the documentation provided. On the basis of these results, we do not believe that the data SBA reports in the GCAR for these five measures provide reliable information on PCRs' and CMRs' actual accomplishments. We provide additional details on the results of our assessment of SBA's GCAR data later in this letter. We conducted our work from October 2010 through June 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: PCRs and CMRs are the primary staff who implement SBA's prime contracts and subcontracting assistance programs, which are intended to increase contracting opportunities for small businesses and help ensure that small businesses receive a fair and equitable opportunity to participate in federal prime contracts and subcontracts. SBA's Office of Government Contracting administers the prime contracts and subcontracting assistance programs through SBA headquarters and six area offices located across the country.[Footnote 11] Area office directors are responsible for supervising these programs in the field, including the PCR and CMR staff. According to SBA, its 57 PCRs are assigned to approximately 868 buying activities (divisions that purchase goods and services) within 21 federal agencies and departments.[Footnote 12] Forty-one PCRs are colocated at a buying activity; other staff with PCR responsibilities are located at SBA offices. SBA assigns its 33 CMRs to prime contractors based on the CMR's location and other considerations. As previously stated, a PCR's key responsibilities include making recommendations to agency contracting officers that proposed contracts be set aside for eligible small businesses--through informal and formal means. A PCR generally will issue an informal Form 70 after discussing and reaching an agreement with a contracting officer to accept a recommended set-aside. If the PCR and contracting officer do not reach an agreement, the PCR can submit a formal Form 70 to the contracting officer, which effectively stops the contract action, pending further consideration. If the contracting officer refuses to accept the recommended set-aside addressed in the Form 70,SBA can appeal the rejection to the head of the contracting activity, then to the agency head (known as a secretarial appeal).[Footnote 13] PCRs and CMRs also may be required to conduct additional duties, including size determinations and certificates of competency. SBA conducts size determinations of businesses against which a protest has been filed (because they are believed to be other than small). SBA must complete these determinations within 15 business days of receiving a protest, if possible. Size cases are assigned to the SBA area office in the area where the business concern is located. Contracting officers may withhold award of a contract to a small business otherwise in line for award if they determine the firm is "nonresponsible."[Footnote 14] Such determinations must be referred to SBA which must, if requested by the small business, complete a certificate of competency review within 15 business days to determine whether the small business is responsible. If SBA issues a certificate of competency, the agency must generally award the contract to the firm. Of the 27 PCRs and CMRs we interviewed, 17 had responsibilities to conduct size determinations or certificates of competency. SBA's GCAR provides information on various PCR and CMR activities. PCRs and CMRs are required to enter details on their activities into the spreadsheet-based report on a monthly basis. According to SBA standard operating procedures, a program manager consolidates the information by area office and forwards the report to the Associate Administrator for Government Contracting. Although SBA Performance Goals and Measures Provide Some Information on Key PCR and CMR Activities, Data Reliability Issues May Limit the Usefulness of These Data: SBA's Office of Government Contracting has performance goals and measures related to key PCR and CMR activities. For example, for fiscal year 2010, PCRs and CMRs were expected to (1) influence $6.7 billion of procurements for small business programs (by making formal and informal recommendations through Form 70s on specific contracts), (2) conduct 42 surveillance and follow-up reviews, (3) conduct 1,220 subcontracting reviews, and (4) conduct 40 training sessions for federal agencies (contracting staff). According to SBA, PCRs and CMRs generally exceeded these goals in fiscal year 2010. However, our review of SBA's GCAR data showed that data reliability issues may limit the usefulness of these measures for monitoring PCR and CMR performance and actual accomplishments. Our comparison of selected GCAR data reported by three SBA area offices in July and August 2010 against documentation maintained by PCRs and CMRs showed that GCAR data often did not match the documentation or could not be verified based on the documentation. Specifically, * Formal Form 70s issued to initiate small business set-asides: We reviewed $32.1 million reported on the GCAR in July 2010 for four formal recommendations that PCRs made. For formal Form 70s, SBA guidance requires PCRs to report the base year value for multiyear contracts or the total value for a one-time or 1-year award. For one recommendation, we found that the GCAR lists an amount of $4.5 million, but the documentation appeared to support an amount of $800,000. The GCAR lists $10 million for another recommendation; however, the documentation appeared to support an amount of $7 million. For the remaining two recommendations, we could not verify the amount reported on the GCAR because the supporting documentation lacked sufficient detail. * Informal Form 70s issued to initiate small business set-asides: We reviewed $68.5 million reported on the GCAR in July 2010 for 36 informal recommendations PCRs made. For informal Form 70s, SBA guidance requires PCRs to report the base year value for multiyear contracts or the total value for a one-time or 1-year award. We found that seven informal recommendations lacked supporting documentation with sufficient detail to determine the base year value or one-time or 1-year award value of the contract. * Surveillance reviews conducted: We reviewed seven surveillance reviews reported on the GCAR for August 2010. We could not verify the date for one review because the supporting documentation lacked sufficient detail. SBA officials told us that a second review was conducted in June 2010 but was incorrectly reported in August 2010. * Subcontracting reviews: We reviewed documentation for the 67 subcontracting reviews reported on the GCAR for July 2010. SBA provided documentation for 41 of these reviews. SBA officials told us they could not locate documentation for 26 reviews that they said were conducted by a staff person who had recently retired. In addition, we could not verify the date for 13 of the 41 reviews because the supporting documentation lacked sufficient detail. One review appeared to have been conducted in September 2010 based on the supporting documentation but was reported in July 2010. * Other training-federal agencies (contracting activities): We reviewed documentation for the 133 federal agency training events reported on the GCAR for July 2010. SBA officials told us that one area office reported the number of attendees (123) rather than the number of events (8), resulting in an overstatement of 115 training events reported for the month. In addition, we could not verify the date or the purpose for three events because the supporting documentation lacked sufficient detail. Four training events also appeared to have taken place in August 2010 based on the supporting documentation but were reported in July 2010. According to our internal control standards, for an agency to run and control its operations, it must have relevant, reliable information relating to internal events.[Footnote 15] This information should be recorded and communicated to management and others within the agency who need it to carry out their responsibilities. SBA's standard operating procedures require PCRs and CMRs to submit a productivity report and other information to their area director each month and maintain backup documentation in their office for 3 years. These procedures also require area directors to review the records that PCRs submit to them and conduct an on-site review of the PCRs' records every other year (as other duties and travel funds permit), prepare a report of their review, and submit a copy to SBA headquarters. [Footnote 16] However, SBA has not communicated standards or consistently applied internal control procedures. More specifically, it has not provided clear and complete guidance for PCRs and CMRs for accurately recording and maintaining the appropriate backup documentation for accomplishments reported in the GCAR. For example, some PCRs we interviewed told us that their area directors relied on a system of trust and did not review the documentation supporting their accomplishments or conduct on-site reviews of their records; other PCRs told us that their area director selectively asked for supporting documentation. In addition, only one of the five area directors we interviewed told us that he reviewed PCRs' records to fully ensure they were maintaining the appropriate documentation supporting their reported accomplishments. Two other area directors told us that they spot-checked the data PCRs report in the GCAR by asking for supporting e-mail correspondence with agency officials or by comparing the reported data with e-mails on which they were "copied" addressees. SBA's ability to monitor the performance of PCRs and CMRs and determine whether established goals have been achieved is compromised when GCAR data are inaccurate. SBA officials told us that the Office of Government Contracting is planning to update the standard operating procedures for PCRs and CMRs by December 2011, including the requirements related to the documentation of data reported into the GCAR. PCRs and CMRs Cited Various Challenges Related to Carrying Out Their Responsibilities: * PCRs and CMRs we interviewed identified seven key challenges to effectively performing their duties. * PCRs and CMRs said conducting size determinations and issuing certificates of competency took priority over and reduced time for PCR and CMR duties. SBA officials told us staff reductions required them to cross-train most PCRs and CMRs on size determinations and certificates of competency. * CMRs told us that the CMR function increasingly has become part- time. According to SBA, more than half the staff with CMR functions spent 25 percent or less of their time on CMR duties as of November 1, 2010. * PCRs and CMRs told us that the lack of in-person interaction with the buying activities and prime contractors limited their ability to influence procurements and subcontracting opportunities. PCRs working at buying activities said their access to procurement planning discussions helped influence procurements. * PCRs told us that contracting officers may not understand the small business contracting provisions in the Federal Acquisition Regulation, and some did not know how to conduct market research to identify qualified small businesses during acquisition planning. A few PCRs said turnover at certain buying activities explained these gaps. * Many PCRs (13 of 22) told us that some agencies would not send procurements to them for review, although the Federal Acquisition Regulation requires agencies to provide certain procurements to SBA for review prior to award.[Footnote 17] SBA officials told us they were meeting with officials from three agencies to resolve this issue. * Some PCRs (5 of 22) told us because purchases made using the Federal Supply Schedule were excluded from set-aside requirements, their ability to influence these purchases was limited. Yet, federal contracting data indicate that in fiscal year 2009, $9 billion (or 40 percent) of schedule purchases went to small businesses. * PCRs and CMRs cited a lack of authority to influence subcontracting opportunities. PCRs told us that they had no means to dispute agency procurements if contracting officers did not use their recommendations on subcontracting plans. PCRs and CMRs also said it was difficult to enforce prime contractor performance under subcontracting plans because determining that a contractor was not acting in good faith was difficult. Anecdotal evidence from agency procurement staff and others provides additional insight into PCR and CMR effectiveness. Officials representing six major buying activities had mixed views of PCR effectiveness. Some said PCRs were useful in training contracting staff, discussing procurement strategies to include small businesses, and generally augmenting the resources of the agency small business office. Several others said working with PCRs slowed the procurement process, citing a lack of interaction between agency staff and PCRs. They cited other limitations on PCR effectiveness, including PCRs' lack of technical knowledge of agencies' specific contracting requirements, and limitations in SBA's server capacity that required some agencies to fax procurement materials to PCRs for review. Conversely, representatives of a group representing large prime contractors told us CMRs effectively helped them understand contracting regulations and provided information on legislative updates. Several Options Could Increase PCR and CMR Effectiveness: From interviews with PCRs, CMRs, and other stakeholders, five options emerged for increasing PCR and CMR effectiveness: (1) increase the PCR and CMR workforce; (2) remove size determination and certificate of competency duties from staff with PCR and CMR responsibilities; (3) increase opportunities for PCRs and CMRs to have in-person interactions with buying activities and large prime contractors; (4) increase SBA's server capacity; (5) increase small business training that PCRs conducted for agency contracting officers; and (6) allow PCRs to dispute a procurement if their recommendations on a subcontracting plan were not implemented. Possible advantages of these options include increasing the number of procurements reviewed or compliance reviews conducted, and improving PCR and CMR influence. Possible disadvantages include the need for increased resources and a change in statute for some options. These options are discussed in greater detail in enclosure I. Conclusions: SBA plays an important role in helping ensure that small businesses gain access to federal contracting opportunities. Among its staff, PCRs and CMRs play important roles in advocating for and advancing prime and subcontracting opportunities for small businesses. Thus, they are in key positions to help SBA achieve the goal of helping ensure such opportunities. However, our review of SBA's GCAR data indicated that SBA may not have accurate information, which is needed to determine how well staff have performed or that intended goals are being achieved. By better communicating standards and consistently applying internal control procedures, SBA could help ensure the accuracy and reliability of its GCAR data. More specifically, without clear and complete guidance for PCRs and CMRs to accurately record and maintain the appropriate backup documentation for accomplishments reported in the GCAR, SBA risks ongoing issues with the quality of its data. Similarly, until SBA takes steps to ensure the verification of data PCRs and CMRs submit monthly and completes periodic reviews of PCR and CMR records, it will lack the controls needed to better ensure the reliability of GCAR data. As a result, the usefulness of the GCAR data SBA uses to evaluate PCR and CMR performance will be limited. Recommendations for Executive Action: To help ensure that GCAR data are accurate and that SBA reliably can use the data to monitor PCR and CMR performance and determine whether established goals have been achieved, we recommend that SBA's Director of Government Contracting take the following two steps: * provide clear and complete guidance to PCRs and CMRs on accurately recording and maintaining the appropriate backup documentation for accomplishments reported in the GCAR monthly report, and: * require that monthly GCAR data are verified and that documentation for PCR and CMR records are periodically reviewed for quality and completeness. Agency Comments and Our Evaluation: We provided a draft of this letter and the attached briefing slides to SBA, DOD, GSA, and VA for review and comment. SBA's Acting Director for the Office of Government Contracting provided written comments that are presented in enclosure II. In addition, VA provided technical comments, which we incorporated where appropriate. DOD and GSA stated that they had no comments on our draft letter. In his letter, SBA's Acting Director for the Office of Government Contracting agreed with our recommendations and stated that SBA has begun to address them. To address the recommendation on enhancing guidance for recording and maintaining documentation, he stated that SBA has been updating the standard operating procedures for the PCR and CMR programs, and that the updates will provide clear instructions on what PCRs and CMRs should report for the GCAR monthly report, how the data should be reported, and how to maintain appropriate backup documentation for verification purposes. To address the recommendation on data verification and PCR and CMR record review, he stated that SBA will develop and implement a verification method that area directors and SBA headquarters staff will use to insure that data in the GCAR are accurate and reliable. We are sending copies of this report to the appropriate congressional committees, and the Administrator of the Small Business Administration and other interested parties. In addition, this report will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributors to this report were Marshall Hamlett (Assistant Director), William R. Chatlos, David Reed, Barbara Roesmann, and Kathryn Supinski. Signed by: William B. Shear: Director, Financial Markets and Community Investment: Enclosures: [End of section] Enclosure I: Briefing slides: The Role of SBA Procurement Center Representatives and Options for Increasing Their Effectiveness: Briefing for the Senate Committee on Small Business and Entrepreneurship and the House Committee on Small Business: March 2011: Introduction: The Small Business Jobs Act of 2010 (P.L. 111-240) includes a requirement for GAO to conduct a study on the Small Business Administration's (SBA) Procurement Center Representatives (PCR) and Commercial Market Representatives (CMR), including ways to improve their effectiveness. SBA's PCRs and CMRs work to increase business opportunities for small businesses and help ensure they receive a fair and equitable opportunity to participate in federal prime contracts and subcontracts. Preliminary data from fiscal year 2010 indicate the federal government awarded about $112 billion in contracts to small businesses. We and the SBA Inspector General previously have reported that resource constraints have limited the ability of PCRs and CMRs to advocate, review, and monitor small business contracting at federal agencies.[Footnote 18] Moreover, questions have been raised regarding the effectiveness of PCRs in breaking up bundled or consolidated federal contracts. Topics to Be Covered: The briefing will cover the following topics: * Objectives; * Scope and Methodology; * Summary of Findings; * Background; * Discussion by Objective. Objectives: In accordance with the mandate and discussions with committee offices, our study addressed the following key questions: * Measures SBA uses to determine the effectiveness of PCRs and CMRs in carrying out their responsibilities; * Key challenges PCRs and CMRs cite related to carrying out their responsibilities; and; * Options PCRs, CMRs, and other key stakeholders cite to increase the effectiveness of PCRs and CMRs, including the likely advantages and disadvantages of different options. Scope and Methodology: To help address our objectives, we took the following actions: * We reviewed SBA's Office of Government Contracting data on PCR and CMR performance for fiscal years 2006-2007 and 2009-2010. We talked with agency officials and staff about data quality control procedures and reviewed relevant documentation. Although we are still conducting audit work at the PCR/CMR level for the final report, we believe the data included in this briefing to be sufficiently reliable for the purposes of this report. We did not include data for fiscal year 2008 because SBA did not retain such data during a transition to a new reporting system. * We reviewed and analyzed relevant laws and regulations, SBA position descriptions, standard operating procedures for SBA's Prime Contracts and Subcontracting Assistance Programs, and SBA's Office of Government Contracting staffing directories from August 2008 and August 2010. * We interviewed SBA officials in charge of managing PCRs and CMRs, including five of six Area Directors. We also interviewed a random sample of 27 SBA staff with PCR or CMR responsibilities that we selected from a population of 77 SBA staff, stratified to include PCR and CMR representation from each of SBA's six area offices. The final sample included 22 staff with PCR responsibilities, 14 with CMR responsibilities, and 9 of the 27 had both. Although the sample was selected randomly with representation, it does not generalize to the total PCR/CMR population. * We interviewed contracting staff from six major buying activities (divisions that purchase goods and services) at three federal agencies where SBA has assigned PCRs—Department of Defense (DOD), General Services Administration (GSA), and the Department of Veterans Affairs (VA). * We interviewed five national small business stakeholders and a regional group representing large prime contractors with active subcontracting plans. * As agreed with your offices, we focused on the policies and procedures SBA has in place for PCRs and CMRs and measures of their performance but did not undertake audit work to independently evaluate the effectiveness of PCRs and CMRs. * We conducted our work from October 2010 through March 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Summary of Findings: SBA has performance goals and measures that provide some information on key PCR and CMR activities. For example, PCRs reported influencing $11.1 billion in procurements awarded to small business in fiscal year 2010 by recommending specific contracts for set-aside, compared with the agency's goal of $6.7 billion. However, SBA does not have goals and measures for three key responsibilities, including reviewing proposed bundled contracts and making recommendations to enhance small business participation. PCRs and CMRs cited various challenges that they believe limit their effectiveness. For example, several PCRs and CMRs who are required to conduct additional duties told us that the time constraints and complexity associated with these additional duties leaves them with less time to complete their core PCR and CMR functions. PCRs, CMRs, and other stakeholders cited several options that could help PCRs and CMRs further increase small business contracting opportunities, such as reducing the number of noncore functions that PCRs and CMRs perform and increasing in-person interactions with agency contracting staff and prime contractors. Background: PCRs and CMRs are the primary staff who implement SBA's Prime Contracts and Subcontracting Assistance Programs, which are intended to increase contracting opportunities for small businesses and help ensure that small businesses receive a fair and equitable opportunity to participate in federal prime contracts and subcontracts. SBA's Office of Government Contracting oversees the agency's PCR and CMR staff. According to SBA, its 57 PCRs are assigned to approximately 868 contracting offices (divisions that purchase goods and services) within 21 federal agencies and departments.[Footnote 19] Forty-one PCRs are colocated at a buying activity; other staff with PCR responsibilities are located at SBA offices. SBA assigns CMRs to prime contractors based on the CMR's location and other considerations. Figure: Background: Overview of PCR and CMR Responsibilities: [Refer to PDF for image: illustration] Procurement Center Representative (PCR): * Counsel and train small businesses on how to obtain federal contracts; * Review federal agency acquisitions, including proposed bundled acquisitions; * Recommend small business set-asides and strategies to enhance small business participation as prime contractors and subcontractors (with Federal agency contracting office); * Conduct surveillance reviews to monitor small business contracting at federal agencies (with Office of Small and Disadvantaged Business Utilization). Commercial Market Representative (CMR): * What contractors can I match together? - Small business subcontractors: Large prime contractors. * Counsel small businesses on marketing themselves to large prime contractors; * Counsel large prime contractors on their responsibilities to maximize subcontracting opportunities for small businesses; * Instruct on how to identify small businesses; * Maintain a portfolio of large prime contractors and conduct reviews (e.g., Subcontracting Orientation and Assistance Reviews, compliance, and performance reviews (desk audits)). Sources: GAO; Art Explosion. [End of figure] PCRs and CMRs also may be required to conduct additional duties, including size determinations and certificates of competency. * Size determinations: SBA conducts size determinations of businesses against which a protest has been filed (because they are believed to be other than small). SBA must complete these determinations within 15 business days of receiving a protest, if possible. Size cases are assigned to the SBA area office in the area where the business concern is located. * Certificates of Competency: Contracting officers may withhold award of a contract to a small business otherwise in line for award if they determine the firm is "nonresponsible."[Footnote 20] Such determinations must be referred to SBA, which must, if requested by the small business, complete a certificate of competency review within 15 business days to determine whether the small business is responsible. If SBA issues a certificate of competency, the agency must generally award the contract to the firm. * Of the 27 PCRs and CMRs we interviewed, 17 had responsibilities to conduct size determinations or certificates of competency. SBA Performance Goals and Measures Provide Some Information on Key PCR and CMR Activities: SBA's Office of Government Contracting has performance goals and measures that provide some information about the key activities PCRs and CMRs conduct—including making set-aside recommendations through informal and formal means, conducting surveillance reviews, conducting subcontracting reviews, and providing training. SBA data show that PCRs and CMRs generally exceeded goals for these activities in fiscal years 2009 and 2010. (See slide 13.) Figure: SBA Performance Goals and Measures Provide Some Information on Key PCR and CMR Activities: [Refer to PDF for image: 4 vertical bar graphs] Values are approximated from the graphs. Procurements influenced for Small business: Goal, 2009: $5.3 billion; Actual performance, 2009: $9.1 billion; Goal, 2010: $6.7 billion; Actual performance, 2010: $11.1 billion. Surveillance reviews: Goal, 2009: 32; Actual performance, 2009: 31; Goal, 2010: 42; Actual performance, 2010: 41. Subcontracting reviews: Goal, 2009: 1,109; Actual performance, 2009: 1,679; Goal, 2010: 1,220; Actual performance, 2010: 1,812. Federal agency trainings[A]: Goal, 2009: 30; Actual performance, 2009: 1,633; Goal, 2010: 40; Actual performance, 2010: 752. Source: GAO analysis. Note: The value of procurements influenced is the sum of appeal actions (Form 70) issued formally and informally. Subcontracting reviews includes data reported on the number of Subcontracting Orientation and Assistance Reviews, performance reviews (desk audits), and compliance reviews. [A] SBA does not report training data separately for PCRs and CMRs, but for all staff in the Office of Government Contracting, including Size Specialists and others. [End of figure] SBA does not have a separate goal and measure related to one key PCR activity—reviewing proposed bundled contracts and making recommendations to enhance small business participation. SBA officials acknowledged that the agency could improve data related to this activity. SBA officials added that the agency had reported more data on this activity in the past but had streamlined the amount of data reported to reduce administrative costs. SBA also does not have goals or measures related to two key CMR responsibilities—to facilitate the matching of large prime contractors with small businesses and instruct large prime contractors on identifying small businesses through the use of various tools. PCRs and CMRs Continue to Operate Under Resource Constraints: PCRs and CMRs continue to operate under resource constraints. We previously reported that overall agency downsizing and budget reductions contributed to a reduced PCR and CMR workforce, which limited their ability to carry out some core responsibilities. [Footnote 21] As of August 2008, SBA had 59 PCRs-16 of whom also had CMR responsibilities—and 31 CMRs-16 of whom had additional functions such as PCR duties, size determinations, and issuing certificates of competency. As of November 2010, SBA had 57 PCRs and 34 CMRs-31 of the CMRs had additional responsibilities. We recommended in our November 2008 report that SBA assess resources allocated for PCR and CMR functions and develop a plan to better ensure that these staff could carry out their responsibilities. [Footnote 22] SBA agreed with our recommendation and stated that it was assessing statutory requirements and resources for PCRs and CMRs to develop such a plan. In September 2010, SBA contracted with a consulting firm to conduct a study to define the optimal environment in which a PCR's efforts would have the maximum impact on directing contracts to qualified and capable small businesses. At the time of this briefing, SBA officials told us that they were reviewing the study results. Summary of Views on PCR Effectiveness: PCRs we interviewed generally rated themselves as highly effective in their ability to influence agency contracting decisions to include small business opportunities. PCRs most often cited their authority to dispute a procurement and the strength of a cooperative relationship with agency staff as factors enabling them to influence agency contracting decisions. In addition, officials from buying activities with whom we met said that their assigned PCR was useful in training contracting staff, discussing procurement strategies to involve small businesses, and augmenting the resources and supporting the decisions of agency small business offices. However, officials from four buying activities told us that a lack of interaction with their assigned PCR or their assigned PCR's lack of technical knowledge about agency procurements slowed the procurement process. According to SBA officials, PCRs are expected to be knowledgeable about federal acquisition regulations applicable to all federal buying activities rather than the technical specifications of individual contract requirements. Officials from buying activities also told us that SBA's limited server capacity slowed the procurement process, as contracting officers had to fax coordination records to SBA instead of transmitting the records via e-mail. SBA officials added that qualitative aspects of PCR work (such as the relationships built with the contracting officials at buying activities) were difficult to quantify but contributed to the effectiveness of these staff. Summary Views on CMR Effectiveness: CMRs we interviewed generally rated their ability to influence subcontracting opportunities for small businesses as moderately effective. CMRs said their influence generally stemmed from prime contractors being concerned about the impression they make with contracting officers (who may consider a contractor's past performance in fulfilling their subcontracting plan as criteria for awarding future contracts). Representatives of one group representing large prime contractors told us CMRs provided valuable information and assistance. They said CMRs provided interpretations of the Federal Acquisition Regulation and information on legislative updates on a regular basis. CMRs also provided them with information on how to report data in SBA's subcontracting reporting system.[Footnote 23] The representatives noted that the CMR role became more important about 5 years ago when the number of DOD regional councils declined and the Defense Contract Management Agency reduced resources to assist large prime contractors on small business subcontracting. PCRs and CMRs Cited Various Challenges Related to Carrying Out Their Responsibilities: Size and Certificate of Competency cases take priority over core PCR and CMR responsibilities: * According to SBA officials, most of its PCRs and CMRs are cross- trained in conducting size and certificate of competency determinations because of staff reductions in other positions. * However, 13 of the 17 PCRs and CMRs we interviewed with size and/or certificate of competency duties told us that the time constraints and complexities associated with completing these cases leave them with less time to review proposed procurement actions, recommend small business set-asides, and conduct compliance reviews of prime contractors. * SBA data show that the number of size determinations requested increased 44 percent in one year (from 488 in fiscal year 2009 to 705 in 2010), while the number of certificate of competency referrals received increased from 294 to 300 over the same time frame. * The number of SBA staff performing the CMR function on a part-time basis continues to increase. * As of August 2008 SBA had 31 CMRs, 16 of whom had additional job functions such as PCR duties, size determinations, and certificate of competency referrals, and these additional roles often took higher priority and reduced the amount of time CMRs spent on subcontracting assistance activities.[Footnote 24] * As of November 2010, SBA had 33 staff performing the CMR function, 30 of whom had additional duties, and 17 of the staff spent 25 percent or less of their time on CMR duties. Interactions with buying activities and prime contractors: * According to13 of the 22 PCRs we interviewed, the lack of in-person interaction at buying activities limits their ability to influence procurements for small businesses. * These PCRs also told us that being co-located at a buying activity enhances their ability to influence procurements for small businesses because it provides them with an opportunity to develop cooperative relationships with contracting staff and attend acquisition planning meetings. * Six CMRs told us that a lack of time and travel funds has limited their ability to meet in-person with prime contractors, which they said is a highly effective way to improve a contractor's small business subcontracting efforts. However, SBA officials told us there has not been a limit on travel funds in FY 2010 and FY 2011. Contracting officers may not fully understand Federal Acquisition Regulation (FAR) requirements: * Eleven of those we interviewed told us that contracting officers at buying activities may not fully understand the small business requirements contained in the FAR. In addition, some contracting officers did not know how to conduct market research to identify qualified small businesses or develop small business subcontracting goals. Three of them noted that this might be due to the turnover of contracting officers at certain buying activities. Buying activities do not submit proposed procurement actions to SBA for review: * The FAR requires that contracting officers provide certain proposed procurement actions to PCRs for review. However, 13 of the 22 PCRs we interviewed described instances in which buying activities did not submit actions to PCRs as required. Federal Supply Schedule (FSS) purchases not subject to small business set-aside requirements: * According to 5 of the 22 PCRs we interviewed, small businesses lose contracting opportunities when federal agencies make purchases using the FSS. * Regulations implementing the Small Business Act and FSS program exclude FSS purchases from small business set-aside requirements; however, federal agencies still may consider small businesses when making purchases using the FSS. * In fiscal year 2009, $9 billion (or 40 percent) of federal agency purchases made through the FSS went to small businesses. Difficulty enforcing prime contractor performance under subcontracting plans and ensuring subcontracting plan recommendations are implemented: * Large prime contractors are required to make a "good faith effort" to achieve the goals in their subcontracting plans. * However, 7 of the 27 PCRs and CMRs told us that determining that a contractor is not acting in good faith is difficult. * In addition, PCRs generally review the proposed subcontracting plans of large prime contractors prior to contract award and can make recommendations, but PCRs told us that they do not have any means of disputing the procurement if the contractors do not implement their recommendations. Options That Could Increase PCR and CMR Effectiveness: Through interviews with SBA Office of Government Contracting officials, PCRs and CMRs, officials from buying activities, and groups that represent small business interests and large prime contractors, we identified several options—and their associated advantages and disadvantages—that could increase the effectiveness of PCRs and CMRs. These options are not intended to be exhaustive. Table: Options That Could Increase PCR and CMR Effectiveness: Option: Increase the size of the PCR and CMR workforce; Advantages: * An increased PCR workforce could cover more buying activities and review more procurement actions; * An increased CMR workforce could conduct more compliance reviews and provide more outreach to prime contractors. implications for other SBA programs; Disadvantages: * These changes would require additional resources or a re-allocation of resources, which could have negative Option: Remove size determination and certificate of competency duties from staff who have PCR and CMR responsibilities; Advantages: * With the additional time, PCR and CMR staff told us they could review more procurements, cover more agencies, conduct more compliance reviews, and train more prime contractors and agency staff; Disadvantages: * Determining where the size determination and certificate of competency responsibilities would shift is unclear. Option: Increase opportunities for PCRs and CMRs to have in-person interactions and increase SBA's server capacity; Advantages: * PCRs and CMRs said increasing the in-person interactions with agency staff and prime contractors could increase their influence over procurements and subcontracting; * Increasing server capacity would result in more efficient procurement coordination; Disadvantages: * These changes would require additional resources or a re-allocation of resources, which could have negative implications for other SBA programs. Any additional travel could take time away from other PCR or CMR duties. Option: Increase small business training for agency contracting officers conducted by PCRs; Advantages: * PCRs informed us that training would be an effective method for maximizing small business participation as it makes contracting officers aware of the small business requirements and improves coordination with the PCR. Disadvantages: * It could be more effective to require an agency's small business office to train contracting officers because the small business staff is already on-site and a PCR would not have to travel to the agency. Option: Allow PCRs to dispute a procurement if recommendations related to its subcontracting plan are not implemented; Advantages: * With the potential for this action, the contracting officer would have incentives to more closely assess PCR advisory comments, which would lead to more meaningful subcontracting goals and increase the integrity of the subcontracting program; Disadvantages: * Agency officials did not think that PCRs had enough information about the overall contract and the prime contractor to accurately assess the subcontracting plan; * This option would require a change in the statute. [End of table] GAO on the Web: Web site: [hyperlink, http://www.gao.gov/] Contact: Chuck Young, Managing Director, Public Affairs, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: Copyright: This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. [End of briefing slides] Enclosure II: Agency comments: U.S. Small Business Administration: Washington, DC 20416: June 8, 2011: William B. Shear: Director, Financial Markets and Community Investment: U S. Government Accountability Office: 441 G Street NW: Washington, D C 20548: Dear Mt. Shear: The U.S. Small Business Administration (SBA) Office of Government Contacting (GC) has reviewed the GAO draft report entitled "Improvements Needed to Ensure Reliability of SBA's Performance Data on Procurement Center Representatives" The report responded to a statutory mandate for GAO to study the effectiveness of SBA's Procurement Center Representatives (PCRs) and Commercial Marketing Representatives (CMRs) We appreciate the time spent by GAO staff to review CMR and PCR functions, interview SBA's GC staff including PCRs and CMRs, interview and consult with the small business staff at other Federal agencies and prime contractors, and develop two resulting recommendations We were pleased to see that officials at major buying activities found SBA PCRs to be helpful resources for developing small business procurement strategies and for training. Furthermore, SBA CMRs were recognized by large prime contracting as resources for interpreting contracting regulations and as knowledgeable on legislative changes. GAO's two recommendations both concern the data quality of SBA's Government Contacting Area Report (GCAR) The GCAR is an internal report used by GC management officials to measure the contribution of each of SBA's six government contracting Area Offices toward. SBA's overall government contracting-related goals. Data for the GCAR is compiled from the reports of individual GC employees, as submitted to SBA's GC Area Directors, GAO found that the data showed reliability issues that limit its usefulness for monitoring performance and accomplishments. SBA recognizes GAO's concerns and welcomes GAO's two recommendations: that SBA provide clear and complete guidance to PCRs and CMRs on accurately recording and maintaining the appropriate backup documentation for accomplishments reported in the GCAR, and that SBA require the GCAR data be verified with documentation periodically reviewed for quality and completeness. SBA will implement these recommendations during SBA's update of the Standard Operation Procedure (SOP) for the PCR and CMR programs The SOP contains internal policies and procedures for accomplishing PCR and CMR functions. As noted in GAO's report, the process of updating the SOP is currently ongoing. In the updated SOP, SBA will provide clear instructions on what PCRs and CMRs should report for the GCAR and how data should be reported SBA will address how to maintain appropriate backup documentation for data verification Finally, SBA will develop and implement a verification method that Area Directors and SBA headquarters staff will use to insure that data in the GCAR is accurate and reliable. Thank you for this opportunity to comment on the rely, draft report. Sincerely, Signed by: John W. Klein: Acting Director: Office of Government Contracting: [End of section] Footnotes: [1] Pub. L. No. 111-240, § 1312(c), 124 Stat. 2504, 2537 (2010). [2] Section 412 of the Small Business Administration Reauthorization Act of 1997 defines the bundling of contract requirements as the consolidation of two or more procurement requirements for goods or services previously provided or performed under separate smaller contracts into a solicitation of offers for a single contract that is likely to be unsuitable for award to a small business concern for various reasons. [3] The PCR may appeal a contracting officer's rejection of a set- aside recommendation to the head of the contracting activity within 2 business days after receipt of notice of the rejection. If the head of the contracting activity agrees that the contracting officer's rejection was appropriate, the PCR may ask the contracting officer to suspend action on the acquisition until the SBA Administrator appeals to the agency head. FAR § 19.505(b). [4] Large prime contractors must submit subcontracting plans that contain explicit goals for subcontracting to small businesses for contracts exceeding $650,000, or $1.5 million for construction projects, and that have subcontracting possibilities. FAR § 19.702(a). [5] Subcontracting Orientation and Assistance Reviews focus on large prime contractors newly assigned to a CMR or that hired a new Small Business Liaison Officer. A CMR may discuss the contractor's subcontracting reports and train the contractor on new small business program issues. The performance review focuses on a contractor's subcontracting achievements, as reported in the Electronic Subcontracting Reporting System. SBA guidance instructs CMRs to conduct at least one performance review annually on each large contractor in their portfolio. Compliance reviews are the key focus of CMR activities and the vehicle by which CMRs validate a contractor's subcontracting activity. [6] SBA, Office of the Inspector General, Audit of the Contract Bundling Process, Audit Report No. 5-20 (Washington, D.C.: May 20, 2005). [7] SBA, Office of the Inspector General, Review of SBA's Subcontracting Assistance Program, Audit Report No. 7-33 (Washington, D.C.: Sept. 28, 2007). [8] GAO, Small Business Administration: Agency Should Assess Resources Devoted to Contracting and Improve Several Processes in the 8(a) Program, [hyperlink, http://www.gao.gov/products/GAO-09-16] (Washington, D.C.: Nov. 21, 2008). [9] [hyperlink, http://www.gao.gov/products/GAO-09-16]. [10] We did not review data for fiscal year 2008 because SBA did not retain such data during a transition to a new reporting system. [11] SBA's Office of Government Contracting developed policies and procedures implementing the prime contracts and subcontracting assistance programs and incorporated them into its standard operating procedures in 2004 and 2006, respectively. [12] In fiscal year 2010, there were 3,415 buying activities with procurement activity. [13] If the contracting officer rejects a recommendation of the PCR, written notice shall be furnished within 5 working days of the contracting officer's receipt of the recommendation. FAR § 19.505(a). [14] Responsibility concerns such factors as whether a prospective contractor is able to comply with the required or proposed delivery or performance schedule, and whether it has the necessary organization, experience, and technical skills (or the ability to obtain them). FAR § 9.104-1. [15] GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999); Internal Control Management and Evaluation Tool, [hyperlink, http://www.gao.gov/products/GAO-01-1008G] (Washington, D.C.: August 2001). [16] If an on-site review is not feasible, SBA standard operating procedures note that the area director should request that PCRs provide them with records (for example, subcontracting plan review checklists or counseling logs) and then verify the method the PCRs used for obtaining socioeconomic reporting information, the proper documentation of activities that were attended at the contracting activity, and method of recording other reporting information required by the monthly PCR Productivity Report. [17] FAR 19.202-1(e)(1). [18] GAO, Small Business Administration: Agency Should Assess Resources Devoted to Contracting and Improve Several Processes in the 8(a) Program, [hyperlink, http://www.gao.gov/products/GAO-09-16] (Washington, D.C.: Nov. 21, 2008); SBA, Review of SBA's Subcontracting Assistance Program, Audit Report No. 7-33 (Washington, D.C., Sept. 28, 2007); and SBA, Audit of the Contract Bundling Process, Audit Report No. 5-20 (Washington, D.C., May 20, 2005). [19] In fiscal year 2010, there were 3,415 contracting offices with procurement activity. [20] Responsibility concerns such factors as whether a prospective contractor is able to comply with the required or proposed delivery or performance schedule, and whether it has the necessary organization, experience, and technical skills (or the ability to obtain them). FAR § 9.104-1. [21] [hyperlink, http://www.gao.gov/products/GAO-09-16]. [22] [hyperlink, http://www.gao.gov/products/GA0-09-16]. [23] Prime contractors self-report subcontracting activities into the electronic Subcontracting Reporting System, which SBA and GSA jointly administer and maintain. [24] See [hyperlink, http://www.gao.gov/products/GAO-09-16]. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: