Small Business Administration:

Office of Advocacy Needs to Improve Controls over Research, Regulatory, and Workforce Planning Activities

GAO-14-525: Published: Jul 22, 2014. Publicly Released: Jul 22, 2014.

Additional Materials:

Contact:

Cindy Brown Barnes
(202) 512-8678
brownbarnesc@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Office of Advocacy (Advocacy) within the Small Business Administration (SBA) fulfills its mission by researching small business issues and providing input into federal rulemaking and related regulatory activities. However, GAO identified key areas in Advocacy's system of internal control that could be improved.

  • Research. GAO found that Advocacy did not ensure that its staff monitored the quality of the information the office disseminated, as required. GAO reviewed 20 selected research products and found that in 16 cases a required quality review had not been documented. Advocacy recently established a review policy for its research, but it does not include procedures for selecting the reviewers or documenting that a review occurred and how reviewer comments are addressed. GAO also found that Advocacy staff had not followed federal information quality guidelines to retain data and could not substantiate the quality of information in two cost-estimation reports—a research product it has contracted for every 5 years. Without better controls over its quality review process and efforts to substantiate the information it disseminates, Advocacy cannot ensure the validity of one of its core activities—research in support of small businesses.
  • Regulatory activities. Advocacy recently updated procedures for its regulatory activities, but these could be strengthened. GAO found the extent to which individual staff maintained records varied, in part, because the procedures lacked policies for documentation. For instance, the procedures state that when staff decide to intervene in the rulemaking process, they must follow up as appropriate with the interested groups to ensure that Advocacy has sufficient information and data to support its case. However, there is no policy that these interactions be documented. Federal internal control standards state that documentation and records should be maintained. If key procedures are not being documented, managers do not have an institutional record that agency goals and objectives in this area are being met. GAO also found that the Federal Advisory Committee Act's transparency and other requirements do not apply to Advocacy's meetings with stakeholders to get input on regulations (roundtables).
  • Workforce planning. Advocacy's workforce efforts include training and mentoring for new staff, but do not include succession planning, which is recommended by the Office of Personnel Management. According to federal internal control standards, effective management of a workforce is essential to achieving program results. Officials told GAO that Advocacy was a small office and that additional staff were hired on an as-needed basis. However, some key staff have been with Advocacy for many years and their experience will be difficult to replace. If Advocacy does not incorporate succession planning strategies into its workforce planning efforts, it is at risk of not having the skills and expertise to meet its mission when key staff leave or retire.

Why GAO Did This Study

Congress created Advocacy within SBA in 1976 as an independent voice for small businesses. Questions have recently been raised about Advocacy's efforts to represent small businesses in regulatory activities and some of its research on small business issues. In light of these questions, GAO was asked to review Advocacy's activities.

This report examines Advocacy's (1) research, (2) regulatory activities, and (3) workforce planning efforts. GAO analyzed Advocacy's research, comment letters, and other regulatory information for fiscal years 2008-2013; assessed Advocacy's policies and procedures against federal standards for internal control and information quality; and interviewed agency officials, and small business and industry representatives. GAO also analyzed the applicability of the Federal Advisory Committee Act to Advocacy roundtables.

What GAO Recommends

GAO makes several recommendations to improve the Office of Advocacy's controls over the quality of its research, the documentation of its regulatory activities, and workforce planning. In commenting on a draft of this report, Advocacy agreed with our recommendations and noted some steps it will take to address them.

For more information, contact Cindy Brown Barnes at (202) 512-8678 or brownbarnesc@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In August 2015, Office of Advocacy (Advocacy) provided GAO with the "Office of Economic Research Additional Information Quality Guidelines and Peer Review Policies" (revised August 2015). The guidance formalizes criteria for determining the type of review that should be assigned to each product. Advocacy's peer reviews are conducted either by internal Advocacy staff or external reviewers in relevant fields such as academics or technical experts. The guidance requires Advocacy staff to document and file the outcome of all such determinations. Furthermore, the guidance includes criteria for selecting the peer reviewers for Advocacy's external research (done by contracted researchers). Specifically, the guidance states that Advocacy staff will select a minimum of three qualified peer reviewers. Peer reviewer qualifications shall be determined according to the subject matter expertise in the relevant research field, as indicated by: educational background; occupation; publications and presentations in the relevant field (such as in professional and/or academic journals and conferences); and other relevant professional factors. Once the peer review is completed, the guidance indicates that the contracted researchers are to respond, via a memo to Advocacy staff, to all peer review comments. The guidance requires Advocacy staff to maintain files of these memos, all related peer review materials, and the related research contract.

    Recommendation: To improve Advocacy's system of internal control, and help to provide reasonable assurance that the office is meeting its mission, the Chief Counsel of Advocacy should strengthen the accountability of its research activities by enhancing its peer review policies and procedures by including written guidance on selecting peer reviewers and documenting all steps of the peer review process--whether a peer review occurred and how and to what extent peer reviewer comments were addressed.

    Agency Affected: Small Business Administration: Office of Advocacy

  2. Status: Closed - Implemented

    Comments: In August 2015, Office of Advocacy (Advocacy) officials revised their "Information Quality Guidelines, SBA Advocacy Office of Economic Research." The Advocacy guidelines now reflect the federal information quality guidelines, which require that all agencies producing and disseminating "influential statistical information" help to ensure a high degree of transparency about the data and methods used. Federal guidelines state that this is achieved in part by storing the data from influential studies to facilitate the reproducibility of such information by qualified third parties. Advocacy's new guidelines state that, for influential economic research, contracted researchers shall be required to disclose data resources in all final products, and Advocacy's internal files will record data robustness checks such as additional model fits and data checks. However, Advocacy will not require that all disseminated data be subjected to a reproducibility requirement and will weigh all the factors discussed in the federal information quality guidelines when deciding which data to retain. Those factors include, for example, the importance of the information and cost of acquiring additional data, data storage, and/or data programming resources, particularly in relation to both Advocacy's overall research budget and individual research contract cost ceilings. In addition, Advocacy's guidelines state that where data utilized in research is already publicly available, cost constraints may weigh in favor of appropriate citation to publicly available data and clear explanation of methodology within the research document rather than toward acquisition and storage of public data. For such data that should not be retained, the guidance outlines steps to substantiate them. Specifically, the guidance states that in addition to requiring contracted researchers to disclose data resources in all final products, internal Advocacy files will record data robustness checks such as additional model fits and data checks. With these revisions, Advocacy's guidance is now consistent with the federal information quality guidelines.

    Recommendation: To improve Advocacy's system of internal control, and help to provide reasonable assurance that the office is meeting its mission, the Chief Counsel of Advocacy should strengthen the accountability of its research activities by developing policies and procedures that reflect the federal information quality guidelines on retaining data for influential studies, and when not retaining data, taking additional steps to substantiate the quality of information disseminated.

    Agency Affected: Small Business Administration: Office of Advocacy

  3. Status: Closed - Implemented

    Comments: On March 26, 2015 Office of Advocacy (Advocacy) officials provided GAO with excerpts of additions to its Interagency Procedures Manual (revised September 2014). These updated procedures state that Advocacy staff are required to maintain records of their small business contacts. Staff are to maintain the contacts in a Due Diligence memo that also details, for example, a description of the rule or policy, the genesis of Advocacy involvement, a plan for outreach to small businesses, small businesses affected, and the economic impact of the rule. The excerpts also detail that the Chief Counsel (the head of Advocacy) must approve proposed roundtable agendas, speakers, agency participants, and discussion topics and the approval must be documented before invitations are sent to participants. Finally, it states that Advocacy's Administrative Support Branch will compile a file that will include a copy of the signed comment letter, due diligence memo, list of small businesses spoke to (name, organization, etc.), roundtable agenda, roundtable invite list, and roundtable sign-in sheet.

    Recommendation: To improve Advocacy's system of internal control, and help to provide reasonable assurance that the office is meeting its mission, the Chief Counsel of Advocacy should strengthen the accountability of its regulatory activities by developing policies and procedures to ensure that key elements of that work--such as sources of input for comment letters and roundtable discussions--are consistently documented.

    Agency Affected: Small Business Administration: Office of Advocacy

  4. Status: Closed - Implemented

    Comments: In December 2014, Office of Advocacy (Advocacy) issued a succession plan aimed at strengthening current and future leadership capacity within the office. The plan outlined a process for assessing the workforce needs of the office, anticipating retirement or other attrition, and preparing for any future skill gaps in Advocacy's workforce. The process includes five steps: (1) Identify critical positions. The plan identified nearly all of Advocacy's positions as mission critical, and notes that it is therefore vital that senior officials continuously assess the needs of the office and the skills of its workforce. This continuous assessment involves, for example, an annual review of federal agencies' regulatory agendas and quarterly meetings between Advocacy's personnel team--the Chief Counsel, the Deputy Chief Counsel, the Senior Advisor to the Chief Counsel and the Director of the Administrative Support Branch--and its senior staff to determine if there are office needs that are not being met. (2) Determine the skills needed for critical positions. Advocacy's personnel team holds an annual meeting to review each of Advocacy's position descriptions to determine if they need to be revised and to identify any needs for new or additional skills. The skill assessment is dependent upon numerous factors discussed in the plan, such as the upcoming federal agencies' regulatory agenda, Congressionally-mandated work, and budgetary increases or decreases. (3) Find and assess potential candidates. Advocacy fills staffing needs by examining both internal candidates and conducting external interviews. This process involves the annual review of current employees' ratings and skills assessments to determine if they may be a fit for any new positions being created within Advocacy or any positions that will soon be vacated due to retirements. To find qualified external candidates, Advocacy has identified several relevant talent pools to draw from, such as federal agencies, legal fellow programs, and private organizations. (4) Hire, train, and plan for filling gaps. These activities are achieved through Advocacy's onboarding process which includes providing internal training on specific subject matters, establishing a trial period for all new hires so that Advocacy can make any needed adjustments, conducting quarterly supervisory reviews, and mentoring. (5) Commitment to developing talent and promoting from within. Advocacy's personnel team meets with senior staff quarterly to determine if there are needs of the office that are not being met and whether there are employees in need of additional training. Each director and manager is asked to assess the skills and production potential of each of their employees as well as to determine if there are employees in need of improvement. The personnel team encourages managers to support their staff and develop lines of communication, training, and evaluation conducive to promoting employees from within.

    Recommendation: To improve Advocacy's system of internal control, and help to provide reasonable assurance that the office is meeting its mission, the Chief Counsel of Advocacy should improve its workforce planning efforts to be better prepared to meet its future workforce needs by incorporating succession planning.

    Agency Affected: Small Business Administration: Office of Advocacy

  5. Status: Closed - Implemented

    Comments: Office of Advocacy (Advocacy) created a public webpage in January 2015 that lists information about its roundtables. As of June 1, 2015, the webpage (https://www.sba.gov/category/advocacy-navigation-structure/regulatory-roundtables) has a link to past and upcoming roundtables (January 21, 2015 to June 10, 2015) and includes the meeting date and time, location, and agenda. In addition, the webpage states that Advocacy's roundtables are open to all interested persons, with the exception of the press, and provides an email address (advocacy@sba.gov) for anyone, including press, to submit a request to receive presentations made at the roundtable or be included in the regular distribution list.

    Recommendation: To improve Advocacy's system of internal control, and help to provide reasonable assurance that the office is meeting its mission, the Chief Counsel of Advocacy should coordinate with SBA officials who oversee website administration to comply with Advocacy's roundtable policy to make information on the events--agendas, presentation materials--publicly available on its website so that its regulatory activities are more transparent to the public.

    Agency Affected: Small Business Administration: Office of Advocacy

 

Explore the full database of GAO's Open Recommendations »

May 26, 2016

Apr 14, 2016

Apr 11, 2016

Mar 21, 2016

Feb 8, 2016

Jan 27, 2016

Nov 16, 2015

Oct 28, 2015

Sep 8, 2015

Jul 27, 2015

Looking for more? Browse all our products here