Federal Protective Service:

Enhancements to Performance Measures and Data Quality Processes Could Improve Human Capital Planning

GAO-16-384: Published: Mar 24, 2016. Publicly Released: Mar 24, 2016.

Additional Materials:

Contact:

Lori Rectanus
(202) 512-2834
rectanusl@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Federal Protective Service (FPS)—which protects about 9,500 federal facilities—developed a Strategic Human Capital Plan ( Plan ) and engaged in related efforts that generally align with most key principles GAO identified for strategic workforce planning. Specifically, FPS

  • solicited input from key stakeholders, such as its employees and the National Protection and Programs Directorate (NPPD)—FPS's parent organization responsible for managing and overseeing FPS's human capital efforts;
  • determined critical skills and competencies;
  • developed human capital strategies (i.e., programs, policies, and processes) tailored to address identified gaps and needs in its workforce; and
  • identified actions that build the organizational capability to support the strategies.

However, FPS has not fully developed performance measures to evaluate progress toward goals, which is also a key principle for strategic workforce planning. For example, FPS has not identified performance measures for all of the Plan's strategies, has not included targets for the identified performance measures (e.g., a desired target for the “attrition rate” measure), and has not linked the measures to FPS's human capital goals. GAO's work on measuring program performance has found that targets and linkages are among the attributes of successful performance measures. FPS and NPPD officials said they plan on developing measures with targets and linkages but have not yet established time frames for completing these tasks. Without performance measures that have targets and linkages, it will be difficult for NPPD and FPS to assess whether the Plan and related efforts are helping achieve FPS's human capital goals and its facility protection mission.

FPS designed its staffing model—which identifies the federal workforce the agency needs to meet its mission—consistent with most key practices GAO identified for the design of staffing models, and FPS uses the model to help make management decisions. Specifically, FPS's model includes:

  • work activities and the time required to perform them;
  • facility risk levels, which determine the frequency with which FPS must complete facility security assessments; and
  • input from key stakeholders, including NPPD and some regional officials.

FPS officials said they took steps, such as reviewing work hour estimates, to ensure the quality of data used in the model—another key practice. FPS currently uses the model to help make human capital planning and other management decisions, but NPPD and FPS have not identified time frames for updating the model since its last update in August 2013. Furthermore, FPS cannot assure data quality in future updates to the model because it has no documented process for ensuring data quality. Without time frames for updating the model and guidance on ensuring data quality, NPPD and FPS may not have accurate estimates of staffing needs to make management decisions.

Why GAO Did This Study

The federal security workforce plays a crucial role in meeting the growing challenges of protecting federal facilities. FPS, within the Department of Homeland Security (DHS), worked with NPPD to develop a staffing model and a Plan in 2013 and 2015 to help FPS manage its workforce. A 2015 Senate Appropriations Committee report included a provision for GAO to review the Plan . The committee also asked GAO to evaluate the staffing model. GAO examined (1) FPS's Plan and related human capital planning efforts and (2) how FPS designed and uses its staffing model. GAO assessed FPS's Plan and model to determine if they aligned with key workforce-planning principles and practices for designing staffing models. GAO identified these principles and practices from prior work and other sources. GAO also interviewed NPPD and FPS officials in headquarters and three regions selected to obtain regional variation such as in the number of FPS staff.

What GAO Recommends

To improve FPS's human capital planning, GAO recommends that the Secretary of DHS direct NPPD and FPS to identify time frames for developing performance measures with targets that are explicitly aligned to FPS's goals, establish a plan and time frames for updating its staffing model, and develop and document guidance for ensuring the quality of staffing model data. DHS concurred with GAO's recommendations and outlined steps it plans to take to address them.

For more information, contact Lori Rectanus at (202) 512-2834 or rectanusl@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: FPS stated that NPPD is working with FPS to develop a recruitment and retention strategy that will include performance measures. This strategy will be complete by December 31, 2016. However, NPPD and FPS have not yet identified timeframes for developing performance measures for other human capital areas identified in FPS's human capital plan, such as leadership and knowledge management and building a results-oriented performance culture. Furthermore, because the recruitment and retention strategy is still under development, it is unclear whether the performance measures will have targets that are explicitly aligned to FPS's stated human capital goals. FPS plans to finalize all actions associated with this recommendation by March 2017.

    Recommendation: To help FPS enhance its strategic human capital planning efforts, the Secretary of Homeland Security should direct the Under Secretary of NPPD to work with the Director of FPS to identify time frames for developing human capital performance measures with targets that are explicitly aligned to FPS's stated human capital goals.

    Agency Affected: Department of Homeland Security

  2. Status: Closed - Implemented

    Comments: In March 2016, GAO reported that the Federal Protective Service (FPS) used the staffing model in conjunction with other management tools, professional judgment, and institutional knowledge to help inform human capital planning and budget requirements. However, GAO found that the staffing model did not reflect regular and unexpected changes in FPS's operating conditions that have occurred since the model's last update in August 2013. Further, GAO found that the National Protection and Programs Directorate (NPPD)--FPS's parent agency--and FPS had not established specific time frames for updating the model because NPPD had not determined whether update responsibility would remain with FPS, shift to NPPD, or become a shared responsibility under NPPD's planned reorganization. GAO, Standards for Internal Control in the Federal Government and associated guidance note that managers need timely analytical information to help make management decisions. Without a clear plan and time frames for updating the staffing model to reflect regular and unexpected changes in operating conditions, NPPD and FPS will have limited assurance of the accuracy of the model's estimates and their decisions regarding the FPS workforce. To address this problem, GAO recommended that NPPD work with FPS to establish a plan and time frames for updating FPS's staffing model regularly and for unexpected changes in operating conditions. According to documentation that FPS provided, FPS completed its Data Collection Plan in September 2016 and updated it through August 2017. This Data Collection Plan identified the frequency with which FPS is expected to update its staffing model. Specifically, the Data Collection Plan stated that FPS would update the staffing model annually to ensure that the model reflects regular and unexpected changes in operating conditions that occurred during the year or reflects changes that are planned to occur within the next year. The Data Collection Plan also stated that FPS would update the model every four years. This update will include a comprehensive review and analysis of assumptions as well as changes to data to reflect FPS priorities. Collectively, these updates will help increase assurance that decisions regarding the FPS workforce are based on accurate model estimates.

    Recommendation: To help FPS enhance its strategic human capital planning efforts, the Secretary of Homeland Security should direct the Under Secretary of NPPD to work with the Director of FPS to establish a plan and time frames for updating FPS's staffing model regularly and for unexpected changes in operating conditions.

    Agency Affected: Department of Homeland Security

  3. Status: Closed - Implemented

    Comments: In March 2016, GAO reported that the Federal Protective Service (FPS) designed its staffing model to align with three of four key practices GAO identified for the design of staffing models. GAO found that FPS took steps to ensure the quality of the data used in the model it developed in August 2013. However, GAO found that FPS could not assure data quality in future updates to its staffing model because it had not documented a process for ensuring data quality. As such, it was not clear whether the model would use quality data that reflect current operations conditions. GAO Standards for Internal Control in the Federal Government and associated guidance note that agencies should use a process to help ensure data quality. Documented processes on data quality can help ensure that data used in the model are reasonably free from error and bias and provide greater assurance to decision makers that they are using reliable and sound information that is produced from the model. Without a documented process for ensuring data quality, FPS may not be able to ensure that future updates to the staffing model will provide accurate estimates of staffing needs, putting FPS at risk of not fully understanding whether it has the staff it needs to perform its mission. To address this problem, GAO recommended that the National Protection and Programs Directorate (NPPD) - FPS's parent agency - work with FPS to develop and document guidance on the process to ensure the quality of its staffing model data, such as guidance on how to collect data, validate assumptions, and perform sensitivity analyses to assess the assumptions. According to documentation that FPS provided, FPS developed and updated a Data Collection Plan between September 2016 and August 2017, which describes the agency's process for ensuring data quality. This process includes methods for collecting and updating data, conducting data quality assessments to reduce data collection bias or errors, and performing a sensitivity analysis to test and validate assumptions. FPS has incorporated this process in its current staffing model. As a result, FPS is in a better position for ensuring data quality to provide assurance that the staffing model provides accurate estimates of staffing needs.

    Recommendation: To help FPS enhance its strategic human capital planning efforts, the Secretary of Homeland Security should direct the Under Secretary of NPPD to work with the Director of FPS to develop and document guidance on the process FPS will use to ensure the quality of its staffing model data, such as guidance on how to collect data, validate assumptions, and perform sensitivity analyses to assess the assumptions.

    Agency Affected: Department of Homeland Security

 

Explore the full database of GAO's Open Recommendations »

Sep 10, 2018

May 30, 2018

Dec 14, 2017

May 24, 2017

May 18, 2017

Apr 27, 2017

Dec 29, 2016

Sep 1, 2016

Aug 19, 2016

Looking for more? Browse all our products here