Digital Service Programs:

Assessing Results and Coordinating with Chief Information Officers Can Improve Delivery of Federal Projects

GAO-16-602: Published: Aug 15, 2016. Publicly Released: Sep 14, 2016.

Additional Materials:

Contact:

David A. Powner
(202) 512-9286
pownerd@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The General Service Administration's (GSA) 18F and Office of Management and Budget's (OMB) U.S. Digital Service (USDS) have provided a variety of services to agencies supporting their information technology (IT) efforts. Specifically, 18F staff helped 18 agencies with 32 projects and generally provided development and consulting services, including software development solutions and acquisition consulting. In addition, USDS provided assistance on 13 projects across 11 agencies and generally provided consulting services, including quality assurance, problem identification and recommendations, and software engineering. Further, according to GAO's survey, managers were generally satisfied with the services they received from 18F and USDS on these projects (see table).

Results of GAO Survey on Satisfaction with Digital Services Projects

Program

Very satisfied

Moderately satisfied

Neither satisfied nor dissatisfied

Moderately dissatisfied

No response

18F

16

7

0

3

5

U.S. Digital Service

6

3

0

0

4

Source: GAO survey of agency project managers that engaged with 18F and U.S. Digital Service. | GAO-16-602

Both 18F and USDS have partially implemented practices to identify and help agencies address problems with IT projects. Specifically, 18F has developed several outcome-oriented goals and related performance measures, as well as procedures for prioritizing projects; however, not all of its goals are outcome-oriented and it has not yet fully measured program performance. Similarly, USDS has developed goals, but they are not all outcome-oriented and it has established performance measures for only one of its goals. USDS has also measured progress for just one goal. Until 18F and USDS fully implement these practices, it will be difficult to hold the programs accountable for results.

Agencies are beginning to establish digital service teams. Of the 25 agencies included in the President's proposed funding for agency digital service teams, OMB has established charters with 6 agencies for their digital service teams. In addition, according to the Deputy USDS Administrator, USDS expects to establish charters with an additional 2 agencies by the end of the fiscal year—the Department of Education and the Small Business Administration. For the remaining 16 agencies, as of April 2016, 8 agencies reported that they plan to establish digital service teams but have yet to establish charters with USDS. The other 9 agencies reported that they do not plan to establish digital service teams by September 2016 and most noted that it was because they did not receive requested funding to do so. Further, of the 4 agencies GAO selected to review, only 1 has defined the relationship between its digital service team and the agency Chief Information Officer (CIO). This is due, in part, to the fact that USDS policy does not describe the expected relationship between CIOs and these teams. Until OMB updates its policy and ensures that the responsibilities between the CIOs and digital services teams are clearly defined, it is unclear whether CIOs will be able to fulfill their statutory responsibilities with respect to IT management of the projects undertaken by the digital service teams.

Why GAO Did This Study

In an effort to improve IT across the federal government, in March 2014 GSA established 18F, which provides IT services (e.g., develop websites) to agencies. In addition, in August 2014 the Administration established USDS, which aims to improve public-facing federal IT services. The President's Budget for fiscal year 2016 also proposed funding for agencies to establish their own digital service teams.

GAO was asked to review 18F and USDS. GAO's objectives were to (1) describe 18F and USDS efforts to address problems with IT projects and agencies' views of services provided, (2) assess these programs' efforts against practices for performance measurement and project prioritization, and (3) assess agency plans to establish their own digital service teams. To do so, GAO reviewed 32 18F projects and 13 USDS projects that were underway or completed as of August 2015 and surveyed agencies about these projects; reviewed 18F and USDS in key performance measurement and project prioritization practices; reviewed 25 agencies' efforts to establish digital service teams; and reviewed documentation from four agencies, which were chosen based on their progress made in establishing digital service teams.

What GAO Recommends

GAO is making two recommendations to GSA and two recommendations to OMB to improve goals and performance measurement. GAO is also recommending that OMB update policy regarding CIOs and digital services teams. GSA and OMB concurred with the recommendations.

For more information, contact David A. Powner at (202) 512-9286 or pownerd@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: The General Services Administration (GSA) agreed with, and took steps to implement, this recommendation. Specifically, GSA developed six outcome-oriented goals for 18F as well as associated performance measures and targets. In addition, one of the program's goals is to fully recover program costs. By ensuring that all of its goals and performance measures are outcome-oriented and that its performance measures have targets, 18F has a more clear definition of what it wants to accomplish. Additionally, by developing performance measures and targets tied to achieving full cost recovery, GSA is better positioned to begin recovering costs for the program.

    Recommendation: To effectively measure 18F's performance, the Administrator of GSA should direct the Commissioner for the Technology Transformation Service to ensure that goals and associated performance measures are outcome-oriented and that performance measures have targets, including (1) performance measures and targets tied to fully recovering program costs; and (2) goals, performance measures, and targets for how the program will achieve its mission after September 2016.

    Agency Affected: General Services Administration

  2. Status: Open

    Comments: The General Services Administration (GSA) agreed with, and has begun to take steps to implement, this recommendation. Specifically, in a March 2020 written response, GSA stated that Technology Transformation Service (TTS) leadership will be briefed on the program's performance measures on a quarterly basis. We are following up with GSA to confirm that its TTS leadership has been briefed on the results on these performance measures. We will continue to evaluate GSA's progress in implementing this recommendation.

    Recommendation: To effectively measure 18F's performance, the Administrator of GSA should direct the Commissioner for the Technology Transformation Service to assess actual results for each performance measure.

    Agency Affected: General Services Administration

  3. Status: Open

    Comments: The Office of Management and Budget (OMB) generally agreed with, and has begun to take steps to implement, this recommendation. Specifically, in its December 2016 report to Congress, OMB developed three goals for U.S. Digital Service (USDS): (1) rethink how the federal government builds and buys digital services; (2) expand the use of common, platforms, services, and tools; and (3) bring top technical talent into public service. In addition, OMB established performance measures with targets for its third goal and for each of the program's major projects. However, OMB has not established performance measures for the first two USDS goals. Further, the program's third goal is not outcome-oriented. In May 2018, an USDS staff member stated that USDS established goals for and measured performance on each of the projects the program supports in its fall 2017 report to Congress. Although measuring performance on projects can provide USDS with valuable information, this effort does not address goals and performance measurement on the overall USDS program. In May 2020, OMB stated that they would provide an update on the agency's efforts to address the recommendation by June 2020. We will continue to evaluate OMB's progress in implementing this recommendation.

    Recommendation: To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to ensure that all goals and associated performance measures are outcome-oriented and that performance measures have targets.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  4. Status: Open

    Comments: The Office of Management and Budget (OMB) generally agreed with, and has begun to take steps to implement, this recommendation. Specifically, in its December 2016 report to Congress, OMB assessed the results of performance measures for one of the U.S. Digital Service (USDS) program's goals--bring top technical talent into public service--and for each of the program's major projects. However, OMB has not established performance measures for the other two USDS goals--rethink how the federal government builds and buys digital services; and expand the use of common, platforms, services, and tools. In May 2018, an USDS staff member stated that USDS established goals for and measured performance on each of the projects the program supports in its fall 2017 report to Congress. As of July 2019, USDS has not publicly released any subsequent reports to Congress or additional information on its goals and performance measures. Although measuring performance on projects can provide USDS with valuable information, this effort does not address performance measurement on the overall USDS program. In May 2020, OMB stated that they would provide an update on the agency's efforts to address the recommendation by June 2020. We will continue to evaluate OMB's progress in implementing this recommendation.

    Recommendation: To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to assess actual results for each performance measure.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  5. Status: Open

    Comments: The Office of Management and Budget (OMB) generally agreed with, and has begun to take steps to implement, this recommendation. In particular, OMB updated its digital service team policy to require that teams appropriately inform their chief information officers (CIO) regarding U.S. Digital Service (USDS) projects. However, the policy does not describe the responsibilities or authorities governing the relationships between CIOs and digital service teams. In May 2018, an USDS staff member stated that the program updated digital service team charters to address the role of agency CIOs. As of May 2020, USDS has yet to provide us with the updated digital service team charters. In May 2020, OMB stated that they would provide an update on the agency's efforts to address the recommendation by June 2020. We will continue to evaluate OMB's progress in implementing this recommendation.

    Recommendation: To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to update USDS policy to clearly define the responsibilities and authorities governing the relationships between CIOs and the digital service teams and require existing agency digital service teams to address this policy. In doing so, the Federal Chief Information Officer should ensure that this policy is aligned with relevant federal law and OMB guidance on CIO responsibilities and authorities.

    Agency Affected: Executive Office of the President: Office of Management and Budget

 

Explore the full database of GAO's Open Recommendations »

Sep 14, 2020

Sep 9, 2020

Sep 8, 2020

Aug 31, 2020

Aug 3, 2020

Jun 1, 2020

Mar 5, 2020

Feb 20, 2020

  • it icon, source: PhotoDisc

    Science & Tech Spotlight:

    Deepfakes
    GAO-20-379SP: Published: Feb 20, 2020. Publicly Released: Feb 20, 2020.

Dec 12, 2019

Looking for more? Browse all our products here