U.S. Postal Service:

Addressing Policy Gaps Could Improve Pilot Design and Evaluation for Postal Innovations

GAO-19-293: Published: Mar 14, 2019. Publicly Released: Mar 14, 2019.

Additional Materials:

Contact:

Lori Rectanus
(202) 512-2834
RectanusL@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Revenue from USPS’s most profitable product—First Class Mail—has declined over the last decade. To help stem its significant financial losses and sustain its operations, USPS has been piloting innovative products and services, such as same-day delivery and grocery delivery.

However, with ever-shrinking financial resources, USPS needs to make sure that its new and innovative ideas generate revenue and enhance customer experience.

We found that the pilot process doesn't consistently give USPS the information it needs to effectively assess and draw conclusions about its pilots. We made 2 recommendations to improve the process.

One of the Postal Service's Innovations Involved Delivering Groceries in Certain Areas

Green tote used to deliver groceries in front of a residence.

Green tote used to deliver groceries in front of a residence.

Additional Materials:

Contact:

Lori Rectanus
(202) 512-2834
RectanusL@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

From fiscal years 2013 through 2017, the U.S. Postal Service (USPS) piloted 24 key innovations intended primarily to generate revenue or improve customers' experience. The following four selected innovations illustrate these efforts:

Same-Day Delivery: USPS delivered goods consumers bought online or in stores. The pilot sought to test the product's feasibility and revenue potential.

Grocery Delivery: USPS delivered groceries to consumers in metropolitan areas. The pilot sought to test the product's feasibility and revenue potential.

Informed Delivery: USPS emailed customers an advance image of the mail they would receive. The pilot sought to test the service's potential benefits, such as generating new revenue from advertisers that may use the service.

Keyless Parcel Lockers: USPS is testing lockers where customers can independently pick up packages at post offices. The pilot seeks to test the service's operation and potential benefits for USPS and customers.

(From Left to Right) Metro Post (same-day delivery), Grocery Delivery Tote Bag, Informed Delivery Email Notification, and Keyless Parcel Locker Unit

\\prod\userdata\FR_Data\taylorcs\Desktop\239\102478 Graphics\Fig0_Highlights-102478_jo.tif

USPS's policies for piloting innovations do not fully reflect the five leading practices for pilot design and evaluation identified in GAO's prior work. The policies fully reflect two of the leading practices because they require articulating a methodology for evaluating pilot performance and documenting lessons learned. The policies do not fully reflect the other three practices because they do not require: (1) linking pilot objectives to identified performance measures; (2) documenting conclusions based on pilot results; or (3) communicating with key external stakeholders, as appropriate. These policy gaps limit the extent to which USPS can ensure that it is making good resource allocation decisions based on pilot experiences. For example, GAO found that USPS did not document its conclusions based on the results of its pilots of same-day delivery, grocery delivery, and Informed Delivery. Documenting conclusions can be especially important when USPS continues to offer the product or service after the pilot has concluded, even though the pilot did not achieve all of its objectives, as was the case with these three innovations. Further, while USPS's policies require documenting lessons learned from its pilots, USPS did not do so for some pilots GAO reviewed. Senior USPS officials said that USPS did not consistently follow this policy because it had not developed tools or training that could help ensure such consistency. As a result, USPS risks losing information that could be relevant to future innovation efforts.

USPS faces a challenging business environment that has led to reduced demand for its traditional services and significant financial losses. USPS aims to address this challenge by offering innovative products and services. The success of these efforts will depend, in part, on how effectively USPS tests each innovation's performance on a small scale to determine whether, how, and when to launch an innovation more broadly—a practice known as “piloting.”

GAO was asked to review USPS's efforts to develop postal innovations. This report (1) describes key innovations that USPS recently piloted and (2) examines the extent to which USPS's policies reflect leading practices for pilot design and evaluation. GAO analyzed information on USPS pilots from fiscal years 2013 through 2017; compared USPS policies for piloting innovations to leading practices for pilot design and evaluation in prior GAO work and relevant standards for internal control; and selected four key innovations based on various characteristics (e.g., innovation type) to serve as illustrative examples of USPS's piloting efforts.

What GAO Recommends

GAO recommends that USPS (1) develop policies that fully reflect leading practices for pilot design and evaluation and (2) develop tools or training to ensure consistent documentation of lessons learned from pilots. USPS neither agreed nor disagreed with the recommendations but described actions it plans to take related to each.

For more information, contact Lori Rectanus at (202) 512-2834 or RectanusL@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: The U.S. Postal Service (USPS) faces a challenging business environment that has led to reduced demand for its traditional services and significant financial losses. USPS aims to address this challenge by offering innovative products and services. The success of these efforts will depend, in part, on how effectively USPS tests each innovation's performance on a small scale to determine whether, how, and when to launch an innovation more broadly--a practice known as "piloting." In 2019, GAO reported that USPS's policies for piloting innovations fully reflected two of the five leading practices for pilot design and evaluation that GAO identified in its prior work and relevant standards for internal control. However, GAO identified gaps in USPS's policies related to linking objectives and performance measures, documenting conclusions, and communicating with key external stakeholders. These gaps limit the extent to which USPS can ensure that it is making good resource allocation decisions based on pilot experiences. GAO recommended that USPS address these gaps to enable its leadership to better assess the outcomes of its pilots and gauge customers' reactions to innovative products and services. In 2020, GAO confirmed that USPS addressed this recommendation. Specifically, USPS issued new guidance and training for its employees on applying leading practices during pilot design and evaluation, including linking objectives with performance measures, documenting conclusions, and communicating with key external stakeholders. With this guidance and training, USPS leadership will be better prepared to assess the results and lessons of its efforts to offer innovative products and services and make good resource allocation decisions.

    Recommendation: The Postmaster General should direct the Vice President of Product Innovation to develop policies that fully reflect leading practices for pilot design and evaluation in areas such as linking objectives and performance measures; documenting conclusions about scalability based on pilot results; and communicating with key external stakeholders, as appropriate. (Recommendation 1)

    Agency Affected: United States Postal Service

  2. Status: Closed - Implemented

    Comments: The U.S. Postal Service (USPS) faces a challenging business environment that has led to reduced demand for its traditional services and significant financial losses. USPS aims to address this challenge by offering innovative products and services. The success of these efforts will depend, in part, on how effectively USPS tests each innovation's performance on a small scale to determine whether, how, and when to launch an innovation more broadly-a practice known as "piloting." In 2019, GAO reported that USPS's policies for piloting innovations required its staff to evaluate pilot performance and identify and document lessons learned-a key leading practice identified by GAO for pilot design and evaluation. However, in practice, GAO found that USPS had not consistently documented lessons learned from its pilots. Specifically, GAO found that USPS had not consistently documented lessons learned across the 24 key innovations that GAO reviewed because USPS had not developed tools or training that could help ensure such consistency. Without consistent documentation of lessons learned from its innovation pilots USPS risks losing information that could inform its future innovation efforts. GAO recommended that USPS develop tools or training to help ensure that USPS consistently documents lessons learned from its pilots. In 2020, GAO confirmed that USPS had developed and implemented training to ensure consistent documentation of lessons learned from its pilots. Such training should help to ensure that USPS has access to key information to inform its decisions about pilots and any future related efforts.

    Recommendation: The Postmaster General should direct the Vice President of Product Innovation to develop tools, such as a template, or training to help ensure USPS consistently documents lessons learned at the conclusion of pilots, as required by USPS policies. (Recommendation 2)

    Agency Affected: United States Postal Service

 

Explore the full database of GAO's Open Recommendations »

Sep 25, 2020

Sep 23, 2020

Sep 10, 2020

Sep 8, 2020

Aug 31, 2020

Aug 27, 2020

Aug 19, 2020

Jul 9, 2020

Looking for more? Browse all our products here