Thrifty Food Plan: Better Planning and Accountability Could Help Ensure Quality of Future Reevaluations
Fast Facts
Over 40 million people rely on the Supplemental Nutrition Assistance Program (SNAP), formerly known as "food stamps," to buy food. Benefits are based on the Thrifty Food Plan, which estimates how much it would cost a family of 4 to eat a healthy diet on a budget.
In response to COVID-19, the U.S. Department of Agriculture expedited a planned update to the Thrifty Food Plan. USDA allowed plan costs to increase beyond inflation for the first time—resulting in a 21% increase in SNAP benefits.
But officials made this update without key project management and quality assurance practices in place. We recommended ways USDA can improve future updates.
EBT accepted here
Highlights
What GAO Found
The Thrifty Food Plan (TFP) describes how much it costs to eat a healthy diet on a limited budget, and is the basis for maximum Supplemental Nutrition Assistance Program (SNAP) benefits. In 2021, the U.S. Department of Agriculture (USDA) reevaluated the Thrifty Food Plan and made decisions that resulted in increased costs and risks for the reevaluated TFP. Specifically, the agency (1) allowed the cost of the TFP—and thus SNAP benefits—to increase beyond inflation for the first time in 45 years, and (2) accelerated the timeline of the reevaluation by 6 months in order to respond to the COVID-19 emergency. The reevaluation resulted in a 21 percent increase in the cost of the TFP and the maximum SNAP benefit. The reevaluation was complex and involved several USDA offices. However, USDA began the reevaluation without three key project management elements in place. First, without a charter, USDA missed an opportunity to identify ways to measure project success and to set clear expectations for stakeholders. Second, USDA developed a project schedule but not a comprehensive project management plan that included certain elements, such as a plan for ensuring quality throughout the process. Third, the agency did not employ a dedicated project manager to ensure that key practices in project management were generally followed.
USDA's Thrifty Food Plan Reevaluation Lacked Key Project Management Elements
USDA gathered external input, but given time constraints, did not fully incorporate this input in its reevaluation. Specifically, USDA substituted a limited internal review of the TFP report for the formal peer review it had initially planned. This review was conducted by USDA officials who had been involved in the TFP reevaluation, and therefore were not independent. The TFP report lacked a comprehensive, external peer review to assess the transparency, clarity, or interpretation of the results. As a result, the review also did not meet relevant Office of Management and Budget (OMB) and USDA guidelines, such as the requirement to publish a report with the results of a peer review.
The complexity of the economic model USDA uses to calculate the TFP led officials to make numerous methodological and policy decisions during the 2021 reevaluation, as they had in past reevaluations. However, GAO found that key decisions did not fully meet standards for economic analysis, primarily due to failure to fully disclose the rationale for decisions, insufficient analysis of the effects of decisions, and lack of documentation. As a result, members of the public and policymakers reading the TFP report may not understand the rationale for decisions made by USDA officials, and external parties would face difficulties reproducing the TFP, which further decreases transparency and accountability.
Why GAO Did This Study
SNAP supplemented the food budgets of more than 41 million people in 2021. Following a provision in the Agriculture Improvement Act of 2018 to reevaluate the TFP by 2022, USDA completed a reevaluation of the TFP in 2021.
GAO was asked to review the reevaluation. This report examines (1) USDA's administrative process for reevaluating the TFP in 2021 and the extent to which the process employed leading project management practices; (2) the extent to which USDA gathered and analyzed external input to inform the reevaluation; and (3) how the methodology and results of the reevaluation compared to methodological standards.
GAO reviewed USDA documents, interviewed officials and external experts, and compared the information collected to key project planning practices; relevant OMB and USDA guidelines for information quality, peer review, and scientific integrity; federal standards for internal control; and GAO's assessment methodology for economic analysis.
Recommendations
GAO is making eight recommendations, including that USDA develop policies to ensure TFP reevaluations follow key project management practices, peer review guidelines, and quality standards; and publish information to allow external parties to reproduce results. USDA did not explicitly agree or disagree with the recommendations but disagreed with GAO's selection and application of certain criteria. GAO believes the criteria were appropriate and stands by the findings, conclusions, and recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Agriculture |
Priority Rec.
The Secretary of Agriculture should develop and document a process to ensure that TFP reevaluations follow the project management practice of establishing a key document at the start of a project, such as a project charter, that includes an overall assessment of risk and measurable objectives and metrics for success related to project requirements and other expectations for the project. (Recommendation 1) |
Open
USDA did not explicitly agree or disagree with this recommendation. USDA did not describe any actions it planned to take to address this recommendation and noted that it plans to respond to each recommendation in detail as part of its required Statement of Action by June 2023.
|
| Department of Agriculture | The Secretary of Agriculture should develop and document a process to ensure that TFP reevaluations follow the project management practice of creating a comprehensive project management plan that describes how the project will be executed, monitored, controlled, and closed and, in addition to a project schedule, includes:
|
Open
USDA did not explicitly agree or disagree with this recommendation. USDA did not describe any actions it planned to take to address this recommendation and noted that it plans to respond to each recommendation in detail as part of its required Statement of Action by June 2023.
|
| Department of Agriculture | The Secretary of Agriculture should develop and document a process to ensure that TFP reevaluations follow the project management practice of designating a project manager, or another member of the project team with project management expertise, to ensure that TFP reevaluations apply generally recognized project management practices, including creating key project documentation. (Recommendation 3) |
Open
USDA did not explicitly agree or disagree with this recommendation. USDA did not describe any actions it planned to take to address this recommendation and noted that it plans to respond to each recommendation in detail as part of its required Statement of Action by June 2023.
|
| Department of Agriculture | The Secretary of Agriculture should develop and document a process to ensure that TFP reevaluations are subject to formal, comprehensive, and independent peer reviews before publication. (Recommendation 4) |
Open
USDA did not explicitly agree or disagree with this recommendation. USDA noted that the Food and Nutrition Service (FNS) will develop formal peer review guidance and clarify which scientific studies will be subject to peer review, how those decisions will be made, and the process for peer review. However, FNS did not specify whether all future TFP reevaluations will be subject to formal, comprehensive, and independent peer reviews before publication. USDA noted that it plans to respond to each recommendation in detail as part of its required Statement of Action by June 2023.
|
| Department of Agriculture | The Secretary of Agriculture should consult with the Office of Information and Regulatory Affairs and designate the TFP as a "highly influential scientific assessment" subject to more stringent OMB guidance concerning peer review. (Recommendation 5) |
Open
USDA did not explicitly agree or disagree with this recommendation. USDA did not describe any actions it planned to take to address this recommendation and noted that it plans to respond to each recommendation in detail as part of its required Statement of Action by June 2023.
|
| Department of Agriculture |
Priority Rec.
The Secretary of Agriculture should establish specific quality assurance guidelines for TFP reevaluations that will ensure methodological decisions meet key quality standards for an analysis that will affect public policy and inform policy makers. These guidelines should summarize applicable USDA and other federal quality standards and should describe how such standards will be embedded in future TFPs. These guidelines should ensure that future TFP reports have:
|
Open
USDA did not explicitly agree or disagree with this recommendation. USDA stated that, for future Thrifty Food Plan reevaluations, it plans to publish a detailed technical document that describes the methodology and decision points for the reevaluation, along with more detail on the quality assurance processes used. However, the agency did not identify any actions regarding the development of specific quality assurance guidelines for future TFP reevaluations. USDA noted that it plans to respond to each recommendation in detail as part of its required Statement of Action by June 2023.
|
| Department of Agriculture | The Secretary of Agriculture should ensure that FNS designs, documents, and implements key internal controls related to data processing, including standards and procedures for review of the computer code used in generating the TFP Market Baskets. (Recommendation 7) |
Open
USDA did not explicitly agree or disagree with this recommendation. USDA stated that future TFP reports will describe the quality assurance methods used, including code review, but did not provide further detail on how it plans to address this recommendation. USDA noted that it plans to respond to each recommendation in detail as part of its required Statement of Action by June 2023.
|
| Department of Agriculture | The Secretary of Agriculture should publish the computer code and raw data used to generate the TFP Market Baskets, to the extent allowable, along with all of the final equations used to create the model, in order to ensure qualified external parties can reproduce and replicate the TFP. (Recommendation 8) |
Open
The agency agreed with this recommendation and stated that it plans to publish code and data supplements for future evaluations. In addition, the agency planned to update its online supplement to make it more accessible to new users. USDA noted that it plans to respond to each recommendation in detail as part of its required Statement of Action by June 2023.
|