Reports & Testimonies

  • GAO’s recommendations database contains report recommendations that still need to be addressed.

    GAO’s recommendations help congressional and agency leaders prepare for appropriations and oversight activities, as well as help improve government operations. Recommendations remain open until they are designated as Closed-implemented or Closed-not implemented. You can explore open recommendations by searching or browsing.

    GAO's priority recommendations are those that we believe warrant priority attention. We sent letters to the heads of key departments and agencies, urging them to continue focusing on these issues. These recommendations are labeled as such. You can find priority recommendations by searching or browsing our open recommendations below, or through our mobile app.

  • Browse Open Recommendations

    Explore priority recommendations by subject terms or browse by federal agency

    Search Open Recommendations

    Search for a specific priority recommendation by word or phrase



  • Governing on the go?

    Our Priorities for Policy Makers app makes it easier for leaders to search our recommendations on the go.

    See the November 10th Press Release


  • Have a Question about a Recommendation?

    • For questions about a specific recommendation, contact the person or office listed with the recommendation.
    • For general information about recommendations, contact GAO's Audit Policy and Quality Assurance office at (202) 512-6100 or apqa@gao.gov.
  • « Back to Results List Sort by   

    Results:

    Subject Term: "Evaluation methods"

    20 publications with a total of 43 open recommendations including 5 priority recommendations
    Director: Robert Goldenkoff
    Phone: (202) 512-2757

    3 open recommendations
    Recommendation: The Secretary of Commerce should direct the Under Secretary of the Economics and Statistics Administration and the Director of the U.S. Census Bureau to use the Bureau's evaluations before 2020 to determine the implications of in-office address canvassing on the cost and quality of address canvassing, and use this information to justify decisions related to its re-engineered address canvassing approach.

    Agency: Department of Commerce
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Recommendation: Early in the next decennial cycle, the Secretary of Commerce should direct the Under Secretary of the Economics and Statistics Administration and the Director of the U.S. Census Bureau to plan and execute more flexible, and perhaps smaller, address canvassing test and evaluation activity needed to support key design decisions having significant effect on the cost and quality of the census.

    Agency: Department of Commerce
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Recommendation: The Secretary of Commerce should direct the Under Secretary of the Economics and Statistics Administration and the Director of the U.S. Census Bureau to use productivity measures that track the progress in completing in-office address canvassing workload and the effectiveness of in-office address canvassing in reducing fieldwork in order to make informed decisions on allocating resources to current and future address canvassing workload so that the operation is completed in a timely and cost-effective manner.

    Agency: Department of Commerce
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Director: Beryl H. Davis
    Phone: (202) 512-2623

    2 open recommendations
    Recommendation: To help ensure that government-wide compliance under IPERA is consistently determined and reported, the Director of OMB should coordinate with CIGIE to develop and issue guidance, either jointly or independently, to specify what procedures should be conducted as part of the IGs' IPERA compliance determinations.

    Agency: Executive Office of the President: Office of Management and Budget
    Status: Open

    Comments: OMB had no comments on the report or the recommendation to coordinate with CIGIE to develop guidance. Although this recommendation was not directed to CIGIE, the CIGIE Chairperson stated that CIGIE would coordinate with OMB as needed and provide feedback on any draft OMB guidance.
    Recommendation: To help fulfill USDA's requirements under IPERA and OMB guidance--that agencies submit proposals to Congress when a program reaches 3 or more consecutive years of noncompliance with IPERA criteria--the Secretary of Agriculture should submit a letter to Congress detailing proposals for reauthorization or statutory changes in response to 3 consecutive years of noncompliance as of fiscal year 2015 for its Farm Security and Rural Investment Act Program. To the extent that reauthorization or statutory changes are not considered necessary to bring a program into compliance, the Secretary or designee should state so in the letter.

    Agency: Department of Agriculture
    Status: Open

    Comments: USDA's Acting Deputy Secretary concurred with this recommendation.
    Director: Gretta L. Goodwin
    Phone: (202) 512-8777

    1 open recommendations
    Recommendation: To ensure that the Capitol Police Board's current and any new approaches help enhance accountability, transparency, and effective external communication with its stakeholders, the Board should revise its Manual of Procedures to fully incorporate each of the leading practices for internal control and governance standards discussed in this report. In so doing, the Board should engage stakeholders in the revision process, such as by soliciting their input on any non-statutory changes that could particularly address the concerns stakeholders have raised, and incorporating their views as appropriate. If, in making revisions to its Manual, the Board determines that statutory changes may be helpful to enhance Board operations, then the Board should also engage with stakeholders on such proposed changes.

    Agency: Capitol Police Board
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Director: Mark Goldstein
    Phone: (202) 512-2834

    2 open recommendations
    including 1 priority recommendation
    Recommendation: Given DOT's new discretionary grant programs and similar challenges we have found with previous DOT programs, the Secretary of Transportation should issue a directive that governs department-wide and modal administration discretionary grant programs. Such a directive should include requirements to: (1) develop a plan for evaluating project proposals in advance of issuing a notice of funding availability that defines the stages of the process, including how the process will be overseen to ensure a consistent review of applications; (2) document key decisions, including the reason for any rating changes and the officials responsible for those changes, and how high-level concerns raised during the process were addressed; and (3) align stated program purpose and policy priorities with the evaluation and selection process.

    Agency: Department of Transportation
    Status: Open
    Priority recommendation

    Comments: DOT concurred with this recommendation. To fully implement this recommendation, DOT must issue a directive, to address our concerns and require DOT's operating administrations to develop and document procedures for all phases of the process for its discretionary grants.
    Recommendation: To better address FTA's stated policy priority of regional resilience and to minimize fragmentation of federal efforts, the Secretary of Transportation should direct the FTA Administrator to (a) examine the projects FTA funded under the Disaster Relief Appropriations Act of 2013 (DRAA) discretionary transit resilience grant program for potential duplication with other resilience efforts and (b) determine whether it is appropriate to realign funds for FTA-supported projects for other purposes authorized under the DRAA, or request a rescission of funds for any of the FTA-supported projects.

    Agency: Department of Transportation
    Status: Open

    Comments: When we confirm what actions DOT has taken in response to this recommendation, we will provide updated information.
    Director: James Cosgrove
    Phone: (202) 512-7114

    2 open recommendations
    Recommendation: The Secretary of Health and Human Services should direct the Administrator of CMS to assess the feasibility of updating the agency's study on the effect of VA-provided Medicare-covered services on per capita county Medicare FFS spending rates by obtaining VA utilization and diagnosis data for veterans enrolled in Medicare FFS under its existing data use agreement or by other means as necessary.

    Agency: Department of Health and Human Services
    Status: Open

    Comments: In July 2016, the U.S. Department of Health and Human Services (HHS) reiterated its disagreement with our recommendation. HHS stated that the Centers for Medicare & Medicaid Services (CMS) uses Medicare fee-or-service(FFS) spending rates when setting the benchmark, which excludes services provided by Department of Veterans Affairs (VA) facilities. In addition, HHS stated that incorporating VA utilization and diagnosis data into CMS's analysis may not materially improve the analysis and the resulting adjustment. HHS indicated that it will continue to review the need for incorporating additional data or for methodology changes in the future. As we note in the report, only VA's utilization and diagnosis data can account for services provided by and diagnoses made by VA. Depending on the number and mix of services provided by and the diagnoses made by VA, risk-adjusted Medicare FFS spending for veterans may either be higher or lower than it would be if CMS accounted for VA-provided services and diagnoses. Therefore, relying exclusively on Medicare FFS spending to estimate the effect of VA spending on Medicare FFS-enrolled veterans could result in an inaccurate estimate of how VA spending on services for Medicare FFS-enrolled veterans affects per capita county Medicare FFS spending. While there may be challenges associated with incorporating VA utilization and diagnosis data into CMS's analysis, we maintain that CMS should work to do so given the implications that not incorporating the data may have on the accuracy of payment to MA plans.
    Recommendation: If CMS makes an adjustment to the benchmark to account for VA spending on Medicare-covered services, the Secretary of Health and Human Services should direct the Administrator of CMS to assess whether an additional adjustment to MA payments is needed to ensure that payments to MA plans are equitable for veterans and nonveterans.

    Agency: Department of Health and Human Services
    Status: Open

    Comments: The Department of Health and Human Services (HHS)has proposed adjusting the benchmark for 2017 to account for the Department of Veterans Affairs (VA) spending on Medicare-covered services. As of July 2016, HHS had not yet completed its assessment of whether an additional adjustment to MA payments is needed to ensure that payment to Medicare Advantage (MA) plans are equitable for veterans and nonveterans. In order to close this recommendation, CMS will need to complete its assessment.
    Director: Brenda S. Farrell
    Phone: (202) 512-3604

    1 open recommendations
    Recommendation: To assist DOD in determining whether CIPP is meeting its intended purpose of enhancing retention and providing greater flexibility in the career path of servicemembers, the Secretary of Defense should direct the Under Secretary of Defense for Personnel and Readiness, in collaboration with the service secretaries, to develop and implement a plan to evaluate the pilot that includes key features such as well-defined, clear, and measurable objectives and standards for determining pilot-program performance.

    Agency: Department of Defense
    Status: Open

    Comments: The Department of Defense (DOD) concurred with our recommendation, and recognized the importance of developing well defined measures to evaluate the effectiveness and utility of the career intermission pilot program (CIPP). In November 2015, DOD acknowledged receipt of our final report and noted that its position on the report has not changed since providing official written comments. In May 2017, we issued GAO-17-623R, which provided an update to the CIPP participant data in our initial report, and noted that DOD had not established performance measures in response to our recommendation. As of September 2017, DOD had still not developed these measures, noting the program was still in a pilot stage, and the final consolidated report due to Congress in June 2023 will include narratives from the service secretaries discussing the effectiveness and value of the program.
    Director: William Shear
    Phone: (202) 512-8678

    6 open recommendations
    including 1 priority recommendation
    Recommendation: To improve management of the Small Business Administration and to ensure that SBA assesses the effectiveness of its programs, the SBA Administrator should prioritize resources to conduct additional program evaluations.

    Agency: Small Business Administration
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Recommendation: To improve management of the Small Business Administration and to ensure that SBA fully meets GPRAMA requirements, the SBA Administrator should use the results of additional evaluations it conducts in its strategic planning process and ensure the agency's next strategic plan includes required information on program evaluations, including a schedule of future evaluations.

    Agency: Small Business Administration
    Status: Open
    Priority recommendation

    Comments: SBA officials stated that, as of October 2016, the agency had taken several steps to prioritize resources and establish an implementation plan for future evaluations, including hiring its first lead program evaluator to develop a long-term evaluation agenda and initiating four program evaluations. They stated that once completed, the evaluations would be incorporated into the agency's fiscal year 2018-2022 strategic plan. As of May 2017, SBA had started reviewing guidance on drafting this plan, which is due in February 2018.
    Recommendation: To improve management of the Small Business Administration and to improve SBA's human capital management, the SBA Administrator should incorporate into its next training plan key principles such as goals and measures for its training programs and input on employee development goals.

    Agency: Small Business Administration
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Recommendation: To improve management of the Small Business Administration and to ensure that SBA's organizational structure helps the agency meet its mission, the SBA Administrator should document the assessment of the agency's organizational structure, including any necessary changes to, for example, better ensure areas of authority, responsibility, and lines of reporting are clear and defined.

    Agency: Small Business Administration
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Recommendation: To improve management of the Small Business Administration and to improve SBA's program and management guidance, the SBA Administrator should set time frames for periodically reviewing and updating its SOPs as appropriate.

    Agency: Small Business Administration
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Recommendation: To improve management of the Small Business Administration and to help ensure that SBA's IT operations and maintenance investments are continuing to meet business and customer needs and the agency's strategic goals, the SBA Administrator should direct the appropriate officials to perform an annual operational analysis on all SBA investments in accordance with OMB guidance.

    Agency: Small Business Administration
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Director: Charles Michael Johnson, Jr.,
    Phone: (202) 512-7331

    2 open recommendations
    Recommendation: Given that countering violent extremism is a priority for the U.S. government in general and State's Bureau of Counterterrorism (CT Bureau), the Secretary of State should take steps to ensure that CVE program efforts abroad are evaluated.

    Agency: Department of State
    Status: Open

    Comments: In June 2017, the CT Bureau indicated that a third-party evaluation of the CVE program has been completed. The evaluation focused on process and programming, including all CVE projects funded between fiscal years 2012 and 2016 and resulted in two related but disparate sets of recommendations and findings. The CT Bureau indicated that it has begun incorporating the recommendations made in the evaluation into its overall CVE efforts.
    Recommendation: To improve State's CT Bureau's program management efforts, the Secretary of State should take steps to ensure the Bureau of Counterterrorism establishes specific time frames for addressing recommendations from program evaluations.

    Agency: Department of State
    Status: Open

    Comments: In June 2017, the CT Bureau indicated it is in the process of reviewing recommendations from the CVE evaluation, and will soon be assigning timelines to those recommendations that the bureau deems relevant and achievable. For other evaluations, the CT Bureau indicated that it has already acknowledged the need to assign specific timelines to evaluation recommendations and has adjusted accordingly.
    Director: Lori Rectanus
    Phone: (202) 512-2834

    1 open recommendations
    Recommendation: To better understand attributable costs for individual Parcel Select NSAs, the Postmaster General should direct the appropriate staff to identify and implement cost-effective methods, such as using a sample, to collect and study information on the costs of delivering Parcel Select packages of varying characteristics in order to develop contract-specific attributable cost estimates.

    Agency: United States Postal Service
    Status: Open

    Comments: USPS developed analysis in response to this recommendation by using a proxy for package dimension rather than a sample of customer-specific dimension data, as GAO recommended. The assumptions used in the methodology by USPS result in only minor cost differences regardless of the dimension or weight of packages shipped under individual Parcel Select NSAs. As of June 13, 2017, this recommendation remains open pending additional discussions with USPS about its methodology and results.
    Director: Kingsbury, Nancy R
    Phone: (202) 512-2700

    3 open recommendations
    including 2 priority recommendations
    Recommendation: In addition to implementing our previous recommendations, to effectively implement key elements of the IRS information security program, the Commissioner of Internal Revenue should ensure contractors receive security awareness training within 5 business days of being granted access to an IRS information system.

    Agency: Department of the Treasury: Internal Revenue Service
    Status: Open

    Comments: During the audit of IRS' FY 2017 financial statements, IRS indicated that it had not yet implemented this recommendation. When IRS indicates that it has implemented this recommendation, we will review its actions.
    Recommendation: In addition to implementing our previous recommendations, to effectively implement key elements of the IRS information security program, the Commissioner of Internal Revenue should ensure that control testing methodology and results fully meet the intent of the control objectives being tested.

    Agency: Department of the Treasury: Internal Revenue Service
    Status: Open
    Priority recommendation

    Comments: During the audit of IRS's FY 2017 financial statements, IRS indicated that it has not completed these actions. When IRS indicates that it has implemented the recommendation, we will evaluate the effectiveness of its actions.
    Recommendation: In addition to implementing our previous recommendations, to effectively implement key elements of the IRS information security program, the Commissioner of Internal Revenue should update the remedial action verification process to ensure actions are fully implemented.

    Agency: Department of the Treasury: Internal Revenue Service
    Status: Open
    Priority recommendation

    Comments: During the audit of IRS's FY 2017 financial statements, IRS indicated that it has not completed actions to implement the recommendation. When IRS indicates that it has implemented the recommendation, we will evaluate the effectiveness of its actions.
    Director: Chaplain, Cristina T
    Phone: (202) 512-4841

    1 open recommendations
    Recommendation: To ensure that satellites storage is fully considered at the beginning of the acquisition process for all satellite programs and sufficient detailed cost data are maintained, the Secretary of Defense should provide guidance regarding when and how to use storage in the acquisition process, and establish mechanisms so that more detailed data are maintained for use in evaluating the reasonableness of contractors' storage cost proposals and for informing DOD's oversight of satellite acquisitions.

    Agency: Department of Defense
    Status: Open

    Comments: In its response to the report, DOD concurred with the recommendation and noted that it is important to develop guidance regarding the use of satellite storage in the acquisition process. In addition, DOD agreed that it is important to establish mechanisms such that more detailed data are available to evaluate storage cost proposals and inform the oversight of satellite acquisitions. In October 2015, DOD provided GAO with draft language that it planned to include in the Space Systems chapter of the Defense Acquisition Guidebook (DAG) when the final chapter was to be published. In an August 24, 2016, response to a GAO inquiry regarding the language not appearing in the on-line version of the DAG, the office of the Under Secretary of Defense for Acquisition, Technology and Logistics (USD/AT&L) explained that the Space Systems chapter of the DAG had been deleted. USD/AT&L stated it was working to incorporate the proposed language in the next revision of the DAG, scheduled to be completed in December 2016. A September 12, 2017, search of DOD's on-line guidance did not locate any guidance related to satellite storage. DOD liaison was contacted, but no information has been provided yet as of September 2017.
    Director: Lawrance L. Evans, Jr.
    Phone: (202) 512-8678

    5 open recommendations
    Recommendation: To improve FSOC's control activities and help ensure that it better manages its determination process and achieves intended results, the Secretary of the Treasury, in his capacity as the Chairperson of FSOC and in consultation with FSOC members, should systematically record the staff contributing to determination evaluations, and monitor such information to help assess the progress and efficiency of determination evaluations..

    Agency: Department of the Treasury: Financial Stability Oversight Council
    Status: Open

    Comments: FSOC has created a document to track agencies participating in evaluations and has said that it will record and monitor information as new companies are evaluated. Although FSOC has created a template to record agency participation, FSOC still needs to collect and monitor such information including information on the agency staff participating in determination evaluations.
    Recommendation: To enhance disclosure and strengthen transparency, the Secretary of the Treasury, in consultation with FSOC members, for future determinations, to the maximum extent possible, should include additional details in its public basis documentation about why FSOC determined that the company met one or both of the statutory determination standards. Specifically, in addition to identifying that the size, significance, or other attributes of the company's characteristics could pose a threat to U.S. financial stability, FSOC should explain--without revealing sensitive information--how it concluded that the characteristics were sufficiently large or significant enough, or had other attributes, to meet one or both of the statutory determination standards.

    Agency: Department of the Treasury: Financial Stability Oversight Council
    Status: Open

    Comments: FSOC stated that it intends to include more detail in its public basis document while meeting its legal obligations to protect confidential information. In February 2015, FSOC issued supplemental procedures for nonbank financial company designations that stated its commitment to continuing to set forth sufficient information in its public bases to provide the public with an understanding of the Council's analysis while protecting sensitive, confidential information submitted by the company to the Council. FSOC's public basis document for its most recent designation, issued on December 18, 2014, included additional information compared to prior basis documents. However, the recent basis document did not fully explain how FSOC concluded that the company's characteristics were sufficiently large or significant enough, or had other attributes, to meet a determination standard.
    Recommendation: To help ensure that FSOC is comprehensively identifying and considering companies, the Secretary of the Treasury in consultation with FSOC members, should establish procedures to evaluate companies in Stage 2 and Stage 3 under both statutory determination standards when an evaluation in either stage concludes that a company does not meet one of the standards, or document--on a company-specific or more general basis--why the second determination standard is not relevant for determination evaluations.

    Agency: Department of the Treasury: Financial Stability Oversight Council
    Status: Open

    Comments: FSOC conducted a review of its nonbank designation procedures, including consideration of this recommendation, which resulted in issuance of supplemental procedures in February 2015. FSOC stated that it has not adopted formal changes to implement this recommendation but noted that the Council will continue to work to identify and evaluate potential changes to its practices and procedures and will revisit this recommendation in mid-2016.
    Recommendation: To help ensure that FSOC is comprehensively identifying and considering companies, the Secretary of the Treasury in consultation with FSOC members, should develop a process to collect information necessary for Stage 1 analysis, as appropriate, from certain nonbank financial companies for which public or regulatory information is otherwise unavailable. For example, FSOC could have companies for which such information is unavailable and that meet certain characteristics (such as quantitative thresholds similar to those used in Stage 1) report necessary information to the Office of Financial Research.

    Agency: Department of the Treasury: Financial Stability Oversight Council
    Status: Open

    Comments: FSOC staff are currently reviewing potential ways to supplement the existing public and regulatory information available to identify companies for evaluation in Stage 1. FSOC stated that it will revisit this recommendation in mid-2016.
    Recommendation: To improve FSOC's control activities and help ensure that it better manages its determination process and achieves intended results, the Secretary of the Treasury, in his capacity as the Chairperson of FSOC and in consultation with FSOC members, should systematically record the dates of key process steps.

    Agency: Department of the Treasury: Financial Stability Oversight Council
    Status: Open

    Comments: FSOC has created a document to centrally track key dates in each evaluation and has said that it will record and monitor the information as new companies are evaluated. However, FSOC has not yet recorded these dates in the document that it has created for this purpose.
    Director: Debra A. Draper
    Phone: (202) 512-7114

    1 open recommendations
    Recommendation: To help ensure adequate and qualified nurse staffing at VAMCs, the Secretary of Veterans Affairs should direct the Interim Under Secretary for Health to enhance VHA's internal controls by completing evaluations of Phase I and Phase II and make any necessary changes to policies and procedures before national implementation of Phase II in all VAMCs.

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: VA agreed with this recommendation and, according to an update received April 2017, VA is taking actions towards addressing this recommendation and expects to complete those actions Fall 2017. When additional information is received GAO will update the recommendation as appropriate.
    Director: Fleming, Susan A
    Phone: 202-512-2834

    1 open recommendations
    Recommendation: The Secretary of Transportation should establish additional accountability measures for management of the TIGER program to (1) establish clear procedures for addressing late-arriving applications and communicate its decisions to either accept or reject these applications to the public, and (2) document and approve major decisions in the application evaluation and project selection process through a decision memorandum or similar mechanism that provides a clear rationale for decisions to: (a) advance for senior review applications other than those rated as highly recommended; (b) not advance applications rated as highly recommended; and (c) change the technical evaluation rating of an application.

    Agency: Department of Transportation
    Status: Open

    Comments: GAO reviewed DOT's TIGER application evaluation process for the 2014 and 2015 funding rounds and found that DOT established guidance that it would no longer accept late-arriving TIGER applications and it improved its internal procedures for documenting major decisions in its application evaluation database, including decisions to advance for senior review applications other than those rated as "highly recommended." However, DOT did not document these procedures through a decision memorandum or similar mechanism to ensure a consistent and transparent process for approving major decisions in these and future TIGER rounds. Moreover, in reviewing the 2015 TIGER application evaluation process, GAO noted that DOT altered its procedures for evaluating applications while the review process was ongoing. According to DOT's Financial Assistance Guidance Manual, grant evaluations must be conducted in accordance with the evaluation plan created prior to the announcement, and may not be changed once applications have been received. As of August 2017, the Department's Inspector General is conducting an audit of the TIGER program to assess DOT's application evaluation procedures. GAO will review the Inspector General's findings when they become available.
    Director: Andrew Sherrill
    Phone: (202) 512-7215

    2 open recommendations
    Recommendation: To better ensure servicemember participation in and completion of TAP, the Secretary of Defense should direct the Under Secretary for Personnel and Readiness to require that all services provide unit commanders and their leaders information on TAP participation levels of servicemembers under their command, similar to that provided by the Army and Air Force. Such information could be used to help hold leaders accountable for ensuring TAP participation and completion.

    Agency: Department of Defense
    Status: Open

    Comments: DOD disagreed with this recommendation and, as of January 2016, has not taken action to address it. In its written comments published in GAO's 2014 report, DOD questioned GAO's concern that not having a mechanism to ensure a servicemember's completion of TAP by commander would result in commanders not supporting the release of servicemembers to attend TAP. DOD noted that GAO's concern is based on observations that preceded full implementation of the capstone event. In October 2013, DOD launched the capstone as a mandatory process by which commanders verify TAP participation and positively affirm that servicemembers have or have not met career readiness standards. In addition, DOD said that capstone event implementation was accompanied by a communications campaign to inform both commanders and the services. Finally, DOD said that capstone event completion is monitored departmentwide to ensure compliance. However, GAO believes that such monitoring does not provide routine information to commanders and their leaders and not all TAP locations will be monitored routinely. GAO's report noted that DOD reported that ensuring servicemember participation in capstone events was a challenge for most of the services possibly due, in part, to lack of commanders' support. Commander support for the program has been a long-standing challenge for the program. Therefore, we continue to believe that an accountability mechanism is needed for TAP that mirrors the level at which responsibility has been assigned. Since commanders are responsible for ensuring that eligible servicemembers have full access to and successfully complete required TAP components, providing them and their leaders information on TAP participation levels of servicemembers under their command could promote accountability and oversight. Servicemember participation in TAP is generally required by law and DOD policy, and also relates to a Cross-Agency Priority Goal, reinforcing the need for such a mechanism. For GAO to close this recommendation, DOD should require all services to provide unit commanders and their leaders information on TAP participation levels of servicemembers under their command.
    Recommendation: To provide information on the extent to which the revamped TAP is effective, the Secretary of Defense should direct the Under Secretary for Personnel and Readiness to work with the partner agencies to develop a written strategy for determining which components and tracks to evaluate and the most appropriate evaluation methods. This strategy should include a plan to use the results of evaluations to modify or redesign the program, as appropriate.

    Agency: Department of Defense
    Status: Open

    Comments: DOD has partially addressed GAO's recommendation on evaluating the effectiveness of the revamped TAP, information that is currently limited. DOD's May 2014 presentation to OMB on the status of TAP implementation and evaluation efforts lists specific TAP components and tracks in which VA, DOL, and SBA planned to assess TAP effectiveness or post-transition outcomes. Although the presentation to OMB indicated that VA planned a higher level evaluation, as of December 2015, VA officials said they are focusing on lower level evaluations, such as gauging participant reactions to training and monitoring. As of March 2016, DOL has been conducting a quasi-experimental evaluation of the Employment Workshop, which it expects to complete in 2017. Further, SBA officials said they are surveying Entrepreneurial track participants who agreed to a post-TAP, follow-up survey. However, DOD and the agencies have not fully demonstrated a strategic approach to planning higher level evaluations. Specifically, DOD and its partner agencies have not provided a rationale for why they chose to evaluate certain aspects of TAP and not others. A written rationale, along with how the agencies plan to use the results of the evaluations, is required for GAO to consider this recommendation closed.
    Director: Fleming, Susan A
    Phone: (202) 512-2834

    2 open recommendations
    including 1 priority recommendation
    Recommendation: To improve the CSA program, the Secretary of Transportation should direct the FMCSA Administrator to revise the SMS methodology to better account for limitations in drawing comparisons of safety performance information across carriers; in doing so, the Secretary of Transportation should direct the FMCSA Administrator to conduct a formal analysis that specifically identifies: (1) limitations in the data used to calculate SMS scores including variability in the carrier population and the quality and quantity of data available for carrier safety performance assessments, and (2) limitations in the resulting SMS scores including their precision, confidence, and reliability for the purposes for which they are used.

    Agency: Department of Transportation
    Status: Open
    Priority recommendation

    Comments: As of October 2016, FMCSA continues to maintain that they do not agree with our methodology or conclusions. While FMCSA's position about our specific recommendation is unchanged, FMCSA noted that Section 5221 of the FAST Act directed the National Academies of Science (NAS) to conduct a safety correlation study of the CSA program, and specifically FMCSA's Safety Measurement System's (SMS) methodology. FMCSA stated that if the outcome of the NAS study results in recommendations for SMS changes, they will address those recommendations accordingly. We continue to believe this recommendation has merit and could help the agency better target FMCSA's resources to the carriers that pose the highest risk of crashing, as we demonstrate in our report. For example, we reported that FMCSA requires a minimum level of information for a carrier to receive an SMS score; however, this requirement is not strong enough to produce sufficiently reliable scores. As a result, GAO found that FMCSA identified many carriers as high risk that were not later involved in a crash, potentially causing FMCSA to miss opportunities to intervene with carriers that were involved in crashes. FMCSA's methodology is limited because of insufficient information, which reduces the precision of SMS scores. GAO found that by scoring only carriers with more information, FMCSA could better identify high risk carriers likely to be involved in crashes. This illustrative approach involves trade-offs; it would assign SMS scores to fewer carriers, but these scores would generally be more reliable and thus more useful in targeting FMCSA's scarce resources.
    Recommendation: To improve the CSA program, the Secretary of Transportation should direct the FMCSA Administrator to ensure that any determination of a carrier's fitness to operate properly accounts for limitations we have identified regarding safety performance information.

    Agency: Department of Transportation
    Status: Open

    Comments: While FMCSA does not agree with our methodology or conclusions, we believe this recommendation has merit and could help the agency better target FMCSA's resources to the carriers that pose the highest risk of crashing, as we demonstrate in our report. For example, we reported that FMCSA requires a minimum level of information for a carrier to receive an SMS score; however, this requirement is not strong enough to produce sufficiently reliable scores. As a result, GAO found that FMCSA identified many carriers as high risk that were not later involved in a crash, potentially causing FMCSA to miss opportunities to intervene with carriers that were involved in crashes. FMCSA's methodology is limited because of insufficient information, which reduces the precision of SMS scores. GAO found that by scoring only carriers with more information, FMCSA could better identify high risk carriers likely to be involved in crashes. This illustrative approach involves trade-offs; it would assign SMS scores to fewer carriers, but these scores would generally be more reliable and thus more useful in targeting FMCSA's scarce resources.
    Director: St James, Lorelei
    Phone: (202) 512-2834

    1 open recommendations
    Recommendation: Because USPS faces a deteriorating financial situation, and to help ensure that future promotions generate net revenue for USPS, the Chairman of the PRC should direct staff to evaluate USPS's data collection and analysis plans for USPS's proposed mail promotions and discuss these evaluations in the PRC decisions for those mail promotions.

    Agency: Postal Regulatory Commission
    Status: Open

    Comments: The Postal Regulatory Commission (PRC) said that it agreed with our recommendation. PRC has also noted the U.S. Postal Service (USPS) is not required by statute to collect data on promotions and provide it to PRC. Based on the information it received, PRC approved USPS's Calendar Year 2017 promotions. However, USPS did not provide PRC with information regarding its plans to estimate the net financial results of the proposed promotions, as we recommended. As no such information was received, no evaluation of these plans was prepared by PRC, and this recommendation remains open.
    Director: Trimble, David C
    Phone: (202)512-9338

    4 open recommendations
    Recommendation: To increase EPA's consistency, transparency, and clarity in implementing the Safe Drinking Water Act in a way that better assures the public of safe drinking water, and to systematically implement the statutory requirement to consider for regulation the contaminants that present the greatest public health concern, the EPA Administrator should require that the Office of Water to develop a coordinated process for obtaining both the occurrence and health effects data that may be needed for the agency to make informed regulatory determinations on these priority contaminants.

    Agency: Environmental Protection Agency
    Status: Open

    Comments: EPA has not demonstrated that it has developed a coordinated process for obtaining both occurrence and health effects data to make informed determinations on priority contaminants. EPA's response to this recommendation is that it will continue the status quo of obtaining occurrence data through UCMRs and work with the Office of Research and Development. As we reported, the approach EPA currently uses does not provide the agency with all of the data it needs in a timely manner to support determinations for some priority contaminants.
    Recommendation: To increase EPA's consistency, transparency, and clarity in implementing the Safe Drinking Water Act in a way that better assures the public of safe drinking water, and to support the development of regulatory determinations that are transparent, clear, and consistent and that follow applicable agency policy, the EPA Administrator should require the Office of Water to expeditiously develop, and make available to the public, policies or guidance that clearly articulates the agency's interpretation of the act's broad statutory criteria for making regulatory determinations and provides a protocol for making such determinations. In particular, the guidance should establish a process to ensure that the presentation of health effects and occurrence information in regulatory determination notices and support documents is comprehensive, consistent, informative, and understandable and that it includes clear explanations of key information, such as any exceptions to existing guidance reflected in the agency's support for its regulatory determinations.

    Agency: Environmental Protection Agency
    Status: Open

    Comments: EPA's Preliminary Regulatory Determinations for Contaminants on the Third Drinking Water Contaminant Candidate List (Reg Det 3) did not appear to include any cases in which EPA made an exception to existing guidance, negating the need for an explanation. EPA has not developed policies or guidance that establishes a process to ensure that the presentation of health effects and occurrence information in regulatory determination notices and support documents is comprehensive, consistent, informative, and understandable, and that it includes clear explanations of key information and the Protocol for Regulatory Determinations 3 does not appear to address this circumstance. Without policies or guidance establishing a process to ensure the comprehensive, consistent, informative, and understandable presentation of its regulatory determinations, there is no guarantee at this time that EPA would include this information in future regulatory determinations.
    Recommendation: To increase EPA's consistency, transparency, and clarity in implementing the Safe Drinking Water Act in a way that better assures the public of safe drinking water, and to support the development of regulatory determinations that are transparent, clear, and consistent and that follow applicable agency policy, the EPA Administrator should require the Office of Water to expeditiously develop, and make available to the public, policies or guidance that clearly articulates the agency's interpretation of the act's broad statutory criteria for making regulatory determinations and provides a protocol for making such determinations. In particular, the guidance should specify that appropriate stakeholders--that is, EPA offices with relevant expertise such as the Office of Children's Health Protection and regional offices that have known or likely occurrence of the contaminants being evaluated in public water systems within their areas of jurisdiction--be encouraged and have the opportunity to participate in the regulatory determination work groups.

    Agency: Environmental Protection Agency
    Status: Open

    Comments: EPA's response to this recommendation reflects the practice that was in place at the time we issued our report including this recommendation. As a result, it does not appear that EPA has taken any additional action to satisfy this recommendation at this time.
    Recommendation: In light of EPA's decisions to issue health advisories in conjunction with determinations to not regulate certain contaminants that have been detected in some public water systems at levels of public health concern, the EPA Administrator should (1) determine whether the Office of Water's use of health advisories provides sufficient information on these unregulated contaminants to support timely and effective actions by states, localities, public water systems, and the public to ensure the safety of public drinking water, and (2) if not, direct the Office of Water to develop a plan to more effectively communicate such information to these entities.

    Agency: Environmental Protection Agency
    Status: Open

    Comments: EPA indicated that it has begun a process of developing concise updates to carry forward into its health advisory table and that the process will be completed gradually over the next year, with updates continuing in the future, but has not yet published any updates for non-microbial contaminants. In addition, EPA has not addressed the call in the recommendation for the documentation of the process to update the advisories. The agency did indicate its plan to update the Drinking Water Standards and Health Advisory table in the fall of 2016, so we will leave this recommendation open and continue to monitor for future actions.
    Director: White, James R
    Phone: (202)512-3000

    2 open recommendations
    Recommendation: In order to ensure the most efficient, fair, and consistent administration of civil tax penalties, and that penalties are achieving their purpose of encouraging voluntary compliance, the Commissioner of Internal Revenue should direct the Office of Servicewide Penalties (OSP) to evaluate penalty administration and penalties' effect on voluntary compliance.

    Agency: Department of the Treasury: Internal Revenue Service
    Status: Open

    Comments: The Office of Servicewide Penalties (OSP) initiated a plan to comprehensively evaluate penalty administration and the impact of penalties on voluntary compliance. They stated that they understand such a plan will be useful in identifying priorities and determining additional potential resource needs. However, OSP put this plan on hold while they developed a business case for obtaining more staff and resources. OSP is in the process of obtaining feedback to refine their business case for final submission. While they await approval of the business case they have not done any work on the comprehensive plan. As the comprehensive plan remains incomplete, OSP has not yet undertaken an evaluation of penalties' effect on voluntary compliance. In December 2011 OSP formed a Civil Penalties Administrative Improvement Initiative team that has the goal of making improvements to civil penalty administration. Activities underway as of February 2013 include developing measures to improve taxpayer consistency and taking actions to improve the Reasonable Cause Assistant computer system. We learned in February 2015 that the program is now under a new executive. When we met with the new agency officials in April 2015 they told us that they had a new effort underway to develop a comprehensive strategy that examines the impact of penalties on voluntary compliance while ensuring quality. IRS finalized a Penalty Performance Plan on March 6, 2017, which we are currently reviewing.
    Recommendation: In order to ensure the most efficient, fair, and consistent administration of civil tax penalties, and that penalties are achieving their purpose of encouraging voluntary compliance, the Commissioner of Internal Revenue should direct OSP to develop and implement a plan to collect and analyze penalty-related data. The plan should address the constraints officials have identified as impeding progress in analyzing penalties.

    Agency: Department of the Treasury: Internal Revenue Service
    Status: Open

    Comments: Office of Servicewide Penalties (OSP) initiated a plan to comprehensively evaluate penalty administration and the impact of penalties on voluntary compliance. However, OSP put this plan on hold while they developed a business case for obtaining more staff and resources. As of February 2013 the business case has not progressed and OSP has taken no additional action to complete the comprehensive plan. We learned in February 2015 that the program is now under a new executive. When we met with the new agency officials in April 2015 they told us that they had a new effort underway to develop a comprehensive strategy that examines the impact of penalties on voluntary compliance while ensuring quality. IRS finalized a Penalty Performance Plan on March 6, 2017, which we are currently reviewing.
    Director: Steinwald, Alan Bruce
    Phone: (202)512-3000

    1 open recommendations
    Recommendation: Given the contribution of physicians to Medicare spending in total, the Administrator of CMS should develop a profiling system that identifies individual physicians with inefficient practice patterns and, seeking legislative changes as necessary, use the results to improve the efficiency of care financed by Medicare. The profiling system should include methods for measuring the impact of physician profiling on program spending and physician behavior.

    Agency: Department of Health and Human Services: Centers for Medicare and Medicaid Services
    Status: Open

    Comments: Physician feedback reporting was initiated under section 131(c) of the Medicare Improvements for Patients and Providers Act of 2008 (MIPPA), and was expanded by section 3003 of the Patient Protection and Affordable Care Act (PPACA). In addition, PPACA required the Department of Health and Human Services to coordinate the physician feedback program with a Value Modifier (VM) that will adjust fee-for-service physician payments for the relative quality and cost of care provided to beneficiaries. In 2012, CMS provided Quality and Resource Use Reports (QRUR) to large providers nationwide and physician-focused QRURs to groups with 25 or more eligible providers in 9 states; by 2014, CMS sent QRURs to all group practices and solo practitioners. Also, as required in the act, CMS applied the VM to select physicians in 2015, with all physicians being subject to VM by 2017. The Act requires the VM to be implemented in a budget neutral manner, meaning that any upward payment adjustments for high performance must balance the downward payment adjustments applied for poor performance. CMS officials said they develop and will continue to develop experience reports related to each year's QRUR/VM cycle. In 2015, CMS used VM results for physicians in groups of 100 or more in public engagement of stakeholders, encouraging them to report quality, because quality performance was the driver of the payment adjustments in 2015. CMS is working with its Center for Clinical Standards and Quality to think of how to better engage physicians and groups in reporting, to avoid the automatic downward adjustment. As the program gains experience, CMS will use the experience reports to examine the impact of QRURs and VM on the Medicare program. In order for this recommendation to be closed as implemented, CMS will need to expand its efforts to measure the impact of QRURs and VM on program spending and physician behavior.