Reports & Testimonies

  • GAO’s recommendations database contains report recommendations that still need to be addressed.

    GAO’s recommendations help congressional and agency leaders prepare for appropriations and oversight activities, as well as help improve government operations. Recommendations remain open until they are designated as Closed-implemented or Closed-not implemented. You can explore open recommendations by searching or browsing.

    GAO's priority recommendations are those that we believe warrant priority attention. We sent letters to the heads of key departments and agencies, urging them to continue focusing on these issues. These recommendations are labeled as such. You can find priority recommendations by searching or browsing our open recommendations below, or through our mobile app.

  • Browse Open Recommendations

    Explore priority recommendations by subject terms or browse by federal agency

    Search Open Recommendations

    Search for a specific priority recommendation by word or phrase



  • Governing on the go?

    Our Priorities for Policy Makers app makes it easier for leaders to search our recommendations on the go.

    See the November 10th Press Release


  • Have a Question about a Recommendation?

    • For questions about a specific recommendation, contact the person or office listed with the recommendation.
    • For general information about recommendations, contact GAO's Audit Policy and Quality Assurance office at (202) 512-6100 or apqa@gao.gov.
  • « Back to Results List Sort by   

    Results:

    Subject Term: "Systems testing"

    6 publications with a total of 16 open recommendations
    Director: Michele Mackin
    Phone: (202) 512-4841

    3 open recommendations
    Recommendation: To mitigate the risk of poor acquisition outcomes and strengthen the department's investment decisions, the Secretary of Homeland Security should direct the Undersecretary for Management to update the acquisition policy to require that major acquisition programs' technical requirements are well defined and key technical reviews are conducted prior to approving programs to initiate product development and establishing Acquisition Program Baselines, in accordance with acquisition best practices.

    Agency: Department of Homeland Security
    Status: Open

    Comments: In providing comments on this report, DHS concurred with our recommendation and stated that it planned to initiate a study to assess how to better align its processes for technical reviews and acquisition decisions. Upon completion of the study, DHS plans to update its acquisition policies, as appropriate.
    Recommendation: To mitigate the risk of poor acquisition outcomes and strengthen the department's investment decisions, the Secretary of Homeland Security should direct the Undersecretary for Management to update the acquisition policy to specify that acquisition decision memorandums clearly document the rationale of decisions made by DHS leadership, such as, but not limited to, the reasons for allowing programs to deviate from the requirement to obtain department approval for certain documents at Acquisition Decision Events and the results of considerations or trade-offs.

    Agency: Department of Homeland Security
    Status: Open

    Comments: In providing comments on this report, DHS concurred with our recommendation and stated that it had begun expanding the content included in Acquisition Decision Memorandums (ADM) to include greater detail and that future ADMs would address the status of the acquisition documentation. DHS also said it had updated the guidance for writing ADMs in a handbook for Office of Program Accountability and Risk Management staff, thus making progress toward satisfying the recommendation. However, we did not close the recommendation because the updated guidance was not incorporated into the department's official acquisition policy, which may limit DHS's ability to implement the changes consistently over time. We will continue to review ADMs to assess whether the department's actions address the intent of this recommendation.
    Recommendation: To mitigate the risk of poor acquisition outcomes and strengthen the department's investment decisions, the Secretary of Homeland Security should direct the Undersecretary for Management to update the acquisition policy to specify at what point minimum standards for KPPs should be met, and clarify the performance data that should be used to assess whether or not a performance breach has occurred.

    Agency: Department of Homeland Security
    Status: Open

    Comments: In providing comments on this report, DHS concurred with our recommendation and stated that it had updated guidance related to its performance breaches in a handbook for Office of Program Accountability and Risk Management staff, thus moving toward satisfying the intent of this recommendation. Specifically, DHS identified that programs' KPPs should be met and verified no later than initial operational test and evaluation conducted prior to Acquisition Decision Event 3, the point at which DHS leadership approves the program to transition into sustainment. If programs have not met a KPP by this point, they will be required to declare a performance breach and submit a remediation plan documenting the root cause of the breach, along with how and when the breach will be resolved. However, we did not close the recommendation because the department's official acquisition policy has yet to be updated. DHS will fully address this recommendation when it incorporates the changes into its acquisition policy to ensure that the updated guidance on performance breaches is communicated and implemented consistently throughout the department.
    Director: Carol C. Harris
    Phone: (202) 512-4456

    3 open recommendations
    Recommendation: To help improve the management of DOD's MAIS programs, the Secretary of Defense should direct the Secretary of the Army to direct the program manager for Global Combat Support System-Army Increment 1 to establish standard operating procedures for managing risks that include guidance for establishing thresholds and bounds for key risk areas.

    Agency: Department of Defense
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Recommendation: To help improve the management of DOD's MAIS programs, the Secretary of Defense should direct the Secretary of the Air Force to direct the program manager for Air and Space Operations Center-Weapon System Increment 10.2 to develop an overall risk mitigation plan to guide the implementation of individual risk mitigation and contingency plan activities.

    Agency: Department of Defense
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Recommendation: To help improve the management of DOD's MAIS programs, the Secretary of Defense should direct the Secretary of the Air Force to direct the program manager for Joint Space Operations Center, Mission System Increment 2 to appoint a chief developmental tester to oversee systems testing and integration activities.

    Agency: Department of Defense
    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
    Director: David Powner
    Phone: (202) 512-9286

    5 open recommendations
    Recommendation: In order to institutionalize sound IT management practices and build FSA's IT management capacity while improving service to the Nation's farmers and ranchers, the Secretary of Agriculture should direct the FSA Administrator to establish and implement an improvement plan to guide the agency in adopting recognized best practices and following agency policy.

    Agency: Department of Agriculture
    Status: Open

    Comments: FSA developed a Strategic IT Roadmap to assist the agency's business and IT leadership in prioritizing IT investments. In addition, FSA stated that it will develop and document a comprehensive improvement plan that is to delineate tactical steps, timelines, and performance metrics to track incremental progress in adopting recognized best practices and program management capabilities. We will continue to monitor the agency's progress in documenting and implementing its improvement plan.
    Recommendation: In order to institutionalize sound IT management practices and build FSA's IT management capacity while improving service to the Nation's farmers and ranchers, the Secretary of Agriculture should direct the FSA Administrator to adhere to recognized best practices and agency policy in developing and managing system requirements before proceeding with any further system development to deliver previously envisioned MIDAS functionality. Specifically, the Administrator should ensure that requirements are complete, unambiguous, and prioritized; commitment to requirements is obtained through a formal requirements baseline; differences (or gaps) between the requirements and capabilities of the intended solution (including commercial off-the-shelf solutions) are analyzed; strategies to address any gaps are developed; and requirements are traced forward and backward among development products.

    Agency: Department of Agriculture
    Status: Open

    Comments: FSA reported that it will improve the rigor and adherence to requirements management processes for all IT projects, utilizing processes and tools that will support the integrity of the requirements throughout the lifecycle, to ensure that requirements are complete, formally baselined, gaps are analyzed, and fully traceable forward and backward. FSA also noted that it is pursuing an enhanced, more comprehensive governance structure that will further support its commitment to increasing rigor and adherence to defined requirements management processes. We will continue to monitor the agency's implementation of these efforts.
    Recommendation: In order to institutionalize sound IT management practices and build FSA's IT management capacity while improving service to the Nation's farmers and ranchers, the Secretary of Agriculture should direct the FSA Administrator to adhere to recognized best practices and agency policy in planning and monitoring projects. Specifically, the Administrator should ensure that project plans include predefined expectations for cost, schedule, and deliverables before proceeding with any further system development; updates to the project plan are made through change control processes; and progress against the project plan, including work performed by contractors, is monitored.

    Agency: Department of Agriculture
    Status: Open

    Comments: FSA noted that it began an initiative to improve the agency's use of capital planning guidance from the Office of Management and Budget and would prepare corrective action plans to address identified weaknesses in fiscal year 2016. FSA also noted that it was conducting a series of training classes on capital planning and IT project management across the agency, developing a risk management program, and strengthening the use of earned value management. We will continue to monitor the agency's progress on its project planning efforts.
    Recommendation: In order to institutionalize sound IT management practices and build FSA's IT management capacity while improving service to the Nation's farmers and ranchers, the Secretary of Agriculture should direct the FSA Administrator to adhere to recognized best practices and agency policy in system testing. Specifically, the Administrator should establish well-defined test plans before proceeding with any further system development, and ensure that testing of (a) individual system components, (b) the integration of system components, and (c) the end-to-end system are conducted.

    Agency: Department of Agriculture
    Status: Open

    Comments: FSA stated that going forward the agency will adhere to recognized best practices and agency policy in pursuing consistent or increased rigor around system testing. The agency noted that it plans to demonstrate that its testing capabilities are consistent and repeatable across all FSA IT projects. We will continue to monitor the agency's implementation of these efforts.
    Recommendation: In order to institutionalize sound IT management practices and build FSA's IT management capacity while improving service to the Nation's farmers and ranchers, the Secretary of Agriculture should direct the FSA Administrator to adhere to recognized best practices and agency policy in executive-level IT governance before proceeding with any further system development. Specifically, an executive-level governance board should (1) review and approve a comprehensive business case that includes a life cycle cost estimate, a cost-benefit analysis, and an analysis of alternatives for proposed solutions that are to provide former MIDAS requirements prior to their implementation; (2) ensure that any programs that are to accommodate former MIDAS requirements are fully implementing the IT program management disciplines and practices identified in this report; (3) conduct a post-implementation review and document lessons learned for the MIDAS investment; and (4) reassess the viability of the MIDAS technical solution before investing in further modernization technologies.

    Agency: Department of Agriculture
    Status: Open

    Comments: FSA stated that, as part of its organizational transformation efforts, the CIO is evaluating its governance structure and updating the charter for the agency-wide IT investment review board with the support of the agency's Executive Leadership Council. FSA also noted that it will adhere to the department's governance framework and processes. We will continue to monitor the agency's implementation of these efforts and how they address our recommendation.
    Director: Fleming, Susan A
    Phone: (202) 512-2834

    1 open recommendations
    Recommendation: To help ensure that the Federal Railroad Administration manages its limited resources and provides flexibility to railroads in implementing PTC, Congress should consider amending RSIA as requested in the FRA's August 2012 PTC Implementation Status Report to Congress, including granting FRA the authority to approve the use of alternative safety technologies in lieu of PTC to allow railroads to improve safety and meet many of the functions of PTC through other means.

    Agency: Congress
    Status: Open

    Comments: As of August 2017, Congress had not taken any action regarding this Matter for Congressional Consideration.
    Director: White, James R
    Phone: (202)512-5594

    1 open recommendations
    Recommendation: To ensure that IRS can adequately enforce certain tax provisions, Congress may wish to consider providing IRS with MEA to use tax return information from previous years to ensure that taxpayers do not improperly claim credits or deductions in excess of lifetime limits where applicable.

    Agency: Congress
    Status: Open

    Comments: As of April 2017, Congress had not yet provided IRS with math error authority (MEA) to use tax return information from previous years to ensure that taxpayers do not improperly claim credits or deductions in excess of lifetime limits.
    Director: Melvin, Valerie C
    Phone: (202)512-6304

    3 open recommendations
    Recommendation: To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to ensure implementation of a requirements management plan that reflects leading practices for requirements development and management. Specifically, implementation of the plan should include analyzing requirements to ensure they are complete, verifiable, and sufficiently detailed to guide development, and maintaining requirements traceability from high-level operational requirements through detailed low-level requirements to test cases.

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: In written comments on our report, VA concurred with this recommendation and in August 2014, identified initial actions the department had taken in response. Specifically, as part of its plans to issue a request for proposals to acquire a replacement scheduling system under its Medical Appointment Scheduling System (MASS) project, VA developed a Business Requirements Document that defines its specific business needs, capabilities, features, and constraints. Additionally, the department reported that it intends to manage and document requirements using processes supported by a Web-based tool called Rational Doors. In August 2015, VA's Office of Acquisition, Logistics, and Construction awarded a contract for the MASS project. However, in April 2016, the department paused MASS to evaluate an alternative project to enhance its legacy scheduling system. Subsequently, in early 2017, the department restarted the MASS project. Nevertheless, as of June 2017, the department had not developed a requirements management plan for MASS. Thus, the MASS project has not yet reached the point where the effectiveness of the requirements management activities we recommended can be assessed.
    Recommendation: To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to adhere to the department's guidance for system testing including (1) performing testing incrementally and (2) resolving defects of average and above severity prior to proceeding to subsequent stages of testing.

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: In written comments on our report, VA concurred with this recommendation and stated that testing was managed using documented, repeatable processes that are included in the department's ProPath Web-based tool. According to the Acting Deputy Chief Information Officer for Product Development, the Medical Appointment Scheduling System (MASS) project is expected to incorporate Agile software development practices, including the use of incremental testing. In August 2015, the department awarded a contract for the MASS project that included task orders for the development of test plans. However, in April 2016, the department paused MASS to evaluate an alternative to enhance its legacy scheduling system. In early 2017, the department restarted the MASS project, but as of June 2017, had not developed a test plan for MASS. Thus, the project has not yet reached the point where adherence to the department's system testing guidance can be assessed.
    Recommendation: To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to ensure that the policies and procedures VA is establishing to provide meaningful program oversight are effectively executed and that they include (1) robust collection methods for information on project costs, benefits, schedule, risk assessments, performance metrics, and system functionality to support executive decision making; (2) the establishment of reporting mechanisms to provide this information in a timely manner to department IT oversight control boards; and (3) defined criteria and documented policies on actions the department will take when development deficiencies for a project are identified.

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: The Department of Veterans Affairs concurred with this recommendation and identified various actions it has taken in response. Specifically, the department awarded a contract for its Medical Appointment Scheduling System (MASS) project in August 2015. However, in April 2016, it paused MASS to evaluate an alternative to enhance its legacy scheduling system. In June 2017, VA reported that the MASS project had been resumed and indicated that it would adhere to the department's Veteran-focused Intake Process (VIP), which is intended to ensure oversight, accountability, and traceability of all program activity. Also, the department reported that MASS had met its first VIP milestone, Critical Decision 1, in January 2017. However, key future activities, including product development and testing, have not yet been demonstrated, while VIP milestones (e.g., Critical Decision 2), have not yet been met. Thus, MASS has not reached the point where the effectiveness of project oversight can be fully assessed.