Reports & Testimonies

  • GAO’s recommendations database contains report recommendations that still need to be addressed.

    GAO’s recommendations help congressional and agency leaders prepare for appropriations and oversight activities, as well as help improve government operations. Recommendations remain open until they are designated as Closed-implemented or Closed-not implemented. You can explore open recommendations by searching or browsing.

    GAO's priority recommendations are those that we believe warrant priority attention. We sent letters to the heads of key departments and agencies, urging them to continue focusing on these issues. These recommendations are labeled as such. You can find priority recommendations by searching or browsing our open recommendations below, or through our mobile app.

  • Browse Open Recommendations

    Explore priority recommendations by subject terms or browse by federal agency

    Search Open Recommendations

    Search for a specific priority recommendation by word or phrase



  • Governing on the go?

    Our Priorities for Policy Makers app makes it easier for leaders to search our recommendations on the go.

    See the November 10th Press Release


  • Have a Question about a Recommendation?

    • For questions about a specific recommendation, contact the person or office listed with the recommendation.
    • For general information about recommendations, contact GAO's Audit Policy and Quality Assurance office at (202) 512-6100 or apqa@gao.gov.
  • « Back to Results List Sort by   

    Results:

    Subject Term: "Systems conversions"

    7 publications with a total of 36 open recommendations
    Director: Valerie Melvin
    Phone: (202) 512-6304

    4 open recommendations
    Recommendation: To address weaknesses in the department's financial management systems modernization efforts, the Secretary of HUD should direct the Chief Financial Officer to work with the Chief Information Officer in managing subsequent initiatives to define a high-level depiction of the IT systems anticipated in the future state, a description of the operations that must be performed and who must perform them, and an explanation of where and how the operations are to be carried out.

    Agency: Department of Housing and Urban Development
    Status: Open

    Comments: In its comments on our draft report, HUD neither agreed nor disagreed with our recommendations, but noted that it planned to improve management practices and IT governance for future modernization efforts. In May 2017, HUD's Deputy Chief Information Officer reported that that the office was managing multiple enterprise-level initiatives no longer classified as financial management modernization efforts, but which are intended to address certain previously reported financial systems modernization needs. The department provided early high-level requirements and a solution architecture for one such initiative, including a future requirement to support data required for HUD's financial reporting needs from Treasury. However, HUD does not yet have a plan to develop a high-level concept of operations for IT systems anticipated in the future state. We intend to follow up on HUD's actions.
    Recommendation: To address weaknesses in the department's financial management systems modernization efforts, the Secretary of HUD should direct the Chief Financial Officer to work with the Chief Information Officer in managing subsequent initiatives to develop comprehensive plans for scope, schedule and cost.

    Agency: Department of Housing and Urban Development
    Status: Open

    Comments: In its comments on our draft report, HUD neither agreed nor disagreed with our recommendations, but noted that it planned to improve management practices and IT governance for future modernization efforts. In May 2017, the department provided an early project oversight plan and critical task schedule for one initiative related to enterprise data management, but these plans are not comprehensive and do not include, among other things, detailed cost estimates. We intend to follow up on HUD's actions.
    Recommendation: To address weaknesses in the department's financial management systems modernization efforts, the Secretary of HUD should direct the Chief Financial Officer to work with the Chief Information Officer in managing subsequent initiatives to ensure requirements are fully documented and traceable.

    Agency: Department of Housing and Urban Development
    Status: Open

    Comments: In its comments on our draft report, HUD neither agreed nor disagreed with our recommendations, but noted that it planned to improve management practices and IT governance for future modernization efforts. In March 2017, the department reported that the Chief Financial Officer and the Chief Information Officer intend to partner on future departmental financial management systems modernization efforts to fully document requirements and trace requirements to the functionality in the modernized system. In May 2017, department officials reported that the subsequent initiatives underway were following an Agile process yielding product-release backlogs as documentation of requirements for ongoing initiatives. They provided the initial backlog for an enterprise data management initiative. However, HUD could not demonstrate that these requirements were complete and traceable to mission needs. We intend to follow up on HUD's actions.
    Recommendation: The Secretary of HUD should also direct the Deputy Secretary to ensure that the Chief Information Officer takes action to improve IT governance control activities used for monitoring programs and identifying needed corrective actions, and strengthen investment oversight by improving coordination with stakeholders and alignment among IT modernization efforts.

    Agency: Department of Housing and Urban Development
    Status: Open

    Comments: In its comments on our draft report, HUD neither agreed nor disagreed with our recommendations, but noted that it planned to improve management practices and IT governance for future modernization efforts. In March 2017, the department reported on its fiscal year 2016 updates to charters of its IT governance boards, which provide oversight of all its IT investments, including financial management initiatives, and noted that business cases for proposed development and modernization initiatives had been discussed at governance meetings. HUD also reported that it had set up steering committees to supplement board governance and monitoring two enterprise-level modernization efforts and planned to apply mechanisms, such as project health assessments, intended to establish effective investment oversight. However, HUD has not yet demonstrated that the updated governance control activities have improved program monitoring and identified any needed corrective actions or that planned oversight mechanisms have improved coordination with stakeholders or alignment of modernization efforts. We intend to follow up on HUD's actions to ensure that planned improvements to governance and oversight mechanisms are effectively implemented and institutionalized.
    Director: David A. Powner
    Phone: (202) 512-9286

    22 open recommendations
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings.

    Agency: Department of Agriculture
    Status: Open

    Comments: When we confirm what actions have been taken, we will update the recommendation status.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings.

    Agency: Department of Education
    Status: Open

    Comments: The Department agreed with the recommendation, but has not provided an update on its actions to address it. When we confirm what actions have been taken, we will update.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings.

    Agency: Department of Energy
    Status: Open

    Comments: The Department agreed with the recommendation and, in a written response, stated that the Office of the CIO will update the CIO's OMB IT Dashboard Standard Operating Procedure to include the evaluation and assessment of active risks. This effort is to be completed by the end of December 2016. We will continue to monitor the implementation of this recommendation.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings.

    Agency: Department of Health and Human Services
    Status: Open

    Comments: The Department agreed with the recommendation and, in a written response, stated that it updated its CIO evaluation methodology to measure active risks in areas such as budget variance, performance, policy and governance compliance, risk management, and contract risk. When we confirm what actions have been taken, we will update.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings.

    Agency: Department of the Interior
    Status: Open

    Comments: The Department agreed with this recommendation and, in a written response, stated that it plans to address this recommendation with the following actions: (1) developing a method to review and assign ratings for active risks that will be incorporated into CIO ratings and (2) integrating the risk rating methodology into a new process for all major investments' CIO ratings. We will continue to monitor the implementation of this recommendation.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings.

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: The Department agreed with the recommendation and, in a written response, stated that it is amending its current monthly review process to ensure that risks are factored into its IT Dashboard CIO ratings. VA expects to complete this effort during the first quarter of 2017. We will continue to monitor the implementation of this recommendation.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings.

    Agency: Department of State
    Status: Open

    Comments: The Department agreed with the recommendation, but has not provided an update on its actions to address the recommendation. When we confirm what actions have been taken, we will update.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings.

    Agency: Office of Personnel Management
    Status: Open

    Comments: When we confirm what actions have been taken, we will update the recommendation status.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Defense, Education, and Homeland Security; and the Commissioner of the Social Security Administration should direct their CIOs to update their CIO ratings at least as frequently as required in OMB's guidance.

    Agency: Department of Defense
    Status: Open

    Comments: The Department of Defense (DOD) disagreed with this recommendation. In its written response, the Department noted that its semi-annual reporting is consistent with FITARA requirements and is documented in its OMB-approved FITARA Implementation Plan. After the publication of our report in June 2016, OMB issued its "Fiscal Year 2018 IT Budget-Capital Planning Guidance." This guidance removes the mandatory reporting frequency, but states that OMB expects that the CIOs would evaluate and rate their investments at specific times, including when the investment business cases are submitted to OMB in the agency budget request and when the business cases are prepared for the President's Budget release. In light of this new guidance, we analyzed the Department's update frequency for its 34 major investments (as listed on the IT Dashboard in June 2017). From June 2016 through May 2017, we found that 26 of the investments' ratings were updated once: in May 2017. The other 8 investments were not updated during this timeframe. Prior to this, the last DOD rating updates were made in March 2016, over a year beforehand. This analysis shows that DOD is not adhering to either its own semi-annual reporting requirements or to OMB's expectations. As such, we are not closing the recommendation at this time. We will continue to monitor the IT Dashboard for changes to DOD's update frequency. We maintain that frequent rating updates help ensure that the information on the Dashboard is timely and accurately reflects recent changes. Without such updates, the CIO ratings on the IT Dashboard may not reflect the current level of investment risk.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of Homeland Security
    Status: Open

    Comments: The Department agreed with the recommendation and, in a written response, stated that the Office of the CIO Enterprise Business Management Office is updating its program assessment guideline. The updated guideline will include risk-based scores as the basis for its investment ratings. The Department expects to release this new guideline by the end of December 2016. We will continue to monitor the implementation of this recommendation.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of Agriculture
    Status: Open

    Comments: When we confirm what actions have been taken, we will update the recommendation status.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of Education
    Status: Open

    Comments: The Department agreed with the recommendation, but has not provided an update on its actions to address it. When we confirm what actions have been taken, we will update.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of Commerce
    Status: Open

    Comments: The Department agreed with our recommendation and, in a written response, stated that the CIO has revised the IT Dashboard assessment criteria to directly incorporate the degree of risk represented in the investments' Business Case documents. We will continue to monitor the implementation of this recommendation.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of Defense
    Status: Open

    Comments: When we confirm what actions have been taken, we will update the recommendation status.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of Energy
    Status: Open

    Comments: The Department agreed with the recommendation and, in a written response, stated that the Office of the CIO will update its IT Dashboard Standard Operating Procedure to include an active risk sub-criteria comprised of probability and impact scores. This effort is to be completed by the end of December 2016. We will continue to monitor the implementation of this recommendation.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of Health and Human Services
    Status: Open

    Comments: The Department agreed with the recommendation and, in a written response, stated that it updated its CIO evaluation methodology to measure active risks in areas such as budget variance, performance, policy and governance compliance, risk management, and contract risk. According to HHS, these risk areas reflect both internal and external risks that affect an investment's ability to accomplish its goals. When we confirm what actions have been taken, we will update.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Social Security Administration
    Status: Open

    Comments: The agency partially agreed with our recommendation and, in a written response, stated that its CIO rating criteria includes a review of the level of risk facing an investment relative to that investment's ability to accomplish its goals. The written statement also notes that the CIO receives regular updates from key stakeholders on investment risks and mitigation plans. When we confirm what actions have been taken, we will update.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of Transportation
    Status: Open

    Comments: When we confirm what actions have been taken, we will update the recommendation status.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of the Treasury
    Status: Open

    Comments: When we confirm what actions have been taken, we will update the recommendation status.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: The Department agreed with the recommendation and, in a written response, stated that it plans to require investment managers to assess operational risks detailing the probability and impact of pending threats to success. VA expects to complete this effort during the first quarter of 2017. We will continue to monitor the implementation of this recommendation.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Department of State
    Status: Open

    Comments: The Department agreed with the recommendation, but has not provided an update on its actions to address the recommendation. When we confirm what actions have been taken, we will update.
    Recommendation: To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals.

    Agency: Environmental Protection Agency
    Status: Open

    Comments: The agency disagreed with the recommendation and has not provided an update on its actions to address the recommendation. We will continue to monitor the implementation of this recommendation.
    Director: Carol R.Cha
    Phone: (202) 512-4456

    2 open recommendations
    Recommendation: The Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology, and Logistics to require MAIS programs to establish their first acquisition program baseline within 2 years of beginning work on the programs.

    Agency: Department of Defense
    Status: Open

    Comments: The Department developed a draft process document that states that business system (e.g. financial management, logistics management) programs should start development on at least one release within 24 months after programs have identified the needed capabilities and received approval to conduct further analysis into the potential delivery of the capabilities. We will follow-up with the Department for the final process document and guidance, when available.
    Recommendation: The Secretary of Defense should direct the Secretary of the Army to direct the Army (Financial Management and Comptroller) to complete a plan for conducting auditability testing of LMP Increment 2 functionality to ensure that such testing occurs prior to the LMP program management office deploying future functionality.

    Agency: Department of Defense
    Status: Open

    Comments: According to DOD officials, in response to our recommendation, the department developed a plan to conduct system testing on LMP Increment 2 in accordance with the Federal Information System Controls Audit Manual. The officials stated that the department's plan was to conduct this testing both prior to and after the deployment of new functionality to users. We have requested additional information and documentation from DOD regarding these LMP Increment 2 test plans in order to determine whether the testing associated with auditability of the system was to be conducted before deployment to users.
    Director: Asif A. Khan
    Phone: (202) 512-9869

    2 open recommendations
    Recommendation: To help improve the implementation of GCSS-Army, the Secretary of the Army should ensure that the Under Secretary of the Army, in his capacity as the Chief Management Officer, directs the GCSS-Army Program Management Office to develop an updated schedule that fully incorporates best practices, including (1) assigning resources to all activities, (2) establishing durations of all activities, (3) confirming that the critical path is valid, and (4) ensuring reasonable total float.

    Agency: Department of Defense: Department of the Army
    Status: Open

    Comments: We are in the process of obtaining an updated integrated master schedule from DOD to determine if Army fully incorporated best practices. As of June 2017,Army officials told us that the integrated master schedule and revised cost estimate will not be available until December 2017. This recommendation remains open.
    Recommendation: To help improve the implementation of GCSS-Army, the Secretary of the Army should ensure that the Under Secretary of the Army, in his capacity as the Chief Management Officer, directs the GCSS-Army Program Management Office to update the cost estimate to fully incorporate best practices by documenting the results of (1) a risk and uncertainty analysis, (2) the cross-checking of major cost elements to see if results are similar, and (3) a sensitivity analysis.

    Agency: Department of Defense: Department of the Army
    Status: Open

    Comments: We are in the process of obtaining an updated cost estimate from DOD to determine if Army fully incorporated best practices. As of June 2017, Army officials told us that the integrated master schedule and revised cost estimate will not be available until December 2017. This recommendation remains open.
    Director: Farrell, Brenda S
    Phone: (202)512-3604

    1 open recommendations
    Recommendation: To promote an efficient use of resources and to better plan for the design of a new performance management system, the Secretary of Defense should direct the Deputy Assistant Secretary of Defense for Civilian Personnel Policy to, in conjunction with the DOD Comptroller, help ensure that information identifying and supporting the costs of the NSPS termination and new performance management system is documented, reliable, traceable to a source document, and readily available for examination.

    Agency: Department of Defense
    Status: Open

    Comments: As of September 2015, there has been no evidence that the department has taken action to address this recommendation.
    Director: Melvin, Valerie C
    Phone: (202)512-6304

    3 open recommendations
    Recommendation: To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to ensure implementation of a requirements management plan that reflects leading practices for requirements development and management. Specifically, implementation of the plan should include analyzing requirements to ensure they are complete, verifiable, and sufficiently detailed to guide development, and maintaining requirements traceability from high-level operational requirements through detailed low-level requirements to test cases.

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: In written comments on our report, VA concurred with this recommendation and in August 2014, identified initial actions the department had taken in response. Specifically, as part of its plans to issue a request for proposals to acquire a replacement scheduling system under its Medical Appointment Scheduling System (MASS) project, VA developed a Business Requirements Document that defines its specific business needs, capabilities, features, and constraints. Additionally, the department reported that it intends to manage and document requirements using processes supported by a Web-based tool called Rational Doors. In August 2015, VA's Office of Acquisition, Logistics, and Construction awarded a contract for the MASS project. However, in April 2016, the department paused MASS to evaluate an alternative project to enhance its legacy scheduling system. Subsequently, in early 2017, the department restarted the MASS project. Nevertheless, as of June 2017, the department had not developed a requirements management plan for MASS. Thus, the MASS project has not yet reached the point where the effectiveness of the requirements management activities we recommended can be assessed.
    Recommendation: To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to adhere to the department's guidance for system testing including (1) performing testing incrementally and (2) resolving defects of average and above severity prior to proceeding to subsequent stages of testing.

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: In written comments on our report, VA concurred with this recommendation and stated that testing was managed using documented, repeatable processes that are included in the department's ProPath Web-based tool. According to the Acting Deputy Chief Information Officer for Product Development, the Medical Appointment Scheduling System (MASS) project is expected to incorporate Agile software development practices, including the use of incremental testing. In August 2015, the department awarded a contract for the MASS project that included task orders for the development of test plans. However, in April 2016, the department paused MASS to evaluate an alternative to enhance its legacy scheduling system. In early 2017, the department restarted the MASS project, but as of June 2017, had not developed a test plan for MASS. Thus, the project has not yet reached the point where adherence to the department's system testing guidance can be assessed.
    Recommendation: To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to ensure that the policies and procedures VA is establishing to provide meaningful program oversight are effectively executed and that they include (1) robust collection methods for information on project costs, benefits, schedule, risk assessments, performance metrics, and system functionality to support executive decision making; (2) the establishment of reporting mechanisms to provide this information in a timely manner to department IT oversight control boards; and (3) defined criteria and documented policies on actions the department will take when development deficiencies for a project are identified.

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: The Department of Veterans Affairs concurred with this recommendation and identified various actions it has taken in response. Specifically, the department awarded a contract for its Medical Appointment Scheduling System (MASS) project in August 2015. However, in April 2016, it paused MASS to evaluate an alternative to enhance its legacy scheduling system. In June 2017, VA reported that the MASS project had been resumed and indicated that it would adhere to the department's Veteran-focused Intake Process (VIP), which is intended to ensure oversight, accountability, and traceability of all program activity. Also, the department reported that MASS had met its first VIP milestone, Critical Decision 1, in January 2017. However, key future activities, including product development and testing, have not yet been demonstrated, while VIP milestones (e.g., Critical Decision 2), have not yet been met. Thus, MASS has not reached the point where the effectiveness of project oversight can be fully assessed.
    Director: Dillingham, Gerald L
    Phone: (202)512-3000

    2 open recommendations
    Recommendation: Because of the importance of the successful planning and implementation of NGATS and the need for effective collaboration between diverse organizations, the Secretary of Transportation should direct JPDO to better ensure the involvement of all key stakeholders in the NGATS planning process, by determining whether key stakeholders and expertise are not represented on JPDO's integrated product teams, divisions, or elsewhere within its organization. For example, JPDO should consider the addition of active, subject matter expert air traffic controllers to its integrated product teams.

    Agency: Department of Transportation
    Status: Open

    Comments: In multiple reports and testimonies on Air Traffic Control Modernization and subsequently the Next Generation Air Transportation System (NextGen) GAO has reported that a lack of stakeholder or expert involvement early and throughout a project can lead to cost increases and delays. We also found that the Joint Planning and Development Office within the Federal Aviation Administration (FAA) had not adequately involved air traffic controllers or the National Air Traffic Controllers Association (NATCA) in NextGen planning efforts, and have since reported on challenges facing FAA in involving controllers and other stakeholders as they begin to implement various NextGen improvements. GAO recommended that the Secretary of Transportation consider various actions to better ensure the involvement of all key stakeholders in planning and implementing NextGen and specifically recommended further involvement of active, subject matter expert air traffic controllers. Article 114 of the recently signed contract between the FAA and NATCA addresses this recommendation by directing full participation by NATCA to further the development and implementation of NextGen. The specifics of the agreement between FAA and NATCA regarding how and in what manner controllers will be included has yet to be determined, but the inclusion of Article 114 is a step in the right direction to ensuring active involvement. With greater active participation from air traffic controllers and NATCA, issues that cause cost increase and delays can be identified earlier and their effects mitigated.
    Recommendation: Given the technical complexity of the implementation of NGATS and FAA's past experiences, the Secretary of Transportation should direct the FAA to undertake a formal exploration of FAA's strengths and weaknesses with regard to the technical expertise and contract management expertise that will be required to define, implement, and integrate the numerous complex programs and systems inherent in the transition to NGATS. For example, FAA should work to determine whether it will need to contract with an LSI, federally-funded not-for-profit corporation, or other technical or managerial entity to assist in the implementation of NGATS.

    Agency: Department of Transportation
    Status: Open

    Comments: According to the Vice President for Planning, FAA, on June 13, 2007, FAA awarded a contract to the National Academy of Public Administration (NAPA) to do an independent assessment of FAA's technical and program management (i.e., contract management) skills required for NextGen implementation. The Vice President directly attributed FAA's ability to move the contract award to GAO's recommendation.