Skip to Highlights
Highlights

The Department of Veterans Affairs (VA) provides medical care, disability compensation, and vocational rehabilitation to veterans. The Veterans Health Administration (VHA)--a component of VA--provides care to over 5 million patients in more than 1,500 facilities. VHA relies on an outpatient scheduling system that is over 25 years old. In 2000, VHA began the Scheduling Replacement Project to modernize this system as part of a larger departmentwide modernization effort called HealtheVet. However, in February 2009, VA terminated a key contract supporting the project. GAO was asked to (1) determine the status of the Scheduling Replacement Project, (2) determine the effectiveness of VA's management and oversight of the project, and (3) assess the impact of the project on VA's overall implementation of its HealtheVet initiative. To do so, GAO reviewed project documentation and interviewed VA and contractor officials.

Skip to Recommendations

Recommendations

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Veterans Affairs To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the Chief Information Officer (CIO) to ensure acquisition plans document how competition will be sought, promoted, and sustained or identify the basis of authority for not using full and open competition.
Closed - Implemented

Recommendation status is Closed - Implemented.

In written comments on our report, VA concurred with this recommendation and stated that it would work closely with contracting officers to ensure future acquisition plans clearly identify an acquisition strategy that promotes full and open competition. The department subsequently issued a request for proposals to industry in November 2014, for the Medical Appointment Scheduling System (MASS) project that was informed by a request for information and market analysis. VA prepared an acquisition strategy for MASS that was intended to promote full and open competition and, in January 2015, the department received multiple proposals. VA subsequently awarded a contract for the MASS project in August 2015. As a result, VA has taken actions to ensure that its effort to replace its scheduling system will benefit from activities taken to promote full and open competition that were not taken for previous scheduling replacement efforts.
Department of Veterans Affairs To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to ensure implementation of a requirements management plan that reflects leading practices for requirements development and management. Specifically, implementation of the plan should include analyzing requirements to ensure they are complete, verifiable, and sufficiently detailed to guide development, and maintaining requirements traceability from high-level operational requirements through detailed low-level requirements to test cases.
Closed - Implemented

Recommendation status is Closed - Implemented.

In written comments on our report, VA concurred with this recommendation and awarded a contract for its Medical Appointment Scheduling System (MASS) project in August 2015. In July 2018, VA indicated that it followed its own guidance by documenting requirements for MASS in its internal requirements management system. Additionally, management of requirements is documented in the MASS Configuration and Change Management Plan. According to the MASS project manager, this plan and the MASS Master Test Plan ensure requirements traceability from high-level operational requirements through detailed low-level requirements to test cases. As a result, VA has taken actions to ensure the MASS effort will benefit from activities taken to reflect leading practices for requirements development and management.
Department of Veterans Affairs To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to adhere to the department's guidance for system testing including (1) performing testing incrementally and (2) resolving defects of average and above severity prior to proceeding to subsequent stages of testing.
Closed - Implemented

Recommendation status is Closed - Implemented.

In written comments on our report, VA concurred with this recommendation and in August 2015, the department awarded a contract for the Medical Appointment Scheduling System (MASS) project that included task orders for the development of test plans. In August 2018, the department provided its test plan for MASS and test results indicating that testing on the project had been performed incrementally with all defects of average and above severity resolved prior to proceeding to subsequent stages of testing. Additionally, the April 2018, MASS Critical Decision Point 2 briefing indicated that there were no open severity 1 or 2 defects prior to proceeding with deployment of MASS to its first pilot site. As a result, VA has taken actions to ensure that its effort to replace its scheduling system will benefit from activities taken to conduct incremental testing and resolve severe defects.
Department of Veterans Affairs To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to ensure effective implementation of earned value management (EVM) by making certain that the: (1) EVM reporting systems for the scheduling project are certified for compliance with American National Standards Institute/Electronic Industries Alliance (ANSI/EIA) Standard 748 and data resulting from the systems are reliable; (2) project status reports based on EVM data are reliable in their portrayal of the project's cumulative and current cost and schedule performance; and (3) officials responsible for managing and overseeing the project use earned value data as an input to their decision-making processes.
Closed - Not Implemented

Recommendation status is Closed - Not Implemented.

In written comments on our report, the Department of Veterans Affairs stated that it "concurred in principle" with our recommendation and noted that its Project Management Accountability System (PMAS) requires monthly analysis and reporting of project performance. However, PMAS project status reporting does not include EVM data.
Department of Veterans Affairs To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to identify risks related to the scheduling project moving forward and prepare plans and strategies to mitigate them.
Closed - Implemented

Recommendation status is Closed - Implemented.

The Department of Veterans Affairs concurred with our recommendation. The department created a risk register for its Medical Appointment Scheduling System (MASS) project that identifies risks as well as their probability and impact. In addition, the department instituted the use of its Project Management Accountability System and its ProPath Web-based tool to help mitigate risks related to the department's effort to replace its scheduling system.
Department of Veterans Affairs To enhance VA's effort to successfully fulfill its forthcoming plans for the outpatient scheduling system replacement project and the HealtheVet program, the Secretary of Veterans Affairs should direct the CIO to ensure that the policies and procedures VA is establishing to provide meaningful program oversight are effectively executed and that they include (1) robust collection methods for information on project costs, benefits, schedule, risk assessments, performance metrics, and system functionality to support executive decision making; (2) the establishment of reporting mechanisms to provide this information in a timely manner to department IT oversight control boards; and (3) defined criteria and documented policies on actions the department will take when development deficiencies for a project are identified.
Closed - Implemented

Recommendation status is Closed - Implemented.

The Department of Veterans Affairs concurred with this recommendation and awarded a contract for its Medical Appointment Scheduling System (MASS) project in August 2015. In June 2017, VA reported that the MASS project would adhere to the department's Veteran-focused Intake Process (VIP), which is intended to ensure oversight, accountability, and traceability of all program activity. The department has since followed this process and reported that MASS had met the VIP milestones of Critical Decision 1 in January 2017, and Critical Decision 2 in April 2018, both of which included oversight briefings. Additionally, according to the MASS project manager and project documentation, oversight is executed through weekly meetings where, among other things, cost, schedule, and critical risk and dependencies are discussed. As a result, VA has taken actions to ensure that its effort to replace its scheduling system will benefit from activities taken to provide meaningful program oversight.

Full Report