Military Disability System: Improved Monitoring Needed to Better Track and Manage Performance
Highlights
What GAO Found
Case processing times under the Integrated Disability Evaluation System (IDES) have increased over time, and measures of servicemember satisfaction have shortcomings. Since 2008, annual average processing times for IDES cases have steadily climbed, while the percentage of cases meeting established timeliness goals declined. Average case processing times reached 394 and 420 days for active and reserve component members in fiscal year 2011--compared to goals of 295 and 305 days, respectively, and just 19 percent of active duty and 18 percent of guard or reserve servicemembers completed the process and received benefits within established goals. Of the four phases comprising IDES, the medical evaluation board phase increasingly fell short of timeliness goals, while the physical evaluation board phase, although meeting goals, was taking increasingly more time to complete. With respect to servicemember satisfaction with the IDES process, GAO found shortcomings in how these data are collected and reported, such as unduly limiting who is eligible to receive a survey and computing average satisfaction scores in a manner that may overstate them. Department of Defense (DOD) officials told GAO they are considering alternatives for gauging satisfaction with the process. DOD and Veterans Affairs (VA) are taking steps to improve IDES performance, but progress to date is uneven and it is too early to assess their overall impact. For example, VA increased resources for completing exams and disability ratings while the Army is hiring additional staff for its medical evaluation boards. VA has met exam timeliness goals in the past several months, but other resources have yet to translate into lower processing times. DOD and VA are pursuing system upgrades so that staff and managers at IDES facilities can better track and manage the progress of servicemembers' cases. IDES officials have been working with the military services to correct case data that were inaccurately entered into VA's IDES tracking system, but have not yet identified a permanent solution to improve the accuracy of data input. Finally, DOD, with VA's assistance, is in the early stages of an in-depth review of the entire IDES process and its supporting IT systems. This effort is intended to improve understanding of how each step contributes to overall processing times and identify opportunities to streamline the process and supporting systems. However, timeframes for completing the review or issuing recommendations have yet to be established.
Why GAO Did This Study
Since 2007, DOD and VA have jointly operated IDES--which is intended to expedite benefits for injured servicemembers. IDES replaced the departments' separate processes for evaluating servicemembers for fitness and disability. Initially a pilot at 3 military treatment facilities, IDES is now in place at military treatment facilities worldwide. In previous reports, GAO identified a number of challenges as IDES expanded to more facilities, including staffing shortages and difficultly meeting timeliness goals. In light of IDES' expansion, GAO was asked to examine: (1) the extent to which DOD and VA are meeting IDES timeliness and servicemember satisfaction performance goals, and (2) steps the agencies are taking to improve IDES performance. GAO analyzed IDES timeliness and customer satisfaction data, visited six IDES sites with varying performance, and interviewed DOD and VA officials.
Recommendations
To improve monitoring of IDES timeliness and satisfaction, GAO recommends that DOD and VA work together to (1) develop plans for completing the ongoing business process review and implementing any resulting recommendations and (2) improve the accuracy of case information at the point of data entry; and that (3) DOD consider alternative approaches to measuring satisfaction. DOD and VA concurred with GAO's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status Sort descending |
|---|---|---|
| Department of Defense | To ensure that IDES management decisions continue to be based upon reliable and accurate data, the Secretaries of Defense and Veterans Affairs work together to develop a strategy to continuously monitor and remedy issues with VTA timeliness information. This could include issuing guidance to facilities or developing best practices on preventing and correcting data entry errors; and developing reporting capabilities in VTA to alert facilities to potential issues with their data. |
Closed – Implemented
VA and DOD have implemented various efforts to improve their oversight of Veterans Tracking Application (VTA) timeliness data and address accuracy issues. For example, the departments regularly held joint training sessions on the use of VTA, which were modified over time to address data error trends in VTA. VA also conducted training sessions in 2013 on how to properly enter data and correct any errors, and DOD reported that it conducted additional training at IDES sites as needed regarding unique or advanced VTA challenges. Further, VA published revised IDES guidance in March 2014 in its Compensation and Pension Manual, and expects this guidance to increase consistency in IDES, including with VTA compliance. In addition, both departments reported efforts to increase monitoring and oversight of VTA timeliness data. For example, VA held regular teleconferences in which it shared best practices regarding VTA and discussed error trends. VA and DOD also worked together to identify additional VTA functionality that would help users monitor their cases. As a result, VA reported releasing an update to VTA in February 2014 that contained new capabilities that will help staff track their caseload. VA also built into VTA the ability to automatically alert VA Military Service Coordinators when they have cases that are nearing the claims development stage deadline. For its part, DOD worked to lessen both delays and errors between the transition and benefit phases by creating the capability to enter servicemembers' separation dates directly into VTA and upload DD Form 214 discharge documentation into VA's systems. In addition to these improvements and ongoing measures, VA reported undertaking several one-time reviews to confirm the accuracy of insufficient exam report data and make corrections to the status of IDES cases as needed.
|
| Department of Veterans Affairs | To ensure that IDES management decisions continue to be based upon reliable and accurate data, the Secretaries of Defense and Veterans Affairs work together to develop a strategy to continuously monitor and remedy issues with VTA timeliness information. This could include issuing guidance to facilities or developing best practices on preventing and correcting data entry errors; and developing reporting capabilities in VTA to alert facilities to potential issues with their data. |
Closed – Implemented
VA and DOD have implemented various efforts to improve their oversight of Veterans Tracking Application (VTA) timeliness data and address accuracy issues. For example, the departments regularly held joint training sessions on the use of VTA, which were modified over time to address data error trends in VTA. VA also conducted training sessions in 2013 on how to properly enter data and correct any errors, and DOD reported that it conducted additional training at IDES sites as needed regarding unique or advanced VTA challenges. Further, VA published revised IDES guidance in March 2014 in its Compensation and Pension Manual, and expects this guidance to increase consistency in IDES, including with VTA compliance. In addition, both departments reported efforts to increase monitoring and oversight of VTA timeliness data. For example, VA held regular teleconferences in which it shared best practices regarding VTA and discussed error trends. VA and DOD also worked together to identify additional VTA functionality that would help users monitor their cases. As a result, VA reported releasing an update to VTA in February 2014 that contained new capabilities that will help staff track their caseload. VA also built into VTA the ability to automatically alert VA Military Service Coordinators when they have cases that are nearing the claims development stage deadline. For its part, DOD worked to lessen both delays and errors between the transition and benefit phases by creating the capability to enter servicemembers' separation dates directly into VTA and upload DD Form 214 discharge documentation into VA'?s systems. In addition to these improvements and ongoing measures, VA reported undertaking several one-time reviews to confirm the accuracy of insufficient exam report data and make corrections to the status of IDES cases as needed.
|
| Department of Defense | To ensure that servicemember cases are processed and are awarded benefits in a timely manner, we recommend that the Secretaries of Defense and Veterans Affairs work together to develop timeframes for completing the IDES business process review and implementing any resulting recommendations. |
Closed – Implemented
Since our 2012 report, DOD has taken concrete steps to evaluate potential changes to the IDES and plan their implementation, and VA has provided input as appropriate. DOD collaborated with the military services and VA to identify capability gaps and business needs in the IDES, and documented these needs in a joint IDES problem statement in 2014. This statement identified the need for improvement in areas, including case management and tracking, reporting, and electronic data exchange. DOD subsequently decided to pursue a joint IDES IT solution to help address these identified issues. In January 2015, DOD completed a Business Requirements Document, which defines the requirements for capabilities in such an IT system. By September 2015, DOD hopes to complete an Analysis of Alternatives study which will assess the cost, risk, and capabilities of government, commercial off-the-shelf, and existing DES support systems used by the military services and VA for their potential as the Joint IDES IT solution. DOD plans to conduct a proof of concept and prototype IT effort between September 2015 and May 2016, and have an initial operating capability for the system in late 2016 or early 2017. DOD also reported continuing its efforts to develop and refine computer models to simulate the effect of potential changes in the IDES. In March 2015, DOD completed an update of the model that incorporates recent efforts from the military services to streamline their processes as well as historical processing times from VA's Veterans Tracking Application (VTA). Further, DOD states that it has been working to standardize, assess, and monitor the services' disability evaluation system quality assurance programs as directed by Section 524 of the National Defense Authorization Act for fiscal year 2013. According to DOD, this effort will allow it to standardize quality assurance requirements across the services, better monitor and improve the consistency of decisions across Medical and Physical Evaluation Boards, and monitor the staff who perform these duties. DOD expects to implement these changes in 2016. Lastly, DOD reported working to develop an implementation manual for an Expedited Disability Evaluation System for servicemembers designated as catastrophically wounded, ill, or injured. DOD plans to coordinate development of this manual with VA to ensure it adheres to past agreements between the agencies, and publish the manual by early 2016.
|
| Department of Veterans Affairs | To ensure that servicemember cases are processed and are awarded benefits in a timely manner, we recommend that the Secretaries of Defense and Veterans Affairs work together to develop timeframes for completing the IDES business process review and implementing any resulting recommendations. |
Closed – Implemented
Since our 2012 report, DOD has taken concrete steps to evaluate potential changes to the IDES and plan their implementation, and VA has provided input as appropriate. DOD collaborated with the military services and VA to identify capability gaps and business needs in the IDES, and documented these needs in a joint IDES problem statement in 2014. This statement identified the need for improvement in areas, including case management and tracking, reporting, and electronic data exchange. DOD subsequently decided to pursue a joint IDES IT solution to help address these identified issues. In January 2015, DOD completed a Business Requirements Document, which defines the requirements for capabilities in such an IT system. By September 2015, DOD hopes to complete an Analysis of Alternatives study which will assess the cost, risk, and capabilities of government, commercial off-the-shelf, and existing DES support systems used by the military services and VA for their potential as the Joint IDES IT solution. DOD plans to conduct a proof of concept and prototype IT effort between September 2015 and May 2016, and have an initial operating capability for the system in late 2016 or early 2017. DOD also reported continuing its efforts to develop and refine computer models to simulate the effect of potential changes in the IDES. In March 2015, DOD completed an update of the model that incorporates recent efforts from the military services to streamline their processes as well as historical processing times from VA's Veterans Tracking Application (VTA). Further, DOD states that it has been working to standardize, assess, and monitor the services' disability evaluation system quality assurance programs as directed by Section 524 of the National Defense Authorization Act for fiscal year 2013. According to DOD, this effort will allow it to standardize quality assurance requirements across the services, better monitor and improve the consistency of decisions across Medical and Physical Evaluation Boards, and monitor the staff who perform these duties. DOD expects to implement these changes in 2016. Lastly, DOD reported working to develop an implementation manual for an Expedited Disability Evaluation System for servicemembers designated as catastrophically wounded, ill, or injured. DOD plans to coordinate development of this manual with VA to ensure it adheres to past agreements between the agencies, and publish the manual by early 2016.
|
| Department of Defense | To improve DOD's ability to measure servicemembers' satisfaction with the IDES process, the Secretary of Defense should develop alternative approaches for collecting more meaningful and representative information in a cost effective manner. |
Closed – Implemented
DOD resumed its customer satisfaction survey in July 2013 and has taken several steps to help ensure the information it collects and presents is more meaningful and representative. DOD implemented a quarterly reporting mechanism that includes data from the current and preceding quarters to allow reporting of results from more low density treatment facilities. According to DOD, the Defense Manpower Data Center allows reporting only from facilities with at least 10 cases, and combining quarterly data on a rolling basis allows DOD to meet this reporting threshold at more facilities. Additionally, DOD revised its satisfaction reports to provide information on the distribution of positive, neutral, and negative responses related to different portions of the IDES process and to individual survey questions. DOD also revised its survey administration procedures to ensure more servicemembers are eligible to participate. All servicemembers who complete the Medical Evaluation Board (MEB) stage are eligible to take the MEB survey and all who complete the Physical Evaluation Board (PEB) can take the PEB survey. Further, DOD began surveying a representative sample servicemembers at treatment facilities whose annual caseload is expected to exceed 800, while continuing to survey all servicemembers at smaller facilities. However, DOD reported that its move to probability sampling has not resulted in any significant reductions to its overall cost of administering the survey.
|