DMDC Data on Officers' Commissioning Programs is Insufficiently Reliable and Needs to be Corrected
GAO-07-372R: Published: Mar 8, 2007. Publicly Released: Mar 8, 2007.
The Defense Manpower Data Center (DMDC) is a key Department of Defense (DOD) support organization that, among other things, generates reports for defense organizations such as the military services, the Office of the Secretary of Defense, and the Joint Staff. External organizations such as GAO and federally funded research and development centers also rely on DMDC for quantitative data and analyses pertaining to a wide variety of issues, including the numbers of DOD personnel in specified occupations or demographic groups, servicemembers' attitudes, and compensation. DMDC reports to DOD's Office of the Under Secretary of Defense for Personnel and Readiness. In describing its databases, DMDC states that it maintains the largest archive of personnel, manpower, training, and financial data in DOD. It also notes that the personnel data are broad in scope and extend back to the early 1970s covering all services, all components of the total force (active duty, guard, reserve, and civilian), and all phases of the personnel life cycle (accession, separation, and retirement). DMDC data serve as the basis for DOD's annual Population Representation in the Military Forces which, among other things, provides information on the numbers of officers who were accessed into the military from each service's various commissioning programs: military academies, Reserve Officers' Training Corps (ROTC), Officer Candidate Schools (OCS: for the Army, Navy, and Air Force) and Officer Training School (OTS: for the Air Force). On January 19, 2007, we issued a report on officer accessions, retention, and foreign language training. The purpose of this report is to bring to your attention reliability issues with DMDC data that we encountered while preparing our report and to provide you with our recommendations to address these issues.
We found the information that DMDC provided to us on the number of officers accessed from DOD's various commissioning programs to be insufficiently reliable for use in our January 2007 report. Government auditing standards, which are applicable to all federal agencies including DOD, require that data be valid and reliable when the data are significant to the auditor's findings. More specifically, federal internal control standards require that data control activities, such as edit checks, verification, and reconciliation, be conducted and documented to help provide reasonable assurance that agency objectives are being met. We found discrepancies when we compared the DMDC-provided information on the number of officers accessed from DOD's commissioning programs (the academies, ROTC, and OCS/OTS) to information provided by the services. In the most extreme example of a discrepancy, DMDC-provided information indicated that 17 officers were accessed from the Marine Corps' ROTC program in fiscal year 2005, but Marine Corps-supplied information indicated that 160 officers were assessed. DMDC also provided us with information on the total number of officer accessions in fiscal year 2005, which were 6 per cent to 39 percent higher than the total numbers the four services provided us. Until DMDC corrects these data problems, it will be unable to provide policymakers with sufficiently reliable data upon which to base decisions related to officers. Therefore, we are recommending that DMDC reconcile its data with the services' data on officer accessions from the various commissioning programs for current and past officers and verify the accuracy of findings produced with the corrected data. In responding to a draft of this report, DOD partially concurred with our recommendation to reconcile DMDC and service data and did not concur with our recommendation to verify the corrected data. DOD stated, among other things, that we provided DMDC and the services with apparently different specifications in our requests for analyses. While we initially requested information on "all officers" from DMDC versus "commissioned officers" from the services, we clarified our specifications with DMDC staff before any analyses were begun. DOD also stated that DMDC has a longstanding policy to not correct historical data. In our evaluation of DOD's comments, we cite DOD and governmentwide policies that run contrary to DMDC's policy and emphasize the need for valid and reliable data.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: In this report, we identified problems with the data analyses and databases that the Defense Manpower Data Center (DMDC) used to provide information to us. In an April 2007 meeting, DMDC's Director stated that DMDC is taking steps to address the recommendations from this and other GAO reports. During the meeting the DMDC Director identified actions that DMDC had taken since the publication of this and other reports, as well as other actions still being implemented to improve DMDC data quality overall--rather than just in the databases where we had identified problems. For example, DMDC staff reviewed the Government Auditing Standards to understand GAO's need for data reliability metrics and completed three guides to enhance and standardize data management and data reporting procedures/processes. Also, the DMDC Director indicated that the agency would focus more effort on data reliability questionnaires for each GAO engagement in the future. In 2008 and again in 2009, DMDC implemented the changes that they had discussed with GAO in 2007.
Recommendation: The Secretary of Defense should direct the Under Secretary of Defense for Personnel and Readiness to require the Defense Manpower Data Center to reconcile its data with data from the services on accessions from the services' commissioning programs for current and past officers.
Agency Affected: Department of Defense
Status: Closed - Not Implemented
Comments: DOD strongly nonconcurred with this recommendation and they closed the recommendation in their system in June 2007. Based upon DOD's initial rejection of this recommendation and their lack of action regarding the recommendation to date, the status of this recommendation is listed as closed and not implemented. However, GAO continues to believe that DOD's Defense Manpower Data Center should verify the accuracy of it personnel data base through further analyses for each officer commissioning program.
Recommendation: The Secretary of Defense should direct the Under Secretary of Defense for Personnel and Readiness to require the Defense Manpower Data Center to verify the accuracy of its corrected database through further analyses such as demographic breakouts (such as race, gender, and year accessed) for each officer commissioning program.
Agency Affected: Department of Defense