Defense Transportation: Opportunities Exist to Enhance the Credibility of the Current and Future Mobility Capabilities Studies
Highlights
We are reviewing the processes the Department of Defense (DOD) is using to conduct its Mobility Capabilities Study (MCS). The MCS is to address changes in DOD's transportation force structure and mobility requirements due to changes in threats and certain national security and military strategies. The study results may underpin decisions on future strategic airlift, aerial refueling aircraft, and sealift procurements. The study relies on the use of various models and data inputs to develop and evaluate transportation alternatives, including variations in alternative transportation modes (air, land, sea) and sources (military, civilian, foreign), as well as factors that affect transportation mode and source decisions. The Senate Armed Services Committee directed us to monitor the conduct of the MCS and report on the adequacy and completeness of the report no later than 30 days after DOD completes the study. DOD plans to issue the MCS report during 2005. This letter is intended to bring to the Secretary of Defense's attention preliminary observations on certain aspects of the MCS methodology to permit you to ensure the credibility of this and future studies. In our letter, we address the adequacy of the department's verification, validation, and accreditation (VV&A) of the models and simulations being used to conduct the MCS--that is, the process the MCS team is using to identify the models' capabilities, limitations, and performance relative to the real world events they simulate. We will continue to monitor the MCS and will report on the adequacy and completeness of the methodology after DOD issues its report.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | The Secretary of Defense should direct the Director, Office of Program Analysis and Evaluation, to develop documentation that describes the equivalent VV&A process used to verify and validate the mobility models and baseline data used to conduct the MCS prior to publishing any portion of the study results. |
Closed – Not Implemented
DOD states that these models have been used for years and their strengths and weaknesses are known and adjusted for. Officials don't see the need to do a VV&A on older models.
|
| Department of Defense | The Secretary of Defense should direct the Director, Office of Program Analysis and Evaluation, to disclose in the published MCS report the equivalent VV&A process used on the models and baseline data. |
Closed – Not Implemented
DOD did not disclose the equivalent process in the MCS report.
|
| Department of Defense | The Secretary of Defense should direct the Director, Office of Program Analysis and Evaluation, to work with the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics to evaluate the current DOD VV&A guidance to determine its relevance for use with legacy models and to change the guidance if appropriate. |
Closed – Not Implemented
DOD has no plans for reviewing the VV&A guidance.
|