Military Education: DOD Needs to Develop Performance Goals and Metrics for Advanced Distributed Learning in Professional Military Education
GAO-04-873
Published: Jul 30, 2004. Publicly Released: Jul 30, 2004.
Skip to Highlights
Highlights
As part of its transformation to prepare the armed forces to meet current and future challenges, the Department of Defense (DOD) is expanding its use of advanced distributed learning (ADL) techniques in senior- and intermediate-level officer professional military education (PME). ADL instruction does not require an instructor's presence, and it facilitates the use of varied learning management systems. To date, the application of ADL has been targeted to nonresident students. To determine whether DOD uses a systematic process for evaluating the results of ADL application, GAO was asked to examine DOD's metrics for assessing program effectiveness, to compare DOD's criteria for converting courses to ADL with those of private-sector institutions, and to identify the challenges to ADL implementation.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status Sort descending |
|---|---|---|
| Department of Defense | The Secretary of Defense should direct the Under Secretary of Defense for Personnel and Readiness, in concert with the Joint Staff, service headquarters, and the PME schools, to promote the use of ADL technologies to capture data to provide knowledge about learning outcomes. |
Closed – Implemented
DOD concurred with this recommendation and the most recent DAMIS report and discussions with an OSD official responsible for ADL, indicates that a number of actions have been completed. First, a comprehensive analysis of evaluations of learning outcomes in ADL was completed, establishing a baseline for comparisons of ongoing and future ADL courses. Additionally, a database of 25 variables that influence the effectiveness of ADL will be updated and maintained as new information on the ADL is reported. Also, a Ph.D. level evaluation analyst to head an Evaluation Practices cell at the ADL Co-Laboratory has been hired to in part develop best practices for evaluating ADL programs. Finally, as a part of an ongoing effort to evaluate the value of ADL programs, the department issued on 03/24/2005 a Broad Agency Announcement to obtain competitive bids seeking technologies, methodologies and techniques for continuously collecting metrics online about a learner's performance. These bids have been reviewed and contracts awarded.
|
| Department of Defense | The Secretary of Defense should direct the Under Secretary of Defense for Personnel and Readiness, in concert with the Joint Staff, service headquarters, and the PME schools, to promote the development of specific performance effectiveness goals for ADL. |
Closed – Not Implemented
While DOD partially concurred with this recommendation, they did not indicate any corrective action has been taken and officially closed the recommendation as of 10/1/2004. DOD stated in the most recent DAMIS report that they support the use of specific performance effectiveness goals for professional military education but do not believe it appropriate to develop such goals for a specific delivery method such as ADL. They further stated that educational outcomes are based on common standards defined in the Professional Military Education Policy, and that ADL is one delivery system that will help them accomplish these outcomes. A discussion with an OSD official responsible for ADL stated that DOD's position has not changed and there is no plan to develop specific performance effectiveness goals for ADL. Based on this response and lack of DOD action, GAO does not consider it a reasonable expectation that DOD will implement this recommendation.
|
Full Report
Office of Public Affairs
Topics
Comparative analysisEducationEducation program evaluationEducational facilitiesEvaluation criteriaEvaluation methodsMilitary officersMilitary personnelMilitary trainingPerformance measuresPrivate sector practicesSchoolsKnowledge, skills and abilities