Recommendation: We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, develop a method to regularly collect feedback from SWOs across the fleet, such as in a survey, regarding the quality of their classroom, simulator, and at-sea training on Division Officer performance; and evaluates trends in the feedback received for the purpose of improving SWO training. (Recommendation 1)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: The Navy concurred with our recommendation. In March 2020, the Navy provided an estimated implementation date of March 2023, noting that it was considering a fleet-wide survey, timed for a later date when more Surface Warfare Officers have completed new training courses and implemented their training. In addition, the Navy listed other means it employs to collect feedback, such as student surveys at the end of training courses, leadership visits and conferences, and Commanding Officer updates. Our emphasis on collection of fleet-wide feedback from all Surface Warfare Officers and trend analysis remains critical to help the Navy understand the value of its training programs at various career stages and in the diverse operating environments across the fleet.

Recommendation: We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, routinely conduct regular Officer of the Deck competency assessments using samples of sufficient size and using selection methods to gauge the level of fleet-wide ship-driving proficiency trends following the implementation of the planned ship-driving training programs. (Recommendation 2)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: The Navy concurred with our recommendation. In March 2020, the Navy provided an estimated implementation date of March 2023, noting that it was conducting the planned fleet-wide Officer of the Deck competency checks in 2020, and that it intends to use a system of ten career milestone assessments for future performance measurement. The Navy stated that it may or may not hold subsequent rounds of the Officer of the Deck competency assessments depending on performance indicated in other career milestone assessments. In our report we identified the importance of continuing the current Officer of the Deck competency assessments through at least 2024 because that is when new officers that complete the full set of new initial ship-driving training courses will be eligible for assessment. The Navy used the Officer of the Deck competency assessments in 2018 to establish a performance baseline, and we believe that the Navy should apply the same standard to measure performance changes for Surface Warfare Officers that complete new training courses moving forward.

Recommendation: We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, provide Commanding Officers with standard criteria to inform their evaluation of candidates for their Officer of the Deck qualification and incorporates these criteria into surface fleet guidance. (Recommendation 3)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: The Navy concurred with our recommendation. In March 2020, the Navy provided an estimated implementation date of March 2023. However, in its official comments on the report and in subsequent correspondence, the Navy indicated that its existing policies already meet the intent of the recommendation. Specifically, the Navy stated that its Officer of the Deck Underway Personnel Qualification Standards provide standard evaluation criteria for Officer of the Deck qualification. In our report, we noted that while the Personnel Qualification Standards provide a common list of required experiences, they do not provide a common understanding of proficiency in completing these experiences. Instead, proficiency determination is left to the discretion of the ship's Commanding Officer, which has led to wide variation in ship-driving proficiency across the fleet. Therefore, we continue to believe that the Navy should provide Commanding Officers with standard criteria to inform their evaluation of candidates for their Officer of the Deck qualification and incorporate these criteria into surface fleet guidance.

Recommendation: We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, develop a plan to analyze and use Mariner Skills Logbook information to inform decision-making. (Recommendation 4)

Agency: Department of Defense: Department of the Navy
Status: Open

Comments: The Navy concurred with our recommendation. In March 2020, the Navy provided an estimated implementation date of March 2023. In official comments on the report and in subsequent correspondence the Navy stated that its Surface Warfare Career Manual establishes guidance for the implementation and use of the Mariner Skills Logbook, and that the logbook will contribute information to allow proficiency trend analysis over time. However, while the Surface Warfare Career Manual identifies the offices responsible for logbook activities, it does not include a specific plan for the use of logbook data to analyze proficiency trends over time or to benefit individual officers. Our emphasis that the Navy develop a plan to analyze and use Surface Warfare Mariner Skills Logbook data to aid decision-making remains valid, and when implemented should assist the Navy in determining the relationship between SWO experience and ship-driving proficiency.

Recommendation: In order to better integrate virtual training devices into operational training, the Secretary of Defense should direct the Secretary of the Army to specify in Army guidance for developing virtual training device requirements that training developers consider and document the time available to train with the devices and intended usage rates to achieve training tasks and proficiency goals during operational training.

Agency: Department of Defense
Status: Open

Comments: The Department of Defense (DOD) concurred with this recommendation. As of December 2019, the Army had taken some steps to improve its guidance, as GAO recommended in August 2016, but did not plan to fully address the recommendation until 2021. Officials stated that the Army established target usage rates for existing virtual training devices and issued guidance and tracking tools for recording device usage. However, the Army had not modified the guidance, cited in GAO's August 2016 report, to require that training developers consider the amount of time available to train with or expected usage rates of new virtual training devices. According to Army officials, they will implement GAO's recommendation in a planned update to guidance on the justification and validation of new virtual training devices scheduled for 2021. By updating this guidance, the Army will have the information it requires to evaluate the amount of virtual training capabilities needed to achieve training tasks and proficiency goals during operational training.

Recommendation: To improve decision makers' abilities to make fully informed decisions concerning whether training requirements can be met with live and simulation-based training and determine optimal mixes of live and simulation-based training, the Secretary of Defense should direct the Secretary of the Army and the Commandant of the Marine Corps to develop outcome-oriented performance metrics that can be used to assess the impact of simulation-based training on improving the performance or proficiency of servicemembers and units.

Agency: Department of Defense
Status: Open

Comments: As of 18 Aug 2014, the Army and Marine Corps actions for this recommendation are currently ongoing and the recommendation status currently remains open. On 14 June 2014, the DOD Inspector General reported in the Defense Audit Management Information System that "the Office of the Deputy Assistant Secretary of Defense(Readiness) developed a decision algorithm to determine which military tasks could be taught virtually and which military tasks should only be taught in classroom or field environments (i.e., live). The algorithm was provided to the Services for peer-review and possible implementation. The Army is reviewing its progressive training models through a process called Training Summit IV (TS IV). These models establish how virtual and constructive based training is integrated with live training to optimize training readiness. The TS IV will include training model review by proponent schools, as well as a cross-section of unit commanders and leaders. This effort will be completed in Fiscal Year (FY) 2014 and presented for validation and G-3/5/7 approval at the Army Training General Officer Steering Committee in November 2014. Also, the Marine Corps initiated a request for an internal servicewide study of existing and potential approaches to this topic (4th Quarter FY 2013). The initial focus is in determining how metrics can be better used to assess the impact of simulation based on meeting Marine Corps Training Standards. Furthermore, a targeted study began in the 1st Quarter FY 2014 and is focused initially on enhancing the methodology for assessing individual based simulators against Training and Readiness (T&R) Standards. In FY 2015, the study results will shape policy on how future T&R manuals will identify the appropriateness of simulators and simulations for training."

Recommendation: To improve decision makers' abilities to make fully informed decisions concerning whether training requirements can be met with live and simulation-based training and determine optimal mixes of live and simulation-based training, the Secretary of Defense should direct the Secretary of the Army and the Commandant of the Marine Corps to develop a methodology--to include identifying the costs that should be included and how these costs should be captured--for comparing the costs associated with the use of live and simulation-based training.

Agency: Department of Defense
Status: Open

Comments: As of 18 Aug 2014, the Army and Marine Corps actions for this recommendation are currently ongoing and the recommendation status currently remains open. On 14 June 2014, the DOD Inspector General reported in the Defense Audit Management Information System that "the Office of the Deputy Assistant Secretary of Defense (Readiness) has coordinated with the Army and Marine Corps to identify standard approaches to capture costs and cost benefit analysis that could be used DoD-wide. The Army has undertaken a "cost of training" analysis that is an on-going action to determine cost of readiness and/or training. One area of concentration is to look at the "Other Burdened Resources Required for Training Readiness." This area is further broken down into two areas: Investment/Modernization and Installation Services. The Investment/Modernization area will look at Non-System Training Aids, Devices, Simulators and Simulations while Installation Services will look at Post Deployment Software Support. In addition, the Army is gathering data to validate an existing model developed by the Simulations to Mission Command Interoperability Director (Program Executive Office for Simulation, Training, and Instrumentation) for the Value of Simulation Study consisting of five phases: Phase one focused on development of a working methodology to assess both quantitative and qualitative value of simulations used to support collective training (completed). Phase two is currently gathering data for model validation. Phase three will be an expansion to other simulation capabilities. Phase four is data gathering and validation. Phase five is expanded testing/methodology use case study/validation for return on investment use. The Marine Corps established a study, described in response to Recommendation 1, which will evaluate and propose the initial cost factors not currently captured during Programming yet would be relevant in determining the appropriate mix of live and simulated training. The initial results are expected in FY 2015."