Improved Performance Measures and Information Needed for Assessing Asset Visibility Initiatives
GAO-17-183: Published: Mar 16, 2017. Publicly Released: Mar 16, 2017.
What GAO Found
The Department of Defense (DOD) has identified performance measures for the eight selected asset visibility initiatives GAO reviewed, but these performance measures generally cannot be used to monitor progress. Specifically, GAO found that the measures for the eight initiatives reviewed did not generally include key attributes of successful performance measures. For example, for six initiatives there were no baseline and trend data associated with the measures. While DOD's 2014 and 2015 Strategy for Improving DOD Asset Visibility (Strategies) called for performance measures to be identified for the initiatives, the Strategies lacked complete direction on how to develop performance measures that would allow DOD to assess the progress of the initiatives toward their intended outcomes. GAO also found that after-action reports for the initiatives did not always include key information needed to determine the success of the initiatives in achieving the goals described in the Strategies. Without improved performance measures and information to support that progress has been made, DOD may not be able to monitor and show progress in improving asset visibility.
DOD has made progress and meets the criteria related to capacity and its corrective action plan but needs to take additional actions to monitor implementation and demonstrate progress to meet GAO's two remaining criteria for removal from the High Risk List, as shown in the figure. For the capacity criterion, in its draft update to the 2015 Strategy, DOD provides guidance on how to document cases where the funding for the initiatives is embedded within the overall program funding. Also, for the action plan criterion, DOD included matrixes in its 2015 Strategy to link ongoing initiatives to the Strategy's goals and objectives. DOD has also taken steps to monitor the status of initiatives. However, the performance measures for the selected initiatives that GAO reviewed generally cannot be used to track progress and are not consistently incorporated into reports to demonstrate results. Until these criteria are met, DOD will have limited ability to demonstrate sustained progress in improving asset visibility.
DOD's Progress in Meeting GAO's High-Risk Criteria for Asset Visibility
Why GAO Did This Study
GAO designated DOD's supply chain management as a high-risk area in 1990 and in February 2011 reported that limitations in asset visibility make it difficult to obtain timely and accurate information on assets that are present in a theater of operations. DOD defines asset visibility as the ability to provide timely and accurate information on the location, quantity, condition, movement, and status of items in its inventory. In 2015, GAO found that DOD had demonstrated leadership commitment and made considerable progress in addressing weaknesses in its supply chain management.
This report addresses the extent to which DOD has (1) identified performance measures that allow it to monitor the progress of selected asset visibility initiatives identified in its Strategies; and (2) addressed the five criteria—leadership commitment, capacity, corrective action plan, monitoring, and demonstrated progress—for removing asset visibility from the High Risk List. GAO reviewed documents associated with selected initiatives, surveyed DOD officials, and observed demonstrations.
What GAO Recommends
GAO recommends that DOD use key attributes of successful performance measures in refining measures in updates to the Strategy and incorporate information related to performance measures into after-action reports for the asset visibility initiatives. DOD partially concurred with both recommendations. The actions DOD proposed are positive steps, but GAO believes the recommendations should be fully implemented, as discussed in the report.
For more information, contact Zina Merritt at (202) 512-5257 or firstname.lastname@example.org.
Recommendations for Executive Action
Comments: DOD, in its comments on the draft report, partially concurred. DOD stated that it recognizes the need for performance measures to ensure the success of an asset visibility improvement effort but noted that the level of complexity and granularity of the metrics we suggest may not be suitable for all initiatives. DOD also stated that the purpose of the Strategy is to create a framework whereby the components can work collaboratively to coordinate and integrate department-wide efforts to improve asset visibility, not to provide complete direction on how to define, implement, and oversee these initiatives. Additionally, DOD stated that the next edition of the Strategy will encourage the adoption of our six key attributes for asset visibility initiatives to the extent appropriate, but will not mandate their use. As discussed in our report, use of the key attributes in measuring the performance of asset visibility initiatives would help DOD to better assess department-wide progress against the goals in its Strategy and clarify what additional steps need to be taken to enable decision makers to exercise effective oversight. Encouraging adoption of the key attributes, as DOD plans to do, is a positive step, but we continue to believe that DOD needs to use these key attributes to refine its performance measures to monitor the initiatives in the future.
Recommendation: The Secretary of Defense should direct the Assistant Secretary of Defense for Logistics and Materiel Readiness, in collaboration with the Director, Defense Logistics Agency; the Secretaries of the Army, Navy, and Air Force; the Commandant of the Marine Corps; the Commander of the United States Transportation Command; and the Chairman of the Joint Chiefs of Staff, to use the key attributes of successful performance measures--including clarity, measurable target, objectivity, reliability, baseline and trend data, and linkage--in refining the performance measures in subsequent updates to the Strategy to improve DOD's efforts to monitor asset visibility initiatives.
Agency Affected: Department of Defense
Comments: DOD, in its comments on the draft report, partially concurred. DOD stated that it is important to capture and review performance data prior to a component closing an asset visibility initiative, but that the Strategy after-action report is not intended to be used to evaluate the success of an asset visibility initiative or to determine if an initiative has met its intended objectives. According to DOD, documentation and information to support the evaluation of initiatives is defined by and executed in accordance with component-level policy and procedures. DOD agreed to update its Strategy to clarify the purpose and use of the after-action reports and to ensure that the Strategy specifies roles and responsibilities for evaluating and closing initiatives. DOD?s response, however, did not state whether and how these updates to the Strategy would result in more consistent incorporation of information relating to performance measures when closing initiatives in the future. As we noted previously in this report, according to the 2015 Strategy, the after-action report for closed initiatives should include information on the objectives met, problems or gaps resolved, and measures of success obtained. We believe our recommendation is consistent with this guidance. Without incorporating this information, DOD does not have assurance that closed initiatives have been fully assessed and have resulted in achieving the goals and objectives of the Strategies. Therefore, we continue to believe that full implementation of our recommendation is needed.
Recommendation: The Secretary of Defense should direct the Assistant Secretary of Defense for Logistics and Materiel Readiness, in collaboration with the Director, Defense Logistics Agency; the Secretaries of the Army, Navy, and Air Force; the Commandant of the Marine Corps; the Commander of the United States Transportation Command; and the Chairman of the Joint Chiefs of Staff, to incorporate into after-action reports information relating to performance measures for the asset visibility initiatives when evaluating and closing these initiatives to ensure that implemented initiatives will achieve the goals and objectives in the Strategies.
Agency Affected: Department of Defense