Department of Defense (DOD) weapon programs often experience significant cost and schedule problems because they are allowed to start with too many technical unknowns and not enough knowledge about the development and production risks they entail. GAO was asked to review the department's Analysis of Alternatives (AOA) process--a key first step in the acquisition process intended to assess the operational effectiveness, costs, and risks of alternative weapon system solutions for addressing a validated warfighting need. This report (1) examines whether AOAs have been effective in identifying the most promising options and providing a sound rationale for weapon program initiation, (2) determines what factors have affected the scope and quality of AOAs, and (3) assesses whether recent DOD policy changes will enhance the effectiveness of AOAs. To meet these objectives, GAO efforts included collecting information on AOAs from 32 major defense acquisition programs, reviewing guidance and other documents, and interviewing subject matter experts.

Although an AOA is just one of several inputs required to initiate a weapon system program, a robust AOA can be a key element to ensure that new programs have a sound, executable business case. Many of the AOAs that GAO reviewed did not effectively consider a broad range of alternatives for addressing a warfighting need or assess technical and other risks associated with each alternative. For example, the AOA for the Future Combat System program, one of DOD's large and most complex development efforts, analyzed the operational performance and cost of its alternatives but failed to compare the technical feasibility and risks, assuming that the technologies would perform as forecasted. Without a sufficient comparison of alternatives and focus on technical and other risks, AOAs may identify solutions that are not feasible and decision makers may approve programs based on limited knowledge. While many factors can affect cost and schedule outcomes, we found that programs that had a limited assessment of alternatives tended to have poorer outcomes than those that had more robust AOAs. The narrow scope and limited risk analyses in AOAs can be attributed in part to program sponsors choosing a solution too early in the process, the compressed timeframes that AOAs are conducted under, and the lack of guidance for conducting AOAs. While AOAs are supposed to provide a reliable and objective assessment of viable weapon solutions, we found that service sponsors sometimes identify a preferred solution or a narrow range of solutions early on, before an AOA is conducted. The timing of AOAs has also been problematic. Some AOAs are conducted under compressed timeframes in order to meet a planned milestone or weapon system fielding date and are conducted concurrently with other key activities required to become a program of record. This can short-change a comprehensive assessment of risks and preclude effective cost, schedule, and performance trade offs from taking place prior to beginning development. Furthermore, while DOD has an opportunity to influence the scope and quality of AOAs, it has not always provided guidance for conducting individual AOAs. Recognizing the need for more discipline in weapon systems acquisition, DOD recently revised its overall acquisition and requirements policies. If implemented properly, the revised policies could provide a better foundation for planning and starting new programs with sound, knowledge-based business cases. Included in the revised acquisition policy are several mechanisms to improve the AOA process. For example, the policy revisions should help ensure that DOD direction is provided before AOAs are started and that they are conducted at an early point in the acquisition process where their results can inform decisions affecting program initiation. While these policy changes are promising, DOD must ensure that they are consistently implemented and reflected in decisions on individual programs. Furthermore, more specific criteria and guidance for how AOAs should be conducted may need to be developed to ensure they meet their intended objectives and provide an in-depth assessment of alternatives.