Effectiveness of DOD Development Test and Evaluation
PSAD-79-86: Published: Jun 25, 1979. Publicly Released: Jun 25, 1979.
- Full Report:
In 1978, GAO reviewed the military services' development test and evaluation (DT&E) of six weapons systems totaling an estimated $12 billion in development and procurement costs. Inadequate testing is principally due to the expense, the difficulty of staging and execution, the time consumed, and belief that negative test results may jeopardize or delay a program. To overcome the unenthusiastic attitude towards testing, the type of results expected from the testing must be defined. Possibly, a strong testing activity is the only means of accomplishing this objective. The Department of Defense spelled out testing procedures in a 1973 policy directive; when it is followed, it discloses problems and risks before production and verifies solutions.
As design and development phases progressed, the programs experienced cost growth, schedule slippages, and a reduction in performance levels. To remedy these effects, the responsible agencies, without formal assessment of risks and benefits, reduced the scope of, and hardware for, DT&E during critical tests; advanced programs into operational testing and production before completion of DT&E; and approved concurrent development and production; all without formal assessment of risks and benefits. In five of the six programs, planned tests for resolving technical uncertainties either were reduced or not performed, and final test reports either were unavailable or omitted the critical issues to be resolved before a given development phase could be considered complete. These practices could result in the systems' entering production before their performance capabilities are known. Judging from past experience, this is an unsatisfactory procedure because systems have emerged from testing with degraded performance or with additional costs incurred in raising performance levels to acceptable standards. Actions which would reduce development time and costs include the elimination or reduction of development tests of systems and subsystems, the combination of DT&E with initial operational test and evaluation (IOT&E), and a clear assessment of the risks involved in concurrent development and production.
Recommendation for Executive Action
Comments: Please call 202/512-6100 for additional information.
Recommendation: The Secretary of Defense should require that DT&E reports identify any deleted, reduced, or substituted tests, and state clearly the risks and implications of these actions. The Secretary should also require test plans to show the additional risks of joint DT&E/IOT&E test schedules and allow time to correct discovered deficiencies before proceeding to the next testing phase. Formal assessment of risks and benefits should be required before approval of concurrent development and production, and developers should be ordered to monitor sensitive system elements and high-risk areas closely, and provide schedule and funding reserves to resolve problems as they are identified.