Weapons Acquisition Reform:

Actions Needed to Address Systems Engineering and Developmental Testing Challenges

GAO-11-806: Published: Sep 19, 2011. Publicly Released: Sep 19, 2011.

Additional Materials:

Contact:

Michael J. Sullivan
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

For the past 2 years, the Department of Defense (DOD) has been implementing the Weapon Systems Acquisition Reform Act (Reform Act) requirements for systems engineering and developmental testing. These activities are important to DOD's ability to control acquisition costs, which increased by $135 billion over the past 2 years for 98 major defense acquisition programs. GAO was asked to determine (1) DOD's progress in implementing the Reform Act's requirements and (2) whether there are challenges at the military service level that could affect their systems engineering and developmental testing activities. To do this, GAO analyzed implementation status documents, discussed developmental testing office concerns with current and former DOD officials, and analyzed military service workforce growth plans and test range funding data..

The new offices for systems engineering and developmental test and evaluation are continuing to make progress implementing Reform Act requirements. Since GAO's 2010 report on this topic, the Deputy Assistant Secretaries for Systems Engineering and Developmental Test and Evaluation have issued additional policies and guidance, assisted more weapons acquisition programs in the development of acquisition plans, and provided input to senior leaders at Defense Acquisition Board meetings. DOD also designated the Deputy Assistant Secretary for Developmental Test and Evaluation for concurrent service as the Director of the Test Resource Management Center. This was an optional Reform Act provision, which places oversight of testing resources and acquisition program developmental testing activities under one official. Despite these steps, the developmental test and evaluation office reports having difficulty covering its portfolio of about 250 defense acquisition programs with its current authorized staff of 63 people. Current and former testing officials believe the office needs more influence and resources to be effective, but they said thorough analysis has not been done to determine the appropriate office size. Further, according to the Deputy Assistant Secretary for Developmental Test and Evaluation, a statutory provision that designates the Test Resource Management Center as a field activity may limit his ability to achieve management and reporting efficiencies that could be obtained by combining or shifting resources between the two organizations. GAO has a matter for Congressional consideration to allow shifting resources between the Test Resource Management Center and the developmental test and evaluation office. The military services are facing workforce challenges that could curb systems engineering and developmental testing efforts, if not properly addressed. The services planned to increase their systems engineering and test and evaluation career fields by about 5,000 people (14 percent) and about 300 people (4 percent), respectively, between fiscal years 2009 and 2015 through hiring actions and converting contractor positions to government positions. The services have increased the systems engineering career field by about half of its projections and exceeded its planned growth for the test and evaluation career field through the end of fiscal year 2010. However, future growth may be difficult because of budget cuts and a clarification in DOD's insourcing approach, which may make civilian hiring more difficult. For example, the services now plan to hire about 800 fewer systems engineers by 2015 than they originally projected. Further, cuts to development test ranges' fiscal year 2012 budgets of nearly $1.2 billion (17 percent) over the next 5 years could offset some of the workforce gains already achieved. Currently, the services lack metrics that could be used to justify funding levels, effectively allocate funding cuts, make workforce decisions, or make difficult decisions related to mothballing, closing, or consolidating test capabilities, if future budget cuts are necessary. To the extent DOD cannot provide adequate systems engineering and developmental testing support to its weapon systems portfolio, the risks of executing the portfolio within cost and schedule are increased. GAO recommends that DOD assess the resources needed by the developmental test and evaluation office, develop a plan to implement the assessment, develop metrics to aid funding decisions, and report the effect budget cuts are having on the services' ability to meet program office needs. GAO also has a matter for congressional consideration. DOD concurred with two recommendations, and offered clarifying language, which GAO incorporated, on the other two recommendations for which DOD partially concurred.

Matter for Congressional Consideration

  1. Status: Closed - Not Implemented

    Comments: The assessment of developmental test resources that GAO recommended was completed in November 2011 and briefed to the Under Secretary for Acquisition, Technology, and Logistics. The organizational structure of the Test Resource Management Center and the Developmental Test and Evaluation office was modified based on this study. This reorganization took effect in January 2012 without requiring any statutory changes.

    Matter: Contingent upon the results of DOD's assessment, the Congress may wish to consider revising any applicable statutory provisions necessary to allow for DOD to combine or shift resources between the Test Resource Management Center and the office of the Deputy Assistant Secretary for Developmental Test and Evaluation.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In response to our recommendation, the Deputy Assistant Secretary of Defense for Developmental Test & Evaluation commissioned a study that reviewed the Test Resource Management Center and the Developmental Test & Evaluation office to identify efficiencies and develop alternative organizational structures. This study was finalized in November 2011 and briefed to the Under Secretary of Defense for Acquisition, Technology and Logistics. The study presented organizational and staffing options for a stand-alone Test Resource Management Center as well as a unified framework that included central management of the two organizations, but allowed for a variety of implementation approaches below the management level.

    Recommendation: The Secretary of Defense should assess the resources and influence needed by the developmental test and evaluation office to assist and oversee defense acquisition programs, including (1) the number of defense acquisition programs that can be supported by different developmental test and evaluation office staffing levels, including specifying the total number of personnel, the mix of government and contractor employees, and the number of senior executive service personnel needed for each of these staffing levels; (2) whether the Test Resource Management Center and the office of the Deputy Assistant Secretary for Developmental Test and Evaluation should be combined or resources shifted between organizations to more effectively support the activities of both organizations and if so, identify for Congress any statutory revisions that would be necessary; and (3) the proper reporting channel, taking into account the decision on whether or not to combine the organizations, the statutory oversight requirements, and the level of influence needed to oversee and assess program office developmental testing and service budgeting activities.

    Agency Affected: Department of Defense

  2. Status: Closed - Implemented

    Comments: GAO recommended (GAO-11-806) that the Secretary of Defense develop a plan to implement the results of a study which examined the resources needed for the developmental test and evaluation office. The study was completed in November 2011 and subsequently briefed to the Under Secretary of Defense for Acquisition, Technology, and Logistics. The study examined alternatives for the reorganization of the Test Resource Management Center as a stand-alone entity. It also developed a unified framework which included central management of the Test Resource Management Center and the Developmental Test and Evaluation Office, but allowed for a variety of implementation approaches below the management level. According to the Deputy Assistant Secretary of Defense for Developmental Test and Evaluation, the Under Secretary selected a hybrid approach based on the study. The reorganization reduced the staffing level of the Test Resource Management Center and added a senior executive position in the developmental test and evaluation office. The reorganization did not increase the action officer staffing level of the developmental test and evaluation office, however. The current reporting channels and organizational alignment of the two offices remains unchanged. This reorganization took effect in January 2012.

    Recommendation: The Secretary of Defense should develop a plan to implement the results of the assessment.

    Agency Affected: Department of Defense

  3. Status: Open

    Comments: In its March 2013 annual report, the Deputy Assistant Secretary for Developmental Test and Evaluation stated that the Test Resource Management Center was tasked with assembling an expert team to determine how test and evaluation resources can be reduced to provide adequate and robust testing. Specifically, the team looked at the areas of infrastructure requirements, capability gaps, funding alternatives, operation and support of test range alternatives, and other areas for improvement. This study identified several recommendations and findings. The 2014 Strategic Plan for DoD Test and Evaluation (T&E) Resources (that characterizes the funding, investments, and manpower trends for DoD T &E ranges and facilities) is scheduled for a summer publication. The Test Resource Management Center has systematically implemented a phased process for developing performance measures on Major Range and Test Facility Base (MRTFB) resources. The first phase (data collection) was completed for all of the 23 MRTFB locations and the data analysis is ongoing. An Integrated Product Team has been established to develop common performance standards across the MRTFB and implementation is planned to begin by the end of 2014. GAO will continue to monitor this recommendation.

    Recommendation: The Secretary of Defense, with input from the military services, should develop metrics to assess the Major Range and Test Facility Base (MRTFB) test capabilities (expanding to DOD non-MRTFB, and non-DOD government test facilities once an approved set of metrics are in place supporting the MRTFB), justify funding, and assist in making decisions on the right-sizing of personnel, how best to allocate funding, or make future decisions on whether to mothball, shut down, or consolidate test facilities. These efforts should be coordinated with the Test Resource Management Center.

    Agency Affected: Department of Defense

  4. Status: Closed - Implemented

    Comments: In the March 2012 joint annual report, the Deputy Assistant Secretary of Defense for Systems Engineering reported on the impact these budget cuts had on the systems engineering workforce. The report indicated that the impact of the cuts is not fully known, but the Deputy will continue to work closely with the military services to guide, oversee, and advocate for the systems engineering workforce. Regarding service resources, the report stated that the Army's objectives for growing the workforce were reduced and insourcing was suspended. In contrast, the Navy's fiscal year 2012 budget is sufficient to support the planned program. The Air Force's civilian personnel reductions are contingent upon implementation of the fiscal year 2012 budget. The Deputy reported that Air Force needs to continue targeted hiring to avoid potential experience gaps in the workforce.

    Recommendation: The Secretary of Defense, with input from the military services, should report the impact budget cuts reflected in the fiscal year 2012 budget, as well as the insourcing policy clarification, will have on their (1) total workforce (civilians, military, and contractors) that support both of these activities and (2) ability to meet program office systems engineering and developmental test and evaluation needs.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Oct 30, 2014

Oct 16, 2014

Oct 10, 2014

Oct 8, 2014

Sep 30, 2014

Sep 25, 2014

Sep 23, 2014

Sep 19, 2014

Sep 18, 2014

Looking for more? Browse all our products here