Leading Practices: Agency Acquisition Policies Could Better Implement Key Product Development Principles
Fast Facts
Together, the Departments of Defense and Homeland Security and NASA invest hundreds of billions of dollars each year to develop and deliver a wide range of systems, from stealth jets to lunar rovers.
Leading companies rely on certain principles to ensure product development success. But we found that these federal agencies aren't generally required to meet such principles. For example, DOD's policies don't require acquisition staff to consider dropping less urgent requirements to stay on schedule—a key development principle.
We recommended that the agencies create or update guidance to reflect key principles of product development.
Highlights
Why This Matters
Each year, the Departments of Defense (DOD) and Homeland Security (DHS) and the National Aeronautics and Space Administration (NASA) together invest hundreds of billions of dollars to buy stealth jets, cutters and ships, and lunar rovers, among other things, all with complex software. However, GAO’s annual reviews of these agencies’ major acquisitions find they often take longer and spend more money than planned to deliver capabilities to users.
Key Takeaways
Leading companies take a disciplined approach to develop innovative products that satisfy their customers’ needs, and to deliver them to market on time and within planned costs. The 13 leading companies GAO interviewed perform similar activities when developing new products, such as iterative design in hardware and software development. These activities in the development process align with the four key principles that help project teams deliver innovative products to market quickly and efficiently (see figure). GAO found that the department-wide acquisition policies of DOD, DHS, and NASA implement some key product development principles. But, they have yet to fully implement others. This gap limits agencies from ensuring a consistent approach to developing and delivering products with speed and efficiency.
Leading Companies Use Four Key Principles for Product Development
For example, leading companies focus on designing a minimum marketable product—one with the minimum capabilities needed for customers to recognize value. Leading companies also prioritize a project’s schedule: they release the features most critical to the customer and will off-ramp non-critical product features—an industry term for removing them from the current release—as necessary, in order to maintain schedule. Leading companies have mechanisms to solicit and implement feedback from customers early and often throughout development to ensure the product is relevant to customer needs, among other things.
Primary DOD, DHS, and NASA acquisition policies incorporate many aspects of the four key principles, to varying degrees. However, agencies miss opportunities for positive outcomes by not addressing some sub-principles in their policies.
- DOD’s policies do not require all programs to consider off-ramping non-critical capabilities in order to achieve schedule, hindering programs’ best chance of maintaining time frames.
- DHS’s policies do not require all programs to utilize modern design tools during hardware and software development, limiting consistent opportunities for programs to successfully improve revisions to the design.
- NASA’s policies do not include mechanisms for programs to obtain and utilize product feedback from stakeholders or end users—such as astronauts using spacecraft or the science community benefiting from NASA projects—in order to identify challenges or new features to include in subsequent projects.
GAO previously found that other factors beyond policies can affect agency outcomes, including structural differences between government and private industry. However, GAO’s prior work also demonstrates that key principles from private industry can be thoughtfully applied to government acquisition to improve outcomes, even with the different cultures and incentives.
How GAO Did This Study
This report examines principles that guide leading companies’ product development efforts and the extent to which primary, department-wide DOD, DHS, and NASA acquisition policies reflect the companies’ key principles and result in similar outcomes. GAO identified the 13 leading product development companies based on rankings in well-recognized lists; interviewed company representatives; analyzed department-wide acquisition policies from DOD, DHS, and NASA; and interviewed agency officials. The report is the first product in a planned body of work. In future work, GAO will explore how government agencies can apply some of the key principles outlined in this report.
Recommendations
GAO is making nine recommendations to DOD, DHS, and NASA to update acquisition policies to fully implement key principles of product development. All three agencies concurred with our recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status Sort descending |
|---|---|---|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment update DOD acquisition policies to fully implement the following principle throughout development: incorporating feedback from users of initial capabilities (Recommendation 4). |
Open
DOD concurred with our recommendation. In August 2022, the Under Secretary of Defense for Acquisition and Sustainment noted it would give full consideration to the further application of key product development principles when it next formally updates its overarching acquisition policy and the other individual pathways and functional acquisition policies, anticipated for completion by June 2024. We will continue to track DOD's progress in this area.
|
| National Aeronautics and Space Administration | The NASA Administrator should ensure that the NASA Office of the Chief Engineer update NASA acquisition policies to fully implement the following principle throughout development: incorporating feedback from users of initial capabilities (Recommendation 9). |
Open
NASA concurred with our recommendation. In September 2023, NASA reiterated its earlier position that that its policies require feedback to address challenges and inform designs for future programs and projects. NASA referred to policies we previously reviewed for our report but did not provide evidence of updates that fully implement the leading principle of incorporating feedback from users of initial capabilities. While NASA's policies require programs to examine and incorporate lessons learned into the planning process, they do not require programs to incorporate feedback from users, in particular, to inform improvements or changes in capabilities for subsequent projects. We will continue to track NASA's progress in this area.
|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment update DOD acquisition policies to fully implement the following principle throughout development: attaining a sound business case (Recommendation 1). |
Open
DOD concurred with our recommendation. In August 2022, the Under Secretary of Defense for Acquisition and Sustainment noted it would give full consideration to the further application of key product development principles when it next formally updates its overarching acquisition policy and the other individual pathways and functional acquisition policies, anticipated for completion by June 2024. We will continue to track DOD's progress in this area.
|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment update DOD acquisition policies to fully implement the following principle throughout development: applying iterative design approaches (Recommendation 2). |
Open
DOD concurred with our recommendation. In August 2022, the Under Secretary of Defense for Acquisition and Sustainment noted it would give full consideration to the further application of key product development principles when it next formally updates its overarching acquisition policy and the other individual pathways and functional acquisition policies, anticipated for completion by June 2024. We will continue to track DOD's progress in this area.
|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment update DOD acquisition policies to fully implement the following principle throughout development: off-ramping capabilities when needed to maintain schedule (Recommendation 3). |
Open
DOD concurred with our recommendation. In August 2022, the Under Secretary of Defense for Acquisition and Sustainment noted it would give full consideration to the further application of key product development principles when it next formally updates its overarching acquisition policy and the other individual pathways and functional acquisition policies, anticipated for completion by June 2024. We will continue to track DOD's progress in this area.
|
| Department of Homeland Security | The Secretary of Homeland Security should ensure that the DHS Undersecretary for Management update DHS acquisition policies to fully implement the following principle throughout development: attaining a sound business case (Recommendation 5). |
Closed – Implemented
DHS concurred with our recommendation. We found that DHS policies did not implement the leading sub-principle of terminating product development promptly if the product no longer has a sound business case. In January 2023, DHS updated Instruction 102-01-001 Acquisition Management to include guidance for communicating and documenting the decision to cancel or terminate an acquisition program because of lack of funding, extreme schedule breaches, changes in mission need, or inability to meet the performance requirements, for example. The additional guidance clarifies when to consider a program for termination and should help DHS avoid continuing to invest in acquisition programs that no longer have a sound business case.
|
| Department of Homeland Security | The Secretary of Homeland Security should ensure that the DHS Undersecretary for Management update DHS acquisition policies to fully implement the following principle throughout development: off-ramping capabilities when needed to maintain schedule. (Recommendation 7). |
Closed – Implemented
DHS concurred with our recommendation. In January 2023, DHS updated Instruction 102-01-001 Acquisition Management to provide for identifying trade-offs to maintain affordability and considering trade-offs to implement acquisition programs within the approved cost, schedule, and performance parameters of the acquisition program baseline. This additional guidance should help DHS ensure acquisition programs provide users with needed capabilities within cost and schedule goals.
|
| Department of Homeland Security | The Secretary of Homeland Security should ensure that the DHS Undersecretary for Management update DHS acquisition policies to fully implement the following principle throughout development: applying iterative design approaches (Recommendation 6). |
Open – Partially Addressed
DHS concurred with our recommendation. In January 2023, DHS updated Instruction 102-01-001 Acquisition Management to include guidance for program managers to use modern engineering design tools, which is responsive to the recommendation. However, the updated Instruction also refers to incremental development, rather than iterative design or development, as the desired methodology for program managers to use. We have found that these two terms carry different definitions among leading companies and in government acquisition programs. Iterative development lends itself to ongoing user input throughout a design effort that culminates with identifying and agreeing upon requirements that constitute a minimum viable product. Alternatively, incremental development revolves around setting detailed performance requirements at the start of the design effort and then delivering different increments of capability to meet those requirements. We will continue to track DHS's implementation of policy changes in this area.
|
| National Aeronautics and Space Administration | The NASA Administrator should ensure that the NASA Office of the Chief Engineer update NASA acquisition policies to fully implement the following principle throughout development: applying iterative design approaches (Recommendation 8). |
Open – Partially Addressed
NASA concurred with our recommendation. In September 2023, NASA reiterated its earlier position that a complete, iterative design approach is currently documented in its policy. NASA referred to the same policies we reviewed for our report, as well as the NASA Systems Modeling Handbook for Systems Engineering, which NASA released more recently, in November 2022. The handbook shows how system modeling using the Systems Modeling Language can be integrated into NASA's systems engineering processes, which applies to systems that include hardware and software. The release of this handbook partially addresses the recommendation by partially implementing the leading sub-principle of using modern design tools during both hardware and software development. To be considered policy, however, the relevant policy document(s) would need to indicate the handbook should be used for systems engineering. In addition, NASA's policies have yet to be updated to fully implement the leading sub-principle of using iterative design and testing to identify a minimum marketable product that can be followed by successive updates. NASA noted that the agency operates in a high-risk environment with a portfolio that consists primarily of unique, one-of-a-kind spaceflight projects. However, leading companies we interviewed also operate in similar high-risk environments with unique products and successfully use feedback from initial capabilities to inform subsequent efforts. We will continue to monitor NASA's progress in this area.
|