Information Technology Reform:

Agencies Need to Increase Their Use of Incremental Development Practices

GAO-16-469: Published: Aug 16, 2016. Publicly Released: Sep 15, 2016.

Additional Materials:

Contact:

David A. Powner
(202) 512-9286
pownerd@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

For fiscal year 2016, 22 agencies reported 64 percent of their software development projects would deliver useable functionality every 6 months on the Information Technology (IT) Dashboard, as required by the Office of Management and Budget (OMB). However, shortcomings with OMB's guidance—the lack of clarity regarding the types of projects where incremental development would not apply and how the status of these non-software projects should be reported—have affected the accuracy of the data on the IT Dashboard.

GAO reviewed 7 departments' software projects and found approximately half of these projects reported delivery of functionality every 6 months. However, there are significant differences in the data reported to GAO and on the IT Dashboard (see figure below) due to inconsistencies in reporting non-software projects, the timing of reporting, and a lack of support for reported delivery, which affects the accuracy of reported rates. Department officials also reported that management and organizational challenges and project complexity and uniqueness impact their ability to deliver incrementally. It is critical that departments continue to improve their use of incremental development to deliver functionality and reduce the risk that these projects will not meet cost, schedule, and performance goals.

Comparison of Software Development Projects' Percentage of Planned Delivery Every Six Months Reported on IT Dashboard and to GAO for Fiscal Year 2016

Comparison of Software Development Projects' Percentage of Planned Delivery Every Six Months Reported on IT Dashboard and to GAO for Fiscal Year 2016

ªDefense did not provide the requested information in time to verify the information reported to GAO.

Although OMB's requirement has been in place since June 2015, only three departments (Department of Commerce, Department of Homeland Security, and the Department of Transportation) had policies and processes to ensure that the chief information officer would certify IT investments are adequately implementing incremental development. Officials from three departments reported they were updating their existing policies to address certification, but had not yet finalized these efforts, and one department stated its current processes were sufficient.

Why GAO Did This Study

Federal agencies plan to spend more than $89 billion on IT in fiscal year 2017. Historically, long-term system development projects have often produced disappointing results. To help address these issues, OMB now requires agencies to deliver useable functionality every 6 months to reduce risk and deliver capabilities more quickly. GAO's objectives were to (1) describe the number of major IT investments primarily in development identified on the IT Dashboard as reporting the delivery of functionality every 6 months; (2) assess selected departments' and investments' delivery of incremental functionality and determine the factors affecting delivery rates; and (3) assess the quality and completeness of selected departments' plans to employ incremental development practices. GAO analyzed information on the IT Dashboard, analyzed reported project data and policies of 7 selected departments with the most investments primarily in development, and interviewed OMB and department officials.

What GAO Recommends

GAO is making 12 recommendations to 8 agencies to improve reporting of incremental data on the IT Dashboard and policies for CIO certification of adequate incremental development. Five departments agreed with our recommendations, the Department of Defense partially agreed with one and disagreed with another, OMB did not agree or disagree, and the Department of the Treasury did not comment on the recommendations. GAO continues to believe its recommendations are appropriate.

For more information, contact David A. Powner at (202) 512-9286 or pownerd@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: The Office of Management and Budget (OMB) has taken initial steps to implement our recommendation. Specifically, OMB's August 2017 annual capital planning guidance for fiscal year 2019 included instructions on what types of investments were required to adhere to incremental development requirements related to the delivery of usable functionality and how CIOs are to report the status of nonsoftware projects. Specifically, the guidance stated that all software development projects must produce usable functionality at intervals of no more than six months and projects that do not involve software development are not required to leverage iterative/agile methodologies. In addition, the guidance noted that all projects within investments are required to use modular development principles. Lastly, the guidance included language in the requirement to report the project's software development life cycle methodology that identified which methodologies were considered software or systems development and also included an option which allowed agencies to indicate the project was not a software development or systems development project. By implementing our recommendation, OMB has helped ensure agencies more accurately report the status of nonsoftware development investment data on the Dashboard.

    Recommendation: In order to improve the accuracy of IT Dashboard incremental development data, the Director of OMB should direct the Federal Chief Information Officer (CIO) to clarify existing guidance regarding what IT investments are and are not subject to requirements on the use of incremental development and how CIOs should report the status of projects that are not subject to these requirements.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  2. Status: Closed - Implemented

    Comments: The Department of Homeland Security (DHS) has taken steps to address our recommendation. In April 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, DHS has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Homeland Security

  3. Status: Closed - Implemented

    Comments: The Department of Education (Education) has taken steps to address our recommendation. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Education has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Education

  4. Status: Closed - Implemented

    Comments: The Department of Commerce (Commerce) has taken steps to address our recommendation. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Commerce has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Commerce

  5. Status: Closed - Implemented

    Comments: The Department of Defense (Defense) partially concurred with our recommendation. In its written response to our draft report, the department stated that it was taking action to update IT Dashboard data as appropriate, but the department asserted that it considered other factors and information from key department stakeholders in rating investments on the IT Dashboard. In addition, the department stated that it does not require that each investment be developed incrementally and provide capability every 6 months. We reported that there was a significant discrepancy in the incremental data reported on the Dashboard versus the data reported to us, noting that on the Dashboard, Defense reported that only 8 percent of its projects delivered functionality in fiscal year 2016 while the projects that we reviewed reported a 63 percent delivery rate to us-a 55 percentage point difference. We also noted that OMB's guidance, clearly states that all projects, regardless of whether they use incremental development principles, must produce usable functionality at least every 6 months. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Defense has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Defense

  6. Status: Closed - Implemented

    Comments: The Department of Health and Human Services (HHS) has taken steps to address our recommendation. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, HHS has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Health and Human Services

  7. Status: Closed - Implemented

    Comments: The Department of Transportation (Transportation) has taken steps to address our recommendation. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Transportation has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Transportation

  8. Status: Open

    Comments: The Department of the Treasury (Treasury) did not comment on our recommendation. In June 2018, a review of the IT Dashboard found that while the department had updated 55 out of 67 of its major IT investment projects to address the key areas identified in our report, 12 projects that were currently in-progress still lacked information regarding whether the project is a software development project and 11 projects did not provide information on the status of the project's delivery of incremental functionality. Until Treasury implements our recommendation, the department may be at risk that OMB and other key stakeholders will not have the most accurate and current information about the department's investments in order to make decisions. We will continue to evaluate Treasury's progress in implementing this recommendation.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of the Treasury

  9. Status: Open

    Comments: The Department of Education (Education) concurred with our recommendation to establish a departmentwide certification policy. Education officials reported in June 2018 that the department will complete changes to its guidance by October 2018. However, until this guidance is finalized, Education will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate Education's progress in implementing this recommendation.

    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency Affected: Department of Education

  10. Status: Closed - Implemented

    Comments: The Department of Defense (Defense) did not concur with our recommendation. In its written response to our draft report, the department stated that its existing acquisition system includes policy and processes defined in Defense Directive 5000.01 and Defense Instruction 5000.02 regarding the use of incremental development, which was documented in the department's approved FITARA implementation plan. We reported that OMB's guidance clearly states that each department should define processes and policies to ensure that the CIO certifies the use of adequate incremental development for its IT investments, and that both of the department's directives were issued prior to OMB's guidance. In November 2017, Defense issued its financial management regulation 7000.14-R, Volume 2B, Budget Formulation and Presentation (Chapters 4-19), which requires each component CIO to certify that IT investments were adequately implementing incremental development. In addition, the guidance states that the component CIOs are to document the certification in a memorandum using their agency's letterhead and include explanations for any investments which do not conform to department budget guidance. These memos will be submitted to the Defense CIO on an annual basis. By implementing our recommendation, Defense will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.

    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency Affected: Department of Defense

  11. Status: Open

    Comments: The Department of Health and Human Services (HHS) concurred with our recommendation to establish a departmentwide certification policy. However, HHS officials reported in June 2018 that they did not have a timeframe for when the department's new certification guidance would be completed. Until this guidance is finalized, HHS will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate HHS's progress in implementing this recommendation.

    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency Affected: Department of Health and Human Services

  12. Status: Open

    Comments: The Department of the Treasury (Treasury) did not comment on our recommendations. As of June 2018, Treasury officials reported that they were still working to assess our recommendation. Until the department establishes a CIO certification policy, Treasury will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate Treasury's progress in implementing this recommendation.

    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency Affected: Department of the Treasury

 

Explore the full database of GAO's Open Recommendations »

Aug 2, 2018

Jun 13, 2018

May 24, 2018

May 23, 2018

May 22, 2018

Mar 14, 2018

Jan 30, 2018

Jan 10, 2018

Nov 21, 2017

Looking for more? Browse all our products here