Information Technology Reform:

Agencies Need to Increase Their Use of Incremental Development Practices

GAO-16-469: Published: Aug 16, 2016. Publicly Released: Sep 15, 2016.

Additional Materials:

Contact:

David A. Powner
(202) 512-9286
pownerd@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

For fiscal year 2016, 22 agencies reported 64 percent of their software development projects would deliver useable functionality every 6 months on the Information Technology (IT) Dashboard, as required by the Office of Management and Budget (OMB). However, shortcomings with OMB's guidance—the lack of clarity regarding the types of projects where incremental development would not apply and how the status of these non-software projects should be reported—have affected the accuracy of the data on the IT Dashboard.

GAO reviewed 7 departments' software projects and found approximately half of these projects reported delivery of functionality every 6 months. However, there are significant differences in the data reported to GAO and on the IT Dashboard (see figure below) due to inconsistencies in reporting non-software projects, the timing of reporting, and a lack of support for reported delivery, which affects the accuracy of reported rates. Department officials also reported that management and organizational challenges and project complexity and uniqueness impact their ability to deliver incrementally. It is critical that departments continue to improve their use of incremental development to deliver functionality and reduce the risk that these projects will not meet cost, schedule, and performance goals.

Comparison of Software Development Projects' Percentage of Planned Delivery Every Six Months Reported on IT Dashboard and to GAO for Fiscal Year 2016

Comparison of Software Development Projects' Percentage of Planned Delivery Every Six Months Reported on IT Dashboard and to GAO for Fiscal Year 2016

ªDefense did not provide the requested information in time to verify the information reported to GAO.

Although OMB's requirement has been in place since June 2015, only three departments (Department of Commerce, Department of Homeland Security, and the Department of Transportation) had policies and processes to ensure that the chief information officer would certify IT investments are adequately implementing incremental development. Officials from three departments reported they were updating their existing policies to address certification, but had not yet finalized these efforts, and one department stated its current processes were sufficient.

Why GAO Did This Study

Federal agencies plan to spend more than $89 billion on IT in fiscal year 2017. Historically, long-term system development projects have often produced disappointing results. To help address these issues, OMB now requires agencies to deliver useable functionality every 6 months to reduce risk and deliver capabilities more quickly. GAO's objectives were to (1) describe the number of major IT investments primarily in development identified on the IT Dashboard as reporting the delivery of functionality every 6 months; (2) assess selected departments' and investments' delivery of incremental functionality and determine the factors affecting delivery rates; and (3) assess the quality and completeness of selected departments' plans to employ incremental development practices. GAO analyzed information on the IT Dashboard, analyzed reported project data and policies of 7 selected departments with the most investments primarily in development, and interviewed OMB and department officials.

What GAO Recommends

GAO is making 12 recommendations to 8 agencies to improve reporting of incremental data on the IT Dashboard and policies for CIO certification of adequate incremental development. Five departments agreed with our recommendations, the Department of Defense partially agreed with one and disagreed with another, OMB did not agree or disagree, and the Department of the Treasury did not comment on the recommendations. GAO continues to believe its recommendations are appropriate.

For more information, contact David A. Powner at (202) 512-9286 or pownerd@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: The Office of Management and Budget (OMB) has taken initial steps to implement our recommendation. Specifically, OMB's June 2016 annual capital planning guidance for fiscal year 2018 included instructions on what types of investments were required to adhere to incremental development requirements related to the delivery of usable functionality. The guidance stated that all software development projects are required to produce usable functionality at intervals of no more than six months. Further, all major development projects within investments are required to use modular/agile principles. However, OMB's guidance still lacks direction on how CIOs are to report the status of nonsoftware projects, as we recommended. In the absence of our recommended guidance clarification, OMB is at risk of agencies continuing to be unclear about how nonsoftware development investment data are to be reported on the Dashboard, increasing the risk that data on the IT Dashboard will not always be accurate. We will continue to evaluate OMB's progress in clarifying its guidance and considering a change to provide more detailed guidance related to the reporting of nonsoftware development investment data.

    Recommendation: In order to improve the accuracy of IT Dashboard incremental development data, the Director of OMB should direct the Federal Chief Information Officer (CIO) to clarify existing guidance regarding what IT investments are and are not subject to requirements on the use of incremental development and how CIOs should report the status of projects that are not subject to these requirements.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  2. Status: Open

    Comments: The Department of Homeland Security (DHS) concurred with our recommendation and stated that the Enterprise Business Management Office within the Office of the Chief Information Officer will validate each investment reported on the Dashboard and work with program officials to ensure they appropriately update the data for the IT Dashboard. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Homeland Security

  3. Status: Open

    Comments: The Department of Education (Education) concurred with our recommendation and stated that the department will ensure that the data is kept current using their IT portfolio management process. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Education

  4. Status: Open

    Comments: The Department of Commerce (Commerce) concurred with our recommendation and stated that these changes would be incorporated into the department?s Dashboard reporting. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Commerce

  5. Status: Open

    Comments: The Department of Defense (Defense) partially concurred with our recommendation and stated that the department is taking action to update the Dashboard data as appropriate. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Defense

  6. Status: Open

    Comments: The Department of Health and Human Services (HHS) concurred with our recommendation. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Health and Human Services

  7. Status: Open

    Comments: The Department of Transportation (Transportation) concurred with our recommendation and stated the department was committed to ensuring the information on the IT Dashboard reflects up to date information. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of Transportation

  8. Status: Open

    Comments: The Department of the Treasury (Treasury) did not comment on our recommendation. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.

    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency Affected: Department of the Treasury

  9. Status: Open

    Comments: The Department of Education (Education) concurred with our recommendation to establish a departmentwide certification policy. Education officials reported in March 2017 that the department will complete changes to its guidance by November 2017. However, until this guidance is finalized, Education will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate Education's progress in implementing this recommendation.

    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency Affected: Department of Education

  10. Status: Open

    Comments: The Department of Defense (Defense) did not concur with our recommendation, stating that its existing guidance was adequate in this area. However, in August 2016, Defense issued its fiscal year 2018 budget submission guidance which required each component CIO to certify that IT investments were adequately implementing incremental development. The component CIOs were to document the certification in a statement of compliance memorandum, using their agency's letterhead, and submit the memorandum to the Defense CIO. Defense officials report that this same guidance will be added to the Financial Management Regulations during summer 2017. Until this annual guidance has been updated and incorporated into the department's standing policies, Defense is at risk of overlooking this requirement in subsequent years. We will continue to evaluate Defense's progress in implementing this recommendation.

    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency Affected: Department of Defense

  11. Status: Open

    Comments: The Department of Health and Human Services (HHS) concurred with our recommendation to establish a departmentwide certification policy. However, HHS officials reported in April 2017 that they did not have a timeframe for when the department's new certification guidance would be completed. Until this guidance is finalized, HHS will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate HHS's progress in implementing this recommendation.

    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency Affected: Department of Health and Human Services

  12. Status: Open

    Comments: The Department of the Treasury (Treasury) did not comment on our recommendations. Further, Treasury officials reported in March 2017 that it had no plans to revise its policies, as we recommended. Until the department establishes a CIO certification policy, Treasury will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate Treasury's progress in implementing this recommendation.

    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency Affected: Department of the Treasury

 

Explore the full database of GAO's Open Recommendations »

Sep 18, 2017

Sep 6, 2017

Jul 13, 2017

Jun 21, 2017

Jun 13, 2017

May 18, 2017

May 15, 2017

Apr 11, 2017

Looking for more? Browse all our products here