Recommendation: The Commissioner of the Social Security Administration should ensure the Office of Acquisition and Grants, in consultation with the Chief Information Officer, revises relevant guidance to identify the role of contracting officials in the Agile software development process.

Agency: Social Security Administration
Status: Open

Comments: The Social Security Administration agreed with the recommendation but has not yet taken actions to implement it.

Recommendation: The Secretary should ensure that the Director of Strategic Technology Management (STM), in collaboration with other members of the Information Technology Program Management Center of Excellence (ITPM COE), identifies the skills and resources needed to complete the work intended for the upcoming fiscal year, including the availability of supplementary staff, such as subject matter experts. (Recommendation 1)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary should ensure that the Executive Steering Committee overseeing the activities of the ITPM COE establishes target measures for the department's desired outcomes of its transition to Agile development. (Recommendation 2)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary should ensure that the DHS Chief Information Officer (CIO) defines a process and associated set of controls to ensure that Agile programs and projects are reporting a set of core required performance metrics for monitoring and measuring Agile adoption. (Recommendation 3)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary should ensure that the ITPM COE, in coordination with the CIO, begins measuring results associated with the transition to Agile and the success of the transition based on its impact on the department. (Recommendation 4)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary should ensure that the CIO, in collaboration with the Chief Procurement Officer, through the Homeland Security Acquisition Institute, establish Agile training requirements for senior stakeholders. (Recommendation 5)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary should ensure that the Chief Human Capital Officer, in collaboration with the CIO, consider modifications to the current employee recognition and performance management governance to ensure that teamwork and team performance of Agile programs and projects are incentivized. (Recommendation 6)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary should ensure that the CIO, in collaboration with the Chief Procurement Officer, through the Homeland Security Acquisition Institute, establish Agile training requirements for staff outside of the acquisition workforce but assigned to Agile programs. (Recommendation 7)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary should ensure that the CIO, upon establishing a set of core performance metrics, tracks and monitors the pace of Agile team development. (Recommendation 8)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary should ensure that the CIO, in collaboration with the Executive Director of the Office of Program Accountability and Risk Management (PARM), update or develop new guidance on Agile methodologies to describe how Agile teams can estimate the relative complexity of user stories. (Recommendation 9)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary should ensure that the CIO, upon establishing a set of core performance metrics, sets expectations for automated testing and code quality, and tracks and monitors against those expectations. (Recommendation 10)

Agency: Department of Homeland Security
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Acting Director of BLM should develop a documented process to consistently implement the APD prioritization process outlined in Instruction Memorandum 2013-104 at all field offices. (Recommendation 1)

Agency: Department of the Interior: Bureau of Land Management
Status: Open

Comments: The Department of the Interior, on behalf of BLM, did not concur with this recommendation. As of August 2020, BLM has not taken steps to address this recommendation.

Recommendation: The Acting Director of BLM should instruct agency staff to document formal change management processes for the rollout of future AFMSS II modules consistent with leading software development practices. (Recommendation 2)

Agency: Department of the Interior: Bureau of Land Management
Status: Open

Comments: The Department of the Interior, on behalf of BLM, did not concur with this recommendation--although, it did state that it agreed with the premise of the need to further improve its formal change management procedures. As of August 2020, BLM has not taken actions to address this recommendation.

Recommendation: The Acting Director of BLM should document and implement an action plan that identifies potential corrective actions based on previous lessons learned to address any challenges with the rollout of future AFMSS II modules. (Recommendation 3)

Agency: Department of the Interior: Bureau of Land Management
Status: Open

Comments: The Department of the Interior, on behalf of BLM, agreed with this recommendation. In its agency comment letter, the Department said that the BLM AFMSS team would implement adjustments and improvement based on the collected lessons learned. They also stated that upon completion of the last Field Office implementation and conversion to AFMSS II, BLM will conduct a formal program review and an annual operational assessment, which will also present opportunities for annual corrective actions and additional lessons learned considerations. As of August 2020, BLM has not taken actions to address this recommendation.

Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops a program-wide process for measuring, collecting, and tracking information on how ALIS is affecting the performance of the F-35 fleet to include, but not be limited to, its effects on mission capability rates. (Recommendation 1)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Secretary of Defense should ensure the Under Secretary of Defense for Acquisition and Sustainment, in consultation with the F-35 Program Executive Officer, develops and implements a strategy for the re-design of ALIS. The strategy should be detailed enough to clearly identify and assess the goals, key risks or uncertainties, and costs of re-designing the system. (Recommendation 2)

Agency: Department of Defense: Office of the Secretary of Defense
Status: Open

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: The Under Secretary of Defense for Acquisition and Sustainment should ensure that the Air Force's finalized Space C2 program's acquisition strategy includes, at a minimum, the following elements:

• acquisition and contracting approach;
• program management structure, including authorities and oversight responsibilities;
• plans for platform and infrastructure development;
• requirements management and development approach, and plans for prioritization;
• risk management plans, including how the program will identify and mitigate risks;
• metrics for measuring quality of software, and how those results will be shared with external stakeholders;
• manpower assessment identifying program workforce needs and state of expertise in Agile methods;
• requirements for reporting program progress to decision makers; and
• yearly funding levels. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with the recommendation and stated that the Under Secretary of Defense for Acquisition and Sustainment directed the Air Force (this work has now been moved to the Space Force) to provide an Acquisition Strategy for approval in November 2019. DOD noted that a strategy template provided to the Air Force included the elements identified by GAO. As of July 2020, the Acquisition Strategy had been submitted to the office of the Under Secretary of Defense for Acquisition and Sustainment, but officials stated that the strategy is still in review and has not yet been finalized.

Recommendation: The Under Secretary of Defense for Acquisition and Sustainment should ensure that the Air Force's Space C2 program conducts periodic independent reviews to assess the program's approach to developing software and provide, as needed, advice to the program and recommendations for improving the program's development and progress. Participants could include, but are not limited to, officials from the Defense Innovation Board, the Defense Digital Service, the office of the Air Force Chief Software Advisor, and the Under Secretary of Defense for Acquisition and Sustainment's Special Assistant for Software Acquisition. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation and stated that the Under Secretary of Defense for Acquisition and Sustainment will assess the need for future periodic and independent reviews of the program. As of July 2020, the Office of the Under Secretary of Defense for Acquisition and Sustainment stated that it had planned to direct an independent review of the program to be conducted by a Federally Funded Research and Development Center and to be completed by September 2020. However, lack of funding and restrictions related to COVID-19 impacted planning. The office still plans to direct this review, but details are pending.

Recommendation: The Secretary of Defense should ensure the department implements the pilot program by releasing at least 20 percent of newly custom-developed code as open source software (OSS). (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense did not concur with this recommendation and as of July 2020 has not yet implemented it. According to a December 2019 department letter provided to GAO, the 20 percent software release target is unlikely achievable due to the nature of code that is custom developed by the department. However, the department is mandated by law to implement the open source software pilot program established by the Office of Management and Budget's memorandum M-16-21. Releasing at least 20 percent of newly custom-developed code is a requirement of this program. GAO will continue to follow-up on the status of the pilot program.

Recommendation: The Secretary of Defense should ensure the department identifies a measure to calculate the percentage of code released to gauge its progress on implementing the pilot program. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: The Department of Defense partially agreed with this recommendation and as of July 2020 has not yet implemented it. According to a December 2019 department letter sent to GAO, the department intends to release updated guidance on the release of custom-developed code as open-source software and will include metrics. The department estimated that the updated policy will be completed in the 3rd quarter of fiscal year 2020. GAO will follow-up with the agency to obtain the status of the updated guidance.

Recommendation: The Secretary of Defense should direct the Director, Office of Cost Assessment and Program Evaluation to conduct an independent schedule assessment of the full program schedule for the Global Positioning System's next generation operational control system based on progress made through the end of calendar year 2019. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD did not concur with our recommendation to conduct an independent assessment of the full OCX program schedule based on progress made through the end of calendar year 2019, citing an independent cost and schedule estimate conducted in September 2018 and other ongoing program assessment and monitoring efforts. GAO continues to affirm that the recommendation is necessary given that DOD has not conducted an assessment of the full schedule since June 2018, since which time program risks have evolved. Additionally, ongoing oversight efforts are limited in scope and do not include the developmental test period after product delivery. As of August 2020, the Air Force position remains a non-concur, based on the same rationale previously noted.

Recommendation: The FEMA Administrator should ensure that the GMM program management office finalizes the organizational change management plan and time frames for implementing change management actions. (Recommendation 1)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the GMM program management office updates the program schedule to address the leading practices for a reliable schedule identified in this report. (Recommendation 4)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the FEMA Office of the Chief Information Officer (OCIO) defines sufficiently detailed planned evaluation methods and actual evaluation methods for assessing security controls. (Recommendation 5)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the FEMA OCIO approves a security assessment plan before security assessment reviews are conducted. (Recommendation 6)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the GMM program management office follows DHS guidance on preparing corrective action plans for all security vulnerabilities. (Recommendation 7)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The FEMA Administrator should ensure that the GMM program management office fully tests all of its security controls for the system. (Recommendation 8)

Agency: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open

Comments: DHS concurred with this recommendation. We will continue to monitor the department's efforts to implement it.

Recommendation: The Secretary of Defense should ensure the department's guidance that addresses software development provides specific, required direction on when and how often to involve users so that such involvement is early and continues through the development of the software and related program components. (Recommendation 1)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation and issued an interim Software Acquisition Pathway policy in January 2020 that addresses software development, including direction on user involvement. As of August 2020, this interim policy has not yet been finalized. According to DOD officials, a final policy is currently under development and is expected to be issued by the end of December 2020.

Recommendation: The Secretary of Defense should ensure the department's guidance that addresses software development provides specific, required direction on documenting and communicating user feedback to stakeholders during software system development. (Recommendation 2)

Agency: Department of Defense
Status: Open

Comments: DOD concurred with this recommendation. As of August 2020, DOD has issued an interim Software Acquisition Pathway that addresses software development, including direction on user involvement. According to DOD officials, this interim pathway is planned to be replaced by a final policy that is currently under development and is expected to be issued by the end of December 2020.

Recommendation: The Secretary of Agriculture should ensure that the CIO of USDA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 9)

Agency: Department of Agriculture
Status: Open

Comments: In September 2019, a Department of Agriculture official stated that the department was working to establish a policy to include the information noted in our recommendation and planned to finalize a policy by the end of December 2019. We will continue to monitor the department's progress on these efforts.

Recommendation: The Secretary of Veterans Affairs (VA) should ensure that the CIO of VA updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 10)

Agency: Department of Veterans Affairs
Status: Open

Comments: The Department of Veterans Affairs (VA) has taken action, and stated that it would draft a policy to address our recommendation. In November 2019, a VA official stated that the department is working to address our recommendation but did not identify timeframes for when all activities would be completed. We will continue to evaluate the department's progress in implementing this recommendation.

Recommendation: The Administrator of the Environmental Protection Agency (EPA) should ensure that the CIO of EPA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 11)

Agency: Environmental Protection Agency
Status: Open

Comments: The Environmental Protection Agency (EPA) concurred with our recommendation and stated that it planned to develop a policy to implement this recommendation and other FITARA issues. Specifically, EPA officials reported in July 2019 that the agency was continuing to work to address the recommendation but did not provided a time frame for when a policy would be finalized. We will continue to monitor EPA's progress on these efforts.

Recommendation: The Administrator of the National Aeronautics and Space Administration (NASA) should ensure that the CIO of NASA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 13)

Agency: National Aeronautics and Space Administration
Status: Open

Comments: The National Aeronautics and Space Administration (NASA) concurred with our recommendation and reported that the agency was in the process of addressing it. Specifically, NASA officials reported in June 2020 that its guidance is currently being updated to include the information noted in our recommendation and will be finalized by September 2020. We will continue to monitor NASA's progress on these efforts.

Recommendation: The Director of the Office of Personnel Management (OPM) should ensure that the CIO of OPM updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 16)

Agency: Office of Personnel Management
Status: Open

Comments: The Office of Personnel Management (OPM) concurred with our recommendation and stated that it would update its policies and processes to include the elements we recommended. Specifically, OPM officials reported in November 2019 that guidance on CIO certification was being developed but the agency had not yet determined a time frame for finalizing the policy. We will continue to monitor OPM's progress on these efforts.

Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

Agency: Department of Health and Human Services
Status: Open

Comments: The Department of Health and Human Services (HHS) concurred with our recommendation and has taken steps to establish a department policy and process for the certification of major IT investments' use of incremental development. Specifically, in September 2020, HHS officials reported that they have established a draft policy and anticipate publishing the finalized guidance by March 2021. We will continue to evaluate HHS's progress in implementing this recommendation.

Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

Agency: Department of the Treasury
Status: Open

Comments: In September 2020, an official from the Department of the Treasury (Treasury) reported that the department had developed draft guidance to address our recommendation, but did not provide time frames for when the guidance would be finalized. Until the department establishes a CIO certification policy, Treasury will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate Treasury's progress in implementing this recommendation.

Recommendation: To effectively measure 18F's performance, the Administrator of GSA should direct the Commissioner for the Technology Transformation Service to assess actual results for each performance measure.

Agency: General Services Administration
Status: Open

Comments: The General Services Administration (GSA) agreed with, and has begun to take steps to implement, this recommendation. Specifically, in a March 2020 written response, GSA stated that Technology Transformation Service (TTS) leadership will be briefed on the program's performance measures on a quarterly basis. We are following up with GSA to confirm that its TTS leadership has been briefed on the results on these performance measures. We will continue to evaluate GSA's progress in implementing this recommendation.

Recommendation: To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to ensure that all goals and associated performance measures are outcome-oriented and that performance measures have targets.

Agency: Executive Office of the President: Office of Management and Budget
Status: Open

Comments: The Office of Management and Budget (OMB) generally agreed with, and has begun to take steps to implement, this recommendation. Specifically, in its December 2016 report to Congress, OMB developed three goals for U.S. Digital Service (USDS): (1) rethink how the federal government builds and buys digital services; (2) expand the use of common, platforms, services, and tools; and (3) bring top technical talent into public service. In addition, OMB established performance measures with targets for its third goal and for each of the program's major projects. However, OMB has not established performance measures for the first two USDS goals. Further, the program's third goal is not outcome-oriented. In May 2018, an USDS staff member stated that USDS established goals for and measured performance on each of the projects the program supports in its fall 2017 report to Congress. Although measuring performance on projects can provide USDS with valuable information, this effort does not address goals and performance measurement on the overall USDS program. In May 2020, OMB stated that they would provide an update on the agency's efforts to address the recommendation by June 2020. We will continue to evaluate OMB's progress in implementing this recommendation.

Recommendation: To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to assess actual results for each performance measure.

Agency: Executive Office of the President: Office of Management and Budget
Status: Open

Comments: The Office of Management and Budget (OMB) generally agreed with, and has begun to take steps to implement, this recommendation. Specifically, in its December 2016 report to Congress, OMB assessed the results of performance measures for one of the U.S. Digital Service (USDS) program's goals--bring top technical talent into public service--and for each of the program's major projects. However, OMB has not established performance measures for the other two USDS goals--rethink how the federal government builds and buys digital services; and expand the use of common, platforms, services, and tools. In May 2018, an USDS staff member stated that USDS established goals for and measured performance on each of the projects the program supports in its fall 2017 report to Congress. As of July 2019, USDS has not publicly released any subsequent reports to Congress or additional information on its goals and performance measures. Although measuring performance on projects can provide USDS with valuable information, this effort does not address performance measurement on the overall USDS program. In May 2020, OMB stated that they would provide an update on the agency's efforts to address the recommendation by June 2020. We will continue to evaluate OMB's progress in implementing this recommendation.

Recommendation: To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to update USDS policy to clearly define the responsibilities and authorities governing the relationships between CIOs and the digital service teams and require existing agency digital service teams to address this policy. In doing so, the Federal Chief Information Officer should ensure that this policy is aligned with relevant federal law and OMB guidance on CIO responsibilities and authorities.

Agency: Executive Office of the President: Office of Management and Budget
Status: Open

Comments: The Office of Management and Budget (OMB) generally agreed with, and has begun to take steps to implement, this recommendation. In particular, OMB updated its digital service team policy to require that teams appropriately inform their chief information officers (CIO) regarding U.S. Digital Service (USDS) projects. However, the policy does not describe the responsibilities or authorities governing the relationships between CIOs and digital service teams. In May 2018, an USDS staff member stated that the program updated digital service team charters to address the role of agency CIOs. As of May 2020, USDS has yet to provide us with the updated digital service team charters. In May 2020, OMB stated that they would provide an update on the agency's efforts to address the recommendation by June 2020. We will continue to evaluate OMB's progress in implementing this recommendation.

Recommendation: To provide the Congress and NASA a reliable estimate of program cost and schedule that are useful to support management and stakeholder decisions, the NASA Administrator should direct the Orion program to perform an updated JCL analysis including updating cost and schedule estimates in adherence with cost and schedule estimating best practices.

Agency: National Aeronautics and Space Administration
Status: Open
Priority recommendation

Comments: NASA partially agreed with this recommendation, stating that the agency reviewed, in detail, the Orion integrated cost/schedule and risk analysis methodology and determined the rigor to be a sufficient basis for the agency commitments. In November 2019, Orion program officials told us that in response to a recent policy change, the program office will update its joint confidence level analysis when the program has its Key Decision Point D review. This review occurs before the program enters the system assembly integration and test, and launch phase and is not scheduled to occur until December 2020. To fully implement this recommendation, NASA will need to provide evidence that it updated its joint confidence level analysis when the Orion program holds its Key Decision Point D review.