Reports & Testimonies

  • GAO’s recommendations database contains report recommendations that still need to be addressed.

    GAO’s recommendations help congressional and agency leaders prepare for appropriations and oversight activities, as well as help improve government operations. Recommendations remain open until they are designated as Closed-implemented or Closed-not implemented. You can explore open recommendations by searching or browsing.

    GAO's priority recommendations are those that we believe warrant priority attention. We sent letters to the heads of key departments and agencies, urging them to continue focusing on these issues. These recommendations are labeled as such. You can find priority recommendations by searching or browsing our open recommendations below, or through our mobile app.

  • Browse Open Recommendations

    Explore priority recommendations by subject terms or browse by federal agency

    Search Open Recommendations

    Search for a specific priority recommendation by word or phrase



  • Governing on the go?

    Our Priorities for Policy Makers app makes it easier for leaders to search our recommendations on the go.

    See the November 10th Press Release


  • Have a Question about a Recommendation?

    • For questions about a specific recommendation, contact the person or office listed with the recommendation.
    • For general information about recommendations, contact GAO's Audit Policy and Quality Assurance office at (202) 512-6100 or apqa@gao.gov.
  • « Back to Results List Sort by   

    Results:

    Subject Term: "Software development"

    6 publications with a total of 60 open recommendations including 7 priority recommendations
    Director: David A. Powner
    Phone: (202) 512-9286

    19 open recommendations
    Recommendation: The Secretary of Energy should ensure that the CIO of Energy reports major IT investment information related to incremental development accurately in accordance with OMB guidance. (Recommendation 1)

    Agency: Department of Energy
    Status: Open

    Comments: In comments on our report, the Department of Energy (Energy) concurred with our recommendation. The department described actions by the Office of the Chief Information Officer to review the accuracy of Energy's major IT investment project reporting related to incremental development as part of monthly IT Dashboard and Investment Review Board meetings. We will continue to monitor Energy's reporting of its incremental data on the IT Dashboard.
    Recommendation: The Secretary of Agriculture should ensure that the CIO of U.S. Department of Agriculture (USDA) reports major IT investment information related to incremental development accurately in accordance with OMB guidance. (Recommendation 2)

    Agency: Department of Agriculture
    Status: Open

    Comments: The U.S. Department of Agriculture (USDA) has not yet taken any actions to implement our recommendation. We will continue to monitor USDA's progress in implementing this recommendation.
    Recommendation: The Commissioner of the Social Security Administration (SSA) should ensure that the CIO of SSA reports major IT investment information related to incremental development accurately in accordance with OMB guidance. (Recommendation 3)

    Agency: Social Security Administration
    Status: Open

    Comments: In comments on our report, the Social Security Administration (SSA) concurred with our recommendation and stated that it had implemented two new processes related to incremental certification, which would be included in the agency's upcoming revision to its Capital Planning and Investment Control Guide. We will follow up with SSA to ascertain whether the guide has been updated during fiscal year 2018.
    Recommendation: The Secretary of Housing and Urban Development (HUD) should ensure that the CIO of HUD establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 4)

    Agency: Department of Housing and Urban Development
    Status: Open

    Comments: In comments on our report, the Department of Housing and Urban Development (HUD) concurred with our recommendation and stated that it planned to develop more definitive timelines on how to address our recommendation. We will follow up with HUD to ascertain whether an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development has been established during fiscal year 2018.
    Recommendation: The Secretary of the Interior should ensure that the CIO of Interior updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development, consistent with OMB guidance. (Recommendation 5)

    Agency: Department of the Interior
    Status: Open

    Comments: In comments on our report, the Department of the Interior (Interior) concurred with our recommendation and stated that it planned to update its existing policy to include the elements we recommended. We will follow up with Interior to ascertain whether an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development has been updated during fiscal year 2018.
    Recommendation: The Attorney General of the United States should ensure that the CIO of Justice establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 6)

    Agency: Department of Justice
    Status: Open

    Comments: In comments on our report, the Department of Justice (Justice) concurred with our recommendation and stated that it planned to amend existing policy to implement this recommendation. We will follow up with Justice to ascertain whether an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development has been updated during fiscal year 2018.
    Recommendation: The Secretary of Labor should ensure that the CIO of Labor updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 7)

    Agency: Department of Labor
    Status: Open

    Comments: The Department of Labor (Labor) has not yet taken any actions to implement our recommendation. We will continue to monitor Labor's progress in implementing this recommendation.
    Recommendation: The Secretary of State should ensure that the CIO of State updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 8)

    Agency: Department of State
    Status: Open

    Comments: In comments on our report, the Department of State (State) reported that it has developed an incremental development policy that addresses our recommendation and it is currently in the process of being approved. We will follow up with State to ascertain whether an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development has been developed during fiscal year 2018.
    Recommendation: The Secretary of Agriculture should ensure that the CIO of USDA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 9)

    Agency: Department of Agriculture
    Status: Open

    Comments: The U.S. Department of Agriculture (USDA) has not yet taken any actions to implement our recommendation. We will continue to monitor USDA's progress in implementing this recommendation.
    Recommendation: The Secretary of Veterans Affairs (VA) should ensure that the CIO of VA updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 10)

    Agency: Department of Veterans Affairs
    Status: Open

    Comments: In comments on our report, the Department of Veterans Affairs (VA) partially concurred with our recommendation and stated that it would draft a policy to address our recommendation. We will follow up with VA to ascertain whether an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development has been developed during fiscal year 2018.
    Recommendation: The Administrator of the Environmental Protection Agency (EPA) should ensure that the CIO of EPA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 11)

    Agency: Environmental Protection Agency
    Status: Open

    Comments: In comments on our report, the Environmental Protection Agency (EPA) concurred with our recommendation and stated that it planned to develop a policy to implement this recommendation and other FITARA issues. We will follow up with EPA to ascertain whether an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development has been developed during fiscal year 2018.
    Recommendation: The Administrator of the General Services Administration (GSA) should ensure that the CIO of GSA updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 12)

    Agency: General Services Administration
    Status: Open

    Comments: In comments on our report, the General Services Administration (GSA) concurred with our recommendation and stated that it would develop and implement a plan to address it. We will follow up with GSA to ascertain whether an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development has been updated during fiscal year 2018.
    Recommendation: The Administrator of the National Aeronautics and Space Administration (NASA) should ensure that the CIO of NASA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 13)

    Agency: National Aeronautics and Space Administration
    Status: Open

    Comments: In comments on our report, the National Aeronautics and Space Administration (NASA) concurred with our recommendation and stated that it is currently updating its policies to address it, which will be completed by March 2018. We will continue to monitor NASA's progress on these efforts.
    Recommendation: The Director of the National Science Foundation (NSF) should ensure that the CIO of NSF updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 14)

    Agency: National Science Foundation
    Status: Open

    Comments: The National Science Foundation (NSF) has not yet taken any actions to implement our recommendation. We will continue to monitor NSF's progress in implementing this recommendation.
    Recommendation: The Chairman of the Nuclear Regulatory Commission (NRC) should ensure that the CIO of NRC establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 15)

    Agency: Nuclear Regulatory Commission
    Status: Open

    Comments: In comments on our report, the Nuclear Regulatory Commission (NRC) reported that it plans to establish agency-wide, formalized processes and procedures for the CIO to approve the incremental development of major IT investments by December 31, 2017. We will continue to monitor NRC's progress on this effort.
    Recommendation: The Director of the Office of Personnel Management (OPM) should ensure that the CIO of OPM updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 16)

    Agency: Office of Personnel Management
    Status: Open

    Comments: In comments on our report, the Office of Personnel Management (OPM) concurred with our recommendation and stated that it would update its policies and processes to include the elements we recommended. We will follow up with OPM to ascertain whether an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development has been updated during fiscal year 2018.
    Recommendation: The Administrator of the Small Business Administration (SBA) should ensure that the CIO of SBA establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 17)

    Agency: Small Business Administration
    Status: Open

    Comments: The Small Business Administration (SBA) has not yet taken any actions to implement our recommendation. We will continue to monitor SBA's progress in implementing this recommendation.
    Recommendation: The Commissioner of the Social Security Administration should ensure that the CIO of SSA updates the agency's policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes a description of the CIO's role in the certification process and a description of how CIO certification will be documented. (Recommendation 18)

    Agency: Social Security Administration
    Status: Open

    Comments: In comments on our report, the Social Security Administration (SSA) concurred with our recommendation and stated that it had implemented two new processes related to incremental certification, which would be included in the agency's upcoming revision to its Capital Planning and Investment Control Guide. We will follow up with SSA to ascertain whether the guide has been updated during fiscal year 2018.
    Recommendation: The Administrator of the U.S. Agency for International Development (USAID) should ensure that the CIO of USAID establishes an agency-wide policy and process for the CIO's certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA, and confirm that it includes: a description of the CIO's role in the certification process; a description of how CIO certification will be documented; and a definition of incremental development and time frames for delivering functionality, consistent with OMB guidance. (Recommendation 19)

    Agency: United States Agency for International Development
    Status: Open

    Comments: In comments on our report, the U.S. Agency for International Development (USAID) reported that it is in the process of establishing an agency-wide policy and process for the CIO's certification of adequate incremental development. It estimates that this policy will be implemented by August 31, 2018. We will continue to monitor USAID's progress in this effort.
    Director: David A. Powner
    Phone: (202) 512-9286

    12 open recommendations
    Recommendation: In order to improve the accuracy of IT Dashboard incremental development data, the Director of OMB should direct the Federal Chief Information Officer (CIO) to clarify existing guidance regarding what IT investments are and are not subject to requirements on the use of incremental development and how CIOs should report the status of projects that are not subject to these requirements.

    Agency: Executive Office of the President: Office of Management and Budget
    Status: Open

    Comments: The Office of Management and Budget (OMB) has taken initial steps to implement our recommendation. Specifically, OMB's June 2016 annual capital planning guidance for fiscal year 2018 included instructions on what types of investments were required to adhere to incremental development requirements related to the delivery of usable functionality. The guidance stated that all software development projects are required to produce usable functionality at intervals of no more than six months. Further, all major development projects within investments are required to use modular/agile principles. However, OMB's guidance still lacks direction on how CIOs are to report the status of nonsoftware projects, as we recommended. In the absence of our recommended guidance clarification, OMB is at risk of agencies continuing to be unclear about how nonsoftware development investment data are to be reported on the Dashboard, increasing the risk that data on the IT Dashboard will not always be accurate. We will continue to evaluate OMB's progress in clarifying its guidance and considering a change to provide more detailed guidance related to the reporting of nonsoftware development investment data.
    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency: Department of Homeland Security
    Status: Open

    Comments: The Department of Homeland Security (DHS) concurred with our recommendation and stated that the Enterprise Business Management Office within the Office of the Chief Information Officer will validate each investment reported on the Dashboard and work with program officials to ensure they appropriately update the data for the IT Dashboard. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.
    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency: Department of Education
    Status: Open

    Comments: The Department of Education (Education) concurred with our recommendation and stated that the department will ensure that the data is kept current using their IT portfolio management process. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.
    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency: Department of Commerce
    Status: Open

    Comments: The Department of Commerce (Commerce) concurred with our recommendation and stated that these changes would be incorporated into the department?s Dashboard reporting. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.
    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency: Department of Defense
    Status: Open

    Comments: The Department of Defense (Defense) partially concurred with our recommendation and stated that the department is taking action to update the Dashboard data as appropriate. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.
    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency: Department of Health and Human Services
    Status: Open

    Comments: The Department of Health and Human Services (HHS) concurred with our recommendation. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.
    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency: Department of Transportation
    Status: Open

    Comments: The Department of Transportation (Transportation) concurred with our recommendation and stated the department was committed to ensuring the information on the IT Dashboard reflects up to date information. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.
    Recommendation: To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels.

    Agency: Department of the Treasury
    Status: Open

    Comments: The Department of the Treasury (Treasury) did not comment on our recommendation. However, after our report was issued in August 2016, the IT Dashboard was not publicly updated from the end of August 2016 until the end of May 2017, during the formulation of the President's budget request. Now that the Dashboard is being publicly updated again, we will continue to analyze and monitor the department's progress in updating investment information on the Dashboard and the implementation of our recommendation.
    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency: Department of Education
    Status: Open

    Comments: The Department of Education (Education) concurred with our recommendation to establish a departmentwide certification policy. Education officials reported in March 2017 that the department will complete changes to its guidance by November 2017. However, until this guidance is finalized, Education will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate Education's progress in implementing this recommendation.
    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency: Department of Defense
    Status: Open

    Comments: The Department of Defense (Defense) did not concur with our recommendation, stating that its existing guidance was adequate in this area. However, in August 2016, Defense issued its fiscal year 2018 budget submission guidance which required each component CIO to certify that IT investments were adequately implementing incremental development. The component CIOs were to document the certification in a statement of compliance memorandum, using their agency's letterhead, and submit the memorandum to the Defense CIO. Defense officials report that this same guidance will be added to the Financial Management Regulations during summer 2017. Until this annual guidance has been updated and incorporated into the department's standing policies, Defense is at risk of overlooking this requirement in subsequent years. We will continue to evaluate Defense's progress in implementing this recommendation.
    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency: Department of Health and Human Services
    Status: Open

    Comments: The Department of Health and Human Services (HHS) concurred with our recommendation to establish a departmentwide certification policy. However, HHS officials reported in April 2017 that they did not have a timeframe for when the department's new certification guidance would be completed. Until this guidance is finalized, HHS will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate HHS's progress in implementing this recommendation.
    Recommendation: To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act.

    Agency: Department of the Treasury
    Status: Open

    Comments: The Department of the Treasury (Treasury) did not comment on our recommendations. Further, Treasury officials reported in March 2017 that it had no plans to revise its policies, as we recommended. Until the department establishes a CIO certification policy, Treasury will not be able to fully ensure adequate implement of, or benefit from, incremental development practices. We will continue to evaluate Treasury's progress in implementing this recommendation.
    Director: David A. Powner
    Phone: (202) 512-9286

    5 open recommendations
    Recommendation: To effectively measure 18F's performance, the Administrator of GSA should direct the Commissioner for the Technology Transformation Service to ensure that goals and associated performance measures are outcome-oriented and that performance measures have targets, including (1) performance measures and targets tied to fully recovering program costs; and (2) goals, performance measures, and targets for how the program will achieve its mission after September 2016.

    Agency: General Services Administration
    Status: Open

    Comments: The General Services Administration (GSA) agreed with, and has begun to take steps to implement, this recommendation. Specifically, GSA developed a quarterly performance report for fiscal year 2017 that includes an outcome-oriented goal for 18F as well as associated performance measures and targets. According to a Technology Transformation Service official, GSA plans to expand its quarterly performance report for fiscal year 2018 to reflect additional 18F goals and performance measures, including measures tied to fully recovering program costs. We will continue to evaluate GSA's progress in implementing this recommendation.
    Recommendation: To effectively measure 18F's performance, the Administrator of GSA should direct the Commissioner for the Technology Transformation Service to assess actual results for each performance measure.

    Agency: General Services Administration
    Status: Open

    Comments: The General Services Administration (GSA) agreed with, and has begun to take steps to implement, this recommendation. Specifically, GSA developed a quarterly performance report for fiscal year 2017 that includes an outcome-oriented goal for 18F as well as associated performance measures with targets. Additionally, GSA has assessed actual results of the performance measures for the first two quarters of fiscal year 2017. According to a Technology Transformation Service official, GSA plans to expand its quarterly performance report for fiscal year 2018 to include additional 18F goals and performance measures. We will continue to evaluate GSA's progress in implementing this recommendation.
    Recommendation: To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to ensure that all goals and associated performance measures are outcome-oriented and that performance measures have targets.

    Agency: Executive Office of the President: Office of Management and Budget
    Status: Open

    Comments: The Office of Management and Budget (OMB) generally agreed with, and has begun to take steps to implement, this recommendation. Specifically, in its December 2016 report to Congress, OMB developed three goals for U.S. Digital Service (USDS): (1) rethink how the federal government builds and buys digital services; (2) expand the use of common, platforms, services, and tools; and (3) bring top technical talent into public service. In addition, OMB established performance measures with targets for its third goal and for each of the program's major projects. However, OMB has not established performance measures for the first two USDS goals. Further, the program's third goal is not outcome-oriented. We will continue to evaluate OMB's progress in implementing this recommendation.
    Recommendation: To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to assess actual results for each performance measure.

    Agency: Executive Office of the President: Office of Management and Budget
    Status: Open

    Comments: The Office of Management and Budget (OMB) generally agreed with, and has begun to take steps to implement, this recommendation. Specifically, in its December 2016 report to Congress, OMB assessed the results of performance measures for one of the U.S. Digital Service (USDS) program's goals--bring top technical talent into public service--and for each of the program's major projects. However, OMB has not established performance measures for the other two USDS goals--rethink how the federal government builds and buys digital services; and expand the use of common, platforms, services, and tools. We will continue to evaluate OMB's progress in implementing this recommendation.
    Recommendation: To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to update USDS policy to clearly define the responsibilities and authorities governing the relationships between CIOs and the digital service teams and require existing agency digital service teams to address this policy. In doing so, the Federal Chief Information Officer should ensure that this policy is aligned with relevant federal law and OMB guidance on CIO responsibilities and authorities.

    Agency: Executive Office of the President: Office of Management and Budget
    Status: Open

    Comments: The Office of Management and Budget (OMB) generally agreed with, and has begun to take steps to implement, this recommendation. In particular, OMB updated its digital service team policy to require that teams appropriately inform their chief information officers (CIO) regarding U.S. Digital Service (USDS) projects. However, the policy does not describe the responsibilities or authorities governing the relationships between CIOs and digital service teams. We will continue to evaluate OMB's progress in implementing this recommendation.
    Director: Chaplain, Cristina T
    Phone: (202) 512-4841

    2 open recommendations
    including 1 priority recommendation
    Recommendation: To provide the Congress and NASA a reliable estimate of program cost and schedule that are useful to support management and stakeholder decisions, the NASA Administrator should direct the Orion program to perform an updated JCL analysis including updating cost and schedule estimates in adherence with cost and schedule estimating best practices.

    Agency: National Aeronautics and Space Administration
    Status: Open
    Priority recommendation

    Comments: NASA partially agreed with this recommendation, stating that the agency reviewed, in detail, the Orion integrated cost/schedule and risk analysis methodology and determined the rigor to be a sufficient basis for the agency commitments. We still contend that NASA should update its analysis that informed its baseline because we found that the cost and schedule estimates underlying those baselines are not reliable as they did not conform to best practices.
    Recommendation: To have a full understanding of the cost, schedule, and safety impact of deferring work, the NASA Administrator should direct the Orion program to perform an analysis on the cost of deferred work in relation to levels of management reserves and unallocated future expenses and actual contractor performance, and report the results of that analysis to NASA management.

    Agency: National Aeronautics and Space Administration
    Status: Open

    Comments: NASA concurred with this recommendation, but characterized its deferral of work to date as task-level deferrals, lasting only several months and not affecting major program milestone or the critical path. NASA did agree to include an analysis of how these deferrals affect budget reserves and program performance in future routine management reporting. NASA officials told us that they are currently evaluating work flow for the first and second mission as the agency revisits the launch date for the first mission. Given this is currently being analyzed, officials were not able to provide any analysis at this time about the potential cost impact of changes in scheduled work.
    Director: Carol C. Harris
    Phone: (202) 512-4456

    9 open recommendations
    Recommendation: To provide reasonable assurance that the program executes Agile software development for USCIS ELIS consistent with its own policies and guidance and follows applicable leading practices, the Secretary of the Department of Homeland Security (DHS) should direct the Director of USCIS to direct the USCIS Chief Information Officer (CIO), in coordination with the DHS CIO and the Chief of the Office of Transformation Coordination (OTC), to review and update, as needed, existing policies and guidance and consider additional controls to complete planning for software releases prior to initiating development and ensure software meets business expectations prior to deployment.

    Agency: Department of Homeland Security
    Status: Open

    Comments: As of July 2017, the U.S. Citizenship and Immigration Services (USCIS) within the Department of Homeland Security (DHS) had taken steps to address this recommendation. In particular, in June 2017, USCIS provided an updated policy, dated April 2017, governing planning and deploying software releases. USCIS also demonstrated partial compliance with that policy. For example, it provided some release planning review documentation for recent releases that are required by the updated policy, including readiness review memos for releases 7.2 and 8.1. However, USCIS did not demonstrate that the program responsible for developing the USCIS Electronic Immigration System (USCIS ELIS) was consistently following its updated policy. For example, USCIS did not demonstrate that the program was completing all planning activities prior to initiating development, as called for in its updated policy. Moreover, the agency did not demonstrate compliance with its previous policy for all software releases planned and deployed since our July 2016 report. We will continue to work with USCIS to monitor actions the agency is taking to address this recommendation.
    Recommendation: To provide reasonable assurance that the program executes Agile software development for USCIS ELIS consistent with its own policies and guidance and follows applicable leading practices, the Secretary of DHS should direct the Director of USCIS to direct the USCIS CIO, in coordination with the DHS CIO and the Chief of OTC, to review and update, as needed, existing policies and guidance and consider additional controls to consistently implement the principles of the framework adopted for Agile software development.

    Agency: Department of Homeland Security
    Status: Open

    Comments: As of July 2017, USCIS had taken steps to address this recommendation. For example, in May 2017, USCIS provided updated policy governing the development of software releases, dated April 2017, along with release planning artifacts specific to USCIS ELIS. The updated policy included an appendix devoted to generally accepted agency practices and applying Agile principles in the agency. However, USCIS had not clearly indicated if USCIS ELIS was to implement the practices described in the policy. For example, the updated policy did not require program compliance with the generally accepted agency practices. Moreover, supporting artifacts from the release planning process did not always define a commitment to a particular development methodology or set of development practices. For example, the team process agreements, which describe how members of individual teams will work with each other, did not indicate if developers were to adhere to the practices described in updated USCIS policy. We will continue to work with USCIS to obtain additional documentation about actions it is taking to address this recommendation.
    Recommendation: To provide reasonable assurance that the program executes Agile software development for USCIS ELIS consistent with its own policies and guidance and follows applicable leading practices, the Secretary of DHS should direct the Director of USCIS to direct the USCIS CIO, in coordination with the DHS CIO and the Chief of OTC, to review and update, as needed, existing policies and guidance and consider additional controls to define and consistently execute appropriate roles and responsibilities for individuals responsible for development activities consistent with its selected development framework.

    Agency: Department of Homeland Security
    Status: Open

    Comments: As of July 2017, USCIS had taken steps to address this recommendation. For example, in June 2017, USCIS provided updated policy, dated April 2017, governing the development of software releases and release planning artifacts. The updated policy and release documentation defined some roles and responsibilities that were previously only described by USCIS in its informal November 2014 management model, such as the authority and responsibility of a product owner. However, program documentation and policy did not define all of the roles and responsibilities. For example, program documentation and policy did not define the roles and responsibilities of a facilitator, or Scrum Master, which is a position identified in leading practices for software development using Scrum, the development methodology previously identified by the program. In addition, USCIS did not demonstrate that it had defined and committed to an updated development methodology for software releases. Such a defined methodology will impact expectations for the roles and responsibilities in software development. Without such a defined methodology or approach to Agile software development, it is not clear if roles and responsibilities defined by previously documented approach to Agile software development are still applicable for the current development approach. Moreover, documentation associated with program releases and updated policy did not define all of the roles and responsibilities for positions described by USCIS in its May 2017 written response to GAO. We will continue to work with USCIS to obtain additional documentation about actions it is taking to address this recommendation.
    Recommendation: To provide reasonable assurance that the program executes Agile software development for USCIS ELIS consistent with its own policies and guidance and follows applicable leading practices, the Secretary of DHS should direct the Director of USCIS to direct the USCIS CIO, in coordination with the DHS CIO and the Chief of OTC, to review and update, as needed, existing policies and guidance and consider additional controls to identify all system users and involve them in release planning activities.

    Agency: Department of Homeland Security
    Status: Open

    Comments: As of July 2017, DHS and USCIS had not provided information demonstrating that the department has addressed this recommendation. In October 2016, DHS provided a written response stating that the USCIS Office of Information Technology and Office of Transformation Coordination were working closely with the various USCIS directorates to obtain and integrate feedback through regular review sessions with the end users and through additional end user testing. However, as of July 2017, DHS and USCIS have not provided new information about the status of this recommendation.
    Recommendation: To provide reasonable assurance that the program executes Agile software development for USCIS ELIS consistent with its own policies and guidance and follows applicable leading practices, the Secretary of DHS should direct the Director of USCIS to direct the USCIS CIO, in coordination with the DHS CIO and the Chief of OTC, to review and update, as needed, existing policies and guidance and consider additional controls to write user stories that identify user roles, include estimates of complexity, take no longer than one sprint to complete, and describe business value.

    Agency: Department of Homeland Security
    Status: Open

    Comments: As of July 2017, USCIS had provided GAO with documentation intended to demonstrate that the agency had taken steps to address this recommendation. For example, in May 2017, USCIS provided updated policy governing the development of software releases along with release planning artifacts specific to USCIS ELIS and an Independent Verification and Validation assessment. The agency also provided a series of backlogs that captured user stories for some software releases. In addition, the Independent Verification and Validation assessment indicated that the program was tracking user story quality as part of assessing whether value was continuously discovered and aligned to the mission. However, the assessment report provided to GAO indicated a negative trend for this outcome. Moreover, USCIS policy no longer set expectations regarding user story development. In addition, supporting artifacts from the release planning process did not always define a commitment to a particular development methodology, which is turn impacts the expectations for writing user stories. Finally, backlogs provided by USCIS did not cover all releases in development since our July 2016 report and did not include enough detail to assess all aspects of the user story process (e.g., story size and user involvement). We will continue to work with USCIS to obtain additional documentation about actions it is taking to address this recommendation.
    Recommendation: To provide reasonable assurance that the program executes Agile software development for USCIS ELIS consistent with its own policies and guidance and follows applicable leading practices, the Secretary of DHS should direct the Director of USCIS to direct the USCIS CIO, in coordination with the DHS CIO and the Chief of OTC, to review and update, as needed, existing policies and guidance and consider additional controls to establish outcomes for Agile software development.

    Agency: Department of Homeland Security
    Status: Open

    Comments: As of July 2017, USCIS had taken steps to address this recommendation. For example, in April 2017, USCIS issued updated policy governing software development at the agency. The updated policy included an appendix devoted to generally accepted agency practices and applying Agile principles in the agency. This appendix also included a set of ten outcomes associated with using Agile practices at USCIS. For example, outcomes included that value is continuously discovered and aligned to the mission. However, the updated policy did not require program compliance with the practices and principles described in the appendix. Moreover, the agency did not demonstrate that USCIS ELIS had committed to achieving a specific set of outcomes for Agile software development, such as the outcomes described in the USCIS policy. We will continue to work with USCIS to obtain additional documentation about actions it is taking to address this recommendation.
    Recommendation: To provide reasonable assurance that the program executes Agile software development for USCIS ELIS consistent with its own policies and guidance and follows applicable leading practices, the Secretary of DHS should direct the Director of USCIS to direct the USCIS CIO, in coordination with the DHS CIO and the Chief of OTC, to review and update, as needed, existing policies and guidance and consider additional controls to monitor program performance and report to appropriate entities through the collection of reliable metrics.

    Agency: Department of Homeland Security
    Status: Open

    Comments: As of July 2017, USCIS had taken steps to address this recommendation. For example, in May 2017, USCIS provided updated policy governing the development of software that called for teams to prepare an Operations Monitoring Plan or dashboard showing the practices, tools, and measures that will monitor applications in production. The agency also provided a series of documents from internal systems and processes intended to monitor performance, such as a product dashboard for analyzing code quality (i.e., SonarQube) and a report from its Independent Verification and Validation team. However, the program was undergoing a re-baseline and had yet to document updated cost, schedule, and performance expectations against which to monitor. Moreover, the agency did not demonstrate that other metrics, such as customer satisfaction and team velocity, were being reliably collected. We will continue to work with USCIS to obtain additional documentation about actions it is taking to address this recommendation.
    Recommendation: To help manage the USCIS ELIS system, the Secretary of DHS should direct the Director of USCIS to direct the USCIS CIO, in coordination with the DHS CIO and the Chief of OTC, to review and update existing policies and guidance and consider additional controls to conduct unit and integration, and functional acceptance tests, and code inspection consistent with stated program goals.

    Agency: Department of Homeland Security
    Status: Open

    Comments: As of July 2017, USCIS had taken steps to address this recommendation. For example, in May 2017, USCIS provided artifacts from internal systems in place to monitor software development performance. These metrics monitored aspects of testing, such as code quality and code coverage. However, the program did not provide an updated Test and Evaluation Master Plan, which is a document it will produce as part of its ongoing effort to re-baseline. A Test and Evaluation Master Plan sets the testing expectations for the program as agreed upon with its stakeholders in DHS and USCIS. The updated plan will provide a basis for further evaluation of the steps DHS and USCIS have taken to address this recommendation. Moreover, the agency did not demonstrate that functional acceptance tests were being conducted in accordance with stated program goals. For example, the agency did not provide acceptance criteria or the associated tests demonstrating that user stories passed the defined acceptance criteria. We will continue to work with USCIS to obtain additional documentation about actions it is taking to address this recommendation.
    Recommendation: To help manage the USCIS ELIS system, the Secretary of DHS should direct the Director of USCIS to direct the USCIS CIO, in coordination with the DHS CIO and the Chief of OTC, to review and update existing policies and guidance and consider additional controls to develop complete test plans and cases for interoperability and end user testing, as defined in the USCIS Transformation Program Test and Evaluation Master Plan, and document the results.

    Agency: Department of Homeland Security
    Status: Open

    Comments: As of July 2017, DHS and USCIS had not provided information demonstrating that they had addressed this recommendation. In October 2016, DHS provided a written response indicating that an internal process for revisiting the USCIS ELIS Test and Evaluation Master Plan had been initiated, with participation from all relevant stakeholder groups. A Test and Evaluation Master Plan sets the testing expectations for the program as agreed upon with its stakeholders in DHS and USCIS. The updated plan will provide a basis for further evaluation of the steps DHS and USCIS have taken to address this recommendation. The letter also stated that USCIS had begun to work on a policy for new interoperability test procedures. Moreover, the letter added that end user testing is a continuing activity, including providing feedback of observed issues into the development queue, with the slow launch of the naturalization capabilities in USCIS ELIS being a model. However, as of July 2017, DHS and USCIS had not provided new information about the status of this recommendation. We will continue to work with DHS and USCIS to obtain additional documentation about actions they are taking to address this recommendation.
    Director: Timothy J. DiNapoli
    Phone: (202) 512-4841

    13 open recommendations
    including 6 priority recommendations
    Recommendation: To improve efforts to strategically source IT services within the Army, the Secretary of the Army should direct its strategic sourcing accountable official to conduct a comprehensive analysis of Army IT services spending to determine the extent to which requirements can be addressed by Computer Hardware, Enterprise Software and Solutions (CHESS) or other strategic sourcing approaches, and based on this analysis, consider opportunities to reduce duplicative contracts.

    Agency: Department of Defense: Department of the Army
    Status: Open
    Priority recommendation

    Comments: DOD concurred with our recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within the Army, the Secretary of the Army should direct its strategic sourcing accountable official to implement utilization metrics and mandatory use or consideration policies.

    Agency: Department of Defense: Department of the Army
    Status: Open

    Comments: DOD concurred with our recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within the Army, the Secretary of the Army should direct its strategic sourcing accountable official to develop guidance and overarching goals and metrics for savings.

    Agency: Department of Defense: Department of the Army
    Status: Open

    Comments: DOD concurred with our recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within the Army, the Secretary of the Army should direct its strategic sourcing accountable official to conduct a review of the benefits and disadvantages of standardized labor categories for CHESS or future contracts.

    Agency: Department of Defense: Department of the Army
    Status: Open

    Comments: DOD concurred with our recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within the Navy, the Secretary of the Navy should direct its strategic sourcing accountable official to conduct a comprehensive analysis of IT services spending to determine the extent to which requirements can be addressed by the existing contracts or other strategic sourcing approaches and based on this analysis, reduce duplicative contracts.

    Agency: Department of Defense: Department of the Navy
    Status: Open
    Priority recommendation

    Comments: DOD concurred with our recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within the Navy, the Secretary of the Navy should direct its strategic sourcing accountable official to implement utilization metrics and monitor agency efforts to comply with the Navy's existing use policies for IT services.

    Agency: Department of Defense: Department of the Navy
    Status: Open

    Comments: DOD concurred with our recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within the Air Force, the Secretary of the Air Force should direct its strategic sourcing accountable to conduct a comprehensive analysis of IT services spending to determine the extent to which requirements can be addressed by Network-Centric Solutions (NETCENTS) or other strategic sourcing approaches, and based on this analysis, reduce duplicative contracts.

    Agency: Department of Defense: Department of the Air Force
    Status: Open
    Priority recommendation

    Comments: DOD concurred with our three recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within the Air Force, the Secretary of the Air Force should direct its strategic sourcing accountable to implement utilization metrics.

    Agency: Department of Defense: Department of the Air Force
    Status: Open

    Comments: DOD concurred with our recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within the Air Force, the Secretary of the Air Force should direct its strategic sourcing accountable to develop guidance and overarching goals and metrics for savings.

    Agency: Department of Defense: Department of the Air Force
    Status: Open

    Comments: DOD concurred with our recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within the Air Force, the Secretary of the Air Force should direct its strategic sourcing accountable to conduct a review of the benefits and disadvantages of standardized labor categories for primary strategic sourcing vehicles such as NETCENTS.

    Agency: Department of Defense: Department of the Air Force
    Status: Open

    Comments: DOD concurred with our recommendations. To fully implement these recommendations, DOD should complete analyses of information technology services spending, reduce duplicative contracts where appropriate, and establish metrics to monitor progress and assess compliance with existing IT services use policies.
    Recommendation: To improve efforts to strategically source IT services within NASA, the Administrator of NASA should direct its strategic sourcing accountable official to use its 2014 spend analysis to determine the extent to which requirements can be addressed by the IT Infrastructure Integration Program or other strategic sourcing approaches, and based on this analysis, reduce duplicative contracts.

    Agency: National Aeronautics and Space Administration
    Status: Open
    Priority recommendation

    Comments: NASA agreed with this recommendation. To fully implement it, NASA needs to successfully implement its planned actions, including (1) implement new strategic sourcing policies in the NASA federal acquisition regulation supplement, (2) revise the 2014 spend analysis by December 14, 2017, and (3) require strategic sourcing of IT services by December 2018 for services such as mobile communications, telecommunications, cloud computing, and seat management.
    Recommendation: To improve efforts to strategically source IT services within NASA, the Administrator of NASA should direct its strategic sourcing accountable official to implement utilization metrics and mandatory use policies.

    Agency: National Aeronautics and Space Administration
    Status: Open
    Priority recommendation

    Comments: NASA partially agreed with this recommendation. Specifically, NASA agreed to establish metrics, but sought to employ mandatory consideration policies, where applicable, instead of mandatory use policies. We agreed that the proposed approach would meet the intent of our recommendation. To fully implement this recommendation, NASA needs to successfully implement its planned actions, including (1) revising the NASA strategic sourcing guide to include establishment of utilization metrics, and (2) issuing updated strategic sourcing policies in the NASA federal acquisition regulation supplement to include mandatory use policies.
    Recommendation: To improve efforts to strategically source IT services within NASA, the Administrator of NASA should direct its strategic sourcing accountable official to develop guidance and overarching goals and metrics for savings.

    Agency: National Aeronautics and Space Administration
    Status: Open
    Priority recommendation

    Comments: NASA agreed with this recommendation. To fully implement this recommendation, NASA needs to successfully implement its planned actions, including (1) issuing updated strategic sourcing policies in the NASA federal acquisition regulation supplement, (2) updating its strategic sourcing website, and (3) updating the NASA strategic sourcing guide to include the setting of goals or baselines as a method of evaluating the strategic sourcing approach.