Cloud Computing:

Agencies Need to Incorporate Key Practices to Ensure Effective Performance

GAO-16-325: Published: Apr 7, 2016. Publicly Released: Apr 7, 2016.

Additional Materials:

Contact:

David A. Powner
(202) 512-9286
pownerd@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Federal and private sector guidance highlights the importance of federal agencies using a service level agreement (SLA) in a contract when acquiring information technology (IT) services through a cloud computing services provider. An SLA defines the level of service and performance expected from a provider, how that performance will be measured, and what enforcement mechanisms will be used to ensure the specified performance levels are achieved. GAO identified ten key practices to be included in an SLA, such as identifying the roles and responsibilities of major stakeholders, defining performance objectives, and specifying security metrics. The key practices, if properly implemented, can help agencies ensure services are performed effectively, efficiently, and securely. Under the direction of the Office of Management and Budget (OMB), guidance issued to agencies in February 2012 included seven of the ten key practices described in this report that could help agencies ensure the effectiveness of their cloud services contracts.

GAO determined that the five agencies and the 21 cloud service contracts it reviewed had included a majority of the ten key practices. Specifically, of the 21 cloud service contracts reviewed from the Departments of Defense, Health and Human Services, Homeland Security, Treasury, and Veterans Affairs, 7 had fulfilled all 10 of the key practices, as illustrated in the figure. The remaining 13 contracts had incorporated 5 or more of the 10 key practices and 1 had not included any practices.

Figure 1: Number of Cloud Service Contracts That Met All 10 Key Practices

Figure 1: Number of Cloud Service Contracts That Met All 10 Key Practices

Agency officials gave several reasons for why they did not include all elements of the key practices into their cloud service contracts, including that guidance directing the use of such practices had not been created when the cloud services were acquired. Unless agencies fully implement SLA key practices into their SLAs, they may not be able to adequately measure the performance of the services, and, therefore, may not be able to effectively hold the contractors accountable when performance falls short.

Why GAO Did This Study

Cloud computing is a means for delivering computing services via IT networks. When executed effectively, cloud-based services can allow agencies to pay for only the IT services used, thus paying less for more services. An important element of acquiring cloud services is a service level agreement that specifies, among other things, what services a cloud provider is to perform and at what level.

GAO was asked to examine federal agencies' use of SLAs. GAO's objectives were to (1) identify key practices in cloud computing SLAs and (2) determine the extent to which federal agencies have incorporated such practices into their SLAs. GAO analyzed research, studies, and guidance developed by federal and private entities to develop a list of key practices to be included in SLAs. GAO validated its list with the entities, including OMB, and analyzed 21 cloud service contracts and related documentation of five agencies (with the largest fiscal year 2015 IT budgets) against the key practices to identify any variances, their causes, and impacts.

What GAO Recommends

GAO recommends that OMB include all ten key practices in future guidance to agencies and that Defense, Health and Human Services, Homeland Security, Treasury, and Veterans Affairs implement SLA guidance and incorporate applicable key practices into their SLAs. In commenting on a draft of this report, OMB and one agency had no comment, the remaining four agencies concurred with GAO's recommendations.

For more information, contact David A. Powner at (202) 512-9286 or pownerd@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: As of April 2018, a staff member from the Office of Management and Budget (OMB) reported that OMB has not yet taken action to address our recommendation and did not provide any timeframes for when the recommendation might be addressed. We will continue to monitor the agency's progress in implementing this recommendation.

    Recommendation: To ensure that agencies are provided with more complete guidance for contracts for cloud computing services, the Director of OMB should include all ten key practices in future guidance to agencies.

    Agency Affected: Executive Office of the President: Office of Management and Budget

  2. Status: Closed - Implemented

    Comments: The Department of Defense (Defense) agreed with our recommendation and stated that the department would update its cloud computing guidance and contracting guidance. In August 2017, Defense finalized its updated guidance on service level agreements in its Defense Acquisition Guidebook, which incorporates the key practices identified in our report. For example, the Guidebook specifying the roles and responsibilities of all parties involved in the agreement, including department personnel and service provider staff; defining clear measures for performance by the cloud service provider, such as service availability and response time; and providing how data and networks are to be managed and maintained. The guidance also identifies that enforceable consequences, such as penalties, in the case of non-compliance with the performance measures should be included. Subsequently, in May 2018, Defense provided evidence that the guidance has been incorporated into cloud service contracts. For example, a review of the department's contract documentation for milCloud found that language was included that the contractor would demonstrate that they complied with sections of the service level agreement and that the cloud provider would maintain the cloud environment in accordance with applicable department polices and guidance. By updating its guidance to address these key practices, Defense is better positioned to more effectively measure the performance of the services it receives, and, therefore, more likely to ensure the delivery and effective implementation of services for which it has contracted.

    Recommendation: To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretary of Defense should direct the appropriate officials to ensure key practices are fully incorporated for cloud services as the contracts and associated SLAs expire. These efforts should include updating the Department of Defense memorandum on acquiring cloud services and current Defense Acquisition Regulations System to more completely include the key practices.

    Agency Affected: Department of Defense

  3. Status: Closed - Implemented

    Comments: The Department of Homeland Security (DHS) agreed with our recommendation and stated that the department would establish common cloud computing service level agreement guidance. In August 2017, DHS finalized its service level agreement template, which provides a consistent format for agreements between DHS and cloud service providers and incorporates the key practices identified in our report. For example, the template ensures the delivery and effective implementation of services by specifying the roles and responsibilities of all parties involved in the agreement, including department personnel and service provider staff; defining clear measures for performance by the contractor, such as service availability and quality; and providing planning and testing for disaster recovery and continuity of operations. The template also identifies a range of enforceable consequences, such as penalties, in the case of non-compliance with the performance measures by the contractors. In December 2017, DHS' Office of the Chief Procurement Officer notified all the heads of contracting activities and component acquisition executives that the template had been added to the Homeland Security Acquisition Manual, and requested that acquisition staff and financial personnel be notified of the change as well. Subsequently, in February 2018, DHS provided evidence that the template has been incorporated into cloud service contracts. For example, Immigrations and Customs Enforcement confirmed that the contractor's service level agreement for a new contract was aligned with the DHS template, and that proper parameters and commitments are in place to ensure that the DHS requirements are met. By finalizing a standard service level agreement to address these key practices, DHS is better positioned to more effectively measure the performance of the services it receives, and, therefore, more likely to ensure the delivery and effective implementation of services for which it has contracted.

    Recommendation: To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretaries of Health and Human Services, Homeland Security, Treasury, and Veterans Affairs should direct appropriate officials to develop SLA guidance and ensure key practices are fully incorporated as the contract and associated SLAs expire.

    Agency Affected: Department of Homeland Security

  4. Status: Open

    Comments: The Department of Health and Human Services (HHS) has taken initial steps to address our recommendation. As of March 2018, according to a HHS official, HHS is developing a cloud computing service level agreement, which it anticipates completing by the end of December 2018. We will continue to evaluate the department's progress in implementing this recommendation.

    Recommendation: To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretaries of Health and Human Services, Homeland Security, Treasury, and Veterans Affairs should direct appropriate officials to develop SLA guidance and ensure key practices are fully incorporated as the contract and associated SLAs expire.

    Agency Affected: Department of Health and Human Services

  5. Status: Open

    Comments: The Department of the Treasury (Treasury) has taken initial steps to address our recommendation. As of February 2018, according to a Treasury official, Treasury stakeholders, including the Office of the Chief Information Officer and the Treasury Procurement Office, are in the process of reviewing the Federal Acquisition Regulation and may have a cloud computing service level agreement guidance policy by the end of December 2018. We will continue to evaluate the department's progress in implementing this recommendation.

    Recommendation: To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretaries of Health and Human Services, Homeland Security, Treasury, and Veterans Affairs should direct appropriate officials to develop SLA guidance and ensure key practices are fully incorporated as the contract and associated SLAs expire.

    Agency Affected: Department of the Treasury

  6. Status: Open

    Comments: As of March 2018, according to a VA official, the department is working to address our recommendation but did not identify timeframes for when all activities would be completed. We will continue to evaluate the department's progress in implementing this recommendation.

    Recommendation: To help ensure continued progress in the implementation of effective cloud computing SLAs, the Secretaries of Health and Human Services, Homeland Security, Treasury, and Veterans Affairs should direct appropriate officials to develop SLA guidance and ensure key practices are fully incorporated as the contract and associated SLAs expire.

    Agency Affected: Department of Veterans Affairs

 

Explore the full database of GAO's Open Recommendations »

Aug 2, 2018

Jun 13, 2018

May 24, 2018

May 23, 2018

May 22, 2018

Mar 14, 2018

Jan 30, 2018

Jan 10, 2018

Nov 21, 2017

Looking for more? Browse all our products here