Cloud vendors are quick to tout the benefits offered by on demand IT compared to traditional on premise IT service models. Their marketing materials offer many examples of the benefits realized from moving to the cloud for increased service agility, efficiency, and cost savings. The lure of cost savings generated by moving from a capital expenditure to operating expense based, on-demand service is often cited by IT leaders as a key factor in their decision to move to cloud service support.

Indeed many federal agencies have committed significant dollars to the promise of cloud efficiencies and IT cost savings. As of 2014, it is estimated that cloud investments represent 5% of total IT spending, the equivalent of approximately $3B annually (IDC Press Release from 9/16/14).

In the rush to gain on demand IT efficiencies and savings and meet federal mandates such as the Cloud First Initiative, many have forgone careful planning to ensure their goals are met and promises from cloud vendors are delivered. Unfortunately, failure of cloud projects is not uncommon and often federal timelines for realizing benefits from cloud services are overly optimistic. To date, OMB auditors have issued over 747 recommendations as of January 2015 to address the trend of failing IT acquisitions. OMB has confirmed only ~23% have been fully addressed to date (Next Gov, 2/11/15).

In addition to addressing audit recommendations, agencies also need objective insight to better evaluate proposals and contract terms from cloud vendors. Without independent outside analysis, it is unlikely agencies will be able to establish a criteria for identification of best value or effective control and oversight mechanisms. The future of cloud acquisition will remain bleak unless organizations begin to address the root issues of evaluating cloud services and match rigorous independent analytical review with cloud acquisition planning practices.

The heart of the issue lies in the source of most cloud savings and benefit data: the cloud service provider (CSP). Since cloud services are still relatively new to the federal government, there are few independent benchmarks to establish how quickly or effectively cloud services benefits should be realized in a mission environment. As a result, agencies are dependent on cost and performance projections from competing CSPs to set targets for their service performance. This dependency gives the advantage to the CSP in evaluation of proposals and negotiation of cloud contract terms and conditions. Further, it incentivizes CSPs to bundle their prices in such a way that make it difficult for the government to compare costs or determine relative value of CSP proposals. The inevitable outcome of this dependency is that government leadership is at a disadvantage in evaluating the performance of cloud services in their mission environment.

Define True Benefits in Mission and Business Context

Benefits defined against on premise to off premise IT movements can offer valuable insight into the cost savings resulting from adding virtualized IT capacity, but does not reflect the true value of cloud services to an enterprise. This shortcoming can be attributed to the inability of traditional key performance indicators (KPI) to show downstream value of virtualized IT services to meet user needs. Often performance objectives are limited to KPIs that compare traditional IT technical measures with cloud solutions (Figure1).

If a cloud service customer only uses traditional IT KPIs to evaluate cloud performance, they are limiting themselves in three key areas:

• Myopic lifecycle: A cloud service customer will only see benefit of implementing cloud services against legacy data center costs, missing the evaluation of post-implementation performance in the IT operations environment.

• Overlooking efficiency and productivity: Focusing on the savings against legacy costs overlooks the largest potential gain from cloud service: increasing the efficiency of IT service delivery (match capacity to demand) and productivity gains resulting from efficient delivery.

• Missing service usage: Traditional IT performance measures overlook adoption as a key factor in the ability of an enterprise to meet performance goals. If new cloud services are not used, benefits will not be realized.

Although performance measures related to availability, throughput, load size, and memory distribution are important for comparing initial savings compared to legacy IT, they do not provide enough information to understand progress relative to long term business and mission goals.

What is missing is a consideration for cloud economics to complement traditional IT measures with insight into cost and value created in the short term and long-term by evaluating factors as shown in Figure 2.

The additional layer of insight into the return of cloud services in context of mission will enable decision makers to better evaluate whether resources expended and budgetary dollars spent (service level agreements, contracts, and direct costs) are producing planned returns, savings, and benefits. Also, it will give decision makers the ability to objectively evaluate whether their cloud deployment model is putting the right emphasis on risk mitigation and control functions to achieve planned results.

Planning to Leap: Gaining Performance Advantage by Considering Cloud Economics

Government executives are under great pressure to achieve cost savings using cloud solutions. In the rush to implement, many may lose a critical opportunity to set terms and conditions that allow them to establish independent, objective performance targets and ultimately manage expectations for what can be achieved using cloud on demand services in their enterprise. There is great value in taking the time to carefully plan and assess requirements and needs of the organization and the potential scope of change required to fully integrate cloud functionality into IT operations. Taking the time to plan should also include a full assessment of the costs and benefits throughout the IT service lifecycle so that cloud customers can understand the independent, objective value of cloud service to their endusers and mission without dependence on service providers for performance data.