In the previous section, we learned how to break down the cloud forecasting challenge into addressable parts and went over its essential components. Now we are going to look at how to align the forecasting methodologies with the practice maturity.

The forecast frequency, accuracy, and the models used will depend on the maturity phase of your organization and may differ by cloud providers. A reasonable first step for smaller workloads is trend-based forecasting, with exponential smoothing providing good results whether or not trend or seasonality is available. To incorporate trend and seasonality Holt's exponential smoothing provides reasonably accurate results. We are going to dive into these in a moment.

Driver-based forecasting is more involved but produces better results for larger or more mature workloads. Here the business forecasts Key Performance Indicators, or KPIs, taking into account organic growth, like more people on the Internet, and inorganic growth, like new features and marketing campaigns. These KPIs are then applied to actuals to forecast future cloud spend.

Rate reduction, cost avoidance, and future workloads will need to be layered on top of existing cloud forecasts. A good starting point is to layer these on top of forecasts using spreadsheets. This process can be automated as the practice matures.

Using Exponential Smoothing for cloud forecasting where there is no trend or seasonality.

Let's get started on how to use simple exponential smoothing to forecast cloud spend where we do not have historic trend or seasonality available. Here we use the following formula: ( F sub t ) equals ( alpha ) times ( A sub t minus one ) plus ( one minus alpha ) times ( F sub t minus one ). Let's go over the components of the formula. ( F sub t ) is the forecast for month t. 

Alpha is the smoothing constant you choose that works best for your organization. Its value is a floating-point number between zero and one. A smaller number has a greater smoothing effect but is less responsive to recent changes, while a larger number gives greater weight to recent changes but has less of a smoothing effect. To get started use zero point five and experiment from there.

 ( A sub t minus one ) is the previous month's actual cloud cost. And ( F sub t minus one ) is the previous month's forecasted value. If the previous month has no forecasted value use the previous month's actual cloud cost to get the iteration started.

Using Holt's Exponential Smoothing for cloud forecasting if trend or seasonality is available

Now let's look at how we can incorporate trend and seasonality using Holt's exponential smoothing. To do so we need to introduce the concepts of level and trend. Level is the average value in a series while trend is the increasing or decreasing value in a series.

To calculate the level we use the following formula: ( L sub t ) equals ( alpha ) times ( A sub t ) plus ( one minus alpha ) times ( L sub t minus one ) plus ( T sub t minus one ). Let's go over the components of the formula. ( L sub t ) is the level for month t.

Alpha is the smoothing constant used for leveling. Its value is a floating-point number between zero and one. ( A sub t ) is the actual cloud cost for month t. ( L sub t minus one ) is the previous month's level value. And ( T sub t minus one ) is the previous month's trend value.

To calculate the trend we use the following formula: ( T sub t ) equals ( beta ) times ( L sub t ) minus ( L sub t minus one ) plus ( one minus beta ) times ( T sub t minus one ). Let's go over the components of the formula. ( T sub t ) is the trend for month t.

Beta is the smoothing constant used for trending. Its value is a floating-point number between zero and one. ( L sub t ) is the level for month t and ( L sub t minus one ) is the level for the previous month. Lastly ( T sub t minus one ) is the previous month's trend.

To calculate our cloud forecast we use the following formula: ( F sub t ) equals ( L sub t ) plus ( T sub t ) where ( F sub t ) is our forecast for month t.

The smoothing constants can be dynamically adjusted based on historic trends or where seasonality is expected. For example, a larger number can be used when cost spikes are anticipated.

Holt's exponential smoothing provides a good starting point to forecast trend and seasonality. More advanced methods exist like Holt Winters triple exponential smoothing where a third equation is introduced to calculate seasonality separately. Many open-source packages exist in various programming languages we can leverage in our cloud forecasting journey.

Driver-based forecasting.

Now let's switch gears and look at driver-based forecasting. We need to work with the business to define drivers like for example active users, widgets sold, or website visits. The business then needs to build methodologies to forecast these business drivers similar to what we went over previously.

Then we leverage existing or introduce new tagging of cloud resources and map these to business drivers. For example, a database that grows with active users or a web application that grows with website visits. We have more flexibility when we keep the mapping separate from the tagging, for example in a spreadsheet or database, as changing tags requires engineering involvement.

To forecast cloud spending we apply the growth of the business drivers to actual cloud cost based on the driver mapping. For example, if a cloud workload is mapped to active accounts and the driver predicts a five percent growth next month, we take the current month's actual cost for that specific workload and increase it by five percent to forecast next month's cloud cost.

Driver-based forecasting generally produces more accurate results, however not all workloads grow by a floating-point number. For example, to grow a database we can increase the number of virtual machines or use larger virtual machines. A database can't grow by an arbitrary number.

To identify gaps in our forecasting methodology, we need to regularly compare actuals to forecasts. When reaching out to workload owners to identify root causes for substantially over or under performing against their targets, we will also learn use cases for which the forecasting algorithm does not perform well. This will require short-term adjustments or building additional functionality in the forecasting algorithm to account for these corner cases.

Static vs Rolling Forecast.

Now let's look at static forecasts versus rolling forecasts. Static forecasts predict for the fiscal year only with no adjustments being made at a later time. Rolling forecasts predict a window of time and can be adjusted based on shifts in the business such as economic changes. As the business changes, a rolling forecast is adjusted to forecast that change and allow leadership to alter their plans using new data.

Special Projects.

Lastly, in this section let's talk about so-called special projects. The term special project covers any inorganic growth such as new features, products, or countries being released.

Impact to cloud cost needs to be estimated for each special project on a monthly basis and the effect may differ from month to month. For example, a marketing campaign may have a slow ramp up for a few months, experience a peak over several more months, and then the effect may taper off quickly after its completion.

The owner of the special project needs to provide a monthly estimate of the cloud cost. The effects of each special project need to be aggregated on a monthly basis and layered on top of the existing forecast methodology to produce more accurate cloud forecasts.

Let's look at an example where we identified three special projects affecting next month's cloud cost. The first is a rate reduction change of five thousand Dollars due to a large reservation purchase. The second is a waste reduction effort where a legacy workload is terminated resulting in a three thousand Dollar per month saving. And the third is a product launch adding a new cloud workload estimated at a ten thousand Dollar monthly cost. When we add up these three special projects we get a net increase of two thousand Dollars in cloud cost for the next month.

Our existing forecasting algorithm is unable to predict these changes as these are new efforts that do not exist in historic data.

To summarize "Align forecasting methodologies with practice maturity", we learned that forecast frequency, accuracy, and models need to be aligned with the maturity of our organization. Exponential smoothing is a good first step when no trend or seasonality is available while Holt's exponential smoothing can be used otherwise. Driver-based forecasting is more advanced as it requires the business to forecast growth drivers but produces more accurate results. Special Projects are future efforts that need to be layered on top of existing cloud forecasts.