How to predict growth areas and risks in a marketing plan based on data

We recently announced a new product — OWOX BI Insights — that’s able to predict the realization of marketing goals based on your data, overall market data, and machine learning. It gives marketers answers to important questions:

• Will we fulfill the plan for sales, leads, and other indicators this month?
• If we don’t change anything, how many sales, leads, etc. will we get by the end of the year?
• In which channels, regions, and segments are the goals unreached, and where are the key areas for growth?
• How are we doing relative to the market? Has the market declined or are we just doing poorly?

 In this article, we tell you why to build forecasts and how to find growth zones and risks in order to be one step ahead of your competitors.

Order your forecast now!

Marketing analytics play an important role in modern businesses.

In 2018, the average share of a company’s marketing budget spent on marketing technologies (29%) for the first time exceeded the share spent on the people who use these technologies (24%).

If the company’s growth forecast isn’t fulfilled, most managers will first of all dismiss their marketing director. Therefore, CMOs should be very interested in introducing technologies and looking for better solutions.

The majority of CMOs (78%) have increased their ROI by using analytics when forming a marketing strategy.

Here’s a short story of how it all began a few years ago and how we came to realize that our clients needed forecasts and insights.

OWOX BI: Would you like us to set up Google Analytics?
Customer: Sure, but we can’t see ROI in Google Analytics for our Facebook and Twitter marketing campaigns and can’t compare their effectiveness.
OWOX BI: Here’s automated data import. Just add access to your advertising services and all data will automatically be imported into Google Analytics. You’ll get ROI for all ad campaigns.
Customer: Great, but the data in Google Analytics doesn’t match what’s in our CRM. Our management believes the data in the CRM, and we can’t reprocess Google Analytics data to ensure it’s correct. We also can’t combine this data with internal data retroactively, especially not without query time import.
OWOX BI: Okay, here’s all your data in Google BigQuery. We’ve collected it from Google Analytics, advertising services, call tracking, and mobile applications. Now you can combine it with data from your CRM and build any reports.
Client: Awesome! But we need to find underestimated campaigns in all of this. Using a simple SQL query doesn’t help because users make a number of clicks along the path to a purchase. We need to somehow attribute the value of all these steps, which means we need to hire developers.
OWOX BI: Agreed. How about ML funnel-based attribution and ready-made reports? Just specify these steps and you’ll see undervalued campaigns.
Client: Can you possibly identify our growth zones in all this data? Just say where we should look to fulfill the sales plan.
OWOX BI: Soooo... Let’s discuss what you expect from your marketing analyst.

1. Shows where the marketing goals are fulfilled and unfulfilled.
During a meeting at the beginning of each month, the manager asks the marketing director: «Why hasn’t it been fulfilled?» Which region or advertising channel did we have? " And hears in response: «I do not know yet. I need a couple of days — I’ll see, come back and tell. » Common situation?

We decided that enough is enough — let the whole process be automated. Everything is simple here: we take retrospective data, that is, a fact, and compare it with the planned one. It is important that this is done automatically for different segments.

2. Shows how you will fulfill the annual planned goals if you go further at the same pace as now.
If you find out in advance that the marketing team falls short in a category or region, you can fix your marketing strategy quickly and improve the results. In order for our clients to have such an opportunity, we take their data and build a forecasting model in the horizon up to a year. Here the forecast is compared with the plan.

3. Finds growth areas and risks for the next month.
To do this, OWOX BI Insights automatically compares the marketing goals and forecast at the level of regions, channels, parameters and within the areas of responsibility of each decision maker in the company.

4. Helps to answer the question: «Is this what we are not working on, or has the market subsided?».
This is always an open and relevant question in any company, because it’s not enough to answer its own data. For example, if the plan is exceeded, it is interesting to know: these are marketers — well done or something happened on the market and everyone ran to spend money. Or, let’s say, the goals were not fulfilled by 2%. Is this a good or bad result for a marketing team? If the market has grown, then it is bad, and if it has sunk by 15%, then it is good.

Market data is critical to answer these questions. That’s why we’re building a machine learning model based on data from tens of thousands of projects to provide better forecasts than you could achieve using only your own historical data.

5. Creates reports in the usual and convenient services for you.
Our experience shows that when it comes to a new product, most companies do not want to introduce another service. You may be thinking: «I have Excel (or Google Sheets, Data Studio, Tableau, Power BI). Can I add a forecast there? » Of course you can, because almost any visualization service can get data from Google BigQuery, where we save the results of calculations.

To begin with, let’s look at the final reports. Then we’ll tell you how they’re made.

The first and simplest thing is to get three columns for each month: Plan, Actual, and Forecast.

Making the plan is a simple matter. It can be made by any analyst in Excel. We can also calculate the actual values, as the business has this data. Our task is to add a forecast.

Note: The report contains data for the beginning of April. As you can see, we have a small actual value (the month has just begun) and there is some kind of forecast for the rest of April. Accordingly, for the current month, the plan should be compared with the sum of the actual and forecast values.

In the graph above, the yellow line is the goals that the client has set, the dark blue columns are the actual values that have already been fulfilled, and the light blue columns are forecasts.

Once we know the goals, the forecast, and the actual values in the context of the parameters needed for making decisions, we can build a classic pivot table in Google Sheets.

There are three parameters in this table: reporting month, channel group (organic search, marketplace, email, etc.), and region. This is the basic grouping with which most businesses work. Next, we show the key metrics that are needed to manage the marketing budget:

From the report, we see that paid search in Region 1 is likely to generate less revenue than planned. We also see that the email channel may lag behind transactions. This information allows you to make decisions before your budget is suboptimally spent — and before the growth zone is in the past and is lost.

And the best thing is that we have a forecast for the future. It shows what will happen if the current trend continues.

Let’s look in detail at some of the elements in this table. The percentage deviation, which is highlighted in red and green, is calculated using a simple formula:

We take the fact with the forecast, that is what we will most likely receive at the end of this month. Subtract from this the goals for the segment and divide the result by the plan for the period. The resulting number answers the question to what extent a particular segment deviates from the forecast.

For example, for April transactions we see a deviation of −4.8%:

This means that in April, we’re 4.8% behind the transaction plan. If you add up the values ​​for all channels above this figure, the total will be −4.8%.

Why do we look at this? Why don’t we look at relative deviation? For example, say we have a small Referral channel. According to the plan, there should be, say, 50 transactions. A difference of even 20 transactions will give us a 40% deviation. But this isn’t what you need to pay attention to in the first place, because there are much larger channels. And if a larger channel shows a deviation of 10%, it will be much more important for the business. The red and green colors in the table show how important it is for the business to pay attention to this segment.

In the screenshot below, we’ve expanded the data for April:

Here we see that data can be decomposed into channels and regions. We can select those segments that we want to look at first. Red is a risk zone, green is a growth zone. The conversion rate is marked in blue, and at the bottom there’s a summary.

Now we’ll introduce the most important and interesting report, which shows eight time periods that are used to compare the forecast with the goals.

1. This Year Actual shows the actual figures from the beginning of the year to the current moment. We can compare these values with the goals to understand how well we’re meeting them.
2. Last Month Actual shows actual data for the past month. We closed the previous month, so we can easily see in each section and segment whether the actual numbers coincide with the planned ones.
3. Last Week Actual shows actual data for the past week.
4. This Week Forecast shows the forecast for the current week. The week has begun, and we see what happens to us. Where the person responsible, for example, for paid search, pay attention right now, to achieve a better result at the end of the week. To do this, you just need to avoid risks (highlighted in red) and implement growth zones (highlighted in green).
5. Next Week Forecast shows the forecast for the next week.
6. This Month Forecast shows forecast for the current month.
7. Next Month Forecast shows forecast for the next month. In our experience, it’s better to build this forecast only based on the Primary Dimension. That is, splitting it up to the second level (in our case, regions) is no longer advisable because there’s a lot of noise and poor forecasting quality.
8. This Year Forecast shows the forecast through the end of the year. We build it only for the key metric, income. Experience shows that predicting the distribution of traffic across regions and channels and other segments before the end of the year is not a fascinating exercise and leads to large errors and loss of confidence in forecasts.
   

You can view the table in Google Sheets at any time. However, decision-makers usually expect to receive a ready-made report and say, Look here. It seems we have a growth zone.

Creating this report isn’t an easy task, and it’s one we’re still working on. We want our customers to get clear, meaningful messages written in human language just as a good analyst would write them.

There are two problems. The first is a lot of nuances. Crying wolf about unmanaged direct channels isn’t very useful. What should you do if Referral or Direct have fallen? You need to focus on what you can manage, and for this, you need more knowledge for the program when searching for insights.

The second problem is that these emails might be answered with: «Give me more detail.» But noreply@owox.com never planned to enter into a conversation. We do believe that communication from an analytical service should be as similar as possible to communication with a marketing analyst, however. Therefore, we plan to send insights on growth zones and risks via messenger in the future.

At the end of the email, there’s an interesting table in which the results for the main periods are highlighted in red and green. This immediately shows whether it’s necessary to open Google Sheets and look at the details for this period.

Before you start to build forecasts, you need to answer a few important questions:

As a result of this implementation, you’ll get a table like this:

The first column indicates the area of ​​responsibility. In our example, there are three: marketing in general, paid PPC campaigns, and SEO (organic).

The second column is KPIs, which are used for setting goals and evaluating performance in each of the areas of responsibility. For some of these indicators, there’s a plan. We recommend that you not change your existing goals when you decide to take on predictions. It’s clear that in business you can always improve something, and you may want to add new indicators to your reports. But you can’t do such things on the move. First, it’s better to automate what you already have, then add new metrics.

In the third and fourth columns, we mark the metrics for which there’s a plan and for which we can build a forecast.

In the fifth column, we mark the quantitative and qualitative metrics because they need to be displayed differently and should be treated differently.

The sixth column is the segmentation criteria. In different areas of responsibility, the same metric is segmented by different criteria. For example, marketing sessions are more interesting to segment by channel; SEO should be segmented by search engine.

The seventh column is the secondary dimension. There may be more than two dimensions, but in that case the requirements for the volume of data should be higher. The more segmentation parameters, the more data is needed for statistically significant forecasts.

The last column is the number of values ​​for a particular parameter that you want to see in the report. Why do you need this? If you didn’t specify a limit by region and if you expanded the list, for example, you would have not five regions but five hundred. Many of them would be too small and most likely not informative. We recommend adding limits so this entire label can somehow fit into Google Sheets and work with it.

So you’ve built a table like this and collected all your data (planned and actual) in Google BigQuery (or some other cloud storage). What should we do next? The first and easiest thing to do is contact OWOX BI. Our AI algorithm will analyze your data, compare it with market data, make predictions, and show where your growth zones and risks are.

If you want to build forecasts by yourself, there are several ways you can do so. There are several ways. We’ll share how to do it in Google Cloud, as this platform is most familiar to us. Clearly you may use other tools in your company.

There are three basic ways to build forecasts in Google Cloud:

1. TensorFlow and CloudML are the methods most often used by Data Scientists who aren’t too lazy to spend time customizing models. These are difficult tools, but they allow you to achieve good results.
2. BigQuery ML — If you’ve never worked with machine learning but you already have data in Google BigQuery, we recommend starting with this method. If you have data from Google Analytics 360, you can already train a model, for example to calculate the probability of conversion, churn, or any other parameter. It’s very easy to choose predictors, features, and test models. This is a smart way for an analyst who’s familiar with SQL.
3. The AutoML and CloudML APIs are the most basic ways for developers. They make it easier to deploy models, compare them, and roll back versioning. This is ideal if you’re a developer and your task is to roll out the model that you built in production.

This is a large table with data from which you can easily calculate the deviation, conversion because you have all the components for this and they’re grouped into three blocks: plan, actual, and forecast.

1. If you’re told «We have marketing goals,» at best this means an Excel spreadsheet that an analyst manually updates once a week.
2. Automated forecasting doesn’t replace analytics — it’s just a way to increase its efficiency. In our experience, manual adjustments are always needed. For example, say that in October we open a large store in Houston. The forecasting model will never account for this because it doesn’t have this knowledge.
3. The quality of forecasts is significantly improved if, in addition to your own data, you use market data. This is what makes OWOX BI’s approach to forecasting qualitatively different. We use data from tens of thousands of projects to train our model. As a result, it shows market trends more accurately. For example, we know the market share of organic search and paid search for projects in the same niche as our client’s, and we know what these numbers lead to. Enriching the model with this data allows you to better understand trends and know whether you’re doing great or the market is just growing.
4. It’s necessary to separate calculations from data visualization. When you’ve formulated expectations for accuracy and depth, you can build a forecast and calculate data in a strictly allotted time. As for data visualization, this is an endless task, and there’s no limit to perfection.
5. To get high-quality forecasts with a granularity of two parameters, you need website data on 3 million sessions per month. It’s possible to build a forecast with a smaller amount without segmentation, but the question is why. If you see, for example, that the goals aren’t being realized, can you say in what segment it isn’t working? What regions? You most likely won’t find the answer because there’s little data.

If you have historical data for two years from Google Analytics, traffic from 3 million sessions per month, and want to fulfill your marketing goals and grow faster thanks to insights, write to us by filling out a form on our site. We’ll discuss the details with you and help you build a forecast.