Optimize 360 proves to be accurate no-coding solution for testing user experience on your site

Goal

OWOX specializes in reselling and supporting Analytics 360, along with developing and implementing business intelligence services which are based on Google Cloud Platform. We also organize and conduct the largest conferences on web analytics in Ukraine and other countries. One of the key sources of lead generation is the company’s website. We are constantly working to improve the elements of the website, including content, structure, design and lead generation forms. These are the elements we apply to get users engaged in our services and to increase conversion rate. As a rule, a large number of hypotheses has to be tested within a tight time frame and requires constant involvement of several developers. Due to this, we had to give up on some experiments. The other experiments were either delayed or their results had little value. We needed a testing tool which would meet the following requirements:

1. Minimal developer involvement
2. Ease of implementation
3. Integration with Analytics 360, in order to:
   • create reports of any complexity in Analytics 360 interface;
   • enable targeting for experiments using Analytics 360 data.

Solution

In order to test a number of hypotheses, we developed a Test Plan under which we ran four experiments. The full cycle for each experiment was comprised of the following stages:

1. Defining the problem
2. Determining the hypothesis
3. Setting up and running the experiment
   • Developing the alternatives
   • Targeting
   • Running the experiment
4. Determining the best option
5. Implementing the best option
6. Measuring the results

We used Optimize 360 and Analytics 360 to run the experiments, test, and analyze our assumptions.

Following the development and coordination of the test plan, we performed the installation and configuration of Optimize 360 with the following actions:

1. Creating an Optimize account;
2. Placing the Optimize code on all pages of the website in the beginning of the tag, right after the Data Layer variable as you can see.

   

3. Linking Optimize 360 and Analytics 360 accounts.

At the second stage, we set up experiments for each hypothesis:

1. Changing the design of the Request Demo button. In this case, we used A/B testing, with the alternatives created in Optimize Visual Editor without involving developers:

   

   The efficiency of the alternatives was tested by the number of clicks on the Request Demo button.
2. Adaptive website design. The adaptive variation of the website was tested using Redirect test. As you can see on the screenshot below, we evaluated the following data: Pageviews, Session Duration, Bounce Rate, and also an Analytics 360 Goal measuring the number of specific actions of the website.

   

   To ensure objective results, one of the conditions for the targeting was to include only users who visit the site on a tablet or mobile phone.

   

3. Additional Request Demo button. In this case, we also used A/B testing, with the alternative page created using Optimize Visual Editor without the help of the developers. In this experiment, the evaluation criteria were the numbers of open and successfully submitted Request Demo forms. The extra Request Demo button broke the adaptive layout, so we needed to exclude users of the adaptive design by adding a condition in the Targeting tab. Since the adaptive website layout depended on the screen resolution, and not on the type of the device, we needed to create a JavaScript variable. With this variable, we determined the need for user participation in the experiment.

   

   Custom JavaScript variable clientWidth was used to determine the website layout.
4. The new version of the catalogue page with a description of OWOX BI products. In this experiment we used the Redirect test. The success rate of the new page was determined by the number of transitions made to a particular product page, bounce rate and exit rate. This experiment looked at participants who were new users or users who had never seen the old catalogue page before:

   

   To achieve this, we created a segment in Analytics 360 and used it as one of the targeting conditions:

   

   As shown below, we used built-in reports in Optimize 360 and Analytics 360 to determine the best option and the best time to end the experiment:

   

   

   If the winning option had not been determined within several weeks of testing, we continued with the experiment, changing the targeting conditions to involve more people and ensure statistically significant results.

Results

By utilizing Optimize 360 as a key tool for conducting experiments the following results were achieved:

• Reduced the involvement of developers during test implementation and execution in more than 50% of experiments.
• Ran 100% tests in accordance with the plan.
• More than doubled the number of experiments conducted in a timely manner.

As a result of the decisions made on the basis of two completed experiments, we have succeeded in increasing the conversion rate of the Request Demo form by 21% or 1.4 percentage points. The best option was not specified in the other two tests, so we decided to change the targeting conditions and continue experimenting.