ISM eCompany Helped Staatsloterij Increase CTR using VWO

Staatsloterij’s primary objective was to provide comprehensive information about its XL option and get more people to select it during the checkout process to increase the lottery ticket size.  

The XL option allows an individual to play twice with the same ticket number, win prizes from the monthly prize package, and also win prizes from the XL prize package.

The execs at Staatsloterij saw that XL transactions were consistently low on both desktop and mobile. Despite the presence of a tooltip, not many people selected the option. They approached Loef and his team to understand the problem and seek assistance on increasing XL selections and lottery ticket sizes.

Loef and his team conducted an in-depth analysis of the XL option using VWO’s capabilities. They used heatmaps, sessions, and other qualitative research tools to understand why visitors did not select the XL option. 

They discovered that visitors often encountered the XL option for the first time on the shopping cart page. Many even clicked on the XL toggle and tooltip out of curiosity to learn more about it. However, the tooltip hardly provided any valuable information about the XL option because of which most visitors switched off the XL option and proceeded to check out.   

Loef and his team hypothesized that by adding more information about the XL option in the tooltip and explaining why visitors should choose the option, they should be able to increase the XL option selection and the lottery sale amount.  

They used VWO Testing to run a Split URL test. They created a new version of the checkout page for both desktop and mobile and added two elements on the checkout page.

1. An arrow with text to increase the focus on the XL toggle option.
2. More information about the XL option in the tooltip and why visitors should select the option. 

Here are the control and variation for desktop and mobile, respectively.

Desktop Version

Mobile Version

The team ran the test for 17 days on both the platforms, with traffic split equally between the control and variation. Here’s what the team analyzed:

• Desktop:
  • The number of transactions increased by 1.04%.
  • Visitors clicked less on the tooltip, possibly because the arrow focused much attention on the XL toggle option than the definition of XL.
  • Visitors switched on the XL toggle more than switching it off (+ 8.16%).
  • The arrow with text focused on the extra chances of winning, which served as a motivator.

• Mobile:
  • The XL explanation had a negative impact on the number of transactions (-1.10%.)
  • Visitors clicked on the tooltip less often.
  • Most visitors switched on the XL toggle more than switching it off (+ 8.35%).
  • The number of transactions compared to the desktop was less, possibly because of the arrow and text position. 

After observing an uplift on the desktop, the team deployed the change and it’s currently live on the website. Meanwhile, an uplift wasn’t observed on their mobile platform, and hence, the team is currently contemplating to run more experiments with different designs to achieve their objective of more XL selections. 

ISM eCompany runs every experiment with one primary end goal: getting an uplift in revenue through successful A/B test implementation. They plan and strategize in a manner that every experiment run provides enough learnings for them to grow and achieve their goals.

Start your free trial today to know how A/B testing works.