Skip to main content
mixed-group-problem

[Product Release] Kameleoon puts an end to the Mixed Group Problem

December 7, 2016
You already know why A/B testing is so popular with marketers: it’s an efficient means for user experience optimizations. But you probably don’t know their is one major problem with A/B testing (it’s a very well-kept secret), a technical limit that can well skew the results of all your tests results. It’s called mixed group problem. Up to now, A/B testing solutions weren’t able to recognize a visitor who logs in via different devices, and couldn’t guarantee that this visitor would always see the same variation of an A/B test. Our teams were aware that this was an important issue for our customers, as users access their sites via desktop, smartphone or tablet. And thanks to Kameleoon’s innovation mindset, we developped the first platform ever to come up with a solution to offer reliable data for your business decisions under any circumstances.
Jean-Noël Rivasseau - Mixed Group Problem“Kameleoon’s aim has always been to make marketing easier, allowing to create tailored experiences for their visitors. A solid, agile solution should be able to recognize visitors on whatever device they are using, in order to make sure they will always see the same variation when being tested. We are very proud to have created the first optimization platform worldwide to put an end to the mixed group problem.” -Jean-Nöel Rivasseau, Founder & CTO, Kameleoon

The root of all evil: the mixed group problem

Why is it important to recognize a visitor on several devices? Slowly but surely, we’re moving towards “mobile first”. Mobile Internet traffic will soon become more important than desktop traffic. According to Salesforce we’ll buy more via smartphone than via any other device by the end of 2017, and mobile Internet will represent 60% of overall traffic (source: EBG, Task Force #9, Les Pure Players). graph - mixed group problem Marketers optimize their sites through A/B tests because they deliver reliable data for reliable decisions. But they ignore that the cross-device factor can bias results and make them take the wrong decision. How come? It’s the cookies. A visitor connecting via desktop receives a visitor ID, or cookie. If he connects via a smartphone, he receives a different cookie. Although the visitor is the same, he is not recognized and may be exposed to different variations of the same A/B test.
Example :  Let’s imagine that a site has an audience of 100 visitors, who all possess two devices. Each time they log in, they are randomly allocated to one of the variations. Over time, your A/B testing solution will have executed 200 redirections for only 100 visitors. Ultimately, the mixed group problem can invalidate a test by biasing the number of visitors – and their behavior. Seeing two different variations definitely impacts the visitor’s behavior.

The mixed group problem definitely solved

At Kameleoon, we could hardly contain our excitement when our R&D team added a unique feature to our solution. Kameleoon solved the mixed group problem by allowing the recognition of cookies from one device to another.
Frédéric - Mixed Group Problem“By solving the mixed group problem, we put an end to two inconsistencies: First, the user journey becomes more relevant because we are sure that the visitor sees only one variation. Secondly, that implies that our customers are sure to obtain statistically reliable data.” -Frédéric De Todaro, Head of Consulting, Kameleoon.
Use case example: Auchan ends e-mail conflict Like many other e-merchants, Auchan uses e-mails to contact its customers off-site and to encourage them to buy.
Akim Demora - Mixed group Problem“We send reminder e-mails 1, 2 and 24 hours after a visit and make product recommendations according to the visitor’s basket and customer journey (products and product categories viewed). Evidently, we want to make the offer as relevant as possible. That’s why it is so important to consolidate cross-device visitor journeys.” -Akim Demora, Had of E-Commerce Marketing , Auchan.

In practical terms: A visitor who doesn’t validate his basket will receive a reminder e-mail some time later that encourages him to finalize his purchase. Auchan - Mixed Group ProblemThis type of e-mail helps Auchan raise its conversion rate. So far, so good. But if the visitor logs in on his smartphone to finalize his purchase, he will nevertheless receive a reminder e-mail asking him to validate his basket. Thanks to Kameleoon’s innovation, Auchan can make sure that this won’t happen, that test results will not be biased and that the user journey will not be disturbed. Voyage Privé unifies its customers’ user experiences A trip is a very engaging experience and must be well prepared. At the same time, part of the decision to travel is always impulsive. In order to encourage the conversion of their visitors, tour operators must create an atmosphere of evasion. That is why Voyage Privé wanted to create a high-quality immersive experience. Unifying this experience over different devices helps the tour operator to optimize its conversion funnel. Voyage Prive - Mixed Group Problem A unified experience helps the visitor understand his experience. He recognizes the site and feels “at home” more easily. Kameleoon puts an end to inconsistency caused by the mixed group problem. We promised you lots of innovation this year - we keep our promises (and the year isn’t even over yet)! If you want to see this new feature work or learn more about our solution, ask for a personalized demo!

demo request kameleoon

Topics covered by this article