How Intelligent Tracking Prevention (ITP) impacts A/B testing – and how marketers can overcome its restrictions

How Intelligent Tracking Prevention (ITP) impacts A/B testing – and how marketers can overcome its restrictions

Can marketers trust their A/B testing tool anymore?

That’s the key question given the introduction of Apple’s latest Intelligent Tracking Prevention (ITP) update, which is poised to have an enormous impact on CRO practices.

At Kameleoon, we’re surprised that ITP has not yet made the digital marketing headlines. While it is a very technical topic it can potentially have an enormous impact on marketers carrying out A/B testing experiments.

To find out more, we spoke to our CTO, Jean-Noël Rivasseau, about ITP, and in particular the latest version, 2.1 which introduces major restrictions on the operations JavaScript-based software can perform, especially regarding cookies.

ITP is implemented within the Safari browser, and while this has a relatively low percentage of the desktop market, it is a completely different picture when it comes to mobile devices, where Apple has much higher market share (closer to 40% to 50%). This means that almost half of your web traffic could be impacted by ITP!

Here are the key points we captured from our conversation with Jean-Noël:

  • What ITP 2.1 is
  • How it impacts A/B testing tools
  • How we overcome this issue at Kameleoon

What is Intelligent Tracking Prevention?

Apple’s ITP aims to increase user privacy by preventing unwanted tracking by websites and the various scripts installed on them, and is implemented in all Safari browsers. The latest version, ITP 2.1, introduces some major restrictions on JavaScript-based software. Essentially, it means that JavaScript set cookies (which weren’t 100% trustworthy even before ITP because users could manually wipe them), basically become totally unreliable with ITP.

While previous ITP versions mainly targeted online advertising systems, 2.1 is much more aggressive and impacts web analytics software such as Google Analytics (GA) and A/B testing tools in a very serious way.

The main reason for this is that it limits all JavaScript based cookies to a seven day lifespan. The exact wording from Apple’s WebKit developer’s blog is: “With ITP 2.1, all persistent client-side cookies, i.e. persistent cookies created through document.cookie, are capped to a seven day expiry.”

Capping JavaScript based cookies at 7 days is very aggressive. Almost all web analytics platforms (including Google Analytics) are now JavaScript/Front-end based, and they all use a cookie to store the visitor’s unique identifier.

This identifier is randomly generated on the first pageview of the first visit of a user to your website. It is then reused on all subsequent page views and visits to identify this particular user. For example, this allows Google Analytics to know if a visitor is new (as a new identifier/cookie has been generated) or returning (as it finds the cookie that has been set previously).

Given that cookies now have a lifetime of seven days in Safari, this overrides the much longer default expiry date normally set by GA. So, a visitor making a first visit on Monday, then returning on Tuesday of the next week will be considered as a totally new visitor.

Their new visit won’t be linked to the previous one. As a result your new visitor metric, as reported by Google Analytics, is completely false for any Safari traffic. Many other metrics (such as number of unique visitors, time to buy, etc) are also completely distorted. Essentially, it’s hard to trust any number from cookie-based analytics solutions anymore!

What are the main impacts of ITP 2.1 on A/B testing tools?

The majority of testing and conversion optimization platforms also bundle web analytics capabilities within the tool. This means they’ll suffer from the exact same issues we just described, which is problematic in itself. However, for A/B experiments, an even more serious obstacle appears.

This is because each time a visitor is bucketed into an experiment, a variation is selected for them. This variation must be remembered, since if they return, they must see the same variation as before.

To achieve this all front-end testing solutions store this information (the association between experiment and variation) in a cookie. So, with ITP, a visitor coming back more than seven days after they initially triggered an experiment runs the risk of seeing a different variation. A/B testing in these conditions is useless and does not produce reliable results.

To overcome the problem, A/B testing software vendors can be tempted to move away from setting cookies in their JavaScript code, but instead store it on the server via an HTTP header. However, to achieve this they would need access to the back-end code, which adds a lot of complexity. Most existing testing solutions are installed on the promise of being extremely simple to setup, providing a static JavaScript file to include in the HTML source code, or deployed even more easily via a Tag Manager. If a client has to make extensive efforts to install and maintain them, the likelihood is that these solutions will probably not be used at all. This means any workarounds that rely on HTTP headers is not really viable for analytics vendors.

Why is Kameleoon the only conversion optimization platform that automatically and transparently handles Apple ITP?

In short, because our engineering team has implemented a cross-domain Local Storage (LS) for Kameleoon.

Local Storage, a standard Web technology supported by virtually all browsers in 2019, is basically a data store in the browser, so it can handle data in the same way that cookies do.

Given all of its characteristics, Local Storage seems to be an ideal candidate to replace cookies. And ITP 2.1 does not place restrictions on Local Storage.

Actually, several testing platforms already implement LS based storage instead of cookie based storage. However, there is a huge problem with Local Storage: it is limited to a single exact subdomain.

This is because of the subdomain / protocol partitioning scheme used by Local Storage. Unlike cookies, where JavaScript code running on http://www.example.com can create a cookie associated with the domain *.example.com (which means that you can access this cookie from buy.example.com, or even buy.now.really.example.com), Local Storage is partitioned by exact subdomain AND protocol. If you write data on the LS on the subdomain http://www.example.com, you cannot access it later from a page at http://buy.example.com, or even from https://www.example.com. Both protocol and domain have to exactly match.

For this very reason, unless you’re lucky enough to have a single site and a single subdomain throughout this entire site, you will still suffer from serious issues if you use a solution relying on a simple Local Storage implementation. These issues would appear similar to those caused by ITP, although the reason is totally different.

As an example, let’s imagine that most of your e-commerce website is hosted on https://www.randomshop.com, but the conversion funnel is on https://transaction.randomshop.com. If your visitor ID is persisted on the LS, your analytics solution will report two different visitors when a complete purchase occurs, one seen on www.randomshop.com and the other on transaction.randomshop.com. Data does not transition consistently from a domain to another, and such a solution would see the as two completely separate sessions, not a single one of the same visitor.

For A/B testing, this is – again – even worse. If you run an experiment modifying the navigation menu, with two variations, one user could very well be exposed to the first variation on the main site, but to the second one down the funnel! This is of course disastrous in terms of user experience, while also voiding the results of the experiment.

So we went further and implemented a full cross-domain Local Storage implementation, as even a small percentage of pages on a different subdomain on your website can mean unreliable and even false results for A/B experiments.

We actually started our Local Storage project back in 2014! Then, we just needed a large storage space on the browser, as we wanted to store all data gathered by Kameleoon on the front-end. This enables us to run our predictive machine learning algorithms on this information – yes, we actually run neural networks algorithms directly on the browser, in JavaScript! It works very well, but that’s a topic for a different article.

When the first versions of ITP came out, we actually had to perform some minor changes to our system. Oddly, external Local Storage domains are actually treated as third-party context by all ITP versions, like cookies. We initially hosted the common iFrame file on our own (Kameleoon) domain, not on the customer domain (mostly to facilitate setup), so the common Local Storage data was considered to be third-party and was blocked by Safari. To fix this, we just changed the hosting of the iFrame so that it became hosted on the customer’s domain.

So to enjoy ITP-free A/B testing and analytics with Kameleoon, make sure you use our Dual JavaScript & iFrame files integration method. That’s all there is to it, really!

Two additional closing notes:

Although we can guarantee that our analytics core is unaffected by ITP, and that all our reported traffic data is reliable, of course we cannot guarantee the same for third party web analytics platforms. Our integration bridges with partner tools continue to work in the same way as before. We recommend that customers discuss the situation with these vendors directly.

Our server-side SDKs (and Mobile SDKs) are completely unaffected by ITP, as analytics tracking calls are made server-to-server, never on the front. If you use the Back-End/Front-End bridge, a cookie will be used to synchronize the kameleoonVisitorCode, but it will be set via the server (HTTP header), not via the front, and will be considered a first-party cookie, so is not affected by ITP.

The Kameleoon team is extremely proud to be able to provide our customers with this painless solution to the ITP 2.1 challenge. Thanks to it, our customers can continue to enjoy stable, reliable A/B testing on their mobile traffic, without having to spend precious time implementing custom hacks to mitigate ITP. As a software vendor, we strongly believe we should do the maximum to ease the lives of our customers and this is a perfect example of our philosophy.

We will continue to monitor the evolution of Apple’s ITP technology, as well as other initiatives from other browser vendors (Firefox has a similar technology to ITP, but this is not yet as extreme). Any new changes will be swiftly analyzed and we’ll deliver solutions as soon as possible. We currently have a very sizable head start over competing platforms in this area, and we intend to keep it.

For a full technical understanding of ITP 2.1 and how Kameleoon overcomes it, visit our documentation section.

Jean-Baptiste Alarcon

Jean-Baptiste is Growth Marketer at Kameleoon. Aside from reading a lot and drinking coffee like his life depends on it, he leads Kameleoon's growth on English markets.