[SOLVED] How bad are redirect A/B tests for performance?

How bad are redirect A/B tests for performance?

My company is asking to A/B test our main menu navigation which is available on all of our pages. We currently use a platform called Optimizely to redirect users for any variations.

I know that redirects are especially bad for mobile performance. Are experiments like this generally not recommended? Seems like Optimizely tries to mitigate the impact of redirects but could we potentially lose users?

I see a server-side option for Optimizely, but it is not something we have investigated yet. This is a broad question but any insight is appreciated. I'd be happy to post more info.

Solution

Have you considered defining a "Page" in Optimizely that uses Global URL targeting (docs)? This page will activate on every page where the optimizely JS snippet lives. This is the typical means of deploying a navigation menu A/B test.

If this is not possible due to the technical complexities of your navigation, you can safely run a redirect experiment for this test.

Optimizely's redirect uses a window.location.replace call, and as long as the snippet is installed high in the <head> of the page, the user will be bucketed and redirected quickly.

To test out the performance on mobile, I recommend the following steps:

Create the experiment and redirect variation
Create a "QA" audience (docs).
Visit your site, appending the necessary query parameter to become eligible for the experiment

This will allow you to QA the redirect behavior as if it were a live experiment on your site - "Preview" mode in Optimizely is merely a preview, and not representative of the true production performance once 'published'.

A few important SEO measures to take when launching a redirect experiment:

Canonical tag - You should add a link tag to the head of your post-redirect page which tells Google that the canonical version of the page lives at a different URL. (docs)
Robots.txt entry - search engines that respect this convention will skip over URLs that follow the disallow directive. Add your post-redirect page URL to this file.