We’ve developed a new site search for our site, but before I deploy the new one, I’d kinda like to run some split testing to see if it actually gives better conversions.
I had been planning to use Google anlaytics for the split test, just because I use Google for a lot of other analytics. But I’m not sure if Google’s split testing is suitable for this task, as I really want to deliver a separate page, rather than some slightly different content, which I think is how Google’s test works (though I’m prepared to be corrected on this).
Is it possible to use Google analytics for this, and to count in the test only those users who used the search? Or would I have to use a server-side split test for this?
Yes you can use Google analytics for this however they don’t call it split testing but Content Experiments. The name is a little misleading because you can test multiple things and not just content. You can set up the experiment and test multiple versions of a page to see which one is most effective. It does this using goals that you have set up in the analytics dashboard.
OK I’ve set up an experiment, and it works pretty much like I suspected, in that the server-side processing is carried out before any of the test scripting is run.
So in my example, where I have two versions of the search algorithm, the order of execution would be:
user submits search
server processes search
server outputs page to browser, including the analytics javascript code
the client-side code executes and (conditionally) redirects to the page to the alternate search page
So the main processing page will run regardless. This is fine from the point of view of Google analytics collecting data, but not so great for the user, as they may have to wait for two complete searches to execute, which is a couple of seconds for each page. This would also make the test page appear slower to the user, which would skew the results.
Any suggestions? At this point I’m thinking my best bet would be to serve a generic page which does no server-side processing, and have the google analytics code sort out all the content. However I then have all the content (of the search page, anyway) dependant on javascript, which I never like doing.