Content A/B split testing improvements

In this article we will review content A/B testing improvements, which will be released within upcoming version 8. Version 8 is scheduled to be released at the end of Q1/2014 and you can see additional roadmap details here: http://www.kentico.com/Product/Roadmap.


Please note that screenshots of A/B test improvements below might differ from what will be released within version 8.
 









Improvements overview
Let’s start our tour of improvements in the content tree:
Running tests are marked by a new icon so that content editors can easily determine where the tests are located.
We have also reviewed statuses and other data available on the A/B test list page:
 
When creating a new A/B test, you have several new options available, such as “Visitor targeting”. Here, you can define complex conditions which visitors of the tested page must meet in order to be included in the test and see the test variants. You can also restrict the amount of the page’s traffic included in the test by using “Included traffic”:  
Finally, there’s the Overview tab which gives you summary of all the data about running and finished tests:
 
Here, you can see new types of reports and you can filter the results according to different criteria, such as Conversion goals, Success metrics or Counting methodology. You can also view a summary of a test or see data for each of its variants. Additionally, you are able to evaluate confidence of a test data using metrics such as Chance to beat original, Improvement or Conversion rate overlap. Note also, that tests can be launched and finished only once - this will help evaluate confidence of results of a test.
 
Backward compatibility
Some of the reports are backwards compatible, so if you have some tests that finished prior to v8, you will be able to see its results even after the upgrade. Please also note that some of the new reports won’t display any results for tests that finished prior to v8. These are based on different data—unavailable in the previous versions. 
 
MVT
As you may know, an MVT content testing tool has also been released as a part of the previous versions, together with A/B testing. For v8, we have only improved A/B tests and MVT remains the same as in the previous version. We are waiting to hear your feedback – please let us know if you want the MVT content testing tool to be upgraded or if you are fine with just A/B testing.
 
Please, send us your feedback and comments on the A/B test improvements! You can leave your suggestions below in the comments sections or you can use http://kentico.uservoice.com for any new ideas on A/B. We are keen to hear what you like, what you are missing and if you need similar improvements for the MVT content testing tool.