About this page
This page describes the cycle of a test-round.
Before starting a round
In Kaizen Platform K2, a single A/B test is called a Round.
Find more about this here:
Start of a round
After completing the actions to prepare for a round, the test can begin. How? On the right side of the Variations-tab is the button to select: -> Start New Round.
When the round has successfully started, the Report-tab will show like this:
Which design Variation will show
When the round starts, Variations will be shown to visitors at random. The shown Variation is recorded by Kaizen Platform, so visitors returning to the page will always see the same Variation.
For example, for the test-page
http://example.com/test.html, the URLs to test each Variation are like this:
- Original design
- 1st design Variation
- 2nd design Variation
- nth design Variation（n can be 1-9）
Design Variations are usually named 0,1,2,3… so to see them, simply use one URL and adjust the last value. Also the different Variations used in a Split URL test can be viewed by adjusting the GET parameters like this.
End of a round
After a significant difference has been
determined, a round can be stopped to move on to a next round. By comparing the conversion of the original design with the conversion of each design Variation, the design with the highest conversion rate is found. This will be set as the new base design for a next round.
Manual controls in an Experiment
Kaizen Platform K2 looks for the siginificant difference in a round by itself, and displays the results automatically. But there are manual controls too.
Stop testing a specific design Variation
To stop a specific design Variation for the rest of a round, select that Variation and click the Pause-button in the top right. This way, that Variation will not be shown to visitors anymore.
An Experiment runs with the 3 Variations A B and C. The conversion of B is significant lower. B could be manually withdrawn from the round by selecting Pause. By doing that, the round will continue with A and C. Note that the design set as base design cannot be paused.
On the Report and Variations-tabs of Active Experiments, the (sliding) top bar shows 3 buttons: Pause, Clear and End.
Pause the round
Pause a round at any time. While paused, visitors get to see the base design of a previous round, or the original. Pausing also holds all measurements, like visitors and conversions. Already measured data will be saved.
When a round is paused, a Resume-button will replace the Pause-button. Use it to restart the round at any time.
Clear the round
To return to the situation before a round was started, use the Clear-function. This stops the Experiment and all gathered data will be lost.
Use this function for example when an Experiment was started by accident, or to restart when an Experiment is not performing as expected.
End the round
End a round at any time. Once End is clicked, the Variation with the highest conversion rate will be set as the new base design for a next round, regardless wether a significant difference was reached or not. If there would be no difference in conversion rate, the base design of the this round will be used again as the base design for a next round.
My new design(s) doesn’t show up in the Experiment. What is the problem?
An Experiment on Kaizen Platform starts about 5 minutes from selecting the -> Start New Round-button. There may be a technical issue if the Experiment is not showing the new design Variation(s) after 5 minutes. In that case, please contact our support-team for assistance.
Can I stop an A/B test anytime I want to?
Yes. Select the End-button from the (sliding) top bar on the Report or Variations-tab of the Experiment you would like to stop.