A/B test is a powerful tool of marketing which can be used to evaluate how effective is your message and how leads will react to a different version of it. Imagine that your goal is to understand which one works better in mailing: "Hello" or "Hi"?
Any message can be used in A/B testing. You can check how changes in messages (and even in JS) can reflect on your conversion.
A/B test setting
Any message can be used in A/B testing. You can set A/B testing for created and launched auto messages. Lets see how to ckeate A/B test in a new message.
Create new message, choose the trigger, audience and sending conditions. You can indicate percent of control group (these leads won't receive this message at all) in sending conditions.
Set form and condition and click on "add A/B test". Version A will hide so you will be able to set up version B.
When both versions are ready you can move to the next step.
If you don't want to add A/B test just skip this step.
The control group
The control group will help you to understand how your leads will act without getting this message. Select the percentage of leads who will not see the message.
You can select control group in "Sending conditions" step, by default it is 10%. You can set your own value.
Important: The percentage of control group can be set even if you do not configure A/B test. In this case you will see how your leads will act without getting any auto message at all.
Logic of A/B test and control group work
Example: you created two versions of message (A/B test) which are sent to all leads from London in 10 minutes after signing up.
Among all leads who have signed up (completed triggered event) in 10 minutes (timeout) will be selected leads from London (audience). If you configured a control group then a selected percentage of leads won't receive this auto message. A/B test doesn't affect on the work of control group. If there is no A/B test then the rest of audience will receive a message. If A/B test is configured then the rest of audience will be randomly divided 50/50. leads in control group won't see the message at all, the first group will see version A and the second one will see version B.
Important: leads from audience are selected randomly, so the order in which they will receive the message may differ from A-B-A-B-A-... There are more chances that the ratio will be 50:50 when the audience is big.
Numbers are everything. Numbers will help you to determine what message is more preferable to send. Every message can have it's own success indicators. What message to choose: which has more clicks or which were opened more times? The decision is up to you. You can also decide when the collected data is enough to make a conclusion and to stop A/B test.
Statistics will help you to check the following: number of sent messages, how many of them were delivered, read, replied, in how many of them leads clicked on a link, marked as spam and unsubscripted.
Data is displayed side by side so you could compare numbers and charts as well.
Moreover, you can even check how much money is brought by any variant (and control group). In order to do this just indicate a goal (the next step after Form and Content). This event should be done by a lead after receiving an auto message (e.g. buy something). Statistics will show you the conversion of goal achievement if you configured goal value and income.
A/B test disabling
Once you collected enough statistics and chose the best variant you can disable A/B test.
In order to disable A/B test click Finish A/B test button and select version which you want to disable. Example: you are not satisfied with version B, then select it and click on "Finish A/B test". A/B test statistics will be moved to archive, you can check it at any time. Version A will be still working and will be sent to all leads who match all filters of this auto messages.
Once you need to compare version A with another version you can create a new A/B test. One message can have unlimited number of A/B tests. You can check the history of all closed A/B tests in archive.
Important: If you will disable auto message and then will launch it again, A/B test will be continued.