In this article, we'll be looking at how to conduct A/B tests using HubSpot and Seventh Sense. If you're new to Seventh Sense, it may be helpful to familiarize yourself with the following articles before attempting an A/B test using Seventh Sense and HubSpot:
The problem with traditional email A/B testing
Conducting A/B tests to optimize content, subject lines, style of email or other variables is an important part of running a successful email marketing program. Unfortunately, most marketing automation platforms don't give you very much control over picking winners or analyzing the results.
In addition, using traditional blast A/B methods usually results in some, or most emails going out with poor timing since the system has to wait for a certain amount of opens to occur before sending the rest of the emails.
With Seventh Sense, your emails are automatically spread out over time to optimize each person's delivery time. This allows you to conduct A/B tests without compromising on send time or deliverability.
For example, if you start to see one subject line start to really take off (and be sure to check statistical relevance), you can then switch the email that is being sent by modifying your smart campaign mid-delivery.
How to set up an A/B test using Seventh Sense and HubSpot
Setting up your A/B test in HubSpot: The first step is to create two independent emails that have the different variable you're testing. Once you have one template built, you can also simply clone the email and change the variable that you're testing. When the emails are built, we recommend calling them something similar to
Assign each email to it's own workflow. You'll set these workflows up like you normally do when initiating deliveries from Seventh Sense with manual enrollment, then send an email step and finally turn the workflow on. We recommend calling the workflows something similar to:
- <your_email_name>_Cohort_A which is associated with the <your_email_name>_Cohort_A email
- <your_email_name>_Cohort_B which is associated with the <your_email_name>_Cohort_B email
Splitting your Source List: Navigate to the lists screen in Seventh Sense and next to the list you want to send the campaign to, click "Split". This will create a cohort A and cohort B list that are of equality, not only in list size, but also amount of engagement and last period of engagement.
Depending on the list size, it can take a little time, but once the lists show up (they'll also have a "count" next to them), they'll be ready to use. You can also track the progress of the split in the spinning wheel on the menu bar of the screen.
Finally, use the Cohort A list and personalize it to the Cohort A workflow and Use the Cohort B list and personalize it to the Cohort B workflow.
While there are a few extra steps the advantage of doing it this way is that you have more control. For example, if you start to see one subject line start to really take off (and be sure to check statistical relevance), you can then switch the email that is being sent mid-delivery. For example, if Cohort A's subject line is performing really well and Cohort B's is lagging, simply switch the email in the Cohort B workflow to the Cohort A email.