A/B testing

With A/B testing, you can send up to 4 different test emails to see which one generates the most response, based on the variables you are comparing. Your contacts' actions can tell you which email content, subject line names, and From Domain names that they prefer.

Not supported

A/B Testing does not support the following features:

Email-level frequency control
Post to social
Create CRM settings

Errors, warnings, and spam scores

View the errors, warning, and spam scores for each template. You must return to your email draft to address errors and warnings.

Errors show the items for each template that you must fix before you can send the A/B test email.
Warnings show the items for each template that you might want to fix before you send the A/B test email.
Click each tab to see the spam score for each template.

Summary and status

You can view the summary and status of the email after you send the A/B test.

Note: The A/B testing report doesn't display until all emails that were sent from the test are completed.

After you configure and send the email, click Sent Email at the bottom of the Confirm and Send page. A list of sent emails are displayed on the Email and templates page.

Note: Click Scheduled to see the emails that will be sent. You can also stop and unschedule emails on this page.

The A/B Test Status page provides data and metrics for your test. On the Sent tab, click the A/B test email name link. For A/B test status, click View.

If you configured the test to notify you when the test is complete, click A/B Test Status page in the email notification.

The following table describes the states of the A/B test:

State	Description
Test not started	The A/B test is scheduled for a time in the future, and it has not yet started.
In progress	The A/B test started but has not yet determined a winner. You see this even if you manually selected a segment to send to before the configured test completion time is reached.
Winner determined	The A/B test determined a winner. You see this regardless of whether the remainder segment has been sent to.
Canceled by user	The A/B test was canceled. To cancel a test, click End Test Without Sending Remainder at the bottom of the A/B Test Status page.
Failed - system error	If the A/B test fails for some reason, it doesn't become scheduled or get started. It cleans itself up, and the only way you will know about the failure is by looking at the status on the A/B Test data job.

For information about the results of an A/B test, watch this video: http://cdn.training.silverpop.com/training/eng/email/mailings/abtesting/index.html.

A/B testing validation errors

A/B testing uses field-level validations. If you do not configure a field correctly, an error displays.

The following table describes common validation errors:

Error	Details
All email bodies must match the ones in the delivery options tab.	Email body settings for all templates in the A/B test must be the same. It is based on template A's draft email bodies (what's selected in delivery options, not A's template-level email body.)
The tracking level of your template is set to no tracking.	Occurs when the template tracking is off (Tracking level = No tracking) in any of the templates' settings and when A/B testing is on.
This information entered in Test Panel [A] is identical to Test Panel [C].	Occurs when any two variants in the A/B test tabs are identical.
Post to social settings are not applied to your A/B test	Occurs during the health check validation process. Review your post to social settings.
Create CRM campaign setting is not applied to your A/B test	Occurs during the health check validation process. Review your CRM settings.
Maximum messages per contact settings are not applied to your A/B test	Occurs during the health check validation process. Review your contact source settings.
A/B test schedule buttons grayed out	You can't edit this page if A/B test is on. You must click Change A/B Test Schedule to change the schedule for the A/B test.
Please provide at least 2 subject lines	Occurs when only one subject line is created.
Please select a metric to determine the winner	Occurs when the user does not select a metric from the drop-down list.
This time has already passed. Your email will be sent immediately.	Occurs if the user does not change the start date and returns to the A/B testing page an hour later.
This end date occurs before the start date. Select a date in the future.	Occurs if a scheduled end date happens before the scheduled start date.
You cannot apply an A/B test to a contact list. Please select a database or query.	Occurs when a contact list is attached to the email.
Amount of time to determine winner must be set for a longer period than your Send Time Optimization (STO).	Occurs if the STO is set to a pre-defined value for the contact source and the time to determine the winner is less than the STO value.