Does FeedbackFive have A/B testing?

Use the Analytics feature to compare the performance of two or more campaigns.

In FeedbackFive, you can set up two campaigns and use the Analytics feature to compare the resulting reviews or feedback.

Amazon policy prohibits sending more than one request for a review or feedback for the same order. This guide explains how to set up your test (either for seller feedback or product reviews) and choose a winner without overlapping your campaigns.

Click the links below to skip to a section:

1. Choose what to test.
2. Set up a test for seller feedback results.
   Or
   Set up a test for product review results.
3. Switch to the winning campaign.

What to Test?

It is best to test only one aspect of the email at a time, while all other aspects remain the same.

• To test two versions of your email content or two subject lines, you will first need to create two templates, then create two campaigns to go with them according to the directions below.
  • Testing two subject lines is a good place to start. Buyers won't see the content of your email unless they find the subject line interesting enough to open it.
  • When testing content, test just one specific part of the content, e.g. a product review button vs. a product review link with stars.
• To test sending at different times or on different days, use the same template on two different campaigns.
  • You can test one campaign sending in the evening and one in the morning, or one campaign sending closer to the delivery date and the other a few days later.

Setup: Testing for Seller Feedback

To test seller feedback results, set up two campaigns that split your orders based on order number:

1. Select FeedbackFive in the top navigation and hover on the left side of the screen to open the navigation menu. Click Campaigns and select "Create New Campaign."
2. Add the template, title, and description.
   • Note: Add "A" or "1" to the title to remind yourself that this campaign is part of a test. Use the description to remind yourself what is being tested, e.g. "Sending at 8:00 AM."
3. Add the rule "Send for order numbers ending in."
4. Select only the even numbers: 0, 2, 4, 6, 8.
5. Add any other rules, such as "Send to refunded orders?" > "Never" (recommended). 
   • Note: If you have ever sent requests for feedback or reviews from FeedbackFive before that might overlap with this campaign, add the rule "Send based on number of previous emails" > "= 0" or "Send if email sent from specific campaign" > "Send if an email has not been sent from ANY of these campaigns" > type the name(s) of previous feedback/review request campaigns.
6. Save changes.
7. Back on the Campaigns page, hover over the title of your newly created campaign and click the "Copy" icon.  This creates a clone of the campaign and opens the Campaign Rules page.
8. Edit the aspect you are testing:
   • If you are testing subject line or content, switch to the other template you created (see above).
   • If you are testing timing, change the timing.
9. Edit "Send for order numbers ending in" to select only the odd numbers: 1, 3, 5, 7, 9.
10. Edit the title and description, using "B" or "2" to show that this is the other half of the test. Use the description to remind yourself of the difference between the campaigns.
11. Save changes.
12. Activate both campaigns.

Setup: Testing for Product Reviews

To test product review and rating results, separate all your SKUs or ASINs into two groups and target each campaign to about 50% of your products (or products that will generate about 50% of your orders). For best results, make sure that you include high-priced and low-priced items, as well as items from every category you sell, in each group. If you have items with variations, split up variations so that about half the variations for a product are in each group.

Once you have your items separated:

1. Select FeedbackFive in the top navigation and hover on the left side of the screen to open the navigation menu. Click Campaigns and select "Create New Campaign."
2. Add the template, title, and description.
3. Add the custom rule, "Send for specific products."
4. Choose "contain" and either SKUs or ASINs from the rule options.
5. Add 50% of your products to the list.
6. Add any other rules, such as "Send or not send to refunded orders" > "Never" (recommended).
   • Note: If you have ever sent requests for feedback or reviews from FeedbackFive before that might overlap with this campaign, add the rule "Send based on number of previous emails" > "= 0" or "Send if email sent from specific campaign" > "Send if an email has not been sent from ANY of these campaigns" > type the name(s) of previous feedback/review request campaigns.
7. Save changes.
8. Back on the Campaigns page, hover over the title of your newly created campaign and click the "Copy" icon.  This creates a clone of the campaign and opens the Campaign Rules page.
9. Edit the aspect you are testing:
   • If you are testing the subject line or content, switch to the other template you created (see above).
   • If you are testing timing, change the timing.
10. Change the rule "Send for specific products" to "do NOT contain." Do not edit the list of products. (This ensures there is no overlap between the campaigns and that no items will be left out of both campaigns by mistake.)
11. Edit the title and description, using "B" or "2" to show that this is the other half of the test. Use the description to remind yourself of the difference between the campaigns.
12. Save changes.
13. Activate both campaigns.

Check Results

After at least 30 days and/or when a few hundred emails have gone out from each campaign, check results:

1. Select FeedbackFive in the top navigation and hover on the left side of the screen to open the navigation menu. Click Campaign Analytics.
2. Using the drop-downs at the top of the page, select your A or 1 campaign and 30 days.
3. Using the drop-downs on the graph at the middle of the page, select Emails and either Product Reviews or Seller Feedback (depending on what you were testing for).
4. Observe the results, then use the campaign drop-down at the top of the page to switch to your second campaign and compare results.

Switch to the Winner

If one campaign performed better than the other, it’s time to combine the campaigns back together. Follow these steps carefully to avoid overlapping your campaigns, which could lead Amazon to restrict or block your emails:

1. Deactivate the campaign that performed worse.
2. Click the title of the better campaign.
3. Remove the products rule by clicking the X beside it.
4. Add the rule “Email sent from specific campaign?” if it isn’t already there. Choose “Send if an email has not been sent from…” and add the name of the campaign that performed worse.
5. Save changes.

Further Help

If you need help deciding what to test, setting up your test, or choosing a winner, contact us for assistance.