Intelligent A/B Testing of Push Notifications for Shopify

A/B testing lets you send two versions of a push broadcast and compare which one performs better. You create two notification variants, and PushEngage can send each to a sample of your audience, then automatically send the winning version to everyone else. This guide shows you how to set up an A/B test on a push broadcast in the PushEngage Shopify app.

Innan du börjar

  • Install and connect PushEngage on your Shopify store, and build up an audience before you test.
  • Have at least two ideas to compare, such as two titles, two messages, or two images.
  • For Intelligent A/B testing, your targeted audience needs to be large enough to sample, with a minimum of 50 subscribers in the test phase to see best results.

How A/B Testing Works

You start from a push broadcast and add A/B testing to it. The broadcast then has two content variants, Notification A and Notification B, which share the same audience and schedule. The content of each variant is independent, so you can change the title, message, URL, image, and action buttons per variant.

With Intelligent AB Test turned on, PushEngage sends each variant to a sample of your audience, waits for a delay you set, then automatically sends the winning variant to the rest of the audience. You control the sample size and the delay before the winner goes out.

Create an A/B Test

1. Go to Push Notifications » Campaigns. Push Broadcasts is the default view.

2. Click Create to start a new broadcast and fill in the details. Then Click Add A/B Testing to turn on the test.

3. Now, switch to the Notification B tab and write the second version.

4. Set your audience and schedule, which both variants share.

5. Configure the Intelligent AB Test options if you want PushEngage to pick and send the winner automatically.

6. Save as Draft or Schedule the broadcast.

Set up the Intelligent AB Test

In the Intelligent AB Test card, PushEngage sends each variant to a sample, then automatically sends the winner to the rest. Set the Sample Size (%) to control how much of your audience receives the test, and set the Delay to control how long PushEngage waits before sending the winner to everyone else. The sample must reach at least 50 subscribers for the test to run.

Review Test Results

After a test runs, open the broadcast from the Push Broadcasts list to see the results. Each variant shows its own performance, and the winning variant is marked so you can see which version your audience responded to.

Vanliga frågor

What can I change between the two variants?

You can change any content field, including the title, message, URL, icon, large image, and action buttons. The audience and schedule are shared, so the only difference your subscribers experience is the content you vary.

How does PushEngage decide the winner?

With Intelligent AB Test enabled, PushEngage sends each variant to the sample you set, waits for your delay, then sends the better-performing variant to the remaining audience.

Can I A/B test a drip or triggered campaign?

A/B testing applies to push broadcasts. For drips and triggered campaigns, create separate campaigns to compare approaches.

Om du stöter på några problem är du välkommen att kontakta oss genom att klicka här. Vårt supportteam kommer att kunna hjälpa dig.

Last updated on August 2nd, 2026

Engagera och behåll besökare efter att de har lämnat din webbplats

Öka värdet av varje webbesök med push-notiser som är svåra att missa.

  • Evigt gratis-plan
  • Enkel installation
  • 5-stjärnig support