A/B tests: what you can test and how to start one

A/B tests has its own section in the sidebar, right under Analytics. An A/B test shows different versions of one product to different visitors and tells you which version earns the most.

What you can test (choose it first in New test):

  • Landing page vs product page: some visitors see a landing page, the others the normal product page.
  • Landing pages: two or three of your landing pages for the same product.
  • Product images: another set of photos, or another main photo.
  • Product title: another name for the product.
  • Product description: another description on the product page.
  • Store theme: your published theme against one or two gallery themes, across the whole store. It is not tied to a product, and every order counts.

Prices and delivery fees are never tested, so a customer never sees two prices for the same product.

The A/B pill: the quickest way. A small white A/B pill with a flask sits next to everything you can test: the Title, Description and Media headings in the product editor, the top-right corner of the Storefront preview card (product page vs landing pages), every landing page in Online Store → Pages, and the Draft themes heading in Online Store → Themes. Tap it and a window opens with everything the test needs on one screen: the versions, the visitors split, the goal, Pick the winner automatically and the name, then Save draft or Start test. While a test runs, the pill shows a dot (green and pulsing while testing, amber when paused, grey for a draft) and opens the results in the same window.

From the section: A/B tests → New test (or one of the five tiles under Start a test). Two quick choices, then the same one-screen window the A/B pill opens:

  1. What to test.
  2. Which product (skipped for a theme test). A test belongs to one product, and a product can only be in one running test at a time. A landing page can also only be in one running test, and a store runs one theme test at a time.

The window then has everything on one screen:

  • Versions. Version A is the control: the product as it is today, or the first page. Add versions with the + button next to Versions (up to 3 on Try, 4 on Starter, 6 on Growth, 8 on Scale, version A included); remove one with its trash button. Set each version's share in the Visitors split card (drag or type, 1% to 99%; the others adjust so the split adds up to 100%), and choose whether visitors shift to the leader automatically. Landing pages must be published before the test starts. Images come from your media library. For a theme test, version A is your published theme and B / C are picked from the gallery themes (an AI store design can't be version B or C yet).
  • Goal (see "Reading A/B test results"), the Pick the winner automatically switch and the test name. Then Save draft or Start test.

The A/B tests page: a summary band at the top shows how many tests are Running, the Visitors in live tests, how many tests are Ready to decide and the Winners kept so far. When you have tests in more than one state, the All / Running now / Drafts / Ended filter narrows the list. Each running or paused test is a card: one bar per version showing its chance to win (or its share of visitors until the first result arrives), the version in front marked Leading, the gain or loss against version A in green or red, and the goal's number per visitor. Under the bars, four steps show how far the test is from a result: 7 days, Visitors, Finished orders, Clear leader, with one sentence saying what is still missing. A test that has met all four moves to the top with a green Ready to decide badge. Drafts and ended tests are compact rows; an ended test shows its winner, how long it ran and its visitors. Tap any card or row to open the test.

What shoppers see: each visitor is put in one version and keeps seeing it every time they come back on that device. The address does not change: the link in your ads opens the visitor's version under the same URL, so you don't need to edit your ads. Search engines always see version A. In a theme test the visitor sees the same theme on every page of the store.

How many tests at once (running or paused): Try 3, Starter 5, Growth 20, Scale unlimited. Drafts and ended tests don't count. When the limit is reached, end a test or upgrade in Settings → Plan.

Who can use it: team members need View A/B tests to open the section and Manage A/B tests to create, start, pause or end tests (Team → member → Access → A/B tests). Admins, website managers and marketing managers have both by default.

More in A/B tests

Still need help?

Ask the assistant from the search box in your dashboard. It answers from these same guides, in English, French or Arabic, and can make changes for you. Or write to us and a person will reply.