Skip to main content

How to read A/B test results and declare a winner

Before You Start​

  • The test should be running (or archived with historical data).
  • Real visitors must have hit the entry URL — your own visit in a private window works for a first check.

For a deeper explanation of every metric, see Reading A/B test results.

Step 1: Open results​

From the platform A/B Tests page:

  1. Find the experiment card.
  2. Click View in Analytics.

Atomicat opens the Analytics app on the A/B Tests tab with that experiment preselected.

You can also open Analytics → A/B Tests, choose the experiment from the searchable dropdown, set the period, and click Refresh if needed.

Step 2: Read the summary​

Check:

  • Visitors — unique visitors assigned to variants
  • Conversions — unique visitors who completed the primary goal at least once
  • Conversion rate — unique converters ÷ visitors (N/A for traffic-split-only tests)
  • Best variant — declared winner if one exists; otherwise the highest conversion rate including control
  • SRM — whether traffic split looks healthy vs configured weights

Step 3: Compare variants​

In the variant details table:

  • Control is labeled explicitly.
  • Lift shows relative change vs control (hover the info icon for the formula).
  • Decision can be collecting data, leading, losing, or inconclusive.

Early traffic often stays in Collecting data until there are enough visitors for a statistical read.

Step 4: Declare a winner (optional)​

When you are ready to ship a variant:

  1. Return to platform A/B Tests.
  2. On the running test, click Declare winner.
  3. Select the winning variant in the popup and confirm.

Atomicat applies the winning page to the public entry URL (for page-based tests), stores the winner, and archives the experiment so splitting stops for that path.

After that you should see:

  • A Winner badge on the platform test card
  • Winner / Not selected in the Analytics Decision column
  • The declared winner in the Analytics status row and Best variant card

Step 5: Archive without a winner​

If you want to stop the test without applying a winner, click Archive. Other active tests on different entry paths of the same site keep running.

FAQ​

What does an SRM warning mean?

Traffic allocation looks unusually imbalanced compared with your configured weights. Redeploy the test and confirm the entry URL still serves variants correctly.

Can I keep editing a running test?

Only the name and hypothesis. Site, entry URL, variants, traffic split, and goal are locked. Archive and create a new test if you need a different setup. Use Redeploy only to refresh edge routing or republished variant pages.

Why does Decision still say Collecting data?

That means there is not enough traffic for a statistical call yet — or you archived without declaring a winner. After you declare a winner, Decision should show Winner / Not selected.