Help Center › A/B Experiments

A/B Experiments

Last updated: August 2026

Experiments let you show different versions of a page to different visitors and measure which one converts better. Traffic is split by the widget you already installed, so a simple test needs no code at all.

Creating an experiment

  1. Go to Experiments in the dashboard and click New experiment
  2. Give it a name and a key — the key is what you reference in code, and it can't be changed later
  3. Set Target pages — one pattern per line, e.g. /pricing or /landing/*. Leave it empty to run everywhere
  4. Choose a conversion goal (see below)
  5. Define your variants and how traffic splits between them — the percentages must add up to 100%
  6. Save, then press Start experiment when you're ready to collect data

Experiments start as a draft and collect nothing until you start them. Available on the Unlimited plan.

Conversion goals

A goal is what a visitor must do to count as converted. There are two kinds:

  • Custom event — fires when an event with that name is tracked. Either call Announcer.track('signup_completed') in your code, or set up a no-code click tracker under Settings › Events and use its name here
  • Page visit — the visitor lands on a matching page, e.g. /thank-you

Each visitor converts at most once per experiment, so your conversion rate is the percentage of people who converted, not the number of times the goal fired.

Split-URL tests (no code)

The simplest test: two URLs, no code changes. Give each variant a redirect URL and the widget sends visitors to the right one automatically.

  • Two separate pages — set the target page to /pricing, leave the control's redirect empty (the page itself is the control), and point variant B at /pricing-b
  • One page, a query parameter — if both versions live on the same page, give the control /pricing?v=a and variant B /pricing?v=b, then render based on the parameter

Existing query parameters such as UTM tags are carried across the redirect, and visitors who land directly on a variant URL still stay in the variant they were assigned, so the split stays clean.

Code and CSS variants

To vary a page in place, the widget gives you three options. All of them are shown with your real experiment and variant keys on the experiment's results page.

JavaScript:

Announcer.ready(function () {
  if (Announcer.experiment('new-pricing') === 'b') {
    // show variant B
  }
});

CSS — the widget puts the assigned variant on the <html> tag, so a test can be pure CSS:

html[data-announcer-exp-new-pricing="b"] .old-headline { display: none; }

Event — for frameworks that render late, listen for announcer:experiments on document, which carries every active experiment and its assigned variant.

How visitors are assigned

  • Assignment is sticky — a visitor keeps the same variant on every visit, including anonymous visitors
  • Changing the traffic split mid-test doesn't move anyone who has already seen the experiment
  • A visitor only counts once they actually saw the experiment on a targeted page, so people who never reached the test don't dilute your results

Reading the results

Open an experiment to see, per variant:

  • Visitors and conversions
  • Conversion rate, with a bar for quick comparison
  • Uplift against the control
  • Confidence — how sure we are the difference is real, not noise. 95% or higher is marked Significant

Below a few hundred visitors per variant, results are usually noise — the page will tell you when there isn't enough data yet. Resist calling a winner early.

Pausing and finishing

  • Pause stops the split; visitors go back to seeing the control page as it is
  • Resume picks up where you left off, keeping existing assignments and data
  • Complete ends the test and lets you mark a winning variant, keeping the results for reference

After completing a test, roll the winner into your page for real and delete the experiment's redirect, so visitors stop being bounced through it.