Every number we publish, and how it was produced.

Experiments run in Intelligems or Convert.com depending on the store's setup. The tool is named in every proof pack, because they do not treat significance identically and a reader is entitled to know which one produced the number.

We report at 90% confidence.

That is a deliberate choice and it is lower than the 95% often quoted as a default. Ninety percent carries more false-positive risk. We use it because the alternative, at the traffic volumes most stores run, is waiting so long for significance that the test blocks the roadmap for a quarter and the business decision gets made without evidence anyway.

The honest framing is that 90% is a tradeoff between statistical caution and commercial usefulness, not a stricter standard than 95%. Anyone who tells you their 90% is as safe as a 95% is selling.

Our test windows run from a few days to sixty. A longer window is more defensible, because a short one cannot see day-of-week effects, campaign mix changes or seasonality.

So the window sits next to every number. When a result comes from a five-day window we call it a short-window directional projection, even when the sample is large. Session volume does not neutralise a calendar.

It is an estimate extrapolated from the test window, not revenue banked.

A test that ran eight days and won gets scaled to a monthly figure using the observed lift and the store's own traffic and order economics. That is a model. It assumes the lift holds, the traffic mix stays comparable and nothing else changes, and all three of those assumptions can be wrong.

We label it projected every time it appears. If you ever see one of our numbers presented as money a client collected, it has been misquoted.

Fewer than one in three of our experiments produces a win. That is normal in experimentation, and it is the reason the choice of what to test carries more weight than how many tests you run.

A loss is recorded with the same detail as a win: what changed, what moved, how it was measured, and what it ruled out. A ruled-out hypothesis is worth paying for, because the alternative is testing it again in eighteen months.

Some results never get published.

We exclude anomalies. A result from a single day on a small sample, showing a lift far outside anything the store has produced before, is more likely to be a measurement artefact than a finding. We have excluded our single largest recorded result on exactly those grounds.

We also hold back results where two internal measurements disagree. Where a stricter probability-to-beat-baseline threshold reclassifies an earlier win as unclear, the claim stays unavailable until the delivery team resolves it. Several currently sit in that state.

Naming a client requires their recorded permission, separately from permission to publish the number. Where we have the result but not the permission, we describe the change and omit the brand.

A logo on its own is not proof of anything, so we do not use one that way.

Every quantified statement we publish is registered internally before it goes public, with its source, its required caveat and its approved wording. If a claim is not registered, it does not appear on this site, in a deck, or in an email.

That is also why our published range across winning tests starts at a small number rather than a flattering one. The range is what the data says.