What is Incrementality Testing?
Incrementality testing measures the true causal impact of a marketing campaign by comparing the behaviour of exposed users against a control group that didn't see the campaign. Rather than relying on last-click attribution, it answers the fundamental question: "What sales or conversions happened because of this campaign?"
The methodology typically involves randomly withholding a campaign from a statistically significant control group whilst running it normally for a test group. By comparing conversion rates, purchase behaviour, or other KPIs between the two groups, marketers isolate the genuine uplift attributable to the campaign alone – not confounding factors.
Why It Matters
UK media buyers increasingly face pressure to prove ROI in a privacy-first landscape where third-party cookies are disappearing and attribution becomes harder. Incrementality testing fills this gap by providing clean, causal evidence of campaign effectiveness independent of tracking limitations.
Without incrementality testing, marketing teams risk overstating campaign value. A user who converts after seeing an ad might have converted anyway – a phenomenon known as "cannibalisation." Budget allocated based on inflated attribution claims gets misallocated away from truly high-performing channels.
For FMCG brands, ecommerce players, and financial services – major sectors in the UK – incrementality testing reveals whether campaigns drive genuine incremental revenue or merely capture existing demand at higher cost.
When to Use Incrementality Testing
Incrementality testing works best for:
- High-volume campaigns with sufficient budget and scale to create meaningful control groups (typically £50k+)
- Direct response channels where conversion data is trackable (paid search, display, email, social)
- Mature campaigns where you need precision ROI rather than vanity metrics
- Major budget decisions where understanding true effectiveness justifies test costs
- Privacy-first environments where first-party data and causal testing replace attribution models
Key Considerations
Duration: Most tests run 2–4 weeks minimum to capture enough conversions and account for purchase cycles.
Scale: Smaller campaigns may lack statistical power; control groups typically need 10,000–50,000+ users depending on conversion rate.
Cost: Testing reduces short-term performance for the control group, effectively costing the incremental revenue forgone during the test period.
Holdout fatigue: Running repeated tests on the same audience may reduce control group relevance over time.
Incrementality vs. Attribution
Attribution models (first-touch, last-touch, data-driven) assign credit to touchpoints within a journey. Incrementality testing asks a different question: does the campaign itself drive uplift? This makes it more reliable for ROI measurement, particularly for awareness and mid-funnel campaigns where attribution struggles.
Best Practice
Combine incrementality testing with attribution data for a complete picture: use attribution to optimise within a channel, use incrementality to validate whether that channel truly drives incremental business value.