Every attribution model has the same fundamental limitation: it measures correlation, not causation.

When your multi-touch attribution model tells you that a channel drove 15% of conversions, it is telling you that users who converted were exposed to that channel. It is not telling you whether those users would have converted anyway. It is not telling you whether the channel caused the conversion, or whether it simply happened to be present in the journey of users who were already going to buy.

Incrementality testing answers the question attribution cannot: what would have happened if you had not run that campaign?

The Distinction That Changes Everything

Consider a retargeting campaign targeting users who visited your product pages. Your attribution model shows strong performance — high conversion rates, low cost per acquisition, excellent ROAS.

But ask a different question: of the users who saw your retargeting ads and converted, how many would have converted anyway without the ad?

If your retargeting audience consists primarily of users who were already deep in the purchase funnel — users who had added items to their cart, visited the checkout page, or returned to your site multiple times — the answer might be: most of them. The retargeting ad reached users who were already going to buy. It did not cause the conversion. It just happened to be there.

This is the incrementality problem. And it is not unique to retargeting. It affects every channel that targets users with demonstrated intent.

How Incrementality Testing Works

Incrementality testing uses controlled experiments to measure the causal effect of advertising. The core design is simple: split your audience into a treatment group that sees your ads and a holdout group that does not, then compare conversion rates between the two groups.

The difference in conversion rates — controlling for all other variables — is the incremental lift. It is the percentage of conversions that would not have happened without the advertising.

Geo-Lift Testing

For campaigns where user-level holdout groups are difficult to implement — particularly for upper-funnel awareness campaigns — geo-lift testing uses geographic regions as the unit of randomization.

You select matched pairs of geographic markets, run your campaign in one market of each pair, and measure the difference in outcomes between the treatment and control markets. The lift is the incremental effect of the campaign, isolated from baseline trends.

Geo-lift testing is particularly valuable for television, out-of-home, and digital campaigns where you cannot easily suppress ads for individual users.

User-Level Holdout Testing

For digital campaigns where you have more control over ad delivery, user-level holdout testing is more precise. You randomly assign users to treatment and control groups before the campaign launches, suppress ads for the control group, and measure the conversion rate difference.

The challenge with user-level holdouts is that most advertising platforms make it difficult to suppress ads for specific users. You need either platform-level holdout functionality or a technical implementation that prevents your holdout users from entering your campaign audiences.

Ghost Ad Testing

Ghost ad testing is a variant of user-level holdout testing where the control group sees a placeholder ad — typically a public service announcement or a non-competing brand ad — instead of your campaign creative. This controls for the attention effect of seeing an ad, isolating the specific effect of your campaign message.

Designing Tests That Produce Actionable Results

The mechanics of incrementality testing are straightforward. The hard part is designing tests that produce results you can act on.

Statistical power. Incrementality tests require sufficient conversion volume to detect meaningful lift. For campaigns with low conversion rates or small audiences, you may need to run tests for weeks or months to achieve statistical significance. Plan your test duration before you start, not after.

Holdout contamination. If users in your holdout group are exposed to your ads through other channels — organic social, word of mouth, or cross-device exposure — your test results will underestimate the true lift. Design your holdouts to minimize contamination, and account for it in your analysis.

Market selection for geo tests. The validity of a geo-lift test depends on how well your treatment and control markets match each other. Use historical data to select markets with similar baseline conversion rates, seasonality patterns, and demographic profiles. A poorly matched market pair will produce unreliable results regardless of how well you execute the test.

The counterfactual question. Be precise about what you are testing. Are you measuring the lift of running the campaign versus not running it? Or are you measuring the lift of one campaign strategy versus another? The counterfactual determines what your results mean and what decisions they support.

Integrating Incrementality with Attribution

Incrementality testing and attribution modeling are complementary, not competing.

Attribution models are useful for understanding the relative contribution of different touchpoints within a converting journey. They are good at answering questions like: given that a user converted, which channels were most important in their path?

Incrementality testing is useful for understanding the causal effect of advertising investment. It answers questions like: if we increase spend in this channel by 20%, how many additional conversions will we generate?

The most sophisticated enterprise marketing organizations use both. They use attribution to understand the customer journey and optimize touchpoint sequencing. They use incrementality testing to validate their attribution model's recommendations and calibrate their budget allocation decisions.

The Organizational Challenge

Incrementality testing requires something that is genuinely difficult in most enterprise marketing organizations: the willingness to withhold advertising from a portion of your audience in order to learn.

The holdout group is, by definition, a group of potential customers who are not seeing your ads. In the short term, this costs conversions. The argument for accepting that cost is that the knowledge you gain allows you to make better decisions that more than offset the short-term loss.

Making that argument to stakeholders who are evaluated on quarterly conversion targets requires organizational alignment and executive sponsorship. The teams that have built incrementality testing into their standard operating procedures have typically done so with explicit support from leadership and a clear framework for communicating the value of the learning investment.

What Incrementality Testing Reveals

The results of incrementality testing are often surprising — and almost always actionable.

Teams that have run systematic incrementality programs typically find that their highest-ROAS channels have lower incremental lift than expected, and their lowest-ROAS channels have higher incremental lift than expected. The channels that look most efficient in attribution are often the ones capturing intent that was built elsewhere. The channels that look least efficient are often the ones actually building that intent.

This does not mean you should shift all your budget to low-ROAS channels. It means you should understand what each channel is actually doing in your marketing system — and make budget decisions based on that understanding rather than on attribution metrics that conflate correlation with causation.

That is the difference between reporting and strategy.