From Attributed to Caused: Operationalizing Always-on Audience Holdouts for Omnichannel Incrementality in Big-Box Retail

Abstract

Advertising measurement across networks is difficult to compare because platforms use different exposure definitions, attribution windows, identity rules, and counterfactuals. This paper describes an operational protocol for always-on audience holdouts in a large omnichannel retail setting. Eligible first-party customers are assigned by stable randomization to a persistent portfolio holdout or the addressable media population; paid delivery is suppressed to the holdout as completely as platform capabilities permit, and outcomes are read from retailer-owned omnichannel transaction data, including in-store sales. The primary causal estimand is the intent-to-treat difference in sales between randomized populations, not a customer-level propensity residual or a platform-reported conversion count. Match rate, suppression leakage, and exposure timing are treated as compliance and interpretation diagnostics rather than substitutes for randomization. The global holdout identifies portfolio-level incrementality. It does not, by itself, identify additive channel contributions; channel-specific causal effects require separate randomized channel holdouts under the same outcome and read-window rules. Channels that cannot support person-level suppression are evaluated through geo experiments or remain outside the strict protocol. Because suppression can be imperfect, estimates may be attenuated toward zero under stated assumptions and are reported with confidence intervals and leakage diagnostics. No quantitative outcomes are reported because the paper focuses on the design and governance of the framework rather than an evaluation of a named deployment. The contribution is a methods and governance architecture for making omnichannel incrementality reproducible and retailer-owned.