What is a CTV holdout test and how does it work?

A holdout test (also called an incrementality test or lift test) is a controlled experiment that measures whether CTV advertising caused additional conversions beyond what would have happened without it. The methodology: divide the target market into two groups — a test group exposed to CTV ads and a holdout group deliberately not exposed. After the campaign, compare conversion rates between the two groups. The difference is the incremental lift that CTV caused. In India, holdout tests are typically structured by geography (test cities vs holdout cities) rather than by individual user, because user-level random assignment in CTV is not technically reliable.

View-through attribution requires linking the CTV device that received an impression to the mobile device that converted — a match that only succeeds for 20–40% of India CTV impressions due to device graph gaps and dynamic IP addresses. A holdout test does not require this link. It simply compares real conversion outcomes between two groups — one exposed to CTV, one not. The comparison captures the full effect of CTV exposure including unmeasured household members, co-viewers, and conversions that happened without a traceable device ID link. Holdout tests are causal; VTA is correlational.

India CTV holdout tests are typically feasible for campaigns above ₹50 lakh. Below this level, the campaign may not deliver enough impressions across test cities (typically 5–10 million impressions) to generate statistically significant lift signals. The test should run for at least 4–6 weeks, with 8–12 weeks of pre-campaign baseline data for matching test and control city pairs. The outcome metric must be measurable at the city level — app installs via AppsFlyer, e-commerce orders from the advertiser's platform, or organic search volume from Google Search Console are the most commonly used city-level metrics in India.