A holdout test (also called a geo holdout, incrementality test, or lift test) is the most reliable method for measuring CTV's causal contribution to conversions in India. Unlike view-through attribution, which depends on household identity matching with 20–40% match rates, holdout testing does not require linking CTV device IDs to converting devices. It measures whether the group exposed to CTV advertising converts at a higher rate than an equivalent unexposed group — the difference is the incremental lift attributable to CTV.
How holdout testing works for India CTV
The core methodology: divide the target audience or geography into two groups — a test group that receives CTV ads and a holdout group that does not. After the campaign flight, compare conversion rates between the two groups. The incremental conversion rate is the lift that CTV caused above the counterfactual (no CTV exposure).
In India, holdout tests are typically structured by geography rather than by individual user, because user-level random assignment in CTV is not technically reliable. A geo holdout test: select 20 Indian cities as the test market (CTV ads run) and 10 matched cities as the holdout (no CTV ads). Both groups receive other media. After the campaign, compare app installs, e-commerce orders, or website conversions between city groups. The difference — adjusted for baseline conversion rate differences — is CTV's incremental contribution.
Matching test and holdout markets
The validity of a geo holdout test depends on how well test and control cities are matched. Matching criteria for India CTV holdout tests:
- City tier (keep Tier 1 vs Tier 2 similar across test and control)
- Baseline conversion rate in the pre-campaign period
- Category penetration index (how over/under-indexed the city is for the category)
- Competitive CTV activity (avoid putting a city with heavy competitor CTV in the holdout)
- Media exposure similarity (test and holdout cities should have similar linear TV and digital pressure)
Minimum scale requirements for India CTV holdout tests
Minimum impressions: Typically 5–10 million impressions across test cities to generate detectable lift for a brand with moderate category penetration.
Minimum spend: India CTV holdout tests are typically feasible for campaigns above ₹50 lakh. Below this level, the test may lack statistical power to detect meaningful lift.
Minimum duration: 4–6 weeks is typically needed for conversion signals to accumulate, particularly for categories with longer purchase consideration cycles (BFSI, automotive, education).
Data requirements for holdout test analysis
The outcome metric must be measurable at the city level with sufficient granularity. Suitable outcome metrics for India CTV holdout tests:
- App installs (geo-attributed via AppsFlyer or Adjust city-level data)
- E-commerce orders (city-level order data from the advertiser's platform)
- Website sessions from organic search or direct traffic (city-level from GA4)
- Brand search volume (city-level from Google Search Console or SEM data)
Interpreting holdout test results
A 4% higher app install rate in test cities vs holdout cities, with statistical significance at 95% confidence, means CTV drove 4% incremental lift in app installs in the exposed geography. This number is more credible than VTA-based attribution because it does not depend on device ID matching — it is a direct comparison of real outcomes.
India-specific caution: geo holdout tests in India are noisier than in the US or UK because India cities have higher baseline conversion rate variance. Use the longest pre-campaign baseline period available (8–12 weeks) and match on category penetration, not just city tier. City-to-city economic differences in India are large enough to confound shorter baseline periods.