What is a holdout test in CTV advertising and how do you run one in India?

A holdout test (also called a ghost bidding or geo holdout test) measures the true incremental impact of a CTV campaign by comparing outcomes between a group exposed to the ad (test group) and a comparable group that was deliberately withheld from the campaign (holdout group). Unlike standard post-campaign measurement, which attributes all observed outcomes to the ad, a holdout test isolates the causal effect by controlling for baseline behaviour. For example: if a CTV campaign appears to drive a 15% lift in brand search queries, a holdout test can determine how much of that lift would have happened anyway — revealing the true incremental lift (which might be 8%, not 15%).

For an India CTV holdout test: (1) Split the target audience randomly at device ID level — 80% in the test group (receives the ad), 20% in the holdout (suppressed from all campaign delivery). The holdout must be enforced at the DSP level. (2) Run the campaign for a minimum of 4–6 weeks to accumulate sufficient exposure events. (3) After the campaign, compare the business outcome metric (brand search, app install, site visit) between test and holdout groups. The difference between the two is the incremental lift attributable to the CTV campaign. Minimum holdout group size for statistical significance: 50,000 devices in each group. For India CTV, DSPs that support holdout testing include DV360 (via Brand Lift) and The Trade Desk (via Koa measurement).

Three practical limitations for India CTV holdout tests: (1) SSAI inventory exclusions — JioHotstar’s SSAI environment means the DSP cannot reliably suppress specific devices from delivery, making clean holdout groups harder to maintain on JioHotstar-heavy campaigns. Holdout tests work most cleanly on CSAI inventory. (2) Cross-channel contamination — if the holdout group is exposed to the same brand on other channels (TV, digital, radio) during the test period, the measured incremental lift is understated. India’s multi-channel media environment makes clean holdouts difficult without isolating all channels simultaneously. (3) Co-viewing — a device in the holdout group may be in the same household as a device in the test group, contaminating the holdout through household ad exposure.