# Strong evidence for ads versus content

## Conclusion

Existing public research does not provide a credible universal estimate of how many paying customers $100 of content produces versus $100 of advertising. The defensible research conclusion is that channel performance is heterogeneous and that platform attribution or before–after comparisons cannot establish incrementality. A business-specific randomized experiment is required.

## Evidence

Large-scale paid-search experiments at eBay found that observational attribution substantially overstated causal returns. Brand-keyword advertising produced no measurable short-term revenue benefit, while non-brand advertising benefited new and infrequent users but had negative average returns because frequent users absorbed much of the spending without changing their purchasing behavior ([Blake, Nosko, and Tadelis 2015](https://doi.org/10.3982/ECTA12423)).

This is not an isolated measurement problem. Fifteen Facebook advertising experiments covering roughly 500 million user-experiment observations showed that common observational approaches often failed to reproduce randomized estimates even after extensive demographic and behavioral adjustment ([Gordon, Zettelmeyer, Bhargava, and Chapsky 2019](https://doi.org/10.1287/mksc.2018.1135)).

Even randomized advertising studies require very large samples because incremental sales effects are generally small relative to natural variation in customer spending. Evidence from 25 large field experiments, collectively involving $2.8 million in digital advertising expenditure, showed that return-on-advertising estimates can remain imprecise despite millions of observations ([Lewis and Rao 2015](https://doi.org/10.1093/qje/qjv023)).

The content-marketing literature is weaker for this question. Some randomized interventions show that branded content can affect attitudes, loyalty, or purchase intentions, but these outcomes are not acquired customers or incremental revenue. For example, a randomized online intervention involving YouTube users measured purchase intentions rather than observed sales ([Lou, Xie, Feng, and Kim 2019](https://doi.org/10.1108/JPBM-07-2018-1948)). Field experiments show that particular content elements can affect actual purchases—for example, alternative product images increased purchase likelihood in an e-commerce experiment—but such treatment effects do not estimate the return to an entire content-marketing program ([Overmars and Poels 2019](https://doi.org/10.1016/j.dss.2019.04.008)).

Consequently, the available evidence cannot justify the general claim that spending $100 on content acquires more customers than spending $100 on ads.

## Defensible experiment

Randomly assign comparable geographic areas, accounts, or audience clusters to three conditions:

1. **Ads:** incremental paid-media budget with normal content held constant.
2. **Content:** incremental content-production and distribution budget with paid media held constant.
3. **Control:** business-as-usual marketing.

Pre-register the primary outcome as incremental first-time paying customers per $100 of fully loaded cost. Content costs should include staff and contractor time, production, software, and distribution. Advertising costs should include media, creative production, and management fees.

Run the experiment long enough to capture content maturation and delayed conversion—normally at least three to six months, and longer for infrequent-purchase or B2B products. Use the same attribution window for every arm, but estimate effects from randomized assignment rather than last-click attribution.

Report:

- incremental customers per $100;
- incremental gross profit per $100;
- customer acquisition cost;
- confidence intervals;
- results by new versus existing customer;
- sensitivity to the treatment period and post-treatment carryover.

The winner should be the arm with the larger statistically credible incremental customer or gross-profit effect—not the arm with more platform-attributed leads.
