A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook

@article{Gordon2019ACO,
  title={A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook},
  author={Brett R. Gordon and Florian Zettelmeyer and Neha Bhargava and Daniel Chapsky},
  journal={Managerial Marketing eJournal},
  year={2019}
}
Observational methods often fail to accurately recover the treatment effects generated from randomized advertising experiments on Facebook. 

Figures from this paper

Spillovers from Mass Advertising: An Identification Strategy
TLDR
Estimates from this “spillover strategy” agree with recent literature that suggests many standard approaches to estimating the response to advertising may produce biased results because of unobservables and suggest that some recent empirical strategies, such as the DMA-border strategy, can produce biased estimates for seasonal products.
Exploration in Action: The Role of Randomized Control Trials in Online Demand Generation
Randomized control trials (RCTs) are widely lauded as the “gold standard” for inference, providing unbiased estimates of the causal effect of firms’ policies and allowing for better decisions, e.g.,
Display Ad Measurement Using Observational Data: A Reinforcement Learning Approach
TLDR
An observational framework derived from Markov Decision Processes is introduced for measuring the effects of display advertising and a novel estimation method is developed that recovers the incremental impact on outcomes attributable to a display advertising campaign and the individual impressions served.
Fake News and Advertising on Social Media: A Study of the Anti-Vaccination Movement
Online sources sometimes publish information that is false or intentionally misleading. We study the role of social networks and advertising on social networks in the dissemination of false news
Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness
TLDR
The authors show that, relative to public service announcement and intent-to-treat A/B tests, ghost ads can reduce the cost of experimentation, improve measurement precision, deliver the relevant strategic baseline, and work with modern ad platforms that optimize ad delivery in real time.
Appendix O: measurement issues in digital advertising
  • Business
  • 2020
1. For there to be effective competition between buyers and suppliers of digital advertising, advertisers need to be able to assess and evaluate the quality and effectiveness of the digital
Causal Inference Struggles with Agency on Online Platforms
TLDR
Four large-scale within-study comparisons on Twitter are conducted aimed at assessing the effectiveness of observational studies derived from user self-selection on online platforms, suggesting that observational studiesderived from userSelf-selection are a poor alternative to randomized experimentation on online Platforms.
Competitive Advertising on Brand Search: Traffic Stealing and Click Quality
Competitive advertising on brand keywords of rivals in sponsored search is effective in generating traffic, if rivals do not defend their traffic.
Inefficiencies in Digital Advertising Markets
Digital advertising markets are growing and attracting increased scrutiny. This article explores four market inefficiencies that remain poorly understood: ad effect measurement, frictions between and
Attribution Inference for Digital Advertising using Inhomogeneous Poisson Models
TLDR
This work presents a new observational attribution method based on a successful model of neural spiking that learns the temporal interactions between event-based time series and shows that it can accurately recover the true causal attribution.
...
...

References

SHOWING 1-10 OF 91 REFERENCES
Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness
TLDR
The authors show that, relative to public service announcement and intent-to-treat A/B tests, ghost ads can reduce the cost of experimentation, improve measurement precision, deliver the relevant strategic baseline, and work with modern ad platforms that optimize ad delivery in real time.
Ghost Ads: Improving the Economics of Measuring Ad Effectiveness
To measure the effects of advertising, marketers must know how consumers would behave had they not seen the ads. We develop a methodology we call ‘Ghost Ads,’ which facilitates this comparison by
Online Display Advertising: Targeting and Obtrusiveness
TLDR
It is found that matching an ad to website content and increasing an ad's obtrusiveness independently increase purchase intent, however, in combination, these two strategies are ineffective.
Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!
A randomized experiment with 1.6 million customers measures positive causal effects of online advertising for a major retailer. The advertising profitably increases purchases by 5%. 93% of the
Here, there, and everywhere: correlated online behaviors can lead to overestimates of the effects of advertising
TLDR
Using three controlled experiments, it is shown that observational data frequently lead to incorrect estimates of adFX, and how and why observational methods lead to a massive overestimate of adfx in such circumstances.
When Less Is More: Data and Power in Advertising Experiments
Yahoo! Research partnered with a nationwide retailer to study the effects of online display advertising on both online and in-store purchases. We use a randomized field experiment on 3 million Yahoo!
Native Advertising, Sponsorship Disclosure and Consumer Deception: Evidence from Mobile Search-Ad Experiments
Recent advances in advertising technology have lead to the development of “native advertising”, which is a format of advertising that mimics the other non-sponsored content on the medium. While
Bias and High-Dimensional Adjustment in Observational Studies of Peer Effects
TLDR
It is shown that high-dimensional adjustment of a nonexperimental control group using propensity score models produces estimates of peer effects statistically indistinguishable from those using a large randomized experiment, which demonstrates how detailed records of behavior can improve studies of social influence, information diffusion, and imitation.
Effect of temporal spacing between advertising exposures: Evidence from online field experiments
This paper aims to understand the impact of temporal spacing between ad exposures on the likelihood of a consumer purchasing the advertised product. I create an individual-level data set with
Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment
Internet advertising has been the fastest growing advertising channel in recent years, with paid search ads comprising the bulk of this revenue. We present results from a series of large‐scale field
...
...