Exploitation and exploration in a performance based contextual advertising system


The dynamic marketplace in online advertising calls for ranking systems that are optimized to consistently promote and capitalize better performing ads. The streaming nature of online data inevitably makes an advertising system choose between maximizing its expected revenue according to its current knowledge in short term (exploitation) and trying to learn more about the unknown to improve its knowledge (exploration), since the latter might increase its revenue in the future. The exploitation and exploration (EE) tradeoff has been extensively studied in the reinforcement learning community, however, not been paid much attention in online advertising until recently. In this paper, we develop two novel EE strategies for online advertising. Specifically, our methods can adaptively balance the two aspects of EE by automatically learning the optimal tradeoff and incorporating confidence metrics of historical performance. Within a deliberately designed offline simulation framework we apply our algorithms to an industry leading performance based contextual advertising system and conduct extensive evaluations with real online event log data. The experimental results and detailed analysis reveal several important findings of EE behaviors in online advertising and demonstrate that our algorithms perform superiorly in terms of ad reach and click-through-rate (CTR).

DOI: 10.1145/1835804.1835811

Extracted Key Phrases

9 Figures and Tables

Citations per Year

82 Citations

Semantic Scholar estimates that this publication has 82 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Li2010ExploitationAE, title={Exploitation and exploration in a performance based contextual advertising system}, author={Wei Li and Xuerui Wang and Ruofei Zhang and Ying Cui and Jianchang Mao and Rong Jin}, booktitle={KDD}, year={2010} }