Top

Published in:

2022 | OriginalPaper | Chapter

Online Advertising Incrementality Testing: Practical Lessons, Paid Search and Emerging Challenges

Authors : Joel Barajas, Narayan Bhamidipati, James G. Shanahan

Published in: Advances in Information Retrieval

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Online advertising has historically been approached as an ad-to-user matching problem within sophisticated optimization algorithms. As the research and ad tech industries have progressed, advertisers have increasingly emphasized the causal effect estimation of their ads (incrementality) using controlled experiments (A/B testing). With low lift effects and sparse conversion, the development of incrementality testing platforms at scale suggests tremendous engineering challenges in measurement precision. Similarly, the correct interpretation of results addressing a business goal requires significant data science and experimentation research expertise.

We propose a practical tutorial in the incrementality testing landscape, including:

– The business need

– Literature solutions and industry practices

– Designs in the development of testing platforms

– The testing cycle, case studies, and recommendations

– Paid search effectiveness in the marketplace

– Emerging privacy challenges for incrementality testing and research solutions

We provide first-hand lessons based on the development of such a platform in a major combined DSP and ad network, and after running several tests for up to two months each over recent years. With increasing privacy constraints, we survey literature and current practices. These practices include private set union and differential privacy for conversion modeling, and geo-testing combined with synthetic control techniques.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter ROMCIR 2022: Overview of the 2nd Workshop on Reducing Online Misinformation Through Credible Information Retrieval

next chapter From Fundamentals to Recent Advances: A Tutorial on Keyphrasification

Abadie, A., Diamond, A., Hainmueller, J.: Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J. Am. Stat. Assoc. 105(490), 493–505 (2010)MathSciNetCrossRef

Barajas, J., Akella, R., Holtan, M., Flores, A.: Experimental designs and estimation for online display advertising attribution in marketplaces. Mark. Sci. 35(3), 465–483 (2016)CrossRef

Barajas, J., Bhamidipati, N.: Incrementality testing in programmatic advertising: enhanced precision with double-blind designs. In: Proceedings of the Web Conference 2021, pp. 2818–2827. WWW 2021, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3442381.3450106

Barajas, J., Bhamidipati, N., Shanahan, J.G.: Online advertising incrementality testing and experimentation: industry practical lessons. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 4027–4028. KDD 2021, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3447548.3470819

Barajas, J., Bhamidipati, N., Shanahan, J.G.: Online advertising incrementality testing: practical lessons and emerging challenges. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 4838–4841. CIKM 2021, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3459637.3482031

Barajas, J., Zidar, T., Bay, M.: Advertising incrementality measurement using controlled geo-experiments: the universal app campaign case study (2020)

Blake, T., Nosko, C., Tadelis, S.: Consumer heterogeneity and paid search effectiveness: a large-scale field experiment. Econometrica 83(1), 155–174 (2015)MathSciNetCrossRefMATH

Chan, D., Ge, R., Gershony, O., Hesterberg, T., Lambert, D.: Evaluating online ad campaigns in a pipeline: causal models at scale. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 7–16. KDD 2010, ACM, New York, NY, USA (2010). https://doi.org/10.1145/1835804.1835809,http://doi.acm.org/10.1145/1835804.1835809

Dai, D., Luca, M.: Effectiveness of paid search advertising: Experimental evidence. Technical report, Harvard Business School (October 2016). workin Paper No. 17–025

10.

Farahat, A., Shanahan, J.: Econometric analysis and digital marketing: how to measure the effectiveness of an ad. In: Proceedings of the sixth ACM International Conference on Web Search and Data Mining, pp. 785–785 (2013)

11.

Frangakis, C., Rubin, D.: Principal stratification in causal inference. Biometrics 58(1), 21–29 (2002). https://doi.org/10.1111/j.0006-341X.2002.00021.xMathSciNetCrossRefMATH

12.

Gordon, B.R., Zettelmeyer, F., Bhargava, N., Chapsky, D.: A comparison of approaches to advertising measurement: evidence from big field experiments at Facebook. Mark. Sci. 38(2), 193–225 (2019)CrossRef

13.

Imbens, G.W., Rubin, D.B.: Bayesian inference for causal effects in randomized experiments with noncompliance. Ann. Stat. 25(1), 305–327 (1997). http://www.jstor.org/stable/2242722

14.

Johnson, G.A., Lewis, R.A., Nubbemeyer, E.I.: Ghost ads: improving the economics of measuring online ad effectiveness. J. Mark. Res. 54(6), 867–884 (2017). https://doi.org/10.1509/jmr.15.0297

15.

Kireyev, P., Pauwels, K., Gupta, S.: Do display ads influence search? Attribution and dynamics in online advertising. Int. J. Res. Mark. 33(3), 475–490 (2016)CrossRef

16.

Kohavi, R., Tang, D., Xu, Y.: Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press, Cambridge (2020)

17.

Lewis, R.A., Rao, J.M.: The unfavorable economics of measuring the returns to advertising *. Q. J. Econ. 130(4), 1941–1973 (2015)CrossRef

18.

Lewis, R.A., Rao, J.M., Reiley, D.H.: Here, there, and everywhere: correlated online behaviors can lead to overestimates of the effects of advertising. In: Proceedings of the 20th International Conference on World Wide Web, pp. 157–166. WWW 2011, ACM, New York, NY, USA (2011). https://doi.org/10.1145/1963405.1963431,http://doi.acm.org/10.1145/1963405.1963431

19.

Li, H.A., Kannan, P.: Attributing conversions in a multichannel online marketing environment: an empirical model and a field experiment. J. Mark. Res. 51(1), 40–56 (2014)CrossRef

20.

Lin, T., Misra, S.: The identity fragmentation bias (2020)

21.

Moshary, S.: Sponsored search in equilibrium: evidence from two experiments. Available at SSRN 3903602 (2021)

22.

Rubin, D.B.: Causal inference using potential outcomes. J. Am. Stat. Assoc. 100(469), 322–331 (2005). https://doi.org/10.1198/016214504000001880

Title: Online Advertising Incrementality Testing: Practical Lessons, Paid Search and Emerging Challenges
Authors: Joel Barajas
Narayan Bhamidipati
James G. Shanahan
Publisher: Springer International Publishing
Book: Advances in Information Retrieval
Print ISBN: 978-3-030-99738-0

Electronic ISBN: 978-3-030-99739-7

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-3-030-99739-7_72

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"