Skip to main content
Top

2022 | OriginalPaper | Chapter

Exploiting Data from Field Experiments

Authors : Martin Artz, Hannes Doering

Published in: Handbook of Market Research

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This chapter gives an introduction on how to exploit data from field experiments and aims to provide an intuitive understanding for managers and researchers alike. We outline the relevance and hurdles in identifying causal effects compared to observing purely correlational associations in studies which take place in the real world. We further provide a framework to classify different kinds of field experiments, such as quasi field experiments and natural field experiments. The core of this chapter focuses on giving an understanding of three standard econometric methods to exploit data from field experiments: difference-in-differences, regression discontinuity, and instrumental variables. For each method, we provide an intuitive understanding of the core features and its critical assumptions. We complement those explanations with an in-depth look at one practical application of each method in a field experiment setting and with a variety of practical examples from recently published research. Lastly, we provide a brief overview on how to implement each method in standard software packages such as STATA, R, and SPSS.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
go back to reference Abadie, A. (2020). Using synthetic controls: Feasibility, data requirements, and methodological aspects. Journal of Economic Literature. (forthcoming). Abadie, A. (2020). Using synthetic controls: Feasibility, data requirements, and methodological aspects. Journal of Economic Literature. (forthcoming).
go back to reference Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490), 493–505.CrossRef Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490), 493–505.CrossRef
go back to reference Acemoglu, D., Johnson, S., Kermani, A., Kwak, J., & Mitton, T. (2016). The value of connections in turbulent times: Evidence from the United States. Journal of Financial Economics, 121(2), 368–391.CrossRef Acemoglu, D., Johnson, S., Kermani, A., Kwak, J., & Mitton, T. (2016). The value of connections in turbulent times: Evidence from the United States. Journal of Financial Economics, 121(2), 368–391.CrossRef
go back to reference Aghamolla, C., & Li, N. (2018). Debt contract enforcement and conservatism: Evidence from a natural experiment. Journal of Accounting Research, 56(5), 1383–1416.CrossRef Aghamolla, C., & Li, N. (2018). Debt contract enforcement and conservatism: Evidence from a natural experiment. Journal of Accounting Research, 56(5), 1383–1416.CrossRef
go back to reference Angrist, J. D., & Krueger, A. B. (1991). Does compulsory school attendance affect schooling and earnings? Quarterly Journal of Economics, 106(4), 989–1014.CrossRef Angrist, J. D., & Krueger, A. B. (1991). Does compulsory school attendance affect schooling and earnings? Quarterly Journal of Economics, 106(4), 989–1014.CrossRef
go back to reference Angrist, J. D., & Krueger, A. B. (1999). Empirical strategies in labor economics. In O. Ashenfelter & D. Card (Eds.), Handbook of labor economics (Vol. 3, pp. 1277–1366). Amsterdam: North-Holland. Angrist, J. D., & Krueger, A. B. (1999). Empirical strategies in labor economics. In O. Ashenfelter & D. Card (Eds.), Handbook of labor economics (Vol. 3, pp. 1277–1366). Amsterdam: North-Holland.
go back to reference Angrist, J. D., & Pischke, J. S. (2009). Mostly harmless econometrics. Princeton: Princeton University Press.CrossRef Angrist, J. D., & Pischke, J. S. (2009). Mostly harmless econometrics. Princeton: Princeton University Press.CrossRef
go back to reference Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.CrossRef Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.CrossRef
go back to reference Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). On making causal claims: A review and recommendations. The Leadership Quarterly, 21(1), 1086–1120.CrossRef Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). On making causal claims: A review and recommendations. The Leadership Quarterly, 21(1), 1086–1120.CrossRef
go back to reference Armstrong, C. S., Glaeser, S., & Huang, S. (2019). Controllability of risk and the design of incentive-compensation contracts. Working paper. Available at SSRN 2896147. Armstrong, C. S., Glaeser, S., & Huang, S. (2019). Controllability of risk and the design of incentive-compensation contracts. Working paper. Available at SSRN 2896147.
go back to reference Atanasov, V., & Black, B. (2016). Shock-based causal inference in corporate finance and accounting research. Critical Finance Review, 5(1), 207–304.CrossRef Atanasov, V., & Black, B. (2016). Shock-based causal inference in corporate finance and accounting research. Critical Finance Review, 5(1), 207–304.CrossRef
go back to reference Bennedsen, M., Nielsen, K. M., Perez-Gonzalez, F., & Wolfenzon, D. (2007). Inside the family firm: The role of families in succession decisions and performance. The Quarterly Journal of Economics, 122(2), 647–691.CrossRef Bennedsen, M., Nielsen, K. M., Perez-Gonzalez, F., & Wolfenzon, D. (2007). Inside the family firm: The role of families in succession decisions and performance. The Quarterly Journal of Economics, 122(2), 647–691.CrossRef
go back to reference Bertrand, M., Karlan, D., Mullainathan, S., Shafir, E., & Zinman, J. (2010). What’s advertising content worth? Evidence from a consumer credit marketing field experiment. The Quarterly Journal of Economics, 125(1), 263–306.CrossRef Bertrand, M., Karlan, D., Mullainathan, S., Shafir, E., & Zinman, J. (2010). What’s advertising content worth? Evidence from a consumer credit marketing field experiment. The Quarterly Journal of Economics, 125(1), 263–306.CrossRef
go back to reference Blake, T., Nosko, C., & Tadelis, S. (2015). Consumer heterogeneity and paid search effectiveness: A large scale field experiment. Econometrica, 83(1), 155–174.CrossRef Blake, T., Nosko, C., & Tadelis, S. (2015). Consumer heterogeneity and paid search effectiveness: A large scale field experiment. Econometrica, 83(1), 155–174.CrossRef
go back to reference Casas-Arce, P., Lourenço, S. M., & Martínez-Jerez, F. (2017a). The performance effect of feedback frequency and detail: Evidence from a field experiment in customer satisfaction. Journal of Accounting Research, 55(5), 1051–1088.CrossRef Casas-Arce, P., Lourenço, S. M., & Martínez-Jerez, F. (2017a). The performance effect of feedback frequency and detail: Evidence from a field experiment in customer satisfaction. Journal of Accounting Research, 55(5), 1051–1088.CrossRef
go back to reference Casas-Arce, P., Martínez-Jerez, F., & Narayanan, V. G. (2017b). The impact of forward-looking metrics on employee decision-making: The case of customer lifetime value. The Accounting Review, 92(3), 31–56.CrossRef Casas-Arce, P., Martínez-Jerez, F., & Narayanan, V. G. (2017b). The impact of forward-looking metrics on employee decision-making: The case of customer lifetime value. The Accounting Review, 92(3), 31–56.CrossRef
go back to reference Flammer, C. (2015). Does corporate social responsibility lead to superior financial performance? A regression discontinuity approach. Management Science, 61(11), 2549–2568.CrossRef Flammer, C. (2015). Does corporate social responsibility lead to superior financial performance? A regression discontinuity approach. Management Science, 61(11), 2549–2568.CrossRef
go back to reference Gassen, J., & Muhn, M. (2018). Financial transparency of private firms: Evidence from a randomized field experiment. Working paper. Available at SSRN 3290710. Gassen, J., & Muhn, M. (2018). Financial transparency of private firms: Evidence from a randomized field experiment. Working paper. Available at SSRN 3290710.
go back to reference Gelman, A., & Imbens, G. W. (2019). Why high-order polynomials should not be used in regression discontinuity designs. Journal of Business and Economic Statistics, 37(3), 447–456.CrossRef Gelman, A., & Imbens, G. W. (2019). Why high-order polynomials should not be used in regression discontinuity designs. Journal of Business and Economic Statistics, 37(3), 447–456.CrossRef
go back to reference Gill, M., Sridhar, S., & Grewal, R. (2017). Return on engagement initiatives: A study of a business-to-business mobile app. Journal of Marketing, 81(4), 45–66.CrossRef Gill, M., Sridhar, S., & Grewal, R. (2017). Return on engagement initiatives: A study of a business-to-business mobile app. Journal of Marketing, 81(4), 45–66.CrossRef
go back to reference Giroud, X., Mueller, H. M., Stomper, A., & Westerkamp, A. (2012). Snow and leverage. The Review of Financial Studies, 25(3), 680–710.CrossRef Giroud, X., Mueller, H. M., Stomper, A., & Westerkamp, A. (2012). Snow and leverage. The Review of Financial Studies, 25(3), 680–710.CrossRef
go back to reference Goldfarb, A., & Tucker, C. E. (2011). Privacy regulation and online advertising. Management Science, 57(1), 57–71.CrossRef Goldfarb, A., & Tucker, C. E. (2011). Privacy regulation and online advertising. Management Science, 57(1), 57–71.CrossRef
go back to reference Goldfarb, A., & Tucker, C. E. (2014). Conducting research with quasi-experiments: A guide for marketers. Working paper. Available at SSRN 2420920. Goldfarb, A., & Tucker, C. E. (2014). Conducting research with quasi-experiments: A guide for marketers. Working paper. Available at SSRN 2420920.
go back to reference Harrison, G. W., & List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009–1055.CrossRef Harrison, G. W., & List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009–1055.CrossRef
go back to reference Heese, J., & Pérez-Calvazos, G. (2020). When the boss comes to town: The effect of headquarter’s visits on facility-level misconduct. The Accounting Review, 95(6), 235–261.CrossRef Heese, J., & Pérez-Calvazos, G. (2020). When the boss comes to town: The effect of headquarter’s visits on facility-level misconduct. The Accounting Review, 95(6), 235–261.CrossRef
go back to reference Imbens, G. W., & Kalyanaraman, K. (2012). Optimal bandwidth choice for the regression discontinuity estimator. The Review of Economic Studies, 79(3), 933–959.CrossRef Imbens, G. W., & Kalyanaraman, K. (2012). Optimal bandwidth choice for the regression discontinuity estimator. The Review of Economic Studies, 79(3), 933–959.CrossRef
go back to reference Keele, L. (2015). The discipline of identification. PS: Political Science & Politics, 48(1), 102–106. Keele, L. (2015). The discipline of identification. PS: Political Science & Politics, 48(1), 102–106.
go back to reference Kenny, D. A. (1979). Correlation and causality. New York: Wiley-Interscience. Kenny, D. A. (1979). Correlation and causality. New York: Wiley-Interscience.
go back to reference Keys, B. J., Mukherjee, T., Seru, A., & Vig, V. (2010). Did securitization lead to lax screening? Evidence from subprime loans. The Quarterly Journal of Economics, 125(1), 307–362.CrossRef Keys, B. J., Mukherjee, T., Seru, A., & Vig, V. (2010). Did securitization lead to lax screening? Evidence from subprime loans. The Quarterly Journal of Economics, 125(1), 307–362.CrossRef
go back to reference Ladika, T., & Sautner, Z. (2020). Managerial short-termism and investment: Evidence from accelerated option vesting. Review of Finance, 24(2), 305–344. Ladika, T., & Sautner, Z. (2020). Managerial short-termism and investment: Evidence from accelerated option vesting. Review of Finance, 24(2), 305–344.
go back to reference Lawrence, A., Ryans, J., Sun, E., & Laptev, N. (2018). Earnings announcement promotions: A Yahoo Finance field experiment. Journal of Accounting and Economics, 66(2–3), 399–414.CrossRef Lawrence, A., Ryans, J., Sun, E., & Laptev, N. (2018). Earnings announcement promotions: A Yahoo Finance field experiment. Journal of Accounting and Economics, 66(2–3), 399–414.CrossRef
go back to reference Lourenço, S. M. (2019). Field experiments in managerial accounting research. Foundations and Trends in Accounting, 14(1), 1–72.CrossRef Lourenço, S. M. (2019). Field experiments in managerial accounting research. Foundations and Trends in Accounting, 14(1), 1–72.CrossRef
go back to reference McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142(2), 698–714.CrossRef McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142(2), 698–714.CrossRef
go back to reference Michels, J. (2017). Disclosure versus recognition: Inferences from subsequent events. Journal of Accounting Research, 55(1), 3–34.CrossRef Michels, J. (2017). Disclosure versus recognition: Inferences from subsequent events. Journal of Accounting Research, 55(1), 3–34.CrossRef
go back to reference Pattabhiramaiah, A., Sriram, S., & Manchanda, P. (2019). Paywalls: Monetizing online content. Journal of Marketing, 83(2), 19–36.CrossRef Pattabhiramaiah, A., Sriram, S., & Manchanda, P. (2019). Paywalls: Monetizing online content. Journal of Marketing, 83(2), 19–36.CrossRef
go back to reference Presslee, A., Vance, T. W., & Webb, R. A. (2013). The effects of reward type on employee goal setting, goal commitment, and performance. The Accounting Review, 88(5), 1805–1831.CrossRef Presslee, A., Vance, T. W., & Webb, R. A. (2013). The effects of reward type on employee goal setting, goal commitment, and performance. The Accounting Review, 88(5), 1805–1831.CrossRef
go back to reference Roberts, M. R., & Whited, T. M. (2013). Endogeneity in empirical corporate finance. In G. M. Constantinides, M. Harris, & R. M. Stulz (Eds.), Handbook of the economics of finance (Vol. 2(A), pp. 493–572). Oxford: North Holland.CrossRef Roberts, M. R., & Whited, T. M. (2013). Endogeneity in empirical corporate finance. In G. M. Constantinides, M. Harris, & R. M. Stulz (Eds.), Handbook of the economics of finance (Vol. 2(A), pp. 493–572). Oxford: North Holland.CrossRef
go back to reference Sanderson, E., & Windmeijer, F. (2016). A weak instrument F-test in linear IV models with multiple endogenous variables. Journal of Econometrics, 190(2), 212–221.CrossRef Sanderson, E., & Windmeijer, F. (2016). A weak instrument F-test in linear IV models with multiple endogenous variables. Journal of Econometrics, 190(2), 212–221.CrossRef
go back to reference Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
go back to reference Shriver, S. K., Nair, H. S., & Hofstetter, R. (2013). Social ties and user-generated content: Evidence from an online social network. Management Science, 59(6), 1435–1443.CrossRef Shriver, S. K., Nair, H. S., & Hofstetter, R. (2013). Social ties and user-generated content: Evidence from an online social network. Management Science, 59(6), 1435–1443.CrossRef
go back to reference Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72–101.CrossRef Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72–101.CrossRef
go back to reference Stock, J. H., & Yogo, M. (2005). Asymptotic distributions of instrumental variables statistics with many instruments. In D. W. K. Andrews & J. H. Stock (Eds.), Identification and inference for econometric models. New York: Cambridge University Press. Stock, J. H., & Yogo, M. (2005). Asymptotic distributions of instrumental variables statistics with many instruments. In D. W. K. Andrews & J. H. Stock (Eds.), Identification and inference for econometric models. New York: Cambridge University Press.
go back to reference Verbeek, M. (2008). A guide to modern econometrics (3rd ed.). West Sussex: John Wiley & Sons Ltd. Verbeek, M. (2008). A guide to modern econometrics (3rd ed.). West Sussex: John Wiley & Sons Ltd.
go back to reference Wooldridge, J. M. (2012). Introductory econometrics: A modern approach (4th ed.). Mason: South-Western Cengage. Wooldridge, J. M. (2012). Introductory econometrics: A modern approach (4th ed.). Mason: South-Western Cengage.
Metadata
Title
Exploiting Data from Field Experiments
Authors
Martin Artz
Hannes Doering
Copyright Year
2022
DOI
https://doi.org/10.1007/978-3-319-57413-4_36