Skip to main content
SpringerLink
Log in

Effect sizes and the interpretation of research results in international business

  • Research Note
  • Published:
Journal of International Business Studies Aims and scope Submit manuscript

Abstract

Journal editors and academy presidents are increasingly calling on researchers to evaluate the substantive, as opposed to the statistical, significance of their results. To measure the extent to which these calls have been heeded, I aggregated the meta-analytically derived effect size estimates obtained from 965 individual samples. I then surveyed 204 studies published in the Journal of International Business Studies. I found that the average effect size in international business research is small, and that most published studies lack the statistical power to detect such effects reliably. I also found that many authors confuse statistical with substantive significance when interpreting their research results. These practices have likely led to unacceptably high Type II error rates and invalid inferences regarding real-world effects. By emphasizing p values over their effect size estimates, researchers are under-selling their results and settling for contributions that are less than what they really have to offer. In view of this, I offer four recommendations for improving research and reporting practices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abraham, W. T., & Russell, D. W. 2008. Statistical power analysis in psychological research. Social and Personality Compass, 2 (1): 283–301.

    Article  Google Scholar 

  • AERA. 2006. Standards for reporting on empirical social science research in AERA publications. http://www.aera.net/opportunities/?id=1850. Accessed 11 September 2008.

  • APA. 2001. Publication manual of the American Psychological Association, (5th ed.). Washington, DC: American Psychological Association.

  • APA. 2010. Publication manual of the American Psychological Association, (6th ed.). Washington, DC: American Psychological Association.

  • Baggs, J., & Brander, J. A. 2006. Trade liberalization, profitability, and financial leverage. Journal of International Business Studies, 37 (2): 196–211.

    Article  Google Scholar 

  • Barden, J. Q., Steensma, H. K., & Lyles, M. A. 2005. The influence of parent control structure on parent conflict in Vietnamese international joint ventures: An organizational justice-based contingency approach. Journal of International Business Studies, 36 (2): 156–174.

    Article  Google Scholar 

  • Brock, J. 2003. The “power” of international business research. Journal of International Business Studies, 34 (1): 90–99.

    Article  Google Scholar 

  • Campbell, J. P. 1982. Editorial: Some remarks from the outgoing editor. Journal of Applied Psychology, 67 (6): 691–700.

    Article  Google Scholar 

  • Campion, M. A. 1993. Article review checklist: A criterion checklist for reviewing research articles in applied psychology. Personnel Psychology, 46 (3): 705–718.

    Article  Google Scholar 

  • Cano, C. R., Carrillat, F. A., & Jaramillo, F. 2004. A meta-analysis of the relationship between market orientation and business performance: Evidence from five continents. International Journal of Research in Marketing, 21 (2): 179–200.

    Article  Google Scholar 

  • Carver, R. P. 1978. The case against statistical significance testing. Harvard Educational Review, 48 (3): 378–399.

    Article  Google Scholar 

  • Cashen, L. H., & Geiger, S. W. 2004. Statistical power and the testing of null hypotheses: A review of contemporary management research and recommendations for future studies. Organizational Research Methods, 7 (2): 151–167.

    Article  Google Scholar 

  • Child, J., Chung, L., & Davies, H. 2003. The performance of cross-border units in China: A test of natural selection, strategic choice and contingency theories. Journal of International Business Studies, 34 (3): 242–254.

    Article  Google Scholar 

  • Cohen, J. 1988. Statistical power for the behavioral analysis, (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Combs, J. G. 2010. Big samples and small effects: Let's not trade relevance and rigor for power. Academy of Management Journal, 53 (1): 9–13.

    Article  Google Scholar 

  • Cortina, J. M., & Dunlap, W. P. 1997. On the logic and purpose of significance testing. Psychological Methods, 2 (2): 161–172.

    Article  Google Scholar 

  • Cumming, G., & Finch, S. 2005. Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist, 60 (2): 170–180.

    Article  Google Scholar 

  • Cummings, T. G. 2007. 2006 presidential address: Quest for an engaged academy. Academy of Management Review, 32 (2): 355–360.

    Article  Google Scholar 

  • Ellis, P. D. 2007. Distance, dependence and diversity of markets: Effects on market orientation. Journal of International Business Studies, 38 (3): 374–386.

    Article  Google Scholar 

  • Ellis, P. D. 2008. Does psychic distance moderate the market size–entry sequence relationship? Journal of International Business Studies, 39 (3): 351–369.

    Article  Google Scholar 

  • Ellis, P. D. 2010a. Effect size FAQs. http://www.effectsizefaq.com. Accessed 2 June 2010.

  • Ellis, P. D. 2010b. The essential guide to effect sizes: An introduction to statistical power, meta-analysis and the interpretation of research results. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Hambrick, D. C. 1994. 1993 Presidential Address: What if the academy actually mattered? Academy of Management Review, 19 (1): 11–16.

    Google Scholar 

  • Hoenig, J. M., & Heisey, D. M. 2001. The abuse of power: The pervasive fallacy of power calculations for data analysis. The American Statistician, 55 (1): 19–24.

    Article  Google Scholar 

  • Iacobucci, D. 2005. From the editor. Journal of Consumer Research, 32 (1): 1–6.

    Article  Google Scholar 

  • JARS. 2008. Reporting standards for research in psychology. American Psychologist, 63 (9): 839–851.

  • JEP. 2003. Instructions to authors. Journal of Educational Psychology, 95 (1): 201.

  • Luk, C. L., Yau, O., Sin, L., Tse, A., Chow, R., & Lee, J. 2008. The effects of social capital and organizational innovativeness in different institutional contexts. Journal of International Business Studies, 39 (4): 589–612.

    Article  Google Scholar 

  • Mazen, A. M., Graf, L. A., Kellogg, L. A., & Hemmasi, M. 1987. Statistical power in contemporary management research. Academy of Management Journal, 30 (2): 369–380.

    Article  Google Scholar 

  • Parboteeah, K. P., Hoegl, M., & Cullen, J. B. 2008. Managers’ gender role attitudes: A country institutional profile approach. Journal of International Business Studies, 39 (5): 795–813.

    Article  Google Scholar 

  • Qian, G. M., Li, L., Li, J., & Qian, Z. M. 2008. Regional diversification and firm performance. Journal of International Business Studies, 39 (2): 197–214.

    Article  Google Scholar 

  • Rynes, S. L. 2007. Editors afterword: Let's create a tipping point – What academics and practitioners can do, alone and together. Academy of Management Journal, 50 (5): 1046–1054.

    Article  Google Scholar 

  • Scherbaum, C. A., & Ferreter, J. M. 2009. Estimating statistical power and required sample sizes for organizational research using multilevel modeling. Organizational Research Methods, 12 (2): 347–367.

    Article  Google Scholar 

  • Shaver, J. M. 2006. Interpreting empirical findings. Journal of International Business Studies, 37 (4): 451–452.

    Article  Google Scholar 

  • Shaver, J. M. 2008. Organizational significance. Strategic Organization, 6 (2): 185–193.

    Article  Google Scholar 

  • Van de Vliert, E. V. 2003. Thermoclimate, culture, and poverty as country-level roots of workers’ wages. Journal of International Business Studies, 34 (1): 40–52.

    Article  Google Scholar 

  • Zedeck, S. 2003. Editorial. Journal of Applied Psychology, 88 (1): 3–5.

    Article  Google Scholar 

Download references

Acknowledgements

I am grateful to Juergen Brock, Sylvie Chetty, Flora Gu, Evert Van de Vliert, Robert Wright, Consulting Editor Myles Shaver, and three anonymous JIBS referees for providing feedback and constructive criticism on earlier drafts of this paper. The research reported in this paper was supported by a Internal Research Grant provided by the Hong Kong Polytechnic University (Project 4-ZZ9V).

Author information

Authors and Affiliations

  1. Department of Management & Marketing, Hong Kong Polytechnic University, Kowloon, Hong Kong

    Paul D Ellis

Authors
  1. Paul D Ellis
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Accepted by Myles Shaver, Consulting Editor, 25 July 2010. This paper has been with the author for three revisions.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ellis, P. Effect sizes and the interpretation of research results in international business. J Int Bus Stud 41, 1581–1588 (2010). https://doi.org/10.1057/jibs.2010.39

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/jibs.2010.39

Keywords

Search

Navigation