Skip to main content
Erschienen in: TechTrends 5/2018

18.06.2018 | Original Paper

Meta-Analysis and Meta-Synthesis Methodologies: Rigorously Piecing Together Research

verfasst von: Heather Leary, Andrew Walker

Erschienen in: TechTrends | Ausgabe 5/2018

Einloggen, um Zugang zu erhalten

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

For a variety of reasons, education research can be difficult to summarize. Varying contexts, designs, levels of quality, measurement challenges, definition of underlying constructs, and treatments as well as the complexity of research subjects themselves can result in variability. Education research is voluminous and draws on multiple methods including quantitative, as well as, qualitative approaches to answer key research questions. With increased numbers of empirical research in Instructional Design and Technology (IDT), using various synthesis methods can provide a means to more deeply understand trends and patterns in research findings across multiple studies. The purpose of this article is to illustrate structured review or meta-synthesis procedures for qualitative research, as well as, novel meta-analysis procedures for the kinds of multiple treatment designs common to IDT settings. Sample analyses are used to discuss key methodological ideas as a way to introduce researchers to these techniques.
Literatur
Zurück zum Zitat Attride-Stirling, J. (2001). Thematic networks: An analytic tool for qualitative research. Qualitative Research, 1(3), 385–405.CrossRef Attride-Stirling, J. (2001). Thematic networks: An analytic tool for qualitative research. Qualitative Research, 1(3), 385–405.CrossRef
Zurück zum Zitat Barnett-Page, E., & Thomas, J. (2009). Methods for the synthesis of qualitative research: a critical review. ESRC National Centre for Research Methods, Social Science Research Unit, Institute of Education, University of London, London, 01/09. Barnett-Page, E., & Thomas, J. (2009). Methods for the synthesis of qualitative research: a critical review. ESRC National Centre for Research Methods, Social Science Research Unit, Institute of Education, University of London, London, 01/09.
Zurück zum Zitat Barton, E. E., Pustejovsky, J. E., Maggin, D. M., & Reichow, B. (2017). Technology-aided instruction and intervention for students with ASD: A meta-analysis using novel methods of estimating effect sizes for single-case research. Remedial and Special Education, 38(6), 371–386.CrossRef Barton, E. E., Pustejovsky, J. E., Maggin, D. M., & Reichow, B. (2017). Technology-aided instruction and intervention for students with ASD: A meta-analysis using novel methods of estimating effect sizes for single-case research. Remedial and Special Education, 38(6), 371–386.CrossRef
Zurück zum Zitat Beal, C. R., Arroyo, I., Cohen, P. R., & Woolf, B. P. (2010). Evaluation of AnimalWatch: An intelligent tutoring system for arithmetic and fractions. Journal of Interactive Online Learning, 9(1), 64–77. Beal, C. R., Arroyo, I., Cohen, P. R., & Woolf, B. P. (2010). Evaluation of AnimalWatch: An intelligent tutoring system for arithmetic and fractions. Journal of Interactive Online Learning, 9(1), 64–77.
Zurück zum Zitat Belland, B., Walker, A., Kim, N., & Lefler, M. (2017a). Synthesizing results from empirical research on computer-based scaffolding in STEM education: A meta-analysis. Review of Educational Research, 82(2), 309–344.CrossRef Belland, B., Walker, A., Kim, N., & Lefler, M. (2017a). Synthesizing results from empirical research on computer-based scaffolding in STEM education: A meta-analysis. Review of Educational Research, 82(2), 309–344.CrossRef
Zurück zum Zitat Belland, B., Walker, A., & Kim, N. (2017b). A bayesian network meta-analysis to synthesize the influence of contexts of scaffolding use on cognitive outcomes in STEM education. Review of Educational Research, 87(6), 1042–1081.CrossRef Belland, B., Walker, A., & Kim, N. (2017b). A bayesian network meta-analysis to synthesize the influence of contexts of scaffolding use on cognitive outcomes in STEM education. Review of Educational Research, 87(6), 1042–1081.CrossRef
Zurück zum Zitat Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester: Wiley.CrossRef Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester: Wiley.CrossRef
Zurück zum Zitat Bulu, S., & Pedersen, S. (2010). Scaffolding middle school students’ content knowledge and illstructured problem solving in a problem-based hypermedia learning environment. Educational Technology Research & Development, 58(5), 507–529.CrossRef Bulu, S., & Pedersen, S. (2010). Scaffolding middle school students’ content knowledge and illstructured problem solving in a problem-based hypermedia learning environment. Educational Technology Research & Development, 58(5), 507–529.CrossRef
Zurück zum Zitat Cooper, H. (2009). Research synthesis and meta-analysis: A step-by-step approach (applied social research methods). Thousand Oaks: Sage Publications.CrossRef Cooper, H. (2009). Research synthesis and meta-analysis: A step-by-step approach (applied social research methods). Thousand Oaks: Sage Publications.CrossRef
Zurück zum Zitat Cooper, H. (2016). Research synthesis and meta-analysis: A step-by-step approach (5th ed.). Thousand Oaks: Sage publications. Cooper, H. (2016). Research synthesis and meta-analysis: A step-by-step approach (5th ed.). Thousand Oaks: Sage publications.
Zurück zum Zitat Cooper, H., & Hedges, L. V. (1994). The handbook of research synthesis. New York: Russell Sage Foundation. Cooper, H., & Hedges, L. V. (1994). The handbook of research synthesis. New York: Russell Sage Foundation.
Zurück zum Zitat Finfgeld, D. L. (2003). Metasynthesis: The state of the art - so far. Qualitative Health Research, 13(7), 893–904.CrossRef Finfgeld, D. L. (2003). Metasynthesis: The state of the art - so far. Qualitative Health Research, 13(7), 893–904.CrossRef
Zurück zum Zitat Finfgeld-Connett, D. (2010). Generalizability and transferability of meta-synthesis research findings. Journal of Advanced Nursing, 66(2), 246–254.CrossRef Finfgeld-Connett, D. (2010). Generalizability and transferability of meta-synthesis research findings. Journal of Advanced Nursing, 66(2), 246–254.CrossRef
Zurück zum Zitat Finfgeld-Connett, D. (2016). The future of theory-generating meta-synthesis research. Qualitative Health Research, 26(3), 291–293.CrossRef Finfgeld-Connett, D. (2016). The future of theory-generating meta-synthesis research. Qualitative Health Research, 26(3), 291–293.CrossRef
Zurück zum Zitat Finlayson, K., & Dixon, A. (2008). Qualitative meta-synthesis: A guide for the novice. Nurse Researcher, 15(2), 59–71.CrossRef Finlayson, K., & Dixon, A. (2008). Qualitative meta-synthesis: A guide for the novice. Nurse Researcher, 15(2), 59–71.CrossRef
Zurück zum Zitat Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3–8.CrossRef Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3–8.CrossRef
Zurück zum Zitat Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018). Meta-analysis and the science of research synthesis. Nature, 555, 175–182.CrossRef Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018). Meta-analysis and the science of research synthesis. Nature, 555, 175–182.CrossRef
Zurück zum Zitat Harden, A. (2010). Mixed-methods systematic reviews: Integrating quantitative and qualitative findings. Focus Technical Brief, 25, 1–8. Harden, A. (2010). Mixed-methods systematic reviews: Integrating quantitative and qualitative findings. Focus Technical Brief, 25, 1–8.
Zurück zum Zitat Heller, R. (1990). The role of hypermedia in education: A look at the research issues. Journal of Research on Computing in Education, 22, 431–441.CrossRef Heller, R. (1990). The role of hypermedia in education: A look at the research issues. Journal of Research on Computing in Education, 22, 431–441.CrossRef
Zurück zum Zitat Higgins, S. (2016). Meta-synthesis and comparative meta-analysis of education research findings: Some risks and benefits. Review of Education, 4(1), 31–53.CrossRef Higgins, S. (2016). Meta-synthesis and comparative meta-analysis of education research findings: Some risks and benefits. Review of Education, 4(1), 31–53.CrossRef
Zurück zum Zitat Karich, A. C., Burns, M. K., & Maki, K. E. (2014). Updated meta-analysis of learner control within educational technology. Review of Educational Research, 84(3), 392–410.CrossRef Karich, A. C., Burns, M. K., & Maki, K. E. (2014). Updated meta-analysis of learner control within educational technology. Review of Educational Research, 84(3), 392–410.CrossRef
Zurück zum Zitat Lachal, J., Revah-Levy, A., Orri, M., & Moro, M. R. (2017). Metasynthesis: An original method to synthesize qualitative literature. Frontiers in Psychiatry, 8, 1–9.CrossRef Lachal, J., Revah-Levy, A., Orri, M., & Moro, M. R. (2017). Metasynthesis: An original method to synthesize qualitative literature. Frontiers in Psychiatry, 8, 1–9.CrossRef
Zurück zum Zitat Lam, S. K. H., & Owen, A. (2007). Combined resynchronisation and implantable defibrillator therapy in left ventricular dysfunction: Bayesian network meta-analysis of randomised controlled trials. BMJ: British Medical Journal, 335(7626), 925–928.CrossRef Lam, S. K. H., & Owen, A. (2007). Combined resynchronisation and implantable defibrillator therapy in left ventricular dysfunction: Bayesian network meta-analysis of randomised controlled trials. BMJ: British Medical Journal, 335(7626), 925–928.CrossRef
Zurück zum Zitat Leary, H., Shelton, B. E., & Walker, A. (2010). Rich visual media meta-analyses for learning: An approach at meta-synthesis. Paper presented at the American Educational Research Association annual meeting, Denver, CO. Leary, H., Shelton, B. E., & Walker, A. (2010). Rich visual media meta-analyses for learning: An approach at meta-synthesis. Paper presented at the American Educational Research Association annual meeting, Denver, CO.
Zurück zum Zitat Lipsey, M., & Wilson, D. (2001). Practical meta-analysis (Applied social research methods). Thousand Oaks: Sage Publications. Lipsey, M., & Wilson, D. (2001). Practical meta-analysis (Applied social research methods). Thousand Oaks: Sage Publications.
Zurück zum Zitat Liu, Y., Cornish, A., & Clegg, J. (2007). ICT and special educational needs: Using meta-synthesis for bridging the multifaceted divide. In Y. Shi, G. D. van Albada, J. Dongarra, & P. M. A. Sloot (Eds.), Computational science–ICCS 2007. ICCS 2007. Lecture notes in computer science (Vol. 4490). Berlin, Heidelberg: Springer. Liu, Y., Cornish, A., & Clegg, J. (2007). ICT and special educational needs: Using meta-synthesis for bridging the multifaceted divide. In Y. Shi, G. D. van Albada, J. Dongarra, & P. M. A. Sloot (Eds.), Computational science–ICCS 2007. ICCS 2007. Lecture notes in computer science (Vol. 4490). Berlin, Heidelberg: Springer.
Zurück zum Zitat Ludvigsen, M. S., Hall, E. O. C., Meyer, G., Fegran, L., Aagaard, H., & Uhrenfeldt, L. (2016). Using Sandelowski and Barroso’s meta-synthesis method in advancing qualitative evidence. Qualitative Health Research, 26(3), 320–329.CrossRef Ludvigsen, M. S., Hall, E. O. C., Meyer, G., Fegran, L., Aagaard, H., & Uhrenfeldt, L. (2016). Using Sandelowski and Barroso’s meta-synthesis method in advancing qualitative evidence. Qualitative Health Research, 26(3), 320–329.CrossRef
Zurück zum Zitat Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009). The BUGS project: Evolution, critique and future directions. Statistics in Medicine, 28(25), 3049–3067.CrossRef Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009). The BUGS project: Evolution, critique and future directions. Statistics in Medicine, 28(25), 3049–3067.CrossRef
Zurück zum Zitat McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. London: Routledge. McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. London: Routledge.
Zurück zum Zitat Mumtaz, S. (2000). Factors affecting teachers’ use of information and communications technology: A review of the literature. Journal of Information Technology for Teacher Education, 9(3), 319–342.CrossRef Mumtaz, S. (2000). Factors affecting teachers’ use of information and communications technology: A review of the literature. Journal of Information Technology for Teacher Education, 9(3), 319–342.CrossRef
Zurück zum Zitat Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. London: Sage.CrossRef Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. London: Sage.CrossRef
Zurück zum Zitat Paterson, B. L., Thorne, S. E., Canam, C., & Jillings, C. (2001). Meta-study of qualitative health research: A practical guide to meta-analysis and meta-synthesis. London: Sage.CrossRef Paterson, B. L., Thorne, S. E., Canam, C., & Jillings, C. (2001). Meta-study of qualitative health research: A practical guide to meta-analysis and meta-synthesis. London: Sage.CrossRef
Zurück zum Zitat Puhan, M. A., Schünemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., … Guyatt, G. H. (2014). A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 349, g5630. Puhan, M. A., Schünemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., … Guyatt, G. H. (2014). A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 349, g5630.
Zurück zum Zitat Reeves, T., & Oh, E. (2017). The goals and methods of educational technology research over a quarter century (1989–2014). Educational Technology Research & Development, 65(2), 325–339.CrossRef Reeves, T., & Oh, E. (2017). The goals and methods of educational technology research over a quarter century (1989–2014). Educational Technology Research & Development, 65(2), 325–339.CrossRef
Zurück zum Zitat Salanti, G., Giovane, C. D., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014). Evaluating the quality of evidence from a network meta-analysis. PLOS ONE, 9(7), e99682.CrossRef Salanti, G., Giovane, C. D., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014). Evaluating the quality of evidence from a network meta-analysis. PLOS ONE, 9(7), e99682.CrossRef
Zurück zum Zitat Sandelowski, M., & Barroso, J. (2003). Toward a metasynthesis of qualitative findings on motherhood in HIV-positive women. Research in Nursing and Health, 26(2), 153–170.CrossRef Sandelowski, M., & Barroso, J. (2003). Toward a metasynthesis of qualitative findings on motherhood in HIV-positive women. Research in Nursing and Health, 26(2), 153–170.CrossRef
Zurück zum Zitat Sandelowski, M., Docherty, S., & Emden, C. (1997). Quality metasynthesis: Issues and techniques. Research in Nursing and Health, 20, 365–371.CrossRef Sandelowski, M., Docherty, S., & Emden, C. (1997). Quality metasynthesis: Issues and techniques. Research in Nursing and Health, 20, 365–371.CrossRef
Zurück zum Zitat Taquero, J. M. (2011). A meta-ethnographic synthesis of support services in distance learning programs. Journal of Information Technology Education: Innovations in Practice, 10, 157–179. Taquero, J. M. (2011). A meta-ethnographic synthesis of support services in distance learning programs. Journal of Information Technology Education: Innovations in Practice, 10, 157–179.
Zurück zum Zitat Thorne, S., Jensen, L., Kearney, M. H., Noblit, G., & Sandelowski, M. (2004). Qualitative metasynthesis: Reflection on methodological orientation and ideological agenda. Qualitative Health Research, 14(10), 1342–1365.CrossRef Thorne, S., Jensen, L., Kearney, M. H., Noblit, G., & Sandelowski, M. (2004). Qualitative metasynthesis: Reflection on methodological orientation and ideological agenda. Qualitative Health Research, 14(10), 1342–1365.CrossRef
Zurück zum Zitat Tondeur, J., van Braak, J., Sang, G., Voogt, J., Fisser, P., & Ottenbreit-Lewftwich, A. (2012). Preparing pre-service teachers to integrate technology in education: A synthesis of qualitative evidence. Computers & Education, 59(1), 134–144.CrossRef Tondeur, J., van Braak, J., Sang, G., Voogt, J., Fisser, P., & Ottenbreit-Lewftwich, A. (2012). Preparing pre-service teachers to integrate technology in education: A synthesis of qualitative evidence. Computers & Education, 59(1), 134–144.CrossRef
Zurück zum Zitat Torraco, R. J. (2016). Writing integrative literature reviews: Using the past and present to explore the future. Human Resource Development Review, 15(4), 404–428.CrossRef Torraco, R. J. (2016). Writing integrative literature reviews: Using the past and present to explore the future. Human Resource Development Review, 15(4), 404–428.CrossRef
Zurück zum Zitat Walker, A., Belland, B., Kim, N., & Lefler, M. (2016). Searching for structure: A bayesian network meta-analysis of computer-based scaffolding research in STEM education. Paper presented at the American Educational Research Association, San Antonio, TX. Walker, A., Belland, B., Kim, N., & Lefler, M. (2016). Searching for structure: A bayesian network meta-analysis of computer-based scaffolding research in STEM education. Paper presented at the American Educational Research Association, San Antonio, TX.
Zurück zum Zitat Walsh, D., & Downe, S. (2005). Meta-synthesis method for qualitative research: A literature review. Methodological Issues in Nursing Education, 50(2), 204–211. Walsh, D., & Downe, S. (2005). Meta-synthesis method for qualitative research: A literature review. Methodological Issues in Nursing Education, 50(2), 204–211.
Zurück zum Zitat Young, J. (2017). Technology-enhanced mathematics instruction: A second-order meta-analysis of 30 years of research. Educational Research Review, 22, 19–33.CrossRef Young, J. (2017). Technology-enhanced mathematics instruction: A second-order meta-analysis of 30 years of research. Educational Research Review, 22, 19–33.CrossRef
Metadaten
Titel
Meta-Analysis and Meta-Synthesis Methodologies: Rigorously Piecing Together Research
verfasst von
Heather Leary
Andrew Walker
Publikationsdatum
18.06.2018
Verlag
Springer US
Erschienen in
TechTrends / Ausgabe 5/2018
Print ISSN: 8756-3894
Elektronische ISSN: 1559-7075
DOI
https://doi.org/10.1007/s11528-018-0312-7

Weitere Artikel der Ausgabe 5/2018

TechTrends 5/2018 Zur Ausgabe

Editor’s Notes

Editor’s Notes