Abstract
This paper reports the findings of a Stage I meta-analysis exploring the achievement effects of computer-based technology use in higher education classrooms (non-distance education). An extensive literature search revealed more than 6,000 potentially relevant primary empirical studies. Analysis of a representative sample of 231 studies (k = 310) yielded a weighted average effect size of 0.28 surrounded by wide variability. A mixed effects model was adopted to explore coded moderators of effect size. Research design was found to be not significant across true, quasi- and pre-experimental designs, so the designs were combined. The variable “degree of technology use” (i.e., low, medium, and high) was found to be significant, with low and medium use performing significantly higher than high use. For the variable “type of use” (i.e., cognitive support tools, presentational tools, and multiple uses), cognitive support (g+ = 0.40) was greater than presentational and multiple uses.
Similar content being viewed by others
Notes
This acknowledges that studies published in 1990 were informed by data collected prior to the 1990 Internet milestone, sometimes many years prior (see inclusion criteria policy below).
References
Note: A list of studies included in the meta-analysis is available upon request from the authors and online as electronic supplementary material.
Abrami, P. C., & Bernard, R. M. (2006). Research on distance education: In defense of field experiments. Distance Education, 27(1), 5–26.
Abrami, P.C., & Bernard, R.M. (2009). Statistical control vs. classification of study quality in meta-analysis. Manuscript submitted for publication.
Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist, 40(4), 199–209.
Azevedo, R., & Bernard, R. M. (1995). A meta-analysis of the effects of feedback in computer-based instruction. Journal of Educational Computing Research, 13(2), 111–127.
Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004). How does distance education compare to classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74(3), 379–439.
Bernard, R.M., Abrami, P.C., Borokhovski, E., Wade, C.A., Tamim, R., Surkes, M.A., & Bethel, E.C. (2009). A meta-analysis of three types of interaction treatments in distance education. Manuscript in press, Review of Educational Research.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. (2005). Comprehensive meta-analysis version 2. Englewood, NJ: Biostat.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. (2009). Introduction to meta-analysis. Chichester, UK: Wiley & Sons.
Campbell, D., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand, McNally.
Christmann, E. P., & Badgett, J. L. (2000). The comparative effectiveness of CAI on collegiate academic performance. Journal of Computing in Higher Education, 11(2), 91–103.
Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445–459.
Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(3), 21–29.
Clark, R.E., Yates, K., Early, S., & Moulton, K. (2009). An analysis of the failure of electronic media and discovery-based learning: Evidence for the performance benefits of guided training methods. In K. H. Silber, & R. Foshay, (Eds.). Manuscript in press: Handbook of training and improving workplace performance, Volume I: Instructional design and training delivery. Somerset, NJ: Wiley.
Cobb, T. (1997). Cognitive efficiency: Toward a revised theory of media. Educational Technology Research and Development, 45(4), 21–35.
Cohen, P. A., & Dacanay, L. S. (1992). Computer-based instruction and health professions education: A meta-analysis of outcomes. Evaluation and the Health Professions, 15(3), 259–281.
Dabbagh, N., & Kitsantas, A. (2005). Using web-based pedagogical tools as scaffolds for self-regulated learning. Instructional Science, 33(5–6), 513–540.
Fletcher-Flinn, C. M., & Gravatt, B. (1995). The efficacy of computer assisted instruction (CAI): A meta-analysis. Journal of Educational Computing Research, 12(3), 219–242.
Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
Hedges, L.V., Shymansky, J.A., & Woodworth, G. (1989). A practical guide to modern methods of meta-analysis. (ERIC Document Reproduction Service No. ED 309 952).
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analysis. British Medical Journal, 327(7414), 557–560.
Hsu, Y.-c. (2003). The effectiveness of computer-assisted instruction in statistics education: A meta-analysis. Unpublished doctoral dissertation, University of Arizona.
Jonassen, D. H., & Reeves, T. C. (1996). Learning with technology: Using computers as cognitive tools. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology. New York, NY: Scholastic Press.
Jonassen, D. H., Howland, J., Moore, J., & Marra, R. M. (2003). Learning to solve problems with technology: A constructivist perspective (2nd ed.). Upper Saddle River, NJ: Merrill Prentice Hall.
Koufogiannakis, D., & Wiebe, N. (2006). Effective methods for teaching information literacy skills to undergraduate students: A systematic review and meta-analysis. Evidence Based Library and Information Practice, 1(3), 3–43.
Kozma, R. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7–19.
Lou, Y., Abrami, P. C., & D’Appollonia, S. (2001). Small group and individual learning with technology: A meta-analysis. Review of Educational Research, 71, 449–521.
Lou, Y., Bernard, R. M., & Abrami, P. C. (2006). Media and pedagogy in undergraduate distance education: A theory-based meta-analysis of empirical literature. Educational Technology Research &Development, 54(2), 141–176.
Lowerison, G., Tamim, R., Nicolaidu, I., & Schmid, R. F. (2006, April). Implications of student technology use for knowledge building. Paper presented at the annual meeting of the American Educational Research Association, San Francisco: CA.
Michko, G. M. (2007). A meta-analysis of the effects of teaching and learning with technology on student outcomes in undergraduate engineering education. Unpublished doctoral dissertation, University of Houston, Houston, TX.
Rosen, Y., & Salomon, G. (2007). The differential learning achievements of constructivist technology-intensive learning environments as compared with traditional ones: A meta-analysis. Journal of Educational Computing Research, 36(1), 1–14.
Schenker, J. D. (2007). The effectiveness of technology use in statistics instruction in higher education: A meta-analysis using hierarchical linear modeling. Unpublished doctoral dissertation, Kent State University, Kent, OH.
Schmid, R. F., Borokhovski, E., Tamim, R., Bernard, R. M., Abrami, P. C., Wade, C. A., Surkes, M. A., Lowerison, G., Lysenko, L., & Galofre. (2009). Effect of computer-based technology use in postsecondary education: Preliminary findings of a meta-analysis. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
Tamim, R. M. (2009). Effects of technology on students’ achievement: A second-order meta-analysis. Unpublished doctoral dissertation, Concordia University, Montreal, QC, Canada.
Timmerman, C. E., & Kruepke, K. A. (2006). Computer-assisted instruction, media richness, and college student performance. Communication Education, 55, 73–104.
Zhao, Y. (2003). Recent developments in technology and language learning: A literature review and meta-analysis. CALICO Journal, 21(10), 7–27.
Acknowledgments
An earlier version of this paper was presented at the American Educational Research Association’s Annual Meeting, San Diego, CA on April 14, 2009. This study was supported by grants from the Social Sciences and Humanities Research Council of Canada to Schmid, Abrami and Bernard. The authors express appreciation to Christina Galofre, Carol Sparkes, Larysa Lysenko, Julia Barkhouse, Anna Sokolovskaya, Katherine Hanz, Nancy Acemian, Dai Zhang and Lucie A. Ranger. Contact: Dr. Richard F. Schmid, CSLP, LB-583-3, Department of Education, Concordia University, 1455 de Maisonneuve Blvd. W., Montreal, QC, Canada H3G 1M8. E-mail: schmid@education.concordia.ca, Website: http://doe.concordia.ca/cslp.
Author information
Authors and Affiliations
Corresponding authors
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Schmid, R.F., Bernard, R.M., Borokhovski, E. et al. Technology’s effect on achievement in higher education: a Stage I meta-analysis of classroom applications. J Comput High Educ 21, 95–109 (2009). https://doi.org/10.1007/s12528-009-9021-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12528-009-9021-8