ReviewHow features of educational technology applications affect student reading outcomes: A meta-analysis
Highlights
► The findings suggest that educational technology produced a positive, though small, effect. ► There were differential impacts of various types of educational technology applications. ► Traditional computer-assisted instruction programs produced a small effect. ► Applications with the support of professional development showed more promising evidence.
Introduction
The classroom use of educational technology such as computers, interactive whiteboards, multimedia, and the internet, has been growing at a phenomenal rate in the last two decades. According to a recent survey conducted by the U.S. Department of Education (SETDA, 2010) on the use of educational technology in U.S. public schools, almost all public schools had one or more instructional computers with internet access, and the ratio of students to instructional computers with internet access was 3.1–1. In addition, 97% of schools had one or more instructional computers located in classrooms and 58% of schools had laptops on carts. A majority of public schools surveyed also indicated their schools provided various educational technology devices for instruction: LCD (liquid crystal display) and DLP (digital light processing) projectors (97%), digital cameras (93%), and interactive whiteboards (73%). The U.S. Department of Education provides generous grants to state education agencies to support the use of educational technology in K-12 classrooms. For example, in fiscal year 2009, the Department made a $900 million investment in educational technology in elementary and secondary schools (SETDA, 2010).
The debate around the effectiveness of educational technology for improving student learning has been carried on for over three decades. Perhaps the most widely cited debate was between Clark (1983) and Kozma (1994). Clark (1983) first argued that educational technology had no impact on student learning under any condition and that “media are mere vehicles that deliver instruction but do not influence student achievement any more than the truck that delivers our groceries causes changes in our nutrition”. He continued to argue that the impact of technology on student learning was mainly due to novelty effects or instructional strategies, but not technology itself. Kozma (1994) responded to Clark’s argument by saying the analogy of “delivery truck” creates an “unnecessary schism between medium and method”. Kozma believed that technology had an actual impact on student learning and played an important role in student learning.
The Clark–Kozma debate of the 1980’s has been overtaken by the extraordinary developments in technology applications in education in recent years. It may be theoretically interesting to ask whether the impact of technology itself can be separated from the impact of particular applications, but as a practical matter, machine and method are intertwined. As is the case for many educational interventions with many components, currently available technology applications can be seen as packages of diverse elements and evaluated as such. If a particular combination of hardware, software, print materials, professional development for teachers, and other elements can be reliably replicated in many classrooms, then it is worth evaluating as a potential means of enhancing student outcomes. Components of effective multi-element treatments can be varied to find out which elements contribute to effectiveness and to advance theory, but it is also of value for practice and policy to know the overall impact for students even if the theoretical mechanisms are not yet fully understood. Technology is here to stay, and pragmatically, the question is how to make the best use of the many technologies now available.
Section snippets
Research on educational technology applications
Research on the effectiveness of various forms of educational technology applications such as computer-assisted instruction (CAI) for improving learning outcomes has been abundant since the 1980s. Several major meta-analyses of the impact of educational technology on reading have also been conducted in the past two decades (Becker, 1992, Blok et al., 2002, Fletcher-Finn and Gravatt, 1995, Kulik and Kulik, 1991, Kulik, 2003, Ouyang, 1993, Soe et al., 2000). Overall, all came to a similar
Method
The current review employed meta-analytic techniques proposed by Glass, McGaw, and Smith (1981) and Lipsey and Wilson (2001). Comprehensive Meta-analysis Software Version 2 (Borenstein, Hedges, Higgins, & Rothstein, 2005) was used to calculate effect sizes and to carry out various meta-analytical tests, such as Q statistics and sensitivity analyses. Like many previous meta-analyses, this study follows several key steps: (1) locating all possible studies; (2) screening potential studies for
Overall effects
A total of 84 qualifying studies based on 60,553 K-12 participants were included in the final analysis: 8kindergarten studies (N = 2068), 59elementary studies (N = 34,200), and 18 secondary studies (N = 24,285). As indicated in Table 2, the overall mean effect size for the 84 qualifying studies is +0.16. The distribution of effect sizes in this collection of studies is highly heterogeneous (Q = 362.52, df = 83, p < 0.00), indicating that the variance of study effect sizes is larger than can be explained by
Limitations
It is important to mention several limitations in this review. First, due to the scope of this review, only studies with quantitative measures of reading were included. There is much to be learned from other non-experimental studies such as qualitative and correlational research that can add depth and insight to understanding the effects of these educational technology programs. Second, the review focuses on replicable programs used in realistic school settings over periods of at least 12 weeks,
Discussion
The purpose of this review was to examine the overall effectiveness of educational technology applications on reading outcomes in K-12 classrooms. Consistent with previous reviews of similar focus (Kulik, 2003, Kulik and Kulik, 1991, Soe et al., 2000), the findings of this study suggest that applications of educational technology generally produced a positive, though small, effect (ES = +0.16) in comparison to traditional methods. This effect is much larger than those reported in the recent
Practical implications for designing effective technology applications
The findings of this study have some practical implications for designing effective technology applications for reading. First, it is important to discuss the potential benefits and drawbacks of technology applications to illustrate how design of software and the human systems in which technology applications operate might determine reading outcomes. It may be impossible to determine the unique contribution of technology itself, but it should be possible to learn how to maximize technology’s
Conclusions
The findings of this review support those of earlier reviews by other researchers. The classroom use of educational technology will undoubtedly continue to expand and play an increasingly significant role in public education in the years to come as technology becomes more sophisticated and more cost-effective. This review highlights the need for more randomized studies. In addition, schools and districts should make concerted efforts to identify and adopt research-proven educational technology
References (128)
Mindless or mindful use of integrated learning systems
International Journal of Educational Research
(1994)- et al.
Effectiveness of computer-based instruction: An update analysis
Computers in Human Behavior
(1991) Quantitative analyses of student outcome measures
International Journal of Educational Research
(1997)- et al.
Putting computerized instruction to the test: A randomized evaluation of a “scientifically based” reading program
Economics of Education Review
(2004) - ∗Abram, S. L. (1984). The effect of computer assisted instruction on first grade phonics and mathematics achievement...
- et al.
Research on distance education: In defense of field experiments
Distance Education
(2006) - Adams (2010). Technology for developing children’s language and literacy: Bringing speech-recognition to the classroom....
- et al.
Multimedia for learning: Methods and development
(2001) An integrated learning system in an elementary school: Implementation, attitudes, and results
Journal of Computing in Childhood Education
(1991)- Atkinson, R. C. (1968). Computerized instruction and the learning process. American Psychologist, 23(4),...
Teaching children to read with a computer
The Reading Teacher
Teaching basic skills through microcomputer assisted instruction
Journal of Educational Computing Research
Computer-based integrated learning systems in the elementary and middle grades: A critical review and synthesis of evaluation reports
Journal of Educational Computing Research
Using an interactive singing software program: A comparative study of struggling middle school readers
Reading Psychology
Computer-assisted instruction in support of beginning reading instruction: A review
Review of Educational Research
Introduction to meta-analysis
Why do kids love computers?
Scholastic Early Childhood Today
The impact of a structured integrated learning system on first grade students’ reading gains
Reading and Writing Quarterly
Achievement effects of embedded multimedia in a Success for all reading program
Journal of Educational Psychology
Small-group computer-assisted tutoring to improve reading outcomes for struggling first and second graders
Elementary School Journal
Technology infusion in success for all: Reading outcomes for first graders
Elementary School Journal
Reconsidering research on learning from media
Review of Educational Research
Confounding in education computing research
Journal of Educational Computing Research
Evidence for confounding in computer-based instruction studies: Analyzing the meta-analyses
Educational Communication and Technology Journal
Writing to read: An evaluation of a Canadian installation of a computer-supported initial language environment
Journal of Educational Computing Research
Scholastic’s READ 180: A heritage of research
Reading rescue: An effective tutoring intervention model for language-minority students who are struggling readers in first grade
American Educational Research Journal
The impact of a year of computer assisted instruction on the development of first grade reading skills
Journal of Educational Computing Research
The efficacy of computer-assisted instruction (CAI): A meta-analysis
Journal of Educational Computing Research
Voyager universal literacy system: Results from a study of Kindergarten students in inner-city schools
Journal of Education for Students Placed at Risk
Cited by (190)
How to improve reading and writing skills in primary schools: A comparison between gamification and pen-and-paper training
2024, International Journal of Child-Computer InteractionStimulating preschoolers’ early literacy development using educational technology: A systematic literature review
2024, International Journal of Child-Computer InteractionExploring the impact of personalized and adaptive learning technologies on reading literacy: A global meta-analysis
2024, Educational Research ReviewSpecial Issue: Rethinking educational practices and responsibilities in the light of digitalisation
2023, International Journal of Educational ResearchComputer-assisted word reading intervention effects throughout the primary grades: A meta-analysis
2022, Educational Research Review
- 1
Tel.: +1 410 616 2310.