Skip to main content
SpringerLink
Log in

Measuring cognitive load with subjective rating scales during problem solving: differences between immediate and delayed ratings

  • Published:
Instructional Science Aims and scope Submit manuscript

Abstract

Subjective cognitive load (CL) rating scales are widely used in educational research. However, there are still some open questions regarding the point of time at which such scales should be applied. Whereas some studies apply rating scales directly after each step or task and use an average of these ratings, others assess CL only once after the whole learning or problem-solving phase. To investigate if these two approaches are comparable indicators of experienced CL, two experiments were conducted, in which 168 and 107 teacher education university students, respectively, worked through a sequence of six problems. CL was assessed by means of subjective ratings of mental effort and perceived task difficulty after each problem and after the whole process. Results showed that the delayed ratings of both effort and difficulty were significantly higher than the average of the six ratings made during problem solving. In addition, the problems we assumed to be of higher complexity seemed to be the best predictors for the delayed ratings. Interestingly, for ratings of affective variables, such as interest and motivation, the delayed rating did not differ from the average of immediate ratings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Antonenko, P., & Niederhauser, D. (2010). The influence of leads on cognitive load and learning in a hypertext-assisted learning environment. Computers in Human Behavior, 26, 140–150.

    Article  Google Scholar 

  • Antonenko, P., Paas, F., Grabner, R., & Van Gog, T. (2010). Using electroencephalography (EEG) to measure cognitive load. Educational Psychology Review, 22, 425–438.

    Article  Google Scholar 

  • Ayres, P. (2006). Using subjective measures to detect variations of intrinsic load within problems. Learning and Instruction, 16, 389–400.

    Article  Google Scholar 

  • Bratfisch, O., Borg, G., & Dornic, S. (1972). Perceived item difficulty in three tests of intellectual performance capacity. Stockholm: Institute of Applied Psychology, Report No. 29.

  • Brünken, R., Plass, J. L., & Leutner, D. (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38, 53–61.

    Article  Google Scholar 

  • Brünken, R., Plass, J. L., & Leutner, D. (2004). Assessment of cognitive load in multimedia learning with dual-task methodology: Auditory load and modality effects. Instructional Science, 32, 115–132.

    Article  Google Scholar 

  • Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293–332.

    Article  Google Scholar 

  • Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62, 233–246.

    Article  Google Scholar 

  • Chandler, P., & Sweller, J. (1996). Cognitive load while learning to use a computer program. Applied Cognitive Psychology, 10, 1–20.

    Article  Google Scholar 

  • de Jong, T. (2010). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science, 38, 105–134.

    Article  Google Scholar 

  • Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371.

    Article  Google Scholar 

  • Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93, 579–588.

    Article  Google Scholar 

  • Kühl, T., Scheiter, K., Gerjets, P., & Edelmann, J. (2011). The influence of text modality on learning with static and dynamic visualizations. Computers in Human Behavior, 27, 29–35.

    Article  Google Scholar 

  • Kuusela, H., & Paul, P. (2000). A comparison of concurrent and retrospective verbal protocol analysis. American Journal of Psychology, 113, 387–404.

    Article  Google Scholar 

  • Leutner, D., Leopold, C., & Sumfleth, E. (2009). Cognitive load and science text comprehension: Effects of drawing and mentally imagining text content. Computers in Human Behavior, 25, 284–289.

    Article  Google Scholar 

  • Logie, R. H. (1995). Visuo-spatial working memory. Hove, UK: Lawrence Erlbaum Associates.

    Google Scholar 

  • Marcus, N., Cooper, M., & Sweller, J. (1996). Understanding instructions. Journal of Educational Psychology, 88, 49–63.

    Article  Google Scholar 

  • Opfermann, M. (2008). There’s more to it than instructional design: The role of individual learner characteristics for hypermedia learning. Berlin, Germany: Logos.

    Google Scholar 

  • Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429–434.

    Article  Google Scholar 

  • Paas, F., Ayres, P., & Pachman, M. (2008). Assessment of cognitive load in multimedia learning: Theory, methods and applications. In D. Robinson & G. Schraw (Eds.), Recent innovations in educational technology that facilitate student learning (pp. 20–36). Charlotte, NC: Information Age Publishing.

  • Paas, F., Renkl, A., & Sweller, J. (2003a). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38, 1–4.

    Article  Google Scholar 

  • Paas, F., Tuovinen, J., Tabbers, H. K., & Van Gerven, P. W. M. (2003b). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 63–71.

    Article  Google Scholar 

  • Paas, F., & van Merriënboer, J. J. G. (1994). Variability of worked examples and transfer of geometrical problem solving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122–133.

    Article  Google Scholar 

  • Paas, F., van Merriënboer, J. J. G., & Adam, J. J. (1994). Measurement of cognitive load in instructional research. Perceptual and Motor Skills, 79, 419–430.

    Article  Google Scholar 

  • Park, B., Moreno, R., Seufert, T., & Brünken, R. (2011). Does cognitive load moderate the seductive details effect? A multimedia study. Computers in Human Behavior, 27, 5–10.

    Article  Google Scholar 

  • Plass, J. L., Moreno, R., & Brünken, R. (Eds.). (2010). Cognitive load: Theory & application. Cambridge: Cambridge University Press.

    Google Scholar 

  • Renkl, A., Gruber, H., Weber, S., Lerche, T., & Schweizer, K. (2003). Cognitive Load beim Lernen aus Lösungsbeispielen [Cognitive load of learning from worked-out examples]. Zeitschrift für Pädagogische Psychologie, 17, 93–101.

    Article  Google Scholar 

  • Rheinberg, F., Vollmeyer, R., & Burns, B. D. (2001). FAM: Ein Fragebogen zur Erfassung aktueller Motivation in Lern- und Leistungssituationen [QCM: A questionnaire to assess current motivation in learning situations]. Diagnostica, 47, 57–66.

    Article  Google Scholar 

  • Schwamborn, A., Thillmann, H., Opfermann, M., & Leutner, D. (2011). Cognitive load and instructionally supported learning with provided and learner-generated visualizations. Computer in Human Behavior, 27, 89–93.

    Article  Google Scholar 

  • Seufert, T., & Brünken, R. (2006). Cognitive load and the format of instructional aids for coherence formation. Applied Cognitive Psychology, 20, 321–331.

    Article  Google Scholar 

  • Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.

    Article  Google Scholar 

  • Sweller, J. (1993). Some cognitive processes and their consequences for the organization and presentation of information. Australian Journal of Psychology, 45, 1–8.

    Article  Google Scholar 

  • Sweller, J. (2010). Element interactivity and intrinsic, extraneous and germane cognitive load. Educational Psychology Review, 22, 123–138.

    Article  Google Scholar 

  • Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York: Springer.

    Book  Google Scholar 

  • Sweller, J., van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296.

    Article  Google Scholar 

  • Tabbers, H. K., Martens, R. L., & van Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology, 74, 71–81.

    Article  Google Scholar 

  • Taylor, K. L., & Dionne, J.-P. (2000). Accessing problem-solving strategy knowledge: The complementary use of concurrent verbal protocols and retrospective debriefing. Journal of Educational Psychology, 92, 413–425.

    Article  Google Scholar 

  • Van Gerven, P. W. M., Paas, F., van Merriënboer, J. J. G., & Schmidt, H. G. (2004). Memory load and the cognitive pupillary response in aging. Psychophysiology, 41, 167–174.

    Article  Google Scholar 

  • Van Gog, T., & Jarodzka, H. (2013). Eye tracking as a tool to study and enhance cognitive and metacognitive processes in computer-based learning environments. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 143–156). New York: Springer.

    Google Scholar 

  • Van Gog, T., Kirschner, F., Kester, L., & Paas, F. (2012). Timing and frequency of mental effort measurement: Evidence in favor of repeated measures. Applied Cognitive Psychology, 26, 833–839.

    Article  Google Scholar 

  • Van Gog, T., & Paas, F. (2008). Instructional efficiency: Revisiting the original construct in educational research. Educational Psychologist, 43, 16–26.

    Article  Google Scholar 

  • Van Gog, T., Paas, F., & van Merriënboer, J. J. G. (2006). Effects of process-oriented worked examples on troubleshooting transfer performance. Learning and Instruction, 16, 154–164.

    Article  Google Scholar 

  • Van Merriënboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17, 147–177.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Instructional Psychology, Faculty for Educational Sciences, University of Duisburg-Essen, 45117, Essen, Germany

    Annett Schmeck, Maria Opfermann & Detlev Leutner

  2. Capaciteitsgroep Psychologie, Faculteit der Sociale Wetenschappen, Erasmus University Rotterdam, Burgemeester Oudlaan 50, T-gebouw, P.O. Box 1738, 3000 DR, Rotterdam, The Netherlands

    Tamara van Gog & Fred Paas

Authors
  1. Annett Schmeck
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maria Opfermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tamara van Gog
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fred Paas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Detlev Leutner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Annett Schmeck.

Appendix

Appendix

Weekday problems (according to order of presentation in Experiment 1)

  1. (a)

    Suppose today is Tuesday. What day of the week is the day after tomorrow?

  2. (b)

    Suppose today is Thursday. What day of the week is in three days?

  3. (c)

    Suppose yesterday was Wednesday. What day of the week was four days before the day before yesterday?

  4. (d)

    What day was yesterday if the day after the day after tomorrow is three days before Sunday?

  5. (e)

    Suppose five days after the day before yesterday is Friday. What day of the week is tomorrow?

  6. (f)

    Suppose last Tuesday was the 2nd day. What day of the week is it in 17 days, if the 8th day is in two days?

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schmeck, A., Opfermann, M., van Gog, T. et al. Measuring cognitive load with subjective rating scales during problem solving: differences between immediate and delayed ratings. Instr Sci 43, 93–114 (2015). https://doi.org/10.1007/s11251-014-9328-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11251-014-9328-3

Keywords

Search

Navigation