Skip to main content
SpringerLink
Log in

Individual differences in decision-making and confidence: capturing decision tendencies in a fictitious medical test

  • Published:
Metacognition and Learning Aims and scope Submit manuscript

Abstract

Decision-making is a complex process that is largely studied from an experimental perspective or in specific organizational contexts. As such, no generalizable framework exists with which to study decision-making from an individual differences perspective for predictive/selection purposes. By generalising a context-specific decision model proposed by Koriat and Goldsmith (1996), the focus of this research was to therefore test a novel framework for studying individual differences in decision-making tendencies. Utilising this framework within a fictitious Medical Decision-Making Test (MDMT) yielded five novel variables that provided unique insight into individuals’ decision tendencies: Optimal, Realistic, Incompetent, Hesitant and Congruent. Metacognitive confidence and its calibration (bias and CAQ) were used as predictor variables to validate this framework. One hundred ninety-three undergraduate students completed the MDMT and three cognitive ability tests with confidence ratings, a personality questionnaire, and the Need for Closure questionnaire. All decision tendency variables demonstrated excellent internal consistency and were predicted by the metacognitive variables incrementally to the remaining variables as hypothesized. Additionally, the metacognitive indices were found to generalize across the decision-making and cognitive tests. The results imply that this novel framework and MDMT reliably capture individuals’ decision behaviour that shares a meaningful relationship with their general confidence and calibration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. This variable can also be thought of as the inverse proportion of erroneous decisions, as it is equal to 1 – (B + C)/(A + B + C + D).

  2. The lower end of the confidence scale is defined by the minimum probability of being correct. For example, in a five option multiple-choice test the probability of being correct is at least 20 %. In this case, the scale will range from 20 to 100 %. The scale can request any value, or make categories available such as an 11-point scale ranging from 0 to 100 %.

  3. Ten minutes learning time was deemed appropriate and tested in a pilot study (N = 19). Feedback received indicated that 10 min was a sufficient amount of time to learn and that the instructions had been understood. 10 min was therefore retained as the learning time in the present study.

  4. Given that the confidence scales for the MDMT and the APM were presented as categorical responses representing 10 % increments (e.g., 50 %, 60 % etc.), we chose to represent the lower bound of the scale with the value that was closest to the actual probability associated with guessing. That is, 30 % was used as the closest estimate to the 33 % chance of being correct in the MDMT, and 10 % as the closest estimate to the 12.5 % chance in the APM. Given the small deviations here (3 % and 2.5 %) and previous results in our laboratory, there was no reason to suspect that this would alter the results.

  5. Marker tests were selected to be consistent with the Horn-Cattell Gf/Gc theory of intelligence (Horn and Cattell 1982): a hierarchical model defining intelligence in terms of two broad independent abilities (Carroll 1993). Fluid intelligence (Gf) reflects basic reasoning abilities, whilst crystallized intelligence (Gc) reflects the ability to learn and use information acculturated through education and experience.

  6. CAQ cannot be calculated when overall accuracy equals 0 % or 100 % because average confidence for correct or incorrect items is not assigned by the participants, thus is not available (Shaughnessy 1979). CAQ was therefore calculated for any individual with at least one correct and incorrect answer. CAQ scores were not computed for five participants in the MDMT, one in Raven’s Advanced Progressive Matrices, and two in Esoteric Analogies due to 100 % accuracy (no participants scored 0 %).

  7. An initial exploratory factor analysis (EFA; Principal Components with PROMAX-rotation) was run without constraints. The latent root criterion suggested 3 factors, but the scree plot clearly suggested the presence of two distinct factors only. Considering the moderate to high intercorrelations among many variables, a single factor solution was also run. However, a single factor only accounted for 44.158 % of the variance (a drop of almost 16 %), and one communality was below 0.20. Hence, the two-factor solution was retained and used in the final analysis.

References

  • Allwood, C. M., Granhag, P. A., & Jonsson, A.-C. (2006). Child witnesses’ metamemory realism. Scandinavian Journal of Psychology, 47(6), 461–470. doi:10.1111/j.1467-9450.2006.00530.x.

    Article  Google Scholar 

  • Appelt, K. C., Milch, K. F., Handgraaf, M. J., & Weber, E. U. (2011). The decision making individual differences inventory and guidelines for the study of individual differences in judgment and decision-making research. Judgment and Decision Making, 6(3), 252–262.

    Google Scholar 

  • Azevedo, R. (2009). Theoretical, conceptual, methodological, and instructional issues in research on metacognition and self-regulated learning: a discussion. Metacognition Learning, 4, 87–95.

    Article  Google Scholar 

  • Bavolar, J. (2013). Validation of the adult decision-making competence in Slovak students. Judgment and Decision Making, 8(3), 386–392.

    Google Scholar 

  • Boekaerts, M., & Rozendaal, J. S. (2010). Using multiple calibration indices in order to capture the complex picture of what affects students’ accuracy of feeling of confidence. Learning and Instruction, 20(5), 372–382.

    Article  Google Scholar 

  • Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press.

    Book  Google Scholar 

  • Cattell, R. B. (1987). Intelligence: Its structure, growth and action. Amsterdam: North Holland.

    Google Scholar 

  • Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioural sciences (3rd ed.). Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Costermans, J., Lories, G., & Ansay, C. (1992). Confidence level and feeling of knowing in question answering: the weight of inferential processes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(1), 142–150. doi:10.1037/0278-7393.18.1.142.

    Google Scholar 

  • Cowley, E. (2004). Recognition confidence, recognition accuracy and choice. Journal of Business Research, 57(6), 641–646. doi:10.1016/s0148-2963(02)00307.

    Article  Google Scholar 

  • Crawford, J. D., & Stankov, L. (1996). Age differences in the realism of confidence judgements: a calibration study using tests of fluid and crystallized intelligence. Learning and Individual Differences, 8(2), 83–103.

    Article  Google Scholar 

  • DeMarree, K. G., & Petty, R. E. (2007). Self-certainty: parallels to attitude certainty. International Journal of Psychology and Psychological Therapy, 7(2), 159–188.

    Google Scholar 

  • Edwards, W. (1954). The theory of decision making. Psychological Bulletin, 51(4), 380–417.

    Article  Google Scholar 

  • Edwards, W. (1961). Behavioral decision theory. Annual Review of Psychology, 12, 473–498.

    Article  Google Scholar 

  • Efklides, A. (2008). Metacognition: defining its facets and levels of functioning in relation to self-regulation and co-regulation. European Psychologist, 13(4), 277–287. doi:10.1027/1016-9040.13.4.277.

    Article  Google Scholar 

  • Efklides, A. (2009). The role of metacognitive experiences in the learning process. Psicothema, 21(1), 76–82.

    Google Scholar 

  • Flavell, J. H. (1979). Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906–911.

    Article  Google Scholar 

  • Gilovich, T., Griffin, D. W., & Kahneman, D. (Eds.). (2002). Heuristics and biases: The psychology of intiuitive judgement. New York: Cambridge University Press.

    Google Scholar 

  • Glaser, M., & Weber, M. (2007). Overconfidence and trading volume. The Geneva Risk and Insurance Review, 32(1), 1–36.

    Google Scholar 

  • Hacker, D. J., Bol, L., & Keener, M. C. (2008). Metacognition in education: A focus on calibration. Handbook of metamemory and memory, 429–455.

  • Harvey, N. (1997). Confidence in judgment. Trends in Cognitive Sciences, 1(2), 78–82.

    Article  Google Scholar 

  • Harvey, N. (2001). Studying judgement: general issues. Thinking & Reasoning, 7(1), 103–118. doi:10.1080/13546780042000064.

    Article  Google Scholar 

  • Horn, J. L., & Cattell, R. B. (1982). Whimsy and misunderstanding of gf-gc theory: a comment on Guilford. Psychological Bulletin, 91(3), 623–633.

    Article  Google Scholar 

  • Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge, and job performance. Journal of Vocational Behavior, 29(3), 340–362. doi:10.1016/0001-8791(86)90013-8.

    Article  Google Scholar 

  • Juslin, P., Winman, A., & Olsson, H. (2000). Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect. Psychological Review, 107(2), 384–396.

    Article  Google Scholar 

  • Kaplan, R. M., & Saccuzzo, D. P. (2005). Psychological testing; principles, applications, and issues (6th ed.). Belmont: Thomson Wadsworth.

    Google Scholar 

  • Kelemen, W. L., Frost, P. J., & Weaver, C. A. (2000). Individual differences in metacognition: Evidence against a general metacognitive ability. Memory & Cognition, 28(1), 92–107.

    Google Scholar 

  • Kleitman, S. (2008). Metacognition in the rationality debate. Saarbrücken: VDM Verlag Dr. Müller Aktiengesellschaft & Co. KG.

    Google Scholar 

  • Kleitman, S., & Gibson, J. (2011). Metacognitive beliefs, self-confidence and primary learning environment of sixth grade students. Learning and Individual Differences, 21, 728–735.

    Article  Google Scholar 

  • Kleitman, S., & Mascrop, T. (2010). Self-confidence and academic achievements in primary-school children: Their relationships and links to parental bonds, intelligence, age, and gender. In A. Efklides & P. Misailidi (Eds.), Trends and prospects in metacognition research (pp. 293–326). US: Springer.

    Chapter  Google Scholar 

  • Kleitman, S., & Stankov, L. (2001). Ecological and person-oriented aspects of metacognitive processes in test-taking. Applied Cognitive Psychology, 15(3), 321–341. doi:10.1002/acp.705.

    Article  Google Scholar 

  • Kleitman, S., & Stankov, L. (2007). Self-confidence and metacognitive processes. Learning and Individual Differences, 17, 161–173.

    Article  Google Scholar 

  • Kleitman, S., Stankov, L., Allwood, C. M., Young, S., & Mak, K. (2011). Metacognitive self-confidence in school-aged children educators. Unpublished Book Chapter.

  • Knight, F. H. (1921). Risk, uncertainty and profit. Boston: Houghton Mifflin.

    Google Scholar 

  • Koriat, A., & Goldsmith, M. (1996). Monitoring and control processes in the strategic regulation of memory accuracy. Psychological Review, 103(3), 490.

    Article  Google Scholar 

  • Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Performance, 20(2), 159–183. doi:10.1016/0030-5073(77)90001-0.

    Article  Google Scholar 

  • McKenzie, C. R. (1998). Taking into account the strength of an alternative hypothesis. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24(3), 771–792.

    Google Scholar 

  • Mellers, B., Schwartz, A., & Cooke, A. (1998). Judgment and decision making. Annual Review of Psychology, 49, 447–477.

    Article  Google Scholar 

  • Mengelkamp, C., & Bannert, M. (2010). Accuracy of confidence judgments: stability and generality in the learning process and predictive validity for learning outcome. Memory & Cognition, 38(4), 441–451.

    Article  Google Scholar 

  • Moore, D. A., & Healy, P. J. (2008). The trouble with overconfidence. Psychological Review, 115(2), 502–517.

    Article  Google Scholar 

  • Nelson, T. O. (1996). Consciousness and metacognition. American Psychologist, 51(2), 102–116.

    Article  Google Scholar 

  • Pallier, G., Wilkinson, R., Danthir, V., Kleitman, S., Knezevic, G., Stankov, L., & Roberts, R. D. (2002). The role of individual differences in the accuracy of confidence judgments. Journal of General Psychology, 129(3), 257–299. doi:10.1080/00221300209602099.

    Article  Google Scholar 

  • Parker, A. M., Bruine de Bruin, W., Yoong, J., & Willis, R. (2012). Inappropriate confidence and retirement planning: four studies with a national sample. Journal of Behavioral Decision Making, 25(4), 382–389.

    Article  Google Scholar 

  • Raven, J. C. (1938-65). Progressive matrices. New York: The Psychological Corporation.

  • Roets, A., & Van Hiel, A. (2011). Item selection and validation of a brief, 15-item version of the need for closure scale. Personality and Individual Differences, 50(1), 90–94. doi:10.1016/j.paid.2010.09.004.

    Article  Google Scholar 

  • Saucier, G. (2008). Measures of personality factors found recurrently in human lexicons. In G. J. Boyle, G. Matthews, & D. Sakloske (Eds.), Handbook of personality theory and testing (Vol. 2). London: Sage.

    Google Scholar 

  • Schraw, G. (2009). Measuring metacognitive judgments. Handbook of metacognition in education (pp. 415–429). New York: Routledge/Taylor & Francis Group.

    Google Scholar 

  • Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19(4), 460–475. doi:10.1006/ceps.1994.1033.

    Article  Google Scholar 

  • Schraw, G., Dunkle, M. E., Bendixen, L. D., & Roedel, T. D. (1995). Does a general monitoring skill exist? Journal of Educational Psychology, 87(3), 433–444. doi:10.1037/0022-0663.87.3.433.

    Article  Google Scholar 

  • Shaughnessy, J. J. (1979). Confidence-judgment accuracy as a predictor of test performance. Journal of Research in Personality, 13(4), 505–514. doi:10.1016/0092-6566(79)90012-6.

    Article  Google Scholar 

  • Slovic, P., Fischoff, B., & Lichtenstein, S. (1977). Behavioral decision theory. Annual Review of Psychology, 28, 1–39.

    Article  Google Scholar 

  • Stankov, L. (1997). The Gf/Gc quickie test battery. School of Psychology: The University of Sydney. Unpublished test battery.

  • Stankov, L. (1999). Mining on the “no man’s land” between intelligence and personality. In P. L. Ackerman, P. C. Kyllonen, & R. D. Roberts (Eds.), Learning and individual differences: Process, trait, and content determinants (pp. 315–337). Washington, DC: American Psychological Association.

    Chapter  Google Scholar 

  • Stankov, L. (2000). Complexity, metacognition, and fluid intelligence. Intelligence, 28, 121–143.

    Article  Google Scholar 

  • Stankov, L., & Crawford, J. D. (1996). Confidence judgments in studies of individual differences. Personality and Individual Differences, 21(6), 971–986.

    Article  Google Scholar 

  • Stankov, L., & Crawford, J. D. (1997). Self-confidence and performance on tests of cognitive abilities. Intelligence, 25(2), 93–109.

    Article  Google Scholar 

  • Stankov, L., & Kleitman, S. (2008). Processes on the borderline between cognitive abilities and personality: Confidece and its realism. In G. J. Boyle, G. Matthews & D. Saklofske (Eds.), The SAGE handbook of personality theory and assessment (pp. 541–555). London, England: SAGE Publications.

  • Stankov, L., & Lee, J. (2008). Confidence and cognitive test performance. Journal of Educational Psychology, 100(4), 961–976. doi:10.1037/a0012546.

    Article  Google Scholar 

  • Stankov, L., Lee, J., Luo, W., & Hogan, D. J. (2012). Confidence: a better predictor of academic achievement than self-efficacy, self-concept and anxiety? Learning and Individual Differences, 22(6), 747–758.

    Google Scholar 

  • Stankov, L., Kleitman, S., & Jackson, S. A. (2013). Measures of the trait of confidence: In G. Boyle, & D. Saklofske (Eds.), Measures of personality and social psychological attitudes. Amsterdam, Netherlands: Elsevier (in press).

  • Veenman, M. V., Wilhelm, P., & Beishuizen, J. J. (2004). The relation between intellectual and metacognitive skills from a developmental perspective. Learning and Instruction, 14(1), 89–109.

    Article  Google Scholar 

  • Want, J., & Kleitman, S. (2006). Imposter phenomenon and self-handicapping: links with parenting styles and self-confidence. Personality and Individual Differences, 40(5), 961–971.

    Article  Google Scholar 

  • West, R. F., & Stanovich, K. E. (1997). The domain specificity and generality of overconfidence: individual differences in performance estimation bias. Psychonomic Bulletin & Review, 4(3), 387–392.

    Article  Google Scholar 

  • Williams, E. F., & Gilovich, T. (2008). Do people really believe they are above average? Journal of Experimental Social Psychology, 44(4), 1121–1128.

    Article  Google Scholar 

  • Yang, H., & Thompson, C. (2010). Nurses’ risk assessment judgements: a confidence calibration study. Journal of Advanced Nursing, 66(12), 2751–2760. doi:10.1111/j.1365-2648.2010.05437.x.

    Article  Google Scholar 

  • Yates, J. F. (1990). Judgment and decision making. Englewood Cliffs: Prentice-Hall.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Psychology, The University of Sydney, Sydney, NSW, 2006, Australia

    Simon A. Jackson & Sabina Kleitman

Authors
  1. Simon A. Jackson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sabina Kleitman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon A. Jackson.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jackson, S.A., Kleitman, S. Individual differences in decision-making and confidence: capturing decision tendencies in a fictitious medical test. Metacognition Learning 9, 25–49 (2014). https://doi.org/10.1007/s11409-013-9110-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11409-013-9110-y

Keywords

Search

Navigation