Skip to main content
main-content
Top

Hint

Swipe to navigate through the articles of this issue

Published in: Journal of Classification 3/2019

29-03-2019

Card Sorting Data Collection Methodology: How Many Participants Is Most Efficient?

Authors: Ethan Lantz, Jared W. Keeley, Michael C. Roberts, Maria Elena Medina-Mora, Pratap Sharan, Geoffrey M. Reed

Published in: Journal of Classification | Issue 3/2019

Login to get access
share
SHARE

Abstract

Pairwise similarity judgments and card sorting methodologies are different ways of generating data for similarity matrices used in various analyses such as multidimensional scaling and cluster analysis. Pairwise similarity judgments are considered the gold standard methodology, but can be cumbersome for large numbers of stimuli given the geometric increase in number of judgments necessary to fill the matrix. Card sorting methods provide a more expedient means of gathering this information, although they typically generate only binary data. Nonetheless, aggregated matrices generated from card sorts approximate pairwise similarity matrices. The current study used pairwise similarity and card sorting results from two existing studies that used the same stimuli to determine the optimal number of participants needed in a card sorting task to approximate the similarity matrix of pairwise data collection. In these studies, approximately 10–15 participants provided optimal estimation of the similarity matrix, with minimal increases for higher numbers of participants.
Literature
go back to reference Clatworthy, J., Buick, D., Hankins, M., Weinman, J., & Horne, R. (2005). The use of reporting and cluster analysis in health psychology: a review. British Journal of Health Psychology, 10(3), 329–358. CrossRef Clatworthy, J., Buick, D., Hankins, M., Weinman, J., & Horne, R. (2005). The use of reporting and cluster analysis in health psychology: a review. British Journal of Health Psychology, 10(3), 329–358. CrossRef
go back to reference Danilova, D., Diekhoff, G. M., & Vandehey, M. A. (2013). A multidimensional scaling analysis of male body perception in men with muscle dysmorphia: “The Adonis Complex”. International Journal of Men’s Health, 12(2), 83–105. CrossRef Danilova, D., Diekhoff, G. M., & Vandehey, M. A. (2013). A multidimensional scaling analysis of male body perception in men with muscle dysmorphia: “The Adonis Complex”. International Journal of Men’s Health, 12(2), 83–105. CrossRef
go back to reference Hout, M. C., Papesh, M. H., & Goldinger, S. D. (2012). Multidimensional scaling. Wiley Interdisciplinary Reviews (WIREs): Cognitive Science, 4, 93–103. Hout, M. C., Papesh, M. H., & Goldinger, S. D. (2012). Multidimensional scaling. Wiley Interdisciplinary Reviews (WIREs): Cognitive Science, 4, 93–103.
go back to reference Lantz, E., Diekhoff, G., & Carlston, D. (2014). How many participants are needed for card sort data collection? Poster presented at the southwestern psychological association. TX: San Antonio. Lantz, E., Diekhoff, G., & Carlston, D. (2014). How many participants are needed for card sort data collection? Poster presented at the southwestern psychological association. TX: San Antonio.
go back to reference Reed, G., Roberts, M., Keeley, J., Hooppell, C., Matsumoto, C., Sharan, P., Robles, R., Carvalho, H., Wu, C., Leal-Leturia, I., Flanagan, E., Correia, J., Ayuso-Mateos, J., Mari, J., Xiao, Z., Evans, S., Saxena, S., & Medina-Mora, M. (2013). Mental health professionals’ natural taxonomies of mental disorders: implications for clinical utility of ICD-11 and DSM-5. Journal of Clinical Psychology, 69, 1191–1212. CrossRef Reed, G., Roberts, M., Keeley, J., Hooppell, C., Matsumoto, C., Sharan, P., Robles, R., Carvalho, H., Wu, C., Leal-Leturia, I., Flanagan, E., Correia, J., Ayuso-Mateos, J., Mari, J., Xiao, Z., Evans, S., Saxena, S., & Medina-Mora, M. (2013). Mental health professionals’ natural taxonomies of mental disorders: implications for clinical utility of ICD-11 and DSM-5. Journal of Clinical Psychology, 69, 1191–1212. CrossRef
go back to reference Roberts, M., Reed, G., Medina-Mora, M., Keeley, J., Sharan, P., Johnson, D., Mari, J., Ayuso-Mateos, J., Gureje, O., Xiao, Z., Maruta, T., Khoury, B., Robles, R., & Saxena, S. (2012). A global clinicians’ map of mental disorders to improve ICD-11: analysing meta-structure to enhance clinical utility. International Review of Psychiatry, 24(6), 578–590. CrossRef Roberts, M., Reed, G., Medina-Mora, M., Keeley, J., Sharan, P., Johnson, D., Mari, J., Ayuso-Mateos, J., Gureje, O., Xiao, Z., Maruta, T., Khoury, B., Robles, R., & Saxena, S. (2012). A global clinicians’ map of mental disorders to improve ICD-11: analysing meta-structure to enhance clinical utility. International Review of Psychiatry, 24(6), 578–590. CrossRef
go back to reference Tullis, T., & Wood, L. (June, 2004). How many users are enough for a card-sorting study? Paper presented at the usability professional association. MN: Minneapolis. Tullis, T., & Wood, L. (June, 2004). How many users are enough for a card-sorting study? Paper presented at the usability professional association. MN: Minneapolis.
go back to reference Tullis, T., & Wood, L. (June, 2005). How can you do a card-sorting study with LOTS of cards? Paper presented at the usability professional association. MN: Minneapolis. Tullis, T., & Wood, L. (June, 2005). How can you do a card-sorting study with LOTS of cards? Paper presented at the usability professional association. MN: Minneapolis.
go back to reference Whaley, A., & Longoria, R. (2009). Preparing card Sort data for multidimensional scaling analysis in social psychological research: a methodological approach. The Journal of Social Psychology, 149(1), 105–115. CrossRef Whaley, A., & Longoria, R. (2009). Preparing card Sort data for multidimensional scaling analysis in social psychological research: a methodological approach. The Journal of Social Psychology, 149(1), 105–115. CrossRef
Metadata
Title
Card Sorting Data Collection Methodology: How Many Participants Is Most Efficient?
Authors
Ethan Lantz
Jared W. Keeley
Michael C. Roberts
Maria Elena Medina-Mora
Pratap Sharan
Geoffrey M. Reed
Publication date
29-03-2019
Publisher
Springer US
Published in
Journal of Classification / Issue 3/2019
Print ISSN: 0176-4268
Electronic ISSN: 1432-1343
DOI
https://doi.org/10.1007/s00357-018-9292-8

Other articles of this Issue 3/2019

Journal of Classification 3/2019 Go to the issue

Premium Partner