Skip to main content
SpringerLink
Log in

A knowledge-tracing model of learning from a social tagging system

  • Original Paper
  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

We propose a user model to support personalized learning paths through online material. Our approach is a variant of student modeling using the computer tutoring concept of knowledge tracing. Knowledge tracing involves representing the knowledge required to master a domain, and, from traces of online user behavior, diagnosing user knowledge states as a profile over those elements. The user model is induced from documents tagged by an expert in a social tagging system. Tags identified with “expertise” in a domain can be used to identify a corpus of domain documents. That corpus can be fed to an automated process that distills a topic model representation characteristic of the domain. As a learner navigates and reads online material, inferences can be made about the degree to which topics in the target domain have been learned. We validate this knowledge tracing approach against data from a social tagging study. As part of this evaluation, we match the predictions of the knowledge-tracing model to individual participant responses made to individual question items used to test domain knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abel, F., Baldoni, M., Baroglio, C., Henze, N., Krause, D., Patti, V.: Context-based ranking in folksonomies. Paper presented at the proceedings of the 20th ACM conference on hypertext and hypermedia (HT ’09), pp. 209–218 (2009)

  • Abel, F., Herder, E., Houben, G.-J., Henze, N., Krause, D.: Cross-system User Modeling and Personalization on the Social Web. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9131-2

  • Aleven V., McLaren B.M., Sewall J., Koedinger K.R.: A new paradigm for intelligent tutoring systems: example-tracing tutors. J. Artif. Intell. Educ 19(2), 105–154 (2009)

    Google Scholar 

  • Allen, I.E., Seaman, J.: Learning on demand: online education in the United States, 2009: Sloan Consortium (2010)

  • Anderson J.R.: Learning to program in LISP. Cogn. Sci. 8, 87–129 (1984)

    Article  Google Scholar 

  • Anderson J.R.: The Adaptive Character of Thought. Lawrence Erlbaum Associates, Hillsdale (1990)

    Google Scholar 

  • Anderson, J.R.: Rules of the Mind. Lawrence Erlbaum Associates, Hillsdale (1993)

  • Anderson J.R., Bothell D., Byrne M.D., Douglass S., Lebiere C., Qin Y.: An integrated theory of mind. Psychol Rev 11(4), 1036–1060 (2004)

    Article  Google Scholar 

  • Anderson J.R., Boyle C.F., Corbett A., Lewis M.W.: Cognitive modelling and intelligent tutoring. Artif. Intell. 42, 7–49 (1990)

    Article  Google Scholar 

  • Balog, K., Azzopardi, L., de Rijke, M.: Formal models for expert finding in enterprise corpora. Paper presented at the 29th annual ACM SIGIR conference on research and development in information retrieval (SIGIR ’06) (2006)

  • Bernstein, M., Suh, B., Hong, L., Chen, J., Kairam, S., Chi, E.H.: Interactive topic-based browsing of social status streams. Paper presented at the proceedings of the 23rd symposium on user interface software and technology (UIST ’10) (2010)

  • Blei D.M., Ng A.Y., Jordan M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  • Bloom B.: The 2 sigma problem: the search for methods of instruction as effective as one-to-one tutoring. Educ. Res. 13(6), 4–16 (1984)

    Google Scholar 

  • Brin S., Page L.: The anatomy of a large-scale hypertextual web search engine. Comput. Netw. ISDN Syst. 30(1–7), 107–117 (1998)

    Article  Google Scholar 

  • Brusilovsky P., Peylo C.: Adaptive and intelligent web-based educational systems. Int. J. Artif. Intell. Educ. 13, 156–169 (2003)

    Google Scholar 

  • Budura, A., Bourges-Waldegg, D., Riordan, J.: Deriving expertise profiles from tags. Paper presented at the international conference on computational science and engineering (CSE ’09) (2009)

  • Campbell, C.S., Maglio, P.P., Cozzi, A., Dom, B.: Expertise identification using email communications. Paper presented at the proceedings of the 2003 ACM CIKM international conference on information and knowledge management (CIKM ’03) (2003)

  • Carbonell J.R.: AI in CAI: an artificial-intelligence approach to computer-assisted instruction. IEEE Trans. Man Mach. Syst. 11(4), 190–202 (1970)

    Article  Google Scholar 

  • Champ, H.: (2009, August 13, 2010). 4,000,000,000. http://blog.flickr.net/en/2009/10/12/4000000000

  • Conati C., Gertner A., Vanlehn K.: Using Bayesian networks to manage uncertainty in student modeling. User Model. User Adapt. Interact. 12(4), 371–417 (2002)

    Article  MATH  Google Scholar 

  • Corbett, A.T.: Cognitive computer tutors: solving the two-sigma problem. Paper presented at the user modeling 2001: 8th international conference, Berlin (2001)

  • Corbett A.T., Anderson J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Model. User Adapt. Interact. 4(4), 253–278 (1995)

    Article  Google Scholar 

  • Corbett, A. T., Anderson, J. R., O’Brien, A. T.: Student modeling in the ACT Programming Tutor. In: Nichols P. D., Chipman S. F., Brennan R. L. (eds.) Cognitively diagnostic assessment, pp. 19–41. Lawrence Erlbaum Associates, Hillsdale, NJ (1995)

  • Delicious.: (2008, August 13, 2010). Oh Happy Day. http://blog.delicious.com/blog/2008/07/oh-happy-day.html

  • Foltz, P.W., Laham, D., Landauer, T.K.: Automated essay scoring: application to educational technology. Paper presented at the world conference on education, multimedia, hypermedia, and telecommunications, Seattle, WA (1999)

  • Fox, S., Fallows, D.: Internet health resources. Retrieved December, 2003, from http://www.pewinternet.org/reports/pdfs/PIP_Health_Report_July_2003.pdf (2003, August)

  • Fox, S., Jones, S.: The social life of health information. Pew Internet & American Life Project. Retrieved June 27, 2009, from http://www.pewinternet.org/Reports/2009/8-The-Social-Life-of-Health-Information.aspx (2009, June 11)

  • Furnas G.W., Landauer T.K., Gomez L.M., Dumais S.T.: The vocabulary problem in human-system communication. Commun. ACM 30, 964–971 (1987)

    Article  Google Scholar 

  • Gayo-Avello, D., Brenes D.J.: Overcoming spammers in twitter—-a tale of five algorithms. Paper presented at the Congreso Español de Recuperación de Información (CERI 2010) (2010)

  • Gena, C., Cena, F., Vernero, F., Grillo, P.: The evaluation of a social adaptive web site for cultural events. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9129-9

  • Golder S.A., Huberman B.A.: The structure of collaborative tagging systems. J. Inf. Sci. 32, 198–208 (2006)

    Article  Google Scholar 

  • Griffiths T.L., Steyvers M., Tenenbaum J.B.: Topics in semantic representation. Psychol. Rev. 114(2), 211–244 (2007)

    Article  Google Scholar 

  • Han S.-G., Lee S.-G., Jo G.-S.: Case-based tutoring systems for procedural problem solving on the www. Expert Syst. Appl. Int. J. 29(3), 573–582 (2005)

    Article  Google Scholar 

  • Hartley J.R., Sleeman D.H.: Towards more intelligent teaching systems. Int. J. Man Mach. Stud. 5(2), 215–236 (1973)

    Article  Google Scholar 

  • Hofman, T.: Probabilistic latent semantic indexing. Paper presented at the twenty-second international SIGIR conference on research and development in information retrieval (SIGIR-99) (1999)

  • Hong, L., Chi, E. H., Budiu, R., Pirolli, P., Nelson, L.: SparTag.us: a low cost tagging system for foraging of web content. Paper presented at the working conference on advanced visual interfaces (AVI ’08) (2008)

  • Horrigan, J.: The Internet as a resource for news and information about science. Pew Internet & American Life Project. Retrieved June 27, 2009, from http://www.pewinternet.org/Reports/2006/The-Internet-as-a-Resource-for-News-and-Information-about-Science.aspx (2006, November 20)

  • Hotho, A., Jäschke, R., Schmitz, C., Stumme, G.: Information retrieval in folksonomies: search and ranking. Paper presented at the 3rd European semantic web conference (ESWC ’06) (2006)

  • John, A., Seligmann, D.: Collaborative tagging and expertise in the enterprise. Paper presented at the collaborative web tagging workshop in conjunction with 15th international conference on world wide web (WWW ’06) (2006)

  • Junker, B.: Some statistical models and computational methods that may be useful for cognitively-relevant assessment. National Research Council (1999)

  • Kakkonen, T., Myller, N., Timonen, J., Sutinen, E.: Automatic essay grading with probabilistic latent semantic analysis. Paper presented at the proceedings of the second workshop on building educational applications Using NLP (2005)

  • Kammerer, Y., Nairn, R., Pirolli, P., Chi, E. H.: Signpost from the masses: learning effects in an exploratory social tag search browser. Paper presented at the 27th conference on human factors in computing systems (CHI ’09) (2009)

  • Kim, H.-N., El Saddik, A.: Exploring social tagging for personalized community recommendations. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9130-3

  • Kleinberg J.: Authoritative sources in a hyperlinked environment. J. ACM 46(5), 604–632 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Kline T.J.B.: Psychological Testing: A Practical Approach to Design and Evaluation. Sage Publications, Thousand Oaks (2005)

    Google Scholar 

  • Koedinger K.R., Anderson J.R., Hadley W.H., Mark M.A.: Intelligent tutoring goes to the big city. Int. J. Artif. Intell. Educ. 8, 30–43 (1997)

    Google Scholar 

  • Landauer T.K., Dumais S.T.: A solution to Plato’s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104, 211–240 (1997)

    Article  Google Scholar 

  • Lenhart, A.: Adults and social network websites. Pew Internet & American Life Project. Retrieved June 27, 2009, from http://www.pewinternet.org/Reports/2009/Adults-and-Social-Network-Websites.aspx (2009, January 14)

  • Loizou, S. K., Dimitrova, V.: Adaptive notifications to support knowledge sharing in close-knit virtual communities. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9127-y

  • Mimno, D., McCallum, A.: Expertise modeling for matching papers with reviewers. Paper presented at the 13th international conference on knowledge discovery and data mining (KDD ’07) (2007)

  • Murray T.: Authoring intelligence tutoring systems: an analysis of the state of the art. Int. J. Artif. Intell. Educ. 10, 98–129 (1999)

    Google Scholar 

  • Murray T.: An overview of intelligent tutoring system authoring tools: updated analysis of the state of the art. In: Murray, T., Blessing, S., Aisworth, S. (eds) Authoring Tools for Advanced Technology Environments, pp. 493–546. Kluwer, Dordrecht (2003)

    Google Scholar 

  • Nelson, L., Held, C., Pirolli, P., Hong, L., Schiano, D., Chi, E. H.: With a little help from my friends: examining the impact of social annotations in sensemaking tasks. Paper presented at the proceedings of the 27th international conference on human factors in computing systems (2009)

  • Noll, M. G., Au Yeung, C.-M., Gibbins, N., Meinel, C., Shadbolt, N.: Telling experts from spammers: expertise ranking in folksonomies. Paper presented at the 32nd annual ACM SIGIR conference on research and development in information retrieval (SIGIR ’09) (2009)

  • Noll, M. G., Yeung, C.-m. A., Gibbins, N., Meinel, C., Shadbolt, N.: Telling experts from spammers: expertise ranking in folksonomies. Paper presented at the proceedings of the 32nd international ACM SIGIR conference on research and development in information retrieval (2009)

  • PACT.: Pittsburgh advanced cognitive tutor center home. Retrieved August 13, 2010, from http://pact.cs.cmu.edu (2005)

  • Petkova, D., Croft, B.W.: Hierarchical language models for expert finding in enterprise corpora. Paper presented at the 18th IEEE conference on tools with artificial intelligence (ICTAI ’06) (2006)

  • Pirolli P., Wilson M.: A theory of the measurement of knowledge content, access, and learning. Psychol. Rev. 105, 58–82 (1998)

    Article  Google Scholar 

  • Robu V., Halpin H., Shepherd H.: Emergence of consensus and shared vocabularies in collaborative tagging systems. ACM Trans. Web 3(4), 1–34 (2009)

    Article  Google Scholar 

  • Rosen-Zvi, M., Griffiths, T.L., Steyvers, M., Smyth, P.: The author-topic model for authors and documents. Paper presented at the 20th conference on uncertainty in artificial intelligence (2004)

  • Shapira, B., Rokach, L., Freilichman, S.: Utilizing facebook single and cross domain data for recommendation systems. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9128-x

  • Shute V.J., Psotka J.: Intelligent tutoring systems: past, present, and future. In D.H. Jonassen (ed.) Handbook of Research on Educational Communications and Technology, pp. 570–600. Simon & Schuster Macmillan, New York (1996)

  • Steyvers M., Griffiths, T.L., Dennis, S.: Probabilistic inference in human semantic memory. TRENDS Cogn. Sci. 10(7), 327–334 (2006)

    Google Scholar 

  • Weng, J., Lim, E.-P., Jiang, J., He, Q.: TwitterRank: finding topic-sensitive influential twitterers. Paper presented at the proceedings of the 3rd ACM international conference on web search and data mining (WSDM ’10) (2010)

  • Yin, Z., Li, R., Mei, Q., Han, J.: Exploring social tagging graph for web object classification. Paper presented at the 15th ACM SIGKDD conference on knowledge discovery and data mining (KDD ’09) (2009)

  • YouTube. YouTube Statistics. Retrieved January 16, 2012, from http://www.youtube.com/t/press_statistics

  • Zhang, J., Ackerman, M.S.: Searching for expertise in social networks: a simulation of potential strategies. Paper presented at the 2005 international ACM SIGGROUP conference on supporting group work (Group ’05) (2005)

  • Zhang, J., Ackerman, M. S., Adamic, L.: Expertise networks in online communities: structure and algorithms. Paper presented at the 16th international world wide web conference (WWW ’07) (2007)

Download references

Author information

Authors and Affiliations

  1. Palo Alto Research Center, 3333 Coyote Hill Rd, Palo Alto, CA, 94304, USA

    Peter Pirolli & Sanjay Kairam

  2. Department of Computer Science, Stanford University, 353 Serra Mall, Office 376, Stanford, CA, 94305, USA

    Sanjay Kairam

Authors
  1. Peter Pirolli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanjay Kairam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Pirolli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pirolli, P., Kairam, S. A knowledge-tracing model of learning from a social tagging system. User Model User-Adap Inter 23, 139–168 (2013). https://doi.org/10.1007/s11257-012-9132-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-012-9132-1

Keywords

Search

Navigation