Skip to main content
SpringerLink
Log in

Evaluating Communities of Practice and Knowledge Networks: A Systematic Scoping Review of Evaluation Frameworks

  • Review
  • Published:
EcoHealth Aims and scope Submit manuscript

Abstract

Communities of Practice (CoPs) are increasingly considered a part of ecohealth and other sectors such as health care, education, and business. However, there is little agreement on approaches to evaluate the influence and effectiveness of CoPs. The purpose of this review was to understand what frameworks and methods have been proposed or used to evaluate CoPs and/or knowledge networks. The review searched electronic databases in interdisciplinary, health, education, and business fields, and further collected references and forward citations from relevant articles. Nineteen articles with 16 frameworks were included in the synthesis. The purposes of the evaluation frameworks varied; while some focused on assessing the performance of CoPs, several frameworks sought to learn about CoPs and their critical success factors. Nine of the frameworks had been applied or tested in some way, most frequently to guide a case study. With limited applications of the frameworks, strong claims about generalizability could not be made. The review results can inform the development of tailored frameworks. However, there is a need for more detailed and targeted CoP evaluation frameworks, as many imperative CoP evaluation needs would be unmet by the available frameworks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

References

  • Arksey H, O’Malley L (2005) Scoping studies: towards a methodological framework. International Journal of Social Research Methodology 8:19–32

    Article  Google Scholar 

  • Barragán-Ocaña A, Quijano-Solís Á, Vega-Díaz G, Sánchez-Lara B (2012) Communities of practice: A focus from complex systems. International Journal of General Systems 41:741-755

    Article  Google Scholar 

  • Barwick MA, Peters J, Boydell K (2009) Getting to uptake: Do communities of practice support the implementation of evidence-based practice? Journal of the Canadian Academy of Child and Adolescent Psychiatry 18:16

    PubMed Central  PubMed  Google Scholar 

  • Bertone MP, Meessen B, Clarysse G, Hercot D, Kelley A, Kafando Y, Lange I, Pfaffmann J, Ridde V, Sieleunou I (2013) Assessing communities of practice in health policy: A conceptual framework as a first step towards empirical research. Health Research Policy and Systems 11:39

    Article  PubMed Central  PubMed  Google Scholar 

  • Botkin JW (1999). Smart business: How knowledge communities can revolutionize your company. New York: The Free Press

    Google Scholar 

  • Charron DF (2012) Ecohealth Research in Practice: Innovative Applications of an Ecosystem Approach to Health, Insight and Innovation in Development, New York: Springer and Ottawa, Canada: International Development Research Centre

    Google Scholar 

  • Chu M-T, Khosla R (2009) Index evaluations and business strategies on communities of practice. Expert Systems with Applications 36:1549-1558

    Article  Google Scholar 

  • Chu M-T, Khosla R, Nishida T (2012) Communities of practice model driven knowledge management in multinational knowledge based enterprises. Journal of Intelligent Manufacturing 23:1707-1720

    Article  Google Scholar 

  • Conklin J, Stolee P (2008) A model for evaluating knowledge exchange in a network context. CJNR (Canadian Journal of Nursing Research) 40:116-124

    Google Scholar 

  • Cox A (2005) What are communities of practice? A comparative review of four seminal works. Journal of Information Science 31:527-540

    Article  Google Scholar 

  • Creech H, Ramji A (2004) Knowledge networks: Guidelines for assessment. International Institute for Sustainable Development. http://www.iisd.org/pdf/2004/networks_guidelines_for_assessment.pdf. Accessed 7 Oct, 2013.

  • Creech H, Laurie M, Paas L, Parry J-E (2012) Performance improvement and assessment of collaboration: Starting points for networks and communities of practice. International Institute for Sustainable Development. http://www.iisd.org/pdf/2012/performance_improvement_networks.pdf. Accessed 7 Oct 2013.

  • Davis K, Drey N, and Gould D (2009) What are scoping studies? A review of the nursing literature. International journal of nursing studies 46:1386-1400.

    Article  PubMed  Google Scholar 

  • De Plaen R, Kilelu C (2004) From multiple voices to a common language: Ecosystem approaches to human health as an emerging paradigm. EcoHealth 1:SU8-SU15

    Article  Google Scholar 

  • Dubé L, Bourhis A, Jacob R (2006) Towards a typology of virtual communities of practice. Interdisciplinary Journal of Information, Knowledge, and Management 1:69–93

  • Fenton E, Harvey J, Sturt J (2007) Evaluating primary care research networks. Health Services Management Research 20:162-173

    Article  PubMed  Google Scholar 

  • Fischer G (2001) Communities of interest: Learning through the interaction of multiple knowledge systems. In Proceedings of the 24th IRIS Conference.

  • Flynn-Dapaah K (2003) Ecosystem approaches to human health global community of practice: Report on the design phase consultations. International Development Research Centre, Ottawa

    Google Scholar 

  • Grootveld A, Helms R (2008) Development and application of a factor framework to diagnose possible failure of communities of practice. In The 9th European Conference on Knowledge Management: Eckm 2008.

  • Haas PM (1992) Introduction: epistemic communities and international policy coordination. International organization 46(1):1-35.

    Article  Google Scholar 

  • Kim S, Hong J, Suh E (2012) A diagnosis framework for identifying the current knowledge sharing activity status in a community of practice. Expert Systems with Applications 39:130893

    Google Scholar 

  • Kitson A, Harvey G, McCormack B (1998) Enabling the implementation of evidence based practice: a conceptual framework. Quality in Health care 7:149-158

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Lave J, Wenger E (1991) Situated learning: Legitimate peripheral participation. Cambridge university press

    Google Scholar 

  • Lee J, Suh E-h, Hong J (2010) A maturity model based cop evaluation framework: A case study of strategic cops in a Korean company. Expert Systems with Applications 37:2670-2681

    Article  Google Scholar 

  • Lesser EL, Storck J (2001) Communities of practice and organizational performance. IBM systems journal 40:831-841

    Article  Google Scholar 

  • Levac D, Colquhoun H, and O’Brien KK (2010) Scoping studies: advancing the methodology. Implementation Science 5:1-9

    Article  Google Scholar 

  • Li LC, Grimshaw JM, Nielsen C, Judd M, Coyte PC, Graham ID (2009a) Evolution of Wenger’s concept of community of practice. Implementation Science 4:11

    Article  PubMed Central  PubMed  Google Scholar 

  • Li LC, Grimshaw JM, Nielsen C, Judd M, Coyte PC, Graham ID (2009b) Use of communities of practice in business and health care sectors: A systematic review. Implementation Science 4:16

    Article  Google Scholar 

  • Loyarte E, Rivera O (2007) Communities of practice: A model for their cultivation. Journal of Knowledge Management 11:67-77

    Article  Google Scholar 

  • McDermott R (1999) Nurturing three-dimensional communities of practice. Knowledge Management Review: 11:26-29

    Google Scholar 

  • McDermott R (2002) Measuring the impact of communities what happens when you need to show proof that the communities are working and are worth the investment? This article explains how to make sense of a chain of community activities, outcomes and value in the language of the business. Knowledge Management Review 5:26-29

    Google Scholar 

  • Meessen B, Bertone MP (2012) Assessing performance of communities of practice in health policy.

  • Parkes MW (2011) Diversity, emergence, resilience: Guides for a new generation of ecohealth research and practice. EcoHealth 8:137-139

    Article  PubMed  Google Scholar 

  • Parkes MW, Charron DF, Sanchez A (2012) Better together: field-building networks at the frontiers of ecohealth research. In: Ecohealth Research in Practice: Innovative Applications of an Ecosystem Approach to Health, Charron DF (editor), Insight and Innovation in Development, New York, NY: Springer and Ottawa, Canada: International Development Research Centre, pp 231–253.

  • Patton MQ (2011) Developmental evaluation: Applying coplexity concepts to enhance innovation and use. New York: The Guilford Press.

    Google Scholar 

  • Perrin B (2002) How to—and how not to—evaluate innovation. Evaluation 8:13-28

    Article  Google Scholar 

  • Ranmuthugala G, Cunningham FC, Plumb JJ, Long J, Georgiou A, Westbrook JI, Braithwaite J (2011a) A realist evaluation of the role of communities of practice in changing healthcare practice. Implementation Science 6:49

    Article  PubMed Central  PubMed  Google Scholar 

  • Ranmuthugala G, Plumb JJ, Cunningham FC, Georgiou A, Westbrook JI, Braithwaite J (2011b) How and why are communities of practice established in the healthcare sector? A systematic review of the literature. BMC health services research 11:273

    Article  PubMed Central  PubMed  Google Scholar 

  • Rogers PJ (2008) Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation 14:29-48

    Article  Google Scholar 

  • Scarso E, Bolisani E (2008) Communities of practice as structures for managing knowledge in networked corporations. Journal of Manufacturing Technology Management 19:374-390

    Article  Google Scholar 

  • Scarso E, Bolisani E, Salvador L (2009) A systematic framework for analysing the critical success factors of communities of practice. Journal of Knowledge Management 13:431-447

    Article  Google Scholar 

  • Serrat O (2010) Knowledge solutions: tools, methods and approaches to drive development forward and enhance its effects. Asian Development Bank.

  • Spiegel J, Breilh J, Beltran E, Parra J, Solis F, Yassi A, et al (2011) Establishing a community of practice of researchers, practitioners, policy-makers and communities to sustainably manage environmental health risks in Ecuador. BMC international health and human rights 11:S5

  • Stein J, Stren R, Fitzgibbon J, MacLean M (2001) Networks of knowledge: Collaborative innovation in international learning. University of Toronto Press, Toronto

    Google Scholar 

  • Stolee P, Conklin J (n.d.) Assessment of impact of SHRTN activities. http://seniorshealthknowledgenetwork.ca/sites/seniorshealthknowledgenetwork.ca/files/Model_to_guide_the_evaluation_of_SHRTN_2_.pdf. Accessed 7 Oct 2013.

  • Van Eerd D, Cole D, Keown K, Irvin E, Kramer D, Brenneman Gibson J, Kohn M, Mahood Q, Slack T, Amick III B, Phipps D, Garcia J, Morassaei S (2011) Report on knowledge transfer and exchange practices: A systematic review of the quality and types of instruments used to assess kte implementation and impact. Institute for Work & Health, Toronto

    Google Scholar 

  • Verburg RM, Andriessen J (2006) The assessment of communities of practice. Knowledge and Process Management 13:13-25

    Article  Google Scholar 

  • Webb JC, Mergler D, Parkes MW, Saint-Charles J, Spiegel J, Waltner-Toews D, Yassi A, Woollard RF (2010) Tools for thoughtful action: The role of ecosystem approaches to health in enhancing public health. Canadian Journal of Public Health 101:439-441

    PubMed  Google Scholar 

  • Wenger E (1998) Communities of practice: Learning, meaning, and identity. Boston: Cambridge university press

    Book  Google Scholar 

  • Wenger E (2004) Knowledge management as a doughnut: Shaping your knowledge strategy through communities of practice. Ivey Business Journal 68:1-8

    Google Scholar 

  • Wenger E, McDermott R, Snyder WM (2002) Cultivating communities of practice: A guide to managing knowledge. Boston: Harvard Business School Press

    Google Scholar 

  • Wenger E, Trayner B, de Laat M (2011) Promoting and assessing value creation in communities and networks: a conceptual framework. http://wenger-trayner.com/wp-content/uploads/2011/12/11-04-Wenger_Trayner_DeLaat_Value_creation.pdf. Accessed 7 Oct 2013.

  • Yap R, Robben J (2012) Measuring the value of online communities and networks of practice for business. In: Proceedings of the 8th International Conference on Networked Learning.

  • Zhang J, Luna-Reyes LF, Nakashima M, Gil-Garcia JR, Sayogo DS, Mellouli S (2011) Building and sustaining a transnational and interdisciplinary research group: lessons learned from a North American experience. In: System Sciences (HICSS), 2011 44th Hawaii International Conference on System Sciences.

Download references

Author information

Authors and Affiliations

  1. Institute of Health Policy, Management and Evaluation, Faculty of Medicine, University of Toronto, 155 College Street, Suite 425, Toronto, ON, M5T 3M6, Canada

    Kaileah A. McKellar, Kristen B. Pitzul & Juliana Y. Yi

  2. Community of Practice in Ecosystems Approaches to Health, 155 College St, Toronto, ON, Canada

    Kaileah A. McKellar & Donald C. Cole

  3. Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, ON, Canada

    Donald C. Cole

Authors
  1. Kaileah A. McKellar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristen B. Pitzul
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juliana Y. Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Donald C. Cole
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kaileah A. McKellar.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 25 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McKellar, K.A., Pitzul, K.B., Yi, J.Y. et al. Evaluating Communities of Practice and Knowledge Networks: A Systematic Scoping Review of Evaluation Frameworks. EcoHealth 11, 383–399 (2014). https://doi.org/10.1007/s10393-014-0958-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10393-014-0958-3

Keywords

Search

Navigation