1 Introduction
2 Method
2.1 Selecting articles (inclusion)
2.2 Screening articles (exclusion)
-
Concept maps were used as a research instrument;
-
The study was an empirical study;
-
Respondents made their own concept map; and
-
An open-ended concept map assignment was applied.
2.3 Data selection from articles
Aspect | Definition | Selection criteria | Signal words | Coding of clusters |
---|---|---|---|---|
General information | Authors and year of publication | Authors Year of publication | ||
Research design | How the research is performed, i.e. the empirical or (quasi) experimental setting in terms of the number of respondents, the number of measurements and the inclusions of a control group | The number of respondents The number of measurements The inclusion of a control group | Research design Respondents Measurement Control group | One test, one group One test, more groups Pre/post-test one group Pre/post-test more groups More than two tests, one group More than two tests, more groups |
Research object | The phenomena or subject under study, for instance students’ perception of leadership or patterns of learning. All references to the research object (for instance knowledge, learning or mental model) are included | Research object in title Research object in abstract Keywords How studies refer to what they study or explore The objects as described in the results The objects as described in the conclusion | Knowledge Knowledge structure Learning Perception Mental model Systems thinking | Knowledge Knowledge structure Learning Perception Mental model Systems thinking |
Method of analysis | The way in which the method of analysis is referred to, or how data is considered, as described in the article, for instance traditional counting, comparative analysis or holistic scoring | Method of analysis described Ways in which data is explored How studies evaluate concept maps/measure concept map Why these (rationale) | Analysis Method Score/scoring | Quantitative Qualitative Holistic Similarity |
Concept map characteristics | What measures or aspects of the concept map are taken into account, for instance the number of nodes, node validity, the terms used, breadth and depth of the map, number and level of hierarchies, or overlap with reference map | Concept map measures described Concept map characteristics measured Why these (rationale) | Measure Node Link Number Overall Overlap | Size Structural complexity Type of structure Semantic sophistication Category representation Interlinkage Complexity index |
Conclusions | The results presented (about measures or characteristics included) and conclusions drawn (about the research object) based on these results | The results presented and conclusions drawn based on these results | Increase Decrease More Less Understanding Growth | Quantitative Qualitative Holistic Similarity |
2.4 Data analysis
3 Results
3.1 Methods of analysis
Focus of analysis | Application of analysis | References to method of analysis |
---|---|---|
Nodes | Describing and counting the nodes belonging to categories of nodes | “Analysis of categories” by Segalàs, Mulder and Ferrer-Balas (2012, p. 295) “Categorical scoring” by Watson, Pelkey, Noyes and Rodgers (2016b, p. 549) “Thematic analysis” by Ward and Haigh (2017, p. 1243) “Representational level analysis” by Yaman and Ayas (2015, p. 844) |
Nodes and links | Counting the number of nodes and/or links | “Quantitative analysis” by Beyerbach (1988, p. 339), Blackwell and Williams (2007, p. 704), Ifenthaler et al. (2011, p. 56), Weiss, Pellegrino and Brigham (2017, p. 65) “Traditional analysis” by Watson and colleagues (2016b, p. 549) “Traditional quantitative analysis” by Besterfield-Sacre, Gerchak, Lyons, Shuman and Wolfe (2004, p. 105) “Structural scoring” by West and colleagues (2002, p. 1107) |
Nodes and links | Scoring nodes and links based on a rubric | |
Nodes and links | Scoring nodes and links based on a reference map | “Scoring overlap” by Freeman and Urbaczewski (2002, p. 41) “Similarity scoring” by Beyerbach (1988, p. 342) “Existential assessment” by Gregoriades, Pampaka and Michail 2009, p. 423) |
Nodes and propositions | The number and validity of nodes or propositions | “Relational analysis” by Gregoriades et al. (2009, p. 423) “Quantitative criteria analysis” by Yaman and Ayas (2015, p. 846) |
Nodes and propositions | Describing nodes and propositions inductively | “Qualitative analysis” by Blackwell and Williams (2007, p. 703), Weiss et al. (2017, p. 71), Wormnaes Mkumbo, Skaar, and Refseth (2015, p. 371) “Content analysis” by Beyerbach (1988, p. 342), Van den Boogaart, Hummel and Kirschner (2018, p. 298), Kostromina, Gnedykh and Ruschack (2017, p. 316) “Inductive coding” by Ritchhart, Turner and Hadar (2009, p. 151) |
Nodes and propositions | Describing nodes and propositions based on a reference map | “Qualitative analysis” by Freeman and Urbaczewski (2002, p. 51) |
Type of structure | Describing the type of structure | |
Links | Describing or counting the interlinkage between old and new nodes |
3.2 Concept map characteristics
Concept map characteristic | Operationalization of the concept map characteristic or measures | Rationale behind the concept map characteristic or operationalization |
---|---|---|
Size | Quantitative: counting the number of nodes or propositions, including or excluding invalid nodes | Size is a basic measure for knowledge extent or specificity and is easy to determine |
Structural complexity | Quantitative: structural complexity was explored by scoring of hierarchies, cross-links and examples, but also by counting the number of links and levels, or counting measures from graph theory Holistic: scoring structural complexity based on a rubric Similarity: scoring the structural complexity in comparison with a reference map | Structural complexity is interpreted as an indicator of respondents’ understanding and was the main focus of the first scoring system as proposed by Novak and Gowin (1984). It is considered as a relatively objective measurement that is related to the complexity of respondents’ knowledge structure or understanding |
Type of structure | Qualitative: categorizing the type of structure of the map as a whole based on common global morphologies | The type of structure of the map provides a more holistic view of the structure and is easy to score |
Semantic sophistication | Qualitative: semantic sophistication was explored by describing and interpreting the terms used Holistic: terms used were scored based on a rubric Similarity: terms used were compared to a reference map | The semantic sophistication shows which concepts are considered and how these are described, providing insights into the content of maps and making maps with different terms more comparable |
Category representation | Based on the terms used, categories of nodes and/or links were distinguished inductively or deductively (qualitative) Qualitative: the categories are described Quantitative: category representation is calculated as the number of nodes per category, or as a percentage of total number of nodes in the concept map | Category representation is interpreted as an indicator of knowledge coverage or balance, considering the representation of relevant categories |
Interlinkage | Quantitative: counting the number of links between categories Qualitative: describing the type of links (e.g. validity of links) between categories | The number of links between categories provides insights into the interconnectedness or integration of categories in concept maps |
Complexity index | Quantitative: the complexity index is a particularization of interlinkage, dividing the number of interlinks by the number of categories, multiplied by the number of nodes | The complexity index is interpreted as the overall coverage and connectedness of concept maps, combining category representation and interlinkage |