1 Introduction
2 Mixing methods with SNA
3 Combining SNA with qualitative methods
Participants (countries as receivers) | |||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
BE | BG | CZ | HR | EE | FI | FR | DE | EL | HU | IE | IT | LT | PL | SI | ES | ||
Speakers (countries as contributors) | BE | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | |
BG | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
CZ | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
HR | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | ||
EE | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | ||
FI | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
FR | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | ||
DE | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | ||
EL | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | ||
HU | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
IE | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | ||
IT | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | ||
LT | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
PL | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | ||
SI | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
ES | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 |
4 Combining SNA with quantitative methods
FI | FR | DE | IT | NL | SE | UK | |
---|---|---|---|---|---|---|---|
FI | 21 | 21 | 16 | 21 | 21 | 23 | |
FR | 21 | 20 | 15 | 20 | 21 | 20 | |
DE | 21 | 20 | 16 | 21 | 21 | 21 | |
IT | 16 | 15 | 16 | 16 | 16 | 16 | |
NL | 21 | 20 | 21 | 16 | 20 | 21 | |
SE | 21 | 21 | 21 | 16 | 20 | 21 | |
UK | 23 | 20 | 21 | 16 | 21 | 21 |
4.1 Word frequency analysis
Managerial term | Count | Stemmed terms | Procedural term | Count | Stemmed terms |
---|---|---|---|---|---|
evaluation | 208 | evaluate, evaluated, evaluating, evaluations | decree | 183 | decrees |
assessments | 103 | assess, assessed, assesses, assessing, assessment | order | 101 | |
quality | 103 | law | 98 | laws | |
improve | 65 | improved, improvement, improvements, improving | procedures | 86 | procedure |
financial | 65 | financially | regulations | 84 | regulate, regulated, regulates, regulating, regulation |
management | 53 | managed, management, manager, managers, manages, managing | principle | 75 | principles |
scientific | 53 | responsible | 75 | responsibilities, responsibility, responsiveness | |
legally | 74 | legal | |||
contract | 60 | contracting, contracts | |||
provisions | 55 | provision | |||
requirements | 54 | require, required, requires, requiring | |||
TOT | 650 | 945 |
Managerial term | Count | Stemmed terms | Procedural term | Count | Stemmed terms |
---|---|---|---|---|---|
quality | 200 | process | 116 | processes | |
qaa | 120 | requirements | 71 | require, required, requires | |
assessment | 84 | assess, assessed, assessing, assessments | legislative | 53 | legislates, legislation |
skills | 84 | skill, skilled | responsible | 49 | response, responses, responsibilities, responsibility, responsive |
standards | 62 | standard, standardized | procedures | 48 | procedure |
science | 47 | science | provisions | 46 | provisions |
improve | 46 | improved, improvement, improvements, improves, improving | code | 44 | code |
outcomes | 43 | outcome | |||
experiments | 43 | experience, experiences, experiment | |||
audits | 42 | audit, audited | |||
TOT | 771 | 427 |
4.2 Topic modeling
4.3 Cluster analysis
Year | Jaccard index |
---|---|
2003 | 0.266466 |
2005 | 0.28813 |
2007 | 0.315357 |
2009 | 0.333989 |
Year | Managerial countries | Procedural countries | ||
---|---|---|---|---|
Procedural language | Managerial language | Procedural language | Managerial language | |
2003 | 0.77 | 0.78 | 0.98 | 0.79 |
2005 | 0.71 | 0.75 | 0.95 | 0.85 |
2007 | 0.67 | 1.00 | 1.06 | 0.92 |
2009 | 0.68 | 0.97 | 1.05 | 0.65 |
4.4 Topic classification
Country type | Reporting year | Topic code | Topic(Culture) | Absolute relevance | Relative relevancea |
---|---|---|---|---|---|
Managerial | 2003 | 200 | Managerial | 0.76371622832716 | 100 |
2003 | 100 | Procedural | 0.72239408649603 | 95 | |
Procedural | 2003 | 100 | Procedural | 0.71181774349084 | 100 |
2003 | 200 | Managerial | 0.29312920013586 | 41 | |
Managerial | 2005 | 100 | Procedural | 0.71752448453608 | 100 |
2005 | 200 | Managerial | 0.68636489579114 | 96 | |
Procedural | 2005 | 100 | Procedural | 0.71121876370013 | 100 |
2005 | 200 | Managerial | 0.33448263578157 | 47 | |
Managerial | 2007 | 200 | Managerial | 0.78661582474291 | 100 |
2007 | 100 | Procedural | 0.51852132933845 | 66 | |
Procedural | 2007 | 100 | Procedural | 0.71120211492418 | 100 |
2007 | 200 | Managerial | 0.32930227026205 | 46 | |
Managerial | 2009 | 100 | Procedural | 0.71230478087649 | 100 |
2009 | 200 | Managerial | 0.71000083199562 | 100 | |
Procedural | 2009 | 100 | Procedural | 0.7107722323049 | 100 |
2009 | 200 | Managerial | 0.26547817309187 | 37 |
5 Systematic comparison and discussion
Time employeda | Human resources | Research resources | Unobtrusiveness | Effectiveness | |
---|---|---|---|---|---|
Qualitative S1 | |||||
Minutes-based SNA | 24 h | 1 researcher with SNA and software-specific skills | Publicly available meeting minutes, PDF editor, file storage space, SNA software | High | High |
Qualitative text analysis | 80 h | 1 researcher with qualitative analysis skills | Publicly available meeting minutes, PDF editor, file storage space | High | High |
Interviews | 9 h | 1 researcher with qualitative analysis skills; 5 informants | List of email addresses, teleconference softwareb | Medium | High |
Total S1 | 113 h | 1 researcher with SNA, software-specific, and qualitative analysis skills; 5 informants | Publicly available documents, PDF editor, file storage space, email addresses, teleconference software, SNA software | High-medium | High |
Quantitative S2 | |||||
Minutes-based SNA | 24 h | 1 researcher with SNA and software-specific skills | Publicly available meeting minutes, PDF editor, file storage space, SNA software | High | Low |
Word frequency analysis | 2 h | 1 researcher with basic quantitative analysis skills | Publicly available governmental reports, file storage space, text analysis software | High | Low |
Topic modeling in R | 24 h | 1 researcher with programming skills | Publicly available governmental reports, file storage space, (considerable) random access memory (RAM) space | High | Medium |
Cluster analysis (and Jaccard index) | 8 h | 1 researcher with software-specific skills | Publicly available governmental reports, file storage space, PDF editor, text analysis software | High | Medium |
Topic classification | 6 h | 1 researcher with software-specific skills | Publicly available governmental reports, file storage space, PDF editor, text analysis software | High | Medium |
Total S2 | 64 h | 1 researcher with SNA, software-specific, quantitative analysis, and programming skills | Publicly available documents, file storage space, (considerable) RAM space, SNA software, text analysis software, PDF editor | High | Medium–low |