Skip to main content
Top

2018 | OriginalPaper | Chapter

Using Altmetrics to Support Research Evaluation

Author : Mike Thelwall

Published in: Altmetrics for Research Outputs Measurement and Scholarly Information Management

Publisher: Springer Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Altmetrics are indicators that have been proposed as alternatives to citation counts for academic publication impact assessment. Altmetrics may be valued for their speed or ability to reflect the non-scholarly or societal impacts of research. Evidence supports these claims for some altmetrics but many are limited in coverage (the proportion of outputs that have non-zero values) or ability to reflect societal impact. This article describes data sources for altmetrics, indicator formulae, and strategies for applying them for different tasks. It encompasses traditional altmetrics as well webometric and usage indicators.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
2.
go back to reference Holmberg, K.: Altmetrics for Information Professionals Past, Present and Future. Chandos, Oxford (2015) Holmberg, K.: Altmetrics for Information Professionals Past, Present and Future. Chandos, Oxford (2015)
3.
go back to reference Vaughan, L., Hysen, K.: Relationship between links to journal Web sites and impact factors. Aslib Proc. 54(6), 356–361 (2002)CrossRef Vaughan, L., Hysen, K.: Relationship between links to journal Web sites and impact factors. Aslib Proc. 54(6), 356–361 (2002)CrossRef
6.
go back to reference Kousha, K., Thelwall, M.: Web indicators for research evaluation, Part 3: books and non-standard outputs. El Profesional de la Información 24(6), 724–736 (2015)CrossRef Kousha, K., Thelwall, M.: Web indicators for research evaluation, Part 3: books and non-standard outputs. El Profesional de la Información 24(6), 724–736 (2015)CrossRef
7.
go back to reference Sugimoto, C.R., Work, S., Larivière, V., Haustein, S.: Scholarly use of social media and altmetrics: a review of the literature. J. Assoc. Inf. Sci. Technol. 68(9), 2037–2062 (2017)CrossRef Sugimoto, C.R., Work, S., Larivière, V., Haustein, S.: Scholarly use of social media and altmetrics: a review of the literature. J. Assoc. Inf. Sci. Technol. 68(9), 2037–2062 (2017)CrossRef
8.
go back to reference Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S.: The Metric Tide: Independent Review of the Role of Metrics in Research Assessment and Management. HEFCE, London (2015)CrossRef Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S.: The Metric Tide: Independent Review of the Role of Metrics in Research Assessment and Management. HEFCE, London (2015)CrossRef
9.
go back to reference Haustein, S., Siebenlist, T.: Applying social bookmarking data to evaluate journal usage. J. Informetr. 5, 446–457 (2011) Haustein, S., Siebenlist, T.: Applying social bookmarking data to evaluate journal usage. J. Informetr. 5, 446–457 (2011)
10.
go back to reference Piwowar, H., Priem, J.: The power of altmetrics on a CV. Bull. Assoc. Inf. Sci. Technol. 39(4), 10–13 (2013)CrossRef Piwowar, H., Priem, J.: The power of altmetrics on a CV. Bull. Assoc. Inf. Sci. Technol. 39(4), 10–13 (2013)CrossRef
11.
go back to reference Merton, R.K.: The Sociology of Science: Theoretical and Empirical Investigations. University of Chicago Press, Chicago (1973) Merton, R.K.: The Sociology of Science: Theoretical and Empirical Investigations. University of Chicago Press, Chicago (1973)
12.
go back to reference Krampen, G., Becker, R., Wahner, U., Montada, L.: On the validity of citation counting in science evaluation: content analyses of references and citations in psychological publications. Scientometrics 71(2), 191–202 (2007)CrossRef Krampen, G., Becker, R., Wahner, U., Montada, L.: On the validity of citation counting in science evaluation: content analyses of references and citations in psychological publications. Scientometrics 71(2), 191–202 (2007)CrossRef
13.
go back to reference Campbell, F.M.: National bias: a comparison of citation practices by health professionals. Bull. Med. Libr. Assoc. 78(4), 376 (1990) Campbell, F.M.: National bias: a comparison of citation practices by health professionals. Bull. Med. Libr. Assoc. 78(4), 376 (1990)
14.
go back to reference Pasterkamp, G., Rotmans, J., de Kleijn, D., Borst, C.: Citation frequency: a biased measure of research impact significantly influenced by the geographical origin of research articles. Scientometrics 70(1), 153–165 (2007)CrossRef Pasterkamp, G., Rotmans, J., de Kleijn, D., Borst, C.: Citation frequency: a biased measure of research impact significantly influenced by the geographical origin of research articles. Scientometrics 70(1), 153–165 (2007)CrossRef
15.
go back to reference Seglen, P.O.: Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthopaedica Scandinavica 69(3), 224–229 (1998)CrossRef Seglen, P.O.: Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthopaedica Scandinavica 69(3), 224–229 (1998)CrossRef
16.
go back to reference Althouse, B.M., West, J.D., Bergstrom, C.T., Bergstrom, T.: Differences in impact factor across fields and over time. J. Assoc. Inf. Sci. Technol. 60(1), 27–34 (2009)CrossRef Althouse, B.M., West, J.D., Bergstrom, C.T., Bergstrom, T.: Differences in impact factor across fields and over time. J. Assoc. Inf. Sci. Technol. 60(1), 27–34 (2009)CrossRef
17.
go back to reference van Driel, M.L., Maier, M., Maeseneer, J.D.: Measuring the impact of family medicine research: scientific citations or societal impact? Fam. Pract. 24(5), 401–402 (2007)CrossRef van Driel, M.L., Maier, M., Maeseneer, J.D.: Measuring the impact of family medicine research: scientific citations or societal impact? Fam. Pract. 24(5), 401–402 (2007)CrossRef
18.
go back to reference Glänzel, W., Schubert, A.: A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics 56(3), 357–367 (2003)CrossRef Glänzel, W., Schubert, A.: A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics 56(3), 357–367 (2003)CrossRef
19.
go back to reference Dinsmore, A., Allen, L., Dolby, K.: Alternative perspectives on impact: the potential of ALMs and altmetrics to inform funders about research impact. PLoS Biol. 12(11), e1002003 (2014)CrossRef Dinsmore, A., Allen, L., Dolby, K.: Alternative perspectives on impact: the potential of ALMs and altmetrics to inform funders about research impact. PLoS Biol. 12(11), e1002003 (2014)CrossRef
20.
go back to reference Fenner, M.: What can article-level metrics do for you? PLoS Biol. 11(10), e1001687 (2013)CrossRef Fenner, M.: What can article-level metrics do for you? PLoS Biol. 11(10), e1001687 (2013)CrossRef
21.
go back to reference Maggio, L.A., Meyer, H.S., Artino, A.R.: Beyond citation rates: a real-time impact analysis of health professions education research using altmetrics. Acad. Med. 92(10), 1449–1455 (2017)CrossRef Maggio, L.A., Meyer, H.S., Artino, A.R.: Beyond citation rates: a real-time impact analysis of health professions education research using altmetrics. Acad. Med. 92(10), 1449–1455 (2017)CrossRef
24.
go back to reference Mas-Bleda, A., Thelwall, M.: Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research. Scientometrics 109(3), 2007–2030 (2016)CrossRef Mas-Bleda, A., Thelwall, M.: Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research. Scientometrics 109(3), 2007–2030 (2016)CrossRef
25.
go back to reference Ravenscroft, J., Liakata, M., Clare, A., Duma, D.: Measuring scientific impact beyond academia: an assessment of existing impact metrics and proposed improvements. PLoS ONE 12(3), e0173152 (2017)CrossRef Ravenscroft, J., Liakata, M., Clare, A., Duma, D.: Measuring scientific impact beyond academia: an assessment of existing impact metrics and proposed improvements. PLoS ONE 12(3), e0173152 (2017)CrossRef
26.
go back to reference Wouters, P., Costas, R.: Users, narcissism and control: tracking the impact of scholarly publications in the 21st century. In: Science and Technology Indicators 2012 (STI 2012), pp. 847–857. SURF Foundation, Utrecht (2012) Wouters, P., Costas, R.: Users, narcissism and control: tracking the impact of scholarly publications in the 21st century. In: Science and Technology Indicators 2012 (STI 2012), pp. 847–857. SURF Foundation, Utrecht (2012)
29.
go back to reference Robinson-García, N., Torres-Salinas, D., Zahedi, Z., Costas, R.: New data, new possibilities: exploring the insides of Altmetric.com. El Profesional de La Información 23(4), 359–366 (2014)CrossRef Robinson-García, N., Torres-Salinas, D., Zahedi, Z., Costas, R.: New data, new possibilities: exploring the insides of Altmetric.com. El Profesional de La Información 23(4), 359–366 (2014)CrossRef
30.
go back to reference Waltman, L., van Eck, N.J., van Leeuwen, T.N., Visser, M.S., van Raan, A.F.: Towards a new crown indicator: an empirical analysis. Scientometrics 87(3), 467–481 (2011)CrossRef Waltman, L., van Eck, N.J., van Leeuwen, T.N., Visser, M.S., van Raan, A.F.: Towards a new crown indicator: an empirical analysis. Scientometrics 87(3), 467–481 (2011)CrossRef
32.
go back to reference Haustein, S., Larivière, V., Thelwall, M., Amyot, D., Peters, I.: Tweets vs. Mendeley readers: how do these two social media metrics differ? IT Inf. Technol. 56(5), 207–215 (2014) Haustein, S., Larivière, V., Thelwall, M., Amyot, D., Peters, I.: Tweets vs. Mendeley readers: how do these two social media metrics differ? IT Inf. Technol. 56(5), 207–215 (2014)
33.
go back to reference Erdt, M., Nagarajan, A., Sin, S.C.J., Theng, Y.L.: Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media. Scientometrics 109(2), 1117–1166 (2016)CrossRef Erdt, M., Nagarajan, A., Sin, S.C.J., Theng, Y.L.: Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media. Scientometrics 109(2), 1117–1166 (2016)CrossRef
35.
go back to reference Borrego, A., Fry, J.: Measuring researchers’ use of scholarly information through social bookmarking data: a case study of BibSonomy. J. Inf. Sci. 38(3), 297–308 (2012)CrossRef Borrego, A., Fry, J.: Measuring researchers’ use of scholarly information through social bookmarking data: a case study of BibSonomy. J. Inf. Sci. 38(3), 297–308 (2012)CrossRef
36.
go back to reference Costas, R., Zahedi, Z., Wouters, P.: Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. J. Assoc. Inf. Sci. Technol. 66(10), 2003–2019 (2015)CrossRef Costas, R., Zahedi, Z., Wouters, P.: Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. J. Assoc. Inf. Sci. Technol. 66(10), 2003–2019 (2015)CrossRef
38.
go back to reference Thelwall, M., Kousha, K.: Online presentations as a source of scientific impact? An analysis of PowerPoint files citing academic journals. J. Am. Soc. Inf. Sci. Technol. 59(5), 805–815 (2008)CrossRef Thelwall, M., Kousha, K.: Online presentations as a source of scientific impact? An analysis of PowerPoint files citing academic journals. J. Am. Soc. Inf. Sci. Technol. 59(5), 805–815 (2008)CrossRef
39.
go back to reference Thelwall, M., Kousha, K.: SlideShare presentations, citations, users and trends: a professional site with academic and educational uses. J. Assoc. Inf. Sci. Technol. 68(8), 1989–2003 (2017)CrossRef Thelwall, M., Kousha, K.: SlideShare presentations, citations, users and trends: a professional site with academic and educational uses. J. Assoc. Inf. Sci. Technol. 68(8), 1989–2003 (2017)CrossRef
40.
go back to reference Kousha, K., Thelwall, M.: Patent citation analysis with Google. J. Assoc. Inf. Sci. Technol. 68(1), 48–61 (2017)CrossRef Kousha, K., Thelwall, M.: Patent citation analysis with Google. J. Assoc. Inf. Sci. Technol. 68(1), 48–61 (2017)CrossRef
41.
go back to reference Shema, H., Bar-Ilan, J., Thelwall, M.: Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. J. Am. Soc. Inf. Sci. Technol. 65(5), 1018–1027 (2014)CrossRef Shema, H., Bar-Ilan, J., Thelwall, M.: Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. J. Am. Soc. Inf. Sci. Technol. 65(5), 1018–1027 (2014)CrossRef
42.
go back to reference Thelwall, M., Kousha, K., Abdoli, M.: Is medical research informing professional practice more highly cited? Evidence from AHFS DI Essentials in Drugs.com. Scientometrics 112(1), 509–527 (2017)CrossRef Thelwall, M., Kousha, K., Abdoli, M.: Is medical research informing professional practice more highly cited? Evidence from AHFS DI Essentials in Drugs.com. Scientometrics 112(1), 509–527 (2017)CrossRef
43.
go back to reference Thelwall, M., Maflahi, N.: Guideline references and academic citations as evidence of the clinical value of health research. J. Assoc. Inf. Sci. Technol. 67(4), 960–966 (2016)CrossRef Thelwall, M., Maflahi, N.: Guideline references and academic citations as evidence of the clinical value of health research. J. Assoc. Inf. Sci. Technol. 67(4), 960–966 (2016)CrossRef
44.
go back to reference Haustein, S., Bowman, T.D., Holmberg, K., Tsou, A., Sugimoto, C.R., Larivière, V.: Tweets as impact indicators: examining the implications of automated “bot” accounts on Twitter. J. Assoc. Inf. Sci. Technol. 67(1), 232–238 (2016)CrossRef Haustein, S., Bowman, T.D., Holmberg, K., Tsou, A., Sugimoto, C.R., Larivière, V.: Tweets as impact indicators: examining the implications of automated “bot” accounts on Twitter. J. Assoc. Inf. Sci. Technol. 67(1), 232–238 (2016)CrossRef
45.
go back to reference Sud, P., Thelwall, M.: Evaluating altmetrics. Scientometrics 98(2), 1131–1143 (2014)CrossRef Sud, P., Thelwall, M.: Evaluating altmetrics. Scientometrics 98(2), 1131–1143 (2014)CrossRef
49.
go back to reference Halevi, G., Moed, H.F.: Usage patterns of scientific journals and their relationship with citations. In: Context Counts: Pathways to Master Big and Little Data, pp. 241–251 (2014) Halevi, G., Moed, H.F.: Usage patterns of scientific journals and their relationship with citations. In: Context Counts: Pathways to Master Big and Little Data, pp. 241–251 (2014)
51.
go back to reference Mohammadi, E., Thelwall, M., Haustein, S., Larivière, V.: Who reads research articles? An altmetrics analysis of Mendeley user categories. J. Assoc. Inf. Sci. Technol. 66(9), 1832–1846 (2015)CrossRef Mohammadi, E., Thelwall, M., Haustein, S., Larivière, V.: Who reads research articles? An altmetrics analysis of Mendeley user categories. J. Assoc. Inf. Sci. Technol. 66(9), 1832–1846 (2015)CrossRef
54.
go back to reference Tsou, A., Bowman, T.D., Ghazinejad, A., Sugimoto, C.R.: Who tweets about science? In: Proceedings of ISSI 2015 - 15th International Conference of the International Society for Scientometrics and Informetricspp, pp. 95–100. Boğaziçi University Printhouse, Istanbul (2015) Tsou, A., Bowman, T.D., Ghazinejad, A., Sugimoto, C.R.: Who tweets about science? In: Proceedings of ISSI 2015 - 15th International Conference of the International Society for Scientometrics and Informetricspp, pp. 95–100. Boğaziçi University Printhouse, Istanbul (2015)
56.
go back to reference Schloegl, C., Gorraiz, J.: Comparison of citation and usage indicators: the case of oncology journals. Scientometrics 82(3), 567–580 (2010)CrossRef Schloegl, C., Gorraiz, J.: Comparison of citation and usage indicators: the case of oncology journals. Scientometrics 82(3), 567–580 (2010)CrossRef
57.
go back to reference Moed, H.F.: Statistical relationships between downloads and citations at the level of individual documents within a single journal. J. Assoc. Inf. Sci. Technol. 56(10), 1088–1097 (2005)CrossRef Moed, H.F.: Statistical relationships between downloads and citations at the level of individual documents within a single journal. J. Assoc. Inf. Sci. Technol. 56(10), 1088–1097 (2005)CrossRef
58.
go back to reference Wilkinson, D., Sud, P., Thelwall, M.: Substance without citation: evaluating the online impact of grey literature. Scientometrics 98(2), 797–806 (2014)CrossRef Wilkinson, D., Sud, P., Thelwall, M.: Substance without citation: evaluating the online impact of grey literature. Scientometrics 98(2), 797–806 (2014)CrossRef
59.
go back to reference Moed, H.F., Halevi, G.: On full text download and citation distributions in scientific-scholarly journals. J. Assoc. Inf. Sci. Technol. 67(2), 412–431 (2016)CrossRef Moed, H.F., Halevi, G.: On full text download and citation distributions in scientific-scholarly journals. J. Assoc. Inf. Sci. Technol. 67(2), 412–431 (2016)CrossRef
60.
go back to reference Kudlow, P., Cockerill, M., Toccalino, D., Dziadyk, D.B., Rutledge, A., Shachak, A., Eysenbach, G.: Online distribution channel increases article usage on Mendeley: a randomized controlled trial. Scientometrics 112(3), 1537–1556 (2017)CrossRef Kudlow, P., Cockerill, M., Toccalino, D., Dziadyk, D.B., Rutledge, A., Shachak, A., Eysenbach, G.: Online distribution channel increases article usage on Mendeley: a randomized controlled trial. Scientometrics 112(3), 1537–1556 (2017)CrossRef
61.
go back to reference Larivière, V., Archambault, É., Gingras, Y.: Long-term variations in the aging of scientific literature: from exponential growth to steady-state science (1900–2004). J. Assoc. Inf. Sci. Technol. 59(2), 288–296 (2008)CrossRef Larivière, V., Archambault, É., Gingras, Y.: Long-term variations in the aging of scientific literature: from exponential growth to steady-state science (1900–2004). J. Assoc. Inf. Sci. Technol. 59(2), 288–296 (2008)CrossRef
65.
go back to reference Eysenbach, G.: Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J. Med. Internet Res. 13(4), e123 (2011)CrossRef Eysenbach, G.: Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J. Med. Internet Res. 13(4), e123 (2011)CrossRef
66.
go back to reference Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., Rafols, I.: The Leiden Manifesto for research metrics. Nature 520(7548), 429–431 (2015)CrossRef Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., Rafols, I.: The Leiden Manifesto for research metrics. Nature 520(7548), 429–431 (2015)CrossRef
67.
go back to reference Kousha, K., Thelwall, M.: Assessing the impact of disciplinary research on teaching: an automatic analysis of online syllabuses. J. Am. Soc. Inform. Sci. Technol. 59(13), 2060–2069 (2008)CrossRef Kousha, K., Thelwall, M.: Assessing the impact of disciplinary research on teaching: an automatic analysis of online syllabuses. J. Am. Soc. Inform. Sci. Technol. 59(13), 2060–2069 (2008)CrossRef
Metadata
Title
Using Altmetrics to Support Research Evaluation
Author
Mike Thelwall
Copyright Year
2018
Publisher
Springer Singapore
DOI
https://doi.org/10.1007/978-981-13-1053-9_2

Premium Partner