Top

Social Network Analysis and Mining

Published in:

01-12-2021 | Original Article

Text documents streams with improved incremental similarity

Authors: Rui Portocarrero Sarmento, Douglas O. Cardoso, Kemmily Dearo, Pavel Brazdil, João Gama

Published in: Social Network Analysis and Mining | Issue 1/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

There has been a significant effort by the research community to address the problem of providing methods to organize documentation, with the help of Information Retrieval methods. In this paper, we present several experiments with stream analysis methods to explore streams of text documents. This paper also presents possible architectures of the Text Document Stream Organization, with the use of incremental algorithms like Incremental Sparse TF-IDF and Incremental Similarity. Our results show that with this architecture, significant improvements are achieved, regarding efficiency in grouping of similar documents. These improvements are important since it is of general knowledge that great amounts of text analysis are a high dimensional and complex subject of study, in the data analysis area.

previous article Artificial neural networks applied for predicting and explaining the education level of Twitter users

next article Design and analysis of microblog-based summarization system

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Aggarwal CC, Han J, Wang J, Yu PS (2004) A framework for projected clustering of high dimensional data streams, VLDB ’04, 852–863 (VLDB Endowment). http://dl.acm.org/citation.cfm?id=1316689.1316763

Aggarwal CC, Yu PS (2006) A framework for clustering massive text and categorical data streams 479–483. https://doi.org/10.1137/1.9781611972764.44

Aggarwal CC, Yu PS (2005) Online analysis of community evolution in data streams 56–67. https://doi.org/10.1137/1.9781611972757.6

Blondel VD, Guillaume J-L, Lambiotte R, Lefebvre E. Fast unfolding of community hierarchies in large networks. CoRR abs/0803.0476 (2008)

Carmona Cejudo JM (2013) Nuevas tendencias en fundamentos teóricos aplicaciones de la minería de datos aplicada a la clasificación de textos en lenguaje natural. Ph.D. thesis, U. Salamanca

Cordeiro M, Sarmento R, Gama J (2016) Dynamic community detection in evolving networks using locality modularity optimization. Social Netw Analys Mining 6(1):15:1-15:20. https://doi.org/10.1007/s13278-016-0325-1CrossRef

Cordeiro M, Sarmento RP, Gama J (2016) Dynamic community detection in evolving networks using locality modularity optimization. Soc Netw Anal Min 6(1):1–20. https://doi.org/10.1007/s13278-016-0325-1CrossRef

Corney D, Albakour D, Martinez M, Moussa S (2016) What do a million news articles look like? 42–47. http://ceur-ws.org/Vol-1568/paper8.pdf

Cossu J-V, Labatut V, Dugué N (2016) A review of features for the discrimination of twitter users: application to the prediction of offline influence. Soc Netw Anal Min 6(1):25. https://doi.org/10.1007/s13278-016-0329-xCrossRef

Csardi G, Nepusz T (2006) The igraph software package for complex network research. InterJournal Complex Systems, 1695. http://igraph.org

Eddelbuettel D, Balamuta JJ (2017) Extending extitR with extitC++: A Brief Introduction to extitRcpp. PeerJ Preprints 5:e3188v1. https://doi.org/10.7287/peerj.preprints.3188v1CrossRef

Eddelbuettel D, François R (2011) SRcpp: eamless R and C++ integration. J Stat Softw 40(8):1–18. https://doi.org/10.18637/jss.v040.i08CrossRef

Eddelbuettel D (2013) Seamless R and C++ Integration with Rcpp. Springer, New York. ISBN 978-1-4614-6867-7

Feinerer I, Hornik K, Meyer D (2008) Text mining infrastructure in r. J Stat Softw 25(5):1–54CrossRef

Feinerer I, Hornik K (2018) tm: Text Mining Package. https://CRAN.R-project.org/package=tm. R package version 0.7-5

Feldman R, Sanger J (2006) Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data. Cambridge University Press, New YorkCrossRef

Fortunato S (2010) Community detection in graphs. Phys Rep 486(3–5):75–174. https://doi.org/10.1016/j.physrep.2009.11.002. arXiv:0906.0612v2MathSciNetCrossRef

Gama J (2010) Knowledge Discovery from Data Streams, 1st edn. Chapman & Hall/CRC, CaliforniaCrossRef

Iacobucci D (1994) Graphs and Matrices. In: Wasserman S (ed) Social network analysis: methods and applications. Cambridge University Press, New York, pp 92–166CrossRef

Mersmann O (2018) microbenchmark: Accurate Timing Functions. https://CRAN.R-project.org/package=microbenchmark. R package version 1.4-6

Oliveira MDB, Guerreiro A, Gama J (2014) Dynamic communities in evolving customer networks: an analysis using landmark and sliding windows. Social Netw Analys Mining 4(1):208. https://doi.org/10.1007/s13278-014-0208-2CrossRef

Pons P, Latapy M (2005) Computing communities in large networks using random walks, ISCIS‘05. Springer-Verlag, Berlin, pp 284–293. https://doi.org/10.1007/11569596_31CrossRef

Sarmento RP, Lemos L, Cordeiro M, Rossetti G, Cardoso D (2019) Dyncomm R package - dynamic community detection for evolving networks. CoRR abs/1905.01498. arXiv:1905.01498

Sarmento R, Cordeiro M, Gama J (2015) Streaming networks sampling using top-k networks 228–234. https://doi.org/10.5220/0005341402280234

Trigo L, Víta M, Sarmento R, Brazdil P (2015) Retrieval, visualization and validation of affinities between documents. INSTICC (SciTePress), pp 452–459

Trigo L, Brazdil P (2014) Affinity analysis between researchers using text mining and differential analysis of graphs. https://phdsession-ecmlpkdd2014.greyc.fr/sites/phdsession-ecmlpkdd2014.greyc.fr/files/papers/Paper_20702.pdf

Urbanek S (2017) fastmatch: Fast match() function. https://CRAN.R-project.org/package=fastmatch. R package version 1.1-0

Wasserman S, Faust K (1994) Social network analysis: Methods and applications, vol 8. Cambridge University Press, CambridgeCrossRef

Title: Text documents streams with improved incremental similarity
Authors: Rui Portocarrero Sarmento
Douglas O. Cardoso
Kemmily Dearo
Pavel Brazdil
João Gama
Publication date: 01-12-2021
Publisher: Springer Vienna
Published in: Social Network Analysis and Mining / Issue 1/2021
Print ISSN: 1869-5450
Electronic ISSN: 1869-5469
DOI: https://doi.org/10.1007/s13278-021-00826-z

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 1/2021

Integrating location and textual information for detecting affected people in a crisis

Misleading information in Spanish: a survey

Modeling signed social networks using spectral embedding

A survey on the use of data and opinion mining in social media to political electoral outcomes prediction

Probabilistic reasoning system for social influence analysis in online social networks

Using targeted betweenness centrality to identify bridges to neglected users in the Twitter conversation on veteran suicide

Premium Partner