Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2021 | OriginalPaper | Chapter

Cbow Training Time and Accuracy Optimization Using SkipGram

Authors: Toufik Mechouma, Ismail Biskri, Jean Guy Meunier, Alaidine Ben Ayed

Published in: Advances in Computational Collective Intelligence

Publisher: Springer International Publishing

share
SHARE

Abstract

Most word embedding techniques get their theoretical foundation from distributional semantics theory. They have been among the most popular trends of natural language processing for the last two decades. They have a large range of application. The present paper presents an overview of recent word embedding techniques. Furthermore, it proposes an optimized continuous bag of word (Cbow) model. The experiments we conducted show that the proposed approach outperforms the classic Cbow technique in terms of accuracy and training time.
Literature
1.
go back to reference Berners, T., Hendler, J., Lassila, O.: The semantic web. Sci. Am. 284(5), 34–43 (2001) CrossRef Berners, T., Hendler, J., Lassila, O.: The semantic web. Sci. Am. 284(5), 34–43 (2001) CrossRef
3.
go back to reference Bobrow, D.: Natural Language Input for a Computer Problem Solving System, Massachusetts Institute of Technology 201 Vassar Street, W59–200 Cambridge, MA, USA (1964) Bobrow, D.: Natural Language Input for a Computer Problem Solving System, Massachusetts Institute of Technology 201 Vassar Street, W59–200 Cambridge, MA, USA (1964)
4.
go back to reference Weizenbaum, J.: Computer Power and Human Reason, pp. 188–189. From Judgment to Calculation W. H. Freeman and Company, San Francisco (1976). ISBN 0-7167-0463-3 Weizenbaum, J.: Computer Power and Human Reason, pp. 188–189. From Judgment to Calculation W. H. Freeman and Company, San Francisco (1976). ISBN 0-7167-0463-3
5.
go back to reference Schank, R.: A conceptual dependency parser for natural language. In: Proceedings of the 1969 Conference on Computational Linguistics, Sång-Säby, pp. 1–3. Sweden (1969) Schank, R.: A conceptual dependency parser for natural language. In: Proceedings of the 1969 Conference on Computational Linguistics, Sång-Säby, pp. 1–3. Sweden (1969)
6.
go back to reference Aaronson, D.: Computer use in cognitive psychology. Behav. Res. Meth. Instrum. Comput. 26, 81–93 (1994) CrossRef Aaronson, D.: Computer use in cognitive psychology. Behav. Res. Meth. Instrum. Comput. 26, 81–93 (1994) CrossRef
7.
go back to reference Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. (1990) Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. (1990)
8.
go back to reference Salton, G., Wong, A., Yang, C.: A vector space model for automatic indexing [archive]. Commun. ACM 18(11), 613–620 (1975) CrossRef Salton, G., Wong, A., Yang, C.: A vector space model for automatic indexing [archive]. Commun. ACM 18(11), 613–620 (1975) CrossRef
9.
go back to reference Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality (2013) Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality (2013)
10.
11.
go back to reference Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014) Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
12.
go back to reference Bojanowski, P., Grave, P., Joulin, E., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017) CrossRef Bojanowski, P., Grave, P., Joulin, E., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017) CrossRef
13.
go back to reference Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5 an open multilingual graph of general knowledge. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017) Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5 an open multilingual graph of general knowledge. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)
14.
go back to reference Speer, R., J Duda, J.: ConceptNet extending word embeddings with multilingual relational knowledge. In: SemEval-2017 (2017) Speer, R., J Duda, J.: ConceptNet extending word embeddings with multilingual relational knowledge. In: SemEval-2017 (2017)
15.
go back to reference Faruqui, M., Sujay, J., Jauhar, K., Hovy, C.E., Smith, N.A.: Retrofitting word vectors to semantic lexicons. In: Proceedings of NAACL (2015) Faruqui, M., Sujay, J., Jauhar, K., Hovy, C.E., Smith, N.A.: Retrofitting word vectors to semantic lexicons. In: Proceedings of NAACL (2015)
17.
go back to reference Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71 (1988) CrossRef Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71 (1988) CrossRef
18.
go back to reference McDonald, S., Ramscar, M.: Testing the distributional hypothesis: the influence of context on judgements of semantic similarity. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 23(23) (2001) McDonald, S., Ramscar, M.: Testing the distributional hypothesis: the influence of context on judgements of semantic similarity. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 23(23) (2001)
Metadata
Title
Cbow Training Time and Accuracy Optimization Using SkipGram
Authors
Toufik Mechouma
Ismail Biskri
Jean Guy Meunier
Alaidine Ben Ayed
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-88113-9_46

Premium Partner