Skip to main content
main-content

Tipp

Weitere Kapitel dieses Buchs durch Wischen aufrufen

2021 | OriginalPaper | Buchkapitel

Cbow Training Time and Accuracy Optimization Using SkipGram

verfasst von : Toufik Mechouma, Ismail Biskri, Jean Guy Meunier, Alaidine Ben Ayed

Erschienen in: Advances in Computational Collective Intelligence

Verlag: Springer International Publishing

share
TEILEN

Abstract

Most word embedding techniques get their theoretical foundation from distributional semantics theory. They have been among the most popular trends of natural language processing for the last two decades. They have a large range of application. The present paper presents an overview of recent word embedding techniques. Furthermore, it proposes an optimized continuous bag of word (Cbow) model. The experiments we conducted show that the proposed approach outperforms the classic Cbow technique in terms of accuracy and training time.
Literatur
1.
Zurück zum Zitat Berners, T., Hendler, J., Lassila, O.: The semantic web. Sci. Am. 284(5), 34–43 (2001) CrossRef Berners, T., Hendler, J., Lassila, O.: The semantic web. Sci. Am. 284(5), 34–43 (2001) CrossRef
3.
Zurück zum Zitat Bobrow, D.: Natural Language Input for a Computer Problem Solving System, Massachusetts Institute of Technology 201 Vassar Street, W59–200 Cambridge, MA, USA (1964) Bobrow, D.: Natural Language Input for a Computer Problem Solving System, Massachusetts Institute of Technology 201 Vassar Street, W59–200 Cambridge, MA, USA (1964)
4.
Zurück zum Zitat Weizenbaum, J.: Computer Power and Human Reason, pp. 188–189. From Judgment to Calculation W. H. Freeman and Company, San Francisco (1976). ISBN 0-7167-0463-3 Weizenbaum, J.: Computer Power and Human Reason, pp. 188–189. From Judgment to Calculation W. H. Freeman and Company, San Francisco (1976). ISBN 0-7167-0463-3
5.
Zurück zum Zitat Schank, R.: A conceptual dependency parser for natural language. In: Proceedings of the 1969 Conference on Computational Linguistics, Sång-Säby, pp. 1–3. Sweden (1969) Schank, R.: A conceptual dependency parser for natural language. In: Proceedings of the 1969 Conference on Computational Linguistics, Sång-Säby, pp. 1–3. Sweden (1969)
6.
Zurück zum Zitat Aaronson, D.: Computer use in cognitive psychology. Behav. Res. Meth. Instrum. Comput. 26, 81–93 (1994) CrossRef Aaronson, D.: Computer use in cognitive psychology. Behav. Res. Meth. Instrum. Comput. 26, 81–93 (1994) CrossRef
7.
Zurück zum Zitat Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. (1990) Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. (1990)
8.
Zurück zum Zitat Salton, G., Wong, A., Yang, C.: A vector space model for automatic indexing [archive]. Commun. ACM 18(11), 613–620 (1975) CrossRef Salton, G., Wong, A., Yang, C.: A vector space model for automatic indexing [archive]. Commun. ACM 18(11), 613–620 (1975) CrossRef
9.
Zurück zum Zitat Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality (2013) Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality (2013)
10.
11.
Zurück zum Zitat Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014) Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
12.
Zurück zum Zitat Bojanowski, P., Grave, P., Joulin, E., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017) CrossRef Bojanowski, P., Grave, P., Joulin, E., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017) CrossRef
13.
Zurück zum Zitat Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5 an open multilingual graph of general knowledge. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017) Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5 an open multilingual graph of general knowledge. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)
14.
Zurück zum Zitat Speer, R., J Duda, J.: ConceptNet extending word embeddings with multilingual relational knowledge. In: SemEval-2017 (2017) Speer, R., J Duda, J.: ConceptNet extending word embeddings with multilingual relational knowledge. In: SemEval-2017 (2017)
15.
Zurück zum Zitat Faruqui, M., Sujay, J., Jauhar, K., Hovy, C.E., Smith, N.A.: Retrofitting word vectors to semantic lexicons. In: Proceedings of NAACL (2015) Faruqui, M., Sujay, J., Jauhar, K., Hovy, C.E., Smith, N.A.: Retrofitting word vectors to semantic lexicons. In: Proceedings of NAACL (2015)
17.
Zurück zum Zitat Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71 (1988) CrossRef Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71 (1988) CrossRef
18.
Zurück zum Zitat McDonald, S., Ramscar, M.: Testing the distributional hypothesis: the influence of context on judgements of semantic similarity. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 23(23) (2001) McDonald, S., Ramscar, M.: Testing the distributional hypothesis: the influence of context on judgements of semantic similarity. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 23(23) (2001)
Metadaten
Titel
Cbow Training Time and Accuracy Optimization Using SkipGram
verfasst von
Toufik Mechouma
Ismail Biskri
Jean Guy Meunier
Alaidine Ben Ayed
Copyright-Jahr
2021
DOI
https://doi.org/10.1007/978-3-030-88113-9_46

Premium Partner