Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Interaction with Formal Mathematical Documents in Isabelle/PIDE Read chapter Read first chapter

2019 | OriginalPaper | Chapter

Explorations into the Use of Word Embedding in Math Search and Math Semantics

Authors : Abdou Youssef, Bruce R. Miller

Published in: Intelligent Computer Mathematics

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Word embedding, which represents individual words with semantically rich numerical vectors, has made it possible to successfully apply deep learning to NLP tasks such as semantic role modeling, question answering, and machine translation. As math text consists of natural text as well as math expressions that similarly exhibit linear correlation and contextual characteristics, word embedding can be applied to math documents as well. On the other hand, math terms also show characteristics (e.g., abstractions) that are different from textual words. Accordingly, it is worthwhile to explore the use and effectiveness of word embedding in math language processing and MKM.
In this paper, we present exploratory investigations of math embedding by testing it on some basic tasks such as (1) math-term similarity, (2) analogy, (3) basic numerical concept-modeling using a novel approach based on computing the (weighted) centroid of the keywords that characterize a concept, and (4) math search, especially query expansion using the weighted centroid of the query keywords and then expanding the query with new keywords that are most similar to the centroid. Due to lack of benchmarks, our investigations were done using carefully selected illustrations on the DLMF. We draw from our investigations some general observations and lessons that form a trajectory for future statistically significant testing on large benchmarks. Our preliminary results and observations show that math embedding holds much promise but also point to the need for more robust embedding.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Integrating Semantic Mathematical Documents and Dynamic Notebooks
Literature
1.
Bordes, A., et al.: Joint learning of words and meaning representations for open-text semantic parsing. In: AISTATS (2012)
2.
Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. In: EMNLP (2015)
3.
4.
Chelba, C., et al.: One billion word benchmark for measuring progress in statistical language modeling. In: INTERSPEECH (2014)
5.
Chiu, J., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016)CrossRef
6.
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: EMNLP (2014)
7.
Clark, C., Gardner, M.: Simple and effective multi-paragraph reading comprehension. In: ACL 2018, Melbourne, Australia, 15–20 July 2018, pp. 845–855 (2018)
8.
Chen, Q., et al.: Enhanced LSTM for natural language inference. In: ACL (2017)
9.
Devlin, J., et al.: Fast and robust neural network joint models for statistical machine translation. In: Proceedings of the ACL (2014)
10.
Gao, L., et al.: Preliminary exploration of formula embedding for mathematical information retrieval: can mathematical formulae be embedded like a natural language? arXiv:​1707.​05154 (2017)
11.
Gong, Y., Luo, H., Zhang, J.: Natural language inference over interaction space. In: ICLR (2018)
12.
He, L., Lee, K., Lewis, M., Zettlemoyer, L.S.: Deep semantic role labeling: what works and what’s next. In: ACL (2017)
13.
Iacobacci, I., Pilehvar, M.T., Navigli, R.: Embeddings for word sense disambiguation: an evaluation study. In: ACL (2016)
14.
Kim, Y.: Convolutional neural networks for sentence classification. In: EMNLP, Doha, Qatar, pp. 1746–1751, October 2014
15.
Krishna, A., Youssef, A., et al.: Query Expansion for Patent Searching using Word Embedding and Professional Crowdsourcing (in submission)
16.
17.
Lai, S., Liu, K., He, S., Zhao, J.: How to generate a good word embedding. IEEE Intell. Syst. 31(6), 5–14 (2016)CrossRef
18.
Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: International Conference on Machine Learning, pp. 1188–1196, January 2014
19.
Lee, K., He, L., Lewis, M., Zettlemoyer, L.S.: End-to-end neural coreference resolution. In: EMNLP (2017)
20.
21.
Mikolov, T., et al.: Efficient estimation of word representations in vector space. In: Workshops Track, International Conference on Learning Representations (2013)
22.
Mikolov, T., et al.: Distributed representations of words and phrases and their compositionality. In: NIPS, pp. 3111–3119 (2013)
23.
Nickel, M., Kiela, D.: Poincare embeddings for learning hierarchical representations. In: Advances in NIPS (2017)
24.
25.
Palmer, M., et al.: The proposition bank: an annotated corpus of semantic roles. Comput. Linguist. 31, 71–106 (2005)CrossRef
26.
Piotr, B., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. ACL 5, 135–146 (2017)
27.
Peters, M.E., et al.: Deep contextualized word representations. In: Proceedings of NAACL-HLT, pp. 2227–2237 (2018)
28.
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. InL EMNLP, 25–29 October 2014, pp. 1532–1543 (2014)
29.
Raganato, A., Bovi, C.D., Navigli, R.: Neural sequence learning models for word sense disambiguation. In: EMNLP (2017)
30.
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. In: EMNLP (2016)
31.
Rudolph, M., Ruiz, F., Athey, S., Blei, D.M.: Structured embedding models for grouped data. In: NIPS, pp. 250–260 (2017)
32.
Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: EMNLP (2013)
33.
Wiseman, S., Rush, A.M., Shieber, S.M.: Learning global features for coreference resolution. In: HLT-NAACL (2016)
34.
Zhou, J., Xu, W.: End-to-end learning of semantic role labeling using recurrent neural networks. In: ACL (2015)
Metadata
Title
Explorations into the Use of Word Embedding in Math Search and Math Semantics
Authors
Abdou Youssef
Bruce R. Miller
Copyright Year
2019
Book
Intelligent Computer Mathematics

Print ISBN: 978-3-030-23249-8

Electronic ISBN: 978-3-030-23250-4

Copyright Year: 2019

DOI
https://doi.org/10.1007/978-3-030-23250-4_20

Premium Partner

    Image Credits