nach oben

International Journal of Machine Learning and Cybernetics

Erschienen in:

05.07.2023 | Original Article

UniSAr: a unified structure-aware autoregressive language model for text-to-SQL semantic parsing

verfasst von: Longxu Dou, Yan Gao, Mingyang Pan, Dingzirui Wang, Wanxiang Che, Jian-Guang Lou, Dechen Zhan

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 12/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Existing text-to-SQL semantic parsers are typically designed for particular settings such as handling queries that span multiple tables, domains, or turns which makes them ineffective when applied to different settings. We present UniSAr (Unified Structure-Aware Autoregressive Language Model), which benefits from directly using an off-the-shelf language model architecture and demonstrates consistently high performance under different settings. Specifically, UniSAr extends existing autoregressive language models to incorporate two non-invasive extensions to make them structure-aware: (1) adding structure mark to encode database schema, conversation context, and their relationships; (2) constrained decoding to decode well-structured SQL for a given database schema. On seven well-known text-to-SQL datasets covering multi-domain, multi-table, and multi-turn, UniSAr demonstrates highly comparable or better performance to the most advanced specifically-designed text-to-SQL models.

Vorheriger Artikel Feature selection using symmetric uncertainty and hybrid optimization for high-dimensional data

Nächster Artikel Robust stability of complex-valued fractional-order neural networks with uncertain parameters based on new integral inequalities

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

ATZelectronics worldwide

ATZlectronics worldwide is up-to-speed on new trends and developments in automotive electronics on a scientific level with a high depth of information.

Order your 30-days-trial for free and without any commitment.

Jetzt informieren

ATZelektronik

Die Fachzeitschrift ATZelektronik bietet für Entwickler und Entscheider in der Automobil- und Zulieferindustrie qualitativ hochwertige und fundierte Informationen aus dem gesamten Spektrum der Pkw- und Nutzfahrzeug-Elektronik.

Lassen Sie sich jetzt unverbindlich 2 kostenlose Ausgabe zusenden.

Jetzt informieren

https://lucene.apache.org/.

https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html.

Zhong V, Xiong C, Socher R (2017) Seq2sql: generating structured queries from natural language using reinforcement learning. arXiv:1709.00103

Yu T, Zhang R, Yang K, Yasunaga M, Wang D, Li Z, Ma J, Li I, Yao Q, Roman S, Zhang Z, Radev D (2018) Spider: a large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. EMNLP2018. https://aclanthology.org/D18-1425

Guo J, Zhan Z, Gao Y, Xiao Y, Lou J-G, Liu T, Zhang D (2019) Towards complex text-to-SQL in cross-domain database with intermediate representation. Association for Computational Linguistics, Florence, Italy. https://aclanthology.org/P19-1444. Accessed 2 Aug 2019

Yu T, Zhang R, Yasunaga M, Tan YC, Lin XV, Li S, Er H, Li I, Pang B, Chen T, Ji E, Dixit S, Proctor D, Shim S, Kraft J, Zhang V, Xiong C, Socher R, Radev D (2019) SParC: cross-domain semantic parsing in context. Association for Computational Linguistics, Florence, Italy. https://aclanthology.org/P19-1443. Accessed 2 Aug 2019

Sun N, Yang X, Liu Y (2020) TableQA: a large-scale Chinese text-to-SQL dataset for table-aware SQL generation. arXiv:2006.06434

Wang L, Zhang A, Wu K, Sun K, Li Z, Wu H, Zhang M, Wang H (2020) DuSQL: a large-scale and pragmatic Chinese text-to-SQL dataset. Association for Computational Linguistics, Online. https://aclanthology.org/2020.emnlp-main.562. Accessed 20 Nov 2020

Yu T, Zhang R, Er H, Li S, Xue E, Pang B, Lin XV, Tan YC, Shi T, Li Z, Jiang Y, Yasunaga M, Shim S, Chen T, Fabbri A, Li Z, Chen L, Zhang Y, Dixit S, Zhang V, Xiong C, Socher R, Lasecki W, Radev D (2019) CoSQL: a conversational text-to-SQL challenge towards cross-domain natural language interfaces to databases. Association for Computational Linguistics, Hong Kong, China. https://aclanthology.org/D19-1204. Accessed 7 Nov 2019

Guo J, Si Z, Wang Y, Liu Q, Fan M, Lou J-G, Yang Z, Liu T (2021) Chase: a large-scale and pragmatic Chinese dataset for cross-database context-dependent text-to-SQL. Association for Computational Linguistics, Online. https://doi.org/10.18653/v1/2021.acl-long.180. https://aclanthology.org/2021.acl-long.180

Yin P, Neubig G (2018) TRANX: a transition-based neural abstract syntax parser for semantic parsing and code generation. Association for Computational Linguistics, Brussels, Belgium. https://aclanthology.org/D18-2002. Accessed 4 Nov 2018

10.

Wang B, Shin R, Liu X, Polozov O, Richardson M (2020) RAT-SQL: relation-aware schema encoding and linking for text-to-SQL parsers. Association for Computational Linguistics, Online. https://doi.org/10.18653/v1/2020.acl-main.677. https://aclanthology.org/2020.acl-main.677

11.

Bogin B, Gardner M, Berant J (2019) Global reasoning over database structures for text-to-SQL parsing. Association for Computational Linguistics, Hong Kong, China. https://aclanthology.org/D19-1378. Accessed 7 Nov 2019

12.

Cai Y, Wan X (2020) IGSQL: database schema interaction graph based neural model for context-dependent text-to-SQL generation. Association for Computational Linguistics, Online. https://doi.org/10.18653/v1/2020.emnlp-main.560. https://aclanthology.org/2020.emnlp-main.560

13.

Zhang R, Yu T, Er H, Shim S, Xue E, Lin XV, Shi T, Xiong C, Socher R, Radev D (2019) Editing-based SQL query generation for cross-domain context-dependent questions. Association for Computational Linguistics, Hong Kong, China. https://aclanthology.org/D19-1537. Accessed 7 Nov 2019

14.

Hwang W, Yim J, Park S, Seo M (2019) A comprehensive exploration on wikisql with table-aware word contextualization. arXiv:1902.01069

15.

Xu X, Liu C, Song D (2017) Sqlnet: Generating structured queries from natural language without reinforcement learning. arXiv preprint. arXiv:1711.04436

16.

Zhang X, Yin F, Ma G, Ge B, Xiao W (2020) F-sql: fuse table schema and table content for single-table text2sql generation. IEEE Access 8:136409–136420. https://doi.org/10.1109/ACCESS.2020.3011747CrossRef

17.

Shin R, Lin CH, Thomson S, Chen C, Roy S, Platanios EA, Pauls A, Klein D, Eisner J, Durme BV (2021) Constrained language models yield few-shot semantic parsers. CoRR. arXiv:2104.08768

18.

Scholak T, Schucher N, Bahdanau D, Bahdanau D (2021) PICARD: parsing incrementally for constrained auto-regressive decoding from language models. EMNLP2021. https://aclanthology.org/2021.emnlp-main.779

19.

Xie T, Wu CH, Shi P, Zhong R, Scholak T, Yasunaga M, Wu C-S, Zhong M, Yin P, Wang SI, Zhong V, Wang B, Li C, Boyle C, Ni A, Yao Z, Radev D, Xiong C, Kong L, Zhang R, Smith NA, Zettlemoyer L, Yu T (2022) UnifiedSKG: unifying and multi-tasking structured knowledge grounding with text-to-text language models. arXiv preprint. arXiv:2201.05966

20.

Xuan K, Wang Y, Wang Y, Wen Z, Dong Y (2021) Sead: End-to-end text-to-sql generation with schema-aware denoising. CoRR. arXiv:2105.07911

21.

Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, Stoyanov V, Zettlemoyer L (2020) BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. Association for Computational Linguistics, Online. https://aclanthology.org/2020.acl-main.703. Accessed 10 Jul 2020

22.

Liu Y, Wan Y, He L, Peng H, Yu PS (2020) KG-BART: knowledge graph-augmented BART for generative commonsense reasoning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 35, no 7, pp 6418–6425

23.

Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(140):1–67MathSciNet

24.

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Part of Advances in Neural Information Processing Systems 30 (NIPS 2017)

25.

Lei W, Wang W, Ma Z, Gan T, Lu W, Kan M-Y, Chua T-S (2020) Re-examining the role of schema linking in text-to-SQL. Association for Computational Linguistics, Online. https://aclanthology.org/2020.emnlp-main.564. Accessed 20 Nov 2020

26.

Aghajanyan A, Okhonko D, Lewis M, Joshi M, Xu H, Ghosh G, Zettlemoyer L (2022) Htlm: hyper-text pre-training and prompting of language models. ICLR

27.

Chen X, Zhang N, Xie X, Deng S, Yao Y, Tan C, Huang F, Si L, Chen H (2021) Knowprompt: knowledge-aware prompt-tuning with synergistic optimization for relation extraction. arXiv:2104.07650

28.

Liu Q, Yang D, Zhang J, Guo J, Zhou B, Lou J-G (2021) Awakening latent grounding from pretrained language models for semantic parsing. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Association for Computational Linguistics, pp 1174–1189, Online. https://aclanthology.org/2021.findings-acl.100

29.

Sun Y, Tang D, Duan N, Ji J, Cao G, Feng X, Qin B, Liu T, Zhou M (2018) Semantic parsing with syntax- and table-aware SQL generation. Association for Computational Linguistics, Melbourne, Australia. https://aclanthology.org/P18-1034. Accessed 20 Jul 2018

30.

Zheng Y, Wang H, Dong B, Wang X, Li C (2022) HIE-SQL: history information enhanced network for context-dependent text-to-SQL semantic parsing. ACL2022. arXiv:2203.07376

31.

Cao ND, Izacard G, Riedel S, Petroni F (2021) Autoregressive entity retrieval. ICLR2021. https://openreview.net/forum?id=5k8F6UU39V

32.

Cormen TH, Leiserson CE, Rivest RL, Stein C (2001) Introduction to algorithms. The MIT Press, Cambridge

33.

Dong L, Lapata M (2018) Coarse-to-fine decoding for neural semantic parsing. Association for Computational Linguistics, Melbourne, Australia. https://aclanthology.org/P18-1068. Accessed 20 Jul 2018

34.

He P, Mao Y, Chakrabarti K, Chen W (2019) X-SQL: reinforce schema representation with context. arXiv:1908.08113

35.

Lyu Q, Chakrabarti K, Hathi S, Kundu S, Zhang J, Chen Z (2020) Hybrid ranking network for text-to-SQL. arXiv:2008.04759

36.

Lin XV, Socher R, Xiong C (2020) Bridging textual and tabular data for cross-domain text-to-SQL semantic parsing. Association for Computational Linguistics, Online. https://aclanthology.org/2020.findings-emnlp.438. Accessed 20 Nov 2020

37.

Choi D, Shin M, Kim E, Shin DR (2020) RYANSQL: recursively applying sketch-based slot fillings for complex text-to-SQL in cross-domain databases. CoRR. arXiv:2004.03125

38.

Ott M, Edunov S, Baevski A, Fan A, Gross S, Ng N, Grangier D, Auli M (2019) fairseq: A fast, extensible toolkit for sequence modeling. Association for Computational Linguistics, Minneapolis, Minnesota. https://aclanthology.org/N19-4009. Accessed 7 Jun 2019

39.

Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958MathSciNetMATH

40.

Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. CoRR. arXiv:1412.6980

41.

Rasley J, Rajbhandari S, Ruwase O, He Y (2020) DeepSpeed: system optimizations enable training deep learning models with over 100 billion parameters. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3394486.3406703

42.

Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Davison J, Shleifer S, von Platen P, Ma C, Jernite Y, Plu J, Xu C, Le Scao T, Gugger S, Drame M, Lhoest Q, Rush A (2020) Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics, pp 38–45, Online. https://aclanthology.org/2020.emnlp-demos.6

43.

Shazeer N, Stern M (2018) Adafactor: adaptive learning rates with sublinear memory cost. PMLR. https://proceedings.mlr.press/v80/shazeer18a.html

44.

Loshchilov I, Hutter F (2019) Decoupled weight decay regularization. ICLR 2019. https://openreview.net/forum?id=Bkg6RiCqY7

45.

Cao R, Chen L, Chen Z, Zhao Y, Zhu S, Yu K (2021) LGESQL: line graph enhanced text-to-SQL model with mixed local and non-local relations. ACL 2021

Titel: UniSAr: a unified structure-aware autoregressive language model for text-to-SQL semantic parsing
verfasst von: Longxu Dou
Yan Gao
Mingyang Pan
Dingzirui Wang
Wanxiang Che
Jian-Guang Lou
Dechen Zhan
Publikationsdatum: 05.07.2023
Verlag: Springer Berlin Heidelberg
Erschienen in: International Journal of Machine Learning and Cybernetics / Ausgabe 12/2023
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI: https://doi.org/10.1007/s13042-023-01898-3

Neuer Inhalt

Bildnachweise

VDI-Icon, Profil Icon, inhalt2, Springer Professional Modul/© Springer Fachmedien Wiesbaden GmbH, Internationaler Motorenkongress/© [M] ATZlive | Chisnikov / Fotolia.com, Search Icon, Banner Hanser, Customer Experience/© © oatawa / Getty Images / iStock, Erdgasmotor 1.5 TGI evo von Volkswagen/© Volkswagen AG, Thorsten Mücke/© Alexandra Bachran, Zeitschrift Wissensmanagement Cover, PatentFit-Logo/© Springer Fachmedien Wiesbaden GmbH, 2023_Antrieb/© supervisuell, ATZ-Webinar: Prototypenfreie Entwicklung durch Offline- und Driver-in-the-Loop-HiL-Tests /© (c) VI-grade, chassis.tech plus 2023/© [M] ATZlive / TÜV SÜD PRODUCT SERVICE GMBH

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

ATZelectronics worldwide

ATZelektronik

Weitere Artikel der Ausgabe 12/2023

A fuzzy clustering ensemble selection based on active full-link similarity

Feature selection using symmetric uncertainty and hybrid optimization for high-dimensional data

IST-PTEPN: an improved pedestrian trajectory and endpoint prediction network based on spatio-temporal information

Unsupervised concept drift detection method based on robust random cut forest

General fine-grained event detection based on fusion of multi-information representation and attention mechanism

Structured deep learning based object-specific distance estimation from a monocular image

Neuer Inhalt

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.