Top

Published in:

2025 | OriginalPaper | Chapter

DUNKS: Chunking and Summarizing Large and Heterogeneous Data for Dataset Search

Authors : Qiaosheng Chen, Xiao Zhou, Zhiyang Zhang, Gong Cheng

Published in: The Semantic Web – ISWC 2024

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

With the vast influx of open data on the Web, dataset search has become a trending research problem which is crucial to data discovery and reuse. Existing methods for dataset search either employ only the unstructured metadata of datasets but ignore their actual data, or cater to structured data in a single format such as RDF despite the diverse formats of open data. In this paper, to address the magnitude of large datasets, we decompose RDF data into data chunks, and then, to accommodate big chunks to the limited input capacity of dense ranking models based on pre-trained language models, we propose a multi-chunk summarization method that extracts representative data from representative chunks. Moreover, to handle heterogeneous data formats beyond RDF, we transform other formats into chunks to be processed in a uniform way. Experiments on two test collections for dataset search demonstrate the effectiveness of our dense ranking over summarized data chunks.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Compiling SHACL Into SQL

next chapter CRAWD: Sampling-Based Estimation of Count-Distinct SPARQL Queries

https://github.com/nju-websoft/ACORDAR.

https://huggingface.co/BAAI/bge-large-en-v1.5.

https://huggingface.co/BAAI/bge-reranker-large.

https://huggingface.co/castorini/monobert-large-msmarco.

https://open.bigmodel.cn/dev/api#glm-4.

https://github.com/AmenRa/ranx.

https://ntcir.datasearch.jp/data_search_1/.

https://github.com/mpkato/ntcir-datasearch.

https://github.com/castorini/pyserini.

Amsterdamer, Y., Davidson, S.B., Milo, T., Razmadze, K., Somech, A.: Selecting sub-tables for data exploration. In: ICDE 2023, pp. 2496–2509 (2023). https://doi.org/10.1109/ICDE55515.2023.00192

Bassani, E.: ranx: A blazing-fast python library for ranking evaluation and comparison. In: ECIR 2022, pp. 259–264 (2022). https://doi.org/10.1007/978-3-030-99739-7_30

Bassani, E., Romelli, L.: ranx.fuse: A python library for metasearch. In: CIKM 2022, pp. 4808–4812 (2022). https://doi.org/10.1145/3511808.3557207

Benjelloun, O., Chen, S., Noy, N.F.: Google dataset search by the numbers. In: ISWC 2020, pp. 667–682 (2020). https://doi.org/10.1007/978-3-030-62466-8_41

Brickley, D., Burgess, M., Noy, N.F.: Google dataset search: building a search engine for datasets in an open Web ecosystem. In: WWW 2019, pp. 1365–1375 (2019). https://doi.org/10.1145/3308558.3313685

Castelo, S., Rampin, R., Santos, A.S.R., Bessa, A., Chirigati, F., Freire, J.: Auctus: a dataset search engine for data discovery and augmentation. Proc. VLDB Endow. 14(12), 2791–2794 (2021). https://doi.org/10.14778/3476311.3476346

Cebiric, S., et al.: Summarizing semantic graphs: a survey. VLDB J. 28(3), 295–327 (2019). https://doi.org/10.1007/S00778-018-0528-3CrossRef

Chapman, A., Simperl, E., Koesten, L., Konstantinidis, G., Ibáñez, L., Kacprzak, E., Groth, P.: Dataset search: a survey. VLDB J. 29(1), 251–272 (2020). https://doi.org/10.1007/s00778-019-00564-xCrossRef

Chen, J., Wang, X., Cheng, G., Kharlamov, E., Qu, Y.: Towards more usable dataset search: from query characterization to snippet generation. In: CIKM 2019, pp. 2445–2448 (2019). https://doi.org/10.1145/3357384.3358096

10.

Chen, Q., Huang, Z., Zhang, Z., Luo, W., Lin, T., Shi, Q., Cheng, G.: Dense re-ranking with weak supervision for RDF dataset search. In: ISWC 2023, pp. 23–40 (2023). https://doi.org/10.1007/978-3-031-47240-4_2

11.

Chen, Z., Jia, H., Heflin, J., Davison, B.D.: Leveraging schema labels to enhance dataset search. In: ECIR 2020, pp. 267–280 (2020). https://doi.org/10.1007/978-3-030-45439-5_18

12.

Cheng, G., Jin, C., Ding, W., Xu, D., Qu, Y.: Generating illustrative snippets for open data on the Web. In: WSDM 2017, pp. 151–159 (2017). https://doi.org/10.1145/3018661.3018670

13.

Dai, Z., Callan, J.: Deeper text understanding for IR with contextual neural language modeling. In: SIGIR 2019, pp. 985–988 (2019). https://doi.org/10.1145/3331184.3331303

14.

Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL 2019, pp. 4171–4186 (2019). https://doi.org/10.18653/v1/n19-1423

15.

Gunaratna, K., Thirunarayan, K., Sheth, A.P., Cheng, G.: Gleaning types for literals in RDF triples with application to entity summarization. In: ESWC 2016, pp. 85–100 (2016). https://doi.org/10.1007/978-3-319-34129-3_6

16.

Hochbaum, D.S.: Approximating covering and packing problems: set cover, vertex cover, independent set, and related problems, pp. 94–143. PWS Publishing Co., USA (1996)

17.

Hofstätter, S., Mitra, B., Zamani, H., Craswell, N., Hanbury, A.: Intra-document cascading: Learning to select passages for neural document ranking. In: SIGIR 2021, pp. 1349–1358 (2021). https://doi.org/10.1145/3404835.3462889

18.

Hofstätter, S., Zamani, H., Mitra, B., Craswell, N., Hanbury, A.: Local self-attention over long text for efficient document retrieval. In: SIGIR 2020, pp. 2021–2024 (2020). https://doi.org/10.1145/3397271.3401224

19.

Karpukhin, V., Oguz, B., Min, S., Lewis, P.S.H., Wu, L., Edunov, S., Chen, D., Yih, W.: Dense passage retrieval for open-domain question answering. In: EMNLP 2020, pp. 6769–6781 (2020). https://doi.org/10.18653/v1/2020.emnlp-main.550

20.

Kato, M.P., Ohshima, H., Liu, Y., Chen, H.: A test collection for ad-hoc dataset retrieval. In: SIGIR 2021, pp. 2450–2456 (2021). https://doi.org/10.1145/3404835.3463261

21.

Khattab, O., Zaharia, M.: ColBERT: efficient and effective passage search via contextualized late interaction over BERT. In: SIGIR 2020, pp. 39–48 (2020). https://doi.org/10.1145/3397271.3401075

22.

Koesten, L.M., Kacprzak, E., Tennison, J.F.A., Simperl, E.: The trials and tribulations of working with structured data: -a study on information seeking behaviour. In: CHI 2017, pp. 1277–1289 (2017). https://doi.org/10.1145/3025453.3025838

23.

Kroll, H., Nagel, D., Balke, W.T.: Bafrec: balancing frequency and rarity for entity characterization in open linked data. In: EYRE 2018 (2018)

24.

Li, C., Yates, A., MacAvaney, S., He, B., Sun, Y.: PARADE: passage representation aggregation for document reranking. CoRR abs/2008.09093 (2020)

25.

Li, M., Gaussier, É.: KeyBLD: selecting key blocks with local pre-ranking for long document information retrieval. In: SIGIR 2021, pp. 2207–2211 (2021). https://doi.org/10.1145/3404835.3463083

26.

Li, M., Popa, D.N., Chagnon, J., Cinar, Y.G., Gaussier, É.: The power of selecting key blocks with local pre-ranking for long document information retrieval. ACM Trans. Inf. Syst. 41(3), 73:1–73:35 (2023). https://doi.org/10.1145/3568394

27.

Lin, J., Ma, X., Lin, S., Yang, J., Pradeep, R., Nogueira, R.F.: Pyserini: a python toolkit for reproducible information retrieval research with sparse and dense representations. In: SIGIR 2021, pp. 2356–2362 (2021). https://doi.org/10.1145/3404835.3463238

28.

Lin, J., Nogueira, R.F., Yates, A.: Pretrained Transformers for Text Ranking: BERT and Beyond. Morgan & Claypool Publishers (2021). https://doi.org/10.2200/S01123ED1V01Y202108HLT053

29.

Lin, T., et al.: ACORDAR: a test collection for ad hoc content-based (RDF) dataset retrieval. In: SIGIR 2022, pp. 2981–2991 (2022). https://doi.org/10.1145/3477495.3531729

30.

Liu, D., Cheng, G., Liu, Q., Qu, Y.: Fast and practical snippet generation for RDF datasets. ACM Trans. Web 13(4), 19:1–19:38 (2019). https://doi.org/10.1145/3365575

31.

Liu, Q., Cheng, G., Gunaratna, K., Qu, Y.: Entity summarization: state of the art and future challenges. J. Web Semant. 69, 100647 (2021). https://doi.org/10.1016/J.WEBSEM.2021.100647

32.

Luo, W., Chen, Q., Zhang, Z., Huang, Z., Cheng, G.: An empirical investigation of implicit and explicit knowledge-enhanced methods for ad hoc dataset retrieval. In: Findings of EMNLP 2023, pp. 14349–14360 (2023). https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.957

33.

Muennighoff, N., Tazi, N., Magne, L., Reimers, N.: MTEB: massive text embedding benchmark. In: EACL 2023, pp. 2006–2029 (2023). https://doi.org/10.18653/V1/2023.EACL-MAIN.148

34.

Neumaier, S., Umbrich, J., Polleres, A.: Automated quality assessment of metadata across open data portals. ACM J. Data Inf. Qual. 8(1), 2:1–2:29 (2016). https://doi.org/10.1145/2964909

35.

Nguyen, P., et al.: Nii table linker at the ntcir-15 data search task: Re-ranking with pre-trained contextualized embeddings, data content, entity-centric, and cluster-based approaches. In: NTCIR 2020 (2020)

36.

Nguyen, T., et al.: MS MARCO: a human generated machine reading comprehension dataset. In: Workshop on Cognitive Computation (NIPS 2016), vol. 1773 (2016)

37.

Nogueira, R.F., Cho, K.: Passage re-ranking with BERT. CoRR abs/1901.04085 (2019)

38.

Okamoto, T., Miyamori, H.: Ksu systems at the ntcir-15 data search task. In: NTCIR 2020 (2020)

39.

Ota, M., Mueller, H., Freire, J., Srivastava, D.: Data-driven domain discovery for structured datasets. Proc. VLDB Endow. 13(7), 953–965 (2020). https://doi.org/10.14778/3384345.3384346

40.

Pietriga, E., Gözükan, H., Appert, C., Destandau, M., Cebiric, S., Goasdoué, F., Manolescu, I.: Browsing linked data catalogs with lodatlas. In: ISWC 2018, pp. 137–153 (2018). https://doi.org/10.1007/978-3-030-00668-6_9

41.

Quarati, A.: Open government data: Usage trends and metadata quality. J. Inf. Sci., 1–24 (2021). https://doi.org/10.1177/01655515211027775

42.

Silva, L., Barbosa, L.: Improving dense retrieval models with LLM augmented data for dataset search. Knowl. Based Syst. 294, 111740 (2024). https://doi.org/10.1016/j.knosys.2024.111740

43.

Sun, W., et al.: Is ChatGPT good at search? investigating large language models as re-ranking agents. In: EMNLP 2023, pp. 14918–14937 (2023). https://doi.org/10.18653/V1/2023.EMNLP-MAIN.923

44.

Trabelsi, M., Chen, Z., Zhang, S., Davison, B.D., Heflin, J.: Strubert: structure-aware BERT for table search and matching. In: WWW 2022, pp. 442–451 (2022). https://doi.org/10.1145/3485447.3511972

45.

Wang, X., Cheng, G.: A survey on extractive knowledge graph summarization: applications, approaches, evaluation, and future directions. In: IJCAI 2024 (2024)

46.

Wang, X., et al.: PCSG: pattern-coverage snippet generation for RDF datasets. In: ISWC 2021, pp. 3–20 (2021). https://doi.org/10.1007/978-3-030-88361-4_1

47.

Wang, X., Cheng, G., Pan, J.Z., Kharlamov, E., Qu, Y.: BANDAR: benchmarking snippet generation algorithms for (RDF) dataset search. IEEE Trans. Knowl. Data Eng. 35(2), 1227–1241 (2023). https://doi.org/10.1109/TKDE.2021.3095309

48.

Wang, X., Lin, T., Luo, W., Cheng, G., Qu, Y.: CKGSE: a prototype search engine for Chinese knowledge graphs. Data Intell. 4(1), 41–65 (2022). https://doi.org/10.1162/dint_a_00118

49.

Xiao, S., Liu, Z., Zhang, P., Muennighof, N.: C-pack: packaged resources to advance general Chinese embedding (2023). https://doi.org/10.48550/ARXIV.2309.07597

50.

Xiong, L., et al.: Approximate nearest neighbor negative contrastive learning for dense text retrieval. In: ICLR 2021 (2021)

51.

Yang, E., Hao, F., Yang, Y., Maio, C.D., Nasridinov, A., Min, G., Yang, L.T.: Incremental entity summarization with formal concept analysis. IEEE Trans. Serv. Comput. 15(6), 3289–3303 (2022). https://doi.org/10.1109/TSC.2021.3090276

52.

Zeng, A., et al.: ChatGLM: a family of large language models from GLM-130B to GLM-4 all tools. CoRR abs/2406.12793 (2024). https://doi.org/10.48550/ARXIV.2406.12793

53.

Zhao, W.X., Liu, J., Ren, R., Wen, J.: Dense text retrieval based on pretrained language models: a survey. CoRR abs/2211.14876 (2022). https://doi.org/10.48550/ARXIV.2211.14876

54.

Zhiltsov, N., Kotov, A., Nikolaev, F.: Fielded sequential dependence model for ad-hoc entity retrieval in the Web of data. In: SIGIR 2015, pp. 253–262 (2015). https://doi.org/10.1145/2766462.2767756

55.

Zneika, M., Vodislav, D., Kotzinos, D.: Quality metrics for RDF graph summarization. Semantic Web 10(3), 555–584 (2019). https://doi.org/10.3233/SW-190346CrossRef

Title: DUNKS: Chunking and Summarizing Large and Heterogeneous Data for Dataset Search
Authors: Qiaosheng Chen
Xiao Zhou
Zhiyang Zhang
Gong Cheng
Publisher: Springer Nature Switzerland
Book: The Semantic Web – ISWC 2024
Print ISBN: 978-3-031-77849-0

Electronic ISBN: 978-3-031-77850-6

Copyright Year: 2025
DOI: https://doi.org/10.1007/978-3-031-77850-6_5

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"