Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
DHBFT: Dynamic Hierarchical Byzantine Fault-Tolerant Consensus Mechanism Based on Credit Read chapter Read first chapter

2020 | OriginalPaper | Chapter

Pipelined Query Processing Using Non-volatile Memory SSDs

Authors : Xinyu Liu, Yu Pan, Wenxiu Fang, Rebecca J. Stones, Gang Wang, Yusen Li, Xiaoguang Liu

Published in: Web and Big Data

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

NVM Optane SSDs are faster than traditional flash-based SSDs and more economical than DRAM main memory, so we explore query processing with the inverted index on NVM aiming at reducing costs, but this leads to NVM-to-DRAM I/O which negatively affects the search engine’s responsiveness. To alleviate this problem, we propose a pipelining scheme to overlap CPU computation with NVM-to-DRAM I/O. We further propose some optimizations: variable coalesced block size, data prefetching, and block skipping.
The experiments on the Gov2 and ClueWeb document corpuses indicate a reduction in CPU waiting time caused by NVM-to-DRAM I/O by around 85% for Maxscore, Wand, and BlockMaxWand queries vs. not using pipelining, while maintaining comparable query throughput (loss within 6%) vs. an in-memory inverted index (DRAM-based scheme). For RankAnd queries, we occupy 3% of the inverted index in memory for caching to achieve similar query efficiency (within 6%) vs. the DRAM-based scheme.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Quantitative Contention Generation for Performance Evaluation on OLTP Databases
next chapter Sorting-Based Interactive Regret Minimization
Literature
1.
Anh, V.N., Moffat, A.: Index compression using 64-bit words. Softw. Pract. Exp. 40(2), 131–147 (2010)
2.
Ao, N., et al.: Efficient parallel lists intersection and index compression algorithms using graphics processing units. PVLDB 4(8), 470–481 (2011)
3.
Broder, A.Z., Carmel, D., Herscovici, M., Soffer, A., Zien, J.Y.: Efficient query evaluation using a two-level retrieval process. In: Proceedings of CIKM. pp. 426–434 (2003)
4.
Büttcher, S., Clarke, C.L.A., Cormack, G.V.: Information Retrieval-Implementing and Evaluating Search Engines. MIT Press, Cambridge (2010)MATH
5.
Cambazoglu, B.B., Baeza-Yates, R.A.: Scalability challenges in web search engines. Synth. Lect. Inf. Concepts Retr. Serv. 7, 1–138 (2015)
6.
Cambazoglu, B.B., Kayaaslan, E., Jonassen, S., Aykanat, C.: A term-based inverted index partitioning model for efficient distributed query processing. TWEB 7(3), 15:1–15:23 (2013)
7.
Cutting, D.R., Pedersen, J.O.: Optimizations for dynamic inverted index maintenance. In: Proceedings of SIGIR, pp. 405–411 (1990)
8.
Ding, S., Suel, T.: Faster top-k document retrieval using block-max indexes. In: Proceedings of SIGIR, pp. 993–1002 (2011)
9.
Eisenman, A., et al.: Reducing DRAM footprint with NVM in Facebook. In: Proceedings of EuroSys, pp. 42:1–42:13 (2018)
10.
11.
Lemire, D., Boytsov, L.: Decoding billions of integers per second through vectorization. Softw. Pract. Exp. 45(1), 1–29 (2015)CrossRef
12.
Liu, X., Peng, Z.: An efficient random access inverted index for information retrieval. In: Proceedings of WWW, pp. 1153–1154 (2010)
13.
Moffat, A., Stuiver, L.: Binary interpolative coding for effective index compression. Inf. Retr. 3(1), 25–47 (2000)
14.
Ottaviano, G., Tonellotto, N., Venturini, R.: Optimal space-time tradeoffs for inverted indexes. In: Proceedings of WSDM, pp. 47–56 (2015)
15.
Risvik, K.M., Chilimbi, T.M., Tan, H., Kalyanaraman, K., Anderson, C.: Maguro, a system for indexing and searching over very large text collections. In: Proceedings of WSDM, pp. 727–736 (2013)
16.
Robertson, S.E., Jones, K.S.: Relevance weighting of search terms. JASIS 27(3), 129–146 (1976)
17.
Stepanov, A.A., Gangolli, A.R., Rose, D.E., Ernst, R.J., Oberoi, P.S.: SIMD-based decoding of posting lists. In: Proceedings of CIKM, pp. 317–326 (2011)
18.
Turtle, H.R., Flood, J.: Query evaluation: strategies and optimizations. Inf. Process. Manag. 31(6), 831–850 (1995)
19.
Wang, J., Lo, E., Yiu, M.L., Tong, J., Wang, G., Liu, X.: The impact of solid state drive on search engine cache management. In: Proceedings of SIGIR, pp. 693–702 (2013)
20.
Xia, F., Jiang, D., Xiong, J., Sun, N.: HIKV: a hybrid index key-value store for DRAM-NVM memory systems. In: Proceedings of USENIX, pp. 349–362 (2017)
21.
Xu, J., Swanson, S.: NOVA: a log-structured file system for hybrid volatile/non-volatile main memories. In: Proceedings of FAST, pp. 323–338 (2016)
22.
Yan, H., Ding, S., Suel, T.: Inverted index compression and query processing with optimized document ordering. In: Proceedings of WWW, pp. 401–410 (2009)
23.
Zhang, J., Long, X., Suel, T.: Performance of compressed inverted list caching in search engines. In: Proceedings of WWW, pp. 387–396 (2008)
24.
Zhang, R., Sun, P., Tong, J., Stones, R.J., Wang, G., Liu, X.: Compact snippet caching for flash-based search engines. In: Proceedings of SIGIR, pp. 1015–1018 (2015)
25.
Zukowski, M., Héman, S., Nes, N., Boncz, P.A.: Super-scalar RAM-CPU cache compression. In: Proceedings of ICDE (2006)
Metadata
Title
Pipelined Query Processing Using Non-volatile Memory SSDs
Authors
Xinyu Liu
Yu Pan
Wenxiu Fang
Rebecca J. Stones
Gang Wang
Yusen Li
Xiaoguang Liu
Copyright Year
2020
Book
Web and Big Data

Print ISBN: 978-3-030-60289-5

Electronic ISBN: 978-3-030-60290-1

Copyright Year: 2020

DOI
https://doi.org/10.1007/978-3-030-60290-1_35