Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
The Journal of Supercomputing
Published in: The Journal of Supercomputing 8/2016

01-08-2016

Exploring large-scale small file storage for search engines

Authors: Weizhe Zhang, Gangzhao Lu, Hui He, Qizhen Zhang, Chuanliang Yu

Published in: The Journal of Supercomputing | Issue 8/2016

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Large-scale small file storage for original pages degrades performance of search engines. In this paper, we first analyze the disadvantages of the existing EXT3 file system in accessing small files. Then, the rate and speed of compression algorithms are verified to choose a proper storage compression algorithm. Meanwhile, we design an original page oriented file organization structure and a read–write query tree to store the large-scale small files which need no modification. The accessing response time and disk space waste are remarkably decreased when search engines use these techniques to store original-page small files.
next article QuaCentive: a quality-aware incentive mechanism in mobile crowdsourced sensing (MCS)

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
2.
3.
Welch TA (1984) A technique for high-performance data compression. Computer 17(6):8–19CrossRef
4.
5.
Tweedie SC (1998) Journaling the Linux ext2fs filesystem. In: Proceedings of the 4th Annual LinuxExpo, Durham, NC
6.
7.
8.
9.
Rosenblum M, Ousterhout JK (1992) The design and implementation of a log-structured file system. ACM Trans Comput Syst (TOCS) 10(1):26–52CrossRef
10.
Zhang WZ, Chen HX, He H, Chen G (2014) A two-tier distributed full-text indexing system. Appl Math 8(1):321–326
11.
Zhang WZ, He H, Ye J (2013) A two-level cache for distributed information retrieval in search engines. Sci World J (2013)
12.
Zhang WZ, He H, Zhang Q (2012) Original-page small file oriented EXT3 file storage system. ASTL 5 (Software Technology)
Metadata
Title
Exploring large-scale small file storage for search engines
Authors
Weizhe Zhang
Gangzhao Lu
Hui He
Qizhen Zhang
Chuanliang Yu
Publication date
01-08-2016
Publisher
Springer US
Published in
The Journal of Supercomputing / Issue 8/2016
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-015-1394-z

Other articles of this Issue 8/2016

The Journal of Supercomputing 8/2016 Go to the issue

OriginalPaper

VMSA: a performance preserving online VM splitting and placement algorithm in dynamic cloud environments

OriginalPaper

Large-scale image colorization based on divide-and-conquer support vector machines

OriginalPaper

Extracting significant pattern histories from timestamped texts using MapReduce

OriginalPaper

An intelligent system for predicting and preventing MERS-CoV infection outbreak

OriginalPaper

Feature selection based on an improved cat swarm optimization algorithm for big data classification

OriginalPaper

A general perspective of Big Data: applications, tools, challenges and trends

Premium Partner

    Image Credits