Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben

2017 | Buch

Introduction Kapitel lesen Erstes Kapitel lesen

Traffic Measurement for Big Network Data

verfasst von: Shigang Chen, Min Chen, Qingjun Xiao

Verlag: Springer International Publishing

Buchreihe : Wireless Networks

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten
insite
SUCHEN

Über dieses Buch

This book presents several compact and fast methods for online traffic measurement of big network data. It describes challenges of online traffic measurement, discusses the state of the field, and provides an overview of the potential solutions to major problems.
The authors introduce the problem of per-flow size measurement for big network data and present a fast and scalable counter architecture, called Counter Tree, which leverages a two-dimensional counter sharing scheme to achieve far better memory efficiency and significantly extend estimation range.
Unlike traditional approaches to cardinality estimation problems that allocate a separated data structure (called estimator) for each flow, this book takes a different design path by viewing all the flows together as a whole: each flow is allocated with a virtual estimator, and these virtual estimators share a common memory space. A framework of virtual estimators is designed to apply the idea of sharing to an array of cardinality estimation solutions, achieving far better memory efficiency than the best existing work.
To conclude, the authors discuss persistent spread estimation in high-speed networks. They offer a compact data structure called multi-virtual bitmap, which can estimate the cardinality of the intersection of an arbitrary number of sets. Using multi-virtual bitmaps, an implementation that can deliver high estimation accuracy under a very tight memory space is presented.
The results of these experiments will surprise both professionals in the field and advanced-level students interested in the topic. By providing both an overview and the results of specific experiments, this book is useful for those new to online traffic measurement and experts on the topic.

Inhaltsverzeichnis

Frontmatter
Chapter 1. Introduction
Abstract
Traffic measurement has many important applications in capacity planning, accounting and billing, anomaly detection, service provision, etc. In the era of big network data, traffic measurement becomes a daunting task that requires tremendous resources. To keep up with the line speeds of modern routers, the measurement modules should be implemented in the limited on-chip cache memory, thereby minimizing the per-packet processing time. This book aims to develop new compact and fast online measurement methods that reduce big network data to measurement summaries orders-of-magnitude smaller than what the traditional methods can do. The new methods hold the promise of allowing routers to perform measurement on large network traffic in real time using small cache memory on network processors.
Shigang Chen, Min Chen, Qingjun Xiao
Chapter 2. Per-Flow Size Measurement
Abstract
Per-flow size measurement, which is to count the number of packets for each active flow during a certain measurement period, has many applications in usage accounting, traffic engineering, service provision, and anomaly detection. In order to maintain the high throughput of routers or switchers, the per-flow size measurement module should use high-bandwidth SRAM that allows fast memory accesses. Due to the limited SRAM space, exact counting, which requires to keep a counter for each flow, does not scale to measure big network data consisting of numerous flows. Some recent work takes a different design path to accurately estimate the flow sizes using counter architectures that can fit into tight SRAM. However, existing counter architectures have some limitations, either still requiring considerable SRAM space, or having a very small estimation range. This chapter presents a scalable counter architecture, called Counter Tree, which leverages a two-dimensional counter sharing technique to achieve far better memory efficiency and significantly extend estimation range. Furthermore, we improve the performance of Counter Tree by adding a status bit to each counter. The extensive experiments with real network trace demonstrate that the new architecture can produce accurate estimates for flows of all sizes even under a very tight memory space, e.g., 1 bit per flow.
Shigang Chen, Min Chen, Qingjun Xiao
Chapter 3. Per-Flow Cardinality Measurement
Abstract
Per-flow cardinality measurement over big network data consisting of numerous flows is a fundamental problem with many practical applications. Traditionally the research on this problem focused on using a small amount of memory to estimate each flow’s cardinality from a large range (up to 109). However, although the memory needed for each flow has been greatly compressed, when there is an extremely large number of flows, the overall memory demand can still be very high, exceeding the availability under some important scenarios, such as implementing online measurement modules in network processors using only on-chip cache memory. In this chapter, instead of allocating a separated data structure (called estimator) for each flow, we take a different path by viewing all the flows together as a whole: Each flow is allocated with a virtual estimator, and these virtual estimators share a common memory space. We show that sharing at the register (multi-bit) level is superior than sharing at the bit level. We present a framework of virtual estimators that allows us to apply the idea of sharing to an array of cardinality estimation solutions, achieving far better memory efficiency than the best existing work. Experimental results show that the new solution can work in a tight memory space of less than 1 bit per flow or even one tenth of a bit per flow—a quest that has never been realized before.
Shigang Chen, Min Chen, Qingjun Xiao
Chapter 4. Persistent Spread Measurement
Abstract
The persistent spread of a destination host is the number of distinct sources that have contacted it persistently in predefined t measurement periods. A persistent spread estimator is a software/hardware component in a router that inspects the arrival packets and estimates the persistent spread of each destination. This is a new primitive for network measurement that can be used to detect long-term stealthy malicious activities, which cannot be recognized by the traditional superspreader detectors that are designed only for “elephant” activities. This chapter presents an estimator that can use very tight memory space to deliver high estimation accuracy: Its memory expense is less than one bit per-flow element in each time period; Its estimation accuracy is over 90 % better than a continuous variant of Flajolet–Martin sketches; Its operating range to produce effective measurements is hundreds of times broader than the traditional bitmap. These advantages originate from a new data structure called multi-virtual bitmap, which is designed to estimate the cardinality of the intersection of an arbitrary number of sets. The effectiveness of the new estimator is verified using the real network traffic traces from CAIDA.
Shigang Chen, Min Chen, Qingjun Xiao
Metadaten
Titel
Traffic Measurement for Big Network Data
verfasst von
Shigang Chen
Min Chen
Qingjun Xiao
Copyright-Jahr
2017
Electronic ISBN
978-3-319-47340-6
Print ISBN
978-3-319-47339-0
DOI
https://doi.org/10.1007/978-3-319-47340-6

Neuer Inhalt

    Bildnachweise