Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top

2013 | Book

High-Performance Network Traffic Processing Systems Using Commodity Hardware Read chapter Read first chapter

Data Traffic Monitoring and Analysis

From Measurement, Classification, and Anomaly Detection to Quality of Experience

Editors: Ernst Biersack, Christian Callegari, Maja Matijasevic

Publisher: Springer Berlin Heidelberg

Book Series : Lecture Notes in Computer Science

Part of: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Login to get access
insite
SEARCH

About this book

This book was prepared as the Final Publication of COST Action IC0703 "Data Traffic Monitoring and Analysis: theory, techniques, tools and applications for the future networks". It contains 14 chapters which demonstrate the results, quality,and the impact of European research in the field of TMA in line with the scientific objective of the Action. The book is structured into three parts: network and topology measurement and modelling, traffic classification and anomaly detection, quality of experience.

Table of Contents

Frontmatter

Network Measurement

Frontmatter
High-Performance Network Traffic Processing Systems Using Commodity Hardware
Abstract
The Internet has opened new avenues for information accessing and sharing in a variety of media formats. Such popularity has resulted in an increase of the amount of resources consumed in backbone links, whose capacities have witnessed numerous upgrades to cope with the ever-increasing demand for bandwidth. Consequently, network traffic processing at today’s data transmission rates is a very demanding task, which has been traditionally accomplished by means of specialized hardware tailored to specific tasks. However, such approaches lack either of flexibility or extensibility—or both. As an alternative, the research community has pointed to the utilization of commodity hardware, which may provide flexible and extensible cost-aware solutions, ergo entailing large reductions of the operational and capital expenditure investments. In this chapter, we provide a survey-like introduction to high-performance network traffic processing using commodity hardware. We present the required background to understand the different solutions proposed in the literature to achieve high-speed lossless packet capture, which are reviewed and compared.
José Luis García-Dorado, Felipe Mata, Javier Ramos, Pedro M. Santiago del Río, Victor Moreno, Javier Aracil
Active Techniques for Available Bandwidth Estimation: Comparison and Application
Abstract
There are various parameters for analyzing the quality of network communication links and paths, one attracting particular attention is available bandwidth. In this chapter we describe a platform for the available bandwidth estimation, a comparison of different tools for the estimation of this parameter, and an application of such estimation in a real-world application. In details, we describe a novel platform called UANM, capable of properly choosing, configuring, and using different available bandwidth tools and techniques in an autonomic fashion. Moreover, thanks to UANM, we show the results of a comparison of the performance of several tools in terms of accuracy, probing time and intrusiveness. Finally, we show a practical example of the use of the available bandwidth measurement: we describe an approach for server selection and admission control in a content distribution network based on the available bandwidth estimation.
Alessio Botta, Alan Davy, Brian Meskill, Giuseppe Aceto
Internet Topology Discovery
Abstract
Since the nineties, the Internet has seen an impressive growth, in terms of users, intermediate systems (such as routers), autonomous systems, or applications. In parallel to this growth, the research community has been looking for obtaining and modeling the Internet topology, i.e., how the various elements of the network interconnect between themselves. An impressive amount of work has been done regarding how to collect data and how to analyse and model it.
This chapter reviews main approaches for gathering Internet topology data. We first focus on hop limited probing, i.e., traceroute-like probing. We review large-scale tracerouting projects and discuss traceroute limitations and how they are mitigated by new techniques or extensions. Hop limited probing can reveal an IP interface vision of the Internet. We next focus on techniques for aggregating several IP interfaces of a given router into a single identifier. This leads to a router level vision of the topology. The aggregation can be done through a process called alias resolution. We also review a technique based on IGMP probing that silently collect all multicast interfaces of a router into a single probe. We next refine the router level topology by adding subnet information. We finish this chapter by discussing the AS level topology, in particular the relationships between ASes and the induced hierarchy.
Benoit Donnet
Internet PoP Level Maps
Abstract
Inferring the Internet Point of Presence (PoP) level maps is gaining interest due to its importance to many areas, e.g., for tracking and studying properties of the Internet. In this chapter we survey research towards the generation of PoP level maps. The chapter introduces different approaches to automatically classify IP addresses to PoPs and discusses their strengths and weaknesses. Special attention is devoted to the challenge of validating the generated PoP maps in the absence of ground truth. The chapter next describes general IP geolocation techniques, points out weaknesses in geolocation databases, as well as, in constraint-based approaches, and concentrates on PoPs geolocation techniques, discussing validation and lack of ground truth availability. The third part of the chapter describes how to generate maps with PoP-to-PoP connectivity and analyzes some of their properties. At the end of the chapter, some applications of PoP level maps, such as Internet distance maps, evolution models and homeland security are introduced and discussed.
Yuval Shavitt, Noa Zilberman
Analysis of Packet Transmission Processes in Peer-to-Peer Networks by Statistical Inference Methods
Abstract
Applying advanced statistical techniques, we characterize the peculiarities of a locally observed peer population in a popular P2P overlay network. The latter is derived from a mesh-pull architecture. Using flow data collected at a single peer, we show how Pareto and Generalized Pareto models can be applied to classify the local behavior of the population feeding a peer. Our approach is illustrated both by file sharing data of a P2P session generated by a mobile BitTorrent client in a WiMAX testbed and by video data streamed to a stationary client in a SopCast session. These techniques can help us to cope with an efficient adaptation of P2P dissemination protocols to mobile environments.
Natalia M. Markovich, Udo R. Krieger

Traffic Classification and Anomaly Detection

Frontmatter
Reviewing Traffic Classification
Abstract
Traffic classification has received increasing attention in the last years. It aims at offering the ability to automatically recognize the application that has generated a given stream of packets from the direct and passive observation of the individual packets, or stream of packets, flowing in the network. This ability is instrumental to a number of activities that are of extreme interest to carriers, Internet service providers and network administrators in general. Indeed, traffic classification is the basic block that is required to enable any traffic management operations, from differentiating traffic pricing and treatment (e.g., policing, shaping, etc.), to security operations (e.g., firewalling, filtering, anomaly detection, etc.).
Up to few years ago, almost any Internet application was using well-known transport layer protocol ports that easily allowed its identification. More recently, the number of applications using random or non-standard ports has dramatically increased (e.g. Skype, BitTorrent, VPNs, etc.). Moreover, often network applications are configured to use well-known protocol ports assigned to other applications (e.g. TCP port 80 originally reserved for Web traffic) attempting to disguise their presence.
For these reasons, and for the importance of correctly classifying traffic flows, novel approaches based respectively on packet inspection, statistical and machine learning techniques, and behavioral methods have been investigated and are becoming standard practice. In this chapter, we discuss the main trend in the field of traffic classification and we describe some of the main proposals of the research community.
We complete this chapter by developing two examples of behavioral classifiers: both use supervised machine learning algorithms for classifications, but each is based on different features to describe the traffic. After presenting them, we compare their performance using a large dataset, showing the benefits and drawback of each approach.
Silvio Valenti, Dario Rossi, Alberto Dainotti, Antonio Pescapè, Alessandro Finamore, Marco Mellia
A Methodological Overview on Anomaly Detection
Abstract
In this Chapter we give an overview of statistical methods for anomaly detection (AD), thereby targeting an audience of practitioners with general knowledge of statistics. We focus on the applicability of the methods by stating and comparing the conditions in which they can be applied and by discussing the parameters that need to be set.
Christian Callegari, Angelo Coluccia, Alessandro D’Alconzo, Wendy Ellens, Stefano Giordano, Michel Mandjes, Michele Pagano, Teresa Pepe, Fabio Ricciato, Piotr Z̊uraniewski
Changepoint Detection Techniques for VoIP Traffic
Abstract
The control of communication networks critically relies on procedures capable of detecting unanticipated load changes. In this chapter we present an overview of such techniques, in a setting in which each connection consumes roughly the same amount of bandwidth (with VoIP as a leading example). For the situation of exponential holding times an explicit analysis can be performed in a large-deviations regime, leading to approximations of the test statistic of interest (and, in addition, to results for the transient of the M/M/∞ queue, which are of independent interest). This procedure being applicable to exponential holding times only, and also being numerically rather involved, we then develop an approximate procedure for general holding times. In this procedure we record the number of trunks occupied at equidistant points in time Δ,2Δ,…, where Δ is chosen sufficiently large to safely assume that the samples are independent; this procedure is backed by results on the transient of the M/G/∞ queue. The validity of the testing procedures is demonstrated through set of numerical experiments; it is also pointed out how diurnal patterns can be dealt with. An experiment with real data illustrates the proposed techniques.
Michel Mandjes, Piotr Żuraniewski
Distribution-Based Anomaly Detection in Network Traffic
Abstract
In this Chapter we address the problem of detecting “anomalies” in the global network traffic produced by a large population of end-users. Empirical distributions across users are considered for several traffic variables at different timescales, and the goal is to identify statistically-significant deviations from the past behavior. This problem is casted into the framework of hypothesis testing. We first address the methodology for dynamically identifying a reference for the null hypothesis (“normal” traffic) that takes into account the typical non-stationarity of real traffic in volume and composition. Then, we illustrate two general distribution-based detection approaches based on both heuristic and formal methods. We discuss also operational criteria for dynamically tuning the detector, so as to track the physiological variation of traffic profiles and number of active users. The Chapter includes a final evaluation based on the analysis of a dataset from an operational 3G network, so as to show in practice the detection of real-world traffic anomalies.
Angelo Coluccia, Alessandro D’Alconzo, Fabio Ricciato

Quality of Experience

Frontmatter
From Packets to People: Quality of Experience as a New Measurement Challenge
Abstract
Over the course of the last decade, the concept of Quality of Experience (QoE) has gained strong momentum, both from an academic research and an industry perspective. Being linked very closely to the subjective perception of the end user, QoE is supposed to enable a broader, more holistic understanding of the qualitative performance of networked communication systems and thus to complement the traditional, more technology-centric Quality of Service (QoS) perspective.
The purpose of this chapter is twofold: firstly, it introduces the reader to QoE by discussing the origins and the evolution of the concept. Secondly, it provides an overview of the current state of the art of QoE research, with focus on work that particularly addresses QoE as a measurement challenge on the technology as well as on the end-user level. This is achieved by surveying the different streams of QoE research that have emerged in the context of Video, Voice and Web services with respect to the following aspects: fundamental relationships and perceptual principles, QoE assessment, modeling and monitoring.
Raimund Schatz, Tobias Hoßfeld, Lucjan Janowski, Sebastian Egger
Internet Video Delivery in YouTube: From Traffic Measurements to Quality of Experience
Abstract
This chapter investigates HTTP video streaming over the Internet for the YouTube platform. YouTube is used as concrete example and case study for video delivery over the Internet, since it is not only the most popular online video platform, but also generates a large share of traffic on today’s Internet. We will describe the YouTube infrastructure as well as the underlying mechanisms for optimizing content delivery. Such mechanisms include server selection via DNS as well as application-layer traffic management. Furthermore, the impact of delivery via the Internet on the user experienced quality (QoE) of YouTube video streaming is quantified. In this context, different QoE monitoring approaches are qualitatively compared and evaluated in terms of the accuracy of QoE estimation.
Tobias Hoßfeld, Raimund Schatz, Ernst Biersack, Louis Plissonneau
Quality Evaluation in Peer-to-Peer IPTV Services
Abstract
Modern IPTV services are comprised of multiple comprehensive service elements in the entire content delivery chain to maximise the efficiency in networking. Audio-visual content may experience various types of impairments during content ingest, processing, distribution and reception. While some impairments do not cause noticeable distortions to the delivered content, many others such as the network transmission loss can be highly detrimental to the user experience in content consumption. In order to optimise service quality and to provide a benchmarking platform to evaluate the designs for future audio-visual content distribution system, a quality evaluation framework is essential. We introduce such an evaluation framework to assess video service with respect of user perception, while supporting service diagnosis to identify root-causes of any detected quality degradation. Compared with existing QoE frameworks, our solution offers an advanced but practical design for the real-time analysis of IPTV services in multiple service layers.
Mu Mu, William Knowles, Panagiotis Georgopoulos, Steven Simpson, Eduardo Cerqueira, Nicholas Race, Andreas Mauthe, David Hutchison
Cross-Layer FEC-Based Mechanism for Packet Loss Resilient Video Transmission
Abstract
Real-time video transmission over wireless networks is now a part of the daily life of users, since it is the vehicle that delivers a wide range of information. The challenge of dealing with the fluctuating bandwidth, scarce resources and time-varying error levels of these networks, reveals the need for packet-loss resilient video transport. Given these conditions, Forward Error Correction (FEC) approaches are desired to ensure the delivery of video services for wireless users with Quality of Experience (QoE) assurance. This work proposes a Cross-layer Video-Aware FEC-based mechanism with Unequal Error Protection (UEP) scheme for packet loss resilient video transmission in wireless networks, which can increase user satisfaction and improve the use of resources. The advantages and disadvantages of the developed mechanism are highlighted through simulations and assessed by means of both subjective and objective QoE metrics.
Roger Immich, Eduardo Cerqueira, Marilia Curado
Approaches for Utility-Based QoE-Driven Optimization of Network Resource Allocation for Multimedia Services
Abstract
Taking jointly into account end-user QoE and network resource allocation optimization provides new opportunities for network and service providers in improving user perceived service performance. In this chapter, we discuss state-of-the-art with regards to QoE-driven utility-based optimization of network resource allocation, in particular related to multimedia services. We present two general types of approaches: those which are primarily user-centric and those which are primarily network-centric. Finally, we provide a comparison of the analyzed approaches and present open issues for future research.
Lea Skorin-Kapov, Krunoslav Ivesic, Giorgos Aristomenopoulos, Symeon Papavassiliou
Backmatter
Metadata
Title
Data Traffic Monitoring and Analysis
Editors
Ernst Biersack
Christian Callegari
Maja Matijasevic
Copyright Year
2013
Publisher
Springer Berlin Heidelberg
Electronic ISBN
978-3-642-36784-7
Print ISBN
978-3-642-36783-0
DOI
https://doi.org/10.1007/978-3-642-36784-7

Premium Partner

    Image Credits