Skip to main content
Top

2022 | Book

Robust Network Compressive Sensing

share
SHARE
insite
SEARCH

About this book

This book investigates compressive sensing techniques to provide a robust and general framework for network data analytics. The goal is to introduce a compressive sensing framework for missing data interpolation, anomaly detection, data segmentation and activity recognition, and to demonstrate its benefits. Chapter 1 introduces compressive sensing, including its definition, limitation, and how it supports different network analysis applications. Chapter 2 demonstrates the feasibility of compressive sensing in network analytics, the authors we apply it to detect anomalies in the customer care call dataset from a Tier 1 ISP in the United States. A regression-based model is applied to find the relationship between calls and events. The authors illustrate that compressive sensing is effective in identifying important factors and can leverage the low-rank structure and temporal stability to improve the detection accuracy. Chapter 3 discusses that there are several challenges in applying compressive sensing to real-world data. Understanding the reasons behind the challenges is important for designing methods and mitigating their impact. The authors analyze a wide range of real-world traces. The analysis demonstrates that there are different factors that contribute to the violation of the low-rank property in real data. In particular, the authors find that (1) noise, errors, and anomalies, and (2) asynchrony in the time and frequency domains lead to network-induced ambiguity and can easily cause low-rank matrices to become higher-ranked. To address the problem of noise, errors and anomalies in Chap. 4, the authors propose a robust compressive sensing technique. It explicitly accounts for anomalies by decomposing real-world data represented in matrix form into a low-rank matrix, a sparse anomaly matrix, an error term and a small noise matrix. Chapter 5 addresses the problem of lack of synchronization, and the authors propose a data-driven synchronization algorithm. It can eliminate misalignment while taking into account the heterogeneity of real-world data in both time and frequency domains. The data-driven synchronization can be applied to any compressive sensing technique and is general to any real-world data. The authors illustrates that the combination of the two techniques can reduce the ranks of real-world data, improve the effectiveness of compressive sensing and have a wide range of applications.

The networks are constantly generating a wealth of rich and diverse information. This information creates exciting opportunities for network analysis and provides insight into the complex interactions between network entities. However, network analysis often faces the problems of (1) under-constrained, where there is too little data due to feasibility and cost issues in collecting data, or (2) over-constrained, where there is too much data, so the analysis becomes unscalable. Compressive sensing is an effective technique to solve both problems. It utilizes the underlying data structure for analysis. Specifically, to solve the under-constrained problem, compressive sensing technologies can be applied to reconstruct the missing elements or predict the future data. Also, to solve the over-constraint problem, compressive sensing technologies can be applied to identify significant elements

To support compressive sensing in network data analysis, a robust and general framework is needed to support diverse applications. Yet this can be challenging for real-world data where noise, anomalies and lack of synchronization are common. First, the number of unknowns for network analysis can be much larger than the number of measurements. For example, traffic engineering requires knowing the complete traffic matrix between all source and destination pairs, in order to properly configure traffic and avoid congestion. However, measuring the flow between all source and destination pairs is very expensive or even infeasible. Reconstructing data from a small number of measurements is an underconstrained problem. In addition, real-world data is complex and heterogeneous, and often violate the low-level assumptions required by existing compressive sensing techniques. These violations significantly reduce the applicability and effectiveness of existing compressive sensing methods. Third, synchronization of network data reduces the data ranks and increases spatial locality. However, periodic time series exhibit not only misalignment but also different frequencies, which makes it difficult to synchronize data in the time and frequency domains.

The primary audience for this book is data engineers, analysts and researchers, who need to deal with big data with missing anomalous and synchronization problems. Advanced level students focused on compressive sensing techniques will also benefit from this book as a reference.

Table of Contents

Frontmatter
Chapter 1. Introduction
Abstract
Wireless networks and sensor networks are constantly generating an enormous amount of rich and diverse information. Such information creates exciting opportunities for network analytics. Network analytics can provide deep insights into the complex interactions among network entities, and has a wide range of applications in wireless networks across all protocol layers.
Guangtao Xue, Yi-Chao Chen, Feng Lyu, Minglu Li
Chapter 2. Event Detection System
Abstract
In this chapter, we show how compressive sensing technique can be applied to build an event detection system in a major cellular network using customer care call data.
Guangtao Xue, Yi-Chao Chen, Feng Lyu, Minglu Li
Chapter 3. Limitation of Compressive Sensing
Abstract
Real-world data are complicated and heterogeneous, and often violate the low-rank assumption required by existing compressive sensing techniques.
Guangtao Xue, Yi-Chao Chen, Feng Lyu, Minglu Li
Chapter 4. Robust Compressive Sensing
Abstract
In this chapter, we develop LENS decomposition, a novel technique to accurately decompose network data represented in the form of a matrix into a low-rank matrix, a sparse anomaly matrix, an error term, and a small noise matrix.
Guangtao Xue, Yi-Chao Chen, Feng Lyu, Minglu Li
Chapter 5. Data-Driven Synchronization
Abstract
In this chapter, we present the data-driven synchronization algorithm and apply it to improve interpolation, enable segmentation, and perform activity recognition.
Guangtao Xue, Yi-Chao Chen, Feng Lyu, Minglu Li
Chapter 6. Conclusion and Future Research Direction
Abstract
In this monograph we focus on applying compressive sensing for network analytics and building systems that leverage the advantage of compressive sensing, such as anomaly detection, missing value interpolation, and future data prediction.
Guangtao Xue, Yi-Chao Chen, Feng Lyu, Minglu Li
Backmatter
Metadata
Title
Robust Network Compressive Sensing
Authors
Guangtao Xue
Yi-Chao Chen
Feng Lyu
Minglu Li
Copyright Year
2022
Electronic ISBN
978-3-031-16829-1
Print ISBN
978-3-031-16828-4
DOI
https://doi.org/10.1007/978-3-031-16829-1

Premium Partner