Managing and Mining Sensor Data

The increasing advances in hardware technology for sensor processing and mobile technology has resulted in greater access and availability of sensor data from a wide variety of applications. For example, the commodity mobile devices contain a wide variety of sensors such as GPS, accelerometers, and other kinds of data. Many other kinds of technology such as RFID-enabled sensors also produce large volumes of data over time. This has lead to a need for principled methods for efficient sensor data processing. This chapter will provide an overview of the challenges of sensor data analytics and the different areas of research in this context. We will also present the organization of the chapters in this book in this context.

Recently, with the fast development of sensing and wireless communication technology, wireless sensor networks (WSNs) have been applied to monitor the physical world. A WSN consists of a set of sensor nodes, which are small sensing devices with limited computational resources able to communicate with each other located in their radio range. Network protocols ensure the effectiveness of communication between sensor nodes and provide the foundation for WSN applications. The characteristics of WSNs, including the limited energy supply and computational resources, render the design of WSN algorithms challenging and interesting. Both the Database and Network communities have dedicated considerable efforts to make WSNs more effective and efficient. In this chapter, we survey the problems arisen in practical applications of WSNs, focusing on various query processing techniques over captured sensing data.

Sensors including RFID tags have been widely deployed for measuring environmental parameters such as temperature, humidity, oxygen concentration, monitoring the location and velocity of moving objects, tracking tagged objects, and many others. To support effective, efficient, and near real-time phenomena probing and objects monitoring, streaming sensor data have to be gracefully managed in an event processing manner. Different from the traditional events, sensor events come with temporal or spatio-temporal constraints and can be non-spontaneous. Meanwhile, like general event streams, sensor event streams can be generated with very high volumes and rates. Primitive sensor events need to be filtered, aggregated and correlated to generate more semantically rich complex events to facilitate the requirements of up-streaming applications. Motivated by such challenges, many new methods have been proposed in the past to support event processing in sensor event streams. In this chapter, we survey state-of-the-art research on event processing in sensor networks, and provide a broad overview of major topics in.

This chapter surveys fundamental tools for dimensionality reduction and filtering of time series streams, illustrating what it takes to apply them efficiently and effectively to numerous problems. In particular, we show how least-squares based techniques (auto-regression and principal component analysis) can be successfully used to discover correlations both across streams, as well as across time. We also broadly overview work in the area of pattern discovery on time series streams, with applications in pattern discovery, dimensionality reduction, compression.

In recent years, advances in hardware technology have facilitated new ways of collecting data continuously. One such application is that of sensor data, which may continuously monitor large amounts of data for storage and processing. In this paper, we will discuss the general issues which arise in mining large amounts of sensor data. In many cases, the data patterns may evolve continuously, as a result of which it is necessary to design the mining algorithms effectively in order to account for changes in underlying structure of the data stream. This makes the solutions of the underlying problems even more difficult from an algorithmic and computational point of view. In this chapter we will provide an overview of the problem of data stream mining and the unique challenges that data stream mining poses to different kinds of sensor applications.

The proliferation of Wireless Sensor Networks (WSNs) in the past decade has provided the bridge between the physical and digital worlds, enabling the monitoring and study of physical phenomena at a granularity and level of detail that was never before possible. In this study, we review the efforts of the research community with respect to two important problems in the context of WSNs: real-time collection of the sensed data, and real-time processing of these data series.

Wireless sensor networks (WSNs) consist of a collection of low cost and low powered sensor devices capable of communicating with each other via an ad-hoc wireless network. Due to their rapid proliferation, sensor networks are currently used in a plethora of applications such as earth sciences, systems health, military applications etc. These sensors collect the data about the environment and this data can be mined for a variety of analysis. Unfortunately, post analysis of the data extracted from the WSN incurs high sensor communication cost for sending the raw data to the base station and at the same time runs the risk of delayed analysis. To overcome this, researchers have proposed several distributed algorithms which can deal with the data in situ – these data mining algorithms utilize the computing power at each node to first do some local computations and then exchange messages with its neighbors to come to a consensus regarding a global model. These algorithms reduce the communication cost vastly and also are extremely efficient in terms of model computation and event detection. In this chapter we focus on such distributed data mining algorithms for data clustering, classification and outlier detection tasks.

A number of sensor applications in recent years collect data which can be directly associated with human interactions. Some examples of such applications include GPS applications on mobile devices, accelerometers, or location sensors designed to track human and vehicular traffic. Such data lends itself to a variety of rich applications in which one can use the sensor data in order to model the underlying relationships and interactions. This requires the development of trajectory mining techniques, which can mine the GPS data for interesting social patterns. It also leads to a number of challenges, since such data may often be private, and it is important to be able to perform the mining process without violating the privacy of the users. Given the open nature of the information contributed by users in social sensing applications, this also leads to issues of trust in making inferences from the underlying data. In this chapter, we provide a broad survey of the work in this important and rapidly emerging field. We also discuss the key problems which arise in the context of this important field and the corresponding solutions.

Recent advances in affordable positioning hardware and software have made the availability of location data ubiquitous. Personal devices such as tablet PCs, smart phones and even sport watches are all able to collect and store a user’s location over time, providing an ever-growing supply of spatiotemporal data. Managing this plethora of data is a relatively new challenge and there has been a great deal of research in the recent years devoted to the problems that arise from spatiotemporal data. This book chapter surveys recent developments in the techniques used for the management and mining of spatiotemporal data. We focus our survey on three main areas: (i) data management, which includes indexing and querying mobile objects, (ii) tracking, making use of noisy location observations to infer an object’s actual or future position, and (iii) mining, extracting interesting patterns from spatiotemporal data. First, we cover recent advances in database systems for managing spatiotemporal data, including index structures and efficient algorithms for processing queries. Next, we review the problem of tracking for mobile objects to estimate an object’s location given a sequence of noisy observations. We discuss some of the common approaches used for tracking and examine some recent work which focuses specifically on tracking vehicles using a road network. Then we review the recent literature on mining spatiotemporal data. We conclude by discussing some interesting areas of future research.

Radio Frequency Identification (RFID) is a new technology which allows a sensor (reader) to read, from a distance, and without line of sight, a unique product identification code (EPC) associated with a tag. Such tags are very useful in inventory management and logistics, because they can be used in order to track the movement and locations of large volumes of items in a cost effective way. This leads to massive streams of noisy data, which can be used in the context of a variety of data management and event processing algorithms. The use of RFID also has a number of privacy challenges associated with it, because a tag on an item being carried by a person, also becomes a unique location tag for that person. Therefore, methods need to to be designed to increase the privacy and security of RFID technology. This chapter will provide a broad overview and survey of a variety of RFID data management, mining and processing techniques. We will also discuss the privacy and security issues associated with the use of RFID technology.

Advances in sensor data collection technology, such as pervasive and embedded devices, and RFID Technology have lead to a large number of smart devices which are connected to the net and continuously transmit their data over time. It has been estimated that the number of internet connected devices has overtaken the number of humans on the planet, since 2008. The collection and processing of such data leads to unprecedented challenges in mining and processing such data. Such data needs to be processed in real-time and the processing may be highly distributed in nature. Even in cases, where the data is stored offline, the size of the data is often so large and distributed, that it requires the use of big data analytical tools for processing. In addition, such data is often sensitive, and brings a number of privacy challenges associated

Historically, healthcare has been mainly provided in a reactive manner that limits its usefulness. With progress in sensor technologies, the instrumentation of the world has offered unique opportunities to better observe patients physiological signals in order to provide healthcare in a more proactive manner. To reach this goal, it is essential to be able to analyze patient data and turn it into actionable information using data mining. This chapter surveys existing applications of sensor data mining technologies in healthcare. It starts with a description of healthcare data mining challenges before presenting an overview of applications of data mining in both clinical and non clinical settings.

Advances in earth observation technologies have led to the acquisition of vast volumes of accurate, timely and reliable environmental data which encompass a multitude of information about the land, ocean and atmosphere of the planet. Earth science sensor datasets capture multiple facets of information about natural processes and human activities that shape the physical landscape and environmental quality of our planet, and thus, offer an opportunity to monitor and understand the diverse phenomena affecting earth’s complex system. The monitoring, analysis and understanding of these rich sensor datasets is thus of prime importance for the efficient planning and management of critical resources, since the societal costs of mitigation or adaptation decisions for natural or human-induced adverse events are significant. Hence, a thorough understanding of earth science sensor datasets has a direct impact on a range of societally relevant issues. Moreover, earth science sensor datasets possess unique domain-specific properties that distinguish them from sensor datasets used in other domains, and thus demand the need for novel tools and techniques to be developed for their analysis, adhering to their characteristic issues and challenges.