On Efficient Processing of Linked Stream Data

Today, many application areas require continuous processing of data streams in an efficient manner and real-time fashion. Processing these continuous flows of data, integrating dynamic data with other data sources, and providing the required semantics lead to real challenges. Thus, Linked Stream Data (LSD) has been proposed which combines two concepts: Linked Open Data and Data Stream Processing (DSP). Recently, several LSD engines have been developed, including C-SPARQL and CQELS, which are based on SPARQL extensions for continuous query processing. However, this SPARQL-centric view makes it difficult to express complex processing pipelines. In this paper, we propose a LSD engine based on a more general stream processing approach. Instead of a variant of SPARQL, our engine provides a dataflow specification language called

PipeFlow

which is compiled into native code.

PipeFlow

supports native stream processing operators (e.g., window, aggregates, and joins), complex event processing as well as RDF data transformation operators such as tuplifier and triplifier to efficiently support LSD queries and provide a higher degree of expressiveness. We discuss the main concepts addressing the challenges of LSD processing and describe the usage of these concepts for processing queries from LSBench and SRBench. We show the effectiveness of our system in terms of query execution times through a comparison with existing systems as well as through a detailed performance analysis of our system implementation.