nach oben

Erschienen in:

2015 | OriginalPaper | Buchkapitel

Real-Time Anomaly Detection from Environmental Data Streams

verfasst von : Sergio Trilles, Sven Schade, Óscar Belmonte, Joaquín Huerta

Erschienen in: AGILE 2015

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Modern sensor networks monitor a wide range of phenomena. They are applied in environmental monitoring, health care, optimization of industrial processes, social media, smart city solutions, and many other domains. All in all, they provide a continuously pulse of the almost infinite activities that are happening in the physical space—and in cyber space. The handling of the massive amounts of generated measurements poses a series of (Big Data) challenges. Our work addresses one of these challenges: the detection of anomalies in real-time. In this paper, we propose a generic solution to this problem, and introduce a system that is capable of detecting anomalies, generating notifications, and displaying the recent situation to the user. We apply CUSUM a statistical control algorithm and adopt it so that it can be used inside the Storm framework—a robust and scalable real-time processing framework. We present a proof of concept implementation from the area of environmental monitoring.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Enabling Semantic Search and Knowledge Discovery for ArcGIS Online: A Linked-Data-Driven Approach

Nächstes Kapitel Towards Real-Time Processing of Massive Spatio-temporally Distributed Sensor Data: A Sequential Strategy Based on Kriging

Storm, Apache Incubator, http://storm.apache.org.

Apache S4, Apache Incubator, http://incubator.apache.org/s4.

Samza, Apache Incubator, http://samza.incubator.apache.org.

Spark Streaming, Apache Incubator, http://spark.apache.org/streaming.

Storm Trident, Apache Incubator, http://storm.apache.org/documentation/Trident-API-Overview.html.

ActiveMQ framework, Apache Software Foundation, http://activemq.apache.org.

OpenWire protocol, Apache Incubator, http://activemq.apache.org/openwire.html.

STOMP protocol, Apache Incubator, http://activemq.apache.org/stomp.html.

http://www.citma.gva.es/web/calidad-ambiental/datos-on-line.

Leaflet: An Open-Source JavaScript Library for Mobile-Friendly Interactive Maps, http://leafletjs.com.

ESRI, http://www.esri.com.

Bootstrap, Twitter http://getbootstrap.com.

jQuery, jQuery Foundation http://jquery.com.

Highcharts JS, Highcharts AS http://www.highcharts.com.

Barratt, B., & Fuller, G. (2014). Intervention assessments in the control of PM10 emissions from an urban waste transfer station. Environmental Science: Processes and Impacts, 16(6), 1328–1337.

Barratt, B., Atkinson, R., Anderson, H., Beevers, S., Kelly, F., Mudway, I., & Wilkinson, P. (2007). Investigation into the use of the cusum technique in identifying changes in mean air pollution levels following introduction of a traffic management scheme. Atmospheric Environment, 41(8), 1784–1791.CrossRef

Buschmann, F., Meunier, R., Rohnert, H., & Sommerlad, P. (1996). Pattern-oriented software architecture: A system of patterns (Vol. 1). New York: Wiley.

Carslaw, D., Ropkins, K., & Bell, M. (2006). Change-point detection of gaseous and particulate traffic-related pollutants at a roadside location. Environmental Science and Technology, 40(22), 6912–6918.CrossRef

Charles, J., & Jeh-Nan, P. (2002). Evaluating environmental performance using statistical process control techniques. European Journal of Operational Research, 139(1), 68–83.CrossRef

Chelani, A. (2011). Change detection using cusum and modified cusum method in air pollutant concentrations at traffic site in Delhi. Stochastic Environmental Research and Risk Assessment, 25(6), 827–834.CrossRef

Chuen-Sheng, C. (1995). A multi-layer neural network model for detecting changes in the process mean. Computers and Industrial Engineering, 28(1), 51–61.CrossRef

Cox, S. (2007). Observations and measurements part 1—observation schema. Technical report, Open Geospatial Consortium (OGC).

De Francisci Morales, G. (2013). Samoa: A platform for mining big data streams. In Proceedings of the 22nd International Conference on World Wide Web Companion, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, WWW ’13 Companion (pp. 777–778).

Fielding, R. (2000). Representational state transfer (rest) (Chap. 5). Fielding Dissertation.

Grigg, O., Farewell, V., & Spiegelhalter, D. (2003). Use of risk-adjusted CUSUM and RSPRTcharts for monitoring in medical contexts. Statistical Methods in Medical Research, 12(2), 147–170.

Guh, R., & Hsieh, Y. (1999). A neural network based model for abnormal pattern recognition of control charts. Computers and Industrial Engineering, 36(1), 97–108.CrossRef

Hapner, M., Burridge, R., Sharma, R., Fialli, J., & Stout, K. (2002). Java message service. Santa Clara, CA: Sun Microsystems Inc.

Hickson, I. (2011). The WebSocket API. W3C Working Draft WD-websockets-20110929, September.

Jeske, D., Montes De Oca, V., Bischoff, W., & Marvasti, M. (2009). Cusum techniques for timeslot sequences with applications to network surveillance. Computational Statistics Data Analysis, 53(12), 4332–4344.CrossRef

Kortuem, G., Kawsar, F., Fitton, D., & Sundramoorthy, V. (2010). Smart objects as building blocks for the internet of things. IEEE Internet Computing, 14(1), 44–51.CrossRef

Lucas, J. (1982). Combined Shewhart-CUSUM quality control schemes. Journal of Quality Technology, 14(2).

Manovich, L. (2012). Trending: The promises and the challenges of big social data. In M. K. Gold (Ed.), Debates in the digital humanities (pp. 460–475). Minneapolis: U of Minnesota P.

Mesnil, B., & Petitgas, P. (2009). Detection of changes in time-series of indicators using cusum control charts. Aquatic Living Resources, 22(02), 187–192.

Morris, S., & Paradiso, J. (2002). Shoe-integrated sensor system for wireless gait analysis and real-time feedback. In Engineering in Medicine and Biology, 2002. 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society EMBS/BMES Conference, 2002. Proceedings of the Second Joint, IEEE (Vol. 3, pp. 2468–2469).

Nativi, S., Craglia, M., & Pearlman, J. (2012). The brokering approach for multidisciplinary interoperability: A position paper. International Journal of Spatial Data Infrastructures Research, 7, 1–15.

Osanaiye, P., & Talabi, C. (1989). On some non-manufacturing applications of counted data cumulative sum (CUSUM) control chart schemes. The Statistician, 38(4), 251–257.

Page, E. (1954). Continuous inspection schemes. Biometrika, 41(1/2), 100–115.CrossRef

Shvachko, K., Kuang, H., Radia, S., & Chansler, R. (2010). The Hadoop distributed file system. In 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), IEEE (pp. 1–10).

Simoncelli, D., Dusi, M., Gringoli, R., & Niccolini, S. (2013). Stream-monitoring with blockmon: Convergence of network measurements and data analytics platforms. SIGCOMM Computer Communication Review, 43(2), 29–36.CrossRef

Sitaram, D., Srinivasaraghavan, H., Agarwal, K., Agrawal, A., Joshi, N., & Ray, D. (2013). Pipelining acoustic model training for speech recognition using storm. In 2013 Fifth International Conference on Computational Intelligence, Modelling and Simulation (CIMSim) (pp. 219–224).

Sunderrajan, A., Aydt, H., & Knoll, A. (2014). Real time load prediction and outliers detection using storm. In Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, ACM, New York, NY, USA, DEBS ’14 (pp. 294–297).

Trilles, S., Belmonte, O., Diaz, L., & Huerta, J. (2014). Mobile access to sensor networks by using GIS standards and restful services. IEEE Sensors Journal, 14(12), 4143–4153.CrossRef

Yutan, D., Jun, L., Fang, L., & Luying, C. (2014). A real-time anomalies detection system based on streaming technology. In 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC) (Vol. 2, pp. 275–279).

Titel: Real-Time Anomaly Detection from Environmental Data Streams
verfasst von: Sergio Trilles
Sven Schade
Óscar Belmonte
Joaquín Huerta
Verlag: Springer International Publishing
Buch: AGILE 2015
Print ISBN: 978-3-319-16786-2

Electronic ISBN: 978-3-319-16787-9

Copyright-Jahr: 2015
DOI: https://doi.org/10.1007/978-3-319-16787-9_8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner