Skip to main content
Erschienen in: The Journal of Supercomputing 1/2019

24.05.2018

A pattern-based outlier region detection method for two-dimensional arrays

verfasst von: Ki Yong Lee, Young-Kyoon Suh

Erschienen in: The Journal of Supercomputing | Ausgabe 1/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Recently, with the prevalence of various sensing devices and numerical simulation software, a large amount of data is being generated in the form of a two-dimensional (2D) array. One of the important tasks for analyzing such arrays is to find anomalous or outlier regions in such a 2D array. In this article, we propose an effective method for detecting outlier regions in an arbitrary 2D array, which show a significantly different pattern from that of their surrounding regions. Unlike most existing methods that determine the outlierness of a region based on how different its average is from that of its neighboring elements, our method exploits the regression models of a region in determining its outlierness. More specifically, this method first divides the array into a number of small subarrays and then builds a regression model for each subarray. In turn, the method iteratively merges adjacent subarrays with similar regression models into larger clusters. After the clustering, the proposed method reports very small clusters as outlier regions at the final step. Lastly, we demonstrate in our experiments the effectiveness of the proposed method on synthetic and real datasets.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Amidan BG, Ferryman TA, Cooley SK (2005) Data outlier detection using the Chebyshev theorem. In: Proceedings of 2005 IEEE Aerospace Conference. IEEE, pp 3814–3819 Amidan BG, Ferryman TA, Cooley SK (2005) Data outlier detection using the Chebyshev theorem. In: Proceedings of 2005 IEEE Aerospace Conference. IEEE, pp 3814–3819
2.
Zurück zum Zitat Bouguettaya A, Yu Q, Liu X, Zhou X, Song A (2015) Efficient agglomerative hierarchical clustering. Expert Syst Appl 42(5):2785–2797CrossRef Bouguettaya A, Yu Q, Liu X, Zhou X, Song A (2015) Efficient agglomerative hierarchical clustering. Expert Syst Appl 42(5):2785–2797CrossRef
3.
Zurück zum Zitat Chawla M, Sharma S, Sivaswamy J, Kishore LA (2009) Method for automatic detection and classification of stroke from brain CT images. In: Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, pp 3581–3584 Chawla M, Sharma S, Sivaswamy J, Kishore LA (2009) Method for automatic detection and classification of stroke from brain CT images. In: Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, pp 3581–3584
4.
Zurück zum Zitat Chawla S, Sun P (2006) SLOM: a new measure for local spatial outliers. Knowl Inf Syst 9(4):412–429CrossRef Chawla S, Sun P (2006) SLOM: a new measure for local spatial outliers. Knowl Inf Syst 9(4):412–429CrossRef
5.
6.
Zurück zum Zitat Franke C, Gertz M (2009) ORDEN: outlier region detection and exploration in sensor networks. In: Proceedings of the 2009 ACM SIGMOD International Conference on Management of data. ACM, pp 1075–1078 Franke C, Gertz M (2009) ORDEN: outlier region detection and exploration in sensor networks. In: Proceedings of the 2009 ACM SIGMOD International Conference on Management of data. ACM, pp 1075–1078
7.
Zurück zum Zitat Friedman JH, Fisher NI (1999) Bump hunting in high-dimensional data. Stat Comput 9(2):123–143CrossRef Friedman JH, Fisher NI (1999) Bump hunting in high-dimensional data. Stat Comput 9(2):123–143CrossRef
8.
Zurück zum Zitat Janeja VP, Atluri V (2008) Random walks to identify anomalous free-form spatial scan windows. IEEE Trans Knowl Data Eng 20(10):1378–1392CrossRef Janeja VP, Atluri V (2008) Random walks to identify anomalous free-form spatial scan windows. IEEE Trans Knowl Data Eng 20(10):1378–1392CrossRef
9.
Zurück zum Zitat Jim J, Lee W, Song JJ, Lee SB (2017) Optimized combinatorial clustering for stochastic processes. Clust Comput 20(2):1135–1148CrossRef Jim J, Lee W, Song JJ, Lee SB (2017) Optimized combinatorial clustering for stochastic processes. Clust Comput 20(2):1135–1148CrossRef
11.
Zurück zum Zitat Kulldorff M, Huang L, Pickle L, Duczmal L (2006) An elliptic spatial scan statistic. Stat Med 25(22):3929–3943MathSciNetCrossRef Kulldorff M, Huang L, Pickle L, Duczmal L (2006) An elliptic spatial scan statistic. Stat Med 25(22):3929–3943MathSciNetCrossRef
12.
Zurück zum Zitat Kutner MH, Nachtsheim CJ, Neter J, Li W (2005) Applied linear statistical models, 5th edn. McGraw Hill, New York Kutner MH, Nachtsheim CJ, Neter J, Li W (2005) Applied linear statistical models, 5th edn. McGraw Hill, New York
13.
Zurück zum Zitat Langfelder P, Zhang B, Horvath S (2007) Defining clusters from a hierarchical cluster tree: the dynamic tree cut package for R. Bioinformatics 24(5):719–720CrossRef Langfelder P, Zhang B, Horvath S (2007) Defining clusters from a hierarchical cluster tree: the dynamic tree cut package for R. Bioinformatics 24(5):719–720CrossRef
14.
Zurück zum Zitat Li X, Lv J, Yi Z (2018) An efficient representation-based method for boundary point and outlier detection. IEEE Trans Neural Netw Learn Syst 29(1):51–62MathSciNetCrossRef Li X, Lv J, Yi Z (2018) An efficient representation-based method for boundary point and outlier detection. IEEE Trans Neural Netw Learn Syst 29(1):51–62MathSciNetCrossRef
15.
Zurück zum Zitat Lu CT, Kou Y, Zhao J, Chen L (2007) Detecting and tracking regional outliers in meteorological data. Inf Sci 177(7):1609–1632CrossRef Lu CT, Kou Y, Zhao J, Chen L (2007) Detecting and tracking regional outliers in meteorological data. Inf Sci 177(7):1609–1632CrossRef
16.
Zurück zum Zitat Lu CT, SANTOS JR RFD, Liu X, Kou Y (2011) A graph-based approach to detect abnormal spatial points and regions. Int J Artif Intell Tools 20(04):721–751CrossRef Lu CT, SANTOS JR RFD, Liu X, Kou Y (2011) A graph-based approach to detect abnormal spatial points and regions. Int J Artif Intell Tools 20(04):721–751CrossRef
17.
Zurück zum Zitat Luts J, Laudadio T, Idema AJ, Simonetti AW, Heerschap A, Vandermeulen D, Suykens JA, Van Huffel S (2009) Nosologic imaging of the brain: segmentation and classification using MRI and MRSI. NMR Biomed 22(4):374–390CrossRef Luts J, Laudadio T, Idema AJ, Simonetti AW, Heerschap A, Vandermeulen D, Suykens JA, Van Huffel S (2009) Nosologic imaging of the brain: segmentation and classification using MRI and MRSI. NMR Biomed 22(4):374–390CrossRef
18.
Zurück zum Zitat Mao J, Wang T, Jin C, Zhou A (2017) Feature grouping-based outlier detection upon streaming trajectories. IEEE Trans Knowl Data Eng 29(12):2696–2709CrossRef Mao J, Wang T, Jin C, Zhou A (2017) Feature grouping-based outlier detection upon streaming trajectories. IEEE Trans Knowl Data Eng 29(12):2696–2709CrossRef
21.
Zurück zum Zitat Neill DB, Moore AW, Cooper GF (2006) A Bayesian spatial scan statistic. In: Weiss Y, Schölkopf B, Platt JC (eds) Advances in Neural Information Processing Systems 18 (NIPS 2005). Neural Information Processing Systems Foundation, Inc, pp 1003–1010 Neill DB, Moore AW, Cooper GF (2006) A Bayesian spatial scan statistic. In: Weiss Y, Schölkopf B, Platt JC (eds) Advances in Neural Information Processing Systems 18 (NIPS 2005). Neural Information Processing Systems Foundation, Inc, pp 1003–1010
22.
Zurück zum Zitat Patil GP, Taillie C (2004) Upper level set scan statistic for detecting arbitrarily shaped hotspots. Environ Ecol Stat 11(2):183–197MathSciNetCrossRef Patil GP, Taillie C (2004) Upper level set scan statistic for detecting arbitrarily shaped hotspots. Environ Ecol Stat 11(2):183–197MathSciNetCrossRef
23.
Zurück zum Zitat Prastawa M, Bullitt E, Ho S, Gerig G (2004) A brain tumor segmentation framework based on outlier detection. Med Image Anal 8(3):275–283CrossRef Prastawa M, Bullitt E, Ho S, Gerig G (2004) A brain tumor segmentation framework based on outlier detection. Med Image Anal 8(3):275–283CrossRef
24.
Zurück zum Zitat Ramteke R, Monali YK (2012) Automatic Medical image classification and abnormality detection using K-nearest neighbour. Int J Adv Comput Res 2(4):190–196 Ramteke R, Monali YK (2012) Automatic Medical image classification and abnormality detection using K-nearest neighbour. Int J Adv Comput Res 2(4):190–196
25.
Zurück zum Zitat Reed IS, Yu X (1990) Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution. IEEE Trans Acoust Speech Signal Proces 38(10):1760–1770CrossRef Reed IS, Yu X (1990) Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution. IEEE Trans Acoust Speech Signal Proces 38(10):1760–1770CrossRef
27.
Zurück zum Zitat Stein DW, Beaven SG, Hoff LE, Winter EM, Schaum AP, Stocker AD (2002) Anomaly detection from hyperspectral imagery. IEEE Signal Proces Mag 19(1):58–69CrossRef Stein DW, Beaven SG, Hoff LE, Winter EM, Schaum AP, Stocker AD (2002) Anomaly detection from hyperspectral imagery. IEEE Signal Proces Mag 19(1):58–69CrossRef
28.
Zurück zum Zitat Suhail Z, Sarwar M, Murtaza K (2015) Automatic detection of abnormalities in mammograms. BMC Med Imaging 15(1):53CrossRef Suhail Z, Sarwar M, Murtaza K (2015) Automatic detection of abnormalities in mammograms. BMC Med Imaging 15(1):53CrossRef
29.
Zurück zum Zitat Tango T, Takahashi K (2005) A flexibly shaped spatial scan statistic for detecting clusters. Int J Health Geogr 4(1):11CrossRef Tango T, Takahashi K (2005) A flexibly shaped spatial scan statistic for detecting clusters. Int J Health Geogr 4(1):11CrossRef
30.
Zurück zum Zitat Telang A, Deepak P, Joshi S, Deshpande P, Rajendran R (2014) Detecting localized homogeneous anomalies over spatio-temporal data. Data Mining Knowl Discov 28(5–6):1480–1502MathSciNetCrossRef Telang A, Deepak P, Joshi S, Deshpande P, Rajendran R (2014) Detecting localized homogeneous anomalies over spatio-temporal data. Data Mining Knowl Discov 28(5–6):1480–1502MathSciNetCrossRef
31.
Zurück zum Zitat Tran L, Fan L, Shahabi C (2016) Distance-based outlier detection in data streams. Proc VLDB Endow 9(12):1089–1100CrossRef Tran L, Fan L, Shahabi C (2016) Distance-based outlier detection in data streams. Proc VLDB Endow 9(12):1089–1100CrossRef
32.
Zurück zum Zitat You C, Robinson DP, Vidal R (2017) Provable self-representation based outlier detection in a union of subspaces. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp 1–10 You C, Robinson DP, Vidal R (2017) Provable self-representation based outlier detection in a union of subspaces. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp 1–10
33.
Zurück zum Zitat Zheng G, Brantley SL, Lauvaux T, Li Z (2017) Contextual spatial outlier detection with metric learning. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 2161–2170 Zheng G, Brantley SL, Lauvaux T, Li Z (2017) Contextual spatial outlier detection with metric learning. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 2161–2170
34.
Zurück zum Zitat Zhu M, Aggarwal CC, Ma S, Zhang H, Huai J (2017) Outlier detection in sparse data with factorization machines. In: Proceedings of the 2017 ACM Conference on Information and Knowledge Management, pp 817–826 Zhu M, Aggarwal CC, Ma S, Zhang H, Huai J (2017) Outlier detection in sparse data with factorization machines. In: Proceedings of the 2017 ACM Conference on Information and Knowledge Management, pp 817–826
Metadaten
Titel
A pattern-based outlier region detection method for two-dimensional arrays
verfasst von
Ki Yong Lee
Young-Kyoon Suh
Publikationsdatum
24.05.2018
Verlag
Springer US
Erschienen in
The Journal of Supercomputing / Ausgabe 1/2019
Print ISSN: 0920-8542
Elektronische ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-018-2418-2

Weitere Artikel der Ausgabe 1/2019

The Journal of Supercomputing 1/2019 Zur Ausgabe

Premium Partner