nach oben

Data Mining and Knowledge Discovery

Erschienen in:

29.03.2016

Outlying property detection with numerical attributes

verfasst von: Fabrizio Angiulli, Fabio Fassetti, Giuseppe Manco, Luigi Palopoli

Erschienen in: Data Mining and Knowledge Discovery | Ausgabe 1/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The outlying property detection problem (OPDP) is the problem of discovering the properties distinguishing a given object, known in advance to be an outlier in a database, from the other database objects. This problem has been recently analyzed focusing on categorical attributes only. However, numerical attributes are very relevant and widely used in databases. Therefore, in this paper, we analyze the OPDP within a context where also numerical attributes are taken into account, which represents a relevant case left open in the literature. As major contributions, we present an efficient parameter-free algorithm to compute the measure of object exceptionality we introduce, and propose a unified framework for mining exceptional properties in the presence of both categorical and numerical attributes.

Vorheriger Artikel Adversarial classification using signaling games with an application to phishing detection

Nächster Artikel Evidence-based uncertainty sampling for active learning

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

For the sake of simplicity and without loss of generality, we are assuming that an arbitrary ordering of the attributes in A has been fixed.

We point out that \(k_\theta \) has a twofold function: it allows the analyst to control the complexity of the mined patterns and it speeds up the algorithm execution. However, by setting \(k_\theta \) to m the algorithm is able to detect explanations of any length, while pruning the search space and avoiding overfitting by means of the threshold support.

In practice, if a tuple is detected as an outlier in a given iteration, it gets a positive score. Scores are then summarized in the combine function, and tuples are sorted according to the scores.

Experiments were performed on an Intel Core i7 2.3 GHz based computer by using the Java programming language.

Aggarwal C (2013) Outlier analysis. Springer, New YorkCrossRefMATH

Aggarwal CC, Yu PS (2001) Outlier detection for high dimensional data. In: Proceeding of the ACM SIGMOD conference on managment of data (SIGMOD’01), pp 37–46

Agrawal R, Srikant R (1994) Fast algorithms for mining association rules in large databases. In: Proceedings of the international conference on very large data bases (VLDB’94), pp 487–499

Angiulli F, Fassetti F (2009) Dolphin: an efficient algorithm for mining distance-based outliers in very large datasets. ACM Trans Knowl Discov Data 3(1):Article 4

Angiulli F, Fassetti F, Palopoli L (2009) Detecting outlying properties of exceptional objects. ACM Trans Database Syst 34(1):1–62CrossRef

Arning A, Aggarwal C, Raghavan P (1996) A linear method for deviation detection in large databases. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining (KDD’96), pp 164–169

Barnett V, Lewis T (1994) Outliers in statistical data. Wiley, ChichesterMATH

Bay S, Pazzani M (1999) Detecting change in categorical data: mining constrast sets. In: Proceedings of the ACM conference on knowledge discovery in data (KDD’99), pp 302–306

Breunig MM, Kriegel H, Ng RT, Sander J (2000) Lof: identifying density-based local outliers. In: Proceedings of the ACM SIGMOD conference on managment of data (SIGMOD’00), pp 93–104

Caroni C (2000) Outlier detection by robust principal component analysis. Commun Stat Simul Comput 29:129–151CrossRefMATH

Chawla NV, Japkowicz N, Kotcz A (2004) Editorial: special issue on learning from imbalanced data sets. SIGKDD Explor 6(1):1–6CrossRef

Costa G, Fassetti F, Guarascio M, Manco G, Ortale R (2010) Mining models of exceptional objects through rule learning. In: Proceedings of the ACM symposium on applied computing (SAC’10), pp 1078–1082

Dang XH, Assent I, Ng RT, Zimek A, Schubert E (2014) Discriminative features for identifying and interpreting outliers. In: Proceedings of the IEEE international conference on data engineering, (ICDE’14), pp 88–99

Dang XH, Micenkov B, Assent I, Ng RT (2013) Local outlier detection with interpretation. In: Proceedings of the joint European conference on machine learning and knowledge discovery in databases (ECML-PKDD’13). Lecture Notes in Computer Science, vol 8190. pp 304–320

De Vries, T, Chawla, S, Houle M (2010) Finding local anomalies in very high dimensional space. In: Proceedings of the IEEE international confence on data mining (ICDM’10), pp 128–137

Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B 39

Duan L, Tang G, Pei J, Bailey J, Campbell A, Tang C (2015) Mining outlying aspects on numeric data. Data Min Knowl Dis 29(5):1116–1151MathSciNetCrossRef

Eskin E (2000) Anomaly detection over noisy data using learned probability distributions. In: Proceedings of the international conference on machine learning (ICML’00), pp 255–262

Figueiredo MAT, Jain AK (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24:381–396CrossRef

Fox J (1990) Describing univariate distributions. In: Fox J, Long JS (eds) Modern methods of data analysis. Sage Publications, Newbury Park, pp 58–125

Ghoting A, Parthasarathy S, Otey ME (2015) Fast mining of distance-based outliers in high-dimensional datasets. Data Min Knowl Dis 16:349–364MathSciNetCrossRef

Greco A, Perri S (2014) Identification of high shears and compressive discontinuities in the inner heliosphere. Astrophys J 784(2):163CrossRef

Jin W, Tung AKH, Han J (2001) Mining top-n local outliers in large databases. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining (KDD’01), pp 293–298

Jones MC, Henderson DA (2009) Maximum likelihood kernel density estimation: on the potential of convolution sieves. Comput Stat Data Anal 53:3726–3733MathSciNetCrossRefMATH

Knorr E, Ng R (1998) Algorithms for mining distance-based outliers in large datasets. In: Proceedings of the international conference on very large databases (VLDB’98), pp 392–403

Knorr E, Ng R (1999) Finding intensional knowledge of distance-based outliers. In: Proceedings of the international conference on very large databases (VLDB’99), pp 211–222

Kriegel H-P, Schubert M, Zimek A (2008) Angle-based outlier detection in high-dimensional data. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining (KDD’08), pp 444–452

Kriegel H-P, Kröger P, Schubert E, Zimek A (2009) Loop: local outlier probabilities. In: Proceedings of the ACM international conference on information and knowledge management (CIKM’09), pp 1649–1652

Kriegel HP, Kroger P, Schubert E, Zimek A (2009) Outlier detection in axis-parallel subspaces of high dimensional data. In: Proceedings of the Pacific-Asia conference on knowledge discovery and data mining (PAKDD’09), pp 831–838

Lazarevic A, Kumar V (2005) Feature bagging for outlier detection. In: Proceedings of the ACM SIGKDD conference on knowledge discovery in data (KDD’05), pp 157–166

Lichman M (2013) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine. http://archive.ics.uci.edu/ml

Liu FT, Ting KM, Zhou Z-H (2008) Isolation forest. In: Proceedings of the IEEE international conference on data mining (ICDM’08), pp 413–422

Micenková B, Ng RT, Dang XH, Assent I (2013) Explaining outliers by subspace separability. In: Proceedings of the IEEE international conference on data mining (ICDM’13), pp 518–527

Nguyen H, Gopalkrishnan V, Assent I (2011) An unbiased distance-based outlier detection approach for high dimensional data. In: Proceedings of the international conference on database systems for advanced applications (DASFAA), pp 138–152

Papadimitriou S, Kitagawa H, Gibbons PB, Faloutsos C (2003) Loci: fast outlier detection using the local correlation integral. In: Proceedings of the IEEE international conference on data enginnering (ICDE’03), pp 315–326

Rousseeuw P, Leroy A (2003) Robust regression and outlier detection. Wiley, New YorkMATH

Salgado-Ugarte IH, Pérez-Hernández MA (2003) Exploring the use of variable bandwidth kernel density estimators. Stata J 3(2):133–147

Schölkopf B, Burges C, Vapnik V (1995) Extracting support data for a given task. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining (KDD), pp 252–257

Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, LondonCrossRefMATH

Vinh NX, Chan J, Bailey J, Leckie C, Ramamohanarao K, Pei J (2015) Scalable outlying-inlying aspects discovery via feature ranking. In: Proceedings of the Pacific-Asia conference on knowledge discovery and data (PAKDD’15), pp 422–434

Xiong L, Chen X, Schneider J (2011) Direct robust matrix factorization for anomaly detection. In: Proceedings of the IEEE international confence on data mining (ICDM’11), pp 844 – 853

Yang X, Latecki LJ, Pokrajac D (2009) Outlier detection with globally optimal exemplar-based GMM. In: Proceedings of the SIAM conference on data mining (SDM’09), pp 145–154

Titel: Outlying property detection with numerical attributes
verfasst von: Fabrizio Angiulli
Fabio Fassetti
Giuseppe Manco
Luigi Palopoli
Publikationsdatum: 29.03.2016
Verlag: Springer US
Erschienen in: Data Mining and Knowledge Discovery / Ausgabe 1/2017
Print ISSN: 1384-5810
Elektronische ISSN: 1573-756X
DOI: https://doi.org/10.1007/s10618-016-0458-x

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 1/2017

Generalizing DTW to the multi-dimensional case requires an adaptive approach

Reliable early classification of time series based on discriminating the classes over time

Evidence-based uncertainty sampling for active learning

TBM, a transformation based method for microaggregation of large volume mixed data

SimUSF: an efficient and effective similarity measure that is invariant to violations of the interval scale assumption

Hierarchical evolving Dirichlet processes for modeling nonlinear evolutionary traces in temporal data