Entropy conserving probability transforms and the entailment principle

doi:10.1016/j.fss.2007.01.019

Fuzzy Sets and Systems

Volume 158, Issue 12, 16 June 2007, Pages 1397-1405

https://doi.org/10.1016/j.fss.2007.01.019 Get rights and content

Abstract

Our main result here is the development of a general procedure for transforming some initial probability distribution into a new probability distribution in a way that the resulting distribution has entropy at least as great as the original distribution.A significant aspect of our approach is that it makes use of Zadeh's entailment principle which is itself a general procedure for going from an initial possibility distribution to a new possibility distribution so that the resulting possibility has an uncertainty at least as great as the original.

References (26)

M. Delgado et al.
On the concept of possibility–probability consistency
Fuzzy Sets and Systems
(1987)
D. Dubois et al.
Unfair coins and necessary measures: a possible interpertation of histograms
Fuzzy Sets and Systems
(1983)
D. Dubois et al.
Fuzzy sets in approximate reasoning. Part 1: inference with possibility distributions
Fuzzy Sets and Systems
(1991)
D. Dubois et al.
Fuzzy sets in approximate reasoning. Part 2: logical approaches
Fuzzy Sets and Systems
(1991)
R.R. Yager
Default knowledge and measures of specificity
Inform. Sci.
(1992)
L.A. Zadeh
Toward a perception-based theory of probabilistic seasoning with imprecise probabilities
J. Statist. Plann. Inference
(2002)
L.A. Zadeh
Toward a generalized theory of uncertainty (GTU)—an outline
Inform. Sci.
(2005)
J. Aczel et al.
On Measures of Information and their Characterizations
(1975)
B. Buck
Maximum Entropy in Action: a Collection of Expository Essays
(1991)
D. Dubois et al.
A notion of comparative probabilistic entropy based on the possibilistic specificity ordering

D. Dubois et al.

On several representations of an uncertain body of evidence

D. Dubois et al.

The principle of minimum specificity as a basis for evidential reasoning

D. Dubois et al.

Possibility Theory: an Approach to Computerized Processing of Uncertainty

(1988)

Cited by (9)

Likelihood-fuzzy analysis: From data, through statistics, to interpretable fuzzy classifiers
2018, International Journal of Approximate Reasoning
Citation Excerpt :
However, the confidence given by a fuzzy system is not straightforwardly meaningful. In fact, some works focused on bridges between fuzzy and statistic interpretations [48–53], and there are ways to convert probability [54–59] or likelihood [60,61] into possibility, and a possibility distribution can be viewed as a fuzzy set [62], but the inverse is not true, i.e. it is not straightforward that any fuzzy measure represents a possibility nor a probability. Therefore, the confidence measure given as output by a fuzzy system is generally meaningless, unless the system is modeled properly.
Data-driven extraction of knowledge naturally takes advantage from the use of statistics, since statistical approaches enable to summarize information embedded in the dataset. On the other hand, fuzzy logic has gained increasing importance in data analytics, since it can enable to build a transparent and interpretable knowledge base. In this paper, a method named Likelihood-Fuzzy Analysis (LFA) for translating statistical information coming from labeled data into a fuzzy classification system is proposed. The characteristics of the method and of the resulting fuzzy classifiers are underlined. They range from the capacity of managing a dataset eventually comprising heterogeneous variables and missing data, to the high predictability, good confidence measure in terms of class probabilities, and interpretability of the fuzzy classification model, by means of semantically interpretable fuzzy partitions and if–then rules. The application of the method on a number of benchmark datasets is presented, showing high performances and semantic power, with respect to well-established methods, including fuzzy systems and non-fuzzy approaches.
An intelligent quality-based approach to fusing multi-source probabilistic information
2016, Information Fusion
Our objective here is to obtain quality-fused values from multiple sources of probabilistic distributions, where quality is related to the lack of uncertainty in the fused value and the use of credible sources. We first introduce a vector representation for a probability distribution. With the aid of the Gini formulation of entropy, we show how the norm of the vector provides a measure of the certainty, i.e., information, associated with a probability distribution. We look at two special cases of fusion for source inputs those that are maximally uncertain and certain. We provide a measure of credibility associated with subsets of sources. We look at the issue of finding the highest quality fused value from the weighted aggregations of source provided probability distributions.
Fuzzy partitioning for clinical DSSs using statistical information transformed into possibility-based knowledge
2014, Knowledge-Based Systems
Citation Excerpt :
The core of this approach relies on the transformation from clinical data, acquired in the form of a training dataset and encoded statistically, or promptly available in the form of probability distributions (together with prior probabilities), to fuzzy knowledge, to be used in DSSs in medical field. The definition of fuzzy sets starting from statistical information is typically accomplished by using probability distributions [18,21,28,42,53,54], but, among these approaches, only that proposed in [42] aims to construct fuzzy sets to be used for classification; however, using only class probability distributions hides a lack of knowledge regarding prior probabilities. A different approach involves the use of likelihood functions [19,43], which take into account class distributions and prior probabilities at the same time; however, in [19], no transformation is applied to obtain fuzzy sets from likelihood functions, thus resulting in a system reflecting a probabilistic interpretation of fuzzy sets.
Fuzzy logic has gained much importance for its applications in Decision Support Systems (DSSs), especially in fields like medicine, where the final user has to handle uncertain data and vague concepts, and needs an intelligible system based on clear rule bases. In medical applications, physicians are often skilled to reason using statistical approaches, since this type of information is often known, or can be extracted from data. However, since decisions have to be applied to single patients, clinical data items have to be classified in order to obtain the plausibility of conclusions, rather than their probability. Therefore, statistical information can be used, in order to define fuzzy sets and rules for constructing DSSs based on fuzzy logic. While the transformation of probability distributions is well known in literature, here, an approach is presented for transforming likelihood functions into fuzzy sets, based on possibility theory, which is actually instanced into four different new methods for knowledge representation. A comparison among different methods is shown, as well as the analysis of transformation properties and resulting fuzzy sets characteristics are considered, by using synthetic and real data. Finally, some considerations about the most suitable method to be used in the context of clinical DSSs are given.
A Possibilistic Information Fusion-Based Unsupervised Feature Selection Method Using Information Quality Measures
2023, IEEE Transactions on Fuzzy Systems
A fuzzy supplier selection application using large survey datasets of delivery performance
2015, Advances in Fuzzy Systems
A Fuzzy Set Based Evaluation of Suppliers on Delivery, Front Office Quality and Value-Added Services
2014, Communications in Computer and Information Science

View all citing articles on Scopus

View full text

Entropy conserving probability transforms and the entailment principle

Abstract

Fuzzy Sets and Systems

Fuzzy Sets and Systems

Fuzzy Sets and Systems

Fuzzy Sets and Systems

Inform. Sci.

J. Statist. Plann. Inference

Inform. Sci.

On Measures of Information and their Characterizations

Maximum Entropy in Action: a Collection of Expository Essays

A notion of comparative probabilistic entropy based on the possibilistic specificity ordering

On several representations of an uncertain body of evidence

The principle of minimum specificity as a basis for evidential reasoning

Possibility Theory: an Approach to Computerized Processing of Uncertainty