Support vector interval regression networks for interval regression analysis

doi:10.1016/S0165-0114(02)00570-5

Fuzzy Sets and Systems

Volume 138, Issue 2, 1 September 2003, Pages 283-300

https://doi.org/10.1016/S0165-0114(02)00570-5 Get rights and content

Abstract

In this paper, the support vector interval regression networks (SVIRNs) are proposed for the interval regression analysis. The SVIRNs consist of two radial basis function networks. One network identifies the upper side of data interval, and the other network identifies the lower side of data intervals. Because the support vector regression (SVR) approach is equivalent to solving a linear constrained quadratic programming problem, the number of hidden nodes and the initial values of adjustable parameters can be easily obtained. Since the selection of a parameter ε in the SVR approach may seriously affect the modeling performance, a two-step approach is proposed to properly select the ε value. After the SVR approach with the selected ε, an initial structure of SVIRNs can be obtained. Besides, outliers will not significantly affect the upper and lower bound interval obtained through the proposed two-step approach. Consequently, a traditional back-propagation (BP) learning algorithm can be used to adjust the initial structure networks of SVIRNs under training data sets without or with outliers. Due to the better initial structure of SVIRNs are obtained by the SVR approach, the convergence rate of SVIRNs is faster than the conventional networks with BP learning algorithms or with robust BP learning algorithms for interval regression analysis. Four examples are provided to show the validity and applicability of the proposed SVIRNs.

References (21)

C.B. Cheng et al.
Fuzzy regression with radial basis function network
Fuzzy Sets and Systems
(2001)
B. Heshmaty et al.
Fuzzy linear regression and its applications to forecasting in uncertain environment
Fuzzy Sets and Systems
(1985)
L. Huang et al.
Robust interval regression analysis using neural networks
Fuzzy Sets and Systems
(1998)
H. Ishibuchi et al.
An architecture of neural networks with interval weights and its application to fuzzy regression analysis
Fuzzy Sets and Systems
(1993)
D.S. Chen et al.
A robust back-propagation learning algorithm for function approximation
IEEE Trans. Neural Networks
(1994)
C.C. Chuang et al.
The annealing robust back-propagation (ARBP) learning algorithm
IEEE Trans. Neural Networks
(2000)
A. Cichocki et al.
Neural Networks for Optimization and Signal Processing
(1993)
H. Drucker
Support Vector Regression Machines
(1997)
M. Evans et al.
Statistical Distributions
(1993)
J.E. Freund et al.
Dictionary/outline of Basic Statistics
(1966)

There are more references available in the full text version of this article.

Cited by (124)

A new algorithm for support vector regression with automatic selection of hyperparameters
2023, Pattern Recognition
The hyperparameters in support vector regression (SVR) determine the effectiveness of the support vectors with fitting and predictions. However, the choice of these hyperparameters has always been challenging in both theory and practice. The $ν$ -support vector regression eliminates the need to specify an $ϵ$ value elegantly, but at the cost of specifying or postulating a $ν$ value. We propose an extended primal objective function arising from probability regularization leading to an automatic selection of $ϵ$ , and we can express $ν$ as an explicit function of $ϵ$ . The resultant hyperparameter values can be interpreted as ‘working’ values required only in training but not testing or prediction. This regularized algorithm, namely $ϵ^{*}$ -SVR, automatically provides a data-dependent $ϵ$ and is found to have a close connection to the $ν$ -support vector regression in the sense that $ν$ as a fraction is a sensible function of $ϵ$ . The $ϵ^{*}$ -SVR automatically selects both $ν$ and $ϵ$ values. We illustrate these findings with some public benchmark datasets.
On pairing Huber support vector regression
2020, Applied Soft Computing
In this paper, a novel and efficient pairing support vector regression learning method using $ε -$ insensitive Huber loss function (PHSVR) is proposed where the $ε -$ insensitive zone having flexible shape is determined by tightly fitting the training samples. Our approach leads to solving a pair of unconstrained minimization problems in primal and the solutions are obtained by two algorithms: a functional iterative (FPHSVR) and Newton iterative (NPHSVR) algorithms. The finite termination of the Newton method to its global minimum solution is proved. The significant advantages of the proposed method are the robustness, generalization ability and learning speed. Experiments performed on a series of synthetic data sets, polluted by different types of noise including heteroscedastic noise and outliers, and on real-world benchmark data sets confirm the effectiveness and superiority of the proposed method.
Fuzzy regression analysis: Systematic review and bibliography
2019, Applied Soft Computing Journal
Statistical regression analysis is a powerful and reliable method to determine the impact of one or several independent variable(s) on a dependent variable. It is the most widely used of all statistical methods and has broad applicability to numerous practical problems. However, various problems can arise, when for instance the sample size is too small, distributional assumptions are not fulfilled, the relationship between independent and dependent variables is vague or when there is an ambiguity of events. Moreover, the complexity of real-life problems often makes the underlying models inadequate, since information is frequently imprecise in many ways. To relax these rigidities, numerous researchers have modified and extended concepts of statistical regression analysis by means of concepts of fuzzy set theory. By now, there is a large number of papers on the topic of fuzzy regression analysis, especially concerning possibilistic, fuzzy least squares or machine learning approaches. Additionally, the variety of approaches includes probabilistic, logistic, type-2 and clusterwise fuzzy regression methods, among many others. Besides papers mainly devoted to advances in methodology, there are also several papers presenting case studies in various research fields. To structure this diversity of papers, proposals and applications we give in this paper a comprehensive systematic review and provide a bibliography on the topic of fuzzy regression analysis. Thus, the paper intends to consolidate the topic in order to aid new researchers in this area, focuses the field’s attention on key open questions, and highlights possible directions for future research.
Simplex basis function based sparse least squares support vector regression
2019, Neurocomputing
Citation Excerpt :
A novel geometric framework, based on which new SVR models and algorithms are proposed in [6], these use convex hull algorithms for data reduction. The SVR has also been applied in applications in nonstationary settings, e.g., time-series prediction in nonlinear environment [7], and interval regression analysis in time-varying environment [8]. A major advance for computational efficiency is the introduction of least squares support vector machine (LSSVM) and least squares support vector regression (LSSVR), which uses equality constraints [9] so that closed form least squares type solutions are available.
In this paper, a novel sparse least squares support vector regression algorithm, referred to as LSSVR-SBF, is introduced which uses a new low rank kernel based on simplex basis function, which has a set of nonlinear parameters. It is shown that the proposed model can be represented as a sparse linear regression model based on simplex basis functions. We propose a fast algorithm for least squares support vector regression solution at the cost of O(N) by avoiding direct kernel matrix inversion. An iterative estimation algorithm has been proposed to optimize the nonlinear parameters associated with the simplex basis functions with the aim of minimizing model mean square errors using the gradient descent algorithm. The proposed fast least square solution and the gradient descent algorithm are alternatively applied. Finally it is shown that the model has a dual representation as a piecewise linear model with respect to the system input. Numerical experiments are carried out to demonstrate the effectiveness of the proposed approaches.
Development of new correlations for the oil formation volume factor in oil reservoirs using artificial intelligent white box technique
2018, Petroleum
Citation Excerpt :
Support Vector machine is the type of supervised learning that is mostly used for regression and pattern recognition purposes [28,29]. Based on soft margin hyper-plane support vector machine have been introduced as new artificial intelligence tool framework for both classification and function approximation [30,31]. Instead of sigmoidal type transfer function like in artificial neural network, support vector machine stands on the kernel neuron function which definitely allows projection to higher planes and able to solve more complicated and complex highly nonlinear problems [32].
Oil formation volume factor (OFVF) is considered one of the main parameters required to characterize the crude oil. OFVF is needed in reservoir simulation and prediction of the oil reservoir performance. Existing correlations apply for specific oils and cannot be extended to other oil types. In addition, big errors were obtained when we applied existing correlations to predict the OFVF. There is a massive need to have a global OFVF correlation that can be used for different oils with less error.
The objective of this paper is to develop a new empirical correlation for oil formation volume factor (OFVF) prediction using artificial intelligent techniques (AI) such as; artificial neural network (ANN), adaptive neuro-fuzzy inference system (ANFIS), and support vector machine (SVM). For the first time we changed the ANN model to a white box by extracting the weights and the biases from AI models and form a new empirical equation for OFVF prediction. In this paper we present a new empirical correlation extracted from ANN based on 760 experimental data points for different oils with different compositions.
The results obtained showed that the ANN model yielded the highest correlation coefficient (0.997) and lowest average absolute error (less than 1%) for OFVF prediction as a function of the specific gravity of gas, the dissolved gas to oil ratio, the oil specific gravity, and the temperature of the reservoir compared with ANFIS and SVM. The developed empirical equation from the ANN model outperformed the previous empirical correlations and AI models for OFVF prediction. It can be used to predict the OFVF with a high accuracy.
A Systematic Survey on Implementation of Fuzzy Regression Models for Real Life Applications
2024, Archives of Computational Methods in Engineering

View all citing articles on Scopus

^☆: This work was supported by National Science Council Under Grant NSC89-2218-E-146-001.

View full text

Support vector interval regression networks for interval regression analysis☆

Abstract

Fuzzy Sets and Systems

Fuzzy Sets and Systems

Fuzzy Sets and Systems

Fuzzy Sets and Systems

A robust back-propagation learning algorithm for function approximation

IEEE Trans. Neural Networks

The annealing robust back-propagation (ARBP) learning algorithm

IEEE Trans. Neural Networks

Neural Networks for Optimization and Signal Processing

Support Vector Regression Machines

Statistical Distributions

Dictionary/outline of Basic Statistics