Support vector regression to predict porosity and permeability: Effect of sample size
Highlights
► In this research, the capability of SVR to predict porosity and permeability in a heterogeneous sandstone reservoir under the effect of small sample size is evaluated. ► Particularly, the impact of Vapnik's ε-insensitivity loss function and least-modulus loss function on generalization performance were empirically investigated. ► The results are compared to the Multilayer Perception (MLP) neural network, a widely used regression method, which operates under the ERM principle. ► The mean square error and correlation coefficients were used to measure the quality of predictions. ► The results demonstrate that SVR yields consistently better predictions of the porosity and permeability with small sample size than the MLP method.
Introduction
One of the most important tasks in modeling geoscience data is development of robust and accurate correlation models calibrated to small sample size problems. For permeability estimation, correlation models between porosity and permeability are often built based on the relationship between geophysical logs and core-measured porosity and permeability. In typical practice, core plugs are extracted from a few key wells during drilling whereas geophysical logs are run for all wells in the oil/gas field. The limited number of core plug data poses a challenging problem to existing empirical techniques that employ the empirical risk minimization (ERM) principle such as linear regression and neural networks. Statistical learning theory (SLT) shows that these techniques can be safely used as a measure of the true risk when the sample size is sufficiently large. SLT calls for introducing a structure to match the complexity of the predictive learning technique to the available training data. The structure is composed of elements of increasing complexity that need to be chosen to imitate the response of the learning problem using a limited number of data (Cherkassky and Mulier, 2007).
Artificial neural networks (ANNs) is a universal approximator that is capable of approximating any nonlinear function to any degree of accuracy provided that there are a sufficient number of neurons in the network (Hornik et al., 1989). The structure implemented by ANN may be captured by the number and size of the hidden layers, which are controlled explicitly by the user. This structure may also lead to an overfitting problem during learning, particularly in the presence of a small sample size, which potentially yields a poor generalization model. Although ANN has shown some successful applications to porosity and permeability (Helle and Ursin, 2001; Huang et al., 2001), the underlying learning algorithm has been developed for learning problems of large sample sizes. Hence, for a given small sample size, extensive experiments with several different learning techniques are required to devise an accurate ANN-based regression model (Kaviani et al., 2008).
Recently, support vector machines (SVMs) have been gaining popularity in regression and classification due to their excellent generalization performance. The SVM approach has been successfully applied to several different applications such as face recognition, object detection, handwriting recognition, text detection, speech recognition and prediction, lithology identification, and porosity and permeability determination from log data (Li et al., 2000, Lu et al., 2001, Choisy and Belaid, 2001, Gao et al., 2001, Kim et al., 2001, Ma et al., 2001, Van Gestel et al., 2001, Al-Anazi and Gates, 2010a, Al-Anazi and Gates, 2010b, Al-Anazi and Gates, 2010c, Al-Anazi and Gates, 2010d). The SVM formulation is based on the structural risk minimization (SRM) inductive principle where the empirical risk minimization (ERM) inductive principle and the Vapnik and Chervonenkis (VC) confidence interval are simultaneously minimized (Vapnik and Chervonenkis, 1974, Vapnik, 1982, Vapnik, 1995). The SRM principle introduces a structure where each element of the structure is indexed by a measure of complexity defined by the margin size between two classes in a classification learning problem and by an insensitivity zone size in a regression problem (Cherkassky and Mulier, 2007). The SVM optimization formulation implicitly matches a suitable structure of certain complexity to the available small size sample. This type of structure is controlled independently of the dimension of the problem, which is an advantage over classical learning techniques. In regression applications, the empirical error (the training error) is minimized by Vapnik's ε-insensitivity loss function rather than the quadratic error and absolute-value loss functions used in neural networks and classical regression methods. To generalize to nonlinear regression, kernel functions are used to project the input space into a feature space where a linear or nearly linear regression hypersurface results. A regularization term is used to determine a trade-off between the training error and the VC confidence term. The learning problem is formulated as a constrained convex optimization problem whose solution is used to construct the mapping function between the empirical input and the output data (Kecman, 2005).
Previously, our research demonstrated the generalization capability of SVM in lithology classification and porosity and permeability predictions with sensitivity analysis of kernel function types and SVM regularization parameters. In this research, however, the sensitivity of the SVM-based prediction of porosity and permeability to sample size and empirical loss functions are examined and compared to a multilayer perceptron network. The empirical evaluation of the generalization performance under small sample setting is conducted for two loss functions: first, the ε-insensitivity loss function, and second, the least-modulus (or absolute value) loss function.
Section snippets
Multilayer perceptron neural network model
ANN has been frequently used as an intelligent regression technique in petrophysical properties estimation (Rogers et al., 1995, Huang et al., 1996, Huang et al., 2001, Fung et al., 1997, Helle and Ursin, 2001, Helle and Bhatt, 2002). Backpropagation multilayer perceptron neural networks are considered to be universal approximators: it has been mathematically proven that a network with a hidden layer of an arbitrary large number of nonlinear neurons can approximate any continuous nonlinear
Multilayer perceptron neural network
In MLP neural networks, the topology of the networks consists of input layer, single hidden layer, and output layer. The size of the hidden layer is measured by the number of processing neurons where a sigmoid kernel function is used. The size of the hidden layer determines the complexity of the regression model. Increasing the complexity may lead to overfitting the training data and lead in turn to potentially poor generalization to testing data. Therefore, the size of the hidden layer must be
Reservoir description
The raw input data were obtained from three wells completed in a heterogeneous sandstone oil reservoir deposited in a fluvial dominated deltaic environment of Middle Cretaceous Albian age. The reservoir, consisting of a sequence of sandstone, siltstone, and shale with thin intervals of limestone, coal, and varying amounts of ironstone, can be divided into three main lithologic units. The first one is mainly shale and sands that are thinly interbedded with small amount of siltstones, green mud,
Small sample size testing procedure
The modeling data were prepared as follows:
- 1.
The data were randomly subdivided into divisions as percentages of the data as shown in Table 2 in nine cases.
- 2.
A total of 50 realizations were randomly generated for each case. Since the regression model itself depends on a particular (random) realization of a fixed size training sample, its estimated prediction accuracy is also a random variable. Therefore, the experiment calls for repeating the experimental procedure with different realizations of
Small sample size analysis with the ε-insensitivity loss function
To analyze the results, notched box plots of the mean squared error from the Gaussian-based SVR (SVR–RBF), sigmoid-based SVR (SVR-Sigmoid), and MLP models are compared. The notched box plot is an exploratory data analysis tool that provides statistical summaries of the underlying prediction error distribution. The tops and bottoms of box represent the 25th and 75th percentiles of the error, respectively. The red line in each box is the median of the error. The whiskers are the lines that extend
Small sample size analysis with least-modulus loss function
Although it was demonstrated that the prediction accuracy of the SVM using ε-insensitivity loss function outperforms ordinary least-squares and least-modulus methods for linear regression problems under small size high-dimensional datasets (Cherkassky and Ma, 2004), it would be of practical importance to investigate the impact of different loss functions on prediction accuracy of nonlinear SVM regression with RBF Gaussian and sigmoid kernel functions. As previously stated, SVM simultaneously
Conclusions
The capability of support vector regression to model porosity and permeability in a heterogeneous reservoir with finite sample size has been investigated. The result of the prediction of SVM models with Gaussian radial basis and sigmoid kernel functions have been compared to multilayer perceptron neural networks. The results reveal that
- 1.
the SVM based on radial basis and sigmoid kernel functions is capable of modeling small datasets of porosity and permeability outperforms the MLP method;
- 2.
the SVM
References (31)
- et al.
A support vector machine algorithm to classify lithofacies and model permeability in heterogeneous reservoirs
Engineering Geology
(2010) - et al.
Support vector regression for porosity prediction in a heterogeneous reservoir: a comparative study
Computers and Geosciences
(2010) - et al.
Multilayer feedforward networks are universal approximators
Neural Networks
(1989) - et al.
An integrated neural-fuzzy-genetic-algorithm using hyper-surface membership functions to predict permeability in petroleum reservoirs
Engineering Applications of Artificial Intelligence
(2001) - et al.
Support vector machine-based text detection in digital video
Pattern Recognition
(2001) - et al.
On the capability of support vector machines to classify lithology
Natural Resources Research
(2010) - et al.
Support vector regression for permeability prediction in a heterogeneous reservoir: a comparative study
SPE Reservoir Evaluation and Engineering
(2010) - et al.
Selecting of the loss function for robust linear regression
Neural Computation
(2004) - et al.
Learning from Data. Concepts, Theory, and Methods
(2007) - Choisy, C., Belaid, A., 2001. Handwriting recognition using local methods for normalization and global methods for...