Elsevier

Computers & Geosciences

Volume 39, February 2012, Pages 64-76
Computers & Geosciences

Support vector regression to predict porosity and permeability: Effect of sample size

https://doi.org/10.1016/j.cageo.2011.06.011Get rights and content

Abstract

Porosity and permeability are key petrophysical parameters obtained from laboratory core analysis. Cores, obtained from drilled wells, are often few in number for most oil and gas fields. Porosity and permeability correlations based on conventional techniques such as linear regression or neural networks trained with core and geophysical logs suffer poor generalization to wells with only geophysical logs. The generalization problem of correlation models often becomes pronounced when the training sample size is small. This is attributed to the underlying assumption that conventional techniques employing the empirical risk minimization (ERM) inductive principle converge asymptotically to the true risk values as the number of samples increases. In small sample size estimation problems, the available training samples must span the complexity of the parameter space so that the model is able both to match the available training samples reasonably well and to generalize to new data. This is achieved using the structural risk minimization (SRM) inductive principle by matching the capability of the model to the available training data. One method that uses SRM is support vector regression (SVR) network. In this research, the capability of SVR to predict porosity and permeability in a heterogeneous sandstone reservoir under the effect of small sample size is evaluated. Particularly, the impact of Vapnik's ε-insensitivity loss function and least-modulus loss function on generalization performance was empirically investigated. The results are compared to the multilayer perception (MLP) neural network, a widely used regression method, which operates under the ERM principle. The mean square error and correlation coefficients were used to measure the quality of predictions. The results demonstrate that SVR yields consistently better predictions of the porosity and permeability with small sample size than the MLP method. Also, the performance of SVR depends on both kernel function type and loss functions used.

Highlights

► In this research, the capability of SVR to predict porosity and permeability in a heterogeneous sandstone reservoir under the effect of small sample size is evaluated. ► Particularly, the impact of Vapnik's ε-insensitivity loss function and least-modulus loss function on generalization performance were empirically investigated. ► The results are compared to the Multilayer Perception (MLP) neural network, a widely used regression method, which operates under the ERM principle. ► The mean square error and correlation coefficients were used to measure the quality of predictions. ► The results demonstrate that SVR yields consistently better predictions of the porosity and permeability with small sample size than the MLP method.

Introduction

One of the most important tasks in modeling geoscience data is development of robust and accurate correlation models calibrated to small sample size problems. For permeability estimation, correlation models between porosity and permeability are often built based on the relationship between geophysical logs and core-measured porosity and permeability. In typical practice, core plugs are extracted from a few key wells during drilling whereas geophysical logs are run for all wells in the oil/gas field. The limited number of core plug data poses a challenging problem to existing empirical techniques that employ the empirical risk minimization (ERM) principle such as linear regression and neural networks. Statistical learning theory (SLT) shows that these techniques can be safely used as a measure of the true risk when the sample size is sufficiently large. SLT calls for introducing a structure to match the complexity of the predictive learning technique to the available training data. The structure is composed of elements of increasing complexity that need to be chosen to imitate the response of the learning problem using a limited number of data (Cherkassky and Mulier, 2007).

Artificial neural networks (ANNs) is a universal approximator that is capable of approximating any nonlinear function to any degree of accuracy provided that there are a sufficient number of neurons in the network (Hornik et al., 1989). The structure implemented by ANN may be captured by the number and size of the hidden layers, which are controlled explicitly by the user. This structure may also lead to an overfitting problem during learning, particularly in the presence of a small sample size, which potentially yields a poor generalization model. Although ANN has shown some successful applications to porosity and permeability (Helle and Ursin, 2001; Huang et al., 2001), the underlying learning algorithm has been developed for learning problems of large sample sizes. Hence, for a given small sample size, extensive experiments with several different learning techniques are required to devise an accurate ANN-based regression model (Kaviani et al., 2008).

Recently, support vector machines (SVMs) have been gaining popularity in regression and classification due to their excellent generalization performance. The SVM approach has been successfully applied to several different applications such as face recognition, object detection, handwriting recognition, text detection, speech recognition and prediction, lithology identification, and porosity and permeability determination from log data (Li et al., 2000, Lu et al., 2001, Choisy and Belaid, 2001, Gao et al., 2001, Kim et al., 2001, Ma et al., 2001, Van Gestel et al., 2001, Al-Anazi and Gates, 2010a, Al-Anazi and Gates, 2010b, Al-Anazi and Gates, 2010c, Al-Anazi and Gates, 2010d). The SVM formulation is based on the structural risk minimization (SRM) inductive principle where the empirical risk minimization (ERM) inductive principle and the Vapnik and Chervonenkis (VC) confidence interval are simultaneously minimized (Vapnik and Chervonenkis, 1974, Vapnik, 1982, Vapnik, 1995). The SRM principle introduces a structure where each element of the structure is indexed by a measure of complexity defined by the margin size between two classes in a classification learning problem and by an insensitivity zone size in a regression problem (Cherkassky and Mulier, 2007). The SVM optimization formulation implicitly matches a suitable structure of certain complexity to the available small size sample. This type of structure is controlled independently of the dimension of the problem, which is an advantage over classical learning techniques. In regression applications, the empirical error (the training error) is minimized by Vapnik's ε-insensitivity loss function rather than the quadratic error and absolute-value loss functions used in neural networks and classical regression methods. To generalize to nonlinear regression, kernel functions are used to project the input space into a feature space where a linear or nearly linear regression hypersurface results. A regularization term is used to determine a trade-off between the training error and the VC confidence term. The learning problem is formulated as a constrained convex optimization problem whose solution is used to construct the mapping function between the empirical input and the output data (Kecman, 2005).

Previously, our research demonstrated the generalization capability of SVM in lithology classification and porosity and permeability predictions with sensitivity analysis of kernel function types and SVM regularization parameters. In this research, however, the sensitivity of the SVM-based prediction of porosity and permeability to sample size and empirical loss functions are examined and compared to a multilayer perceptron network. The empirical evaluation of the generalization performance under small sample setting is conducted for two loss functions: first, the ε-insensitivity loss function, and second, the least-modulus (or absolute value) loss function.

Section snippets

Multilayer perceptron neural network model

ANN has been frequently used as an intelligent regression technique in petrophysical properties estimation (Rogers et al., 1995, Huang et al., 1996, Huang et al., 2001, Fung et al., 1997, Helle and Ursin, 2001, Helle and Bhatt, 2002). Backpropagation multilayer perceptron neural networks are considered to be universal approximators: it has been mathematically proven that a network with a hidden layer of an arbitrary large number of nonlinear neurons can approximate any continuous nonlinear

Multilayer perceptron neural network

In MLP neural networks, the topology of the networks consists of input layer, single hidden layer, and output layer. The size of the hidden layer is measured by the number of processing neurons where a sigmoid kernel function is used. The size of the hidden layer determines the complexity of the regression model. Increasing the complexity may lead to overfitting the training data and lead in turn to potentially poor generalization to testing data. Therefore, the size of the hidden layer must be

Reservoir description

The raw input data were obtained from three wells completed in a heterogeneous sandstone oil reservoir deposited in a fluvial dominated deltaic environment of Middle Cretaceous Albian age. The reservoir, consisting of a sequence of sandstone, siltstone, and shale with thin intervals of limestone, coal, and varying amounts of ironstone, can be divided into three main lithologic units. The first one is mainly shale and sands that are thinly interbedded with small amount of siltstones, green mud,

Small sample size testing procedure

The modeling data were prepared as follows:

  • 1.

    The data were randomly subdivided into divisions as percentages of the data as shown in Table 2 in nine cases.

  • 2.

    A total of 50 realizations were randomly generated for each case. Since the regression model itself depends on a particular (random) realization of a fixed size training sample, its estimated prediction accuracy is also a random variable. Therefore, the experiment calls for repeating the experimental procedure with different realizations of

Small sample size analysis with the ε-insensitivity loss function

To analyze the results, notched box plots of the mean squared error from the Gaussian-based SVR (SVR–RBF), sigmoid-based SVR (SVR-Sigmoid), and MLP models are compared. The notched box plot is an exploratory data analysis tool that provides statistical summaries of the underlying prediction error distribution. The tops and bottoms of box represent the 25th and 75th percentiles of the error, respectively. The red line in each box is the median of the error. The whiskers are the lines that extend

Small sample size analysis with least-modulus loss function

Although it was demonstrated that the prediction accuracy of the SVM using ε-insensitivity loss function outperforms ordinary least-squares and least-modulus methods for linear regression problems under small size high-dimensional datasets (Cherkassky and Ma, 2004), it would be of practical importance to investigate the impact of different loss functions on prediction accuracy of nonlinear SVM regression with RBF Gaussian and sigmoid kernel functions. As previously stated, SVM simultaneously

Conclusions

The capability of support vector regression to model porosity and permeability in a heterogeneous reservoir with finite sample size has been investigated. The result of the prediction of SVM models with Gaussian radial basis and sigmoid kernel functions have been compared to multilayer perceptron neural networks. The results reveal that

  • 1.

    the SVM based on radial basis and sigmoid kernel functions is capable of modeling small datasets of porosity and permeability outperforms the MLP method;

  • 2.

    the SVM

References (31)

  • C. Fung et al.

    Modular artificial neural network for prediction of petrophysical properties from well log data

    IEEE Transactions on Instrumentation and Measurement

    (1997)
  • Gao, D., Zhou, J., Xin, L., 2001. SVM-based detection of moving vehicles for automatic traffic monitoring. In:...
  • T. Hastie et al.

    The Elements of Statistical Learning: Data Mining, Inference, and Prediction

    (2001)
  • H. Helle et al.

    Porosity and permeability prediction from wireline logs using artificial neural networks: a North Sea case study

    Geophysical Prospecting

    (2001)
  • H. Helle et al.

    Fluid saturation from well logs using committee neural networks

    Petroleum Geoscience

    (2002)
  • Cited by (0)

    View full text