Entropy-Based Parameter Estimation in Hydrology

Clausius coined the term ‘entropy’ from the Greek meaning transformation. Thus, entropy originated in physics and occupies an exceptional position among physical quantities. It does not appear in the fundamental equations of motion. Its nature is, rather, a statistical or probabilistic one, for it can be interpreted as a measure of the amount of chaos within a quantum mechanical mixed state. It is an extensive property like mass, energy, volume, momentum, charge, number of atoms of chemical species, etc., but, unlike these quantities, it does not obey a conservation law. Entropy is not an observable; rather it is a function of state. For example, if the state is described by the density matrix, its entropy is given by the van Neumann formula. In physical sciences, entropy relates macroscopic and microscopic aspects of nature and determines the behavior of macroscopic systems in equilibrium.

There is a multitude of methods for estimating parameters of hydrologic frequency models. Some of the popular methods used in hydrology include (1) method of moments (Nash, 1959; Dooge, 1973; Harley, 1967; O’Meara, 1968; Van de Nes and Hendriks, 1971; Singh, 1988); (2) method of probability weighted moments (Greenwood, et al., 1979); (3) method of mixed moments (Rao, 1980, 1983; Shrader, et al., 1981); (4) L-moments (Hosking, 1986, 1990, 1992); (5) maximum likelihood estimation (Douglas, et al., 1976; Sorooshian, et al., 1983; Phien and Jivajirajah, 1984); and (6) least squares method (Jones, 1971; Snyder, 1972; Bree, 1978a, 1978b). A brief review of these methods is given here.

Uniform distribution is the simplest statistical distribution. Although there is hardly any hydrologic variable that follows a uniform probability distribution, it is invoked in a variety of applications. For example, in Bayesian statistical modeling in hydrology it is frequently used as a prior distribution. In systems hydrology, uniform distribution is the pulse function obtained by subtracting two step functions lagged by the length of the uniform distribution. The pulse function is a key to deriving the unit hydrograph theory. The instantaneous unit hydrograph of the rational method, used in urban hydrology, is a uniform distribution (Singh, 1988). Of all the statistical distributions, uniform distribution has the highest entropy. In river morphology, when a river approaches equilibrium or dynamic equilibrium, its characteristics tend to follow a uniform distribution. Under equilibrium, rivers follow the minimum rate of energy dissipation. Furthermore, a river constantly adjusts its cross-sectional geometry and longitudinal profile to accommodate the influx of water and sediment coming from its drainage basin, and this adjustment is in accordance with the principle of maximum entropy. Thus, there is a close link between equilibrium and uniform distribution and then between maximum entropy (uniform distribution) and minimum rate of energy dissipation. This link plays a fundamental role in river engineering and training works, river morphology, evolution of deltas, etc.

The exponential distribution is a basic distribution for constructing a number of other distributions. For example, the gamma distribution is obtained from the distribution of the sum of random variables where each variable follows an exponential distribution. Indeed, it is the simplest member of the gamma family of distributions and can be considered as a special case of the two-parameter gamma distribution. It is a one-parameter distribution and has found widespread application in hydrology and water resources. The instantaneous unit hydrograph of a linear reservoir, frequently used in systems hydrology, is exponential (Singh, 1988). The exponential distribution is often used for frequency analysis of rainfall depth, intensity and duration, and number of rainfall events (Eagleson, 1982). It is frequently used in biology, genetics, quantum mechanics, reliability engineering, to name but a few.

The normal distribution is probably the most popular statistical distribution. It is also known as the Gaussian distribution or error function. Many statistical parameters are found to be approximately normally distributed; therefore, the normal distribution is often used for statistical inferences. A variety of natural phenomena either approximately follow a normal distribution or can be transformed to follow a normal distribution. One of the earliest applications of the normal distribution in hydrology was made by Hazen (1914), who introduced the normal probability paper for ananlysis of hydrologic data. Markovic (1965) fitted the normal distribution to annual rainfall and runoff data Slack et al. (1975) showed that when the information about the distribution of floods and economic losses associated with the design of flood retardation structures was lacking, it was better to use the normal distribution than other distributions such as extreme value, lognormal, Weibull, etc. The other advantages of the normal distribution are that it is extensively tabulated and the standardized normal variate is the same as the frequency factor.

The logarithmic normal probability law is widely used to describe the distribution of annual maximum values of hourly or daily precipitation (Weiss, 1957), flood flows (Chow, 1951, 1954), hydraulic conductivity (Freeze and Cherry, 1979), soil properties (physical, chemical and microbiological) (Parkin and Robinson, 1993), etc. Kalinske (1946) found that many times river discharge data and sand sizes followed the normal law if they were logarithmically transformed. Chow (1951, 1954, 1959) gave a historical background of the log-probability law and discussed its wide-ranging application in engineering, and exensively worked with the lognormal distribution. Aitchison and Brown (1957) presented a comprehensive statistical treatment of the lognormal distribution. Parkin et al. (1988) evaluated statistical methods for log-normally distributed variables, including the method of moments, maximum likelihood, and Finney’s method. Parkin and Robinson (1993) evaluated soil properties using log-normal distribution. Brakensiek (1958) employed the least squares method for fitting the log-normal distribution to annual runoff. Moran (1957) fitted a log-normal distribution to fifty annual values of extreme monthly flow of the River Murray in Australia. Lewis (1979) applied log-normal distribution to maximum measured discharges of River Kafue in Africa.Weiss (1957) developed a nomogram for log-normal frequency analysis.Alexander et al. (1969) discussed statistical properties of lognormal distribution. Using mean square error of estimation as a criterion, Stedinger (1980) evaluated the efficiency of alternative methods of fitting the lognormal distribution. Charbeneau (1978) compared two- and three-parameter log-normal distributions for simulation of stream flow.

The three-parameter lognormal (TPLN)distribution is frequently used in hydrologic analysis of extreme floods, seasonal flow volumes, duration curves for daily streamflow, rainfall intensity-duration, soil water retention, etc. It is also popular in synthetic streamflow generation. Properties of this distribution are discussed by Aitchison and Brown (1957), and Johnson and Kotz (1970). Its applications are discussed by Slade (1936), Chow (1954), Matalas (1967), Sangal and Biswas (1970), Fiering and Jackson (1971), Snyder and Wallace (1974), Burges et al. (1975), Burges and hoshi (1978), Charbeneau (1978), Stedinger (1980), Singh and Singh (1987), Kosugi (1994), among others. Burges et al. (1975) discussed properties of the three-parameter lognormal distribution and compared two methods of estimation of the third parameter “a”. Kosugi (1994) applied the three-parameter lognormal distribution to the pore radius distribution function and to the water capacity function which was taken to be the pore capillary distribution function. He found that three parameters were closely related to the statistics of the pore capillary pressure distribution function, including the bubbling pressure, the mode of capillary pressure, and the standard deviation of transformed capillary distribution function. Burges and Hoshi (1978) proposed approximating the normal populations with 3-parameter lognormal distributions to facilitate multivariate hydrologic disaggregation or generation schemes in cases where mixed normal and lognormal populations existed.

The extreme value type 1 (EV 1) distribution is one of the most popularly used distributions for frequency analysis of extreme values of meteorologic or climatic and hydrologic variables, such as floods, rainfall, droughts, etc. This distribution was derived by Fisher and Tippett (1928) as a limiting form of the frequency distribution of the largest or smallest of a sample. In a series of papers Gumbel (1941a, b, 1942a, b, 1948) derived the EV1 distribution for flood flows and applied it to frequency analysis of floods, droughts, and meteorological data. Gumbel (1958) published a treatise on statistics of extremes, which contains a comprehensive treatment of EV1 distribution. Bardsley and Manly (1987) examined the transformations under which non-Gumbel distributions of annual flood flow maxima would converge to the Gumbel distribution. Smith (1986) presented a family of statistical distributions and estimators based on a fixed number (greater than one) of the largest annual events. Jenkinson (1955) found a general solution of the function equation derived by Fisher and Tippett (1928) for extreme values and showed that the Gumbel distribution was a special case of the general solution. Singh et al. (1986) derived this distribution using the principle of maximum entropy. Al-Mashidini et al. (1978) presented a simplified form of EV 1 distribution for flood estimation.

The logarithmically transformed extreme value type 1 (LEV 1) distribution is the log-Gumbel distribution. The logarithmic version of EV 1 distribution is not as popular as the original EV 1 distribution. Reich (1970) employed log-Gumbel distribution to analyze annual series of maximum instantaneous flood peaks from 26 Pennsylvanian watersheds smaller than 200 square miles in area. He found consistently overestimation of long-period extremes from use of the log-Gumbel distribution. Using the principle of maximum entropy Singh (1985) derived the log-Gumbel distribution and its parameters. Heo and Salas (1996) estimated quantiles and confidence intervals for the log-Gumbel distribution. They used the methods of moments, maximum likelihood and probability weighted moments for parameter estimation.

The extreme value type (EV) III distribution has been employed for frequency analysis of low river flows (Gumbel, 1963; Matalas, 1963; Condie and Nix, 1975; Kite, 1978; Loganathan et al., 1985). Otten and Van Montefort (1978) discussed tests for the EV distributions. Gumbel (1963) estimated the EV III parameters using the method of moments (MOM). Matalas (1963) estimated them using MOM and the method of maximum likelihood estimation (MLE). Condie and Nix (1975) also used MOM and MLE. Kite (1978) described both MLE and MOM for the EV III distribution Singh (1987) employed the principle of maximum entropy (POME) to estimate the EV III parameters and compared it with MOM and MLE.

The generalized extreme-value (GEV) distribution was introduced by Jenkinson (1955, 1969) and recommended by Natural Environment Research Council (1975) of Great Britain. The GEV distribution is the most widely accepted distribution for describing flood frequency data from the United Kingdom (Sinclair and Ahmad, 1988) and has also become popular elsewhere (Otten and van Montfort, 1980; Prescott and Walden, 1980, 1983; Turkman, 1985; Hosking et al., 1985; Arne11 et al., 1986). Sinclair and Ahmad (1988) introduced location-invariance in the context of using plotting positions in estimating parameters of the GEV distribution by the method of probability-weighted moments. They emphasized that this was an important factor in the selection of an appropriate plotting position, for otherwise the estimate of the shape parameter might not be independent of location. Tawn (1988) presented a method of filtering the original time series containing dependent data to obtain independent extremes. He then used the limiting joint generalized extreme value distribution for the r largest order statistics.

The Weibull distribution is commonly used for frequency analysis as well as risk and reliability analysis of the life times of systems and their components. Its applications have been reported frequently in hydrology and meteorology. Grace and Eagleson (1966) fitted this distribution to the wet and dry sequences and obtained satisfactory results. Rao and Chenchayya (1974) applied it to short-term increment urban precipitation characteristics in various parts of the U.S.A. and obtained satisfactory fit to the durations of wet and dry periods as well as other characteristics. Singh (1987) derived the Webull distribution and estimated its parameters using the principle of maximum entropy (POME). For the precipitation data used, he found POME-based parameter estimates to be either superior or at least comparable to those obtained with the methods of moments and maximum likelihood. Nathan and McMahon (1990) considered some practical aspects concerning the application of the Weibull distribution to low-flow frequency analysis on 134 catchments located in southeastern Australia. They examined the relative performance of the methods of moments, maximum likelihood, and probability weighted moments. They found that different estimation methods provided distinct sets of quantile estimates and the differences between estimation methods decreased as the sample size increased. While fitting the Weibull distribution to annual minimum low flows of different durations, Polarski (1989) found that occasionally the frequency distributions for different durations crossed, in which case the distribution parameters were constrained by adding to the likelihood function the conditions to prevent the curves from crossing. Vogel and Kroll (1989) developed probability-plot correlation coefficient (PPCC) tests for the Weibull distribution. He then used PPCC tests to discriminate among both competing distributional hypotheses for the distribution of fixed shape and competing parameter estimation methods for distributions with variable shape. Durrans (1996) applied the Weibull distribution to obtain estimates of low-flow quantiles, such as 7-day, 10-year low flow. For developing a stochastic flood model Eknayake and Cruise (1993) compared Weibull and exponentially-based models for flood exceedances. They found that the Weibullbased model possessed predictive properties to those of the exponential model when samples exhibited coefficients of variation less than 1.5 and sample sizes were on the order of two events per year. Using Monte Carlo simulation, Singh et al. (1990) made a comparative evaluation of different estimators of the Weibull distribution parameters, including the methods of Moments, probability-weighted moments, maximum likelihood (MLE), least squares, and POME, with the objective of identifying the most robust estimator. Their analysis showed that MLE and POME demonstrated the most robustness.

The two-parameter gamma distribution is commonly employed for synthesis of instantaneous or finite-period unit hydrographs (Dooge, 1973) and also for flood frequency analysis (Haan, 1977; Phien and Jivajirajah, 1984; Yevjevich and Obseysekera, 1984). By making two hydrologic postulates, Edson (1951) was perhaps the first to derive it for describing a unit hydrograph (UH). Using the theory of linear systems Nash (1957, 1959, 1960) showed that the mathematical equation of the instantaneous unit hydrograph (IUH) of a basin represented by a cascade of equal linear reservoirs would be a gamma distribution. This also resulted as a special case of the general unit hydrograph theory developed by Dooge (1959). On the other hand, using statistical and mathematical reasoning, Lienhard and associates (Lienhard, 1964; Lienhard and Davis, 1971; Lienhard and Meyer, 1967) derived this distribution as a basis for describing the IUH. Thus, these investigators laid the foundation of a hydrophysical basis underlying the use of this distribution in synthesizing the direct runoff. There has since been a plethora of studies employing this distribution in surface water hydrology (Gray, 1961; Wu, 1963; DeCoursey, 1966; Dooge, 1973; Gupta and Moin, 1974; Gupta, et al., 1974; Croley, 1980; Aron and White, 1982; Singh, 1982a, 1982b, 1988; Collins, 1983).

The Pearson type (PT) III distribution is the generalized gamma distribution and is one of the most popular distributions for hydrologic frequency analysis. Bobee and Robitaille (1977) compared PT III and log PT III distributions using several long-term records of annual flood flows and found PT III distribution to be preferable, especially when the method of moments (MOM) was applied to observed sample data. Bobee (1973), Chang and Moore (1983), among others, used it for flood frequency analysis. Markovic found practically no difference in fitting of Pearson and lognormal distributions to annual precipitation and runoff data. Matalas (1963) found PT III distribution to be representative of low flows. Obeyesekera and Yevjevich (1985) presented a procedure for generation of samples of an autoregressive scheme that has an exact Pearson type III distribution with given mean, variance and skewness. Harter (1958) prepared tables for percentage points of the PT III distribution. Wilk et al. (1962) described a procedure for preparing probability plots for randon samples from an assumed PT III distribution Haktanir (1991) developed a practical method for computation of PT III frequency factors. Shaligram and Lele (1978) analyzed hydrologic data using PT III distribution and showed that the confidence intervals for this distribution were larger than for the Gumbel distribution.

The log-Pearson type 3 (LP3) distribution has been one of the most frequently used distributions for hydrologic frequency analyses since the recommendation of the Water Resources Council (1967, 1982) of the United States as to its use as the base method. The Water Resources Council also recommended that this distribution be fitted to sample data by using mean, standard deviation and coefficient of skewness of the logarithms of flow data [i.e., the method of moments (MOM)]. A large volume of literature on the LP3 distribution has since been published with regard to its accuracy and methods of fitting or parameter estimation. McMahon and Srikanthan (1981) and Srikanthan and McMahon (1981) examined the applicability of LP3 distribution to Australian rivers and questioned the assumption of setting to zero the coefficient of skewness of logarithms of peak discharges that were not statistically different from zero. They evaluated the effect of sample size, distribution parameters and dependence on peak annual flood estimates. Gupta and Deshpande (1974) applied LP3 distribution to evaluate design earthquake magnitudes. Phien and Jivajirajah (1984) applied LP3 distribution to annual maximum rainfall, annual streamflow and annual rainfall. Wallis and Wood (1985) found, based on Monte Carlo experiments, that the flood quantile estimates obtained by using an index flood type approach with either a generalized extreme value distribution or a Wakeby distribution fitted by PWM were superior to those obtained by LP3 distribution with MOM -based parameters. This finding was challenged later by several investigators (Beard, 1986; Landwehr et al., 1986).

In reliability safety analysis of civil engineering systems, we encounter parameters which are generally bounded and skewed random quantities. Exemplifying these parameters are factors of safety or safety indexes, variables representing strength of materials, intensity of loads, etc. Harr (1977) demonstrated the ability of the beta (or Pearson type 1) distribution to approximate most of the geotechnical parameters. Obini and Bourdeau (1985) simplified use of the beta distribution and investigated its sensitivity to the bound locations. Fielitz and Myers (1975) argued for the method of moments (MOM) to estimate the parameters of the beta distribution for ease of computation. Romesburg (1976) commented that formulation of the problem in terms of smallest order statistics would allow the use of the method of maximum likelihood estimation (MLE) to estimate the parameters of the beta distribution with little more effort than MOM. In multivariate cases, however, MOM would be the only practical method for parameter estimation.

The log-logistic distribution (LLD) is obtained by applying the logarithmic transformation to the logistic distribution (LD) in much the same way as the log-normal distribution is obtained from normal distribution or the log-Pearson distribution from the Pearson distribution. The log-logistic distribution is a special case of Burr’s type-XII distribution (Burr, 1942), and also a special case of the “Kappa distributions” (Mielke and Johnson, 1973), that have been applied to precipitation and streamflow data.

Some general aspects of the log-logistic distribution (LLD) are discussed in Chapter 17. The three-parameter log-logistic distribution (LLD3) is a generalization of the two-parameter log-logistic distribution. The LLD3 has been applied to frequency analysis of precipitation and streamflow data. Ahmad, et al. (1988) employed it for flood frequency analysis of annual maximum series for part of Scotland, and compared its performance with the generalized extreme value, three-parameter log-normal and Pearson-type 3 distributions. They found LLD3 to consistently perform better than these three distributions. In a comparative study on statistical modeling of annual maximum flows of 112 Turkish rivers, Haktanir (1991) observed that LLD3 and log-Pearson type 3 distribution provided better fits than log-normal, Gumbel, SMEMAX, log-Boughton, and Pearson-type 3 distributions. He concurred with the findings of Ahmad, et al. (1988) as far as Turkish rivers were concerned.

The Pareto distribution was introduced by Pickands (1975) and has since been applied to a number of areas including socio-economic phenomena, physical and biological processes (Saksena and Johnson, 1984), reliability studies and the analysis of environmental extremes. Davison and Smith (1990) pointed out that the Pareto distribution might form the basis of a broad modeling approach to high-level exceedances. DuMouchel (1983) applied it to estimate the stable index a to measure tail thickness, whereas Davison (1984a, 1984b) modeled contamination due to long-range atmospheric transport of radionuclides. van Montfort and Witter (1985, 1986, 1991) applied the Pareto distribution to model the peaks over threshold (POT) streamflows and rainfall series, and Smith (1984, 1987) applied it to analyze flood frequencies. Similarly, Joe (1987) employed it to estimate quantiles of the maximum of a set of observations. Wang (1991) applied it to develop a peak over threshold (POT) model for flood peaks with Poisson arrival time, whereas Rosbjerg et al. (1992) compared the use of the 2-parameter Pareto and exponential distributions as distribution models for exceedances with the parent distribution being a generalized Pareto distribution. In an extreme value analysis of the flow of Burbage Brook, Barrett (1992) used the Pareto distribution to model the POT flood series with Poisson interarrival times. Davison and Smith (1990) presented a comprehensive analysis of the extremes of data by use of the Pareto distribution for modeling the sizes and occurrences of exceedances over high thresholds.

The Pareto distribution has been introduced in Chapter 19. Also discussed in the chapter are a brief review of literature and the methods of estimating its parameters. Kotz and Johnson (1985) provided a detailed discusson of the Pareto distributin. Methods for estimating parameters of the 2-parameter generalized Pareto (GP2) distribution were reviewed by Hosking and Wallis (1987). The method of moments (MOM), maximum likelihood estimation (MLE), and probability weighted moments (PWM) were included in the review. Ashkar and Ouarda (1997) presented some methods of fitting the GP2 distribution using Monte Carlo generated data. They discussed six versions of the generalized method of moments. Wang (1991) derived PWMs for both known and unknown thresholds. van Montfort and Witter (1991) used the MLE method to fit the GP2 distribution to represent the Dutch POT rainfall series, and used an empirical correction formula to reduce bias of the scale and shape parameter estimates. Davison and Smith (1990) used MLE, PWM, and a graphical method to estimate the GP2 distribution parameters. Guo and Singh (1992) and Singh and Guo (1997) employed the principle of maximum entropy (POME) to derive a new method of parameter estimation (Singh and Rajagopal, 1986) for the GP2 distribution. They used Monte Carlo simulated data to evaluate this method and compare it with he MOM, PWM, and MLE methods. The parameter estimates yielded by POME were comparable or better within certain ranges of sample size and coefficient of variation.

The Pareto distribution has been introduced in Chapter 19. Also discussed there is a brief review of literature and methods of estimating its parameters. Further elaboration of the distribution is given in Chapter 20. Methods of parameter estimation were reviewed by Hosking and Wallis (1987). The methods of moments (MOM), maximum likelihood estimation (MLE) and probability weighted moments (PWM) were included in the review. Guo and Singh (1992) and Singh and Guo (1995) employed the principle of maximum entropy (POME) to develop a new competitive method of parameter estimation (Singh and Rajagopal, 1986) for the 3-parameter generalized Pareto (GP3) distribution and compared it with MOM, MLE and PWM using Monte Carlo simulated data The parameter estimates yielded by POME were either superior or comparable for high skewness.

It is well known that floods may be generated by different physical mechanisms. For instance, most of the annual flood maxima at a particular site might be the result of a primary mechanism, such as frontal storms. A smaller fraction of the events, however, might be associated with a secondary mechanism, such as rain on snow with frozen soils, that occasionally gives rise to floods larger than those associated with the primary mechanism. In this regard, Rossi et al. (1984) proposed a two-component extreme value distribution. This distribution belongs to the family of distributions of the annual maxima of a compound Poisson process, which forms a theoretical basis for annual flood series analysis. Single-component distribution methods of estimating return periods and probabilities of flood events do not work well when runoff originates from nonhomogeneous sources, i.e., when a mixture random variables is involved. The most important consideration in selecting a distribution for use in flood frequency analysis is the behavior of the right tail of the distribution. It is from the right tail that return periods and probabilities of rare events are determined. The two-component extreme value (TCEV) distribution permits a reasonable interpretation of the physical phenomenon which generates floods and is able to account for most of the characteristics of the real world flood data, important among them being the large variability of the sample skewness coefficient which mostly gives rise to the poor performance of many of the commonly used flood frequency distributions. The two component extreme value (TCEV) distribution has been shown to account for most of the characteristics of the real flood experience. The TCEV distribution also offers a practical approach to regional flood frequency estimation. Theoretical properties of the TCEV distribution have been widely investigated (Rossi, et al., 1984; Berm, et al., 1986; Rossi, et al., 1986; Fiorentino et al., 1987a, b). In his extensive review of a large number of commonly used distributions, Cunnane (1986) concluded that only the two-component extreme value distribution and the Wakeby distribution satisfied the important reproductive criterion--an ideal distribution must reproduce at least as much variability in flood characteristics as is observed in empirical data.