Neural networks meet least squares Monte Carlo at internal model data

We have undertaken the task of providing realistic data for the area of proxy modeling in risk management within the Solvency II context. To the best of our knowledge, so far no data based on a model actually implemented in the industry has been published. For our use case developed for the Actuarial Data Science Committee of the German Actuarial Association we have produced realistic data stemming from life and health cash flow projection models (CFPMs) and containing all the complexities of real-life productive models. Data are only scaled and fully anonymized. In the GitHub repository [15] we have uploaded data for two life and one health insurance portfolios, projected 60 years into the future in an actuarial CFPM.

The amount of samples we provide as well as the accuracy of actuarial valuations for our samples is higher than what we have seen in most internal models in the industry. We have done this intentionally in order to allow meaningful statistical and convergence analyses.

For years the authors have worked in the area of life and health insurance proxy function modeling. Since 2015 we have been developing and testing various machine learning methods for the risk capital approximation. For a comprehensive description of the framework of risk capital proxy modeling as well as various machine learning approaches that can be used for it, we refer the readers to [18‐20].

The regression task refers to the value of a portfolio of insurance policies, which must be evaluated in a Monte Carlo manner using stochastic simulations. An appropriate way of measuring errors for our framework is given in Sect. 3.

Our analyses so far are in line with Krah et al. [19] and indicate that neural networks combined with the hyperparameter tuning and an ensemble approach yield very accurate approximation results. This demonstrates anew the versatility of neural networks and their virtually universal application possibilities.

In the original publication of our data we have presented a rudimentary Jupyter notebook [15] showcasing a simple least squares regression for a given set of polynomial terms as well as a neural network training with a pre-defined architecture of the neural network in terms of e. g. the number of layers, nodes and activation functions used. In this paper we explain these two methods in more detail. Besides that, we disclose our best regression results as an invitation for the actuarial data science community to try to improve them. Besides describing the approach that has resulted in the high approximation quality, we focus on an efficient implementation of neural networks. By doing this, we recognize the fact that the computational resources in companies are limited, even in the times when many companies move such calculations to the cloud.

The least squares Monte Carlo (LSMC) framework, which relies on data as ours, is described and formally introduced in Krah et al. [18]. Our polynomial proxy function fitting follows the approach described in this article, that is the adaptive Akaike information criterion (AIC)-based algorithm. Our implementation of neural networks follows Krah et al. [19]. The analysis of an efficient approach using less data in the last part of this article is specifically produced for this article.

An overview of several proxy methods can be found in Kopczyk [16]. Neural networks and random forests outperform the linear methods for the data analyzed in this article. The result for neural networks is in line with our results.

The seminal paper for the introduction of the LSMC method is Longstaff and Schwartz [21], where it was proposed for pricing of American options. The transfer to the insurance industry happened some 10 years later; one of the first descriptions of the new application was in Koursaris [17].

Recently, in Castellani et al. [5] several methods were compared and again deep neural networks have demonstrated the best performance.

An application of neural networks for an valuation of variable annuities with guaranteed minimum income benefit with three risk factors is included in Cheridito et al. [6], the example follows the previous work with a classical LSMC solution in Bauer and Ha [2].

The remainder of this article is structured as follows: in Sect. 2, we briefly motivate the data analysis problem associated to the solvency capital requirement (SCR) calculation, which is the starting point for our use case. The underlying data set of the use case is then described in detail in Sect. 3. This not only serves to better understand the results presented in the following, but should also facilitate application and interpretation by inclined users. Finally, the data set is analyzed using different models, which are briefly introduced in Sect. 4. The numerical results of this analysis and its interpretation are discussed in detail in Sect. 5, and in Sect. 6 we conclude our use case.

It is well known that Solvency II aims at implementing a set of robust solvency rules for insurance companies, which takes the most material risks into account in an adequate way. One of the key concepts is hereby the calculation of the SCR.

According to Article 101 (3) of Solvency II Directive 2009/138/EC (see European Parliament and European Council [8])

In the following we focus our attention on internal models.

To start with, let us consider the functionality of an internal model for calculating the SCR. As motivated above, the SCR derivation relies on the re-evaluation of the economic balance sheet in multiple risk scenarios as displayed in Fig. 1.

As introduced in Bauer and Ha [2] technically these risk scenarios can be thought of as the realization of a multi-dimensional state process under the physical measure (aka real-world measure). This state process represents the uncertain development of influencing factors (risk factors) of the company’s assets and liabilities. These risks are typically split into classes such as market, credit or underwriting risks; operational risk might be a further risk class. The market consistent re-evaluation of balance sheet items is then formally associated with the derivation of the conditional expectation of future cash flows under the risk-neutral measure. The liability items of the economic balance sheet of life and health insurers are driven by options, asymmetries and complex dependencies which do not allow for a simple closed-form solution. Therefore, the valuation is typically carried out by involving Monte Carlo methods based on an actuarial CFPM, which determines the distribution of future cash flows along risk-neutral valuation paths between all stakeholders, like policy holders, company’s employees and shareholders.

As illustrated in Fig. 2 a logical approach is to use a simulation within simulation technique; this nested simulation approach is also often applied in the field of financial mathematics, see e.g. Glasserman [11] or Andersen and Broadie [1]. This requires the generation of a large number of risk scenarios (also called outer scenarios) under the real-world measure; in doing so, the dependencies between the various risk factors have to be reflected in an appropriate way. In each of those outer scenarios the re-evaluation of the balance sheet is then carried out by another simulation based on risk-neutral valuation scenarios (also called inner scenarios) that are fed into the CFPM. Statistically, the more simulations are generated the more accurate the approximation of the OF distribution is. This approach is attractive due to its simplicity, but leads to exponentially growing computational efforts and is therefore expensive; moreover, high storage capacities are necessary. Thus, we move on with more efficient methods which are preferred in industry.

The state-of-the-art answer to avoid the time-consuming nested simulations approach are regression-based methods; these methods need significantly lower storage and computational time. There has already been a few publications which have dealt with this problem, see for instance Bettels et al. [3], Hejazi and Jackson [14], Nikolić et al. [22], Fernandez-Arjona [9], Krah et al. [19, 20].² The key idea is to define an appropriate fitting space, denoted by \(I_{1}\times \cdots \times I_{D}\) in the following, which is given by the intervals \(I_{d}\ni RF_{d}\) for the corresponding D risk factors \(RF_1, \ldots , RF_D\). Then, only a low number of inner scenarios are generated for each outer scenario from the cube \(I_{1}\times \cdots \times I_{D}\). The inaccurate mean of the low number of inner scenarios leads to an inaccurate but unbiased fitting point for each outer scenario, which is interpreted as a rough estimation of a balance sheet position. In the following we focus on the balance sheet item OF.³

Based on this approach we receive a point cloud of OF estimations, such that the ultimate task is to find a sound functional relationship between the risk factors and OF. Let us call f this functional relationship which represents the approximation \({\widehat{OF}}\) of the true OF. Then, for \(n=1, \ldots , N\), denoting the estimated OF samples and the simulated risk factor samples (outer scenarios) by \(OF_n\) and \(RF_{1n}, \ldots , RF_{Dn}\), respectively, we are able to determine the function f via least squares, i.e. minimizingAn exemplary visualisation of an OF surface for two risk factors is shown in Fig. 3.

The focus of this use case is on the application of neural networks for regression. By doing so, we want to challenge the ordinary least squares approach—working with polynomials—and, based on our comprehensive data set, an adequate comparative analysis is guaranteed. In the sense of open data science our data set can be used by the interested readers for performing own meaningful studies.

Coming back to the regression problem (1) a simple, yet powerful model function f for approximating the OF is given by a linear combination of basis functions \(\{\phi _m(\cdot )\}^M_{m=1}\) with coefficients \(a_m\in {\mathbb {R}}\):For what concerns the basis functions, we are free in the selection but polynomials or radial functions have shown well-working approximation quality; using e.g. monomials as basis functions allows for a direct interpretation of the functional shape with respect to several risk factors, whereas the constant term is the base case value and the residual term shows the stress behavior for any evaluated scenario. Then, in order to determine the function f the corresponding optimization problem can be solved by numerically stable methods such as the QR or singular value decomposition, see Golub and van Loan [12]. The advantage of this approach is clearly its simplicity, good results interpretation and an easy implementation as a number of standard software packages provide solvers for deriving the coefficients of the assumed proxy model. This approach goes under the name LSMC, called LSMC-Pol in the following, where Pol stands for polynomial functions.

Lately the neural networks have become ubiquitous in the scientific and practical publications. The attempts to describe neural networks in mathematical terms have proved to be somehow tedious. This comes primarily from the notational complexity, not from a conceptual difficulty in explaining neural networks as mathematical functions.

Often the natural model, that is the brain, is used as a motivation for an introduction of neural networks. Although there are some striking analogies between “natural” learning and its “machine” counterpart,⁵ this perspective is not really useful for a mathematical understanding of the underlying functions and algorithms.

We do not intend to fully introduce the concept of neural networks. Instead, we clarify the terminology, which we employ, and focus our attention on the features, which are relevant for our application of neural networks.

Let \(N_0, \ldots ,N_{L+1}\in {\mathbb {N}}\). Following Definition 1 and Definition 2 in Krah et al. [19], a fully connected feed forward neural network with \(L \in {\mathbb {N}}\) hidden layers is a function \(f : {\mathbb {R}}^{N_{0}} \rightarrow {\mathbb {R}}^{N_{L+1}}\) defined aswhere \(\circ\) denotes the concatenation, andare affine mappings represented by matrices \(W_l\) of dimension \(N_{l-1} \times N_l\) and vectors \(b_l \in {\mathbb {R}}^{N_l}\). \(N_0\) is the input dimension, \(N_{L+1}\) the output dimension and \(N_l\) the number of neurons or nodes in the hidden layer l. For each \(l \in \{1, \ldots , L+1\}\), the functions \(\Phi _l\) are defined aswith real-valued functions \(\phi _l: {\mathbb {R}} \rightarrow {\mathbb {R}}\), which are called activation functions of the neural network and for which it is required to be monotone and Lipschitz continuous (see Definition 1 in Krah et al. [19]⁶). In our use case all \(\phi _l, l \in \{1, \ldots , L \}\), are equal and denoted by \(\phi\). Only the function \(\phi _{L+1}\), which is called output activation function, may differ from \(\phi\). Moreover, in our application we also set \(N_1 = \cdots = N_L\), that is the number of neurons in each hidden layer is the same.

In the numerical results in Sect. 5 we use the following activation functions:The functions \(a_l\) with \(l \in \{1, \ldots , L+1\}\) are affine transformations between the layers of the neural network and the parameters of the affine transformations are weights between the nodes of the layers. The change of these weights by an algorithm constitutes the actual training for given input and output data.

For instance, for Portfolio 1 with 13 risk factors \(RF_1,\ldots ,RF_{13}\) (denoted by x), the input dimension is \(N_0 = 13\) and the first affine transformation iswith a \(13 \times N_1\) matrix \(W_1\) and a vector \(b_1 \in {\mathbb {R}}^{N_1}\). Hence, \(a_1\) maps the 13-dimensional space of risk factors to the first layer consisting of \(N_1\) neurons.

As already noted above, in our applications \(N_1 = \cdots = N_L\) and \(N_{L+1} = 1\) as we work with the same number of nodes for all hidden layers and our data have only one output variable.

The last affine transformation takes the form \(a_{L+1} : {\mathbb {R}}^{N_L} \rightarrow {\mathbb {R}}\), so it maps the outputs of the nodes in the last hidden layer to a real number. After a transformation with the output activation function \(\phi _{L+1}\) we obtain an estimate for the Own Funds \({\widehat{OF}}(x) = f(x) \in {\mathbb {R}}\), where the input x represents the input vector of the risk factors \(RF_1,\ldots ,RF_D\).

Notify that the vector x is 13-dimensional for Portfolios 1 and 2, and 12-dimensional for Portfolio 3. For a more thorough description of the risk factors which may represent the input data, we refer to Table 1 in Krah et al. [18]. We do not disclose the exact meaning of the input data, i.e. what each risk factor represents, as we do not want to reveal details about exact implementations in the industry. Our application with neural networks is illustrated in Fig. 5.

For a more comprehensive introduction to neural networks, also including many other possible architectures, we refer the readers to Goodfellow et al. [13] or Efron and Hastie [7]. In this paper we only refer to fully connected feed forward neural networks.

In the course of the preparation and publication of our data we have developed various machine learning models. The most promising models have been developed by neural networks. After testing various options we have defined a rich set of 300 quasi-randomly selected combinations of hyperparameters⁷: In total we have performed more than 800 different pre-training tests with neural networks. Even after keeping all but one hyperparameters fixed, the variation of the results when varying the remaining hyperparameter was considerably high. Most results did not show a completely clear dependency of the approximation quality (measured in sum of errors and sum of squared errors for OOS validation and SCR region data) from the changes in a hyperparameter. For instance, we did not observe that increasing the drop out rate always leads to a better result or vice versa. There were some tendencies, which we explain below when we define the efficient set of hyperparameters.

For each of the hyperparameters listed above we have derived the sets from which to choose the hyperparameter values (e.g. from 2 to 10 for the number of hidden layers) by performing these pre-training tests in which we have allowed much larger ranges for all hyperparameters. To give an example, in this pre-training stage we have trained neural networks with the number of hidden layers from 1 until 11. After completing the training we have plotted the out-of-sample results depending only on one hyperparameter, i.e. here depending only on the number of hidden layers. The results have shown that 1 hidden layer leads to notably lower accuracy and the same holds for 11 hidden layers. As a consequence we have selected [2, 10] as the range for the number of hidden layers. Furthermore, in each training we have allowed a new random seed. Since the training of neural networks converges towards a local minimum we should always keep in mind that another set of initial weights for the optimization method might lead to another model. We take this into account by working with randomly selected values for the initial weights.

Working with LSMC-Pol the toolbox is much simpler. Instead of varying several hyperparameters we employ the forward adaptive step-wise algorithm which gradually builds up a polynomial adding terms of monomial basis functions. For the model selection as well as the stopping criterion we use the AIC where we restrict the number of admissible terms to the number of 150 and the maximum polynomial degree to eight for both, the single and cross-terms, albeit the highest degree chosen by the algorithm was six. For a detailed explanation of the procedure, we refer the readers to Krah et al. [20]. We refer to the results derived by this benchmark technique as LSMC-Pol2 in the following. For the sake of comparability, we additionally report results from an even simpler model, namely a simple model consisting only of linear terms \(RF_{d}\), denoted by LSMC-Pol1.

With reference to the approach outlined above it is worth noting that one might try deriving LSMC proxy functions with higher quality, but we think that the goodness of the selected polynomials is satisfactory. In particular, we do not want to explore the limits of the LSMC approach. Rather, we investigate whether an alternative technique can reliably beat the goodness of an easy-to-implement existing approach, such as the LSMC-Pol approach.

Let us focus on the results of our first trial, in which we run a number of neural networks based on the previously mentioned comprehensive set of hyperparameters.

For all three portfolios, Fig. 6 shows box plots of the mean errors (ME) calculated for the OOS validation and SCR region points.⁸ We clearly see a huge variety of accuracy: driven by the choice of hyperparameters we yield results with high approximation quality, but also models with a poor goodness of fit; some outliers can be observed for each validation set. This observation is in line with our expectation as the hyperparameter choice might be more art than science.

Taking the mean of the models with the lowest mean squared error (MSE) for the OOS validaton set might be applied to get a higher approximation quality and to solve the curse of hyperparameters choice. By doing so, Fig. 7 shows the box plots for the ten best neural networks, measured regarding the MSE of the OOS validation set, out of the 300 calculated models. We observe tighter box plots without outliers. While the tighter box plots on the OOS validation sets are a logical result of taking out nets with low accuracy (measured as MSE on the very same set), the results also show that by doing this we are able to reduce the bias of the SCR estimate and in particular reduce the prediction error on unseen data.

Following, for our final comparison with the LSMC-Pol approach we take the average of these best 10 nets, which is based on the idea of the ensemble technique for weak learners. As can be seen in Table 1 we get a high approximation quality for our NN ensemble; this is true for all Portfolios. In particular, we see that LSMC-NN is superior to LSMC-Pol, both in terms of ME and mean absolute errors (MAE). While the MAE gives evidence for a lowered variability of errors, also the ME can be of particular interest, especially for the SCR region set. After all, in the sense of the original task—the SCR calculation—we are able to receive a lower bias using LSMC-NN. Fairly enough, we should also take the computational effort into account as the calculation of the comprehensive set of 300 different combinations of hyperparameters is much more expensive than deriving a well-working polynomial with a good approximation quality. Therefore, we go on by considering some further techniques to make the LSMC-NN approach more efficient, which is, in particular, important for applications in practice.

By acknowledging that the amount of data we provide, as outlined in Sect. 3, exceeds the computational resources of most insurance companies reporting under Solvency II we intend to show in this chapter how good proxy functions can be developed using just a fraction of the available data.

In order to facilitate an approach which may actually be feasible for most insurance companies, we have trained neural networks in the following seven settings. The ultimate target is that we arrive at Step VI and Step VII to settings which are actually computationally feasible for most companies. These seven steps can be found in Table 2.

The columns in Table 2 denote different settings we have varied in our numerical experiments: HypCho is the possible set for hyperparameters. With Wide we allow a broad range of hyperparameters, e.g. number of layers from 2 to 10 as outlined above. The Efficient choice means that prior to the final run we have analyzed which configurations lead on average to lower errors and allow only them in the hyperparameter search space; here we go into detail below. NoHypComb is the number of calculated hyperparameter combinations. Early denotes the data set that is used for the early stopping within the neural network training. If Vali is chosen, the MSE on the OOS validation data is monitored during training, and, if it does not decrease in the last 200 epochs, the training of the neural network stops; we have chosen 200 epochs as the maximum number, whereas one is free to adapt this threshold. Fitting means that the fitting data set is used to monitor the MSE and stop the training. EnsCho shows which data set is used when deciding which 10 neural networks are the best and should be used in the ensemble. Finally, NoFit refers to the number of fitting points used for the neural network training.

Summing up all seven steps, we can represent the settings in the following list, in which we describe the full setting for Step I and mention for each subsequent step the feature that has changed: In the following we focus our attention on measuring the quality of results by reducing complexity as previously outlined.

In order to investigate the potential loss of accuracy by moving from Step I to Step VII in the following we concentrate on Portfolio 1 and both error metrics, ME and MAE, for the OOS validation and SCR region data sets. In order to ensure a fair comparison between LSMC-Pol and LSMC-NN for Step VII, we have also derived LSMC-Pol1 and LSMC-Pol2 based on 8192 fitting points, denoted by LSMC-Pol1 8k and LSMC-Pol2 8k.

As mentioned above, to execute Step II we have to decide which values for hyperparameters can be seen as efficient. For that we have analyzed the MSE with respect to changes in several hyperparameters with the goal of identifying whether some hyperparameter values are favorable in terms of good approximations. In Fig. 8 we clearly see there is a significant impact of the number of layers on the magnitude of errors. Neural networks with a low number of layers tend to produce smaller errors and with an increasing number of layers the errors systematically become larger; a small error cannot be guaranteed for four or more layers. The reason for this may lie in the vanishing gradient effect. To underline this, let us consider the mean of the MSE for an increasing number of hidden layers and the chosen activation functions (in the inner nodes) in Fig. 9. We observe that the Sigmoid function systematically leads to higher errors for more layers.⁹

The activation functions ReLU and Leaky ReLU also produce higher errors with an increasing number of layers, albeit their errors are much lower. By performing this analysis for all hyperparameters it has turned out that some hyperparameter values have systematically led to strong approximation results (in the sense of small MSE values measured for the OOS validation set). On the other hand, different values of some hyperparameters such as the optimization method, learning rate or dropout rate have not given any insight regarding the approximation quality, see e.g. Fig. 10.

Following, to determine an efficient set of hyperparameters we have focused on the most promising values. For Portfolio 1, this procedure has led to the following search space: It seems to be natural to automatize such an efficient hyperparameter search. Figures 11 and 12 plot the ME and MAE, respectively, for polynomials—with good statistical properties—and neural networks resulting from all steps.

At first glance it can be highlighted that the neural networks have outperformed the LSMC-Pol approach for all steps regarding the MAE for the OOS validation data set, apart from Step VII, where the MAE is lower for the polynomial Pol2; furthermore, the polynomial Pol2 shows good results for OOS validation data if the ME is the error metric. Considering the approximation quality for the SCR region data set we have observed a significant improvement by applying neural networks; this is true for both error metrics, MAE and ME. Using an efficient hyperparameter search approach could further reduce the ME from \(-1.39\times 10^{-2}\) to \(-0.83\times 10^{-2}\) for the SCR region points, whereas the quality for the OOS validation data set has slightly declined. Even if we reduce the number of hyperparameter sets from 90 resp. to 30 we have still achieved strong results. In many machine learning applications the initial data set is split into a training data set and a test data set. As mentioned above we are in the quite comfortable situation to have a rich set of validation points. A more realistic case in line with practice is thus to use also the fitting set for early stopping and the ensemble building. Moving from Step III to Step IV does not give any rise of concerns with respect to approximation quality as the errors have still been on a moderately low level. This has also held for the ensemble where a choice of the best 10 nets based on the fitting data rather than OOS validation data has still given very good results. As a last step we have reduced the number of fitting points and we also see here that the neural networks have clearly outperformed the LSMC-Pol approach for the SCR region data set. Let us finally note that the LSMC-Pol proxies largely maintain the goodness of fit by reducing the number of fitting points from 32k to 8k, i.e. Pol1 32k results are comparable to Pol1 8k results; the same holds for Pol2.

Following the described procedure from Step I to Step VII we expect that similar results may be achieved for Portfolios 2 and 3. We invite the interested reader to perform the procedure with her or his best choice of hyperparameters.

To sum up, we can conclude that even if we reduce the number of training data points in our use case, we still get good results, outperforming the LSMC-Pol approach. Implementing an efficient hyperparameter search algorithm leads to less computational time and strong approximation quality, which is an important insight for practice.

In addition to a purely quantitative evaluation of the different machine learning techniques as carried out above, selected qualitative aspects can also be taken into account when assessing the practical suitability of the various methods. These aspects include the smoothness of risk profiles and the ability to provide a meaningful extrapolation, i.e. a reasonable behavior of the derived proxy functions beyond the training area.

In the present case, the former aspect arises as a requirement from the economic interpretation of the risk factors and their effect on the selected insurance portfolios. The latter aspect arises as a general problem in the context of the risk evaluation on the basis of proxy functions, that are necessarily derived on a bounded training space that does not cover the entire domain of the potentially unbounded underlying statistical distribution of the risk factors.

In the LSMC-Pol approach, depending on the choice of the basis functions, some conclusions on these two aspects can already be drawn on the basis of analytical considerations. In the case of generalized proxy functions that are derived via alternative machine learning techniques, at least a visual or rather empirical inspection of selected risk profiles is possible. In the context of the investigations presented here, no obvious undesired behaviors related to the above mentioned aspects have been detected. The extent to which the extrapolation delivers meaningful results would have to be checked in each individual case on the basis of an extended OOS validation carried out on a further enlarged data set. In principle, however, the results found in this investigation allow the conclusion that neural networks are able to depict a reasonable behavior beyond the training space. This can also be seen as an advantage over other alternative machine learning techniques such as tree-based methods, which cannot provide any meaningful extrapolation by construction due to the way how the risk factor space is partitioned.