1 Introduction
2 Stochastic analysis of a heterogeneous porous medium
2.1 Darcy equation for groundwater flow problem
2.2 Generate the hydraulic conductivity fields
2.3 Defining numerical experimental model
-
One-dimensional groundwater flow \(\rightarrow [0,25]\).
-
Two-dimensional groundwater flow \(\rightarrow [0,20]\times [0,20]\).
-
Three-dimensional groundwater flow \(\rightarrow [0,5]\times [0,2]\times [0,1]\).
2.4 Manufactured solutions
3 Deep learning-based neural architecture search method
3.1 Modified neural architecture search (NAS) model
3.1.1 Components of convolutional NAS
-
Search Space. The search space defines the architecture that can be represented. Combined with a priori knowledge of the typical properties of architectures well suited to the underlying task, this can reduce the size of the search space and simplify the search. For the model in this study, the priori knowledge of search space is gained from the global sensitive analysis. Figure 4b shows a common global search space with a chain structure. The chain-structured neural network architecture can be written as a sequence of n layers, where the ith layer \(L_i\) receives input from layer \(i-1\) and its output is used as input for layer \(i+1\):where \(\odot \) are operations.$$\begin{aligned} output = L_n\odot L_{n-1}\odot ... L_1\odot L_0, \end{aligned}$$(27)
-
Search Method. The search method is an initial filtering step narrowing down the search space. In this paper, hyperparameter optimizers will be used. The choice of the search space largely determines the difficulty of the optimization problem, which may result in the optimization problem remaining (i) noncontinuous and (ii) high-dimensional. Thus, some prior knowledge of the model features is needed.
-
Performance Estimation Strategy. The simplest option for a performance estimation strategy is standard training and validation of the data for the architecture. As pointed out in Sect. 2.4, we define the relative error of manufactured solution for the performance estimation strategy:$$\begin{aligned} \delta h=\frac{\Vert {\hat{h}}-{\hat{h}}_{MMS}\Vert _2}{\Vert {\hat{h}}_{MMS}\Vert _2}. \end{aligned}$$(28)
3.1.2 Modified NAS
3.2 Neural networks generator
3.2.1 Physics-informed neural network
3.2.2 Deep collocation method
3.3 Sensitivity analyses (SA)
3.4 Search methods for NNs
3.5 Transfer learning (TL)
4 Numerical examples
4.1 Comparison of Gaussian and exponential correlations
4.1.1 One-dimensional groundwater flow with both correlations
\(\sigma ^2\) | ||||||
---|---|---|---|---|---|---|
N | 0.1 | 1 | 3 | |||
without TL | with TL | without TL | with TL | without TL | with TL | |
500 | 1.184e-3 | 1.797e-4 | 1.100e-2 | 4.884e-4 | 1.159e-1 | 5.360e-4 |
1000 | 2.437e-2 | 2.354e-4 | 9.026e-3 | 5.282e-4 | 3.752e-2 | 1.754e-3 |
2000 | 5.789e-4 | 1.007e-4 | 3.813e-3 | 5.939e-4 | 3.532e-2 | 4.316e-3 |
\(\sigma ^2\) | ||||||
---|---|---|---|---|---|---|
N | 0.1 | 1 | 3 | |||
without TL | with TL | without TL | with TL | without TL | with TL | |
500 | 1.211e-4 | 1.137e-4 | 9.065e-4 | 1.204e-4 | 7.539e-3 | 1.690e-4 |
1000 | 1.317e-4 | 1.133e-4 | 8.312e-4 | 1.200e-4 | 2.864e-3 | 3.662e-4 |
2000 | 1.158e-4 | 1.333e-4 | 2.811e-4 | 1.538e-4 | 2.904e-3 | 5.756e-4 |
4.1.2 Two-dimensional groundwater flow with both correlations
\(\sigma ^2\) | ||||||
---|---|---|---|---|---|---|
N | 0.1 | 1 | 3 | |||
Without TL | With TL | Without TL | With TL | Without TL | With TL | |
500 | 6.777e-2 | 9.345e-2 | 3.817e-2 | 4.635e-2 | 2.560e-1 | 5.5080e-2 |
1000 | 1.479e-2 | 4.832e-2 | 1.790e-3 | 8.157e-2 | 9.739e-2 | 7.201e-2 |
2000 | 7.147e-3 | 4.829e-2 | 4.471e-2 | 4.924e-2 | 9.357e-2 | 1.187e-1 |
\(\sigma ^2\) | ||||||
---|---|---|---|---|---|---|
N | 0.1 | 1 | 3 | |||
Without TL | With TL | Without TL | With TL | Without TL | With TL | |
500 | 9.974e-4 | 9.842e-4 | 3.530e-3 | 7.900e-4 | 3.053e-2 | 2.475e-3 |
1000 | 2.980e-4 | 6.954e-4 | 6.270e-3 | 1.527e-3 | 3.855e-2 | 2.904e-3 |
2000 | 7.299e-4 | 5.717e-4 | 7.719e-3 | 1.704e-3 | 7.486e-2 | 2.506-2 |
4.1.3 Three-dimensional groundwater flow with both correlations
\(\sigma ^2\) | ||||||
---|---|---|---|---|---|---|
N | 0.1 | 1 | 3 | |||
Without TL | With TL | Without TL | With TL | Without TL | With TL | |
500 | 3.419e-3 | 1.529e-2 | 1.131e0 | 5.885e-2 | 6.264e-1 | 7.340e-2 |
1000 | 1.219e-1 | 8.333e-3 | 3.257e-1 | 5.668e-2 | 1.055e0 | 8.982e-2 |
2000 | 5.667e-2 | 1.230e-2 | 4.287e-1 | 6.161e-2 | 1.204e0 | 5.313e-2 |
\(\sigma ^2\) | ||||||
---|---|---|---|---|---|---|
N | 0.1 | 1 | 3 | |||
Without TL | With TL | Without TL | With TL | Without TL | With TL | |
500 | 1.161e-2 | 5.439e-3 | 6.247e-3 | 1.540e-2 | 9.294e-2 | 1.078e-2 |
1000 | 1.187e-3 | 5.620e-3 | 4.087e-2 | 1.205e-2 | 3.004e-1 | 1.996e-2 |
2000 | 8.342e-3 | 1.661e-2 | 1.218e-2 | 1.278e-2 | 1.562e-1 | 1.952-2 |
Dimension | ||||||
---|---|---|---|---|---|---|
Correlation | 1 | 2 | 3 | |||
Without TL | With TL | Without TL | With TL | Without TL | With TL | |
Exponential | 30s | 3.0s | 97s | 6.5s | 58s | 7.8s |
Gaussian | 28s | 3.0s | 58s | 9.8s | 52s | 5.9s |
4.2 Sensitivity analysis results
Hyper-parameters | Intervals |
---|---|
Layers of NNs | [2, 30] |
Neurons per layer | [10, 50] |
Number of iterations | [1500, 3000] |
Number of collation points | [800, 2000] |
Maximum line search of L-BFGS algorithm | [30, 300] |
4.3 Hyperparameter optimizations method comparison
Algorithms | Time | Relative error |
---|---|---|
RSM | 1830s | 0.00051 |
Bayesian | 1395s | 0.00032 |
Hyperband | 1449s | 0.00058 |
Jaya | 1757s | 0.00139 |
Dimension | Layer | Neurons |
---|---|---|
1D | 2 | 37 |
2D | 6 | 17 |
3D | 2 | 14 |
4.4 Model validation in different dimensions
4.4.1 One-dimensional case model validation
4.4.2 Two-dimensional case model validation
4.4.3 Three-dimensional case model validation
Dimension | |||
---|---|---|---|
Results | 1 | 2 | 3 |
Relative error | 6.443e-5 | 0.017 | 5.711 |
Time | 2.8s | 180s | 1245s |
Dimension | ||||||
---|---|---|---|---|---|---|
Results | 1 | 2 | 3 | |||
without TL | with TL | without TL | with TL | without TL | with TL | |
Relative error | 1.369e-4 | 1.195e-4 | 4.262e-3 | 4.405e-3 | 8.915e-3 | 8.864e-3 |
Time | 14.3s | 1.5s | 108.5s | 9.8s | 32.2s | 1.9s |