Weight initialization algorithm for physics-informed neural networks using finite differences

With physics-informed neural networks (PINNs), inverse problems involving differential equations can be solved despite noise, sparsity, and varying levels of fidelity. The objective of PINNs is to minimize a loss function using all available information, including physical laws, initial conditions, boundary conditions, and observed quantities. This is accomplished by turning the original problem into a problem of optimization based on the available data. Our paper presents a method for initializing weights in PINNs using finite differences. We consider two neural networks, feed-forward neural network (FNN) \(u_1\) and PINN \(u_2\) with the same architecture. First, we construct a dataset and fit FNN \(u_1\) to it. To generate our dataset, we use an approximate solution of the partial differential equation using finite differences. In our PINN, the initial weights are determined by the weights obtained from fitting FNN \(u_1\) to the constructed dataset. The use of coarse mesh resolutions has proven to be viable as well. In contrast to the vanilla PINN training strategy, the proposed weight initialization strategy reduces PINN solution errors and training loss. We also propose a method named Progressive Unfreezing (PU) to reduce the computational cost of training our model. PU gradually fine-tunes the layers of the PINN \(u_2\) while keeping already fine-tuned layers frozen. This allows the model to leverage pre-trained knowledge and adapt to new task-specific data while reducing computational cost. We solve the one-dimensional diffusion (heat) equation, Fisher equation also known as the Kolmogorov–Petrovsky–Piscounov, and Burgers non-linear partial differential equation.