by Nirav Vasant Shah
Arti ficial Neural Network (ANN) has shown promising capabilities for solving Partial Di fferential Equations (PDEs). ANN based approaches can be combined with conventional methods and can give accurate predictions also with noisy data. Despite several known computational advantages of ANN based approaches over conventional approaches, there is limited understanding about “why the ANN based methods work or do not work”. On the contrary, we know about “what ANN is able to learn”.
ANN can act as universal function approximator i.e. ANN can learn mapping function from inputs to outputs. In classical approaches, ANN is used to identify mapping between two finite dimensional spaces. By using proper architecture and a properly defi ned loss function, ANN can compute the solution fi eld in computationally efficient manner by learning from data. ANN based approaches can be modifi ed to utilize knowledge of the governing PDE [7]. However, such methods require training of ANN for each new instance of parameter or coefficient. Additionally, such networks might have limited generalization capabilities. Accuracy of ANN can be divided into three components: training error, optimization error and generalization error. The universal approximation theorem guarantees only small approximation error but it does not consider optimization error and generalization error [5].
Alternatively, ANN can be used to learn non-linear continuous operators. Some of the recent approaches have focused on learning mapping between in finite dimensional spaces i.e. learning the operator. Deep Operator Network (DeepONet) [5] can learn operator accurately from relatively small dataset and has shown promising generalization capabilities. It uses “Branch” sub-network for encoding input function and “Trunk” sub-network for encoding locations of output functions. Graph Kernel Network (GKN) [3] has also been used to learn mapping between infinite dimensional spaces. GKN uses iterative architecture, which includes learning kernel by a neural network. The approach is discretization invariant and is able to train and to generalize on diff erent meshes. Fourier Neural Operator (FNO) [4] is another approach for learning mapping between two in finite dimensional spaces. FNO also uses iterative updates, replacing kernel integral operator by a convolution operator in Fourier space. It is shown that if operator is approximated properly, the error will be constant at any resolution of the data. For any new instance of coefficient or parameter of the governing equation, GKN and FNO only require forward pass of the ANN.
ANN has been used as function approximator for model order reduction of parametric PDEs [2], in areas such as thermomechanical problems of industrial interest [8, 9] and complex problems in computational fluid dynamics [6]. One of the key challenges for deep learning-based model order reduction techniques is to extract more information from the high- fidelity solutions in a non-intrusive manner. Non-intrusive methods are convenient, especially in case the high- fidelity solution is computed with commercial software. The extension of Physics Informed Neural Network [7] to Physics Reinforced Neural Network [1] is an example of modifying ANN-based approaches for application to model order reduction. Similarly, it is important to identify opportunities to extend operator learning to model order reduction techniques.
Temperature and displacement prole for thermomechanical problem [9]
Bibliography
[1] W. Chen, Q. Wang, J. S. Hesthaven, and C. Zhang. Physics-informed machine learning for reduced-order modeling of nonlinear problems. Journal of Computational Physics, 446:110666, 2021.
[2] J.S. Hesthaven and S. Ubbiali. Non-intrusive reduced order modeling of nonlinear problems using neural networks. Journal of Computational Physics, 363:55-78, 2018.
[3] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Neural operator: Graph kernel network for partial differential equations, 2020.
[4] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Fourier neural operator for parametric partial differential equations, 2021.
[5] L. Lu, P. Jin, and G. E. Karniadakis. Deeponet: Learning nonlinear operators for identifying di fferential equations based on the universal approximation theorem of operators, 2020.
[6] F. Pichi, F. Ballarin, G. Rozza, and J. S. Hesthaven. An arti ficial neural network approach to bifurcating phenomena in computational fluid dynamics, 2021.
[7] M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial di fferential equations. Journal of Computational Physics, 378:686-707, 2019.
[8] N. V. Shah, M. Girfoglio, P. Quintela, G. Rozza, A. Lengomn, F. Ballarin,
and P. Barral. Finite element based model order reduction for parametrized one-way coupled steady state linear thermomechanical problems, 2021.
[9] N. V. Shah, M. Girfoglio, and G. Rozza. Thermomechanical modelling for industrial applications, 2021.