Feasibility Study of Physics-Informed Neural Network Modeling in Computational Fluid Dynamics
Open AccessSolving the Navier-Stokes equations with deep learning has been a popular research topic since the term physics-informed neural networks (PINNs) was coined by Raissi et al. in 2017. A search on Google Scholar using ”physics-informed neural networks” as the keyword revealed a rapid growth from 60 results in 2017 to 3340 results in 2021. Such a blooming in PINNs raised our interest in how ready PINNs are for computational fluid dynamics (CFD) applications in practical engineering.This work aims to understand data-free PINNs’ capability to solve CFD problems from the perspectives of predictability and controllability. Predictability helps engineering practitioners estimate the cost for the desired accuracy level, and controllability means whether practitioners can control the cost and accuracy. With the help of NVIDIA’s Modulus—a Python tool set for PINNs, we conducted computational experiments to understand PINNs’ numerical behaviors and limitations in solving the incompressible Navier-Stokes equations.The computational experiments consist of two groups of benchmarks. Benchmarks in the first group help us understand the following behaviors of PINNs when solving the incompressible Navier-Stokes equations: (1) cost-performance ratios, (2) convergence of errors with respect to model complexity of MLP (multilayer perceptron) networks, and (3) weak and strong parallel scalability. Further, we benchmarked several training strategies that have not been seen in PINNs’ literature or have not been widely examined by other researchers. These strategies include annealing loss aggregation, cyclical learning rate, stochastic weight averaging, and nonlinear conjugate-gradient methods. We used the 2D Taylor-Green vortex at Re = 100 for all benchmarks within this group.In the second group of the benchmarks, we investigated PINNs’ capability to solve flow problems with instabilities by conducting computational experiments with a 2D cylinder flow at Re = 200. In both physical experiments and conventional CFD simulations, such a flow eventually reaches an unsteady solution with periodic vortex shedding due to instability. We wanted to understand PINNs’ accuracy in predicting such unsteady solutions. Benchmarks in this group used three types of PINNs: (1) a steady data-free PINN, (2) an unsteady data-free PINN, and (3) an unsteady data-driven PINN trained against both the Navier-Stokes equations and simulation data from PetIBM. In addition, we used a 2D cylinder flow at Re = 40, which does not have instabilities, to validate the steady and unsteady data-free PINNs’ capabilities in flow with immersed bodies.Here we list the key findings from the first group of our benchmarks:1. Under a given desired error level, the time cost of a PINN solver is several orders of magnitude slower than our in-house conventional CFD solver, PetIBM. For example, 20 seconds with PetIBM versus 2 hours with the PINN for an error level of 1e − 3. 2. PINNs with the same training loss may give different prediction accuracies. This nondeterministic behavior grows when MLP networks’ model complexity increases. We found that training losses may keep decreasing even when PDE residuals stop improving. In such cases, only the residuals of initial conditions are improving, which does not help with prediction accuracy. This finding suggests that PDE residuals are better options for monitoring training progress. 3. Errors decrease at a similar rate when we either increase the number of hidden layers by one or double the number of per-layer neurons in MLP networks. It suggests that the former is a better option to improve prediction accuracy because it needs less computational cost. 4. Unlike conventional numerical methods, PINNs’ errors do not monotonically decrease when increasing MLP networks’ degrees of freedom (DoFs). PINNs can have different accuracies under the same number of DoFs, indicating that DoFs are not a proper metric for PINNs’ model complexity. 5. The per-batch number of training points substantially affects PINNs’ time costs more than the model complexity. However, the former has less influence than the latter on prediction accuracy. It suggests that increasing per-batch training points should come after increasing hidden layers or neurons when improving PINNs’ accuracy. 6. PINNs’ weak scaling is above 90% on average (with 1 to 8 GPUs), which helps scale per-batch training points on GPUs with limited memory space. 7. If we defined workload using the number of training points, PINNs have strong-scaling efficiencies ranging from 91% with 2 GPUs in the best case to 27% with 8 GPUs in the worst case. 8. Defining a proper quantitative metric to evaluate model complexity is necessary to further study PINNs’ convergence and scaling properties. 9. All new training strategies we tried did not significantly improve time-to-solution and prediction accuracy.In the second group of our benchmarks, the validations of both the steady and unsteady data-free PINN solvers were successful against the 2D cylinder flow at Re=40. PINNs’ drag coefficients and surface pressure distribution were within the range of experimental data and conventional CFD simulations.As for the 2D cylinder flow at Re = 200, the unsteady data-free PINN solver could not predict vortex shedding within a simulation time of t ∈ [0, 200]. The unsteady data-free PINN solver converged to a steady-state solution instead. The data-driven PINN solver—trained against simulation results from PetIBM in t ∈ [125, 140] and the Navier-Stokes equations in t ∈ [125, 200]—also converged to a steady-state solution when t > 140. Moreover, the data-driven PINN solver failed to give meaningful predictions for t < 125. Nevertheless, all PINNs’ drag coefficients agreed with conventional steady-state simulation results.The spectral analysis of the data-driven PINN’s results in t ∈ [125, 140] shows that the PINN used in our work is both dispersive and dissipative. The dispersion and dissipation may work as dampers in the dynamical system of the fluid flow. Such dampers inhibit the instabilities and the oscillating from happening. Eventually, the vortex shedding did not occur in the data-free PINN and disappeared ultimately in the data-driven PINN.Besides the benchmarks, this work briefly introduces PINNs. Starting from the difference between data-driven and data-free PINNs, we continue by introducing the history and the development of solving PDEs with data-free PINNs since 1994. We discussed the open questions and known issues of data-free PINNs, which further motivated the present work. In a later chapter, we detail the mathematical formulations related to PINNs. Moreover, we compare PINNs to conventional numerical methods and show that PINNs are special cases of the least-square finite elementmethods.We describe our in-house high-performance CFD code and associated verification and validation (V&V;) to compare conventional CFD methods and data-free PINNs. The V&V; confirmed the correctness of our CFD code, and the subsequent performance benchmarks demonstrated its computational performance. The in-house CFD code served as a baseline for PINNs’ benchmarks.In a nutshell, our results show that PINNs is a relatively new research area. As we investigated the feasibility of PINNs, we found more open questions and challenges for PINNs to become an engineering tool. While most studies in the literature focused on the theoretical capability of PINNs and toy PDEs, we hope our work can arouse others’ interest in solving those challenges. All the code, case configurations, scripts for pre-/post-processing, and definition files of containers are open and available online (DOI: 10.5281/zenodo.7388327).
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.