An analysis and solution of ill-conditioning in physics-informed neural networks

Physics-informed neural networks (PINNs) have recently emerged as a novel and popular approach for solving forward and inverse problems involving partial differential equations (PDEs). However, ensuring stable training and obtaining accurate results remain challenging in many scenarios, often attributed to the ill-conditioning of PINNs. Despite this, a deeper analysis is still lacking, which hampers progress and application of PINNs in complex engineering problems. Drawing inspiration from the ill-conditioning analysis in traditional numerical methods, we establish a strong connection between the ill-conditioning of PINNs and the Jacobian matrix of the PDE system. Specifically, for any given PDE system, we construct a controlled system that allows for the adjustment of the Jacobian matrix's condition number while retaining the same solution as the original system. Our numerical experiments show that as the condition number of the Jacobian matrix decreases, PINNs exhibit faster convergence and higher accuracy. Building upon this principle and the extension of controlled systems, we propose a general approach to mitigate the ill-conditioning in PINNs, leading to successful simulations of three-dimensional flow around the M6 wing at a Reynolds number of 5,000. To the best of our knowledge, this is the first time that PINNs have successfully simulated such complex systems, offering a promising new technique for addressing industrial complexity problems. Our findings also provide valuable insights to guide the future development of PINNs.