Stabilität, Genauigkeit und Effizienz hybrider Finite Elemente / neuronaler Netzwerk - Ansätze zur Lösung partieller Diffe- rentialgleichungen

The solution of partial differential equations is a central subject of numerical analysis and an indispens- able tool in science and engineering. Existing approaches, such as finite elements, can provide solutions efficiently and robustly in many applications. Deep neural networks, nonetheless, emerged in the last few years as an alternative approach with promising results. Techniques that are completely or partially based on neural networks, however, currently lack the mathematical guarantees and insights available for estab- lished approaches. Furthermore, their relative performance and practical robustness in applications is at the moment unclear, even in the case of standard problems such as those from three dimensional fluid mechanics.
In the proposed project, we will work towards a mathematical theory for numerical techniques that combine finite elements and deep neural networks for the solution of partial differential equations. Informed by our preliminary work, our hypothesis is that a combination of (adaptive multigrid) finite elements with deep neural networks can provide a computationally more efficient and more accurate solution than either approach alone. Concretely, we will consider the Stokes and Navier-Stokes equations and the neural networks will represent fine scale behavior not resolved by a finite element solution. The networks will be trained using high-resolution reference data, which was sufficient to attain accurate and efficient solutions of standard flow problems in 2d and 3d in our preliminary work. We will therefore not pursue physical or mathematical constraints on the solutions, as, e.g., in PINNs, and consider it an important but orthogonal research direction to our planned work. Although it is a central objective of the proposed project to develop mathematically rigorous analyses, we consider it also as important to study the practicality of our results through implementations. As part of the project, a research code for hybrid fluid flow solvers in 2d and 3d will therefore be implemented and made publicly available at the end of the project.
We will build on recent work that showed that the mathematical analysis of deep neural networks is possible using tools that have been developed for the analysis of finite element methods. We will extend these results to numerical time stepping schemes for the Stokes and Navier-Stokes equations that combine a classical discretization with neural networks and consider practically relevant setups in 2d and 3d, e.g. by including relevant boundary conditions. We will also extend existing results to state-of-the art network architectures used in the machine learning literature, e.g. transformers. These are one of the most powerful architectures used in practice and at the same time well suited for scientific computing and a mathematical analysis.
The first questions we want to address in the proposed project are stability and accuracy of the hybrid simulations, i.e. that they remain bounded and that a neural network is able to improve the accuracy. For a hybrid solver, this requires, among other things, neural networks that are stable for admissible inputs but also a coupling to the finite element part that preserves the stability. Second, we will explore adaptive solution schemes where a posteriori or neural network-based error estimates are used to refine a solution if necessary to meet pre-defined error criteria. We believe that the results obtained in the proposed project will also be of relevance for a more complete theory for neural network-based simulations.