Solving Partial Differential Equations : from Finite Elements to Neural Networks

In the vast majority of theoretical and real-world applications, the solution to a partial differential equation can not be computed analytically. The aim of this course is to explore different methods allowing to compute approximate solutions. The first one, "traditional” one, is the Finite Element Method. The other ones make use of the recent developments in Deep Learning leading including the Physics Informed Neural Networks and Neural Operators. At the end of the semester, the students have to implement a project using both the FEM and NN-based methods.

This course has been given as an Arbeitsgemeinschaft (working group) in the University of Würzburg during the summer semesters of 2024, 2025 and 2026. This page aims at providing necessary resources for the students, starting with the lecture notes!

Overview of the course

1. Finite Element Method

1.1. Introduction to the variational formulation

Consider the prototypal Poisson equation \begin{equation} \begin{cases} - \Delta u = f &\mbox{ in } \Omega \\ u = 0 & \mbox{ on } \partial \Omega \end{cases} \end{equation} on a domain $\Omega$. Using Stokes formula (that we will recall), we can show that $u \in C^2$ is a solution of this equation if and only if it verifies $$ \int_\Omega \nabla u \cdot \nabla v = \int_\Omega fv $$ for all $v\in C^1$ such that $v=0$ in $\partial \Omega$. This motivates the introduction of the Lax-Milgram theorem, an abstract framework to prove the existence and uniqueness of solutions of such problems.

However, the theorem can not be applied in $C^n$ function spaces since it is not complete for the natural scalar product, making necessary to generalize the meaning of the Poisson equation to the so-called Sobolev spaces.

1.2. A short introduction to Sobolev spaces: the case of $H^1$

After a quick reminder of the $L^2$ space, we introduce the notion of weak derivatives. This allows to define the Sobolev space $H^1$ of square-integrable weakly differentiable functions, which can be proved to be an Hilbert space. While the structure of Hilbert space is amazingly useful, the functions of this space might not even be continous anymore. This will add further difficulties, like defining the notion of trace of an $H^1$ function on $\partial \Omega$ to make sense of the boundary conditions.

We then proceed to study canonical examples, such as the Poisson equation with various boundary conditions and other elliptic PDEs. We will show that often, the solutions to such problems can be seen as the minimizers of a certain energy.

1.3. The Finite Element Method

Solving a PDE algorithmically requires to approximate it by a problem of finite dimension. The most elementary way to do so may be the finite differences method, which amounts at discretizing the domain $\Omega$. The Finite Element Method consists in another approach, namely discretizing the Sobolev space $H^1(\Omega)$ (which, in turn, need to mesh $\Omega$). We will detail the principle of internal approximation of a variationnal problem. Then we proceed to explain the principles of the FEM, first for $P_1$ elements in $1$D, writing explicitly the mass and rigidity matrix corresponding to the Poisson problem. We then generalize to $n$D, with triangular meshes.

At last, we will use the theoretical knowledge of the previous sections to numerically approximate otherwise unsolvable problems. We will use Fenicsx, a Python FEM framwork.

2. Neural Networks for PDEs

2.1. A brief introduction to Neural Networks

We provide a brief overview of Deep Learning and its uses (supervised, unsupervised, reinforcement, generative, regression, classification). We define the notion of a dense neural network and state and prove the Universal Approximation Theorem, which state that every function can be approximated by a NN.

Introducing the concept of loss function, gradient descent and backpropagation, the training of a NN is then tackled.

2.2. Using Neural Networks as PDE solvers

Through FEM, we have seen a precise and elegant way to approximate the solution of a PDE. Using approximation capabilities of NNs, we see how to solve complex PDEs using little to no data. We will discuss the recent developments in the field, such as "vanilla" PINNs and the Deep Ritz Method, along with error estimates. In a second part, we will discuss Neural Operators.

Project

The course will end with a project, by groups of 2-3 students. Each group has the freedom to chose between:

• Comparing the performance of the FEM and PINNs on a given PDE;
• Implementing a Neural Operator and train it on a dataset generated using the FEM.

The NN part has to be implemented in PyTorch. The FEM library is free to choose, but I would suggest Fenicsx, FreeFem++ or scikit-fem.

Examples of previous projects (I warmly thank my students for allowing me to share those):

2024:

• Testing Physics-Informed Neural Networks for the Solution of Hyperbolic Conservation Laws, Leon Jacobi and Simon Krotsch (report) (code)

2025:

• On the Numerical Treatment of the Time-Dependent Schrödinger Equation in Two Dimensions, Lasse Kreimendahl and Simon Wenchel (report) (GitHub)
• A Comparison of FEM and PINN with Variations of the Stokes Equation, Christoph Roth and Florian Eich (report) (GitHub)