Textbook
Physics-Informed Neural Networks, from scratch
From the first derivative to a solver you can defend.
An open textbook. It starts from machine learning and neural networks with nothing assumed, then builds the physics-informed method piece by piece: differential equations, automatic differentiation, residual and boundary losses, collocation points, optimisers and the failures to watch for.
Every chapter has the maths written out rather than gestured at, code you can paste into a notebook (and that has been run), exercises with answers, and a recap. Papers the text draws on are cited with links.
1 What machine learning is
Functions with adjustable knobs, a number that says how wrong they are, and the difference between fitting data and understanding it.
2 Neural networks from scratch
The neuron, the layer, the activation function and the forward pass, first in plain NumPy and then in PyTorch.
3 Learning by gradient descent
The loss as a landscape, the gradient as the way downhill, backpropagation as the way to compute it, and a full training loop.
4 Differential equations
What an equation in derivatives asks for, which conditions pin down one answer, the three families of second-order PDE, how to check a solution, and why units matter to a network.
5 Differentiating a network
Three ways to take a derivative, how forward and reverse mode work, and how to get first, second and mixed derivatives of a network with respect to its inputs in PyTorch, with the mistakes that break them.
6 What a PINN actually is
The residual as a loss, a complete PINN in thirty lines, boundary and initial conditions as loss terms or built into the network, and how to tell whether the answer is right.
7 Sampling the domain
Where the equation is checked: how many collocation points, how to spread them, whether to move them, how to put them where the network is wrong, and how to sample shapes that are not rectangles.
8 Time and unknowns
Time as one more input, why evolution problems can fail by ignoring causality, how to recover an unknown coefficient from measurements, how much noisy data can tell you, and networks with several outputs.
9 Optimisers for PINNs
Why PINN losses are hard to minimise, how to get the most out of Adam, when and how to finish with L-BFGS, and what second-order methods offer beyond it.
10 Balancing the loss
Why the weights of the loss terms change the answer, how to set them from the scales of the problem, and the methods that adjust them during training: gradient balancing, neural tangent kernel weights and self-adaptive weights.
11 Architectures
What the network should look like for a PINN: scaled inputs and outputs, Fourier features against spectral bias, sine activations and trainable slopes, and how depth, width and gated architectures compare.
12 When training fails
How to read a PINN's loss curves, a catalogue of the ways training fails and what each looks like, a checklist that finds bugs before they cost a week, and an honest comparison with classical solvers.
13 Worked problems
Poisson in one and two dimensions, then heat, manufactured solutions, waves, a shock and a curved domain. Each is built in the Studio, run, and checked against an exact or independent reference.
- 13.1 Poisson in one dimension
- 13.2 Poisson on a square
- 13.3 Heat in a bar: adding time to a PINN
- 13.4 Manufactured solutions: invent the answer, then build the problem
- 13.5 A standing wave: second derivatives in time
- 13.6 A shock wave: Burgers' equation and the second optimiser
- 13.7 Laplace's equation on a disc: a curved boundary
- 13.8 Heat in a plate: a hot spot cooling in two dimensions
- 13.9 Lid-driven cavity: a viscous flow in a box
- 13.10 A simply supported beam: PINN against FEM