Scientific ML Studio
Learn/Physics-Informed Neural Networks, from scratch

Textbook

Physics-Informed Neural Networks, from scratch

From the first derivative to a solver you can defend.

An open textbook. It starts from machine learning and neural networks with nothing assumed, then builds the physics-informed method piece by piece: differential equations, automatic differentiation, residual and boundary losses, collocation points, optimisers and the failures to watch for.

Every chapter has the maths written out rather than gestured at, code you can paste into a notebook (and that has been run), exercises with answers, and a recap. Papers the text draws on are cited with links.

Part 1

AI and neural networks from zero

Models, losses, networks and gradient descent. Enough machine learning to follow everything after.

1 What machine learning is

Functions with adjustable knobs, a number that says how wrong they are, and the difference between fitting data and understanding it.

2 Neural networks from scratch

The neuron, the layer, the activation function and the forward pass, first in plain NumPy and then in PyTorch.

3 Learning by gradient descent

The loss as a landscape, the gradient as the way downhill, backpropagation as the way to compute it, and a full training loop.

Part 2

Equations for ML people

What a differential equation says, which conditions it needs, how to check an answer, and how to differentiate a network with respect to its inputs.

4 Differential equations

What an equation in derivatives asks for, which conditions pin down one answer, the three families of second-order PDE, how to check a solution, and why units matter to a network.

5 Differentiating a network

Three ways to take a derivative, how forward and reverse mode work, and how to get first, second and mixed derivatives of a network with respect to its inputs in PyTorch, with the mistakes that break them.

Part 3

PINN fundamentals

The residual as a loss, a first complete PINN, boundary and initial conditions, collocation points, time and unknowns.

6 What a PINN actually is

The residual as a loss, a complete PINN in thirty lines, boundary and initial conditions as loss terms or built into the network, and how to tell whether the answer is right.

7 Sampling the domain

Where the equation is checked: how many collocation points, how to spread them, whether to move them, how to put them where the network is wrong, and how to sample shapes that are not rectangles.

8 Time and unknowns

Time as one more input, why evolution problems can fail by ignoring causality, how to recover an unknown coefficient from measurements, how much noisy data can tell you, and networks with several outputs.

Part 4

Making it work

Optimisers, loss balancing, architectures, and where training fails.

9 Optimisers for PINNs

Why PINN losses are hard to minimise, how to get the most out of Adam, when and how to finish with L-BFGS, and what second-order methods offer beyond it.

10 Balancing the loss

Why the weights of the loss terms change the answer, how to set them from the scales of the problem, and the methods that adjust them during training: gradient balancing, neural tangent kernel weights and self-adaptive weights.

11 Architectures

What the network should look like for a PINN: scaled inputs and outputs, Fourier features against spectral bias, sine activations and trainable slopes, and how depth, width and gated architectures compare.

12 When training fails

How to read a PINN's loss curves, a catalogue of the ways training fails and what each looks like, a checklist that finds bugs before they cost a week, and an honest comparison with classical solvers.

Part 5

Worked problems

Complete problems from equation to result. Each has a notebook you can open in the Studio and the exact script it generates.