Adaptive refinement
Putting points where the network is wrong.
1 min read
Residual-based adaptive refinement is a loop: train, evaluate $|r_\theta|$ on a dense candidate set, add the worst points, repeat.
When to resample
Every 1000–5000 iterations is typical. Resampling too often makes the loss surface non-stationary and the optimiser never settles.