Randomness is how we simulate the world. Derivatives are how we invert it. This course sits where the two meet — and teaches you to write estimators that see backwards, from the image to the scene, from the measurement to the cause.
This course explores the theory and practice of differentiating Monte Carlo estimates and solving the inverse problems that follow from them. We build the machinery from first principles: general methods for gradient estimation, including boundary integrals for discontinuous integrands, then differentiable light transport, then applications in inverse rendering, Bayesian inference, and scientific computing.
Half of the work is analytical — deriving estimators and proving them unbiased. The other half is on the GPU, where you will implement what you derive and watch a scene reconstruct itself from a handful of photographs.
The course is open to students in Computer Science, Applied Mathematics, Computational Physics, and neighbouring fields. If you have ever wanted to run a simulation backwards, this is the course.
Change of variables, Jacobians, and differentiation under the integral sign.
Conditioning, least squares, and the vocabulary of ill-posed systems.
Estimators, variance, convergence, and Markov chains.
Working fluency; prior exposure to PyTorch or Mitsuba helps but is not assumed.
Open a week to see its topics.
Weeks 1–3
Monte Carlo estimator properties — unbiasedness, variance, convergence rate. Fredholm integral equations. Forward versus inverse problems, conditioning, and regularization (Tikhonov, TV).
Control variates, stratified sampling, Multiple Importance Sampling (MIS), weight window techniques, and variance trade-offs inside optimization loops.
Metropolis–Hastings, Hamiltonian Monte Carlo, posterior estimation in high-dimensional state spaces, and pseudo-marginal MCMC.
Weeks 4–6
The score-function estimator (likelihood ratio method) and its derivation, control variates for variance reduction, and application to discrete and continuous distributions.
The reparameterization trick as a pushforward of measures, pathwise differentiation, and a head-to-head comparison of score-function and pathwise estimators in variance and compute cost.
The boundary problem in MC differentiation. Reynolds transport theorem and the Leibniz rule in higher dimensions, edge sampling, Dirac delta formulations, and continuous versus discontinuous transport boundary differentiation.
Weeks 7–9
The path integral formulation of radiative transport, differentiation along ray paths, primary-sample-space versus screen-space differentiation, and reverse-mode automatic differentiation inside a Monte Carlo estimator.
Differentiable path tracing, handling occlusion discontinuities, material parameter recovery, shape optimization, and inverse appearance modelling.
The volumetric path integral, differentiable null-collision algorithms (delta tracking), inverse scattering, diffuse optical tomography in medicine, and atmospheric transport.
Weeks 10–13
Neural importance sampling, neural control variates, continuous normalizing flows for proposal sampling, and learned path guiding.
Differentiable neutron transport, climate and ocean radiative transfer, particle transport in plasma physics, and parameter calibration in complex physical simulations.
Measure-valued derivatives, unbiased gradient estimators with bias removal (Rhee–Glynn, Russian roulette differentiation), and hardware-aware implementation — symbolic versus trace-based compilation.
Final project presentations, peer review, and a look at where differentiable stochastic simulation goes next.