Differentiable and Inverse Monte Carlo Methods

At a high level, a random walk terminating on a domain boundary and a light path terminating on an emitter are the same object; the derivative of each decomposes into an interior term and a boundary term for the same reason; and techniques cross over in both directions.

A course built around getting derivatives exactly right owes an honest account of when that is the wrong goal. Some forward models are black boxes with no adjoint; some have discrete parameters; and some have objectives so flat that the exact derivative is correctly zero and tells you nothing. A parallel literature answers with approximation — zeroth-order and evolution-strategy estimators, deliberate smoothing of the objective, relaxed forward models, and learned local surrogates that replace the simulator with a differentiable neural stand-in. We treat these as a genuine methodological alternative rather than a footnote (L15), because knowing when the exact gradient is not worth its price is part of knowing how to compute it.

A gradient serves more than one master. We compute $\partial(\text{simulation})/\partial\theta$; that object then feeds (i) optimization, descending a loss, and (ii) sampling, since Langevin and Hamiltonian Monte Carlo use gradients to propose moves, and since $\nabla \log \pi$ is what a score-based generative prior supplies.

1. Theory — how to differentiate an integral whose integrand and whose domain depend on the parameters, and what a correct estimator of that derivative looks like.
2. Algorithms and systems — how to make such estimators fast enough and low-variance enough to be usable, and how they interact with autodiff frameworks, GPUs, and memory limits.
3. Applications and inference — inverse rendering, appearance acquisition, PDE-constrained shape optimization, transient and non-line-of-sight imaging, biomedical and tomographic transport, manufacturing and optical design, and Bayesian recovery with uncertainty quantification.

Students are expected to come from graphics, vision, machine learning, or scientific computing. No single background is assumed, and the course is deliberately built so that each community contributes something the others lack. Graphics students already know light transport but often not adjoint methods; scientific computing students know adjoint methods and boundary value problems but not path space; ML students know autodiff and stochastic optimization but not why their gradients are biased.

A prior rendering course, a numerical PDE course, or practical experience with PyTorch or JAX.

Try these in week 1. If two or more are unfamiliar, use the bridging readings below — I do not expect all of you to be comfortable with all of them.

1. Write down an unbiased estimator of $\int_0^1 f(x)\,dx$ using samples from a density $p$, and state its variance.
2. State the rendering equation, and separately state a screened Poisson boundary value problem.
3. Explain the difference between forward-mode and reverse-mode automatic differentiation, and why reverse mode is preferred when there are many parameters.
4. Explain why $\frac{d}{d\theta}\mathbb{E}[X_\theta] \neq \mathbb{E}\!\left[\frac{d}{d\theta}X_\theta\right]$ in general.

Each 75-minute session is structured as one of three kinds:

The hybrid structure is deliberate. Papers in this area are dense and assume a formalism that is not in any textbook; a discussion that starts cold usually stalls. The lecture segment installs the vocabulary, and the discussion segment stress-tests it against a real paper.

The remaining ~20 minutes are open discussion, moderated by me. You will serve as Presenter once and Respondent once over the semester. Sign-ups open in week 2. You should choose papers away from your existing expertise.

Everyone else submits a reading response by 9:00 pm the night before each hybrid session. In half a page:

1. One sentence: what is the paper’s contribution?
2. The single step in the derivation or algorithm you found least convincing, or least clear.
3. One question you want raised in class.

There is no textbook. Primary sources are listed per session in the schedule above. Three resources recur: