Monte Carlo estimators for differential light transport

SIGGRAPH 2021

Differentiable rendering of a scene featuring specular interreflection between metallic surfaces of varying roughness. We differentiate the image with respect to the combined roughness of all objects, which produces the gradients shown in the first column with insets. A disconcertingly large number of differential estimators can solve this problem, albeit with drastically different statistical efficiency: the following four columns highlight the standard deviation of emitter sampling and three material-based strategies. An overview of the exhaustive set of combinations (21 methods) and results for an additional four estimators are provided in the supplemental material, which also contains uncropped images. The objective of our work is to provide intuition on how to navigate the large design space of differential Monte Carlo estimators.

Abstract

Physically based differentiable rendering algorithms propagate derivatives through realistic light transport simulations and have applications in diverse areas including inverse reconstruction and machine learning. Recent progress has led to unbiased methods that can simultaneously compute derivatives with respect to millions of parameters. At the same time, elementary properties of these methods remain poorly understood.

Current algorithms for differentiable rendering are constructed by mechanically differentiating a given primal algorithm. While convenient, such an approach is simplistic because it leaves no room for improvement. Differentiation produces major changes in the integrals that occur throughout the rendering process, which indicates that the primal and differential algorithms should be decoupled so that the latter can suitably adapt.

This leads to a large space of possibilities: consider that even the most basic Monte Carlo path tracer already involves several design choices concerning the techniques for sampling materials and emitters, and their combination, e.g. via multiple importance sampling (MIS). Differentiation causes a veritable explosion of this decision tree: should we differentiate only the estimator, or also the sampling technique? Should MIS be applied before or after differentiation? Are specialized derivative sampling strategies of any use? How should visibility-related discontinuities be handled when millions of parameters are differentiated simultaneously? In this paper, we provide a taxonomy and analysis of different estimators for differential light transport to provide intuition about these and related questions.

Downloads and links

Media

Supplemental video

Fast-forward video

Summary video

Talk video

BibTeX reference

@article{Zeltner:2021:DifferentialEstimators,
  author = {Tizian Zeltner and S\'{e}bastien Speierer and Iliyan Georgiev and Wenzel Jakob},
  title = {Monte Carlo Estimators for Differential Light Transport},
  journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH)},
  volume = {40},
  number = {4},
  year = {2021},
  month = aug,
  keywords = {specular light paths, SDS paths, caustics, glints},
  doi = {10.1145/3450626.3459807}
}