Understanding NUTS and HMC

1 · George Ho · Jan. 7, 2021, midnight
“Bayesian modeling is harder than deep learning” is a sentiment I’ve been hearing a lot lately. While I’m skeptical of sweeping statements like that, I agree when it comes to the central inference algorithm — how MCMC samplers work (especially the de facto standard samplers, NUTS and HMC) is one of the most difficult concepts I’ve tried to learn, and is certainly harder than autodifferentiation or backpropagation. So I thought I’d share what worked for me when I tried to teach myself NUTS and HM...