Deep Learning from Scratch to GPU - 15 - Weight Decay

In this article we explore a simple but useful technique for keeping weights from growing too big. Weight Decay is useful as a regularization technique that improves generalization, and can help with improving even the basic learning on the technical level. If you haven't yet, read my introduction to this series in Deep Learning in Clojure from Scratch to GPU - Part 0 - Why Bother?. The previous article, Part 14, is here: Learning a Regression. To run the code, you need a Clojure project ...

Submit