Entropy, Cross-Entropy and KL-Divergence

1 · Shane Mulligan · Nov. 1, 2019, 11 a.m.
Original video A Short Introduction to Entropy, Cross-Entropy and KL-Divergence - YouTube Related reading https://blog.floydhub.com/knowledge-distillation/ Glossary 1 2 3 4 5 6 7 8 9 10 11 12 13 marginalized Treated as insignificant or peripheral. marginal likelihood function integrated likelihood model evidence evidence [#statistics] [#bayesian statistics] A likelihood function in which some parameter variables have been marginalized. Predicted distribution vs true distribution Predicted di...