Honey, my pdf is greater than 1!

This is a confusing concept because of the density function for discrete distributions. These have a probability mass function, not a probability density function. So the interpretation here is straightforward, if we evalute the PMF at any point, $X = x$, that value corresponds to the probability of $X = x$ occurring. So if we toss a coin, then $p(X = \text{Head})$ is the probability of observing a head. When we try to extend this logic to probability density functions, weird things happen. Beca...