Optimizing over multinomial distributions

Sometimes you have to maximize some function f(w_1, w_2, ldots, w_n) where w_1 + w_2 + ldots + w_n = 1 and 0 le w_i le 1 . Usually, f is concave and differentiable, so there’s one unique global maximum and you can solve it by applying gradient ascent. The presence of the constraint makes it a little tricky, but we can solve it using the method of Lagrange multipliers. In particular, since the surface w_1 + w_2 + ldots + w_n has the normal (1, 1, ldots, 1) , the following optimization procedu...

Submit