bayesianapproximationexpectation-maximization

updates in mean-field variational inference

I'm wondering if ELBO should be increasing monotonically as every update of the variational parameter q_i? I understand that it should increase at every iteration, but is this also true within one iteration for each q_i?

Solution

  • Yes, it is also true. Thinks it is a coordinate ascent method:

    q_i^t = argmax_{q_i} \cal{L}(q_1^{t}, ...,q_{i-1}^{t}, q_i, q_{i+1}^{t-1}, ..., q_n^{t-1}).

    so you are always maximizing.