Mean-Field Variational Inference
Approximate posteriors that factorize — fast, but blind to correlations.
Motivation
Mean-field variational inference approximates an intractable posterior with a fully factorized distribution, turning inference into optimization. It is fast and scalable, at the cost of ignoring dependencies between latent variables.
Method
Under the mean-field assumption the ELBO admits coordinate-ascent updates in closed form for conjugate models: each factor is updated to the exponentiated expected log-joint under the others.
\[ q^\star(z) = \prod_i q_i(z_i), \qquad \log q_j^\star(z_j) \propto \mathbb{E}_{-j}\big[\log p(x,z)\big] \]
Mathematical core
Minimizing the reverse KL from approximation to posterior is what gives mean-field its characteristic under-dispersion. Understanding that bias — and when it matters — motivates richer families and the automatic methods below.