MMSE Bounds Under Kullback–Leibler Divergence Constraints on the Joint Input-Output Distribution

This paper proposes new lower and upper bounds on the minimum mean squared error (MMSE). The key idea is to minimize/maximize the MMSE subject to the constraint that the joint distribution of the input-output statistics lies in a Kullback–Leibler divergence ball centered at some Gaussian distribution. The bounds are shown to hold for a larger family of distributions than the Cramér–Rao bound and are shown to be sharper than the Cramér–Rao bound in some regimes.