MMSE Bounds Under Kullback-Leibler Divergence Constraints on the Joint\n Input-Output Distribution

This paper proposes a new family of lower and upper bounds on the minimum\nmean squared error (MMSE). The key idea is to minimize/maximize the MMSE\nsubject to the constraint that the joint distribution of the input-output\nstatistics lies in a Kullback-Leibler divergence ball centered at some Gaussian\nreference distribution. Both bounds are tight and are attained by Gaussian\ndistributions whose mean is identical to that of the reference distribution and\nwhose covariance matrix is determined by a scalar parameter that can be\nobtained by finding the root of a monotonic function. The upper bound\ncorresponds to a minimax optimal estimator and provides performance guarantees\nunder distributional uncertainty. The lower bound provides an alternative to\nwell-known inequalities in estimation theory, such as the Cram\\'er-Rao bound,\nthat is potentially tighter and defined for a larger class of distributions.\nExamples of applications in signal processing and information theory illustrate\nthe usefulness of the proposed bounds in practice.\n