<i>L</i> ¹ Estimation: On the Optimality of Linear Estimators

Consider the problem of estimating a random variable X from noisy observations <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Y = X+ Z$ </tex-math></inline-formula>, where Z is standard normal, under the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L^{1}$ </tex-math></inline-formula> fidelity criterion. It is well known that the optimal Bayesian estimator in this setting is the conditional median. This work shows that the only prior distribution on X that induces linearity in the conditional median is Gaussian. Along the way, several other results are presented. In particular, it is demonstrated that if the conditional distribution <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$P_{X|Y=y}$ </tex-math></inline-formula> is symmetric for all y, then X must follow a Gaussian distribution. Additionally, we consider other <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L^{p}$ </tex-math></inline-formula> losses and observe the following phenomenon: for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p \in [{1,2}]$ </tex-math></inline-formula>, Gaussian is the only prior distribution that induces a linear optimal Bayesian estimator, and for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p \in (2,\infty)$ </tex-math></inline-formula>, infinitely many prior distributions on X can induce linearity. Finally, extensions are provided to encompass noise models leading to conditional distributions from certain exponential families.