Conditional Mean Estimation in Gaussian Noise: A Meta Derivative Identity With Applications

Consider a channel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {Y}= \mathbf {X}+ \mathbf {N}$ </tex-math></inline-formula> where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {X}$ </tex-math></inline-formula> is an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> -dimensional random vector, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {N}$ </tex-math></inline-formula> is a multivariate Gaussian vector with a full-rank covariance matrix <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {\mathsf {K}}_{ \mathbf {N}}$ </tex-math></inline-formula> . The object under consideration in this paper is the conditional mean of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {X}$ </tex-math></inline-formula> given <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {Y}={\mathbf{y}}$ </tex-math></inline-formula> , that is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${\mathbf{y}} \mapsto \mathbb {E} [\mathbf {X}| \mathbf {Y}={\mathbf{y}}]$ </tex-math></inline-formula> . Several identities in the literature connect <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbb {E}[\mathbf {X}| \mathbf {Y}={\mathbf{y}}]$ </tex-math></inline-formula> to other quantities such as the conditional variance, score functions, and higher-order conditional moments. The objective of this paper is to provide a unifying view of these identities. In the first part of the paper, a general derivative identity for the conditional mean estimator is derived. Specifically, for the Markov chain <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {U}\leftrightarrow \mathbf {X}\leftrightarrow \mathbf {Y}$ </tex-math></inline-formula> , it is shown that the Jacobian matrix of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbb {E}[\mathbf {U}| \mathbf {Y}={\mathbf{y}}]$ </tex-math></inline-formula> is given by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {\mathsf {K}}_{ \mathbf {N}}^{-1} \boldsymbol {\mathsf {Cov}} (\mathbf {X}, \mathbf {U}| \mathbf {Y}={\mathbf{y}})$ </tex-math></inline-formula> where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {\mathsf {Cov}} (\mathbf {X}, \mathbf {U}| \mathbf {Y}={\mathbf{y}})$ </tex-math></inline-formula> is the conditional covariance. In the second part of the paper, via various choices of the random vector <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {U}$ </tex-math></inline-formula> , the new identity is used to recover and generalize many of the known identities and derive some new identities. First, a simple proof of the Hatsel and Nolte identity for the conditional variance is shown. Second, a simple proof of the recursive identity due to Jaffer is provided. The Jaffer identity is then further explored, and several equivalent statements are derived, such as an identity for the higher-order conditional expectation (i.e., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbb {E}[\mathbf {X}^{k}| \mathbf {Y}]$ </tex-math></inline-formula> ) in terms of the derivatives of the conditional expectation. Third, a new fundamental connection between the conditional cumulants and the conditional expectation is demonstrated. In particular, in the univariate case, it is shown that the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -th derivative of the conditional expectation is proportional to the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(k+1)$ </tex-math></inline-formula> -th conditional cumulant. A similar expression is derived in the multivariate case.