When high-level features (such as concepts or aggregated input features) are provided by an explanans, they should capture as much information as possible about the model's output. In turn, the prediction should be reflected in the explanans, implying a high degree of mutual dependency between both.
To quantify this relationship, [Nguyen and Martínez (2020)] leverage the Mutual Information (MI) (see [Cover (1999)]) between the explanans and the output: $MI(e_{\theta,x,\hat{y}}, \theta(x))$. Since MI is symmetric, a high score indicates that the explanation both reflects (i.e. correctness) and encompasses (i.e., completeness) the relevant reasoning of the model.