Abstract
While uncertainty estimation for graphs recently gained traction, most methods rely on homophily and deteriorate in heterophilic settings. We address this by analyzing message passing neural networks from an information-theoretic perspec tive and developing a suitable analog to data processing inequality to quantify information throughout the model’s layers. In contrast to non-graph domains, information about the node level prediction target can increase with model depth if a node’s features are semantically differ ent from its neighbors. Therefore, on heterophilic graphs, the latent embeddings of an MPNN each provide different information about the data distri bution – different from homophilic settings. This reveals that considering all node representations simultaneously is a key design principle for epis temic uncertainty estimation on graphs beyond homophily. We empirically confirm this with a simple post-hoc density estimator on the joint node embedding space that provides state-of-the art uncertainty on heterophilic graphs. At the same time, it matches prior work on homophilic graphs without explicitly exploiting homophily through post-processing
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
"Uncertainty Estimation for Heterophilic Graphs Through the Lens of Information Theory", Technical Disclosure Commons, (July 08, 2025)
https://www.tdcommons.org/dpubs_series/8325