Graphormer

Graphormer asked whether the plain Transformer could be the best graph learner if you simply teach it about graph structure. The answer turned out to be yes, with three structural encodings doing the teaching: centrality encoding adds node degree information to the input, spatial encoding biases attention by shortest-path distance between node pairs, and edge encoding injects edge features along those paths. With those additions, an otherwise standard Transformer encoder captures both local neighborhoods and long-range dependencies that message-passing GNNs struggle to reach.\n\nGraphormer won first place in the quantum-prediction track of the OGB Large-Scale Challenge (KDD Cup 2021), and the authors' repo reports 0.1234 validation MAE on PCQM4M-LSC, 31.39% AP on MolPCBA, 80.51% AUC on MolHIV and 0.122 MAE on ZINC, surpassing prior models by large margins. The NeurIPS 2021 paper launched the graph-transformer wave that now dominates molecular property prediction, with GPS, GraphGPS and successors all building on its encodings.

Repo/paper-reported: 0.1234 valid MAE on PCQM4M-LSC (1st place, OGB-LSC KDD Cup 2021); 31.39% AP MolPCBA, 80.51% AUC MolHIV, 0.122 MAE ZINC. Competition placement provides independent validation of the PCQM4M result.