Graph Attention Network (GAT)

GAT asked a simple question about GCN: why weight every neighbor the same way? Instead of the fixed renormalization of the adjacency matrix, a GAT layer computes attention coefficients between a node and each of its neighbors with a small shared feed-forward network, then aggregates as a weighted sum. Multi-head attention (8 heads on the first layer in the paper's transductive setup) stabilizes training, mirroring the Transformer recipe. The mechanism is computationally efficient, needs no costly matrix operations or full knowledge of the graph structure upfront, and works inductively on unseen nodes.\n\nThe ICLR 2018 paper reports 83.0% on Cora, 72.5% on Citeseer, 79.0% on Pubmed and 0.973 micro-F1 on PPI, beating GCN on every benchmark and setting a new state of the art at the time. The paper's ablation is telling: a constant-attention variant of the same architecture drops PPI F1 to 0.934, showing the attention itself does the work. Attention over graphs became a standard primitive after GAT, appearing in GATv2, graph transformers and countless domain applications.

Paper-reported: 83.0% Cora, 72.5% Citeseer, 79.0% Pubmed, 0.973 micro-F1 on PPI (Velickovic et al., ICLR 2018). Independent community reimplementations reproduce the paper's Table 2 within run variance.