Graph Attention Network (GAT)

图注意力网络：一类带学习式边权重的图神经网络，股票 = 节点，相关性/行业/因子暴露 = 边。

Definition

A Graph Attention Network (Veličković et al. 2018) is a message-passing neural network in which, for each edge (i,j), a learned attention coefficient \alpha_{ij}^{(h)} (head h) replaces the fixed aggregation weight used in vanilla Graph Convolutional Networks. Each layer updates a node's latent embedding by attending to its neighbours, with attention computed from the node features and (in extended variants) from edge attributes.

GAT is the relational encoder in the Temporal-Heterogeneous GNN (THGNN) of Fanshawe Masih Cameron 2025 THGNN Correlation Forecasting; the same architecture is used for the equity-network propagation step that follows the Transformer temporal encoder.

Why attention matters for financial graphs

Fixed-weight GNNs (GCN) treat every neighbour as equally informative. In equity networks this is wrong: a stock's correlation with the energy sector matters much more when oil is regime-defining (2020 oil crash, 2022 Russia-Ukraine) than when it is not. GAT's learned attention gives the model a way to dynamically reweight neighbours per regime.

Empirically, in finance tasks GAT-based models outperform GCN and LSTM baselines on Sharpe and ranking metrics (Xiang et al. 2023; Sawhney et al. 2021).

Core mechanism — edge-aware multi-head attention

For head h at layer l, node i's update aggregates neighbours via

h_i^{(l+1, h)} = \sigma\!\left(\sum_{j \in \mathcal{N}(i)} \alpha_{ij}^{(l, h)} \, W^{(l, h)} h_j^{(l)}\right)

where the attention coefficient itself is conditioned on edge state, node features, and (in THGNN) edge attributes a_{ij} and a discrete relation class \tau_{ij} \in \{\text{neg}, \text{mid}, \text{pos}\}:

\alpha_{ij}^{(l, h)} = \text{softmax}_{j}\!\left(\text{LeakyReLU}\!\left(\mathbf{a}^{T}\!\left[W h_i | W h_j | g_{ij}^{(l)}\right]\right)\right)

with the edge gate g_{ij}^{(l)} a function of edge attributes, sector/subindustry flags, and a persistent edge state that propagates relational context across layers. The heads' outputs are concatenated, projected, residual-added, layer-normalised.

Heterogeneous / edge-aware extensions for finance

Three extensions matter for the Fanshawe Masih Cameron 2025 THGNN Correlation Forecasting application:

Edge attributes as input to attention: baseline correlation \rho_{\text{base}}(i,j,t), |\rho_{\text{base}}|, sign indicator, sector / subindustry membership.
Relation-class routing: pairwise edge embeddings are routed to one of k expert MLP heads by a discretised relation class (negative / mid / positive). This is what lets the model learn that the dynamics of "stocks that are weakly correlated" are categorically different from "stocks that are strongly correlated".
Persistent edge state: e_{ij}^{(l)} carries across layers, accumulating higher-order relational information (e.g., "stock i and j share two common neighbours with high attention to k").

Practical choices in THGNN

The Fanshawe–Masih–Cameron GAT uses L_g = 3 layers, H = 4 heads, d = 512 node embeddings. The choice of fewer but wider heads in the GAT (4 heads vs 8 in the Transformer) is deliberate: each node has many dense neighbours and the conditioning surface is edge-features, not time, so per-head capacity matters more than head count.

Interpretability — and its limits

GAT attention weights are intrinsically interpretable in a way Transformer attention is not: they act directly on the final node embeddings, so they can be read as "how much did neighbour j contribute to i's prediction?". Averaging across heads and aggregating to sector level gives time series of sector-to-sector attention. THGNN's sector attention reveals:

Energy: persistent attention to Financials / Health Care / IT; intra-sector attention spikes 0.05 → 0.35 during COVID-19 oil crash, reverts within months.
Financials: broad attention, top-5 sectors cluster 12–16%; idiosyncratic volatility shifts attention toward IT / Consumer Discretionary in late 2020.
Real Estate: highly volatile with spikes >0.7, often to Health Care without obvious economic mechanism — likely a regime-shift indicator.

Caveat (Jain & Wallace 2019): high attention ≠ causal importance, especially in deep multi-layer architectures. Attention is an informative summary of internal weighting, not a formal attribution.

When GAT alone is not enough

GAT handles cross-asset structure at a snapshot. It does not, by itself, model temporal evolution of dependencies — and that is precisely the limitation rolling-window correlation has. THGNN's contribution is the joint stacking: a Transformer first compresses 30 days of per-stock features into a context-aware 512-d embedding, then GAT propagates those embeddings across the equity network. Either component alone underfits; together they reduce MAE from 0.307 (persistence baseline) to 0.230 and lift Pearson r from 0.310 to 0.778 on 269 M out-of-sample edges.

Notation summary

Symbol	Meaning
h_i^{(l)}	Node i's embedding at layer l
\alpha_{ij}^{(l,h)}	Attention coefficient from i to j, head h, layer l
W^{(l,h)}	Head-specific linear projection at layer l
a_{ij}	Edge attributes (baseline corr, sign, sector flags)
\tau_{ij}	Discrete relation class {neg, mid, pos} for expert routing
e_{ij}^{(l)}	Persistent edge state, propagated across layers
g_{ij}^{(l)}	Edge-conditioned gate function