The architecture. TabPFN (Hollmann et al., 2022; v2 2025) is a transformer with a permutation-invariant attention pattern over feature columns and example rows. At training time, it is fed millions of synthetic datasets drawn from a hypothesis space defined by structural causal models. The training objective is approximate Bayesian inference: predict the held-out target given the context examples, marginalised over plausible data-generating processes.

In-context learning, not fine-tuning. At inference time, the model is given the labelled training rows as part of its input prompt, along with the unlabelled test row. It produces a posterior over the target. No gradients are computed; no parameters are updated. The "learning" is purely conditional inference within a single forward pass.

Theoretical foundation. Müller et al. (2022) showed PFNs perform approximate Bayesian inference for the prior they were trained on. The prior over synthetic SCMs is broad enough to cover most real-world tabular patterns, which is why the approach transfers. Where the prior is wrong, the model degrades gracefully or catastrophically depending on the mismatch.

Operational characteristics. Strong on small datasets (≤10K rows, ≤100 features) — frequently competitive with tuned XGBoost on standard benchmarks. Limited by context window: typical implementations support a few thousand context rows. TabPFN v2 (2025) extends to larger contexts and adds support for missing data, categorical features, and regression.

Limitations. Cannot capture bank-specific patterns absent from its synthetic prior. Inference is more expensive than a fitted XGBoost (transformer forward pass vs tree traversal). Not yet routinely used for high-volume real-time prediction at bank scale; better as a "first-look" model or for low-volume use cases.

The system. Kumo (Kumo.AI, 2022 founding; KumoRFM commercial release 2024) is a foundation model architecture for relational data — explicitly designed to accept multi-table relational schemas (or knowledge graphs) and answer predictive queries without per-task fine-tuning. Founded by authors of PyTorch Geometric (Matthias Fey, Jure Leskovec).

Architecture. Combines graph-transformer encoders over the relational schema with temporal attention for time-series-shaped queries. Pre-trained on a large corpus of public relational datasets across e-commerce, telecom, healthcare, and finance — the analogue of GPT’s text corpus, but for relational structure. The published architecture is a member of the broader "relational deep learning" family (Fey et al. 2023).

PQL — Predictive Query Language. A SQL-flavoured DSL for declaring predictive tasks: PREDICT <target> FOR <entity> WHERE <filter> AT <time-cutoff>. PQL handles the canonical hard parts of ML productionisation: temporal leakage prevention (target after cutoff), entity scoping, target definition. The compilation target is a graph encoding plus an attention-based readout.

Zero-shot operation. The model is shipped pre-trained. To deploy on a new bank’s graph, you connect the data sources, declare the schema (Kumo can auto-infer much of this), and write PQL queries. No labelled training set is required for the first prediction, though labels improve accuracy as they accrue.

Reasoning trails. A first-class feature: each prediction is accompanied by feature-importance breakdowns at the graph level (which neighbours mattered, which relationship paths contributed). This explainability story is what makes the model viable for regulated industries; it is also the most direct point of comparison versus opaque GNN architectures.

Limitations. Vendor-managed and priced. Less transparent than open-source alternatives for what exactly the model has learned. Best-fit when the bank values cycle-time and managed-service economics over open-source control. Performance gap vs custom-trained classical ML on stable use cases is typically 2-5 percentage points.

The library. PyTorch Geometric (Fey & Lenssen 2019) is the dominant open-source library for graph neural networks. Implements ~70 GNN layers, scalable mini-batching via NeighborLoader, heterogeneous graph support (HeteroData), temporal graphs, and integrations with PyTorch Lightning. Active development by NVIDIA, Stanford, and the Kumo team.

GraphSAGE (Hamilton, Ying, Leskovec 2017). Sample-and-aggregate inductive GNN. At each layer k, for each node v: sample a fixed-size set of neighbours N(v), aggregate (mean/max/LSTM), concatenate with v’s own previous representation, project through learned weights. The inductive property — generalises to unseen nodes — is what makes it deployable in production.

GAT (Veličković et al. 2018). Attention mechanism replaces uniform aggregation. α_{ij} = softmax_j(LeakyReLU(a^T [W h_i || W h_j])). Multi-head attention for stability (typically 4-8 heads). Provides per-edge interpretability — the attention coefficient α_{ij} is the model’s claim about how much neighbour j mattered for node i’s prediction.

R-GCN (Schlichtkrull et al. 2018). Generalises GCN to multi-relational graphs. Separate weight matrices W_r per relation type r; the update aggregates over each relation separately, then sums. Basis-decomposition regularisation keeps parameter counts tractable when there are many relations (the standard banking case).

Heterogeneous graphs in production. A bank’s graph has multiple node types (Customer, Account, Advisor, Product, Transaction) and edge types. PyG’s HeteroData class plus HGT (Heterogeneous Graph Transformer, Hu et al. 2020) is the modern default for production deployments.

Operational considerations. Training requires labelled data — typically 100K-1M labelled examples for stable churn / fraud / next-product models. Inference is fast (milliseconds per node). Standard MLOps applies: drift monitoring, periodic retraining, A/B testing. Cost-effective when the bank has an ML platform team; expensive in engineering time if it doesn’t.

The design pattern. Predictive Graph Networks combine three architectural choices: (i) a graph encoder (typically GraphSAGE, GAT, or HGT) that produces node embeddings, (ii) temporal-aware message passing that weights recent edges more than old ones, and (iii) multiple task-specific prediction heads that share the encoder. The pattern is more important than any single named architecture.

Temporal edge weighting. Each edge carries a timestamp. During message aggregation, the contribution of neighbour j to node i is scaled by a decay function δ(t_now − t_{ij}) — exponential decay being the simplest, learned decay being the most expressive. TGN (Rossi et al. 2020) and TGAT (Xu et al. 2020) are the canonical reference architectures.

Multi-task heads. Given a shared node embedding z_v ∈ ℝ^d, separate task-specific heads h_k: ℝ^d → ℝ^{o_k} compute each target. Training combines losses: L = Σ_k λ_k L_k(h_k(z_v), y_k). The λ_k weights balance tasks; uncertainty weighting (Kendall et al. 2018) is the standard automated approach. The shared encoder learns a representation that’s useful for all tasks, which improves data efficiency.

Coherence guarantees. Because predictions share an encoder, they are mutually consistent in a way that independent per-task models are not. If the encoder thinks Marcus looks like a churn risk, his next-product prediction will also reflect that — no "we predict churn but also predict mortgage upsell" incoherence.

Operational characteristics. Higher upfront engineering investment than single-task models. Lower amortised cost when running 3+ predictions per customer. Faster inference per prediction (encoder runs once). Better cold-start behaviour on new customers (the shared encoder transfers across tasks).

What this is not. "PGN" is not a single library or product. It’s a design pattern implemented many ways — pyg-team examples, internal bank architectures, components of Kumo’s commercial offering. The term in the article is a useful umbrella; treat it as a category, not a specific tool.

The library. PyTorch Frame (pyg-team, 2024) is an open-source library for deep learning on multi-table relational data. Companion project to PyTorch Geometric, designed by the same team to lower the on-ramp for engineers who think tabular-first. Implements stype-aware column handling (numerical, categorical, text-embedded, timestamp, multicategorical) and standard neural tabular architectures.

Architectures shipped. ResNet (Gorishniy et al. 2021 baseline), FT-Transformer (Gorishniy et al. 2021), TabTransformer (Huang et al. 2020), Trompt (Chen et al. 2023). Each can be combined with PyG-extracted features from a graph backbone — the integration is first-class.

Graph-augmented tabular features. The library’s practical sweet spot. Given a customer table and a graph of relations, extract neighbourhood-aggregated features as additional columns: mean AUM of advisor’s clients, variance of household income, count of products held by similar customers. These columns plug into XGBoost, LightGBM, CatBoost, or PyTorch Frame neural models interchangeably.

Foundation model components. PyTorch Frame integrates with HuggingFace tabular models and ships its own pre-trained checkpoints for some tasks. The pre-training corpus is smaller than TabPFN’s but more bank-relevant for the financial-services-trained checkpoints.

Operational adoption pattern. Phase 1 — augment existing XGBoost pipelines with graph features (no model change, lift comes from better features). Phase 2 — try neural tabular models (FT-Transformer) on the augmented dataset (often a 1-2 point lift). Phase 3 — graduate to native GNNs when the team has the engineering bandwidth. The library supports all three phases without rewriting data pipelines.

Limitations and honest framing. PyTorch Frame is younger than PyG; the documentation and community are smaller. Pre-trained checkpoint quality lags behind TabPFN on small tabular benchmarks and behind native GNNs on large graph benchmarks. It is not the peak-performance answer; it is the practical-adoption answer.

The benchmark. RelBench (Robinson, Ranjan, Hu, Yeh, Shirzad, Hilger, Fey, Leskovec; NeurIPS 2024) is an open benchmark for evaluating relational deep learning on real-world multi-table datasets with temporal structure. Datasets cover e-commerce (rel-amazon), Q&A (rel-stack), sports (rel-f1), healthcare (rel-trial), and others. Each dataset ships with standardised predictive tasks, train/val/test splits respecting temporal cutoffs, and reference baselines.

What it measures. Standard predictive metrics (AUC, accuracy, MAE) plus latency and resource consumption. Crucially, it provides a fair comparison surface: same task, same data split, same evaluation protocol across classical ML (LightGBM, XGBoost) and relational deep learning (GraphSAGE, Hetero GNNs, foundation models).

Published results pattern. Across the RelBench tasks: tuned LightGBM/XGBoost is competitive everywhere and leads on roughly half the tasks. GNN-based RDL leads on the other half, particularly tasks with rich relational structure and temporal dependencies. Foundation-model zero-shot approaches close the gap to within a few points on most tasks without per-task training. No method dominates universally.

Why the benchmark matters strategically. Before RelBench, banks evaluating zero-shot approaches had to rely on vendor benchmarks (with predictable biases) or build their own evaluation infrastructure (costly and slow). RelBench provides an open, citable baseline that lets the bank evaluate "would this approach work on our data?" by anchoring on published comparisons.

The honest five-metric table for the bank. For each candidate use case, score: accuracy, time-to-deploy, cost-of-ownership, explainability, rare-event coverage. The right model often differs by metric. A mature 2026 bank’s ML portfolio uses classical ML for credit/fraud (regulatory, stable, high-volume) and zero-shot for new products / cold-start / rapid experimentation (where cycle-time dominates). The portfolio mix — not the per-task winner — is the strategic decision.

What this lab’s comparison demo shows. An illustrative side-by-side on Marcus’s churn: TabPFN, Kumo, GraphSAGE, GAT, PGN, PyTorch Frame, and XGBoost (12-week baseline). The numbers are representative of typical patterns in published benchmarks, not from a specific RelBench run.