Cluster is about turning racks into one machine. Scale-out and scale-across networks, the optics that connect them, and keeping 100K GPU training jobs and large inference fleets running. From the teams who design and operate AI clusters at hyperscalers and neoclouds.
Who it's for
VP, AI InfrastructureDirector, ML PlatformPrincipal Network ArchitectHead of GPU Cloud EngineeringCluster Operations LeadSite Reliability Engineering Manager
Cluster track topics at Digital Infrastructure World 2027
The Cluster track at Digital Infrastructure World 2027, held in collaboration with the Open Compute Project, home of OCP Tech Day. San Jose, February 3-4, 2027. Sessions are being confirmed; these are the topics on the agenda.
01 · Training & Reliability
Training and reliability: goodput beats peak FLOPS
Meta trained Llama 3 405B on 16,384 H100 GPUs over 54 days and logged 419 unexpected interruptions, roughly one every three hours. At 100,000 GPUs and beyond, failure stops being an event and becomes the weather.
This block is about the clusters built to keep training through it: fault tolerance that restarts a small group rather than the whole job, faster checkpointing, straggler detection, silent data corruption, and the goodput and utilization numbers that decide whether a training cluster earns back its capex. At the 2026 OCP Global Summit Meta described Parallelism-Aware Fault Tolerance, which isolates restarts to smaller parts of the cluster.
Agentic inference: many agents, long contexts, tokens per megawatt
Agents are taking over inference. SemiAnalysis told the 2026 OCP Global Summit that about 64% of the traffic it measured came from agents, with tools such as Claude Code and Codex running many sub-agents in parallel.
Agentic work changes the shape of the load: long contexts, many parallel sessions and users who wait on every token. Sessions cover disaggregated serving, with compute-bound prefill and memory-bound decode on separate pools, KV-cache transfer and offload, interactivity measured in tokens per second per user, and the metric the industry is converging on: tokens per provisioned megawatt.
Topics include
Agentic AI inference
Disaggregated prefill and decode
KV cache offload
Tokens per second per user
Tokens per megawatt
03 · Scale-Out & Across
Scale-out and scale-across: when one building is not enough
The cluster has outgrown the building. Meta's Prometheus spans several buildings at 1 GW, and Microsoft links its Fairwater sites in Wisconsin and Atlanta, roughly 700 miles apart, over a dedicated AI WAN.
This block follows the back-end network from the rack outward: 102.4T switch silicon, Ultra Ethernet (UEC Specification 1.0, June 2025), packet spraying and scheduled fabrics, deep-buffer data center interconnect, and the speed-of-light budget for training across sites. Scale-up inside the rack belongs to the Rack track at OCP Tech Day.
Neoclouds have gone from renting out GPUs to building some of the largest clusters in the market, and they design for one number above all: the days between hardware delivery and a paying workload.
The block compares reference designs for clusters of 10,000 to 100,000+ GPUs: topology and oversubscription, storage tiers, multi-tenancy and isolation, and what it takes to hand a customer a validated cluster rather than a hall full of servers.
Topics include
Neocloud AI cluster design
100K GPU cluster
Multi-tenant AI clusters
Oversubscription ratios
Cluster reference architectures
05 · Test & Validation
Test, validation and operations: congestion, software and checkpoints
When Meta scaled RoCE for AI training, standard ECMP load balancing struggled with the low flow entropy of collective traffic. Reworking routing improved AllReduce performance by up to 40%, by Meta's account. Most cluster gains are won in operations, not in the purchase order.
Four topics: congestion control in AI fabrics (RoCEv2, PFC, DCQCN and receiver-driven alternatives), open networking software with OCP roots such as SAI and SONiC, cluster test and validation before handover, and storage and checkpointing, including asynchronous and in-memory checkpoints that cut recovery time.
In a million-GPU estate, every point of utilization is worth about 10,000 GPUs. That is why the bill of materials and the cost per token are now the same conversation.
The closing panel follows the money from components to tokens: GPUs, network, optics, power and cooling as shares of cluster capex, where open designs take cost out, and how utilization and depreciation decide who makes money on AI infrastructure.
Kiran ThakkarMaster Principal Cloud ArchitectOracle
Jigar SavlaSilicon Architect and Product ManagerTensordyne