InfiniBand NDR / XDR
The lowest-latency fabric for the largest Bare Metal training jobs
Overview
InfiniBand NDR is the premium fabric option for Bare Metal deployments of AMD MI350X, MI300X and NVIDIA H200 — the GPUs used in our largest training jobs. Choose InfiniBand when your workload demands the absolute lowest AllReduce latency across hundreds of GPUs. InfiniBand is available on Bare Metal instances in our Glasgow, London and Texas data centres, with Managed Slurm providing native Slurm scheduler support for IB-aware job placement.
Key specifications
- Bandwidth
- 400 Gbps NDR / 800 Gbps XDR
- Latency
- <1 µs port-to-port
- Topology
- Fat-tree with full bisection
- Routing
- Adaptive routing with SHARP v2
- Collectives
- NCCL (NVIDIA) / RCCL (AMD)
- Available form factors
- Bare Metal only
Core advantages
Optimised for Bare Metal MI350X, MI300X and H200
Our 8-way Bare Metal servers for AMD MI350X (288GB HBM3e), MI300X (192GB HBM3) and NVIDIA H200 (141GB HBM3e) are wired with 400G NDR InfiniBand for maximum collective throughput.
Managed Slurm with IB-aware scheduling
Slurm places jobs with InfiniBand topology awareness — NCCL for NVIDIA and RCCL for AMD tune their transport automatically to the IB subnet, with SHARP in-network computing available for AllReduce offload.
Direct-attached in Glasgow, London and Texas
Each data centre runs a dedicated IB subnet managed by a Subnet Manager, with full-bisection fat-tree topology — no cross-fabric bridging required within a region.
Paired with our shared filesystem
InfiniBand-connected Bare Metal nodes mount our Shared Filesystem (500+ GB/s) over RDMA, so checkpoint writes and dataset reads stay on the fabric without TCP overhead.
Ideal for
Ready to put Technova to work?
Talk to our team about a custom GPU cluster, managed Slurm or one of our vertical AI solutions.