NetWarren_

Lessons › AI infrastructure networking

Why AI needs a special network

11 min lesson · AI infrastructure networking · free

▶ Start the interactive lesson

A company has bought 512 GPUs to train its own AI model. They're expensive: each GPU server costs more than a house. Week one, the training job runs at half the expected speed. The GPUs are fine. The servers are fine. The problem is the network. Sam: "In AI clusters, the network isn't just…

GPUs work as one big team

AI models are too big to train on one GPU. The work is split across hundreds or thousands of GPUs, in many servers.

Why normal networks aren't enough

Normal data centre traffic is lots of small, random flows. A few dropped packets just get resent and nobody notices.

Two networks, not one

AI clusters usually have separate networks: a front-end network (users, storage, management, like a normal data centre) and a back-end or compute fabric just for GPU-to-GPU traffic.

Two ways to build the GPU fabric

InfiniBand: a network built for supercomputers. Lossless, very low latency, but its own technology and tools.

Key points

Words you'll learn

Collective communication · All-reduce · Front-end network · Back-end / compute fabric · NVLink / scale-up network · Scale-out network · Tail latency matters · GPU cluster / AI factory / SuperPOD

The full lesson has more: animations, a hands-on exercise, 3 quiz questions, a practical scenario test and spaced review cards. Open it free →