The End of TCP for AI Clusters — John Ousterhout, Stanford
AI workloads are shifting towards smaller transfers where latency is crucial.
“Homa...can reduce tail latency by an order of magnitude or more.”
John Ousterhout discussed the changing nature of AI workloads, emphasizing the increasing importance of latency over throughput. He introduced Homa, a new protocol developed at Stanford that significantly reduces tail latency for small data transfers, addressing limitations of legacy protocols like TCP.