Elon Musk's xAI Colossus: How 200,000 GPUs Produced Grok 3, Grok 4, and a New AI Contender

Colossus: From Announcement to Production

When xAI unveiled Colossus in September 2024, it was already operational - a 100,000 H100 GPU cluster built in Memphis, Tennessee in just eight months. Located in a repurposed industrial facility, the supercomputer draws up to 150 megawatts of power, requiring a dedicated energy arrangement to guarantee uninterrupted supply. For context: OpenAI's GPT-4 was reportedly trained on approximately 25,000 A100 GPUs. Colossus represented a fourfold increase in raw training compute and signalled that xAI was prepared to compete with the largest AI labs in the world on infrastructure as well as capability.

The cluster has since been expanded to 200,000 GPUs - a scale that makes it one of the largest AI training installations in existence. The expansion, completed in early 2025, doubled the original capacity and was the foundation for xAI's accelerated model release cadence.

What Colossus Produced: Grok 3 and Grok 4

Grok 3, launched on February 17, 2025, was the first flagship model trained on the full Colossus infrastructure. xAI claimed it was trained with ten times the compute of its predecessor Grok 2, and benchmarks showed strong performance on mathematics and reasoning tasks. The model scored 93.3% on the American Invitational Mathematics Examination and 84.6% on graduate-level GPQA reasoning - competitive with OpenAI and Google's leading models at the time of release.

By July 9, 2025, xAI had released Grok 4 - its current flagship. Grok 4 introduced native tool use, real-time search integration, and a multi-agent "Heavy" variant aimed at the most demanding professional workflows. The pace of release - two major flagship models in five months - would have been inconceivable without the Colossus infrastructure underpinning the training pipeline.

The Energy Question

At full load, Colossus consumes as much electricity as a mid-sized city. xAI's initial reliance on natural gas to guarantee power reliability drew criticism from climate advocates. The company has pointed to future nuclear power agreements as a pathway to net-zero operations, though confirmed timelines remain unclear. The energy footprint of frontier AI training has become one of the most significant environmental policy questions of the decade - and Colossus, as the largest such facility, sits at the centre of that debate.