AI has a performance problem. I’m not talking about the models themselves. 

In the nearly three years since the launch of ChatGPT, models have gotten better, smarter, and faster… but the AI infrastructure they’re trained on has remained stagnant and is lagging behind. This widespread problem is costing teams months of delays and millions in wasted compute. 

The industry’s legacy approach to infrastructure was never meant to support the unique stresses of long-duration, synchronized GPU workloads at scale. To move the industry forward at the pace of innovation, cloud solutions for AI model training need to evolve. They need to be purpose built.

What’s wrong with training on the cloud today? 

Legacy hyperscaler clouds and on-prem supercomputers tend to treat infrastructure like a closed box: they hand over a prebuilt environment, hope it is stable, and give customers limited visibility into how it behaves under real stress. There is little ability to proactively detect or mitigate issues before they impact jobs, and even less ability to continuously optimize during a run.

This means issues are inevitable. 

It’s a major reason why general-purpose clouds fall short of what you need to get breakthroughs to market quickly. Bottlenecks are not just raw throughput; they are interrupted runs, network contention, data loading or transfer slowdowns, and a constellation of micro-failures that quietly eat away at efficiency.

AI clouds break the old training model. One of the world’s largest AI research teams, reporting on a multi-thousand-GPU run last year, put it bluntly: “The complexity and potential failure scenarios of large-scale GPU training surpass those of much larger CPU clusters.” If they’re hitting these limits, everyone should anticipate a similar challenge. 

The quote above captures the driving force behind CoreWeave’s purpose-built approach to AI. Solving the pitfalls of cloud infrastructure requires a complete reimagining of AI infrastructure from the ground to the cloud. 

Vertical integration: The foundation of the Essential Cloud for AI

You cannot stitch together commodity components and hope they behave under the pressure of a 30-day, thousand-GPU training run. Instead, imagine a coordinated, purpose-built environment where every layer is designed to work together as a single, integrated system. 

At CoreWeave, vertical integration means that every part of the stack—from data center architecture and hardware selection to networking, storage, orchestration, observability, and support—is engineered to work together seamlessly. 

CoreWeave Cloud’s purpose-built approach makes it possible to:

• Continuously validate hardware to ensure readiness before it enters production
• Proactively replace components that show early signs of failure
• Perform rolling maintenance without disrupting active workloads
• Apply optimizations such as topology-aware GPU placement and asynchronous checkpointing, benefits of which compound over long training runs and improve efficiency

This is what makes CoreWeave the Essential Cloud for AI: a platform built from the ground up for the realities of large-scale, high-stakes AI workloads. Vertical integration is not just an architectural choice. It is the foundation for the performance and reliability required to train the world’s most advanced models. 

Measure what makes a training cluster performant 

First, a few definitions to ensure we’re all on the same page. To measure the value of a compute cluster, the following metrics must be considered.

• Time to Market (TTM): How fast or slow teams can stand up hardware, software, and other requirements to get a healthy cluster that’s ready to deploy workloads. 
• Mean Time to Failure (MTTF): The average amount of time a job can run before it is interrupted by a failure.
• Model FLOPs Utilization (MFU): The percentage of a GPU’s theoretical peak performance that is actually used for training.
• Effective Training Time Ratio (ETTR) or Goodput: The amount of compute time spent doing meaningful work.

If you are training models, you should be capturing these measurements and evaluating your cloud provider. These metrics are critical for companies competing at the bleeding edge of AI. A service delivered fast but rife with errors will slow progress just as much as one that is rock solid but arrives a year late. Any infrastructure failures or suboptimal supporting services will diminish your results, adding cost, reducing efficiency, and slowing time to market.

Take a look at the full whitepaper we published in August for a deeper understanding of these metrics and how CoreWeave measures against general-purpose legacy hyperscalers.

Building and testing a cloud purpose-built for AI at scale

Understanding the metrics that define a high-performing cluster is one thing. Building a platform that can consistently deliver on them at production scale is another, which is where CoreWeave’s unique approach comes in.

We uniquely designed every layer of our platform for AI. That means:

• Bare-metal access to GPU clusters for full performance and control
• Dual network fabrics to eliminate contention between compute and storage traffic
• Automated, topology-aware orchestration to detect and evict unhealthy nodes before they can take down a job
• High-speed data pipelines and interconnects that keep GPUs fed without bottlenecks
• Deep observability into both hardware and workload performance, so we can predict and prevent failures proactively rather than react to them

We have pioneered this approach for years, but we also needed proof that it worked—hard numbers collected under real-world training conditions.

In May and June 2025, our engineering team ran a large-scale pretraining benchmark for a 30-billion-parameter large language model across 1,024 GPUs. This was not a lab demo. It was a full-scale, production-quality run designed to measure how our infrastructure performs when every system is pushed to its limits.

The results speak for themselves:

• 51–52% MFU: up to 20% higher than typical public benchmarks
• 97–98% ETTR: up from the 90% industry average
• 10x improved MTTF: 3.66 days (average) until failure compared to the industry benchmark 0.33 days

These results translate into shorter training calendars, fewer wasted GPU-hours, and faster iteration cycles for AI teams pushing the frontier. In the scenario of a 30-day training cycle, these performance and reliability advantages translate to 7-15 days faster to market when training on CoreWeave.

This is the most compelling validation yet that when you build for AI from the ground up, performance and reliability can scale together instead of trading off.

The future of AI training belongs to purpose-built clouds

The lesson is simple: AI training infrastructure needs its own blueprint. General-purpose infrastructure will always hit a wall on reliability and efficiency at scale.

To push this industry forward and empower pioneers everywhere, we must continue to evaluate and elevate the infrastructure that supports it. That means more testing, more transparency, and more willingness to rethink the fundamentals.

Our full 30-plus-page whitepaper includes all the methodology, raw data, and lessons learned from this benchmark. If you are running or planning to run large-scale AI training, I encourage you to read it and compare these results to your own. You can also watch the webinar, which features two of the authors discussing the project.

If you want to learn more about our vertically integrated approach to AI model training and inference, schedule a call with our team.

‍