CoreWeave Tops NVIDIA Blackwell Inference Throughput, Cuts TCO Up to 65%

CoreWeave Tops NVIDIA Blackwell Inference Throughput, Cuts TCO Up to 65%

Enterprise AI programs increasingly run on rented infrastructure. Rather than absorbing the capital costs, long lead times, and ongoing maintenance of standing up hardware on-premises, organizations are securing dedicated GPU capacity in the cloud. But what does AI infrastructure cost at scale?

A new study by Mitch Lewis, Performance Analyst at Signal65, CoreWeave Delivers Higher Throughput and Lower Cost per Token on NVIDIA Blackwell, answers this question. The report examines cloud economics and tokenomics, modeling a 5,000-GPU NVIDIA Blackwell deployment over three years. CoreWeave delivered the lowest total cost of ownership (TCO) in six of seven configurations, and the highest per-GPU inference throughput, compared with three traditional hyperscaler clouds.

The three-year TCO advantage at scale

Signal65’s analysis starts with the total multi-year contract cost of deploying clusters at scale. As detailed in Figure 1, when configuring NVIDIA GB200 NVL72 systems, CoreWeave costs less than all three evaluated hyperscalers. In the most extreme case, Hyperscaler C's projected three-year cost came to $3.9 billion. Comparatively, an identical NVIDIA GB200 NVL72 deployment on CoreWeave is 65% less expensive. CoreWeave also delivered a 36% lower cost than Hyperscaler A and a 22% lower cost than Hyperscaler B, based on an estimated rate because Hyperscaler B doesn’t publish GB200 NVL72 pricing.

Three-year cloud costs for a 5,000-GPU NVIDIA GB200 NVL72 deployment
Figure 1: Total 3-Year Cloud Costs (NVIDIA GB200 NVL72 Systems)

However, the sticker price is only part of the story. A Signal65 commentary in the report notes that the model leaves out some fees common on hyperscaler clouds: per-operation storage charges and early-deletion penalties on archive tiers. CoreWeave charges neither, so in practice the TCO gap is likely even wider than the modeled figures suggest.

Performance-fueled savings on DeepSeek-R1

Inference economics come down to throughput: how efficiently your infrastructure runs a model directly shapes your real cost per token. Figure 2 and Figure 3 of the report illustrate this by translating fully loaded cluster costs into a per-million-token price, using MLPerf® Inference v6.0 results for the 671-billion-parameter DeepSeek-R1 reasoning model.

Cost per million tokens running DeepSeek-R1 on NVIDIA GB200 NVL72
Figure 2: Cost per Million Tokens Comparison

CoreWeave’s stack delivered 4,867 tokens per second per GPU running DeepSeek-R1 on NVIDIA GB200 NVL72. Price and performance compound. Compared to the most competitively priced hyperscaler, CoreWeave’s per-token cost advantage widens from 22% (based on infrastructure pricing alone) to 65% once measured throughput is factored in. CoreWeave’s cost per million tokens comes to $0.75, compared with $1.23 to $2.22 on the hyperscalers. Put another way, a $1 million annual commitment buys about 1.33 trillion tokens on CoreWeave, versus 451 billion to 814 billion elsewhere. That’s up to 2.9x the tokens for the same spend. As Signal65 notes, tokens are the core unit of AI work, so more tokens for the same spend means more work completed.

Tokens processed per $1 million of annual spend
Figure 3: Tokens Processed per $1 Million

CoreWeave inference pricing

For teams that prefer to skip infrastructure management entirely and consume hosted models via an API, Figure 4 evaluates CoreWeave Serverless Inference token costs. On gpt-oss-120b, the one model available on every cloud in the study, CoreWeave input tokens cost 67% to 80% less and output tokens 53% to 72% less. Across 17 directly comparable models, CoreWeave Serverless Inference is lower-priced on 11 and identically priced on four. CoreWeave delivers 26% lower median input token costs and 15% lower median output token costs.

Serverless inference input and output token prices for gpt-oss-120b
Figure 4: Serverless Inference Costs (gpt-oss-120b)

The CoreWeave Cloud infrastructure behind these numbers

These TCO and throughput results come from a cloud platform optimized at every layer, from metal to model:

  • Bare Metal Servers: CoreWeave provides direct hardware access without a hypervisor layer, removing the overhead and latency a virtualized environment adds to the serving path.
  • Networking: High-bandwidth, low-latency nonblocking fabrics. Every leaf in our network carries enough uplink capacity to sustain full line rate from every connected node, at every layer, at a 1:1 ratio. That discipline holds on every fabric we've built, and it's why CoreWeave operates some of the largest NVIDIA Quantum InfiniBand and RDMA over Converged Ethernet (RoCE) fabrics running today with 800 Gbps of full-duplex bandwidth.
  • Storage: Local Object Transport Accelerator (LOTA) is an integral part of CoreWeave AI Object Storage, keeping reads local to the GPU nodes. It runs as a caching proxy on every GPU node in your CoreWeave Kubernetes Service (CKS) cluster. In the study, CoreWeave storage costs 28% to 55% less than the hyperscalers.
  • Serverless Inference: For those wanting pay-per-token access to open-source models with no clusters to manage, CoreWeave Serverless Inference runs on this same stack.

Signal65’s conclusion is that comparing hourly rates alone understates the real gap. By combining bare-metal performance with transparent pricing, CoreWeave ensures you spend your budget on intelligence, not overhead.

Attend the webinar: The True Price of Intelligence: An Economic Study by Signal65

Read the full report: CoreWeave Delivers Higher Throughput and Lower Cost per Token on NVIDIA Blackwell: A Three-Year TCO and Tokenomics Analysis.

Related blogs

GPU Compute,
Inference,
Copy code
Copied!
GPU Compute,
Inference,