LLM Inference

Generative text inference — tokens produced per second per GPU. · llama2-70b · Offline + Server scenarios · v4.1 / v5.0 / v5.1 / v6.0 · on-demand pricing

Throughput — per GPU

GPUScenariotok/s per GPU
NVIDIA B300-SXM-270GBOffline14,119 tok/s
NVIDIA B300-SXM-270GBServer13,549 tok/s
NVIDIA B200-SXM-180GBOffline12,699 tok/s
NVIDIA B200-SXM-180GBServer12,413 tok/s
NVIDIA GB300Server12,059 tok/s
NVIDIA H200-SXM-141GBOffline4,415 tok/s
NVIDIA H200-SXM-141GBServer4,156 tok/s
NVIDIA H200-NVL-141GBOffline4,000 tok/s
NVIDIA H100-SXM-80GBOffline3,913 tok/s
NVIDIA H100-SXM-80GBServer3,888 tok/s
NVIDIA H200-NVL-141GBServer3,636 tok/s
NVIDIA GB300Offline0 tok/s

Efficiency — tok/$ · ranked

ProviderGPUScenarioPricetok/$
Lambda LabsNVIDIA H200-SXM-141GBOffline$2.29/GPU·hr
19m ago		6,939,963
Lambda LabsNVIDIA B200-SXM-180GBOffline$6.69/GPU·hr
19m ago		6,833,744
RunPodNVIDIA B200-SXM-180GBOffline$6.79/GPU·hr
19m ago		6,733,100
Lambda LabsNVIDIA B200-SXM-180GBServer$6.69/GPU·hr
19m ago		6,679,692
RunPodNVIDIA B200-SXM-180GBServer$6.79/GPU·hr
19m ago		6,581,316
Lambda LabsNVIDIA H200-SXM-141GBServer$2.29/GPU·hr
19m ago		6,532,713
NebiusNVIDIA B300-SXM-270GBOffline$7.85/GPU·hr
19m ago		6,475,070
NebiusNVIDIA B200-SXM-180GBOffline$7.15/GPU·hr
19m ago		6,394,091
NebiusNVIDIA B200-SXM-180GBServer$7.15/GPU·hr
19m ago		6,249,949
NebiusNVIDIA B300-SXM-270GBServer$7.85/GPU·hr
19m ago		6,213,569
HyperstackNVIDIA H200-SXM-141GBOffline$2.79/GPU·hr
19m ago		5,696,242
GCP (us-central1)NVIDIA B200-SXM-180GBOffline$8.05/GPU·hr
19m ago		5,675,698
GCP (us-central1)NVIDIA B200-SXM-180GBServer$8.05/GPU·hr
19m ago		5,547,751
HyperstackNVIDIA H200-SXM-141GBServer$2.79/GPU·hr
19m ago		5,361,976
CoreWeaveNVIDIA B200-SXM-180GBOffline$8.60/GPU·hr
19m ago		5,316,017
CoreWeaveNVIDIA B200-SXM-180GBServer$8.60/GPU·hr
19m ago		5,196,179
HyperstackNVIDIA H100-SXM-80GBOffline$2.72/GPU·hr
19m ago		5,179,434
HyperstackNVIDIA H100-SXM-80GBServer$2.72/GPU·hr
19m ago		5,146,260
RunPodNVIDIA H100-SXM-80GBOffline$3.29/GPU·hr
19m ago		4,282,085
RunPodNVIDIA H100-SXM-80GBServer$3.29/GPU·hr
19m ago		4,254,658
NebiusNVIDIA H100-SXM-80GBOffline$3.85/GPU·hr
19m ago		3,659,236
NebiusNVIDIA H100-SXM-80GBServer$3.85/GPU·hr
19m ago		3,635,799
CrusoeNVIDIA H100-SXM-80GBOffline$3.90/GPU·hr
19m ago		3,612,323
CrusoeNVIDIA H100-SXM-80GBServer$3.90/GPU·hr
19m ago		3,589,186
DigitalOceanNVIDIA H200-SXM-141GBOffline$4.47/GPU·hr
19m ago		3,555,372
NebiusNVIDIA H200-SXM-141GBOffline$4.50/GPU·hr
19m ago		3,531,670
Lambda LabsNVIDIA H100-SXM-80GBOffline$3.99/GPU·hr
19m ago		3,530,842
Lambda LabsNVIDIA H100-SXM-80GBServer$3.99/GPU·hr
19m ago		3,508,227
RunPodNVIDIA H200-SXM-141GBOffline$4.59/GPU·hr
19m ago		3,462,422
OCINVIDIA B300-SXM-270GBOffline$15.00/GPU·hr
19m ago		3,388,620
CrusoeNVIDIA H200-NVL-141GBOffline$4.29/GPU·hr
19m ago		3,357,063
DigitalOceanNVIDIA H200-SXM-141GBServer$4.47/GPU·hr
19m ago		3,346,737
NebiusNVIDIA H200-SXM-141GBServer$4.50/GPU·hr
19m ago		3,324,425
OCINVIDIA B200-SXM-180GBOffline$14.00/GPU·hr
19m ago		3,265,554
RunPodNVIDIA H200-SXM-141GBServer$4.59/GPU·hr
19m ago		3,259,240
OCINVIDIA B300-SXM-270GBServer$15.00/GPU·hr
19m ago		3,251,768
DigitalOceanNVIDIA H100-SXM-80GBOffline$4.41/GPU·hr
19m ago		3,194,571
OCINVIDIA B200-SXM-180GBServer$14.00/GPU·hr
19m ago		3,191,938
DigitalOceanNVIDIA H100-SXM-80GBServer$4.41/GPU·hr
19m ago		3,174,110
CrusoeNVIDIA H200-NVL-141GBServer$4.29/GPU·hr
19m ago		3,050,860
CrusoeNVIDIA H200-SXM-141GBOffline$6.00/GPU·hr
19m ago		2,648,752
CoreWeaveNVIDIA H200-SXM-141GBOffline$6.31/GPU·hr
19m ago		2,518,624
CrusoeNVIDIA H200-SXM-141GBServer$6.00/GPU·hr
19m ago		2,493,319
OCINVIDIA GB300Server$18.00/GPU·hr
19m ago		2,411,883
CoreWeaveNVIDIA H200-SXM-141GBServer$6.31/GPU·hr
19m ago		2,370,826
CoreWeaveNVIDIA H100-SXM-80GBOffline$6.16/GPU·hr
19m ago		2,287,023
CoreWeaveNVIDIA H100-SXM-80GBServer$6.16/GPU·hr
19m ago		2,272,374
OCINVIDIA H200-SXM-141GBOffline$10.00/GPU·hr
19m ago		1,589,251
Azure (eastus2)NVIDIA H200-SXM-141GBOffline$10.60/GPU·hr
19m ago		1,499,294
OCINVIDIA H200-SXM-141GBServer$10.00/GPU·hr
19m ago		1,495,991
GCP (us-central1)NVIDIA H200-SXM-141GBOffline$10.84/GPU·hr
19m ago		1,465,425
Azure (eastus2)NVIDIA H200-SXM-141GBServer$10.60/GPU·hr
19m ago		1,411,312
GCP (us-central1)NVIDIA H200-SXM-141GBServer$10.84/GPU·hr
19m ago		1,379,431
AWS (us-east-1)NVIDIA H200-NVL-141GBOffline$10.60/GPU·hr
19m ago		1,358,660
OCINVIDIA H100-SXM-80GBOffline$10.75/GPU·hr
19m ago		1,310,517
OCINVIDIA H100-SXM-80GBServer$10.75/GPU·hr
19m ago		1,302,123
AWS (us-east-1)NVIDIA H200-NVL-141GBServer$10.60/GPU·hr
19m ago		1,234,735
GCP (us-central1)NVIDIA H100-SXM-80GBOffline$11.53/GPU·hr
19m ago		1,222,202
GCP (us-central1)NVIDIA H100-SXM-80GBServer$11.53/GPU·hr
19m ago		1,214,374
AWS (us-east-1)NVIDIA H100-SXM-80GBOffline$12.29/GPU·hr
19m ago		1,146,303
Azure (eastus)NVIDIA H100-SXM-80GBOffline$12.29/GPU·hr
19m ago		1,146,303
Azure (eastus2)NVIDIA H100-SXM-80GBOffline$12.29/GPU·hr
19m ago		1,146,303
AWS (us-east-1)NVIDIA H100-SXM-80GBServer$12.29/GPU·hr
19m ago		1,138,961
Azure (eastus)NVIDIA H100-SXM-80GBServer$12.29/GPU·hr
19m ago		1,138,961
Azure (eastus2)NVIDIA H100-SXM-80GBServer$12.29/GPU·hr
19m ago		1,138,961
OCINVIDIA GB300Offline$18.00/GPU·hr
19m ago		3

Generation Efficiency Ratios

AWS (us-east-1) · Offline · H200 vs H1001.19×more tok/$
AWS (us-east-1) · Server · H200 vs H1001.08×more tok/$
Azure (eastus2) · Offline · H200 vs H1001.31×more tok/$
Azure (eastus2) · Server · H200 vs H1001.24×more tok/$
CoreWeave · Offline · B200 vs H1002.32×more tok/$
CoreWeave · Offline · H200 vs H1001.10×more tok/$
CoreWeave · Server · B200 vs H1002.29×more tok/$
CoreWeave · Server · H200 vs H1001.04×more tok/$
Crusoe · Offline · H200 vs H1000.93×more tok/$
Crusoe · Server · H200 vs H1000.85×more tok/$
DigitalOcean · Offline · H200 vs H1001.11×more tok/$
DigitalOcean · Server · H200 vs H1001.05×more tok/$
GCP (us-central1) · Offline · B200 vs H1004.64×more tok/$
GCP (us-central1) · Offline · H200 vs H1001.20×more tok/$
GCP (us-central1) · Server · B200 vs H1004.57×more tok/$
GCP (us-central1) · Server · H200 vs H1001.14×more tok/$
Hyperstack · Offline · H200 vs H1001.10×more tok/$
Hyperstack · Server · H200 vs H1001.04×more tok/$
Lambda Labs · Offline · B200 vs H1001.94×more tok/$
Lambda Labs · Offline · H200 vs H1001.97×more tok/$
Lambda Labs · Server · B200 vs H1001.90×more tok/$
Lambda Labs · Server · H200 vs H1001.86×more tok/$
Nebius · Offline · B200 vs H1001.75×more tok/$
Nebius · Offline · H200 vs H1000.97×more tok/$
Nebius · Server · B200 vs H1001.72×more tok/$
Nebius · Server · H200 vs H1000.91×more tok/$
OCI · Offline · B200 vs H1002.49×more tok/$
OCI · Offline · H200 vs H1001.21×more tok/$
OCI · Server · B200 vs H1002.45×more tok/$
OCI · Server · H200 vs H1001.15×more tok/$
RunPod · Offline · B200 vs H1001.57×more tok/$
RunPod · Offline · H200 vs H1000.81×more tok/$
RunPod · Server · B200 vs H1001.55×more tok/$
RunPod · Server · H200 vs H1000.77×more tok/$