LLM Inference

Generative text inference — tokens produced per second per GPU. · llama2-70b · Offline + Server scenarios · v4.1 / v5.0 · on-demand pricing

Throughput — per GPU

GPUScenariotok/s per GPU
NVIDIA B200-SXM-180GBOffline12,357 tok/s
NVIDIA B200-SXM-180GBServer12,305 tok/s
NVIDIA H200-SXM-141GBOffline4,432 tok/s
NVIDIA H200-SXM-141GBServer4,134 tok/s
NVIDIA H100-SXM-80GBOffline3,913 tok/s
NVIDIA H200-NVL-141GBOffline3,894 tok/s
NVIDIA H100-SXM-80GBServer3,888 tok/s
NVIDIA H200-NVL-141GBServer3,606 tok/s

Efficiency — tok/$ · ranked

ProviderGPUScenarioPricetok/$
NebiusNVIDIA B200-SXM-180GBOffline$3.95/GPU·hr
16m ago		11,262,304
NebiusNVIDIA B200-SXM-180GBServer$3.95/GPU·hr
16m ago		11,215,059
RunPodNVIDIA B200-SXM-180GBOffline$5.89/GPU·hr
16m ago		7,552,818
RunPodNVIDIA B200-SXM-180GBServer$5.89/GPU·hr
16m ago		7,521,135
Lambda LabsNVIDIA H200-SXM-141GBOffline$2.29/GPU·hr
16m ago		6,966,747
Lambda LabsNVIDIA B200-SXM-180GBOffline$6.69/GPU·hr
16m ago		6,649,641
Lambda LabsNVIDIA B200-SXM-180GBServer$6.69/GPU·hr
16m ago		6,621,747
NebiusNVIDIA H100-SXM-80GBOffline$2.15/GPU·hr
16m ago		6,552,586
NebiusNVIDIA H200-SXM-141GBOffline$2.45/GPU·hr
16m ago		6,511,776
NebiusNVIDIA H100-SXM-80GBServer$2.15/GPU·hr
16m ago		6,510,617
Lambda LabsNVIDIA H200-SXM-141GBServer$2.29/GPU·hr
16m ago		6,498,759
NebiusNVIDIA H200-SXM-141GBServer$2.45/GPU·hr
16m ago		6,074,350
HyperstackNVIDIA H200-SXM-141GBOffline$2.79/GPU·hr
16m ago		5,718,226
GCP (us-central1)NVIDIA B200-SXM-180GBOffline$8.05/GPU·hr
16m ago		5,522,793
GCP (us-central1)NVIDIA B200-SXM-180GBServer$8.05/GPU·hr
16m ago		5,499,626
HyperstackNVIDIA H200-SXM-141GBServer$2.79/GPU·hr
16m ago		5,334,106
HyperstackNVIDIA H100-SXM-80GBOffline$2.72/GPU·hr
16m ago		5,179,434
CoreWeaveNVIDIA B200-SXM-180GBOffline$8.60/GPU·hr
16m ago		5,172,802
CoreWeaveNVIDIA B200-SXM-180GBServer$8.60/GPU·hr
16m ago		5,151,103
HyperstackNVIDIA H100-SXM-80GBServer$2.72/GPU·hr
16m ago		5,146,260
RunPodNVIDIA H100-SXM-80GBOffline$2.99/GPU·hr
16m ago		4,711,726
RunPodNVIDIA H100-SXM-80GBServer$2.99/GPU·hr
16m ago		4,681,547
DigitalOceanNVIDIA H200-SXM-141GBOffline$3.44/GPU·hr
16m ago		4,637,747
DigitalOceanNVIDIA H200-SXM-141GBServer$3.44/GPU·hr
16m ago		4,326,208
DigitalOceanNVIDIA H100-SXM-80GBOffline$3.39/GPU·hr
16m ago		4,155,770
DigitalOceanNVIDIA H100-SXM-80GBServer$3.39/GPU·hr
16m ago		4,129,152
RunPodNVIDIA H200-SXM-141GBOffline$4.39/GPU·hr
16m ago		3,634,134
CrusoeNVIDIA H100-SXM-80GBOffline$3.90/GPU·hr
16m ago		3,612,323
CrusoeNVIDIA H100-SXM-80GBServer$3.90/GPU·hr
16m ago		3,589,186
Lambda LabsNVIDIA H100-SXM-80GBOffline$3.99/GPU·hr
16m ago		3,530,842
Lambda LabsNVIDIA H100-SXM-80GBServer$3.99/GPU·hr
16m ago		3,508,227
RunPodNVIDIA H200-SXM-141GBServer$4.39/GPU·hr
16m ago		3,390,013
CrusoeNVIDIA H200-NVL-141GBOffline$4.29/GPU·hr
16m ago		3,267,472
OCI (us-ashburn-1)NVIDIA B200-SXM-180GBOffline$14.00/GPU·hr
56d ago		3,177,579
OCINVIDIA B200-SXM-180GBOffline$14.00/GPU·hr
16m ago		3,177,579
OCI (us-ashburn-1)NVIDIA B200-SXM-180GBServer$14.00/GPU·hr
56d ago		3,164,249
OCINVIDIA B200-SXM-180GBServer$14.00/GPU·hr
16m ago		3,164,249
CrusoeNVIDIA H200-NVL-141GBServer$4.29/GPU·hr
16m ago		3,026,056
CrusoeNVIDIA H200-SXM-141GBOffline$6.00/GPU·hr
16m ago		2,658,975
CoreWeaveNVIDIA H200-SXM-141GBOffline$6.31/GPU·hr
16m ago		2,528,344
CrusoeNVIDIA H200-SXM-141GBServer$6.00/GPU·hr
16m ago		2,480,360
CoreWeaveNVIDIA H200-SXM-141GBServer$6.31/GPU·hr
16m ago		2,358,503
CoreWeaveNVIDIA H100-SXM-80GBOffline$6.16/GPU·hr
16m ago		2,287,023
CoreWeaveNVIDIA H100-SXM-80GBServer$6.16/GPU·hr
16m ago		2,272,374
OCI (us-ashburn-1)NVIDIA H200-SXM-141GBOffline$10.00/GPU·hr
56d ago		1,595,385
OCINVIDIA H200-SXM-141GBOffline$10.00/GPU·hr
16m ago		1,595,385
Azure (eastus2)NVIDIA H200-SXM-141GBOffline$10.60/GPU·hr
16m ago		1,505,080
OCI (us-ashburn-1)NVIDIA H200-SXM-141GBServer$10.00/GPU·hr
56d ago		1,488,216
OCINVIDIA H200-SXM-141GBServer$10.00/GPU·hr
16m ago		1,488,216
GCP (us-central1)NVIDIA H200-SXM-141GBOffline$10.84/GPU·hr
16m ago		1,471,080
Azure (eastus2)NVIDIA H200-SXM-141GBServer$10.60/GPU·hr
16m ago		1,403,977
GCP (us-central1)NVIDIA H200-SXM-141GBServer$10.84/GPU·hr
16m ago		1,372,261
AWS (us-east-1)NVIDIA H200-NVL-141GBOffline$10.60/GPU·hr
16m ago		1,322,401
OCI (us-ashburn-1)NVIDIA H100-SXM-80GBOffline$10.75/GPU·hr
56d ago		1,310,517
OCINVIDIA H100-SXM-80GBOffline$10.75/GPU·hr
16m ago		1,310,517
OCI (us-ashburn-1)NVIDIA H100-SXM-80GBServer$10.75/GPU·hr
56d ago		1,302,123
OCINVIDIA H100-SXM-80GBServer$10.75/GPU·hr
16m ago		1,302,123
AWS (us-east-1)NVIDIA H200-NVL-141GBServer$10.60/GPU·hr
16m ago		1,224,696
GCP (us-central1)NVIDIA H100-SXM-80GBOffline$11.53/GPU·hr
16m ago		1,222,202
GCP (us-central1)NVIDIA H100-SXM-80GBServer$11.53/GPU·hr
16m ago		1,214,374
AWS (us-east-1)NVIDIA H100-SXM-80GBOffline$12.29/GPU·hr
16m ago		1,146,303
Azure (eastus)NVIDIA H100-SXM-80GBOffline$12.29/GPU·hr
16m ago		1,146,303
Azure (eastus2)NVIDIA H100-SXM-80GBOffline$12.29/GPU·hr
16m ago		1,146,303
AWS (us-east-1)NVIDIA H100-SXM-80GBServer$12.29/GPU·hr
16m ago		1,138,961
Azure (eastus)NVIDIA H100-SXM-80GBServer$12.29/GPU·hr
16m ago		1,138,961
Azure (eastus2)NVIDIA H100-SXM-80GBServer$12.29/GPU·hr
16m ago		1,138,961

Generation Efficiency Ratios

AWS (us-east-1) · Offline · H200 vs H1001.15×more tok/$
AWS (us-east-1) · Server · H200 vs H1001.08×more tok/$
Azure (eastus2) · Offline · H200 vs H1001.31×more tok/$
Azure (eastus2) · Server · H200 vs H1001.23×more tok/$
CoreWeave · Offline · B200 vs H1002.26×more tok/$
CoreWeave · Offline · H200 vs H1001.11×more tok/$
CoreWeave · Server · B200 vs H1002.27×more tok/$
CoreWeave · Server · H200 vs H1001.04×more tok/$
Crusoe · Offline · H200 vs H1000.90×more tok/$
Crusoe · Server · H200 vs H1000.84×more tok/$
DigitalOcean · Offline · H200 vs H1001.12×more tok/$
DigitalOcean · Server · H200 vs H1001.05×more tok/$
GCP (us-central1) · Offline · B200 vs H1004.52×more tok/$
GCP (us-central1) · Offline · H200 vs H1001.20×more tok/$
GCP (us-central1) · Server · B200 vs H1004.53×more tok/$
GCP (us-central1) · Server · H200 vs H1001.13×more tok/$
Hyperstack · Offline · H200 vs H1001.10×more tok/$
Hyperstack · Server · H200 vs H1001.04×more tok/$
Lambda Labs · Offline · B200 vs H1001.88×more tok/$
Lambda Labs · Offline · H200 vs H1001.97×more tok/$
Lambda Labs · Server · B200 vs H1001.89×more tok/$
Lambda Labs · Server · H200 vs H1001.85×more tok/$
Nebius · Offline · B200 vs H1001.72×more tok/$
Nebius · Offline · H200 vs H1000.99×more tok/$
Nebius · Server · B200 vs H1001.72×more tok/$
Nebius · Server · H200 vs H1000.93×more tok/$
OCI · Offline · B200 vs H1002.42×more tok/$
OCI · Offline · H200 vs H1001.22×more tok/$
OCI · Server · B200 vs H1002.43×more tok/$
OCI · Server · H200 vs H1001.14×more tok/$
OCI (us-ashburn-1) · Offline · B200 vs H1002.42×more tok/$
OCI (us-ashburn-1) · Offline · H200 vs H1001.22×more tok/$
OCI (us-ashburn-1) · Server · B200 vs H1002.43×more tok/$
OCI (us-ashburn-1) · Server · H200 vs H1001.14×more tok/$
RunPod · Offline · B200 vs H1001.60×more tok/$
RunPod · Offline · H200 vs H1000.77×more tok/$
RunPod · Server · B200 vs H1001.61×more tok/$
RunPod · Server · H200 vs H1000.72×more tok/$