Extractive language understanding — queries answered per second per GPU. · BERT-Large · reading comprehension · MLPerf v4.1/v5.0 · on-demand pricing
| GPU | Scenario | samp/s per GPU | |
|---|---|---|---|
| NVIDIA H200-SXM-141GB | Offline | 9,224 samp/s | |
| NVIDIA H100-SXM-80GB | Offline | 9,110 samp/s | |
| NVIDIA H100-SXM-80GB | Server | 7,366 samp/s | |
| NVIDIA H200-SXM-141GB | Server | 7,261 samp/s |
| Provider | GPU | Scenario | Price | samp/$ | |
|---|---|---|---|---|---|
| RunPod | NVIDIA H100-SXM-80GB | Offline | $2.99/GPU·hr 1d ago | 10,967,960 | |
| RunPod | NVIDIA H100-SXM-80GB | Server | $2.99/GPU·hr 1d ago | 8,868,844 | |
| Crusoe | NVIDIA H100-SXM-80GB | Offline | $3.90/GPU·hr 1d ago | 8,408,769 | |
| Lambda Labs | NVIDIA H100-SXM-80GB | Offline | $3.99/GPU·hr 1d ago | 8,219,098 | |
| Crusoe | NVIDIA H200-SXM-141GB | Offline | $4.29/GPU·hr 1d ago | 7,740,315 | |
| RunPod | NVIDIA H200-SXM-141GB | Offline | $4.31/GPU·hr 1d ago | 7,704,397 | |
| Crusoe | NVIDIA H100-SXM-80GB | Server | $3.90/GPU·hr 1d ago | 6,799,447 | |
| Lambda Labs | NVIDIA H100-SXM-80GB | Server | $3.99/GPU·hr 1d ago | 6,646,076 | |
| Crusoe | NVIDIA H200-SXM-141GB | Server | $4.29/GPU·hr 1d ago | 6,093,490 | |
| RunPod | NVIDIA H200-SXM-141GB | Server | $4.31/GPU·hr 1d ago | 6,065,214 | |
| CoreWeave | NVIDIA H100-SXM-80GB | Offline | $6.16/GPU·hr 1d ago | 5,323,734 | |
| CoreWeave | NVIDIA H200-SXM-141GB | Offline | $6.31/GPU·hr 1d ago | 5,262,433 | |
| CoreWeave | NVIDIA H100-SXM-80GB | Server | $6.16/GPU·hr 1d ago | 4,304,845 | |
| CoreWeave | NVIDIA H200-SXM-141GB | Server | $6.31/GPU·hr 1d ago | 4,142,801 | |
| OCI | NVIDIA H200-SXM-141GB | Offline | $10.00/GPU·hr today | 3,320,595 | |
| Azure (eastus2) | NVIDIA H200-SXM-141GB | Offline | $10.60/GPU·hr today | 3,132,637 | |
| GCP (us-central1) | NVIDIA H200-SXM-141GB | Offline | $10.60/GPU·hr today | 3,132,382 | |
| OCI | NVIDIA H100-SXM-80GB | Offline | $10.75/GPU·hr today | 3,050,623 | |
| GCP (us-central1) | NVIDIA H100-SXM-80GB | Offline | $11.06/GPU·hr today | 2,964,782 | |
| AWS (us-east-1) | NVIDIA H100-SXM-80GB | Offline | $12.29/GPU·hr today | 2,668,365 | |
| Azure (eastus2) | NVIDIA H100-SXM-80GB | Offline | $12.29/GPU·hr today | 2,668,365 | |
| Azure (eastus) | NVIDIA H100-SXM-80GB | Offline | $12.29/GPU·hr today | 2,668,365 | |
| OCI | NVIDIA H200-SXM-141GB | Server | $10.00/GPU·hr today | 2,614,107 | |
| OCI | NVIDIA H100-SXM-80GB | Server | $10.75/GPU·hr today | 2,466,776 | |
| Azure (eastus2) | NVIDIA H200-SXM-141GB | Server | $10.60/GPU·hr today | 2,466,139 | |
| GCP (us-central1) | NVIDIA H200-SXM-141GB | Server | $10.60/GPU·hr today | 2,465,938 | |
| GCP (us-central1) | NVIDIA H100-SXM-80GB | Server | $11.06/GPU·hr today | 2,397,364 | |
| AWS (us-east-1) | NVIDIA H100-SXM-80GB | Server | $12.29/GPU·hr today | 2,157,676 | |
| Azure (eastus2) | NVIDIA H100-SXM-80GB | Server | $12.29/GPU·hr today | 2,157,676 | |
| Azure (eastus) | NVIDIA H100-SXM-80GB | Server | $12.29/GPU·hr today | 2,157,676 |