Extractive language understanding — queries answered per second per GPU. · BERT-Large · reading comprehension · v4.1 / v5.0 / v5.1 / v6.0 / v6.1 · on-demand pricing
| GPU | Scenario | samp/s per GPU | |
|---|---|---|---|
| NVIDIA H200-SXM-141GB | Offline | 9,224 samp/s | |
| NVIDIA H100-SXM-80GB | Offline | 9,110 samp/s | |
| NVIDIA H100-SXM-80GB | Server | 7,366 samp/s | |
| NVIDIA H200-SXM-141GB | Server | 7,261 samp/s | |
| NVIDIA H100-PCIe-80GB | Offline | 5,879 samp/s | |
| NVIDIA H100-PCIe-80GB | Server | 4,573 samp/s | |
| NVIDIA L40S | Offline | 3,476 samp/s | |
| NVIDIA L40S | Server | 3,463 samp/s |
| Provider | GPU | Scenario | Price | samp/$ | |
|---|---|---|---|---|---|
| Lambda Labs | NVIDIA H200-SXM-141GB | Offline | $2.29/GPU·hr 18m ago | 14,500,415 | |
| Latitude.sh | NVIDIA H100-PCIe-80GB | Offline | $1.68/GPU·hr 18m ago | 12,598,500 | |
| Hyperstack | NVIDIA H100-SXM-80GB | Offline | $2.72/GPU·hr 18m ago | 12,056,691 | |
| Hyperstack | NVIDIA H200-SXM-141GB | Offline | $2.79/GPU·hr 18m ago | 11,901,774 | |
| Lambda Labs | NVIDIA H200-SXM-141GB | Server | $2.29/GPU·hr 18m ago | 11,415,315 | |
| Crusoe | NVIDIA H100-PCIe-80GB | Offline | $2.00/GPU·hr 18m ago | 10,582,740 | |
| Latitude.sh | NVIDIA H100-PCIe-80GB | Server | $1.68/GPU·hr 18m ago | 9,798,954 | |
| Hyperstack | NVIDIA H100-SXM-80GB | Server | $2.72/GPU·hr 18m ago | 9,749,207 | |
| Hyperstack | NVIDIA H200-SXM-141GB | Server | $2.79/GPU·hr 18m ago | 9,369,560 | |
| Crusoe | NVIDIA H100-PCIe-80GB | Server | $2.00/GPU·hr 18m ago | 8,231,121 | |
| RunPod | NVIDIA H100-SXM-80GB | Offline | $3.99/GPU·hr 18m ago | 8,219,098 | |
| Lambda Labs | NVIDIA H100-SXM-80GB | Offline | $3.99/GPU·hr 18m ago | 8,219,098 | |
| Vultr | NVIDIA L40S | Offline | $1.54/GPU·hr 18m ago | 8,135,106 | |
| Vultr | NVIDIA L40S | Server | $1.54/GPU·hr 18m ago | 8,105,792 | |
| DigitalOcean | NVIDIA L40S | Offline | $1.57/GPU·hr 18m ago | 7,970,331 | |
| DigitalOcean | NVIDIA L40S | Server | $1.57/GPU·hr 18m ago | 7,941,611 | |
| DigitalOcean | NVIDIA H100-SXM-80GB | Offline | $4.41/GPU·hr 18m ago | 7,436,327 | |
| DigitalOcean | NVIDIA H200-SXM-141GB | Offline | $4.47/GPU·hr 18m ago | 7,428,624 | |
| Crusoe | NVIDIA H100-SXM-80GB | Offline | $4.50/GPU·hr 18m ago | 7,287,600 | |
| Nebius | NVIDIA H100-SXM-80GB | Offline | $4.50/GPU·hr 18m ago | 7,287,600 | |
| RunPod | NVIDIA H100-SXM-80GB | Server | $3.99/GPU·hr 18m ago | 6,646,076 | |
| Lambda Labs | NVIDIA H100-SXM-80GB | Server | $3.99/GPU·hr 18m ago | 6,646,076 | |
| RunPod | NVIDIA H200-SXM-141GB | Offline | $5.29/GPU·hr 18m ago | 6,277,117 | |
| Crusoe | NVIDIA H200-SXM-141GB | Offline | $5.40/GPU·hr 18m ago | 6,149,250 | |
| Nebius | NVIDIA H200-SXM-141GB | Offline | $5.40/GPU·hr 18m ago | 6,149,250 | |
| DigitalOcean | NVIDIA H100-SXM-80GB | Server | $4.41/GPU·hr 18m ago | 6,013,116 | |
| Crusoe | NVIDIA H100-SXM-80GB | Server | $4.50/GPU·hr 18m ago | 5,892,854 | |
| Nebius | NVIDIA H100-SXM-80GB | Server | $4.50/GPU·hr 18m ago | 5,892,854 | |
| DigitalOcean | NVIDIA H200-SXM-141GB | Server | $4.47/GPU·hr 18m ago | 5,848,114 | |
| CoreWeave | NVIDIA H100-SXM-80GB | Offline | $6.16/GPU·hr 18m ago | 5,323,734 | |
| CoreWeave | NVIDIA H200-SXM-141GB | Offline | $6.31/GPU·hr 18m ago | 5,262,433 | |
| RunPod | NVIDIA H200-SXM-141GB | Server | $5.29/GPU·hr 18m ago | 4,941,601 | |
| Crusoe | NVIDIA H200-SXM-141GB | Server | $5.40/GPU·hr 18m ago | 4,840,939 | |
| Nebius | NVIDIA H200-SXM-141GB | Server | $5.40/GPU·hr 18m ago | 4,840,939 | |
| CoreWeave | NVIDIA H100-SXM-80GB | Server | $6.16/GPU·hr 18m ago | 4,304,845 | |
| CoreWeave | NVIDIA H200-SXM-141GB | Server | $6.31/GPU·hr 18m ago | 4,142,801 | |
| OCI | NVIDIA H200-SXM-141GB | Offline | $10.00/GPU·hr 18m ago | 3,320,595 | |
| Azure (eastus2) | NVIDIA H200-SXM-141GB | Offline | $10.60/GPU·hr 18m ago | 3,132,637 | |
| GCP (us-central1) | NVIDIA H200-SXM-141GB | Offline | $10.84/GPU·hr 18m ago | 3,061,870 | |
| OCI | NVIDIA H100-SXM-80GB | Offline | $10.75/GPU·hr 18m ago | 3,050,623 | |
| GCP (us-central1) | NVIDIA H100-SXM-80GB | Offline | $11.53/GPU·hr 18m ago | 2,845,043 | |
| AWS (us-east-1) | NVIDIA H100-SXM-80GB | Offline | $12.29/GPU·hr 18m ago | 2,668,365 | |
| Azure (eastus) | NVIDIA H100-SXM-80GB | Offline | $12.29/GPU·hr 18m ago | 2,668,365 | |
| Azure (eastus2) | NVIDIA H100-SXM-80GB | Offline | $12.29/GPU·hr 18m ago | 2,668,365 | |
| OCI | NVIDIA H200-SXM-141GB | Server | $10.00/GPU·hr 18m ago | 2,614,107 | |
| OCI | NVIDIA H100-SXM-80GB | Server | $10.75/GPU·hr 18m ago | 2,466,776 | |
| Azure (eastus2) | NVIDIA H200-SXM-141GB | Server | $10.60/GPU·hr 18m ago | 2,466,139 | |
| GCP (us-central1) | NVIDIA H200-SXM-141GB | Server | $10.84/GPU·hr 18m ago | 2,410,428 | |
| GCP (us-central1) | NVIDIA H100-SXM-80GB | Server | $11.53/GPU·hr 18m ago | 2,300,541 | |
| AWS (us-east-1) | NVIDIA H100-SXM-80GB | Server | $12.29/GPU·hr 18m ago | 2,157,676 | |
| Azure (eastus) | NVIDIA H100-SXM-80GB | Server | $12.29/GPU·hr 18m ago | 2,157,676 | |
| Azure (eastus2) | NVIDIA H100-SXM-80GB | Server | $12.29/GPU·hr 18m ago | 2,157,676 |