Compare GPU Performance on AI Workloads
H100 80GB SXM
H100 80GB SXMVs.
A100 80GB SXM
A100 80GB SXMLLM Benchmarks
Benchmarks were run on RunPod gpus using vllm. For more details on vllm, check out the vllm github repository.
Output Token Throughput (tok/s)
Llama 8b Instruct
1
Output Token Throughput (tok/s)
Get started with RunPod
today.
We handle millions of gpu requests a day. Scale your machine learning workloads while keeping costs low with RunPod.
Get Started