Serving · KernelIndex

31 comparison groups · 151 feasible · 0 excluded

Ranked only within one model, workload, protocol, hardware, and quality bar; a run missing a bounded metric is left out, not guessed.

Llama-3.1-405B (MLPerf reference) · llama3.1-405b · Offline

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 13 feasible · TTFT p99 reported for 0 of 13

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Dell · Dell PowerEdge XE9685L (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.8

1,661

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0028Run detail →

Nebius · Nebius B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

1,660

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0073Run detail →

Google · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

1,655

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0047Run detail →

Supermicro · SYS-422GS-NBRT-LCC (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

1,651

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0088Run detail →

Broadcom_Supermicro · Supermicro SYS-422GA-NBRT-LCC (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.8

1,650

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0009Run detail →

Lambda · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

1,649

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0060Run detail →

Dell · Dell PowerEdge XE9680L (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

1,648

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0024Run detail →

Supermicro · SYS-422GA-NBRT-LCC (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

1,641

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0087Run detail →

Show all 13 configurations ›

Lenovo · ThinkSystem SR780a V3 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.8, CUDA 12.8

1,637

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0062Run detail →

GigaComputing · G894-SD1

TensorRT 10.11, CUDA 12.9

1,626

not reported

Llama-3.1-405B (MLPerf reference) · llama3.1-405b · Server

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 13 feasible · TTFT p99 reported for 0 of 13

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Nebius · Nebius B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

1,280

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0073Run detail →

Lenovo · ThinkSystem SR780a V3 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.8, CUDA 12.8

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Offline

8× NVIDIA B200-SXM-180GB · mlperf_closed_99_9

Ranked only inside this comparison group · 11 feasible · TTFT p99 reported for 0 of 11

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Lenovo · ThinkSystem SR680a V3 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.8, CUDA 12.8

102,909

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99_9 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0061Run detail →

Dell · Dell PowerEdge XE9685L (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.8

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Server

8× NVIDIA B200-SXM-180GB · mlperf_closed_99_9

Ranked only inside this comparison group · 11 feasible · TTFT p99 reported for 0 of 11

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Nebius · Nebius B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

101,611

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99_9 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0073Run detail →

Dell · Dell PowerEdge XE9685L (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.8

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Offline

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 10 feasible · TTFT p99 reported for 0 of 10

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Lenovo · ThinkSystem SR680a V3 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.8, CUDA 12.8

102,909

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0061Run detail →

Lambda · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

Llama-3.1-405B (MLPerf reference) · llama3.1-405b · Interactive

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 10 feasible · TTFT p99 reported for 0 of 10

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Nebius · Nebius B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

771

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0073Run detail →

Lenovo · ThinkSystem SR780a V3 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.8, CUDA 12.8

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Server

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 10 feasible · TTFT p99 reported for 0 of 10

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Nebius · Nebius B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

101,611

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0073Run detail →

Lambda · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

Llama-3.1-8B (MLPerf reference) · llama3.1-8b · Server

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 9 feasible · TTFT p99 reported for 0 of 9

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Dell · Dell PowerEdge XE9685L (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.8

128,794

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0028Run detail →

Dell · Dell PowerEdge XE9680L (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

Llama-3.1-8B (MLPerf reference) · llama3.1-8b · Offline

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 9 feasible · TTFT p99 reported for 0 of 9

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Lenovo · ThinkSystem SR780a V3 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.8, CUDA 12.8

146,960

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0062Run detail →

Oracle · BM.GPU.B200.8

TensorRT 10.11, CUDA 12.9

145,789

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Interactive

8× NVIDIA B200-SXM-180GB · mlperf_closed_99_9

Ranked only inside this comparison group · 8 feasible · TTFT p99 reported for 0 of 8

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

GigaComputing · G894-SD1

TensorRT 10.11, CUDA 12.9

62,851

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99_9 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0046Run detail →

Supermicro · SYS-422GA-NBRT-LCC (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

60,140

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Interactive

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 6 feasible · TTFT p99 reported for 0 of 6

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

GigaComputing · G894-SD1

TensorRT 10.11, CUDA 12.9

62,851

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0046Run detail →

Dell · Dell PowerEdge XE9680L (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

60,138

DeepSeek-R1 (MLPerf reference) · deepseek-r1 · Server

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 5 feasible · TTFT p99 reported for 0 of 5

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

NVIDIA · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

18,592

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0069Run detail →

GigaComputing · G894-SD1

TensorRT 10.11, CUDA 12.9

15,978

19 more comparison groups. Narrow by model, workload, or hardware to see them.