Serving · KernelIndex

6 comparison groups · 10 feasible · 0 excluded

Ranked only within one model, workload, protocol, hardware, and quality bar; a run missing a bounded metric is left out, not guessed.

llama3.1-8b · Interactive

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Lambda_SIT · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.14, CUDA 13.1

128,750

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0062Run detail →

HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT)

TensorRT 10.11, CUDA 13.0

126,143

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0048Run detail →

llama3.1-8b · Interactive

2× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

HPE · HPE ProLiant DL385 Gen11 (2x NVIDIA RTX PRO 6000 Blackwell Server Edition-600W, TensorRT)

TensorRT 10.14.1.48, CUDA 13.0

10,589

not reported

2× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0052Run detail →

HPE · HPE ProLiant DL385 Gen11 (2x NVIDIA RTX PRO 6000 Blackwell Server Edition-600W, TensorRT)

llama3.1-8b · Interactive

4× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

HPE · HPE ProLiant DL385 Gen11 (4x NVIDIA RTX PRO 6000 Blackwell Server Edition-450W, TensorRT)

TensorRT 10.14.1.48, CUDA 13.0

20,119

not reported

4× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0054Run detail →

HPE · HPE ProLiant DL385 Gen11 (4x NVIDIA RTX PRO 6000 Blackwell Server Edition-450W, TensorRT)

llama3.1-8b · Interactive

8× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

HPE · HPE ProLiant Compute DL380a Gen12 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT)

TensorRT 10.14.1.48, CUDA 13.0

44,087

not reported

8× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0047Run detail →

ASUSTeK · ESC8000A_E13P_RTX_PRO_6000_PCIE_96GBx8

TensorRT 10.14, CUDA 13.1

llama3.1-8b · Interactive

10× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

HPE · HPE ProLiant Compute DL380a Gen12 (10x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT)

TensorRT 10.14.1.48, CUDA 13.0

53,055

not reported

10× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0045Run detail →

llama3.1-8b · Interactive

8× NVIDIA B300-SXM-270GB · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

GigaComputing · G894-SD3-AAX7

TensorRT 10.14, CUDA 13.1

128,633

not reported

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0039Run detail →