Serving · KernelIndex

22 comparison groups · 41 feasible · 0 excluded

Ranked only within one model, workload, protocol, hardware, and quality bar; a run missing a bounded metric is left out, not guessed.

llama3.1-8b · Offline

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 9 feasible · TTFT p99 reported for 0 of 9

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Lenovo · ThinkSystem SR780a V3 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.8, CUDA 12.8

146,960

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0062Run detail →

Oracle · BM.GPU.B200.8

TensorRT 10.11, CUDA 12.9

145,789

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0076Run detail →

Dell · Dell PowerEdge XE9685L (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.8

143,904

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0028Run detail →

Dell · Dell PowerEdge XE9680L (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

143,309

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0024Run detail →

Supermicro · AS-4126GS-NBR-LCC (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

142,955

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0086Run detail →

GigaComputing · G894-SD1

TensorRT 10.11, CUDA 12.9

142,136

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0046Run detail →

Google · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

141,138

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0047Run detail →

Supermicro · SYS-422GS-NBRT-LCC (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

141,056

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0088Run detail →

Show all 9 configurations ›

NVIDIA · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

136,752

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0069Run detail →

llama3.1-8b · Offline

8× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_closed_99

Ranked only inside this comparison group · 4 feasible · TTFT p99 reported for 0 of 4

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

HPE · HPE ProLiant Compute DL380a Gen12 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT)

TensorRT 10.14.1.48, CUDA 13.0

49,580

not reported

8× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0047Run detail →

Dell · PowerEdge XE7745 (8x RTXPro6000-PCIe-96GB, TensorRT)

TensorRT 10.14, CUDA 13.0

llama3.1-8b · Offline

8× NVIDIA B300-SXM-270GB · mlperf_closed_99

Ranked only inside this comparison group · 3 feasible · TTFT p99 reported for 0 of 3

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

ASUSTeK · XA NB3I-E12

TensorRT 10.14, CUDA 13.1

166,745

not reported

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0006Run detail →

NVIDIA · NVIDIA DGX B300 (8x B300-SXM-270GB, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

llama3.1-8b · Offline

8× NVIDIA H200-SXM-141GB · mlperf_closed_99

Ranked only inside this comparison group · 3 feasible · TTFT p99 reported for 0 of 3

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

HPE · HPE Cray XD670 (8x H200-SXM-141GB)

TensorRT 10.11, CUDA 12.9

66,037

not reported

8× NVIDIA H200-SXM-141GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0049Run detail →

Cisco · Cisco UCS C885A M8 (8x H200-SXM-141GB, TensorRT)

TensorRT 10.10, CUDA 12.9

58,304

llama3.1-8b · Offline

4× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_closed_99

Ranked only inside this comparison group · 3 feasible · TTFT p99 reported for 0 of 3

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Cisco · Cisco UCS X215c M8 - X580P PCIe Node (4x RTXPro6000-PCIe-96GB, TensorRT)

TensorRT 10.11, CUDA 12.9

24,014

not reported

4× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0014Run detail →

HPE · HPE ProLiant DL385 Gen11 (4x NVIDIA RTX PRO 6000 Blackwell Server Edition-450W, TensorRT)

TensorRT 10.14.1.48, CUDA 13.0

llama3.1-8b · Offline

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Lambda_SIT · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.14, CUDA 13.1

160,403

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0062Run detail →

HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT)

TensorRT 10.11, CUDA 13.0

152,625

llama3.1-8b · Offline

2× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

HPE · HPE ProLiant DL385 Gen11 (2x NVIDIA RTX PRO 6000 Blackwell Server Edition-600W, TensorRT)

TensorRT 10.14.1.48, CUDA 13.0

12,495

not reported

2× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0052Run detail →

HPE · HPE ProLiant DL385 Gen11 (2x NVIDIA RTX PRO 6000 Blackwell Server Edition-600W, TensorRT)

llama3.1-8b · Offline

4× Intel(R) Arc Pro(R) B50 · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Intel · 1-node-4x-BMG-B50

PyTorch vLLM

1,722

not reported

4× Intel(R) Arc Pro(R) B50 · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0058Run detail →

llama3.1-8b · Offline

10× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

HPE · HPE ProLiant Compute DL380a Gen12 (10x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT)

TensorRT 10.14.1.48, CUDA 13.0

60,680

not reported

10× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0045Run detail →

llama3.1-8b · Offline

8× NVIDIA H200-NVL-141GB · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Dell · PowerEdge XE7740 (8x H200-NVL-141GB, TensorRT)

TensorRT 10.13, CUDA 13.0

53,194

not reported

8× NVIDIA H200-NVL-141GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0021Run detail →

llama3.1-8b · Offline

4× Intel(R) Arc Pro(R) B60 · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Intel · 1-node-4x-BMG-B60

PyTorch vLLM

3,347

not reported

4× Intel(R) Arc Pro(R) B60 · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0059Run detail →

llama3.1-8b · Offline

4× Intel(R) Arc Pro(R) B70 · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Intel · 1-node-4x-BMG-B70

PyTorch vLLM

4,617

not reported

4× Intel(R) Arc Pro(R) B70 · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

Availability: preview.

MLPerf Inference · entry v6.0/6.0-0101Run detail →

10 more comparison groups. Narrow by model, workload, or hardware to see them.