Serving · KernelIndex

17 comparison groups · 28 feasible · 0 excluded

Ranked only within one model, workload, protocol, hardware, and quality bar; a run missing a bounded metric is left out, not guessed.

gpt-oss-120b · Offline

8× NVIDIA B300-SXM-270GB · mlperf_closed_99

Ranked only inside this comparison group · 6 feasible · TTFT p99 reported for 0 of 6

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT)

TensorRT 10.13, CUDA 13.0

111,496

not reported

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0010Run detail →

ASUSTeK · XA NB3I-E12

TensorRT 10.14, CUDA 13.1

110,077

not reported

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0006Run detail →

Lenovo · ThinkSystem SR680a V4 (8x B300-SXM-270GB, TensorRT)

TensorRT 10.14, CUDA 13.1

109,141

not reported

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0066Run detail →

GigaComputing · G894-SD3-AAX7

TensorRT 10.14, CUDA 13.1

107,696

not reported

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0039Run detail →

Nebius · Nebius B300 n1 (8x B300-SXM-270GB, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

106,885

not reported

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0084Run detail →

NVIDIA · NVIDIA DGX B300 (8x B300-SXM-270GB, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

103,961

not reported

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0073Run detail →

gpt-oss-120b · Offline

8× AMD Instinct MI355X 288GB HBM3e · mlperf_closed_99

Ranked only inside this comparison group · 5 feasible · TTFT p99 reported for 0 of 5

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC)

PyTorch 2.9.1+git8907517, ROCm 7.0.0

95,004

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0002Run detail →

GigaComputing · G893-ZX1-AAX4

PyTorch 2.9.1+git8907517, ROCm 7.0.0

92,332

gpt-oss-120b · Offline

4× NVIDIA GB300 × 18 nodes · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Nebius · Nebius GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, Driver 580.105.08, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo

1,046,150

not reported

4× NVIDIA GB300 × 18 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0081Run detail →

NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

gpt-oss-120b · Offline

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

RedHat · LLM-D v0.5.0,Openshift 4.20.12,NVIDIA 8xB200-SXM-180GB

LLM-D v0.5.0 ,vLLM 0.14.1, RHEL 9.6

93,071

not reported

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0091Run detail →

Nebius · Nebius B200 n1 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

gpt-oss-120b · Offline

4× Intel(R) Arc Pro(R) B70 · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Intel · 1-node-4x-BMG-B70

PyTorch vLLM

1,537

not reported

4× Intel(R) Arc Pro(R) B70 · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

Availability: preview.

MLPerf Inference · entry v6.0/6.0-0101Run detail →

gpt-oss-120b · Offline

4× NVIDIA GB300 · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Lambda_SIT · NVIDIA GB300 NVL72 (4x GB300-288GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1

60,220

not reported

4× NVIDIA GB300 · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0063Run detail →

gpt-oss-120b · Offline

4× MS-Intel Arc Pro B60 Dual 48G Turbo · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Intel · 1-node-4x-BMG-Pro-B60-Dual

PyTorch vLLM

1,602

not reported

4× MS-Intel Arc Pro B60 Dual 48G Turbo · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0060Run detail →

gpt-oss-120b · Offline

4× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

HPE · HPE ProLiant DL385 Gen11 (4x NVIDIA RTX PRO 6000 Blackwell Server Edition-450W, TensorRT)

TensorRT 10.14.1.48, CUDA 13.0

7,297

not reported

4× NVIDIA RTX PRO 6000 Blackwell Server Edition · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0054Run detail →

gpt-oss-120b · Offline

4× NVIDIA GB200 × 16 nodes · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

CoreWeave · CoreWeave GB200 NVL72 (64x GB200-186GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1

774,539

not reported

4× NVIDIA GB200 × 16 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0015Run detail →

gpt-oss-120b · Offline

8× AMD Instinct MI355X 288GB HBM3e × 12 nodes · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · AMD MI355X Custom Cluster

PyTorch 2.9.1+git8907517, ROCm 7.0.0

1,031,070

not reported

8× AMD Instinct MI355X 288GB HBM3e × 12 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0003Run detail →

gpt-oss-120b · Offline

4× Intel(R) Arc Pro(R) B60 · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

Intel · 1-node-4x-BMG-B60

PyTorch vLLM

841

not reported

4× Intel(R) Arc Pro(R) B60 · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0059Run detail →

gpt-oss-120b · Offline

4× NVIDIA GB300 × 16 nodes · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

CoreWeave · CoreWeave GB300 NVL72 (64x GB300-288GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1

910,468

not reported

4× NVIDIA GB300 × 16 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0016Run detail →

5 more comparison groups. Narrow by model, workload, or hardware to see them.