Serving · KernelIndex

24 comparison groups · 66 feasible · 0 excluded

Ranked only within one model, workload, protocol, hardware, and quality bar; a run missing a bounded metric is left out, not guessed.

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Offline

8× AMD Instinct MI355X 288GB HBM3e · mlperf_closed_99

Ranked only inside this comparison group · 7 feasible · TTFT p99 reported for 0 of 7

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC)

PyTorch 2.9.1+git8907517, ROCm 7.0.0

103,480

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0002Run detail →

Supermicro · QuantaGrid D74A-7U

Supermicro submission (software not stated) TODO

100,754

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0097Run detail →

GigaComputing · G893-ZX1-AAX4

PyTorch 2.9.1+git8907517, ROCm 7.0.0

100,310

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0037Run detail →

HPE · HPE ProLiant Compute XD685 (8x AMD Instinct MI355X 288GB, vLLM)

vLLM

100,019

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0049Run detail →

Dell · PowerEdge XE9785L (8x MI355X)

vLLM 0.14.0rc3.dev30+gb026cf14e.d20260212.rocm700, Pytorch 2.9.1+git8907517, ROCm 7.0.0.70000-38~22.04

100,008

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0020Run detail →

ORACLE · BM.GPU.MI355X.8

Oracle submission (software not stated)

99,874

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0086Run detail →

Dell_MangoBoost · PowerEdge XE9785

ROCm

97,535

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0032Run detail →

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Offline

8× AMD Instinct MI355X 288GB HBM3e · mlperf_closed_99_9

Ranked only inside this comparison group · 7 feasible · TTFT p99 reported for 0 of 7

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC)

PyTorch 2.9.1+git8907517, ROCm 7.0.0

103,480

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99_9 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0002Run detail →

Supermicro · QuantaGrid D74A-7U

Supermicro submission (software not stated) TODO

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Server

8× AMD Instinct MI355X 288GB HBM3e · mlperf_closed_99

Ranked only inside this comparison group · 7 feasible · TTFT p99 reported for 0 of 7

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC)

PyTorch 2.9.1+git8907517, ROCm 7.0.0

100,282

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0002Run detail →

ORACLE · BM.GPU.MI355X.8

Oracle submission (software not stated)

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Server

8× AMD Instinct MI355X 288GB HBM3e · mlperf_closed_99_9

Ranked only inside this comparison group · 7 feasible · TTFT p99 reported for 0 of 7

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC)

PyTorch 2.9.1+git8907517, ROCm 7.0.0

100,282

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99_9 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0002Run detail →

ORACLE · BM.GPU.MI355X.8

Oracle submission (software not stated)

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Interactive

8× AMD Instinct MI355X 288GB HBM3e · mlperf_closed_99

Ranked only inside this comparison group · 6 feasible · TTFT p99 reported for 0 of 6

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC)

PyTorch 2.9.1+git8907517, ROCm 7.0.0

73,608

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0002Run detail →

GigaComputing · G893-ZX1-AAX4

PyTorch 2.9.1+git8907517, ROCm 7.0.0

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Interactive

8× AMD Instinct MI355X 288GB HBM3e · mlperf_closed_99_9

Ranked only inside this comparison group · 6 feasible · TTFT p99 reported for 0 of 6

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC)

PyTorch 2.9.1+git8907517, ROCm 7.0.0

73,608

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99_9 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0002Run detail →

GigaComputing · G893-ZX1-AAX4

PyTorch 2.9.1+git8907517, ROCm 7.0.0

GPT-OSS-120B (MLPerf reference) · gpt-oss-120b · Offline

8× AMD Instinct MI355X 288GB HBM3e · mlperf_closed_99

Ranked only inside this comparison group · 5 feasible · TTFT p99 reported for 0 of 5

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC)

PyTorch 2.9.1+git8907517, ROCm 7.0.0

95,004

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0002Run detail →

GigaComputing · G893-ZX1-AAX4

PyTorch 2.9.1+git8907517, ROCm 7.0.0

GPT-OSS-120B (MLPerf reference) · gpt-oss-120b · Server

8× AMD Instinct MI355X 288GB HBM3e · mlperf_closed_99

Ranked only inside this comparison group · 5 feasible · TTFT p99 reported for 0 of 5

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC)

PyTorch 2.9.1+git8907517, ROCm 7.0.0

82,136

not reported

8× AMD Instinct MI355X 288GB HBM3e · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0002Run detail →

GigaComputing · G893-ZX1-AAX4

PyTorch 2.9.1+git8907517, ROCm 7.0.0

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Server

8× AMD Instinct MI355X 288GB HBM3e × 11 nodes · mlperf_closed_99_9

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · AMD MI355X Custom Cluster

PyTorch 2.9.1+git8907517, ROCm 7.0.0

1,016,380

not reported

8× AMD Instinct MI355X 288GB HBM3e × 11 nodes · mlperf_loadgen v6.0 · mlperf_closed_99_9 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0001Run detail →

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Interactive

8× AMD Instinct MI355X 288GB HBM3e × 11 nodes · mlperf_closed_99_9

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · AMD MI355X Custom Cluster

PyTorch 2.9.1+git8907517, ROCm 7.0.0

785,522

not reported

8× AMD Instinct MI355X 288GB HBM3e × 11 nodes · mlperf_loadgen v6.0 · mlperf_closed_99_9 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0001Run detail →

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Interactive

8× AMD Instinct MI355X 288GB HBM3e × 11 nodes · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · AMD MI355X Custom Cluster

PyTorch 2.9.1+git8907517, ROCm 7.0.0

785,522

not reported

8× AMD Instinct MI355X 288GB HBM3e × 11 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0001Run detail →

Llama-2-70B (MLPerf reference) · llama2-70b-99 · Offline

8× AMD Instinct MI355X 288GB HBM3e × 11 nodes · mlperf_closed_99_9

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

AMD · AMD MI355X Custom Cluster

PyTorch 2.9.1+git8907517, ROCm 7.0.0

1,042,110

not reported

8× AMD Instinct MI355X 288GB HBM3e × 11 nodes · mlperf_loadgen v6.0 · mlperf_closed_99_9 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0001Run detail →

12 more comparison groups. Narrow by model, workload, or hardware to see them.