Kernels Serving

Serving

722 results · all from MLPerf Inference, shown as published

11 comparison groups · 26 feasible · 0 excluded

Ranked only within one model, workload, protocol, hardware, and quality bar; a run missing a bounded metric is left out, not guessed.

deepseek-r1 · Server

8× NVIDIA B300-SXM-270GB · mlperf_closed_99

Ranked only inside this comparison group · 7 feasible · TTFT p99 reported for 0 of 7

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

Nebius · Nebius B300 n1 (8x B300-SXM-270GB, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

60,413

not reported

not reported

8

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0084Run detail →

2

Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT)

TensorRT 10.13, CUDA 13.0

58,553

not reported

not reported

8

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0010Run detail →

3

Dell · PowerEdge XE9780 (8x B300-SXM-270GB, TensorRT)

TensorRT 10.14, CUDA 13.0

46,513

not reported

not reported

8

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0027Run detail →

4

Dell · PowerEdge XE9780L (8x B300-SXM-270GB, TensorRT)

TensorRT 10.14, CUDA 13.1

46,511

not reported

not reported

8

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0024Run detail →

5

NVIDIA · NVIDIA DGX B300 (8x B300-SXM-270GB, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

42,721

not reported

not reported

8

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0073Run detail →

6

GigaComputing · G894-SD3-AAX7

TensorRT 10.14, CUDA 13.1

35,553

not reported

not reported

8

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0039Run detail →

7

Quanta_Cloud_Technology · QuantaGrid D75H-10U (8x B300-SXM-270GB, TensorRT)

TensorRT 10.14, CUDA 13.1

35,436

not reported

not reported

8

8× NVIDIA B300-SXM-270GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0089Run detail →

deepseek-r1 · Server

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 5 feasible · TTFT p99 reported for 0 of 5

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

NVIDIA · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.11, CUDA 12.9

18,592

not reported

not reported

8

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0069Run detail →

2

GigaComputing · G894-SD1

TensorRT 10.11, CUDA 12.9

15,978

deepseek-r1 · Server

8× NVIDIA B200-SXM-180GB · mlperf_closed_99

Ranked only inside this comparison group · 3 feasible · TTFT p99 reported for 0 of 3

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

Nebius · Nebius B200 n1 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

51,693

not reported

not reported

8

8× NVIDIA B200-SXM-180GB · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0083Run detail →

2

Google · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT)

TensorRT 10.14, CUDA 13.1

deepseek-r1 · Server

4× NVIDIA GB200 × 18 nodes · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

336,106

not reported

not reported

72

4× NVIDIA GB200 × 18 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0075Run detail →

2

Google · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1

deepseek-r1 · Server

4× NVIDIA GB300 × 2 nodes · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

Nebius · Nebius GB300 NVL72 (4x GB300-288GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, Driver 580.105.08, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo

64,510

not reported

not reported

8

4× NVIDIA GB300 × 2 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0082Run detail →

2

CoreWeave · CoreWeave GB300 NVL72 (8x GB300-288GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1

deepseek-r1 · Server

4× NVIDIA GB300 × 18 nodes · mlperf_closed_99

Ranked only inside this comparison group · 2 feasible · TTFT p99 reported for 0 of 2

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

Nebius · Nebius GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, Driver 580.105.08, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo

575,580

not reported

not reported

72

4× NVIDIA GB300 × 18 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0081Run detail →

2

NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

deepseek-r1 · Server

4× NVIDIA GB300 × 18 nodes · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

NVIDIA · (72x GB300-288GB_aarch64, TensorRT)

TensorRT 10.13, CUDA 13.0

209,328

not reported

not reported

72

4× NVIDIA GB300 × 18 nodes · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0072Run detail →

deepseek-r1 · Server

4× NVIDIA GB300 × 72 nodes · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM

1,555,110

not reported

not reported

288

4× NVIDIA GB300 × 72 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0076Run detail →

deepseek-r1 · Server

4× NVIDIA GB200 × 16 nodes · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

CoreWeave · CoreWeave GB200 NVL72 (64x GB200-186GB_aarch64, TensorRT)

TensorRT 10.14, CUDA 13.1

311,531

not reported

not reported

64

4× NVIDIA GB200 × 16 nodes · mlperf_loadgen v6.0 · mlperf_closed_99 · Reported evidence · observed 2026-03-26

MLPerf Inference · entry v6.0/6.0-0015Run detail →

deepseek-r1 · Server

4× NVIDIA GB300 × 2 nodes · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

CoreWeave · (8x GB300-288GB_aarch64, TensorRT)

TensorRT 10.13, CUDA 13.0

22,545

not reported

not reported

8

4× NVIDIA GB300 × 2 nodes · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

Availability: preview.

MLPerf Inference · entry v5.1/5.1-0097Run detail →

deepseek-r1 · Server

4× NVIDIA GB200 × 18 nodes · mlperf_closed_99

Ranked only inside this comparison group · 1 feasible · TTFT p99 reported for 0 of 1

#

Configuration

Stack

tokens/s

TTFT p99

TPOT p99

GPUs

1

NVIDIA · (72x GB200-186GB_aarch64, TensorRT)

TensorRT 10.11, CUDA 12.9

167,578

not reported

not reported

72

4× NVIDIA GB200 × 18 nodes · mlperf_loadgen v5.1 · mlperf_closed_99 · Reported evidence · observed 2025-09-04

MLPerf Inference · entry v5.1/5.1-0071Run detail →