FP8 shared expert MLP

29 eligible runs

moe

FP8-quantized shared expert MLP for DeepSeek-V3 MoE. Processes ALL tokens unconditionally using SwiGLU activation: output = down_proj(silu(gate_proj(x)) * up_proj(x)). Uses blockwise FP8 quantization with 128x128 block scaling for weights and per-row scaling for activations.

Fastest reported

164.6µsmean · 89% of SOL

Amir M. Mir | SF TensorLicense unknown · unknown

Reported evidence · last observed 2026-08-01. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyblock_size = 128 · hidden_size = 7168 · hidden_blocks = 56 · intermediate_size = 2048 · intermediate_blocks = 1616 cases

num_tokens

dtype

128bf16/fp32 384bf16/fp32 512bf16/fp32 1280bf16/fp32 1408bf16/fp32 1792bf16/fp32 1920bf16/fp32 2176bf16/fp32 2304bf16/fp32 2432bf16/fp32 2816bf16/fp32 3072bf16/fp32 3200bf16/fp32 3456bf16/fp32 3584bf16/fp32 3712bf16/fp32

scrolls · 16 cases total

Aggregatessuite of 16 cases · mean latency

HardwareBest knownImplementationRuns

NVIDIA B200 · env 1164.6 µsAmir M. Mir | SF Tensor23 NVIDIA B200 · env 2195.6 µsRecursive6

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 23 results · last observed 2026-08-23Record history →

#

Implementation

Latency

vs #1

Trust

Observed

1

Amir M. Mir | SF Tensor

164.6µs89% SOL

1.00×

Reported · license unknown · no source

2026-08-01

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

2

171.6µs89% SOL

1.04×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

3

182.2µs88% SOL

1.11×

Reported · license unknown · no source

2026-08-01

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

4

197.5µs87% SOL

1.20×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

5

198.6µs87% SOL

1.21×

Reported · license unknown · no source

2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

6

200.0µs87% SOL

1.22×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

7

Amir M. Mir | SF Tensor

200.7µs86% SOL

1.22×

Reported · license unknown · no source

2026-07-28

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

8

209.6µs86% SOL

1.27×

Reported · license unknown · no source

2026-08-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

9

219.1µs85% SOL

1.33×

Reported · license unknown · no source

2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

10

224.6µs85% SOL

1.36×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

11

227.9µs85% SOL

1.38×

Reported · license unknown · no source

2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

12

233.3µs84% SOL

1.42×

Reported · license unknown · no source

2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

13

239.5µs84% SOL

1.46×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

14

241.4µs84% SOL

1.47×

Reported · license unknown · no source

2026-08-08

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

15

241.6µs84% SOL

1.47×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

16

292.2µs80% SOL

1.78×

Reported · license unknown · no source

2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

17

292.2µs80% SOL

1.78×

Reported · license unknown · no source

2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

18

314.2µs78% SOL

1.91×

Reported · license unknown · no source

2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

19

Amir M. Mir | SF Tensor

317.1µs78% SOL

1.93×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

20

337.4µs77% SOL

2.05×

Reported · license unknown · no source

2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

21

390.5µs74% SOL

2.37×

Reported · license unknown · no source

2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

22

454.3µs70% SOL

2.76×

Reported · license unknown · no source

2026-07-25

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

23

725.6µs59% SOL

4.41×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation

Runtime

Best latency

Evidence

Availability

unknown

195.6µs

1.19×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

164.6µs

1.00×

Reported

license unknown · no source

unknown

182.2µs

1.11×

Reported

license unknown · no source

unknown

171.6µs

1.04×

Reported

license unknown · no source

unknown

197.5µs

1.20×

Reported

license unknown · no source

unknown

211.9µs

1.29×

Reported

license unknown · no source

unknown

233.3µs

1.42×

Reported

license unknown · no source

unknown

712.8µs

4.33×

Reported

license unknown · no source

Show all 29 implementations ›

unknown

198.6µs

1.21×

Reported

license unknown · no source

unknown

200.0µs

1.22×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

200.7µs

1.22×

Reported

license unknown · no source

unknown

209.6µs

1.27×

Reported

license unknown · no source

unknown

219.1µs

1.33×

Reported

license unknown · no source

unknown

224.6µs

1.36×

Reported

license unknown · no source

unknown

227.9µs

1.38×

Reported

license unknown · no source

unknown

231.5µs

1.41×

Reported

license unknown · no source

unknown

239.5µs

1.46×

Reported

license unknown · no source

unknown

241.4µs

1.47×

Reported

license unknown · no source

unknown

241.6µs

1.47×

Reported

license unknown · no source

unknown

292.2µs

1.78×

Reported

license unknown · no source

unknown

292.2µs

1.78×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

301.4µs

1.83×

Reported

license unknown · no source

unknown

314.2µs

1.91×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

317.1µs

1.93×

Reported

license unknown · no source

unknown

337.4µs

2.05×

Reported

license unknown · no source

unknown

390.5µs

2.37×

Reported

license unknown · no source

unknown

454.3µs

2.76×

Reported

license unknown · no source

unknown

725.6µs

4.41×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

4.68ms

28.45×

Reported

license unknown · no source

Semantics

Inputs and outputs

xbf16 [num_tokens, hidden_size]

gate_proj_weightbf16 [intermediate_size, hidden_size]

gate_proj_weight_scalesfp32 [hidden_blocks, intermediate_blocks]

up_proj_weightbf16 [intermediate_size, hidden_size]

up_proj_weight_scalesfp32 [hidden_blocks, intermediate_blocks]

down_proj_weightbf16 [hidden_size, intermediate_size]

down_proj_weight_scalesfp32 [intermediate_blocks, hidden_blocks]

outputbf16 [num_tokens, hidden_size]

Axes and behavior

block_sizeconstant = 128

num_tokensvariable

hidden_sizeconstant = 7168

hidden_blockshidden_blocks = hidden_size // block_size

intermediate_sizeconstant = 2048

intermediate_blocksintermediate_blocks = intermediate_size // block_size

determinismunspecified

constraintsNo mutation or aliasing

Identity

alias017_fp8_shared_expert_mlpmodeldeepseek-v3sha256468e75d1cac5…

Sources: NVIDIA SOL-ExecBench (2026-08-23)last observed 2026-08-23How records are decided JSON