Skip to content
KernelIndex
Search⌘K
Sign in

FP8 MoE router projection

28 eligible runs
moe

FP8-quantized MoE router gate projection that computes expert routing logits. Projects hidden states to expert scores using blockwise FP8 quantization with BlockWise1x128 scaling for activations and BlockWise128x128 scaling for weights. The weight matrix is padded to 128 experts for BlockWise128x128 compatibility.

Fastest reported
8.69µsmean · 81% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-22. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhidden_size = 3584 · num_experts = 64 · hidden_blocks = 28 · padded_experts = 12816 cases
m
dtype
256fp32/fp8_e4m3768fp32/fp8_e4m3896fp32/fp8_e4m31024fp32/fp8_e4m31280fp32/fp8_e4m31408fp32/fp8_e4m32048fp32/fp8_e4m33072fp32/fp8_e4m33328fp32/fp8_e4m33584fp32/fp8_e4m34096fp32/fp8_e4m34352fp32/fp8_e4m34864fp32/fp8_e4m35376fp32/fp8_e4m36144fp32/fp8_e4m38192fp32/fp8_e4m3
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 18.69 µsRIAC as well21NVIDIA B200 · env 29.68 µsRecursive7

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 21 results · last observed 2026-08-24Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
8.69µs81% SOL
1.00×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
9.16µs80% SOL
1.05×
Reported · license unknown · no source
2026-08-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Amir M. Mir | SF Tensor
10.1µs78% SOL
1.16×
Reported · license unknown · no source
2026-08-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Laser
10.2µs78% SOL
1.17×
Reported · license unknown · no source
2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Amir M. Mir | SF Tensor
10.5µs77% SOL
1.21×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Geometric
10.7µs77% SOL
1.23×
Reported · license unknown · no source
2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Q H
10.8µs77% SOL
1.24×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
ac4k
11.3µs76% SOL
1.30×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Gleaming Coyote
11.4µs75% SOL
1.31×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Rapid Tiger
11.4µs75% SOL
1.31×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Electric Cheetah
11.4µs75% SOL
1.31×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Geometric
12.0µs74% SOL
1.38×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Amir M. Mir | SF Tensor
12.0µs74% SOL
1.38×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Amir M. Mir | SF Tensor
12.9µs73% SOL
1.49×
Reported · license unknown · no source
2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Hyra
13.3µs72% SOL
1.53×
Reported · license unknown · no source
2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
doubleAI
13.4µs72% SOL
1.55×
Reported · license unknown · no source
2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Amir M. Mir | SF Tensor
15.8µs68% SOL
1.82×
Reported · license unknown · no source
2026-07-27

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
Amir M. Mir | SF Tensor
22.1µs60% SOL
2.54×
Reported · license unknown · no source
2026-07-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
19
Cursor
24.3µs57% SOL
2.79×
Reported · license unknown · no source
2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
20
Amir M. Mir | SF Tensor
26.9µs55% SOL
3.10×
Reported · license unknown · no source
2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
21
IcySky
45.3µs41% SOL
5.21×
Reported · license unknown · no source
2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Q H
unknown
10.8µs
1.24×
Reported
license unknown · no source
ac4k
unknown
11.3µs
1.30×
Reported
license unknown · no source
Gleaming Coyote
unknown
11.4µs
1.31×
Reported
license unknown · no source
RIAC as well
unknown
8.69µs
1.00×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
9.16µs
1.05×
Reported
license unknown · no source
Laser
unknown
10.2µs
1.17×
Reported
license unknown · no source
Geometric
unknown
10.7µs
1.23×
Reported
license unknown · no source
Recursive
unknown
9.68µs
1.11×
Reported
license unknown · no source
Show all 28 implementations ›
Rapid Tiger
unknown
11.4µs
1.31×
Reported
license unknown · no source
Electric Cheetah
unknown
11.4µs
1.31×
Reported
license unknown · no source
doubleAI
unknown
12.8µs
1.48×
Reported
license unknown · no source
Hyra
unknown
13.3µs
1.53×
Reported
license unknown · no source
Cursor
unknown
23.0µs
2.65×
Reported
license unknown · no source
IcySky
unknown
31.6µs
3.63×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
10.1µs
1.16×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
10.5µs
1.21×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
12.0µs
1.38×
Reported
license unknown · no source
Geometric
unknown
12.0µs
1.38×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
12.9µs
1.49×
Reported
license unknown · no source
doubleAI
unknown
13.4µs
1.55×
Reported
license unknown · no source
doubleAI
unknown
13.8µs
1.59×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
15.8µs
1.82×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
22.1µs
2.54×
Reported
license unknown · no source
Cursor
unknown
24.3µs
2.79×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
26.5µs
3.05×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
26.9µs
3.10×
Reported
license unknown · no source
IcySky
unknown
45.3µs
5.21×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
185.1µs
21.30×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesfp8_e4m3 [m, hidden_size]
gate_weightfp8_e4m3 [padded_experts, hidden_size]
scale_hiddenfp32 [m, hidden_blocks]
scale_weightfp32 [1, hidden_blocks]
router_logitsbf16 [m, num_experts]
Axes and behavior
mvariable
hidden_sizeconstant = 3584
num_expertsconstant = 64
hidden_blockshidden_blocks = hidden_size // 128
padded_expertsconstant = 128
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias005_fp8_moe_router_projectionmodelqwen3-vl-30b-a3b-instruct-fp8sha256885aeac1e200…
Sources: NVIDIA SOL-ExecBench (2026-08-24)last observed 2026-08-24How records are decidedJSON