Skip to content
KernelIndex
Search⌘K
Sign in

MoE sparse routing and dispatch backward

22 eligible runs
moe

Backward pass for MoE sparse routing and dispatch. Computes gradients through softmax, top-k selection (straight-through estimator), normalization, and router linear projection.

Fastest reported
15.2µsmean · 78% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-23. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencytop_k = 8 · hidden_size = 2048 · num_experts = 12816 cases
num_tokens
dtype
1024bf16/fp321056bf16/fp321087bf16/fp321088bf16/fp321120bf16/fp321152bf16/fp321163bf16/fp321184bf16/fp321216bf16/fp321248bf16/fp321249bf16/fp321280bf16/fp322048bf16/fp323557bf16/fp328192bf16/fp3265536bf16/fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 115.2 µsRIAC as well14NVIDIA B200 · env 217.2 µsRecursive8

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 14 results · last observed 2026-08-23Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
15.2µs78% SOL
1.00×
Reported · license unknown · no source
2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Hyra
16.0µs77% SOL
1.05×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Recursive
17.5µs75% SOL
1.15×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
ac4k
19.0µs74% SOL
1.25×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
doubleAI
19.1µs73% SOL
1.26×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Proud Wren @ NVIDIA
20.8µs72% SOL
1.37×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Geometric
21.1µs71% SOL
1.39×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Amir M. Mir | SF Tensor
23.6µs69% SOL
1.56×
Reported · license unknown · no source
2026-07-28

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Amir M. Mir | SF Tensor
25.3µs67% SOL
1.67×
Reported · license unknown · no source
2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Geometric
25.5µs67% SOL
1.68×
Reported · license unknown · no source
2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Amir M. Mir | SF Tensor
26.3µs66% SOL
1.73×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Q H
26.8µs65% SOL
1.76×
Reported · license unknown · no source
2026-08-08

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Geometric
26.9µs65% SOL
1.77×
Reported · license unknown · no source
2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Cursor
28.3µs64% SOL
1.87×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
RIAC as well
unknown
15.2µs
1.00×
Reported
license unknown · no source
doubleAI
unknown
18.3µs
1.21×
Reported
license unknown · no source
ac4k
unknown
19.0µs
1.25×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
20.8µs
1.37×
Reported
license unknown · no source
Geometric
unknown
21.1µs
1.39×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
23.6µs
1.56×
Reported
license unknown · no source
Q H
unknown
26.8µs
1.76×
Reported
license unknown · no source
Cursor
unknown
27.6µs
1.82×
Reported
license unknown · no source
Show all 22 implementations ›
doubleAI
unknown
19.1µs
1.26×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
20.7µs
1.36×
Reported
license unknown · no source
doubleAI
unknown
20.9µs
1.38×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
25.2µs
1.66×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
25.3µs
1.67×
Reported
license unknown · no source
Geometric
unknown
25.5µs
1.68×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
26.3µs
1.73×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
26.6µs
1.75×
Reported
license unknown · no source
Geometric
unknown
26.9µs
1.77×
Reported
license unknown · no source
Cursor
unknown
28.3µs
1.87×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
34.5µs
2.27×
Reported
license unknown · no source
Hyra
unknown
16.0µs
1.05×
Reported
license unknown · no source
Recursive
unknown
17.2µs
1.13×
Reported
license unknown · no source
Recursive
unknown
17.5µs
1.15×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesbf16 [num_tokens, hidden_size]
gate_weightbf16 [num_experts, hidden_size]
router_logitsbf16 [num_tokens, num_experts]
routing_probsfp32 [num_tokens, num_experts]
selected_expertsint64 [num_tokens, top_k]
routing_weights_sumfp32 [num_tokens, 1]
grad_routing_weightsbf16 [num_tokens, top_k]
grad_expert_maskbf16 [num_experts, top_k, num_tokens]
grad_router_logitsbf16 [num_tokens, num_experts]
grad_hidden_statesbf16 [num_tokens, hidden_size]
grad_gate_weightbf16 [num_experts, hidden_size]
Axes and behavior
top_kconstant = 8
num_tokensvariable
hidden_sizeconstant = 2048
num_expertsconstant = 128
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias011_moe_sparse_routing_and_dispatch_backwardmodelqwen3-coder-30b-a3b-instructsha256fe1f4f88e04a…
Sources: NVIDIA SOL-ExecBench (2026-08-23)last observed 2026-08-23How records are decidedJSON