MoE layer complete forward with residual
Complete MoE layer forward pass that fuses routing, expert dispatch/computation, aggregation, shared expert computation. This represents the entire MoE layer computation from input hidden states through router sigmoid scoring, hierarchical group-based top-8 expert selection, parallel expert execution (160 routed experts with 1536 intermediate size), weighted aggregation, and shared expert computation (also 1536 intermediate size).
Reported evidence · last observed 2026-08-14. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyn_group = 1 · topk_group = 1 · hidden_size = 5120 · n_routed_experts = 160 · num_experts_per_tok = 8 · shared_intermediate = 1536 · moe_intermediate_size = 153616 cases
Not measured on H100 for this workload. Challenges →
11.00ms97% SOL1.00×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
21.03ms97% SOL1.03×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
31.06ms96% SOL1.06×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
41.18ms96% SOL1.18×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
51.20ms95% SOL1.20×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
61.22ms94% SOL1.22×Reported · license unknown · no source2026-07-19
Measured exactly what you asked. Reported by source; not independently reproduced.
71.32ms92% SOL1.31×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
81.32ms92% SOL1.32×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
91.36ms90% SOL1.36×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
101.44ms88% SOL1.43×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
111.56ms83% SOL1.56×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
121.75ms80% SOL1.74×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
131.80ms78% SOL1.80×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.
141.82ms78% SOL1.82×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
151.85ms76% SOL1.85×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
162.40ms66% SOL2.39×Reported · license unknown · no source2026-07-25
Measured exactly what you asked. Reported by source; not independently reproduced.
172.48ms64% SOL2.47×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
182.87ms59% SOL2.86×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
193.39ms52% SOL3.38×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
203.70ms49% SOL3.69×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2112.7ms18% SOL12.62×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.