MoE expert batched execution with capacity factor
Optimized batched MoE expert execution with capacity-based token batching. Groups tokens by assigned experts using capacity factors, executes all experts in parallel using batched matrix operations, and performs efficient scatter-gather for weighted aggregation. Uses SwiGLU activation (silu(gate) * up) for expert computation. Capacity factor is fixed at 1.25.
Reported evidence · last observed 2026-08-19. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhidden_size = 6144 · num_experts = 160 · num_experts_per_tok = 8 · moe_intermediate_size = 256016 cases
Not measured on H100 for this workload. Challenges →
12.14ms94% SOL1.00×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
22.24ms94% SOL1.05×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
32.43ms84% SOL1.14×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
42.44ms84% SOL1.14×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
52.52ms80% SOL1.18×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
62.64ms76% SOL1.23×Reported · license unknown · no source2026-08-05
Measured exactly what you asked. Reported by source; not independently reproduced.
72.65ms76% SOL1.24×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
82.67ms75% SOL1.25×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
92.87ms73% SOL1.34×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
103.18ms61% SOL1.49×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
113.30ms60% SOL1.54×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
123.48ms55% SOL1.63×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
133.75ms51% SOL1.75×Reported · license unknown · no source2026-07-25
Measured exactly what you asked. Reported by source; not independently reproduced.
143.93ms48% SOL1.84×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
154.00ms47% SOL1.87×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
164.04ms47% SOL1.89×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
174.12ms46% SOL1.93×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
187.96ms23% SOL3.72×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.