MoE sparse routing and dispatch
Mixture-of-Experts sparse routing and expert dispatch system. Routes each token to top-8 experts out of 128 total experts using gating network, performs expert computation with dynamic batching, and aggregates results weighted by routing probabilities.
Reported evidence · last observed 2026-08-20. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhidden_size = 2048 · num_experts = 128 · intermediate_size = 768 · num_experts_per_tok = 816 cases
Not measured on H100 for this workload. Challenges →
1218.3µs95% SOL1.00×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
2225.3µs94% SOL1.03×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
3233.1µs94% SOL1.07×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
4234.9µs93% SOL1.08×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
5310.7µs84% SOL1.42×Reported · license unknown · no source2026-08-05
Measured exactly what you asked. Reported by source; not independently reproduced.
6326.9µs83% SOL1.50×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
7327.9µs83% SOL1.50×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
8332.0µs82% SOL1.52×Reported · license unknown · no source2026-07-23
Measured exactly what you asked. Reported by source; not independently reproduced.
9361.9µs80% SOL1.66×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
10408.3µs77% SOL1.87×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
11438.2µs74% SOL2.01×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
12446.2µs74% SOL2.04×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
13456.2µs73% SOL2.09×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
14464.3µs72% SOL2.13×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
15479.7µs71% SOL2.20×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
16524.9µs68% SOL2.40×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
17543.6µs67% SOL2.49×Reported · license unknown · no source2026-08-07
Measured exactly what you asked. Reported by source; not independently reproduced.
18853.8µs52% SOL3.91×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
19858.2µs52% SOL3.93×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
201.02ms47% SOL4.68×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
211.65ms33% SOL7.57×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
225.75ms12% SOL26.33×Reported · license unknown · no source2026-08-07
Measured exactly what you asked. Reported by source; not independently reproduced.