Expert weighted aggregation with shared expert
Expert computation and weighted aggregation in Sparse MoE with shared expert. Performs parallel expert forward passes (SwiGLU MLPs), applies routing weights, aggregates outputs, and adds gated shared expert contribution.
Reported evidence · last observed 2026-08-16. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhidden_size = 2048 · num_experts = 512 · num_experts_per_tok = 10 · moe_intermediate_size = 512 · shared_expert_intermediate_size = 51216 cases
Not measured on H100 for this workload. Challenges →
1714.4µs83% SOL1.00×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
2728.9µs82% SOL1.02×Reported · license unknown · no source2026-08-05
Measured exactly what you asked. Reported by source; not independently reproduced.
3733.5µs82% SOL1.03×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
4739.6µs82% SOL1.04×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
5749.9µs81% SOL1.05×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
6767.0µs80% SOL1.07×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
7769.8µs80% SOL1.08×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
8785.9µs80% SOL1.10×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
9814.6µs78% SOL1.14×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
10846.6µs77% SOL1.19×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
11863.8µs76% SOL1.21×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
12912.6µs74% SOL1.28×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
131.09ms68% SOL1.53×Reported · license unknown · no source2026-07-20
Measured exactly what you asked. Reported by source; not independently reproduced.
141.16ms66% SOL1.62×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
151.16ms65% SOL1.63×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
161.21ms64% SOL1.70×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
171.52ms56% SOL2.13×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
181.89ms49% SOL2.65×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
192.03ms46% SOL2.85×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.