Batched expert forward
Batched MoE expert forward pass for inference that processes all experts in parallel. Includes gate_up projection, custom gated GLU activation with clamping, down projection, and routing weight application.
Reported evidence · last observed 2026-08-23. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 14 cases · mean latencyexpert_dim = 2880 · gate_up_dim = 5760 · hidden_size = 2880 · num_experts = 128 · intermediate_size = 288014 cases
Not measured on H100 for this workload. Challenges →
17.27ms76% SOL1.00×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
27.49ms75% SOL1.03×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
37.61ms75% SOL1.05×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
47.69ms74% SOL1.06×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
57.70ms74% SOL1.06×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
67.73ms74% SOL1.06×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
77.74ms74% SOL1.06×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
87.93ms73% SOL1.09×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
98.00ms73% SOL1.10×Reported · license unknown · no source2026-08-24
Measured exactly what you asked. Reported by source; not independently reproduced.
108.14ms73% SOL1.12×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
118.64ms71% SOL1.19×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
1212.3ms59% SOL1.69×Reported · license unknown · no source2026-07-23
Measured exactly what you asked. Reported by source; not independently reproduced.
1313.4ms62% SOL1.84×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1422.6ms41% SOL3.10×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.