MoE expert parallel execution backward
Backward pass for MoE expert parallel execution. Computes gradients for hidden_states, topk_weights, gate_weights, up_weights, and down_weights through the SwiGLU MLP expert computation with token dispatch and weighted aggregation.
Reported evidence · last observed 2026-07-22. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhidden_size = 4096 · n_routed_experts = 256 · num_experts_per_tok = 8 · moe_intermediate_size = 204816 cases
Not measured on H100 for this workload. Challenges →
112.6ms54% SOL1.00×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
212.7ms53% SOL1.00×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
312.7ms53% SOL1.00×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
412.8ms53% SOL1.01×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
512.8ms53% SOL1.01×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
612.8ms53% SOL1.01×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
713.0ms53% SOL1.03×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
813.1ms52% SOL1.04×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
913.1ms52% SOL1.04×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
1013.6ms51% SOL1.08×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1114.2ms50% SOL1.12×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
1214.2ms50% SOL1.13×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
1314.3ms50% SOL1.13×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
1414.4ms50% SOL1.14×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
1514.6ms49% SOL1.15×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
1616.3ms46% SOL1.29×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1721.6ms39% SOL1.71×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
1821.7ms38% SOL1.72×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
1921.9ms38% SOL1.73×Reported · license unknown · no source2026-07-25
Measured exactly what you asked. Reported by source; not independently reproduced.
2021.9ms38% SOL1.73×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
2121.9ms38% SOL1.74×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
2221.9ms38% SOL1.74×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
2322.3ms38% SOL1.77×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
2438.8ms25% SOL3.08×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
25440.0ms3% SOL34.87×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.