MoE expert SwiGLU with down projection backward
Backward pass for MoE expert SwiGLU with down projection. Computes gradients through the complete expert MLP chain: down_proj <- SwiGLU <- [gate_proj, up_proj]. Returns gradients for input x and all three weight matrices (gate, up, down).
Reported evidence · last observed 2026-07-14. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhidden_size = 2048 · intermediate_size = 76816 cases
Not measured on H100 for this workload. Challenges →
142.2µs71% SOL1.00×Reported · license unknown · no source2026-07-14
Measured exactly what you asked. Reported by source; not independently reproduced.
245.8µs69% SOL1.09×Reported · license unknown · no source2026-07-30
Measured exactly what you asked. Reported by source; not independently reproduced.
346.9µs68% SOL1.11×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
447.0µs69% SOL1.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
549.3µs68% SOL1.17×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
649.9µs67% SOL1.18×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
750.9µs66% SOL1.21×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
851.2µs67% SOL1.21×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
954.1µs66% SOL1.28×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
1054.8µs66% SOL1.30×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
1156.5µs65% SOL1.34×Reported · license unknown · no source2026-08-24
Measured exactly what you asked. Reported by source; not independently reproduced.
1256.5µs65% SOL1.34×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
1357.1µs64% SOL1.35×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
1457.3µs64% SOL1.36×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
1557.6µs64% SOL1.36×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
1657.6µs64% SOL1.37×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
1758.2µs64% SOL1.38×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
1858.3µs65% SOL1.38×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
1962.9µs61% SOL1.49×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2067.4µs59% SOL1.60×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.
2167.6µs59% SOL1.60×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2269.2µs59% SOL1.64×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
2369.6µs58% SOL1.65×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
2469.7µs58% SOL1.65×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
2570.5µs58% SOL1.67×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
2672.6µs57% SOL1.72×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
2773.8µs56% SOL1.75×Reported · license unknown · no source2026-08-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2874.1µs57% SOL1.76×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
2974.7µs56% SOL1.77×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
3075.0µs56% SOL1.78×Reported · license unknown · no source2026-08-07
Measured exactly what you asked. Reported by source; not independently reproduced.
3175.3µs55% SOL1.79×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
3275.4µs56% SOL1.79×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
3388.0µs51% SOL2.09×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
34326.0µs20% SOL7.73×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.