Embedding with initial LayerNorm backward
Backward pass for fused embedding lookup and RMSNorm. Computes gradients for embedding table (scatter-add) and RMSNorm weight (reduction). The forward pass performs: embed_lookup -> RMSNorm(x * rsqrt(mean(x^2) + eps) * weight).
Reported evidence · last observed 2026-07-16. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyvocab_size = 65536 · hidden_size = 409616 cases
Not measured on H100 for this workload. Challenges →
134.8µs90% SOL1.00×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
242.5µs87% SOL1.22×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
343.5µs86% SOL1.25×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
446.5µs85% SOL1.33×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
547.5µs85% SOL1.36×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
648.4µs84% SOL1.39×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
751.3µs83% SOL1.47×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
857.5µs81% SOL1.65×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
958.3µs81% SOL1.67×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1061.6µs80% SOL1.77×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
11111.8µs66% SOL3.21×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
12141.1µs60% SOL4.05×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.
13141.7µs60% SOL4.07×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
14142.6µs60% SOL4.09×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
15142.7µs60% SOL4.10×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
16144.5µs59% SOL4.15×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.
17145.2µs59% SOL4.17×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.
18147.3µs59% SOL4.23×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
19148.6µs59% SOL4.27×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
20148.6µs59% SOL4.27×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
21152.1µs58% SOL4.37×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
22152.3µs58% SOL4.37×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
23152.9µs58% SOL4.39×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
24160.4µs57% SOL4.60×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
25165.4µs56% SOL4.75×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
26167.5µs56% SOL4.81×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
27173.3µs55% SOL4.98×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
28175.6µs55% SOL5.04×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
29175.7µs55% SOL5.04×Reported · license unknown · no source2026-08-10
Measured exactly what you asked. Reported by source; not independently reproduced.
30180.5µs54% SOL5.18×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
31181.1µs54% SOL5.20×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
32181.3µs54% SOL5.20×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
33212.0µs49% SOL6.09×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
34231.0µs48% SOL6.63×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
35232.3µs48% SOL6.67×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
36232.3µs48% SOL6.67×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
37234.6µs48% SOL6.74×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
38267.6µs45% SOL7.69×Reported · license unknown · no source2026-08-07
Measured exactly what you asked. Reported by source; not independently reproduced.
39290.4µs42% SOL8.34×Reported · license unknown · no source2026-08-10
Measured exactly what you asked. Reported by source; not independently reproduced.