RMSNorm h4096
Root Mean Square Normalization with hidden_size=4096. Captured from Llama-3.1-8B. Epsilon is fixed at 1e-5.
Reported evidence · last observed 2026-08-03. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 14 cases · mean latency28 cases
Not measured on H100 for this workload. Challenges →
15.75µs81% SOL1.00×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
26.00µs82% SOL1.04×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
36.07µs82% SOL1.05×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
46.08µs81% SOL1.06×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
4=6.08µs80% SOL1.06×Reported · license unknown · no source2026-08-11
Measured exactly what you asked; tied ranks share a number. Reported by source; not independently reproduced.
56.09µs81% SOL1.06×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
66.12µs80% SOL1.06×Reported · license unknown · no source2026-07-30
Measured exactly what you asked. Reported by source; not independently reproduced.
76.12µs80% SOL1.06×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
86.24µs77% SOL1.08×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
96.28µs80% SOL1.09×Reported · license unknown · no source2026-08-01
Measured exactly what you asked. Reported by source; not independently reproduced.
106.29µs79% SOL1.09×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
116.33µs78% SOL1.10×Reported · license unknown · no source2026-07-31
Measured exactly what you asked. Reported by source; not independently reproduced.
126.35µs79% SOL1.10×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
136.38µs77% SOL1.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
146.42µs78% SOL1.12×Reported · license unknown · no source2026-07-30
Measured exactly what you asked. Reported by source; not independently reproduced.
156.46µs78% SOL1.12×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
166.59µs77% SOL1.14×Reported · license unknown · no source2026-07-29
Measured exactly what you asked. Reported by source; not independently reproduced.
176.64µs77% SOL1.15×Reported · license unknown · no source2026-08-01
Measured exactly what you asked. Reported by source; not independently reproduced.
186.75µs75% SOL1.17×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
196.78µs77% SOL1.18×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
206.79µs76% SOL1.18×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
217.00µs75% SOL1.22×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
227.07µs74% SOL1.23×Reported · license unknown · no source2026-07-18
Measured exactly what you asked. Reported by source; not independently reproduced.
237.18µs76% SOL1.25×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
247.21µs74% SOL1.25×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
257.68µs72% SOL1.34×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
267.73µs72% SOL1.34×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
277.74µs71% SOL1.35×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
288.09µs70% SOL1.41×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
298.34µs69% SOL1.45×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
309.64µs62% SOL1.67×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
Implementations
Show all 56 implementations ›Showing all 56 implementations ⌄
Semantics
Also indexed as 025-rmsnorm-h4096 · reviewed as the same computation; every definition's cohorts are shown here, ranked separately.