Skip to content
KernelIndex
Search⌘K
Sign in

Fused add RMSNorm h2048

131 eligible runs
rmsnorm

Fused Add + RMSNorm with hidden_size=2048 for Qwen3-30B-A3B. Epsilon is fixed at 1e-6.

Fastest reported
4.33µsmean · 68% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-22. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 7 cases · mean latency14 cases
batch_size
hidden_size
dtype
1—bf1612048bf166—bf1662048bf1634—bf16342048bf1664—bf16642048bf1679—bf16792048bf1612383—bf16123832048bf1616254—bf16162542048bf16
scrolls · 14 cases total
Aggregatessuite of 7 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 14.33 µsRIAC as well43NVIDIA B200 · env 24.81 µsRecursive32

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 7 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 43 results · last observed 2026-08-24Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
4.33µs68% SOL
1.00×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Qwen
4.47µs67% SOL
1.03×
Reported · license unknown · no source
2026-07-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Qwen
4.51µs66% SOL
1.04×
Reported · license unknown · no source
2026-07-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
ac4k
4.54µs67% SOL
1.05×
Reported · license unknown · no source
2026-08-01

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
ac4k
4.57µs63% SOL
1.06×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Geometric
4.59µs66% SOL
1.06×
Reported · license unknown · no source
2026-08-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Geometric
4.63µs65% SOL
1.07×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Geometric
4.67µs65% SOL
1.08×
Reported · license unknown · no source
2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Geometric
4.68µs64% SOL
1.08×
Reported · license unknown · no source
2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Geometric
4.70µs65% SOL
1.09×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
ac4k
4.70µs62% SOL
1.09×
Reported · license unknown · no source
2026-07-30

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Geometric
4.71µs64% SOL
1.09×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Amir M. Mir | SF Tensor
4.80µs64% SOL
1.11×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Recursive
4.82µs63% SOL
1.11×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Geometric
4.89µs63% SOL
1.13×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
Lunar Wolf
4.91µs61% SOL
1.14×
Reported · license unknown · no source
2026-07-29

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Geometric
4.93µs60% SOL
1.14×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
Geometric
4.93µs60% SOL
1.14×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
19
doubleAI
4.99µs60% SOL
1.15×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
20
Geometric
5.00µs59% SOL
1.16×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
21
Geometric
5.04µs59% SOL
1.17×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
22
Geometric
5.09µs58% SOL
1.18×
Reported · license unknown · no source
2026-07-29

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
23
Amir M. Mir | SF Tensor
5.09µs62% SOL
1.18×
Reported · license unknown · no source
2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
24
Coral@HumanAgentSociety
5.12µs62% SOL
1.18×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
25
Geometric
5.26µs58% SOL
1.22×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
26
Geometric
5.35µs57% SOL
1.24×
Reported · license unknown · no source
2026-07-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
27
Geometric
5.35µs57% SOL
1.24×
Reported · license unknown · no source
2026-07-29

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
28
Geometric
5.39µs57% SOL
1.25×
Reported · license unknown · no source
2026-07-26

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
29
Geometric
5.43µs56% SOL
1.26×
Reported · license unknown · no source
2026-07-28

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
30
Geometric
5.50µs56% SOL
1.27×
Reported · license unknown · no source
2026-07-27

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
31
Amir M. Mir | SF Tensor
5.51µs56% SOL
1.27×
Reported · license unknown · no source
2026-07-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
32
Amir M. Mir | SF Tensor
5.52µs57% SOL
1.28×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
33
Amir M. Mir | SF Tensor
5.59µs56% SOL
1.29×
Reported · license unknown · no source
2026-07-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
34
Lucky Phoenix
5.70µs55% SOL
1.32×
Reported · license unknown · no source
2026-07-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
35
xuanzhengdu
5.71µs55% SOL
1.32×
Reported · license unknown · no source
2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
36
Hyra
5.81µs54% SOL
1.34×
Reported · license unknown · no source
2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
37
Geometric
5.92µs54% SOL
1.37×
Reported · license unknown · no source
2026-07-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
38
Stellar Flamingo
5.93µs55% SOL
1.37×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
39
Qin Liu @ UCD
5.99µs53% SOL
1.39×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
40
Geometric
6.21µs52% SOL
1.44×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
41
Theoric.com
6.41µs51% SOL
1.48×
Reported · license unknown · no source
2026-07-29

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
42
Amir M. Mir | SF Tensor
6.44µs49% SOL
1.49×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
43
Cursor
9.28µs41% SOL
2.15×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
ac4k
unknown
4.54µs
1.05×
Reported
license unknown · no source
Geometric
unknown
4.67µs
1.08×
Reported
license unknown · no source
Geometric
unknown
4.68µs
1.08×
Reported
license unknown · no source
Jayce Su
unknown
6.32µs
1.46×
Reported
license unknown · no source
Lunar Wolf
unknown
4.91µs
1.14×
Reported
license unknown · no source
ac4k
unknown
4.57µs
1.06×
Reported
license unknown · no source
RIAC as well
unknown
4.33µs
1.00×
Reported
license unknown · no source
Qwen
unknown
4.47µs
1.03×
Reported
license unknown · no source
Show all 84 implementations ›
Geometric
unknown
4.59µs
1.06×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
4.80µs
1.11×
Reported
license unknown · no source
Recursive
unknown
4.82µs
1.11×
Reported
license unknown · no source
Coral@HumanAgentSociety
unknown
4.98µs
1.15×
Reported
license unknown · no source
doubleAI
unknown
4.88µs
1.13×
Reported
license unknown · no source
Stellar Flamingo
unknown
5.93µs
1.37×
Reported
license unknown · no source
xuanzhengdu
unknown
5.71µs
1.32×
Reported
license unknown · no source
Lucky Phoenix
unknown
5.70µs
1.32×
Reported
license unknown · no source
Hyra
unknown
5.81µs
1.34×
Reported
license unknown · no source
XG
unknown
6.02µs
1.39×
Reported
license unknown · no source
Qin Liu @ UCD
unknown
5.99µs
1.39×
Reported
license unknown · no source
Jayce Su
unknown
6.08µs
1.41×
Reported
license unknown · no source
Noble Pegasus
unknown
6.17µs
1.43×
Reported
license unknown · no source
Theoric.com
unknown
6.41µs
1.48×
Reported
license unknown · no source
Neural Jackal
unknown
6.29µs
1.46×
Reported
license unknown · no source
Neon Panda
unknown
6.08µs
1.41×
Reported
license unknown · no source
Cursor
unknown
9.28µs
2.15×
Reported
license unknown · no source
Qwen
unknown
4.51µs
1.04×
Reported
license unknown · no source
Geometric
unknown
4.63µs
1.07×
Reported
license unknown · no source
Geometric
unknown
4.70µs
1.09×
Reported
license unknown · no source
Geometric
unknown
4.71µs
1.09×
Reported
license unknown · no source
Recursive
unknown
4.81µs
1.11×
Reported
license unknown · no source
Geometric
unknown
4.89µs
1.13×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
5.09µs
1.18×
Reported
license unknown · no source
ac4k
unknown
4.70µs
1.09×
Reported
license unknown · no source
Coral@HumanAgentSociety
unknown
5.12µs
1.18×
Reported
license unknown · no source
doubleAI
unknown
4.99µs
1.15×
Reported
license unknown · no source
doubleAI
unknown
4.93µs
1.14×
Reported
license unknown · no source
Geometric
unknown
4.93µs
1.14×
Reported
license unknown · no source
Geometric
unknown
4.93µs
1.14×
Reported
license unknown · no source
Geometric
unknown
5.00µs
1.16×
Reported
license unknown · no source
Geometric
unknown
5.04µs
1.17×
Reported
license unknown · no source
doubleAI
unknown
5.14µs
1.19×
Reported
license unknown · no source
Geometric
unknown
5.09µs
1.18×
Reported
license unknown · no source
Geometric
unknown
5.26µs
1.22×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
5.52µs
1.28×
Reported
license unknown · no source
Geometric
unknown
5.35µs
1.24×
Reported
license unknown · no source
Geometric
unknown
5.35µs
1.24×
Reported
license unknown · no source
Geometric
unknown
5.39µs
1.25×
Reported
license unknown · no source
Geometric
unknown
5.43µs
1.26×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
5.51µs
1.27×
Reported
license unknown · no source
Geometric
unknown
5.50µs
1.27×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
5.59µs
1.29×
Reported
license unknown · no source
Stellar Flamingo
unknown
5.96µs
1.38×
Reported
license unknown · no source
Stellar Flamingo
unknown
6.04µs
1.40×
Reported
license unknown · no source
Stellar Flamingo
unknown
6.04µs
1.40×
Reported
license unknown · no source
Stellar Flamingo
unknown
5.94µs
1.37×
Reported
license unknown · no source
Stellar Flamingo
unknown
5.95µs
1.38×
Reported
license unknown · no source
Stellar Flamingo
unknown
5.94µs
1.37×
Reported
license unknown · no source
Stellar Flamingo
unknown
5.95µs
1.38×
Reported
license unknown · no source
Stellar Flamingo
unknown
5.97µs
1.38×
Reported
license unknown · no source
Stellar Flamingo
unknown
5.97µs
1.38×
Reported
license unknown · no source
Geometric
unknown
5.92µs
1.37×
Reported
license unknown · no source
Geometric
unknown
6.21µs
1.44×
Reported
license unknown · no source
Stellar Flamingo
unknown
6.02µs
1.39×
Reported
license unknown · no source
Jayce Su
unknown
6.39µs
1.48×
Reported
license unknown · no source
Jayce Su
unknown
6.42µs
1.48×
Reported
license unknown · no source
Jayce Su
unknown
6.48µs
1.50×
Reported
license unknown · no source
Jayce Su
unknown
6.53µs
1.51×
Reported
license unknown · no source
Jayce Su
unknown
6.57µs
1.52×
Reported
license unknown · no source
Jayce Su
unknown
6.60µs
1.53×
Reported
license unknown · no source
Jayce Su
unknown
6.75µs
1.56×
Reported
license unknown · no source
Jayce Su
unknown
6.37µs
1.47×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
6.43µs
1.49×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
6.44µs
1.49×
Reported
license unknown · no source
Neon Panda
unknown
6.55µs
1.51×
Reported
license unknown · no source
Cursor
unknown
9.48µs
2.19×
Reported
license unknown · no source
claude-opus-4-1 / cuda8eba35claude-opus-4-1-20250805
cuda
7.11µs
1.64×
Reproduction-ready
Apache-2.0 · source
claude-opus-4-1 / tritonc9eea2claude-opus-4-1-20250805
triton
12.4µs
2.87×
Reported
Apache-2.0 · source
gemini-2.5-pro / cuda4597e0gemini-2.5-pro
cuda
—
—
Apache-2.0 · source
gemini-2.5-pro / tritonxtl8hxgemini-2.5-pro
triton
6.26µs
1.45×
Reported
Apache-2.0 · source
gpt-5 / cudab963ecgpt-5-2025-08-07
cuda
6.27µs
1.45×
Reproduction-ready
Apache-2.0 · source
gpt-5 / triton679e13gpt-5-2025-08-07
triton
13.3µs
3.08×
Reported
Apache-2.0 · source
gpt-o3 / cuda3def09gpt-o3
cuda
7.28µs
1.68×
Reproduction-ready
Apache-2.0 · source
gpt-o3 / triton2e18c2gpt-o3
triton
6.19µs
1.43×
Reported
Apache-2.0 · source
flashinfer / wrapper74a870FlashInfer-Bench baselines
python
7.67µs
1.77×
Reported
Apache-2.0 · source

Semantics

Inputs and outputs
hidden_statesbf16 [batch_size, hidden_size]
residualbf16 [batch_size, hidden_size]
weightbf16 [hidden_size]
outputbf16 [batch_size, hidden_size]
Axes and behavior
batch_sizevariable
hidden_sizeconstant = 2048
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias001_fused_add_rmsnorm_h2048sha25696c78d50a2e5…

Also indexed as fused-add-rmsnorm-h2048 · reviewed as the same computation; every definition's cohorts are shown here, ranked separately.

Sources: NVIDIA SOL-ExecBench (2026-08-24) · FlashInfer-Bench (2025-10-21) · Apache-2.0last observed 2026-08-24How records are decidedJSON