Skip to content
KernelIndex
Search⌘K
Sign in

Fused RoPE with QK norm and KV cache update

50 eligible runs
rmsnorm

Fused operation combining per-head RMS normalization on Q and K, RoPE (Rotary Position Embedding) application, and KV cache update. This eliminates 5+ separate kernel launches by keeping intermediate results in registers/shared memory.

Fastest reported
9.43µsmean · 91% of SOL
ac4kLicense unknown · unknown

Reported evidence · last observed 2026-07-10. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 13 cases · mean latencyhead_dim = 128 · half_head_dim = 64 · num_attention_heads = 96 · num_key_value_heads = 8 · max_position_embeddings = 26214413 cases
seq_len
cache_len
batch_size
dtype
101bf16/fp32120484bf16/fp3212801bf16/fp3212802bf16/fp3212808bf16/fp322561281bf16/fp3225610248bf16/fp322931311bf16/fp325122562bf16/fp3254101bf16/fp3210245121bf16/fp32204801bf16/fp32409620481bf16/fp32
scrolls · 13 cases total
Aggregatessuite of 13 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 19.43 µsac4k37NVIDIA B200 · env 29.38 µsac4k13

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 13 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 37 results · last observed 2026-08-20Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
ac4k
9.43µs91% SOL
1.00×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
doubleAI
9.54µs91% SOL
1.01×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
multiagent
9.87µs91% SOL
1.05×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
multiagent
9.95µs91% SOL
1.06×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Hyra
10.0µs90% SOL
1.06×
Reported · license unknown · no source
2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
multiagent
10.1µs91% SOL
1.07×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
multiagent
10.1µs91% SOL
1.07×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Geometric
10.2µs90% SOL
1.08×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
multiagent
10.2µs90% SOL
1.08×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Geometric
10.2µs90% SOL
1.08×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
multiagent
10.2µs90% SOL
1.09×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
xuanzhengdu
10.3µs90% SOL
1.09×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Recursive
10.3µs90% SOL
1.09×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
xuanzhengdu
10.3µs90% SOL
1.10×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
xuanzhengdu
10.4µs90% SOL
1.10×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
xuanzhengdu
10.5µs90% SOL
1.11×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
xuanzhengdu
10.5µs90% SOL
1.11×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
xuanzhengdu
10.5µs90% SOL
1.12×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
19
xuanzhengdu
10.6µs90% SOL
1.12×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
20
xuanzhengdu
10.6µs90% SOL
1.12×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
21
Amir M. Mir | SF Tensor
10.6µs89% SOL
1.12×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
22
Databricks
10.7µs89% SOL
1.14×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
23
xuanzhengdu
11.4µs88% SOL
1.21×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
24
xuanzhengdu
11.5µs88% SOL
1.22×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
25
Igor Bragaia
12.0µs87% SOL
1.27×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
26
xuanzhengdu
12.3µs87% SOL
1.30×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
27
bl-rr
14.0µs83% SOL
1.48×
Reported · license unknown · no source
2026-07-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
28
TH3CHARLie | TileGym
14.0µs83% SOL
1.49×
Reported · license unknown · no source
2026-07-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
29
xuanzhengdu
14.3µs83% SOL
1.52×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
30
Geometric
14.7µs82% SOL
1.56×
Reported · license unknown · no source
2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
31
Amir M. Mir | SF Tensor
16.0µs80% SOL
1.70×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
32
Geometric
16.3µs80% SOL
1.72×
Reported · license unknown · no source
2026-08-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
33
bl-rr
16.3µs79% SOL
1.73×
Reported · license unknown · no source
2026-07-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
34
Cursor
16.9µs81% SOL
1.79×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
35
bl-rr
17.1µs81% SOL
1.82×
Reported · license unknown · no source
2026-07-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
36
bl-rr
17.4µs78% SOL
1.85×
Reported · license unknown · no source
2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
37
TH3CHARLie | TileGym
30.3µs63% SOL
3.21×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
ac4k
unknown
9.38µs
1.00×
Reported
license unknown · no source
multiagent
unknown
9.87µs
1.05×
Reported
license unknown · no source
Recursive
unknown
10.1µs
1.07×
Reported
license unknown · no source
xuanzhengdu
unknown
10.3µs
1.09×
Reported
license unknown · no source
Hyra
unknown
10.0µs
1.07×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
10.6µs
1.13×
Reported
license unknown · no source
Databricks
unknown
10.5µs
1.12×
Reported
license unknown · no source
Igor Bragaia
unknown
12.0µs
1.27×
Reported
license unknown · no source
Show all 50 implementations ›
bl-rr
unknown
14.0µs
1.49×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
14.0µs
1.49×
Reported
license unknown · no source
Cursor
unknown
16.6µs
1.77×
Reported
license unknown · no source
Zerg AI
unknown
1.15ms
122.82×
Reported
license unknown · no source
multiagent
unknown
9.95µs
1.06×
Reported
license unknown · no source
doubleAI
unknown
9.59µs
1.02×
Reported
license unknown · no source
multiagent
unknown
10.1µs
1.07×
Reported
license unknown · no source
multiagent
unknown
10.1µs
1.07×
Reported
license unknown · no source
multiagent
unknown
10.2µs
1.09×
Reported
license unknown · no source
multiagent
unknown
10.2µs
1.09×
Reported
license unknown · no source
xuanzhengdu
unknown
10.3µs
1.10×
Reported
license unknown · no source
Geometric
unknown
10.2µs
1.09×
Reported
license unknown · no source
xuanzhengdu
unknown
10.4µs
1.11×
Reported
license unknown · no source
Recursive
unknown
10.3µs
1.10×
Reported
license unknown · no source
xuanzhengdu
unknown
10.5µs
1.12×
Reported
license unknown · no source
xuanzhengdu
unknown
10.5µs
1.12×
Reported
license unknown · no source
xuanzhengdu
unknown
10.5µs
1.12×
Reported
license unknown · no source
xuanzhengdu
unknown
10.6µs
1.13×
Reported
license unknown · no source
xuanzhengdu
unknown
10.6µs
1.13×
Reported
license unknown · no source
Databricks
unknown
10.7µs
1.14×
Reported
license unknown · no source
xuanzhengdu
unknown
11.4µs
1.22×
Reported
license unknown · no source
xuanzhengdu
unknown
11.5µs
1.22×
Reported
license unknown · no source
xuanzhengdu
unknown
12.3µs
1.31×
Reported
license unknown · no source
Igor Bragaia
unknown
12.1µs
1.29×
Reported
license unknown · no source
xuanzhengdu
unknown
14.3µs
1.53×
Reported
license unknown · no source
doubleAI
unknown
14.3µs
1.52×
Reported
license unknown · no source
Geometric
unknown
14.7µs
1.57×
Reported
license unknown · no source
Cursor
unknown
16.9µs
1.80×
Reported
license unknown · no source
bl-rr
unknown
17.1µs
1.83×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
15.9µs
1.69×
Reported
license unknown · no source
Geometric
unknown
16.3µs
1.73×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
16.0µs
1.71×
Reported
license unknown · no source
bl-rr
unknown
16.3µs
1.74×
Reported
license unknown · no source
bl-rr
unknown
17.4µs
1.86×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
22.0µs
2.35×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
30.3µs
3.23×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
122.5µs
13.06×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
143.7µs
15.32×
Reported
license unknown · no source
Igor Bragaia
unknown
276.4µs
29.47×
Reported
license unknown · no source
ac4k
unknown
9.43µs
1.01×
Reported
license unknown · no source
doubleAI
unknown
9.54µs
1.02×
Reported
license unknown · no source
Geometric
unknown
10.2µs
1.09×
Reported
license unknown · no source

Semantics

Inputs and outputs
querybf16 [batch_size, num_attention_heads, seq_len, head_dim]
keybf16 [batch_size, num_key_value_heads, seq_len, head_dim]
valuebf16 [batch_size, num_key_value_heads, seq_len, head_dim]
position_idsint64 [batch_size, seq_len]
key_cachebf16 [batch_size, num_key_value_heads, max_position_embeddings, head_dim]
value_cachebf16 [batch_size, num_key_value_heads, max_position_embeddings, head_dim]
cache_positionint64 [seq_len]
q_norm_weightbf16 [head_dim]
k_norm_weightbf16 [head_dim]
inv_freqfp32 [half_head_dim]
rms_norm_epsfp32 scalar
query_rotatedbf16 [batch_size, num_attention_heads, seq_len, head_dim]
key_rotatedbf16 [batch_size, num_key_value_heads, seq_len, head_dim]
key_cache_outbf16 [batch_size, num_key_value_heads, max_position_embeddings, head_dim]
value_cache_outbf16 [batch_size, num_key_value_heads, max_position_embeddings, head_dim]
Axes and behavior
seq_lenvariable
head_dimconstant = 128
cache_lenvariable
batch_sizevariable
half_head_dimconstant = 64
num_attention_headsconstant = 96
num_key_value_headsconstant = 8
max_position_embeddingsconstant = 262144
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias018_fused_rope_with_qk_norm_and_kv_cache_updatemodelqwen3-coder-480b-a35b-instructsha2563dfe4d15332e…
Sources: NVIDIA SOL-ExecBench (2026-08-20)last observed 2026-08-20How records are decidedJSON