Skip to content
KernelIndex
Search⌘K
Sign in

KV shared attention with RoPE

27 eligible runs
attention

KV-Sharing Attention with RoPE for Gemma3n. Implements novel KV-sharing mechanism where later layers reuse key-value states from earlier non-shared layers. Supports both global and sliding window attention patterns with RoPE embeddings, query/key/value normalization, and attention logit soft-capping.

Fastest reported
126.8µsmean · 82% of SOL
HyraLicense unknown · unknown

Reported evidence · last observed 2026-07-15. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 256 · kv_out_dim = 256 · hidden_size = 2304 · qkv_out_dim = 2048 · num_attention_heads = 8 · num_key_value_heads = 1 · num_key_value_groups = 816 cases
seq_len
batch_size
dtype
12816bf1612832bf1612864bf161311bf162562bf162564bf1625616bf1625632bf1625664bf164492bf165122bf1699716bf1610241bf1610242bf16116332bf1620481bf16
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1126.8 µsHyra18NVIDIA B200 · env 2132.3 µsRecursive9

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 18 results · last observed 2026-08-21Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
Hyra
126.8µs82% SOL
1.00×
Reported · license unknown · no source
2026-07-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
135.8µs80% SOL
1.07×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Recursive
136.0µs80% SOL
1.07×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Geometric
143.6µs79% SOL
1.13×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
ac4k
144.2µs79% SOL
1.14×
Reported · license unknown · no source
2026-08-05

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Geometric
145.3µs79% SOL
1.15×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Amir M. Mir | SF Tensor
146.5µs79% SOL
1.16×
Reported · license unknown · no source
2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
ac4k
155.2µs77% SOL
1.22×
Reported · license unknown · no source
2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Amir M. Mir | SF Tensor
164.8µs76% SOL
1.30×
Reported · license unknown · no source
2026-07-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Geometric
164.9µs76% SOL
1.30×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
doubleAI
168.4µs75% SOL
1.33×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Proud Wren @ NVIDIA
174.3µs74% SOL
1.37×
Reported · license unknown · no source
2026-07-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Geometric
183.9µs73% SOL
1.45×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Amir M. Mir | SF Tensor
185.9µs73% SOL
1.47×
Reported · license unknown · no source
2026-07-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Geometric
195.0µs72% SOL
1.54×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
genproof.com
244.7µs65% SOL
1.93×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Amir M. Mir | SF Tensor
486.7µs47% SOL
3.84×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
Cursor
956.8µs30% SOL
7.54×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Amir M. Mir | SF Tensor
unknown
135.8µs
1.07×
Reported
license unknown · no source
ac4k
unknown
144.2µs
1.14×
Reported
license unknown · no source
Geometric
unknown
143.6µs
1.13×
Reported
license unknown · no source
doubleAI
unknown
147.2µs
1.16×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
174.3µs
1.37×
Reported
license unknown · no source
genproof.com
unknown
235.8µs
1.86×
Reported
license unknown · no source
Cursor
unknown
836.7µs
6.60×
Reported
license unknown · no source
Geometric
unknown
145.3µs
1.15×
Reported
license unknown · no source
Show all 27 implementations ›
Amir M. Mir | SF Tensor
unknown
146.5µs
1.16×
Reported
license unknown · no source
ac4k
unknown
155.2µs
1.22×
Reported
license unknown · no source
Geometric
unknown
164.9µs
1.30×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
164.8µs
1.30×
Reported
license unknown · no source
doubleAI
unknown
168.4µs
1.33×
Reported
license unknown · no source
Geometric
unknown
183.9µs
1.45×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
185.9µs
1.47×
Reported
license unknown · no source
Geometric
unknown
195.0µs
1.54×
Reported
license unknown · no source
genproof.com
unknown
244.7µs
1.93×
Reported
license unknown · no source
doubleAI
unknown
287.0µs
2.26×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
460.5µs
3.63×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
475.9µs
3.75×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
486.7µs
3.84×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
727.3µs
5.73×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
871.2µs
6.87×
Reported
license unknown · no source
Cursor
unknown
956.8µs
7.54×
Reported
license unknown · no source
Hyra
unknown
126.8µs
1.00×
Reported
license unknown · no source
Recursive
unknown
132.3µs
1.04×
Reported
license unknown · no source
Recursive
unknown
136.0µs
1.07×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesbf16 [batch_size, seq_len, hidden_size]
position_idsint64 [batch_size, seq_len]
attention_maskbf16 [batch_size, 1, seq_len, seq_len]
q_proj_weightbf16 [qkv_out_dim, hidden_size]
k_proj_weightbf16 [kv_out_dim, hidden_size]
v_proj_weightbf16 [kv_out_dim, hidden_size]
o_proj_weightbf16 [hidden_size, qkv_out_dim]
q_norm_weightbf16 [head_dim]
k_norm_weightbf16 [head_dim]
rope_thetafp32 scalar
softcapfp32 scalar
rms_norm_epsfp32 scalar
attn_outputbf16 [batch_size, seq_len, hidden_size]
key_states_outbf16 [batch_size, num_key_value_heads, seq_len, head_dim]
value_states_outbf16 [batch_size, num_key_value_heads, seq_len, head_dim]
Axes and behavior
seq_lenvariable
head_dimconstant = 256
batch_sizevariable
kv_out_dimkv_out_dim = num_key_value_heads * head_dim
hidden_sizeconstant = 2304
qkv_out_dimqkv_out_dim = num_attention_heads * head_dim
num_attention_headsconstant = 8
num_key_value_headsconstant = 1
num_key_value_groupsconstant = 8
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias039_kv_shared_attention_with_ropemodelgemma-3n-e2bsha2566f4c31a2b644…
Sources: NVIDIA SOL-ExecBench (2026-08-21)last observed 2026-08-21How records are decidedJSON