Skip to content
KernelIndex
Search⌘K
Sign in

Grouped query attention with YaRN RoPE and QK norm

26 eligible runs
rmsnorm

Grouped Query Attention with YARN RoPE scaling and Q/K normalization. 40 query heads, 8 key-value heads (5:1 GQA ratio), head dimension 128, YARN RoPE with 8x scaling for 65536 max positions, sliding window attention 4096 tokens, RMS normalization on Q/K before attention.

Fastest reported
298.5µsmean · 57% of SOL
RecursiveLicense unknown · unknown

Reported evidence · last observed 2026-07-10. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 128 · hidden_size = 5120 · half_head_dim = 64 · kv_hidden_size = 1024 · num_attention_heads = 40 · num_key_value_heads = 8 · num_key_value_groups = 516 cases
seq_len
batch_size
dtype
1284bf16/fp3212832bf16/fp3212864bf16/fp321311bf16/fp321318bf16/fp322568bf16/fp3225616bf16/fp322931bf16/fp325412bf16/fp327734bf16/fp329971bf16/fp3210242bf16/fp3215712bf16/fp3220481bf16/fp3220482bf16/fp3221314bf16/fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1298.5 µsRecursive15NVIDIA B200 · env 2292.1 µsdoubleAI11

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 15 results · last observed 2026-08-19Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
Recursive
298.5µs57% SOL
1.00×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
301.3µs56% SOL
1.01×
Reported · license unknown · no source
2026-07-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
ac4k
306.4µs56% SOL
1.03×
Reported · license unknown · no source
2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Geometric
306.8µs56% SOL
1.03×
Reported · license unknown · no source
2026-08-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Proud Wren @ NVIDIA
309.9µs55% SOL
1.04×
Reported · license unknown · no source
2026-07-30

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
doubleAI
310.8µs55% SOL
1.04×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Geometric
319.6µs54% SOL
1.07×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Proud Wren @ NVIDIA
326.4µs53% SOL
1.09×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Hyra
327.9µs53% SOL
1.10×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Geometric
331.9µs52% SOL
1.11×
Reported · license unknown · no source
2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Cursor
348.7µs50% SOL
1.17×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
genproof.com
356.1µs49% SOL
1.19×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Amir M. Mir | SF Tensor
370.6µs47% SOL
1.24×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Geometric
573.0µs32% SOL
1.92×
Reported · license unknown · no source
2026-07-27

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Geometric
758.2µs24% SOL
2.54×
Reported · license unknown · no source
2026-07-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Amir M. Mir | SF Tensor
unknown
301.3µs
1.03×
Reported
license unknown · no source
Geometric
unknown
306.8µs
1.05×
Reported
license unknown · no source
ac4k
unknown
306.4µs
1.05×
Reported
license unknown · no source
Cursor
unknown
318.1µs
1.09×
Reported
license unknown · no source
Hyra
unknown
327.9µs
1.12×
Reported
license unknown · no source
genproof.com
unknown
335.5µs
1.15×
Reported
license unknown · no source
gundam00
unknown
667.2µs
2.28×
Reported
license unknown · no source
bismuth83
unknown
669.0µs
2.29×
Reported
license unknown · no source
Show all 26 implementations ›
Recursive
unknown
298.5µs
1.02×
Reported
license unknown · no source
doubleAI
unknown
310.8µs
1.06×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
309.9µs
1.06×
Reported
license unknown · no source
Geometric
unknown
319.6µs
1.09×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
326.4µs
1.12×
Reported
license unknown · no source
Geometric
unknown
331.9µs
1.14×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
350.1µs
1.20×
Reported
license unknown · no source
Cursor
unknown
348.7µs
1.19×
Reported
license unknown · no source
genproof.com
unknown
356.1µs
1.22×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
370.6µs
1.27×
Reported
license unknown · no source
doubleAI
unknown
379.4µs
1.30×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
465.6µs
1.59×
Reported
license unknown · no source
Geometric
unknown
573.0µs
1.96×
Reported
license unknown · no source
gundam00
unknown
777.3µs
2.66×
Reported
license unknown · no source
Geometric
unknown
758.2µs
2.60×
Reported
license unknown · no source
doubleAI
unknown
292.1µs
1.00×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
294.0µs
1.01×
Reported
license unknown · no source
Recursive
unknown
293.1µs
1.00×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesbf16 [batch_size, seq_len, hidden_size]
position_idsint64 [batch_size, seq_len]
attention_maskbf16 [batch_size, 1, seq_len, seq_len]
q_proj_weightbf16 [hidden_size, hidden_size]
k_proj_weightbf16 [kv_hidden_size, hidden_size]
v_proj_weightbf16 [kv_hidden_size, hidden_size]
o_proj_weightbf16 [hidden_size, hidden_size]
q_norm_weightbf16 [hidden_size]
k_norm_weightbf16 [kv_hidden_size]
inv_freqfp32 [half_head_dim]
rms_norm_epsfp32 scalar
attention_factorfp32 scalar
scalingfp32 scalar
outputbf16 [batch_size, seq_len, hidden_size]
Axes and behavior
seq_lenvariable
head_dimconstant = 128
batch_sizevariable
hidden_sizeconstant = 5120
half_head_dimhalf_head_dim = head_dim // 2
kv_hidden_sizekv_hidden_size = num_key_value_heads * head_dim
num_attention_headsconstant = 40
num_key_value_headsconstant = 8
num_key_value_groupsnum_key_value_groups = num_attention_heads // num_key_value_heads
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias027_grouped_query_attention_with_yarn_rope_and_qk_normmodelolmo-3-32b-thinksha256db0f2f926778…
Sources: NVIDIA SOL-ExecBench (2026-08-19)last observed 2026-08-19How records are decidedJSON