Skip to content
KernelIndex
Search⌘K
Sign in

Grouped query self attention with RoPE

43 eligible runs
rope

Grouped Query Self-Attention with integrated RoPE position embeddings. Supports both encoder (non-causal, 16 heads, 4 KV heads) and decoder (causal, 24 heads, 8 KV heads) configurations. Fuses Q/K/V projections, RoPE application, GQA computation, and output projection.

Fastest reported
73.1µsmean · 79% of SOL
TH3CHARLie | TileGymLicense unknown · unknown

Reported evidence · last observed 2026-07-27. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 64 · num_heads = 16 · hidden_size = 1024 · q_proj_size = 1024 · kv_proj_size = 256 · num_key_value_heads = 4 · num_key_value_groups = 416 cases
seq_len
batch_size
dtype
1284fp321311fp322112fp322561fp322564fp322568fp323738fp3244916fp325121fp325124fp325128fp3251216fp325411fp3210242fp3220481fp3240961fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 173.1 µsTH3CHARLie | TileGym31NVIDIA B200 · env 283.9 µsDatabricks12

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 31 results · last observed 2026-08-23Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
TH3CHARLie | TileGym
73.1µs79% SOL
1.00×
Reported · license unknown · no source
2026-07-27

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
85.4µs76% SOL
1.17×
Reported · license unknown · no source
2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Geometric
90.3µs75% SOL
1.24×
Reported · license unknown · no source
2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Geometric
91.0µs75% SOL
1.24×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Amir M. Mir | SF Tensor
97.8µs74% SOL
1.34×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Geometric
98.1µs74% SOL
1.34×
Reported · license unknown · no source
2026-08-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Geometric
98.7µs73% SOL
1.35×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Geometric
99.0µs73% SOL
1.35×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Geometric
106.1µs72% SOL
1.45×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
ac4k
107.3µs72% SOL
1.47×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
ac4k
110.2µs71% SOL
1.51×
Reported · license unknown · no source
2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Databricks
111.4µs70% SOL
1.52×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Recursive
111.8µs71% SOL
1.53×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Electric Cheetah
116.4µs70% SOL
1.59×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Rapid Tiger
116.4µs70% SOL
1.59×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
Gleaming Coyote
116.5µs70% SOL
1.59×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
ac4k
117.0µs70% SOL
1.60×
Reported · license unknown · no source
2026-07-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
Hyra
123.1µs69% SOL
1.68×
Reported · license unknown · no source
2026-07-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
19
Geometric
124.0µs68% SOL
1.70×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
20
Geometric
124.4µs68% SOL
1.70×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
21
Geometric
129.2µs68% SOL
1.77×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
22
Geometric
143.1µs65% SOL
1.96×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
23
Amir M. Mir | SF Tensor
167.8µs61% SOL
2.29×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
24
Geometric
204.6µs56% SOL
2.80×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
25
Geometric
229.0µs53% SOL
3.13×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
26
ac4k
267.3µs50% SOL
3.65×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
27
doubleAI
485.4µs37% SOL
6.64×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
28
Zerg AI
491.5µs35% SOL
6.72×
Reported · license unknown · no source
2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
29
Zerg AI
559.5µs32% SOL
7.65×
Reported · license unknown · no source
2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
30
Zerg AI
567.7µs31% SOL
7.76×
Reported · license unknown · no source
2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
31
Cursor
605.3µs30% SOL
8.28×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Gleaming Coyote
unknown
116.5µs
1.59×
Reported
license unknown · no source
Hyra
unknown
123.1µs
1.68×
Reported
license unknown · no source
Zerg AI
unknown
236.1µs
3.23×
Reported
license unknown · no source
Argus
unknown
276.2µs
3.78×
Reported
license unknown · no source
Cursor
unknown
504.7µs
6.90×
Reported
license unknown · no source
Geometric
unknown
91.0µs
1.24×
Reported
license unknown · no source
Zerg AI
unknown
559.5µs
7.65×
Reported
license unknown · no source
Databricks
unknown
83.9µs
1.15×
Reported
license unknown · no source
Show all 43 implementations ›
Geometric
unknown
90.3µs
1.24×
Reported
license unknown · no source
Recursive
unknown
100.7µs
1.38×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
73.1µs
1.00×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
85.4µs
1.17×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
97.8µs
1.34×
Reported
license unknown · no source
ac4k
unknown
100.4µs
1.37×
Reported
license unknown · no source
doubleAI
unknown
105.0µs
1.44×
Reported
license unknown · no source
Electric Cheetah
unknown
116.4µs
1.59×
Reported
license unknown · no source
Zerg AI
unknown
567.7µs
7.76×
Reported
license unknown · no source
Cursor
unknown
605.3µs
8.28×
Reported
license unknown · no source
Rapid Tiger
unknown
116.4µs
1.59×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
93.2µs
1.27×
Reported
license unknown · no source
Geometric
unknown
98.1µs
1.34×
Reported
license unknown · no source
Geometric
unknown
98.7µs
1.35×
Reported
license unknown · no source
Geometric
unknown
99.0µs
1.35×
Reported
license unknown · no source
Geometric
unknown
106.1µs
1.45×
Reported
license unknown · no source
ac4k
unknown
107.3µs
1.47×
Reported
license unknown · no source
ac4k
unknown
110.2µs
1.51×
Reported
license unknown · no source
Recursive
unknown
111.8µs
1.53×
Reported
license unknown · no source
Databricks
unknown
111.4µs
1.52×
Reported
license unknown · no source
ac4k
unknown
117.0µs
1.60×
Reported
license unknown · no source
Geometric
unknown
124.0µs
1.70×
Reported
license unknown · no source
Geometric
unknown
124.4µs
1.70×
Reported
license unknown · no source
Geometric
unknown
129.2µs
1.77×
Reported
license unknown · no source
Geometric
unknown
143.1µs
1.96×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
163.4µs
2.23×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
165.3µs
2.26×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
167.8µs
2.29×
Reported
license unknown · no source
doubleAI
unknown
175.0µs
2.39×
Reported
license unknown · no source
Geometric
unknown
204.6µs
2.80×
Reported
license unknown · no source
Geometric
unknown
229.0µs
3.13×
Reported
license unknown · no source
ac4k
unknown
267.3µs
3.65×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
419.9µs
5.74×
Reported
license unknown · no source
doubleAI
unknown
485.4µs
6.64×
Reported
license unknown · no source
Zerg AI
unknown
491.5µs
6.72×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesfp32 [batch_size, seq_len, hidden_size]
position_idsint64 [batch_size, seq_len]
q_proj_weightfp32 [q_proj_size, hidden_size]
k_proj_weightfp32 [kv_proj_size, hidden_size]
v_proj_weightfp32 [kv_proj_size, hidden_size]
o_proj_weightfp32 [hidden_size, q_proj_size]
inv_freqfp32 [32]
is_causalbool scalar
attn_outputfp32 [batch_size, seq_len, hidden_size]
attn_weightsfp32 [batch_size, num_heads, seq_len, seq_len]
Axes and behavior
seq_lenvariable
head_dimconstant = 64
num_headsconstant = 16
batch_sizevariable
hidden_sizeconstant = 1024
q_proj_sizeq_proj_size = num_heads * head_dim
kv_proj_sizekv_proj_size = num_key_value_heads * head_dim
num_key_value_headsconstant = 4
num_key_value_groupsnum_key_value_groups = num_heads // num_key_value_heads
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias075_grouped_query_self_attention_with_ropemodelparakeet-tdt-0-6b-v3sha2563aee7f4def75…
Sources: NVIDIA SOL-ExecBench (2026-08-23)last observed 2026-08-23How records are decidedJSON