Skip to content
KernelIndex
Search⌘K
Sign in

Grouped query attention with RoPE backward

32 eligible runs
rope

Backward pass for Grouped Query Attention with Rotary Position Embeddings. Computes gradients through output projection, attention computation, RoPE, and Q/K/V projections.

Fastest reported
316.3µsmean · 70% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-15. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 128 · hidden_size = 5120 · q_proj_size = 4096 · kv_proj_size = 1024 · num_attention_heads = 32 · num_key_value_heads = 816 cases
seq_len
batch_size
dtype
1281bf161282bf1612816bf161311bf162561bf1625616bf162931bf163848bf164492bf165121bf165124bf166911bf1610241bf1610242bf1610244bf1620481bf16
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1316.3 µsRIAC as well19NVIDIA B200 · env 2328.9 µsdoubleAI13

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 19 results · last observed 2026-08-22Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
316.3µs70% SOL
1.00×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
343.8µs67% SOL
1.09×
Reported · license unknown · no source
2026-07-26

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
doubleAI
348.3µs67% SOL
1.10×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Geometric
359.8µs66% SOL
1.14×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Geometric
362.1µs66% SOL
1.14×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
ac4k
363.8µs65% SOL
1.15×
Reported · license unknown · no source
2026-07-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Geometric
365.0µs65% SOL
1.15×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Recursive
371.0µs65% SOL
1.17×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Geometric
385.4µs64% SOL
1.22×
Reported · license unknown · no source
2026-08-07

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Hyra
397.9µs63% SOL
1.26×
Reported · license unknown · no source
2026-07-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Amir M. Mir | SF Tensor
421.1µs61% SOL
1.33×
Reported · license unknown · no source
2026-07-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Proud Wren @ NVIDIA
421.2µs62% SOL
1.33×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Geometric
450.9µs59% SOL
1.43×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Q H
470.5µs57% SOL
1.49×
Reported · license unknown · no source
2026-08-08

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
bismuth83
472.1µs57% SOL
1.49×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
Gilded Quail
486.0µs55% SOL
1.54×
Reported · license unknown · no source
2026-07-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Amir M. Mir | SF Tensor
496.9µs56% SOL
1.57×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
Geometric
543.9µs53% SOL
1.72×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
19
Cursor
631.7µs47% SOL
2.00×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
RIAC as well
unknown
316.3µs
1.00×
Reported
license unknown · no source
Geometric
unknown
359.8µs
1.14×
Reported
license unknown · no source
Recursive
unknown
362.3µs
1.15×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
375.2µs
1.19×
Reported
license unknown · no source
ac4k
unknown
363.8µs
1.15×
Reported
license unknown · no source
Hyra
unknown
397.9µs
1.26×
Reported
license unknown · no source
bismuth83
unknown
417.2µs
1.32×
Reported
license unknown · no source
Q H
unknown
470.5µs
1.49×
Reported
license unknown · no source
Show all 32 implementations ›
Gilded Quail
unknown
492.3µs
1.56×
Reported
license unknown · no source
AKO
unknown
521.6µs
1.65×
Reported
license unknown · no source
Cursor
unknown
551.7µs
1.74×
Reported
license unknown · no source
doubleAI
unknown
357.5µs
1.13×
Reported
license unknown · no source
Geometric
unknown
362.1µs
1.14×
Reported
license unknown · no source
Geometric
unknown
365.0µs
1.15×
Reported
license unknown · no source
Recursive
unknown
371.0µs
1.17×
Reported
license unknown · no source
Geometric
unknown
385.4µs
1.22×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
421.2µs
1.33×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
421.1µs
1.33×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
428.5µs
1.35×
Reported
license unknown · no source
Geometric
unknown
450.9µs
1.43×
Reported
license unknown · no source
bismuth83
unknown
472.1µs
1.49×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
496.9µs
1.57×
Reported
license unknown · no source
Gilded Quail
unknown
486.0µs
1.54×
Reported
license unknown · no source
Gilded Quail
unknown
494.3µs
1.56×
Reported
license unknown · no source
Gilded Quail
unknown
499.0µs
1.58×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
510.3µs
1.61×
Reported
license unknown · no source
Geometric
unknown
543.9µs
1.72×
Reported
license unknown · no source
AKO
unknown
530.1µs
1.68×
Reported
license unknown · no source
Cursor
unknown
631.7µs
2.00×
Reported
license unknown · no source
doubleAI
unknown
328.9µs
1.04×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
343.8µs
1.09×
Reported
license unknown · no source
doubleAI
unknown
348.3µs
1.10×
Reported
license unknown · no source

Semantics

Inputs and outputs
grad_outputbf16 [batch_size, seq_len, hidden_size]
hidden_statesbf16 [batch_size, seq_len, hidden_size]
cosbf16 [batch_size, seq_len, head_dim]
sinbf16 [batch_size, seq_len, head_dim]
q_weightbf16 [q_proj_size, hidden_size]
k_weightbf16 [kv_proj_size, hidden_size]
v_weightbf16 [kv_proj_size, hidden_size]
o_weightbf16 [hidden_size, q_proj_size]
query_statesbf16 [batch_size, num_attention_heads, seq_len, head_dim]
key_statesbf16 [batch_size, num_attention_heads, seq_len, head_dim]
value_statesbf16 [batch_size, num_attention_heads, seq_len, head_dim]
attn_weightsbf16 [batch_size, num_attention_heads, seq_len, seq_len]
attn_outputbf16 [batch_size, seq_len, q_proj_size]
scalingfp32 scalar
grad_hidden_statesbf16 [batch_size, seq_len, hidden_size]
grad_q_weightbf16 [q_proj_size, hidden_size]
grad_k_weightbf16 [kv_proj_size, hidden_size]
grad_v_weightbf16 [kv_proj_size, hidden_size]
grad_o_weightbf16 [hidden_size, q_proj_size]
Axes and behavior
seq_lenvariable
head_dimconstant = 128
batch_sizevariable
hidden_sizeconstant = 5120
q_proj_sizeq_proj_size = num_attention_heads * head_dim
kv_proj_sizekv_proj_size = num_key_value_heads * head_dim
num_attention_headsconstant = 32
num_key_value_headsconstant = 8
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias003_grouped_query_attention_with_rope_backwardmodeldeephermes-3-mistral-24b-previewsha2562afa1d8c8129…
Sources: NVIDIA SOL-ExecBench (2026-08-22)last observed 2026-08-22How records are decidedJSON