Skip to content
KernelIndex
Search⌘K
Sign in

GQA attention with QK norm

30 eligible runs
rmsnorm

Grouped Query Attention with QK normalization for GLM-4. Includes Q/K/V projections, RMSNorm on Q/K, RoPE, GQA with 96 query heads and 8 KV heads (12x repetition), scaled dot-product attention with causal mask, and output projection.

Fastest reported
403.5µsmean · 58% of SOL
RecursiveLicense unknown · unknown

Reported evidence · last observed 2026-07-10. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 128 · hidden_size = 4096 · q_out_features = 12288 · kv_out_features = 1024 · num_attention_heads = 96 · num_key_value_heads = 816 cases
seq_len
batch_size
dtype
1281bf161284bf1612816bf162562bf1625616bf1625632bf162934bf163738bf165121bf165122bf165124bf165128bf1610241bf1610244bf1620481bf1620482bf16
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1403.5 µsRecursive18NVIDIA B200 · env 2379.7 µsRecursive12

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 18 results · last observed 2026-08-20Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
Recursive
403.5µs58% SOL
1.00×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
doubleAI
408.1µs58% SOL
1.01×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Databricks
412.2µs56% SOL
1.02×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Amir M. Mir | SF Tensor
412.4µs57% SOL
1.02×
Reported · license unknown · no source
2026-08-07

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
ac4k
419.8µs56% SOL
1.04×
Reported · license unknown · no source
2026-07-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Geometric
425.3µs55% SOL
1.05×
Reported · license unknown · no source
2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Hyra
429.6µs55% SOL
1.06×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Amir M. Mir | SF Tensor
436.0µs54% SOL
1.08×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Geometric
443.3µs53% SOL
1.10×
Reported · license unknown · no source
2026-08-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
ac4k
443.4µs53% SOL
1.10×
Reported · license unknown · no source
2026-07-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Geometric
444.9µs52% SOL
1.10×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Geometric
459.5µs51% SOL
1.14×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Geometric
462.4µs51% SOL
1.15×
Reported · license unknown · no source
2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Argus
465.8µs51% SOL
1.15×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Cursor
469.1µs50% SOL
1.16×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
ac4k
513.8µs46% SOL
1.27×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
TH3CHARLie | TileGym
515.1µs45% SOL
1.28×
Reported · license unknown · no source
2026-07-28

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
ac4k
834.3µs30% SOL
2.07×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Recursive
unknown
379.7µs
1.00×
Reported
license unknown · no source
doubleAI
unknown
386.7µs
1.02×
Reported
license unknown · no source
ac4k
unknown
385.4µs
1.02×
Reported
license unknown · no source
Argus
unknown
414.5µs
1.09×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
413.4µs
1.09×
Reported
license unknown · no source
Geometric
unknown
425.3µs
1.12×
Reported
license unknown · no source
Cursor
unknown
426.6µs
1.12×
Reported
license unknown · no source
Hyra
unknown
429.6µs
1.13×
Reported
license unknown · no source
Show all 30 implementations ›
Chrome Lion
unknown
440.9µs
1.16×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
515.1µs
1.36×
Reported
license unknown · no source
doubleAI
unknown
414.7µs
1.09×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
412.4µs
1.09×
Reported
license unknown · no source
Databricks
unknown
412.2µs
1.09×
Reported
license unknown · no source
ac4k
unknown
419.8µs
1.11×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
433.9µs
1.14×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
436.6µs
1.15×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
436.0µs
1.15×
Reported
license unknown · no source
ac4k
unknown
443.4µs
1.17×
Reported
license unknown · no source
Geometric
unknown
443.3µs
1.17×
Reported
license unknown · no source
Geometric
unknown
444.9µs
1.17×
Reported
license unknown · no source
Geometric
unknown
459.5µs
1.21×
Reported
license unknown · no source
Geometric
unknown
462.4µs
1.22×
Reported
license unknown · no source
Argus
unknown
465.8µs
1.23×
Reported
license unknown · no source
Cursor
unknown
469.1µs
1.24×
Reported
license unknown · no source
ac4k
unknown
513.8µs
1.35×
Reported
license unknown · no source
ac4k
unknown
834.3µs
2.20×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
1.02ms
2.68×
Reported
license unknown · no source
Databricks
unknown
388.8µs
1.02×
Reported
license unknown · no source
Recursive
unknown
403.5µs
1.06×
Reported
license unknown · no source
doubleAI
unknown
408.1µs
1.07×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesbf16 [batch_size, seq_len, hidden_size]
q_proj_weightbf16 [q_out_features, hidden_size]
q_proj_biasbf16 [q_out_features]
k_proj_weightbf16 [kv_out_features, hidden_size]
k_proj_biasbf16 [kv_out_features]
v_proj_weightbf16 [kv_out_features, hidden_size]
v_proj_biasbf16 [kv_out_features]
o_proj_weightbf16 [hidden_size, q_out_features]
q_norm_weightbf16 [head_dim]
k_norm_weightbf16 [head_dim]
cosbf16 [batch_size, seq_len, head_dim]
sinbf16 [batch_size, seq_len, head_dim]
rms_norm_epsfp32 scalar
outputbf16 [batch_size, seq_len, hidden_size]
Axes and behavior
seq_lenvariable
head_dimconstant = 128
batch_sizevariable
hidden_sizeconstant = 4096
q_out_featuresq_out_features = num_attention_heads * head_dim
kv_out_featureskv_out_features = num_key_value_heads * head_dim
num_attention_headsconstant = 96
num_key_value_headsconstant = 8
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias092_gqa_attention_with_qk_normmodelglm-4-5-airsha256e2d4154b5378…
Sources: NVIDIA SOL-ExecBench (2026-08-20)last observed 2026-08-20How records are decidedJSON