Attention QK matmul with GQA repeat and scaling
Fused attention Q@K^T computation with GQA key repetition and scaling. Takes query (batch, num_heads=4, seq_len, head_dim=256) and key (batch, num_kv_heads=1, seq_len, head_dim=256), repeats KV heads 4x to match query heads, computes Q@K^T, and applies scaling (1/sqrt(256)=0.0625).
Reported evidence · last observed 2026-06-07. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 256 · num_attention_heads = 4 · num_key_value_heads = 1 · num_key_value_groups = 416 cases
Not measured on H100 for this workload. Challenges →
111.3µs59% SOL1.00×Reported · license unknown · no source2026-06-07
Measured exactly what you asked. Reported by source; not independently reproduced.
211.7µs58% SOL1.03×Reported · license unknown · no source2026-07-01
Measured exactly what you asked. Reported by source; not independently reproduced.
312.1µs57% SOL1.07×Reported · license unknown · no source2026-06-28
Measured exactly what you asked. Reported by source; not independently reproduced.
412.6µs56% SOL1.11×Reported · license unknown · no source2026-07-05
Measured exactly what you asked. Reported by source; not independently reproduced.
512.9µs56% SOL1.14×Reported · license unknown · no source2026-06-30
Measured exactly what you asked. Reported by source; not independently reproduced.
613.6µs54% SOL1.20×Reported · license unknown · no source2026-07-08
Measured exactly what you asked. Reported by source; not independently reproduced.
714.4µs53% SOL1.27×Reported · license unknown · no source2026-05-20
Measured exactly what you asked. Reported by source; not independently reproduced.
814.7µs52% SOL1.29×Reported · license unknown · no source2026-04-14
Measured exactly what you asked. Reported by source; not independently reproduced.
915.8µs51% SOL1.40×Reported · license unknown · no source2026-07-05
Measured exactly what you asked. Reported by source; not independently reproduced.
1015.9µs50% SOL1.41×Reported · license unknown · no source2026-05-17
Measured exactly what you asked. Reported by source; not independently reproduced.
1129.5µs34% SOL2.61×Reported · license unknown · no source2026-04-22
Measured exactly what you asked. Reported by source; not independently reproduced.
1233.4µs31% SOL2.95×Reported · license unknown · no source2026-04-21
Measured exactly what you asked. Reported by source; not independently reproduced.
1342.9µs25% SOL3.79×Reported · license unknown · no source2026-05-28
Measured exactly what you asked. Reported by source; not independently reproduced.