Repeat KV attention matmul
Fused Grouped Query Attention (GQA) key-value repetition and attention score computation. This operation repeats key tensors from num_key_value_heads to num_attention_heads (4x expansion), then computes scaled attention scores via Q@K^T. The fusion avoids materializing the expanded key tensor, saving 75% of intermediate memory bandwidth.
Reported evidence · last observed 2026-07-10. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 128 · num_attention_heads = 32 · num_key_value_heads = 32 · num_key_value_groups = 116 cases
Not measured on H100 for this workload. Challenges →
125.0µs59% SOL1.00×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
226.7µs55% SOL1.07×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
327.0µs55% SOL1.08×Reported · license unknown · no source2026-07-31
Measured exactly what you asked. Reported by source; not independently reproduced.
427.5µs55% SOL1.10×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
527.6µs56% SOL1.11×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
628.7µs54% SOL1.15×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
728.7µs55% SOL1.15×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
828.8µs53% SOL1.15×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
929.1µs53% SOL1.16×Reported · license unknown · no source2026-07-20
Measured exactly what you asked. Reported by source; not independently reproduced.
1029.8µs52% SOL1.19×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
1131.3µs51% SOL1.25×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1233.0µs49% SOL1.32×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1338.2µs42% SOL1.53×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
1440.2µs44% SOL1.61×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1547.7µs42% SOL1.91×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1661.8µs30% SOL2.47×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
17333.1µs8% SOL13.33×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
18354.2µs7% SOL14.17×Reported · license unknown · no source2026-07-29
Measured exactly what you asked. Reported by source; not independently reproduced.