Attention QKV projection with GQA reshape
Fused QKV projection with Grouped Query Attention (GQA) reshaping. Performs three linear projections (Q: 2048->2048, K: 2048->512, V: 2048->512) followed by reshape and transpose to prepare tensors for attention computation. Uses GQA pattern with 16 query heads and 4 key-value heads (4x compression).
Reported evidence · last observed 2026-08-16. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyq_size = 2048 · kv_size = 512 · head_dim = 128 · num_heads = 16 · hidden_size = 2048 · num_kv_heads = 416 cases
Not measured on H100 for this workload. Challenges →
125.5µs68% SOL1.00×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
225.8µs68% SOL1.01×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
327.6µs66% SOL1.08×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
427.9µs65% SOL1.09×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
529.4µs63% SOL1.15×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
629.9µs62% SOL1.17×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
730.0µs64% SOL1.18×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
830.0µs64% SOL1.18×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
930.3µs64% SOL1.19×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
1030.5µs62% SOL1.20×Reported · license unknown · no source2026-08-24
Measured exactly what you asked. Reported by source; not independently reproduced.
1130.8µs61% SOL1.21×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
1230.9µs61% SOL1.21×Reported · license unknown · no source2026-08-24
Measured exactly what you asked. Reported by source; not independently reproduced.
1330.9µs62% SOL1.21×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1431.3µs61% SOL1.23×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
1531.4µs60% SOL1.23×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
1632.3µs60% SOL1.27×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1733.4µs58% SOL1.31×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
1833.4µs58% SOL1.31×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
1934.1µs57% SOL1.34×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
2034.2µs57% SOL1.34×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
2134.9µs56% SOL1.37×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
2234.9µs57% SOL1.37×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2336.6µs54% SOL1.44×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
2436.7µs54% SOL1.44×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
2536.7µs54% SOL1.44×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
2637.4µs53% SOL1.47×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
2745.2µs48% SOL1.77×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2851.6µs44% SOL2.03×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.