Attention QKV projection with bias and reshape
Fused QKV projection with bias and reshape for Gemma3 attention. Performs three separate linear projections (Q, K, V) with bias addition, then reshapes outputs from (batch, seq_len, hidden_size) to (batch, seq_len, num_heads, head_dim) format for multi-head attention.
Reported evidence · last observed 2026-07-12. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 256 · q_out_dim = 1024 · kv_out_dim = 256 · hidden_size = 640 · num_attention_heads = 4 · num_key_value_heads = 116 cases
Not measured on H100 for this workload. Challenges →
19.07µs71% SOL1.00×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
210.2µs68% SOL1.13×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
310.4µs67% SOL1.15×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
410.8µs66% SOL1.19×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
511.6µs64% SOL1.28×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
612.5µs62% SOL1.38×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
712.5µs62% SOL1.38×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
812.6µs61% SOL1.39×Reported · license unknown · no source2026-08-24
Measured exactly what you asked. Reported by source; not independently reproduced.
912.6µs62% SOL1.39×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
1012.6µs61% SOL1.39×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
1113.1µs60% SOL1.44×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
1213.2µs60% SOL1.45×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
1313.2µs60% SOL1.45×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
1413.4µs59% SOL1.47×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
1513.5µs59% SOL1.49×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
1613.7µs59% SOL1.51×Reported · license unknown · no source2026-07-18
Measured exactly what you asked. Reported by source; not independently reproduced.
1714.0µs59% SOL1.54×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
1814.1µs59% SOL1.56×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1914.3µs58% SOL1.57×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2014.7µs57% SOL1.62×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
2114.8µs57% SOL1.63×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
2214.8µs57% SOL1.63×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
2314.9µs56% SOL1.64×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
2414.9µs57% SOL1.64×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2515.1µs57% SOL1.67×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
2615.1µs57% SOL1.67×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
2719.2µs50% SOL2.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2821.2µs47% SOL2.34×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
2932.0µs36% SOL3.53×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
3036.1µs33% SOL3.98×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
3139.8µs31% SOL4.39×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.