Attention output reshape and projection
Fused attention output reshape and projection. Takes multi-head attention output from [batch, num_heads, seq_len, v_head_dim] format, transposes to [batch, seq_len, num_heads, v_head_dim], reshapes to [batch, seq_len, num_heads * v_head_dim] (requires contiguous copy), then projects to [batch, seq_len, hidden_size] via output projection. This is a critical memory bandwidth bottleneck due to non-contiguous transpose, memory reallocation for contiguous, and large intermediate dimension (16384) before projection to hidden_size (7168).
Reported evidence · last observed 2026-08-18. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencynum_heads = 128 · v_head_dim = 128 · hidden_size = 7168 · intermediate_size = 1638416 cases
Not measured on H100 for this workload. Challenges →
1235.6µs69% SOL1.00×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
2245.9µs67% SOL1.04×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
3246.2µs66% SOL1.05×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
4250.4µs66% SOL1.06×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
5250.9µs65% SOL1.07×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
6251.4µs65% SOL1.07×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
7255.5µs63% SOL1.08×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
8256.6µs63% SOL1.09×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
9259.3µs63% SOL1.10×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
10269.4µs60% SOL1.14×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
11274.6µs59% SOL1.17×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
12277.7µs58% SOL1.18×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
13279.3µs57% SOL1.19×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
14282.7µs57% SOL1.20×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
15283.7µs56% SOL1.20×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
16300.1µs54% SOL1.27×Reported · license unknown · no source2026-07-25
Measured exactly what you asked. Reported by source; not independently reproduced.
17304.3µs55% SOL1.29×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
18312.3µs53% SOL1.33×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
19553.4µs49% SOL2.35×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.