Decoder complete layer
Complete decoder transformer layer combining pre-norm RMSNorm, GQA self-attention with RoPE (16 query heads, 4 KV heads), residual connection, pre-norm RMSNorm for cross-attention, GQA cross-attention (16 query heads, 16 KV heads), residual connection, pre-norm RMSNorm for MLP, SiLU-gated MLP (8192 intermediate), and final residual. This represents the full decoder layer that repeats 18 times.
Reported evidence · last observed 2026-08-11. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 128 · hidden_size = 2048 · cross_head_dim = 128 · self_attn_q_dim = 2048 · cross_attn_q_dim = 2048 · self_attn_kv_dim = 512 · cross_attn_kv_dim = 2048 · cross_hidden_size = 1024 · intermediate_size = 8192 · num_attention_heads = 16 · num_key_value_heads = 4 · cross_num_attention_heads = 16 · cross_num_key_value_heads = 1616 cases
Not measured on H100 for this workload. Challenges →
1348.0µs64% SOL1.00×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
2358.5µs64% SOL1.03×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
3386.7µs61% SOL1.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
4396.3µs60% SOL1.14×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
5397.4µs60% SOL1.14×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
6398.1µs60% SOL1.14×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
7398.3µs60% SOL1.14×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
8408.8µs59% SOL1.17×Reported · license unknown · no source2026-08-04
Measured exactly what you asked. Reported by source; not independently reproduced.
9410.6µs59% SOL1.18×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
10412.2µs59% SOL1.18×Reported · license unknown · no source2026-07-19
Measured exactly what you asked. Reported by source; not independently reproduced.
11413.9µs59% SOL1.19×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
12428.9µs58% SOL1.23×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
13437.8µs57% SOL1.26×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
14442.7µs56% SOL1.27×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
15461.5µs56% SOL1.33×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
16462.4µs55% SOL1.33×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
17474.6µs53% SOL1.36×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
18479.6µs53% SOL1.38×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
19502.3µs52% SOL1.44×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
20506.9µs51% SOL1.46×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
21540.4µs50% SOL1.55×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
22556.6µs49% SOL1.60×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
23558.5µs49% SOL1.60×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
24563.5µs48% SOL1.62×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
25563.9µs48% SOL1.62×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
26587.9µs47% SOL1.69×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
27590.6µs47% SOL1.70×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
28645.4µs45% SOL1.85×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
29782.8µs38% SOL2.25×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
30835.6µs36% SOL2.40×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
31851.4µs35% SOL2.45×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
321.96ms17% SOL5.64×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.
333.67ms12% SOL10.53×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.