Decoder layer full block
Complete Qwen3 decoder layer block fusing: input RMSNorm -> QKV projection with QK norm -> RoPE -> GQA attention -> output projection -> residual -> post-attention RMSNorm -> SwiGLU MLP -> final residual. Hidden size 5120, 40 query heads, 8 KV heads, head dim 128, intermediate size 17408.
Reported evidence · last observed 2026-07-17. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 128 · hidden_size = 5120 · half_head_dim = 64 · intermediate_size = 17408 · num_attention_heads = 40 · num_key_value_heads = 8 · num_heads_x_head_dim = 5120 · num_kv_heads_x_head_dim = 102416 cases
Not measured on H100 for this workload. Challenges →
1877.8µs70% SOL1.00×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
2883.1µs69% SOL1.01×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
3907.8µs65% SOL1.03×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
4923.0µs64% SOL1.05×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
5925.0µs68% SOL1.05×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
6936.6µs65% SOL1.07×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
7946.9µs63% SOL1.08×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
8979.0µs63% SOL1.12×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
91.01ms59% SOL1.15×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
101.03ms58% SOL1.17×Reported · license unknown · no source2026-07-20
Measured exactly what you asked. Reported by source; not independently reproduced.
111.05ms57% SOL1.19×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
121.09ms55% SOL1.25×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
131.13ms52% SOL1.28×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
141.16ms53% SOL1.32×Reported · license unknown · no source2026-07-30
Measured exactly what you asked. Reported by source; not independently reproduced.
151.27ms47% SOL1.45×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
161.29ms46% SOL1.47×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
171.32ms45% SOL1.51×Reported · license unknown · no source2026-07-19
Measured exactly what you asked. Reported by source; not independently reproduced.
181.40ms43% SOL1.59×Reported · license unknown · no source2026-07-25
Measured exactly what you asked. Reported by source; not independently reproduced.
191.42ms42% SOL1.62×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
201.42ms41% SOL1.62×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
211.51ms39% SOL1.71×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.