Decoder layer with residual connections
Complete decoder layer with pre-normalization, self-attention (GQA with Q/K norm), MoE MLP, and residual connections. Includes RMSNorm before attention and MLP, rotary position embeddings, and sparse mixture of experts routing.
Reported evidence · last observed 2026-08-20. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 128 · hidden_size = 2048 · num_experts = 128 · q_out_features = 4096 · kv_out_features = 512 · num_attention_heads = 32 · num_experts_per_tok = 8 · num_key_value_heads = 4 · moe_intermediate_size = 76816 cases
Not measured on H100 for this workload. Challenges →
1959.2µs71% SOL1.00×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
21.01ms69% SOL1.05×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
31.02ms69% SOL1.06×Reported · license unknown · no source2026-08-04
Measured exactly what you asked. Reported by source; not independently reproduced.
41.03ms69% SOL1.08×Reported · license unknown · no source2026-08-01
Measured exactly what you asked. Reported by source; not independently reproduced.
51.07ms68% SOL1.11×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
61.07ms68% SOL1.11×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
71.08ms68% SOL1.12×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
81.16ms66% SOL1.21×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
91.18ms65% SOL1.23×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
101.18ms65% SOL1.23×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
111.21ms64% SOL1.27×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
121.25ms63% SOL1.31×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
131.31ms62% SOL1.37×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
141.57ms57% SOL1.64×Reported · license unknown · no source2026-07-19
Measured exactly what you asked. Reported by source; not independently reproduced.
152.58ms44% SOL2.69×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
163.05ms39% SOL3.17×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
173.07ms39% SOL3.20×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
183.11ms39% SOL3.24×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
193.24ms38% SOL3.38×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
203.28ms38% SOL3.42×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
213.30ms38% SOL3.44×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
223.43ms37% SOL3.58×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
233.48ms36% SOL3.63×Reported · license unknown · no source2026-07-20
Measured exactly what you asked. Reported by source; not independently reproduced.