Encoder layer dual residual norm chain
Complete encoder layer processing chain with two sequential residual-norm patterns: (1) self-attention with pre-norm and residual, (2) MLP with pre-norm and residual. This pattern appears in all 12 encoder layers and represents a critical memory bandwidth bottleneck due to repeated residual additions and RMSNorm operations.
Reported evidence · last observed 2026-07-16. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 64 · num_heads = 16 · hidden_size = 1024 · intermediate_size = 4096 · num_key_value_heads = 4 · intermediate_size_x2 = 819216 cases
Not measured on H100 for this workload. Challenges →
1155.2µs64% SOL1.00×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
2160.5µs63% SOL1.03×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
3162.8µs62% SOL1.05×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
4163.5µs62% SOL1.05×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
5171.1µs61% SOL1.10×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
6173.7µs60% SOL1.12×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
7177.0µs60% SOL1.14×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
8184.7µs58% SOL1.19×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
9191.3µs57% SOL1.23×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
10192.0µs57% SOL1.24×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
11200.9µs55% SOL1.29×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
12208.2µs54% SOL1.34×Reported · license unknown · no source2026-07-25
Measured exactly what you asked. Reported by source; not independently reproduced.
13211.6µs54% SOL1.36×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
14222.6µs53% SOL1.43×Reported · license unknown · no source2026-07-19
Measured exactly what you asked. Reported by source; not independently reproduced.
15495.9µs30% SOL3.20×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.