Adaptive LayerNorm continuous with modulation
AdaLayerNormContinuous normalization with adaptive scale and shift modulation conditioned on timestep embeddings. Computes layer normalization, projects temb through a linear layer to get scale/shift, then modulates: normalized * (1 + scale) + shift.
Reported evidence · last observed 2026-08-23. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyinner_dim = 2432 · inner_dim_x2 = 486416 cases
Not measured on H100 for this workload. Challenges →
121.1µs81% SOL1.00×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
222.0µs80% SOL1.04×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
323.6µs79% SOL1.12×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
423.9µs79% SOL1.13×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
524.4µs78% SOL1.15×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
624.4µs78% SOL1.15×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
725.7µs77% SOL1.22×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
825.9µs77% SOL1.23×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
926.8µs76% SOL1.27×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
1027.5µs76% SOL1.30×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
1128.2µs75% SOL1.33×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
1228.3µs75% SOL1.34×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
1328.4µs75% SOL1.34×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1430.3µs74% SOL1.43×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1530.6µs73% SOL1.45×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
1631.8µs73% SOL1.51×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1732.1µs72% SOL1.52×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1847.0µs63% SOL2.22×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1980.2µs50% SOL3.80×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.