Fused encoder final norm to decoder cross attention KV projection
Fused encoder final RMSNorm with cross-attention K/V projection. Applies RMSNorm to encoder hidden states then projects to cross-attention keys and values, avoiding intermediate materialization of normalized states.
Reported evidence · last observed 2026-07-10. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 64 · hidden_size = 1024 · num_kv_heads = 2 · kv_hidden_size = 12816 cases
Not measured on H100 for this workload. Challenges →
110.2µs84% SOL1.00×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
210.3µs83% SOL1.01×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
310.5µs84% SOL1.02×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
410.6µs83% SOL1.04×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
510.6µs83% SOL1.04×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
610.9µs82% SOL1.07×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
711.3µs82% SOL1.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
811.4µs81% SOL1.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
912.1µs80% SOL1.19×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
1012.1µs80% SOL1.19×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
1112.9µs79% SOL1.26×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
1213.6µs77% SOL1.33×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1313.6µs77% SOL1.34×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
1413.8µs77% SOL1.35×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
1513.8µs77% SOL1.35×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
1614.1µs76% SOL1.38×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
1714.4µs76% SOL1.42×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
1816.0µs74% SOL1.56×Reported · license unknown · no source2026-08-07
Measured exactly what you asked. Reported by source; not independently reproduced.
1916.4µs72% SOL1.61×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2022.9µs64% SOL2.25×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
2123.3µs64% SOL2.29×Reported · license unknown · no source2026-07-23
Measured exactly what you asked. Reported by source; not independently reproduced.
2241.6µs47% SOL4.07×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.