MTP shifted embedding with dual RMS norm fusion
Multi-Token Prediction (MTP) shifted embedding with dual RMS normalization fusion. Rolls input_ids by -1, embeds shifted ids, applies RMSNorm to both embeddings and hidden states, concatenates, and projects back to hidden_size.
Reported evidence · last observed 2026-08-18. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyvocab_size = 157184 · hidden_size = 4096 · hidden_size_x2 = 819216 cases
Not measured on H100 for this workload. Challenges →
1115.9µs76% SOL1.00×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
2123.4µs73% SOL1.06×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
3123.4µs72% SOL1.06×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
4126.8µs72% SOL1.09×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
5127.0µs72% SOL1.10×Reported · license unknown · no source2026-08-07
Measured exactly what you asked. Reported by source; not independently reproduced.
6130.1µs70% SOL1.12×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
7130.3µs70% SOL1.12×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
8130.8µs70% SOL1.13×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
9132.3µs70% SOL1.14×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
10135.5µs68% SOL1.17×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
11140.9µs67% SOL1.22×Reported · license unknown · no source2026-07-23
Measured exactly what you asked. Reported by source; not independently reproduced.
12141.6µs67% SOL1.22×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
131.30ms12% SOL11.22×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.