Rotary embedding with attention scaling backward
Backward pass for rotary position embedding with attention scaling. Computes gradients with respect to position_ids through the RoPE embedding generation pipeline including inverse frequency expansion, frequency computation, and cos/sin with attention scaling.
Reported evidence · last observed 2026-07-10. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhalf_dim = 64 · head_dim = 12816 cases
Not measured on H100 for this workload. Challenges →
14.93µs93% SOL1.00×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
25.12µs93% SOL1.04×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
35.34µs93% SOL1.08×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
45.34µs93% SOL1.08×Reported · license unknown · no source2026-08-24
Measured exactly what you asked. Reported by source; not independently reproduced.
55.36µs93% SOL1.09×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
65.37µs92% SOL1.09×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
75.39µs93% SOL1.09×Reported · license unknown · no source2026-07-14
Measured exactly what you asked. Reported by source; not independently reproduced.
85.42µs92% SOL1.10×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
96.02µs90% SOL1.22×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
106.03µs89% SOL1.22×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
116.07µs89% SOL1.23×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
126.25µs91% SOL1.27×Reported · license unknown · no source2026-07-29
Measured exactly what you asked. Reported by source; not independently reproduced.
136.31µs89% SOL1.28×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
146.34µs89% SOL1.28×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
156.34µs89% SOL1.29×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
166.72µs88% SOL1.36×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
176.94µs88% SOL1.41×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
187.11µs90% SOL1.44×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.