Chunk gated delta rule linear attention
Chunk-based Gated Delta Rule linear attention computation that processes sequences in chunks of 64 tokens. Implements delta rule update with exponential decay gating, chunk-wise attention with intra-chunk causal masking, inter-chunk recurrent state propagation, and cumulative decay tracking.
Reported evidence · last observed 2026-08-20. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencychunk_size = 64 · head_k_dim = 128 · head_v_dim = 128 · num_k_heads = 4 · num_v_heads = 1616 cases
Not measured on H100 for this workload. Challenges →
145.2µs93% SOL1.00×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
254.6µs92% SOL1.21×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
357.7µs91% SOL1.27×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
459.3µs91% SOL1.31×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
560.1µs91% SOL1.33×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
660.4µs91% SOL1.34×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
763.5µs90% SOL1.40×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
864.9µs90% SOL1.43×Reported · license unknown · no source2026-08-07
Measured exactly what you asked. Reported by source; not independently reproduced.
965.2µs90% SOL1.44×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
1072.5µs89% SOL1.60×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
1172.9µs89% SOL1.61×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1289.2µs86% SOL1.97×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
1394.2µs86% SOL2.08×Reported · license unknown · no source2026-08-07
Measured exactly what you asked. Reported by source; not independently reproduced.
1497.1µs85% SOL2.15×Reported · license unknown · no source2026-07-30
Measured exactly what you asked. Reported by source; not independently reproduced.
15126.9µs81% SOL2.80×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
16274.2µs66% SOL6.06×Reported · license unknown · no source2026-08-04
Measured exactly what you asked. Reported by source; not independently reproduced.
17597.2µs47% SOL13.20×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
181.49ms26% SOL32.96×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
195.22ms10% SOL115.46×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.