AMD mixed-precision MLA decode
DeepSeek-R1 forward_absorb MLA decode attention for MI355X: absorbed bf16 queries against a compressed latent KV cache with indptr-based variable-length batching, one decode step (qseqlen 1, kvseqlen up to 8k). The KV buffer is supplied in bf16, fp8, and mxfp4 formats; the full 576 dims act as keys and the first 512 as values, and a config dict accompanies the tensors listed here.
Reported evidence · last observed 2026-04-07. Reported by source; not independently reproduced. License unknown.
Current records
Not measured on H100, B200 for this workload. Challenges →
1submission 754087Amo-Zeng8.05µs1.00×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
2submission 753797Barry_zhang14.5µs1.80×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
3submission 754992josusanmartin21.2µs2.63×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
4submission 736894NinoHeather22.1µs2.75×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
5submission 736348josusanmartin22.2µs2.75×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
6submission 675331josusanmartin23.0µs2.86×Reported · license unknown · source2026-03-30stale
Measured exactly what you asked. Reported by source; not independently reproduced.
7submission 673003josusanmartin23.5µs2.93×Reported · license unknown · source2026-03-30stale
Measured exactly what you asked. Reported by source; not independently reproduced.
8submission 670399josusanmartin24.2µs3.00×Reported · license unknown · source2026-03-30stale
Measured exactly what you asked. Reported by source; not independently reproduced.
9submission 670118josusanmartin25.9µs3.22×Reported · license unknown · no source2026-03-30stale
Measured exactly what you asked. Reported by source; not independently reproduced.
10submission 669133josusanmartin27.5µs3.41×Reported · license unknown · source2026-03-30stale
Measured exactly what you asked. Reported by source; not independently reproduced.
11submission 755173Danny Estenberg28.6µs3.55×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
12submission 749874John Hahn29.2µs3.63×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
13submission 693347John Hahn29.3µs3.64×Reported · license unknown · source2026-04-01stale
Measured exactly what you asked. Reported by source; not independently reproduced.
14submission 755183Will Fisher29.3µs3.65×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
15submission 742733LunNova29.4µs3.65×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
16submission 748977Jingze29.8µs3.70×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
17submission 743988mmk15030.0µs3.73×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
18submission 661294josusanmartin31.2µs3.87×Reported · license unknown · source2026-03-29stale
Measured exactly what you asked. Reported by source; not independently reproduced.
19submission 754421divc1331.2µs3.87×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
20submission 754926ftyghome31.4µs3.90×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
21submission 747749divc1331.7µs3.95×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
22submission 755102Law191231.9µs3.96×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
23submission 746709divc1331.9µs3.96×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
24submission 702682Danishlynx31.9µs3.97×Reported · license unknown · source2026-04-02stale
Measured exactly what you asked. Reported by source; not independently reproduced.
25submission 648412John Hahn32.0µs3.98×Reported · license unknown · source2026-03-27stale
Measured exactly what you asked. Reported by source; not independently reproduced.
26submission 755196Ananda Sai A32.3µs4.01×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
27submission 741445Maxwell Cipher32.4µs4.03×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
28submission 754379ftyghome32.5µs4.03×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
29submission 742458Ananda Sai A32.5µs4.04×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
30submission 745253augustus202432.5µs4.05×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
31submission 739598johnny.t.shi32.6µs4.05×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
32submission 751073mars-compute32.6µs4.05×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
33submission 745692divc1332.7µs4.06×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
34submission 700499Danishlynx32.7µs4.07×Reported · license unknown · source2026-04-02stale
Measured exactly what you asked. Reported by source; not independently reproduced.
35submission 618775Nicky Pochinkov32.7µs4.07×Reported · license unknown · source2026-03-24stale
Measured exactly what you asked. Reported by source; not independently reproduced.
36submission 723187Hamza32.8µs4.07×Reported · license unknown · source2026-04-04stale
Measured exactly what you asked. Reported by source; not independently reproduced.
37submission 753932guojun2132.9µs4.09×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
38submission 601703Ananda Sai A33.0µs4.10×Reported · license unknown · source2026-03-21stale
Measured exactly what you asked. Reported by source; not independently reproduced.
39submission 723086.jonnss33.0µs4.10×Reported · license unknown · source2026-04-04stale
Measured exactly what you asked. Reported by source; not independently reproduced.
40submission 744389divc1333.1µs4.11×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
41submission 635670josusanmartin33.1µs4.11×Reported · license unknown · source2026-03-26stale
Measured exactly what you asked. Reported by source; not independently reproduced.
42submission 677054johnny.t.shi33.2µs4.13×Reported · license unknown · source2026-03-31stale
Measured exactly what you asked. Reported by source; not independently reproduced.
43submission 607363Nicky Pochinkov33.4µs4.15×Reported · license unknown · source2026-03-22stale
Measured exactly what you asked. Reported by source; not independently reproduced.
44submission 716743ooousay33.4µs4.15×Reported · license unknown · source2026-04-03stale
Measured exactly what you asked. Reported by source; not independently reproduced.
45submission 735284Borui Xu33.5µs4.16×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
46submission 606366Nicky Pochinkov33.5µs4.16×Reported · license unknown · source2026-03-21stale
Measured exactly what you asked. Reported by source; not independently reproduced.
47submission 737883JIANG Jiaxin33.6µs4.18×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
48submission 737583mars-compute33.7µs4.18×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
49submission 740330divc1333.7µs4.19×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
50submission 739084Maxwell Cipher33.8µs4.20×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
51submission 687679NinoHeather33.9µs4.21×Reported · license unknown · source2026-04-01stale
Measured exactly what you asked. Reported by source; not independently reproduced.
52submission 679268Danishlynx33.9µs4.22×Reported · license unknown · source2026-03-31stale
Measured exactly what you asked. Reported by source; not independently reproduced.
53submission 714276internetrat34.0µs4.23×Reported · license unknown · source2026-04-03stale
Measured exactly what you asked. Reported by source; not independently reproduced.
54submission 651367johnny.t.shi34.1µs4.23×Reported · license unknown · source2026-03-28stale
Measured exactly what you asked. Reported by source; not independently reproduced.
55submission 734135willfisher34.3µs4.26×Reported · license unknown · source2026-04-05
Measured exactly what you asked. Reported by source; not independently reproduced.
56submission 747410Brian Sun34.3µs4.27×Reported · license unknown · source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
57submission 753497guojun2134.4µs4.27×Reported · license unknown · no source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
58submission 641982johnny.t.shi34.5µs4.28×Reported · license unknown · source2026-03-26stale
Measured exactly what you asked. Reported by source; not independently reproduced.
59submission 745713Brian Sun34.5µs4.28×Reported · license unknown · no source2026-04-06
Measured exactly what you asked. Reported by source; not independently reproduced.
60submission 754986flower212334.5µs4.29×Reported · license unknown · source2026-04-07
Measured exactly what you asked. Reported by source; not independently reproduced.
540 more rows in this comparison group. Open all in search →