FP8 MLA attention output projection

44 eligible runs

mla-paged-attention

FP8-quantized Multi-Latent Attention (MLA) output projection. Projects concatenated attention outputs (num_heads * v_head_dim = 16384) back to hidden_size (7168). Uses BlockWise1x128 scaling for activations and BlockWise128x128 scaling for weights.

Fastest reported

387.8µsmean · 76% of SOL

RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-21. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyinput_dim = 16384 · v_head_dim = 128 · hidden_size = 7168 · num_input_blocks = 128 · num_hidden_blocks = 56 · num_attention_heads = 12816 cases

batch_size

seq_len

dtype

1128fp32/fp8_e4m3 11024fp32/fp8_e4m3 14096fp32/fp8_e4m3 18192fp32/fp8_e4m3 2128fp32/fp8_e4m3 2512fp32/fp8_e4m3 41024fp32/fp8_e4m3 42048fp32/fp8_e4m3 8256fp32/fp8_e4m3 81024fp32/fp8_e4m3 16128fp32/fp8_e4m3 16256fp32/fp8_e4m3 16512fp32/fp8_e4m3 32128fp32/fp8_e4m3 32512fp32/fp8_e4m3 64256fp32/fp8_e4m3

scrolls · 16 cases total

Aggregatessuite of 16 cases · mean latency

HardwareBest knownImplementationRuns

NVIDIA B200 · env 1387.8 µsRIAC as well38 NVIDIA B200 · env 2484.2 µsdoubleAI6

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 38 results · last observed 2026-08-24Record history →

#

Implementation

Latency

vs #1

Trust

Observed

1

387.8µs76% SOL

1.00×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

2

416.5µs73% SOL

1.07×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

3

AccountForCollege

453.1µs69% SOL

1.17×

Reported · license unknown · no source

2026-08-09

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

4

Amir M. Mir | SF Tensor

459.3µs68% SOL

1.18×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

5

Amir M. Mir | SF Tensor

462.8µs68% SOL

1.19×

Reported · license unknown · no source

2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

6

470.0µs67% SOL

1.21×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

7

478.6µs66% SOL

1.23×

Reported · license unknown · no source

2026-08-01

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

8

522.3µs63% SOL

1.35×

Reported · license unknown · no source

2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

9

Amir M. Mir | SF Tensor

523.7µs63% SOL

1.35×

Reported · license unknown · no source

2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

10

571.0µs60% SOL

1.47×

Reported · license unknown · no source

2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

11

574.6µs59% SOL

1.48×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

12

577.3µs59% SOL

1.49×

Reported · license unknown · no source

2026-08-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

13

579.3µs59% SOL

1.49×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

14

579.5µs59% SOL

1.49×

Reported · license unknown · no source

2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

15

584.8µs59% SOL

1.51×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

16

584.9µs59% SOL

1.51×

Reported · license unknown · no source

2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

17

585.6µs59% SOL

1.51×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

18

585.9µs59% SOL

1.51×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

19

590.8µs58% SOL

1.52×

Reported · license unknown · no source

2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

20

594.5µs58% SOL

1.53×

Reported · license unknown · no source

2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

21

596.2µs58% SOL

1.54×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

22

599.6µs58% SOL

1.55×

Reported · license unknown · no source

2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

23

604.5µs57% SOL

1.56×

Reported · license unknown · no source

2026-08-09

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

24

606.8µs57% SOL

1.56×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

25

615.5µs57% SOL

1.59×

Reported · license unknown · no source

2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

26

650.0µs55% SOL

1.68×

Reported · license unknown · no source

2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

27

673.0µs53% SOL

1.74×

Reported · license unknown · no source

2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

28

682.3µs53% SOL

1.76×

Reported · license unknown · no source

2026-08-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

29

699.5µs52% SOL

1.80×

Reported · license unknown · no source

2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

30

Amir M. Mir | SF Tensor

712.6µs51% SOL

1.84×

Reported · license unknown · no source

2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

31

733.0µs50% SOL

1.89×

Reported · license unknown · no source

2026-08-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

32

757.0µs49% SOL

1.95×

Reported · license unknown · no source

2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

33

758.0µs49% SOL

1.95×

Reported · license unknown · no source

2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

34

764.1µs49% SOL

1.97×

Reported · license unknown · no source

2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

35

1.40ms31% SOL

3.61×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

36

1.46ms30% SOL

3.77×

Reported · license unknown · no source

2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

37

1.63ms27% SOL

4.19×

Reported · license unknown · no source

2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

38

6.96ms7% SOL

17.94×

Reported · license unknown · no source

2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation

Runtime

Best latency

Evidence

Availability

AccountForCollege

unknown

453.1µs

1.17×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

462.8µs

1.19×

Reported

license unknown · no source

unknown

470.0µs

1.21×

Reported

license unknown · no source

unknown

387.8µs

1.00×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

459.3µs

1.18×

Reported

license unknown · no source

unknown

478.6µs

1.23×

Reported

license unknown · no source

unknown

484.2µs

1.25×

Reported

license unknown · no source

unknown

582.9µs

1.50×

Reported

license unknown · no source

Show all 44 implementations ›

unknown

650.0µs

1.68×

Reported

license unknown · no source

unknown

758.0µs

1.95×

Reported

license unknown · no source

unknown

416.5µs

1.07×

Reported

license unknown · no source

unknown

490.0µs

1.26×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

523.7µs

1.35×

Reported

license unknown · no source

unknown

522.3µs

1.35×

Reported

license unknown · no source

unknown

571.0µs

1.47×

Reported

license unknown · no source

unknown

574.6µs

1.48×

Reported

license unknown · no source

unknown

577.3µs

1.49×

Reported

license unknown · no source

unknown

579.3µs

1.49×

Reported

license unknown · no source

unknown

579.5µs

1.49×

Reported

license unknown · no source

unknown

584.8µs

1.51×

Reported

license unknown · no source

unknown

584.9µs

1.51×

Reported

license unknown · no source

unknown

585.6µs

1.51×

Reported

license unknown · no source

unknown

585.9µs

1.51×

Reported

license unknown · no source

unknown

590.8µs

1.52×

Reported

license unknown · no source

unknown

594.5µs

1.53×

Reported

license unknown · no source

unknown

596.2µs

1.54×

Reported

license unknown · no source

unknown

599.6µs

1.55×

Reported

license unknown · no source

unknown

604.5µs

1.56×

Reported

license unknown · no source

unknown

606.8µs

1.56×

Reported

license unknown · no source

unknown

615.5µs

1.59×

Reported

license unknown · no source

unknown

663.8µs

1.71×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

667.4µs

1.72×

Reported

license unknown · no source

unknown

673.0µs

1.74×

Reported

license unknown · no source

unknown

682.3µs

1.76×

Reported

license unknown · no source

unknown

699.5µs

1.80×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

712.6µs

1.84×

Reported

license unknown · no source

unknown

733.0µs

1.89×

Reported

license unknown · no source

unknown

757.0µs

1.95×

Reported

license unknown · no source

unknown

764.1µs

1.97×

Reported

license unknown · no source

unknown

1.40ms

3.61×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

1.40ms

3.60×

Reported

license unknown · no source

unknown

1.46ms

3.77×

Reported

license unknown · no source

unknown

1.63ms

4.19×

Reported

license unknown · no source

unknown

6.96ms

17.94×

Reported

license unknown · no source

Semantics

Inputs and outputs

hidden_statesfp8_e4m3 [batch_size, seq_len, input_dim]

weightfp8_e4m3 [hidden_size, input_dim]

scale_xfp32 [batch_size, seq_len, num_input_blocks]

scale_wfp32 [num_input_blocks, num_hidden_blocks]

outputbf16 [batch_size, seq_len, hidden_size]

Axes and behavior

seq_lenvariable

input_diminput_dim = num_attention_heads * v_head_dim

batch_sizevariable

v_head_dimconstant = 128

hidden_sizeconstant = 7168

num_input_blocksnum_input_blocks = input_dim // 128

num_hidden_blocksnum_hidden_blocks = hidden_size // 128

num_attention_headsconstant = 128

determinismunspecified

constraintsNo mutation or aliasing

Identity

alias015_fp8_mla_attention_output_projectionmodeldeepseek-v3sha25663d28ae3350f…

Sources: NVIDIA SOL-ExecBench (2026-08-24)last observed 2026-08-24How records are decided JSON