FP8 attention QKV projection

48 eligible runs

quantization

FP8-quantized QKV projection for multi-head attention with Q/K RMSNorm normalization. Projects hidden states to query, key, and value tensors with blockwise FP8 quantization (1x128 for activations, 128x128 for weights), followed by RMSNorm on Q and K heads. Uses GQA pattern with 60 query heads and 8 key/value heads.

Fastest reported

233.2µsmean · 75% of SOL

RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-19. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyq_size = 7680 · kv_size = 1024 · head_dim = 128 · hidden_size = 7680 · num_attention_heads = 60 · num_key_value_heads = 816 cases

batch_size

seq_len

dtype

1128bf16 1256bf16 1384bf16 11024bf16 14096bf16 2128bf16 21024bf16 22048bf16 4128bf16 4384bf16 8128bf16 81024bf16 16256bf16 32256bf16 64128bf16 64256bf16

scrolls · 16 cases total

Aggregatessuite of 16 cases · mean latency

HardwareBest knownImplementationRuns

NVIDIA B200 · env 1233.2 µsRIAC as well37 NVIDIA B200 · env 2255.6 µsArgus11

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 37 results · last observed 2026-08-24Record history →

#

Implementation

Latency

vs #1

Trust

Observed

1

233.2µs75% SOL

1.00×

Reported · license unknown · no source

2026-08-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

2

Amir M. Mir | SF Tensor

238.1µs74% SOL

1.02×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

3

Amir M. Mir | SF Tensor

276.7µs69% SOL

1.19×

Reported · license unknown · no source

2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

4

324.9µs65% SOL

1.39×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

5

326.6µs65% SOL

1.40×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

6

332.6µs64% SOL

1.43×

Reported · license unknown · no source

2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

7

332.8µs64% SOL

1.43×

Reported · license unknown · no source

2026-08-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

8

335.1µs64% SOL

1.44×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

9

335.2µs64% SOL

1.44×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

10

336.9µs64% SOL

1.44×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

11

338.6µs63% SOL

1.45×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

12

339.0µs63% SOL

1.45×

Reported · license unknown · no source

2026-07-30

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

13

339.4µs63% SOL

1.46×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

14

343.9µs63% SOL

1.47×

Reported · license unknown · no source

2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

15

344.0µs63% SOL

1.48×

Reported · license unknown · no source

2026-07-31

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

16

349.4µs63% SOL

1.50×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

17

357.9µs62% SOL

1.53×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

18

361.6µs62% SOL

1.55×

Reported · license unknown · no source

2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

19

362.9µs61% SOL

1.56×

Reported · license unknown · no source

2026-08-09

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

20

376.0µs60% SOL

1.61×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

21

388.8µs59% SOL

1.67×

Reported · license unknown · no source

2026-08-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

22

396.5µs58% SOL

1.70×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

23

397.8µs59% SOL

1.71×

Reported · license unknown · no source

2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

24

405.8µs58% SOL

1.74×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

25

406.6µs58% SOL

1.74×

Reported · license unknown · no source

2026-08-05

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

26

421.3µs56% SOL

1.81×

Reported · license unknown · no source

2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

27

429.3µs56% SOL

1.84×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

28

438.0µs56% SOL

1.88×

Reported · license unknown · no source

2026-08-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

29

Amir M. Mir | SF Tensor

439.6µs55% SOL

1.89×

Reported · license unknown · no source

2026-07-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

30

442.9µs56% SOL

1.90×

Reported · license unknown · no source

2026-08-09

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

31

452.6µs55% SOL

1.94×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

32

Amir M. Mir | SF Tensor

536.4µs50% SOL

2.30×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

33

624.8µs46% SOL

2.68×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

34

674.8µs43% SOL

2.89×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

35

927.0µs35% SOL

3.98×

Reported · license unknown · no source

2026-07-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

36

972.4µs34% SOL

4.17×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

37

978.2µs34% SOL

4.19×

Reported · license unknown · no source

2026-07-25

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation

Runtime

Best latency

Evidence

Availability

Amir M. Mir | SF Tensor

unknown

276.7µs

1.19×

Reported

license unknown · no source

unknown

344.0µs

1.48×

Reported

license unknown · no source

unknown

339.0µs

1.45×

Reported

license unknown · no source

unknown

233.2µs

1.00×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

238.1µs

1.02×

Reported

license unknown · no source

unknown

255.6µs

1.10×

Reported

license unknown · no source

unknown

324.9µs

1.39×

Reported

license unknown · no source

unknown

325.3µs

1.40×

Reported

license unknown · no source

Show all 48 implementations ›

unknown

308.6µs

1.32×

Reported

license unknown · no source

unknown

361.2µs

1.55×

Reported

license unknown · no source

unknown

601.7µs

2.58×

Reported

license unknown · no source

unknown

796.7µs

3.42×

Reported

license unknown · no source

unknown

326.6µs

1.40×

Reported

license unknown · no source

unknown

332.8µs

1.43×

Reported

license unknown · no source

unknown

332.6µs

1.43×

Reported

license unknown · no source

unknown

335.2µs

1.44×

Reported

license unknown · no source

unknown

335.1µs

1.44×

Reported

license unknown · no source

unknown

336.9µs

1.44×

Reported

license unknown · no source

unknown

338.6µs

1.45×

Reported

license unknown · no source

unknown

339.4µs

1.46×

Reported

license unknown · no source

unknown

343.9µs

1.47×

Reported

license unknown · no source

unknown

349.4µs

1.50×

Reported

license unknown · no source

unknown

357.9µs

1.53×

Reported

license unknown · no source

unknown

361.6µs

1.55×

Reported

license unknown · no source

unknown

362.9µs

1.56×

Reported

license unknown · no source

unknown

376.0µs

1.61×

Reported

license unknown · no source

unknown

388.8µs

1.67×

Reported

license unknown · no source

unknown

397.8µs

1.71×

Reported

license unknown · no source

unknown

405.8µs

1.74×

Reported

license unknown · no source

unknown

396.5µs

1.70×

Reported

license unknown · no source

unknown

406.6µs

1.74×

Reported

license unknown · no source

unknown

429.3µs

1.84×

Reported

license unknown · no source

unknown

421.3µs

1.81×

Reported

license unknown · no source

unknown

438.0µs

1.88×

Reported

license unknown · no source

unknown

442.9µs

1.90×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

439.6µs

1.89×

Reported

license unknown · no source

unknown

452.6µs

1.94×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

519.6µs

2.23×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

536.4µs

2.30×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

565.4µs

2.42×

Reported

license unknown · no source

unknown

624.8µs

2.68×

Reported

license unknown · no source

unknown

674.8µs

2.89×

Reported

license unknown · no source

unknown

927.0µs

3.98×

Reported

license unknown · no source

unknown

978.2µs

4.19×

Reported

license unknown · no source

unknown

972.4µs

4.17×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

7.18ms

30.81×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

7.60ms

32.61×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

12.7ms

54.57×

Reported

license unknown · no source

Semantics

Inputs and outputs

hidden_statesbf16 [batch_size, seq_len, hidden_size]

q_weightbf16 [q_size, hidden_size]

k_weightbf16 [kv_size, hidden_size]

v_weightbf16 [kv_size, hidden_size]

q_biasbf16 [q_size]

k_biasbf16 [kv_size]

v_biasbf16 [kv_size]

q_norm_weightbf16 [head_dim]

k_norm_weightbf16 [head_dim]

rms_norm_epsfp32 scalar

query_statesbf16 [batch_size, num_attention_heads, seq_len, head_dim]

key_statesbf16 [batch_size, num_key_value_heads, seq_len, head_dim]

value_statesbf16 [batch_size, num_key_value_heads, seq_len, head_dim]

Axes and behavior

q_sizeq_size = num_attention_heads * head_dim

kv_sizekv_size = num_key_value_heads * head_dim

seq_lenvariable

head_dimconstant = 128

batch_sizevariable

hidden_sizeconstant = 7680

num_attention_headsconstant = 60

num_key_value_headsconstant = 8

determinismunspecified

constraintsNo mutation or aliasing

Identity

alias002_fp8_attention_qkv_projectionmodelqwen3-vl-30b-a3b-instruct-fp8sha256cb6dbcc18adb…

Sources: NVIDIA SOL-ExecBench (2026-08-24)last observed 2026-08-24How records are decided JSON