NVFP4 attention QKV projection

21 eligible runs

quantization

FP4-quantized Q, K, V projections for multi-head attention with GQA (32 query heads, 2 key-value heads). Performs three large matrix multiplications with FP4 weights and block-scaled quantization. Transforms hidden_size (2688) to num_heads * head_dim (4096) for queries and num_key_value_heads * head_dim (256) for keys and values.

Fastest reported

47.3µsmean · 95% of SOL

Amir M. Mir | SF TensorLicense unknown · unknown

Reported evidence · last observed 2026-07-17. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyq_size = 4096 · kv_size = 256 · head_dim = 128 · hidden_sf_k = 168 · hidden_size = 2688 · sf_vec_size = 16 · hidden_packed = 1344 · num_attention_heads = 32 · num_key_value_heads = 216 cases

batch_size

seq_len

dtype

1128fp4/fp32/fp8_e4m3 1256fp4/fp32/fp8_e4m3 11024fp4/fp32/fp8_e4m3 12048fp4/fp32/fp8_e4m3 18192fp4/fp32/fp8_e4m3 2256fp4/fp32/fp8_e4m3 2512fp4/fp32/fp8_e4m3 4256fp4/fp32/fp8_e4m3 4512fp4/fp32/fp8_e4m3 42048fp4/fp32/fp8_e4m3 8512fp4/fp32/fp8_e4m3 16256fp4/fp32/fp8_e4m3 32128fp4/fp32/fp8_e4m3 32256fp4/fp32/fp8_e4m3 32512fp4/fp32/fp8_e4m3 64128fp4/fp32/fp8_e4m3

scrolls · 16 cases total

Aggregatessuite of 16 cases · mean latency

HardwareBest knownImplementationRuns

NVIDIA B200 · env 147.3 µsAmir M. Mir | SF Tensor16 NVIDIA B200 · env 293.1 µsdoubleAI5

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 16 results · last observed 2026-08-17Record history →

#

Implementation

Latency

vs #1

Trust

Observed

1

Amir M. Mir | SF Tensor

47.3µs95% SOL

1.00×

Reported · license unknown · no source

2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

2

50.3µs95% SOL

1.06×

Reported · license unknown · no source

2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

3

50.6µs94% SOL

1.07×

Reported · license unknown · no source

2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

4

52.9µs94% SOL

1.12×

Reported · license unknown · no source

2026-08-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

5

70.4µs93% SOL

1.49×

Reported · license unknown · no source

2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

6

87.4µs91% SOL

1.85×

Reported · license unknown · no source

2026-07-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

7

90.7µs90% SOL

1.92×

Reported · license unknown · no source

2026-08-05

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

8

94.9µs90% SOL

2.01×

Reported · license unknown · no source

2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

9

100.5µs88% SOL

2.13×

Reported · license unknown · no source

2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

10

107.5µs89% SOL

2.28×

Reported · license unknown · no source

2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

11

121.2µs87% SOL

2.56×

Reported · license unknown · no source

2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

12

122.5µs86% SOL

2.59×

Reported · license unknown · no source

2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

13

178.6µs81% SOL

3.78×

Reported · license unknown · no source

2026-07-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

14

236.4µs77% SOL

5.00×

Reported · license unknown · no source

2026-07-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

15

242.1µs77% SOL

5.12×

Reported · license unknown · no source

2026-07-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

16

573.3µs57% SOL

12.13×

Reported · license unknown · no source

2026-07-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation

Runtime

Best latency

Evidence

Availability

Amir M. Mir | SF Tensor

unknown

47.3µs

1.00×

Reported

license unknown · no source

unknown

50.3µs

1.06×

Reported

license unknown · no source

unknown

70.4µs

1.49×

Reported

license unknown · no source

unknown

50.6µs

1.07×

Reported

license unknown · no source

unknown

87.4µs

1.85×

Reported

license unknown · no source

unknown

93.1µs

1.97×

Reported

license unknown · no source

unknown

107.5µs

2.28×

Reported

license unknown · no source

unknown

97.2µs

2.06×

Reported

license unknown · no source

Show all 21 implementations ›

unknown

119.0µs

2.52×

Reported

license unknown · no source

unknown

52.9µs

1.12×

Reported

license unknown · no source

unknown

90.7µs

1.92×

Reported

license unknown · no source

unknown

94.9µs

2.01×

Reported

license unknown · no source

unknown

100.5µs

2.13×

Reported

license unknown · no source

unknown

121.2µs

2.56×

Reported

license unknown · no source

unknown

122.5µs

2.59×

Reported

license unknown · no source

unknown

178.6µs

3.78×

Reported

license unknown · no source

unknown

236.4µs

5.00×

Reported

license unknown · no source

unknown

237.8µs

5.03×

Reported

license unknown · no source

unknown

242.1µs

5.12×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

576.3µs

12.20×

Reported

license unknown · no source

unknown

573.3µs

12.13×

Reported

license unknown · no source

Semantics

Inputs and outputs

hidden_statesfp32 [batch_size, seq_len, hidden_size]

q_weight_fp4float4_e2m1fn_x2 [q_size, hidden_packed]

q_scalefp8_e4m3 [q_size, hidden_sf_k]

q_global_decode_scalefp32 scalar

k_weight_fp4float4_e2m1fn_x2 [kv_size, hidden_packed]

k_scalefp8_e4m3 [kv_size, hidden_sf_k]

k_global_decode_scalefp32 scalar

v_weight_fp4float4_e2m1fn_x2 [kv_size, hidden_packed]

v_scalefp8_e4m3 [kv_size, hidden_sf_k]

v_global_decode_scalefp32 scalar

query_statesfp16 [batch_size, seq_len, q_size]

key_statesfp16 [batch_size, seq_len, kv_size]

value_statesfp16 [batch_size, seq_len, kv_size]

Axes and behavior

q_sizeconstant = 4096

kv_sizeconstant = 256

seq_lenvariable

head_dimconstant = 128

batch_sizevariable

hidden_sf_kconstant = 168

hidden_sizeconstant = 2688

sf_vec_sizeconstant = 16

hidden_packedconstant = 1344

num_attention_headsconstant = 32

num_key_value_headsconstant = 2

determinismunspecified

constraintsNo mutation or aliasing

Identity

alias025_nvfp4_attention_qkv_projectionmodelnvidia-nemotron-3-nano-30b-a3b-nvfp4sha2562159763a2082…

Sources: NVIDIA SOL-ExecBench (2026-08-17)last observed 2026-08-17How records are decided JSON