NVFP4 attention QKV projection with RoPE

26 eligible runs

rope

FP4-quantized attention QKV projection with RMSNorm, RoPE application, and KV cache update. Fuses Q/K/V projections (FP4 block-scaled), per-head RMSNorm on Q and K, rotary position embeddings, for GQA attention with 32 query heads and 8 KV heads.

Fastest reported

86.1µsmean · 94% of SOL

Amir M. Mir | SF TensorLicense unknown · unknown

Reported evidence · last observed 2026-08-01. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyq_size = 4096 · kv_size = 1024 · head_dim = 128 · q_blocks = 256 · kv_blocks = 64 · hidden_size = 4096 · q_size_half = 2048 · kv_size_half = 512 · hidden_blocks = 256 · hidden_size_half = 2048 · num_attention_heads = 32 · num_key_value_heads = 816 cases

seq_len

batch_size

dtype

1282bf16/fp4/fp8_e4m3 1284bf16/fp4/fp8_e4m3 12832bf16/fp4/fp8_e4m3 2568bf16/fp4/fp8_e4m3 25616bf16/fp4/fp8_e4m3 5121bf16/fp4/fp8_e4m3 5122bf16/fp4/fp8_e4m3 5124bf16/fp4/fp8_e4m3 10241bf16/fp4/fp8_e4m3 10248bf16/fp4/fp8_e4m3 20481bf16/fp4/fp8_e4m3 20482bf16/fp4/fp8_e4m3 40961bf16/fp4/fp8_e4m3 40962bf16/fp4/fp8_e4m3 40964bf16/fp4/fp8_e4m3 81921bf16/fp4/fp8_e4m3

scrolls · 16 cases total

Aggregatessuite of 16 cases · mean latency

HardwareBest knownImplementationRuns

NVIDIA B200 · env 186.1 µsAmir M. Mir | SF Tensor20 NVIDIA B200 · env 2101.9 µsRecursive6

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 20 results · last observed 2026-08-23Record history →

#

Implementation

Latency

vs #1

Trust

Observed

1

Amir M. Mir | SF Tensor

86.1µs94% SOL

1.00×

Reported · license unknown · no source

2026-08-01

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

2

89.1µs93% SOL

1.04×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

3

106.7µs92% SOL

1.24×

Reported · license unknown · no source

2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

4

107.3µs92% SOL

1.25×

Reported · license unknown · no source

2026-08-01

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

5

107.9µs92% SOL

1.25×

Reported · license unknown · no source

2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

6

111.6µs91% SOL

1.30×

Reported · license unknown · no source

2026-08-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

7

114.1µs91% SOL

1.33×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

8

139.5µs89% SOL

1.62×

Reported · license unknown · no source

2026-08-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

9

142.2µs89% SOL

1.65×

Reported · license unknown · no source

2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

10

146.5µs89% SOL

1.70×

Reported · license unknown · no source

2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

11

151.3µs88% SOL

1.76×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

12

Amir M. Mir | SF Tensor

153.8µs88% SOL

1.79×

Reported · license unknown · no source

2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

13

172.7µs87% SOL

2.01×

Reported · license unknown · no source

2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

14

179.0µs86% SOL

2.08×

Reported · license unknown · no source

2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

15

196.2µs85% SOL

2.28×

Reported · license unknown · no source

2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

16

229.9µs83% SOL

2.67×

Reported · license unknown · no source

2026-08-09

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

17

234.4µs83% SOL

2.72×

Reported · license unknown · no source

2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

18

326.7µs77% SOL

3.80×

Reported · license unknown · no source

2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

19

338.6µs76% SOL

3.93×

Reported · license unknown · no source

2026-07-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

20

1.21ms47% SOL

14.08×

Reported · license unknown · no source

2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation

Runtime

Best latency

Evidence

Availability

Amir M. Mir | SF Tensor

unknown

86.1µs

1.00×

Reported

license unknown · no source

unknown

107.3µs

1.25×

Reported

license unknown · no source

unknown

106.7µs

1.24×

Reported

license unknown · no source

unknown

89.1µs

1.04×

Reported

license unknown · no source

unknown

101.9µs

1.18×

Reported

license unknown · no source

unknown

104.5µs

1.21×

Reported

license unknown · no source

unknown

111.6µs

1.30×

Reported

license unknown · no source

unknown

234.4µs

2.72×

Reported

license unknown · no source

Show all 26 implementations ›

unknown

879.7µs

10.22×

Reported

license unknown · no source

unknown

107.9µs

1.25×

Reported

license unknown · no source

unknown

114.1µs

1.33×

Reported

license unknown · no source

unknown

125.3µs

1.46×

Reported

license unknown · no source

unknown

139.5µs

1.62×

Reported

license unknown · no source

unknown

142.2µs

1.65×

Reported

license unknown · no source

unknown

146.5µs

1.70×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

152.2µs

1.77×

Reported

license unknown · no source

unknown

151.3µs

1.76×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

153.8µs

1.79×

Reported

license unknown · no source

unknown

172.7µs

2.01×

Reported

license unknown · no source

unknown

179.0µs

2.08×

Reported

license unknown · no source

unknown

196.2µs

2.28×

Reported

license unknown · no source

unknown

229.9µs

2.67×

Reported

license unknown · no source

unknown

326.7µs

3.80×

Reported

license unknown · no source

unknown

338.6µs

3.93×

Reported

license unknown · no source

unknown

1.21ms

14.08×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

1.53ms

17.82×

Reported

license unknown · no source

Semantics

Inputs and outputs

hidden_statesbf16 [batch_size, seq_len, hidden_size]

cosbf16 [batch_size, seq_len, head_dim]

sinbf16 [batch_size, seq_len, head_dim]

q_proj_weight_fp4float4_e2m1fn_x2 [q_size, hidden_size_half]

q_proj_scalefp8_e4m3 [q_size, hidden_blocks]

q_proj_global_decode_scalefp32 scalar

k_proj_weight_fp4float4_e2m1fn_x2 [kv_size, hidden_size_half]

k_proj_scalefp8_e4m3 [kv_size, hidden_blocks]

k_proj_global_decode_scalefp32 scalar

v_proj_weight_fp4float4_e2m1fn_x2 [kv_size, hidden_size_half]

v_proj_scalefp8_e4m3 [kv_size, hidden_blocks]

v_proj_global_decode_scalefp32 scalar

q_norm_weightbf16 [head_dim]

k_norm_weightbf16 [head_dim]

rms_norm_epsfp32 scalar

query_statesbf16 [batch_size, num_attention_heads, seq_len, head_dim]

key_statesbf16 [batch_size, num_key_value_heads, seq_len, head_dim]

value_statesbf16 [batch_size, num_key_value_heads, seq_len, head_dim]

Axes and behavior

q_sizeq_size = num_attention_heads * head_dim

kv_sizekv_size = num_key_value_heads * head_dim

seq_lenvariable

head_dimconstant = 128

q_blocksq_blocks = q_size // 16

kv_blockskv_blocks = kv_size // 16

batch_sizevariable

hidden_sizeconstant = 4096

q_size_halfq_size_half = q_size // 2

kv_size_halfkv_size_half = kv_size // 2

hidden_blockshidden_blocks = hidden_size // 16

hidden_size_halfhidden_size_half = hidden_size // 2

num_attention_headsconstant = 32

num_key_value_headsconstant = 8

determinismunspecified

constraintsNo mutation or aliasing

Identity

alias028_nvfp4_attention_qkv_projection_with_ropemodelqwen3-8b-nvfp4sha2563a67f193e0aa…

Sources: NVIDIA SOL-ExecBench (2026-08-23)last observed 2026-08-23How records are decided JSON