Skip to content
KernelIndex
Search⌘K
Sign in

NVFP4 grouped query attention

23 eligible runs
quantization

FP4-quantized Grouped Query Attention (GQA) with 64 query heads and 8 key-value heads (8:1 ratio). Performs Q/K/V projections with FP4 weights, applies RoPE embeddings, computes attention with repeat_kv for GQA, and produces output via FP4 output projection.

Fastest reported
540.5µsmean · 90% of SOL
LaserLicense unknown · unknown

Reported evidence · last observed 2026-08-23. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 128 · q_out_half = 4096 · hidden_half = 4096 · hidden_size = 8192 · q_out_blocks = 512 · hidden_blocks = 512 · q_out_features = 8192 · kv_out_features = 1024 · num_attention_heads = 64 · num_key_value_heads = 816 cases
batch_size
seq_len
dtype
11024bf16/fp4/fp8_e4m32128bf16/fp4/fp8_e4m32256bf16/fp4/fp8_e4m32512bf16/fp4/fp8_e4m322048bf16/fp4/fp8_e4m324096bf16/fp4/fp8_e4m34128bf16/fp4/fp8_e4m34256bf16/fp4/fp8_e4m342048bf16/fp4/fp8_e4m38256bf16/fp4/fp8_e4m38512bf16/fp4/fp8_e4m381024bf16/fp4/fp8_e4m316256bf16/fp4/fp8_e4m332128bf16/fp4/fp8_e4m332256bf16/fp4/fp8_e4m364128bf16/fp4/fp8_e4m3
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1540.5 µsLaser17NVIDIA B200 · env 2712.7 µsRecursive6

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 17 results · last observed 2026-08-23Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
Laser
540.5µs90% SOL
1.00×
Reported · license unknown · no source
2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
575.1µs89% SOL
1.06×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
ac4k
603.1µs89% SOL
1.12×
Reported · license unknown · no source
2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Geometric
702.4µs87% SOL
1.30×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Geometric
703.4µs87% SOL
1.30×
Reported · license unknown · no source
2026-08-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Hyra
712.5µs87% SOL
1.32×
Reported · license unknown · no source
2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Recursive
728.7µs86% SOL
1.35×
Reported · license unknown · no source
2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Yatimai
761.2µs86% SOL
1.41×
Reported · license unknown · no source
2026-07-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
doubleAI
783.7µs85% SOL
1.45×
Reported · license unknown · no source
2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Geometric
892.3µs83% SOL
1.65×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Geometric
923.7µs83% SOL
1.71×
Reported · license unknown · no source
2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Geometric
954.4µs82% SOL
1.77×
Reported · license unknown · no source
2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Geometric
1.32ms76% SOL
2.45×
Reported · license unknown · no source
2026-08-05

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Geometric
1.92ms68% SOL
3.56×
Reported · license unknown · no source
2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Geometric
2.14ms66% SOL
3.96×
Reported · license unknown · no source
2026-07-26

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
Geometric
2.15ms66% SOL
3.97×
Reported · license unknown · no source
2026-07-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Amir M. Mir | SF Tensor
3.10ms56% SOL
5.73×
Reported · license unknown · no source
2026-07-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Amir M. Mir | SF Tensor
unknown
575.1µs
1.06×
Reported
license unknown · no source
ac4k
unknown
603.1µs
1.12×
Reported
license unknown · no source
Geometric
unknown
702.4µs
1.30×
Reported
license unknown · no source
Laser
unknown
540.5µs
1.00×
Reported
license unknown · no source
Hyra
unknown
712.5µs
1.32×
Reported
license unknown · no source
doubleAI
unknown
723.9µs
1.34×
Reported
license unknown · no source
Recursive
unknown
712.7µs
1.32×
Reported
license unknown · no source
Yatimai
unknown
761.2µs
1.41×
Reported
license unknown · no source
Show all 23 implementations ›
Cursor
unknown
4.28ms
7.91×
Reported
license unknown · no source
Geometric
unknown
703.4µs
1.30×
Reported
license unknown · no source
Recursive
unknown
728.7µs
1.35×
Reported
license unknown · no source
doubleAI
unknown
783.7µs
1.45×
Reported
license unknown · no source
Geometric
unknown
892.3µs
1.65×
Reported
license unknown · no source
Geometric
unknown
923.7µs
1.71×
Reported
license unknown · no source
Geometric
unknown
954.4µs
1.77×
Reported
license unknown · no source
doubleAI
unknown
1.06ms
1.96×
Reported
license unknown · no source
Geometric
unknown
1.32ms
2.45×
Reported
license unknown · no source
Geometric
unknown
1.92ms
3.56×
Reported
license unknown · no source
Geometric
unknown
2.14ms
3.96×
Reported
license unknown · no source
Geometric
unknown
2.15ms
3.97×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
3.09ms
5.73×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
3.10ms
5.73×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
12.5ms
23.13×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesbf16 [batch_size, seq_len, hidden_size]
cosbf16 [batch_size, seq_len, head_dim]
sinbf16 [batch_size, seq_len, head_dim]
q_weight_fp4float4_e2m1fn_x2 [q_out_features, hidden_half]
q_scalefp8_e4m3 [q_out_features, hidden_blocks]
q_global_decode_scalefp32 scalar
k_weight_fp4float4_e2m1fn_x2 [kv_out_features, hidden_half]
k_scalefp8_e4m3 [kv_out_features, hidden_blocks]
k_global_decode_scalefp32 scalar
v_weight_fp4float4_e2m1fn_x2 [kv_out_features, hidden_half]
v_scalefp8_e4m3 [kv_out_features, hidden_blocks]
v_global_decode_scalefp32 scalar
o_weight_fp4float4_e2m1fn_x2 [hidden_size, q_out_half]
o_scalefp8_e4m3 [hidden_size, q_out_blocks]
o_global_decode_scalefp32 scalar
attn_outputbf16 [batch_size, seq_len, hidden_size]
Axes and behavior
seq_lenvariable
head_dimconstant = 128
batch_sizevariable
q_out_halfq_out_half = q_out_features // 2
hidden_halfhidden_half = hidden_size // 2
hidden_sizeconstant = 8192
q_out_blocksq_out_blocks = q_out_features // 16
hidden_blockshidden_blocks = hidden_size // 16
q_out_featuresq_out_features = num_attention_heads * head_dim
kv_out_featureskv_out_features = num_key_value_heads * head_dim
num_attention_headsconstant = 64
num_key_value_headsconstant = 8
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias019_nvfp4_grouped_query_attentionmodelnova-70b-nvfp4sha2569c5ef184ee8d…
Sources: NVIDIA SOL-ExecBench (2026-08-23)last observed 2026-08-23How records are decidedJSON