NVFP4 grouped query attention with KV repeat
FP4-quantized Grouped Query Attention (GQA) with KV cache repeat operation. Implements attention with 40 query heads and 8 key-value heads (5x repeat factor). Uses FP4 block-scaled quantization for Q@K^T and Attn@V matrix multiplications with softmax in between. Note: seq_len must be a multiple of 128 for FP4 quantization compatibility.
Reported evidence · last observed 2026-08-20. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 128 · num_attention_heads = 40 · num_key_value_heads = 8 · num_key_value_groups = 516 cases
Not measured on H100 for this workload. Challenges →
1290.0µs99% SOL1.00×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
2297.2µs99% SOL1.02×Reported · license unknown · no source2026-07-23
Measured exactly what you asked. Reported by source; not independently reproduced.
3301.6µs99% SOL1.04×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
4321.1µs99% SOL1.11×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
5324.0µs99% SOL1.12×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
6332.0µs99% SOL1.14×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
7406.6µs98% SOL1.40×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
8411.7µs98% SOL1.42×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
9432.7µs98% SOL1.49×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
10503.7µs97% SOL1.74×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
11530.2µs97% SOL1.83×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
12598.2µs96% SOL2.06×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
13607.4µs96% SOL2.09×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
14707.1µs96% SOL2.44×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
15745.1µs95% SOL2.57×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
16881.7µs95% SOL3.04×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
17885.3µs94% SOL3.05×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
18947.3µs94% SOL3.27×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
19967.7µs94% SOL3.34×Reported · license unknown · no source2026-08-04
Measured exactly what you asked. Reported by source; not independently reproduced.
201.04ms93% SOL3.57×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
211.06ms93% SOL3.65×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
221.33ms91% SOL4.57×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
231.66ms89% SOL5.72×Reported · license unknown · no source2026-07-18
Measured exactly what you asked. Reported by source; not independently reproduced.
242.11ms86% SOL7.28×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
252.46ms82% SOL8.48×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
263.52ms79% SOL12.13×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
278.29ms61% SOL28.59×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2890.7ms14% SOL312.76×Reported · license unknown · no source2026-08-05
Measured exactly what you asked. Reported by source; not independently reproduced.