GQA ragged prefill causal h32 kv8 d128
Batched Grouped Query Attention prefill with ragged (variable-length) inputs. Causal mask is applied. Captured from Llama-3.1-8B during total prefill.
Reported evidence · last observed 2026-08-07. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 21 cases · mean latency25 cases
Not measured on H100 for this workload. Challenges →
17.50µs92% SOL1.00×Reported · license unknown · no source2026-08-07
Measured exactly what you asked. Reported by source; not independently reproduced.
27.53µs92% SOL1.00×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
37.77µs92% SOL1.04×Reported · license unknown · no source2026-07-30
Measured exactly what you asked. Reported by source; not independently reproduced.
48.19µs92% SOL1.09×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
58.70µs91% SOL1.16×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
68.90µs91% SOL1.19×Reported · license unknown · no source2026-07-30
Measured exactly what you asked. Reported by source; not independently reproduced.
79.08µs91% SOL1.21×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
89.15µs91% SOL1.22×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
99.16µs91% SOL1.22×Reported · license unknown · no source2026-08-05
Measured exactly what you asked. Reported by source; not independently reproduced.
109.19µs91% SOL1.23×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
119.20µs91% SOL1.23×Reported · license unknown · no source2026-08-01
Measured exactly what you asked. Reported by source; not independently reproduced.
129.24µs91% SOL1.23×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
139.27µs90% SOL1.24×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
149.48µs90% SOL1.26×Reported · license unknown · no source2026-08-01
Measured exactly what you asked. Reported by source; not independently reproduced.
159.66µs90% SOL1.29×Reported · license unknown · no source2026-07-31
Measured exactly what you asked. Reported by source; not independently reproduced.
169.90µs90% SOL1.32×Reported · license unknown · no source2026-07-18
Measured exactly what you asked. Reported by source; not independently reproduced.
1710.0µs90% SOL1.34×Reported · license unknown · no source2026-07-30
Measured exactly what you asked. Reported by source; not independently reproduced.
1810.1µs89% SOL1.35×Reported · license unknown · no source2026-07-30
Measured exactly what you asked. Reported by source; not independently reproduced.
1910.2µs89% SOL1.36×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
2010.3µs90% SOL1.38×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
2110.9µs88% SOL1.46×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
2210.9µs88% SOL1.46×Reported · license unknown · no source2026-07-25
Measured exactly what you asked. Reported by source; not independently reproduced.
2311.6µs88% SOL1.55×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
2411.7µs88% SOL1.56×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
2511.9µs87% SOL1.59×Reported · license unknown · no source2026-07-29
Measured exactly what you asked. Reported by source; not independently reproduced.
2612.1µs88% SOL1.61×Reported · license unknown · no source2026-07-23
Measured exactly what you asked. Reported by source; not independently reproduced.
2712.7µs87% SOL1.69×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2814.3µs86% SOL1.91×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2914.7µs85% SOL1.95×Reported · license unknown · no source2026-07-29
Measured exactly what you asked. Reported by source; not independently reproduced.
3021.2µs77% SOL2.83×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
3122.0µs80% SOL2.93×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
3224.9µs77% SOL3.32×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
Implementations
Show all 54 implementations ›Showing all 54 implementations ⌄
Semantics
Also indexed as gqa-ragged-prefill-causal-h32-kv8-d128 · reviewed as the same computation; every definition's cohorts are shown here, ranked separately.