GQA paged prefill causal h5 kv1 d128 ps1
14 eligible runs
gqa-paged-attention
Batched Grouped Query Attention prefill with a paged KV cache. Causal mask is applied. Captured from Llama 4 Scout 17B-16E with tensor parallel size 8 (40/8=5 q-heads, 8/8=1 kv-head).
Source baseline · unbeaten
105.3µsmean
flashinfer / wrapper484d2cFlashInfer-Bench baselines · Apache-2.0 · python
Reproduction-ready evidence · last observed 2026-04-12. The source's designated baseline implementation. Reported by source; not independently reproduced.
Current records
Workloadtotal_q = 1 · num_pages = 167 · len_indptr = 2 · num_kv_indices = 33 · bf16total_q = 14355 · num_pages = 14522 · len_indptr = 44 · num_kv_indices = 14398 · bf16total_q = 11809 · num_pages = 26331 · len_indptr = 21 · num_kv_indices = 11829 · bf16total_q = 16 · num_pages = 33288 · len_indptr = 2 · num_kv_indices = 19 · bf16total_q = 1 · num_pages = 33417 · len_indptr = 2 · num_kv_indices = 19 · bf16total_q = 10847 · num_pages = 44264 · len_indptr = 55 · num_kv_indices = 10945 · bf16total_q = 8846 · num_pages = 53110 · len_indptr = 10 · num_kv_indices = 8860 · bf16
Not measured on H100, B200 for this workload. Challenges →
Source-native comparison · GPU NVIDIA GB200 · Workload total_q = 1 · num_pages = 167 · len_indptr = 2 · num_kv_indices = 33 · bf16 · CUDA 13.0 · Framework pytorch 2.9.1+cu130 · Protocol flashinfer-bench · mean · 2 results · last observed 2026-04-12Record history →
Estimated floor 11 ns · record 9690.42× above itestimate, not evidence ›
DRAM 11 ns · bandwidth-bound on GB200
every declared tensor crosses HBM exactly once (8,000 GB/s, GB200 datasheet)
no arithmetic formula for this family: bandwidth floor only
headroom-v1: a lower bound from declared tensors and datasheet peaks. A kernel can sit well above it for good reasons.
#
Implementation
Latency
vs #1
Trust
Observed
1105.3µs1.00×Reproduction-ready · Apache-2.0 · source2026-04-12
Measured exactly what you asked. The source's designated baseline implementation. Reported by source; not independently reproduced.
source mirroredApache-2.0no install recipeView source →Run detail →
2113.3µs1.08×Reproduction-ready · Apache-2.0 · source2026-04-12
Measured exactly what you asked. The source's designated baseline implementation. Reported by source; not independently reproduced.
source mirroredApache-2.0no install recipeView source →Run detail →
Implementations
Implementation
Runtime
Best latency
Evidence
Availability
flashinfer / wrapper484d2cFlashInfer-Bench baselines
python
105.3µs
1.00×
Reproduction-ready
Apache-2.0 · source
Semantics
Inputs and outputs
qbf16 [total_q, num_qo_heads, head_dim]
k_cachebf16 [num_pages, page_size, num_kv_heads, head_dim]
v_cachebf16 [num_pages, page_size, num_kv_heads, head_dim]
qo_indptrint32 [len_indptr]
kv_indptrint32 [len_indptr]
kv_indicesint32 [num_kv_indices]
sm_scalefp32 scalar
outputbf16 [total_q, num_qo_heads, head_dim]
lsefp32 [total_q, num_qo_heads]
Axes and behavior
total_qvariable
head_dimconstant = 128
num_pagesvariable
page_sizeconstant = 1
len_indptrvariable
num_kv_headsconstant = 1
num_qo_headsconstant = 5
num_kv_indicesvariable
determinismunspecified
constraintsNo mutation or aliasing
Identity
Sources: FlashInfer-Bench (2026-04-12) · Apache-2.0last observed 2026-04-12How records are decidedJSON