Cu seqlens variable length vision attention
Variable-length vision attention using cumulative sequence lengths (cu_seqlens). Processes multiple images/videos in a single batch by splitting Q/K/V according to cu_seqlens boundaries, applying rotary position embeddings, performing separate attention computations for each sequence, then concatenating results. Used in vision transformer layers for efficient batched inference with mixed image/video inputs.
Reported evidence · last observed 2026-08-16. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyqkv_dim = 3456 · head_dim = 72 · embed_dim = 1152 · num_heads = 1616 cases
Not measured on H100 for this workload. Challenges →
151.8µs76% SOL1.00×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
257.0µs75% SOL1.10×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
358.5µs74% SOL1.13×Reported · license unknown · no source2026-07-18
Measured exactly what you asked. Reported by source; not independently reproduced.
463.3µs72% SOL1.22×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
566.3µs70% SOL1.28×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
669.9µs69% SOL1.35×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
770.0µs71% SOL1.35×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
874.1µs68% SOL1.43×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
975.6µs68% SOL1.46×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
1099.3µs61% SOL1.92×Reported · license unknown · no source2026-08-05
Measured exactly what you asked. Reported by source; not independently reproduced.
11109.5µs60% SOL2.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
12273.8µs37% SOL5.29×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
13275.0µs37% SOL5.31×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
14284.4µs36% SOL5.49×Reported · license unknown · no source2026-07-26
Measured exactly what you asked. Reported by source; not independently reproduced.
15294.1µs36% SOL5.68×Reported · license unknown · no source2026-07-24
Measured exactly what you asked. Reported by source; not independently reproduced.
16419.8µs27% SOL8.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.