Skip to content
KernelIndex
Search⌘K
Sign in

KV cache update with RoPE

22 eligible runs
attention

Fused KV cache update with RoPE application. Applies rotary position embeddings to incoming key states and updates both key and value caches in a single operation. Critical for autoregressive generation performance.

Fastest reported
5.19µsmean · 94% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-20. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 128 · num_kv_heads = 1016 cases
current_seq_len
batch_size
new_seq_len
dtype
011bf1601128bf1601512bf1602512bf16021024bf1604256bf1612711bf1613021bf16255321bf16255641bf1637241bf1651021bf1651111bf1676741bf16996161bf16204711bf16
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 15.19 µsRIAC as well11NVIDIA B200 · env 25.46 µsRecursive11

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 11 results · last observed 2026-08-20Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
5.19µs94% SOL
1.00×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Recursive
5.43µs93% SOL
1.05×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
TH3CHARLie | TileGym
5.55µs93% SOL
1.07×
Reported · license unknown · no source
2026-07-27

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
doubleAI
5.57µs93% SOL
1.07×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
TH3CHARLie | TileGym
5.70µs93% SOL
1.10×
Reported · license unknown · no source
2026-07-27

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
ac4k
5.70µs93% SOL
1.10×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Databricks
5.71µs93% SOL
1.10×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Amir M. Mir | SF Tensor
5.80µs93% SOL
1.12×
Reported · license unknown · no source
2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Hyra
5.87µs92% SOL
1.13×
Reported · license unknown · no source
2026-07-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Amir M. Mir | SF Tensor
6.04µs92% SOL
1.16×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Cursor
14.4µs76% SOL
2.77×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
RIAC as well
unknown
5.19µs
1.00×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
5.55µs
1.07×
Reported
license unknown · no source
ac4k
unknown
5.58µs
1.08×
Reported
license unknown · no source
Databricks
unknown
5.71µs
1.10×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
5.80µs
1.12×
Reported
license unknown · no source
Hyra
unknown
5.87µs
1.13×
Reported
license unknown · no source
Argus
unknown
6.28µs
1.21×
Reported
license unknown · no source
Cursor
unknown
14.0µs
2.70×
Reported
license unknown · no source
Show all 22 implementations ›
doubleAI
unknown
5.57µs
1.07×
Reported
license unknown · no source
ac4k
unknown
5.70µs
1.10×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
5.70µs
1.10×
Reported
license unknown · no source
Databricks
unknown
5.78µs
1.11×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
6.04µs
1.16×
Reported
license unknown · no source
doubleAI
unknown
6.00µs
1.16×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
6.18µs
1.19×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
10.8µs
2.08×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
12.5µs
2.40×
Reported
license unknown · no source
Cursor
unknown
14.4µs
2.77×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
25.5µs
4.91×
Reported
license unknown · no source
doubleAI
unknown
5.55µs
1.07×
Reported
license unknown · no source
Recursive
unknown
5.43µs
1.05×
Reported
license unknown · no source
Recursive
unknown
5.46µs
1.05×
Reported
license unknown · no source

Semantics

Inputs and outputs
key_statesbf16 [batch_size, num_kv_heads, new_seq_len, head_dim]
value_statesbf16 [batch_size, num_kv_heads, new_seq_len, head_dim]
cosbf16 [batch_size, 1, new_seq_len, head_dim]
sinbf16 [batch_size, 1, new_seq_len, head_dim]
key_cachebf16 [batch_size, num_kv_heads, current_seq_len, head_dim]
value_cachebf16 [batch_size, num_kv_heads, current_seq_len, head_dim]
updated_key_cachebf16 [batch_size, num_kv_heads, updated_seq_len, head_dim]
updated_value_cachebf16 [batch_size, num_kv_heads, updated_seq_len, head_dim]
Axes and behavior
head_dimconstant = 128
batch_sizevariable
new_seq_lenvariable
num_kv_headsconstant = 10
current_seq_lenvariable
updated_seq_lenupdated_seq_len = current_seq_len + new_seq_len
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias071_kv_cache_update_with_ropemodelphi-4-reasoning-plus-nvfp4sha256c5e742b41204…
Sources: NVIDIA SOL-ExecBench (2026-08-20)last observed 2026-08-20How records are decidedJSON