Skip to content
KernelIndex
Search⌘K
Sign in

Time decay exponential stabilization

22 eligible runs
ssm

RWKV's time decay exponential computation with numerical stabilization for the WKV attention mechanism. Computes -exp(time_decay) and manages exponential stability through max-state tracking for linear-time attention.

Fastest reported
59.8µsmean · 100% of SOL
doubleAILicense unknown · unknown

Reported evidence · last observed 2026-07-10. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyattention_hidden_size = 204816 cases
seq_len
batch_size
dtype
1281fp321311fp322562fp3244916fp325124fp3254132fp3261364fp326911fp329198fp3210248fp3214898fp32172164fp3218792fp3219734fp32213116fp32409632fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 159.8 µsdoubleAI12NVIDIA B200 · env 259.8 µsdoubleAI10

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 12 results · last observed 2026-08-14Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
doubleAI
59.8µs100% SOL
1.00×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Hyra
64.0µs100% SOL
1.07×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Databricks
65.1µs100% SOL
1.09×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Amir M. Mir | SF Tensor
68.2µs100% SOL
1.14×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Geometric
70.6µs100% SOL
1.18×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Geometric
70.7µs100% SOL
1.18×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
ac4k
70.9µs100% SOL
1.19×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
TH3CHARLie | TileGym
73.6µs100% SOL
1.23×
Reported · license unknown · no source
2026-07-28

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Recursive
85.8µs100% SOL
1.43×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Cursor
244.1µs99% SOL
4.08×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Geometric
534.8µs99% SOL
8.94×
Reported · license unknown · no source
2026-07-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Qin Liu @ UCD
543.6µs99% SOL
9.08×
Reported · license unknown · no source
2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Recursive
unknown
63.0µs
1.05×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
67.9µs
1.13×
Reported
license unknown · no source
Hyra
unknown
64.0µs
1.07×
Reported
license unknown · no source
Geometric
unknown
70.7µs
1.18×
Reported
license unknown · no source
ac4k
unknown
70.9µs
1.19×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
73.6µs
1.23×
Reported
license unknown · no source
Argus
unknown
128.4µs
2.15×
Reported
license unknown · no source
Cursor
unknown
244.1µs
4.08×
Reported
license unknown · no source
Show all 22 implementations ›
Qin Liu @ UCD
unknown
543.6µs
9.08×
Reported
license unknown · no source
Databricks
unknown
65.6µs
1.10×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
68.2µs
1.14×
Reported
license unknown · no source
ac4k
unknown
71.1µs
1.19×
Reported
license unknown · no source
Geometric
unknown
70.6µs
1.18×
Reported
license unknown · no source
doubleAI
unknown
78.7µs
1.31×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
85.7µs
1.43×
Reported
license unknown · no source
ac4k
unknown
79.2µs
1.32×
Reported
license unknown · no source
Recursive
unknown
85.8µs
1.43×
Reported
license unknown · no source
Cursor
unknown
245.7µs
4.11×
Reported
license unknown · no source
Geometric
unknown
534.8µs
8.94×
Reported
license unknown · no source
doubleAI
unknown
59.8µs
1.00×
Reported
license unknown · no source
doubleAI
unknown
59.8µs
1.00×
Reported
license unknown · no source
Databricks
unknown
65.1µs
1.09×
Reported
license unknown · no source

Semantics

Inputs and outputs
time_decayfp32 [attention_hidden_size]
keyfp32 [batch_size, seq_len, attention_hidden_size]
time_firstfp32 [attention_hidden_size]
valuefp32 [batch_size, seq_len, attention_hidden_size]
max_statefp32 [batch_size, attention_hidden_size]
num_statefp32 [batch_size, attention_hidden_size]
den_statefp32 [batch_size, attention_hidden_size]
outputfp32 [batch_size, seq_len, attention_hidden_size]
max_state_outfp32 [batch_size, attention_hidden_size]
num_state_outfp32 [batch_size, attention_hidden_size]
den_state_outfp32 [batch_size, attention_hidden_size]
Axes and behavior
seq_lenvariable
batch_sizevariable
attention_hidden_sizeconstant = 2048
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias094_time_decay_exponential_stabilizationmodelrwkv-v7-1-5b-g1-ggufsha256597b49a62e85…
Sources: NVIDIA SOL-ExecBench (2026-08-14)last observed 2026-08-14How records are decidedJSON