Skip to content
KernelIndex
Search⌘K
Sign in

Masked softmax with attention dropout backward

22 eligible runs
attention

Backward pass for masked softmax with attention dropout. Computes gradients through dropout -> softmax -> masked_fill in reverse order.

Fastest reported
24.3µsmean · 77% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-19. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latency16 cases
seq_len
batch_size
n_heads
dtype
128642fp32128648fp3213188fp3225624fp32256416fp32256328fp3229328fp32373816fp32512132fp3251248fp32541216fp3261384fp32919116fp32204812fp322048168fp32409644fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 124.3 µsRIAC as well13NVIDIA B200 · env 226.7 µsDatabricks9

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 13 results · last observed 2026-08-19Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
24.3µs77% SOL
1.00×
Reported · license unknown · no source
2026-08-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
25.8µs76% SOL
1.06×
Reported · license unknown · no source
2026-07-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Databricks
26.8µs75% SOL
1.10×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Hyra
26.9µs75% SOL
1.11×
Reported · license unknown · no source
2026-07-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Recursive
27.3µs74% SOL
1.12×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
ac4k
27.6µs74% SOL
1.13×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
doubleAI
27.9µs74% SOL
1.15×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Geometric
28.1µs74% SOL
1.16×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Geometric
29.7µs72% SOL
1.22×
Reported · license unknown · no source
2026-08-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
TH3CHARLie | TileGym
31.5µs72% SOL
1.30×
Reported · license unknown · no source
2026-07-27

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Argus
32.2µs71% SOL
1.32×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Amir M. Mir | SF Tensor
33.3µs70% SOL
1.37×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Cursor
49.3µs61% SOL
2.03×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
RIAC as well
unknown
24.3µs
1.00×
Reported
license unknown · no source
Hyra
unknown
26.9µs
1.11×
Reported
license unknown · no source
ac4k
unknown
27.3µs
1.12×
Reported
license unknown · no source
doubleAI
unknown
27.7µs
1.14×
Reported
license unknown · no source
Geometric
unknown
28.1µs
1.16×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
31.5µs
1.30×
Reported
license unknown · no source
Argus
unknown
32.2µs
1.32×
Reported
license unknown · no source
Cursor
unknown
48.4µs
1.99×
Reported
license unknown · no source
Show all 22 implementations ›
Databricks
unknown
26.8µs
1.10×
Reported
license unknown · no source
Recursive
unknown
27.3µs
1.12×
Reported
license unknown · no source
ac4k
unknown
27.6µs
1.13×
Reported
license unknown · no source
doubleAI
unknown
27.9µs
1.15×
Reported
license unknown · no source
Geometric
unknown
29.7µs
1.22×
Reported
license unknown · no source
Argus
unknown
32.5µs
1.34×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
32.7µs
1.35×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
33.3µs
1.37×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
33.8µs
1.39×
Reported
license unknown · no source
Cursor
unknown
49.3µs
2.03×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
59.8µs
2.46×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
25.8µs
1.06×
Reported
license unknown · no source
Databricks
unknown
26.7µs
1.10×
Reported
license unknown · no source
Recursive
unknown
27.0µs
1.11×
Reported
license unknown · no source

Semantics

Inputs and outputs
grad_outputfp32 [batch_size, n_heads, seq_len, seq_len]
p_attnfp32 [batch_size, n_heads, seq_len, seq_len]
maskbool [batch_size, 1, seq_len, seq_len]
dropout_maskbool [batch_size, n_heads, seq_len, seq_len]
p_dropoutfp32 scalar
grad_scoresfp32 [batch_size, n_heads, seq_len, seq_len]
Axes and behavior
n_headsvariable
seq_lenvariable
batch_sizevariable
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias066_masked_softmax_with_attention_dropout_backwardmodelopenvoicesha2569fca7062c8e4…
Sources: NVIDIA SOL-ExecBench (2026-08-19)last observed 2026-08-19How records are decidedJSON