Skip to content
KernelIndex
Search⌘K
Sign in

Attention output projection with reshape backward

53 eligible runs
other

Backward pass for attention output projection with reshape. Computes gradients w.r.t. attn_output and weight from grad_output. Forward: transpose (B,H,S,D)->(B,S,H,D), reshape (B,S,H,D)->(B,S,H*D), linear projection.

Fastest reported
39.4µsmean · 75% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-10. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 64 · num_heads = 32 · hidden_size = 204816 cases
batch_size
seq_len
dtype
1256bf1611024bf1621024bf1624096bf164128bf164256bf164512bf1641024bf168128bf168256bf1681024bf1616128bf1616256bf1632128bf1632256bf1664128bf16
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 139.4 µsRIAC as well28NVIDIA B200 · env 244.4 µsRecursive25

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 28 results · last observed 2026-08-16Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
39.4µs75% SOL
1.00×
Reported · license unknown · no source
2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
42.1µs73% SOL
1.07×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
OpenKernels-Ruibo
43.0µs73% SOL
1.09×
Reported · license unknown · no source
2026-07-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Geometric
44.5µs70% SOL
1.13×
Reported · license unknown · no source
2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Hyra
44.6µs70% SOL
1.13×
Reported · license unknown · no source
2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Amir M. Mir | SF Tensor
45.3µs69% SOL
1.15×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Recursive
45.8µs69% SOL
1.16×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
ac4k
46.5µs68% SOL
1.18×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
OpenKernels-Ruibo
47.6µs67% SOL
1.21×
Reported · license unknown · no source
2026-07-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Amir M. Mir | SF Tensor
47.6µs67% SOL
1.21×
Reported · license unknown · no source
2026-07-30

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Amir M. Mir | SF Tensor
47.6µs67% SOL
1.21×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
OpenKernels-Ruibo
47.8µs67% SOL
1.22×
Reported · license unknown · no source
2026-07-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
OpenKernels-Ruibo
49.1µs65% SOL
1.25×
Reported · license unknown · no source
2026-07-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Databricks
49.7µs66% SOL
1.26×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Igor Bragaia
49.7µs65% SOL
1.26×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
Amir M. Mir | SF Tensor
49.9µs65% SOL
1.27×
Reported · license unknown · no source
2026-07-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Geometric
50.5µs64% SOL
1.28×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
OpenKernels-Ruibo
51.0µs64% SOL
1.30×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
19
xuanzhengdu
53.1µs62% SOL
1.35×
Reported · license unknown · no source
2026-08-07

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
20
Amir M. Mir | SF Tensor
54.2µs61% SOL
1.38×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
21
xuanzhengdu
54.4µs62% SOL
1.38×
Reported · license unknown · no source
2026-08-07

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
22
multiagent
54.4µs62% SOL
1.38×
Reported · license unknown · no source
2026-08-07

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
23
TH3CHARLie | TileGym
56.6µs60% SOL
1.44×
Reported · license unknown · no source
2026-07-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
24
Geometric
59.1µs59% SOL
1.50×
Reported · license unknown · no source
2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
25
Geometric
59.2µs58% SOL
1.50×
Reported · license unknown · no source
2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
26
doubleAI
59.7µs56% SOL
1.52×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
27
Cursor
61.5µs56% SOL
1.56×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
28
Runic Kingfisher
98.6µs39% SOL
2.50×
Reported · license unknown · no source
2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Amir M. Mir | SF Tensor
unknown
42.1µs
1.07×
Reported
license unknown · no source
Geometric
unknown
44.5µs
1.13×
Reported
license unknown · no source
Recursive
unknown
44.4µs
1.13×
Reported
license unknown · no source
Databricks
unknown
46.2µs
1.17×
Reported
license unknown · no source
ac4k
unknown
45.0µs
1.14×
Reported
license unknown · no source
Igor Bragaia
unknown
47.7µs
1.21×
Reported
license unknown · no source
doubleAI
unknown
48.0µs
1.22×
Reported
license unknown · no source
Bragaia
unknown
49.8µs
1.27×
Reported
license unknown · no source
Show all 53 implementations ›
AKO4ALL
unknown
50.9µs
1.29×
Reported
license unknown · no source
xuanzhengdu
unknown
53.1µs
1.35×
Reported
license unknown · no source
multiagent
unknown
54.4µs
1.38×
Reported
license unknown · no source
Argus
unknown
53.7µs
1.36×
Reported
license unknown · no source
Cursor
unknown
55.2µs
1.40×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
56.6µs
1.44×
Reported
license unknown · no source
Zerg AI
unknown
57.3µs
1.46×
Reported
license unknown · no source
wychi888
unknown
58.0µs
1.47×
Reported
license unknown · no source
Gilded Quail
unknown
82.0µs
2.08×
Reported
license unknown · no source
Clever Scorpion
unknown
79.0µs
2.01×
Reported
license unknown · no source
Theoric.com
unknown
91.5µs
2.33×
Reported
license unknown · no source
Runic Kingfisher
unknown
98.6µs
2.50×
Reported
license unknown · no source
Quick Impala
unknown
595.8µs
15.14×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
45.3µs
1.15×
Reported
license unknown · no source
Recursive
unknown
45.8µs
1.16×
Reported
license unknown · no source
ac4k
unknown
46.5µs
1.18×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
47.6µs
1.21×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
47.6µs
1.21×
Reported
license unknown · no source
OpenKernels-Ruibo
unknown
47.6µs
1.21×
Reported
license unknown · no source
OpenKernels-Ruibo
unknown
47.8µs
1.22×
Reported
license unknown · no source
Igor Bragaia
unknown
48.8µs
1.24×
Reported
license unknown · no source
Databricks
unknown
49.7µs
1.26×
Reported
license unknown · no source
OpenKernels-Ruibo
unknown
49.1µs
1.25×
Reported
license unknown · no source
Bragaia
unknown
49.9µs
1.27×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
49.9µs
1.27×
Reported
license unknown · no source
Igor Bragaia
unknown
49.7µs
1.26×
Reported
license unknown · no source
doubleAI
unknown
50.8µs
1.29×
Reported
license unknown · no source
Geometric
unknown
50.5µs
1.28×
Reported
license unknown · no source
OpenKernels-Ruibo
unknown
51.0µs
1.30×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
51.4µs
1.31×
Reported
license unknown · no source
Databricks
unknown
51.7µs
1.31×
Reported
license unknown · no source
xuanzhengdu
unknown
54.4µs
1.38×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
54.2µs
1.38×
Reported
license unknown · no source
Zerg AI
unknown
57.0µs
1.45×
Reported
license unknown · no source
Geometric
unknown
59.1µs
1.50×
Reported
license unknown · no source
Geometric
unknown
59.2µs
1.50×
Reported
license unknown · no source
doubleAI
unknown
59.7µs
1.52×
Reported
license unknown · no source
Cursor
unknown
61.5µs
1.56×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
69.0µs
1.75×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
71.5µs
1.82×
Reported
license unknown · no source
Zerg AI
unknown
70.1µs
1.78×
Reported
license unknown · no source
Igor Bragaia
unknown
71.6µs
1.82×
Reported
license unknown · no source
RIAC as well
unknown
39.4µs
1.00×
Reported
license unknown · no source
OpenKernels-Ruibo
unknown
43.0µs
1.09×
Reported
license unknown · no source
Hyra
unknown
44.6µs
1.13×
Reported
license unknown · no source

Semantics

Inputs and outputs
grad_outputbf16 [batch_size, seq_len, hidden_size]
reshapedbf16 [batch_size, seq_len, hidden_size]
weightbf16 [hidden_size, hidden_size]
grad_attn_outputbf16 [batch_size, num_heads, seq_len, head_dim]
grad_weightbf16 [hidden_size, hidden_size]
Axes and behavior
seq_lenvariable
head_dimconstant = 64
num_headsconstant = 32
batch_sizevariable
hidden_sizeconstant = 2048
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias004_attention_output_projection_with_reshape_backwardmodellfm2-1-2bsha2562bef7b0c188a…
Sources: NVIDIA SOL-ExecBench (2026-08-16)last observed 2026-08-16How records are decidedJSON