Skip to content
KernelIndex
Search⌘K
Sign in

FFN GELU projection fused backward

30 eligible runs
mlp

Backward pass for fused FFN block with GELU activation, residual connection, and layer normalization. Computes gradients for hidden_states, fc1_weight, fc1_bias, fc2_weight, fc2_bias, ln_weight, and ln_bias.

Fastest reported
115.1µsmean · 75% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-18. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhidden_size = 512 · intermediate_size = 204816 cases
seq_len
batch_size
dtype
1281fp321311fp322112fp323738fp3244916fp3254132fp326131fp327734fp328538fp3291916fp32108764fp3211631fp3214238fp3218012fp3218794fp32409632fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1115.1 µsRIAC as well23NVIDIA B200 · env 2130.2 µsdoubleAI7

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 23 results · last observed 2026-08-24Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
115.1µs75% SOL
1.00×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
120.3µs74% SOL
1.04×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
doubleAI
133.8µs71% SOL
1.16×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
ac4k
149.4µs68% SOL
1.30×
Reported · license unknown · no source
2026-07-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Rapid Tiger
149.9µs68% SOL
1.30×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Gleaming Coyote
150.2µs68% SOL
1.30×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Electric Cheetah
150.5µs68% SOL
1.31×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Hyra
157.5µs68% SOL
1.37×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Amir M. Mir | SF Tensor
158.6µs67% SOL
1.38×
Reported · license unknown · no source
2026-07-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Amir M. Mir | SF Tensor
162.3µs67% SOL
1.41×
Reported · license unknown · no source
2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Geometric
183.3µs64% SOL
1.59×
Reported · license unknown · no source
2026-08-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Proud Wren @ NVIDIA
184.5µs63% SOL
1.60×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Geometric
187.7µs63% SOL
1.63×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Amir M. Mir | SF Tensor
228.0µs58% SOL
1.98×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Geometric
236.0µs57% SOL
2.05×
Reported · license unknown · no source
2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
Geometric
258.7µs54% SOL
2.25×
Reported · license unknown · no source
2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Geometric
263.8µs54% SOL
2.29×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
Geometric
347.6µs46% SOL
3.02×
Reported · license unknown · no source
2026-08-07

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
19
Geometric
360.5µs45% SOL
3.13×
Reported · license unknown · no source
2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
20
Geometric
436.7µs40% SOL
3.79×
Reported · license unknown · no source
2026-07-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
21
Cursor
577.8µs33% SOL
5.02×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
22
Geometric
769.3µs27% SOL
6.68×
Reported · license unknown · no source
2026-08-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
23
Runic Kingfisher
1.56ms19% SOL
13.54×
Reported · license unknown · no source
2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
RIAC as well
unknown
115.1µs
1.00×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
120.3µs
1.04×
Reported
license unknown · no source
Rapid Tiger
unknown
149.9µs
1.30×
Reported
license unknown · no source
Gleaming Coyote
unknown
150.2µs
1.30×
Reported
license unknown · no source
Electric Cheetah
unknown
150.5µs
1.31×
Reported
license unknown · no source
Hyra
unknown
157.5µs
1.37×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
180.5µs
1.57×
Reported
license unknown · no source
Geometric
unknown
183.3µs
1.59×
Reported
license unknown · no source
Show all 30 implementations ›
Recursive
unknown
216.8µs
1.88×
Reported
license unknown · no source
Cursor
unknown
528.1µs
4.59×
Reported
license unknown · no source
Runic Kingfisher
unknown
1.56ms
13.54×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
158.6µs
1.38×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
162.3µs
1.41×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
184.5µs
1.60×
Reported
license unknown · no source
Geometric
unknown
187.7µs
1.63×
Reported
license unknown · no source
doubleAI
unknown
186.6µs
1.62×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
226.5µs
1.97×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
228.0µs
1.98×
Reported
license unknown · no source
Geometric
unknown
236.0µs
2.05×
Reported
license unknown · no source
Geometric
unknown
258.7µs
2.25×
Reported
license unknown · no source
Geometric
unknown
263.8µs
2.29×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
316.3µs
2.75×
Reported
license unknown · no source
Geometric
unknown
347.6µs
3.02×
Reported
license unknown · no source
Geometric
unknown
360.5µs
3.13×
Reported
license unknown · no source
Geometric
unknown
436.7µs
3.79×
Reported
license unknown · no source
Cursor
unknown
577.8µs
5.02×
Reported
license unknown · no source
Geometric
unknown
769.3µs
6.68×
Reported
license unknown · no source
doubleAI
unknown
130.2µs
1.13×
Reported
license unknown · no source
doubleAI
unknown
133.8µs
1.16×
Reported
license unknown · no source
ac4k
unknown
149.4µs
1.30×
Reported
license unknown · no source

Semantics

Inputs and outputs
grad_outputfp32 [batch_size, seq_len, hidden_size]
hidden_statesfp32 [batch_size, seq_len, hidden_size]
fc1_weightfp32 [intermediate_size, hidden_size]
fc1_outputfp32 [batch_size, seq_len, intermediate_size]
gelu_outputfp32 [batch_size, seq_len, intermediate_size]
fc2_weightfp32 [hidden_size, intermediate_size]
residual_outputfp32 [batch_size, seq_len, hidden_size]
normalizedfp32 [batch_size, seq_len, hidden_size]
varfp32 [batch_size, seq_len, 1]
ln_weightfp32 [hidden_size]
epsfp32 scalar
grad_hidden_statesfp32 [batch_size, seq_len, hidden_size]
grad_fc1_weightfp32 [intermediate_size, hidden_size]
grad_fc1_biasfp32 [intermediate_size]
grad_fc2_weightfp32 [hidden_size, intermediate_size]
grad_fc2_biasfp32 [hidden_size]
grad_ln_weightfp32 [hidden_size]
grad_ln_biasfp32 [hidden_size]
Axes and behavior
seq_lenvariable
batch_sizevariable
hidden_sizeconstant = 512
intermediate_sizeconstant = 2048
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias042_ffn_gelu_projection_fused_backwardmodelkokoro-82msha256f9004b1bfeaf…
Sources: NVIDIA SOL-ExecBench (2026-08-24)last observed 2026-08-24How records are decidedJSON