Skip to content
KernelIndex
Search⌘K
Sign in

GELU approximate feedforward backward

19 eligible runs
mlp

Backward pass for GELU approximate feedforward network. Computes gradients through Linear2 -> GELU_approx -> Linear1 chain. The forward pass is: hidden_states -> Linear1 (expansion to 4x) -> GELU_approx -> Linear2 (contraction) -> output.

Fastest reported
194.4µsmean · 67% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-19. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencydim = 1536 · hidden_dim = 614416 cases
seq_len
batch_size
dtype
1282fp3212832fp322561fp322564fp322568fp3225616fp325121fp325122fp325124fp325128fp326138fp3210241fp3210244fp3215712fp3220484fp3240962fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1194.4 µsRIAC as well11NVIDIA B200 · env 2218.8 µsRecursive8

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 11 results · last observed 2026-08-19Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
194.4µs67% SOL
1.00×
Reported · license unknown · no source
2026-08-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
214.0µs63% SOL
1.10×
Reported · license unknown · no source
2026-07-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
ac4k
219.2µs62% SOL
1.13×
Reported · license unknown · no source
2026-08-05

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Recursive
223.0µs61% SOL
1.15×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Hyra
227.0µs61% SOL
1.17×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Proud Wren @ NVIDIA
234.6µs59% SOL
1.21×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Geometric
247.7µs57% SOL
1.27×
Reported · license unknown · no source
2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Amir M. Mir | SF Tensor
265.5µs54% SOL
1.37×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
doubleAI
278.3µs52% SOL
1.43×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Geometric
318.2µs49% SOL
1.64×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Cursor
400.9µs40% SOL
2.06×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
RIAC as well
unknown
194.4µs
1.00×
Reported
license unknown · no source
Recursive
unknown
218.8µs
1.13×
Reported
license unknown · no source
ac4k
unknown
219.2µs
1.13×
Reported
license unknown · no source
Hyra
unknown
227.0µs
1.17×
Reported
license unknown · no source
Geometric
unknown
247.7µs
1.27×
Reported
license unknown · no source
Cursor
unknown
379.3µs
1.95×
Reported
license unknown · no source
Recursive
unknown
223.0µs
1.15×
Reported
license unknown · no source
doubleAI
unknown
224.9µs
1.16×
Reported
license unknown · no source
Show all 19 implementations ›
Proud Wren @ NVIDIA
unknown
234.6µs
1.21×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
261.1µs
1.34×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
265.5µs
1.37×
Reported
license unknown · no source
doubleAI
unknown
278.3µs
1.43×
Reported
license unknown · no source
Geometric
unknown
318.2µs
1.64×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
324.9µs
1.67×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
326.3µs
1.68×
Reported
license unknown · no source
Cursor
unknown
400.9µs
2.06×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
214.0µs
1.10×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
220.2µs
1.13×
Reported
license unknown · no source
doubleAI
unknown
220.7µs
1.13×
Reported
license unknown · no source

Semantics

Inputs and outputs
grad_outputfp32 [batch_size, seq_len, dim]
hidden_statesfp32 [batch_size, seq_len, dim]
weight1fp32 [hidden_dim, dim]
weight2fp32 [dim, hidden_dim]
linear1_outfp32 [batch_size, seq_len, hidden_dim]
tanh_innerfp32 [batch_size, seq_len, hidden_dim]
gelu_outfp32 [batch_size, seq_len, hidden_dim]
grad_hidden_statesfp32 [batch_size, seq_len, dim]
grad_weight1fp32 [hidden_dim, dim]
grad_weight2fp32 [dim, hidden_dim]
Axes and behavior
dimconstant = 1536
seq_lenvariable
batch_sizevariable
hidden_dimconstant = 6144
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias068_gelu_approximate_feedforward_backwardmodelstable-diffusion-3-5-mediumsha25696c09ab092c0…
Sources: NVIDIA SOL-ExecBench (2026-08-19)last observed 2026-08-19How records are decidedJSON