Skip to content
KernelIndex
Search⌘K
Sign in

Feedforward MLP backward

21 eligible runs
mlp

Backward pass for feedforward MLP: computes gradients w.r.t. input hidden states, weight1, bias1, weight2, bias2 given upstream gradient. Forward: hidden_states -> Linear1 -> GELU -> Linear2 -> output.

Fastest reported
403.1µsmean · 72% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-18. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhidden_size = 2048 · intermediate_size = 819216 cases
batch_size
seq_len
dtype
11024fp322128fp322256fp322512fp3224096fp324256fp324512fp324541fp328128fp328256fp328773fp3216512fp32161024fp3232128fp3232256fp3264128fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1403.1 µsRIAC as well15NVIDIA B200 · env 2421.6 µsdoubleAI6

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 15 results · last observed 2026-08-18Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
403.1µs72% SOL
1.00×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
403.9µs71% SOL
1.00×
Reported · license unknown · no source
2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Electric Cheetah
432.5µs68% SOL
1.07×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Gleaming Coyote
433.0µs68% SOL
1.07×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Rapid Tiger
433.1µs68% SOL
1.07×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
ac4k
439.1µs67% SOL
1.09×
Reported · license unknown · no source
2026-08-05

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Hyra
446.6µs67% SOL
1.11×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
doubleAI
447.0µs67% SOL
1.11×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Recursive
471.2µs65% SOL
1.17×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Geometric
477.3µs65% SOL
1.18×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Proud Wren @ NVIDIA
480.8µs64% SOL
1.19×
Reported · license unknown · no source
2026-07-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Amir M. Mir | SF Tensor
576.9µs58% SOL
1.43×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Geometric
801.1µs45% SOL
1.99×
Reported · license unknown · no source
2026-07-26

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Cursor
836.8µs44% SOL
2.08×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Geometric
1.09ms36% SOL
2.70×
Reported · license unknown · no source
2026-07-25

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
RIAC as well
unknown
403.1µs
1.00×
Reported
license unknown · no source
Electric Cheetah
unknown
432.5µs
1.07×
Reported
license unknown · no source
Gleaming Coyote
unknown
433.0µs
1.07×
Reported
license unknown · no source
Rapid Tiger
unknown
433.1µs
1.07×
Reported
license unknown · no source
Hyra
unknown
446.6µs
1.11×
Reported
license unknown · no source
Recursive
unknown
455.0µs
1.13×
Reported
license unknown · no source
Geometric
unknown
477.3µs
1.18×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
480.8µs
1.19×
Reported
license unknown · no source
Show all 21 implementations ›
Cursor
unknown
780.1µs
1.94×
Reported
license unknown · no source
doubleAI
unknown
447.0µs
1.11×
Reported
license unknown · no source
Recursive
unknown
471.2µs
1.17×
Reported
license unknown · no source
doubleAI
unknown
538.0µs
1.33×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
566.5µs
1.41×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
576.9µs
1.43×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
723.4µs
1.79×
Reported
license unknown · no source
Geometric
unknown
801.1µs
1.99×
Reported
license unknown · no source
Cursor
unknown
836.8µs
2.08×
Reported
license unknown · no source
Geometric
unknown
1.09ms
2.70×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
403.9µs
1.00×
Reported
license unknown · no source
doubleAI
unknown
421.6µs
1.05×
Reported
license unknown · no source
ac4k
unknown
439.1µs
1.09×
Reported
license unknown · no source

Semantics

Inputs and outputs
grad_outputfp32 [batch_size, seq_len, hidden_size]
hidden_statesfp32 [batch_size, seq_len, hidden_size]
weight1fp32 [intermediate_size, hidden_size]
bias1fp32 [intermediate_size]
weight2fp32 [hidden_size, intermediate_size]
bias2fp32 [hidden_size]
intermediatefp32 [batch_size, seq_len, intermediate_size]
intermediate_activatedfp32 [batch_size, seq_len, intermediate_size]
grad_hidden_statesfp32 [batch_size, seq_len, hidden_size]
grad_weight1fp32 [intermediate_size, hidden_size]
grad_bias1fp32 [intermediate_size]
grad_weight2fp32 [hidden_size, intermediate_size]
grad_bias2fp32 [hidden_size]
Axes and behavior
seq_lenvariable
batch_sizevariable
hidden_sizeconstant = 2048
intermediate_sizeconstant = 8192
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias073_feedforward_mlp_backwardmodelstep1x-editsha256b6e04ef979b6…
Sources: NVIDIA SOL-ExecBench (2026-08-18)last observed 2026-08-18How records are decidedJSON