Flux feedforward GELU approximate
FLUX feedforward network with GELU approximate activation (tanh approximation). Performs Linear projection to 4x hidden dimension, GELU-tanh activation, and Linear projection back to hidden dimension. This appears 76 times per forward pass in FLUX transformer blocks.
Reported evidence · last observed 2026-07-10. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhidden_dim = 3072 · mlp_hidden_dim = 1228816 cases
Not measured on H100 for this workload. Challenges →
1518.7µs72% SOL1.00×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
2524.4µs71% SOL1.01×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
3526.9µs71% SOL1.02×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
4529.9µs70% SOL1.02×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
5532.7µs70% SOL1.03×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
6537.1µs70% SOL1.04×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
7553.1µs68% SOL1.07×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
8563.1µs68% SOL1.09×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
9570.5µs67% SOL1.10×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
10575.0µs67% SOL1.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
11601.6µs64% SOL1.16×Reported · license unknown · no source2026-08-10
Measured exactly what you asked. Reported by source; not independently reproduced.
12611.4µs63% SOL1.18×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
13706.9µs57% SOL1.36×Reported · license unknown · no source2026-07-23
Measured exactly what you asked. Reported by source; not independently reproduced.
14766.5µs54% SOL1.48×Reported · license unknown · no source2026-08-04
Measured exactly what you asked. Reported by source; not independently reproduced.
15815.5µs52% SOL1.57×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
16866.5µs48% SOL1.67×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
17868.0µs51% SOL1.67×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.