GeGLU activation
GEGLU (Gated GLU with GELU) activation function. Splits input along last dimension into two halves, applies approximate GELU (tanh-based) to the first half, and multiplies element-wise with the second half. Used in transformer feed-forward networks.
Reported evidence · last observed 2026-08-12. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyinner_dim = 5120 · input_dim = 1024016 cases
Not measured on H100 for this workload. Challenges →
117.0µs68% SOL1.00×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
217.0µs68% SOL1.00×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
317.1µs68% SOL1.01×Reported · license unknown · no source2026-08-24
Measured exactly what you asked. Reported by source; not independently reproduced.
417.2µs67% SOL1.01×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
517.2µs67% SOL1.01×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
617.9µs66% SOL1.05×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
718.3µs65% SOL1.08×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
818.6µs65% SOL1.10×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
918.7µs65% SOL1.10×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1018.9µs64% SOL1.12×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1119.1µs64% SOL1.13×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
1219.3µs64% SOL1.14×Reported · license unknown · no source2026-07-29
Measured exactly what you asked. Reported by source; not independently reproduced.
1319.3µs63% SOL1.14×Reported · license unknown · no source2026-07-14
Measured exactly what you asked. Reported by source; not independently reproduced.
1419.3µs63% SOL1.14×Reported · license unknown · no source2026-07-28
Measured exactly what you asked. Reported by source; not independently reproduced.
1519.6µs63% SOL1.16×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
1619.8µs63% SOL1.16×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.