GeGLU MLP
GeGLU MLP (tanh-approximated GELU gate) over batch×seq×hidden activations.
Reported evidence · last observed 2025-11-11. Reported by source; not independently reproduced.
pip install "liger-kernel==0.6.3"Fastest by languagetriton · Liger · 661.4 µs · #1python · Transformers · 674.3 µs · 1.02×
Current records
Workloadseq = 1024 · batch = 2 · hidden = 2048 · intermediate = 4096 · bf169 cases
Not measured on H100, B200 for this workload. Challenges →
Estimated floor 58.3 µs · record 11.35× above itestimate, not evidence ›
1LigerLiger-Kernel661.4µs1.00×Reported · BSD-2-Clause · source2025-11-11stale
1.02× faster than the baseline. Measured exactly what you asked. Reported by source; not independently reproduced.
2674.3µs1.02×Reported · Apache-2.0 · source2025-11-11stale
Measured exactly what you asked. The source's designated baseline implementation. Reported by source; not independently reproduced.
3LigerLiger-Kernel740.4µs1.12×Reported · BSD-2-Clause · source2025-11-11stale
1.10× slower than the baseline. Measured exactly what you asked. Reported by source; not independently reproduced.
4743.4µs1.12×Reported · Apache-2.0 · source2025-11-11stale
Measured exactly what you asked. The source's designated baseline implementation. Reported by source; not independently reproduced.