Fused gated MLP SiLU
Gated MLP with SiLU activation that fuses gate and up projections into a single linear layer, then splits and applies gated activation before down projection. The pattern is: gate_up_proj -> chunk -> SiLU(gate) * up -> down_proj.
Reported evidence · last observed 2026-07-11. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhidden_size = 1024 · intermediate_size = 4096 · intermediate_size_x2 = 819216 cases
Not measured on H100 for this workload. Challenges →
193.5µs74% SOL1.00×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
297.3µs73% SOL1.04×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
397.9µs72% SOL1.05×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
499.8µs72% SOL1.07×Reported · license unknown · no source2026-08-24
Measured exactly what you asked. Reported by source; not independently reproduced.
5100.7µs71% SOL1.08×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
6101.3µs71% SOL1.08×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
7101.7µs71% SOL1.09×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
8101.9µs71% SOL1.09×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
9102.0µs71% SOL1.09×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
10103.6µs70% SOL1.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
11107.1µs69% SOL1.14×Reported · license unknown · no source2026-08-24
Measured exactly what you asked. Reported by source; not independently reproduced.
12107.3µs69% SOL1.15×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
13108.1µs69% SOL1.16×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
14111.2µs68% SOL1.19×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
15113.9µs67% SOL1.22×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
16118.5µs65% SOL1.27×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
17118.6µs65% SOL1.27×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
18120.2µs65% SOL1.28×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.
19121.6µs65% SOL1.30×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
20124.9µs63% SOL1.34×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
21129.3µs62% SOL1.38×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
22132.8µs61% SOL1.42×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
23145.7µs57% SOL1.56×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
24146.3µs58% SOL1.56×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
25183.3µs50% SOL1.96×Reported · license unknown · no source2026-08-05
Measured exactly what you asked. Reported by source; not independently reproduced.
26268.9µs38% SOL2.87×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.