FP8 Mamba2 in projection
FP8-quantized input projection for Mamba2 mixer. Projects hidden states from hidden_size (2688) to a large combined dimension (13952, padded from 13856 to be multiple of 128) for gated MLP branches, gate, convolution input, and delta/dt. Uses BlockWise1x128 scaling for activations and BlockWise128x128 scaling for weights.
Reported evidence · last observed 2026-08-21. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhidden_size = 2688 · projection_size = 1395216 cases
Not measured on H100 for this workload. Challenges →
1128.4µs75% SOL1.00×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
2146.6µs71% SOL1.14×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
3156.8µs69% SOL1.22×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
4174.6µs65% SOL1.36×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
5181.3µs64% SOL1.41×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
6182.4µs64% SOL1.42×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
7182.5µs64% SOL1.42×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
8182.5µs64% SOL1.42×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
9185.0µs63% SOL1.44×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
10185.9µs63% SOL1.45×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
11187.2µs63% SOL1.46×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
12187.5µs63% SOL1.46×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
13188.4µs63% SOL1.47×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
14188.7µs63% SOL1.47×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
15190.9µs63% SOL1.49×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
16194.0µs62% SOL1.51×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
17194.1µs62% SOL1.51×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
18195.9µs62% SOL1.53×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
19196.3µs62% SOL1.53×Reported · license unknown · no source2026-07-18
Measured exactly what you asked. Reported by source; not independently reproduced.
20196.5µs62% SOL1.53×Reported · license unknown · no source2026-08-01
Measured exactly what you asked. Reported by source; not independently reproduced.
21200.0µs61% SOL1.56×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
22202.0µs61% SOL1.57×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
23213.3µs59% SOL1.66×Reported · license unknown · no source2026-08-06
Measured exactly what you asked. Reported by source; not independently reproduced.
24217.0µs59% SOL1.69×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
25219.5µs58% SOL1.71×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
26272.5µs52% SOL2.12×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
27282.6µs51% SOL2.20×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
28663.9µs29% SOL5.17×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.