NVFP4 linear layer
FP4-quantized linear layer with block-wise scaling (16 elements per block). Computes y = x @ W^T where both x and W are quantized to FP4 with FP8 E4M3FN block-wise scale factors. Uses torch._scaled_mm for efficient FP4 matrix multiplication.
Reported evidence · last observed 2026-08-20. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencysf_k = 512 · in_features = 8192 · sf_vec_size = 16 · out_features = 8192 · in_features_half = 409616 cases
Not measured on H100 for this workload. Challenges →
187.2µs91% SOL1.00×Reported · license unknown · no source2026-08-20
Measured exactly what you asked. Reported by source; not independently reproduced.
288.5µs91% SOL1.02×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
389.3µs90% SOL1.02×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
489.8µs90% SOL1.03×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
5105.6µs89% SOL1.21×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
6107.0µs88% SOL1.23×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
7108.0µs88% SOL1.24×Reported · license unknown · no source2026-08-08
Measured exactly what you asked. Reported by source; not independently reproduced.
8116.7µs87% SOL1.34×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.
9117.1µs87% SOL1.34×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
10125.5µs86% SOL1.44×Reported · license unknown · no source2026-07-18
Measured exactly what you asked. Reported by source; not independently reproduced.
11175.0µs82% SOL2.01×Reported · license unknown · no source2026-08-03
Measured exactly what you asked. Reported by source; not independently reproduced.
12177.3µs81% SOL2.03×Reported · license unknown · no source2026-08-05
Measured exactly what you asked. Reported by source; not independently reproduced.
13193.3µs79% SOL2.22×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
14247.3µs75% SOL2.84×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.