NVFP4 MoE routing with topk selection
FP4-quantized MoE routing gate that computes router logits via FP4 scaled GEMM and performs top-k expert selection with softmax normalization. Input activations are quantized to FP4 on-the-fly, multiplied with pre-quantized gate weights, then softmax and top-k selection are applied.
Reported evidence · last observed 2026-07-22. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencysf_k = 128 · hidden_size = 2048 · num_experts = 512 · hidden_size_half = 1024 · num_experts_per_tok = 1016 cases
Not measured on H100 for this workload. Challenges →
122.4µs97% SOL1.00×Reported · license unknown · no source2026-07-22
Measured exactly what you asked. Reported by source; not independently reproduced.
225.0µs97% SOL1.12×Reported · license unknown · no source2026-08-23
Measured exactly what you asked. Reported by source; not independently reproduced.
327.9µs96% SOL1.25×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
428.7µs96% SOL1.29×Reported · license unknown · no source2026-08-22
Measured exactly what you asked. Reported by source; not independently reproduced.
531.7µs95% SOL1.42×Reported · license unknown · no source2026-08-09
Measured exactly what you asked. Reported by source; not independently reproduced.
632.5µs95% SOL1.45×Reported · license unknown · no source2026-08-18
Measured exactly what you asked. Reported by source; not independently reproduced.
733.5µs95% SOL1.50×Reported · license unknown · no source2026-07-19
Measured exactly what you asked. Reported by source; not independently reproduced.
834.4µs95% SOL1.54×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
934.8µs95% SOL1.56×Reported · license unknown · no source2026-08-19
Measured exactly what you asked. Reported by source; not independently reproduced.
1035.0µs95% SOL1.57×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
1137.0µs94% SOL1.65×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
1243.1µs93% SOL1.93×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
1345.0µs93% SOL2.01×Reported · license unknown · no source2026-08-04
Measured exactly what you asked. Reported by source; not independently reproduced.
1456.9µs91% SOL2.55×Reported · license unknown · no source2026-08-02
Measured exactly what you asked. Reported by source; not independently reproduced.
1586.1µs87% SOL3.85×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
1693.9µs86% SOL4.20×Reported · license unknown · no source2026-07-25
Measured exactly what you asked. Reported by source; not independently reproduced.
17156.4µs77% SOL7.00×Reported · license unknown · no source2026-07-14
Measured exactly what you asked. Reported by source; not independently reproduced.
18160.9µs77% SOL7.20×Reported · license unknown · no source2026-07-14
Measured exactly what you asked. Reported by source; not independently reproduced.
19255.6µs68% SOL11.44×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.