Per-group FP8 quantization
Per-token-group FP8 E4M3 activation quantization as used in W8A8 LLM inference: for each group of group_size contiguous elements, scale = max(absmax, eps) / 448 and x_q = clamp(x / scale, -448, 448). Outputs are fp32 clamped to the FP8 range, written into pre-allocated x_q and per-group scale x_s tensors.
Reported evidence · last observed 2026-03-14. Reported by source; not independently reproduced. License unknown.
Current records
Not measured on H100 for this workload. Challenges →
1submission 554363jiannanWang8.37µs1.00×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
2submission 555541kitrak_rev.8.37µs1.00×Reported · license unknown · source2026-03-15stale
Measured exactly what you asked. Reported by source; not independently reproduced.
3submission 553343kitrak_rev.8.47µs1.01×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
4submission 553239svdrecbd8.56µs1.02×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
5submission 553895suvasis_290479.11µs1.09×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
6submission 555479Ayush109.15µs1.09×Reported · license unknown · source2026-03-15stale
Measured exactly what you asked. Reported by source; not independently reproduced.
7submission 553189brandonin14.8µs1.76×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
8submission 552497dpang15.2µs1.81×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
9submission 552561fluudgate15.3µs1.82×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
10submission 555611yhinai15.5µs1.85×Reported · license unknown · source2026-03-15stale
Measured exactly what you asked. Reported by source; not independently reproduced.
11submission 555582alazarr.m16.0µs1.91×Reported · license unknown · source2026-03-15stale
Measured exactly what you asked. Reported by source; not independently reproduced.
12submission 552685brandonin18.3µs2.19×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
13submission 553151ramizzik24.0µs2.87×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
14submission 553064Narain24.2µs2.89×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
15submission 552575bloomberg938324.4µs2.92×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.
16submission 555533tr00ths3rum24.5µs2.92×Reported · license unknown · source2026-03-15stale
Measured exactly what you asked. Reported by source; not independently reproduced.
17submission 553021CodingMaster28.7µs3.43×Reported · license unknown · source2026-03-14stale
Measured exactly what you asked. Reported by source; not independently reproduced.