Jensen-Shannon divergence loss
48 eligible runs
jsd
Generalized JSD between student and teacher log-probabilities over tokens×vocab; the benchmark generates fp32 log-probs.
Fastest reported · usable today
1.02msmedian · 3.73× faster than baseline
LigerLiger-Kernel · BSD-2-Clause · triton
Reported evidence · last observed 2026-05-27. Reported by source; not independently reproduced.
pip install "liger-kernel==0.8.0"Fastest by languagetriton · Liger · 1.02 ms · #1python · PyTorch · 3.79 ms · 3.73×
Current records
Workloadvocab = 128256 · tokens = 1024 · fp3210 cases
HardwareBest knownImplementationRuns
NVIDIA B200 · env 11.02 msLiger2NVIDIA B200 · env 22.29 msPyTorch2NVIDIA B200 · env 35.92 msPyTorch2Not measured on H100 for this workload. Challenges →
Source-native comparison · GPU NVIDIA B200 · Workload vocab = 128256 · tokens = 1024 · fp32 · Protocol Liger-Kernel benchmark scripts · median · 2 results · last observed 2026-05-27Record history →
Estimated floor 131.3 µs · record 7.73× above itestimate, not evidence ›
DRAM 131.3 µs · bandwidth-bound on B200
every declared tensor crosses HBM exactly once (8,000 GB/s, B200 datasheet)
no arithmetic formula for this family: bandwidth floor only
headroom-v1: a lower bound from declared tensors and datasheet peaks. A kernel can sit well above it for good reasons.
#
Implementation
Latency
vs #1
Trust
Observed
1LigerLiger-Kernel1.02ms1.00×Reported · BSD-2-Clause · source2026-05-27
3.73× faster than the baseline. Measured exactly what you asked. Reported by source; not independently reproduced.
source mirroredBSD-2-ClauseinstallableView source →Run detail →
2PyTorchbaseline3.79ms3.73×Reported · BSD-3-Clause · source2026-05-27
Measured exactly what you asked. The source's designated baseline implementation. Reported by source; not independently reproduced.
source mirroredBSD-3-Clauseno install recipeView source →Run detail →
Scaling by tokens
Implementations
Semantics
Inputs and outputs
student_log_probsfloat [tokens, vocab]
teacher_log_probsfloat [tokens, vocab]
lossfloat [1]
Axes and behavior
vocabvariable
tokensvariable
determinismunspecified
constraintsNo mutation or aliasing
Identity
sha256eccbf4150e60…
Sources: Liger-Kernel benchmarks (2026-05-27) · BSD-2-Clauselast observed 2026-05-27How records are decidedJSON