Skip to content
KernelIndex
Search⌘K
Sign in

Fused linear + KTO loss

20 eligible runs
kto

KTO preference loss: policy and reference batch×seq×hidden activations project through separate biased vocab×hidden LM heads against integer targets, with per-sequence boolean preference labels and a precomputed KL term (beta 0.1).

Best usable
4.00msmedian · 1.03× vs fastest
LigerLiger-Kernel · BSD-2-Clause · triton

Reported evidence · last observed 2025-03-03. Reported by source; not independently reproduced.

pip install "liger-kernel==0.5.4"
View source →Run detail →
Source baseline · unbeaten · not usable as-is
3.88msmedian
TransformersHugging Face Transformers · Apache-2.0 · python

Reported evidence · last observed 2025-03-03. The source's designated baseline implementation. Reported by source; not independently reproduced.

No package · vendor the source →View source →Run detail →

Fastest by languagepython · Transformers · 3.88 ms · #1triton · Liger · 4.00 ms · 1.03×

Current records

Workloadseq = 512 · batch = 4 · vocab = 128256 · hidden = 1024 · bf16seq = 512 · batch = 8 · vocab = 128256 · hidden = 1024 · bf16seq = 512 · batch = 16 · vocab = 128256 · hidden = 1024 · bf16seq = 512 · batch = 2 · vocab = 128256 · hidden = 1024 · bf16seq = 512 · batch = 32 · vocab = 128256 · hidden = 1024 · bf16
HardwareBest knownImplementationRuns
NVIDIA H100 · env 13.88 msTransformers2NVIDIA H100 · env 24.78 msLiger2

Not measured on B200 for this workload. Challenges →

Source-native comparison · GPU NVIDIA H100 · Workload seq = 512 · batch = 2 · vocab = 128256 · hidden = 1024 · bf16 · Protocol Liger-Kernel benchmark scripts · median · 2 results · last observed 2025-03-03Record history →
Estimated floor 158.2 µs · record 24.5× above itestimate, not evidence ›
DRAM 158.2 µs · bandwidth-bound on H100 SXM
every declared tensor crosses HBM exactly once (3,350 GB/s, H100 SXM datasheet)
no arithmetic formula for this family: bandwidth floor only
headroom-v1: a lower bound from declared tensors and datasheet peaks. A kernel can sit well above it for good reasons.
#
Implementation
Latency
vs #1
Trust
Observed
1
TransformersbaselineHugging Face Transformers
3.88ms
1.00×
Reported · Apache-2.0 · source
2025-03-03stale

Measured exactly what you asked. The source's designated baseline implementation. Reported by source; not independently reproduced.

source mirroredApache-2.0no install recipeView source →Run detail →
2
LigerLiger-Kernel
4.00ms
1.03×
Reported · BSD-2-Clause · source
2025-03-03stale

1.03× slower than the baseline. Measured exactly what you asked. Reported by source; not independently reproduced.

source mirroredBSD-2-ClauseinstallableView source →Run detail →

Scaling by batch

20.0 ms40.0 ms60.0 ms2481632batch →3.88 ms7.21 ms13.8 ms27.1 ms54.1 msTransformers4.00 ms7.80 ms15.4 ms30.7 ms61.1 msLiger
TransformersLigerbest per workload · NVIDIA H100 · Liger-Kernel benchmark scripts held constant

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
TransformersHugging Face Transformers
python
3.88ms
1.00×
Reported
Apache-2.0 · source
LigerLiger-Kernel
triton
4.00ms
1.03×
Reported
BSD-2-Clause · source

Semantics

Inputs and outputs
xfloat [batch, seq, hidden]
ref_xfloat [batch, seq, hidden]
weightfloat [vocab, hidden]
biasfloat [vocab]
ref_weightfloat [vocab, hidden]
ref_biasfloat [vocab]
targetint64 [batch, seq]
preference_labelsbool [batch]
klfloat [1]
lossfloat [1]
Axes and behavior
seqvariable
batchvariable
vocabvariable
hiddenvariable
determinismunspecified
constraintsNo mutation or aliasing
Identity
sha256b10257d6984b…
Sources: Liger-Kernel benchmarks (2025-03-03) · BSD-2-Clauselast observed 2025-03-03How records are decidedJSON