Skip to content
KernelIndex
Search⌘K
Sign in

Fused linear + ORPO loss

32 eligible runs
orpo

ORPO preference loss over batch×seq×hidden activations projected through a vocab×hidden LM head against integer targets, computed chunked without materializing logits.

Best usable
31.0msmedian · 2.40× vs fastest
LigerLiger-Kernel · BSD-2-Clause · triton

Reported evidence · last observed 2024-11-13. Reported by source; not independently reproduced.

pip install "liger-kernel==0.4.0"
View source →Run detail →
Source baseline · unbeaten · not usable as-is
12.9msmedian
TransformersHugging Face Transformers · Apache-2.0 · python

Reported evidence · last observed 2024-11-13. The source's designated baseline implementation. Reported by source; not independently reproduced.

No package · vendor the source →View source →Run detail →

Fastest by languagepython · Transformers · 12.9 ms · #1triton · Liger · 31.0 ms · 2.40×

Current records

Workloadseq = 4096 · batch = 2 · vocab = 128256 · hidden = 4096 · bf16seq = 4096 · batch = 4 · vocab = 128256 · hidden = 4096 · bf16seq = 4096 · batch = 8 · vocab = 128256 · hidden = 4096 · bf16seq = 4096 · batch = 16 · vocab = 128256 · hidden = 4096 · bf16seq = 1024 · batch = 2 · vocab = 128256 · hidden = 4096 · bf16seq = 1024 · batch = 4 · vocab = 128256 · hidden = 4096 · bf16seq = 1024 · batch = 8 · vocab = 128256 · hidden = 4096 · bf16seq = 1024 · batch = 16 · vocab = 128256 · hidden = 4096 · bf16
HardwareBest knownImplementationRuns
NVIDIA A100 · env 112.9 msTransformers2NVIDIA A100 · env 232.5 msLiger2

Not measured on H100, B200 for this workload. Challenges →

Source-native comparison · GPU NVIDIA A100 · Workload seq = 1024 · batch = 2 · vocab = 128256 · hidden = 4096 · bf16 · Protocol Liger-Kernel benchmark scripts · median · 2 results · last observed 2024-11-13Record history →
Estimated floor 523.5 µs · record 24.66× above itestimate, not evidence ›
DRAM 523.5 µs · bandwidth-bound on A100 80GB
every declared tensor crosses HBM exactly once (2,039 GB/s, A100 80GB datasheet)
no arithmetic formula for this family: bandwidth floor only
headroom-v1: a lower bound from declared tensors and datasheet peaks. A kernel can sit well above it for good reasons.
#
Implementation
Latency
vs #1
Trust
Observed
1
TransformersbaselineHugging Face Transformers
12.9ms
1.00×
Reported · Apache-2.0 · source
2024-11-13stale

Measured exactly what you asked. The source's designated baseline implementation. Reported by source; not independently reproduced.

source mirroredApache-2.0no install recipeView source →Run detail →
2
LigerLiger-Kernel
31.0ms
2.40×
Reported · BSD-2-Clause · source
2024-11-13stale

2.40× slower than the baseline. Measured exactly what you asked. Reported by source; not independently reproduced.

source mirroredBSD-2-ClauseinstallableView source →Run detail →

Scaling by batch

100.0 ms200.0 ms24816batch →12.9 ms25.6 ms51.8 ms103.9 msTransformers31.0 ms60.9 ms121.1 ms244.4 msLiger
TransformersLigerbest per workload · NVIDIA A100 · Liger-Kernel benchmark scripts held constant

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
TransformersHugging Face Transformers
python
12.9ms
1.00×
Reported
Apache-2.0 · source
LigerLiger-Kernel
triton
31.0ms
2.40×
Reported
BSD-2-Clause · source

Semantics

Inputs and outputs
xfloat [batch, seq, hidden]
weightfloat [vocab, hidden]
targetint64 [batch, seq]
lossfloat [1]
Axes and behavior
seqvariable
batchvariable
vocabvariable
hiddenvariable
determinismunspecified
constraintsNo mutation or aliasing
Identity
sha25678124dd4d43a…
Sources: Liger-Kernel benchmarks (2024-11-13) · BSD-2-Clauselast observed 2024-11-13How records are decidedJSON