Skip to content
KernelIndex
Search⌘K
Sign in

Fused linear + SimPO loss

16 eligible runs
simpo

SimPO preference loss over batch×seq×hidden activations projected through a vocab×hidden LM head against integer targets, computed chunked without materializing logits.

Best usable
30.3msmedian · 2.09× vs fastest
LigerLiger-Kernel · BSD-2-Clause · triton

Reported evidence · last observed 2024-11-15. Reported by source; not independently reproduced.

pip install "liger-kernel==0.4.1"
View source →Run detail →
Source baseline · unbeaten · not usable as-is
14.5msmedian
TransformersHugging Face Transformers · Apache-2.0 · python

Reported evidence · last observed 2024-11-15. The source's designated baseline implementation. Reported by source; not independently reproduced.

No package · vendor the source →View source →Run detail →

Fastest by languagepython · Transformers · 14.5 ms · #1triton · Liger · 30.3 ms · 2.09×

Current records

Workloadseq = 1024 · batch = 2 · vocab = 128256 · hidden = 4096 · bf16seq = 1024 · batch = 4 · vocab = 128256 · hidden = 4096 · bf16seq = 1024 · batch = 8 · vocab = 128256 · hidden = 4096 · bf16seq = 1024 · batch = 16 · vocab = 128256 · hidden = 4096 · bf16
HardwareBest knownImplementationRuns
NVIDIA A100 · env 114.5 msTransformers2NVIDIA A100 · env 231.6 msLiger2

Not measured on H100, B200 for this workload. Challenges →

Source-native comparison · GPU NVIDIA A100 · Workload seq = 1024 · batch = 2 · vocab = 128256 · hidden = 4096 · bf16 · Protocol Liger-Kernel benchmark scripts · median · 2 results · last observed 2024-11-15Record history →
Estimated floor 523.5 µs · record 27.72× above itestimate, not evidence ›
DRAM 523.5 µs · bandwidth-bound on A100 80GB
every declared tensor crosses HBM exactly once (2,039 GB/s, A100 80GB datasheet)
no arithmetic formula for this family: bandwidth floor only
headroom-v1: a lower bound from declared tensors and datasheet peaks. A kernel can sit well above it for good reasons.
#
Implementation
Latency
vs #1
Trust
Observed
1
TransformersbaselineHugging Face Transformers
14.5ms
1.00×
Reported · Apache-2.0 · source
2024-11-15stale

Measured exactly what you asked. The source's designated baseline implementation. Reported by source; not independently reproduced.

source mirroredApache-2.0no install recipeView source →Run detail →
2
LigerLiger-Kernel
30.3ms
2.09×
Reported · BSD-2-Clause · source
2024-11-15stale

2.09× slower than the baseline. Measured exactly what you asked. Reported by source; not independently reproduced.

source mirroredBSD-2-ClauseinstallableView source →Run detail →

Scaling by batch

50.0 ms100.0 ms150.0 ms200.0 ms24816batch →14.5 ms28.8 ms52.6 ms104.4 msTransformers30.3 ms58.8 ms118.0 ms235.6 msLiger
TransformersLigerbest per workload · NVIDIA A100 · Liger-Kernel benchmark scripts held constant

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
TransformersHugging Face Transformers
python
14.5ms
1.00×
Reported
Apache-2.0 · source
LigerLiger-Kernel
triton
30.3ms
2.09×
Reported
BSD-2-Clause · source

Semantics

Inputs and outputs
xfloat [batch, seq, hidden]
weightfloat [vocab, hidden]
targetint64 [batch, seq]
lossfloat [1]
Axes and behavior
seqvariable
batchvariable
vocabvariable
hiddenvariable
determinismunspecified
constraintsNo mutation or aliasing
Identity
sha256f90f215423e3…
Sources: Liger-Kernel benchmarks (2024-11-15) · BSD-2-Clauselast observed 2024-11-15How records are decidedJSON