Skip to content
KernelIndex
Search⌘K
Sign in

Hugging Face Transformers

Library · huggingface/transformers · Apache-2.0
Records held
52as of 2026-10-01 06:01 UTC
Kernels
16
Runs
342
Is this you? Claim this project ›
Verifies your GitHub login against the repository owner, live.

A claim grants attribution and metadata maintenance, never the right to edit evidence. Evidence claims are reviewed by a maintainer.

Current records

Operation · workload
History
Record
Implementation
Hardware
Since
GroupNormfp32 · [128, 32, 512]
84.0µs
49.4% faster · was 165.9 µs
Transformers
B200
2026-02-28
GroupNormfp32 · [128, 64, 512]
112.7µs
34.9% faster · was 173.0 µs
Transformers
B200
2026-02-28
GroupNormfp32 · [128, 32, 512]
55.3µs
16.7% faster · was 66.4 µs
Transformers
B200
2026-02-28
Llama 4 rotary position embeddingbf16 · [1, 2048, 8, 16]
37.6µs
54.4% faster · was 82.5 µs
Transformers
H100
2025-08-07
Llama 4 rotary position embeddingbf16 · [1, 2048, 8, 64]
61.9µs
24.3% faster · was 81.7 µs
Transformers
H100
2025-08-07
Llama 4 rotary position embeddingbf16 · [1, 2048, 8, 16]
137.6µs
10.7% faster · was 154.0 µs
Transformers
H100
2025-08-07
Llama 4 rotary position embeddingbf16 · [1, 2048, 8, 64]
136.0µs
11.5% faster · was 153.6 µs
Transformers
H100
2025-08-07
Llama 4 rotary position embeddingbf16 · [1, 2048, 8, 16]
246.9µs
32.4% faster · was 365.2 µs
Transformers
H100
2025-08-07
Llama 4 rotary position embeddingbf16 · [1, 2048, 8, 64]
247.7µs
31.2% faster · was 359.9 µs
Transformers
H100
2025-08-07
Llama 4 rotary position embeddingbf16 · [1, 1024, 8, 256]
245.9µs
14.4% faster · was 287.4 µs
Transformers
H100
2025-08-07
Show all 40 records ›
Fused linear + KTO lossbf16 · [2, 512, 1024]
3.88ms
3.0% faster · was 4.00 ms
Transformers
H100
2025-03-03
Fused linear + KTO lossbf16 · [4, 512, 1024]
7.21ms
7.6% faster · was 7.80 ms
Transformers
H100
2025-03-03
Fused linear + KTO lossbf16 · [8, 512, 1024]
13.8ms
10.4% faster · was 15.4 ms
Transformers
H100
2025-03-03
Fused linear + KTO lossbf16 · [16, 512, 1024]
27.1ms
11.7% faster · was 30.7 ms
Transformers
H100
2025-03-03
Fused linear + KTO lossbf16 · [32, 512, 1024]
54.1ms
11.4% faster · was 61.1 ms
Transformers
H100
2025-03-03
Fused linear + CPO lossbf16 · [2, 1024, 4096]
14.5ms
54.0% faster · was 31.5 ms
Transformers
A100
2024-11-14
Fused linear + CPO lossbf16 · [4, 1024, 4096]
26.1ms
58.2% faster · was 62.4 ms
Transformers
A100
2024-11-14
Fused linear + CPO lossbf16 · [8, 1024, 4096]
51.9ms
58.1% faster · was 123.6 ms
Transformers
A100
2024-11-14
Fused linear + CPO lossbf16 · [16, 1024, 4096]
105.0ms
57.3% faster · was 245.7 ms
Transformers
A100
2024-11-14
Fused linear + SimPO lossbf16 · [2, 1024, 4096]
14.5ms
52.1% faster · was 30.3 ms
Transformers
A100
2024-11-15
Fused linear + SimPO lossbf16 · [4, 1024, 4096]
28.8ms
51.1% faster · was 58.8 ms
Transformers
A100
2024-11-15
Fused linear + SimPO lossbf16 · [8, 1024, 4096]
52.6ms
55.4% faster · was 118.0 ms
Transformers
A100
2024-11-15
Fused linear + SimPO lossbf16 · [16, 1024, 4096]
104.4ms
55.7% faster · was 235.6 ms
Transformers
A100
2024-11-15
Fused linear + ORPO lossbf16 · [2, 4096, 4096]
39.6ms
65.9% faster · was 116.0 ms
Transformers
A100
2024-11-13
Fused linear + ORPO lossbf16 · [4, 4096, 4096]
80.0ms
65.4% faster · was 230.8 ms
Transformers
A100
2024-11-13
Fused linear + ORPO lossbf16 · [8, 4096, 4096]
160.1ms
65.3% faster · was 462.0 ms
Transformers
A100
2024-11-13
Fused linear + ORPO lossbf16 · [16, 4096, 4096]
321.5ms
65.2% faster · was 923.0 ms
Transformers
A100
2024-11-13
Fused linear + ORPO lossbf16 · [2, 1024, 4096]
12.9ms
58.4% faster · was 31.0 ms
Transformers
A100
2024-11-13
Fused linear + ORPO lossbf16 · [4, 1024, 4096]
25.6ms
58.0% faster · was 60.9 ms
Transformers
A100
2024-11-13
Fused linear + ORPO lossbf16 · [8, 1024, 4096]
51.8ms
57.3% faster · was 121.1 ms
Transformers
A100
2024-11-13
Fused linear + ORPO lossbf16 · [16, 1024, 4096]
103.9ms
57.5% faster · was 244.4 ms
Transformers
A100
2024-11-13
Fused linear + cross-entropybf16 · [4096, 4096]
20.9ms
82.5% faster · was 119.5 ms
Transformers
A100
2024-11-22
Fused linear + cross-entropybf16 · [8192, 4096]
38.0ms
77.4% faster · was 168.1 ms
Transformers
A100
2024-11-22
Fused linear + cross-entropybf16 · [16384, 4096]
76.4ms
72.1% faster · was 274.1 ms
Transformers
A100
2024-11-22
Fused linear + cross-entropybf16 · [32768, 4096]
151.9ms
70.1% faster · was 508.5 ms
Transformers
A100
2024-11-22
Fused linear + cross-entropybf16 · [4096, 4096]
56.0ms
53.9% faster · was 121.4 ms
Transformers
A100
2024-11-22
Fused linear + cross-entropybf16 · [8192, 4096]
111.5ms
33.1% faster · was 166.7 ms
Transformers
A100
2024-11-22
Fused linear + cross-entropybf16 · [16384, 4096]
220.8ms
20.3% faster · was 277.1 ms
Transformers
A100
2024-11-22
Fused linear + cross-entropybf16 · [32768, 4096]
452.5ms
11.5% faster · was 511.1 ms
Transformers
A100
2024-11-22
Fused add + RMSNormfp32 · [2048, 1024]
113.6µs
9.8% faster · was 125.9 µs
Transformers
H100
2026-04-07

12 more records · Open in the records ledger →

Measured kernels

Implementation
Operation
Runtime
Best latency
Evidence
Availability
TransformersHugging Face Transformers
LayerNorm
python
23.6µs
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
GroupNorm
python
30.7µs
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Llama 4 rotary position embedding
python
37.6µs
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Fused add + RMSNorm
python
45.5µs
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
RMSNorm
python
78.2µs
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Rotary position embedding
python
79.7µs
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Embedding lookup
python
144.1µs
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
GeGLU MLP
python
674.3µs
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
SwiGLU MLP
python
674.8µs
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Cross-entropy loss
python
875.1µs
Reported
Apache-2.0 · source
Show all 16 kernels ›
TransformersHugging Face Transformers
Fused linear + KTO loss
python
3.88ms
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Fused MoE expert MLP
python
4.48ms
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Fused linear + ORPO loss
python
12.9ms
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Fused linear + SimPO loss
python
14.5ms
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Fused linear + CPO loss
python
14.5ms
Reported
Apache-2.0 · source
TransformersHugging Face Transformers
Fused linear + cross-entropy
python
20.9ms
Reported
Apache-2.0 · source

Record activity

Nov
Dec
Jan
6Feb
Mar
4Apr
May
Jun
Jul
Aug
Sep
Oct
Sources: Liger-Kernel benchmarks (2026-04-07) · BSD-2-Clauselast observed 2026-04-07JSON