Skip to content
KernelIndex
Search⌘K
Sign in

PyTorch

Library · pytorch/pytorch · BSD-3-Clause, MIT
Records held
591as of 2026-10-01 12:54 UTC
Kernels
509
Runs
1,458
Is this you? Claim this project ›
Verifies your GitHub login against the repository owner, live.

A claim grants attribution and metadata maintenance, never the right to edit evidence. Evidence claims are reviewed by a maintainer.

Current records

Operation · workload
History
Record
Implementation
Hardware
Since
Mamba2ReturnYfp32 · [2048, 128, 8, 64]
7.47ms
39.3% faster · was 12.3 ms
torch.compile (inductor)
H100
2026-03-05
Mamba2ReturnFinalStatefp32 · [2048, 128, 8, 64]
2.66ms
71.0% faster · was 9.18 ms
torch.compile (inductor)
H100
2026-03-05
ReLUSelfAttentionfp32 · [16, 1024, 768]
4.97ms
35.0% faster · was 7.65 ms
torch.compile (inductor)
H100
2026-03-05
MinGPTCausalAttentionfp32 · [128, 512, 768]
15.7ms
20.3% faster · was 19.7 ms
torch.compile (inductor)
H100
2026-03-05
MiniGPTBlockfp32 · [128, 512, 768]
36.7ms
22.9% faster · was 47.6 ms
torch.compile (inductor)
H100
2026-03-05
UNetSoftmaxfp32 · [8, 8, 64, 512]
6.56ms
8.0% faster · was 7.13 ms
torch.compile (inductor)
H100
2026-03-05
NetVladWithGhostClustersfp32 · [2048, 100, 512]
1.50ms
30.6% faster · was 2.16 ms
torch.compile (inductor)
H100
2026-03-05
NetVladNoGhostClustersfp32 · [2048, 100, 512]
1.35ms
30.8% faster · was 1.95 ms
torch.compile (inductor)
H100
2026-03-05
ShuffleNetfp32 · [10, 3, 224, 224]
13.5ms
60.8% faster · was 34.4 ms
torch.compile (inductor)
H100
2026-03-05
RegNetfp32 · [8, 3, 224, 224]
1.60ms
50.8% faster · was 3.25 ms
torch.compile (inductor)
H100
2026-03-05
Show all 40 records ›
VisionTransformerfp32 · [2, 3, 224, 224]
1.59ms
13.1% faster · was 1.83 ms
torch.compile (inductor)
H100
2026-03-05
SwinMLPfp32 · [10, 3, 224, 224]
4.18ms
18.4% faster · was 5.12 ms
torch.compile (inductor)
H100
2026-03-05
SwinTransformerV2fp32 · [10, 3, 224, 224]
6.47ms
35.9% faster · was 10.1 ms
torch.compile (inductor)
H100
2026-03-05
VisionAttentionfp32 · [2, 128, 128, 128]
19.2ms
48.1% faster · was 37.0 ms
torch.compile (inductor)
H100
2026-03-05
ConvolutionalVisionTransformerfp32 · [10, 3, 32, 32]
1.15ms
25.8% faster · was 1.55 ms
torch.compile (inductor)
H100
2026-03-05
VanillaRNNfp32 · [256, 16384]
11.0ms
16.7% faster · was 13.2 ms
torch.compile (inductor)
H100
2026-03-05
VanillaRNNHiddenfp32 · [256, 8, 1024]
10.6ms
15.2% faster · was 12.5 ms
torch.compile (inductor)
H100
2026-03-05
LSTMCnfp32 · [10, 512, 128]
29.5ms
0.3% faster · was 29.6 ms
torch.compile (inductor)
H100
2026-03-05
Matmul BatchNorm BiasAdd Divide Swishfp32 · [1024, 8192]
4.86ms
3.8% faster · was 5.05 ms
torch.compile (inductor)
H100
2026-03-05
Matmul AvgPool GELU Scale Maxfp32 · [1024, 8192]
4.83ms
1.0% faster · was 4.88 ms
torch.compile (inductor)
H100
2026-03-05
ConvTranspose3d Clamp Min Dividefp32 · [16, 64, 24, 48, 48]
12.8ms
37.9% faster · was 20.6 ms
torch.compile (inductor)
H100
2026-03-05
ShallowWideMLPfp32 · [128, 16384]
20.8ms
0.5% faster · was 20.9 ms
torch.compile (inductor)
H100
2026-03-05
DeepNarrowMLPfp32 · [1024, 8192]
2.53ms
9.0% faster · was 2.78 ms
torch.compile (inductor)
H100
2026-03-05
LeNet5fp32 · [4096, 1, 32, 32]
3.71ms
5.6% faster · was 3.93 ms
torch.compile (inductor)
H100
2026-03-05
AlexNetfp32 · [1024, 3, 224, 224]
24.3ms
31.7% faster · was 35.6 ms
torch.compile (inductor)
H100
2026-03-05
GoogleNetInceptionModulefp32 · [10, 480, 224, 224]
10.7ms
17.7% faster · was 13.0 ms
torch.compile (inductor)
H100
2026-03-05
GoogleNetInceptionV1fp32 · [10, 3, 224, 224]
1.82ms
15.3% faster · was 2.15 ms
torch.compile (inductor)
H100
2026-03-05
ResNetBasicBlockfp32 · [10, 3, 224, 224]
1.36ms
40.9% faster · was 2.30 ms
torch.compile (inductor)
H100
2026-03-05
VGG16fp32 · [10, 3, 224, 224]
3.12ms
36.2% faster · was 4.89 ms
torch.compile (inductor)
H100
2026-03-05
VGG19fp32 · [10, 3, 224, 224]
3.64ms
34.4% faster · was 5.55 ms
torch.compile (inductor)
H100
2026-03-05
DenseNet121TransitionLayerfp32 · [128, 32, 256, 256]
5.06ms
54.0% faster · was 11.0 ms
torch.compile (inductor)
H100
2026-03-05
DenseNet121DenseBlockfp32 · [10, 32, 224, 224]
7.84ms
25.3% faster · was 10.5 ms
torch.compile (inductor)
H100
2026-03-05
SqueezeNetFireModulefp32 · [128, 3, 256, 256]
15.3ms
24.3% faster · was 20.2 ms
torch.compile (inductor)
H100
2026-03-05
SqueezeNetfp32 · [64, 3, 512, 512]
21.0ms
46.2% faster · was 39.0 ms
torch.compile (inductor)
H100
2026-03-05
EfficientNetMBConvfp32 · [10, 112, 224, 224]
5.73ms
30.8% faster · was 8.28 ms
torch.compile (inductor)
H100
2026-03-05
ConvTranspose3d BatchNorm AvgPool AvgPoolfp32 · [64, 3, 32, 32, 32]
6.94ms
53.1% faster · was 14.8 ms
torch.compile (inductor)
H100
2026-03-05
Conv2d BatchNorm Scalingfp32 · [128, 8, 128, 128]
1.65ms
53.1% faster · was 3.52 ms
torch.compile (inductor)
H100
2026-03-05
ConvTranspose3d LeakyReLU Multiply LeakyReLU Maxfp32 · [16, 16, 16, 32, 32]
1.28ms
41.0% faster · was 2.17 ms
torch.compile (inductor)
H100
2026-03-05
Gemm GroupNorm Min BiasAddfp32 · [1024, 8192]
4.87ms
6.0% faster · was 5.18 ms
torch.compile (inductor)
H100
2026-03-05
Gemm Add ReLUfp32 · [1024, 8192]
5.05ms
1.2% faster · was 5.11 ms
torch.compile (inductor)
H100
2026-03-05

551 more records · Open in the records ledger →

Measured kernels

Implementation
Operation
Runtime
Best latency
Evidence
Availability
PyTorch
Softmax
python
6.14µs
Reported
BSD-3-Clause · source
PyTorch
ReLU squared
python
8.67µs
Reported
BSD-3-Clause · source
PyTorch
RMSNorm, Megatron layout
python
12.7µs
Reported
BSD-3-Clause · source
PyTorch
SwiGLU activation, Megatron layout
python
23.2µs
Reported
BSD-3-Clause · source
torch.compilePyTorch
Embedding lookup
python
39.7µs
Reported
BSD-3-Clause · source
PyTorch
Multi-token attention
python
63.5µs
Reported
BSD-3-Clause · source
PyTorch
Sparsemax
python
181.2µs
Reported
BSD-3-Clause · source
torch.compile (inductor)PyTorch
MinGPTNewGelu
python · torch_compile_inductor
214.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
CrossEntropyLoss
python · torch_compile_inductor
292.0µs
Reported
MIT · source
PyTorch
Fused neighborhood attention
python
410.8µs
Reported
BSD-3-Clause · source
Show all 40 kernels ›
torch.compile (inductor)PyTorch
LayerNorm
python · torch_compile_inductor
460.0µs
Reported
MIT · source
PyTorch eagerPyTorch
CrossEntropyLoss
python · torch_eager
477.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
Conv3d Multiply InstanceNorm Clamp Multiply Max
python · torch_compile_inductor
508.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
Conv3d Softmax MaxPool MaxPool
python · torch_compile_inductor
621.0µs
Reported
MIT · source
PyTorch
Total variation distance loss
python
728.8µs
Reported
BSD-3-Clause · source
torch.compile (inductor)PyTorch
LeNet5
python · torch_compile_inductor
769.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
KLDivLoss
python · torch_compile_inductor
779.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
Conv2d ReLU HardSwish
python · torch_compile_inductor
814.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
ConvTranspose3d LeakyReLU Multiply LeakyReLU Max
python · torch_compile_inductor
816.0µs
Reported
MIT · source
PyTorch eagerPyTorch
LeNet5
python · torch_eager
821.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
Conv2d HardSwish ReLU
python · torch_compile_inductor
837.0µs
Reported
MIT · source
PyTorch eagerPyTorch
Conv3d Softmax MaxPool MaxPool
python · torch_eager
864.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
Conv3d Min Softmax
python · torch_compile_inductor
867.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
ResNetBasicBlock
python · torch_compile_inductor
875.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
ConvTranspose2d Multiply GlobalAvgPool GlobalAvgPool Mean
python · torch_compile_inductor
911.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
ConvTranspose2d Min Sum GELU Add
python · torch_compile_inductor
924.0µs
Reported
MIT · source
torch.compile (inductor)PyTorch
NetVladNoGhostClusters
python · torch_compile_inductor
953.0µs
Reported
MIT · source
PyTorch eagerPyTorch
Conv3d Multiply InstanceNorm Clamp Multiply Max
python · torch_eager
965.0µs
Reported
MIT · source
PyTorch eagerPyTorch
Conv3d Min Softmax
python · torch_eager
969.0µs
Reported
MIT · source
PyTorch eagerPyTorch
3D tensor matrix multiplication
python · torch_eager
1.04ms
Reported
MIT · source
torch.compile (inductor)PyTorch
3D tensor matrix multiplication
python · torch_compile_inductor
1.05ms
Reported
MIT · source
torch.compile (inductor)PyTorch
TripletMarginLoss
python · torch_compile_inductor
1.06ms
Reported
MIT · source
torch.compile (inductor)PyTorch
ConvTranspose3d Sum ResidualAdd Multiply ResidualAdd
python · torch_compile_inductor
1.13ms
Reported
MIT · source
torch.compile (inductor)PyTorch
ConvolutionalVisionTransformer
python · torch_compile_inductor
1.15ms
Reported
MIT · source
torch.compile (inductor)PyTorch
RegNet
python · torch_compile_inductor
1.17ms
Reported
MIT · source
torch.compile (inductor)PyTorch
Conv2d BatchNorm Scaling
python · torch_compile_inductor
1.25ms
Reported
MIT · source
PyTorch eagerPyTorch
EfficientNetB2
python · torch_eager
1.27ms
Reported
MIT · source
PyTorch eagerPyTorch
Conv3d GroupNorm Mean
python · torch_eager
1.27ms
Reported
MIT · source
torch.compile (inductor)PyTorch
Conv2d Divide LeakyReLU
python · torch_compile_inductor
1.30ms
Reported
MIT · source
PyTorch eagerPyTorch
Matmul with large K dimension
python · torch_eager
1.30ms
Reported
MIT · source

Showing 40 of 509; the full list is in the API dossier below.

Record activity

Nov
Dec
Jan
Feb
807Mar
Apr
5May
16Jun
Jul
Aug
Sep
Oct
Sources: Liger-Kernel benchmarks (2026-07-26) · BSD-2-Clause · KernelBench baseline timings (2026-03-05) · MITlast observed 2026-07-26JSON