Skip to content
KernelIndex
Search⌘K
Sign in

Basic transformer block

25 eligible runs
other

Complete BasicTransformerBlock combining self-attention, cross-attention, and feed-forward network with GEGLU activation. This is the fundamental building block in SDXL Refiner's UNet architecture, processing spatial features through self-attention (spatial coherence), cross-attention (text/image conditioning), and MLP with residual connections.

Fastest reported
703.3µsmean · 59% of SOL
RIAC as wellLicense unknown · unknown

Reported evidence · last observed 2026-08-14. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencydim = 1280 · inner_dim = 3840 · ff_gate_dim = 10240 · encoder_seq_len = 77 · ff_intermediate = 5120 · attention_head_dim = 24 · cross_attention_dim = 1280 · num_attention_heads = 16016 cases
spatial_seq_len
batch_size
dtype
12864fp322564fp3225616fp3225632fp324492fp325122fp325124fp326132fp329971fp3210242fp3210244fp3210248fp3218794fp3220481fp3240932fp3240961fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1703.3 µsRIAC as well17NVIDIA B200 · env 2708.1 µsRecursive8

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 17 results · last observed 2026-08-22Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
RIAC as well
703.3µs59% SOL
1.00×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Amir M. Mir | SF Tensor
721.8µs58% SOL
1.03×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Recursive
730.5µs57% SOL
1.04×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Rapid Tiger
766.0µs56% SOL
1.09×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Hyra
769.6µs55% SOL
1.09×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Amir M. Mir | SF Tensor
785.8µs55% SOL
1.12×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
doubleAI
802.7µs54% SOL
1.14×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Gleaming Coyote
804.5µs54% SOL
1.14×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Electric Cheetah
810.4µs54% SOL
1.15×
Reported · license unknown · no source
2026-08-07

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
ac4k
817.0µs54% SOL
1.16×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Electric Cheetah
817.1µs54% SOL
1.16×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Geometric
844.6µs52% SOL
1.20×
Reported · license unknown · no source
2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Geometric
854.5µs52% SOL
1.22×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Proud Wren @ NVIDIA
1.06ms45% SOL
1.51×
Reported · license unknown · no source
2026-07-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Geometric
1.15ms43% SOL
1.64×
Reported · license unknown · no source
2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
Geometric
1.39ms37% SOL
1.98×
Reported · license unknown · no source
2026-07-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Cursor
1.93ms30% SOL
2.75×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Recursive
unknown
708.1µs
1.01×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
721.8µs
1.03×
Reported
license unknown · no source
Rapid Tiger
unknown
766.0µs
1.09×
Reported
license unknown · no source
doubleAI
unknown
771.4µs
1.10×
Reported
license unknown · no source
Hyra
unknown
769.6µs
1.09×
Reported
license unknown · no source
Gleaming Coyote
unknown
804.5µs
1.14×
Reported
license unknown · no source
Electric Cheetah
unknown
810.4µs
1.15×
Reported
license unknown · no source
ac4k
unknown
817.0µs
1.16×
Reported
license unknown · no source
Show all 25 implementations ›
Geometric
unknown
844.6µs
1.20×
Reported
license unknown · no source
Cursor
unknown
986.3µs
1.40×
Reported
license unknown · no source
koookkk321
unknown
1.03ms
1.46×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
1.06ms
1.51×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
785.8µs
1.12×
Reported
license unknown · no source
doubleAI
unknown
788.6µs
1.12×
Reported
license unknown · no source
doubleAI
unknown
802.7µs
1.14×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
818.6µs
1.16×
Reported
license unknown · no source
Electric Cheetah
unknown
817.1µs
1.16×
Reported
license unknown · no source
Geometric
unknown
854.5µs
1.22×
Reported
license unknown · no source
Geometric
unknown
1.15ms
1.64×
Reported
license unknown · no source
Geometric
unknown
1.39ms
1.98×
Reported
license unknown · no source
Cursor
unknown
1.93ms
2.75×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
14.4ms
20.50×
Reported
license unknown · no source
RIAC as well
unknown
703.3µs
1.00×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
723.3µs
1.03×
Reported
license unknown · no source
Recursive
unknown
730.5µs
1.04×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesfp32 [batch_size, spatial_seq_len, dim]
encoder_hidden_statesfp32 [batch_size, encoder_seq_len, cross_attention_dim]
norm1_weightfp32 [dim]
norm1_biasfp32 [dim]
attn1_to_q_weightfp32 [inner_dim, dim]
attn1_to_k_weightfp32 [inner_dim, dim]
attn1_to_v_weightfp32 [inner_dim, dim]
attn1_to_out_weightfp32 [dim, inner_dim]
attn1_to_out_biasfp32 [dim]
norm2_weightfp32 [dim]
norm2_biasfp32 [dim]
attn2_to_q_weightfp32 [inner_dim, dim]
attn2_to_k_weightfp32 [inner_dim, cross_attention_dim]
attn2_to_v_weightfp32 [inner_dim, cross_attention_dim]
attn2_to_out_weightfp32 [dim, inner_dim]
attn2_to_out_biasfp32 [dim]
norm3_weightfp32 [dim]
norm3_biasfp32 [dim]
ff_linear1_weightfp32 [ff_gate_dim, dim]
ff_linear1_biasfp32 [ff_gate_dim]
ff_linear2_weightfp32 [dim, ff_intermediate]
ff_linear2_biasfp32 [dim]
norm_epsfp32 scalar
outputfp32 [batch_size, spatial_seq_len, dim]
Axes and behavior
dimconstant = 1280
inner_diminner_dim = num_attention_heads * attention_head_dim
batch_sizevariable
ff_gate_dimff_gate_dim = dim * 8
encoder_seq_lenvariable
ff_intermediateff_intermediate = dim * 4
spatial_seq_lenvariable
attention_head_dimconstant = 24
cross_attention_dimconstant = 1280
num_attention_headsconstant = 160
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias070_basic_transformer_blockmodelstable-diffusion-xl-refiner-1-0sha256eec4fc095537…
Sources: NVIDIA SOL-ExecBench (2026-08-22)last observed 2026-08-22How records are decidedJSON