Skip to content
KernelIndex
Search⌘K
Sign in

Flux ada layer norm zero modulation extraction

23 eligible runs
rmsnorm

AdaLayerNormZero modulation parameter extraction for Flux dual-stream transformer. Projects timestep embeddings through a linear layer and chunks into 6 modulation parameters (shift_msa, scale_msa, gate_msa, shift_mlp, scale_mlp, gate_mlp) that control attention and MLP paths.

Fastest reported
78.1µsmean · 61% of SOL
DatabricksLicense unknown · unknown

Reported evidence · last observed 2026-06-29. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyinner_dim = 3072 · output_dim = 1843216 cases
batch_size
dtype
5fp3216fp3232fp3296fp32128fp32131fp32192fp32211fp32373fp32384fp32449fp32512fp32691fp32773fp32853fp32919fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 178.1 µsDatabricks12NVIDIA B200 · env 268.9 µsAmir M. Mir | SF Tensor11

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.0 · mean · 12 results · last observed 2026-08-24Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
Databricks
78.1µs61% SOL
1.00×
Reported · license unknown · no source
2026-06-29

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
doubleAI
78.3µs61% SOL
1.00×
Reported · license unknown · no source
2026-06-25

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
doubleAI
78.4µs61% SOL
1.00×
Reported · license unknown · no source
2026-05-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
ac4k
79.0µs61% SOL
1.01×
Reported · license unknown · no source
2026-06-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Amir M. Mir | SF Tensor
80.3µs60% SOL
1.03×
Reported · license unknown · no source
2026-05-28

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
Recursive
80.4µs60% SOL
1.03×
Reported · license unknown · no source
2026-06-07

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Argus
89.1µs57% SOL
1.14×
Reported · license unknown · no source
2026-07-05

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Cursor
111.5µs49% SOL
1.43×
Reported · license unknown · no source
2026-04-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Amir M. Mir | SF Tensor
155.7µs40% SOL
1.99×
Reported · license unknown · no source
2026-04-30

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Amir M. Mir | SF Tensor
580.5µs19% SOL
7.43×
Reported · license unknown · no source
2026-04-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
Amir M. Mir | SF Tensor
667.6µs15% SOL
8.55×
Reported · license unknown · no source
2026-04-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Igor Bragaia
667.9µs15% SOL
8.55×
Reported · license unknown · no source
2026-05-28

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Amir M. Mir | SF Tensor
unknown
68.9µs
1.00×
Reported
license unknown · no source
RIAC as well
unknown
71.3µs
1.03×
Reported
license unknown · no source
doubleAI
unknown
78.3µs
1.14×
Reported
license unknown · no source
ac4k
unknown
79.0µs
1.15×
Reported
license unknown · no source
Recursive
unknown
80.4µs
1.17×
Reported
license unknown · no source
Argus
unknown
89.1µs
1.29×
Reported
license unknown · no source
TH3CHARLie | TileGym
unknown
88.9µs
1.29×
Reported
license unknown · no source
Cursor
unknown
111.5µs
1.62×
Reported
license unknown · no source
Show all 23 implementations ›
Igor Bragaia
unknown
667.9µs
9.69×
Reported
license unknown · no source
doubleAI
unknown
78.4µs
1.14×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
80.3µs
1.17×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
81.0µs
1.17×
Reported
license unknown · no source
Databricks
unknown
81.1µs
1.18×
Reported
license unknown · no source
ac4k
unknown
81.1µs
1.18×
Reported
license unknown · no source
Recursive
unknown
81.2µs
1.18×
Reported
license unknown · no source
doubleAI
unknown
83.0µs
1.20×
Reported
license unknown · no source
Cursor
unknown
116.9µs
1.70×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
155.7µs
2.26×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
580.5µs
8.42×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
667.6µs
9.68×
Reported
license unknown · no source
Geometric
unknown
77.9µs
1.13×
Reported
license unknown · no source
Databricks
unknown
78.1µs
1.13×
Reported
license unknown · no source
Hyra
unknown
78.2µs
1.13×
Reported
license unknown · no source

Semantics

Inputs and outputs
embfp32 [batch_size, inner_dim]
weightfp32 [output_dim, inner_dim]
biasfp32 [output_dim]
shift_msafp32 [batch_size, inner_dim]
scale_msafp32 [batch_size, inner_dim]
gate_msafp32 [batch_size, inner_dim]
shift_mlpfp32 [batch_size, inner_dim]
scale_mlpfp32 [batch_size, inner_dim]
gate_mlpfp32 [batch_size, inner_dim]
Axes and behavior
inner_dimconstant = 3072
batch_sizevariable
output_dimoutput_dim = inner_dim * 6
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias035_flux_ada_layer_norm_zero_modulation_extractionmodelflux-1-kontext-devsha2561b981ffaf917…
Sources: NVIDIA SOL-ExecBench (2026-08-24)last observed 2026-08-24How records are decidedJSON