Fused final layer upsample with adaptive norm

36 eligible runs

rmsnorm

Fused UNet upsampling path that combines timestep-conditioned adaptive group normalization with ResBlock upsampling and final convolution to convert transformer hidden states back to VAE latent space. This critical operation runs at every diffusion step for image generation.

Fastest reported

1.58msmean · 82% of SOL

Amir M. Mir | SF TensorLicense unknown · unknown

Reported evidence · last observed 2026-07-26. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencypatch_size = 1 · hidden_size = 4096 · emb_channels = 4096 · out_channels = 4 · hidden_channels = 3072 · double_hidden_channels = 614416 cases

token_h

token_w

batch_size

dtype

11111bf16 16164bf16 16168bf16 161664bf16 17172bf16 23238bf16 29291bf16 32328bf16 323216bf16 323232bf16 41412bf16 47478bf16 48482bf16 53531bf16 59592bf16 64641bf16

scrolls · 16 cases total

Aggregatessuite of 16 cases · mean latency

HardwareBest knownImplementationRuns

NVIDIA B200 · env 11.58 msAmir M. Mir | SF Tensor27 NVIDIA B200 · env 23.49 msdoubleAI9

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 27 results · last observed 2026-08-23Record history →

#

Implementation

Latency

vs #1

Trust

Observed

1

Amir M. Mir | SF Tensor

1.58ms82% SOL

1.00×

Reported · license unknown · no source

2026-07-26

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

2

2.59ms63% SOL

1.64×

Reported · license unknown · no source

2026-07-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

3

Amir M. Mir | SF Tensor

3.51ms54% SOL

2.22×

Reported · license unknown · no source

2026-07-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

4

3.54ms55% SOL

2.24×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

5

3.71ms52% SOL

2.35×

Reported · license unknown · no source

2026-07-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

6

Proud Wren @ NVIDIA

3.71ms54% SOL

2.35×

Reported · license unknown · no source

2026-07-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

7

3.79ms52% SOL

2.40×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

8

3.86ms51% SOL

2.44×

Reported · license unknown · no source

2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

9

3.87ms51% SOL

2.45×

Reported · license unknown · no source

2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

10

3.91ms51% SOL

2.48×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

11

3.91ms51% SOL

2.48×

Reported · license unknown · no source

2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

12

3.92ms51% SOL

2.48×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

13

3.97ms51% SOL

2.51×

Reported · license unknown · no source

2026-08-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

14

3.98ms50% SOL

2.52×

Reported · license unknown · no source

2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

15

3.98ms50% SOL

2.52×

Reported · license unknown · no source

2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

16

Amir M. Mir | SF Tensor

4.05ms50% SOL

2.57×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

17

4.08ms50% SOL

2.58×

Reported · license unknown · no source

2026-07-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

18

4.15ms49% SOL

2.63×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

19

4.16ms49% SOL

2.63×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

20

4.27ms48% SOL

2.70×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

21

4.28ms48% SOL

2.71×

Reported · license unknown · no source

2026-08-07

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

22

4.45ms47% SOL

2.82×

Reported · license unknown · no source

2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

23

4.49ms47% SOL

2.85×

Reported · license unknown · no source

2026-08-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

24

4.50ms46% SOL

2.85×

Reported · license unknown · no source

2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

25

4.53ms46% SOL

2.87×

Reported · license unknown · no source

2026-08-09

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

26

4.56ms46% SOL

2.89×

Reported · license unknown · no source

2026-08-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

27

4.58ms46% SOL

2.90×

Reported · license unknown · no source

2026-08-09

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation

Runtime

Best latency

Evidence

Availability

Amir M. Mir | SF Tensor

unknown

1.58ms

1.00×

Reported

license unknown · no source

unknown

2.59ms

1.64×

Reported

license unknown · no source

unknown

3.54ms

2.24×

Reported

license unknown · no source

unknown

3.49ms

2.21×

Reported

license unknown · no source

Proud Wren @ NVIDIA

unknown

3.71ms

2.35×

Reported

license unknown · no source

unknown

3.71ms

2.35×

Reported

license unknown · no source

unknown

3.74ms

2.37×

Reported

license unknown · no source

unknown

3.86ms

2.44×

Reported

license unknown · no source

Show all 36 implementations ›

unknown

4.05ms

2.57×

Reported

license unknown · no source

unknown

4.07ms

2.58×

Reported

license unknown · no source

unknown

4.08ms

2.58×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

3.51ms

2.22×

Reported

license unknown · no source

unknown

3.79ms

2.40×

Reported

license unknown · no source

unknown

3.87ms

2.45×

Reported

license unknown · no source

unknown

3.91ms

2.48×

Reported

license unknown · no source

unknown

3.91ms

2.48×

Reported

license unknown · no source

unknown

3.92ms

2.48×

Reported

license unknown · no source

unknown

3.97ms

2.51×

Reported

license unknown · no source

unknown

3.98ms

2.52×

Reported

license unknown · no source

unknown

3.98ms

2.52×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

4.05ms

2.57×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

4.05ms

2.57×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

4.07ms

2.58×

Reported

license unknown · no source

unknown

4.08ms

2.58×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

4.12ms

2.61×

Reported

license unknown · no source

unknown

4.15ms

2.63×

Reported

license unknown · no source

unknown

4.16ms

2.63×

Reported

license unknown · no source

unknown

4.27ms

2.70×

Reported

license unknown · no source

unknown

4.28ms

2.71×

Reported

license unknown · no source

unknown

4.45ms

2.82×

Reported

license unknown · no source

unknown

4.49ms

2.85×

Reported

license unknown · no source

unknown

4.50ms

2.85×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

4.50ms

2.85×

Reported

license unknown · no source

unknown

4.53ms

2.87×

Reported

license unknown · no source

unknown

4.56ms

2.89×

Reported

license unknown · no source

unknown

4.58ms

2.90×

Reported

license unknown · no source

Semantics

Inputs and outputs

xbf16 [batch_size, seq_len, hidden_size]

timestep_embbf16 [batch_size, emb_channels]

time_emb_mlp_linear_weightbf16 [double_hidden_channels, emb_channels]

time_emb_mlp_linear_biasbf16 [double_hidden_channels]

resblock_in_norm_weightbf16 [hidden_size]

resblock_in_norm_biasbf16 [hidden_size]

resblock_in_conv_weightbf16 [hidden_channels, hidden_size, 3, 3]

resblock_in_conv_biasbf16 [hidden_channels]

resblock_emb_linear_weightbf16 [double_hidden_channels, emb_channels]

resblock_emb_linear_biasbf16 [double_hidden_channels]

resblock_out_norm_weightbf16 [hidden_channels]

resblock_out_norm_biasbf16 [hidden_channels]

resblock_out_conv_weightbf16 [hidden_channels, hidden_channels, 3, 3]

resblock_out_conv_biasbf16 [hidden_channels]

resblock_skip_conv_weightbf16 [hidden_channels, hidden_size, 1, 1]

resblock_skip_conv_biasbf16 [hidden_channels]

final_norm_weightbf16 [hidden_channels]

final_norm_biasbf16 [hidden_channels]

final_conv_weightbf16 [out_channels, hidden_channels, 3, 3]

final_conv_biasbf16 [out_channels]

epsfp32 scalar

outputbf16 [batch_size, out_channels, token_h, token_w]

Axes and behavior

seq_lenseq_len = token_h * token_w

token_hvariable

token_wvariable

batch_sizevariable

patch_sizeconstant = 1

hidden_sizeconstant = 4096

emb_channelsconstant = 4096

out_channelsconstant = 4

hidden_channelsconstant = 3072

double_hidden_channelsdouble_hidden_channels = hidden_channels * 2

determinismunspecified

constraintsNo mutation or aliasing

Identity

alias078_fused_final_layer_upsample_with_adaptive_normmodelhunyuanimage-3-0sha25627828a15952c…

Sources: NVIDIA SOL-ExecBench (2026-08-23)last observed 2026-08-23How records are decided JSON