Decoder layer fused attention MLP

32 eligible runs

mlp

Fused Qwen2VL decoder layer combining input RMSNorm, multimodal 3D RoPE attention with GQA (28 query heads, 4 KV heads), residual connection, post-attention RMSNorm, SwiGLU MLP, and final residual. This represents the complete decoder layer computation path.

Fastest reported

827.2µsmean · 68% of SOL

Amir M. Mir | SF TensorLicense unknown · unknown

Reported evidence · last observed 2026-07-28. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencykv_dim = 512 · qkv_dim = 3584 · head_dim = 128 · num_heads = 28 · hidden_size = 3584 · num_kv_heads = 4 · num_rope_dims = 3 · intermediate_size = 1894416 cases

kv_seq_len

seq_len

batch_size

dtype

1281281fp32 1281284fp32 12812816fp32 12812832fp32 12812864fp32 1311311fp32 2562562fp32 2562568fp32 25625632fp32 3733738fp32 5125128fp32 102410241fp32 157115712fp32 204820481fp32 409640961fp32 819281921fp32

scrolls · 16 cases total

Aggregatessuite of 16 cases · mean latency

HardwareBest knownImplementationRuns

NVIDIA B200 · env 1827.2 µsAmir M. Mir | SF Tensor25 NVIDIA B200 · env 21.15 msdoubleAI7

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 25 results · last observed 2026-08-23Record history →

#

Implementation

Latency

vs #1

Trust

Observed

1

Amir M. Mir | SF Tensor

827.2µs68% SOL

1.00×

Reported · license unknown · no source

2026-07-28

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

2

910.0µs62% SOL

1.10×

Reported · license unknown · no source

2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

3

1.01ms58% SOL

1.22×

Reported · license unknown · no source

2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

4

1.12ms54% SOL

1.35×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

5

1.14ms52% SOL

1.38×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

6

1.15ms52% SOL

1.39×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

7

1.16ms52% SOL

1.40×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

8

1.16ms51% SOL

1.40×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

9

1.16ms51% SOL

1.41×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

10

1.17ms51% SOL

1.41×

Reported · license unknown · no source

2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

11

Amir M. Mir | SF Tensor

1.18ms51% SOL

1.42×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

12

1.19ms50% SOL

1.44×

Reported · license unknown · no source

2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

13

1.19ms50% SOL

1.44×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

14

1.20ms50% SOL

1.44×

Reported · license unknown · no source

2026-08-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

15

1.21ms50% SOL

1.46×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

16

1.21ms49% SOL

1.47×

Reported · license unknown · no source

2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

17

1.22ms49% SOL

1.47×

Reported · license unknown · no source

2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

18

1.23ms49% SOL

1.48×

Reported · license unknown · no source

2026-08-16

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

19

Proud Wren @ NVIDIA

1.23ms49% SOL

1.49×

Reported · license unknown · no source

2026-07-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

20

1.26ms47% SOL

1.52×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

21

1.27ms47% SOL

1.53×

Reported · license unknown · no source

2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

22

1.29ms46% SOL

1.56×

Reported · license unknown · no source

2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

23

1.30ms47% SOL

1.57×

Reported · license unknown · no source

2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

24

1.34ms45% SOL

1.62×

Reported · license unknown · no source

2026-07-28

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

25

1.67ms36% SOL

2.02×

Reported · license unknown · no source

2026-07-25

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation

Runtime

Best latency

Evidence

Availability

unknown

1.12ms

1.35×

Reported

license unknown · no source

unknown

1.16ms

1.40×

Reported

license unknown · no source

unknown

1.15ms

1.39×

Reported

license unknown · no source

unknown

1.21ms

1.46×

Reported

license unknown · no source

Proud Wren @ NVIDIA

unknown

1.23ms

1.49×

Reported

license unknown · no source

unknown

1.14ms

1.38×

Reported

license unknown · no source

unknown

1.15ms

1.39×

Reported

license unknown · no source

unknown

1.16ms

1.40×

Reported

license unknown · no source

Show all 32 implementations ›

unknown

1.16ms

1.40×

Reported

license unknown · no source

unknown

1.16ms

1.41×

Reported

license unknown · no source

unknown

1.17ms

1.41×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

1.18ms

1.42×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

1.19ms

1.43×

Reported

license unknown · no source

unknown

1.19ms

1.44×

Reported

license unknown · no source

unknown

1.19ms

1.44×

Reported

license unknown · no source

unknown

1.20ms

1.44×

Reported

license unknown · no source

unknown

1.21ms

1.46×

Reported

license unknown · no source

unknown

1.21ms

1.47×

Reported

license unknown · no source

unknown

1.22ms

1.47×

Reported

license unknown · no source

unknown

1.23ms

1.48×

Reported

license unknown · no source

unknown

1.25ms

1.51×

Reported

license unknown · no source

unknown

1.26ms

1.52×

Reported

license unknown · no source

unknown

1.27ms

1.53×

Reported

license unknown · no source

unknown

1.30ms

1.57×

Reported

license unknown · no source

unknown

1.29ms

1.56×

Reported

license unknown · no source

unknown

1.34ms

1.62×

Reported

license unknown · no source

unknown

1.67ms

2.02×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

2.82ms

3.41×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

21.7ms

26.24×

Reported

license unknown · no source

Amir M. Mir | SF Tensor

unknown

827.2µs

1.00×

Reported

license unknown · no source

unknown

910.0µs

1.10×

Reported

license unknown · no source

unknown

1.01ms

1.22×

Reported

license unknown · no source

Semantics

Inputs and outputs

hidden_statesfp32 [batch_size, seq_len, hidden_size]

input_layernorm_weightfp32 [hidden_size]

q_proj_weightfp32 [qkv_dim, hidden_size]

q_proj_biasfp32 [qkv_dim]

k_proj_weightfp32 [kv_dim, hidden_size]

k_proj_biasfp32 [kv_dim]

v_proj_weightfp32 [kv_dim, hidden_size]

v_proj_biasfp32 [kv_dim]

o_proj_weightfp32 [hidden_size, qkv_dim]

rope_cosfp32 [num_rope_dims, batch_size, kv_seq_len, head_dim]

rope_sinfp32 [num_rope_dims, batch_size, kv_seq_len, head_dim]

post_attention_layernorm_weightfp32 [hidden_size]

gate_proj_weightfp32 [intermediate_size, hidden_size]

up_proj_weightfp32 [intermediate_size, hidden_size]

down_proj_weightfp32 [hidden_size, intermediate_size]

rms_norm_epsfp32 scalar

outputfp32 [batch_size, seq_len, hidden_size]

Axes and behavior

kv_dimkv_dim = num_kv_heads * head_dim

qkv_dimqkv_dim = num_heads * head_dim

seq_lenvariable

head_dimconstant = 128

num_headsconstant = 28

batch_sizevariable

kv_seq_lenvariable

hidden_sizeconstant = 3584

num_kv_headsconstant = 4

num_rope_dimsconstant = 3

intermediate_sizeconstant = 18944

determinismunspecified

constraintsNo mutation or aliasing

Identity

alias019_decoder_layer_fused_attention_mlpmodelqwen-image-edit-2509sha2560e804a051e29…

Sources: NVIDIA SOL-ExecBench (2026-08-23)last observed 2026-08-23How records are decided JSON