Skip to content
KernelIndex
Search⌘K
Sign in

Text decoder layer with self attention and MLP

37 eligible runs
mlp

Complete Mllama text decoder layer combining self-attention with RoPE, RMSNorm, gated MLP, and residual connections. This is the dominant computational pattern in the text model (32 out of 40 layers). Architecture: (1) RMSNorm + self-attention with RoPE and GQA (32 heads, 8 KV heads) + residual, (2) RMSNorm + gated MLP with SiLU + residual.

Fastest reported
669.2µsmean · 72% of SOL
Gleaming CoyoteLicense unknown · unknown

Reported evidence · last observed 2026-08-14. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 128 · num_heads = 32 · hidden_size = 4096 · kv_head_dim = 1024 · qkv_head_dim = 4096 · intermediate_size = 14336 · num_key_value_heads = 8 · num_key_value_groups = 416 cases
seq_len
batch_size
dtype
1284bf161288bf1612816bf1612864bf161311bf162112bf162561bf162562bf162568bf1625632bf1625664bf165122bf165124bf1610241bf1640961bf1681921bf16
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1669.2 µsGleaming Coyote29NVIDIA B200 · env 2682.0 µsRecursive8

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 29 results · last observed 2026-08-23Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
Gleaming Coyote
669.2µs72% SOL
1.00×
Reported · license unknown · no source
2026-08-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
RIAC as well
674.2µs76% SOL
1.01×
Reported · license unknown · no source
2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
ac4k
700.6µs69% SOL
1.05×
Reported · license unknown · no source
2026-08-05

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
imec.AILabs
701.4µs75% SOL
1.05×
Reported · license unknown · no source
2026-08-03

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Amir M. Mir | SF Tensor
704.5µs69% SOL
1.05×
Reported · license unknown · no source
2026-08-23

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
RIAC as well
705.1µs73% SOL
1.05×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
imec.AILabs
709.9µs70% SOL
1.06×
Reported · license unknown · no source
2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Recursive
711.0µs70% SOL
1.06×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
RIAC as well
711.3µs72% SOL
1.06×
Reported · license unknown · no source
2026-08-19

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Hyra
716.2µs68% SOL
1.07×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
RIAC as well
717.7µs72% SOL
1.07×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
imec.AILabs
726.2µs68% SOL
1.09×
Reported · license unknown · no source
2026-08-02

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Amir M. Mir | SF Tensor
738.6µs67% SOL
1.10×
Reported · license unknown · no source
2026-07-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
doubleAI
754.3µs69% SOL
1.13×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Geometric
759.3µs66% SOL
1.13×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
Geometric
762.1µs66% SOL
1.14×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Proud Wren @ NVIDIA
769.2µs65% SOL
1.15×
Reported · license unknown · no source
2026-07-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
18
Geometric
806.2µs63% SOL
1.20×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
19
Geometric
823.4µs62% SOL
1.23×
Reported · license unknown · no source
2026-07-25

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
20
Geometric
823.6µs62% SOL
1.23×
Reported · license unknown · no source
2026-08-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
21
genproof.com
842.7µs60% SOL
1.26×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
22
Geometric
844.5µs62% SOL
1.26×
Reported · license unknown · no source
2026-08-21

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
23
Geometric
855.8µs60% SOL
1.28×
Reported · license unknown · no source
2026-08-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
24
Geometric
884.6µs59% SOL
1.32×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
25
Amir M. Mir | SF Tensor
889.8µs58% SOL
1.33×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
26
Geometric
896.9µs58% SOL
1.34×
Reported · license unknown · no source
2026-08-18

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
27
Geometric
925.7µs56% SOL
1.38×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
28
Geometric
970.6µs55% SOL
1.45×
Reported · license unknown · no source
2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
29
Geometric
2.83ms21% SOL
4.22×
Reported · license unknown · no source
2026-08-13

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
RIAC as well
unknown
674.2µs
1.01×
Reported
license unknown · no source
Recursive
unknown
682.0µs
1.02×
Reported
license unknown · no source
ac4k
unknown
700.6µs
1.05×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
704.5µs
1.05×
Reported
license unknown · no source
Hyra
unknown
716.2µs
1.07×
Reported
license unknown · no source
Geometric
unknown
759.3µs
1.13×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
769.2µs
1.15×
Reported
license unknown · no source
genproof.com
unknown
805.6µs
1.20×
Reported
license unknown · no source
Show all 37 implementations ›
Cursor
unknown
1.19ms
1.78×
Reported
license unknown · no source
RIAC as well
unknown
705.1µs
1.05×
Reported
license unknown · no source
RIAC as well
unknown
711.3µs
1.06×
Reported
license unknown · no source
RIAC as well
unknown
717.7µs
1.07×
Reported
license unknown · no source
imec.AILabs
unknown
709.9µs
1.06×
Reported
license unknown · no source
Recursive
unknown
711.0µs
1.06×
Reported
license unknown · no source
doubleAI
unknown
754.3µs
1.13×
Reported
license unknown · no source
imec.AILabs
unknown
726.2µs
1.09×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
738.6µs
1.10×
Reported
license unknown · no source
Geometric
unknown
762.1µs
1.14×
Reported
license unknown · no source
Geometric
unknown
806.2µs
1.20×
Reported
license unknown · no source
Geometric
unknown
844.5µs
1.26×
Reported
license unknown · no source
Geometric
unknown
823.4µs
1.23×
Reported
license unknown · no source
Geometric
unknown
823.6µs
1.23×
Reported
license unknown · no source
genproof.com
unknown
842.7µs
1.26×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
853.5µs
1.28×
Reported
license unknown · no source
Geometric
unknown
855.8µs
1.28×
Reported
license unknown · no source
Geometric
unknown
884.6µs
1.32×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
873.6µs
1.31×
Reported
license unknown · no source
Geometric
unknown
896.9µs
1.34×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
889.8µs
1.33×
Reported
license unknown · no source
Geometric
unknown
925.7µs
1.38×
Reported
license unknown · no source
Geometric
unknown
970.6µs
1.45×
Reported
license unknown · no source
doubleAI
unknown
1.02ms
1.52×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
1.72ms
2.56×
Reported
license unknown · no source
Geometric
unknown
2.83ms
4.22×
Reported
license unknown · no source
imec.AILabs
unknown
701.4µs
1.05×
Reported
license unknown · no source
Gleaming Coyote
unknown
669.2µs
1.00×
Reported
license unknown · no source
doubleAI
unknown
729.0µs
1.09×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesbf16 [batch_size, seq_len, hidden_size]
attention_maskbf16 [batch_size, 1, seq_len, seq_len]
q_proj_weightbf16 [qkv_head_dim, hidden_size]
k_proj_weightbf16 [kv_head_dim, hidden_size]
v_proj_weightbf16 [kv_head_dim, hidden_size]
o_proj_weightbf16 [hidden_size, qkv_head_dim]
gate_proj_weightbf16 [intermediate_size, hidden_size]
up_proj_weightbf16 [intermediate_size, hidden_size]
down_proj_weightbf16 [hidden_size, intermediate_size]
input_layernorm_weightbf16 [hidden_size]
post_attention_layernorm_weightbf16 [hidden_size]
rms_norm_epsfp32 scalar
rope_thetafp32 scalar
outputbf16 [batch_size, seq_len, hidden_size]
Axes and behavior
seq_lenvariable
head_dimconstant = 128
num_headsconstant = 32
batch_sizevariable
hidden_sizeconstant = 4096
kv_head_dimkv_head_dim = num_key_value_heads * head_dim
qkv_head_dimqkv_head_dim = num_heads * head_dim
intermediate_sizeconstant = 14336
num_key_value_headsconstant = 8
num_key_value_groupsconstant = 4
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias053_text_decoder_layer_with_self_attention_and_mlpmodelllama-3-2-11b-visionsha2563d5a1395268e…
Sources: NVIDIA SOL-ExecBench (2026-08-23)last observed 2026-08-23How records are decidedJSON