Skip to content
KernelIndex
Search⌘K
Sign in

Dual stream attention with conditional cross attention

24 eligible runs
other

Dual-stream attention block with conditional cross-attention for FIBO model. Implements joint attention between image and text tokens (double layers) or self-attention only (single layers). Includes AdaLN modulation, QKV projection with 24 heads of dimension 128, 3D RoPE positional encoding, attention computation, output projection, and post-attention modulation with residual connection.

Fastest reported
305.2µsmean · 73% of SOL
Amir M. Mir | SF TensorLicense unknown · unknown

Reported evidence · last observed 2026-08-22. Reported by source; not independently reproduced. No public source. License unknown.

No public sourceRun detail →

Current records

Workloadsuite of 16 cases · mean latencyhead_dim = 128 · num_heads = 24 · hidden_size = 3072 · modulation_dim = 18432 · rope_axis0_dim = 16 · rope_axis1_dim = 56 · rope_axis2_dim = 56 · rope_dim_per_axis = 42 · joint_attention_dim = 409616 cases
seq_len
batch_size
text_seq_len
is_joint_block
dtype
12864320fp322561640fp322561641fp322562641fp322568641fp3225632640fp3251211280fp3251211281fp3251241281fp3254141281fp3261311281fp3299712561fp32102442561fp32197315121fp32204825121fp32313615121fp32
scrolls · 16 cases total
Aggregatessuite of 16 cases · mean latency
HardwareBest knownImplementationRuns
NVIDIA B200 · env 1305.2 µsAmir M. Mir | SF Tensor17NVIDIA B200 · env 2359.1 µsRecursive7

Not measured on H100 for this workload. Challenges →

Source-native comparison · GPU NVIDIA B200 · Workload suite of 16 cases · mean latency · Protocol SOL-ExecBench evaluation stack v1.1 · mean · 17 results · last observed 2026-08-22Record history →
#
Implementation
Latency
vs #1
Trust
Observed
1
Amir M. Mir | SF Tensor
305.2µs73% SOL
1.00×
Reported · license unknown · no source
2026-08-22

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
2
Proud Wren @ NVIDIA
345.3µs70% SOL
1.13×
Reported · license unknown · no source
2026-07-11

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
3
Rapid Tiger
346.3µs70% SOL
1.13×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
4
Electric Cheetah
346.3µs70% SOL
1.13×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
5
Gleaming Coyote
346.5µs69% SOL
1.14×
Reported · license unknown · no source
2026-08-15

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
6
ac4k
358.4µs68% SOL
1.17×
Reported · license unknown · no source
2026-08-04

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
7
Recursive
363.0µs68% SOL
1.19×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
8
Hyra
372.9µs67% SOL
1.22×
Reported · license unknown · no source
2026-07-17

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
9
Geometric
387.4µs66% SOL
1.27×
Reported · license unknown · no source
2026-08-20

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
10
Amir M. Mir | SF Tensor
391.3µs66% SOL
1.28×
Reported · license unknown · no source
2026-07-14

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
11
doubleAI
457.0µs61% SOL
1.50×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
12
Geometric
465.0µs61% SOL
1.52×
Reported · license unknown · no source
2026-08-12

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
13
Amir M. Mir | SF Tensor
607.4µs53% SOL
1.99×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
14
Geometric
609.9µs52% SOL
2.00×
Reported · license unknown · no source
2026-08-06

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
15
Cursor
923.5µs41% SOL
3.03×
Reported · license unknown · no source
2026-07-10

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
16
Geometric
1.03ms38% SOL
3.39×
Reported · license unknown · no source
2026-07-26

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →
17
Geometric
1.44ms30% SOL
4.71×
Reported · license unknown · no source
2026-07-24

Measured exactly what you asked. Reported by source; not independently reproduced.

no sourcelicense unknownno install recipeRun detail →

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
Amir M. Mir | SF Tensor
unknown
305.2µs
1.00×
Reported
license unknown · no source
Rapid Tiger
unknown
346.3µs
1.13×
Reported
license unknown · no source
Electric Cheetah
unknown
346.3µs
1.13×
Reported
license unknown · no source
Gleaming Coyote
unknown
346.5µs
1.14×
Reported
license unknown · no source
Hyra
unknown
372.9µs
1.22×
Reported
license unknown · no source
Geometric
unknown
387.4µs
1.27×
Reported
license unknown · no source
doubleAI
unknown
457.0µs
1.50×
Reported
license unknown · no source
Cursor
unknown
820.3µs
2.69×
Reported
license unknown · no source
Show all 24 implementations ›
Recursive
unknown
363.0µs
1.19×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
391.3µs
1.28×
Reported
license unknown · no source
doubleAI
unknown
464.0µs
1.52×
Reported
license unknown · no source
Geometric
unknown
465.0µs
1.52×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
583.5µs
1.91×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
607.4µs
1.99×
Reported
license unknown · no source
Geometric
unknown
609.9µs
2.00×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
833.5µs
2.73×
Reported
license unknown · no source
Cursor
unknown
923.5µs
3.03×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
959.7µs
3.14×
Reported
license unknown · no source
Amir M. Mir | SF Tensor
unknown
1.04ms
3.41×
Reported
license unknown · no source
Geometric
unknown
1.03ms
3.39×
Reported
license unknown · no source
Geometric
unknown
1.44ms
4.71×
Reported
license unknown · no source
Proud Wren @ NVIDIA
unknown
345.3µs
1.13×
Reported
license unknown · no source
Recursive
unknown
359.1µs
1.18×
Reported
license unknown · no source
ac4k
unknown
358.4µs
1.17×
Reported
license unknown · no source

Semantics

Inputs and outputs
hidden_statesfp32 [batch_size, seq_len, hidden_size]
timestep_embeddingfp32 [batch_size, hidden_size]
encoder_hidden_statesfp32 [batch_size, text_seq_len, joint_attention_dim]
adaln_linear_weightfp32 [modulation_dim, hidden_size]
adaln_linear_biasfp32 [modulation_dim]
to_q_weightfp32 [hidden_size, hidden_size]
to_q_biasfp32 [hidden_size]
to_k_weightfp32 [hidden_size, hidden_size]
to_k_biasfp32 [hidden_size]
to_v_weightfp32 [hidden_size, hidden_size]
to_v_biasfp32 [hidden_size]
to_k_context_weightfp32 [hidden_size, joint_attention_dim]
to_k_context_biasfp32 [hidden_size]
to_v_context_weightfp32 [hidden_size, joint_attention_dim]
to_v_context_biasfp32 [hidden_size]
to_out_weightfp32 [hidden_size, hidden_size]
to_out_biasfp32 [hidden_size]
pos_idx_axis0int64 [seq_len]
pos_idx_axis1int64 [seq_len]
pos_idx_axis2int64 [seq_len]
rope_cos_axis0fp32 [rope_axis0_dim, rope_dim_per_axis]
rope_sin_axis0fp32 [rope_axis0_dim, rope_dim_per_axis]
rope_cos_axis1fp32 [rope_axis1_dim, rope_dim_per_axis]
rope_sin_axis1fp32 [rope_axis1_dim, rope_dim_per_axis]
rope_cos_axis2fp32 [rope_axis2_dim, rope_dim_per_axis]
rope_sin_axis2fp32 [rope_axis2_dim, rope_dim_per_axis]
is_joint_blockint32 scalar
outputfp32 [batch_size, seq_len, hidden_size]
Axes and behavior
seq_lenvariable
head_dimconstant = 128
num_headsconstant = 24
batch_sizevariable
hidden_sizeconstant = 3072
text_seq_lenvariable
modulation_dimmodulation_dim = hidden_size * 6
rope_axis0_dimconstant = 16
rope_axis1_dimconstant = 56
rope_axis2_dimconstant = 56
rope_dim_per_axisrope_dim_per_axis = head_dim // 3
joint_attention_dimconstant = 4096
determinismunspecified
constraintsNo mutation or aliasing
Identity
alias032_dual_stream_attention_with_conditional_cross_attentionmodelfibosha256b506349e585a…
Sources: NVIDIA SOL-ExecBench (2026-08-22)last observed 2026-08-22How records are decidedJSON