Conformer relative position attention
Conformer attention with Shaw's relative positional embeddings. Processes sequences in fixed-size context blocks (512 tokens), computes scaled dot-product attention within each block, and adds relative positional bias via learned embeddings. Handles dynamic padding and masking for incomplete blocks.
Reported evidence · last observed 2026-08-17. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencydim_head = 128 · inner_dim = 1024 · num_heads = 8 · hidden_dim = 1024 · max_pos_emb = 512 · num_rel_pos = 1025 · context_size = 512 · inner_dim_x2 = 204816 cases
Not measured on H100 for this workload. Challenges →
1103.6µs73% SOL1.00×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
2111.2µs71% SOL1.07×Reported · license unknown · no source2026-07-15
Measured exactly what you asked. Reported by source; not independently reproduced.
3129.8µs67% SOL1.25×Reported · license unknown · no source2026-08-21
Measured exactly what you asked. Reported by source; not independently reproduced.
4135.4µs66% SOL1.31×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
5136.4µs66% SOL1.32×Reported · license unknown · no source2026-08-04
Measured exactly what you asked. Reported by source; not independently reproduced.
6136.9µs66% SOL1.32×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
7139.0µs65% SOL1.34×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
8160.3µs62% SOL1.55×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
9166.3µs61% SOL1.60×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
10171.0µs60% SOL1.65×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
11203.8µs55% SOL1.97×Reported · license unknown · no source2026-07-25
Measured exactly what you asked. Reported by source; not independently reproduced.
12211.0µs54% SOL2.04×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
13229.1µs52% SOL2.21×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
14267.9µs48% SOL2.59×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
15330.5µs42% SOL3.19×Reported · license unknown · no source2026-07-23
Measured exactly what you asked. Reported by source; not independently reproduced.
16425.7µs36% SOL4.11×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
17510.2µs32% SOL4.92×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
18639.7µs29% SOL6.17×Reported · license unknown · no source2026-08-12
Measured exactly what you asked. Reported by source; not independently reproduced.