Vision encoder layer with gated residuals
Complete vision encoder layer with self-attention, MLP, layer norms, and tanh-gated residual connections. This is the dominant computational pattern in the vision encoder, appearing in 8 global vision layers with gating. The gated residuals use tanh activation on learnable gate parameters to modulate the contribution of attention and MLP outputs.
Reported evidence · last observed 2026-08-16. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyhead_dim = 80 · num_heads = 16 · hidden_size = 1280 · intermediate_size = 512016 cases
Not measured on H100 for this workload. Challenges →
1175.7µs66% SOL1.00×Reported · license unknown · no source2026-08-16
Measured exactly what you asked. Reported by source; not independently reproduced.
2190.5µs63% SOL1.08×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
3216.8µs59% SOL1.23×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
4219.2µs59% SOL1.25×Reported · license unknown · no source2026-07-21
Measured exactly what you asked. Reported by source; not independently reproduced.
5223.5µs58% SOL1.27×Reported · license unknown · no source2026-07-14
Measured exactly what you asked. Reported by source; not independently reproduced.
6226.8µs58% SOL1.29×Reported · license unknown · no source2026-07-17
Measured exactly what you asked. Reported by source; not independently reproduced.
7227.4µs57% SOL1.29×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
8228.7µs58% SOL1.30×Reported · license unknown · no source2026-08-17
Measured exactly what you asked. Reported by source; not independently reproduced.
9253.2µs54% SOL1.44×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
10277.2µs52% SOL1.58×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
11279.1µs50% SOL1.59×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
12334.2µs44% SOL1.90×Reported · license unknown · no source2026-08-15
Measured exactly what you asked. Reported by source; not independently reproduced.
13360.7µs43% SOL2.05×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
14391.4µs40% SOL2.23×Reported · license unknown · no source2026-08-10
Measured exactly what you asked. Reported by source; not independently reproduced.
15495.1µs34% SOL2.82×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
16521.6µs33% SOL2.97×Reported · license unknown · no source2026-07-23
Measured exactly what you asked. Reported by source; not independently reproduced.
17934.9µs21% SOL5.32×Reported · license unknown · no source2026-08-10
Measured exactly what you asked. Reported by source; not independently reproduced.
181.58ms15% SOL8.99×Reported · license unknown · no source2026-08-10
Measured exactly what you asked. Reported by source; not independently reproduced.
191.61ms16% SOL9.18×Reported · license unknown · no source2026-08-13
Measured exactly what you asked. Reported by source; not independently reproduced.
201.70ms15% SOL9.68×Reported · license unknown · no source2026-08-11
Measured exactly what you asked. Reported by source; not independently reproduced.