Whisper decoder output projection
Whisper decoder output projection layer that maps hidden states (d_model=1280) to vocabulary logits (vocab_size=51866). This is a critical bottleneck in autoregressive generation as it runs for every generated token. The operation is C = A @ B.T where A is hidden_states and B is the weight matrix.
Reported evidence · last observed 2026-07-13. Reported by source; not independently reproduced. No public source. License unknown.
Current records
Workloadsuite of 16 cases · mean latencyd_model = 1280 · vocab_size = 5186616 cases
Not measured on H100 for this workload. Challenges →
1159.0µs74% SOL1.00×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
2160.1µs72% SOL1.01×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
3161.5µs73% SOL1.02×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
4161.9µs72% SOL1.02×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
5163.8µs71% SOL1.03×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
6163.9µs71% SOL1.03×Reported · license unknown · no source2026-07-16
Measured exactly what you asked. Reported by source; not independently reproduced.
7164.9µs71% SOL1.04×Reported · license unknown · no source2026-07-12
Measured exactly what you asked. Reported by source; not independently reproduced.
8170.1µs68% SOL1.07×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
9171.6µs68% SOL1.08×Reported · license unknown · no source2026-07-13
Measured exactly what you asked. Reported by source; not independently reproduced.
10173.2µs66% SOL1.09×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
11178.2µs65% SOL1.12×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
12184.0µs62% SOL1.16×Reported · license unknown · no source2026-07-27
Measured exactly what you asked. Reported by source; not independently reproduced.
13194.9µs61% SOL1.23×Reported · license unknown · no source2026-08-14
Measured exactly what you asked. Reported by source; not independently reproduced.
14211.0µs57% SOL1.33×Reported · license unknown · no source2026-07-11
Measured exactly what you asked. Reported by source; not independently reproduced.
15516.9µs26% SOL3.25×Reported · license unknown · no source2026-07-10
Measured exactly what you asked. Reported by source; not independently reproduced.
16754.8µs21% SOL4.75×Reported · license unknown · no source2026-07-29
Measured exactly what you asked. Reported by source; not independently reproduced.