Skip to content
KernelIndex
Search⌘K
Sign in

Top k top p sampling from probs v128256

0 eligible runs
sampling

Top-k top-p (nucleus) sampling from probabilities with vocab_size=128256. Filters probabilities using top-k and top-p constraints, then samples from the filtered distribution. Captured from Llama 3.1 8B.

Current records

No published measurement for the selected workload.

Implementations

Implementation
Runtime
Best latency
Evidence
Availability
claude-opus-4-1-20250805 / cudae7a6dcclaude-opus-4-1-20250805
cuda
—
—
Apache-2.0 · source
claude-opus-4-1-20250805 / tritondf09fdclaude-opus-4-1-20250805
triton
—
—
Apache-2.0 · source
gemini-2.5-pro / cuda54f90agemini-2.5-pro
cuda
—
—
Apache-2.0 · source
gemini-2.5-pro_triton_544238gemini-2.5-pro
triton
—
—
Apache-2.0 · source
gpt-5-2025-08-07 / cudaaec5f2gpt-5-2025-08-07
cuda
—
—
Apache-2.0 · source
gpt-5-2025-08-07 / tritonaf4b72gpt-5-2025-08-07
triton
—
—
Apache-2.0 · source
gpt-o3 / cuda7f55e3gpt-o3
cuda
—
—
Apache-2.0 · source
gpt-o3 / triton2a200egpt-o3
triton
—
—
Apache-2.0 · source
Show all 9 implementations ›
flashinfer_wrapper_211bdd6eFlashInfer-Bench baselines
python
—
—
Apache-2.0 · source

Semantics

Inputs and outputs
probsfp32 [batch_size, vocab_size]
top_kint32 [batch_size]
top_pfp32 [batch_size]
samplesint64 [batch_size]
Axes and behavior
batch_sizevariable
vocab_sizeconstant = 128256
determinismunspecified
constraintsNo mutation or aliasing
Identity
aliastop_k_top_p_sampling_from_probs_v128256modelllama-3-1-8bmodelllama-3-1-70bmodelllama-3-1-405bmodelllama-3-2-3b+1 modelssha256b133675d3a04…
No source imports for this operation yet.How records are decidedJSON