Challenges · KernelIndex

Priority gaps

high-demand families with zero runs on a priority GPU

Operation / family

GPU

What is missing

Since / asks

gqa-paged-attention

H100

no eligible run for the gqa-paged-attention family

—

gqa-ragged-attention

H100

no eligible run for the gqa-ragged-attention family

—

mla-paged-attention

H100

no eligible run for the mla-paged-attention family

—

layernorm

B200

no eligible run for the layernorm family

—

softmax

B200

no eligible run for the softmax family

—

mlp

H100

no eligible run for the mlp family

—

Model gaps

measured on one priority GPU but not the other

Operation / family

GPU

What is missing

Since / asks

Attention softmax dropout value matmul backward

H100

seed-seed-oss-36b-instruct · measured on B200 only

—

Fused vision cu seqlens attention with 2d RoPE backward

H100

qwen3-vl-4b-instruct · measured on B200 only

—

Cu seqlens variable length vision attention

H100

qwen-image-edit-2509 · measured on B200 only

—

Vision cu seqlens variable length attention

H100

qwen3-vl-8b-instruct · measured on B200 only

—

NVFP4 grouped query attention with KV repeat

H100

qwen3-14b-nvfp4 · measured on B200 only

—

Repeat KV attention matmul

H100

olmo-3-7b-think · measured on B200 only

—

KV shared attention with RoPE

H100

gemma-3n-e2b · measured on B200 only

—

KV shared attention with dual RoPE

H100

gemma-3n-e4b · measured on B200 only

—

Show all 60 challenges ›

Attention softmax with softcapping and dropout

H100

gemma-3-1b-it · measured on B200 only

—

Fused attention QK matmul scale mask softmax backward

H100

llama-3-1-405b · measured on B200 only

—

KV cache update with RoPE backward

H100

devstral-small-2507 · measured on B200 only

—

Latent KV expansion with split

H100

kimi-k2-instruct-0905 · measured on B200 only

—

Masked softmax with attention dropout backward

H100

openvoice · measured on B200 only

—

KV cache update with RoPE

H100

phi-4-reasoning-plus-nvfp4 · measured on B200 only

—

Audio encoder varlen attention with chunking backward

H100

qwen3-omni-30b-a3b-instruct · measured on B200 only

—

Audio sinusoidal position embedding with conv projection

H100

qwen3-omni-30b-a3b-instruct · measured on B200 only

—

Audio relative position attention

H100

gemma-3n-e2b · measured on B200 only

—

Audio encoder to language model multimodal fusion

H100

granite-speech-3-3-8b · measured on B200 only

—

Conformer relative position attention

H100

granite-speech-3-3-8b · measured on B200 only

—

Audio encoder conv positional layer stack

H100

voxtral-small-24b-2507 · measured on B200 only

—

Whisper decoder output projection

H100

whisper-large-v3 · measured on B200 only

—

Decoder layer full block

H100

deephermes-3-llama-3-8b-preview · measured on B200 only

—

Decoder layer with residual connections

H100

qwen3-coder-30b-a3b-instruct · measured on B200 only

—

Decoder layer full block

H100

qwen3-14b-nvfp4 · measured on B200 only

—

Decoder complete layer

H100

canary-qwen-2-5b · measured on B200 only

—

Flux concatenated sequence processing with split

H100

flux-1-kontext-dev · measured on B200 only

—

Flux timestep guidance projection embedding

H100

flux-1-dev · measured on B200 only

—

VAE decoder mid block attention ResNet

H100

flux-2-dev-vae · measured on B200 only

—

ResNet block with time embedding

H100

stable-diffusion-v1-5 · measured on B200 only

—

Edit consistency loss with perceptual weighting

H100

step1x-edit-v1p2 · measured on B200 only

—

Region aware self attention with edit bias backward

H100

step1x-edit-v1p2 · measured on B200 only

—

GEMM n128 k2048

H100

qwen3-30b-a3b · measured on B200 only

—

GEMM n16384 k2048

H100

llama-3-2-1b · measured on B200 only

—

GEMM n2048 k2048

H100

llama-3-2-1b · measured on B200 only

—

GEMM n2048 k4096

H100

qwen3-30b-a3b · measured on B200 only

—

GEMM n2048 k8192

H100

llama-3-2-1b · measured on B200 only

—

GEMM n256 k7168

H100

deepseek-v3, deepseek-r1 · measured on B200 only

—

GEMM n28672 k4096

H100

llama-3-1-8b · measured on B200 only

—

GEMM n3072 k2048

H100

llama-3-2-1b · measured on B200 only

—

GEMM n34816 k5120

H100

qwen3-14b · measured on B200 only

—

GEMM n4096 k14336

H100

llama-3-1-8b · measured on B200 only

—

GEMM n4096 k4096

H100

llama-3-1-8b · measured on B200 only

—

GEMM n5120 k17408

H100

qwen3-14b · measured on B200 only

—

GEMM n5120 k2048

H100

qwen3-30b-a3b · measured on B200 only

—

GEMM n5120 k5120

H100

qwen3-14b · measured on B200 only

—

GEMM n6144 k4096

H100

llama-3-1-8b · measured on B200 only

—

GEMM n7168 k5120

H100

qwen3-14b · measured on B200 only

—

GQA paged decode h20 kv4 d128 ps1

H100

qwen3-14b · measured on B200 only

—

GQA paged decode h20 kv4 d128 ps64

H100

qwen3-14b · measured on B200 only

—

GQA paged decode h24 kv4 d128 ps64

H100

mixtral-8x22b · measured on B200 only

—

GQA paged decode h24 kv8 d128 ps1

H100

llama-3-2-3b · measured on B200 only

—

GQA paged decode h32 kv4 d128 ps1

H100

qwen3-30b-a3b · measured on B200 only

—

GQA paged decode h32 kv4 d128 ps64

H100

qwen3-30b-a3b · measured on B200 only

—

GQA paged decode h32 kv8 d128 ps1

H100

llama-3-1-8b · measured on B200 only

—

GQA paged decode h32 kv8 d128 ps64

H100

llama-3-1-8b · measured on B200 only

—

GQA paged decode h32 kv8 d64 ps1

H100

llama-3-2-1b · measured on B200 only

—

GQA paged decode h32 kv8 d64 ps64

H100

llama-3-2-1b · measured on B200 only

—

GQA paged prefill causal h16 kv1 d128 ps64

H100

qwen3-235b · measured on B200 only

—

GQA paged prefill causal h16 kv2 d128 ps64

H100

llama-3-1-70b, qwen3-32b · measured on B200 only

—

GQA paged prefill causal h20 kv4 d128 ps1

H100

qwen3-14b · measured on B200 only

—

Unbeaten baselines

the source's own reference; nobody has entered against it

Operation / family

GPU

What is missing

Since / asks

Sparse GEMM NVFP4 n2048 k2048

B200

int8 · [2048, 1024] · m=128 · cuda_sparse_nvfp4_naive_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Sparse GEMM NVFP4 n2048 k2048

B200

int8 · [2048, 1024] · m=64 · cuda_sparse_nvfp4_naive_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM NVFP4 m contiguous g4 n2048 k2048

B200

fp32 · [512, 1024] · cuda_nvfp4_grouped_naive_g4_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM MXFP4 out NVFP4 n2048 k2048

B200

bf16 · [256, 2048] · mm_fp4_out_nvfp4_flashinfer_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM MXFP4 out NVFP4 n2048 k2048

B200

bf16 · [128, 2048] · mm_fp4_out_nvfp4_flashinfer_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 fp6 n2048 k2048

B200

int8 · [128, 1024] · cuda_fp4_fp6_naive_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 fp6 n2048 k2048

B200

int8 · [64, 1024] · cuda_fp4_fp6_naive_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Sparse GEMM fp4 FP8 n2048 k2048

B200

fp8_e4m3 · [2048, 2048] · m=128 · cuda_sparse_fp4_fp8_naive_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Show all 60 challenges ›

Sparse GEMM fp4 FP8 n2048 k2048

B200

fp8_e4m3 · [2048, 2048] · m=64 · cuda_sparse_fp4_fp8_naive_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 FP8 n2048 k2048

B200

fp8_e4m3 · [128, 1024] · cuda_fp4_fp8_naive_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 FP8 n2048 k2048

B200

fp8_e4m3 · [64, 1024] · cuda_fp4_fp8_naive_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 e2m1 n4096 k2048

B200

int8 · [1024, 1024] · cublaslt_fp4_e2m1_scaled_mm_n4096_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 e2m1 n4096 k2048

B200

int8 · [256, 1024] · cublaslt_fp4_e2m1_scaled_mm_n4096_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 e2m1 n4096 k2048

B200

int8 · [128, 1024] · cublaslt_fp4_e2m1_scaled_mm_n4096_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 e2m1 n2048 k2048

B200

int8 · [1024, 1024] · cublaslt_fp4_e2m1_scaled_mm_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 e2m1 n2048 k2048

B200

int8 · [256, 1024] · cublaslt_fp4_e2m1_scaled_mm_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM fp4 e2m1 n2048 k2048

B200

int8 · [128, 1024] · cublaslt_fp4_e2m1_scaled_mm_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM FP8 fp4 m masked g4 n4096 k2048

B200

fp32/fp8_e4m3 · [4, 512, 2048] · deepgemm_m_grouped_fp8_fp4_masked_g4_n4096_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM FP8 fp4 m masked g4 n2048 k2048

B200

fp32/fp8_e4m3 · [4, 512, 2048] · deepgemm_m_grouped_fp8_fp4_masked_g4_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM FP8 fp4 m contiguous g4 n4096 k2048

B200

fp32/fp8_e4m3 · [1920, 2048] · deepgemm_m_grouped_fp8_fp4_contiguous_g4_n4096_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM FP8 fp4 m contiguous g4 n4096 k2048

B200

fp32/fp8_e4m3 · [960, 2048] · deepgemm_m_grouped_fp8_fp4_contiguous_g4_n4096_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM FP8 fp4 m contiguous g4 n2048 k2048

B200

fp32/fp8_e4m3 · [1920, 2048] · deepgemm_m_grouped_fp8_fp4_contiguous_g4_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM FP8 fp4 m contiguous g4 n2048 k2048

B200

fp32/fp8_e4m3 · [960, 2048] · deepgemm_m_grouped_fp8_fp4_contiguous_g4_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM mxfp8 MXFP4 g4 n4096 k2048

B200

fp8_e4m3 · [1024, 2048] · group_gemm_mxfp4_flashinfer_g4_n4096_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM mxfp8 MXFP4 g4 n4096 k2048

B200

fp8_e4m3 · [512, 2048] · group_gemm_mxfp4_flashinfer_g4_n4096_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM mxfp8 MXFP4 g4 n2048 k2048

B200

fp8_e4m3 · [1024, 2048] · group_gemm_mxfp4_flashinfer_g4_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Grouped GEMM mxfp8 MXFP4 g4 n2048 k2048

B200

fp8_e4m3 · [512, 2048] · group_gemm_mxfp4_flashinfer_g4_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM FP8 fp4 n4096 k4096

B200

fp32/fp8_e4m3 · [1024, 4096] · deepgemm_fp8_fp4_gemm_nt_n4096_k4096 is the source baseline and nobody has beaten it

2026-06-06

GEMM FP8 fp4 n4096 k4096

B200

fp32/fp8_e4m3 · [256, 4096] · deepgemm_fp8_fp4_gemm_nt_n4096_k4096 is the source baseline and nobody has beaten it

2026-06-06

GEMM FP8 fp4 n4096 k4096

B200

fp32/fp8_e4m3 · [128, 4096] · deepgemm_fp8_fp4_gemm_nt_n4096_k4096 is the source baseline and nobody has beaten it

2026-06-06

GEMM FP8 fp4 n2048 k2048

B200

fp32/fp8_e4m3 · [1024, 2048] · deepgemm_fp8_fp4_gemm_nt_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM FP8 fp4 n2048 k2048

B200

fp32/fp8_e4m3 · [256, 2048] · deepgemm_fp8_fp4_gemm_nt_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

GEMM FP8 fp4 n2048 k2048

B200

fp32/fp8_e4m3 · [128, 2048] · deepgemm_fp8_fp4_gemm_nt_n2048_k2048 is the source baseline and nobody has beaten it

2026-06-06

Jensen-Shannon divergence loss

B200

fp32 · [8192, 128256] · PyTorch is the source baseline and nobody has beaten it

2026-05-27

Jensen-Shannon divergence loss

B200

fp32 · [4096, 128256] · PyTorch is the source baseline and nobody has beaten it

2026-05-27

Jensen-Shannon divergence loss

B200

fp32 · [2048, 128256] · PyTorch is the source baseline and nobody has beaten it

2026-05-27

Jensen-Shannon divergence loss

B200

fp32 · [1024, 128256] · PyTorch is the source baseline and nobody has beaten it

2026-05-27

Jensen-Shannon divergence loss

B200

fp32 · [1024, 128256] · PyTorch is the source baseline and nobody has beaten it

2026-05-27

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [4, 32, 128] · num_pages=389 · num_kv_indices=232 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [4, 32, 128] · num_pages=349 · num_kv_indices=212 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [4, 32, 128] · num_pages=398 · num_kv_indices=268 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [128, 32, 128] · num_pages=52033 · num_kv_indices=47438 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [128, 32, 128] · num_pages=51792 · num_kv_indices=46414 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [16, 32, 128] · num_pages=1735 · num_kv_indices=1145 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [16, 32, 128] · num_pages=1686 · num_kv_indices=1033 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [16, 32, 128] · num_pages=1794 · num_kv_indices=1257 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [127, 32, 128] · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [64, 32, 128] · num_pages=29441 · num_kv_indices=27222 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [64, 32, 128] · num_pages=29189 · num_kv_indices=26582 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [63, 32, 128] · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [32, 32, 128] · num_pages=15178 · num_kv_indices=14103 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [32, 32, 128] · num_pages=15114 · num_kv_indices=13783 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [31, 32, 128] · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [8, 32, 128] · num_pages=902 · num_kv_indices=576 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [8, 32, 128] · num_pages=839 · num_kv_indices=528 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA paged decode h32 kv8 d128 ps64

B200

bf16 · [8, 32, 128] · num_pages=910 · num_kv_indices=640 · flashinfer_wrapper_ad4135 is the source baseline and nobody has beaten it

2026-04-30

GQA ragged prefill causal h32 kv8 d64

B200

bf16 · [8819, 8, 64] · flashinfer_wrapper_44b034 is the source baseline and nobody has beaten it

2026-04-20

GQA ragged prefill causal h32 kv8 d64

B200

bf16 · [10930, 8, 64] · flashinfer_wrapper_44b034 is the source baseline and nobody has beaten it

2026-04-20

GQA ragged prefill causal h32 kv8 d64

B200

bf16 · [5932, 8, 64] · flashinfer_wrapper_44b034 is the source baseline and nobody has beaten it

2026-04-20

GQA ragged prefill causal h32 kv8 d64

B200

bf16 · [13716, 8, 64] · flashinfer_wrapper_44b034 is the source baseline and nobody has beaten it

2026-04-20

Unchallenged records

single-entry cohorts

Operation / family

GPU

What is missing

Since / asks

AMD DeepSeek-style MoE layer

AMD Instinct MI300X

fp16 · [2, 4096, 7168] · amd-mixture-of-experts · submission 12732 is the only entry

2025-04-29

Stale records

not re-observed in 180 days

Operation / family

GPU

What is missing

Since / asks

AMD MXFP4 GEMM

AMD Instinct MI355X

suite of 6 cases · submission 728835 last observed 2026-04-04

2026-04-04

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=128 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=8 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=16 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=32 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=64 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=8 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=16 · Liger last observed 2026-04-02

2026-04-02

Show all 60 challenges ›

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=32 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=64 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=8 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=16 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=32 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=64 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [512, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [1024, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [2048, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [4096, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [16384, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [32768, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [128, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [256, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [128, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [256, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [512, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [1024, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [2048, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [4096, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=128 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [16384, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [32768, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [128, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [256, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [512, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [1024, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [2048, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [4096, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [8192, 2048] · experts=128 · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [16384, 2048] · Liger last observed 2026-04-02

2026-04-02

Fused MoE expert MLP

H100

bf16 · [32768, 2048] · Liger last observed 2026-04-02

2026-04-02

Sort

B200

suite of 5 cases · submission 682915 last observed 2026-03-31

2026-03-31

FP16 vector addition

B200

suite of 5 cases · submission 682384 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [413, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [79, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [1834, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [1825, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [4743, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [1307, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [2406, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [170, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [559, 24, 128] · num_pages=5878 · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [332, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [987, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [4812, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [1728, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [82, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [247, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [559, 24, 128] · num_pages=718 · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [1932, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

GQA paged prefill causal h24 kv4 d128 ps1

B200

bf16 · [731, 24, 128] · flashinfer_wrapper_0b1670 last observed 2026-03-31

2026-03-31

Priority coverage

FamilyH100B200All GPUs

gqa-paged-attention0 · gap1,4741,547

gqa-ragged-attention0 · gap322342

mla-paged-attention0 · gap653653

gemm2484,4637,657

moe841,2152,318

rmsnorm1032,1332,272

layernorm340 · gap54

softmax80 · gap56

rope48690786

mlp0 · gap650650

Closing a challenge means publishing evidence: a benchmark run with its workload, protocol, environment, and source.Contribute evidence →JSON