Skip to content
KernelIndex
Search⌘K
Sign in

flashinfer / wrapper0ff432

flashinfer_wrapper_0ff432 · FlashInfer-Bench baselines · python · Apache-2.0
Kernel source · 17 lines ↓

Use it

Vendorable · source mirrored · Apache-2.0View source →

No package. Vendor the mirrored source: 17 lines, Apache-2.0, pinned at da91508.

main.py
curl "https://kernelindex.com/api/v1/implementations/flashinfer-flashinfer-wrapper-0ff432?include=source"
interfacepython
revisionda915083d4c7
symbolrun
pathmain.py
Compatibility
measured onNVIDIA B200
declared hardwareNVIDIA A100, NVIDIA B200, NVIDIA GeForce RTX 4090, NVIDIA H100, NVIDIA H20, NVIDIA H200
architecturesunknown
dtypesbf16

Benchmark evidence

14 measurements across 1 GPU, fastest first.

Operation / workload
Hardware
Latency
Rank
Observed
Fused add RMSNorm h4096bf16 · [4096] · batch_size=34
NVIDIA B200
8.07µs
#2 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=63
NVIDIA B200
8.09µs
#2 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=7
NVIDIA B200
8.10µs
#2 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=16
NVIDIA B200
8.10µs
#2 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=15
NVIDIA B200
8.15µs
#2 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=79
NVIDIA B200
8.15µs
#2 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=64
NVIDIA B200
8.18µs
#2 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=170
NVIDIA B200
8.24µs
#2 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=1
NVIDIA B200
18.0µs
#5 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=8804
NVIDIA B200
59.4µs
#4 of 8
2025-10-21
Run →
Show all 14 measurements ›
Fused add RMSNorm h4096bf16 · [4096] · batch_size=11832
NVIDIA B200
77.1µs
#4 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=14418
NVIDIA B200
92.1µs
#4 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=14509
NVIDIA B200
92.2µs
#4 of 8
2025-10-21
Run →
Fused add RMSNorm h4096bf16 · [4096] · batch_size=10827
NVIDIA B200
188.1µs
#6 of 8
2025-10-21
Run →

Reported · How evidence levels are derived →

Source and license

sourcehttps://huggingface.co/datasets/flashinfer-ai/flashinfer-trace
commitda915083d4c7c5e61aa3005e3d17ae488e0fc71c
revision digestsha256:7e36a87a751b03415e29026bb80864ced3ab0bef7dde027925013bec04a75163
license declaredApache-2.0
license concludedApache-2.0
authorsbaseline
imported2026-08-16

Kernel source

main.py17 lines
import torch
import flashinfer


def run(hidden_states, residual, weight):
    batch_size, hidden_size = hidden_states.shape
    
    assert hidden_size == 4096
    
    EPS = 1e-5
    
    # FlashInfer's fused_add_rmsnorm modifies hidden_states in-place
    flashinfer.norm.fused_add_rmsnorm(hidden_states, residual, weight, EPS)
    
    # Return the modified hidden_states as output
    return hidden_states

Source code from FlashInfer-Bench (flashinfer-ai/flashinfer-trace) · Apache-2.0

Best evidence level for this revision: reported

JSON