Skip to content
KernelIndex
Search⌘K

cuda_nvfp4_grouped_naive_g4_n2048_k2048

FlashInfer-Bench baselines · cuda · Apache-2.0

Use it

Vendorable · source mirrored · Apache-2.0View source →

No package. Vendor the mirrored source: 21 lines, Apache-2.0, pinned at da91508.

main.cpp
curl "https://kernelindex.com/api/v1/implementations/flashinfer-cuda-nvfp4-grouped-naive-g4-n2048-k2048?include=source"
interfacecuda
revisionda915083d4c7
symbolrun
pathmain.cpp
Compatibility
measured onNVIDIA B200
declared hardwareNVIDIA B200
architecturessm_100
dtypesfp32, int32, int8

Benchmark evidence

2 measurements across 1 GPU, fastest first.

Operation / workload
Hardware
Latency
Rank
Observed
NVIDIA B200
19.7ms
#1 of 2
2026-06-06
NVIDIA B200
19.7ms
#2 of 2
2026-06-06

Reproduction-ready · How evidence levels are derived →

Source and license

sourcehttps://huggingface.co/datasets/flashinfer-ai/flashinfer-trace
commitda915083d4c7c5e61aa3005e3d17ae488e0fc71c
revision digestsha256:6f52d7622602ed94cd3c9f5f2dd3e8ab27f508b76b8e76c9108e293e295c469a
license declaredApache-2.0
license concludedApache-2.0
authorsbaseline
imported2026-08-16

Techniques

Extracted from the mirrored source by pattern, never inferred. Each row cites its line.

fp4m.def("run", &run, "grouped nvfp4 gemm",

Kernel source

main.cpp21 lines
#include "kernel.h"
#include <torch/extension.h>

torch::Tensor run(torch::Tensor A_fp4, torch::Tensor A_scale, torch::Tensor B_fp4,
                  torch::Tensor B_scale, torch::Tensor m_indptr, torch::Tensor alpha) {
    auto A = A_fp4.contiguous(); auto As = A_scale.contiguous();
    auto B = B_fp4.contiguous(); auto Bs = B_scale.contiguous();
    auto mi = m_indptr.to(torch::kInt32).contiguous();
    auto al = alpha.to(torch::kFloat32).contiguous();
    int G = B.size(0);
    const auto Mtot = A.size(0); const auto N = B.size(1);
    auto C = torch::empty({Mtot, N}, torch::dtype(at::kBFloat16).device(A.device()));
    launch_nvfp4_grouped(A, As, B, Bs, mi, al, C, G);
    return C;
}
PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
    m.def("run", &run, "grouped nvfp4 gemm",
          py::arg("A_fp4"), py::arg("A_scale"), py::arg("B_fp4"), py::arg("B_scale"),
          py::arg("m_indptr"), py::arg("alpha"));
}

Source code from FlashInfer-Bench (flashinfer-ai/flashinfer-trace) · Apache-2.0

Best evidence level for this revision: reproducible

JSON