lukealonso/MiniMax-M2.5-REAP-139B-A10B-NVFP4

Note: The initial upload was malformed in some way, causing garbage outputs. This has been corrected.

Model Description

MiniMax-M2.5-NVFP4-REAP is an NVFP4-quantized version of cerebras/MiniMax-M2.5-REAP-139B-A10B, a 139B-parameter Mixture-of-Experts language model with 10B active parameters and 154 experts (pruned from 256 via REAP).

The REAP checkpoint's FP8 weights were first dequantized to BF16, then quantized to NVFP4 (4-bit with blockwise FP8 scales per 16 elements) using NVIDIA Model Optimizer.

The 40% expert pruning from REAP combined with NVFP4 quantization makes this model small enough to run on a single 96GB GPU.

What's quantized

Only the MoE expert MLP layers (gate, up, and down projections) are quantized to NVFP4. Attention layers are left in BF16. Since the expert weights constitute the vast majority of model parameters in an MoE architecture, this still yields significant memory savings.

Calibration uses natural top-k routing rather than forcing all experts to activate, so each expert's quantization scales reflect the token distributions it actually sees during inference. To compensate, calibration was run on a vastly larger number of samples than typical to ensure broad expert coverage through natural routing alone.

Calibration dataset

Samples were drawn from a diverse mix of publicly available datasets spanning code generation, function/tool calling, multi-turn reasoning, math, and multilingual (English + Chinese) instruction following. System prompts were randomly varied across samples. The dataset was designed to broadly exercise the model's capabilities and activate diverse token distributions across expert modules.

How to Run

Fits on a single RTX Pro 6000 Blackwell!

SGLang

SGLang image: lmsysorg/sglang:v0.5.8.post1-cu130 Model: lukealonso/MiniMax-M2.5-REAP-139B-A10B-NVFP4 GPU: NVIDIA B200 (single GPU, TP=1) transformers: 4.57.6 (shipped with the sglang image, not upgraded)

python3 -m sglang.launch_server
--model lukealonso/MiniMax-M2.5-REAP-139B-A10B-NVFP4
--served-model-name MiniMax-M2.5-NVFP4
--reasoning-parser minimax
--tool-call-parser minimax-m2
--trust-remote-code
--tp 1
--mem-fraction-static 0.95
--max-running-requests 32
--context-length 50000
--quantization modelopt_fp4
--attention-backend flashinfer
--moe-runner-backend flashinfer_cutlass
--kv-cache-dtype bf16
--enable-flashinfer-allreduce-fusion
--host 0.0.0.0
--port 8000

Acknowledgments

• MiniMaxAI for the original MiniMax-M2.5 model
• Cerebras Research for the REAP expert pruning method and the pruned checkpoint

See Also

• lukealonso/MiniMax-M2.5-NVFP4 — NVFP4 quantization of the full 230B model (needs 2+ GPUs)