ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models
Ultra-low-bit weight-only quantization is essential for deploying sparse Mixture-of-Experts LLMs, but existing mixed-precision PTQ methods collapse under tight bit budgets. ProQuant addresses this with two interventions: a Prune-or-Prioritize allocator (PoP) that treats pruning as a 0-bit option in joint ILP-based bit allocation over candidate set $\{0,2,3\}$, and a Routing-Consistent RMSNorm fine-tuner (RoCo) that compensates quantization error without altering router behavior. Experiments on Qwen3 MoE variants show large generation-quality gains at 1.5–2.5 bits, though evaluation is limited to two model families.
ProQuant offers a practical and empirically strong PTQ pipeline for MoE LLMs. The core insight that 1-bit experts are more harmful than pruned experts because their persistent errors propagate across layers is convincing and well supported. However, the headline comparisons are partly confounded by the fact that ProQuant can drop experts entirely while baseline mixed-precision methods cannot. The gains from RoCo are real but modest, and experimental validation is limited to two Qwen-family models under 35B parameters. Despite these caveats, the work is a clear advance for ultra-low-bit MoE deployment.
The experimental demonstration that extremely low-bit experts destroy rather than help is robust. Figure 2 shows that 1-bit errors in early layers cannot be recovered by later higher-precision layers, and Table 3 confirms that including 1-bit candidates underperforms the $\{0,2,3\}$ strategy by a wide margin. The paper also makes a valuable methodological point: perplexity and zero-shot log-likelihood metrics poorly predict actual generation quality for quantized MoE models.
The strongest concern is comparative fairness. Because PoP adds a 0-bit (pruning) option that baseline methods lack, ProQuant operates in a strictly larger design space. Although the authors argue that the gain comes from avoiding 1-bit experts rather than pruning itself, the freed bit budget is real and directly benefits retained experts. A second limitation is narrow evaluation scope: only two Qwen-family models are tested, and the authors concede scalability to larger models is unverified. Third, RoCo’s routing-consistency mechanism is elegant but its empirical margin over direct RMSNorm fine-tuning is small (65.91 vs 67.30 at 1.5 bits), suggesting that most of the lift comes from PoP’s allocation rather than representation adaptation.
The evidence supports the paper’s central claims and the detailed generation-based benchmarks (GSM8K, MATH-500, GPQA-Diamond, BFCL-v3, etc.) make for a more realistic assessment than PPL alone. Comparisons with MxMoE, PMQ, and GEMQ are thorough under identical GPTQ backends, but the baseline definitions do not include pruning, so the bit-budget comparison is uneven. The authors are transparent about using only Qwen3 variants, yet broader architectural coverage would strengthen generalizability.
The authors provide a public code repository and report hyperparameters in Appendix B, including calibration data (128 C4 sequences, 2048 tokens), GPTQ group size 128, and RoCo fine-tuning with AdamW for one epoch. Because the pipeline needs only calibration data and no backpropagation through expert weights, it should be reproducible on comparable hardware, although the reliance on A800 GPUs and the need for exact environment versions are minor barriers.
Mixture-of-Experts (MoE) large language models reduce computation through sparse activation but still incur substantial storage costs. Mixed-precision post-training quantization (PTQ) reduces these costs by quantizing different model components at different bit widths, but existing methods often struggle to preserve generation quality under tight bit budgets. We identify two key challenges: (1) nonlinear quantization errors at extremely low precision are underestimated when cross-layer propagation is ignored; (2) direct router fine-tuning may cause routing drift, and calibration gains may not generalize to downstream tasks. To address these issues, we propose ProQuant, a weight-only PTQ framework for MoE models. Prune-or-Prioritize Mixed-Precision Allocation (PoP) jointly optimizes expert pruning and bit-width assignment. It allows higher-precision experts to substitute for severely degraded ones while improving bit-budget utilization. Routing-Consistent RMSNorm Finetune (RoCo) adapts expert input representations through mathematically equivalent transformations. It compensates for quantization errors to reduce routing drift. Extensive experiments demonstrate that ProQuant outperforms existing methods across diverse generative tasks. Code is available at https://github.com/ThrowBug/ProQuant.git.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.