LRC-WeatherNet: LiDAR, RADAR, and Camera Fusion Network for Real-time Weather-type Classification in Autonomous Driving
Autonomous vehicles struggle with adverse weather perception. This paper proposes LRC-WeatherNet, a lightweight fusion network combining LiDAR, RADAR, and camera via early BEV fusion and mid-level gating to classify weather conditions in real-time. The approach achieves $86.66\%$ accuracy on the MSU-4S dataset with $7.13\,\mathrm{ms}$ inference, demonstrating that adaptive multi-modal fusion outperforms unimodal baselines, though dataset limitations restrict generalization to rare weather events.
The paper presents a technically sound approach to weather classification using a pragmatic combination of BEV projections and EfficientNet backbones. The gated fusion mechanism effectively balances modalities, and the lightweight design achieves real-time performance on consumer hardware. However, the evaluation relies heavily on the MSU-4S dataset, which contains predominantly seasonal scenes rather than diverse adverse weather conditions, limiting the validation of the core premise regarding robustness in challenging weather.
The architectural design choices are well-justified and effective. Converting LiDAR and RADAR to BEV representations allows using EfficientNet-B0, achieving $4.21\,\mathrm{ms}$ inference for early fusion and $7.13\,\mathrm{ms}$ for the full model. The ablation study clearly shows the value of fusion: Camera-only achieves $77.86\%$ test accuracy while the full LRC-WeatherNet reaches $86.66\%$, with particularly large gains over LiDAR-only ($56.34\%$) and RADAR-only ($27.32\%$). The gating mechanism exhibits adaptive behavior: "The gated fusion mechanism assigned weights of $0.64$ to RGB and $0.35$ to the early fused LiDAR and RADAR, indicating a higher reliance on visual cues in this snowy environment."
The primary limitation lies in the dataset composition. The authors acknowledge that MSU-4S includes "only one class representing rain and two classes corresponding to snowy conditions," with remaining categories being seasonal variants without active precipitation. This contradicts the stated motivation of addressing "adverse weather such as rain, fog, and snow" when the dataset lacks critical edge cases like "dense fog, sleet, hail, or nighttime snow." Furthermore, there is a pronounced generalization gap: LRC-WeatherNet achieves $93.82\%$ validation accuracy versus $86.66\%$ test accuracy, suggesting the model memorizes specific driving routes rather than learning robust weather features.
The quantitative evidence supports relative performance claims but lacks diversity. The comparison against RECNet ($30.91\%$ accuracy) is favorable but potentially unfair given RECNet was designed for different datasets and conditions. The LRC-WeatherNet-PP variant using PointPillars achieves higher test accuracy ($87.77\%$) but with prohibitive latency ($64.40\,\mathrm{ms}$), validating the efficiency argument for the BEV approach. However, the confusion matrix reveals persistent challenges: "LRC-WeatherNet-PP ... exhibits increased confusion among fall-related classes and signs of overfitting in those categories," indicating that seasonal classification relies more on vegetation color than weather phenomena.
Reproducibility is facilitated by the release of trained models and source code at the provided GitHub repository. The paper details hyperparameters including AdamW optimizer settings ($3\times 10^{-4}$ learning rate, $1\times 10^{-4}$ weight decay), data augmentation strategies, and BEV preprocessing ($0.1\,\mathrm{m}$ grid resolution, $50\,\mathrm{m}$ forward range). However, critical details such as the exact learning rate scheduling protocol, batch size, and training duration are omitted. The MSU-4S dataset is publicly available, though the paper's specific train/val/test splits require careful reconstruction based on the described session-aware partitioning method.
Autonomous vehicles face major perception and navigation challenges in adverse weather such as rain, fog, and snow, which degrade the performance of LiDAR, RADAR, and RGB camera sensors. While each sensor type offers unique strengths, such as RADAR robustness in poor visibility and LiDAR precision in clear conditions, they also suffer distinct limitations when exposed to environmental obstructions. This study proposes LRC-WeatherNet, a novel multi-sensor fusion framework that integrates LiDAR, RADAR, and camera data for real-time classification of weather conditions. By employing both early fusion using a unified Bird's Eye View representation and mid-level gated fusion of modality-specific feature maps, our approach adapts to the varying reliability of each sensor under changing weather. Evaluated on the extensive MSU-4S dataset covering nine weather types, LRC-WeatherNet achieves superior classification performance and computational efficiency, significantly outperforming unimodal baselines in adverse conditions. This work is the first to combine all three modalities for robust, real-time weather classification in autonomous driving. We release our trained models and source code in https://github.com/nouralhudaalbashir/LRC-WeatherNet.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.