Reinforcing intra-operative tumour localisation with weak supervision from pre-operative appearances
Intra-operative CT imaging of the liver often shows poor contrast for hepatocellular carcinoma, making tumour localisation difficult and curating labelled datasets expensive. This paper proposes guiding an RL-based search on intra-operative CT using weak supervision from a tumour-presence classifier trained on pre-operative MR images, eliminating the need for paired cross-modal data. The core idea is that an MR-trained classifier still assigns higher tumour probabilities to cancerous windows on CT, and these probabilities drive an agent to explore and aggregate a tumour score map. The work matters because it offers a way to transfer pre-operative appearance knowledge into the operating room without deformable registration or manual CT segmentations.
The paper presents a conceptually interesting cross-modality weak-supervision framework, but its clinical utility remains uncertain. While synthetic transfer outperforms zero-shot, few-shot, and CT-only baselines, the absolute Dice score of 0.31 is low for precise surgical guidance, and the 93% success rate only requires Dice > 0.10. The honest acknowledgement that registration comparisons come from a different cohort tempers the comparative claims, leaving the method as a promising but still preliminary proof-of-concept.
The multi-strategy evaluation is a strength: systematically comparing zero-shot, few-shot, and synthetic transfer with CT-only baselines and registration benchmarks gives a clear picture of where gains come from. The use of unpaired CycleGAN translation to close the modality gap is well-motivated and empirically effective, and the open-source release together with reported training times and hyperparameters supports community verification.
The primary limitation is practical performance: a mean Dice of 0.31 means overlap is rough, and boundary delineation is poor despite high success rate. The weakly-supervised framing is also slightly undermined because threshold selection relies on a grid search over segmentation labels (Table 2b), so the pipeline is not fully label-free during development. Additionally, the comparison to registration methods is explicitly noted as cross-cohort, making direct superiority claims unreliable.
The internal evidence supports the relative ranking (synthetic > few-shot > zero-shot > CT-only), but the margins are small and clinical significance is debatable. Notably, few-shot transfer did not significantly outperform a fully supervised nn-UNet baseline trained only on CT (p > 0.11), suggesting that the cross-modal advantage is modest once some CT labels are available. The contextual comparison to registration is honest but limits the strength of conclusions that can be drawn against the main alternative approach.
Reproducibility is mixed. The authors open-sourced their code and reported hyperparameters, network architectures, and training times. However, the pipeline depends on a mix of public datasets (LLD-MMR, ATLAS, CHAOS, WAW-TACE, TCIA) and an undisclosed internal CT cohort. Replicating the exact data split and preprocessing pipeline will require substantial effort, and reproducing the reported best threshold without voxel-wise validation labels contradicts the weakly-supervised premise. The 48-hour RL training time per run also makes large-scale ablation expensive.
Limited tumour visibility on intra-operative images complicates surgical navigation, including liver hepatocellular carcinoma (HCC) targeting on computed tomography (CT) imaging. This makes labelled datasets difficult to curate for deep learning, and transferring pre-operative delineations through registration remains challenging due to intra-operative deformation and lack of paired pre- and intra-operative data. We propose to transfer tumour appearances learned from pre-operative magnetic resonance (MR) images to guide localisation on intra-operative CT through weak supervision. A classifier is trained to classify tumour presence on pre-operative images. This is applied to windows on intra-operative images, and despite the domain gap, it assigns higher tumour-presence probabilities on average to tumour-containing windows. These weak cross-domain signals guide a reinforcement learning agent to localise tumours on intra-operative images. We compare direct transfer, few-shot adaptation and synthetic transfer (trained using unpaired MR and CT), which translates CT into synthetic MR before localisation. On an external HCC cohort, synthetic transfer outperformed other tested methods and CT-only baselines. Our framework enables cross-modality tumour localisation without paired MR-CT training images.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.