QICOM: Open-Source Software for Reproducible Image Quality Evaluation, Blinded Assessment, and Radiomic Comparison
QICOM is an open-source desktop application that unites quantitative medical image comparison, paired radiomics, and blinded observer assessment into a single reproducible workflow. It tackles the fragmentation problem that forces researchers to juggle separate tools and scripts, losing the link between images, settings, and results. By saving native projects, configurations, and blinded response mappings, it enables any comparison to be reopened and recalculated, which is essential for trustworthy evaluation of image-processing methods and acquisition protocols.
The paper offers a solid technical validation of QICOM’s computational workflows, with rigorous IBSI conformance testing and transparent reporting of limitations. It does not, however, demonstrate clinical or perceptual utility, and the evaluation rests entirely on synthetic phantoms, one public CT case, and scripted rather than human responses. As the authors note, "These experiments document the calculations and data transfers linking the three workflows," but they leave human perceptual validity and cross-site reproducibility for future work.
The radiomics verification is exceptionally thorough: the QICOM/MIRP backend met reference tolerances for all 482 IBSI 1 digital-phantom entries and all 1,622 CT entries across configurations A–E, covering 1,804 feature/aggregation comparisons and 300 preprocessing diagnostics. The authors also sensibly probe implementation subtleties, showing that "With complete kernel support, masked SSIM and HFEN were invariant to the tested excluded-pixel perturbations," a detail many software papers omit.
The study's scope is narrow. Every experiment uses synthetic images or a single public CT case, and the blinded assessment validated only four scripted responses. The authors candidly acknowledge that "Human perceptual validity, clinical utility, cross-site performance, and comparative usability remain untested," and they explicitly state that "QICOM is not a medical device." These limitations mean the paper validates software mechanics, not real-world diagnostic or research utility.
Practical reproduction is also currently impossible. The repository is private, the experiment bundle resides only on the authors' local machine, and essential external resources must be obtained separately. Until the promised public release and archival occur, the reproducibility claims are prospective rather than realized.
The internal evidence strongly supports the narrow claim that QICOM computes metrics and radiomic features in agreement with references. However, comparisons to existing tools remain qualitative. The authors correctly caution that "The small residuals reported here measure computational agreement, rather than the size of an image-quality improvement," yet they do not provide quantitative usability, timing, or coverage benchmarks against the scikit-image/PIQ/PyRadiomics pipeline they describe as fragmented. Without such benchmarks, the practical advantage of integration is asserted but not measured.
Reproducibility is well-documented internally but blocked externally. The paper lists exact dependency versions, source hashes, and configuration recipes, and the software runs offline. Yet the authors state that "The repository is currently private and will be made public before arXiv submission," and that "The experiment bundle is held locally and contains the scripts, synthetic inputs, results, and both source archives." Because these artifacts are not yet publicly archived, an independent group cannot currently reproduce the experiments, even though the methodological description is precise.
QICOM (Quality Image COMparator) is open-source software for quantitative image comparison, paired radiomics, and blinded assessment. It saves processing settings, image identities, measurements, and response mappings so a comparison can be reopened and recalculated. We evaluate reference agreement, responses to controlled perturbations, and recovery of saved analyses. Synthetic experiments cover 37 quantitative descriptors using independentnumerical references and separate upstream integration comparisons. The QICOM/MIRP backend meets the reference tolerances for all 482 numeric IBSI 1 digital-phantom entries and all 1,622 entries across configurations A-E of the public CT reference case; these comprise 1,804 feature/aggregation comparisons and 300 preprocessing diagnostics. Restricting masked SSIM and HFEN to complete kernel support removes the observed dependence on excluded pixels. In a synthetic filtering example, stronger smoothing increases SSIM while worsening MSE and HFEN relative to milder smoothing and changing GLCM contrast more strongly. Reopened radiomics projects reproduce all 43 feature pairs per comparison, and two blinded deliveries return four correctly mapped scripted responses. These experiments document the calculations and data transfers linking the three workflows.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.