A Unified Framework for Image-to-3D Part Generation via Variable Granularity
FlexPart addresses a key limitation in controllable 3D asset creation: existing image-to-3D part generation systems are typically specialized for a single prompt type, forcing users into rigid workflows. The paper introduces a unified Diffusion Transformer framework that simultaneously accepts sparse point prompts, 2D bounding boxes, and dense masks via Gated Adaptive Layer Normalization, while an Asymmetric Geometric Guidance mechanism uses differentiable optimal transport to let dense spatial priors supervise sparse representations during training. By jointly training across prompt granularities, the method aims to improve part-level geometric fidelity and give users flexible control over how they specify structure.
The paper presents a well-motivated and technically coherent approach to controllable 3D part generation. Its strongest empirical result is that joint multi-granularity training improves performance across all prompt types, and the optimal-transport guidance offers a principled mechanism for transferring dense geometric knowledge to sparse inputs. Nevertheless, the evaluation is generous to unmatched parts, and the proclaimed variable-granularity flexibility is undercut by a mandatory point-prompt requirement at inference time.
The multi-granularity synergy ablation is the most compelling evidence: joint training consistently outperforms single-modality baselines, and the unified model improves point-only inference by 7.9% in IoUV and 6.1% in CD by internalizing boundary constraints from dense inputs. The asymmetric geometric guidance also shows clear causal value, boosting point-prompt Part IoUB by 7.9% over the baseline without adding inference overhead. Additionally, the critique of SAM-based cascades—where SAM segments localized fragments rather than complete functional parts—is a practical argument for end-to-end conditioning.
First, the part-level evaluation protocol drops unmatched parts without penalty, which risks masking cardinality errors when the model generates incorrect part counts. Second, the PartObjaverse-Tiny benchmark shares Objaverse lineage with the pretrained TripoSG initialization used by FlexPart, raising data-leakage concerns for generalization claims on that split. Third, despite the variable-granularity framing, points are mandatory at inference, so the system cannot truly accommodate pure count-only or box-only workflows without remapping them into points. Finally, several key baselines differ in training data and input conditions, making the headline comparison tables less controlled than the internal ablations.
The headline comparisons support the claim that direct part generation outperforms post-hoc lifting: under mask conditioning on PartVerseXL, FlexPart achieves Part CD 0.2938 versus OmniPart's 0.3283 and TripoSG+SAM3D's 0.3523, a solid 10.5% reduction versus OmniPart. However, whole-mesh metrics are merely competitive rather than dominant—on PartObjaverse-Tiny, FlexPart's Mean CD is 0.1587, nearly identical to OmniPart (0.1585) and trailing TripoSG (0.1504). The controlled 3D-box comparison with FullPart is more convincing, showing FlexPart outperforming FullPart on most metrics under matched supervision, though FullPart's training overlap with PartVerseXL complicates cross-benchmark interpretation.
Reproducibility is moderate. The architecture builds on public components—a pretrained TripoSG VAE, DINOv2 features, and a DiT backbone—and training hyperparameters are reported in detail, including two-stage learning rates, dropout rates, and token counts. The asymmetric guidance branch is lightweight at only 1.18M parameters. Yet the composite dataset of roughly 250K models derived from Objaverse and PartVerseXL relies on custom watertight conversion, ID-based rendering, and batch-wise part-packing that are not trivial to reconstruct from the text alone. The authors provide a GitHub URL, but no release timestamp or data processing scripts are mentioned in the main paper, leaving a meaningful gap for independent replication.
Controllable 3D part generation is pivotal for digital asset creation, yet existing methods lack the flexibility to accommodate diverse user workflows requiring varying levels of control. In this paper, we introduce FlexPart, a unified image-to-3D part generation framework that supports variable-granularity geometric conditions within a single model. By seamlessly integrating heterogeneous inputs, ranging from sparse points to dense bounding boxes and masks, via gated adaptive modulation, FlexPart unlocks a powerful cross-modal synergy. We also propose an asymmetric geometric guidance mechanism that leverages strong spatial priors to align representations across granularities, improving geometric consistency and facilitating inter-task synergy. Under mask conditioning on PartVerseXL, FlexPart reduces Part CD by 16.6% relative to TripoSG + SAM3D and by 10.5% relative to OmniPart, while maintaining competitive whole-mesh quality. The code can be found at https://github.com/Gucci233/FlexPart
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.