MMKG-R1: Constructing Multimodal Knowledge Graphs via Reinforcement Learning
MMKG-R1 attacks the problem of building useful multimodal knowledge graphs (MMKGs) for downstream Graph RAG. Existing methods either construct graphs without task-aware training or rely on reinforcement learning driven only by final-answer correctness, which can give credit to noisy graphs that happen to retrieve the right image. The core idea is to train a Qwen3-VL-4B constructor with GRPO using three complementary, quality-aware rewards: a graph-level Deducibility Reward and two source-level rewards (Source-Grounded Quality and Graph-Readiness) that directly assess extraction fidelity and structural usability. This matters because a correct downstream answer does not guarantee that the MMKG itself is accurate or reusable across queries.
MMKG-R1 is a solid, well-motivated contribution. The move from answer-level feedback to direct graph-quality rewards is conceptually clean, and the empirical gains over an untrained Base GraphRAG and an F1-based RL baseline are consistent across datasets and retrieval strategies. However, the heavy dependence on frozen 32B judge models for two of the three rewards introduces an unquantified reliability bottleneck, and some absolute improvements are modest (e.g., +1.67% on MMQA Hybrid). The paper makes a credible case that source-grounded rewards help, but it stops short of benchmarking against the strongest prior methods adapted to this multimodal setting.
The reward decomposition is principled and ablated thoroughly. The Deducibility Reward's $0/0.5/1$ scale for whole-bundle and structured-path gaps gives finer-grained signal than binary answer correctness, and the ablation in Figure 3 shows that removing Source-Grounded Quality (SGQ) consistently hurts accuracy. The binary variant of the Deducibility Reward performs worse (Figure 3), confirming that distinguishing incomplete from complete reasoning paths adds useful training signal. Cross-dataset transfer to MMLongBench-Doc and evaluation with a larger 32B answer model (Table 2) suggest the constructor learns generalizable extraction behavior rather than overfitting to a specific QA pipeline.
The framework is bottlenecked by expensive, opaque judge models that are never validated. Both the Deducibility and Source-Grounded Quality rewards depend on Qwen3-VL-32B-Instruct-AWQ judgments, yet the paper reports no judge accuracy, consistency, or inter-annotator agreement, so we cannot tell whether the constructor is learning from reliable signal or overfitting to judge biases. The improvements are also uneven: on MMQA with hybrid retrieval the gain over Base GraphRAG is only 1.67 percentage points (42.50% vs 40.83%), and on MMDocRAG with subgraph retrieval the gain is just 1.36%. The F1-based RL baseline is a weak competitor—it computes token-level F1 against reference answers rather than targeting graph quality—so the comparison overstates the gap to prior RL approaches. A direct adaptation of AutoGraph-R1 or EvoGraph-R1 would be a stronger baseline.
The evidence supports the central claim that quality-aware rewards outperform answer-level F1 rewards and no training, but the comparison set is limited. The paper does not run AutoGraph-R1 or EvoGraph-R1 in its multimodal setting, arguing instead by conceptual distinction. While the ablations are helpful, the lack of a strong graph-level RL baseline adapted to MMKGs makes it hard to isolate the value of the specific reward decomposition. The cross-dataset transfer to MMLongBench-Doc is encouraging, though the evaluation subset is filtered to questions with figures or charts, which may not reflect broader document understanding. Accuracy is evaluated by an independent judge model (Qwen3.5-27B), which is good practice, but the absolute numbers remain low enough (often in the 30-45% range) that the practical ceiling for this pipeline is unclear.
Reproducibility is partial. The authors name public datasets (MMDocRAG, MMQA, MMLongBench-Doc), backbone models (Qwen3-VL-4B constructor, Qwen3-VL-8B/32B judge and answer models), hardware (four NVIDIA RTX A6000 GPUs), and software stack (verl and vLLM), and they disclose sampling temperature (1.0) and max output length (4096 tokens). However, the reward weights $\lambda_Q$ and $\lambda_R$ are stated but not numerically reported in the visible text, and learning rate, batch size, training steps, and exact judge prompts are omitted or deferred to truncated appendices. Fixed parser and assembler modules are described at a high level, but their implementations are not released. No public code repository is mentioned.
Multimodal knowledge graphs (MMKGs) organize information across different modalities to support multimodal Graph Retrieval-Augmented Generation (RAG), which aims to improve the performance of multimodal large language models (MLLMs). The effectiveness of these multimodal Graph RAG methods largely depends on the quality of the constructed MMKGs. However, most existing methods construct MMKGs without explicit guidance on graph quality, while reinforcement learning (RL)-based methods that rely on final-answer correctness may provide only indirect feedback on MMKG quality. To address these limitations, we propose MMKG-R1, a novel MMKG-quality-driven RL framework for training an MLLM as a multimodal knowledge constructor. To be specific, MMKG-R1 introduces multiple quality-aware rewards to directly evaluate both the quality of information extracted from individual knowledge sources and the quality of the assembled MMKG. These rewards are then used to guide the RL training. Experiments across multiple datasets show that MMKG-R1 constructs MMKGs that improve downstream question-answering accuracy.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.