TopicMerge: A Divide and Conquer Approach to Topic Modeling
TopicMerge tackles the scalability bottleneck of LLM-based topic modeling by combining efficient clustering with targeted LLM reasoning. Its core idea is to first over-segment a corpus into fine-grained semantic clusters, then use a lightweight LLM to hierarchically merge them with contextual neighborhood guidance. The authors claim this slashes LLM prompts by over 92% while matching or exceeding state-of-the-art clustering alignment on standard benchmarks.
TopicMerge presents a sensible hybrid design—using geometric clustering for global structure and an LLM for local semantic arbitration—and the ablations do support the value of context-aware merging and high-purity initialization. However, the evaluation relies entirely on external clustering metrics rather than intrinsic topic coherence, the strongest baselines are imported from another study rather than rerun, and the entire pipeline is gated behind proprietary Gemini models and embeddings. The efficiency gains are real in terms of prompt counts, but the paper overstates the practical accessibility of the approach and under-tests the quality of the produced topic labels.
The multi-stage pipeline is well motivated and the ablations empirically validate its key design choices. Table 3 shows that removing local/global context clusters causes a significant drop in ARI (from 0.71 to 0.63 on Wiki), confirming that neighborhood context improves LLM merge decisions. Table 4 further demonstrates that starting with fine-grained, high-purity clusters is important: performance degrades sharply when the initial cluster count drops from 80 to 30. The greedy-independent-set selection and purity-based ranking are sound algorithmic choices that prevent cascading merge errors.
A critical weakness is that every baseline number is taken from Liu et al. (2025) instead of being reproduced under the same experimental setup. This makes the comparison vulnerable to differences in embedding models, LLM backends, and implementation details. Second, the evaluation relies solely on clustering alignment metrics (P1, ARI, NMI) and completely omits intrinsic topic coherence (e.g., NPMI) or human evaluation of label quality. Appendix D reveals that TopicMerge's output is 'more granular than the high-level ground-truth categories,' yet the paper never measures whether this over-specificity hurts interpretability. Third, the method is highly sensitive to the initial cluster count, which is set to 80 because the true topic numbers are known (21 and 15); generalizing to corpora with unknown topic counts is untested.
The evidence supports directional improvement over embedding methods and a large reduction in LLM calls versus per-document approaches, but the comparison to LLM-based rivals is not apples-to-apples. TopicMerge uses Gemini 2.5/3.1 Flash Lite for both embeddings and merging, whereas baselines like LiSA reportedly use GPT-4. Because cost, latency, and prompt capacity differ across providers, reporting raw prompt counts rather than normalized cost or wall-clock time exaggerates the practical savings. Moreover, the oracle experiment in Appendix E shows that perfect merging would yield dramatically higher scores, implying the LLM arbiter—not the pipeline architecture—is the primary bottleneck; the paper frames this as a 'high performance ceiling' rather than a limitation of their chosen backend.
Although the authors release code, independent reproduction is severely hampered by reliance on closed-source Google models: Gemini embedding-001 for document representations and Gemini Flash Lite for all LLM reasoning. The authors explicitly note data-contamination risks and 'inherent stochasticity in outputs.' Temperature is set to 0, but API-level nondeterminism and model versioning remain uncontrolled. Hyperparameters for PaCMAP and agglomerative clustering are reported in detail, yet the 256-dimensional embeddings are not openly available, and the prompt engineering (including few-shot examples) is dataset-specific. A full reproduction would require API access, spending credits, and trusting that the baselines reported from Liu et al. are comparable under these conditions.
Recent advancements in topic modeling have split into two main paths: efficient embedding-based methods that offer scalability but are often outperformed on standard clustering metrics, and Large Language Model (LLM) approaches that achieve state-of-the-art clustering performance but at the cost of prohibitive computational expense and poor scalability. This paper introduces TopicMerge, a novel multi-stage pipeline that bridges these approaches to achieve both state-of-the-art clustering performance and computational efficiency. TopicMerge first employs an efficient clustering pipeline to deliberately over-segment a document corpus into a large set of fine-grained, semantically pure micro-clusters, which are then given initial semantic labels. It then introduces a novel, iterative merging algorithm where an LLM, guided by local neighborhood context, arbitrates on consolidating candidate pairs. These approved merges are then selectively executed and semantically relabeled by the LLM, hierarchically building broader, more coherent topics. Extensive experiments on the Bills and Wiki datasets show that TopicMerge achieves state-of-the-art clustering performance while reducing required LLM calls by over 92% compared to document-level methods, offering a practical and scalable solution.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.