Class-Grounded Batik Captioning under Scarce Text Annotation: Label Retention and Source-Specific Error Propagation
This paper tackles batik image captioning when expert text annotations are scarce by injecting predicted class labels from a LoRA-adapted DINOv2 classifier into a 4-bit NF4 Qwen2.5-VL captioner. The core idea is that compact class supervision can guide vision-language generation in cultural-heritage settings where only sparse metadata exists. The authors evaluate whether the cascade retains correct cultural identities or propagates upstream classification errors across six captioning conditions. Their findings matter for digitizing heritage collections, though they are hedged by overlapping partitions and tiny local test sets.
The paper offers a pragmatic two-stage approach to low-resource cultural-heritage captioning, yet its claims about end-to-end behavior are weakened by unquantified data leakage, minuscule local test sets, and missing single-stage baselines. The authors are notably candid about these constraints, stating in the abstract that "The classifier and captioner use separate partitions, and their training–evaluation overlap remains unquantified," and summarizing that their findings "characterize how class conditioning preserves label information and transfers upstream errors into batik captions under scarce text annotation."
The experimental design cleanly isolates class-injection effects by crossing three captioner adaptation regimes (Baseline, PFT, CIFT) with two inference prompts (NCI, CI), and the resulting $\mathrm{MMR}=0.9067$ across all class-injected variants is a clear, robust signal of label retention. As reported in Section 6.2, "All three CI conditions attain MMR of 0.9067 despite their different adaptation conditions." The source-specific classifier analysis is also well-documented: Section 6.1 notes that "The classifier reaches pooled accuracy of 96.10%," and the authors honestly discuss the large accuracy gap between the public (98.33%) and local (75.76%) subsets rather than hiding it behind aggregate numbers.
The most serious methodological flaw is the unquantified overlap between classifier training images and the 150-image caption-evaluation set, which directly undermines any claim about true cascade generalization. Section 7 explicitly warns that "Some caption-evaluation images may have entered classifier training, and the overlap is unquantified." In addition, the local classifier test is tiny and imbalanced, and the authors concede that "Dataset A's 33-image classifier test, including a single Tulungagung image, further limits the stability of class-level estimates." The BF16/hybrid NaN failure reported in Section 6.5 also prevents any comparison of quantization effects, leaving the reported NF4-only caption quality unverified.
The evidence supports the narrow claim that predicted-class injection controls cultural-identity mention, but it does not establish that the cascade outperforms a single-stage VLM baseline or that lexical metrics reflect visual factuality. The authors correctly observe that "Reference-based caption metrics and image–text compatibility scores produce different model rankings," and highlight that "PFT-NCI leads the reference-based measures while attaining the lowest image–text compatibility score." However, without independent human judgments of attribute-level correctness or a comparison against a single-stage VLM prompted directly for class plus caption, one cannot tell whether the cascade adds value or merely serves as an additional pathway for error propagation.
Reproducibility is hampered by omitted implementation details, incomplete dataset provenance, and the absence of released code or weights. Section 5.2 states that "Metric-library versions, tokenization, BLEU smoothing and aggregation, the BERTScore backbone, and normalization details remain unspecified, limiting independent reproduction of the descriptive scores," while Section 7 adds that "Incomplete Dataset-B provenance and metric-configuration details constrain reproducibility." No code, model checkpoints, exact image identifiers, or duplicate-group mappings are provided, and the independently constructed but potentially overlapping partitions mean an independent reproduction would require rebuilding both splits from scratch.
Class labels offer compact supervision for describing cultural-heritage images when expert text annotations are scarce. We investigate a batik-captioning cascade that integrates a LoRA-adapted DINOv2 classifier with Qwen2.5-VL-7B, adapted using LoRA and loaded in 4-bit NF4. A $3\times2$ evaluation combines three captioner conditions with two inference prompts to examine adaptation and predicted-class injection. Across 150 caption-evaluation images, all three injected conditions achieve a ground-truth class-label mention rate of 0.9067. The classifier's separate test yields 96.10% pooled accuracy, with 98.33% on 300 public-source images and 75.76% on 33 local images. Supported-class macro-F1 and balanced accuracy on the local subset are 0.8493 and 0.8302, respectively. The classifier and captioner use separate partitions, and their training--evaluation overlap remains unquantified. Differences in label semantics and training support also affect interpretation of the source-specific accuracy gap. Reference-based caption metrics and image--text compatibility scores produce different model rankings. Qualitative examples show both retention of correctly predicted identities and propagation of classification errors into fluent descriptions. These findings characterize how class conditioning preserves label information and transfers upstream errors into batik captions under scarce text annotation.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.