Conspiracy Frame: a Semiotically-Driven Approach for Conspiracy Theories Detection
The paper proposes the Conspiracy Frame, a semiotic and frame-semantic representation of conspiratorial narratives with five elements (plan, secret, in-group, out-group, call-to-action), and introduces Con.Fra., a span-annotated Telegram corpus. The core hypothesis is that injecting FrameNet-derived semantic frames into LLM prompts will improve conspiracy detection and explainability. Results show that while frame-guided prompting achieves comparable classification scores to few-shot learning, it does not consistently outperform it, though it reveals interesting abstract semantic patterns.
The paper makes a valuable conceptual contribution by operationalizing semiotic theory into a structured annotation scheme, moving beyond binary classification. However, the central claim that frame-semantics improves detection is not convincingly demonstrated—the frame-guided approach shows mixed results (outperforming few-shot in classification but not span detection) and the authors admit that semantic frame information adds limited discriminative power beyond in-context examples.
The theoretical grounding is rigorous. The Conspiracy Frame successfully translates semiotic theory into a concrete annotation scheme with clearly defined core elements (plan, secret) and non-core elements (in-group, out-group, call-to-action). The span-level annotation achieves substantial inter-annotator agreement ($\kappa = 0.808$ for Plan/Event, $0.750$ for Call-to-Action), validating the scheme's reliability. The FrameNet mapping heuristic successfully identifies abstract patterns like 'Kinship' and 'Ingest_substance' that align with theoretical expectations about conspiracy rhetoric.
First, the binary classification task shows only moderate inter-annotator agreement ($\kappa = 0.41$), raising questions about the ground truth reliability, though the authors mitigate this by releasing disaggregated annotations. Second, the frame-guided prompting strategy shows inconsistent benefits: it improves recall but falls behind few-shot in span detection (winning only 213/1,000 ELO iterations for span detection versus 830/1,000 for classification), suggesting frame hints may introduce noise for fine-grained extraction. Third, the lexical mapping to FrameNet has inherent ambiguity limitations that the authors acknowledge but cannot fully resolve, and notably, the theoretically central frames Execute_Plan and Secrecy_Status do not appear among the top recurring frames, indicating a disconnect between the conceptual framework and the lexical realizations.
The authors appropriately contextualize their work against existing binary classification datasets (LOCO, WICO) and correctly identify their limitation of lacking structural narrative information. The cited claims about annotation difficulties are accurate: Hemm et al. (2024) indeed demonstrate low inter-annotator agreement among experts for CT detection, and Mompelat et al. (2022) confirm challenges in distinguishing conspiracy from mainstream texts. The comparison to related work is fair, though the paper would benefit from stronger baselines beyond Llama variants to validate the generalizability of the frame-semantics approach.
Reproducibility is partially supported but has gaps. The dataset is available at an anonymous repository, and the annotation guidelines and prompts (zero-shot, few-shot, frame-guided) are exhaustively documented in Appendices A and B. The paper specifies FrameNet v1.7, SpaCy en-core-web-lg, and the ELO evaluation methodology (1,000 iterations). However, critical implementation details are missing: LLM inference parameters (temperature, top-p), random seeds, and the specific lexical heuristics for filtering FrameNet mappings beyond the general mention of removing copular verbs and tail frames. The limitation to Llama-3.3 models only (8B and 70B) without testing other architectures limits confidence in the generalizability of the frame-injection findings.
Conspiracy theories are anti-authoritarian narratives that lead to social conflict, impacting how people perceive political information. To help in understanding this issue, we introduce the Conspiracy Frame: a fine-grained semantic representation of conspiratorial narratives derived from frame-semantics and semiotics, which spawned the Conspiracy Frames (Con.Fra.) dataset: a corpus of Telegram messages annotated at span-level. The Conspiracy Frame and Con.Fra. dataset contribute to the implementation of a more generalizable understanding and recognition of conspiracy theories. We observe the ability of LLMs to recognize this phenomenon in-domain and out-of-domain, investigating the role that frames may have in supporting this task. Results show that, while the injection of frames in an in-context approach does not lead to clear increase of performance, it has potential; the mapping of annotated spans with FrameNet shows abstract semantic patterns (e.g., `Kinship', `Ingest\_substance') that potentially pave the way for a more semantically- and semiotically-aware detection of conspiratorial narratives.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.