AI that asks questions helps tell if a social media post is promoting a conspiracy
This paper looks at a hard problem: deciding when a short social media post actually endorses a conspiracy theory. The authors argue that the main difficulty is not spotting conspiracy words, but reading the speaker’s intent — whether the post is sincere, sarcastic, critical, or merely reporting. They propose an “agentic” system: an AI that can call small tools to fetch social context and then decide what evidence it needs for each case.
The team built and tested the approach on a large collection of Hebrew tweets that covers roughly 80–90% of public Hebrew tweets from late 2018 to early 2023. That span includes several election cycles and the COVID pandemic and vaccination debates, which provide many context signals. For any target tweet the agent can request different kinds of context: the user’s profile and recent posts, the surrounding conversation, metadata such as the posting date, or parts of the user’s social network.
At a high level the system works like a careful reader who looks around before judging a sentence. Instead of appending every possible piece of context to the input, the agent decides which small, relevant queries to run and uses the returned evidence in a step-by-step reasoning process. The paper gives a clear example: a tweet that names Pfizer and a “New World Order” looks like conspiracy at first glance, but context (the author’s profile saying they ‘talk nonsense,’ recent weather-related posts, and other users discussing a real storm that day) shows it was sarcastic. A text-only model labeled it a conspiracy; the context-aware agent labeled it correctly.
The experiments use a manually annotated adversarial test set. The authors report that workflows that fetch and use context do better than text-only classification. The agentic framework — which adaptively asks for evidence case by case — performs significantly better than other settings, including a model that is given the same context but does not choose queries dynamically. The paper also analyses common errors and discusses tradeoffs between accuracy and efficiency, for example how much extra data (tokens) the system requests.