Workshop Proceedings of the 17th International AAAI Conference on Web and Social Media
Workshop: Novel Evaluation Approaches for Text Classification Systems (NEATCLasS)
DOI: 10.36190/2023.53An important approach to understanding and mitigating the spread of misinformation is to recognize whether a given social media post aligns with the false information (that is, agreeing with it) or disagreeing with it. In this paper, we present CoMID, a method that detects whether a tweet agrees or disagrees with a misinformation claim, based on the content of the tweet and the author's propensity to spread misinformation. To calculate the propensity of the user, we utilize their past tweets and profile description based on a new model. We evaluate this method on our newly introduced dataset, "COVID-Myths", and compare it to existing state-of-the-art content-only and user & content-based methods. In general, the proposed model, CoMID, is beneficial and achieves a 5% performance gain (in terms of the F1 score) compared to the best-performing baseline. Additionally, we evaluate the generalizability of CoMID in a zero-shot setting by leveraging only the weakly supervised data. CoMID achieves state-of-the-art performance in this setting, which suggests the effectiveness of utilizing user data to capture propensity.