Workshop Proceedings of the 16th International AAAI Conference on Web and Social Media

Workshop: Workshop on News Media and Computational Journalism (MEDIATE)

DOI: 10.36190/2022.51

Published: 2022-06-01
A Structured Analysis of Journalistic Evaluations for News Source Reliability
Manuel Pratelli and Marinella Petrocchi

In today's era of information disorder, many news organizations are moving to verify the veracity of news published on the web and social media. In particular, some agencies are exploring the world of online media and, through a largely manual process, ranking the credibility and transparency of news sources around the world. In this paper, we evaluate two well-known procedures for assessing the risk of online media exposing their readers to m/disinformation. Specifically, considering a fixed set of media outlets, we examine how many of them were rated equally by the two procedures, and which aspects led to disagreement in the assessment. The result of our analysis shows a good degree of agreement, which in our opinion has a double value: it fortifies the correctness of the procedures and lays the groundwork for their automation.