Robust definitions of credibility and its indicators are required for effective solutions, whether that's automated processing or improved media literacy.
Simply rating a story as “credible” because it comes from trusted providers is no longer enough. We believe that it is crucial for researchers, journalists, and the general public to understand what parts of a story are credible and why. We also believe it is important to understand how to discuss credibility effectively.
What is credibility?
Simply put, credibility is information reliability. While individual members of the Coalition may define the factors that make up information reliability with some variability, we believe that it is necessary and beneficial to work together in creating shared understandings.
This is because the assessment of reliability is complex. Credibility is not a boolean flag, or the simple answer to a "yes/no" question.
Credibility is related to accuracy. But our goal is not to weigh into, for example, disagreements among scientists on the precise pace of rising sea levels or historians who differ on the exact number of deaths resulting from genocidal events. There are a number of articles that are more credible than others, and these articles may be equally credible even if there’s disagreement among experts. The same applies for other items in the news, like breaking news in the wake of a natural disaster.
There is an aspect to credibility that is a negotiation, where publishers and authors seek to convey credibility while readers and platforms seek to better ascertain it. The factors that make up credibility are related to one another in ways that are intricate and not yet well understood.
This complexity is why we anticipate differences among particular interpretations of why one story is more credible than the next, but nonetheless believe that the structure — the relationships and signals of credibility — can be productively defined through a community effort. In fact, given the intricacies of information networks and negotiation, we believe that a collaborative and iterative approach is necessary.
At the heart of the Credibility Coalition's endeavor is the definition and refinement of indicators for credibility.
Example indicators include looking at advertisements and revenue models, the structure of an article, and how it quotes and represents outside experts and scientific literature. The first full version of our schema is forthcoming.