A Nordic Effort to Fight Disinformation and Information Disorder

Anja Bechmann of Aarhus University’s Datalab is the chief investigator of the EU-funded NORDIS consortium. NordMedia Network is here to inform and inspire the Nordic research community. And with that in mind, managing editor Knut Risnes sat down to have a quick chat with Bechmann, just as she came off the digital stage as keynote speaker at University of Helsinki’s conference “Media platformisation and small nations”.

Knut Risnes (K): First of all, thank you for your time. As we both know, Datalab at Aarhus University coordinates the NORDIS consortium – the Nordic Observatory for Digital Media and Information Disorder. How did Datalab position itself with its key competence to get such a project, which is relevant for both the Nordics and the EU?

Anja Bechmann (A): Datalab has been researching information disorders online since 2017, so we have been dealing a lot with disinformation, more precisely on social media, which is our area of expertise. In line with that, and in line with our research interest in platform policies, we have throughout the years been more and more focused on the role of fact-checkers – and fact-checking in general – in society. That is a completely new infrastructure, which has an increased policy focus especially highlighted by the EU’s High Level Expert Group on disinformation in 2018, but also in the wake of the 2016 American election campaign and the Brexit referendum.

Datalab has been researching fact checking as an infrastructure. As part of NORDIS, we have investigated how fact checking databases are different and thus hold biases. This was done by measuring overlaps, computing and visualizing differences on various factors such as which labels (degree of false) have been used, which countries the databases cover and who made and fact checked the claims etc. Implementing such accounts regularly we believe will improve transparency and accountability and thus trust in fact checking as an infrastructure. Since 2020, we have been focusing on Covid-19 claims in two of the biggest databases in the world (Poynter and Google) and found significant differences in which area of the world they cover, who is fact checking and which platforms they are covering mostly.

We also have investigated how false and misleading claims and conspiracies around COVID-19 have occured in social media and what the emotional climate around such claims were. In the paper we found false and misleading claims to be a very small portion of the total number of Tweets (4%). Overall false and misleading claims had a neutral emotional valence. Yet, when breaking them down into types of misinformation, we found that particular claims about virus characteristics and conspiracy theories contain negative valence. We know from existing studies that negative Tweets travel faster than positive in the network. Thus, negative valanced Tweets could be a focal point in countering strategies. Datalab will continue to run similar analysis in NORDIS but with a specific view to the Nordic countries in comparison with the rest of Europe through collaborations in EDMO (European Digital Media Observatory).

Covid-19 is an interesting research case also from a fact checking perspective because something that has been checked and labeled false or misleading can turn out later to be true with more information. Fact checkers don’t necessarily have access to conduct the systematic check of sources they would in for instance national elections, where they have better access to sources and can better check the status of such sources and associated claims. However, our fact checking partners in NORDIS are starting to see new issues arise again in the spread of misleading claims that otherwise have been focused on Covid-19 since 2020.


K: How does this crystalise in the NORDIS project? What are you doing in the project itself? 

A: We have three things we are doing. 

Firstly, we are trying to heighten the quality of research, especially focusing on the welfare state model that characterizes the Nordic countries. Led by Datalab and Uppsala University, we want to better understand, monitor, and analyze disinformation and information disorder spread across the region. There is a lot of research focusing on the UK due to Brexit, the US in light of Trump, the eastern and southern part of Europe where they have different media and political systems, but not so much has been done on the welfare state model and the Nordic media system.

We are trying to fill a blind spot here – not a total blind spot, but at least an understudied area. We do not assume that there is a lot of disinformation or it is a big problem, due to the Nordics as a best practice case in terms of the media and political structural conditions. Yet, we still see disinformation spreading, and we also see that the penetration rates of social media, especially Meta (formerly known as Facebook), are some of the highest in Europe. So, we do believe it is important to focus on the Nordics, because there are different structural circumstances that we assume lead to different disinformation logics and dynamics. So that is our research hypothesis. We also aim to unpack whether the amount of misleading information is a good enough indicator for whether the problem is big in Nordic societies and if not, what can create additional insights.

Secondly, regarding fact-checking, led by Bergen University in collaboration with Uppsala University and Faktisk.no, our vision is to help create a more united infrastructure, so that fact-checkers are not looking at disinformation by themselves isolated in each country. It is a small business in each country, and to integrate on a Nordic level would, we believe, strengthen the balance to the rest of the world, for example to Russia. We see that the spread of disinformation is not limited to one country. By having a more united infrastructure, we believe that we would improve fact-checking in general. By creating a more unified and standardized infrastructure, we also improve the baseline for detecting misleading information in research projects. Researchers otherwise rely too much on databases with different and undocumented biases that create downstream problems of research validity where findings vary too much depending on which database researchers check against or if researchers use more databases and the structure vary greatly.

The third focal point, which is policy and digital information literacy, is headed by the University of Helsinki. Here, a special focal point in policy is to what extent and how Public Service Media can play a role in countering disinformation. In regards to digital information literacy, the question is: “How can we improve the knowledge of the logics we find in research and make it visible to society?” For instance, our research finds that whether something is true or false seems less important when people share false and misleading claims. It seems to be more important that claims support the worldview of the person sharing the claim.

This resonates with the labels ‘true’, ‘false’ and especially ‘fake’ to be contested topics in the academic world. The critique often is that it is impossible to decide ontologically what is ‘false’ as what is false to me would be true to another person depending on the worldview and the lived life of a person. In NORDIS research, we instead approach ‘false’ and ‘misleading’ pragmatically by relying on the fact-checkers’ different labels. Not as experts in what is ‘false’ but as a profession that checks claims to find sources that can verify or reject the claim. In this way, we are dealing with ‘false’ epistemically and ask how something came to be evaluated as ‘false’ or ‘misleading’ and are trying to document this method to a larger extent to create better accountability and transparency.

K: If I understand correctly, NORDIS is part of EDMO – the European Digital Media Observatory – and there are similar projects across Europe with variations of the problem definition of NORDIS? 

A: Yes, we are part of the first eight funded national and regional hubs, which refers to EDMO’s central unit, where I’m also a part of the executive board representing academic research. The idea is that by trying to unify this infrastructure, we can potentially create a better baseline for understanding disinformation in Europe. The main idea is that fact-checkers and researchers don’t do things only isolated, but instead collaborate on heightening the level, including the efforts. It also allows us to do more cross-country research, have a shared research design, and investigate whether we detect different logics in different regions, which we assume.

Collaborating with non-academic partners

K: You mentioned that fact-checkers are an important part of this, so the project naturally differs from traditional externally financed research projects. How does it differ – the collaboration with these fact-checkers? What’s the pay-off for everybody involved working together? 

A: I understand the question, but a large part of Horizon Europe is dedicated to research projects where you naturally involve several different university partners and partners from outside academia. So, it is quite a natural fit here. What fact-checkers add to, from a research point of view, is that they improve our knowledge of what is misleading on a larger scale – we also call this our study baseline.

If you do fundamental research, but don’t have any collaborators from the industry, you rely on either existing fact-check lists – what have been deemed false claims – that we find is not sufficient at the moment. There are a lot of issues that can create problems downstream in the research, so we want to improve the baseline. And how do we do that? Normally in research, we would hire student assistants to do the coding first, check the claims, then we have a sample that is manually checked. Or we use detection tools – algorithms – to detect patterns in the claims, and then submit a large sample to automated fact-check, or just outsource it to Amazon Turk – non-trained people doing this from a code book developed by us as researchers. But I would say that the goal is to have experts doing it, trained to do exactly that – and those are the fact-checkers for us. We gain from fact-checkers because they can provide the best possible baseline for us in our work. On the other hand, fact-checkers gain from our work because we are interested in looking at patterns and dynamics. When they heighten their knowledge about such patterns and dynamics, they can identify what the probability is for the claim to spread faster than other claims, and then prioritize their work better. We believe there is a synergy here that is very fruitful for the project, both from the research side, the fact-checking side, and of course the literacy side.

(The interview continues below the fact box)

Project partners

Academic partners
  • Aarhus University (Denmark)
  • University of Bergen (Norway)
  • Uppsala University (Sweden)
  • University of Helsinki (Finland)
Non-academic partners
  • Faktisk.no (Norway)
  • TjekDet (Denmark)
  • Källkritikbyrån (Sweden)
  • Faktabaari (Finland)

Impact in the Nordics

A: In the Nordic countries media and politicians have focused a lot on disinformation during elections, and especially misinformation during Covid-19 on cures and vaccines. These issues are not as big as in other countries, presumably due to our trust level. And I assume also because of our media system – we have strong public service media and newspapers with publicist values. Still, there is a spread of misleading information, information designed to harm and conspiracy theories that can have a negative effect on democracy. We want to investigate the scale of the problem and the underlying logics.

One strategy applied in policy to counter the problem of information disorders is to promote regulation for take down procedures and algorithmic downgrading of false and misleading content. An alternative to downgrade and deleting, or censorship some stakeholders might add, is to promote literacy and increase transparency and accountability, and thereby hopefully trust in information. From a societal point of view, having a bigger overview of the dynamics, especially for the Nordics, and having a stronger focus on transparency and accountability, we believe that we are adding to understand and counter societal issues.

K: In Nordmedia Network, we work to facilitate unity between Nordic media researchers, and you have a Nordic project with several countries and institutions involved. How can the project create academic impact amongst those working with disinformation and data science, but also in other areas?

A: I believe that our cross-national design covering the Nordic welfare states and associated media system in comparison with European countries, together with our cross-disciplinary approach, will resonate internationally as a welcoming and additional contribution to an otherwise US and UK centric field. 

We also have a research-heavy advisory board that connects us to the international research environment and other relevant larger research projects.

Hopefully, we will contribute to an improved baseline for Nordic (and European) researchers, which means that all research on mis- and disinformation will have a better validity going forward. 

Additionally, we are testing new language models, because we have an issue with big data models and small country languages. Our experience with these tests will hopefully inspire other researchers and in time improve minority language (such as the Nordics) models and methods.

And then I believe that the project has the potential to be a good case study of the successful integration of external stakeholders and research. We don’t have to be separated in silos to gain from each other in a better way.

I hope it will also strengthen the Nordic research community, because we are now pulled mostly in the direction of English-speaking countries. Also in the way that we publish. NORDIS is a welcoming opportunity to collaborate with our closest neighbours, which media and communication scholars actually have had a long tradition of doing, especially because of the NordMedia conferences and Nordicom as an institution. I hope this will be yet another example of how this can be done, anchored in an international community.

K: This was very informative, and I believe our readers will appreciate it. Thank you for your time.

A: Thank you.

On December 2nd 2021 NORDIS received the Chydenius Medal to mark its promotion of the principles of openness, cherished by Anders Chydenius, who initiated the world’s first freedom of information act (in 1766).


European Digital Media Observatory

  • EDMO brings together fact-checkers, media literacy experts, and academic researchers to understand and analyse disinformation, in collaboration with media organisations, online platforms and media literacy practitioners.
  • EDMO overnasjonalt samarbeidsprosjekt som først ble skissert av EU i 2018
Visit EDMO arrow_forward

Illustration: EDMO/University of Bergen