AI forces democratic society to rethink

“IT’S NO LONGER JUST Swedish civil servants who exercise public authority,” says informatics researcher Marie Eneman. “Now, it’s algorithms too. This presents a challenge for democracy, as it makes accountability more difficult.”

Sweden’s government administration is based on an organisation with independent agencies that implement the Riksdag’s decisions. Corruption is low, and the ability of Swedes to trust each other and the authorities contributes to a society that is characterised by stability and wellbeing.

“It’s called the Nordic gold, but ongoing technological developments may challenge citizens’ trust in the state. With the advent of AI, transparency is becoming more difficult and it’s no longer possible to follow all the steps through to an official decision, as we allow AI to decide certain things for us with the help of incomprehensible algorithms.”

As a researcher, Marie has long been interested in the legal implications when AI and machine learning are used in the exercise of authority.

WITHIN RESEARCH, the lack of transparency surrounding AI systems is referred to as a ‘black box’. When AI is used in particularly sensitive operations such as policing, social services or defence, this becomes what is known as a ‘double black box’ which makes it even more difficult to understand and appeal an authority’s decisions.

Several government agencies have recently become dependent on AI technology delivered by private companies. The state is keen to maintain security and democratic autonomy while deepening technological dependencies.

“The Swedish police used an AI facial recognition technology that was neither procured nor sanctioned by police management,” adds Marie. “This was a free version from a US company that collects billions of facial images from the open internet. These images form a database that can be used to identify criminals and victims.”

Following a review, the police were forbidden from continuing to use the software and the American company promised to delete the data collected from the Swedish police.

“But can we really trust a company to do so? How can we have transparency when large private actors outside the EU are collecting data on citizens? It would be unreasonable if we didn’t allow the police to develop new methods in their fight against crime. That’s why we need to find a way for authorities to use algorithms that are both efficient and legally secure.”

SURVEILLANCE SHOULD NOT currently be understood as isolated technologies, but as interconnected infrastructures in which different data sources, systems and legal mandates interact. This means that CCTV, body-worn cameras, covert data mining, biometric systems and other information flows are increasingly linked and analysed together.

“The risks rarely lie in a single technology per se. They arise when different forms of surveillance are merged into larger infrastructures where data is shared, combined and made actionable. This also changes the conditions for suspicion, responsibility and democratic control.”

In July, new legislation will be introduced that allows the police to use facial recognition in real time. This means that potential offenders can be identified even before the crime has taken place. The question is what happens when suspicion is no longer based on concrete actions, but on probabilities; when surveillance no longer just sees, but anticipates.

“This raises an urgent question: When surveillance is increasingly about identifying possible behaviours, who decides what counts as a risk, how that risk is calculated, and who is thus subject to suspicion?”

Marie argues that we need to rethink and reimagine what effective and meaningful regulation and oversight of authorities’ use of AI should look like.

“Legislative proposals are evolving at a rapid pace, which makes good oversight extremely important. Just because society is facing a problem, that doesn’t mean we should introduce permanent surveillance. I therefore believe that surveillance legislation should be reviewed after a period of time in order to remain in force. We don’t know who will be running our country in ten years’ time and how they will use surveillance tools.”

ANOTHER GROWING THREAT to trust in society is AI-generated false information in audio, text and images, known as deepfakes. From having been a novelty, these are now widespread and appear in the everyday flow of social media. Many of them are created as entertainment, but deepfakes are increasingly also used in various forms of online fraud and disinformation campaigns.

Children and young adults are particularly vulnerable. Many of them view large numbers of short video clips on platforms like TikTok, where deepfake content is constantly present. The fast-paced scrolling that these platforms encourage does not leave much scope for reflection or critical thinking. Users are encouraged to keep moving on to the next clip, without questioning the authenticity or the source of what they are seeing.

“This is a serious problem,” says social media researcher Chiao-I Tseng, who supports age limits for using social media. “Logical and critical thinking skills are not yet fully developed in young people’s brains. Younger audiences are therefore more vulnerable to misleading and emotive content. They react more based on emotion and intuition.”

Image

Chiao-I studies how news content changes when it ends up on social media platforms and how young groups can be reached that way. She has seen how deepfakes have become an effective tool for the populist mass media. In Germany, for example, far-right groups have used deepfakes to reinforce their message online. By combining emotional appeal with fast-paced visual storytelling, they can reach young viewers.

DONALD TRUMP’S AGGRESSIVE rhetoric on social media is often amplified by the dissemination of manipulated or misleading images, illustrating how political communication is eroding the line between fiction and reality. Chiao-I believes that few leading politicians in Western Europe will emulate Trump to reach a younger audience. Instead, the trend is to distance themselves from the style and tone that the US President has adopted.

“Right now, deepfakes are on the rise. It’s a dangerous phase. At the same time, as people are becoming more cautious and sceptical about what is said and shown, the use of deepfakes will level off. This is often the case when new technologies develop.”

A good example is when moving images first emerged more than a century ago. Many people were worried that films would become a powerful propaganda tool, which they did.

But over time, society has learnt to manage and navigate the impact of film.

How will citizens be able to distinguish between information that comes from authorities and what is actually disinformation and deepfake content?

“The EU is drawing up legislation that sets clear limits on the use of AI and deep-fakes,” continues Chiao-I. “This will enable those who misuse the technology to be held accountable. It’s an important example of boundary setting, and the basis of this implementation is that authorities should stick to not using AI to produce deepfakes, regardless of the benefit.

“Every technology has a downside, but AI is also the most powerful tool we have for detecting deepfakes.”

Text: Olof Lönnehed