AI content has increased people’s doubt about what is true and verifiable

Intro: Protests and violence erupted in two Punjab college campuses last week over fake reports of rape, suicide and missing students. Professor of information at University of Michigan Joyojeet Pal tells Himanshi Dhawan that this is reflective of a disinformation crisis where a small group of people, willing to peddle misinformation, could quickly become the mainstream voice due to rising distrust in institutions.Would you say disinformation has reached alarming levels in India?Three findings matter for the explosion of misinformation in India. First, people do not like to see news they do not agree with; second, bad news travels much faster than good news; and finally, polarisation increases the potential for misinformation. Put together, you have a population that believes another subset of their fellow citizens are in fact their enemies, a population that refuses to accept news that undermines that dislike for the other group, and a social media ecosystem that encourages disagreement. In addition, research shows people are unwilling to challenge those higher in status or older. Also, millions of Indians came online directly through the sudden rise in cheap data and devices. They encountered WhatsApp and YouTube before the older textual web and did not entirely learn to be sceptical of online content.What makes campuses particularly vulnerable to such “viral videos”?Campuses have historically been sites of political protest.They are closed institutions where young people share ideas together. Young people fear authority and consequences less than older people do; they are more likely to be driven by heroism and the belief that their actions are critical to the future. We as a society give young people priority and leeway to articulate their moral positions. We see a student protest very differently than a workers’ union march. Indians are seeing a generational shift in which students who have been raised very differently than their parents on issues of discipline and questioning authority. So, when a young person sees someone like them being shamed online through a value lens that they see as outdated, they are more likely to signal dissent. Their protests are feared by politicians not because of the consequences to the students, but because parents will not take it sitting down when their children are attacked by the govt.WEF’s Global Risks Report ranks India high on misinformation and we continue to be vulnerable to unsubstantiated rumours.The WEF report identified misinformation as India’s leading short-term risk, which means it won’t just be polarisation, it is serious enough to hamper India economically. The drivers revolve around two ideas — identity and institutions. We have tremendous diversity, and at any time one or another of those identities can become more salient, making us willing to believe that those who prioritise a different identity are necessarily our enemies. Second, Indians fundamentally distrust institutions such as the judiciary, police, and bureaucracy. In the past, that distrust was articulated in private conversations, but smartphones have industrialised dissent by giving it channels. What once circulated as a rumour can be weaponised quickly into an idea that reaches millions. This is the key: when a bad idea gets circulated among 100 people, it is possible that only five of them will buy it, and they may be shut down within the group. But when it reaches a million, they can suddenly break off into their own WhatsApp group of 50,000 people, and then give it life with its own truths.What are the political and social consequences of this?India, like many other places, also has a history of rumours preceding real violence. The scary consequence is how this situation will manifest during a crisis. What happens when people distrust institutions, news, and each other outside of narrow issues? We got a hint of this during the covid crisis, when people believed all kinds of things about who was spreading the disease or its cures. But if you look at it from a different perspective, in each case there was someone willing to capitalise on the disinformation. Imagine a new catastrophic crisis — a climate disaster, a new pandemic, or a massive cyber attack. That small 5% group willing to peddle misinformation may quickly become the mainstream voice, and the less we believe the govt or the mainstream media, the more influence fringe beliefs gain.How has the use of AI affected this problem?The problem with AI is not so much that it has created a flood of false material that people believe, but rather that it has increased the level of doubt people have even about content that is true and verifiable. The one danger we are not yet thinking about is that AI does not just create individual artefacts, it can help create an entire backstory that is much harder to debunk. Situations will increasingly arise in which AI can reverse-engineer the queries most likely to be used to debunk an existing artefact and then create new content presenting compelling narratives to support that original fake content. Another misconception with AI is that it will be used to target public figures. Research shows videos about politicians are quickly debunked, while negative content about a random person, presented as a representative of an entire community, will do plenty of damage before it gets debunked.Authorities usually clamp down on the internet and restrict public gatherings. Doesn’t that make matters worse?One of the most enduring research findings about the spread of rumours is that they accelerate during epistemic black holes. The fewer opportunities a community has to confirm or deny what is being floated, the more likely misinformation is to thrive. An internet shutdown serves as precisely such a black hole, in which the average citizen has no way of knowing what is really going on and will blame the govt for taking away their right to find out the truth for themselves. The unintended long-term message is that the state cannot be depended upon during a crisis, since they failed citizens precisely when they most needed reassurance.