From Indias largest newsroom, I'm Arun George and this is the
Times of India podcast. We're just two phases into the
long national elections in India and two Bollywood actors have
already filed complaints over fake videos featuring them being
shared on social media. Aamir Khan and Ranveer Singh
file Police Complaints over videos that was circulated on
social media. They claim they were campaigning
for a political party. Celebrate Kare Hamari.
Dukhi Hui Jeevan. We done this episode at the
beginning of this year about deep fake videos, and we're
bringing back that episode today.
We've spoken with Karen Ribello, who's a deputy editor with fact
checking organization Boom Live, about why it's so hard to tackle
AI generated fake videos and other forms of disinformation on
the Internet. We'd ask Karen what kind of
misinformation is most commonly found in India.
India is a country where you see a lot of misinformation purely
in terms of the volumes of misinformation, it is a lot like
on an average we do about four to six fact checks every day.
And right now I think in the country there may be 15 to 20
fact checkers as well and they do similar volumes as well.
So purely in terms of like, the amount of misinformation is
enormous. What we've generally seen over
the past five to six years, we had a big problem and I would
say we still have a big problem with something called as cheap
fakes. So cheap fakes are something
where you know, it's manipulation which doesn't
involve very high technology As such I crop a video or I speed
up the playback speed of a video or I, you know, morph a photo
using Photoshop or something like that.
Those sort of things would be called as cheap fakes.
Now that doesn't involve AI. What we have seen over the past
one year I'd say is that we are seeing an increasing amount of
AI based misinformation and this trend sort of started I think
around mid last year. So in May last year, you had
that image of wrestlers protesting and they were
detained inside a police van during a protest.
And there was an image which went viral, which showed them
smiling while they were sitting inside that van.
And the narrative around that photo was, look, these guys are
doing it just for a few laughs or that they're not serious
about this protest. We can see that here through
social media posts from netizens like this one, which reads that
the photo is part of a a drama, part of a toolkit in order to
break our country. And of course later on we found
out it was edited or doctored with an app called Face App,
which is an AI based editing app that happened in in May last
year. And since then there has been an
increase in the sort of AI based misinformation, especially that
has been catered to a local audience in India.
The deepfakes that we used to see were mostly Joe Biden or
Zelensky, you know, something related to the Russia, Ukraine
war. What we started seeing last year
is like deepfakes in the Indian context and this sort of
increase. We saw a lot of deepfake
pornographic videos involving Indian actresses.
There were always pictures that were circulated on platforms
like X. And then of course, the Rashmika
Mandana incident blew up. I did a story this month that
during the Madhya Pradesh elections.
So the Madhya Pradesh elections happened in October, November.
We found that a lot of videos were doctored with AI voice
cloning. So you had people make voice
clones of Shivraj Singh Chauhan and Kamal Nath and overlay them
onto real videos of these politicians where you know, you
have them saying completely fabricated statements, things
that they've never said. And those videos went viral.
Just to see how, how effective and how convincing it is, I went
and created 2 audio clones of of those two leaders and it was
terrifying because it was so real.
It sounded just like them. Me barbar Karahu Kamal Nath ko
roko Congress agai to Sabko Pandora surupe mahine or path
surupe. Me gas Denne lagegi, Chodo ugly
Barbie hamara jetna namkin ho jayega.
And you could never tell that a machine had created that or any
sort of AI. So in terms of the use of, you
know, AI to spread misinformation, India has
already seen it last year itself in the context of an election.
So we already have precedence in terms of the general election.
We can expect to see AI based misinformation.
What changes when it's Artificial intelligence based?
You know, these AI voice clones are also being used to scam a
lot of people and to create videos where they're
impersonating the voices of actors.
And they're either, you know, promoting some betting apps or
some fraudulent investment scheme or like diabetes
medicines. And all of these videos are like
sort of rampant. They are almost everywhere on
Facebook, on Instagram and stuff like that.
So now I think what you're seeing is now the first
iteration of how these things are playing out.
And the reason why I guess a lot of people and a lot of fact
checkers and researchers and technologists are worried is
because of the level of sophistication and and the scale
that this sort of technology offers to bad actors.
So it's very easy to do something and you know just
amplify it on a very large scale and it's and it's a lot more
difficult to catch. I would say the the reason why
this thing here is problematic is also because the tools that
we currently have to catch and detect these things are not very
reliable, especially with AI, audio, voice clones.
Just by hearing it the average person or even a fact checker
trained fact checker, you would not be able to spot the
difference. And plus I guess the human brain
has evolved in such a way that it overcompensates for, you
know, discrepancies in audio. So if I tell you that I'm
playing a voice of so and so person, and even if it has a few
glitches, your brain will still make that leap to try to make it
sound like the person I've just told you so.
In that sense, AI voice clones are far more dangerous than AI
based images or videos because it is harder to catch.
Karen, could you talk a bit about the misinformation sort of
industry in India, especially given India is both a massive
creator as well as a massive consumer of misinformation?
The kind of misinformation that we see in India is, is a bit
unique in the sense that the two big baskets if I have to
categorize misinformation in the country, it would be religion
and politics. And now increasingly the the
separation between the two is blurring.
So that is something we don't see in a lot of other countries
where you have so many claims around religion that is like a
unique feature that we see in India.
But generally I would say, you know the the sort of
misinformation that we see tends to come from the right wing
purely on the basis of the fact that there is a nationalistic
government in power and there is a tide of majoritarianism that
is sweeping the country. So the sort of narrative that
accompanies it will also see a lot of misinformation.
That is not to say that there's no misinformation coming from
the left, but it's the scale and the reach are not comparable at
all. It's almost like negligible
majority of it is coming from the right wing ecosystem and
that ecosystem has a lot of different motivations for
creating misinformation. So one is that it's very
lucrative in India to create misinformation.
You know, if you are a YouTube or a content creator, then then
sort of jumping onto this bandwagon, you can get a lot of
traction and make a lot of money.
And that is what we tend to see. Right earlier it was Facebook
pages and accounts. Now it's shifted to YouTube
videos and influencers and and that sort of ecosystem is
financially very rewarding. Then you also have a whole set
of people who ideologically are aligned to this Causeway.
So they may not be doing it for any financial gain as such.
They're doing it just because they 100% believe that what
they're saying is right and so they do it.
So these are the sort of people that we see as well.
And another unique thing I suppose in India and why we do
have a lot of misinformation is that around, you know 2015, 2016
you had a lot of people come online for the first time with
mobile Internet rates falling through the floor.
So then you have a whole section which you know is not that I
would say savvy when it comes to on just in terms of dealing with
online misinformation. So there are number of factors
that have sort of created this very unique situation that we
find ourselves in right now. In terms of India's handling of
misinformation, for one, how has it been so far?
And you know, we now have two laws coming in.
You've got these sort of public announcements against deepfakes
and AI generated imagery and voice cloning.
How much do you see these things?
Solving it in the short term at least?
Yeah, I'm not too optimistic that the laws are going to be of
much help. I think they're going to
complicate things further, especially if they are not, if
they don't take all stakeholders on board and their opinions.
So in my opinion, generally laws have not been very effective
when it comes to curbing misinformation.
You know, a few countries around the world have have sort of
tried it, but it tends to lean towards, you know, either
censoring content or crushing dissent.
We have to be very careful plus the nature of this this problem
is so complex that you can't like have A1 size fits all
solution. So to be honest I'm not very
optimistic. There are some low hanging fruit
which is easy to do. One is the election Commission
should should come out with some sort of guidelines that prevent
the use of generative AI and deepfakes, especially in in any
sort of campaign, purely because we don't know the extent to how
this tech can be abused. We have examples most recently
in Bangladesh where you had deep fakes of candidates and their
voices were cloned and it was overlaid onto a video S it
appeared like the candidates saying that they are dropping
out of the elections and asking people not to vote for them.
So that kind of thing is terrifying when it comes to deep
fake pornography and CSAM, which is child sexual abuse material.
There is no sort of difference of opinion from anyone that this
is bad for for everyone, right? Everybody's on the same page
that this stuff is harmful and it should be taken down.
That should be addressed on war footing.
From my own experience, I'll tell you that I have reported
several handles on Elon Musk's X platform for tweeting and
posting explicit deepfake pornographic material, and the
reply that I've gotten from X has been that this does not
violate our community standards and it has happened on more than
one occasion. Platforms also have wake up to
this and and do more and not just pay lip service to this
problem. The path ahead looks very
difficult. How effective are these platform
based initiatives so far in terms of curbing the spread of
misinformation? So right now you have a very
unique situation where these platforms are almost imploding
like X. It's become this hub of
conspiracy theories and misinformation.
And we saw this during the Israel Hamas war where you had
misinformation on steroids. And it was like a dam breaking
in terms just in terms of the volume of and the amount of
misinformation and all of it was playing out on X.
So X went and changed its blue tick verification system where
it allowed basically anyone to pay $8 and buy that blue tick
mark. And at the same time it
completely decimated its trust and safety teams.
Facebook as well, you know, laid off a lot of people in in those
teams. So now what you're seeing are
like the consequences of those actions.
It brings me to the point that, you know, there is a structural
and fundamental problem with the way these platforms are built.
They are built in a way that it fosters polarization, that the
most divisive content and conversation and material out
there gets traction. I also see there's a lot of
conflict of interest in the sense that a lot of these and
it's not just the platforms, I guess it's all big tech is that
on one hand they are saying that you know they're worried about
AI based misinformation and all of that.
But on the other hand they are coming out with products and
technology and features that enable you to to use AI.
So let's say a Facebook comes out with a feature that enables
AI editing where you can change the background of a photo, you
can insert something that was never there.
You can in fact make a whole video with things that were
never in there. Now they use polar bears and
puppy dogs in in examples for, you know, these products.
But that is not what it's going to be used for, right?
People are going to use it with real life instances of people
and do all sorts of harm. So this conflict of interest
where you know on one hand you're saying oh we are doing
everything we can and we're using our tech and we're
partnering with fact checkers and and all of that.
And then on the other hand, you are building these products
which again fuel the same problem that we are fighting.
Karen Ribello says that these tools that allow the cloning of
face and voice should be more restricted than they are.
She points out that while they may be created for
entertainment, it doesn't take very long for them to be
misused. First of all, I don't think this
technology should be widely available.
Like what is the point of allowing someone to clone
somebody else's voice? And if you are doing it, then
you need to build detectors within your product that are
free and not subscription based where anybody can use it.
These image generation websites, they say they have safeguards,
but if you are determined to sort of bypass those rules, you
can find a way to beat the system.
And we've seen that, right. You have examples of AI based
images used in the Israel Hamas war.
I think they need to really clarify what their sort of
approach is to tackling this problem because as of now, I
think all their efforts are like coming to naughty.
But with a lot of these AI tools, The thing is also they're
seen as tools to foster creativity, a sort of leap
forward in terms of the human imagination, right?
Would say markers like we had, say, a reverse image search
earlier or something like that. Do markers like that actually
help detect a fake? Do people actually invest their
time in finding out it's fake? So from our experience, I mean,
I I don't think people spend that much time and this argument
of these tools have been built for enhancing creativity or
productivity. It sort of reminds me of the
conversation that happened when social media platforms were new
around 15 or 16 years ago. Social media platforms came
into, you know, existence with people promising us the idea of
building communities. Now how much of of community
building has happened over social media than actual
destruction is a conversation for another time.
You know, we are making the same mistakes with generative AI
because the problems that all these social media platforms
have faced. I see all of the same problems
with generative AI as well. When it's in its first few
iterations, right now is the time to talk about it and fix
these problems before it gets worse.
A lot of the debate around the curbing of misinformation has
come to center around artificial intelligence based tools that
are used to create misinformation.
What's spoken of less now is that platforms like X, YouTube,
and WhatsApp are still very active sources to spread
misinformation in spite of all the measures that have been
taken over the years. Karen Rubello explains the
things that platforms can do as a sort of quick fix to curb the
spread of misinformation. There have been studies which
show that, you know, when people come across information that has
been flagged by the platform as misleading, whether it's on X or
on Facebook or anything, there is a sort of slowing down effect
that happens in terms of its velocity.
We can't just say, you know, if whoever wants to forward it will
still forward it. But yeah, it makes a difference
in terms of slowing down its distribution for sure.
In addition, steps like. Enforcing some sort of monetary
penalty or something. So, you know, making sure that
pages that or accounts that share this information can't
earn a lot by sharing misinformation, those steps help
as well because what we saw during the Israel Hamas war is
that a lot of the misinformation came from verified blue tick
accounts. Now a policy that says, you
know, they will not be able to monetize if their content has
been marked or flagged, that certainly helps because it
incentivizes the creator of misinformation.
So small steps like that definitely play a huge role.
There's been a lot written about the work of a fact checker and
the sort of thanklessness that it is.
What has it been like for you to work on curbing the spread of
misinformation, especially given, I mean, even AWF report
kind of points out that you're literally sitting in the hub of
it in some ways. Yeah.
So I was thinking about it the other day, and I figured that
I've spent almost like half my career in this space, which is a
long time, so to say. Misinformation has definitely
evolved in the sense that it's become more pervasive, it's
become more sophisticated and it's definitely more influential
in terms of its effects that we see.
Initially that I used to think that only negative news events
leads to to a lot of misinformation and and that
still holds true to a large extent.
But what we're seeing right now is that, you know, almost every
newsworthy event is sort of accompanied by misinformation in
India. So it's almost become par for
the course. And yeah, it's an uphill battle
and and personally for me it's been hard but completely worth
it. I wouldn't change it any other
way. Today's episode was produced by
Jayaraj Singh and Sahil Gupta. For a daily spotlight on people,
ideas and stories that matter, subscribe to us.
We're available on TOI plus Spotify, Apple, Google Podcasts
and all other platforms of your choice.
For any new steps, e-mail us at toipodcast at Timesinternet in.