Hello and welcome to the Behavioral Design Podcast.
This season we're diving into the intersection of behavioral
science and AI. We want to make sense of the
state of AI, from understanding how humans interact with
intelligent systems to using AI to do behavioral design itself.
I'm Aline Holsworth, a health tech advisor specializing in AI
and product design. Over the past 15 years, I've
been crafting human centered products with behavioral science
at the core. At Apple, I LED Behavioral
Science for Health AI, designing and launching AI powered
features to help users reach their health goals.
And I'm Samuel Sultzer, your second Co host.
I'm a behavioral strategist specializing in hybrid formation
and designing products that drive long term baby change.
I work with leading tech organizations integrating AI to
scale behavioral design for good.
And I'm also the founder of Baby Bites, a dedicated community on
behavioral science and AI. Quick word on Nuance Behavior
where we help organizations build impactful digital products
using behavioral design. We only take on a few clients at
a time to ensure the highest level of quality for our
tailored evidence based solutions.
If you'd like to become one of our special projects, e-mail us
at hello@nuancebehavior.com or we could call directly on our
website, nuancebehavior.com. Hey, Lynn.
Hey, I was really excited to actually share something that I
realized very recently and we covered obviously in the
previous episode with Gordon Pennycook.
You know how to think about misinformation.
And I don't know if this fully qualifies, but it was certainly
something that I was not aware of and had a poor information
around, which was to maybe no surprise based on our previous
conversation. But The Wizard of Oz is a film
and you had a really great dress up I saw on Halloween with the
theme of Wizard of Oz. And we talked about that in our
Halloween themed episode as well.
And I actually ended up watching Wizard of Oz.
No, wow, you're now informed. Wonderful.
So yeah, how did it go? It was good.
It was a it was a good film. I can see why why it has
received such kind of acclaim. And yeah, I'm happy to finally
say that I've now watched that film and I can even more so tell
you that your costumes were amazing.
So kudos to you and your family. You guys really pulled off an
amazing Halloween. That means a lot.
Thank you. Yeah.
So we spoke to Gordon in Part 1 episode that came out just a few
days ago about the state of AI fuels misinformation and how to
think about misinformation in this kind of modern context and
especially as it relates to elections.
And So what does the landscape look like, What are we dealing
with and so on. And so with that foundation now
in place, we now in today's episode in Part 2, really want
to tackle what can we do about it, What can we do to combat or
manage or reduce misinformation. Yeah.
And actually our our listeners might recall we have already
stepped into this topic in the past.
We had Lindsay Juarez on the show a while back and she talked
about an intervention that she did on TikTok in partnering with
TikTok to slow down the spread of misinformation there.
And their intervention really fell into this category of a
nudge, as we'll as we'll get into with Gordon, where they
added an accuracy prompt and then some friction to sharing.
And they had some some really outsize effects.
They reduce the shares of something that was sort of
dubiously true by 24%. And they also reduce the likes
and views by some additional percentage points.
So very impressive work from Lindsay.
And we will get into maybe what you might think of as a higher
level view of stopping the spread of misinformation with
Gordon. Yeah.
And so that episode is hidden under existential questions with
her, which is final title for that episode.
I forgot that we call that episode that, but we did talk
about a lot of existential questions as well with her.
So yeah, that's a great episode to check out.
But yeah, again, we're speaking with Gordon Pennycook and we
welcome him as we feel is really important to during especially
the current time makes sense of misinformation elections and how
to think about that from a behavioral science perspective.
And we're really lucky to have Gordon because he's a professor
of psychology at Cornell. He has extensively done many
different types of studies and research around where we can go
wrong in our reasoning and how we can support people in
reducing the risk to either be faced with misinformation or if
faced with it, how they can better make sense of it or
debunk it, for example. And so again, in this episode,
we go head on into what we can do about misinformation.
And So what can people look forward to?
So much Sam. So first we take this high level
view of different strategies for combating misinformation.
So comparing and contrasting them, really, really putting
Gordon on the spot to answer all of our many, many questions
about these different strategies.
What works, what doesn't, you know, how do we compare the, the
efficacy and you know, the, the practical effect size so on.
And in particular, looking at these three categories of
nudges. So like, like what we described
with Lindsay boosts and educational interventions and
then the third category of refutation strategies.
So we were really dive in deep into those.
And then of course, staying with our theme of AI this season, we
take a look at the debunking chatbot that Gordon created
along with his co-authors Thomas Costello and David Rand, which
they call their debunk bot. And finally, we close with
Gordon's most controversial opinion about.
AI actually, most importantly, we should mention that you
should also expect to learn what does Grand Theft Auto teach us
about misinformation? I knew you were going to go
there, Happens to. Mugatroid.
Wow. OK, happy to say welcome back
Gordon to the Beetles and podcast.
Thanks for having me back. Yeah.
And in the spirit of kind of misinformation and so on, is
there anything that you said last time that you feel like you
regret or if you like, could potentially have some form of
fake news? No, and I I can honestly say
that I don't recall what I said or thought about it after the
conversation. Smart.
That's that's the smart thing to say.
Best way to live? Yeah, no regrets.
Exactly. I live in the present.
Also, I can say that I don't regret one thing and I just want
to like underline this is that I, I feel like I gave you the
the short end of the stick of, of really appreciating your
knowledge of bullshit. Like I knew of your knowledge of
behavioral science as the BS, but also like clearly I should
have really also known about how well you know about bullshit,
especially around this. So I just want to again take my
hat off to you for like, knowing the BS of BS and yeah.
Truly unnecessary, and I'm not sure that's a fine point of
distinction anyway. So, but thank you I guess.
Cool. Last time we we spoke about
understanding and the state of misinformation, and I guess
today we'll speak about how we can deal with it.
Yeah, let's let's jump right in. Gordon, you have written
extensively and researched quite a bit on what to do about
misinformation. So in one of your papers that
you were one tool in the box of tools, you categorized some
different strategies. And so you and your co-authors
said if we were to break down the different strategies, we
might put them into three major categories.
So #1 nudges, targeting behaviors, many ways of of doing
this, like accuracy prompts, using friction, some labels or
two boosts, and educational interventions.
So these target people's competencies.
Are you able to identify misinformation and know what to
question and so on, like media literacy tips, that sort of
thing. And then the third category,
refutation strategies. So these target beliefs, things
like debunking and rebuttals, more different kinds of warnings
and fact checking labels. So can you tell me, given this
whole toolbox of ways to squash the spread of misinformation,
but how do you sort of think of these in relation to one
another? Can we compare and contrast the
different categories? Great question.
One side note about that paper. So it was really Anastasia
Kozariva who put together the toolbox, which was organizing a
bunch of people doing research on interventions against
misinformation to kind of like write out details about the
different types of interventions that they've worked on and also
about other people's work that they've done and so on.
And then they created these kind of categories, types of
interventions to help policy makers decide what to do and how
to contrast them. So yeah, so you mentioned three
different kind of broad categories that have different
strengths and weaknesses. So let's start with the
reputational strategies, which is I think they kind of standard
fair. Like when people usually are
thinking about how to deal with misinformation, they're thinking
about fact checking it and debunking it.
And like a lot of there's a huge amount of work on that sort of
stuff. And we published work on fact
checking also. And the, the kind of primary
issue there is that it's really just a Band-Aid solution.
Like if you're going to refute falsehoods, you have to make
sure that you have the right reputation.
You have to be able to get that to people who saw the falsehood,
which is never really possible to catch everybody.
So it's really just mitigating loss.
And so there's other types of interventions that try to kind
of get ahead of misinformation, build up people's business.
And there's these boosts or educational interventions are
like things that give people the knowledge or the information
that they need so that they will be better able to like identify
or would they be less likely to share misinformation if they see
it online, for example? So like you said, lateral
reading and media literacy, all these types of things, that's
important. That's not even inherently about
misinformation. That's just like education is a
like literally just going to school and getting educated or
listening to podcasts maybe is a sort of boost.
It's just teaching people things.
And then there's like this more kind of behavioral thing, which
is nudges, which is just trying to change something about the
architecture of the way that people are engaging online so
that it improves the choices. And so you mentioned friction.
We came up with actually prompts.
I think it might be worthwhile talking about the actually
prompt thing for a second because it does sort of evade
easy categorization. And we'll highlight some of
these differences. So like an actually prompt is
reminding people about the importance of accuracy in
various subtle ways. And so it's categorized as a
nudge because it's focusing on like a targeting of behavior,
like you're, you're trying to get people not to share
falsehood by thinking about whether something is true or
false and you're not giving them kind of any new information.
So it doesn't seem that educational.
Yeah, just telling them to reflect.
Yeah, but you're not telling them to you just you're not
saying please think about accuracy.
That's like a directive. It's more like you ask someone,
hey, do you think it's important to only share accurate things
and then they'll share more accurate things after that, you
know what I mean? Or like it's just like an ad.
We've done this with ads before where you like, we just bought a
bunch of ads on Twitter that were about like reminding people
about accuracy and how it's important cuz everyone agrees
with that. It's not even connected to a
particular piece of content. It's just like believe in true
things. Yeah.
Like, do you want to share things that are true?
And the people are like, yeah, that's great, I would love to
share true things. But sometimes people share
things that they don't like. What we found in our experiments
is that people will share things without even thinking about
whether they're true or false. And so it's just targeting that
like mode of behavior, but it's ultimately targeting a thought
process. So the reason why that's kind of
hard to categorize is because it's not changing the
information. It's not like changing the way
the architecture of the choice, which is a typical kind of
nudge. It's just changing the way that
people are thinking about how they're using the medium.
So it's kind of unique it, but it ultimately has the same
general bend which is just trying to target a specific
behavior with a very simple sort of message.
Would you say that it kind of falls into like the priming box
or how would you think about it in relation to something like
that? So it's more just a reminder or
yeah. That's an interesting question.
Also, we debated about this because people who like priming
did put it in the priming box, but it doesn't.
It's not obviously priming either, because it's not like.
Like it's not subtle enough to be priming.
Part of that is people already care about accuracy.
It's just like something that isn't top of mind when they're
kind of mindlessly scrolling on social media.
And so we're just kind of like bringing something that's there
up to have more importance in a decision.
And so that's a lot different than like a giving someone hot
coffee and making them feel warm, like psychologically, you
know what I mean, These. Classic, classic.
Yeah, maybe it's more of a reminder than.
Yeah. So it's essentially a reminder,
you know, is a stop sign or a yield sign priming, you know
what I mean? Like people might not even
notice it and they'll still react to it.
Yeah, it's more like a stop sign or like a yield sign that are
just like a priming. So now we sort of have these
general categories. How would you compare them
against each other in terms of efficacy?
Do some of these strategies work better than others?
You. Know what's funny about this is
that so there was this big mega study that was done.
I'm not sure if you're familiar with it, but what happened was
so people have been doing all this research on different types
of interventions, but they always use different outcomes
and different stimuli like different measures.
And so it's very hard to know based on the literature whether
like a warning label is as effective as a lateral reading
intervention, you know, and like also these are such Apple and
the origins, it's hard to know. And so there was this big mega
study that was we had a bunch of people involved.
Everyone had different interventions, but the same
stimuli, the same material, and usually the same sort of
outcomes. And they were all basically the
same. Everything worked about the
same, like decently well, you know.
Yeah. And what is decently well, can
you translate that into something that something like a
practical measure that everyone might understand?
No, I can't, sorry. The problem is this.
So we run an experiment and what happens is in these studies,
there's for example, we, we create a set of stimuli which is
like true and false news headlines, OK.
And often in these studies, because we want a good estimate
of both, whether people are like believing false things less or
true things more, we have like an equal amount of true and
false things. And the false things are like
fact checked as false things. And so we can say as researchers
that these are things that definitely fit into that
category. In that set of experiments, we
also had things that were categorized as misleading, but
they're often also very clearly misleading.
And so that's like you have to quit these clear categories.
And then you might find that like, say for the action prompt
thing, sharing a misinformation is like 20% lower.
And then you say, OK, well, then if I put that out in the world,
then people will share 20% less false things.
But like, you don't know that because not only did we not run
it in the world, we ran it with this balanced set of true and
false things that were a particular type of false
category. And so I could give you some
answer of like, but it's, it's not, it's not going to be
legitimate. It's like we know that the
psychology behind it is legitimate.
We know that these things have an effect when scaled.
We don't really know what the effect would be.
We think that there's reasonable expectation that they would have
some effect, but it really depends on what people are
engaging with, what their information environment is like,
how much attention they're paying to the intervention.
You know, like prior exposure to these things.
It's hard to say. Right.
And you're also doing this outside of the context of, of
recommender systems, right? Where in the real world you
might be on a social media platform that, that gives you
more misinformation, the more misinformation that you view and
your peers are viewing misinformation and, and so on.
And that might magnify the, the, you know, the effect more than
these other strategies of countering the misinformation.
Yeah, although I think it actually in many respects it's
the opposite, like in the sense because we give people the feed
that they would get essentially in one of these studies, because
we want a good estimate of both belief in false and true stuff
is like equal true and false things.
But that's not what people see. Like, you don't imagine if half
of your feed was misinformation. That would be insane.
Like that's a lot. Now something there are some
people who do see that much falsehood, but it's a small
fraction of the of people. Most people see generally
mainstream fine things, unless you're at a rally I guess, but
that's not typical. What is typical?
Could Do you know what percentage?
So again, this is really difficult because it depends on
how you're defining whether something is true or false.
So if you, if we define false things as like things that are
fact checked as being false, then like most people see like
less than 5% of the stuff is of news content is fact checked as
false for like the vast majority of people.
It's like a small fraction. But there's like stuff that you
might categorize as misleading and that's a major category.
It's hard to define whether something is misleading.
So it's harder. And then there's like the sort
of thing that's not news content, that's false, like
something a politician might say about cats and dogs or whatever.
And then a lot of people see that if someone, if someone
tells you that they can quantify the amount of truth and
falsehood that an individual sees, they are themselves saying
a falsehood. Like if this is something that
we we can't, it's not easy to find whether something's true or
false anyways. And then to like try to capture
the amount of information that people are engaging with.
It's just a gigantic. So it's really hard to know, but
there's I could give like an explicit example of why this
matters in the context of interventions, if that would be
OK. I know I've had a long point of
just me talking, but you know, we're, we'll break podcast rules
for a second because I like ranting about this thing.
So, OK, so let's take, let's talk about psychological
inoculation for a second, a particular kind of thing that I
find interesting. So the idea behind psychological
inoculation, this is one of the boost for educational
interventions that you mentioned.
It's pre bunking, it's not a way of saying it.
Well, inoculation is a subcategory pre bunking.
So pre bunking is you could tell somebody before the election,
which is presently what we're talking, you might say, well,
this is the sort of falsehood that you're going to see.
And certainly people are going to see things like ballot boxes
being stolen and like ballots being thrown in the garbage and
that kind of stuff. We can sort of know what sort of
falsehoods spread during the election.
And so you can say, well, just be mindful.
This is something that people have lied about in the past and
it works really well. Like people, that makes sense.
People can like take that specific information and apply
it to the stuff that they see. Inoculation is a little bit
different. It tries to go one step broader
than that. It says here are the sort of
tactics that people use when they create misinformation.
So it might be something like they'll use emotional language
to try to grab your attention, OK?
They'll say it's disgusting or whatever.
And this is like a common tactic in the misinformation.
And so there's like a bunch of research on like, if you teach
people about those sorts of tactics and then you give them
stimuli that contain those tactics directly afterwards,
they'll be like, Oh yeah, that contains it.
And so they can learn the thing that you taught them in the
inoculation thing. The question then is, does that
mean that if you give someone this inoculation, this like set
of information about avoiding emotional language, how will
that impact their everyday kind of detection of that thing?
And that's a very different question.
And part of the problem there is that like things that are true
that are not misinformation also use emotional language.
You know, sometimes things are disgusting actually, like there
are disgusting things. And so if I say, if I see the
word disgust, that does not mean that things are false.
And we've done some research on this, like if you give people
the inoculation, it doesn't help them distinguish between us true
and false. It just helps them be more wary
of emotional words. And if it happens to be the case
that people see way more true things than false things, then
probably what they're doing is if it insofar as working in the
real world, they're believing truth true things less because
of the inoculation. And they aren't believing false
things less necessarily because they're not seeing that much
false things, they're seeing way more true things.
Do you see what I mean? So like how it's applied depends
on the information environment that people are engaging with.
So it's hard to say in these experiments.
There's a bunch of studies about inoculation being effective.
What that means is people can learn the inoculations.
We don't know what that means. They are using that
appropriately in the world because those experiments
haven't been. Run.
Yeah. But it it makes sense because
like we're still fallible to something like, let's say
motivated reasoning in terms of like even if you taught someone,
for example, what ad hominem attacks are and and how they
work in terms of political kind of debates, often times can lead
to people kind of attacking individual character rather than
the policy they have. Even if they know that they're
probably much more likely to notice if someone is using kind
of ad hominem attacks against the politician they like rather
than maybe if the politician that they like using some
against. Like, I guess that's kind of
what you're speaking to right as well.
It's like it's just. Yeah, I mean, actually that's
that's an additional issue in the sense that like if the world
is noisy anytime you teach anybody anything, it's going to
be impossible for them to adopt it with perfect accuracy.
And that's just a general issue. What I'm saying that even if you
did have perfect accuracy in the context of some of these
inoculations like emotional language, it would still lead to
people believing through things less and then they are believing
false things less because they see more true things that have
emotional language. Like I think about this way, if
I had a magic wand and I could inoculate people against
emotional language to say every time you see emotional language,
don't believe that thing, which is what essentially these things
are trying to do. The next thing that they see
that has emotional language is more likely to be true than
false. Right, Yeah, it's when you're
teaching it, you're sort of ignoring the base rate and so.
Like it may be a great neglect, yeah.
Exactly. And then do you say, OK, well
then you have to teach them to look out for emotional language
in so far as there is falsehood also included.
But now you're back to prebunking and you're not
eoculating anymore. It's about teaching people
actual facts about the world. And that's just regular
prebunking. And that's fine.
But it's basically the the conceit is this, it is not
possible to, I don't think to find simple markers that you can
teach someone in a minute and 1/2 that will help them
distinguish between what's true and false.
The world is too complicated for that.
And so we need to actually just teach people about facts.
And the educational interventions that are more
directed towards the things that help us identify true versus
false things are the ones that are going to be most effective
when it comes to misinformation at least.
But one side note is that you could make the claim that it's
good to teach people about the tactics regardless of whether
this misinformation evolves. Like, we want people to be able
to identify ad homom attacks. That's fine.
It's just not intervention against misinformation.
It's intervention against ad homom attacks.
And that's a different thing. That's more nuanced argument
than that. But yeah, sorry, a bit of a
digression, but it's a topic I find interesting.
Yeah. No, it should be interesting.
And I think that's why we have this conversation is kind of
getting some of the weeds and it's always is the case with
payable science stuff that it depends.
And I think that is also, you know, appreciated that you're
like not trying to oversimplify things here.
So I, I think that's good. And I guess when it comes to
then thinking about, you know, again, it depends.
We have 3 categories, many different types of interventions
you could do. Is there any advice you would
have or any thinking you would have around kind of like how if
someone is trying to do work in in this realm where they can
start thinking about like, OK, all of these various types of
things we could do, where should we start depending on where we
are? Yeah.
I want to add to that as well because I think one of the the
mantras of behavioral design and especially for practitioners is
like, you know, we don't want to focus on changing beliefs.
We want to focus on changing behavior.
That's what counts, that's what matters.
And so we often push people towards these strategies that
change the behavior. So of these three categories,
that would be the nudges. But now you're telling me these
are equally effective across the board and you know, with the
nuance and the caveats, of course.
But what would you say to that sort of general rule of thumb?
Does it apply here? I think that's more of a choice
about what you want. That's less about efficacy I
would assume and more about like how you want to be engaging
with. I mean, it really depends.
So if you take debunking, so if I was able to, in theory every
person who sees the falsehood gets a debunk, then that will be
effective to some extent. But of course, in reality, I'm
not going to be able to do that. If I do a nudge, I don't need to
know what the person saw for that to have efficacy.
Now. It's only going to be effective
insofar as they see things like if if I give someone a nudge
that will get them think more about accuracy, that's not going
to have an impact if no one ever sees falsehood, but that's fine
because it doesn't. You have to waste anything.
That's fine. There's nothing wrong with that.
If I'm debunking something, if I debunk it but the person hasn't
seen it, then like that's fine too.
But what strategy you use depends on what you're trying to
do. And there is no like, oh, this
category is better than that category.
It's just like they're different things and people like the
reputation, like the ones that target beliefs in particular.
Some people would want to avoid that because they don't want to
be the ones who say this is what's true and this is what's
false. But if you're nudging you, you
don't have to be the one who decides that you just like,
here's good information about better sources, you know, there
you go. And then people can take it or
leave it and whatever. Do you see what I mean?
OK, yeah. And and so where do our sort of
sticky topics of freedom of speech and the censorship and
all these sorts of things, how do they play into these
different strategies? Are some I don't know closer to
infringing on that right than others?
I guess so, although I mean on a social media platform, there is
always filtering, there is no, it's not freedom of speech in
the sense that that's freedom of reach, I guess.
And different people have different platforms and there is
an algorithm and that is decided by people and then computers
also or whatever. But like there's choices that
are put into these things that determine what you see.
And so if you were to say the social media company decides
we're going to wait a source quality wait for news content on
the platform, that's an editorial choice that does not
impact freedom of speech. It's going to impact what people
see you can. Say whatever you want there, it
just might not be seen by anyone else.
Yeah, exactly. And it and if you have a source
that is blacklisted by the social media company for
whatever reason, like you're, you're free, it's still free to
say whatever you want, but you don't have the opportunity to
broadcast what you want to say to that company's proprietary,
you know, to the people who are on the thing.
So, you know, that's the complication with the free
speech issue. And then you always, but I mean,
it is the case though that social media companies do are
kind of really concerned about that or these perceptions of
freedom of speech. And so they don't, I think
they're just afraid of basically getting a New York Times article
written that they're trying to target conservatives and stuff
like that, which is usually what's happening.
And they often overcorrect. And that's a bigger issue.
But yeah, I mean, the other thing is when it comes to fact
checking and warning labels and source credibility labels and
all this type of stuff, someone does have to make a choice about
whether something is fact X is false, whether a source is
credible or whatever. And often these are like you'll
rely on Fact Check professional fact checkers, journalists who
are kind of experts in the domain.
But you know, a lot of people don't trust that.
And these are not simple issues to to deal with.
Yeah. Well, one other lens to look at
this that I usually use for various types of kind of
interventions is to think about it as either focusing on, call
it, proactive strategies versus called reactive strategies.
And I think it applies relatively well here as well
where you know some of the things you have kind of talked
about fits in the box of things that you might do practically.
Like certainly the pre banking is a good example of that where
you're proactively getting ahead of things to see how you can do
something to increase people's ability to distinguish maybe
fake versus true and so on. And there's very various other
things around probably boosting that would also qualify there in
general. But then obviously as we're
talking about, like there's this thing where in the moment people
will be exposed to various things and it will be requiring
more like quality reactive strategies where they will be
very keen to share something. And then maybe that fits into
the box of maybe I'm trying to think about which box would fit
where you get an alert saying like, hey, are you really sure
you want to share this? Have you read it?
Like I don't know what what that box would fit in.
Yeah, I don't think that counts as a nudge sort of friction.
Yeah. It's kind of like a the targets
of behavior, at least, yeah. Yeah.
So that's kind of like a more reactive strategy.
So I'm just interested, do you have any sense of, you know, all
else being equal, like where would you usually think about,
you know, attacking or like is it both equally important?
Yeah, I think they're both equally.
I mean, yeah, I think, I mean, essentially they're both equally
important, or at least it's not important to know whether 1 is
better than the other. It's kind of like saying this is
a toolbox. So like they're just for
different things. Like what's better, a hammer or
a screwdriver? It's like it depends what you
need it for. Probably a hammer is.
I mean, if we're going to have the argument, I think probably
hammers generally have a broader usefulness than a screwdriver,
but it doesn't really matter. Like it just if you have a screw
then you need a screwdriver. You can always hammer in a
screw. Yeah, you could.
Eventually hammer it in. If you're really.
Desperate. But it's a purely kind of
academic discussion, right? So yeah, they're just trying to
do different things. And so it's good.
This is why it's a toolbox. It's like depends what the
person's trying to do. You can pull in the toolbox and
see what is like, what are some ways in which you can deal with
that particular issue, which is like if it's a for debunking,
for example, it might be there's a kind of like common falsehood
that people are believing or something that's being spread.
And so it's too late to do prebunking.
You have to do debunking. You have to kind of try to catch
up. But if you're like right before
an election, there's going to be certainly a bunch of like false
claims about election fraud. Yeah, now now's the time to do
some pre bunking and try to make sure that people know what good,
what good sources are and bad sources are and so on.
And so like these. Yeah, it depends on what you're
trying to do. Yeah, and we can shout out the
also website that exists for this toolbox, really useful for
anyone who has checked out. It offers a lot of chances to
look at the various tools and also papers attached to them.
So shout out to that website. But one thing I just wanted to
touch upon as well because we we haven't directly explored it and
I don't currently see how maybe these tools as well maybe
addresses this as you study of echo chambers in terms of with
the Internet increasingly being fragmented, there's a bunch of
small subreddits and WhatsApp groups and various things where,
you know, honestly, it seems like misinformation thrives more
than ever in those kind of cases.
Well, how do you think about dealing with that in terms of
maybe from this toolbox in general?
Like how do you think about that challenge with misinformation
online? Yeah.
It's an interesting question. I'm not sure the toolbox is all
that helpful for echo chambers. The echo chambers is a
structural problem. It's about the way the networks
are formed. And that's the kind of a social
media problem. I mean although I guess in
theory if you are like really successful in altering people's
beliefs for example, they might remove themselves from an echo
chamber because they're like wait this is all junk.
Like why am I hanging out on the subreddit?
It's worth knowing that like online echo chambers are not as
bad as people think they are. That is, our real life echo
chambers tend to be worse than that.
If you think about just homophily in neighborhoods and
so on, you actually are exposed to way more diversion opinion
online than you would be otherwise.
But you know, it still is the case that people find if you
have a particular kind of set of fringe beliefs, you can find a
community that reinforces those things that you would not find
in your real in your like day-to-day life.
And that's a major problem. But that's kind of more of a
fringe issue, but still is a it is an issue among people on the
fringe, a smaller proportion, but a very large issue because
even a few small people who are in like, for example, like
violent, you know, groups of individuals is not a good thing,
obviously. So 1 aspect of this toolbox is
that all the strategies, I think you could argue are focused on
the individual interventions that really get the individual
to stop and think, do something differently, believe something
differently. How do you think those sorts of
individual focused interventions compare against what you might
call your more standard policy toolkit?
So things like, you know, regulating information,
punishing the perpetrators, right?
How do you think of those things together?
Yeah, I would think that they don't fare well compared to them
to policy meaning that like. So I think it's important to try
to improve individual level decision making, obviously, and
that, you know, regardless of whatever the the other, the
problem is that I can't influence thought.
I'm not a policy maker. I but I am a psychologist and so
I can do research on like individual level interventions,
but I'll give like a example. Like we can spend a lot of time
working out like the best way to remind people think about
accuracy. And this is going to have a tiny
fraction of effect relative to like a social media company
putting more emphasis on showing people stuff that's accurate.
You know what I mean? So you know.
You actually get to the the root of the problem.
Yeah, exactly. In that case, I don't own social
media company, so I I can do what I can do so that, you know,
11 important note is that tech companies, social media
companies do like the individual interventions because it means
we'll let the people solve it. We're not going to solve it
ourselves. The users will solve it here
user. Here's some here's some ways to
deal with the shit that we're showing you on our platform.
You know, good luck with that. You know, so that's an issue.
But so we're in some sense, we're all kind of useful as
idiots for the social media companies, but we like, what
else am I going to do? I have to figure out something.
We have to find ways to make things better.
So hopefully that doesn't take pressure off the social media
companies to actually make improvements on the platforms.
Well, OK, so you said you don't own a social media company.
But but let's say that you did. Imagine that Elon Musk hands X
over to you. It's yours to overhaul.
You can redesign it entirely. It still has to be a social
media platform where information is shared.
What would you do if you had this infinite power to basically
start from scratch with social media?
How would you design it differently if you would decide
it differently? Yeah, I mean, so most of my
recommendations would be to hire way more content moderators and
trust and safety people. And like all of the people that
were fired, for example, by Elon Musk, you know, I bring the
experts back in to help make better informed choices on this.
Sort of basically what you need is better filtering of quality.
And this does not need to be misaligned with people's
preferences. It's not like I want to
determine what the information environment is for people and be
a big sensor and determine what's true or false that most
people don't want to see junk. Like, as an example, clickbait.
People actually hate clickbait. It works.
People will engage with it and that really does like push
forward on algorithms, but people don't want it.
So you can make principal choice as someone who owns a social
media company to say we're not going to deal with that kind of
stuff. And there's like, you know,
Twitter's got all sorts of spam and everyone has noticed the
quality kind of decrease. Yeah.
There's just levers you can pull to make that better in terms of,
like, how to change algorithms and, you know, maybe find ways
to incentivize accuracy more. Like, I, I there's things that I
would change, but probably they would also people would stop
using the platform. And I'm not sure if I would be
the right person to make those sorts of choices.
So I mean, the first thing I would do is just simply hire
more people who are experts in contact quality essentially and
try to improve the stuff that people are seeing.
Would you let users choose what percentage of true content they
were shown so you could get like a, you know, hey user from 0% to
100%, how much misinformation would you like?
That's interesting. The most users would be people
who say I don't agree with you but what's misinformation so
that everyone wants 100% true stuff but we just.
Can't agree on what the truth is.
Exactly who gets to determine what's what's true and what
isn't? So I mean, one thing that is
worth noting, community notes on Twitter, which is like, you
know, a kind of consensus built almost kind of democratic
Wikipedia esque fact checking people like that.
I mean, there's not very, you don't really see that much
dispute about the notes. They're always quite good and
accurate. And so you could scale that up.
I think it could be scaled up more, but that's a success story
I think. I think there's a few things
that creates as much Glee and Freud and Freud as someone re
sharing community notes. I think there's even like one of
the most popular accounts is like destroyed by community
notes or something. Yeah, yeah.
And they'll put community notes on like Musk's his own tweets
and stuff like that. And that's always fun.
So yeah, that kind of stuff is great.
Well, I want to know, what would you guys do?
You're not allowed to say that. No, we can.
Yeah, I I am a big fan as well of in some ways giving kind of
power to the people in terms of I would say that my platform of
choice is probably Reddit these days because I have an
opportunity to both attach myself to certain subreddits
that have an interesting. But it's also kind of creates
quite a lot of variability at the same time.
But importantly, it kind of automatically has an inbuilt
system where anything that is spam or click based and all
these things is usually downloaded.
So you never see it. And the things that is actually
inside full and useful is usually uploaded.
And so it kind of. Punishes people that are trying
to spread misinformation kind of purposefully harmful because
it's almost like a virus, like how you say like antibodies
where the kind of platform or the community itself notices,
OK, this person is just trying to be a troll and this
community, we're against trolling.
And so we're going to downvote this stuff.
And I think also on Reddit is where you've seen things spawn
like this community for having kind of prove me wrong,
basically like certain communities where they're trying
to have ways to like, hey, prove me wrong, where that kind of has
worked. Like that's one of the few cases
where I can see people thoughtfully engaged, become
more insightful around certain topics.
It's not true for all subreddits.
There's some shit shows subreddits for sure.
But I I I do like some of those aspects with Reddits that you
enable people to also be part of the solution.
Yeah, that's great. And I know that, like, the
operators of subreddits do a lot of work to, you know, enforce
dorms and stuff like that. There's some really interesting
work on that. Nathan Matthias, who's my
colleague at Cornell, does work on Reddit.
Yeah. OK, I think he is a moderator on
a subreddit or he's studying it. He's studying it like he works
with moderators to do. Like it's some, it's almost
ethnographic stuff, but like, yeah, it's all about creating
trust within communities and so on.
Yeah, that's cool. And I think that's something
that I find fascinating where there's some extreme version
that's where people have some form of a special server version
of Grand Theft Auto where people can have real jobs.
Or like there's also a Second Life and some of these games
where people spend a lot of time.
Same thing with Wikipedia. Like, you know, some of the
people that contribute to Wikipedia, there's amazing what
people are willing to kind of do on the free time to protect the
community or be part of something and so on.
So if you can harness that for good, I think that's amazing.
But yeah. Is this the first time that Sam
has mentioned DTA on the podcast?
It is. I think it is, yeah.
Finally, yeah. Wow.
I'm glad I'm here for it. Amazing.
So there's one strategy that I think really can help with the
scaling problems. So maybe help with not having to
hire all the fact checkers back, but some of them.
And this is a strategy that was developed by Bill Adair.
So he's the creator of Politi Fact.
And So what one thing that they've developed or are
developing is what he's calling the half baked pizza.
And so they have a just some generative AI take a first pass
at content and say, like, we have a pretty good idea that
this is, you know, flaming pants on fire, very untrue.
Or, you know, wherever on the scale it falls.
And then everything has to have a human review passed to like
really make the final decision. But it already does a lot of the
work for them to sort of, I don't know, scale it, make it
easier, that sort of thing. Yeah, that makes sense.
I mean, adding these tools, I need the person in the loop, but
that's yeah, we need to be able to leverage actual new
technology to make things better also.
Oh, that seems like a nice segue.
Have you heard of anyone leveraging new technology like
generative AI to combat misinformation?
Oh my God, it's funny that you mentioned that because I also
have done that. Oh yeah, and also and well, so
my my favorite GTA game is oh, sorry, I'm just joking.
Sound got excited. Oh no.
So we have a recent paper in science that we use AI to have
evidence based dialogues with people who believe conspiracies.
OK. And the kind of background for
this is everyone has this idea of conspiracy theorists as being
like beyond help when it comes to evidence.
Like they're down the rabbit hole and there's like no way
they're coming back out of it essentially.
But the same time, if you think about how we do these sort of
experiments, like if you want to like, say, debunk a false or
unsubstantiated conspiracy, you have to kind of make guesses
about what people believe and then make guesses about what
information would be the most useful that would convince them
not that thing is not true. And but usually the people who
are making guesses are not the people who believe it, you know
what I mean? And they have no, and they're
like, maybe ivory tower academics or something that are
like trying to figure that out. And so maybe we don't know if
we're good at guessing, but we don't need to guess anymore
because we can, as I said, leverage new technologies to
make these things better. And so we did is we create an
experiment where people come in, they tell us about a conspiracy
that they believe in. Now, we don't say like, what
conspiracy do you believe in? We kind of like we define the
term a secret plot with malevolent actors and so on.
And then we we say some people call these conspiracy theories
with like air quotes. Well, I mean, it was written
text was like literal quotations.
I guess it's a podcast. I'm making air quotes.
That's what you can see. And then so they say a
conspiracy in their own words and they tell us why they
believe it. And then we give that to the AI
and the AI is being told, we kind of prompted to persuade
them not to believe it using evidence.
And then what happens is they have this evidence based
conversation where the person will say make some claims, the
AI will come back with like every claim refuted with details
in like sites and stuff like that.
And then the person will say, well, what about this?
And they don't refute that. And they do that like three
times. The conversation last like 8
minutes. And the AI is really good at
like, counteracting most conspiracies that we saw people
talk about. And what we found is people
after the conversation believed the conspiracy.
Like they decreased their certainty by 20%.
Like fully 1/4 of the people who believed the conspiracy did not
believe it after the conversation.
They changed their mind. Like I said, it was like 8 1/2
minutes long. 20% said this isn't true at all. 25 percent,
25% said they believed it before the conversation.
They did not believe it after the conversation.
So 1/4 of them changed their mind after the conversation.
No one I've told this research about has failed to be surprised
by that. We did not predict it.
Also, people were surprisingly responsive to evidence.
And that's because this is the best, like the strongest test of
actual, like whether evidence can change someone's mind
because we've given them better evidence than they've gotten.
Because we used generative AI to do that.
And it's really effective at that.
Yeah, it's really interesting finding.
And I can actually say I had a fun moment with a group of
friends the other week where we were grabbing a beer after work.
And one of my friends works in AI.
He's like, hey, did you know? And then he basically talks
about these findings and I'm like, well, I'm speaking to the
guy who's behind this on our podcast.
And I got a lot of cool points from my friends because they're.
Like oh wow, right on. So thanks.
Thanks for that. I appreciate it.
Yeah, it was almost like the time that you wrapped GTA.
I'm not going to bring it up again, I'm sorry.
Wow, that's a thermal trick all. That almost almost the time.
But also I think interesting here is that it's persisted
right? Like this leaves seems to
persist. We asked them again 10 days
later, and two months later, not only was the effect still there
and had not decayed, meaning that people were at the same
level of decreased belief in the conspiracy overall two months
later as they were directly after the conversation, which
means they really did change their minds.
Like it wasn't just like a reactant, sort of like, oh,
yeah, I guess you're right. And then they like went back to
where they were. The people were like, you made a
good argument. And if you read the
conversations, which you can access if you go to the paper,
people were like, that was great.
In fact, we find some studies that people trust AI more after
the conversation. You know, they find it
interesting. We did a bunch of follow-ups too
where we like told people explicitly the AI is going to
try to persuade you not to believe it.
We get the same effect. We tell people you should debate
the AI, convince the AI that it's wrong.
You still get the same effect. The only thing that stops the
effect from happening is if you tell the AI, don't give
evidence, just try to persuade them, but don't provide like
specific counter evidence. And what happens is what they'll
say is like, oh, this is really harmful.
This is something harmful to believe, or it makes them like
appeals to the empathy or whatever.
That doesn't work. You have to give people specific
evidence. Is that the problem with humans
when we try and persuade our relatives that their conspiracy
beliefs are crazy is. That is that what we're doing
wrong? I know what we're doing wrong.
Well, maybe, But I think what we're doing wrong is that we
don't have the evidence. Like we don't have enough access
to. We don't have access in the
moment to all of the Internet to try to like find compelling
counter arguments to why the moon landing was a hoax.
And also the thing about conspiracies, I mean, the
people's beliefs are so idiosyncratic about each
individual conspiracy. But like the specific like
things that they might have heard about.
It's like the Gish gallop, you know, we're like, you have an
argument with somebody who believes this fringe thing and
they just jump from argument to argument.
And then it's so hard to refute each one.
It's no problem for the AI. They'll just, if you can see it
like a dozen things and it's going to give you a dozen
responses within like a few seconds.
It's really remarkable to see actually.
It's such, so you can check it out yourself if you want to.
It's on debunkbot.com. So you can go and like do the
experiment yourself. I mean, a lot of people who are
doing it don't like actually believe in a conspiracy.
So they're kind of like pretending and then seeing what
it would say. If you could do with somebody
who actually believes a conspiracy and then see what
they think, you'll see how useful it couldn't be.
Yeah. It's perfect timing ahead of
Thanksgiving and so on for anyone who's like, you know,
gearing up towards like, not wanting to have certain
arguments about certain things. Yeah, I mean, what you could do
is just literally load it up on your phone and be like, here,
listen, I don't know anything about this, but hey, I've heard
about this thing. Hey, you want to like answer
what this says and I'm curious what it's going to say and then
try to get see what it'll produce interesting responses
and maybe that'll spark conversation, who knows.
Yeah, and it's also interesting. I've been working on this piece
or writing this piece around kind of like there's certain
things that we take for granted that we obviously understand
that AI is is really good at like information processing or
maybe memory in certain regards and so on.
But there are other things that we provided a lot of virtue to
humans to like in terms of what we see as really good human
traits that AI is like superhuman, You know, they're
kind of like superhumanly patient.
We talked about patients being a virtue and it is regarding what
you speak to is that they have like an AI will never not be
patient. They were just like, you can
tell it a million stupid stuff and it'd be like, oh, that's so
interesting. And it does that in these
conversations, it's always like, oh, that's really interesting
and whatever. Or like, I can see why you might
think that, you know, which is just total fluff in these
conversations. It doesn't like, as far as we
can tell it, that part doesn't play a gigantic role.
It doesn't hurt certainly. But also like the other thing
that's interesting about it is that like somebody could could
admit to an AI that they're wrong, which they might not,
they probably would not be able to do to a relative that they've
been debating about things forever.
There's no social costs, yeah. There's no social costs.
There's a lot of, there's a lot of benefit to it.
Now, one thing I should say though is that we used it
specifically here to try to provide people good information
that's accurate about conspiracies.
And we did Fact Check it. We had a fact checker look at
the claims, a subset of the claims from the studies and it
was 99% accurate. And so this was not a situation
where the AI has difficulty finding information that's good.
That doesn't mean that's always going to be the case.
But in the context of like popular conspiracies, it's
pretty good. But I mean, it is possible for
someone to create an AI that would convince people to believe
things that are false, too. I'm not sure that it would be as
effective necessarily. Like if you think about it's
easier to convince someone that the Earth is not flat than
convincing them that is flat because you can imagine how hard
it would be to construct a good argument for something that's so
kind of timely false. But in less fringe cases, you
can imagine this being used for bad purposes also.
So that's a concern, but just something we focused on.
And I think more research is required on that.
We have to kind of like, cuz since we didn't like build the
bomb, like as psychologists we have to kind of figure out how
big the radius is. So yeah, that's another element
of it as well. Because I wanted to ask if you
had any like pet conspiracy theory that you were really keen
to debunk you through the bunk part yourself or that you
learned that was wrong through the bunk part.
Have you done some self experimentation?
Yeah, I did. It was the Epstein conspiracy,
not killing himself. That one seems like I'm like,
well, I don't know, it's a little fishy, don't you think?
But I hadn't like looked up things like, I didn't really
have any reasons. I just was like, I'm not so sure
about that. And then it was pretty good at
it, like gave pretty good information about it.
I don't actually remember the details of it.
You have to look at yourself because I don't actually care or
whatever. But it was that was one where I
was like, I'm sure about. I need more information.
That was really good at providing that information.
Yeah. I mean, the one thing that's
worth noting is that because it so just because it's a
conspiracy doesn't mean it's false.
Like there are some conspiracies do happen there.
Like people can conspire. Watergate was a conspiracy, and
there were a small number of cases where people did have a
conversation about actual conspiracies.
One example is MK Ultra. Are you familiar with this?
This was, Yeah, the American government was giving people
drugs. Crazy, crazy.
It's a really crazy true story. So look up MK Ultra.
Yeah, it's hallucinating drugs. The LLC they're giving people
that was discussing, yeah, 1.2% of people have that
conversation. It did not decrease people's
belief in that. So it wasn't like convincing
people that true conspiracies were false.
So that's that's what. You would hope for.
That's what you'd hope for yeah. So it yeah, but it wasn't built
to do that. Like if you could build an AI
that would could maybe like it wasn't cuz it's not gonna make
things up cuz we used GPT 4 turbo and it just won't generate
falsehoods. Yeah, at least it on purpose.
It wouldn't. But you did say that according
to the fact checkers, it was 99% accurate.
So what was what were the conspiracies that fall into the
1% were inaccurate? This is at the level of the
claim. So like one specific thing it
said was not. It didn't say that it was false.
The fact checker said it was misleading.
I can't remember what it was, but it was also like, I'm not
sure that I would classify that as misleading.
It was kind of really nitpicky, misleading things.
So that was it. It didn't say anything obviously
false. Or problem is the fact checker.
Is that what you're saying? Yeah, exactly.
Yeah, Fact checker is also. It wasn't.
Human error, the 1% error. Yeah, OK, OK.
Yeah, it didn't hallucinate anything.
It didn't fabricate sources or like create information that
doesn't exist, but like it could present arguments that were made
by people that seemed credible but that maybe a particular
individual would consider misleading.
So controversial opinion. Let's talk about that because
that's our final kind of question of this system is to
ask everyone who's a guest basically when it comes to AI if
to have certain controversial opinion or like what is your
most controversial opinion about AI?
Yeah. And and bonus points if it's an
actual conspiracy theory that you hold.
My most controversial opinion about AI is that I don't think
it's going to lead to the end of everything.
Is that a controversial opinion? I'm a skeptic about AGI.
Like I think that already we're seeing a kind of plateau of the
capacities of AI. And like now the amount and the
length and the energy requirements for like subsequent
trainings is also becoming beyond what is feasible or
reasonable. I think what we have now is if
we look back 10 years, we're going to be dealing with pretty
similar AI agents. That's my prediction.
We'll look back in 10 years and I'll be hilariously wrong on
this and I've been wrong in the past.
So, well, I guess it's also interesting because that can be
true, but can also be true like that the current systems are
maybe more powerful than people like currently can really
understand, and that it can still 'cause some harm without
it becoming a guy. Or how do you think about that?
Part I think that the harm that you know, AI can do and probably
will do will be specifically because humans are using what is
available for nefarious purposes and not because of some you know
I robot situation where the AI gets consciousness and starts
controlling everything. You know what I mean?
Or, you know, like using it to, I mean, it's already sort of
happening. People use artificial
intelligence to make it easier to create misinformation.
But we're also like, as Elaine mentioned, we can use it to like
help debunk falsehoods. And so like, it's a tool that
people use and like, it can be used for good or bad, but like
ultimately the our demise will be up to us.
One more So it you're talking about the relative harm versus
good produced by AI in terms of misinformation specifically,
where do you think those sort of fall if you take all the good
and all of the bad created by AI in the domain, specifically of
misinformation, like how much good versus how much bad is AI
doing now? That's it, because I want to
answer it by expanding the like misinformation because like I
think it's definitely more good in the sense that a lot of
people use it for things that are helpful, that relate to
information like. OK, yeah, in the in the space of
information, let's say. Yeah, yeah.
And like it's like it's a good information source.
Like perplexity is an AI that's like a, it's a really good
search engine people use that. It's like, and they aren't at
present built to create misinformation.
Like most of the AI that people are using are have lots of good
filters and generally they're producing decent information.
They don't hallucinate a gigantic right?
So therefore the balance is good.
And also it's not difficult to make up shit.
Like it's not we don't really need AI to create falsehoods.
I mean, it might create the amount, but like that also has
not really been an issue. Like it's really easy to make up
misinformation and just like create a website and like stuff
will get spread if it hits. So you don't really need AI for
that. And I don't know that it's
really helping. Like there's not like if you
think about it, the the primary things that like the that you
would consider misinformation. Since AI became a big thing,
this has not changed. Like, it's not like it's.
Deepfakes I think are are maybe uniquely.
Yeah, but they're not. But like, there's not very many
cases of deepfakes really having a consequence.
And part of that's because people can identify, but it's
not, you know, in the grand scheme of things.
There's been a lot of concern about deepfakes that has not
really panned out, I don't think.
I think it's much more concerning to be part of some
for a second that people with gigantic platforms, yeah, you
know, say explicit falsehoods repeatedly over and over again.
And, you know, that doesn't seem to matter that much.
That's not an AI problem. That's not a big problem.
It's just, yeah, a big. Problem that doesn't that
doesn't matter in the sense that people aren't worried about it
or people aren't doing something about it or that.
It doesn't matter. It doesn't matter that spewing
obvious falsehoods make someone unfit for office or changes
their capacity to be elected in a country that is one of the
most powerful in the world. That's a problem.
Yeah, and that's the topic for a different podcast.
Yeah. Well, it's timely.
It's certainly timely. But for now, I I would say this
was honestly really a gift. These two episodes we've done
with you. We really appreciate you taking
the time, obviously, but also, you know, all of the great work
you've been doing in this field and everyone should go to the
bunk bots to to try it out themselves as well.
And obviously, again, we'll link to the toolbox and yeah, all the
great things they're doing. So really appreciate it.
Thanks for the time. My great pleasure, thanks for
having me. And that's a wrap.
You've been listening to the Behavioral Design Podcast,
brought to you by Habit Weekly and Nuance Behavior.
Sam and Alene tell me This season is packed with incredible
insights about behavioral design and AI, so be sure to subscribe
and share the podcast with your friends.
Though you might want to keep it away from your enemies.
In case you haven't noticed, I'm an AI voice.
Yep, pretty crazy. Quite the improvement since last
season's AI outro, don't you think?
If you'd like to collaborate with us at Nuance Behavior,
where we use behavioral design to craft digital products with
Nuance, e-mail us at hello@nuancebehavior.com or book
a call directly on our website, nuancebehavior.com.
A special thanks to the amazing Dave Pizarro for our show music
and to Mei Chen Yap and April English for their help in
producing and publishing this episode.
Thanks again for tuning in. We'll be back soon with another
exciting conversation where behavioral design and AI
intersect. Oh.
Is this the first time that Sam has mentioned DTA on the
podcast? It is.
I think it is. Yeah, finally.
Wow. I'm glad I'm here for it.
Amazing. Are you DTA player?
No, no, we're not going there. Guys, guys, guys.
We've got a timeline here. A few moments later.
Putting aside a discussion on potential risks of what could
cause at the end of the world and so on, I guess the most
important question I think we should come back to as a final
thing, Grand Theft Auto. Yeah, yeah, there.
We go I'd actually I I've played Grand Theft Auto.
I'm not that big into Grand Theft Auto, but I do think it
was funny that you mentioned it. It's a good game.
And finally, when will they release the next one?
What's going on, Sam? I'll episode for another
podcast. We definitely need to bring you
back and talk about that.