Hello and welcome to the Behavioral Design Podcast.
This season we're diving into the intersection of behavioral
science and AI. We want to make sense of the
state of AI, from understanding how humans interact with
intelligent systems to using AI to do behavioral design itself.
I'm Aline Holsworth, a health tech advisor specializing in AI
and product design. Over the past 15 years, I've
been crafting human centered products with behavioral science
at the core. At Apple, I LED Behavioral
Science for Health AI, designing and launching AI powered
features to help users reach their health goals.
And I'm Samuel Sultzer, your second Co host.
I'm a behavioral strategist specializing in hybrid formation
and designing products that drive long term baby change.
I work with leading tech organizations integrating AI to
scale behavioral design for good.
And I'm also the founder of Baby Bites, a dedicated community on
behavioral science and AI. Quick word on Nuance Behavior
where we help organizations build impactful digital products
using behavioral design. We only take on a few clients at
a time to ensure the highest level of quality for our
tailored evidence based solutions.
If you'd like to become one of our special projects, e-mail us
at hello@nuancebehavior.com or we could call directly on our
website, nuancebehavior.com. Hey, Lin.
Hello. Hello.
Hey. I feel like I owe you a thank
you. You know why?
Tell me, I mean, I can think of a lot of reasons.
Where do we start? That is true.
No, I, I received a package that I was like, what is this?
I didn't order this. And then I looked inside and 1st
it took me a few seconds for like things to click, but then
I, I got so happy, like you made me so happy.
And for the listener, I'm actually wearing the thing that
you sent and it's AT shirt that probably boasts Kremlin duvet
advertising, which probably for like every listener means
nothing. But for me it means a lot.
So thank you. Amazing, amazing.
Do you want to share the background?
What Kremlin advertising is duvet Kremlin?
Yeah, yeah, yeah. Duvet Kremlin.
So I have mentioned I think in an episode with I think it was
Gordon Pennycook, where I mentioned that I'm a die hard
comedy fan in various ways. And I I enjoy comedy in the best
of forms from stand up to TV shows and so on.
And what has been my like really favorite find in the last few
years has been this really, really fantastic show called
Detroiters. And it's a show that two seasons
for some reason got cancelled. But it's with Tim Robinson, who
has been doing his thing with I think should leave on Netflix
and became really big with that. And it's such a really, I don't
know, heartfelt comedy with this struggling to advertisers who
are kind of representing the city of Detroit and their only
jobs is basically to make ads for like the local, you know.
Like the trampoline store the mattresses.
Exactly, like really those like shitty local commercials you
will see on TV? Like they are the ones tasked
with making those kind of shitty commercials and they're the name
of the agency is Kremlin duvet advertising.
And so I think I recommended my fellow Nuance Co workers at some
point like, hey, you should really check out this show.
And then I never thought about it until you sent me this.
So. Well, I take your
recommendations very seriously. So I not only watched the show,
but you know, I kind of enjoyed it at times.
Wow, that says a lot. Did you have a favorite episode
or favorite thing? The Devereaux wigs is definitely
going to be my favorite. This is where they really
emphasize that the wigs from this store are not made from the
hair of dead people. And of course it turns out that
they are. And oh gosh, it's an incredible
ad that they create. And I think here we're going to
have to insert the Jingle if we can find it.
Devereaux wigs. We guarantee our wigs aren't
made from hair off dead bodies. Yeah, we have to Devaru.
Devaru, So I do have to say I felt quite guilty that you got
me two actual presents, one recommended by ChatGPT and one
that you chose yourself after, you know, small
misunderstanding. And yeah, you know, I only got
you a hypothetical gift or two hypothetical gifts.
So I thought, you know, I can deliver.
I can get you a real gift. So there we go.
And so you did. And you know that hypothetical
gifts were nice too, but it's always nice to actually
obviously get a real 1. And this is another example of
like, reciprocity is the way to go.
If you want to receive a gift, give a gift.
Yeah, so now it's your turn to get me a gift.
Oh, it works not OK. This is how it goes.
It just continues forever. Yeah, but it's great to have you
back as well. You were away on a little bit of
a trip and was not with me and Jared as we were kind of trying
to make sense of the past year and the future as well.
And so, yeah, just kind of quickly, what is your current
temperature for 2025? Any thoughts about the state of
things? 2025 I barely have gotten over
2024. I'm mostly terrified of 2025 in
the current political environment and lots of terrible
things happening there, to put it lightly.
And you know, like when it comes to behavioural science and
science in general, not looking very good when it comes to AI,
well, you know, you could argue good in the sense that
innovation will soar with no regulation.
Perhaps some constraints are actually helpful.
Well, there is the EU. What?
What are you talking about? The the US is everything.
So you kind of very quickly said something that I wanted to go
back to, which was something about science and that you're
like things looking dark for science.
What do you mean by that? Like what does that mean for
you? Why do you think that?
Oh, just the pause on all funding that's currently taking
place. Hopefully this gets overwritten.
In the US. In the US, yes.
Yeah, we're recording this end of January.
And yeah, there's been a lot of things happening policy wise in
the US, and I'm mostly receiving updates through you, my sister
and Jon Stewart. So that's all I know about what
happens in the US. All pretty good sources.
Yeah, that's good to know. In terms of where you're at, I
would say maybe I can kind of counterweight what you're
feeling with some level of optimism and excitement for 25.
I don't know exactly where I found that, but either I
currently feel quite excited for this year.
Oh, OK. And specifically what what about
it? No, like I'd love something to
latch onto. That would be great.
Give me anything. Anything.
No, but I, I do think there is a, you know, a possibility of
finding good in any situation I guess.
And I would say 2024 was a pretty dark year for me in some
ways. I think I was struggling with
the various things in terms of just like existentially last
year. And I used to think I have come
to some level of acceptance of what is and then finding some
form of excitement and motivation to work on what I can
control basically. So your optimism is coming from
we've hit rock bottom, it couldn't get any worse.
It must. It can only go up from here.
Is that what you're telling me? No, no.
But yeah, there's things like there I am excited about what
we're kind of working on with new ones.
I'm excited about the podcast, I'm excited about also while for
somehow weekly, we're taking a break a little bit, but we're
doing some really interesting things that we'll hopefully get
back to doing. We're also with Behavior Bites,
that initiative that I'm doing together with Frederick, which
is kind of this community for people specifically within
behavioral science and AI. Kind of hoping for those worlds
to meet outside of this podcast, of course.
And yeah, I do think like all of those various things has given
me some level of excitement. So I have a a couple of hot
takes as we sort of think about 2024 looking back, especially,
you know, at this intersection of AI and behavioral science and
then also like moving forward in 2025.
So we often talk about this sort of slathering of AI on
everything. It was just like this sudden
mindless frenzy where everyone was like.
We've got to. Put a chat bot on the thing
everywhere. I think there's sort of two
effects of this, kind of like the positive and the negative,
the positive and you know, you can debate whether it's actually
positive. But one is that users are
becoming more accustomed to chat bots.
And so there is this growing acceptability where you can use
AI and it's not seen as this weird thing that people aren't
trusting. I think like trust has really
increased overtime so that this just become a part of our world
and we're just accustomed to it. But on the flip side, with that
acceptance have sort of raised the standard.
We have now higher expectations of like what we expect out of a
chat bot. So if you go to any, you know,
sort of random customer service chat bot, you now expect it to
talk to you like ChatGPT does and to be able to interact with
it and get things done. And, and you want your agents,
your AI agents to be useful and productive and to be able to
accomplish things. And that I think is a challenge
because the technology is not quite there yet that you can
like really, really succeed at all of these tasks.
There are still a lot of dumb bots out there with the smart
bots. I think the trend is going to be
that people get even more frustrated with the dumb bots
and maybe are more impatient with them and are less willing
to engage with them as well because they think they have
more experiences of not succeeding with their goals
using them. So that's one prediction.
That's a really good take. Why thank you, I did call it a
hot take. Yeah, it is.
And to add to that, actually what's been on my mind literally
today is something interesting maybe to build on that is as
we're getting frustrated with bots and dumb bots, I think that
will also kind of lead to be more frustrated dumb humans and
faulty humans because yeah, So what I've seen in a project that
I'm working on, I wanted to actually share this with you
anyway, is that in this project is a digital health related
project where patients are communicating with for the most
part, actually humans, but it's text based communication.
And so they're in A and they're communicating via text, but for
the most part on the other side, other than maybe some initial on
boarding template kind of messages they receive, they
currently receive like a lot of messages from humans.
Which Who takes so long to type? Humans are so slow.
They're slow and they even also because of how maybe smart the
smart bulbs have become, then humans are more likely to be
appearing as bad bots or like, or if you give a credit to
humans, maybe they're seen as smart bots.
Because basically what we've seen in our kind of feedback
forms or like from users is that they're complaining about that
some of the people they're interacting with are bots.
They're actually interacting with humans.
And it's because they probably just assume that if someone is
like messaging me, even if it says it's a human, it's in 2025.
Like it's probably a short spot. And I think that's the challenge
of like, how do we navigate that?
Because then basically what becomes the absurd, potentially
challenging thing to have to do is either you have to like find
ways to strengthen that signal to like make it even more human
in a way where you're like put them on a call or you put them
in video mode or text, like a voice mess.
Something that even assists or even better formulated message
that has maybe some imperfections or something like
that really signals that as a human.
Or you might have to use AI to make a human message.
See more human. Yeah.
I mean, there are companies that are literally doing that.
I think we may talk to one of them this season.
Exactly. And that's kind of what we're
kind of experiencing. So that is kind of where we're
finding ourselves in 2025, which is kind of absurd.
It's kind of strange. It truly is.
But yeah, great take. And I would be remiss not to
give you the opportunity for another hot take, and that is
regarding deep seek. Obviously, Deep Seek, if anyone
hasn't followed the news, it's been this breaking news kind of
thing where there's been a team based in China that's developed
this open source model called Deep Seek.
And the news has centered around the fact that it performs quite
closely to open AI and for example, an tropics cloud
models. And it is about I think a 30th
of the price or like it's very, very cheap in comparison.
And again, it is open source, but the online version, the one
that's hosted online is a shot, but that is still very much like
with undertones of Chinese and censorship for example.
You can't access any information about Tiananmen Square, for
example. Or ask about Taiwan.
So yeah, I'm interested here. Elaine what What has been your
take so far? Yes, so definitely breaking
news. All of the headlines have
asserted that the US dominance in AI has been upended.
You know, NVIDIA stock has gone down just like down, down, down
to the ground. I think that is probably not a.
And yeah, I think one of the big concerns, of course, is if you
think that the risk of deregulation is a little bit
scary under a Trump administration, If you just
think about what that means, if China is leading the AI sector,
that probably with 0 regulation at all, or or at least not in
terms of, you know, safety and ensuring the quality of the
output. Also these this model of deep
seek is trained on the output of, you know, the other existing
models. So it doesn't seem, you know,
unless they can continue doing that doesn't seem sustainable.
Yeah, that's also funny meme around this where basically Open
AI are upset that it trained their model based on Open AI,
but obviously Open AI and all the other models have also been
training their models unlike others material.
I have not seen that meme, but yeah it's hard to feel any
sympathy for open AI. Honestly.
I can see this putting a lot of positive pressure to become more
efficient on on the more US based very inefficient models
out there. I also see it doing very
negative things for this crazy AI arms race where you know all
safety, privacy considerations, Everything is just like thrown
to the wayside in order to get there first, wherever there is.
Yeah, and it's very worse on them in terms of feeling not
really that there are adults at the wheel for some of these
things. Like it just feels that they're
somewhat aware of that they're in this arms race.
But at the same time they don't have enough kind of capability
to really get out of the game theory state of like competition
basically. And they're just kind of like
running towards something that they're not really sure of as
long as you. Keep running.
I think that sort of relates to one final thought that I have
about what this landscape is going to look like going
forward. And I think that there will be
maybe as a byproduct of this slathering of AI on everything
is I think there's going to be a bit of a reckoning.
And I think that part of it is the mindless nature of the
slathering, right, is that you can't just make it and they will
come. It's that you have to think
about the use case and how users will interact and, you know, how
it sort of fits into the workflow of the people who it's
designed to, well, in theory, designed to work with it.
And so I think that as adoption is not as high as expected and
as integration doesn't really happen immediately, instantly,
automatically, people are going to realize, oh, we have to
actually think about the system and how this product fits within
the system and how the people using it value it and what value
it does bring to them and so on. And so I think that's really
what behavioral science and or behavioral design can contribute
to AI. That's probably the best
business case that I can come up with, right.
For this intersection between behavioral design and AI is
like, think about the humans that are using your product.
That, I think is going to be really important in 2025.
Yeah, that's a good take. And I think that in that kind of
work, behavioral science can be so useful both to understand
that, but also obviously then through that understanding,
develop better versions of these models and better versions of
these interfaces to really support human good like human
goodness. Yeah.
Here, here, that's. Right.
We have rented now that that was not the plan.
We do have a podcast episode to introduce with a guest.
Yeah, we do. And let me introduce that
fantastic guest because while you're away, I had the pleasure
to speak with Laura de Molier. And she is a very interesting
practitioner who not only founded and LED the Babel
Science team in the UK government's Cabinet Office,
where she kind of honestly, I should mention that she were
involved in a very interesting time both during Brexit and
COVID there. So she was able to work at the
edge of kind of Babel science applies to policy in the most
tumultuous and interesting time in the UK.
You could argue she comes also from an interesting background.
She has a PhD in Cognitive Decision Sciences from
University College London and where I came across her was when
I saw a co-authored framework she had built called Incase,
which is a fantastic framework for understanding unintended
consequences and if we ever needed framework for
understanding unintended consequences maybe.
That is something we really need right now.
So that is nice. And then, yeah, I've had a real
pleasure to be involved with several kind of collaborations
with Laura and from Haverbytes. She was early advisor for that.
She also joined me. We did a Co presentation around
AI in Canada, early days. And yeah, she is fantastic
practitioner and she's doing great work.
And so in the episode, we talked about behavioral science in
policy, we talked about Laura's AI aha moment and like what got
her into using AI more and finding value with using AI, the
promises and perils of AI in behavioral science and so much
more, including synthetic users, importance of counterfactuals,
and of course, her most controversial opinion about AI
happens to Murgatroyd. I'm very happy to say welcome
Laura to the Behavioral Design Podcast.
Hi, thanks for having me. Yeah, I'm really excited to to
chat and there's so much I want to cover with you.
And in general, it's fun to catch up a bit as well.
But to set the scene, I guess I would love to rewind the clock a
little bit a few years because can you remind me when were you
at the Cabinet Office in the UKI?
Moved out of Cabinet Office in 2022 right?
Having been there for about 3 years, starting on Brexit, being
in the middle of COVID and then working across the UK government
on multiple policy areas. Yeah.
So that's already kind of a strange time 2022.
And I guess does it seem like what was the kind of the work
that you've been doing at that point?
Like could you explain a little bit more what you touched upon
there in terms of your kind of policy work?
Or so I think the application of behavioral science obviously is
touching on many different ways of doing it.
And I had developed with my team a quite specific way that was
very much determined by the kind of problems we had.
So imagine you're a behavioral scientist and what dictates your
work in some extent is waking up in the morning, looking at the
BBC app, seeing what went wrong and knowing, OK, probably
there'll be something coming to my desk this morning about that.
So whilst we did quite a few kind of longer term projects
that would span, you know, important big questions like we
looked at reducing violence against women and girls or
worked across the justice system.
A lot of it was also quite reactive in crisis mode.
And that massively determined the kind of a behavioral science
we developed to be efficient there, which was much less about
designing big trials during vast amount of research for a very
long time to understanding what decisions are currently being
made and how can we impact those decisions positively through the
lens of behavioral science. Yeah.
And that's really cool. That's really important work.
And I guess because what I want to kind of tie in from there,
it's kind of Ben knowing you a little bit and knowing a little
bit about your journey into also working with AI.
I realized that one thing I hadn't had a chance to ask you
before is basically I feel like we've all had some level of
color, like an AI aha moment of sort where, you know, it's one
thing to hear about AI. It's one thing to kind of like
see updates on like back in the, you know, the days it was like,
oh, you know, this model by Google has been able to to beat
the chess player or like this thing has been able to basically
understand and diagnose some form of like hard to understand
thing in in medicine. But it's a very different thing
to actually, how do you say experience AI in practice?
So for you, what was kind of your, call it entry into the
rabbit hole of AI, Like what was that aha moment if there was
one? Yeah, I definitely remember that
aha moment. You're yeah, you're very right
that probably everyone has one. A little bit like the question
like where were you at 911? I guess most people also answer
it wrong. But I think that for me, if my
memory serves me right, it was at some point in January or
February in 2023, I think, where obviously Church BT had been
doing the rounds a little bit and I had probably ignored some
of my nerdy friends who pointed me into the direction for some
reason. But I interacted with it for the
first time, I think in the middle of a night, where I took
the perspective of one of the policy makers that I was working
with to try and understand how they could, you know, improve
their reasoning through the use of AI informed behavioral
science. So my first interaction with
church EPT wasn't how can I improve my own reasoning and
decision making here, which I think what kicked out quite a
lot of very well versed behavioral scientists went, this
thing doesn't know more than me. It gets all the literature
wrong. I don't care.
And for me it was the counterfactual or the baseline
was someone who doesn't know very much about behavioral
science, but that could, you know, look at their policy
problems through a different lens.
How would they go about using this?
And this is where my mind was blown because I thought from the
perspective of someone that tries to influence decision
making processes, this could be absolutely incredible.
And I didn't sleep that night and I think had many restless
moments afterwards as well in terms of excitement about it.
Yeah, that's really cool to hear about it.
And I think, yeah, it's very reliable.
And then maybe we can then track your journey from from that
night till today. How is your kind of evolved
understanding or use of AI look like for the last two years or
so? I think actually probably a
little less developed on the professional level in some
senses that I think when I first looked at it and I thought
about, OK, what does it mean for this to improve, you know,
somehow exponentially? Like, what would this mean?
Where would we be in two years? I would probably imagine us now
in a slightly different space. I think I would have expected
tools like, you know, AI enhanced qualitative research
where, you know, we have someone actually giving us live
questions. I would have kind of thought
this would be more mainstream right now, and it isn't.
Nevertheless, I'm applying it a lot particularly, but actually
also with clients, where I say, OK, if you want to think about
the behavioural risks of your interventions, a large language
model is going to be your friend.
I think where I've really grown with it is actually the
application of it in my personal life where I've been finding it
most helpful as someone to correct my own assumption or
challenge my own assumptions about other people's behaviors.
So like I would look at, you know, upload WhatsApp
transcripts to get feedback on myself if I felt like I had a
conflict with someone or I think that aspect probably I for find
a lot and the professional has been growing steadily, but
possibly not as fast as I thought it might.
I'm not sure if you have a different experience with that
if you look back for the last two years or so.
Yeah, I know I, I, I would relate a lot to what you said.
And I think for, for me, it's probably a combination of those
as well. And you know, I think what you
spoke to a little bit before about one of the things that
maybe kind of kicks people off AI or like they start putting a
toe, they kind of like, oh, is this interesting?
And then they kind of abandon it.
There's often times because they engage with their model in a
kind of basic and generic way in some use case.
And then they get a generic and basic answer back for some query
that they prompt. And they feel like, OK, well, I
didn't need to tell me this obvious thing or like this.
I knew this already. As I said, and I think a lot of
the benefit from understanding AI and and making use of it is
really this kind of small, small interactions over and over and
over again in various use cases, both I think professionally, but
also personally. And they're not maybe always as
finally cut. That's where we maybe like to
imagine in terms of like, if we become better understanding how
to prompt and use some of these modest and communicate with them
even in a personal context. Of course, that skill to
promptly communicate and so on, for example, can still be useful
in other contexts too. And like I was reminded what
what you said in some ways about, you know, there's so many
things to as human being today to make sense of.
And I think what I've found myself this week doing actually
is for some of these very complicated geopolitical issues
of the time, it's very easy to want to have like a very strong
opinion about like it's either pro or against something.
And I tried to be someone who who kind of escapes that kind of
either or. And at the same time, I find it
like hard to develop my thinking.
And so I basically set up a way to test the study of, you know,
there's this kind of like idea of I think it's called something
like your own internal board members or something where you
have some people in your life or cables, famous people that you
kind of like hypothetically before it was kind of like
asking them questions like, OK, if your grandma was still alive,
like what would she have said if you were doing this?
Or if Martin Luther King, like it could be technically anyone.
And for long time, that was more like a hypothetical thing that
you were doing. And now I was like doing that
but with real people. So I had like 2 journalists that
I pitted against each other to write like long form the New
York Times kind of articles where they kind of like tried to
steal, manage other arguments and so build the strongest
version of those arguments, but kind of going back and forth on
on this topic. And then I would also interject
things that I was thinking about and kind of things that I was
exploring. And, and this helped me kind of
like following this interchange while contributing to it was
really, really beneficial for me, I think, to make my thinking
more nuanced. And so yeah, I, I feel very
similar to you. Yeah.
Yeah, that, no, that sounds amazing.
And I think it's also worth being aware of some of the
pitfalls, particularly with some of the language models that have
a memory of you as a person, because obviously these models
have been trained to kind of please whoever is, it's fun.
And so I know some person, you know, calling them like
something like Dobby the the House House because it's always
so polite and so kind. I'm going to steal that.
I think it's so true. Yeah, it's cute, isn't it?
But if I want to get some view on my views, if I want to get
some feedback that's more or less objective on say my own
behavior and it could even be something like an exchange on an
e-mail. I have now started to you
pretend I'm the other person So and to avoid that, you know, the
LLM just pleases me and tells me how great and how smart and
fantastic I am. I pretend to be the other person
and I go into, you know, clean slates, different LLMS that I
might have not interacted with before or switch off the memory
and touch BT and ask about my behavior from their perspective
and that I've given me quite a balanced view and also the
satisfaction. If then the LLM actually agrees
with me despite talking to the other person, I get a double.
Yeah, I think I was really right here.
The LLM doesn't even tell sugar coaters to the person I've
argued with. Not that I have lots of
arguments, by the way. It sounds a bit wrong maybe, but
I definitely, you know, sometimes disagreements or like
ways of interactions interacting with each other where I want to
take the other perspective. Yeah.
I think it's really inspiring actually to being able to kind
of take a pause on some of these things and actually taking a
step back and reflecting on that.
Because I think with any kind of yeah, communication, it can
oftentimes be interesting to use like take step back and be like,
well, am I really the one who's on the right side?
Yeah. Or what am I missing?
Yeah. Exactly.
And I think actually the one key for me is an an openness to
understanding in which ways a large language model can
actually support your current thinking and be open to where
that thinking might, you know, not not necessarily be flawed,
but is capable to be helped. And it's quite a similar
approach I actually took when LLMS came first out.
And I was thinking about applying behavioral science and
government. What are current barriers to
more behavioral ways of thinking about the world?
And I kind of split them up. And it's like a proper list and
say, you know, like, we are currently lacking cognitive
empathy, so the ability to take someone else's perspective all
the time. We are, you know, struggling to
switch lenses, to view the problem through different lenses
from different academic fields. We're struggling to imagine what
the future should actually look like.
We're very much constrained in our own imaginability.
And sometimes we're lacking scientific reasoning abilities.
We might be missing trade-offs between departments, you know,
like one department puts out a policy.
We're not really Privy all the time what other aims
behaviorally we're currently impacting through that.
And there's quite a different, few different, I think aspects
where if we're being really systematic, we can enhance our
decision making and quite nice ways to draw on the strength of
an LLM without getting too hung up on what I currently can't do.
We can do better as some sort of self-serving, you know, by, as
I'd say we've I've seen crop up quite a few times.
Yeah. And this brings to something I
want to kind of like talk to you about in terms of we, we
recently kind of had a little bit of a relaunch of Baby Bites
and Baby Bites was an initiative that you were kind of very
helpful in, in getting off the ground.
And so we had this really fun, I guess, November meet up for for
Baby Bites and me and Frederick, who who are now kind of
Frederick is the new person kind of managing this community for
people interested in behavioral science and AI.
And we put together kind of like some form of way to think about
what behavioral science and AI can overlap around and, and how
as behavioral sciences, we can kind of like see potentially
where we can add our value. And we came up with some form of
imperfect, but maybe still useful kind of like quadrant
because I think as behavioral scientists, we have this, I
think interesting two sides to looking at AI.
We have both the kind of the thing that everyone has in terms
of a natural use case, in terms of, OK, could we use AI?
We, the two examples we had was either for like tailoring our
interventions, so make our interventions better and more
personalized and, and so on through various type of AI.
Or could we, for something like automate the things we do and,
you know, make tasks that took a lot of time, a little bit less
time. And maybe things that we did
100% before we maybe do 20% of them outsource some of it to, to
AI. Like those two things in terms
of tailoring and outsourcing or automating is something that
most people can also fit into their equation.
If they're accountant, they would probably think about it
similarly. Maybe they would tailor it the
same way, but but still, But then I think what's really
interesting is that as payroll scientists, we have this other
lens of understanding. And so we explore this both in
terms of like we can help make sense of and understand what is
the impact of AI in the world. Like you mentioned the bias
around, for example, like how people can think about, you
know, interacting with AI, but also like in general algorithmic
impacts on how people view each other and view AI and some of
that stuff. That's love interesting research
on that, of course. And then the second level is
basically how we can understand the machine.
So how we from behavioral science perspective can also
better make sense of these models and understand these
models. And you already started talking
about that as well, which I think was really interesting in
terms of, you know, what you were doing when it came to
understanding how the model operates is to kind of based on
that understanding, removing yourself from from being the
first person that the model was presumably talking to.
And you kind of remove that bias from model and in earlier models
exempt. So that was to kind of like
knowing that these models are there to please you and they're
also kind of trained on human data.
If you threaten them and said, Hey, I have taken your family
hostage, it would do a better job, for example.
And that's not something like you would expect, but obviously
these models are trained on human behavior.
So there's a layer of of human, human aspects.
There's I'm used throwing this at you and seeing what are your
thoughts about kind of thinking about this from this
perspective. Yeah, no, absolutely.
And I actually had some thoughts at the beginning when, you know,
first came out. And I was thinking, I think as a
behavioral scientist, I get more use out of it than my other
science friends, Possibly because there is some implicit
knowledge about behaviors of people, of a specific subset of
people on the Internet. Let's be aware that we're having
the same biases we're used to having.
I guess it's common full circle. Behavioral science, again,
happening in this weird space. Yeah.
Where actually, you know, there are some explanations of
behaviors that are almost baked into like the data because so
much of the data is about, you know, how humans describe and
view the world. But I find that, but also then
that's also one that's hard to handle in reality because we
don't really understand these systems right now, right?
We could be saying, OK, let's do some like proper behavioral
research on them and let's treat them as subjects and let's see
what's happening, which is obviously different from saying
let's do some behavior search with an eye where it pretends to
be lots of different people. And I think that, you know,
you've probably come across some of this data where we can now
simulate different recipients from different demographic
backgrounds to essentially in some cases replicate existing
research that has been done with real humans, which I think is
exciting, particularly for educators that we can't really
communicate with. But what you're just talking
about is also actually for us to try and understand how the
behaviour of of an AI, if we can call it that.
And I'd be very interested to see more behavioral scientists
being involved in this kind of research.
The question is, obviously, does it actually reflect our methods?
Can we actually support it? Mahanch is yes, but I am not
quite sure. I'm not quite sure yet, simply
because we have little understanding of this amount of
complexity and it it might not hold, but it should certainly
try. Yeah.
In terms of the moving to like that instead of the
understanding side of it, really focusing on the applying side of
it. Like what do you see as kind of
from your point of view, the promise and perils of behavioral
scientists reducing more AI in their day-to-day work?
Like where do you see the biggest opportunities and maybe
the risks as well? Yeah.
So I think I see more opportunities and non behavioral
scientists applying AI to think more behaviorally.
And I'm thinking about kind of there are two different ways in
which I'd look at it for public policy where the end goal might
be to make better assumptions about behaviors as part of the
policy making process, right? We can in one way, we can
examine some of the current existing decision making
processes and see very much with the list I previously gave you
where LLMS could enhance existing outcomes.
You know, we can actually have better perspective taking.
We can have an LLM surface trade-offs between different
departments. We can help with all the
ideation around what potential barriers might be way in a way
over behavior. Let's say you'd you'd open up
BBC in the morning and people are stockpiling toilet paper
during COVID instead of relying on behavioral scientists like us
to be present in the room to say it could be due to AB and C
factors. You could have an LLM generate
potential categories of things. And people are very quick to
raise their hand and say, Laura, but what if it's wrong?
What if the LLM tells us something that's really wrong?
And of course that's a possibility, but one of the key
things we're not talking about about enough I think is what the
counterfactual is here. So in other words, we're really
comparing the decision making processes that rely on a
generating of 20 different barriers.
People might have not thought about it before versus them not
thinking about it at all. My current guess and I think
that's what I got so excited about in early 2023 is that the
basic and surfacing more potential barriers and drivers
of behaviors, surfacing more potential behavioral risks of
your interventions. Broadening out your thinking
will have a positive impact because it surfaces much more
things that I have seen from the average policy maker who doesn't
have the headspace he can ask himself this question.
So that's kind of one of my first punches here is that we
can look at the current decision making processes of non experts
and there is so much that could be improved with it.
I think that's a really great point.
And I think that's one of the things that is, I would say, I
don't know if we ever would like to put bias on on labels and
stuff, but I'm very tempted to to find some name for this in
terms of like used, there are in general inability to really
understand the counterfactual, I think has become extremely
prescient in terms of where they are.
Where I think initially the discussion was often times
around, you know, autonomous vehicles where like what is good
enough? Like what is a good enough
autonomous vehicle? Is it when it's better than a
human? Is it when it's twice as good as
a human? Is it when it's perfect and
never does a mistake in any capacity?
And so the counterfactual there has been really hard for people
because like as long as they see a self driving car make a
mistake, then that's the end of the world basically.
Then that's often As for people that's really hard to then
compared to like, well, what would a human do here?
Like how many mistakes are human traffic like people driving
everyday doing? And what you speak to is then
kind of, I think a really recent extension of that with these
large language models we have today where, yeah, I think not
really understanding the counterfactual is what makes it
really tricky. Because I think we had a
discussion on this before on the podcast as well, where, yeah,
there's obviously like a lot of great scenarios where like
ideally we would do so many things in a certain way, but
it's very rarely that it always comes together in in the perfect
scenario. And sometimes good enough with
AI is much, much better than the counterfactual of not doing
something at all. Yeah, it's mainly experts making
that call as well, which I find particularly irritating.
It's mainly, you know, bunch of behavioral scientists going, but
it's not as good as me. How dare you use the LNM and not
me, which I think is just, you know, let's all strive, strive
particularly in something like public policy where the world is
like, or I strongly believe it is the world would be improved
if more behavioral science would flow into decision making.
So let's aim for that and let's up our own game.
I think this is what it has to be about.
But also to say that with regards to this risk, like this
risk asymmetry, almost different kinds of errors made by humans
versus an AI, I think in the way that I am applying large
language models to non exports, it's very much in the divergent
thinking. So opening up the problem space
to say what could be driving this behaviors and it will give
you probably 10 things you haven't thought about.
That's different from convergent thinking where we're asking for
what's the most likely thing to be happening or you know, what
intervention is going to be most successful and it then probably
going to be failing. So I think it's also about
asking questions where we kind of reduce the amount of errors
that could be made. You know, like off the 20
barriers, 5 will be possibly complete rubbish, OK, human
thinking kicks in and possibly, you know, critically assesses
that or it runs onto the risk of removing a barrier that wasn't
there, which I think is quite low, if that makes sense.
Yeah. And like ideally, if we think
about this in being used in the kind of example you mentioned,
it would ideally be set up as some form of vertical AI
solution that is a little bit set up to be better for that use
case in terms of it wouldn't be your off the shelf shaft GPT
ideally, right? Like it would be something that
would be trained and set up in a way that it would kind of maybe
by virtue of its training data and also how it's been kind of
maybe either through prompting or or very others fine tuning to
make it so that all the 20 barriers that it lists, it's not
as likely to ignore edge cases and underserved communities as
maybe a normal language model would be.
Yes. So there's obviously ways to
improve from that side as well. Like you can make something
better there. Absolutely.
But to say that sometimes the stakeholders I'm working with
might be in countries where there's very little access or
like TGBT hasn't, you know, actually even taken hold yet in
as much. And I also think we want to
lower barriers to entry for some of the basic models to not say
you can only use this if you have a very specific thing that
some very smart person developed with vast amount of knowledge
and then sold on for a price. I think particularly when it
comes to running interventions with marginalized communities
where otherwise there might not be many considerations at all
around some of the barriers or some, you know, checking of the
assumptions or a particularly important surfacing, how could
this go wrong? How could this increase
inequalities? I would again, thinking about
the counterfactual, I, I think I'd advocate for using even the
basic model there over nothing. But of course, I absolutely
agree with you. I think that will probably,
hopefully over the next few years, be a lot sharper than it
is right now with regards to these special specialist and
LMS. Yeah.
And something we're kind of been somewhat touching upon, but
maybe not explicitly is that you have synthetic users, for
example, or like some synthetic participants.
Like we've, I think we've talked about it in some ways around
kind of getting, in this case, someone who is maybe in policy
to be able to get some perspectives on something, even
though they might not actually speak to those real people, but
from some perspectives of potentially interesting target
groups. And I think that's a really
interesting discussion point around synthetic users because
my immediate reaction basically was like, that is bullshit.
Like how can we how we get any value from something like
synthetically generated and then drains the kind of the real
human experience from, from our understanding.
And also if that's trained on again, some form of very weird
data sources, we also then kind of reinforce a very weird view
of maybe the context or the thing we're trying to change.
But then I think the main thing that started to shift my
thinking was thinking about the counterfactual in many ways.
And yeah, getting a little bit deeper and exploring how things
that are alternatives to synthetic users that are very
well adopted today, how they would perform.
So for example, I would say something like a focus group is
still very, very widely popular, used to understand user behavior
or like user research. But if I would try to make some
better right now, I would say synthetic users are probably
more useful to get useful data on some form of the context than
a focus group, for example. Yeah, because of the the
limitations of what happens when you put people in the room and
and get them to be prompted to talk about certain things.
It's not usually what comes out of there is not the real world
experience of those people. It's kind of what's been
prompted in that room to get out of them.
So yeah, setting that thing with synthetic users, what do you
think about that? Yeah.
So I think the most excited I've got about synthetic users is
again, if you're thinking about this fast-paced, high pressure
crisis type environment that governments often find
themselves in, where they are incredibly reactive to events,
world events, local events and have to quickly put out messages
that's, you know, inform people of the situation, know what they
should be doing, give an update and so on.
And I don't know about you in Sweden.
I've definitely here in Germany. I don't want to talk bad about
anyone in the UK now I'm joking. I I have seen quite a few of
those kind of government responses to crises that are not
that great. You know, you might
misunderstand some messages. They aren't clearly expressed.
They might actually just not so if your information needs right
now because you're a part of a specific group and you have
specific questions. And for these kind of cases,
that's where I think actually having almost a user test, but a
quick one because you can't, you often don't have time in crisis
to actually do anything else, even if it was marginally
better. That's an empirical question,
right? What gives us better insights, a
focus group versus large base of synthetic users.
Agreeing with you on some of the downfalls of the focus groups,
but also obviously also some downforce with synthetic data,
particularly once it starts feeding itself.
But for these kind of fast decision making processes, I
think this could be actually quite a game changer and
providing clarity to different segments of the population that
you'd usually otherwise not even possibly think about.
So I think that's my kind of like very excited use case I
see. And then I think I'm possibly
not technically advanced to kind of really make a judgement on
whether or not my fantasy is about interacting quote, UN
quote with synthetic users that are very present in the
Internet, in the dark web, but less present in our research
laboratory and definitely not present in our focus groups.
Whether or not there is actually something there that I got kind
of a few years ago. You know, we have so many
conflicts where there are groups that do not talk to the quote UN
quote establishment. And, you know, universities and
research institutions are are seeing a part of that.
And it takes us often a long time if we're looking at certain
misinformation actors in crisis right now, for instance, as
researchers and projects I'm involved in, it's difficult and
dangerous to get involved with these actors.
So are there cases here where we can actually interact with them
with quote, UN quote in different ways through the
choices they left on the Internet, which again, I'm quite
excited about. Yeah, I do think there is.
It's a big one in some ways to unpack in terms of how we, how
we think about that. But I do think that in terms of,
again, we'll go back to kind of limitations of are human.
So I think like what we're good at and what we're bad at in
general. The really tricky thing is that
we're we're good at understanding people that are
similar to ourselves. And we're often times very, very
bad at making some representation of other people's
beliefs that are dissimilar to to ourselves.
And like, I think what you describe is kind of like taking
that to really interesting extreme in some ways where it
becomes even more absurd when you when there's some form of,
yeah, really extremist or type of actors that, yeah, if you're
a very kind of pro democracy liberal, it it's probably really
hard to, to put yourself truly in the perspective of that.
Even if you just had A and start to try to think about it, that's
probably not going to work. And at the same time as well, no
one from that side want to talk to you as well.
So if you wanted to reach out to them and try to have a dialogue
of sort of like, that's probably not going to happen as well.
Yeah. So, yeah, I do think it's it is
interesting, yeah. Yeah, and I often, I mean, if
you were to look at my YouTube feed, in fairness, it looks like
I am an absolute right wing, right wing.
Concluding, a lot of the content I seek out for this purpose, to
be a better behavioral scientist, I suppose, is to try
and expose myself to narratives that I find very puzzling to try
and try and understand the reasoning.
Very similar to your example of what you've done with the LM,
where you, you know, put a different journalistic entrance
around complex topics. I often then purposely also not
just seek out topics I don't understand, but seek out topics
where I just cannot. I cannot possibly understand
someone else's perspective and train myself almost to take
those perspectives, which I think some of my friends and
family find quite irritating because they've been getting
very emotionally upset about public world event.
And I end up saying, Oh well, well, from their perspective,
maybe it looks like I love. That's a bit as I see as well,
of course, because it kind of explains away everything and
takes away most judgments. But I think for behavioral
science, expertise is actually probably quite a useful skill to
be striving for. Yeah, well, I can you say for
most, like on a personal level, I can, I can relate a lot to
that. I think there is, it's
interesting in terms like now there's a big wave of people
moving to to blue sky from from Twitter or X.
And I think the first person who who had like a million more
followers were the AOC, the Alexander Oktas, Cortez and kind
of speaking to the kind of like the liberal side that kind of
flows into the blue sky. And then I was left on on extra
Twitter. It's it's maybe then the other
side in some ways. And for me, like I was never on
Twitter for getting a balanced intellectual perspective that
was very thoughtful. I, I, I've always thought it was
interesting to kind of both follow maybe obviously the
things that I do for my work in terms of some science research
updates or AI updates, but also understanding like what are the
people that I strongly disagree with?
Like how are they commuting getting their points of view?
What are the followers of those people responding to or like,
and how is that happening? Like I, I find that deeply
fascinating as well. And you'll probably find some
similar friend on my on my YouTube feed as well.
Like I, I do find it a little bit, yeah, limiting to just like
I watch someone like Daily Show. I love The Daily Show with Jon
Stewart. It's really fun, really great.
But obviously like they're, they're used to making fun of
the other side and it's just trying to satirize and just make
a straw man in all capacity. And so for that reason, I find
it as interesting to kind of look at the the counter to what
that is, what it does, the Babylon Bee or whatever.
And you speak like, OK, what did, what did they find funny?
Like what are they actually trying to pick up on?
And I think it's really interesting to to form that
perspective. So yeah, I'm with you at least,
yeah. Yeah.
And to draw the parallels of back to behavioral science, I
think apart from helping airlines to try and get some of
this perspective in our current thinking and challenge our own
assumptions as behavioral scientists, it's also about
providing us with assessment of how our interventions could be
perceived by different groups and where it could go wrong.
So I think I'm really a big fan of LMS for things like red
teaming, challenging your exact existing thinking.
I don't know if you have these discussions of like persuade me
of the opposite from myself, you know, to try and cover potential
aspects. That's where I see it as
helpful. But again, I think a lot of it
actually comes down to people's own preferences as well and how
much they like these kind of challenges and how much they
have headspace. My God, headspace.
So important, right? So like, just go around and
circle, have a big old discussion with an LLM about a
topic you might disagree with. That is obviously a big
privilege to be able to, you know, find yourself in a space
where you actually have some of the time to do that kind of
partially self partially professional improvement
activities. Yeah.
And just underline that, I think that is probably what people
often times get wrong maybe is that when we're talking about
this in this context is that we're kind of, yeah, looking to
some more AI model to think on our behalf.
And I think that's the danger. Like it's always dangerous when
you just go there to just be like, hey, I don't know anything
about this, Just tell me what I should think about it.
And honestly, to be fair, though, I don't know how harmful
AI is. Humans are more harmful in that
degree. Humans will probably say
something very, very extreme on one side or the other, like,
yeah, this is good or this is terrible.
Yeah, large sound usually pretty balanced.
They're like, yeah, it depends. It can be like both of this.
And this used to say that. I don't think that probably
that's too bad itself. But I think what we're talking
about here is like when it can be really useful is to when you
have opinions, when you have thoughts, when you have kind of
perspectives to challenge them and kind of build on them and
and get them to become more. Yeah, test that basically and
see in your blind spots. Yeah, What I was just thinking
about as well is in our kind of frameworks of influences on
behaviors. We obviously have things like
knowledge and motivations and whatever else it is in our
framework of choice. But I reckon what will at some
point become a big influence is how large language models
interact with a person about that topic.
If they do choose to, you know, talk to talk to an LLM about it
or to whichever means they are actually influenced through it.
So we should probably also stay on top and you know, the
question of like, how does the LLM behave?
This is also for us as behavioral science to figure out
how to do LLMS actually impact people's thinking.
I can certainly say my thinking processes have been quite
dramatically influenced. I would I would say positively,
but you know, like actually using an LLM almost as a
therapist to talk through personal life situations to talk
through difficult decisions. Again, I often try to have a
blank slate one to avoid this kind of yes, yes Sir, master you
are amazing type interactions or talent explicitly.
I got peace. Don't be.
Don't be so. What's a better word for us?
Licking. Don't be.
Like, I'm not here to be told I'm great.
I'm here to actually get some challenge and some support.
But there are also some moments where all you need is someone to
listen to you and to give you to mirror back as to what you've
said to kind of maybe structure some of your thinking.
And I find it also very, very helpful that, but of course,
this counts as an interaction that if it was between a human
and a human, we would take as an influence on behavior.
So I think there's more that we can probably do as behavioral
scientists and kind of understanding how it takes
behaviors. And I mean, we've seen research
come out that it can persuade people and so on, but in a more
kind of less experimental setting.
That'll be cool to understand. Yeah, Yeah, I agree.
And I guess to in some ways shake your thoughts on various
things that we maybe haven't had a chance to talk about as much
or not segue into our quick fire round of the season called to AI
or not to AI. And so basically, I've written
down a bunch of different things that AI potentially could do.
But I guess my question will be basically to use like should it
do this? This is a good use case for AI.
So yeah. And let's start the quick fire
round with to AI or Not to AI? Deploy AI powered urinals to
provide real time gamified cleanliness feedback.
Yeah, why not? I was trying to look for like,
what is a classic nudge that could be AI and then that that
was the first one. OK, to AI or not to AI?
Calculate individual tax rates based on lifestyle and spending
habits. Oh, that's, but that's a policy
question. That's not an AI or not an AI
question. Well.
It's to use, it's to use it in the policy context.
Yeah. In that context of like having
more personalized taxes based on some form of in the same as you,
maybe you would do some form of actuarial risk assessment for
insurance purposes, like depending on how you spend your
money, like how you should pay taxes.
Interesting. I don't think I can give a clear
yes, no because it would very much depend on the design of it.
Sorry I'm falling back into policy speak, but I that's.
That's. Good, because I like, I think I
will be very open to exploring the idea and to assessing the
associated risks. Yeah.
So I wouldn't. I wouldn't dismiss it entirely,
but annoying answer. But it depends, no?
That's that's that's the payroll science answer.
OK, Evaluate and assign household chores to family
members. Yeah, I already say yes, yes,
yeah, yeah, for sure. I think probably depending on
people's perceptions of their own levels of emotional and
other labor in a family context. No, I, I think this would
actually make sense. And it would also make sense to
try and get it to play towards someone's strengths and, you
know, to pick up like help with scheduling around some of the
tasks and so on. I think there's possibly quite a
lot that could be gained around, you know, improving common
issues in families, yeah. OK, now we're getting into
another policy, one managing disaster response and relief
funds. So, allocation of emergency
resources in disasters. I think I could see how that
could be quite a good one, especially if we're thinking
that we're currently restrained in that through like personal
views, constrained data of hierarchical decision making and
power structures. This could actually lead to some
more fairness if done well, which I'd generally be in favor
of. Nice.
OK, decide when you're ready for promotion based on performance
data. Again, I wouldn't dismiss that
as a thing, because I think performance and promotion
decisions are also so often made on vibes, and it might be
interesting to take a think about what would happen if vibes
aren't part of the equation anymore.
Does it actually become fairer or does the system collapse
because people stop being able to work with each other?
So. Again, very often to
experimentation around it and what would happen as an
unintended side effect. OK, provide hyper personalized
voting reminders tailored to past behavior and preferences.
Yeah, nothing to lose there. Remind them all they want.
It seems like a no brainer to use more personalized
initiatives to get people to vote, right?
Yeah. I mean, I'm sure we can come out
with 10 risks of this as well, of course.
But I think overall, I would think that if, especially if
people are able to opt into it, that'd be great.
Final one, create tax return, not just optimize deductions and
encourage social beneficial contributions.
I'm not sure about that one because who decides what's
socially beneficial? I think that's possibly like the
but I'd get hung up, hung up on on this one and I might come
back to the urinal. Do we mean that there will be a
camera installed? I was just thinking I was way
too quick to say yes to something that wouldn't impact
me. But some people might say, hey,
Laura, that big camera, are we assuming the camera?
In which case I might say no for the protection of men.
Yeah, yeah, no, maybe it would be one of those cameras that is
actually not picking up on exact film footage, but more something
more vaguer or I don't know. Yeah, it's it's, I actually
didn't think about that far as well.
That's a good. Point You're the design guru.
You can make something, make something happen some.
I'll ask Larry David, he has some ideas as well about urinal
improvements and stuff like this.
OK, that was really good, well done.
It was very, I think I selected some hard ones for you, I
realized in hindsight, but kudos for going through them and to
not make any easier. The final question for you is
what do you feel like is your most controversial opinion about
AI? My most controversial opinion, I
think my this opinion was very controversial around 3-4 years
ago, where if you happen to be stuck in a pub with me, you'd
hear me talk about my big fantasy that when I'm old I
don't want to be lonely. And what I want is an AI
companion, someone that I would give them access to all my data,
all my, you know, WhatsApp, social media data over the last.
I don't know when am I going to die in the last 60 years and I'm
going to have the perfect person to interact with someone that's
not just going to laugh about my jokes, but going to tell me
jokes that I find funny. Someone that can remind me of
things that bring me joy. Ideally want some sort of bodily
feedback so it optimizes for my positive emotions.
And I think that was possibly quite controversial back then,
that idea that I'm striving for the perfect AI companion when
I'm old. I think nowadays with people
interacting with the GBT actually seeing some of the
potential benefits that can envision that more.
Although when I say it the way I say it, people still have a bit
of reaction to it. So it's not terribly
controversial, but it's possibly being us round circle in terms
of my my own entry to AI, which was probably actually thinking
about the loneliness question in a in old age.
Yeah, no, I love that. That's a great one.
And honestly, I love this conversation.
It was so much fun. I feel like we could have, I've
talked for many hours about many of these topics, but I I'm so
happy we had a chance to bring onto the podcast to talk to you
and bring your perspective and insights.
Thank you so much for having me. Yeah, thank you.
And thanks also follow the one for work you're doing and thanks
for helping with setting up and bringing paper bytes to life as
well. So yeah, a lot of thank yous.
Yeah, I know. Thank you for including me and
for having me today. Such a good chat.
And that's a wrap. You've been listening to the
behavioral design podcast brought to you by Habit Weekly
and Nuance Behavior. Sam and Aline tell me this
season is packed with incredible insights about behavioral design
and AI, so be sure to subscribe and share the podcast with your
friends. Though you might want to keep it
away from your enemies. In case you haven't noticed, I'm
an AI voice. Yep, pretty crazy.
Quite the improvement since last season's.
AI outro, don't you think? If you'd like to collaborate
with us at Nuance Behavior, where we use behavioral design
to craft digital products with Nuance, e-mail us at
hello@nuancebehavior.com or book a call directly on our website,
nuancebehavior.com. A special thanks to the amazing
Dave Pizarro for our show music and to Mei Chen Yap and April
English for their help in producing and publishing this
episode. Thanks again for tuning in.
We'll be back soon with another exciting conversation where
behavioral design and AI Intersect happens to.
Murgatroyd. Oh.
Oh Devero wigs, we guarantee our
wigs aren't made from hair off dead bodies.
I don't like it. I don't like it at all.
You don't. Why not?
Why bring up the thing about? Dead body hair.
Our wigs aren't made with dead body hair.
Yeah, that's the whole point. So why bring it up?
Advertising one O 1. Answer the question before they
ask it. Who's asking that question,
Rhonda? The main reason people don't buy
wigs is because they're afraid they're made from dead people
here.