Hello and welcome to the Behavioral Design Podcast.
This season we're diving into the intersection of behavioral
science and AI. We want to make sense of the
state of AI, from understanding how humans interact with
intelligent systems to using AI to do behavioral design itself.
I'm Aline Holsworth, a health tech advisor specializing in AI
and product design. Over the past 15 years, I've
been crafting human centered products with behavioral science
at the core. At Apple, I LED Behavioral
Science for Health AI, designing and launching AI powered
features to help users reach their health goals.
And I'm Samuel Sultzer, your second Co host.
I'm a behavioral strategist specializing in hybrid formation
and designing products that drive long term baby change.
I work with leading tech organizations integrating AI to
scale behavioral design for good.
And I'm also the founder of Baby Bites, a dedicated community on
behavioral science and AI. Quick word on Nuance Behavior
where we help organizations build impactful digital products
using behavioral design. We only take on a few clients at
a time to ensure the highest level of quality for our
tailored evidence based solutions.
If you'd like to become one of our special projects, e-mail us
at hello@nuancebehavior.com or we could call directly on our
website, nuancebehavior.com. So Eileen, I want to ask you
something. How often do you speak with an
AI on a daily basis? Like how often are you
conversing with an AI? Conversing as in with my voice
or not at all? I have not done that yet
actually. That's a lie.
That's so funny all the time. Oh my gosh.
I'm thinking of these really innovative new versions of like
ChatGPT voice. I have Siri set a timer for my
coffee every single morning. I ask Alexa to play music
constantly. Yeah, I know.
My my digital voice assistants are a regular part of my life.
I don't know how I just got that.
Yeah, that's interesting. You know, like I assumed that
was probably true. And so I was kind of surprised
when you also said like, no, never.
OK, well that that makes a lot of sense.
So you've then got quite used to using, as you said, like serial
4 timers and so on Alexa, but you haven't used ChatGPT in
terms of the voice mode. I have not know you know I'm
still on the free plan. Is it cheapo?
So no, I have not gotten access to that yet.
OK. Because in that case, that
actually is in a way perfect. Because I have been using the
various voice modes of Chachi, BT and other solutions quite a
bit over the last few years. And one anticipated thing has
been that Open AI was going to launch their Scarlett Johansson
version of their, you know, basically advanced voice mode,
they call it, and it was going to come out in the spring or the
summer. Then Scarlett Johansson sued
them because it was actually using her voice, basically.
Without her consent? What?
Why did? Without her consent, very super
sneaky and super arrogant to, as you say, like without her
consent to assume that they could do it.
But they initially asked her and she said no.
And then they decided let's just do it anyway.
That was the decision. What?
You know, this shows Silicon Valley or something like this.
Parody of what Silicon Valley people do and use the.
Hubris. Hubris.
Yeah, Yeah, it is unbelievable. But anyway, as a mere call it
innocent consumer in this situation, I was still looking
forward to using a more advanced version of this.
And especially as these kind of videos was coming out of people
testing this advanced mode and noticing that it can do so many
cool things with it or disturbing things as well.
I was really excited to get my hands on it.
And this would date episode a little bit would when I say
this, but a few weeks ago or a week ago or so, they said, hey,
we now is going to give this access to everyone.
Everyone's going to get it. And I was refreshing, you know,
on a daily basis, like, OK, when I'm going to get it and so on.
Finally, the day comes when they realize it's not coming to
Europe because of EU law, because of EU law.
They basically it's assumed because this is trying to
register a bit of your emotions. AI in EU can't currently get
access to users emotions as part of their kind of engagement with
the user. And so for that reason this new
version for better or worse can't be rolled out currently as
it is in EU, but it is available in in America.
So I, I am curious if you're up for it, if maybe we can find a
way to get you to test this, try it out and report back to me in
a few episodes and see like, you know, give your take.
Like, is it worth the hype? Was it disturbing?
Was it fun? Was it, you know, not useful at
all? Would you be up for doing some
form of version of this? Yeah, sure.
I, I can give it some different emotions and see if it
accurately, I don't know, like catalogs them.
So related I think is, is there's, there's been a lot of
hype around Google's notebook LM, which of course, probably
everyone else in the world knows by now can create a podcast,
basically a dialogue between two people discussing the, you know,
the topic that you ask it to summarize.
And I was what I I don't usually fear for my job as a result of
AI, but now as a podcast host, I do wonder, like, are we becoming
obsolete? What do you think?
Well, I think actually this is something that we're going to
explore later also in the season probably we're going to actually
put this to tests that is currently in our internal
schedule is to have an episode that tests this a little bit.
And yeah, we'll see. I guess the listener will have
to decide at some point and be the judge.
You know, to be fair, I do think it's very impressive in terms of
like how they've been able to create this very, you know,
again, emotive and engaged sounding hosts that I think the
most viral clip is kind of two AIS kind of coming to the
realization that they're not real.
And they're grasping with the fact that like, wait, like what
I'm calling my wife, you know, is she actually there or like,
you know, and you hear, have them have that conversation in a
yeah. But I as a human would never
have those doubts, so that's how you know the difference.
Well, that separates you from me.
I am still, you know, never sure if I'm in a simulation.
Or not. That's because you are, as in a
simulation. OK, I'm in your simulation,
little did you know. This is the episode where I talk
to an AI. Yeah, yeah, that would have been
the perfect Plot West. But no, what are we actually
doing this episode? This episode we're talking to
Susan Murphy, and I'm a big fan of Susan Murphy.
I've been a big fan of Susan Murphy for a long time.
Actually, in 2018, I went to a webinar that she gave on her
micro randomized trials. So she talked all about how she
was using them in her famous heart steps study.
We'll include the heart Steps study in the show notes because
it is just so cool and it's gone through a lot of iterations over
time, yeah. A pre COVID webinar.
Wow. Yeah, they did exist before
COVID, believe it or not. But one, you know, one of the
things we'll talk about with Susan and what I just think is
so cool is she really developed the micro randomized trial.
And you know, maybe it's worth just getting into a little bit
about what the difference is between a micro randomized trial
and sort of what, what we behavioral scientists think of
as a standard RCT. And you know, I, I don't, I
probably don't need to describe what our, our standard RC TS
are, but just for the sake of the contrast that, you know,
participants, you get assigned to a treatment group, one or two
or or however many groups that you're comparing.
And then you take your dependent variable at time 1 and time 2.
And you say, or you know, what's the difference here?
So for example, if you want to understand the effect of a
reminder to stand, this is this is sort of an easy example.
Everyone in Group one might get a reminder.
Everyone in Group 2 might not get a reminder.
That could be your control group.
And then at the end of the experiment, you say like, OK,
who stood more? Was it the group with the
reminders or the group without the reminders?
But if you think about that as an experiment, you know, say
everyone with the reminders stood more than the people
without the reminders, that's super, super boring.
It's so non specific. You, you don't have much
information at all about, you know, what does this work on an
individual level, under which conditions and and so on.
And so one of the contributions of Susan was to develop this
micro randomized trial where all participants are randomized and
maybe throughout the day, many times throughout the experiment
to either get a standard notification or not.
And so now with other version of a randomized trial, you can
start to understand these contexts where the notification
works and really on an individual level, what works for
each person and when it doesn't. And I remember in this 2018
webinar thinking, Oh my gosh, like why isn't everyone doing
this? This is so cool.
And then a couple years ago I was fortunate enough to get
first hand experience actually using Susan's methods.
And yeah, I'm hoping some of that actually gets published,
then I can share it. But yeah, can now count myself
as one of the lucky few who have been able to be engaged in using
micro randomized trials. I think I agree with you in
terms of like it's so kind of take it for granted that, you
know, RCT is in form of golden standard in many ways.
Where of course in some contexts, I think especially in
the pharmaceutical world, maybe it's often times like you kind
of forced to do something like that because, yeah, you don't
have the ability to test, you know, various drugs at various
times of the day over. It becomes a little bit complex
when you do something traditional like that, but in
our world, the different. Yes, there's a lot it doesn't.
You can't use an MRT for everything, but there's you can
do a lot. So I would encourage our fields
to do more. So let me introduce our guest
today, Susan Murphy. She is professor of statistics
and computer Science at Harvard and is best known for, you know
what we just described her innovative research methods for
building personalized adaptive treatments with AI.
And really anytime anyone asks me about using AI to do
behavioral science research, Susan is the first person that I
recommend. And her heart step study is
always the first paper that I share.
Yeah. And in the episode, we explore
much of this. And if you hear us saying, Jed
Dice, a good thing to remember is that stands for Just in Time,
Adaptive Intervention. So it's purposefully
mispronounced to sound cooler, but actually, if you pronounce
it as it should or like as it's written, it's but yeah, don't be
confused. It's it's quite straightforward.
That's what Jedi's are. And so we'll look at that and
how they contrast and build upon micro randomized trials to
personalize interventions in real time to to have the highest
effectiveness. And this goes into, you know,
looking at specifically, we cover how we can use these ideas
to support people in being more engaged or tailoring solutions
based on personal context, like their activity levels or
external factors. And we, as with many episodes,
this is and we we challenge Susan as well on our quick fire
round, which is to AI or not to AI.
We ask her about her most controversial opinion on AI and
so much more. And finally, Susan explains why
behavioral science should be seen as the frontier of science
today, which is obviously very fun to hear.
And hearing her talk about why it's such an exciting field to
be in is something I think we all kind of need to hear.
Maybe right now it's it's really nice.
So with that said, let's listen to our episode with Susan
Murphy. Happens to Murgatroyd.
I'm excited to say welcome Susan to the Behavioral Sign Podcasts.
How are you today? I'm good.
It's great to be here. Awesome.
Yeah, we're really excited and yeah, been in front of your work
for a long time. That's great.
As have I welcome, Susan, I I'm so excited to dive into this and
really get at some of the the methods that you've both
developed and advanced and are really exciting.
You're really at the forefront at this intersection between AI
and behavioral science. And however, I think some of our
behavioral scientists may not be aware of them.
So I would love if you wouldn't mind giving a little bit of an
explainer on some of your work in terms of developing just in
time adaptive interventions and micro randomized trials.
And maybe you know, when once you've sort of given this the
very introist intro, maybe we can try and understand the
distinction between the two and where 1 ends and when where the
other starts. Yeah.
So I think many of the listeners are familiar with manualized
therapies. Often there's a manual that you
follow in order to maintain consistency and delivery of
different types of content, behavior change, support and so
on. So when you go to a digital
intervention where the individual is going about their
life, for example, maybe they want to increase their physical
activity, the behavior change goal here is to help them,
assist them in doing that. So what do we mean?
What's the analog of a manualized intervention?
And that's this just in time adaptive intervention.
So the reason why we do manualized interventions, again,
it's just to help us maintain a certain standard and a certain
quality of delivery of our behavioral intervention.
And it's the same thing here. The idea is that the just in
time adaptive intervention, it has multiple components, 1
component or what we call tailoring variables.
These are this is information about the individual.
So in the case of someone who's trying to increase their
physical activity, it might be their current location, for
example, are they at home or work or out and about?
It might be the weather, is it crummy?
Is it cold? So on.
It might also be whether or not they've been monitoring their
physical activity recently, how engaged they've been.
So all of these variables might influence how responsive the
person could be to a tip about how they might be more active in
the next 30 minutes. Like it could be a, it's really
a brief tip. You know, it's like suggestions
about taking a walk, maybe walking next time you have to go
to the restroom, walk to the furthest point in the building
and back just to add a little bit of physical activity to your
routine. So when is it useful to send
these messages? So the just in time adaptive
intervention provides essentially something akin to a
manual that says, well, if the weather is really bad, better to
have a tip that's related to walking in the building as
opposed to walking outside. Or if the weather's really bad,
the tip should reframe, attempt to help the person reframe what
it means to go outside in that crummy weather and exercise.
Maybe it's not as bad as you think.
You know, that sort of thing. So the way in which the justice
time adaptive intervention operates is it takes those
tailoring variables and there's a decision rule.
The decision rule inputs the tailoring variables and outputs
what kind of whether to provide support such as this activity
suggestion or activity tip and whether to provide it, and then
what type of activity suggestion to provide.
So would you say the just in time adaptive intervention is
really more of a fixed set of rules?
Exactly. It's a fixed set of rules.
Now, if you and I, Elaine, were in the same, you know, suppose
both of us want to improve our physical activity and in at any
given moment, I might be provided a very different tip
from you, but that's because our tailoring variables are
different. So the just in time adaptive
intervention is personalized in the sense that it takes into
account whether I'm at homework, you know, so on to determine
whether or not it's really useful for me to receive this
particular type of activity suggestion.
Right. And I happen to have for
whatever reason both an or ring and Apple Watch and both of them
give me this notification in some variation of saying like,
hey, you have been sitting for a long time, it's time to stand or
looks like you may be time for a walk.
Would that be an example of this?
That would. Be.
I can't speak to exactly how these two devices work.
You know what software is driving these devices.
Often it is some sort of decision rule based hopefully.
Anyway. I do recall I can't remember
what kind of wrist tracker I was using at the time, but I was
trialing one out and I sprained my ankle and the tracker kept
telling me I needed to go for a walk.
Oh dear. Yeah, it was awful for me, you
know, because I was already pretty upset and I didn't want
to get reminded, you know? So I don't know to what extent,
how sophisticated their rules are, right?
They may be more or less sophisticated.
Well, certainly the the standard notification on the Apple Watch
just goes off at 50 minutes at the hour, every hour.
So that's an example. Of decision rules.
You know, you might prefer a better tailoring variable here
besides the time of the day. For example, we had one study
where we were trying to disrupt sedentary behavior, but you had
to be one of the tailoring variables was how long you had
been sedentary. And you had to have been
sedentary for a certain amount of time.
And you had to have not received a message recently in order for
us to even consider sending you another anti sedentary message.
You know, something like, why don't you stand up and roll your
shoulders or something just to disrupt sitting, right?
So that's the just in time adaptive intervention.
But I I do have to comment. You know it is called a Jedi,
right? Why is it called a Jedi?
Because when we first went down this one of my colleagues, so I
partially work in computer science and one of my colleagues
also ambush Torari works in computer science and we were
running a workshop. And so and there's a many
behavioral scientists there. And it was all about
constructing just in time adaptive interventions.
And we coined that phrase because we were everything was
after after the Jedi's. Yeah, we looked for the words
that would go. Yeah, you come up with the
acronym first. Yeah, and you notice how it's
mispronounced, right? Well, JIT, but I still called
Jedi. Called the Jedi.
OK, well going forward I will correct my pronunciation.
It's not a Jedi, it's a Jedi. Awesome.
And how much of yours Lucas and what not been involved in like
is there any? Did he give it the green light?
Well, we're not trying to make any money.
We're just trying to do science and help people.
So I don't think they have anything to worry about.
Us. The people at the, the, the
little workshop where this phrase was coined, they really
liked it. You know, we had Jedi's all over
the place. What's the name of the little
bitty character that trains the Jedi?
A Yoda, a little. Yoda.
Yoda Yeah, we had little Yodas everywhere.
Baby Yodas. We just didn't, it took us a
while, you know, to start thinking about how to write a
paper about Jedi's. And by the time we wrote a
paper, some of the behavioral scientists had already written a
paper. And we're we're using the term.
It's great. Wow, right?
We were thrilled. Of it, yeah.
So the Jedi's, of course, are these more fixed interventions,
but a lot goes into creating them.
And so that's really where micro randomized trials come in.
So can you tell us a little bit about what micro randomized
trials are, how they work, What are the nuts and bolts of this
intervention? Yeah.
So suppose we don't really know whether or not it's useful to
send an activity suggestion when you're at work in the morning
and the weather's not so good. Suppose we don't have a good
feeling. I mean, remember, the activity
suggestion can just be walking around in the building you're
in, right? And we want to learn whether or
not it would be useful to you, to people, if we would send an
activity suggestion at that time.
So how do you learn? Well, the way you learn an AI,
the dominant way we learn an AI is by what we call exploration.
And what does that mean? That means that sometimes in the
morning at work, when the weather's not so good, you get
an you receive an activity suggestion.
And sometimes in the morning at work, when the weather's not so
good, you don't get a activity suggestion.
And the question is at which of those times, if I compare when
you get an activity suggestion to when you don't, which one
appears to help you in terms of being more active?
And that's what micro randomized trials are all about.
It's to provide that exploration so that we can learn how to form
those decision rules. So if the distinction or maybe
the metaphor that you're making is really one of explorer versus
exploit where micro randomized trails are the exploration and
then the Jedi's are the exploitation.
Yeah, they're the Jedi's are are implicitly saying that we have
enough knowledge, domain expertise, data science
knowledge to be able to exploit that knowledge and know what we
should do in the morning at work when the weather's not so good.
Whereas the micro randomized trial is saying, well, at times
at which you don't have that level of expert knowledge.
Let's explore and let's see which one works.
So for example, in this one physical activity study that I
was part of with Petya Klashnia, we knew that if the sensors
detected that you might be operating a vehicle at that
moment, we did not want to interrupt you and send an
activity suggestion. We exploited our expert opinion
that we should not be interrupting you when you might
be operating a vehicle and ask you to go for a walk.
And on top of that, it doesn't make any sense anyway, you know.
So there are many times at which there is sufficient information
about your current setting so that you know you should not
bother that person. Or even we decided if you'd
received an activity suggestion or any other kind of suggestion
within the last hour, we shouldn't bother you.
And we just decided that was just, that's our domain
expertise. We don't want to overburden
people. We don't want to produce
habituation where they are no longer responsive to the
suggestions. So that constrains the
exploration. So in a micro randomized trial,
you're still exploiting domain scientific expertise and you're
exploring among the options that are still you don't have a good,
you don't have a good understanding of which is best.
So it's not complete exploration.
It's constrained and it has to be ethical, right?
That was, that's relate to the point I made about driving.
You also don't want to overly burden people.
That's why you don't want to send a suggestion too frequently
and so on. Also in that trial, individuals
could turn off the notifications for a certain number of hours.
So if you turned it off, there's no way we're going to send you,
you know, we're just no exploration, man.
Yeah. I want to push on the
methodology just a little bit because I I find this the
two-part process very interesting and I want to
understand better what is necessary about having the Phase
1 micro randomized trial kind of leading into as an input to the
Jedi. It how much of that is essential
that one phase feeds into the other versus you could imagine
some combined intervention where the final product has more of
that experimentation and learning all kind of, you know,
smushed into one product. Actually, what you're doing is
you're going into our second topic about reinforcement
learning because that combines let's do it, but to what?
Just go back to the micro randomization.
So there, what's weird about it or odd seemingly odd about it is
that when you talk about sending, for example, sending
activity suggestions to someone as they go about their life,
there's many times you could do that per day and there's many
days over which that could happen.
So like in our in the study I've been talking about, this is hard
steps. The first study each person
could be micro randomized 210 times, and in further studies it
was over 500 times. And the number of times at which
you might explore just has to do with the number of instances at
which you want to learn what should be done at that moment.
You know what kind of support should be provided or whether
you should even, you know, provide any attempt to provide
any support. So micro randomized trials have
what we would view as randomizations.
But in fact, we really should be thinking of it as exploration
like you were doing. Elaine.
That's a much more accurate statement.
But at the time we developed it, we were focusing on individuals,
scientists who were used to randomized trials.
And so we were connecting everything to randomized trials.
So we framed it from a randomized trial perspective.
Nowadays, it's still framed from a randomized trial perspective,
which as we all know is one of the best ways.
It is probably the best way to obtain evidence about one thing
versus another, send an activity suggestion versus not in a
particular setting. But now we have a more subtle
understanding. So back to the trying to help
people improve their physical activity.
The society is always changing, right?
And there may be, you know, we have new drugs that come out out
that help people who are overweight.
We have pandemics that occur. There's crises in communities,
both good and bad things that happen to our society or our
community as time goes on. And the kinds of people who
might be interested in improving their physical activity change
in unknown ways. They may look similar age group,
similar working habits and so on, but they may be in a
different state, for example, or a different ethnic group, so on.
OK so reinforcement learning combines the micro randomized
trial with the Jedi. And So what it does is that at
each time it trades off this exploration that is learning,
trying to obtain information about whether or not it's useful
in our case to send activity suggestions or not a versus
should you exploit the information you've already
gained. So it trades these two at every
single time as you go through many times experiencing a
digital intervention, you the algorithm is trading off well,
is there enough evidence right now to exploit it and make a, a
decision with certainty that you should send an activity
suggestion or send a different kind of support?
Or is there still some uncertainty and so we need to
explore a little bit to make sure we improve our delivery of
this support. My personal opinion is because
society is changing, it's always changing, that this exploration
can never end. We have to have a more of a
viewpoint of that we're going to continually and you're changing.
If you're experiencing a digital intervention, you know you're
changing. You know you're having things
are happening in your life. Your children maybe get the flu
and all of a sudden it's much harder to move around.
You know you don't have time, you're too busy caregiving or
you may sprain your ankle. Sprain your ankle?
Exactly. So you have to, In reinforcement
learning, we the, we have an algorithm that operates, it's
either operating on the phone or in or on the cloud, but
communicating with the phone, the wearable or your iPad.
And it it does some mixture of exploration and exploitation of
its current understanding of what's best.
Yeah. So, so it's never really fixed,
even if, you know, say for four months or so, you don't respond
to an invitation to get up and walk around while it's crummy
outside and you're at work. But you know, say that was when
your ankle was sprained. And then four months later,
there could still be a reason to try that again in case.
But with very low probability, right, with very low, because
you don't want to bother, you don't want to discourage the
individual, right. But.
And you also like in a really good digital intervention,
you'll be able to provide some feedback.
You'll be able to indicate that this suggestion was completely
inappropriate for you in your life.
You know, it's just like maybe. Well, in our case, we weren't
giving suggestions to swim. But if you don't like swimming
and you got a suggestion to swim, you should have the right
to be able to indicate to the little coach on your phone, IE
the algorithm in our case that this is just not a suitable
suggestion. It is, yeah.
And if the algorithm ever wants to ask you again about it, it
should phrase that in a way. Well, in the past, I understand
you weren't, you didn't really want a suggestion about
swimming. But perhaps might, you might
consider it now or something. You know, it has to take into
account your feedback. It has to use your feedback to
improve its learning about you, about what can be done to help
you. Yeah, And I guess I'm keen to
maybe take a step back because I think one thing that comes up
quite often, I think when I'm trying to communicate around
behavioral science in general, you know, a lot of people are
keen for a long time forever, maybe for the forever how how
long humanity has been around. People have been very happy to
say and tell people what they're supposed to do, like do this, do
that to tell them what to do. And I'll, you know, been trying
to communicate around behavioral science that is important to
understand like what we want people to do.
But oftentimes we think that, OK, if someone is not, you know,
let's say fit enough, they they, the what is go to the gym or eat
vegetables. But it's actually quite hard to
understand, like what is the right what?
That's tricky. But then I also talked about
like, you know, it's not about what it said, but how it's said
and when it's said and even by whom sometimes in terms of like
who's the messenger? And so suddenly we're having a
lot of different colored like variables of sort to consider.
Well, it's not only that. It's like you the person you
should have agencies. So like you have decided, say
for example, you are newly diagnosed with stage 1
hypertension. So you need to improve your
sleep, you need to eat better, you need to be physically more
active, right? At least these three things.
Well then you should be able, you should decide what you want
to work on. And if it's like, if the
algorithm, so in our in the world I live in, what we want is
an algorithm, We want to build these algorithms so that if it
notices that you're becoming increasingly disengaged, IE non
responsive, then it might suggest, well, would you like to
switch things up and try working on your sleep instead?
Or would you I'd like to take a vacation.
Sounds good to me. Yes, yeah, let's go with the
vacation. Yeah.
I mean, in other words, you want the AI agent, that's what I'll
call it. That's Yoda.
You want that to be your partner, right?
A partner. It's not.
And that's the ideal. That's what we're striving
towards is that this artificial agent will be a partner and you
have agency. You know, we're all about
helping you achieve your goals. Not our goals, your goals.
Yeah. And I think that's really
important and sticking to the idea of like, just still
weighing some of these things because the shared goal is that,
you know, this person wants to prove their health and they have
sought out help to increase their chances.
And so knowing that like this feels like, it feels like Asian
science, but it's, it wasn't too long ago that, you know,
interventions in behaval science were just only thinking about,
you know, one like, let's say 5 different text messages that has
files and framing. So it's only about like how
something is said and it only testing how these five different
messages are going to get people to take it like, say, on
average, on average or when like we're just going to remind
people at certain times of the day and then yeah.
Quarter till the hour. Yeah.
And so, you know, in terms of, you know, we're getting into
reinforcement learning, we talked a little bit about and
how we think about like how, how would you start, you know, make
sense of these different variables?
Like what is more important to weigh?
What is the core thing to to focus on, you know, how do you
tackle that conundrum of like, should we get the algorithm to
focus a lot on like, how would just reframes messages rather
than how it sends at certain times of the day or, you know,
trying different behaviors, like what people supposed to do
rather than when they're doing it?
Like, how do you think about that conundrum?
Yeah. Right.
So the issue, the big challenge, OK, so I'm one of the developers
of these algorithms, right? And these reinforcement learning
algorithms, they essentially have two parts.
The first part is it's usually some statistical method.
It's trying to learn about you, how responsive you're likely to
be to these types, different types of support in a particular
setting. So that's the first part of the
reinforcement learning algorithm.
The second part is how do you use that information to decide
whether or not to send a particular type of support or
not send any, not bother the person at all.
OK, so we know like many of the people who have a behavioral
science background, they have had some statistics and they
know that there's a lot of noise in these problems.
You know, there's, and when you're talking about the digital
intervention space, there's this is we're, we're providing
support as someone goes about their life.
So all kinds of things are happening.
You know, someone almost hit you when you were driving to work
today or your boss is not in a good mood or you stubbed your
toe when you walked in the door. You know, there's just one shock
after another. So these are very noisy
settings. And when there's a lot of noise,
it's much harder for that first part of the algorithm, the part
that's learning about you to learn because it has to separate
the signal from the noise. And So what we normally do in
terms of building these algorithms, at least in so far,
we'll use domain science to come up with a whole host of
different types of support, but the algorithm will personalize
whether or not you get bothered at that moment in that setting.
And why do we do that? Because first of all, the the
biggest signal is usually between sending something and
not doing anything at all, something active versus nothing.
It's like providing a medication versus not giving any medication
at all. That's where you normally see.
So the chance of this algorithm learning whether or not it's
worthwhile to interrupt you. Is much higher if we focus on
that one decision then we try to the extent we can to use
behavioral science to ascertain what kind of support should be
sent if a decision is made that support should be sent what kind
of support. And now most of the time in our
in the studies I'm in, we micro randomize among those different
kinds of support. So say if there's ten different
ways in which we could provide support at that moment, we one,
we've probably if the algorithm says send support, it's
worthwhile to the person's receptive, they're probably, you
know, they're more likely to be helped by any kind of support,
then we'll send any one of these supports with chance 110th.
So you can learn a lot more than just the optimization of timing.
You can understand what kind of content at what.
Time. But how do we learn that?
What we, we learned that after you have an implementation
period where you implement, So the reinforcement learning
algorithm is operating while you're experiencing the
intervention. It's learning and selecting
whether or not to send different types of support or in our case
whether to send support or not. It's learning about that, but
it's just doing pure exploration among the kinds of support.
OK, so you finish an implementation, say two months
have gone by, you look at all the data from all the
individuals and now you've explored on those individuals.
Every time you send support, you explored among those ten
options. Now you do analysis to to
determine what was one of those options better in a particular
setting. And then now you go to the
implementation and you can use that information.
So in that setting, the algorithm would just learn
whether or not perhaps to provide support and if so with
high probability would provide the support you learned from the
analysis. So there's two types of learning
going on. There's the algorithm is
attempting to learn as you experience the intervention.
That's the first type that's for you.
And me just on an individual level, right and then?
After a group of individuals, they've experienced the
intervention. Now you have a data, pretty rich
data actually, on a whole group of people.
Now you have a chance to get rid of some of that noise.
You have a lot of people. Now you're going to incur bias
because people are different. You know, we're not the same,
but there may be some overall commonalities between us.
Like for example, when we're at work and we're in a meeting.
You really shouldn't be bothered, I mean.
You should, right? So anyway, you know, so there
may be certain commonalities between all of us that can be
learned between implementations. So what we have now is a
hierarchical setting in which within an implementation, you're
trying to learn on each person, is it worthwhile to interrupt
them or not to try and help them out.
And then between implementations you're trying to learn, well,
what kind of help is useful in which kind of setting?
And actually in our world of digital interventions, this is
sort, this is really a good idea because technology is changing
so fast. So between implementations, a
new kind of tracker may have come out.
How do you integrate the tracker into your digital intervention?
There may be, you know, there's societal changes.
You need to think about the societal changes that are going
on or there may be a new type of intervention that's come out.
And like, you know, right now there's a lot of interest in
positive psychology. I know there's been interest for
a while, but it's a renewed interest in positive psychology.
So maybe you didn't have anything about positive
psychology in the last implementation, but now you want
to try and explore a little bit among positive psychology
messages in the next implementation.
So in a world in which the environment in which we're
learning and trying to personalize is evolving really
rapidly, it makes sense to have this hierarchical approach,
learning within a person over time as they experience the
intervention, and then learning across groups of people between
implementations. And that's essentially what we
do. What would you say is the most
interesting or novel finding that you've found at the
population or sample level across groups that that doesn't
rely on individual idiosyncrasies, but something
that you've found from this rich data that goes across so many?
I think many of your listeners will be acutely aware of this,
particularly if they downloaded a mobile app, but we didn't
realize this at 1st. And that was that disengagement
and habituation is really a big problem in the digital
intervention space. So it's very hard to continue to
persevere when you're trying to change your behavior.
It's just really hard. That's all there is to it.
It's just really hard. And so usually people at first
they can stay engaged, say for, you know, 3 weeks, 2 weeks, and
then their responsivity starts to decrease.
So seeing that across many people, we began to realize now
you have to have interventions which are about helping people
stay motivated, helping them. It's not like you're giving them
ideas about how to be more active.
It's about helping them keep their mind going, you know, keep
on task and like if you, so that's one of the nice things
about if you have multiple things you want to work on, you
can switch, you can take a vacation.
This these are ways that you might be able to help someone
stay motivated and giving people breaks is really useful.
The vacation idea, that was a big one and it's obvious now to
me, but I have to say, it wasn't obvious at the time.
I just thought because no one could ever, in the studies we
had run, no one could ever get the same message twice.
They could never get the same suggestion twice.
So I thought, well, why would their responsivity to these
suggestions versus not getting a suggestion decrease with time?
They're not getting the same thing.
They're never getting the same thing.
But that's not it's just it's behavior change is hard.
Yeah, and and I'm sure there's a bit of notification fatigue.
Right when you. Start to ignore all the
notifications. Yeah, Yeah, exactly.
And so we also try in terms of your current context, we try and
keep track of how many times you've been notified recently
because if after you get to a certain level, it may just not
be worth it, it you may, you may need help, but it's not worth
the price of trying to reach out and provide.
It's not going to be helpful. Yeah, I wonder how much of it
also has to do with users when they're getting lots of small
recommendations. It feels like maybe just the
weight of how much there is to do because they haven't formed a
habit. They haven't set a routine in
place that makes these things. It really happened with
automaticity. It's that everything, every
message is one more thing that they have to do as opposed to,
oh, now I just have a habit where I go for a walk every
morning. And it doesn't feel like
cognitive burden. Yes, you know, you have to
cognate about everything. It's true.
But so you know, I mean, this is one reason why we break goals,
for example, big goals into many small sub goals, right, is to
try and help people and then we try and help them.
Like for example, right before you, you eat, try and go for a 5
minute walk or you try and anchor certain behaviors to
patterns that are already there in your life.
You know, we try and these are all part of that toolbox of way
of supports that I was talking about is trying to help the
person find the right way to keep up.
And in the case I've been talking about, you know,
improving their physical activity.
Yeah. Well first of all my wrist is
vibrated, so it means that you need to.
Stand up. Stand up, but but it's also
maybe we're going to soon get into our kind of quick fire
round. But I wanted to just wrap up
with a kind of final question on your research, which is, you
know, you've kind of LED a very data LED approach to
understanding human behavior. And and I guess it'll be
interesting is to hear what you've learned about humans from
your research so far. Does anything kind of generalize
at all at population or, you know, bigger level?
Because I'm just interested if you like had to say, you know,
in short, like one thing about humans, if you're some aliens
that kind of like studying humans from your research, what
would you say that you have learned?
Well, I think the the thing I've learned most, and it's actually
I've not only learned it, but it's a big attractor is I view
behavioral science, behavior change as the edge of science.
It's the frontier science and it's very exciting to be part of
a frontier science. You know, it's not like, well, I
don't want to diss any other area of science, but there's a
lot of areas of science of which there's been a hundreds of years
of work, you know, and they have all these mechanistic laws, but
we're not. Yeah, I didn't want to say that.
You know, it's not simple. We're really a frontier science.
We're on the very edge, you know, brain science in general
and why people do what they do and how they can change their
behaviors and how they might be maintain changes.
It's just it's very much on the frontier.
And I think we have to always, at least the thing I've learned
is always have to remember that. I can't get discouraged.
I can't say, oh, there's so much noise.
It's hopeless. This is a hopeless problem.
No, I have to remember when you're on a, you're working in a
frontier area, frontier science. This is the way it is.
This is, you know, it's a, there's going to be, you know,
you're going to take two steps forward, one step back, 2 steps
forward, one step, right. And that's just the way it is.
And that's because we're working on the edge.
It's great and then you throw a throw in a dash of AI and you
can't get any more cutting edge than that.
Well, AI is supposed to be of assistance to behavioral.
Space, right, Right. It's a tool.
It's a tool in service of behavior.
That, I believe, is the perfect segue to a little game that
we're going to play. Hey.
I hope I can do it. You can.
So this we have a little quick fire round called to AI or not
to AI. So we're just going to present a
bunch of tasks to you, and then you're going to tell us whether
it is or isn't well suited for AI.
For AI to try to solve or tackle.
Yes, to perform. To perform.
Are you ready? I'm.
Going to do my best. All right, to AI or not to AI
nudge commuters to change their mode of transportation in order
to optimize traffic flow. Oh definitely to AI.
Nice. What about help someone break a
bad habit? To AI, definitely use to a.
All right, pick your bedtime. I think you could use AI for
that too. You could help me decide what my
bedtime should be. Assisting, not deciding.
Yeah, assist, not decide. Nice.
OK, what about on the spot therapy for someone experiencing
high stress? Yeah, you could use AI.
It would be good if you can tailor the support to the
context the person is in. Are they by themselves?
Are they in a social net setting?
Is it in the middle of the night, like at 3:00 AM or, you
know, so on? Is the sun shining?
That's sort of it. You know, maybe they could go
and put their face in the sun and feel a little bit better.
You know it. Get a vitamin D lamp.
Yeah, get a vitamin D. OK, purchase and deliver
groceries. We're already using AI for that.
I'm sorry you got to give me something that AI doesn't have a
chance of helping us. We'll get there.
Yeah, we'll get there. But this one is still a little
bit of like adjacent to what I guess and somewhat is happening.
But we can splice it up with saying that it's from a virtual
A or AI Yoda teaching algebra to 6th graders.
Oh, definitely. You can use that.
I mean, there's already a lot of work on this already.
Online math tutoring, teaching. Yeah, definitely.
And is there something that 6th graders shouldn't be taught by
my guy? Things their parents don't want
them to learn. How to assemble a machine gun?
Or whatever you know their parents decide.
But yeah, I think in all of these cases, AI can be an
assistant. I don't want to say AI can take
over from a human, right? And in my experience, humans,
good therapists are always better than AI.
But AI cannot be better than a really bad therapist.
And AI can be there with you at 3:00 AM and the stairwell in the
middle of the night. Very true, but can an AI, this
is our next one, perform a pap smear?
Not right now, but give it time. OK.
There is AI assisted surgery, you know that?
I'm sure you know that, yeah. Harder than a pap smear in my
opinion. Well, the patient is asleep with
the surgery and the patient's not asleep with the pap smear.
I don't know. It's a little bit more
complicated there, we. Go, OK, you don't realize.
Well, so far, so final one, final one, to AI or not to AI?
Give someone a haircut. Not at this time, but that's
just a matter of time. Well, at least for me, I I don't
have much to cut, so yeah, you. Don't have to worry about how
they can help you with your beard.
Yeah. No, I mean, I think this is
coming. It's not at this time that I
know of. Robotic haircuts.
Yeah. Yeah, no, I wouldn't do it.
I don't know, I might. Why not?
I'd try it once. Yeah.
I mean, you already have this thing where you go to a website
and you tell them your size and you've visually, you know, you,
you see yourself trying on different clothes, you try and
see how you would look with different clothes on and stuff
like that. So I don't know.
But you can always, when you actually try on the clothes, you
can return them if they don't actually fit in real life,
right? The haircut, you're not getting
that back. That's right.
I know exactly what you mean. That's happened to me before.
I've gotten a haircut, but this time, you know, from a human.
And I thought, Oh no, there's no way I can go back in time.
Oh my gosh. All right.
We are almost out of time. So I'm going to throw our very
last question at you, which we ask all of our guests.
What is your most controversial opinion about AII?
Don't know. It depends on who sees it as
controversial, But I do think this idea that we can develop
algorithms that actually learn as a person experiences an
intervention, that is controversial.
It is. And it's controversial among
data scientists because of the enormous amount of noise going
on. And there's a lot of concern
that the noise just swamps any kind of signal.
And we've worked on a whole variety of ways to support that
learning in order to combat that what I would call controversial
idea. So you're describing some form
of like synthetic users or synthetic kind?
Of right, you can develop digital twins that's you can try
and do that. But you can also like using data
from other individuals. You can initialize your
algorithm. You can warm start your
algorithm so that even though the algorithm is not making,
it's still having to do more exploration because you're a new
person. You're not the same as all those
other people. It's closer to where it's closer
to finding what would be a good solution to you than if it was
just a cold start and it hadn't used data on anybody else.
So we do a lot of that. We try to use all the prior
implementations on similar people to warm start each
algorithm so that you can detect the signal in this sea of noise.
Yeah. Yeah.
And I saw maybe it was this this spring, I saw some, I think they
were actually quite young students at Stanford.
But they basically had this a generated some form of, I think
it was some representation of maybe some city in California
like Palo Alto or something like this, where they basically had
like generative agents that represented individual humans to
basically understand what the optimal transportation patterns
and so on. Yeah.
And how they interact with each other.
So they're modeling like multiple decision making agents,
right? Each person is a decision making
agent and how do they interact and how does the behavior of one
agent influence the other agents?
And yeah, yeah. Yeah.
So you, I guess you're then quite pro to what can be done in
that domain like in terms of exploring that further and and
especially as you say maybe early days in terms of like
reducing anatomic consequences as well.
Maybe is that what you're describing like in terms of
before before launching an intervention?
Right. At human level to have some form
of way to. Right.
So remember earlier in the this interview I mentioned that you
don't always do complete exploration.
It's very constrained. And everything I do is embedded
in a behavioral science team. So if there's any, you know,
there's ethical constraints, there's burden constraints.
And I don't the algorithm is only permitted to explore within
those constraints. You have to have equipoise for
exploration. And so the ethics there, they
really come to the fore because we're embedded in a behavioral
science team. I'm not sure if I'm really
answering your question. This is critical.
And so of course, when you build a digital twin of us, say people
who have stage 1 hypertension, you build a digital twin of that
subpopulation. It's akin to what you were
talking about. You're trying to understand,
well, how do people who first get diagnosed with stage 1
hypertension, how might they respond to this digital
intervention in various contexts and what rate at which they
might start to experience overburden and habituation.
Try to understand all of that before you put the intervention
out to you know you provide it to real people.
And I know, Elisa, there was last question, but now I feel
like I want to ask one word relating to this little So so
there's a Black Mirror episode, I think it's called, I'm trying
to think, is this called Hang the DJ or something like this?
It's actually one of the few Black Mirror episodes that is
kind of a positive. Like it's a little bit non
dystopian, but actually kind of positive.
You see this kind of like two people that are dating and they
have in this dating mechanisms. They have some form of devices
that tells them how long they can try to kind of date together
and see how compatible they are. And then if the time runs out,
they would kind of be much to someone else.
And then, OK, this is spoiler alert for anyone who hasn't seen
this episode. But in the end of the episode,
you realize that actually what whatever what happened that you
saw was actually within a higher order device of real people.
And what you have seen was just kind of digital twins of the
real people that were kind of, yeah, they were basically like
test dating other digital twins of other users.
And then recently there was, I think the CEO of, I think it was
Bumble, the dating of Bumble, who basically were kind of like
teasing like, oh, wouldn't this be some interesting feature that
we could launch? Or I don't know exactly if it
was more tongue and chic or if it was actually something that
she suggested that they would try the lunch at some point.
You see, this scares me. This sort of thing scares me
really bad. There could be all kinds of
massive unintended consequences and you could end up like you
could end up nudging someone towards just people like
themselves when they might have ended up with a much richer life
if they'd ended up with someone who was quite different.
And so I don't know, I don't know, slot my cup of tea.
Doesn't doesn't, no no AI like we had our game before us.
This is not AI. To me, when I hear that this is
the problem with AI that is extracted away from the
behavioral science. AI needs to stay completely in
touch, grounded by behavioral science.
That's my opinion. Oh yeah, not argue with that,
yeah. Well, yeah, this was amazing.
Thank you so much, Susan, both for joining today, sharing your
amazing research and, and, and work and, and thoughts with us.
And yeah, and also for your, again, your great work.
It's been really lovely to to follow your work and, and now to
speak to you as well. And yeah.
And remember, just don't get discouraged.
We're all working in a frontier science every time.
Anybody you know, in physics, blah, blah, blah about how we're
not so great? No, no, no.
They're not in a frontier science.
I mean, what can you say? I want to be.
I want to go where no one has gone before.
And that's a wrap. You've been listening to the
Behavioral design podcast brought to you by Habit Weekly
and Nuanced Behavior. Sam and Eileen tell me.
This season is packed with incredible insights about
behavioral design and AI, so be sure to subscribe and share the
podcast with your friends. Though you might want to keep it
away from your enemies. In case you haven't noticed, I'm
an AI voice. Yep, pretty crazy.
Quite the improvement since last season's AI outro, don't you
think? If you'd like to collaborate
with us at Nuance Behavior, where we use behavioral design
to craft digital products with Nuance, e-mail us at
hello@nuancebehavior.com or book a call directly on our website,
nuancebehavior.com. A special thanks to the amazing
Dave Pizarro for our show music and to Mei Chen Yap and April
English for their help in producing and publishing this
episode. Thanks again for tuning in.
We'll be back soon with another exciting conversation where
behavioral design and AI intersect.
Happens to Murgatroyd. Oh.
And remember, just don't get discouraged.
We're all working in a frontier science every time.
Anybody you know in physics, blah, blah blah about how we're
not so great. No, no, no, they're not in a
frontier science.