Hello and welcome to the Behavioral Design Podcast.
This season we're diving into the intersection of behavioral
science and AI. We want to make sense of the
state of AI, from understanding how humans interact with
intelligent systems to using AI to do behavioral design itself.
I'm Aline Holsworth, a health tech advisor specializing in AI
and product design. Over the past 15 years, I've
been crafting human centered products with behavioral science
at the core. At Apple, I LED Behavioral
Science for Health AI, designing and launching AI powered
features to help users reach their health goals.
And I'm Samuel Sultzer, your second Co host.
I'm a behavioral strategist specializing in hybrid formation
and designing products that drive long term baby change.
I work with leading tech organizations integrating AI to
scale behavioral design for good.
And I'm also the founder of Baby Bites, a dedicated community on
behavioral science and AI. Quick word on Nuance Behavior
where we help organizations build impactful digital products
using behavioral design. We only take on a few clients at
a time to ensure the highest level of quality for our
tailored evidence based solutions.
If you'd like to become one of our special projects, e-mail us
at hello@nuancebehavior.com or we could call directly on our
website, nuancebehavior.com. Sam, Hey, Elaine, Do you
remember way, way back in the end of last year, in late 2024,
which in AI time is basically a decade ago?
Yeah. Do you remember we organized a
series of of LinkedIn polls? Do you know what I'm?
Talking about, oh, I don't remember a lot of things from
last year, but I do remember those polls.
Yes, I do remember them. And we were really interested in
using AI for different parts of the behavioural design process.
So if you think of the the process just very generally as
like starting with discovery, so trying to aggregate information
and understand the landscape of a key behaviour, identify that
key behaviour, you know, basically like doing a lit
review and understanding what's out there and then moving on to
diagnosis. So this often is, you know, some
sort of behaviour mapping or a UX audit looking at the the user
journey and so on. And then maybe building on those
two phases to do some design, some behavioral design with the,
you could be creating a product or you know, refining an
existing product, something like that.
And then finally ending in a testing phase.
And of course, we're sort of glossing over the fact that you
can do testing in any of these phases, but let's just say we
end with testing that sort of final product.
So we were curious, given the advent of AI and how how we're
seeing all kinds of different applications out there in the
world, what do our friends and followers think it would be
really well suited to replacement or the help from AI
tools. And so, you know, all the
caveats apply with this is a, you know, a very unscientific,
crowdsourced, convenient sample of like our, you know, not very
many LinkedIn friends. Don't say that.
I think we have pretty good reach on LinkedIn.
OK, sure, but for science's sake, small samples this is
these are not the many thousands that you would get on M jerk or
prolific or whatever. So yes, we did these polls.
It was more of AI think, impetus for conversation than anything
else. We really wanted to see like
what are our friends intuitions in these areas and what did we
find? Do you remember?
No, but I'm not actually super intrigued to hear because it
feels like in hindsight, that was like a really interesting
time to ask this because at least for me, I would say
working closely with AII, think around last fall around October,
November, that's where a lot of AI models became kind of taking
the next level leap into something that we're dealing
with today. And I, I feel like so much has
happened since then. So I, I don't know, I'm super
keen to hear actually what, what did people say?
Well, and and honestly, many of the deep researchers that we
have today, we're not even out when we ran this poll.
So the land, the landscape looked very different then.
However, I don't think that the, you know, this is just my, my
hypothesis, but I don't think the opinions would be different
if we asked people the same question now.
So the vast majority of our respondents said that the
discovery phase, you know, like doing a inventory of the
landscape and seeing what existing literature says,
basically doing a late review that this is the best phase
suited for outsourcing to AI tools.
Does that sort of feel right to you?
Yeah, it is interesting. I, I think in some ways I agree
with you that like probably people would say something
similar today, but it's also interesting, like does that
reflect how AI, what AI is capable of or is it also
reflecting kind of like what people are afraid of it being
capable of? Yeah, yeah, we're we're only
talking about people's subjective opinions with all of
their biases baked in, not the reality of the situation.
Well, I feel scarier that AI could be able to do the second
and third steps, especially rather than the first step in
the journey. Like it feels if AI could do a
really good job around kind of better running interventions,
designing them or understanding and experimenting like those is
probably a little bit feels a little bit more closer to home
as well. It might be reflected, you know,
that AI might not be always as good and evolved in those areas
depending on the context. But I also do think it's it's a
little more scary to to imagine for a lot of behavioral
scientists. I think you're right that it is
maybe threatening to one's identity as a behavioral
scientist to think the later phases could be automated by AI.
However, if you think about what are regularly touted strengths
of at least generative AI, that's really consistent with
the late review phase. So like scanning massive amounts
of, you know, whatever it is data, in this case, it's
research papers. So you just taking like
thousands and thousands of papers and assessing them very
quickly, summarizing research findings like how how great is
ChatGPT at summarizing anything? Like excellent often does a
really, really great job. And then extracting key
insights, right? Organizing based on themes, kind
of grouping similar concepts together, doing this sort of
thematic clustering and what you could call topic modeling,
right? Kind of this like organizing and
structuring and finding similarities between things.
That's really what generative AI is really, really good at.
And so you might think, OK, like, yeah, that that that
should lead to a really, really excellent literature review.
Yeah, no, that is fair. I guess what is interesting as
well though is that we are saying that, you know, I think
it was just this week that there was this kind of like first
paper in. I try to remember if it was in
biology or if it was another science where basically it was
AI who researched the literature review of it.
Kind of like understanding the context, came up with a novel
idea to test, like the hypothesis tested it wrote the
paper and then said oh really for review.
So you know. I don't like that at all.
No, it feels wrong, right? It feels wrong.
And I think I, I guess we talked about it before on the podcast
that like these days, I think it's as important to understand,
you know, what they I can do as, as when it's most useful and,
and important to to use it versus when to have human in the
loop and and so on. And, and when, you know, it's
stronger or weaker and and so on.
But it's still interesting that that happens.
Yeah. I mean, just think of the poor
editors at these journals who are volunteering their time, and
now they're met with this deluge of new article submissions,
these poor, poor academics. I know, I know.
I've never felt more sympathy for academia.
No, it is AI has both made a lot of people in academia their life
easier in many ways in terms of what is helping people do, but
also, you know, AI slop is, is a real thing.
And, and I think we see also a lot of research papers are used
kind of put out in numbers that we've never seen before that
yeah, it's probably not so kind of thoughtfully put together
because they're used incentivize to like generate paper ideas or
paper published. And so, yeah.
But it sounded like in your example that there was truly a
new insight that was generated through this synthesis and
hypothesis generate. So like that seems quite good.
I think it that's kind of a step beyond what I was describing as
the literature review capability of your sort of typical
synthesis of what's out there. And it's sort of like predicting
the the next step forward, identifying gaps, and then
trying to answer those questions.
You know, Google recently launched this, you know, Co
scientist as a kind of initiative to support scientists
who are doing good work and make their life easier with AI and
seeing AI as a collaborator. And I probably would say that
for many of those areas is where AI serves a lot of value as a
collaborator maybe rather than the one you want to give full
freedom to. I do think, you know, it's easy
to say that it's impossible for AI to be original.
You know, it's based on historical data exponency.
But I do think that also brings a little bit of a maybe not a
accurate view of what AI is today.
I think it is moved beyond that. And I think it does kind of
challenge some of those notions in that it is able to, obviously
if given the autonomy and access to certain things, it can run
experiments, it can do certain things.
And you know, one of the things that a lot of labs are working
on now is basically creating a environment that is set up
artificially. So basically like taking some
form of real environment in the world and then setting that
environment in a kind of an artificial setting and is mainly
done now for like robotic training.
So imagine you have to train a robot to walk and move and do
some things with kind of the the idea that they have to
understand how the gravity of the world works and how to catch
a ball and all of these things. But now I think NVIDIA has been
quite proud to announce this, where they basically have been
able to set up environments where these robots can be
trained as kind of digital twins of themselves.
Digital twins? Yeah, exactly.
Yeah, you you have a virtual environment that's mapped to the
real world. But yeah, you can't actually
send your robots over throughout the entire world.
Right. And I guess that that would be
interesting to like what do you think?
Like wouldn't it be possible if you have better and better
environments like that with better and better replicas of
digital twins, if you give access to those environments to
AI, like it will be more able to run and test experiments and do
various things and potentially come up with some findings.
You could say that, well, it's still resulting as they're like,
well, what do we know about the real world?
Because it's been, you know, set up in a virtual lab setting, but
it's still kind of at some point becomes uncomfortably similar to
a physical lab setting where we still ask those questions.
So I don't know what do you think?
For me though, I wonder how much of it is coming to conclusions
based on the new evidence that's been generated rather than
really creating kind of a general mental model of the
world and having this more generalized understanding.
I'm not going to speak for robots, but I am thinking of
maybe synthetic users and doing more behavioral design kind of
research. And the reason I ask this or the
reason that I suspect that there's a difference here is
that I've at least in the examples that I've seen from,
you know, AI generated lit reviews and so on, is that
there's this real like dearth and critical thinking.
There's the ability to really kind of extract from what's
there. But moving beyond that, I'm
pretty cynical about at least current ability to move beyond
that in terms of this like novel insights and going to the like
actually the next step that doesn't directly come out of
what's already there. And this actually, as I was
looking back at our LinkedIn polls, this came through this
idea of like, you know, what's uniquely human.
There were several categories that zero people chose as being
well suited to be replaced by AI.
And those things were like stakeholder alignment,
connecting business to behaviour, product strategy,
basically all of these like larger systems thinking and
critical thinking skills that we think really are uniquely at
least at least our current conceptualization of what's
uniquely human. That is kind of where we seem to
be drawing the line. Yeah, that is interesting.
I think another word for that in some ways is, is maybe a taste.
I think I hear that a lot in AI circles around kind of like,
well, if AI can generate a lot of things, they can do a lot of
things, that is great. But it still seems to be kind of
often times if you have high expertise in something, you have
also developed a high taste of what certain things, how they
work and what works and what doesn't work in in making those
kind of decisions is quite hard. And yeah, I do think that is
maybe like similar to what you're saying, like it's kind of
a lack of taste. Exactly, yeah.
We know immediately when we see something that is just
absolutely wrong, whereas the AI may not.
OK, so let's talk about how AI lit reviews actually perform.
Yeah. We have how people think they
may do, but what do we know about how they really do?
Yeah. So that's a really interesting
thing. So basically I can give a little
back story on this. So I noticed that there's been a
lot of tools for a long time around, kind of like helping you
research. And I've been using a lot of
them myself in terms of, I think we talked about it many times
around some of the early tools like illicit and, and some
others, the research rabbit and so on that kind of like helped
pull together scholar. Yeah, scholar.
So various tools that were kind of like made a little easier to
search for research papers and compile them in ways that would
take much longer before. But then end of last year, I
think a big thing that came out was that Gemini, the Google shot
based kind of model launched this deep research feature where
basically they allowed for you to ask question and then it
would scour the web and the somebody was like published,
researched. And it will basically provide
you a report of like, here's what the science says.
Here's the basically the answer to your question in the report
form. And that was in some ways, I
think for many, including myself, quite exciting.
The idea of like, OK, I can put in a prompt.
It would also like make sure that I'm clear.
So would like follow up with some questions like, is this
what you mean? Did you mean that in what
context and so on. So the record found the prompt
and then it will go to work and you would see it.
And the thing that gave a lot of people kind of this excitement
feeling is that you see the sources kind of like tick up.
So it started with like five sources and a 10/20.
And in some cases like 5075, it's like it's, you see, it's
kind of going through and scouring the web and you're kind
of like anticipating and waiting for what will the report say,
Like what will it do? So I think that was quite
exciting. But what I feel like is that
we're living in a strange time where what used to take three
years now take three months and what took three months take 3
weeks. And kind of the example of that
is, you know, earlier this year, that was like the only tool for
that. There was no other tool that
would like do that kind of call it agent based, sending out,
searching the web and so on. Suddenly then Open AI launched a
tool and they creatively named it the same thing.
Deep research, yeah. Thanks.
And and then, you know, a lot of others follow.
There were Chinese DeepSeek, there was Kimi from China as
well. And then some of the other like
established tools did some versions of this as well.
And then we had Storm from, you know, an open source tool from
Stanford. And so a lot of these tools kind
of like began to promise similar things.
And I saw a lot of hype online. Like we saw like we've talked
about it. We saw a lot of hype and a lot
of people basically declaring that OK, you don't need a PhD
research anymore, you know, these tools can do the job and
and so on. In.
Seconds in seconds. And so basically, I think what
we were interested in is with nuance, we wanted to better
communicate to our clients and the people we work with.
But also I think we tried to give nuance like that's the
whole goal, what we're trying to do.
And so we also thought about as an opportunity to be like, hey,
let's look at these tools, let's compare them and see how well do
they actually do. Like what can we actually say
right now about how good are these tools at doing literature
reviews? So that's a little bit
background on on this project. Oh, so, so, so exciting.
I mean, I, I, this feels like it's so needed because all I see
is the hype. I don't see a systematic
analysis that anyone has done looking through these different
tools. I just see like, Oh well, this
person likes this one and this one.
It's like I just want to get away from the hype and know
what's real. Yeah, and honestly, I was kind
of keen for that as well, because you're always in this
current set of things. You're always feeling like
there's some tool that you haven't tried.
And there's always like, oh, what about this tool?
What about that tool? And you see some some screenshot
and you see some LinkedIn post and each tool is promised to be
like, you know, the, the best thing ever and so on.
So basically what we ended up doing is that we put that as a
challenge, like could we kind of benchmark and test these tools?
It was very much a group effort. You're involved.
And basically I wanted new ones in some ways were involved.
But special shout out to Josh Panjwani, who I think him and I
were the ones kind of leading this.
And what we started doing is basically looking at what are
all tools that are promising this, basically trying to make
sense of. How many are they?
Like, how many are out there? Are they US based, China based?
You know, what's the landscape? And so we did that and then we
basically narrowed down to Indiana 8 tools that we all had
access to that we could use. Some free, some quite expensive
and some you probably heard a lot, maybe some you haven't
heard about. And we then basically went
through and wanted to test five different criteria for basically
how well can it do at creating an AI generated literature
review. So obvious things I wanted to
see like what's the cost of using it?
How much time does it take to to run this tool?
Then importantly, how good is it?
Is it, is it good both in terms of writing like doesn't provide
a good written report or an output, but also citation
quality? Like does it actually provide
reliable good sided sources that is not hallucinated, but real
and good from reputable journals and from also relevant sources?
And then lastly, given that all of these were generated based on
the prompts, we want to see how good are these models at
following the prompts. So within the prompts, we
basically specified certain things like the specific
research question we wanted to have answered, but also the year
in question that we wanted to get research papers from, the
citation style, APA style citations that we wanted to have
and and so on and so forth. So we wanted to see how good is
it at following our instructions basically.
And so, without giving too much away, how did they do?
What did we find? Yeah.
So we're currently compiling like the full report around this
and it should be out very, very soon, probably very close to
this podcast. But I can say quite a lot of
interesting things already that we have seen.
So where to start? I think 1 interesting thing is
that a big worry for people is hallucinations.
Basically that if you ask AI something it's going to give you
an answer whether it's true or not.
And interesting findings. There was no account of any
hallucinations in any report. None of the tools hallucinated
in any way. What is your definition of a
hallucination? So it's making up basically a
citation that is not real, so it's giving you a.
Citation OK specifically for citations.
Exactly. Yeah.
So it's a good, good thing to clarify.
So it might make the wrong conclusion some point, it might
claim certain things, but it. Which I did see that.
OK. Yeah.
But it's not going to claim it based on a made-up citation.
So again, that is not looking at their accuracy in terms of how
well they're making inclusions and using citations.
But it's like, at least we know citations that are used are real
citations in all of the tools we we tested.
And so that should at least give some level of like, hey, we
don't have to worry about at least these tools just making up
citations if we use them, which is kind of a big finding in some
ways. Right?
Yeah. For all the tools that you found
this, all of them, Yeah, that's great.
Yeah, then when we look at how much time it took let a span of
some tools were able to generate a literature review in less than
a minute on average and some it took around 35 plus minutes.
So that was a span that they took, which is pretty big span,
but also like 35 minutes is still relatively short.
That's. Nothing.
I mean this would take you so long otherwise, at least a day.
Yeah, exactly. And I guess going into more
details, I I'm kind of interested, what do you think
did best shyness tools or American tools?
Oh, come on. Best in what regard?
In general, like in All in all categories like we, we can see a
pretty overarching tendency around quality well.
All right, despite my disappointment with the United
States, I still have a America favoritism I would say.
And yeah, I think probably the US based tools performs better.
And yes, they did so. All right.
In recitation quality, in terms of writing quality, in terms of
prompt accuracy and sensitivity, tools like Open AI, Deep
Research, Geminis and the likes performed better than the likes
of Kimi or DeepSeek on those things.
What I would say that both Kimi and DeepSeek, which was the two
Chinese models that we we looked at, they were extremely fast.
So they were the ones that were basically quickest at generating
things and both of them are free.
And so if you wanted to use, you know, either deep research from
Open AI or from Gemini, it would cost you at least $20.00 per
month or something, which is still run to the sheep, right.
But Kimi and Dipstick are both free, but with a caveat that you
might be giving away data to the Chinese government.
So there's there's always a cost, right?
Nothing is completely free. Yeah.
In all cases. Yeah.
I mean, she's like, I guess to hear from you, what would you
want to know because you don't know the results.
So what is you're interested in terms of finding out?
So I'll, I'll just for context, say that I rated many of these
generated lit reviews and was overall pretty disappointed.
There was one that I read that I would actually use a single 11
output. Everything else I would like,
you know, even if I rated it, I don't know, six or seven out of
10, I wouldn't use it. I would not it, it was of no
value to me. Even if I, you know, could would
say like, oh, it's organized OK, Or you know, the citations are,
you know, relevant and has appropriate structure, etcetera.
So what I want to know what's the one?
What's the best? Tell me the one I should use?
That's my big question. I know.
Well, interestingly to your point, what we saw is basically
that across these three quality categories, so citation quality,
writing quality and prompt sensitivity or accuracy, none of
these performed best across the board.
So there were some that were better in one or two categories,
but then really bad in 1/3, for example.
And so there's not one tool that is better than all of them
across the board. So as an example, we can say
that like when it comes to citation quality and prompt
accuracy, elicits and maybe Site AI were the ones that performed
best there, but they're also some of the ones that performed
the worst when it came to writing.
Yeah. So that is currently where we're
at right now, where we're getting some tools that are
better and better at specifically helping us make
sense of the literature, but they might not be the ones that
we want to rely on for also compiling the findings and
writing and making sense of them.
So I think in some ways this reinforces my kind of belief
that I had before this, which basically these tools can be
extremely valuable today. And you will almost be fooled to
completely disregard the tools that are out there, but you
would also be fooled to completely rely on them for
everything. And keeping human in the loop is
basically still a really important factor.
Yeah, I think my main take away from this is that take any of
the outputs, it's like 110th of a literature review.
It's not, you know, there are major gaps, complete
misinterpretations of the data sometimes, which could be really
misleading. So there was one case in one of
the better literature reviews that I reviewed, unfortunately,
where correlation was completely confused with causation.
And it described the impact of financial literacy on financial
well-being. And came to the conclusion that
based on this association between, you know, people having
higher financial literacy and better outcomes in financial
well-being, that like this is an important area to focus
interventions on. Well, like actually, we know
from so, so much research that financial literacy interventions
actually have so negligible, like a fraction of a person
percentage of an impact on one's actual financial decisions and
their well-being. So, you know, without having the
expertise to interpret the results that you're getting, I
think it's really kind of dangerous in leading people
towards the wrong conclusions. I agree and I would say we put a
lot of thought into that. The prompt we would give the
models, and we purposely gave them probably a way beyond
average quality of prompt in yourself details this
instruction clarity around what we wanted and we would expect
that in the wild most people use these tools with much worse
prompts. Yeah, you're really setting it
up for success. Yeah, and and even then, as you
say, this happens, and I like to liken this to them, look back at
like, what would humans do in this context?
What could we expect of humans? And like kind of what we're
seeing now is what we expect from like maybe intern
researchers where they don't really understand the thing
they're being asked to do. And so they kind of make sense
of it. They go out and look for
findings to put together something.
But they don't have enough kind of taste or expertise or context
to know actually what matters and, and what to neglect and
just put everything together and put it into report and give it
to be like here, here, here's what he asked for.
And you're like, well, it's not really what I asked for.
Like this is kind of a mismatch of a bunch of different things
where kind of what I asked for, but not really.
So that is kind of what I would say as a tendency for a lot of
these, and especially the ones, you know, like Open AI, Gemini,
Both of them are also importantly relying on what they
can find online, not what they can find in research papers, but
what they can find in like research abstracts online.
And so you know that whatever they tell you, they haven't read
the methodology, they haven't read the sign of the study, they
haven't read any of that. They've only read the abstract
of the study. That's a huge problem in itself.
And then if you take that a step further, they've not only not
accessed that information, they've certainly not critically
examined it. Like, if you look at, like,
every single behavioral science lab or any kind of science lab
itself, like a very common exercise is some sort of journal
club where you will read an article and then assess, sit and
say, like, OK, like, did you know, what is the validity with
the internal and external validity of this paper?
And you analyze its methods. You say, what are the
shortcomings? What's the sample size?
You know, on and on and on. And every single paper that I've
ever analyzed has something wrong with it.
There's no perfect research study out there.
And all of these literature review tools are completely
glossing over that entirely. There's no critical analysis at
all. It's just we're taking at face
value whatever the abstract says that you found what you said you
found. And so often that's not the
case. Yeah.
And I think that's something that people can relate to in
that lot of their models are trained to be kind of trust what
we say to them. So like if we say that the Earth
is flat, a lot of models traditionally would say, oh,
yeah, sorry, I was wrong, it is flat.
Now they are better. Like a lot of the models are a
little bit more willing to push back, especially the reasoning
models, they're a little bit more able to say, hey, actually,
I don't know if are you joking with me?
Like they would actually kind of understand that it's probably
not true if you say something completely out there.
But again, a lot of these things are possible.
If you don't have a critical eye, you can use gloss over
whatever potential aspects that could be wrong and so on.
So so I agree with you. Like that's a huge issue still
with any of this is that they used take everything at face
value and basically compile everything as it was the same
quality and and so on. And even if we'd instructed to
say like, Hey, weight the ones with higher sample size more
than the ones who don't, that's still not something that usually
this ones are really good at doing.
And that's also like, that's one proxy, but it's, it's like what
makes for good research papers is really massive to determine,
right? Right.
Yeah, yeah. What's the, what's the
experimental design like? Is it an RCT?
What? What were the actual conditions
that were included? How?
What comparisons were made? Yeah, yeah, there's there.
It's hard to find rules of thumb for the quality of a paper.
Yeah. Can I say something fun actually
around this? Yeah.
So one of the two kind of prompts that we gave 11 prompt
was around, as you said, like financial well-being and it's
basically around UK population and younger adults.
So that was the the thing. And it's really funny how
sometimes it took some of those things, like it took either
younger adults or UK and it just came up with completely like
studies that had to do with something with well-being.
Like, like how likely are, you know, people that are 18 to go
to the gym? And then based on based on this
finding, we can say that like. No, yeah.
Yeah. So it wasn't always great at
some of those things. Yeah, and I noticed a lot of
COVID era research was included across many of the reviews and
and I just like that was such a unique time like looking it just
feels a little non generalizable.
Yeah, yeah. No, I agree.
So maybe obviously we'll release all of this very soon and many
more things to talk about. But I think 1 interesting thing
here that is somewhat sad I guess is that we don't know
which model was the best, but we can definitely say which is the
worst. Oh good.
Tell me. It is the storm model by
basically it's a Stanford lab and bummer, it's a bummer.
I was looking forward to trying that one out.
Yeah, yeah, I know once we tested it a little bit deeper
and really evaluated how it performed across the various
settings and and prompts, basically it wasn't able to
perform very well across any of these things.
And it was, yeah, by far the worst of all of the ones.
So the only one that's kind of like open source and free and
American in this case, like it's both American open source and
and free. That was the worst one and by
margin so that like I would almost now having went through
this, I would recommend people not to use coast arm.
That's probably the only one I would like definitely say don't
use it. It's just going to be mainly
going through blog posts rather than research papers and oh.
Wow, that's shocking. Yeah, that is shocking.
Yeah, bummer. It is a bummer.
And they kind of also asked with many of these.
The tricky thing that I think I'm most worried about still is
that the people who are quote, UN quote buying literature
reviews are often the ones that are not like really good at
doing it themselves. Like they don't really know what
a good one looks like. And so they could be like a lot
of people that would say like, oh, actually, I used to hire a
researcher to put together my literature reviews.
Now I'm going to just do them myself.
I heard this tool Storm was out there and I'm just going to do
it with storm and then storm generous report.
It has a lot of citations, but you have to like click on the
citations to see, oh, it's a blog posts, It's a this, it's a
that. So if you squint it looks really
legit because it has cited like 35.
Sources. Wow, that's a shame.
So I guess this in some ways marks a first initiative that
we're going to publish around something fun that we're doing
with Nuance, which is basically we've been working with AI for a
long time now. You know, even before Nuance,
both you and I were involved in a lot of AI related initiatives
and so on. But we're now going to
specifically because we feel like as an example of this,
there's a need for a little bit of thought leadership and help
people navigating the state of AI.
We're going to actually have AAI Lab dedicated to specifically
exploring these kind of questions and hopefully sharing
some important findings around, you know, what works, what
doesn't work, and hopefully as much of what tools to use and
some of those things, but also like really helping make sense
of when to use some of these things because it's becoming
more of a not so much if AI can be useful, but more how it can
be useful and when you should use it.
And so a lot of our initiatives will be looking at stuff like
this, looking at kind of how we can automate aspects of our
work, but also looking at AI adoption.
How can we do that? Well, because that's not a
problem at all, right? Everyone is doing the adoption.
Perfectly, yeah. Not so that is also something
we're going to be sharing more on and we're doing already some
work on that with clients and yeah, and then many other
interesting things. And actually I wanted to shout
out something that you recently posted.
It's maybe not exactly related to our AI lab, but it is a
really fun AI example and and also relates to something that
is also something new for us, which is our behavioral redesign
series. And so kind of reimagining
products if they had a little more of a payroll spice for good
or bad, I guess. Yeah, some are spicier than
others. Yeah.
And and you shared one that I really loved this week where I
think it was a little bit of a personal story as well behind
that. Could you share the Peloton
example that you redesigned? Oh, yeah.
I mean, so this is something I think about a lot is the sort of
flexibility and forgiveness of the various apps out there that
are trying to help you reach some goal that you have.
And, you know, you're committed to reaching that goal.
But also you're a human and life gets in the way and you have
hard days, you have easier days. And on those hard days.
I have found that unfortunately basically 0 apps have any
tolerance or any acknowledgement that like you had a hard day,
you didn't get any sleep last night.
Let's maybe readjust your plan. Yeah.
And I think like my favorite example of that is, you know,
Fitbit has been around for a long time now and it's been like
kind of getting people to get their daily step counts in and
famously gone. A lot of people crazy where they
like, they realize before going to bed they haven't reached
their step count and so they start running in.
David Sedaris in particular. Exactly.
Yeah, But that comes to mind as an example of like, we've been
faced with this for a long time now.
And if it's like still, that is still kind of a burden given to
a lot of users where it's just like, you know, you have a goal
do it doesn't matter if context shifts, you're expected to do
the same thing every day. Yeah, it's always a nudge to get
you to achieve your goal, not to be kind to yourself and maybe
make you more likely to reach your long term goal.
And so the, the idea with this Peloton design was, I mean,
taking in your sleep data and really like presenting that to
the user, acknowledging that you had a really hard night last
night. I can see that, you know, you
didn't get very much sleep. You were, you know, up three
times at night feeding the baby. And, you know, maybe maybe
Peloton doesn't know that you were feeding the baby, But I, I
can add that personal context here.
And then saying, like, let's adjust your plan.
I see that you were going to do a really intense hit workout
this morning. Maybe we should swap a yoga
instead. And that feels like something
that you could actually accomplish, which maybe helps
you stick to your goal in general of regularly exercising,
but isn't so demotivate like there's just zero chance that
you're going to do a hip workout in that state.
So, you know, some sort of compromise and actively
suggesting that as a kindness to the user.
Yeah. And I love that because I think
we've often talked about the city of well, it's great to have
an if time plan around kind of like, you know, if I get tired
and I will do this thing instead or if I get sick, I will do that
thing instead. But I think it's really hard for
people, Like it's hard for me, it's hard for anyone to really
have those backup if then plans always available or always
thought out in advance. And so why shouldn't our
technology help us? Yeah.
And I feel like what oftentimes I see, what we see probably a
lot of people do is they're going to kind of a fuck it mode
instead where they say this heat workout, I don't have energy for
that fuck it. I'm just going to then do the
complete opposite, you know? And so, yeah, I think giving
people this kind of like, you know, if then plan provided for
them and also like really making it easy to do something instead
of nothing. Yeah, I think it's key that this
prompt was to substitute your HIT workout with a yoga.
It wasn't like, you know, forget about your goal altogether.
What, you know, while there is a time and place for that as well.
And you can sort of see that in Marissa Sharif's research with
emergency reserves. Another way to sort of continue
towards towards your long term goal is to just do something so
you like feel like you've progressed and you're sort of
psychologically consistent with your goal.
Even if you didn't do the hard thing, you at least did
something that's consistent. Yeah, I, I, I think this is such
a obvious thing that needs to be in the world.
I mean, to know, you know, how this has been for you on a
personal level, because I know, you know, you've recently had a
baby and you I know you have a Peloton.
So how have you dealt with this without Peloton supporting you?
Like how has this been? Well, yeah.
Well, so maybe I should say the reason that I created this
design is because I think there's a very real need for it.
I have gone the fuck it route and just not done anything but
this is my cry for help. Like I want to get back into it
but it's really really hard. Yeah, yeah.
I don't have a peloton, by the way.
Yeah. That's fine.
So I guess hopefully we'll see some changes around this and
certainly a worthwhile nudge to the industry to reimagine how
this user experience could look like.
And yeah, I think this is one of many redesigns, I think.
How many have we published so far?
Oh my gosh, at least maybe 10:15 so far, and we've got a whole
lot on the way. Yeah, yeah.
So check out I think in the links and then also we we can
see on the Nuance website we list all of them, but also, you
know, on LinkedIn we share them as well.
So I guess I wanted to share a bit what was going on, some
interesting findings, some interesting stuff.
Yeah, lots of fun, exciting initiatives launching with the
AI Lab and Behavioral Redesign series.
So yeah, just stay tuned. And we also have a bunch of
really exciting guests coming up on the podcast soon.
That is true and I'm really looking forward to it.
And I think, honestly, what I felt is that this podcast has
been really a therapy session every time we've recorded an
episode, especially also because we've had so many, I think,
incredible guests. And, you know, I've tried to
make sense personally, you know, what is the state of AI?
What is the state of behavioral science?
And it's been such a gift to, like, having recently talked to,
like, Sandra Matz, Susan Murphy, like all these people who are
really both pioneers but also leaders in a lot of aspects that
we're kind of thinking about. And yeah, as you say, some more
similar conversations coming up soon.
But thanks again for everyone listening and supporting the
podcast. Really appreciate all of the
yeah love we received as well. So that's a wrap up for today,
but see you very soon for more explorations on all things label
science and AI. And that's a wrap.
You've been listening to the Behavioral design podcast
brought to you by Habit Weekly and Nuanced Behavior.
Sam and Alene tell me. This season is packed with
incredible insights about behavioral design and AI, so be
sure to subscribe and share the podcast with your friends.
Though you might want to keep it away from your enemies.
In case you haven't noticed, I'm an AI voice.
Yep, pretty crazy. Quite the improvement since last
season's AI outro, don't you think?
If you'd like to collaborate with us at Nuance Behavior,
where we use behavioral design to craft digital products with
Nuance, e-mail us at hello@nuancebehavior.com or book
a call directly on our website, nuancebehavior.com.
A special thanks to the amazing Dave Pizarro for our show music
and to Mei Chen Yap and April English for their help in
producing and publishing this episode.
Thanks again for tuning in. We'll be back soon with another
exciting conversation where behavioral design and AI
Intersect happens to. Megatroid.
The.