Hey, Sam. Hey, Lane.
So I've been thinking about something and I want to get your
take on it. And I think we have an
interesting situation that's emerging in this landscape of AI
and mental health, which of course, is one of the things
that we think about a lot. On the one hand, we have therapy
and companionship, like really rising to the top as the very
popular use cases. These are common use cases for
people using generative AI. There's evidence that this can
sometimes be helpful, especially in cases where that chatbot is
clinically validated, say like this Thera bot from Dartmouth
that we'll talk about in this episode.
But on the other hand, we see from research like this study
that pretty recently came out from Open AI and MIT that
spending more time with LLMS actually has a detrimental
effect on well-being. So this study showed that people
who spent more time really like heavy users of Chad GBT had an
increase in loneliness and sort of substituted for these real
world interactions that of course are very important for,
you know, being alive and feeling like a human and so on.
So my question for you is, how do we reconcile these findings
that seem to be at odds with one another?
I think it's a very interesting state of things where there's
clear evidence, again, that using AI can be extremely useful
for many things, but again, that it can also, through overuse or
through extreme use, have some form of black fire effects.
And as a Swedish person, I can't help but think about a very
historically significant thing that happened in Sweden.
Can I give you a quick history lesson?
Yes, please do. In the mid 20th century, an idea
was introduced in Sweden. Basically the idea was that how
can you truly love someone let's say a partner if you're
depending on them for their income.
So if you're a woman in in 1950s and most women at the time were
depending on their counterparts salary to survive, like is that
love? Is that true love?
It's really just utilitarian. It's one way love.
Exactly. Well, the Swedish theory of love
was that basically like what the government should be, It should
be what's called folk Hemet, basically the home of the people
that you should only depend on the government, so you don't
have to depend on a partner. Even as a child, you shouldn't
even have to depend on your parents, because if they're bad
parents, you should be rescued from them.
So that's the idea. The government should serve as
this home for the people. The kind of backfire effects of
this is that, of course, Sweden has had a really good Social
Security net and like provide a lot of support for people to
live relatively good lives if compared to most countries.
But the flip side of this is that, well, if you don't need to
depend on any one of them, you might find yourself lonely in
your apartment because you don't need to depend on your
neighbors. You don't need to depend on your
partner, you don't need to depend on anyone.
And there's been a lot of this case in Sweden, kind of always
character of Swedish society where, you know, people are very
cold towards each other. They kind of like isolated and
there's a lot of loneliness and a lot of people that are
basically in the worst of cases dying alone and no one noticing
for months because they don't have to depend on anyone.
This one I'm trying to take back to with AI is that like AI is
amazing and so incredible in what is able to provide, but
it's also risking to some very similar things like that you
could become so easily quite dependent on AI for
companionship, for productivity purposes, for all these things.
And you're less incentivized to do things that are harder, like
getting out of your comfort zone or going out of your apartment
to hang out with a friend. Like all of these things take
more effort. I think it's a really
interesting, I did not think of it that way, but certainly you
can have too much of a good thing, right?
The risk of taking that argument too far perhaps is you get into
the American conservative talking points of like, we
shouldn't have a social safety net at all because people will
take advantage of it and so on. And like, that's not the answer
either. And then if you take that, if
you sort of, you know, copy and paste that argument to AI, we're
not saying that we shouldn't have AI because there are some
potential detrimental consequences of overuse.
Maybe we just need to design systems to prevent people from
overusing it. And I think one thing that I've
been thinking about so much recently is this one design
feature of ChatGPT where every single time you get a response,
instead of just giving you the response.
And, you know, if you have another question, you can ask
another question. And instead of just leaving it
at that, every single time ChatGPT offers to do two or
three more things for you. Would you like me to create a
visual for you? Would you like me to, you know,
turn this into a table? And it's just so irresistible to
continue this chain of work forever because, you know,
sometimes I'm like, yeah, sure, make me a table.
Like, that sounds fun. But then you find yourself just
spending so, so much time going down this trail of things when
you only wanted to do, you know, one thing.
And you end up just lured into this constant cycle of creation
where ChatGPT is just making more and more work for itself.
But we don't have to design systems in that way.
Yeah, No, exactly. Yeah.
I think it's not only in terms of work, but also again, we'll
come back to a companionship. It's the same way there where
it's like you can have the most boring conversation in terms of
like saying the most boring things and it's just going to be
like, that's so interesting, what about this?
It's just going to want to continue talking forever
basically. And I think it's the same thing
there. It's just the sign currently is
used for you to engage as much as possible.
And this whole conversation reminds me so much, actually, of
some of the more nuanced research on social media.
And I think often we're like, we take this blanket approach of
social media is bad. It's like ruining the children.
Everything is terrible about social media.
But actually, there's quite a lot of research showing that
there are, you know, many people who benefit from social media
because it gets them to actually augment their real life
relationships. So these people are not spending
too much time on social media and they're just like using it
to connect with people who they like actually see in real life,
not as a substitute, but as a sort of supplement.
And I think that we can kind of think of LLMS in a similar way
where if you're using them to not entirely substitute your
real life, you know, therapist or your friends and so on, but
perhaps you are using it in smaller ways, then there's like
a healthy way to approach it. So I think it's interesting that
we're currently finding ourself in this, call it sycophantic era
of AI where, you know, AI is super keen to talk forever,
always encouraging, always like if not prompt, otherwise blindly
positive. And I think there will be kind
of an interesting transition at some point where if you truly
want AI to add the value of a human, we don't want it to be
that way. We want it to be a little
different. Like if we had AAI therapist for
example, we would probably need it to tell us some hard truths
and to not, you know, just bake everything into some form of
empty platitudes and sharing. Correcting negative behavior.
If you say I want to do something that is, you know,
socially unacceptable or threatening to other people's
lives or your own life, then yeah, your AI therapist should
tell you that's not a good idea. Yeah, right.
Shouldn't say, yeah, I understand.
Go for it. Yeah, and just be some form of
counterbalance. We want AI to help people form a
better understanding of the world and how they can live a
good life. And currently it's not really
doing that very well. Whatever people bring to the AI
tends to often times to make worse somewhere if someone comes
with AI and have some really wild conspiracy theories at this
point unfortunately, with the exception of using Gordon
Pennycook. 'S the debunk pot.
Yeah, except for the debunk pot, they will find themselves like
probably deeper down that conspiratorial route.
But yeah, I think this is always leads us quite interestingly
into our episode of Today with Allison Cereso.
So Allison is a clinical researcher, psychologist,
professor, and senior vice president of research at
Empathic, overseeing AI tools that supports therapists so that
it can deliver more accurate empathetic care to their
patients. So this really strikes this
interesting balance between understanding when and how to
use AI athletics is today. And what Empathic aims to do is
provide AI tools that gives feedback and suggested text to
therapists to help them be more empathetic to their patients.
So basically maybe again as we talked about like not looking at
the extremes of whether or not using AI at all or using AI
completely, but Morris is a form of middle ground of seeing AI
maybe as a Co therapist. And really interesting given our
episode with Mickey Insleck where we talk about how a
generative AI actually does such an incredible job.
At least, you know, simulating empathy.
Then actually you can use it as a tool to then coach or teach
therapists, be more empathetic. And in our episode with Allison,
we talk all about this, how empathic works as this sort of
Co therapist and the role of AI versus humans in therapy and
where they can really kind of help each other, balance each
other out, where it's important to have a human in the loop.
And finally, how we can use these tools to even mitigate
bias in therapy. So really exciting discussion
and we hope you love it. So I'm very happy to say welcome
Allison to the Bible Design Podcast.
Thank you so much for having me. It's really great to be here
with you both. Yeah.
And we're excited to get into so much of the interesting stuff we
hope to cover with you because you are, I don't say really in
this kind of fascinating space of trying to see what can be
done on this intersection of thoughtful science based design,
combining that with AI and how to leverage AI in good ways.
So tell us about Imperfect and what do you do?
Sure. So Empathic is a conversation
intelligence company, but we think of ourselves as expanding
human understanding to foster clinical precision and to also
foster precision medicine. Our tool is used in a number of
places, but the majority at this time is in the clinical trial
space. So we're often times in the
background helping to make sure that clinicians are sticking to
fidelity of a particular protocol of how they're supposed
to run a trial. But we also are working in
clinical outcome assessments. And so again, providing AI 100%
quality support to ensure that clinicians are able to do their
work in a standardized way and to also have more precision in
the way that they carry out their job.
So I think often times you can think of us as like a Co
supervisor or sometimes people use the word copilot.
But for us, it's the way that we can really like embed or
integrate AI into clinical practice in areas where humans
do need support. I'll just give the example of
like, you know, in a clinical trial space it is, it can be
humanly impossible to review 100% of what happens in every
single session because humans get exhausted.
But also it's just very difficult, you know, time and
resources, whereas an AI can do it.
And so it's a really great place where an AI can provide more of
that coverage and not just do the coverage, but also allow
clinicians or the humans to really focus on different kinds
of parts of the job. So to really maybe focus when an
AI might be able to detect clinical risk than to go in and
uncover what's going on there. You've referred to it as a Co
therapist at times as well. Say that you're a clinician
who's engaging with the tool. How do you sort of interface
with it? What is your experience as a
user? Yeah.
You know, for us, often times a user like it, that can look
different, right? But it could be that it's
somebody who's overseeing a particular site for a clinical
trial or somebody who is overseeing that the assessments
that people are doing in a clinical trial have been done in
a standardized way. So often times, like I think,
our tool is used by the supervisor who can then be
flagged when more support is needed.
The best example I can give is, so I am a former professor.
I worked at UC Santa Barbara for many years.
I had my lab there, but also founded a trauma clinic.
And for a period of time, it was just me and, you know, several
doctoral students who were learning to be clinicians.
And it was impossible for me to do 100% coverage of what was
happening. It was a trauma clinic that was
founded in 2020 during the Black Lives Matter movement, when it
was really getting national recognition.
And so we wanted to be able to be a safe place for Black
residents in the Central Coast, to be able to connect with
therapist and just be able to talk about trauma, but to also
really think of trauma in a more comprehensive way where it isn't
just, you know, exposure to maybe a traumatic event like at
war or something like that or a car accident.
But it was also just the ways that people had to navigate, you
know, living in a world where there was a lot of police
violence at that time and sort of just beating community,
right? And especially for folks who
were being triggered on a regular basis because so much
media exposure to trauma or having to re see, right, like
violent events happen. And so at that time, you know, I
was training this group of doctoral students to do better
assessments around understanding trauma, understanding PTSD, and
at the same time making sure that they also had the basics
down of like, how do you connect with a client right in your
first session? How do you do a comprehensive
assessment intake and just provide support over time?
And so it was impossible to like, you know, sort of do my
job, also get funds for the organization or the clinic, hire
new folks. I do all of that.
And so I was pretty like resource restricted.
I mean, this is an environment where having kind of Technical
Support would have been incredibly helpful, right?
Because at that time, then I would have been able to use a
tool like empathic to be in the background of therapy sessions.
And we do that. We're in the background of
therapy sessions at times, but to be in the background to be
able to flag to me right as the supervising and clinician
moments where we're growing therapists, right?
And junior therapists, like where they need more support
maybe in detecting suicide, responding to suicide, but also
just need more general support in all the different kinds of
ways. And so this is where I feel like
AI can really help clinics like determine like what areas they
can give more support to. I want to unpack.
That a little bit more so when you think about like areas where
you really need a clinician, where that expertise is really
critical versus the types of tasks that you can sort of
offload onto something like an AI, perhaps a clinically
validated AI. How do you sort of think about
this distinction? Yeah, we don't want to replace
humans. That's not the goal at all.
It's more so fostering precision medicine when we're able to.
And so I think like with your question that there's different
ways that you can engage with AI to do some of the more like
rudimentary things. Maybe, you know, getting the
gist of like a clinical note together so that if you only
have a few minutes between clients, the note is generated
for you and you can go in and add, you know, like nuances that
maybe weren't captured. But I also think that there are
moments like with the work that we do where it can help bolster
the work that clinicians are doing, so help them be maybe
more acutely attuned to conversational elements that
would trigger that somebody might be an imminent risk of
suicide or something else. So I think both, right?
Like, I think that you can do the more simple things and I
think in today's world, I think that clinicians are often times
maybe a little bit more comfortable with doing the
things that, you know, that they do need help with and that like
nobody wants to write notes all day like that.
That is hard. Like as a part, you know,
somebody who's like doing clinical work all the time, Like
that was not the fun part of the job.
But at the same time, I think there's also other areas where I
do maybe like, I want to engage with an LLM to understand, like,
could I do this differently? Could I approach the way that
I'm asking about maybe like the grieving process for this
client? Like could I do that
differently? Am I missing anything?
Almost like a training tool. Yeah.
I mean, I think there's so many use cases for it, right, in
psychotherapy and other clinical spaces where psychologists are,
right? Because I think often times,
like I'm very engaged with like the American Psycho Association
and we talk about different ways that people are using AI.
And I feel like often times we just completely forget that
there's so many psychologists engaged in clinical trials as
well or in drug development. And so I'm a clinician that's
really driven to do good, robust science.
And I make sure that the science also informs the way that I do
my clinical work. Yeah.
So I think even with that, like there's so many different ways
that I would use AI to do my job as a clinician, but to also do
my research and. Here's what you describe as
picking up on some of those things that maybe it's missed by
a clinician during a conversation.
Is that based on the words that are said or is it more even
looking at how they are said in terms of the cadence or the
tone? I think it's both, right?
So I think that sometimes we can pick up on literal words or like
therapist utterances or provider utterances, but I think also you
can pick up on things like synchrony or on therapeutic
alliance, right? Or sentiment.
And, you know, one of the best use cases that I can think of as
well is that often times like our tool is in the background of
like medical visits. And so we make sure that we give
a report to the provider within a few minutes after a visit.
And the provider might see 1020, who knows, right, patients a
day. And so they may engage in
certain communications that might be off putting to a
patient or maybe they just didn't pick up on a certain
description of a symptom. And so Rai told, like, you know,
from 7:00 AM to 7:00 PM is going to be just as effective.
But humans get tired over time and so it's our ability to give
you the report on every visit and to make sure that you really
do have a documentation or details that are going to be
hard for you to remember. I.
Think it's interesting in terms of how, as you say, clinicians
are human. We have various kind of things
that we might struggle with. And so when I think one thing
that I know from my good friends as a leading psychologist here
in Sweden highlighted research around that, the best clinical
psychologists are usually the ones that are in their first
five to 10 years of their practice because they're
following the best practices. And actually one of the tricky
things sometimes is when people get a little bit too secure in
their own patterns of things where they kind of lost sight of
what's actually the best practice, the best way of doing
things. And they've used develop some
good habits, but also maybe some bad habits over the years.
And I'm just curious how you think about that as a challenge
and relating to this? Yeah, I feel like it makes me
think of statistics where you would talk about like regressing
to your own being, right? I think you're right.
Like I think in this example, right, like AI can help you
because, yeah, like I used to teach for a second year
therapist and you need to pay attention to synchrony with like
verbal and non verbal cues. And so especially when you're
like a junior therapist, you're paying attention to every single
detail, right? And so, but I think over time,
we become comfortable, we become confident in some of the ways
that we engage in therapy are beautiful.
But then sometimes we might not catch that we're actually doing
things that might be off putting to a client.
And so you're correct in that, you know, getting the AI
feedback, The AI doesn't care if you're like a first or in your
20th year as a therapist or provider, but doesn't care if
you're the attending or the fellow, right?
Like, it's going to give you that feedback in a particular
way, in a way that isn't necessarily biased to who you
are in terms of like, your training or maybe your title.
But of course, like, AI bias is a real thing that we always have
to build, you know, with that in mind.
But I do think that, yeah, I can definitely see what your friend
is talking about. And I think in this way, like,
in the same way that you would do a tune up, right?
Like your car needs a tune up. You know, we need tune ups.
I always talk about that. I think it's wonderful for
everybody, but ADB in therapy, not just when you feel like you
really need it, because it is this emotional and reflective
tune up that a therapeutic space can provide you.
So I think similarly, yeah, like an AI tool can really pick up on
things. And I would say like with our
own platform, what I do love is that we give you a report
according to, you know, certain detections, right, like ensemble
models of collaboration trust, but you get to choose.
We have 200 behaviors that we have models for.
But in addition to that, it does give you the transcript and
it'll show you like when the detections happen in the
conversation that you had. And then based on how you are
connected to our platform, we can either show you that as soon
as you click on in the transcript, it'll show you the
audio or the video feed. So it's a wonderful training
tool, right? And it's really hard to ignore
that you might have minimized something with an older patient
if the transcript and the video or audio feed shows you exactly
what you said, right? So in that way, it can really
shift how people practice. When I was a supervisor, I would
do my best to make sure, especially that junior therapist
for recording those sessions and really paying attention to what
they said, not necessarily the patient, but like how they were
responding or how they were, you know, just engaged in
conversations or if they were affirming or minimizing at
times. And I think similarly, we can do
that with an AI tool. But here you can do it 100% of
the time, right, versus only 1020% of the sessions that
you're doing? I'm also curious, just thinking
about individual differences in terms of the clinician's
response to that feedback. So if I think about, you know,
you just told me I've minimized the patient like, well, no, I
didn't. What is the full range of
responses that you see? I can definitely imagine someone
being maybe more or less open to that sort of quote UN quote
feedback. Yeah, I think it depends on the
person. It could be the case, right?
That like we pick up on something that's minimizing and
you say like, no, I don't think that it was.
And so we have the ability for you to give a thumbs down.
So on our end, we might see that as like a technology failure,
right? That like, oh, we need to go
back and like really make sure that we're capturing the breadth
of nuance and like what it means to minimize.
So you absolutely can engage with the platform to help us
build stronger models over time. But I think with anything,
right, like if you were to create like a psychometric
instrument of social support and you did that in like 1970, it's
going to look dramatically different what social support is
like in 1990, 2010 and whatnot. So I think that's one thing that
folks always should keep in mind is that whenever you're working
with AI, it's not done right. It's an evolving process and
that product should evolve with time as well.
Yeah. When it comes to the feedback
that they receive, is it only afterwards or do they also get
some form of live feedback during the session?
Our platform is able to do live, but for the most part, I don't
think therapists are necessarily ready for live feedback.
We have an NIH grant that we secured and where we are
building out LLMS that are really focused on like synchrony
and precision medicine. And the hope is to get some user
research where we can understand how our therapist engaging with
it live if we if they were to find that to be effective or
distracting. I think currently though,
because we're not really training folks in this way,
right? And I say this is somebody who
was a professor last year. So like, I know we really aren't
training people this way yet. I think with time, yeah,
probably that there will be more openness and receptiveness to
being able to have like, live coaching.
But I think currently that's just not how therapists are
trained. So I'm not sure that we're quite
ready to, like, make that match happen.
I think the technology is there, but I don't know that as humans
were there yet. All right.
I'm going to touch on a touchy subject, which I think is
probably something that you've thought about before.
One thing that you mentioned was like, we don't want to take
therapist jobs, we don't want to replace therapist, right?
I think every therapist would agree.
I think the APA would agree. We probably all agree.
But of course, when we look at the landscape of AI that's very
swiftly evolving, we see, you know, new RCT's coming out, new
tools being tested, some clinically validate, many not
clinically validated. We see people engaging with LLMS
like ChatGPT as if they were therapists.
We see these like, you know, AI health coaches, this huge,
incredible proliferation of tools, some better than others.
In many ways, it feels to me like the reality is that it's a
little bit out of our hands. Whether therapists are at least
some being replaced by artificial intelligence tools,
How do you grapple with this changing environment?
And how are you thinking about the future?
When we talk about working with an AI chat bot, I think that
there are different ways that people are developing them.
And so you could look at Wobot as an example where you have a
deterministic AI, right? And so there are scripts on the
back end. It's not generative.
And so there's like 100% control of how the AI model would
respond to you if you're talking about anxiety or loneliness or
whatnot. And then I do think that there's
other ways that people are able to add some kind of like I'm
thinking of like when you bowl right and you're a kid and you
have these like kind of, you know, the.
Bumpers. Yeah, there you go.
Right. Like, so I love the metaphor,
the visual bumpers of how you would work with, Yeah, with
generative AI. In terms of like working with
therapy chat bots. I do think that if you can have
a chat bot that is able to flag clinical risk and then make sure
that a therapist is connected immediately, right, or flagged,
then I think that's safe. But, you know, it's tough
because I don't think that we have such clear guidelines or
policies of like how these are supposed to roll out.
And I think that, you know that there are organizations like
American Psych Association or American Psychiatric Association
that are really actively working on it.
The challenge is that like, you know, LLM development is so fast
that sometimes the policies are not coming out fast enough to
really map on to the pace of innovation.
But I don't necessarily think that things are good or bad.
I think that if you are somebody who needs therapeutic support at
2:00 AM, there's not a therapist that's going to answer the phone
call, right? Like we have really clear
boundaries for reasons that are important to have, but people
are in crisis at different times.
And sometimes to have a live human is cost prohibitive.
Or maybe you live in a rural area and it is impossible to get
to a live person more immediately.
So I do think that there are times where like virtual care or
even therapy chat bots, like there may be made space for it.
But I think that we have to make sure that the people building
them are clinicians, that they do have expertise.
And if you're in alignment with APA ethics, then yeah, you don't
practice outside of your confidence, right?
It would be inappropriate for me to maybe take a client with the
severe eating disorder today because that isn't an area that
I've like really focused on in the last couple of years.
So I would imagine that there's elements of an assessment that
I'm missing, right? Or like interventions that are
just not actually helpful. So I think similarly when we're
developing these kinds of chat bots, we have to be really
specific that it's, and I saw them the paper, I think we're
talking about the same paper where they talked about working
with major depressive disorder, generalized ID disorder, and I
believe eating disorder, but a very specific type.
And so it's clear that they were really focused on like we are
doing an intake that is comprehensive and we're really
understanding what the symptomatology is.
And I imagine that they also did a comprehensive clinical risk
detection, right? So I think that there are times
where you can do it, but it has to be as careful as humanly
possible in the same way that you would engage with that level
of care in a clinic. I'll add this Therabot paper.
They also assembled custom data sets, so they didn't just Hoover
the Internet as many of our LLMS do.
So they used this data set which was based on evidence based
practices like CBT, cognitive behavioral therapy.
So we have a little more confidence that the data that's
going in is training the model in a more reliable way than some
of the other more unpredictable models out there.
Yeah. And I would say, you know, for
us, like we recently built out a feature where we are doing like
oversight of clinical outcome assessments.
So if we built out this new tool for clinical outcome
assessments, it's really important for us that we make
sure that our AI tool is able to detect that somebody is talking
about reported versus maybe apparent sadness.
And then when they are doing a sort of oversight of the scoring
accuracy, we benchmark it in with licensed clinicians.
So we always want to make sure that there's a really high level
of precision. And I will say that like we have
a much higher level of precision because it is an LLM, right?
Like it's AI then you might expect between humans.
But I think it is that like you have to really be building in a
way that puts patient safety at the heart of the work.
And that's why whenever I'm, I'm talking with clinicians about
like different AI tools, I'm always really focused on whether
there are like clinicians and especially licensed clinicians
on the team that are helping to develop the tool.
Because we're LED with ethics, right?
And we could actually get our license taken away for not
focused on ethics. And so you have to practice and
you have to really build with care.
Yeah. Given that currently what you've
kind of illustrated in terms of the support given to clinicians
in terms of having that kind of sense of someone that gives you
feedback in terms of showing you if you missed something or maybe
helping you detect signs of various kind of serious or maybe
also some more subtle things. That is really useful.
And I feel like in some ways that same guidance could be
given also to not a human, but an AI shotbot, basically giving
fair about the same support as like some form of almost double
AI team where you have a like a clinician AI and then you have
this kind of support guiding expert AI like, and it helps
make sure that the clinician is doing a good job and so on.
Have that been something you've been looking at as a additional
path to provide value towards an AI instead of a human?
In this area, we're talking about like clinical assessments
that we ask our LMM to explain, like why did you choose that by
like, why did you detect this? Like tell us behaviorally what
you picked up on. And for us, it's like an area of
transparency because you have your training data set.
And when we're building, we make sure that our clinicians also
memo, right? Like why they're choosing
particular things. And then we compare that to what
the AI is producing as well. And so you can ask your AI agent
to give you their logic of like why they chose a particular
thing or why they are maybe scoring somebody as having
moderate versus severe depression or something of that
sort. So in that way you can have
checks and balances. It's sort of how you think of
adversarial your Gans, right? Where you have your two models
that are sort of playing off of each other, doing their own
automatic checks and balances. Yeah, but you always have to
have a clinician in the loop. Like, that's so critical all the
time. But I think also because it's
like you can't really ask folks to audit themselves.
Like, we're so biased, right? And so I feel like, you know,
the LLM would likely be less biased, but at the same time,
it's like, no, this is where clinicians really matter.
So I think in two ways, right? One is that ask the AI to
explain its logic and then map that onto what clinicians are
doing. And then at the same time make
sure that clinicians are reviewing portions of the work
or especially areas where you have to handle it with more care
you. Used to be a devil's advocate
here. I guess I'll be interested to
hear your answer with like, why does clinicians matter?
Or like in what way? Because we talked about this day
of like, I think serving at the top of your license as an
example, Like it's really important that we can help
people to serve a top of their license.
And I think it would be really interesting to have a better
shared understanding of what is the things that we do as humans
in various contexts, like in a clinical context or in other
contexts where we can do the most good and add the most value
and so on. So I guess I'm just interested
to hear from your end, like, why does clinicians matter?
Like what is the most important aspects that they add to
treatment? Yeah, I do think clinicians
matter because when we talk about clinical risk or when you
talk about somebody maybe for example, or having suicidal
ideation, the thing that you really have to pay attention to
is whether it's imminent, right? So you have to very explicitly
ask somebody, like, can you go home and be safe, right?
Or you need to not be vague. You want to really ask somebody
the exact question of like, are they going to kill themselves,
right? Like you have to do these kinds
of things. And so we are trained to do
that. We're trained to understand the
nuance. We're trained to, like, remove
vagueness. We're also trained to ensure
that we really do understand, like what imminent risk means.
And I think sometimes conversational output might miss
that to some degree. And so that's where you want a
clinician in the room. I think there's also things I do
think multimodal technologies are being built now, right?
So, you know, AI is going to be like a year from now.
Our conversation could be so incredibly different, but I do
think that they're also things like affect, right?
So you're trained as a clinician to be able to see depression, to
be able to hear depression. It's not just what somebody
says, but it's also how they present.
So I think in those moments, that's where you do want a
clinician in the room. And then the final thing I'll
say is that like clinicians are humans too, and we have biases
and we're going to pick up on things that are salient to us,
not just with our lived experience, but how we are
trained theoretically. And so if you're somebody who,
right, has been part of the team that has like created a
clinically validated training data set, but you are trained as
a psychodynamic therapist, then you might be training an LLM to
be picking up on family dynamics more than maybe behaviors or
being out of alignment with values in the way that you would
if you were doing. So our theories and our
trainings matter a lot. And so that's where I think you
should have different kinds of clinicians in the loop because
it also helps to ensure that your AI is not missing things,
right, because it's been trained to only pick up on certain
characteristics. So one thing from my end when I
get this question, and I think the same is true for things like
Shachi PT will be the death of Duolingo or like whenever
there's like this kind of very sensationalist takes around, he
has developed really high capabilities.
I would say that there's a difference between capabilities
and in this case also a part of treatment is actually having the
social accountability, support and understanding of another
human being. And like knowing that it's not a
human being that cares for you and that's going to follow up
with you and that you will follow up with.
Like that is not something you can also use to replace just
because the capability might exist within an AI.
Even if it feels human, you know that it's not.
Right. But it is interesting because I
feel like Robot has put out some research, right?
They've had a lot of scientific papers where people sometimes
feel like they can say more to an AI agent because they know
they're not being judged in the same way.
And so I do think that there's that.
But then I also think that, like, you know, our company is
named empathic, right? So like our core models are
based on the common factors, which are like the key
ingredients across all different kinds of theoretical
orientations that we know really matter for behavior change.
So things like collaboration and trust and unconditional positive
regard, right? And so that, that isn't the core
of like how we built our models. So we wanted to make sure that
we could pay attention to the nuance in popular media or just
like in the ways that we talk about empathy, like in a more
common way, it's this idea that like you're a nice person,
right? Or that, you know, that you
care, but empathy really is like about, if you really pay
attention to the behaviors, it's about accurate understanding,
right? So ensuring that you really do
accurately understand what somebody is trying to convey to
you about a particular instance. And so I do think that there are
moments where sometimes the way that AI models are built,
especially, you know, because that's why training data is so
critical, that you can train your AI to be extremely
empathetic and more so maybe than a lot of humans.
So it's not just the training of like, hey, you missed this, you
know, this symptom. But it's also like, hey, the way
that you said this was out of alliance, right?
Or it's out of synchrony, it was not empathetic.
And so here's coaching. Here's a report for how you
could do that better. Yeah, I love that study from way
back when with the Reddit doctor responses versus the generative
AI. People found the Gen.
AI response to be more empathetic than the real
doctors. And of course, you know, many
studies have been done since then really replicating this
finding. So I find that extremely
interesting, knowing that the machine is not experiencing
empathy, but it is conveying empathy.
It is mimicking empathy to I guess like more convincing
degree than the humans. Yeah.
And but I would say right, like that's where like AI is not a
black box. Like you have training data,
right? And you train AI like built on
that data. So that's why it matters that
the team building it is diverse because it needs to capture the
breath of how people experience the world and what they respond
to. There's a study that I loved
they did use machine learning to track like Reddit responses
around mental health and they found that.
You know, according to like the typical like, DSM criteria for
depression, it was missing depression for black women
because black women express different kinds of ways of like,
whether that be guilt, shame, somatic complaints that aren't
typically captured in the way that we think about depression.
But because you would imagine that like the training data that
went into the DSM, like the same idea wasn't built around having
enough diversity. But I think that there's really
beautiful ways that you can use machine learning.
And I think you asked this earlier, Sam, but it's not just
like what it was said, but it's how it was said.
And I think that's the thing that I care a lot about, right?
Cuz I've had a long history of being a clinical researcher and
I care a great deal about like how do you work with different
kinds of data? And I've always been somebody
who goes back to like qualitative, quantitative mixed
methodologies. Because I feel like you can with
quantitative, you can tell that a relationship exists, but you
need qualitative to go underneath.
And like, even if you find something is mediating, you need
to understand like what exactly is happening here.
And so I think similarly, like if you use natural language
processing like we use, then it's like paying attention to
conversational cues. And it's not just like a yes or
no that something happened, but it's more so how are people
communicating guilt or how are people communicating empathy and
being able to get at that kind of nuance in communications in a
way that is really hard to do as a human 100% of the time.
Yeah. And especially as you say, you
know, being human that engages with like 20 patients per day
with a limited time, like all these things contribute, I think
to what we talk about empathy is maybe on the best of days, in
the best of moments, you know, humans will be most AI.
But on a regular day and it's all of the other things going
on, it's hard to maintain the same level of empathy.
You know, like they do talk about, like, decisional
exhaustion. So like humans, we do bias all
the time. I do think that there's ways
that AI can really help humans to, yeah, just to do work that
is more precise, but also just more comprehensive that we've
ever been able to do before. So talking about bias,
interestingly, we talked a lot about algorithms going to be
biased, but it's often times, I think the interesting thing is
especially for us as a paper scientist to see both side of
that, like where this human bias, but there's also algorithm
bias and kind of the best of worlds.
You want to kind of mitigate both of them, like you said,
have good training data that supports good algorithms that
are better trained on a wide range of diverse data, but also
understanding how we can support humans in being less biased.
What do you think about the most effective ways to mitigate both
of those sites? Like mitigate bias in the LLMS
but also mitigate bias in the humans?
Yeah. But first I'll say that I think
that they're experts out there, right, who know a lot more about
this than I do. And I still have a lot of
learning to do. But if you're a researcher, I
think that there's ways that like we know how to develop
psychometric instruments, right? Like we understand that.
And I think sometimes using the same logic where it is things
like have you been comprehensive in understanding the content?
And so that's kind of similar to like training data.
Do you have diverse people like that comprise your training
data? But do you also have diverse
people annotating your data to make sure that when you say
trust that is, is it just trust for a middle-aged white man,
right. But it's trust for the breadth
of what human experience can be. And sometimes that can be
impossible to do, right? Cuz it's like you can't get that
for every single person. And we know that intra
variability is oftentimes larger than intergroup.
But I think also you can test the performance of your AI
precision across groups, right? So if you're benchmarking it
against clinicians, you can see, is our AI performing with bias?
Is it more precise for men than women?
Is it more precise for white versus Asian communities or
whatnot? So those are like 2 of the ways
that I think about it more immediately is I always am
paying a great deal of attention to the training data.
And then addition to that, once we have working models, right,
then we do test the precision against different kinds of
groups to see whether they are more or less precise.
Like are they demonstrating bias?
But then I think that there's other ways that you can do it
with user research as well. And I do think that there's
other elements as well. You know, the NIH grant that we
got really was built around synchrony, but also like
elements of cultural responsiveness too.
And so I think that's something that I'm so excited to be able
to focus on in the next few months and to kind of understand
like, what does it mean, right, to really pay attention to like
maybe miss cultural opportunities or to not evoke
how culture might impact how somebody is experiencing their
mental health condition or responding to intervention.
But I love this idea of doing this because I feel like so far
right, like AI amongst clinicians is really focused on
like note taking. And now we're maybe seeing
things, right, of like, you know, deterministic ways of like
how we might respond to depression, which is great.
Like that's a beautiful evolution and I love seeing it.
But I also think that we can use AI in ways that that we've never
done before, right? Like understanding patterns of
maybe how rural youth or youth in like certain communities,
like maybe trans youth, like talk about their mental health
that we can understand like communication patterns in a way
that we've never done before. And so I think that it's about
doing precision medicine, but precision medicine means that
you really are paying attention to like individual differences
and doing your best to really develop interventions that like
are going to be most effective for the patient in front of you,
not like a general kind of intervention that's supposed to
be effective for everybody. I do think that AI can allow us
to do that in a way that we haven't done before.
All right, it is time to move on to our quick fire round, which
we call to AI or not to AI. Are you ready?
I don't know, yes. You must be, you know, it's fun.
You're going to like it. We're just going to give you a
bunch of tasks and you're going to tell us whether you think
it's well suited to AI or not. OK.
All right, first one a digital twin of yourself to ask for
advice. No, I think I already have my
journal for that, so no. OK.
But also because I feel like that's a pretty biased advice,
right? Maybe it's what you want to
hear. Exactly.
I mean. Just give me the permission,
please, OK? A playbook of cultural
expressions of distress. I like that with a caveat
though. I was like who built it?
But yes, I like the theoretically.
I like the idea of it. OK, what about this VR exposure
therapy that is AI powered? Basically providing some form of
personalized exposure scenarios for patients based on their
various phobias, PTSDS and social anxieties.
Yes, and that's already around. And I think it's funny that you
mentioned your journal because the next one is a fact checking
journal. Fact checking I would be really
worried about the training data there.
So I don't know who's doing the fact checking but I do love I
have a journal I use. I think it's reflection and I
love it cuz it also generates AI like gets me deeper.
You can have AI prompts that make you expand on certain
things. But you say, I had a great time
at the library today and it says actually your mood was like 4
out of 10. It's more so like what made it
good, like what would have improved it for the next time,
you know? Yeah, and it brings us maybe to
this one. So predictive nostalgia machine,
basically an AI that projects what you'll be nostalgic about
in the future based on your kind of previous experiences.
So like it knows that you know you've had some experience in
the past. So like it tries to kind of let
you know, like, hey, you should really appreciate this moment
because this is something you'll be nostalgic about in a few
years time. Yeah, the predictive analytics,
Sure, why not? Sounds like it's AI for
mindfulness and I'm about that. I like it.
OK, last one, this is the emotional soundtrack generator.
So this is an AI, listens to your daily conversations,
basically, you know, takes a peek at your life, generates a
soundtrack that reflects your emotional state.
Yes, but I think that I would want it to also generate
soundtracks that Get Me Out of a funk.
They can tell that I'm in one. Yes, good.
There's a lot of soundtracks there, like on Spotify for
example, where it's like you're sad or happy, but there's not
really one from like from sad to excited.
From sad to happiest. Yeah, I love that.
Yeah. And my niece had just the
funniest. I remember when she was young,
at one time, I was like, why are you sad?
She's like, well, the music is sad for that reason.
I feel sad and I want to keep listening to it.
And I was like, OK, yeah. So chicken or egg?
I'm not sure. Right.
And I think you can get us into a mood or sometimes get us out
of 1. So.
Yeah. Yeah, love that.
OK. Well, you made it to the final
question now, which is what is your most controversial opinion
about AII? Would say that I think that
clinicians can be better with AI than without.
I think that's what I would say. And I think that I have been
converted to that now, being able to see that AI can do 100%
oversight. But I think also, yeah, having
founded a clinic, being the slow psychologist for a minute there,
I now see that AI can really help with supervision and yeah,
and quality care in a way that humans, just because life
happens, not just that we get tired, but sometimes we have
tough moments in our lives that can compromise, like our
precision. And so I think that AI can be a
really wonderful tool, but I think it's a tool that that we
need more than we think we do. So you're really let's embrace
AI as collaborator. Let's embrace ethical, robust AI
as collaborator. I love it.
Well thank you, this was so much fun.
Wonderful. All right.
Well, thank you all so much. And that's a wrap.
You've been listening to the Behavioral Design podcast
brought to you by Habit Weekly and Nuanced Behavior.
Sam and Alene tell me. This season is packed with
incredible insights about behavioral design and AI, so be
sure to subscribe and share the podcast with your friends,
though you might want to keep it away from your enemies.
In case you haven't noticed, I'm an AI voice.
Yep, pretty crazy. Quite the improvement since last
season's AI outro, don't you think?
If you'd like to collaborate with us at Nuance Behavior,
where we use behavioral design to craft digital products with
Nuance, e-mail us at hello@nuancebehavior.com or book
a call directly on our website, nuancebehavior.com.
A special thanks to the amazing Dave Pizarro for our show music
and to Mei Chen Yap and April English for their help in
producing and publishing this episode.
Thanks again for tuning in. We'll be back soon with another
exciting conversation where behavioral design and AI
Intersect happens to. Mugatroid.