1
00:00:03,280 --> 00:00:05,880
Hello and welcome to the 
Behavioral Design Podcast. 

2
00:00:05,960 --> 00:00:08,680
This season we're diving into 
the intersection of behavioral 

3
00:00:08,680 --> 00:00:11,080
science and AI. 
We want to make sense of the 

4
00:00:11,080 --> 00:00:14,080
state of AI, from understanding 
how humans interact with 

5
00:00:14,080 --> 00:00:17,520
intelligent systems to using AI 
to do behavioral design itself. 

6
00:00:17,720 --> 00:00:21,280
I'm Aline Holsworth, a health 
tech advisor specializing in AI 

7
00:00:21,280 --> 00:00:23,880
and product design. 
Over the past 15 years, I've 

8
00:00:23,880 --> 00:00:26,720
been crafting human centered 
products with behavioral science

9
00:00:26,720 --> 00:00:29,160
at the core. 
At Apple, I LED Behavioral 

10
00:00:29,160 --> 00:00:32,080
Science for Health AI, designing
and launching AI powered 

11
00:00:32,080 --> 00:00:34,440
features to help users reach 
their health goals. 

12
00:00:35,160 --> 00:00:37,120
And I'm Samuel Sultzer, your 
second Co host. 

13
00:00:37,360 --> 00:00:40,680
I'm a behavioral strategist 
specializing in hybrid formation

14
00:00:40,920 --> 00:00:43,200
and designing products that 
drive long term baby change. 

15
00:00:43,640 --> 00:00:46,880
I work with leading tech 
organizations integrating AI to 

16
00:00:46,880 --> 00:00:48,200
scale behavioral design for 
good. 

17
00:00:48,800 --> 00:00:52,160
And I'm also the founder of Baby
Bites, a dedicated community on 

18
00:00:52,440 --> 00:00:56,240
behavioral science and AI. 
Quick word on Nuance Behavior 

19
00:00:56,240 --> 00:00:59,640
where we help organizations 
build impactful digital products

20
00:00:59,640 --> 00:01:02,640
using behavioral design. 
We only take on a few clients at

21
00:01:02,640 --> 00:01:05,200
a time to ensure the highest 
level of quality for our 

22
00:01:05,200 --> 00:01:07,400
tailored evidence based 
solutions. 

23
00:01:08,280 --> 00:01:11,080
If you'd like to become one of 
our special projects, e-mail us 

24
00:01:11,120 --> 00:01:14,760
at hello@nuancebehavior.com or 
we could call directly on our 

25
00:01:14,760 --> 00:01:23,280
website, nuancebehavior.com. 
Hey, Lin. 

26
00:01:28,760 --> 00:01:29,280
Hello. 
Hello. 

27
00:01:31,480 --> 00:01:35,280
Hey. 
I feel like I owe you a thank 

28
00:01:35,280 --> 00:01:36,200
you. 
You know why? 

29
00:01:37,640 --> 00:01:40,480
Tell me, I mean, I can think of 
a lot of reasons. 

30
00:01:42,920 --> 00:01:46,960
Where do we start? 
That is true. 

31
00:01:47,640 --> 00:01:52,720
No, I, I received a package that
I was like, what is this? 

32
00:01:53,640 --> 00:01:57,880
I didn't order this. 
And then I looked inside and 1st

33
00:01:57,880 --> 00:02:02,360
it took me a few seconds for 
like things to click, but then 

34
00:02:03,000 --> 00:02:07,080
I, I got so happy, like you made
me so happy. 

35
00:02:07,160 --> 00:02:10,479
And for the listener, I'm 
actually wearing the thing that 

36
00:02:10,479 --> 00:02:17,040
you sent and it's AT shirt that 
probably boasts Kremlin duvet 

37
00:02:17,040 --> 00:02:20,880
advertising, which probably for 
like every listener means 

38
00:02:20,880 --> 00:02:24,320
nothing. 
But for me it means a lot. 

39
00:02:24,320 --> 00:02:26,640
So thank you. 
Amazing, amazing. 

40
00:02:26,640 --> 00:02:28,400
Do you want to share the 
background? 

41
00:02:28,800 --> 00:02:31,400
What Kremlin advertising is 
duvet Kremlin? 

42
00:02:31,400 --> 00:02:33,320
Yeah, yeah, yeah. 
Duvet Kremlin. 

43
00:02:33,520 --> 00:02:38,440
So I have mentioned I think in 
an episode with I think it was 

44
00:02:38,440 --> 00:02:43,080
Gordon Pennycook, where I 
mentioned that I'm a die hard 

45
00:02:43,160 --> 00:02:49,440
comedy fan in various ways. 
And I I enjoy comedy in the best

46
00:02:49,480 --> 00:02:52,440
of forms from stand up to TV 
shows and so on. 

47
00:02:53,000 --> 00:02:58,960
And what has been my like really
favorite find in the last few 

48
00:02:58,960 --> 00:03:02,240
years has been this really, 
really fantastic show called 

49
00:03:02,240 --> 00:03:05,560
Detroiters. 
And it's a show that two seasons

50
00:03:05,560 --> 00:03:09,640
for some reason got cancelled. 
But it's with Tim Robinson, who 

51
00:03:09,640 --> 00:03:12,800
has been doing his thing with I 
think should leave on Netflix 

52
00:03:12,800 --> 00:03:17,120
and became really big with that.
And it's such a really, I don't 

53
00:03:17,120 --> 00:03:23,440
know, heartfelt comedy with this
struggling to advertisers who 

54
00:03:23,440 --> 00:03:27,000
are kind of representing the 
city of Detroit and their only 

55
00:03:27,000 --> 00:03:31,560
jobs is basically to make ads 
for like the local, you know. 

56
00:03:32,800 --> 00:03:35,120
Like the trampoline store the 
mattresses. 

57
00:03:35,880 --> 00:03:39,120
Exactly, like really those like 
shitty local commercials you 

58
00:03:39,120 --> 00:03:41,880
will see on TV? 
Like they are the ones tasked 

59
00:03:41,880 --> 00:03:44,600
with making those kind of shitty
commercials and they're the name

60
00:03:44,600 --> 00:03:47,400
of the agency is Kremlin duvet 
advertising. 

61
00:03:47,960 --> 00:03:52,240
And so I think I recommended my 
fellow Nuance Co workers at some

62
00:03:52,240 --> 00:03:54,160
point like, hey, you should 
really check out this show. 

63
00:03:54,320 --> 00:03:57,120
And then I never thought about 
it until you sent me this. 

64
00:03:57,120 --> 00:03:59,120
So. 
Well, I take your 

65
00:03:59,120 --> 00:04:02,560
recommendations very seriously. 
So I not only watched the show, 

66
00:04:02,560 --> 00:04:05,760
but you know, I kind of enjoyed 
it at times. 

67
00:04:06,320 --> 00:04:08,760
Wow, that says a lot. 
Did you have a favorite episode 

68
00:04:08,760 --> 00:04:12,040
or favorite thing? 
The Devereaux wigs is definitely

69
00:04:12,040 --> 00:04:14,200
going to be my favorite. 
This is where they really 

70
00:04:14,200 --> 00:04:18,200
emphasize that the wigs from 
this store are not made from the

71
00:04:18,200 --> 00:04:21,760
hair of dead people. 
And of course it turns out that 

72
00:04:21,760 --> 00:04:26,160
they are. 
And oh gosh, it's an incredible 

73
00:04:26,160 --> 00:04:28,680
ad that they create. 
And I think here we're going to 

74
00:04:28,680 --> 00:04:30,760
have to insert the Jingle if we 
can find it. 

75
00:04:35,320 --> 00:04:40,600
Devereaux wigs. 
We guarantee our wigs aren't 

76
00:04:40,600 --> 00:04:49,640
made from hair off dead bodies. 
Yeah, we have to Devaru. 

77
00:04:50,280 --> 00:04:57,200
Devaru, So I do have to say I 
felt quite guilty that you got 

78
00:04:57,200 --> 00:05:02,360
me two actual presents, one 
recommended by ChatGPT and one 

79
00:05:02,600 --> 00:05:05,880
that you chose yourself after, 
you know, small 

80
00:05:05,880 --> 00:05:09,240
misunderstanding. 
And yeah, you know, I only got 

81
00:05:09,240 --> 00:05:12,280
you a hypothetical gift or two 
hypothetical gifts. 

82
00:05:12,280 --> 00:05:14,440
So I thought, you know, I can 
deliver. 

83
00:05:14,520 --> 00:05:16,840
I can get you a real gift. 
So there we go. 

84
00:05:16,840 --> 00:05:19,040
And so you did. 
And you know that hypothetical 

85
00:05:19,040 --> 00:05:21,800
gifts were nice too, but it's 
always nice to actually 

86
00:05:22,120 --> 00:05:24,760
obviously get a real 1. 
And this is another example of 

87
00:05:24,760 --> 00:05:27,080
like, reciprocity is the way to 
go. 

88
00:05:27,080 --> 00:05:29,400
If you want to receive a gift, 
give a gift. 

89
00:05:29,520 --> 00:05:31,720
Yeah, so now it's your turn to 
get me a gift. 

90
00:05:32,240 --> 00:05:34,400
Oh, it works not OK. 
This is how it goes. 

91
00:05:34,440 --> 00:05:37,440
It just continues forever. 
Yeah, but it's great to have you

92
00:05:37,440 --> 00:05:40,560
back as well. 
You were away on a little bit of

93
00:05:40,560 --> 00:05:46,720
a trip and was not with me and 
Jared as we were kind of trying 

94
00:05:46,720 --> 00:05:50,000
to make sense of the past year 
and the future as well. 

95
00:05:50,400 --> 00:05:54,800
And so, yeah, just kind of 
quickly, what is your current 

96
00:05:54,800 --> 00:05:58,480
temperature for 2025? 
Any thoughts about the state of 

97
00:05:58,480 --> 00:06:03,080
things? 
2025 I barely have gotten over 

98
00:06:03,080 --> 00:06:08,160
2024. 
I'm mostly terrified of 2025 in 

99
00:06:08,160 --> 00:06:12,400
the current political 
environment and lots of terrible

100
00:06:12,400 --> 00:06:14,920
things happening there, to put 
it lightly. 

101
00:06:16,000 --> 00:06:19,000
And you know, like when it comes
to behavioural science and 

102
00:06:19,000 --> 00:06:22,800
science in general, not looking 
very good when it comes to AI, 

103
00:06:23,440 --> 00:06:27,920
well, you know, you could argue 
good in the sense that 

104
00:06:27,920 --> 00:06:31,720
innovation will soar with no 
regulation. 

105
00:06:32,520 --> 00:06:34,800
Perhaps some constraints are 
actually helpful. 

106
00:06:34,800 --> 00:06:36,480
Well, there is the EU. 
What? 

107
00:06:36,640 --> 00:06:39,200
What are you talking about? 
The the US is everything. 

108
00:06:44,760 --> 00:06:48,400
So you kind of very quickly said
something that I wanted to go 

109
00:06:48,400 --> 00:06:50,880
back to, which was something 
about science and that you're 

110
00:06:50,880 --> 00:06:52,440
like things looking dark for 
science. 

111
00:06:52,440 --> 00:06:53,880
What do you mean by that? 
Like what does that mean for 

112
00:06:53,880 --> 00:06:54,560
you? 
Why do you think that? 

113
00:06:55,080 --> 00:06:59,120
Oh, just the pause on all 
funding that's currently taking 

114
00:06:59,120 --> 00:07:01,360
place. 
Hopefully this gets overwritten.

115
00:07:01,400 --> 00:07:03,320
In the US. 
In the US, yes. 

116
00:07:03,520 --> 00:07:05,480
Yeah, we're recording this end 
of January. 

117
00:07:05,680 --> 00:07:08,960
And yeah, there's been a lot of 
things happening policy wise in 

118
00:07:08,960 --> 00:07:14,360
the US, and I'm mostly receiving
updates through you, my sister 

119
00:07:14,720 --> 00:07:17,560
and Jon Stewart. 
So that's all I know about what 

120
00:07:17,560 --> 00:07:20,080
happens in the US. 
All pretty good sources. 

121
00:07:21,080 --> 00:07:23,880
Yeah, that's good to know. 
In terms of where you're at, I 

122
00:07:23,880 --> 00:07:27,680
would say maybe I can kind of 
counterweight what you're 

123
00:07:27,680 --> 00:07:31,200
feeling with some level of 
optimism and excitement for 25. 

124
00:07:31,200 --> 00:07:35,240
I don't know exactly where I 
found that, but either I 

125
00:07:35,240 --> 00:07:37,320
currently feel quite excited for
this year. 

126
00:07:38,400 --> 00:07:41,640
Oh, OK. 
And specifically what what about

127
00:07:41,640 --> 00:07:47,680
it? 
No, like I'd love something to 

128
00:07:47,680 --> 00:07:48,880
latch onto. 
That would be great. 

129
00:07:49,800 --> 00:07:53,200
Give me anything. 
Anything. 

130
00:07:54,560 --> 00:08:02,400
No, but I, I do think there is 
a, you know, a possibility of 

131
00:08:02,600 --> 00:08:05,720
finding good in any situation I 
guess. 

132
00:08:05,720 --> 00:08:10,560
And I would say 2024 was a 
pretty dark year for me in some 

133
00:08:10,560 --> 00:08:12,760
ways. 
I think I was struggling with 

134
00:08:12,960 --> 00:08:16,600
the various things in terms of 
just like existentially last 

135
00:08:16,600 --> 00:08:19,200
year. 
And I used to think I have come 

136
00:08:19,400 --> 00:08:24,920
to some level of acceptance of 
what is and then finding some 

137
00:08:24,920 --> 00:08:30,480
form of excitement and 
motivation to work on what I can

138
00:08:30,480 --> 00:08:34,200
control basically. 
So your optimism is coming from 

139
00:08:35,159 --> 00:08:37,600
we've hit rock bottom, it 
couldn't get any worse. 

140
00:08:37,840 --> 00:08:40,440
It must. 
It can only go up from here. 

141
00:08:40,840 --> 00:08:42,840
Is that what you're telling me? 
No, no. 

142
00:08:42,840 --> 00:08:47,480
But yeah, there's things like 
there I am excited about what 

143
00:08:47,480 --> 00:08:49,320
we're kind of working on with 
new ones. 

144
00:08:49,320 --> 00:08:54,520
I'm excited about the podcast, 
I'm excited about also while for

145
00:08:54,520 --> 00:08:57,040
somehow weekly, we're taking a 
break a little bit, but we're 

146
00:08:57,040 --> 00:09:00,960
doing some really interesting 
things that we'll hopefully get 

147
00:09:00,960 --> 00:09:04,760
back to doing. 
We're also with Behavior Bites, 

148
00:09:04,880 --> 00:09:09,680
that initiative that I'm doing 
together with Frederick, which 

149
00:09:09,680 --> 00:09:12,960
is kind of this community for 
people specifically within 

150
00:09:12,960 --> 00:09:16,600
behavioral science and AI. 
Kind of hoping for those worlds 

151
00:09:16,600 --> 00:09:18,640
to meet outside of this podcast,
of course. 

152
00:09:19,200 --> 00:09:23,520
And yeah, I do think like all of
those various things has given 

153
00:09:23,520 --> 00:09:30,240
me some level of excitement. 
So I have a a couple of hot 

154
00:09:30,240 --> 00:09:35,800
takes as we sort of think about 
2024 looking back, especially, 

155
00:09:35,840 --> 00:09:39,280
you know, at this intersection 
of AI and behavioral science and

156
00:09:39,280 --> 00:09:42,080
then also like moving forward in
2025. 

157
00:09:42,080 --> 00:09:46,720
So we often talk about this sort
of slathering of AI on 

158
00:09:46,720 --> 00:09:48,880
everything. 
It was just like this sudden 

159
00:09:49,040 --> 00:09:51,920
mindless frenzy where everyone 
was like. 

160
00:09:52,120 --> 00:09:55,160
We've got to. 
Put a chat bot on the thing 

161
00:09:55,480 --> 00:09:58,000
everywhere. 
I think there's sort of two 

162
00:09:58,240 --> 00:10:02,360
effects of this, kind of like 
the positive and the negative, 

163
00:10:02,360 --> 00:10:05,360
the positive and you know, you 
can debate whether it's actually

164
00:10:05,360 --> 00:10:08,080
positive. 
But one is that users are 

165
00:10:08,080 --> 00:10:10,560
becoming more accustomed to chat
bots. 

166
00:10:10,560 --> 00:10:16,400
And so there is this growing 
acceptability where you can use 

167
00:10:16,520 --> 00:10:19,960
AI and it's not seen as this 
weird thing that people aren't 

168
00:10:19,960 --> 00:10:21,680
trusting. 
I think like trust has really 

169
00:10:21,680 --> 00:10:26,320
increased overtime so that this 
just become a part of our world 

170
00:10:26,320 --> 00:10:30,160
and we're just accustomed to it.
But on the flip side, with that 

171
00:10:30,160 --> 00:10:34,120
acceptance have sort of raised 
the standard. 

172
00:10:34,320 --> 00:10:38,960
We have now higher expectations 
of like what we expect out of a 

173
00:10:38,960 --> 00:10:41,720
chat bot. 
So if you go to any, you know, 

174
00:10:41,720 --> 00:10:46,440
sort of random customer service 
chat bot, you now expect it to 

175
00:10:46,440 --> 00:10:49,920
talk to you like ChatGPT does 
and to be able to interact with 

176
00:10:49,920 --> 00:10:52,680
it and get things done. 
And, and you want your agents, 

177
00:10:52,920 --> 00:10:57,320
your AI agents to be useful and 
productive and to be able to 

178
00:10:57,320 --> 00:11:00,280
accomplish things. 
And that I think is a challenge 

179
00:11:00,280 --> 00:11:04,200
because the technology is not 
quite there yet that you can 

180
00:11:04,200 --> 00:11:07,560
like really, really succeed at 
all of these tasks. 

181
00:11:08,200 --> 00:11:11,080
There are still a lot of dumb 
bots out there with the smart 

182
00:11:11,080 --> 00:11:13,960
bots. 
I think the trend is going to be

183
00:11:13,960 --> 00:11:18,360
that people get even more 
frustrated with the dumb bots 

184
00:11:18,400 --> 00:11:22,480
and maybe are more impatient 
with them and are less willing 

185
00:11:22,480 --> 00:11:25,160
to engage with them as well 
because they think they have 

186
00:11:25,160 --> 00:11:28,640
more experiences of not 
succeeding with their goals 

187
00:11:28,800 --> 00:11:30,920
using them. 
So that's one prediction. 

188
00:11:31,040 --> 00:11:34,280
That's a really good take. 
Why thank you, I did call it a 

189
00:11:34,280 --> 00:11:36,680
hot take. 
Yeah, it is. 

190
00:11:36,840 --> 00:11:40,760
And to add to that, actually 
what's been on my mind literally

191
00:11:40,760 --> 00:11:46,280
today is something interesting 
maybe to build on that is as 

192
00:11:46,280 --> 00:11:50,760
we're getting frustrated with 
bots and dumb bots, I think that

193
00:11:50,760 --> 00:11:56,920
will also kind of lead to be 
more frustrated dumb humans and 

194
00:11:57,360 --> 00:12:02,120
faulty humans because yeah, So 
what I've seen in a project that

195
00:12:02,120 --> 00:12:04,160
I'm working on, I wanted to 
actually share this with you 

196
00:12:04,640 --> 00:12:09,240
anyway, is that in this project 
is a digital health related 

197
00:12:09,240 --> 00:12:13,080
project where patients are 
communicating with for the most 

198
00:12:13,080 --> 00:12:16,440
part, actually humans, but it's 
text based communication. 

199
00:12:16,800 --> 00:12:22,640
And so they're in A and they're 
communicating via text, but for 

200
00:12:22,640 --> 00:12:25,560
the most part on the other side,
other than maybe some initial on

201
00:12:25,560 --> 00:12:29,080
boarding template kind of 
messages they receive, they 

202
00:12:29,080 --> 00:12:30,760
currently receive like a lot of 
messages from humans. 

203
00:12:31,880 --> 00:12:36,120
Which Who takes so long to type?
Humans are so slow. 

204
00:12:37,480 --> 00:12:44,200
They're slow and they even also 
because of how maybe smart the 

205
00:12:44,200 --> 00:12:49,320
smart bulbs have become, then 
humans are more likely to be 

206
00:12:49,320 --> 00:12:53,640
appearing as bad bots or like, 
or if you give a credit to 

207
00:12:53,640 --> 00:12:55,200
humans, maybe they're seen as 
smart bots. 

208
00:12:55,200 --> 00:12:57,560
Because basically what we've 
seen in our kind of feedback 

209
00:12:57,560 --> 00:13:01,040
forms or like from users is that
they're complaining about that 

210
00:13:01,040 --> 00:13:03,880
some of the people they're 
interacting with are bots. 

211
00:13:03,880 --> 00:13:05,240
They're actually interacting 
with humans. 

212
00:13:05,840 --> 00:13:09,560
And it's because they probably 
just assume that if someone is 

213
00:13:09,560 --> 00:13:14,720
like messaging me, even if it 
says it's a human, it's in 2025.

214
00:13:14,720 --> 00:13:17,800
Like it's probably a short spot.
And I think that's the challenge

215
00:13:17,800 --> 00:13:20,800
of like, how do we navigate 
that? 

216
00:13:20,800 --> 00:13:25,840
Because then basically what 
becomes the absurd, potentially 

217
00:13:25,840 --> 00:13:30,040
challenging thing to have to do 
is either you have to like find 

218
00:13:30,040 --> 00:13:32,840
ways to strengthen that signal 
to like make it even more human 

219
00:13:32,840 --> 00:13:36,720
in a way where you're like put 
them on a call or you put them 

220
00:13:37,000 --> 00:13:39,400
in video mode or text, like a 
voice mess. 

221
00:13:39,760 --> 00:13:44,360
Something that even assists or 
even better formulated message 

222
00:13:44,360 --> 00:13:46,840
that has maybe some 
imperfections or something like 

223
00:13:46,840 --> 00:13:48,280
that really signals that as a 
human. 

224
00:13:48,800 --> 00:13:53,880
Or you might have to use AI to 
make a human message. 

225
00:13:54,040 --> 00:13:55,640
See more human. 
Yeah. 

226
00:13:56,080 --> 00:13:58,360
I mean, there are companies that
are literally doing that. 

227
00:13:58,480 --> 00:14:01,600
I think we may talk to one of 
them this season. 

228
00:14:03,160 --> 00:14:04,960
Exactly. 
And that's kind of what we're 

229
00:14:04,960 --> 00:14:07,520
kind of experiencing. 
So that is kind of where we're 

230
00:14:07,520 --> 00:14:09,880
finding ourselves in 2025, which
is kind of absurd. 

231
00:14:10,120 --> 00:14:12,320
It's kind of strange. 
It truly is. 

232
00:14:12,320 --> 00:14:17,160
But yeah, great take. 
And I would be remiss not to 

233
00:14:17,400 --> 00:14:20,840
give you the opportunity for 
another hot take, and that is 

234
00:14:21,560 --> 00:14:26,000
regarding deep seek. 
Obviously, Deep Seek, if anyone 

235
00:14:26,000 --> 00:14:31,280
hasn't followed the news, it's 
been this breaking news kind of 

236
00:14:31,360 --> 00:14:35,760
thing where there's been a team 
based in China that's developed 

237
00:14:35,760 --> 00:14:37,560
this open source model called 
Deep Seek. 

238
00:14:37,920 --> 00:14:41,880
And the news has centered around
the fact that it performs quite 

239
00:14:41,880 --> 00:14:45,840
closely to open AI and for 
example, an tropics cloud 

240
00:14:46,400 --> 00:14:50,920
models. 
And it is about I think a 30th 

241
00:14:50,920 --> 00:14:53,520
of the price or like it's very, 
very cheap in comparison. 

242
00:14:53,520 --> 00:14:58,360
And again, it is open source, 
but the online version, the one 

243
00:14:58,360 --> 00:15:04,000
that's hosted online is a shot, 
but that is still very much like

244
00:15:04,560 --> 00:15:08,680
with undertones of Chinese and 
censorship for example. 

245
00:15:08,760 --> 00:15:11,680
You can't access any information
about Tiananmen Square, for 

246
00:15:11,680 --> 00:15:13,800
example. 
Or ask about Taiwan. 

247
00:15:13,960 --> 00:15:17,720
So yeah, I'm interested here. 
Elaine what What has been your 

248
00:15:17,720 --> 00:15:20,360
take so far? 
Yes, so definitely breaking 

249
00:15:20,360 --> 00:15:21,920
news. 
All of the headlines have 

250
00:15:22,000 --> 00:15:26,800
asserted that the US dominance 
in AI has been upended. 

251
00:15:26,800 --> 00:15:31,680
You know, NVIDIA stock has gone 
down just like down, down, down 

252
00:15:31,680 --> 00:15:34,640
to the ground. 
I think that is probably not a. 

253
00:15:34,720 --> 00:15:40,240
And yeah, I think one of the big
concerns, of course, is if you 

254
00:15:40,240 --> 00:15:43,640
think that the risk of 
deregulation is a little bit 

255
00:15:43,640 --> 00:15:46,240
scary under a Trump 
administration, If you just 

256
00:15:46,560 --> 00:15:51,120
think about what that means, if 
China is leading the AI sector, 

257
00:15:51,120 --> 00:15:55,400
that probably with 0 regulation 
at all, or or at least not in 

258
00:15:55,400 --> 00:15:59,760
terms of, you know, safety and 
ensuring the quality of the 

259
00:15:59,760 --> 00:16:02,280
output. 
Also these this model of deep 

260
00:16:02,280 --> 00:16:06,680
seek is trained on the output 
of, you know, the other existing

261
00:16:06,680 --> 00:16:09,960
models. 
So it doesn't seem, you know, 

262
00:16:09,960 --> 00:16:13,520
unless they can continue doing 
that doesn't seem sustainable. 

263
00:16:13,720 --> 00:16:17,840
Yeah, that's also funny meme 
around this where basically Open

264
00:16:17,840 --> 00:16:22,000
AI are upset that it trained 
their model based on Open AI, 

265
00:16:22,000 --> 00:16:24,760
but obviously Open AI and all 
the other models have also been 

266
00:16:24,760 --> 00:16:27,840
training their models unlike 
others material. 

267
00:16:28,240 --> 00:16:31,480
I have not seen that meme, but 
yeah it's hard to feel any 

268
00:16:31,480 --> 00:16:33,760
sympathy for open AI. 
Honestly. 

269
00:16:34,840 --> 00:16:38,840
I can see this putting a lot of 
positive pressure to become more

270
00:16:38,840 --> 00:16:43,800
efficient on on the more US 
based very inefficient models 

271
00:16:43,800 --> 00:16:46,480
out there. 
I also see it doing very 

272
00:16:46,480 --> 00:16:53,720
negative things for this crazy 
AI arms race where you know all 

273
00:16:53,720 --> 00:16:58,200
safety, privacy considerations, 
Everything is just like thrown 

274
00:16:58,200 --> 00:17:02,400
to the wayside in order to get 
there first, wherever there is. 

275
00:17:02,400 --> 00:17:05,040
Yeah, and it's very worse on 
them in terms of feeling not 

276
00:17:05,040 --> 00:17:08,119
really that there are adults at 
the wheel for some of these 

277
00:17:08,119 --> 00:17:10,280
things. 
Like it just feels that they're 

278
00:17:10,280 --> 00:17:13,400
somewhat aware of that they're 
in this arms race. 

279
00:17:13,400 --> 00:17:18,000
But at the same time they don't 
have enough kind of capability 

280
00:17:18,000 --> 00:17:23,920
to really get out of the game 
theory state of like competition

281
00:17:23,920 --> 00:17:25,240
basically. 
And they're just kind of like 

282
00:17:25,640 --> 00:17:28,040
running towards something that 
they're not really sure of as 

283
00:17:28,079 --> 00:17:29,720
long as you. 
Keep running. 

284
00:17:30,880 --> 00:17:37,080
I think that sort of relates to 
one final thought that I have 

285
00:17:37,080 --> 00:17:40,320
about what this landscape is 
going to look like going 

286
00:17:40,320 --> 00:17:43,640
forward. 
And I think that there will be 

287
00:17:44,000 --> 00:17:49,080
maybe as a byproduct of this 
slathering of AI on everything 

288
00:17:49,400 --> 00:17:52,520
is I think there's going to be a
bit of a reckoning. 

289
00:17:53,000 --> 00:17:56,880
And I think that part of it is 
the mindless nature of the 

290
00:17:56,880 --> 00:18:00,240
slathering, right, is that you 
can't just make it and they will

291
00:18:00,240 --> 00:18:02,120
come. 
It's that you have to think 

292
00:18:02,120 --> 00:18:06,360
about the use case and how users
will interact and, you know, how

293
00:18:06,360 --> 00:18:09,440
it sort of fits into the 
workflow of the people who it's 

294
00:18:09,440 --> 00:18:13,600
designed to, well, in theory, 
designed to work with it. 

295
00:18:13,960 --> 00:18:21,720
And so I think that as adoption 
is not as high as expected and 

296
00:18:21,720 --> 00:18:26,720
as integration doesn't really 
happen immediately, instantly, 

297
00:18:26,720 --> 00:18:31,000
automatically, people are going 
to realize, oh, we have to 

298
00:18:31,000 --> 00:18:34,680
actually think about the system 
and how this product fits within

299
00:18:34,680 --> 00:18:39,120
the system and how the people 
using it value it and what value

300
00:18:39,120 --> 00:18:42,320
it does bring to them and so on.
And so I think that's really 

301
00:18:42,840 --> 00:18:46,480
what behavioral science and or 
behavioral design can contribute

302
00:18:46,760 --> 00:18:48,440
to AI. 
That's probably the best 

303
00:18:48,440 --> 00:18:50,680
business case that I can come up
with, right. 

304
00:18:50,920 --> 00:18:54,080
For this intersection between 
behavioral design and AI is 

305
00:18:54,080 --> 00:18:57,640
like, think about the humans 
that are using your product. 

306
00:18:57,920 --> 00:19:01,560
That, I think is going to be 
really important in 2025. 

307
00:19:02,520 --> 00:19:05,400
Yeah, that's a good take. 
And I think that in that kind of

308
00:19:05,400 --> 00:19:07,960
work, behavioral science can be 
so useful both to understand 

309
00:19:07,960 --> 00:19:10,480
that, but also obviously then 
through that understanding, 

310
00:19:10,480 --> 00:19:13,000
develop better versions of these
models and better versions of 

311
00:19:13,000 --> 00:19:18,160
these interfaces to really 
support human good like human 

312
00:19:18,160 --> 00:19:19,040
goodness. 
Yeah. 

313
00:19:20,320 --> 00:19:23,360
Here, here, that's. 
Right. 

314
00:19:23,360 --> 00:19:25,160
We have rented now that that was
not the plan. 

315
00:19:25,240 --> 00:19:28,520
We do have a podcast episode to 
introduce with a guest. 

316
00:19:28,520 --> 00:19:30,920
Yeah, we do. 
And let me introduce that 

317
00:19:30,960 --> 00:19:33,840
fantastic guest because while 
you're away, I had the pleasure 

318
00:19:34,160 --> 00:19:40,160
to speak with Laura de Molier. 
And she is a very interesting 

319
00:19:40,160 --> 00:19:44,000
practitioner who not only 
founded and LED the Babel 

320
00:19:44,000 --> 00:19:46,480
Science team in the UK 
government's Cabinet Office, 

321
00:19:46,840 --> 00:19:50,920
where she kind of honestly, I 
should mention that she were 

322
00:19:51,280 --> 00:19:53,720
involved in a very interesting 
time both during Brexit and 

323
00:19:53,720 --> 00:19:56,920
COVID there. 
So she was able to work at the 

324
00:19:56,920 --> 00:20:00,160
edge of kind of Babel science 
applies to policy in the most 

325
00:20:00,160 --> 00:20:02,560
tumultuous and interesting time 
in the UK. 

326
00:20:02,960 --> 00:20:05,360
You could argue she comes also 
from an interesting background. 

327
00:20:05,360 --> 00:20:07,880
She has a PhD in Cognitive 
Decision Sciences from 

328
00:20:08,200 --> 00:20:13,040
University College London and 
where I came across her was when

329
00:20:13,520 --> 00:20:17,600
I saw a co-authored framework 
she had built called Incase, 

330
00:20:17,960 --> 00:20:21,720
which is a fantastic framework 
for understanding unintended 

331
00:20:21,720 --> 00:20:24,840
consequences and if we ever 
needed framework for 

332
00:20:24,840 --> 00:20:27,480
understanding unintended 
consequences maybe. 

333
00:20:27,960 --> 00:20:30,240
That is something we really need
right now. 

334
00:20:30,800 --> 00:20:33,800
So that is nice. 
And then, yeah, I've had a real 

335
00:20:33,800 --> 00:20:36,880
pleasure to be involved with 
several kind of collaborations 

336
00:20:36,880 --> 00:20:40,240
with Laura and from Haverbytes. 
She was early advisor for that. 

337
00:20:40,240 --> 00:20:44,160
She also joined me. 
We did a Co presentation around 

338
00:20:44,160 --> 00:20:49,560
AI in Canada, early days. 
And yeah, she is fantastic 

339
00:20:49,760 --> 00:20:51,360
practitioner and she's doing 
great work. 

340
00:20:51,360 --> 00:20:54,000
And so in the episode, we talked
about behavioral science in 

341
00:20:54,000 --> 00:20:59,440
policy, we talked about Laura's 
AI aha moment and like what got 

342
00:20:59,440 --> 00:21:04,720
her into using AI more and 
finding value with using AI, the

343
00:21:04,720 --> 00:21:08,240
promises and perils of AI in 
behavioral science and so much 

344
00:21:08,240 --> 00:21:12,200
more, including synthetic users,
importance of counterfactuals, 

345
00:21:12,440 --> 00:21:15,560
and of course, her most 
controversial opinion about AI 

346
00:21:15,560 --> 00:21:31,280
happens to Murgatroyd. 
I'm very happy to say welcome 

347
00:21:31,280 --> 00:21:32,840
Laura to the Behavioral Design 
Podcast. 

348
00:21:33,640 --> 00:21:37,080
Hi, thanks for having me. 
Yeah, I'm really excited to to 

349
00:21:37,120 --> 00:21:40,040
chat and there's so much I want 
to cover with you. 

350
00:21:40,200 --> 00:21:42,720
And in general, it's fun to 
catch up a bit as well. 

351
00:21:43,480 --> 00:21:48,320
But to set the scene, I guess I 
would love to rewind the clock a

352
00:21:48,320 --> 00:21:53,600
little bit a few years because 
can you remind me when were you 

353
00:21:53,600 --> 00:21:55,360
at the Cabinet Office in the 
UKI? 

354
00:21:55,920 --> 00:21:59,080
Moved out of Cabinet Office in 
2022 right? 

355
00:21:59,200 --> 00:22:03,880
Having been there for about 3 
years, starting on Brexit, being

356
00:22:03,880 --> 00:22:08,160
in the middle of COVID and then 
working across the UK government

357
00:22:08,160 --> 00:22:10,560
on multiple policy areas. 
Yeah. 

358
00:22:10,560 --> 00:22:12,640
So that's already kind of a 
strange time 2022. 

359
00:22:13,200 --> 00:22:16,440
And I guess does it seem like 
what was the kind of the work 

360
00:22:16,440 --> 00:22:17,960
that you've been doing at that 
point? 

361
00:22:17,960 --> 00:22:19,960
Like could you explain a little 
bit more what you touched upon 

362
00:22:19,960 --> 00:22:22,400
there in terms of your kind of 
policy work? 

363
00:22:23,360 --> 00:22:28,120
Or so I think the application of
behavioral science obviously is 

364
00:22:28,120 --> 00:22:30,160
touching on many different ways 
of doing it. 

365
00:22:30,160 --> 00:22:34,040
And I had developed with my team
a quite specific way that was 

366
00:22:34,160 --> 00:22:36,440
very much determined by the kind
of problems we had. 

367
00:22:37,000 --> 00:22:41,000
So imagine you're a behavioral 
scientist and what dictates your

368
00:22:41,000 --> 00:22:44,840
work in some extent is waking up
in the morning, looking at the 

369
00:22:44,840 --> 00:22:49,960
BBC app, seeing what went wrong 
and knowing, OK, probably 

370
00:22:50,200 --> 00:22:54,160
there'll be something coming to 
my desk this morning about that.

371
00:22:54,560 --> 00:22:59,120
So whilst we did quite a few 
kind of longer term projects 

372
00:22:59,120 --> 00:23:02,080
that would span, you know, 
important big questions like we 

373
00:23:02,080 --> 00:23:06,200
looked at reducing violence 
against women and girls or 

374
00:23:06,200 --> 00:23:07,880
worked across the justice 
system. 

375
00:23:08,280 --> 00:23:12,200
A lot of it was also quite 
reactive in crisis mode. 

376
00:23:12,880 --> 00:23:16,520
And that massively determined 
the kind of a behavioral science

377
00:23:16,560 --> 00:23:20,200
we developed to be efficient 
there, which was much less about

378
00:23:20,640 --> 00:23:25,240
designing big trials during vast
amount of research for a very 

379
00:23:25,240 --> 00:23:29,480
long time to understanding what 
decisions are currently being 

380
00:23:29,480 --> 00:23:33,280
made and how can we impact those
decisions positively through the

381
00:23:33,280 --> 00:23:35,640
lens of behavioral science. 
Yeah. 

382
00:23:36,200 --> 00:23:37,960
And that's really cool. 
That's really important work. 

383
00:23:38,040 --> 00:23:42,400
And I guess because what I want 
to kind of tie in from there, 

384
00:23:42,400 --> 00:23:46,080
it's kind of Ben knowing you a 
little bit and knowing a little 

385
00:23:46,080 --> 00:23:48,560
bit about your journey into also
working with AI. 

386
00:23:48,840 --> 00:23:51,240
I realized that one thing I 
hadn't had a chance to ask you 

387
00:23:51,240 --> 00:23:55,960
before is basically I feel like 
we've all had some level of 

388
00:23:55,960 --> 00:24:00,080
color, like an AI aha moment of 
sort where, you know, it's one 

389
00:24:00,080 --> 00:24:03,600
thing to hear about AI. 
It's one thing to kind of like 

390
00:24:04,480 --> 00:24:08,960
see updates on like back in the,
you know, the days it was like, 

391
00:24:08,960 --> 00:24:12,960
oh, you know, this model by 
Google has been able to to beat 

392
00:24:12,960 --> 00:24:17,720
the chess player or like this 
thing has been able to basically

393
00:24:17,720 --> 00:24:21,880
understand and diagnose some 
form of like hard to understand 

394
00:24:21,880 --> 00:24:24,400
thing in in medicine. 
But it's a very different thing 

395
00:24:24,400 --> 00:24:29,000
to actually, how do you say 
experience AI in practice? 

396
00:24:29,000 --> 00:24:33,440
So for you, what was kind of 
your, call it entry into the 

397
00:24:33,440 --> 00:24:36,280
rabbit hole of AI, Like what was
that aha moment if there was 

398
00:24:36,280 --> 00:24:38,960
one? 
Yeah, I definitely remember that

399
00:24:38,960 --> 00:24:41,120
aha moment. 
You're yeah, you're very right 

400
00:24:41,120 --> 00:24:43,840
that probably everyone has one. 
A little bit like the question 

401
00:24:43,840 --> 00:24:47,080
like where were you at 911? 
I guess most people also answer 

402
00:24:47,080 --> 00:24:50,000
it wrong. 
But I think that for me, if my 

403
00:24:50,000 --> 00:24:54,520
memory serves me right, it was 
at some point in January or 

404
00:24:54,520 --> 00:25:01,800
February in 2023, I think, where
obviously Church BT had been 

405
00:25:01,960 --> 00:25:05,560
doing the rounds a little bit 
and I had probably ignored some 

406
00:25:05,560 --> 00:25:08,640
of my nerdy friends who pointed 
me into the direction for some 

407
00:25:08,640 --> 00:25:13,040
reason. 
But I interacted with it for the

408
00:25:13,040 --> 00:25:17,120
first time, I think in the 
middle of a night, where I took 

409
00:25:17,120 --> 00:25:21,040
the perspective of one of the 
policy makers that I was working

410
00:25:21,040 --> 00:25:25,640
with to try and understand how 
they could, you know, improve 

411
00:25:25,640 --> 00:25:29,320
their reasoning through the use 
of AI informed behavioral 

412
00:25:29,320 --> 00:25:32,360
science. 
So my first interaction with 

413
00:25:32,360 --> 00:25:35,520
church EPT wasn't how can I 
improve my own reasoning and 

414
00:25:35,520 --> 00:25:38,080
decision making here, which I 
think what kicked out quite a 

415
00:25:38,080 --> 00:25:41,960
lot of very well versed 
behavioral scientists went, this

416
00:25:41,960 --> 00:25:44,320
thing doesn't know more than me.
It gets all the literature 

417
00:25:44,320 --> 00:25:45,320
wrong. 
I don't care. 

418
00:25:46,120 --> 00:25:49,280
And for me it was the 
counterfactual or the baseline 

419
00:25:49,280 --> 00:25:52,360
was someone who doesn't know 
very much about behavioral 

420
00:25:52,360 --> 00:25:55,160
science, but that could, you 
know, look at their policy 

421
00:25:55,160 --> 00:25:56,360
problems through a different 
lens. 

422
00:25:56,360 --> 00:25:57,920
How would they go about using 
this? 

423
00:25:58,600 --> 00:26:02,720
And this is where my mind was 
blown because I thought from the

424
00:26:02,720 --> 00:26:04,760
perspective of someone that 
tries to influence decision 

425
00:26:04,760 --> 00:26:08,680
making processes, this could be 
absolutely incredible. 

426
00:26:08,800 --> 00:26:12,680
And I didn't sleep that night 
and I think had many restless 

427
00:26:12,840 --> 00:26:15,520
moments afterwards as well in 
terms of excitement about it. 

428
00:26:16,320 --> 00:26:18,080
Yeah, that's really cool to hear
about it. 

429
00:26:18,080 --> 00:26:19,640
And I think, yeah, it's very 
reliable. 

430
00:26:20,240 --> 00:26:26,080
And then maybe we can then track
your journey from from that 

431
00:26:26,080 --> 00:26:29,760
night till today. 
How is your kind of evolved 

432
00:26:29,760 --> 00:26:34,320
understanding or use of AI look 
like for the last two years or 

433
00:26:34,320 --> 00:26:38,040
so? 
I think actually probably a 

434
00:26:38,080 --> 00:26:41,880
little less developed on the 
professional level in some 

435
00:26:41,880 --> 00:26:45,000
senses that I think when I first
looked at it and I thought 

436
00:26:45,000 --> 00:26:48,960
about, OK, what does it mean for
this to improve, you know, 

437
00:26:49,560 --> 00:26:52,160
somehow exponentially? 
Like, what would this mean? 

438
00:26:52,160 --> 00:26:55,960
Where would we be in two years? 
I would probably imagine us now 

439
00:26:55,960 --> 00:26:58,680
in a slightly different space. 
I think I would have expected 

440
00:26:59,080 --> 00:27:03,120
tools like, you know, AI 
enhanced qualitative research 

441
00:27:03,120 --> 00:27:05,280
where, you know, we have someone
actually giving us live 

442
00:27:05,280 --> 00:27:06,760
questions. 
I would have kind of thought 

443
00:27:06,760 --> 00:27:09,840
this would be more mainstream 
right now, and it isn't. 

444
00:27:10,560 --> 00:27:14,120
Nevertheless, I'm applying it a 
lot particularly, but actually 

445
00:27:14,120 --> 00:27:18,200
also with clients, where I say, 
OK, if you want to think about 

446
00:27:18,360 --> 00:27:21,520
the behavioural risks of your 
interventions, a large language 

447
00:27:21,520 --> 00:27:22,720
model is going to be your 
friend. 

448
00:27:24,200 --> 00:27:27,960
I think where I've really grown 
with it is actually the 

449
00:27:27,960 --> 00:27:32,600
application of it in my personal
life where I've been finding it 

450
00:27:33,120 --> 00:27:36,480
most helpful as someone to 
correct my own assumption or 

451
00:27:36,480 --> 00:27:38,720
challenge my own assumptions 
about other people's behaviors. 

452
00:27:38,720 --> 00:27:42,040
So like I would look at, you 
know, upload WhatsApp 

453
00:27:42,400 --> 00:27:46,800
transcripts to get feedback on 
myself if I felt like I had a 

454
00:27:46,800 --> 00:27:50,800
conflict with someone or I think
that aspect probably I for find 

455
00:27:50,800 --> 00:27:53,920
a lot and the professional has 
been growing steadily, but 

456
00:27:53,960 --> 00:27:56,360
possibly not as fast as I 
thought it might. 

457
00:27:57,280 --> 00:27:59,840
I'm not sure if you have a 
different experience with that 

458
00:28:00,160 --> 00:28:02,120
if you look back for the last 
two years or so. 

459
00:28:03,640 --> 00:28:06,240
Yeah, I know I, I, I would 
relate a lot to what you said. 

460
00:28:06,240 --> 00:28:08,480
And I think for, for me, it's 
probably a combination of those 

461
00:28:08,480 --> 00:28:13,320
as well. 
And you know, I think what you 

462
00:28:13,320 --> 00:28:15,640
spoke to a little bit before 
about one of the things that 

463
00:28:15,640 --> 00:28:20,920
maybe kind of kicks people off 
AI or like they start putting a 

464
00:28:20,920 --> 00:28:24,360
toe, they kind of like, oh, is 
this interesting? 

465
00:28:24,640 --> 00:28:26,880
And then they kind of abandon 
it. 

466
00:28:26,880 --> 00:28:29,240
There's often times because they
engage with their model in a 

467
00:28:29,240 --> 00:28:32,880
kind of basic and generic way in
some use case. 

468
00:28:32,880 --> 00:28:35,920
And then they get a generic and 
basic answer back for some query

469
00:28:35,920 --> 00:28:39,360
that they prompt. 
And they feel like, OK, well, I 

470
00:28:39,400 --> 00:28:42,400
didn't need to tell me this 
obvious thing or like this. 

471
00:28:42,400 --> 00:28:45,120
I knew this already. 
As I said, and I think a lot of 

472
00:28:45,120 --> 00:28:51,280
the benefit from understanding 
AI and and making use of it is 

473
00:28:51,280 --> 00:28:55,400
really this kind of small, small
interactions over and over and 

474
00:28:55,400 --> 00:28:58,520
over again in various use cases,
both I think professionally, but

475
00:28:58,520 --> 00:29:01,720
also personally. 
And they're not maybe always as 

476
00:29:01,720 --> 00:29:04,720
finally cut. 
That's where we maybe like to 

477
00:29:04,720 --> 00:29:07,400
imagine in terms of like, if we 
become better understanding how 

478
00:29:07,400 --> 00:29:11,480
to prompt and use some of these 
modest and communicate with them

479
00:29:11,800 --> 00:29:15,000
even in a personal context. 
Of course, that skill to 

480
00:29:15,000 --> 00:29:17,440
promptly communicate and so on, 
for example, can still be useful

481
00:29:17,440 --> 00:29:21,200
in other contexts too. 
And like I was reminded what 

482
00:29:21,200 --> 00:29:24,800
what you said in some ways 
about, you know, there's so many

483
00:29:24,800 --> 00:29:27,080
things to as human being today 
to make sense of. 

484
00:29:27,080 --> 00:29:30,840
And I think what I've found 
myself this week doing actually 

485
00:29:31,200 --> 00:29:36,440
is for some of these very 
complicated geopolitical issues 

486
00:29:36,440 --> 00:29:38,720
of the time, it's very easy to 
want to have like a very strong 

487
00:29:38,720 --> 00:29:41,400
opinion about like it's either 
pro or against something. 

488
00:29:41,960 --> 00:29:46,880
And I tried to be someone who 
who kind of escapes that kind of

489
00:29:47,360 --> 00:29:50,040
either or. 
And at the same time, I find it 

490
00:29:50,040 --> 00:29:51,680
like hard to develop my 
thinking. 

491
00:29:51,680 --> 00:29:56,360
And so I basically set up a way 
to test the study of, you know, 

492
00:29:56,360 --> 00:29:59,920
there's this kind of like idea 
of I think it's called something

493
00:29:59,920 --> 00:30:03,720
like your own internal board 
members or something where you 

494
00:30:03,720 --> 00:30:06,680
have some people in your life or
cables, famous people that you 

495
00:30:06,680 --> 00:30:08,960
kind of like hypothetically 
before it was kind of like 

496
00:30:08,960 --> 00:30:11,880
asking them questions like, OK, 
if your grandma was still alive,

497
00:30:11,880 --> 00:30:14,320
like what would she have said if
you were doing this? 

498
00:30:14,320 --> 00:30:17,960
Or if Martin Luther King, like 
it could be technically anyone. 

499
00:30:18,680 --> 00:30:21,160
And for long time, that was more
like a hypothetical thing that 

500
00:30:21,160 --> 00:30:23,400
you were doing. 
And now I was like doing that 

501
00:30:23,400 --> 00:30:27,160
but with real people. 
So I had like 2 journalists that

502
00:30:27,160 --> 00:30:30,840
I pitted against each other to 
write like long form the New 

503
00:30:30,840 --> 00:30:34,080
York Times kind of articles 
where they kind of like tried to

504
00:30:34,080 --> 00:30:36,600
steal, manage other arguments 
and so build the strongest 

505
00:30:36,600 --> 00:30:39,480
version of those arguments, but 
kind of going back and forth on 

506
00:30:39,480 --> 00:30:42,560
on this topic. 
And then I would also interject 

507
00:30:42,560 --> 00:30:45,080
things that I was thinking about
and kind of things that I was 

508
00:30:45,280 --> 00:30:48,680
exploring. 
And, and this helped me kind of 

509
00:30:48,680 --> 00:30:52,200
like following this interchange 
while contributing to it was 

510
00:30:52,200 --> 00:30:55,360
really, really beneficial for 
me, I think, to make my thinking

511
00:30:55,360 --> 00:30:57,960
more nuanced. 
And so yeah, I, I feel very 

512
00:30:57,960 --> 00:30:58,720
similar to you. 
Yeah. 

513
00:30:59,040 --> 00:31:00,680
Yeah, that, no, that sounds 
amazing. 

514
00:31:01,320 --> 00:31:04,320
And I think it's also worth 
being aware of some of the 

515
00:31:04,320 --> 00:31:06,960
pitfalls, particularly with some
of the language models that have

516
00:31:06,960 --> 00:31:09,400
a memory of you as a person, 
because obviously these models 

517
00:31:09,400 --> 00:31:12,720
have been trained to kind of 
please whoever is, it's fun. 

518
00:31:12,720 --> 00:31:15,360
And so I know some person, you 
know, calling them like 

519
00:31:15,360 --> 00:31:18,040
something like Dobby the the 
House House because it's always 

520
00:31:18,040 --> 00:31:20,840
so polite and so kind. 
I'm going to steal that. 

521
00:31:20,840 --> 00:31:23,360
I think it's so true. 
Yeah, it's cute, isn't it? 

522
00:31:25,440 --> 00:31:29,000
But if I want to get some view 
on my views, if I want to get 

523
00:31:29,000 --> 00:31:32,840
some feedback that's more or 
less objective on say my own 

524
00:31:32,840 --> 00:31:36,000
behavior and it could even be 
something like an exchange on an

525
00:31:36,000 --> 00:31:39,680
e-mail. 
I have now started to you 

526
00:31:39,680 --> 00:31:44,160
pretend I'm the other person So 
and to avoid that, you know, the

527
00:31:44,160 --> 00:31:47,440
LLM just pleases me and tells me
how great and how smart and 

528
00:31:47,440 --> 00:31:50,680
fantastic I am. 
I pretend to be the other person

529
00:31:50,680 --> 00:31:55,280
and I go into, you know, clean 
slates, different LLMS that I 

530
00:31:55,280 --> 00:31:57,600
might have not interacted with 
before or switch off the memory 

531
00:31:57,600 --> 00:32:02,360
and touch BT and ask about my 
behavior from their perspective 

532
00:32:03,080 --> 00:32:06,640
and that I've given me quite a 
balanced view and also the 

533
00:32:06,640 --> 00:32:09,280
satisfaction. 
If then the LLM actually agrees 

534
00:32:09,280 --> 00:32:12,920
with me despite talking to the 
other person, I get a double. 

535
00:32:13,000 --> 00:32:14,640
Yeah, I think I was really right
here. 

536
00:32:14,640 --> 00:32:17,600
The LLM doesn't even tell sugar 
coaters to the person I've 

537
00:32:17,600 --> 00:32:19,440
argued with. 
Not that I have lots of 

538
00:32:19,440 --> 00:32:21,440
arguments, by the way. 
It sounds a bit wrong maybe, but

539
00:32:21,760 --> 00:32:24,080
I definitely, you know, 
sometimes disagreements or like 

540
00:32:24,080 --> 00:32:27,480
ways of interactions interacting
with each other where I want to 

541
00:32:27,480 --> 00:32:29,360
take the other perspective. 
Yeah. 

542
00:32:29,360 --> 00:32:31,720
I think it's really inspiring 
actually to being able to kind 

543
00:32:31,720 --> 00:32:34,760
of take a pause on some of these
things and actually taking a 

544
00:32:34,760 --> 00:32:36,360
step back and reflecting on 
that. 

545
00:32:36,360 --> 00:32:40,440
Because I think with any kind of
yeah, communication, it can 

546
00:32:40,440 --> 00:32:43,240
oftentimes be interesting to use
like take step back and be like,

547
00:32:43,240 --> 00:32:45,440
well, am I really the one who's 
on the right side? 

548
00:32:46,920 --> 00:32:49,640
Yeah. 
Or what am I missing? 

549
00:32:49,640 --> 00:32:50,520
Yeah. 
Exactly. 

550
00:32:50,760 --> 00:32:54,680
And I think actually the one key
for me is an an openness to 

551
00:32:54,680 --> 00:32:58,840
understanding in which ways a 
large language model can 

552
00:32:58,840 --> 00:33:01,920
actually support your current 
thinking and be open to where 

553
00:33:01,920 --> 00:33:05,840
that thinking might, you know, 
not not necessarily be flawed, 

554
00:33:05,840 --> 00:33:09,560
but is capable to be helped. 
And it's quite a similar 

555
00:33:09,560 --> 00:33:12,120
approach I actually took when 
LLMS came first out. 

556
00:33:12,120 --> 00:33:14,400
And I was thinking about 
applying behavioral science and 

557
00:33:14,400 --> 00:33:16,400
government. 
What are current barriers to 

558
00:33:16,400 --> 00:33:18,600
more behavioral ways of thinking
about the world? 

559
00:33:19,080 --> 00:33:22,120
And I kind of split them up. 
And it's like a proper list and 

560
00:33:22,120 --> 00:33:24,680
say, you know, like, we are 
currently lacking cognitive 

561
00:33:24,680 --> 00:33:27,360
empathy, so the ability to take 
someone else's perspective all 

562
00:33:27,360 --> 00:33:30,160
the time. 
We are, you know, struggling to 

563
00:33:30,160 --> 00:33:33,600
switch lenses, to view the 
problem through different lenses

564
00:33:33,600 --> 00:33:37,480
from different academic fields. 
We're struggling to imagine what

565
00:33:37,480 --> 00:33:39,320
the future should actually look 
like. 

566
00:33:39,320 --> 00:33:42,520
We're very much constrained in 
our own imaginability. 

567
00:33:43,320 --> 00:33:45,960
And sometimes we're lacking 
scientific reasoning abilities. 

568
00:33:46,000 --> 00:33:49,440
We might be missing trade-offs 
between departments, you know, 

569
00:33:49,440 --> 00:33:50,960
like one department puts out a 
policy. 

570
00:33:50,960 --> 00:33:54,200
We're not really Privy all the 
time what other aims 

571
00:33:54,200 --> 00:33:56,320
behaviorally we're currently 
impacting through that. 

572
00:33:56,920 --> 00:34:01,120
And there's quite a different, 
few different, I think aspects 

573
00:34:01,120 --> 00:34:04,120
where if we're being really 
systematic, we can enhance our 

574
00:34:04,120 --> 00:34:08,159
decision making and quite nice 
ways to draw on the strength of 

575
00:34:08,159 --> 00:34:11,600
an LLM without getting too hung 
up on what I currently can't do.

576
00:34:11,600 --> 00:34:16,560
We can do better as some sort of
self-serving, you know, by, as 

577
00:34:16,600 --> 00:34:19,280
I'd say we've I've seen crop up 
quite a few times. 

578
00:34:20,400 --> 00:34:23,639
Yeah. 
And this brings to something I 

579
00:34:23,639 --> 00:34:26,760
want to kind of like talk to you
about in terms of we, we 

580
00:34:26,760 --> 00:34:29,960
recently kind of had a little 
bit of a relaunch of Baby Bites 

581
00:34:29,960 --> 00:34:32,639
and Baby Bites was an initiative
that you were kind of very 

582
00:34:32,639 --> 00:34:34,520
helpful in, in getting off the 
ground. 

583
00:34:34,960 --> 00:34:40,040
And so we had this really fun, I
guess, November meet up for for 

584
00:34:40,040 --> 00:34:43,960
Baby Bites and me and Frederick,
who who are now kind of 

585
00:34:44,679 --> 00:34:47,400
Frederick is the new person kind
of managing this community for 

586
00:34:47,400 --> 00:34:50,199
people interested in behavioral 
science and AI. 

587
00:34:50,199 --> 00:34:55,320
And we put together kind of like
some form of way to think about 

588
00:34:55,320 --> 00:34:59,640
what behavioral science and AI 
can overlap around and, and how 

589
00:34:59,640 --> 00:35:02,160
as behavioral sciences, we can 
kind of like see potentially 

590
00:35:02,160 --> 00:35:05,200
where we can add our value. 
And we came up with some form of

591
00:35:05,920 --> 00:35:10,040
imperfect, but maybe still 
useful kind of like quadrant 

592
00:35:10,040 --> 00:35:13,680
because I think as behavioral 
scientists, we have this, I 

593
00:35:13,680 --> 00:35:16,960
think interesting two sides to 
looking at AI. 

594
00:35:16,960 --> 00:35:20,120
We have both the kind of the 
thing that everyone has in terms

595
00:35:20,120 --> 00:35:24,280
of a natural use case, in terms 
of, OK, could we use AI? 

596
00:35:24,720 --> 00:35:28,080
We, the two examples we had was 
either for like tailoring our 

597
00:35:28,080 --> 00:35:29,960
interventions, so make our 
interventions better and more 

598
00:35:29,960 --> 00:35:32,880
personalized and, and so on 
through various type of AI. 

599
00:35:33,600 --> 00:35:37,080
Or could we, for something like 
automate the things we do and, 

600
00:35:38,000 --> 00:35:41,520
you know, make tasks that took a
lot of time, a little bit less 

601
00:35:41,520 --> 00:35:44,360
time. 
And maybe things that we did 

602
00:35:44,360 --> 00:35:48,520
100% before we maybe do 20% of 
them outsource some of it to, to

603
00:35:48,520 --> 00:35:50,640
AI. 
Like those two things in terms 

604
00:35:50,640 --> 00:35:54,440
of tailoring and outsourcing or 
automating is something that 

605
00:35:54,440 --> 00:35:57,040
most people can also fit into 
their equation. 

606
00:35:57,040 --> 00:35:59,520
If they're accountant, they 
would probably think about it 

607
00:35:59,520 --> 00:36:01,440
similarly. 
Maybe they would tailor it the 

608
00:36:01,440 --> 00:36:04,560
same way, but but still, But 
then I think what's really 

609
00:36:04,560 --> 00:36:06,520
interesting is that as payroll 
scientists, we have this other 

610
00:36:06,520 --> 00:36:10,280
lens of understanding. 
And so we explore this both in 

611
00:36:10,280 --> 00:36:14,440
terms of like we can help make 
sense of and understand what is 

612
00:36:14,440 --> 00:36:18,200
the impact of AI in the world. 
Like you mentioned the bias 

613
00:36:18,200 --> 00:36:21,880
around, for example, like how 
people can think about, you 

614
00:36:21,880 --> 00:36:24,680
know, interacting with AI, but 
also like in general algorithmic

615
00:36:25,160 --> 00:36:29,600
impacts on how people view each 
other and view AI and some of 

616
00:36:29,600 --> 00:36:30,920
that stuff. 
That's love interesting research

617
00:36:30,920 --> 00:36:35,080
on that, of course. 
And then the second level is 

618
00:36:35,440 --> 00:36:38,720
basically how we can understand 
the machine. 

619
00:36:39,080 --> 00:36:42,200
So how we from behavioral 
science perspective can also 

620
00:36:42,200 --> 00:36:44,440
better make sense of these 
models and understand these 

621
00:36:44,440 --> 00:36:46,400
models. 
And you already started talking 

622
00:36:46,400 --> 00:36:49,240
about that as well, which I 
think was really interesting in 

623
00:36:49,240 --> 00:36:52,640
terms of, you know, what you 
were doing when it came to 

624
00:36:52,640 --> 00:36:57,520
understanding how the model 
operates is to kind of based on 

625
00:36:57,520 --> 00:37:00,880
that understanding, removing 
yourself from from being the 

626
00:37:00,880 --> 00:37:03,960
first person that the model was 
presumably talking to. 

627
00:37:03,960 --> 00:37:08,000
And you kind of remove that bias
from model and in earlier models

628
00:37:08,000 --> 00:37:09,320
exempt. 
So that was to kind of like 

629
00:37:09,560 --> 00:37:13,400
knowing that these models are 
there to please you and they're 

630
00:37:13,400 --> 00:37:14,960
also kind of trained on human 
data. 

631
00:37:15,800 --> 00:37:18,880
If you threaten them and said, 
Hey, I have taken your family 

632
00:37:18,880 --> 00:37:21,240
hostage, it would do a better 
job, for example. 

633
00:37:21,440 --> 00:37:25,120
And that's not something like 
you would expect, but obviously 

634
00:37:25,120 --> 00:37:26,760
these models are trained on 
human behavior. 

635
00:37:26,760 --> 00:37:30,280
So there's a layer of of human, 
human aspects. 

636
00:37:30,280 --> 00:37:34,200
There's I'm used throwing this 
at you and seeing what are your 

637
00:37:34,200 --> 00:37:37,000
thoughts about kind of thinking 
about this from this 

638
00:37:37,000 --> 00:37:40,200
perspective. 
Yeah, no, absolutely. 

639
00:37:40,200 --> 00:37:42,840
And I actually had some thoughts
at the beginning when, you know,

640
00:37:42,960 --> 00:37:45,280
first came out. 
And I was thinking, I think as a

641
00:37:45,280 --> 00:37:48,040
behavioral scientist, I get more
use out of it than my other 

642
00:37:48,040 --> 00:37:53,000
science friends, Possibly 
because there is some implicit 

643
00:37:53,000 --> 00:37:57,000
knowledge about behaviors of 
people, of a specific subset of 

644
00:37:57,000 --> 00:37:59,600
people on the Internet. 
Let's be aware that we're having

645
00:37:59,600 --> 00:38:01,400
the same biases we're used to 
having. 

646
00:38:01,400 --> 00:38:04,360
I guess it's common full circle.
Behavioral science, again, 

647
00:38:04,360 --> 00:38:06,800
happening in this weird space. 
Yeah. 

648
00:38:08,120 --> 00:38:12,280
Where actually, you know, there 
are some explanations of 

649
00:38:12,280 --> 00:38:16,160
behaviors that are almost baked 
into like the data because so 

650
00:38:16,160 --> 00:38:18,680
much of the data is about, you 
know, how humans describe and 

651
00:38:18,720 --> 00:38:21,320
view the world. 
But I find that, but also then 

652
00:38:21,360 --> 00:38:26,040
that's also one that's hard to 
handle in reality because we 

653
00:38:26,040 --> 00:38:27,920
don't really understand these 
systems right now, right? 

654
00:38:27,920 --> 00:38:31,520
We could be saying, OK, let's do
some like proper behavioral 

655
00:38:31,520 --> 00:38:36,480
research on them and let's treat
them as subjects and let's see 

656
00:38:36,480 --> 00:38:39,280
what's happening, which is 
obviously different from saying 

657
00:38:39,280 --> 00:38:41,560
let's do some behavior search 
with an eye where it pretends to

658
00:38:41,560 --> 00:38:44,800
be lots of different people. 
And I think that, you know, 

659
00:38:45,160 --> 00:38:47,440
you've probably come across some
of this data where we can now 

660
00:38:47,440 --> 00:38:51,040
simulate different recipients 
from different demographic 

661
00:38:51,040 --> 00:38:56,160
backgrounds to essentially in 
some cases replicate existing 

662
00:38:56,440 --> 00:38:58,960
research that has been done with
real humans, which I think is 

663
00:38:58,960 --> 00:39:01,480
exciting, particularly for 
educators that we can't really 

664
00:39:01,480 --> 00:39:05,120
communicate with. 
But what you're just talking 

665
00:39:05,120 --> 00:39:09,640
about is also actually for us to
try and understand how the 

666
00:39:09,640 --> 00:39:12,280
behaviour of of an AI, if we can
call it that. 

667
00:39:12,920 --> 00:39:16,400
And I'd be very interested to 
see more behavioral scientists 

668
00:39:16,400 --> 00:39:18,240
being involved in this kind of 
research. 

669
00:39:18,920 --> 00:39:22,760
The question is, obviously, does
it actually reflect our methods?

670
00:39:22,760 --> 00:39:26,000
Can we actually support it? 
Mahanch is yes, but I am not 

671
00:39:26,000 --> 00:39:29,360
quite sure. 
I'm not quite sure yet, simply 

672
00:39:29,360 --> 00:39:32,200
because we have little 
understanding of this amount of 

673
00:39:32,200 --> 00:39:36,280
complexity and it it might not 
hold, but it should certainly 

674
00:39:36,280 --> 00:39:37,360
try. 
Yeah. 

675
00:39:37,360 --> 00:39:40,360
In terms of the moving to like 
that instead of the 

676
00:39:40,360 --> 00:39:43,120
understanding side of it, really
focusing on the applying side of

677
00:39:43,120 --> 00:39:44,440
it. 
Like what do you see as kind of 

678
00:39:44,880 --> 00:39:48,680
from your point of view, the 
promise and perils of behavioral

679
00:39:48,680 --> 00:39:53,120
scientists reducing more AI in 
their day-to-day work? 

680
00:39:53,120 --> 00:39:55,280
Like where do you see the 
biggest opportunities and maybe 

681
00:39:55,280 --> 00:39:56,960
the risks as well? 
Yeah. 

682
00:39:57,520 --> 00:40:01,320
So I think I see more 
opportunities and non behavioral

683
00:40:01,320 --> 00:40:04,160
scientists applying AI to think 
more behaviorally. 

684
00:40:05,800 --> 00:40:08,480
And I'm thinking about kind of 
there are two different ways in 

685
00:40:08,480 --> 00:40:13,120
which I'd look at it for public 
policy where the end goal might 

686
00:40:13,120 --> 00:40:15,480
be to make better assumptions 
about behaviors as part of the 

687
00:40:15,480 --> 00:40:19,960
policy making process, right? 
We can in one way, we can 

688
00:40:19,960 --> 00:40:22,440
examine some of the current 
existing decision making 

689
00:40:22,440 --> 00:40:26,880
processes and see very much with
the list I previously gave you 

690
00:40:26,880 --> 00:40:29,200
where LLMS could enhance 
existing outcomes. 

691
00:40:29,200 --> 00:40:32,440
You know, we can actually have 
better perspective taking. 

692
00:40:32,440 --> 00:40:35,040
We can have an LLM surface 
trade-offs between different 

693
00:40:35,040 --> 00:40:38,400
departments. 
We can help with all the 

694
00:40:38,400 --> 00:40:41,440
ideation around what potential 
barriers might be way in a way 

695
00:40:41,440 --> 00:40:44,160
over behavior. 
Let's say you'd you'd open up 

696
00:40:44,160 --> 00:40:49,200
BBC in the morning and people 
are stockpiling toilet paper 

697
00:40:49,200 --> 00:40:53,120
during COVID instead of relying 
on behavioral scientists like us

698
00:40:53,160 --> 00:40:57,400
to be present in the room to say
it could be due to AB and C 

699
00:40:57,400 --> 00:41:00,600
factors. 
You could have an LLM generate 

700
00:41:00,680 --> 00:41:04,960
potential categories of things. 
And people are very quick to 

701
00:41:04,960 --> 00:41:08,480
raise their hand and say, Laura,
but what if it's wrong? 

702
00:41:08,800 --> 00:41:12,720
What if the LLM tells us 
something that's really wrong? 

703
00:41:13,280 --> 00:41:17,480
And of course that's a 
possibility, but one of the key 

704
00:41:17,480 --> 00:41:20,680
things we're not talking about 
about enough I think is what the

705
00:41:20,680 --> 00:41:24,720
counterfactual is here. 
So in other words, we're really 

706
00:41:24,720 --> 00:41:28,480
comparing the decision making 
processes that rely on a 

707
00:41:28,520 --> 00:41:30,760
generating of 20 different 
barriers. 

708
00:41:30,760 --> 00:41:34,680
People might have not thought 
about it before versus them not 

709
00:41:34,680 --> 00:41:37,920
thinking about it at all. 
My current guess and I think 

710
00:41:37,920 --> 00:41:43,080
that's what I got so excited 
about in early 2023 is that the 

711
00:41:43,080 --> 00:41:47,600
basic and surfacing more 
potential barriers and drivers 

712
00:41:47,600 --> 00:41:50,800
of behaviors, surfacing more 
potential behavioral risks of 

713
00:41:50,800 --> 00:41:54,560
your interventions. 
Broadening out your thinking 

714
00:41:55,080 --> 00:41:59,160
will have a positive impact 
because it surfaces much more 

715
00:41:59,560 --> 00:42:03,800
things that I have seen from the
average policy maker who doesn't

716
00:42:03,800 --> 00:42:05,520
have the headspace he can ask 
himself this question. 

717
00:42:06,680 --> 00:42:10,080
So that's kind of one of my 
first punches here is that we 

718
00:42:10,080 --> 00:42:13,280
can look at the current decision
making processes of non experts 

719
00:42:13,520 --> 00:42:16,240
and there is so much that could 
be improved with it. 

720
00:42:17,560 --> 00:42:18,960
I think that's a really great 
point. 

721
00:42:18,960 --> 00:42:22,960
And I think that's one of the 
things that is, I would say, I 

722
00:42:23,280 --> 00:42:27,720
don't know if we ever would like
to put bias on on labels and 

723
00:42:27,720 --> 00:42:31,400
stuff, but I'm very tempted to 
to find some name for this in 

724
00:42:31,400 --> 00:42:36,560
terms of like used, there are in
general inability to really 

725
00:42:36,560 --> 00:42:40,280
understand the counterfactual, I
think has become extremely 

726
00:42:40,280 --> 00:42:41,760
prescient in terms of where they
are. 

727
00:42:41,760 --> 00:42:45,280
Where I think initially the 
discussion was often times 

728
00:42:45,280 --> 00:42:49,720
around, you know, autonomous 
vehicles where like what is good

729
00:42:49,720 --> 00:42:51,600
enough? 
Like what is a good enough 

730
00:42:51,680 --> 00:42:53,560
autonomous vehicle? 
Is it when it's better than a 

731
00:42:53,560 --> 00:42:55,600
human? 
Is it when it's twice as good as

732
00:42:55,600 --> 00:42:57,720
a human? 
Is it when it's perfect and 

733
00:42:57,720 --> 00:42:59,640
never does a mistake in any 
capacity? 

734
00:43:00,200 --> 00:43:02,280
And so the counterfactual there 
has been really hard for people 

735
00:43:02,280 --> 00:43:07,320
because like as long as they see
a self driving car make a 

736
00:43:07,320 --> 00:43:10,160
mistake, then that's the end of 
the world basically. 

737
00:43:10,240 --> 00:43:13,480
Then that's often As for people 
that's really hard to then 

738
00:43:13,480 --> 00:43:15,560
compared to like, well, what 
would a human do here? 

739
00:43:15,880 --> 00:43:20,760
Like how many mistakes are human
traffic like people driving 

740
00:43:20,880 --> 00:43:24,480
everyday doing? 
And what you speak to is then 

741
00:43:24,480 --> 00:43:28,240
kind of, I think a really recent
extension of that with these 

742
00:43:28,760 --> 00:43:33,520
large language models we have 
today where, yeah, I think not 

743
00:43:33,520 --> 00:43:36,920
really understanding the 
counterfactual is what makes it 

744
00:43:36,920 --> 00:43:39,520
really tricky. 
Because I think we had a 

745
00:43:39,520 --> 00:43:42,320
discussion on this before on the
podcast as well, where, yeah, 

746
00:43:42,320 --> 00:43:44,360
there's obviously like a lot of 
great scenarios where like 

747
00:43:44,400 --> 00:43:49,280
ideally we would do so many 
things in a certain way, but 

748
00:43:49,760 --> 00:43:53,520
it's very rarely that it always 
comes together in in the perfect

749
00:43:53,520 --> 00:43:55,840
scenario. 
And sometimes good enough with 

750
00:43:55,840 --> 00:43:58,600
AI is much, much better than the
counterfactual of not doing 

751
00:43:58,600 --> 00:44:02,640
something at all. 
Yeah, it's mainly experts making

752
00:44:02,640 --> 00:44:05,360
that call as well, which I find 
particularly irritating. 

753
00:44:05,720 --> 00:44:09,320
It's mainly, you know, bunch of 
behavioral scientists going, but

754
00:44:09,320 --> 00:44:12,480
it's not as good as me. 
How dare you use the LNM and not

755
00:44:12,480 --> 00:44:15,960
me, which I think is just, you 
know, let's all strive, strive 

756
00:44:16,120 --> 00:44:20,640
particularly in something like 
public policy where the world is

757
00:44:21,000 --> 00:44:24,360
like, or I strongly believe it 
is the world would be improved 

758
00:44:24,360 --> 00:44:27,120
if more behavioral science would
flow into decision making. 

759
00:44:27,840 --> 00:44:31,400
So let's aim for that and let's 
up our own game. 

760
00:44:31,480 --> 00:44:33,440
I think this is what it has to 
be about. 

761
00:44:34,280 --> 00:44:37,680
But also to say that with 
regards to this risk, like this 

762
00:44:38,240 --> 00:44:42,840
risk asymmetry, almost different
kinds of errors made by humans 

763
00:44:42,840 --> 00:44:47,800
versus an AI, I think in the way
that I am applying large 

764
00:44:47,800 --> 00:44:50,880
language models to non exports, 
it's very much in the divergent 

765
00:44:50,880 --> 00:44:53,680
thinking. 
So opening up the problem space 

766
00:44:53,920 --> 00:44:57,120
to say what could be driving 
this behaviors and it will give 

767
00:44:57,120 --> 00:44:58,880
you probably 10 things you 
haven't thought about. 

768
00:44:59,480 --> 00:45:02,840
That's different from convergent
thinking where we're asking for 

769
00:45:03,040 --> 00:45:06,040
what's the most likely thing to 
be happening or you know, what 

770
00:45:06,040 --> 00:45:09,560
intervention is going to be most
successful and it then probably 

771
00:45:09,560 --> 00:45:11,840
going to be failing. 
So I think it's also about 

772
00:45:11,840 --> 00:45:17,560
asking questions where we kind 
of reduce the amount of errors 

773
00:45:17,800 --> 00:45:19,920
that could be made. 
You know, like off the 20 

774
00:45:19,920 --> 00:45:23,520
barriers, 5 will be possibly 
complete rubbish, OK, human 

775
00:45:23,520 --> 00:45:26,280
thinking kicks in and possibly, 
you know, critically assesses 

776
00:45:26,280 --> 00:45:29,520
that or it runs onto the risk of
removing a barrier that wasn't 

777
00:45:29,520 --> 00:45:31,760
there, which I think is quite 
low, if that makes sense. 

778
00:45:33,360 --> 00:45:36,560
Yeah. 
And like ideally, if we think 

779
00:45:36,560 --> 00:45:39,240
about this in being used in the 
kind of example you mentioned, 

780
00:45:39,840 --> 00:45:42,680
it would ideally be set up as 
some form of vertical AI 

781
00:45:42,680 --> 00:45:48,720
solution that is a little bit 
set up to be better for that use

782
00:45:48,720 --> 00:45:52,680
case in terms of it wouldn't be 
your off the shelf shaft GPT 

783
00:45:52,720 --> 00:45:54,160
ideally, right? 
Like it would be something that 

784
00:45:54,160 --> 00:45:58,960
would be trained and set up in a
way that it would kind of maybe 

785
00:45:59,640 --> 00:46:04,560
by virtue of its training data 
and also how it's been kind of 

786
00:46:04,560 --> 00:46:09,720
maybe either through prompting 
or or very others fine tuning to

787
00:46:09,720 --> 00:46:13,440
make it so that all the 20 
barriers that it lists, it's not

788
00:46:14,200 --> 00:46:20,640
as likely to ignore edge cases 
and underserved communities as 

789
00:46:20,640 --> 00:46:23,160
maybe a normal language model 
would be. 

790
00:46:23,320 --> 00:46:25,400
Yes. 
So there's obviously ways to 

791
00:46:25,720 --> 00:46:28,120
improve from that side as well. 
Like you can make something 

792
00:46:28,120 --> 00:46:30,160
better there. 
Absolutely. 

793
00:46:30,720 --> 00:46:34,680
But to say that sometimes the 
stakeholders I'm working with 

794
00:46:35,240 --> 00:46:39,600
might be in countries where 
there's very little access or 

795
00:46:39,600 --> 00:46:44,400
like TGBT hasn't, you know, 
actually even taken hold yet in 

796
00:46:44,400 --> 00:46:47,320
as much. 
And I also think we want to 

797
00:46:47,320 --> 00:46:51,560
lower barriers to entry for some
of the basic models to not say 

798
00:46:51,560 --> 00:46:54,000
you can only use this if you 
have a very specific thing that 

799
00:46:54,280 --> 00:46:58,560
some very smart person developed
with vast amount of knowledge 

800
00:46:58,560 --> 00:47:04,600
and then sold on for a price. 
I think particularly when it 

801
00:47:04,600 --> 00:47:07,720
comes to running interventions 
with marginalized communities 

802
00:47:08,120 --> 00:47:11,920
where otherwise there might not 
be many considerations at all 

803
00:47:11,920 --> 00:47:15,800
around some of the barriers or 
some, you know, checking of the 

804
00:47:15,800 --> 00:47:18,960
assumptions or a particularly 
important surfacing, how could 

805
00:47:18,960 --> 00:47:20,360
this go wrong? 
How could this increase 

806
00:47:20,360 --> 00:47:23,200
inequalities? 
I would again, thinking about 

807
00:47:23,200 --> 00:47:27,320
the counterfactual, I, I think 
I'd advocate for using even the 

808
00:47:27,320 --> 00:47:31,360
basic model there over nothing. 
But of course, I absolutely 

809
00:47:31,360 --> 00:47:33,880
agree with you. 
I think that will probably, 

810
00:47:34,560 --> 00:47:38,160
hopefully over the next few 
years, be a lot sharper than it 

811
00:47:38,160 --> 00:47:41,400
is right now with regards to 
these special specialist and 

812
00:47:41,960 --> 00:47:43,960
LMS. 
Yeah. 

813
00:47:44,440 --> 00:47:47,200
And something we're kind of been
somewhat touching upon, but 

814
00:47:47,200 --> 00:47:51,480
maybe not explicitly is that you
have synthetic users, for 

815
00:47:51,480 --> 00:47:53,600
example, or like some synthetic 
participants. 

816
00:47:53,600 --> 00:47:55,840
Like we've, I think we've talked
about it in some ways around 

817
00:47:55,840 --> 00:48:01,600
kind of getting, in this case, 
someone who is maybe in policy 

818
00:48:01,600 --> 00:48:05,760
to be able to get some 
perspectives on something, even 

819
00:48:05,760 --> 00:48:09,640
though they might not actually 
speak to those real people, but 

820
00:48:09,640 --> 00:48:13,720
from some perspectives of 
potentially interesting target 

821
00:48:13,720 --> 00:48:15,480
groups. 
And I think that's a really 

822
00:48:15,480 --> 00:48:18,080
interesting discussion point 
around synthetic users because 

823
00:48:18,600 --> 00:48:24,640
my immediate reaction basically 
was like, that is bullshit. 

824
00:48:25,360 --> 00:48:28,640
Like how can we how we get any 
value from something like 

825
00:48:28,640 --> 00:48:34,920
synthetically generated and then
drains the kind of the real 

826
00:48:35,240 --> 00:48:38,480
human experience from, from our 
understanding. 

827
00:48:38,480 --> 00:48:42,720
And also if that's trained on 
again, some form of very weird 

828
00:48:43,000 --> 00:48:46,960
data sources, we also then kind 
of reinforce a very weird view 

829
00:48:46,960 --> 00:48:49,200
of maybe the context or the 
thing we're trying to change. 

830
00:48:49,680 --> 00:48:52,280
But then I think the main thing 
that started to shift my 

831
00:48:52,280 --> 00:48:55,760
thinking was thinking about the 
counterfactual in many ways. 

832
00:48:56,360 --> 00:49:02,560
And yeah, getting a little bit 
deeper and exploring how things 

833
00:49:02,560 --> 00:49:05,080
that are alternatives to 
synthetic users that are very 

834
00:49:05,080 --> 00:49:07,680
well adopted today, how they 
would perform. 

835
00:49:07,680 --> 00:49:09,800
So for example, I would say 
something like a focus group is 

836
00:49:09,800 --> 00:49:14,480
still very, very widely popular,
used to understand user behavior

837
00:49:14,480 --> 00:49:18,200
or like user research. 
But if I would try to make some 

838
00:49:18,200 --> 00:49:20,160
better right now, I would say 
synthetic users are probably 

839
00:49:20,160 --> 00:49:25,480
more useful to get useful data 
on some form of the context than

840
00:49:25,600 --> 00:49:28,320
a focus group, for example. 
Yeah, because of the the 

841
00:49:28,320 --> 00:49:30,960
limitations of what happens when
you put people in the room and 

842
00:49:30,960 --> 00:49:34,000
and get them to be prompted to 
talk about certain things. 

843
00:49:34,240 --> 00:49:37,360
It's not usually what comes out 
of there is not the real world 

844
00:49:37,360 --> 00:49:39,080
experience of those people. 
It's kind of what's been 

845
00:49:39,080 --> 00:49:40,880
prompted in that room to get out
of them. 

846
00:49:41,840 --> 00:49:45,000
So yeah, setting that thing with
synthetic users, what do you 

847
00:49:45,000 --> 00:49:46,080
think about that? 
Yeah. 

848
00:49:46,480 --> 00:49:52,760
So I think the most excited I've
got about synthetic users is 

849
00:49:52,840 --> 00:49:56,320
again, if you're thinking about 
this fast-paced, high pressure 

850
00:49:56,320 --> 00:49:58,800
crisis type environment that 
governments often find 

851
00:49:58,800 --> 00:50:02,760
themselves in, where they are 
incredibly reactive to events, 

852
00:50:02,760 --> 00:50:08,080
world events, local events and 
have to quickly put out messages

853
00:50:08,480 --> 00:50:12,240
that's, you know, inform people 
of the situation, know what they

854
00:50:12,240 --> 00:50:14,560
should be doing, give an update 
and so on. 

855
00:50:15,160 --> 00:50:17,320
And I don't know about you in 
Sweden. 

856
00:50:17,320 --> 00:50:20,760
I've definitely here in Germany.
I don't want to talk bad about 

857
00:50:21,120 --> 00:50:25,640
anyone in the UK now I'm joking.
I I have seen quite a few of 

858
00:50:25,640 --> 00:50:29,440
those kind of government 
responses to crises that are not

859
00:50:29,440 --> 00:50:31,240
that great. 
You know, you might 

860
00:50:31,240 --> 00:50:34,400
misunderstand some messages. 
They aren't clearly expressed. 

861
00:50:34,720 --> 00:50:38,320
They might actually just not so 
if your information needs right 

862
00:50:38,320 --> 00:50:40,280
now because you're a part of a 
specific group and you have 

863
00:50:40,280 --> 00:50:44,240
specific questions. 
And for these kind of cases, 

864
00:50:44,360 --> 00:50:49,520
that's where I think actually 
having almost a user test, but a

865
00:50:49,520 --> 00:50:52,720
quick one because you can't, you
often don't have time in crisis 

866
00:50:52,720 --> 00:50:54,960
to actually do anything else, 
even if it was marginally 

867
00:50:54,960 --> 00:50:56,880
better. 
That's an empirical question, 

868
00:50:56,960 --> 00:50:59,400
right? 
What gives us better insights, a

869
00:50:59,560 --> 00:51:02,720
focus group versus large base of
synthetic users. 

870
00:51:03,000 --> 00:51:05,640
Agreeing with you on some of the
downfalls of the focus groups, 

871
00:51:05,640 --> 00:51:08,640
but also obviously also some 
downforce with synthetic data, 

872
00:51:08,640 --> 00:51:10,520
particularly once it starts 
feeding itself. 

873
00:51:12,560 --> 00:51:15,160
But for these kind of fast 
decision making processes, I 

874
00:51:15,160 --> 00:51:18,800
think this could be actually 
quite a game changer and 

875
00:51:18,800 --> 00:51:21,680
providing clarity to different 
segments of the population that 

876
00:51:21,680 --> 00:51:24,640
you'd usually otherwise not even
possibly think about. 

877
00:51:25,240 --> 00:51:29,160
So I think that's my kind of 
like very excited use case I 

878
00:51:29,160 --> 00:51:32,360
see. 
And then I think I'm possibly 

879
00:51:32,360 --> 00:51:35,200
not technically advanced to kind
of really make a judgement on 

880
00:51:35,200 --> 00:51:39,360
whether or not my fantasy is 
about interacting quote, UN 

881
00:51:39,360 --> 00:51:43,080
quote with synthetic users that 
are very present in the 

882
00:51:43,080 --> 00:51:47,000
Internet, in the dark web, but 
less present in our research 

883
00:51:47,000 --> 00:51:49,560
laboratory and definitely not 
present in our focus groups. 

884
00:51:50,240 --> 00:51:53,640
Whether or not there is actually
something there that I got kind 

885
00:51:53,640 --> 00:51:55,840
of a few years ago. 
You know, we have so many 

886
00:51:55,840 --> 00:51:59,940
conflicts where there are groups
that do not talk to the quote UN

887
00:51:59,940 --> 00:52:02,480
quote establishment. 
And, you know, universities and 

888
00:52:02,480 --> 00:52:05,240
research institutions are are 
seeing a part of that. 

889
00:52:05,240 --> 00:52:08,440
And it takes us often a long 
time if we're looking at certain

890
00:52:08,440 --> 00:52:11,800
misinformation actors in crisis 
right now, for instance, as 

891
00:52:11,800 --> 00:52:15,320
researchers and projects I'm 
involved in, it's difficult and 

892
00:52:15,320 --> 00:52:17,960
dangerous to get involved with 
these actors. 

893
00:52:19,040 --> 00:52:22,800
So are there cases here where we
can actually interact with them 

894
00:52:22,800 --> 00:52:25,440
with quote, UN quote in 
different ways through the 

895
00:52:25,440 --> 00:52:28,960
choices they left on the 
Internet, which again, I'm quite

896
00:52:28,960 --> 00:52:35,200
excited about. 
Yeah, I do think there is. 

897
00:52:37,080 --> 00:52:41,160
It's a big one in some ways to 
unpack in terms of how we, how 

898
00:52:41,160 --> 00:52:44,880
we think about that. 
But I do think that in terms of,

899
00:52:44,880 --> 00:52:51,120
again, we'll go back to kind of 
limitations of are human. 

900
00:52:51,640 --> 00:52:54,560
So I think like what we're good 
at and what we're bad at in 

901
00:52:54,560 --> 00:52:56,240
general. 
The really tricky thing is that 

902
00:52:56,240 --> 00:52:59,040
we're we're good at 
understanding people that are 

903
00:52:59,040 --> 00:53:02,320
similar to ourselves. 
And we're often times very, very

904
00:53:02,320 --> 00:53:04,600
bad at making some 
representation of other people's

905
00:53:04,600 --> 00:53:08,240
beliefs that are dissimilar to 
to ourselves. 

906
00:53:08,640 --> 00:53:10,600
And like, I think what you 
describe is kind of like taking 

907
00:53:10,600 --> 00:53:13,720
that to really interesting 
extreme in some ways where it 

908
00:53:13,720 --> 00:53:17,960
becomes even more absurd when 
you when there's some form of, 

909
00:53:18,440 --> 00:53:26,760
yeah, really extremist or type 
of actors that, yeah, if you're 

910
00:53:27,160 --> 00:53:32,320
a very kind of pro democracy 
liberal, it it's probably really

911
00:53:32,320 --> 00:53:36,400
hard to, to put yourself truly 
in the perspective of that. 

912
00:53:36,680 --> 00:53:41,120
Even if you just had A and start
to try to think about it, that's

913
00:53:41,320 --> 00:53:44,080
probably not going to work. 
And at the same time as well, no

914
00:53:44,080 --> 00:53:45,640
one from that side want to talk 
to you as well. 

915
00:53:45,640 --> 00:53:48,600
So if you wanted to reach out to
them and try to have a dialogue 

916
00:53:48,600 --> 00:53:50,600
of sort of like, that's probably
not going to happen as well. 

917
00:53:51,240 --> 00:53:54,480
Yeah. 
So, yeah, I do think it's it is 

918
00:53:54,480 --> 00:53:56,960
interesting, yeah. 
Yeah, and I often, I mean, if 

919
00:53:56,960 --> 00:54:00,280
you were to look at my YouTube 
feed, in fairness, it looks like

920
00:54:00,280 --> 00:54:03,360
I am an absolute right wing, 
right wing. 

921
00:54:03,760 --> 00:54:07,080
Concluding, a lot of the content
I seek out for this purpose, to 

922
00:54:07,840 --> 00:54:10,440
be a better behavioral 
scientist, I suppose, is to try 

923
00:54:10,440 --> 00:54:15,640
and expose myself to narratives 
that I find very puzzling to try

924
00:54:15,640 --> 00:54:17,360
and try and understand the 
reasoning. 

925
00:54:17,360 --> 00:54:20,720
Very similar to your example of 
what you've done with the LM, 

926
00:54:20,720 --> 00:54:22,680
where you, you know, put a 
different journalistic entrance 

927
00:54:22,680 --> 00:54:26,120
around complex topics. 
I often then purposely also not 

928
00:54:26,120 --> 00:54:28,280
just seek out topics I don't 
understand, but seek out topics 

929
00:54:28,280 --> 00:54:31,840
where I just cannot. 
I cannot possibly understand 

930
00:54:31,840 --> 00:54:37,760
someone else's perspective and 
train myself almost to take 

931
00:54:37,760 --> 00:54:40,240
those perspectives, which I 
think some of my friends and 

932
00:54:40,240 --> 00:54:43,280
family find quite irritating 
because they've been getting 

933
00:54:43,280 --> 00:54:46,760
very emotionally upset about 
public world event. 

934
00:54:46,760 --> 00:54:49,280
And I end up saying, Oh well, 
well, from their perspective, 

935
00:54:49,280 --> 00:54:51,720
maybe it looks like I love. 
That's a bit as I see as well, 

936
00:54:51,720 --> 00:54:54,480
of course, because it kind of 
explains away everything and 

937
00:54:54,480 --> 00:54:57,360
takes away most judgments. 
But I think for behavioral 

938
00:54:57,360 --> 00:55:01,000
science, expertise is actually 
probably quite a useful skill to

939
00:55:01,000 --> 00:55:05,080
be striving for. 
Yeah, well, I can you say for 

940
00:55:05,080 --> 00:55:07,520
most, like on a personal level, 
I can, I can relate a lot to 

941
00:55:07,520 --> 00:55:09,560
that. 
I think there is, it's 

942
00:55:10,120 --> 00:55:13,160
interesting in terms like now 
there's a big wave of people 

943
00:55:13,160 --> 00:55:17,200
moving to to blue sky from from 
Twitter or X. 

944
00:55:17,800 --> 00:55:22,280
And I think the first person who
who had like a million more 

945
00:55:22,280 --> 00:55:29,200
followers were the AOC, the 
Alexander Oktas, Cortez and kind

946
00:55:29,200 --> 00:55:31,400
of speaking to the kind of like 
the liberal side that kind of 

947
00:55:31,400 --> 00:55:35,440
flows into the blue sky. 
And then I was left on on extra 

948
00:55:35,440 --> 00:55:39,080
Twitter. 
It's it's maybe then the other 

949
00:55:39,080 --> 00:55:42,880
side in some ways. 
And for me, like I was never on 

950
00:55:43,880 --> 00:55:49,360
Twitter for getting a balanced 
intellectual perspective that 

951
00:55:49,360 --> 00:55:51,600
was very thoughtful. 
I, I, I've always thought it was

952
00:55:51,600 --> 00:55:55,160
interesting to kind of both 
follow maybe obviously the 

953
00:55:55,160 --> 00:55:57,480
things that I do for my work in 
terms of some science research 

954
00:55:57,480 --> 00:56:02,120
updates or AI updates, but also 
understanding like what are the 

955
00:56:02,120 --> 00:56:03,640
people that I strongly disagree 
with? 

956
00:56:03,640 --> 00:56:07,080
Like how are they commuting 
getting their points of view? 

957
00:56:07,320 --> 00:56:11,200
What are the followers of those 
people responding to or like, 

958
00:56:11,680 --> 00:56:14,800
and how is that happening? 
Like I, I find that deeply 

959
00:56:14,800 --> 00:56:16,280
fascinating as well. 
And you'll probably find some 

960
00:56:16,280 --> 00:56:20,240
similar friend on my on my 
YouTube feed as well. 

961
00:56:20,240 --> 00:56:26,600
Like I, I do find it a little 
bit, yeah, limiting to just like

962
00:56:26,600 --> 00:56:29,480
I watch someone like Daily Show.
I love The Daily Show with Jon 

963
00:56:29,480 --> 00:56:31,360
Stewart. 
It's really fun, really great. 

964
00:56:31,360 --> 00:56:33,520
But obviously like they're, 
they're used to making fun of 

965
00:56:33,520 --> 00:56:37,200
the other side and it's just 
trying to satirize and just make

966
00:56:37,200 --> 00:56:41,880
a straw man in all capacity. 
And so for that reason, I find 

967
00:56:41,880 --> 00:56:46,120
it as interesting to kind of 
look at the the counter to what 

968
00:56:46,120 --> 00:56:49,560
that is, what it does, the 
Babylon Bee or whatever. 

969
00:56:49,560 --> 00:56:51,480
And you speak like, OK, what 
did, what did they find funny? 

970
00:56:51,480 --> 00:56:53,840
Like what are they actually 
trying to pick up on? 

971
00:56:53,840 --> 00:56:55,600
And I think it's really 
interesting to to form that 

972
00:56:55,600 --> 00:56:58,240
perspective. 
So yeah, I'm with you at least, 

973
00:56:58,400 --> 00:56:59,200
yeah. 
Yeah. 

974
00:56:59,680 --> 00:57:02,160
And to draw the parallels of 
back to behavioral science, I 

975
00:57:02,160 --> 00:57:05,760
think apart from helping 
airlines to try and get some of 

976
00:57:05,760 --> 00:57:08,760
this perspective in our current 
thinking and challenge our own 

977
00:57:08,760 --> 00:57:13,120
assumptions as behavioral 
scientists, it's also about 

978
00:57:13,280 --> 00:57:16,560
providing us with assessment of 
how our interventions could be 

979
00:57:16,560 --> 00:57:19,440
perceived by different groups 
and where it could go wrong. 

980
00:57:20,040 --> 00:57:25,280
So I think I'm really a big fan 
of LMS for things like red 

981
00:57:25,280 --> 00:57:29,040
teaming, challenging your exact 
existing thinking. 

982
00:57:29,040 --> 00:57:31,280
I don't know if you have these 
discussions of like persuade me 

983
00:57:31,680 --> 00:57:36,040
of the opposite from myself, you
know, to try and cover potential

984
00:57:36,040 --> 00:57:38,800
aspects. 
That's where I see it as 

985
00:57:38,800 --> 00:57:40,520
helpful. 
But again, I think a lot of it 

986
00:57:40,520 --> 00:57:44,000
actually comes down to people's 
own preferences as well and how 

987
00:57:44,000 --> 00:57:45,960
much they like these kind of 
challenges and how much they 

988
00:57:45,960 --> 00:57:47,600
have headspace. 
My God, headspace. 

989
00:57:47,600 --> 00:57:49,920
So important, right? 
So like, just go around and 

990
00:57:49,920 --> 00:57:52,640
circle, have a big old 
discussion with an LLM about a 

991
00:57:52,640 --> 00:57:56,920
topic you might disagree with. 
That is obviously a big 

992
00:57:56,920 --> 00:58:00,360
privilege to be able to, you 
know, find yourself in a space 

993
00:58:00,360 --> 00:58:03,240
where you actually have some of 
the time to do that kind of 

994
00:58:04,360 --> 00:58:06,160
partially self partially 
professional improvement 

995
00:58:06,160 --> 00:58:08,000
activities. 
Yeah. 

996
00:58:08,440 --> 00:58:10,960
And just underline that, I think
that is probably what people 

997
00:58:11,040 --> 00:58:14,240
often times get wrong maybe is 
that when we're talking about 

998
00:58:14,240 --> 00:58:19,600
this in this context is that 
we're kind of, yeah, looking to 

999
00:58:19,800 --> 00:58:22,680
some more AI model to think on 
our behalf. 

1000
00:58:22,680 --> 00:58:24,720
And I think that's the danger. 
Like it's always dangerous when 

1001
00:58:24,720 --> 00:58:27,440
you just go there to just be 
like, hey, I don't know anything

1002
00:58:27,440 --> 00:58:30,480
about this, Just tell me what I 
should think about it. 

1003
00:58:30,480 --> 00:58:32,960
And honestly, to be fair, 
though, I don't know how harmful

1004
00:58:32,960 --> 00:58:35,120
AI is. 
Humans are more harmful in that 

1005
00:58:35,120 --> 00:58:36,480
degree. 
Humans will probably say 

1006
00:58:36,480 --> 00:58:39,440
something very, very extreme on 
one side or the other, like, 

1007
00:58:39,440 --> 00:58:41,600
yeah, this is good or this is 
terrible. 

1008
00:58:41,840 --> 00:58:43,960
Yeah, large sound usually pretty
balanced. 

1009
00:58:43,960 --> 00:58:46,040
They're like, yeah, it depends. 
It can be like both of this. 

1010
00:58:46,040 --> 00:58:49,240
And this used to say that. 
I don't think that probably 

1011
00:58:49,240 --> 00:58:53,160
that's too bad itself. 
But I think what we're talking 

1012
00:58:53,160 --> 00:58:56,880
about here is like when it can 
be really useful is to when you 

1013
00:58:56,880 --> 00:58:59,480
have opinions, when you have 
thoughts, when you have kind of 

1014
00:58:59,480 --> 00:59:02,800
perspectives to challenge them 
and kind of build on them and 

1015
00:59:02,800 --> 00:59:06,720
and get them to become more. 
Yeah, test that basically and 

1016
00:59:06,720 --> 00:59:09,360
see in your blind spots. 
Yeah, What I was just thinking 

1017
00:59:09,360 --> 00:59:13,200
about as well is in our kind of 
frameworks of influences on 

1018
00:59:13,200 --> 00:59:16,920
behaviors. 
We obviously have things like 

1019
00:59:16,920 --> 00:59:20,520
knowledge and motivations and 
whatever else it is in our 

1020
00:59:20,520 --> 00:59:24,440
framework of choice. 
But I reckon what will at some 

1021
00:59:24,440 --> 00:59:28,480
point become a big influence is 
how large language models 

1022
00:59:29,040 --> 00:59:31,160
interact with a person about 
that topic. 

1023
00:59:31,160 --> 00:59:34,560
If they do choose to, you know, 
talk to talk to an LLM about it 

1024
00:59:34,560 --> 00:59:37,560
or to whichever means they are 
actually influenced through it. 

1025
00:59:37,880 --> 00:59:40,480
So we should probably also stay 
on top and you know, the 

1026
00:59:40,480 --> 00:59:42,360
question of like, how does the 
LLM behave? 

1027
00:59:42,960 --> 00:59:45,360
This is also for us as 
behavioral science to figure out

1028
00:59:45,480 --> 00:59:48,200
how to do LLMS actually impact 
people's thinking. 

1029
00:59:48,600 --> 00:59:51,880
I can certainly say my thinking 
processes have been quite 

1030
00:59:51,880 --> 00:59:54,760
dramatically influenced. 
I would I would say positively, 

1031
00:59:54,760 --> 00:59:58,240
but you know, like actually 
using an LLM almost as a 

1032
00:59:58,280 --> 01:00:02,040
therapist to talk through 
personal life situations to talk

1033
01:00:02,040 --> 01:00:05,920
through difficult decisions. 
Again, I often try to have a 

1034
01:00:06,080 --> 01:00:10,960
blank slate one to avoid this 
kind of yes, yes Sir, master you

1035
01:00:10,960 --> 01:00:13,720
are amazing type interactions or
talent explicitly. 

1036
01:00:13,720 --> 01:00:15,440
I got peace. 
Don't be. 

1037
01:00:15,440 --> 01:00:17,800
Don't be so. 
What's a better word for us? 

1038
01:00:17,800 --> 01:00:19,960
Licking. 
Don't be. 

1039
01:00:20,720 --> 01:00:23,120
Like, I'm not here to be told 
I'm great. 

1040
01:00:23,120 --> 01:00:25,520
I'm here to actually get some 
challenge and some support. 

1041
01:00:26,160 --> 01:00:30,520
But there are also some moments 
where all you need is someone to

1042
01:00:30,520 --> 01:00:35,080
listen to you and to give you to
mirror back as to what you've 

1043
01:00:35,080 --> 01:00:38,080
said to kind of maybe structure 
some of your thinking. 

1044
01:00:39,240 --> 01:00:42,200
And I find it also very, very 
helpful that, but of course, 

1045
01:00:42,200 --> 01:00:46,120
this counts as an interaction 
that if it was between a human 

1046
01:00:46,120 --> 01:00:49,040
and a human, we would take as an
influence on behavior. 

1047
01:00:50,320 --> 01:00:53,240
So I think there's more that we 
can probably do as behavioral 

1048
01:00:53,240 --> 01:00:55,960
scientists and kind of 
understanding how it takes 

1049
01:00:55,960 --> 01:00:58,280
behaviors. 
And I mean, we've seen research 

1050
01:00:58,280 --> 01:01:02,200
come out that it can persuade 
people and so on, but in a more 

1051
01:01:02,200 --> 01:01:04,040
kind of less experimental 
setting. 

1052
01:01:04,840 --> 01:01:08,160
That'll be cool to understand. 
Yeah, Yeah, I agree. 

1053
01:01:08,640 --> 01:01:13,880
And I guess to in some ways 
shake your thoughts on various 

1054
01:01:13,880 --> 01:01:16,200
things that we maybe haven't had
a chance to talk about as much 

1055
01:01:16,400 --> 01:01:20,360
or not segue into our quick fire
round of the season called to AI

1056
01:01:20,360 --> 01:01:24,600
or not to AI. 
And so basically, I've written 

1057
01:01:24,600 --> 01:01:27,760
down a bunch of different things
that AI potentially could do. 

1058
01:01:28,080 --> 01:01:30,640
But I guess my question will be 
basically to use like should it 

1059
01:01:30,640 --> 01:01:32,680
do this? 
This is a good use case for AI. 

1060
01:01:34,280 --> 01:01:36,040
So yeah. 
And let's start the quick fire 

1061
01:01:36,040 --> 01:01:42,240
round with to AI or Not to AI? 
Deploy AI powered urinals to 

1062
01:01:42,240 --> 01:01:45,080
provide real time gamified 
cleanliness feedback. 

1063
01:01:45,760 --> 01:01:50,120
Yeah, why not? 
I was trying to look for like, 

1064
01:01:50,120 --> 01:01:54,040
what is a classic nudge that 
could be AI and then that that 

1065
01:01:54,040 --> 01:01:57,800
was the first one. 
OK, to AI or not to AI? 

1066
01:01:57,800 --> 01:02:01,520
Calculate individual tax rates 
based on lifestyle and spending 

1067
01:02:01,520 --> 01:02:05,840
habits. 
Oh, that's, but that's a policy 

1068
01:02:05,840 --> 01:02:07,560
question. 
That's not an AI or not an AI 

1069
01:02:07,560 --> 01:02:08,720
question. 
Well. 

1070
01:02:08,760 --> 01:02:11,040
It's to use, it's to use it in 
the policy context. 

1071
01:02:11,040 --> 01:02:13,400
Yeah. 
In that context of like having 

1072
01:02:13,400 --> 01:02:18,760
more personalized taxes based on
some form of in the same as you,

1073
01:02:18,760 --> 01:02:22,920
maybe you would do some form of 
actuarial risk assessment for 

1074
01:02:22,920 --> 01:02:25,960
insurance purposes, like 
depending on how you spend your 

1075
01:02:25,960 --> 01:02:28,240
money, like how you should pay 
taxes. 

1076
01:02:30,120 --> 01:02:33,600
Interesting. 
I don't think I can give a clear

1077
01:02:33,600 --> 01:02:36,000
yes, no because it would very 
much depend on the design of it.

1078
01:02:36,480 --> 01:02:38,480
Sorry I'm falling back into 
policy speak, but I that's. 

1079
01:02:38,720 --> 01:02:41,160
That's. 
Good, because I like, I think I 

1080
01:02:41,160 --> 01:02:43,840
will be very open to exploring 
the idea and to assessing the 

1081
01:02:43,840 --> 01:02:45,680
associated risks. 
Yeah. 

1082
01:02:45,680 --> 01:02:47,960
So I wouldn't. 
I wouldn't dismiss it entirely, 

1083
01:02:48,240 --> 01:02:50,320
but annoying answer. 
But it depends, no? 

1084
01:02:51,360 --> 01:02:53,640
That's that's that's the payroll
science answer. 

1085
01:02:54,080 --> 01:02:57,880
OK, Evaluate and assign 
household chores to family 

1086
01:02:57,880 --> 01:03:01,520
members. 
Yeah, I already say yes, yes, 

1087
01:03:01,600 --> 01:03:05,720
yeah, yeah, for sure. 
I think probably depending on 

1088
01:03:06,400 --> 01:03:08,640
people's perceptions of their 
own levels of emotional and 

1089
01:03:08,640 --> 01:03:12,680
other labor in a family context.
No, I, I think this would 

1090
01:03:12,680 --> 01:03:14,480
actually make sense. 
And it would also make sense to 

1091
01:03:14,960 --> 01:03:18,040
try and get it to play towards 
someone's strengths and, you 

1092
01:03:18,040 --> 01:03:20,960
know, to pick up like help with 
scheduling around some of the 

1093
01:03:20,960 --> 01:03:23,160
tasks and so on. 
I think there's possibly quite a

1094
01:03:23,160 --> 01:03:27,520
lot that could be gained around,
you know, improving common 

1095
01:03:27,520 --> 01:03:32,840
issues in families, yeah. 
OK, now we're getting into 

1096
01:03:32,840 --> 01:03:36,760
another policy, one managing 
disaster response and relief 

1097
01:03:36,760 --> 01:03:38,960
funds. 
So, allocation of emergency 

1098
01:03:38,960 --> 01:03:44,760
resources in disasters. 
I think I could see how that 

1099
01:03:44,760 --> 01:03:50,120
could be quite a good one, 
especially if we're thinking 

1100
01:03:50,120 --> 01:03:53,320
that we're currently restrained 
in that through like personal 

1101
01:03:53,320 --> 01:03:56,960
views, constrained data of 
hierarchical decision making and

1102
01:03:56,960 --> 01:03:59,800
power structures. 
This could actually lead to some

1103
01:03:59,800 --> 01:04:03,720
more fairness if done well, 
which I'd generally be in favor 

1104
01:04:03,720 --> 01:04:05,640
of. 
Nice. 

1105
01:04:05,800 --> 01:04:10,800
OK, decide when you're ready for
promotion based on performance 

1106
01:04:10,800 --> 01:04:16,920
data. 
Again, I wouldn't dismiss that 

1107
01:04:16,920 --> 01:04:19,920
as a thing, because I think 
performance and promotion 

1108
01:04:19,920 --> 01:04:26,560
decisions are also so often made
on vibes, and it might be 

1109
01:04:26,560 --> 01:04:28,920
interesting to take a think 
about what would happen if vibes

1110
01:04:28,920 --> 01:04:30,080
aren't part of the equation 
anymore. 

1111
01:04:30,080 --> 01:04:32,600
Does it actually become fairer 
or does the system collapse 

1112
01:04:32,600 --> 01:04:34,720
because people stop being able 
to work with each other? 

1113
01:04:36,480 --> 01:04:39,600
So. 
Again, very often to 

1114
01:04:39,600 --> 01:04:42,440
experimentation around it and 
what would happen as an 

1115
01:04:42,440 --> 01:04:47,160
unintended side effect. 
OK, provide hyper personalized 

1116
01:04:47,160 --> 01:04:50,240
voting reminders tailored to 
past behavior and preferences. 

1117
01:04:51,040 --> 01:04:54,040
Yeah, nothing to lose there. 
Remind them all they want. 

1118
01:04:55,440 --> 01:04:58,840
It seems like a no brainer to 
use more personalized 

1119
01:04:59,040 --> 01:05:00,880
initiatives to get people to 
vote, right? 

1120
01:05:02,480 --> 01:05:03,880
Yeah. 
I mean, I'm sure we can come out

1121
01:05:03,880 --> 01:05:06,560
with 10 risks of this as well, 
of course. 

1122
01:05:06,760 --> 01:05:10,600
But I think overall, I would 
think that if, especially if 

1123
01:05:10,600 --> 01:05:13,320
people are able to opt into it, 
that'd be great. 

1124
01:05:15,360 --> 01:05:19,600
Final one, create tax return, 
not just optimize deductions and

1125
01:05:19,600 --> 01:05:21,680
encourage social beneficial 
contributions. 

1126
01:05:26,800 --> 01:05:30,560
I'm not sure about that one 
because who decides what's 

1127
01:05:30,560 --> 01:05:33,800
socially beneficial? 
I think that's possibly like the

1128
01:05:33,800 --> 01:05:37,400
but I'd get hung up, hung up on 
on this one and I might come 

1129
01:05:37,400 --> 01:05:39,880
back to the urinal. 
Do we mean that there will be a 

1130
01:05:39,880 --> 01:05:43,640
camera installed? 
I was just thinking I was way 

1131
01:05:43,640 --> 01:05:45,760
too quick to say yes to 
something that wouldn't impact 

1132
01:05:45,760 --> 01:05:47,680
me. 
But some people might say, hey, 

1133
01:05:47,680 --> 01:05:53,320
Laura, that big camera, are we 
assuming the camera? 

1134
01:05:53,320 --> 01:05:57,240
In which case I might say no for
the protection of men. 

1135
01:05:59,440 --> 01:06:02,560
Yeah, yeah, no, maybe it would 
be one of those cameras that is 

1136
01:06:02,560 --> 01:06:07,360
actually not picking up on exact
film footage, but more something

1137
01:06:07,360 --> 01:06:10,680
more vaguer or I don't know. 
Yeah, it's it's, I actually 

1138
01:06:10,680 --> 01:06:11,960
didn't think about that far as 
well. 

1139
01:06:12,200 --> 01:06:14,000
That's a good. 
Point You're the design guru. 

1140
01:06:14,080 --> 01:06:16,320
You can make something, make 
something happen some. 

1141
01:06:16,800 --> 01:06:20,760
I'll ask Larry David, he has 
some ideas as well about urinal 

1142
01:06:20,760 --> 01:06:22,080
improvements and stuff like 
this. 

1143
01:06:23,160 --> 01:06:26,720
OK, that was really good, well 
done. 

1144
01:06:26,720 --> 01:06:29,600
It was very, I think I selected 
some hard ones for you, I 

1145
01:06:29,600 --> 01:06:34,360
realized in hindsight, but kudos
for going through them and to 

1146
01:06:34,360 --> 01:06:36,600
not make any easier. 
The final question for you is 

1147
01:06:37,200 --> 01:06:39,880
what do you feel like is your 
most controversial opinion about

1148
01:06:40,000 --> 01:06:43,680
AI? 
My most controversial opinion, I

1149
01:06:43,720 --> 01:06:46,840
think my this opinion was very 
controversial around 3-4 years 

1150
01:06:46,840 --> 01:06:50,680
ago, where if you happen to be 
stuck in a pub with me, you'd 

1151
01:06:50,800 --> 01:06:54,480
hear me talk about my big 
fantasy that when I'm old I 

1152
01:06:54,480 --> 01:06:58,200
don't want to be lonely. 
And what I want is an AI 

1153
01:06:58,200 --> 01:07:02,880
companion, someone that I would 
give them access to all my data,

1154
01:07:02,880 --> 01:07:05,200
all my, you know, WhatsApp, 
social media data over the last.

1155
01:07:05,200 --> 01:07:10,280
I don't know when am I going to 
die in the last 60 years and I'm

1156
01:07:10,280 --> 01:07:14,440
going to have the perfect person
to interact with someone that's 

1157
01:07:14,600 --> 01:07:17,000
not just going to laugh about my
jokes, but going to tell me 

1158
01:07:17,000 --> 01:07:19,560
jokes that I find funny. 
Someone that can remind me of 

1159
01:07:19,560 --> 01:07:22,720
things that bring me joy. 
Ideally want some sort of bodily

1160
01:07:22,720 --> 01:07:25,680
feedback so it optimizes for my 
positive emotions. 

1161
01:07:26,480 --> 01:07:29,440
And I think that was possibly 
quite controversial back then, 

1162
01:07:30,080 --> 01:07:33,240
that idea that I'm striving for 
the perfect AI companion when 

1163
01:07:33,240 --> 01:07:35,760
I'm old. 
I think nowadays with people 

1164
01:07:35,760 --> 01:07:37,960
interacting with the GBT 
actually seeing some of the 

1165
01:07:37,960 --> 01:07:40,280
potential benefits that can 
envision that more. 

1166
01:07:40,280 --> 01:07:42,760
Although when I say it the way I
say it, people still have a bit 

1167
01:07:42,760 --> 01:07:45,560
of reaction to it. 
So it's not terribly 

1168
01:07:45,560 --> 01:07:49,840
controversial, but it's possibly
being us round circle in terms 

1169
01:07:49,840 --> 01:07:53,040
of my my own entry to AI, which 
was probably actually thinking 

1170
01:07:53,040 --> 01:07:55,640
about the loneliness question in
a in old age. 

1171
01:07:56,200 --> 01:07:58,160
Yeah, no, I love that. 
That's a great one. 

1172
01:07:58,160 --> 01:08:00,320
And honestly, I love this 
conversation. 

1173
01:08:00,320 --> 01:08:02,480
It was so much fun. 
I feel like we could have, I've 

1174
01:08:02,480 --> 01:08:06,040
talked for many hours about many
of these topics, but I I'm so 

1175
01:08:06,040 --> 01:08:09,200
happy we had a chance to bring 
onto the podcast to talk to you 

1176
01:08:09,360 --> 01:08:11,400
and bring your perspective and 
insights. 

1177
01:08:11,920 --> 01:08:14,440
Thank you so much for having me.
Yeah, thank you. 

1178
01:08:14,440 --> 01:08:17,760
And thanks also follow the one 
for work you're doing and thanks

1179
01:08:17,760 --> 01:08:21,279
for helping with setting up and 
bringing paper bytes to life as 

1180
01:08:21,279 --> 01:08:22,880
well. 
So yeah, a lot of thank yous. 

1181
01:08:23,880 --> 01:08:25,160
Yeah, I know. 
Thank you for including me and 

1182
01:08:25,160 --> 01:08:26,640
for having me today. 
Such a good chat. 

1183
01:08:28,960 --> 01:08:31,080
And that's a wrap. 
You've been listening to the 

1184
01:08:31,080 --> 01:08:34,399
behavioral design podcast 
brought to you by Habit Weekly 

1185
01:08:34,399 --> 01:08:37,279
and Nuance Behavior. 
Sam and Aline tell me this 

1186
01:08:37,279 --> 01:08:40,600
season is packed with incredible
insights about behavioral design

1187
01:08:40,600 --> 01:08:44,240
and AI, so be sure to subscribe 
and share the podcast with your 

1188
01:08:44,240 --> 01:08:46,040
friends. 
Though you might want to keep it

1189
01:08:46,040 --> 01:08:50,040
away from your enemies. 
In case you haven't noticed, I'm

1190
01:08:50,040 --> 01:08:53,439
an AI voice. 
Yep, pretty crazy. 

1191
01:08:53,760 --> 01:08:55,560
Quite the improvement since last
season's. 

1192
01:08:55,720 --> 01:08:59,439
AI outro, don't you think? 
If you'd like to collaborate 

1193
01:08:59,439 --> 01:09:02,560
with us at Nuance Behavior, 
where we use behavioral design 

1194
01:09:02,560 --> 01:09:05,359
to craft digital products with 
Nuance, e-mail us at 

1195
01:09:05,359 --> 01:09:09,760
hello@nuancebehavior.com or book
a call directly on our website, 

1196
01:09:09,920 --> 01:09:14,439
nuancebehavior.com. 
A special thanks to the amazing 

1197
01:09:14,439 --> 01:09:18,040
Dave Pizarro for our show music 
and to Mei Chen Yap and April 

1198
01:09:18,040 --> 01:09:20,840
English for their help in 
producing and publishing this 

1199
01:09:20,840 --> 01:09:23,080
episode. 
Thanks again for tuning in. 

1200
01:09:23,319 --> 01:09:26,040
We'll be back soon with another 
exciting conversation where 

1201
01:09:26,040 --> 01:09:31,800
behavioral design and AI 
Intersect happens to. 

1202
01:09:34,640 --> 01:09:43,920
Murgatroyd. 
Oh. 

1203
01:10:21,520 --> 01:10:36,840
Oh 
Devero wigs, we guarantee our 

1204
01:10:36,840 --> 01:10:40,080
wigs aren't made from hair off 
dead bodies. 

1205
01:10:44,640 --> 01:10:47,200
I don't like it. 
I don't like it at all. 

1206
01:10:48,400 --> 01:10:50,160
You don't. 
Why not? 

1207
01:10:50,200 --> 01:10:52,960
Why bring up the thing about? 
Dead body hair. 

1208
01:10:52,960 --> 01:10:54,960
Our wigs aren't made with dead 
body hair. 

1209
01:10:55,200 --> 01:10:57,600
Yeah, that's the whole point. 
So why bring it up? 

1210
01:10:57,680 --> 01:11:00,840
Advertising one O 1. 
Answer the question before they 

1211
01:11:00,840 --> 01:11:02,560
ask it. 
Who's asking that question, 

1212
01:11:02,680 --> 01:11:05,040
Rhonda? 
The main reason people don't buy

1213
01:11:05,040 --> 01:11:08,760
wigs is because they're afraid 
they're made from dead people 

1214
01:11:08,760 --> 01:11:09,040
here.