1
00:00:23,510 --> 00:00:25,150
Fuck, were you even doing in 
2008? 

2
00:00:25,270 --> 00:00:28,670
So in computer vision, it was 
SIFT features and SVMS that 

3
00:00:28,670 --> 00:00:30,390
that's basically the 
state-of-the-art at the time. 

4
00:00:30,390 --> 00:00:32,350
There were a couple other things
that people are using, but. 

5
00:00:32,750 --> 00:00:37,200
Not even using GPU's. 
I think that some there was, 

6
00:00:37,280 --> 00:00:40,160
yeah, basically Alex was the 
first one to to write 

7
00:00:40,160 --> 00:00:44,040
specialized kernels. 
There was some applications, I'm

8
00:00:44,040 --> 00:00:47,920
sure where it like, you know, 
optical flow, it's highly 

9
00:00:47,920 --> 00:00:49,360
paralyzable. 
And I wouldn't be surprised if 

10
00:00:49,360 --> 00:00:51,640
there were some labs that were 
doing things along those lines. 

11
00:00:51,640 --> 00:00:54,360
But it definitely was not like a
scaling era for machine 

12
00:00:54,360 --> 00:00:56,840
learning. 
Largest data sets you know you 

13
00:00:56,840 --> 00:01:00,960
could fit on a USB stick. 
What gave you the inspiration to

14
00:01:00,960 --> 00:01:05,840
start Code Jet? 
And Like, you must have seen 

15
00:01:05,840 --> 00:01:08,000
something that was like, oh, 
this is getting better. 

16
00:01:08,000 --> 00:01:10,640
We could probably do something 
here because there was a company

17
00:01:10,640 --> 00:01:12,800
that tried it and then they 
folded, right? 

18
00:01:12,840 --> 00:01:14,160
I can't remember the name of 
them. 

19
00:01:14,440 --> 00:01:16,840
The history of cogeneration is 
littered with companies that 

20
00:01:16,840 --> 00:01:19,000
were a little too early but 
like, totally visionary. 

21
00:01:19,000 --> 00:01:20,360
So you're probably thinking of 
Tab 9. 

22
00:01:20,760 --> 00:01:22,760
I think Tab 9's actually still 
around, and they're still. 

23
00:01:22,760 --> 00:01:26,520
Going and then it wasn't them. 
There was one that height I 

24
00:01:26,520 --> 00:01:28,800
think it was kite in like 2022 
right? 

25
00:01:28,800 --> 00:01:32,240
Or 2020 maybe? 
Maybe even before that a little.

26
00:01:32,240 --> 00:01:37,400
Bit like 2021 right? 
They folded right when Chachi BT

27
00:01:37,400 --> 00:01:40,960
started getting really popular 
and it was very confusing to me 

28
00:01:40,960 --> 00:01:44,640
because I'm like wait GitHub 
copilot is printing money? 

29
00:01:44,640 --> 00:01:48,320
Supposedly they just hit like a 
billion in revenue after they 

30
00:01:48,320 --> 00:01:53,400
released and kite shut down. 
You hate to see it. 

31
00:01:53,480 --> 00:01:55,920
I mean, you know, any start up 
dying. 

32
00:01:55,920 --> 00:01:58,760
It's sad to see. 
I think that one of my learnings

33
00:01:58,760 --> 00:02:01,520
from doing startups for about a 
decade is that momentum begets 

34
00:02:01,520 --> 00:02:04,520
momentum. 
And if you are too early, you 

35
00:02:04,520 --> 00:02:07,120
know, you kind of get people all
hyped up on this vision and then

36
00:02:07,120 --> 00:02:09,039
you're actually not the player 
who ends up bringing it to 

37
00:02:09,039 --> 00:02:10,800
market. 
Maybe some people can kind of 

38
00:02:10,800 --> 00:02:12,320
catch on and they end up taking 
off at that point. 

39
00:02:12,320 --> 00:02:15,160
But in the majority of cases it 
ends up being that. 

40
00:02:15,560 --> 00:02:16,840
Yeah, get up. 
Copilot is the one who 

41
00:02:16,840 --> 00:02:18,600
introduced the concept. 
That's what people get excited 

42
00:02:18,600 --> 00:02:19,720
about. 
That's what gets hyped up on 

43
00:02:19,720 --> 00:02:22,040
Twitter. 
Kite is remembered as like the, 

44
00:02:22,520 --> 00:02:25,840
you know, everybody tried it in 
2021 or something like that when

45
00:02:25,840 --> 00:02:28,360
Paul Graham hyped it up and 
they're like, this isn't really 

46
00:02:28,360 --> 00:02:31,080
ready for production yet. 
And get up Copilot was, you 

47
00:02:31,080 --> 00:02:33,360
know, is directly integrated 
into VS Code from the very get 

48
00:02:33,360 --> 00:02:34,480
go. 
And so you don't really have to 

49
00:02:34,480 --> 00:02:36,520
do very much in order to get 
access to it. 

50
00:02:37,760 --> 00:02:41,120
But then the sort of I think the
bigger revelation was cursor, 

51
00:02:41,400 --> 00:02:44,920
Cursor get up Copilot was 2022, 
I want to say is when that came 

52
00:02:44,920 --> 00:02:48,680
out and it immediately it was in
the happy path, which was 

53
00:02:48,680 --> 00:02:50,000
amazing. 
So you don't really have to do 

54
00:02:50,000 --> 00:02:51,400
very much in order to benefit 
from it, right? 

55
00:02:51,400 --> 00:02:53,240
It would just do auto 
completions for you once you 

56
00:02:53,240 --> 00:02:55,960
flipped it on. 
And I think that that same 

57
00:02:55,960 --> 00:02:58,800
experience of basically not 
having to take any additional 

58
00:02:58,800 --> 00:03:01,040
action, but just having AI 
seamlessly injected into your 

59
00:03:01,040 --> 00:03:02,960
workflow that's going to roll 
out to a bunch of other 

60
00:03:02,960 --> 00:03:07,000
industries very soon. 
I think that cursor is sort of 

61
00:03:07,640 --> 00:03:10,200
the, the thing about cursor was 
nobody could tell you not to 

62
00:03:10,200 --> 00:03:12,600
download it. 
So like most of the people who 

63
00:03:12,600 --> 00:03:14,960
ended up adopting it early on, 
these sort of hype guys on, on 

64
00:03:14,960 --> 00:03:17,000
Twitter, which, you know, I'm 
very much so a part of that 

65
00:03:17,000 --> 00:03:20,160
ecosystem where people who are 
like working at, you know, I 

66
00:03:20,160 --> 00:03:21,880
don't know, JP Morgan or 
something like that. 

67
00:03:22,160 --> 00:03:25,080
They download this editor, they 
swipe their personal credit card

68
00:03:25,080 --> 00:03:27,400
for 20 bucks a month. 
And then for me, the revelation 

69
00:03:27,400 --> 00:03:29,560
was the composer mode where as a
sidebar that pops out and you 

70
00:03:29,560 --> 00:03:32,640
can kind of chat with it really 
kind of hit stride with Claude 

71
00:03:32,640 --> 00:03:36,600
Sonic 3.5, which was probably 
mid 2024, something like that. 

72
00:03:36,960 --> 00:03:38,640
Yeah, that's the inflection 
point right there. 

73
00:03:39,800 --> 00:03:43,440
But then code, Jen, what made 
you want to start it? 

74
00:03:43,480 --> 00:03:45,960
And did you see that? 
How did you see the writing on 

75
00:03:45,960 --> 00:03:49,320
the wall? 
Depends how far you want to go 

76
00:03:49,320 --> 00:03:51,440
back, I would say, you know, 
I've always been interested in 

77
00:03:51,440 --> 00:03:54,240
artificial intelligence. 
You know, I think that the idea 

78
00:03:54,240 --> 00:03:57,160
of the machine writing itself 
and creating itself is this sort

79
00:03:57,160 --> 00:03:58,920
of recursive building and 
something that's always really 

80
00:03:58,920 --> 00:04:01,560
appealed to me going way back 
to, I mean, even high school. 

81
00:04:01,720 --> 00:04:04,000
Is the book good to Lecher Bach?
You might be familiar with it 

82
00:04:04,000 --> 00:04:06,400
sort of like a It's a Pulitzer 
Prize winning book and it's. 

83
00:04:06,600 --> 00:04:10,960
Way overestimate my culture. 
Got it. 

84
00:04:11,000 --> 00:04:12,440
I do not know that at all. 
What is? 

85
00:04:13,040 --> 00:04:14,440
It, it's a book called 
Gerdelescherbach. 

86
00:04:14,440 --> 00:04:18,600
It's by this guy Doug 
Hofstadter, who he sort of like 

87
00:04:18,600 --> 00:04:21,880
invented the Simsys major at 
Stanford in a way, or at least 

88
00:04:21,880 --> 00:04:24,160
the the major that Marissa Mayer
and many other people sort of 

89
00:04:24,160 --> 00:04:26,600
made famous. 
It's like based on this book 

90
00:04:26,600 --> 00:04:28,720
almost or directly reflects the 
contents of the book. 

91
00:04:28,720 --> 00:04:32,000
At least it sort of coincided 
with him writing this book went 

92
00:04:32,000 --> 00:04:35,160
on to win the Pulitzer Prize. 
The core concept of it is you 

93
00:04:35,160 --> 00:04:38,720
have MC Escher, the painter, who
you might know the painting of 

94
00:04:38,720 --> 00:04:39,640
him. 
Yeah, exactly. 

95
00:04:40,440 --> 00:04:42,280
Johann Smaschenbach, the 
musician. 

96
00:04:42,280 --> 00:04:44,560
And then Kurt Godel, I'm 
definitely mispronouncing his 

97
00:04:44,560 --> 00:04:48,720
name, but he's a logician. 
And all their works have some 

98
00:04:48,760 --> 00:04:51,680
inherent properties to them or 
substructure that's shared. 

99
00:04:51,960 --> 00:04:53,760
And usually that has to do with 
self reference. 

100
00:04:54,000 --> 00:04:57,640
So for example, there are 
exactly the stairs, the, you 

101
00:04:57,640 --> 00:04:59,840
know, hands writing themselves. 
And there's actually like a lot 

102
00:04:59,840 --> 00:05:02,080
of like really interesting 
mathematical concepts embedded 

103
00:05:02,080 --> 00:05:05,720
both in Bach's work, whether he 
meant it or not, and MC Escher. 

104
00:05:06,440 --> 00:05:09,120
And then Kurt Godel is famous 
for the essential incompleteness

105
00:05:09,120 --> 00:05:12,200
theorem, which is basically this
idea that any sufficiently 

106
00:05:12,200 --> 00:05:16,240
powerful logical system will 
necessarily end up containing 

107
00:05:16,240 --> 00:05:19,560
things that can neither be true,
proved that are true, but cannot

108
00:05:19,560 --> 00:05:22,200
be proven to be true. 
And the way in which it does 

109
00:05:22,200 --> 00:05:25,240
that is basically it contains 
something that looks like the 

110
00:05:25,240 --> 00:05:28,960
epimetitis paradox, which is 
where it's the guy gets up in 

111
00:05:28,960 --> 00:05:31,640
front of, you know, a, a bunch 
of people up amenities is the 

112
00:05:31,640 --> 00:05:33,960
guy gets up in front of a bunch 
of people in Rome or in Greece 

113
00:05:33,960 --> 00:05:36,720
and he says, citizens of Athens,
I am lying to you. 

114
00:05:37,200 --> 00:05:42,280
And because the statement refers
to itself, then it's the self 

115
00:05:42,280 --> 00:05:44,320
reference is basically what 
enables it to be contradictory. 

116
00:05:44,560 --> 00:05:47,680
So it's a paradox. 
And any sufficiently powerful 

117
00:05:47,680 --> 00:05:51,440
number system can contain a self
reference like that that enables

118
00:05:51,440 --> 00:05:54,040
it to basically say this 
statement is not provable. 

119
00:05:54,720 --> 00:05:57,760
And so anyways, it's a little 
bit of a sidetrack here, but the

120
00:05:58,000 --> 00:06:00,320
interesting part about this book
is it ties together all three of

121
00:06:00,320 --> 00:06:04,120
these different concepts or 
these different, you know, you 

122
00:06:04,120 --> 00:06:07,880
know, prolific people's works, 
MC Escher, JS Bach and Kurt 

123
00:06:07,880 --> 00:06:09,360
Godel. 
And basically it's about 

124
00:06:09,880 --> 00:06:11,680
entities that contain self 
reference. 

125
00:06:11,680 --> 00:06:13,800
And the guy uses that as a 
jumping off point to talk about 

126
00:06:13,840 --> 00:06:15,520
AI. 
And eventually he talks about 

127
00:06:15,520 --> 00:06:18,960
the emergence of consciousness 
comes from systems that 

128
00:06:18,960 --> 00:06:21,880
represent the outside world. 
And part of the representation 

129
00:06:21,880 --> 00:06:24,200
of the outside world in this 
symbolic system contains 

130
00:06:24,200 --> 00:06:26,400
representation of self. 
There you go. 

131
00:06:26,400 --> 00:06:28,160
You landed the plane 
beautifully. 

132
00:06:28,200 --> 00:06:34,200
There we go. 
And so code Gen. from that book.

133
00:06:34,200 --> 00:06:36,080
So anyways that that that sort 
of kicked me off. 

134
00:06:36,080 --> 00:06:37,360
I'm getting really interested in
the stuff. 

135
00:06:38,320 --> 00:06:41,080
So Jenny and I went through. 
I worked at Palantir, I studied 

136
00:06:41,120 --> 00:06:44,720
AI in college, I did a bunch of 
machine learning stuff to the 

137
00:06:44,720 --> 00:06:46,320
decades worth of startups, give 
or take. 

138
00:06:47,360 --> 00:06:52,120
And around 2020 is when. 
So I've always kept up with the 

139
00:06:52,120 --> 00:06:53,360
state-of-the-art of yeah, 
research. 

140
00:06:53,360 --> 00:06:55,200
It's just sort of something 
that's very interesting to me. 

141
00:06:55,240 --> 00:06:58,760
2020 is when the GBT 3 paper 
came out, which was essentially 

142
00:06:58,760 --> 00:07:01,240
just scaling up language models 
and then ended up having this 

143
00:07:01,520 --> 00:07:03,520
incredible property. 
So basically few shot learning. 

144
00:07:03,520 --> 00:07:05,880
So you can give it pretty much 
any problem that we were dealing

145
00:07:05,880 --> 00:07:08,040
with in NLP at the time. 
It's just like you few shot 

146
00:07:08,040 --> 00:07:10,720
prompt this thing and bam, it's 
better than whatever you had in 

147
00:07:10,720 --> 00:07:12,480
mind. 
And so I knew this was going to 

148
00:07:12,480 --> 00:07:14,400
be big. 
I remember joking with somebody 

149
00:07:14,400 --> 00:07:17,360
that like between Bitcoin and 
GBT three, it wasn't clear which

150
00:07:17,360 --> 00:07:19,600
would have a bigger impact on 
the world and they were like, 

151
00:07:19,600 --> 00:07:23,000
that's so no, like obviously 
Bitcoin has a bigger impact and 

152
00:07:23,240 --> 00:07:26,520
I guess TBD. 2020, yeah. 2020 it
wasn't clear right? 

153
00:07:27,160 --> 00:07:32,080
And then 2022 is when the API 
opened up from Open AI and I had

154
00:07:32,080 --> 00:07:34,000
just basically sold my previous 
company. 

155
00:07:34,000 --> 00:07:36,280
I had ended up my lock up period
and I ended up leaving for 

156
00:07:36,280 --> 00:07:38,720
various reasons. 
And I entered into this sort of 

157
00:07:38,720 --> 00:07:41,920
flow state of doing what I 
called one demo per week where I

158
00:07:41,920 --> 00:07:44,600
would literally just sit on my 
computer and come up with really

159
00:07:44,600 --> 00:07:46,640
interesting concepts of 
something you can implement 

160
00:07:46,880 --> 00:07:50,280
using the GPT 3 API and some 
image generation stuff as well. 

161
00:07:50,640 --> 00:07:53,320
And just put it on Twitter and 
see what people think and try 

162
00:07:53,320 --> 00:07:54,720
and get, you know, people 
excited about it. 

163
00:07:54,720 --> 00:07:58,200
And so couple notable ones from 
that era for me, where I think I

164
00:07:58,200 --> 00:08:00,080
was the first person to ship 
text to Figma. 

165
00:08:00,480 --> 00:08:02,800
And so you could input a prompt 
and you could say, hey, give me 

166
00:08:02,800 --> 00:08:06,040
an iPhone screen with a bunch of
other stuff on it and it would 

167
00:08:06,040 --> 00:08:08,920
essentially one shot it and 
generate you a schema file. 

168
00:08:09,160 --> 00:08:10,680
And then that would render to 
Figma. 

169
00:08:11,800 --> 00:08:16,240
I did a text to auto tool or 
sorry, text to retool, but it's 

170
00:08:16,240 --> 00:08:18,280
actually known on retools, like 
on an open source one where you 

171
00:08:18,280 --> 00:08:21,120
could describe a dashboard and 
it would have data schema that 

172
00:08:21,120 --> 00:08:24,320
was referencing and it would 
essentially create a data 

173
00:08:24,320 --> 00:08:27,200
dashboard for you. 
And so, you know, what I sort of

174
00:08:27,200 --> 00:08:29,400
discovered over the process of 
building a bunch of these 

175
00:08:29,400 --> 00:08:34,159
applications is that the things 
that made AL one application 

176
00:08:34,159 --> 00:08:37,240
successful were verifiability 
built into the domain. 

177
00:08:37,360 --> 00:08:40,000
And so anytime you're doing some
type of a schema generation like

178
00:08:40,000 --> 00:08:41,520
that, you can essentially lint 
it. 

179
00:08:41,559 --> 00:08:43,559
You can go through and say, is 
this a valid schema? 

180
00:08:43,840 --> 00:08:45,960
Does it compile? 
You know, they're they're like 

181
00:08:46,200 --> 00:08:47,920
specific empirical things you 
can look for. 

182
00:08:48,120 --> 00:08:50,160
And if it's wrong, you can 
actually detect that error and 

183
00:08:50,160 --> 00:08:53,440
basically pass it back to the 
model as a residual and say 

184
00:08:53,440 --> 00:08:58,760
figure it out dude. 
Which is so clear now, but in 

185
00:08:58,760 --> 00:09:01,600
those days that's incredible. 
You probably felt like you 

186
00:09:01,600 --> 00:09:05,000
stumbled on gold. 
I think that I don't want to 

187
00:09:05,000 --> 00:09:06,960
give myself too, too much credit
here, like I think a lot of 

188
00:09:07,000 --> 00:09:09,680
other people were, you know, 
also seeing like, OK, it was 

189
00:09:09,680 --> 00:09:12,560
limited value and like an e-mail
thing for you. 

190
00:09:12,560 --> 00:09:14,000
Yeah. 
Summarization or. 

191
00:09:14,200 --> 00:09:15,680
Yeah, auto complete or something
like that. 

192
00:09:15,680 --> 00:09:18,360
But anybody who's coding, you 
know, with compile at the time 

193
00:09:18,360 --> 00:09:19,560
was like, wow, this thing is 
incredible. 

194
00:09:19,720 --> 00:09:22,000
And you know, it's actually, 
there's so much low hanging 

195
00:09:22,000 --> 00:09:24,200
fruit in code too, where it's 
just boilerplate. 

196
00:09:24,240 --> 00:09:27,560
Like you don't really benefit 
from writing a Java class 

197
00:09:27,560 --> 00:09:30,120
implementation except for a 
couple lines of code in there. 

198
00:09:30,400 --> 00:09:33,160
So it's very well suited. 
And it compiles, which is 

199
00:09:33,160 --> 00:09:36,240
different than a lot of the 
other things that you have to 

200
00:09:36,280 --> 00:09:37,400
eval. 
Exactly. 

201
00:09:37,720 --> 00:09:39,240
Yeah. 
So, you know, and the 

202
00:09:39,240 --> 00:09:41,720
implications of that are pretty 
clear as well with respect to 

203
00:09:41,840 --> 00:09:44,880
our LVR reinforcement learning 
from verifiable rewards, where 

204
00:09:45,280 --> 00:09:47,800
you can have an agent go off and
write code and try and pass the 

205
00:09:47,800 --> 00:09:49,560
unit test. 
If it doesn't pass the unit 

206
00:09:49,560 --> 00:09:51,760
test, you say here's the 
failure, fix it. 

207
00:09:51,960 --> 00:09:54,960
And then eventually it goes off 
and it does pass the unit test. 

208
00:09:54,960 --> 00:09:57,480
And then you say, OK, learn from
your mistakes. 

209
00:09:57,920 --> 00:10:00,480
Or maybe you rephrase your 
trajectory so it's perfect and 

210
00:10:00,480 --> 00:10:04,400
then you fine tune on that. 
I remember seeing one paper 

211
00:10:04,440 --> 00:10:07,520
actually, that was really an aha
moment where they were trying to

212
00:10:07,520 --> 00:10:11,520
train, I think it was for 
performance improvements. 

213
00:10:11,520 --> 00:10:13,960
So they said, OK, here's a 
certain function that we have 

214
00:10:14,640 --> 00:10:17,360
and we want to train a model to 
be good at rewriting the 

215
00:10:17,360 --> 00:10:18,920
function such that it's more 
performant. 

216
00:10:19,400 --> 00:10:21,640
You can run the function and you
can measure the wall time it 

217
00:10:21,640 --> 00:10:24,120
takes to actually the number of 
CBU cycles it takes to run a 

218
00:10:24,120 --> 00:10:25,960
function. 
And so they'd have a try, you 

219
00:10:25,960 --> 00:10:27,480
know, 100 different variants of 
it. 

220
00:10:27,640 --> 00:10:29,480
The one that actually ended up 
being more performant, they'd 

221
00:10:29,480 --> 00:10:31,160
say, OK, this is the one that 
we're going to end up fine 

222
00:10:31,160 --> 00:10:32,880
tuning on. 
And that's now we just call that

223
00:10:32,880 --> 00:10:34,800
RLVR. 
But this is one of the first 

224
00:10:34,800 --> 00:10:38,040
applications to that. 
And so it was also pretty clear.

225
00:10:38,040 --> 00:10:41,000
I guess the point I'm trying to 
make is that the direction that 

226
00:10:41,000 --> 00:10:43,880
research would go in would be 
these things are going to get 

227
00:10:43,880 --> 00:10:47,360
really good at coding really 
fast because the feedback loop 

228
00:10:47,360 --> 00:10:49,360
is instantaneous. 
If you're working at Anthropic, 

229
00:10:49,360 --> 00:10:51,600
like you're writing code with 
these things all day, you can 

230
00:10:51,600 --> 00:10:54,000
figure out what it's lacking and
then that informs the type of, 

231
00:10:54,160 --> 00:10:55,280
you know, emails you put 
together. 

232
00:10:55,520 --> 00:10:59,200
Yeah, all the necessary like 
boundary conditions are set up 

233
00:10:59,320 --> 00:11:02,160
for your success. 
Exactly. 

234
00:11:02,240 --> 00:11:04,520
There's so much data you can 
produce it synthetically. 

235
00:11:05,040 --> 00:11:08,040
And at the time, in 2022, people
like Gary Marcus were getting 

236
00:11:08,040 --> 00:11:10,200
out there and saying like. 
We're running out of data. 

237
00:11:11,240 --> 00:11:14,760
Which everybody knew was, you 
know, first, it's false that we 

238
00:11:14,760 --> 00:11:16,520
didn't run out of data. 
There's still plenty of, you 

239
00:11:16,520 --> 00:11:17,600
know, appreciating data 
available. 

240
00:11:17,600 --> 00:11:20,360
But also it completely ignores 
the fact that you can 

241
00:11:20,360 --> 00:11:25,520
synthetically create. 
There's synthetic data, and the 

242
00:11:25,720 --> 00:11:30,920
Internet is doubled like every 
three days, the amount of data 

243
00:11:30,920 --> 00:11:32,960
that's on there. 
I know that a lot of the new 

244
00:11:32,960 --> 00:11:36,920
data is coming from LMS, but 
this conversation is going to go

245
00:11:36,920 --> 00:11:39,600
out. 
And it wasn't on the Internet 

246
00:11:39,600 --> 00:11:40,240
yesterday. 
Yeah. 

247
00:11:41,320 --> 00:11:44,040
So like that kind of stuff. 
Yeah. 

248
00:11:44,040 --> 00:11:47,120
I was always a little bit 
skeptical on that idea of we're 

249
00:11:47,120 --> 00:11:50,720
running out of data. 
But then you had ILIA came out 

250
00:11:50,720 --> 00:11:52,440
and said it had some nerve apps 
too. 

251
00:11:52,800 --> 00:11:58,520
And it's like this guy's way in 
a different league of brain 

252
00:11:58,520 --> 00:12:01,760
power than me. 
Like, should I question my 

253
00:12:01,760 --> 00:12:03,240
belief? 
So I don't know. 

254
00:12:04,160 --> 00:12:07,000
Yeah, the the Internet doubling 
is probably increasingly not 

255
00:12:07,000 --> 00:12:09,400
from human generated data. 
So you have to be a little bit 

256
00:12:09,400 --> 00:12:13,000
suss of the stuff. 
There's the sexually Speaking of

257
00:12:13,000 --> 00:12:16,880
ML off one of my favorite papers
of all time is it's called 

258
00:12:17,080 --> 00:12:19,320
machine learning, the highest 
interest credit card. 

259
00:12:20,400 --> 00:12:24,080
The one you had to reference, 
any talk that was given around 

260
00:12:24,080 --> 00:12:29,560
machine learning in ML OPS and 
from like 2020 to 2022, it was 

261
00:12:29,840 --> 00:12:32,760
you had to reference that box of
like the models. 

262
00:12:32,800 --> 00:12:34,800
Just a small piece of this, 
right? 

263
00:12:35,240 --> 00:12:36,920
And there's so many 
opportunities to shoot yourself 

264
00:12:36,920 --> 00:12:40,040
in the foot. 
And the example that sticks with

265
00:12:40,040 --> 00:12:43,920
me from that paper is they talk 
about training Google Translate 

266
00:12:44,440 --> 00:12:47,120
and how, you know, they assemble
parallel corpora. 

267
00:12:47,120 --> 00:12:49,120
You get a bunch of English 
pages, you get the Spanish 

268
00:12:49,120 --> 00:12:50,920
translation when they're, you 
know, available in both. 

269
00:12:51,200 --> 00:12:53,040
And then you train it something 
to map between the two. 

270
00:12:53,520 --> 00:12:56,200
And then they realize at a 
certain point that actually most

271
00:12:56,200 --> 00:12:58,320
of the Spanish pages on the 
Internet are the output of 

272
00:12:58,440 --> 00:13:00,440
Google Translate. 
And so they're essentially 

273
00:13:00,440 --> 00:13:03,400
drinking their own backwash. 
And I think that that's 

274
00:13:03,400 --> 00:13:06,120
absolutely what would happen if 
you just try to pre train on all

275
00:13:06,200 --> 00:13:08,840
text on the, you know, on Reddit
2026 onward. 

276
00:13:09,080 --> 00:13:12,200
It's like probably mostly open 
claw at this point or whatever 

277
00:13:12,200 --> 00:13:12,800
he calls it. 
Yeah. 

278
00:13:13,080 --> 00:13:14,360
So you gotta be a little 
careful. 

279
00:13:14,480 --> 00:13:16,480
Yeah. 
And it's essentially getting 

280
00:13:16,480 --> 00:13:19,240
these loops of like confirming 
to itself that the correct 

281
00:13:19,240 --> 00:13:22,120
translation for an idiom is, you
know, whatever it thought was 

282
00:13:22,280 --> 00:13:24,520
like collapses in weird ways. 
Exactly. 

283
00:13:25,080 --> 00:13:28,760
Now then you just sold Cogen to 
click up. 

284
00:13:29,000 --> 00:13:34,680
Walk me through maybe the 
process of selling and then also

285
00:13:34,680 --> 00:13:37,080
now what you're doing. 
For sure. 

286
00:13:37,240 --> 00:13:38,960
Yeah. 
So we built Cogen over three 

287
00:13:38,960 --> 00:13:42,120
years, the original concept 
behind it, we started it when I 

288
00:13:42,120 --> 00:13:44,320
think Cursor had already kind of
hit their strut. 

289
00:13:44,320 --> 00:13:45,880
So it's clear that they were 
going to win. 

290
00:13:45,880 --> 00:13:49,840
I think if you asked me, like in
early 2022 or sorry, early 2023,

291
00:13:50,400 --> 00:13:51,600
what's the landscape going to 
look like? 

292
00:13:51,600 --> 00:13:54,040
I'd say Cursor, this can be the 
editor of choice. 

293
00:13:54,320 --> 00:13:55,680
Maybe the S code will make a 
comeback. 

294
00:13:55,680 --> 00:13:58,320
Didn't happen, You know, maybe 
there'll be like one or two 

295
00:13:58,320 --> 00:14:01,240
other players. 
And so we thought very early on,

296
00:14:01,240 --> 00:14:03,600
OK, we're going to essentially 
leapfrog and we're going to 

297
00:14:03,600 --> 00:14:05,360
shoot to where the puck is 
going, schedule where the puck 

298
00:14:05,360 --> 00:14:08,120
is going and build a fully 
autonomous background agent. 

299
00:14:08,680 --> 00:14:10,480
And so the idea was ticket to 
pull request. 

300
00:14:10,720 --> 00:14:13,320
You essentially create a linear 
ticket or click up ticket or 

301
00:14:13,320 --> 00:14:15,560
something like that would say, I
want this change that. 

302
00:14:15,800 --> 00:14:18,040
It would spin up an agent in a 
sandbox who would go off and 

303
00:14:18,040 --> 00:14:19,880
write the code and then submit a
pull request back. 

304
00:14:21,000 --> 00:14:23,000
So you wouldn't even have to 
deal with the ID. 

305
00:14:23,560 --> 00:14:25,000
No ID. 
You should be able to pull it 

306
00:14:25,000 --> 00:14:26,720
into your ID and iterate on it 
there. 

307
00:14:26,720 --> 00:14:31,080
But I think that the, you know, 
this is in GPT 432 K era. 

308
00:14:31,120 --> 00:14:33,000
That was like the first model 
that came out that could really 

309
00:14:33,000 --> 00:14:37,000
do like agentic sequences. 32K 
is roughly the size of a single 

310
00:14:37,000 --> 00:14:40,760
tool usage for the cloud codes, 
like limited, it's like 15 K. 

311
00:14:40,880 --> 00:14:42,200
It's like 2 tool usages. 
Now. 

312
00:14:42,280 --> 00:14:43,720
It was very difficult to fit 
that all in. 

313
00:14:45,040 --> 00:14:47,960
But yeah, we, we essentially 
were the first people to launch 

314
00:14:48,000 --> 00:14:49,800
a thing. 
We started with linear, actually

315
00:14:50,320 --> 00:14:51,280
something we could give it a 
ticket. 

316
00:14:51,280 --> 00:14:54,320
We go off in the red code. 
In the three years since we 

317
00:14:54,320 --> 00:14:56,360
launched that, the space 
developed a lot. 

318
00:14:56,360 --> 00:14:59,080
And you know, we were sort of 
discussing earlier, it became 

319
00:14:59,080 --> 00:15:02,240
clear that code is such a 
fundamental thing for these 

320
00:15:02,240 --> 00:15:04,880
agents to function properly. 
A lot of people have thought 

321
00:15:04,880 --> 00:15:07,120
maybe there will be code 
specific models and non code 

322
00:15:07,120 --> 00:15:09,120
specific models. 
And really it seems like that's 

323
00:15:09,120 --> 00:15:11,520
not really the case. 
Currently there's GBT 5.3 

324
00:15:11,520 --> 00:15:14,160
codecs, but whatever knowledge 
that model has that's specific 

325
00:15:14,160 --> 00:15:16,760
to code, that'll be folded in a 
GBT 5.4. 

326
00:15:16,760 --> 00:15:19,720
Because there's this notion of 
positive transfer where if you 

327
00:15:19,720 --> 00:15:22,280
get better at coding, coding is 
sort of just an encoding of 

328
00:15:22,280 --> 00:15:23,680
reasoning. 
And so it gets smarter at 

329
00:15:23,680 --> 00:15:27,120
everything else. 
And so, yeah, basically the 

330
00:15:27,120 --> 00:15:29,920
exact product that we rolled out
and we got great customers. 

331
00:15:29,920 --> 00:15:31,840
We had a ton of community 
engagement and whatnot. 

332
00:15:31,840 --> 00:15:34,720
We're sort of the first there. 
We end up seeing increasingly in

333
00:15:34,720 --> 00:15:37,560
the deals that we were going 
into that the Foundation Model 

334
00:15:37,560 --> 00:15:40,200
Labs were offering their product
for free for two years at a 

335
00:15:40,200 --> 00:15:41,920
time. 
And so there's no procurement 

336
00:15:41,920 --> 00:15:43,400
department in the world. 
You can go to where they're 

337
00:15:43,400 --> 00:15:47,440
like, yes, I'll take this 
company that, you know, fly by 

338
00:15:47,440 --> 00:15:51,280
night, San Francisco startup. 
And you're using them in the 

339
00:15:51,280 --> 00:15:53,400
background, I imagine, so 
they're your competition. 

340
00:15:54,040 --> 00:15:55,760
Exactly. 
And many other companies have 

341
00:15:55,760 --> 00:15:58,880
gone up against this windsurf 
famously, you know, have Claude 

342
00:15:58,880 --> 00:16:02,320
yanked, I think cursor, you 
know, they're now training their

343
00:16:02,320 --> 00:16:04,840
own models as well because their
margins are so much worse than 

344
00:16:05,240 --> 00:16:07,960
than what you get in Claude. 
We're sorry that. 

345
00:16:07,960 --> 00:16:08,360
Yeah. 
Claude. 

346
00:16:08,400 --> 00:16:08,840
Good. 
Yeah. 

347
00:16:10,440 --> 00:16:14,720
But yeah, I mean it the I think 
the at a higher level, you know,

348
00:16:14,720 --> 00:16:18,280
this is this is bigger than all 
of us was basically my my take 

349
00:16:18,280 --> 00:16:21,200
away like code is going to be 
solved in a year maybe. 

350
00:16:21,400 --> 00:16:23,120
I mean, I don't think that you 
and I are going to spend that 

351
00:16:23,120 --> 00:16:26,560
much time in an ID very soon. 
I think that is going to be 

352
00:16:26,560 --> 00:16:30,600
largely driven by, you know, 
$100 billion training clusters, 

353
00:16:30,600 --> 00:16:33,200
essentially. 
These people have so much 

354
00:16:33,200 --> 00:16:36,320
capital to invest. 
And I think we're facing the 

355
00:16:36,880 --> 00:16:38,640
automation of knowledge work 
more generally. 

356
00:16:38,920 --> 00:16:40,600
That's a really exciting thing 
to be a part of. 

357
00:16:40,600 --> 00:16:44,600
And as somebody who's spent the 
last 20 years or so working on 

358
00:16:44,640 --> 00:16:48,160
AI, I wanted to be a pivotal 
contributor to a winning team. 

359
00:16:48,640 --> 00:16:51,000
And meanwhile, we had built this
really great partnership with 

360
00:16:51,000 --> 00:16:52,800
Clickup and a couple other 
companies. 

361
00:16:53,160 --> 00:16:54,920
We ended up finding ourselves in
a position where one of our 

362
00:16:54,920 --> 00:16:57,720
business partners offered to buy
the company and it was perfect 

363
00:16:57,720 --> 00:16:59,160
timing. 
I was like, wow, this is 

364
00:16:59,160 --> 00:17:00,800
actually really exciting. 
We went through 1. 

365
00:17:00,800 --> 00:17:02,080
Of your business partners 
company. 

366
00:17:02,080 --> 00:17:03,640
That we work closely with. 
Unfortunately there's some 

367
00:17:03,640 --> 00:17:05,400
things I can't discuss about 
this, just to you know. 

368
00:17:05,599 --> 00:17:08,160
Preserve. 
Keep them anonymous then. 

369
00:17:08,359 --> 00:17:10,280
Company that we work closely 
with and. 

370
00:17:10,280 --> 00:17:13,680
But it wasn't click up. 
This the initial company that 

371
00:17:13,680 --> 00:17:14,640
offered by us was not. 
Click up. 

372
00:17:14,640 --> 00:17:15,920
That's right, but it got you 
thinking. 

373
00:17:16,400 --> 00:17:18,480
Exactly. 
We had a great relationship with

374
00:17:18,640 --> 00:17:20,400
them. 
The CEO is a total visionary and

375
00:17:20,400 --> 00:17:22,359
that spent a lot of time kind of
chatting with him back and 

376
00:17:22,359 --> 00:17:23,440
forth. 
He had been a user of our 

377
00:17:23,440 --> 00:17:25,079
product and given us feedback 
and whatnot. 

378
00:17:25,520 --> 00:17:27,960
And through a sequence of 
events, essentially we were the 

379
00:17:27,960 --> 00:17:30,040
number one agent on Click that 
was an external one. 

380
00:17:31,200 --> 00:17:34,080
It became clear to me that this 
is a winning team and this is a 

381
00:17:34,080 --> 00:17:36,840
really exciting vision to work 
on that goes beyond just 

382
00:17:37,160 --> 00:17:39,840
engineers sitting in San 
Francisco working on code. 

383
00:17:40,200 --> 00:17:41,760
And I'm happy to get into why 
that's the case. 

384
00:17:41,760 --> 00:17:43,960
There's, you know, a lot of my 
personal theses essentially 

385
00:17:43,960 --> 00:17:47,000
lined up with what Click Up was 
going after so. 

386
00:17:47,240 --> 00:17:49,120
OK, go more. 
Yeah. 

387
00:17:49,120 --> 00:17:51,560
But I think that the one of the 
revelations of the last couple 

388
00:17:51,560 --> 00:17:54,480
of years, like I mentioned a 
second ago, is that coding 

389
00:17:54,480 --> 00:17:57,440
agents are generalist agents. 
If you make an agent better at 

390
00:17:57,440 --> 00:17:59,640
coding, then it ends up being 
better at everything else 

391
00:17:59,640 --> 00:18:01,560
because of this notion of 
positive transfer. 

392
00:18:02,600 --> 00:18:05,680
But you know, also, if you look 
at anything that an agent will 

393
00:18:05,680 --> 00:18:09,160
accomplish that has tools or you
know, does various things in 

394
00:18:09,160 --> 00:18:12,040
order to reach out into the 
world, all of those things that 

395
00:18:12,040 --> 00:18:14,080
it uses to reach out into the 
world are comprised of code. 

396
00:18:14,600 --> 00:18:17,760
And so if you have an agent that
can write and execute code, then

397
00:18:17,760 --> 00:18:20,280
basically it ends up being a, 
it's like AGI complete. 

398
00:18:20,440 --> 00:18:23,120
It has the ability to write its 
own tools that has to create 

399
00:18:23,120 --> 00:18:24,960
tool, tool tool, which is write 
a batch script, right? 

400
00:18:25,800 --> 00:18:28,520
And so it seems like me that 
basically coding agents are 

401
00:18:28,520 --> 00:18:31,680
convergent with generalist 
knowledge worker agents. 

402
00:18:31,680 --> 00:18:33,400
And I'm not the only one who's 
knows that. 

403
00:18:34,200 --> 00:18:36,280
You know, a great example of 
this being sort of distributed 

404
00:18:36,280 --> 00:18:40,120
more broadly is clod code used 
to be called clod code and the 

405
00:18:40,120 --> 00:18:42,560
Clod Code SDK was called the 
Clod code SDK. 

406
00:18:42,800 --> 00:18:45,760
Now it's just the Clod agent SDK
and open claw. 

407
00:18:45,960 --> 00:18:48,280
Also a great example of this, 
the way it does stuff is the 

408
00:18:48,280 --> 00:18:50,280
writes code on a sandbox. 
So you just basically have these

409
00:18:50,280 --> 00:18:55,480
primitives of for loop with ALM 
call on it, sandbox, put them 

410
00:18:55,480 --> 00:18:57,880
together and you know, boom, 
you've got a fully general 

411
00:18:57,880 --> 00:19:01,760
station. 
And I think if you assume that 

412
00:19:01,760 --> 00:19:04,120
jet like models are going to get
better, they're going to be 

413
00:19:04,120 --> 00:19:07,240
capable of performing any 
generalist task that relates to 

414
00:19:07,240 --> 00:19:09,120
knowledge work. 
The people who are going to. 

415
00:19:09,120 --> 00:19:11,000
Win in an. 
Environment like that are people

416
00:19:11,000 --> 00:19:14,280
who have the best access to 
context, the best access to 

417
00:19:14,280 --> 00:19:17,240
surfaces on which you interact 
with agents, and the best unit 

418
00:19:17,240 --> 00:19:19,480
economics and existing 
distribution in order to get 

419
00:19:19,480 --> 00:19:22,320
that in front of people. 
And there's essentially no other

420
00:19:22,320 --> 00:19:25,280
player than Clickup that has all
those three things in a way that

421
00:19:25,280 --> 00:19:27,080
I thought was, you know, the 
most compelling. 

422
00:19:27,080 --> 00:19:30,560
So we got hitched and here I am 
now I'm running our AI 

423
00:19:30,560 --> 00:19:33,240
operation. 
But there's so many things that 

424
00:19:33,240 --> 00:19:36,680
you said there that that are 
fascinating to me, especially 

425
00:19:36,680 --> 00:19:40,880
around like just how we interact
with. 

426
00:19:41,200 --> 00:19:44,680
Basically my experience going 
through AI has been most things 

427
00:19:44,680 --> 00:19:46,960
that end up rolling out to the 
world more generally. 

428
00:19:46,960 --> 00:19:49,360
They start in code. 
The whatever pattern ends up 

429
00:19:49,400 --> 00:19:51,360
catching hold, coders do it 
first. 

430
00:19:51,360 --> 00:19:52,360
There's a lot of good reasons 
for that. 

431
00:19:52,360 --> 00:19:54,520
One of them is just programmers 
are working on the stuff 

432
00:19:54,560 --> 00:19:56,920
anyways, but we talked about the
verifiability before. 

433
00:19:57,240 --> 00:20:00,920
So like tab auto complete 
started in code sort of sidebar 

434
00:20:00,920 --> 00:20:03,080
that pops out that you chat with
that also started code. 

435
00:20:03,080 --> 00:20:06,240
Even autonomous agents, arguably
like the first true autonomous 

436
00:20:06,240 --> 00:20:10,080
agents were programming agents. 
And now it's sort of rolling out

437
00:20:10,080 --> 00:20:12,320
to the the the world more 
generally hooks. 

438
00:20:12,320 --> 00:20:16,200
It's another good example. 
And I'm just so excited for the 

439
00:20:16,200 --> 00:20:18,680
rest of the world to go through 
the experience I went to, which 

440
00:20:18,680 --> 00:20:21,240
is, you know, I got cursor and 
Claude code and my whole job 

441
00:20:21,240 --> 00:20:23,160
changed. 
I now essentially will prompt 

442
00:20:23,160 --> 00:20:25,680
things or review code and 
whatnot, but I can my leverage 

443
00:20:25,680 --> 00:20:29,280
has expanded so significantly 
and this idea of cursor for your

444
00:20:29,280 --> 00:20:30,440
whole. 
Job if you are. 

445
00:20:30,440 --> 00:20:33,920
A financial services person or 
you're working in something like

446
00:20:33,920 --> 00:20:37,720
legal services is going to be an
incredible experience for all 

447
00:20:37,720 --> 00:20:39,960
those people to go through and 
the best place to do that is 

448
00:20:39,960 --> 00:20:41,920
click up it's cursor for your 
whole job. 

449
00:20:43,760 --> 00:20:48,000
Yeah, One thing that I've 
noticed myself doing is when I 

450
00:20:48,000 --> 00:20:56,760
create apps now, it's default 
local first, which is different 

451
00:20:56,760 --> 00:20:59,600
than how I have always been 
thinking about it. 

452
00:20:59,600 --> 00:21:01,840
It's like, oh, you need to be 
thinking about production. 

453
00:21:02,080 --> 00:21:05,320
This needs to be battle tested. 
But now it's like, no, this is 

454
00:21:05,320 --> 00:21:08,840
just for me, you know, like I, I
was talking to somebody last 

455
00:21:08,840 --> 00:21:13,960
night, they said I created 40 
apps in the past three months 

456
00:21:14,400 --> 00:21:21,240
and only three of them have I 
actually open sourced and like 

457
00:21:21,480 --> 00:21:26,320
given to the world because 
there's just, it's, it's mine. 

458
00:21:26,800 --> 00:21:31,680
It doesn't need to be, you know,
it's so specific to my need that

459
00:21:32,400 --> 00:21:35,640
I'm not going to put this out 
there. 

460
00:21:36,960 --> 00:21:38,440
I don't think anybody else needs
it. 

461
00:21:39,480 --> 00:21:43,040
And also, you got it. 
Then think about like security. 

462
00:21:43,040 --> 00:21:45,240
If you're going to open source 
stuff, you want to like clean it

463
00:21:45,240 --> 00:21:48,240
up. 
You want to do things that you 

464
00:21:48,240 --> 00:21:50,120
don't want to, just like throw 
anything out there. 

465
00:21:50,320 --> 00:21:53,160
Yeah, yeah, there's totally 
something to be said for 

466
00:21:53,160 --> 00:21:57,760
shifting models of how software 
we're run, given the the case 

467
00:21:57,760 --> 00:22:01,720
that it's so cheap to produce. 
Yeah, that is the wild thing 

468
00:22:01,760 --> 00:22:05,880
that you have these ethermal and
like the generative UI is 

469
00:22:05,880 --> 00:22:10,120
fascinating to me too and the 
MCP apps type of things that'll 

470
00:22:10,560 --> 00:22:13,240
it's now becoming more and more 
a thing. 

471
00:22:13,360 --> 00:22:15,720
And so I imagine you'll start 
adding it and click up soon 

472
00:22:15,720 --> 00:22:19,240
enough where if you don't 
already have it, where, OK, I 

473
00:22:19,240 --> 00:22:24,360
need to do something. 
Boom, you've got the UI right 

474
00:22:24,360 --> 00:22:26,520
there in that moment. 
For sure. 

475
00:22:26,560 --> 00:22:28,080
Yeah. 
I mean, I think that especially 

476
00:22:28,080 --> 00:22:30,920
with an application like click 
up, there's these data models 

477
00:22:30,920 --> 00:22:33,360
that represent everything you 
want to do and work and sort of 

478
00:22:33,360 --> 00:22:36,320
the primitives for interacting 
and getting things done like 

479
00:22:36,320 --> 00:22:38,120
docs and chats and tasks and 
whatnot. 

480
00:22:38,600 --> 00:22:41,240
And depending on what task 
you're trying to accomplish, 

481
00:22:41,880 --> 00:22:44,600
there's wildly different ways of
going about arranging those 

482
00:22:44,600 --> 00:22:46,160
things. 
Click I was incredibly 

483
00:22:46,160 --> 00:22:48,040
configurable and that is a 
blessing and a curse. 

484
00:22:48,040 --> 00:22:50,400
It's a blessing because if you 
configure it correctly, then 

485
00:22:50,400 --> 00:22:53,120
it's, you know, does exactly 
what your workflow is, but the 

486
00:22:53,120 --> 00:22:56,320
curse is takes a while to to 
configure it. 

487
00:22:56,320 --> 00:22:58,200
If you have a very specific 
thing and there's, you know, 

488
00:22:58,200 --> 00:23:00,760
certain things you need to know.
And the amazing thing is 

489
00:23:00,760 --> 00:23:02,120
generative AI is assault for 
that. 

490
00:23:02,120 --> 00:23:05,960
So you can come in and you can 
say, hey, I am the CEO of 

491
00:23:05,960 --> 00:23:09,880
NVIDIA. 
My supply chain is in click up, 

492
00:23:10,360 --> 00:23:13,080
tell me about it and it will 
build you a dashboard. 

493
00:23:13,280 --> 00:23:15,960
You know, it'll, we have a thing
rolling out very soon that is 

494
00:23:15,960 --> 00:23:18,440
going to essentially allow you 
to have completely custom 

495
00:23:18,440 --> 00:23:21,760
JavaScript layers on top of the 
core click up interface in a 

496
00:23:21,760 --> 00:23:24,160
very secure way. 
And so it will be a bespoke 

497
00:23:24,160 --> 00:23:26,160
interface for whatever you're 
trying to accomplish at any 

498
00:23:26,200 --> 00:23:28,400
point in time. 
The amazing thing also about, 

499
00:23:28,480 --> 00:23:31,560
you know, this sort of custom 
finances, I'm sure you've seen 

500
00:23:31,560 --> 00:23:34,720
demos of this of like, you know,
Cerebrus running various models 

501
00:23:34,720 --> 00:23:37,080
of like 15,000,000 tokens per 
second or something. 

502
00:23:37,480 --> 00:23:40,960
It'll be instantaneous. 
In the future Tadarus or 

503
00:23:40,960 --> 00:23:42,200
something? 
I saw that, yeah. 

504
00:23:42,200 --> 00:23:44,560
It's like they. 
So much faster than anything it 

505
00:23:44,560 --> 00:23:47,240
made like Cerebrus look this big
on the graph. 

506
00:23:47,240 --> 00:23:50,000
Yeah, they like compiled it to a
chip basically and that's 

507
00:23:50,000 --> 00:23:53,840
imminently going to happen. 
That was Llama 38 BI, think 3.18

508
00:23:53,840 --> 00:23:56,160
B. 
There's no physical law that 

509
00:23:56,160 --> 00:23:59,560
says you can't run GPT 5 on a 
chip like that. 

510
00:23:59,600 --> 00:24:02,760
And so yeah, that means the the 
amount of time it'll take you to

511
00:24:02,760 --> 00:24:05,960
write, you know, fully custom 
Next JS app that has all this 

512
00:24:05,960 --> 00:24:10,640
custom stuff in it is going to 
be like 100 millisecond, 200 

513
00:24:10,640 --> 00:24:11,680
milliseconds, something like 
that. 

514
00:24:11,880 --> 00:24:14,000
How many tokens are actually in 
a full Next JS app? 

515
00:24:14,000 --> 00:24:16,760
Not that much, which means that 
your interaction with software 

516
00:24:16,760 --> 00:24:18,320
is going to be fully dynamically
configured. 

517
00:24:19,200 --> 00:24:20,480
And I think that's really 
exciting. 

518
00:24:20,520 --> 00:24:22,800
There's a lot of, like, it's 
easy to say that you have to 

519
00:24:22,800 --> 00:24:24,920
actually execute on it. 
And nobody's ever produced an 

520
00:24:24,920 --> 00:24:27,080
interface like that. 
So we don't like, just like 

521
00:24:27,080 --> 00:24:30,280
people said, like, Oh yeah, like
Minority Report style thing with

522
00:24:30,280 --> 00:24:31,800
your hands in the air. 
Like when you actually do that, 

523
00:24:31,800 --> 00:24:33,600
your arms hurt. 
So it never caught on. 

524
00:24:34,000 --> 00:24:36,320
We'll see if this ends up being 
too disorienting, but I think 

525
00:24:36,320 --> 00:24:38,920
there will be a point on the 
spectrum towards fully 

526
00:24:38,920 --> 00:24:41,560
dynamically configured software 
that we end up converging on. 

527
00:24:41,560 --> 00:24:42,880
This is far beyond where we're 
at right now. 

528
00:24:44,440 --> 00:24:47,240
I need to know what I want in 
order to ask for it. 

529
00:24:47,240 --> 00:24:51,920
And so on one hand, it puts a 
lot of like that heavy lifting 

530
00:24:51,920 --> 00:24:56,760
on the user, which is different 
than us in the way that we used 

531
00:24:57,440 --> 00:24:59,800
different pieces of the Internet
before. 

532
00:24:59,800 --> 00:25:01,760
You know, like when I'm 
scrolling Instagram, I'm not 

533
00:25:01,760 --> 00:25:04,560
thinking much. 
It's more like I'm getting fed 

534
00:25:04,880 --> 00:25:09,440
but now I'm having to actually 
like be the chef and cook in the

535
00:25:09,440 --> 00:25:14,280
kitchen. 
Yeah, the be able to tag in 

536
00:25:14,280 --> 00:25:16,120
context. 
I mean, you know, I'm, I'm a 

537
00:25:16,120 --> 00:25:18,160
vlog code and cursor user. 
So that's natively how I can 

538
00:25:18,160 --> 00:25:20,760
think about a lot of this stuff.
Be able to add a file or a 

539
00:25:20,760 --> 00:25:22,880
specific class and say this is 
where you should fix it. 

540
00:25:23,240 --> 00:25:25,640
It's so powerful. 
The thing that makes that 

541
00:25:25,640 --> 00:25:27,960
possible when you step out of. 
So in code, there's like a 

542
00:25:27,960 --> 00:25:30,520
limited number of artifacts that
you need to reference. 

543
00:25:30,520 --> 00:25:33,400
There's you know, maybe there's 
tickets, there's obviously 

544
00:25:33,400 --> 00:25:36,520
files, maybe it's like a century
issue or something like that. 

545
00:25:36,520 --> 00:25:39,560
But there's, you know, we can 
probably enumerate less than a 

546
00:25:39,560 --> 00:25:41,960
dozen things that you would want
to tag if you're going into 

547
00:25:41,960 --> 00:25:44,320
general purpose knowledge work. 
It explodes, right? 

548
00:25:44,360 --> 00:25:47,600
Especially if you take the long 
tail of like PR firms, 

549
00:25:47,600 --> 00:25:51,720
marketers, financial services. 
And that user base is basically 

550
00:25:51,720 --> 00:25:54,440
what comprises click UPS core of
customers. 

551
00:25:55,000 --> 00:25:57,720
And so it's incredibly, you 
know, important if you're doing 

552
00:25:57,720 --> 00:26:00,240
a general purpose knowledge work
job and you want an experience 

553
00:26:00,240 --> 00:26:03,160
like a cursor cloud code where 
it's, you know, very versatile, 

554
00:26:03,160 --> 00:26:05,240
it can do anything. 
Do you have essentially first 

555
00:26:05,240 --> 00:26:07,560
party data integrations that 
allow you to do that type of 

556
00:26:07,560 --> 00:26:09,560
tagging? 
We can reference a certain piece

557
00:26:09,560 --> 00:26:12,200
of content, and there are very 
real constraints that prevent 

558
00:26:12,200 --> 00:26:14,760
you from being able to do that 
in an easy way, just with 

559
00:26:14,760 --> 00:26:18,080
existing sort of the constraints
of fragmentation as we describe 

560
00:26:18,080 --> 00:26:19,560
it. 
So I'll give you a very concrete

561
00:26:19,560 --> 00:26:21,920
example of when you would not be
able to accomplish the flow you 

562
00:26:21,920 --> 00:26:24,200
just described. 
Slack has various things in 

563
00:26:24,200 --> 00:26:27,640
place that prevent you from 
searching past conversations and

564
00:26:27,640 --> 00:26:29,560
from indexing their data if 
you're a verified app. 

565
00:26:30,040 --> 00:26:31,400
This is always a problem we had 
at Cogent. 

566
00:26:31,440 --> 00:26:33,480
You'd be talking Cogent was a 
Slack integration. 

567
00:26:34,000 --> 00:26:36,440
You would say to it, hey Cogent,
go figure out why we made this 

568
00:26:36,440 --> 00:26:39,920
decision and if the decision 
should be updated, please go to 

569
00:26:39,920 --> 00:26:42,960
the code and change it. 
And Cogent wouldn't be able to 

570
00:26:42,960 --> 00:26:44,560
find the decision. 
It would just seem stupid 

571
00:26:44,560 --> 00:26:47,040
because it would try and search 
and it would only be able to see

572
00:26:47,040 --> 00:26:51,000
like 3 hours. 
However, obviously the model is 

573
00:26:51,000 --> 00:26:52,080
not stupid. 
Yeah. 

574
00:26:52,760 --> 00:26:55,600
It's sort of like it's. 
Just this limitation that is put

575
00:26:55,600 --> 00:26:56,840
on. 
I think they did that because of

576
00:26:56,840 --> 00:26:58,800
Glean, right? 
They don't like Glean, I'm sure.

577
00:26:59,040 --> 00:27:03,080
Yeah, I mean it it you kind of 
can't blame them for doing that.

578
00:27:03,080 --> 00:27:05,200
It's bad for the world. 
Yeah, they have to protect. 

579
00:27:05,200 --> 00:27:07,000
Exactly. 
They have to protect their mode 

580
00:27:07,200 --> 00:27:09,800
in a way. 
But it's funny because Slack is 

581
00:27:09,800 --> 00:27:15,160
so quickly becoming the command 
Center for everything outside of

582
00:27:15,160 --> 00:27:18,400
it, right? 
Like you're firing off agents in

583
00:27:18,400 --> 00:27:23,160
so many different ways from 
Slack that in a way it feels 

584
00:27:23,160 --> 00:27:25,280
like they're shooting themselves
in the foot if they're not 

585
00:27:25,280 --> 00:27:28,080
letting you have access to 
everything that's inside. 

586
00:27:28,520 --> 00:27:32,080
I think they would definitely 
like to have you believe that, 

587
00:27:32,080 --> 00:27:34,560
that it will be the command 
Center for agents in the future.

588
00:27:35,040 --> 00:27:36,320
I have my doubts, you know, I 
really. 

589
00:27:37,040 --> 00:27:39,280
Yeah, you know what? 
Company at the scale of 

590
00:27:39,280 --> 00:27:41,240
Salesforce isn't going to be 
able to iterate as quickly. 

591
00:27:41,240 --> 00:27:44,440
I think if they were in a rapid 
iteration mindset, it would have

592
00:27:44,440 --> 00:27:47,400
taken them a lot less time to 
put together something that, you

593
00:27:47,400 --> 00:27:50,640
know, felt agent native. 
There's other companies I would 

594
00:27:50,640 --> 00:27:52,880
point to that I think have done 
a fantastic job that are 

595
00:27:53,240 --> 00:27:54,760
probably operating at some more 
scale. 

596
00:27:55,400 --> 00:27:58,360
I think that it's somewhat 
cannibalistic to their own 

597
00:27:58,360 --> 00:28:00,760
business, though, if they lean 
into agents, right? 

598
00:28:00,760 --> 00:28:03,120
And if they allow other people 
to come in and take data out, 

599
00:28:03,120 --> 00:28:06,640
like you mentioned a second ago,
I think, you know, two years 

600
00:28:06,640 --> 00:28:08,400
ago, they could have had a lot 
of the features that they're 

601
00:28:08,400 --> 00:28:09,600
rolling out today. 
And the only thing that 

602
00:28:09,600 --> 00:28:12,440
prevented them from doing that 
is bureaucracy and some of 

603
00:28:12,440 --> 00:28:15,360
these. 
So my money is on startups that 

604
00:28:15,360 --> 00:28:17,520
can really lean into this and go
all in on AI. 

605
00:28:17,520 --> 00:28:20,520
And I think that one of the 
benefits of a place like click 

606
00:28:20,520 --> 00:28:22,720
up is that they have a chat 
product, which is just like 

607
00:28:22,720 --> 00:28:26,480
slack, just as good, but that is
also literally right next to 

608
00:28:26,560 --> 00:28:28,560
their tasks, which is like a 
linear Jira. 

609
00:28:28,560 --> 00:28:30,680
They've got documents, which is 
like a notion or conflicts. 

610
00:28:30,680 --> 00:28:33,080
They've got white boards just 
like thick my all these things 

611
00:28:33,080 --> 00:28:35,840
in a single place, and you don't
have to ask anybody permission. 

612
00:28:35,840 --> 00:28:39,360
You have the entire history of 
your organization's decisions 

613
00:28:39,360 --> 00:28:43,400
and tasks and chats all along. 
Oh man, I'm the first person to 

614
00:28:43,400 --> 00:28:48,320
say, like I've lived in Slack 
for so long that I would happily

615
00:28:48,320 --> 00:28:51,760
get rid of it if I could. 
I haven't seen an alternative to

616
00:28:51,760 --> 00:28:54,800
that, especially because the way
that I use Slack is through the 

617
00:28:54,800 --> 00:28:56,920
community. 
And so it's in a way it's like 

618
00:28:56,920 --> 00:29:02,680
knowledge sharing and it's 
connection and it's having fun. 

619
00:29:02,720 --> 00:29:05,760
So there's so many. 
The thing if you think about 

620
00:29:05,760 --> 00:29:09,160
like the things to be done, the 
things to be done in that regard

621
00:29:09,160 --> 00:29:15,360
are like, I get a laugh. 
And so it's hard for me to think

622
00:29:15,360 --> 00:29:17,680
that there would be something 
that would replace that. 

623
00:29:17,920 --> 00:29:20,800
However, when it comes to like 
productivity, I know it just 

624
00:29:20,800 --> 00:29:23,560
tanks my productivity. 
Getting pinged on Slack is 

625
00:29:23,760 --> 00:29:26,760
fucking so. 
And especially for stupid shit 

626
00:29:27,120 --> 00:29:30,080
that you're like, well, you 
know, you probably could have 

627
00:29:30,080 --> 00:29:33,200
figured that out. 
I also think like, you're not 

628
00:29:33,200 --> 00:29:36,280
just going to replace Slack. 
What you're going to have to do 

629
00:29:36,280 --> 00:29:40,080
is come at it from a whole 
different way of thinking. 

630
00:29:40,080 --> 00:29:44,040
Like how can we make it? 
Because Slack is for 

631
00:29:44,040 --> 00:29:48,040
communication between people. 
But if you don't need to 

632
00:29:48,040 --> 00:29:50,400
communicate as much between 
people because you're 

633
00:29:50,400 --> 00:29:53,680
communicating through an agent, 
which then will just reference 

634
00:29:53,680 --> 00:29:55,880
context, it's a different 
paradigm. 

635
00:29:56,560 --> 00:30:00,640
Totally, Yeah, Slack is built 
for a model where very dense 

636
00:30:00,640 --> 00:30:03,360
graph between how much 
communication happens. 

637
00:30:03,400 --> 00:30:06,560
And if you think that most of 
the work in the future is going 

638
00:30:06,560 --> 00:30:09,440
to be done by agents, which I 
certainly do, except certainly 

639
00:30:09,440 --> 00:30:11,680
most of the work that we do 
today in an office will be done 

640
00:30:11,680 --> 00:30:15,240
by agents, then maybe 
synchronous or slightly async 

641
00:30:15,240 --> 00:30:17,320
comps between people and Slack 
is less important. 

642
00:30:18,120 --> 00:30:19,640
I think the other thing you 
mentioned is. 

643
00:30:20,000 --> 00:30:22,120
You know, really a. 
Huge issue is work sprawl. 

644
00:30:22,440 --> 00:30:24,520
So the reason it's annoying to 
get pinged in Slack is because 

645
00:30:24,520 --> 00:30:26,640
you're also getting pinged in 
four other places, right? 

646
00:30:27,160 --> 00:30:28,720
And you're like, where should I 
be responding? 

647
00:30:28,720 --> 00:30:32,280
And I think that the number of 
pings that you're going to get 

648
00:30:32,280 --> 00:30:34,880
is going to go up as we scale 
our workforce by bringing in 

649
00:30:34,920 --> 00:30:37,280
anti employees. 
And so there needs to be some 

650
00:30:37,280 --> 00:30:40,280
level of like convergence of 
these platforms into a single 

651
00:30:40,280 --> 00:30:43,240
surface area, but there's only 
one in box that you're going to 

652
00:30:43,280 --> 00:30:44,520
that has all of your pings in 
it. 

653
00:30:45,240 --> 00:30:50,200
That's been the dream for ages. 
I remembered I got a tool that 

654
00:30:50,640 --> 00:30:52,600
was supposed to do that. 
It was like take all your 

655
00:30:52,600 --> 00:30:55,680
Telegram and all your WhatsApp 
and all your e-mail and it's all

656
00:30:56,200 --> 00:31:00,600
going into one inbox. 
But it didn't really work. 

657
00:31:01,120 --> 00:31:04,400
It never stuck for me. 
I don't know why, but it didn't 

658
00:31:04,400 --> 00:31:08,480
stick in the way that cuz as 
soon as you have one that's 

659
00:31:08,520 --> 00:31:12,080
outside of it, cuz I think 
LinkedIn was outside of it, then

660
00:31:12,080 --> 00:31:15,280
you still end up having to check
different places. 

661
00:31:15,280 --> 00:31:18,320
So it defeats the whole purpose.
Yeah, yeah. 

662
00:31:18,320 --> 00:31:21,280
Combine that with. 
It's never been easier to create

663
00:31:21,280 --> 00:31:25,120
a new messaging server because, 
you know, Claude Coat, we'll do 

664
00:31:25,120 --> 00:31:29,120
it for you. 
You get this crazy explosion of,

665
00:31:29,720 --> 00:31:31,560
you know, different channels on 
which to communicate. 

666
00:31:32,000 --> 00:31:35,640
I think that what is likely to 
happen is that you'll probably 

667
00:31:35,640 --> 00:31:38,240
end up seeing some level of 
consolidation at the software 

668
00:31:38,240 --> 00:31:39,640
layer. 
So companies like Ripling, for 

669
00:31:39,640 --> 00:31:43,200
example, I think are very well 
situated, easy for them to add 

670
00:31:43,200 --> 00:31:45,600
another thing like a chat 
service or something like that 

671
00:31:45,840 --> 00:31:48,960
from the procurement and you 
know, sort of the big company 

672
00:31:48,960 --> 00:31:51,720
side, really easy for you to 
green light something people are

673
00:31:51,720 --> 00:31:56,400
already on board as a as a 
vendor, you know, and then 

674
00:31:56,400 --> 00:31:58,520
there's obviously advantages to 
the data integration. 

675
00:31:58,640 --> 00:32:01,640
So if your chat product is 
integrated with everything else 

676
00:32:01,640 --> 00:32:04,160
that you're already using, then 
agents are going to be able to 

677
00:32:04,160 --> 00:32:07,200
navigate it more easily. 
Yeah, that's, that's funny you 

678
00:32:07,200 --> 00:32:10,200
bring that up because I was 
talking to some folks at ifood. 

679
00:32:10,200 --> 00:32:14,400
That's this company out of 
Brazil and they are a food 

680
00:32:14,400 --> 00:32:18,240
delivery app, right? 
Much like, you know, what is it 

681
00:32:18,240 --> 00:32:22,400
DoorDash here. 
But what they started doing is 

682
00:32:22,400 --> 00:32:27,160
they started offering a new Bank
of type of thing where it's got 

683
00:32:27,960 --> 00:32:32,160
the finances for the restaurant 
owners. 

684
00:32:33,520 --> 00:32:36,280
And so because they know 
everything about the 

685
00:32:36,280 --> 00:32:39,080
restaurants, because all of the 
orders come through ifood, 

686
00:32:39,680 --> 00:32:43,880
they're very confident when they
say we can get you these loans 

687
00:32:43,960 --> 00:32:46,840
with these terms. 
And the agents have all that 

688
00:32:46,840 --> 00:32:51,400
context because the bank, they 
are the bank and they're also 

689
00:32:51,640 --> 00:32:53,400
the way that the company makes 
money. 

690
00:32:54,200 --> 00:32:57,120
So it's like they're playing 
both sides. 

691
00:32:57,120 --> 00:33:00,760
And it, it just makes a lot of 
sense that Oh yeah, let's offer 

692
00:33:00,880 --> 00:33:05,240
this financing in these terms 
because we know you're going to 

693
00:33:05,240 --> 00:33:08,000
get it back. 
It's not unless there's some act

694
00:33:08,000 --> 00:33:10,680
of God, you're going to be all 
right. 

695
00:33:10,680 --> 00:33:13,400
And we're confident that you can
fulfill on it. 

696
00:33:13,920 --> 00:33:15,680
Yeah, they're much better 
underwriting because they have 

697
00:33:15,680 --> 00:33:18,600
much better visibility into the,
Yeah, the behaviors of their 

698
00:33:18,600 --> 00:33:20,960
customers. 
And the one way of thinking of 

699
00:33:20,960 --> 00:33:26,120
it is the 20 tens are probably 
like kind of the main force 

700
00:33:26,120 --> 00:33:29,040
driving tech development was 
network effects of communities. 

701
00:33:29,160 --> 00:33:31,240
Yeah. 
So you have, you know, if one 

702
00:33:31,240 --> 00:33:33,920
person's on Facebook, it makes 
more sense for another person. 

703
00:33:33,920 --> 00:33:35,960
Actually, WhatsApp, yeah, 
WhatsApp, etcetera. 

704
00:33:35,960 --> 00:33:38,720
Any app that better owns. 
Yeah, basically. 

705
00:33:38,880 --> 00:33:40,400
And maybe Snapchat we'll throw 
them into. 

706
00:33:40,440 --> 00:33:45,520
The next one, I think that the 
20 twenties is probably going to

707
00:33:45,520 --> 00:33:48,840
be driven by forces like you 
described where there's network 

708
00:33:48,840 --> 00:33:52,480
effects of consolidation of 
software platforms because it's 

709
00:33:52,480 --> 00:33:54,600
never been easy to create a new 
one. 

710
00:33:54,600 --> 00:33:56,200
So there's going to be a million
alternatives. 

711
00:33:56,440 --> 00:33:58,600
The only one that will actually 
stand out is the one that is 

712
00:33:58,600 --> 00:34:01,880
so-called converged or has those
network effects of all the data 

713
00:34:01,880 --> 00:34:06,760
being in one place. 
Yeah, because one thing that you

714
00:34:06,760 --> 00:34:09,679
get, but man, I don't know how 
that's going to work. 

715
00:34:10,440 --> 00:34:14,560
Just being at so many different 
start-ups over my life and 

716
00:34:14,760 --> 00:34:20,360
seeing how much sprawl there is 
for documentation. 

717
00:34:21,440 --> 00:34:25,000
Like, I don't know, it's it's a 
nice theory where you're like, 

718
00:34:25,000 --> 00:34:27,960
yeah, it's all on click up. 
But you know, one team uses 

719
00:34:27,960 --> 00:34:30,679
click up religiously, the other 
team uses git book, the other 

720
00:34:30,679 --> 00:34:34,080
team uses Notion, and it's 
literally everywhere. 

721
00:34:34,480 --> 00:34:37,000
Yeah. 
So how does that like come and 

722
00:34:37,000 --> 00:34:39,560
now we try and centralized but 
we don't? 

723
00:34:40,600 --> 00:34:43,520
Well, I think, you know, being a
realist about it, like any 

724
00:34:43,520 --> 00:34:46,400
company of a certain size is 
going to have seven different 

725
00:34:46,400 --> 00:34:48,880
things competing constantly. 
That's probably a good thing, 

726
00:34:48,920 --> 00:34:50,159
right? 
You, you want a marketplace of 

727
00:34:50,159 --> 00:34:52,639
ideas, you want competition. 
And if one company ends up 

728
00:34:52,639 --> 00:34:55,880
slacking, the other should end 
up taking over slacking and 

729
00:34:56,600 --> 00:34:58,720
slacking offsets. 
I've been playing on words there

730
00:34:58,720 --> 00:35:00,360
though. 
Totally not intentional. 

731
00:35:01,760 --> 00:35:06,080
I think that also the cost of 
migration has never been lower 

732
00:35:06,440 --> 00:35:08,840
for a lot of these systems. 
So there's the human behavior of

733
00:35:08,840 --> 00:35:10,640
you need to adapt a certain 
system. 

734
00:35:11,360 --> 00:35:13,320
But we just said a second ago, 
like you and I are going to 

735
00:35:13,320 --> 00:35:16,680
spend a lot less time in Slack 
responding to notifications. 

736
00:35:16,680 --> 00:35:18,640
Most of the actual work will be 
done by agents. 

737
00:35:18,640 --> 00:35:22,680
And so these platforms end up 
operating like systems of record

738
00:35:23,040 --> 00:35:24,920
with like a thin layer of UI on 
top. 

739
00:35:24,920 --> 00:35:27,160
The cloud code could write for 
you in a very short amount of 

740
00:35:27,160 --> 00:35:29,760
time. 
The other portion of basically 

741
00:35:29,760 --> 00:35:33,160
migrating people over to a 
system is the data integration. 

742
00:35:33,160 --> 00:35:35,920
So you have a bunch of notes in 
Git book, you said how do you 

743
00:35:35,920 --> 00:35:38,720
get those into click up or into 
Confluence? 

744
00:35:38,800 --> 00:35:43,440
And if you give me Claude Co 
work in 20 minutes, I can pump 

745
00:35:43,440 --> 00:35:45,320
it all in there for you, right. 
And make sure it's nice and 

746
00:35:45,320 --> 00:35:47,040
organized. 
Yeah, make sure. 

747
00:35:47,040 --> 00:35:49,680
It is the beautiful thing about 
when you do that, the file 

748
00:35:49,680 --> 00:35:52,800
structure is just impact, yeah. 
Exactly. 

749
00:35:52,800 --> 00:35:55,120
It's so amazing. 
Yeah, our click up agents will 

750
00:35:55,120 --> 00:35:56,360
do it for you, which is. 
Nice. 

751
00:35:56,360 --> 00:36:00,800
So yeah, I think. 
It's never been easier basically

752
00:36:00,800 --> 00:36:03,360
to move between different 
platforms than once you have a 

753
00:36:03,360 --> 00:36:06,880
single consolidated system of 
record that's pretty solid, and 

754
00:36:06,880 --> 00:36:08,880
there's not a great reason to 
move off of that unless it has a

755
00:36:08,880 --> 00:36:14,680
very bad customer experience. 
And are you not worried about 

756
00:36:14,680 --> 00:36:19,920
certain tasks that are done 
outside of click up that you 

757
00:36:19,920 --> 00:36:24,200
don't have Privy into how 
they're done? 

758
00:36:24,760 --> 00:36:28,240
Like how are you expecting to be
able to do those tasks? 

759
00:36:28,560 --> 00:36:34,200
For example, I'm going to take 
marketing and me needing to 

760
00:36:36,000 --> 00:36:40,480
create ads or optimize some paid
ads. 

761
00:36:42,000 --> 00:36:44,240
That's not being done on 
anything. 

762
00:36:44,240 --> 00:36:48,560
Click up, right? 
I guess you're you, you can use 

763
00:36:48,920 --> 00:36:52,040
cloud code to do it. 
I've seen some skills that have 

764
00:36:52,040 --> 00:36:54,560
been created for paid ads or 
paid media. 

765
00:36:55,600 --> 00:36:58,960
Like how do you see yourself 
doing these tasks? 

766
00:36:59,200 --> 00:37:01,680
Sure. 
Yeah, there's a very long tail 

767
00:37:02,040 --> 00:37:03,920
essentially of things and you're
never going to be able to create

768
00:37:03,920 --> 00:37:07,120
like a first party thing for all
of these, even if you have 

769
00:37:07,120 --> 00:37:09,600
agents going off and writing, 
you know, tools for you that's 

770
00:37:09,600 --> 00:37:12,080
like a create ad campaign tool, 
something like that. 

771
00:37:12,080 --> 00:37:14,080
That is actually pretty core to 
what a lot of our customers do. 

772
00:37:14,080 --> 00:37:16,640
And so I wouldn't be surprised 
if, you know, sometime in a 

773
00:37:16,640 --> 00:37:18,920
couple months here, we do have 
like a very good first party 

774
00:37:18,920 --> 00:37:20,240
support for it. 
But I get the point. 

775
00:37:21,520 --> 00:37:22,800
Yeah. 
I mean, I think that they're, 

776
00:37:22,960 --> 00:37:25,560
when you're looking at the long 
tail, there is a couple 

777
00:37:25,560 --> 00:37:27,280
different things. 
One of them is if you give an 

778
00:37:27,280 --> 00:37:31,600
agent a sandbox and you give it 
the ability to run NPX skill 

779
00:37:31,600 --> 00:37:35,200
install, whatever, then yeah, 
there's a huge library of stuff 

780
00:37:35,200 --> 00:37:36,880
out there. 
And as long as you can provide 

781
00:37:36,880 --> 00:37:40,040
it with a credential, it 
probably can use code execution 

782
00:37:40,040 --> 00:37:41,600
in order to create ad campaigns 
for you. 

783
00:37:42,080 --> 00:37:47,560
But you, I guess with ads, it's 
good because you can verify if 

784
00:37:48,120 --> 00:37:52,360
the ad spend or the cost per 
click, cost per acquisition is 

785
00:37:52,360 --> 00:37:55,520
lower, then you know, there's 
something clear that you're 

786
00:37:55,520 --> 00:37:57,720
targeting and that number should
be going down. 

787
00:37:58,520 --> 00:38:00,280
There's a lot of things that 
aren't that clear. 

788
00:38:00,640 --> 00:38:05,200
And so it gets back to like why 
we love code generation is 

789
00:38:05,200 --> 00:38:09,360
because it's, it can compile and
it's verifiable in certain ways 

790
00:38:09,360 --> 00:38:12,360
in certain use cases. 
With a lot of this other stuff 

791
00:38:12,360 --> 00:38:14,680
on this long tail that we're 
talking about, it's not. 

792
00:38:14,880 --> 00:38:18,960
And so how do you ever expect 
for it to be able to do that? 

793
00:38:19,160 --> 00:38:22,760
Get better. 
Yeah, I mean, so this is sort of

794
00:38:22,760 --> 00:38:26,600
a very enelopsy question is, is 
there a way to even measure, to 

795
00:38:26,640 --> 00:38:28,040
eval yourself? 
Getting better. 

796
00:38:28,040 --> 00:38:30,480
Let's take something relatively 
simple, image generation. 

797
00:38:30,920 --> 00:38:33,920
You wanna see a marketer who 
says, hey, I wanna create an 

798
00:38:33,920 --> 00:38:36,440
image, a hero image for this 
Instagram ad I'm gonna create 

799
00:38:36,800 --> 00:38:39,160
and I have these fifty different
tag lines that I've come up 

800
00:38:39,160 --> 00:38:40,360
with. 
Make an image for each of them. 

801
00:38:40,800 --> 00:38:42,960
How do I know that I'm getting 
better at generating images over

802
00:38:42,960 --> 00:38:45,000
time? 
Well, I can tell you that you 

803
00:38:45,000 --> 00:38:47,600
can delete your evals and you 
can wait three months and image 

804
00:38:47,600 --> 00:38:50,320
generation will be better. 
And I know that because I look 

805
00:38:50,320 --> 00:38:53,680
at all these models and yeah, so
good, right? 

806
00:38:54,680 --> 00:38:56,440
That's like not really even a 
hot take. 

807
00:38:56,440 --> 00:38:58,440
Like it will get better and you 
don't really need to do very 

808
00:38:58,440 --> 00:39:00,720
much. 
There are like, I think the, the

809
00:39:00,720 --> 00:39:03,640
place for evals like look at are
these things getting better over

810
00:39:03,640 --> 00:39:05,000
time? 
Is you want to prevent 

811
00:39:05,000 --> 00:39:06,840
regressions, obviously. 
So you want to make sure your 

812
00:39:06,840 --> 00:39:10,040
thing isn't broken, and you want
to make sure it's able to 

813
00:39:10,600 --> 00:39:13,960
perform certain long context 
tasks that are very complex. 

814
00:39:14,000 --> 00:39:17,920
Make an image, then go ahead and
upload it to Instagram or 

815
00:39:17,920 --> 00:39:20,320
something like that and make 
sure the data flows properly and

816
00:39:20,320 --> 00:39:23,200
the agent doesn't get confused, 
which is largely a function of 

817
00:39:23,200 --> 00:39:26,320
your harness. 
But yeah, I mean, I think that 

818
00:39:26,320 --> 00:39:29,480
the brilliant thing about AI in 
2026 is that you know that every

819
00:39:29,480 --> 00:39:32,320
three months there's going to be
a model that gets 20% better and

820
00:39:32,320 --> 00:39:33,480
you can kind of just ride that 
wave. 

821
00:39:34,440 --> 00:39:37,600
And it's almost not a good use 
of your time to write your own 

822
00:39:37,600 --> 00:39:40,600
evals for just the the basic 
model stuff for intelligence. 

823
00:39:40,720 --> 00:39:43,040
Yeah. 
I'm sure the Foundation model 

824
00:39:43,040 --> 00:39:45,040
labs have their own way of going
about eval and this, and a big 

825
00:39:45,040 --> 00:39:47,760
part of it is probably having 
people and you know, I think 

826
00:39:48,240 --> 00:39:50,160
most of the RLHF was done in 
Nigeria. 

827
00:39:50,160 --> 00:39:51,480
They would literally have people
sit there. 

828
00:39:51,480 --> 00:39:54,840
And that's why they're M dash is
so popular. 

829
00:39:54,880 --> 00:39:59,760
Yeah, exactly. 
What else is? 

830
00:39:59,760 --> 00:40:03,200
What else have you been thinking
about or pondering? 

831
00:40:03,440 --> 00:40:08,920
Yeah, I think on evals, I don't 
know if I have that much to add.

832
00:40:08,920 --> 00:40:11,840
I think, you know the the 
wonderful thing like I mentioned

833
00:40:11,840 --> 00:40:14,040
earlier is whatever eval you 
write today is going to be 

834
00:40:14,040 --> 00:40:16,840
obsolete in like 3 months unless
it's incredibly hard. 

835
00:40:16,840 --> 00:40:19,120
But even evals that were 
established in like March of 

836
00:40:19,120 --> 00:40:23,160
last year are completely 
saturated and, and people we're 

837
00:40:23,160 --> 00:40:25,320
saying this will never be solved
by AI. 

838
00:40:25,320 --> 00:40:27,880
This is fundamentally what 
distinguishes humans from, you 

839
00:40:27,880 --> 00:40:29,880
know, AI. 
So if you can make a goal post, 

840
00:40:29,880 --> 00:40:32,960
we'll pass it basically. 
And I think that's a wonderful 

841
00:40:32,960 --> 00:40:35,280
thing because it means that, you
know, most things that you can 

842
00:40:35,280 --> 00:40:37,080
point to that are problems in 
the world, you can, you can 

843
00:40:37,080 --> 00:40:40,760
formulate an eval around it. 
I think one of the, you know, 

844
00:40:40,760 --> 00:40:43,080
underrated, really interesting 
things that's happening in AI 

845
00:40:43,080 --> 00:40:47,280
right now that relates to 
knowledge work is we have 

846
00:40:47,280 --> 00:40:49,440
basically the frontier of 
science at this point, 

847
00:40:49,440 --> 00:40:51,760
especially in like more 
mathematical and verifiable 

848
00:40:51,760 --> 00:40:55,840
fields is being run by LM. 
So, you know, there's obviously 

849
00:40:55,840 --> 00:40:57,680
pure math. 
We've got a bunch of these air 

850
00:40:57,680 --> 00:41:00,040
dish problems are being solved 
on like a weekly basis at this 

851
00:41:00,040 --> 00:41:02,160
point. 
We just got 12 out of 12 on the 

852
00:41:02,160 --> 00:41:04,600
Putnam exam. 
This is like a Google DeepMind 

853
00:41:04,600 --> 00:41:05,840
result. 
That's like the hardest math 

854
00:41:05,840 --> 00:41:08,000
exam that you know, exists 
basically. 

855
00:41:08,360 --> 00:41:10,160
I don't think there's a harder 
math test than that. 

856
00:41:10,160 --> 00:41:13,800
Maybe like a PhD qualifier 
somewhere, and increasingly in 

857
00:41:13,800 --> 00:41:17,040
things like, you know, 
mathematical physics and signal 

858
00:41:17,040 --> 00:41:19,360
processing biology as well. 
Yeah. 

859
00:41:19,360 --> 00:41:22,120
I mean, some of these things are
not strictly LMS, but I know 

860
00:41:22,120 --> 00:41:25,800
there's an initiative to fully 
simulate a cell, which is pretty

861
00:41:25,800 --> 00:41:27,320
incredible. 
So you'd be able to, you know, 

862
00:41:27,320 --> 00:41:30,320
basically say what would the 
impact of this new, you know, 

863
00:41:30,320 --> 00:41:34,080
pathogen be on a cell? 
Yeah, I saw something about that

864
00:41:34,080 --> 00:41:40,960
where it's the reversing of the 
aging and being able to simulate

865
00:41:40,960 --> 00:41:43,480
that. 
It is now more effective because

866
00:41:43,480 --> 00:41:47,440
of these different techniques 
that we've got, which I know 

867
00:41:47,440 --> 00:41:50,400
nothing about. 
Like to be clear, that is like 

868
00:41:50,400 --> 00:41:52,560
way outside of what I'm normally
in thinking about. 

869
00:41:52,560 --> 00:41:55,680
But it is cool that you bring it
up because it's true. 

870
00:41:56,120 --> 00:42:02,720
The cutting edge is now hand in 
hand like researcher plus AI. 

871
00:42:03,280 --> 00:42:05,640
Yeah. 
And there's a real crisis in 

872
00:42:05,640 --> 00:42:07,840
academia that is sort of already
happening. 

873
00:42:07,840 --> 00:42:11,080
I guess the wave is about to 
kind of come up where I'm not 

874
00:42:11,080 --> 00:42:13,920
sure that you're going to need 
as many mathematicians in the 

875
00:42:13,920 --> 00:42:16,520
future to prove math things, or 
maybe we're all mathematicians 

876
00:42:16,520 --> 00:42:17,880
in the future. 
But I don't think that you and I

877
00:42:17,880 --> 00:42:20,240
are going to be in the details. 
And I think that'll be 

878
00:42:20,240 --> 00:42:22,400
increasingly true of a bunch of 
different professions. 

879
00:42:22,880 --> 00:42:27,080
I saw a guy, he's a fellow at 
the Hoover Institute, recently 

880
00:42:27,080 --> 00:42:29,920
published a thing saying even in
economics papers, like you could

881
00:42:29,960 --> 00:42:33,400
essentially have Claude Code go 
off and write hundreds, if not 

882
00:42:33,400 --> 00:42:36,080
thousands of empirical economics
papers. 

883
00:42:36,080 --> 00:42:39,040
Because essentially the practice
of writing these is you find a 

884
00:42:39,040 --> 00:42:42,200
data set online, you do some 
analysis on it, you connect it, 

885
00:42:42,880 --> 00:42:45,200
you know, to various theories 
about how the world works and 

886
00:42:46,120 --> 00:42:48,400
uses evidence. 
There's a lot of data out there,

887
00:42:48,680 --> 00:42:50,600
you know, being collected on a 
regular basis. 

888
00:42:50,640 --> 00:42:53,840
And, you know, I guess, are you 
going to be vibrating economics 

889
00:42:53,840 --> 00:42:55,680
papers? 
Seems like the answer is yes. 

890
00:42:56,280 --> 00:43:01,280
Dude I had this meme that I 
created which was like it's a 

891
00:43:01,280 --> 00:43:07,560
slippery slope from going to 
write a react app to now 

892
00:43:07,560 --> 00:43:14,000
becoming a quantum physicist. 
I feel like at 12:00 PM at night

893
00:43:14,000 --> 00:43:17,160
after I just vibe coded this 
front end I'm like you know 

894
00:43:17,160 --> 00:43:19,840
what, fuck it, let's try quantum
physics. 

895
00:43:19,880 --> 00:43:21,200
Yeah, I've had this experience 
too. 

896
00:43:21,200 --> 00:43:23,840
I I love visualizations of 
relativity. 

897
00:43:24,080 --> 00:43:25,520
They're really beautiful. 
I'm sure you've seen them 

898
00:43:25,520 --> 00:43:26,880
before. 
Like you're seeing space-time 

899
00:43:26,880 --> 00:43:30,400
bend and Oh yeah. 
Like the blanket of oh those are

900
00:43:30,400 --> 00:43:31,960
so cool. 
The rubber sheet is a common 

901
00:43:31,960 --> 00:43:34,200
one. 
Yeah, I saw that actually, when 

902
00:43:34,200 --> 00:43:38,080
I took a lot of acid, I saw that
in my head of like just actual 

903
00:43:38,480 --> 00:43:41,640
things where we're playing 
music, and it felt like the 

904
00:43:41,760 --> 00:43:44,600
sound waves were kind of like 
those blankets. 

905
00:43:44,600 --> 00:43:47,800
Makes you think Einstein must 
have been, you know, on several 

906
00:43:47,800 --> 00:43:49,760
tabs of acid in order to come up
with this. 

907
00:43:49,760 --> 00:43:52,760
He didn't even need it was like 
that was his. 

908
00:43:54,400 --> 00:43:55,800
That was just a regular 
Einstein. 

909
00:43:55,800 --> 00:43:57,000
Yeah. 
So what would we have gotten if 

910
00:43:57,000 --> 00:44:01,280
we gave Einstein acid? 
Somebody call up John von 

911
00:44:01,280 --> 00:44:04,200
Neumann and ask him if he's down
to try this new stuff I just 

912
00:44:04,200 --> 00:44:04,920
got. 
Yeah. 

913
00:44:07,360 --> 00:44:08,960
Yeah. 
But the I mean, the the crazy 

914
00:44:08,960 --> 00:44:10,880
thing about all of this is that 
like, it's the same model. 

915
00:44:10,880 --> 00:44:14,120
Like the same model that is. 
I, there's a guy who went on the

916
00:44:14,120 --> 00:44:16,320
dwarkeshpod, Adam Brown, I think
it's his name. 

917
00:44:16,920 --> 00:44:19,760
And he teaches relativity at 
Stanford. 

918
00:44:19,760 --> 00:44:23,520
So he he has like the class they
teach to PhDs and this was now 

919
00:44:23,520 --> 00:44:25,080
six months ago or maybe more 
than that. 

920
00:44:25,320 --> 00:44:27,160
And he said the latest 
generation of models, which at 

921
00:44:27,160 --> 00:44:30,920
the time was probably just GPT 
5, maybe was Claude four or 

922
00:44:30,920 --> 00:44:34,280
something was acing his exam on 
relativity. 

923
00:44:34,440 --> 00:44:36,240
And this is the same one that 
writes your React app, right? 

924
00:44:36,240 --> 00:44:38,080
And so I had this experience 
recently. 

925
00:44:38,080 --> 00:44:40,280
I was just building my homepage.
I was trying to like vibe code 

926
00:44:40,280 --> 00:44:42,280
some stuff and test out, you 
know, the codecs app or 

927
00:44:42,280 --> 00:44:43,800
something. 
I was like, all right, now let's

928
00:44:43,800 --> 00:44:45,600
make a visualization of 
relativity. 

929
00:44:46,120 --> 00:44:47,880
And it just did it, you know, 
and it nailed it. 

930
00:44:47,920 --> 00:44:49,920
It was amazing. 
Did you at least clear the 

931
00:44:49,920 --> 00:44:53,320
context first? 
No, it's like it goes from yeah.

932
00:44:53,880 --> 00:44:55,760
That is so wild to think about 
her. 

933
00:44:55,920 --> 00:44:59,200
Yeah, so I there are vanishingly
few things at this point that 

934
00:44:59,200 --> 00:45:01,800
I'm better at than these models.
Maybe the only thing is going 

935
00:45:01,800 --> 00:45:04,040
beyond a. 
Million, but it is so spiky. 

936
00:45:04,120 --> 00:45:07,520
So there's it's the you probably
saw the car wash one that's 

937
00:45:07,520 --> 00:45:09,560
coming around. 
How should I walk to the car 

938
00:45:09,560 --> 00:45:12,240
wash? 
Yeah, where you're like, there's

939
00:45:12,240 --> 00:45:17,920
still those moments. 
So I think about that spikiness 

940
00:45:17,920 --> 00:45:20,200
and what you're talking about, 
how the models are going to be 

941
00:45:20,200 --> 00:45:21,840
able to do a lot of knowledge 
work. 

942
00:45:22,400 --> 00:45:26,120
Yeah, but there's going to be 
maybe these moments where they 

943
00:45:26,120 --> 00:45:29,560
hit a wall and it doesn't 
understand why it's hitting a 

944
00:45:29,560 --> 00:45:30,440
wall. 
Yeah. 

945
00:45:31,280 --> 00:45:34,480
But potentially we can get 
around that if it can write a 

946
00:45:34,480 --> 00:45:38,120
tool. 
I think that writing a tool is 

947
00:45:38,120 --> 00:45:40,320
probably not the solve to the 
car wash thing. 

948
00:45:40,320 --> 00:45:43,040
So just you know, for anybody 
listening, the the prompt you 

949
00:45:43,040 --> 00:45:45,040
give it is I'm 50 meters from a 
car wash. 

950
00:45:45,600 --> 00:45:47,040
Should I drive there or walk 
there? 

951
00:45:47,280 --> 00:45:50,800
And if you ask GPT 5.2, I ran. 
Because I need to wash my car. 

952
00:45:51,600 --> 00:45:54,640
Yeah, and if you ask GPT 5.2, 
it's like, well, obviously you 

953
00:45:54,640 --> 00:45:56,720
should walk there because you 
know, the cost of starting up 

954
00:45:56,720 --> 00:45:59,160
your car and driving it there is
going to be too expensive and. 

955
00:45:59,160 --> 00:46:01,120
Global warming and all that fun 
stuff. 

956
00:46:01,120 --> 00:46:03,440
Yeah, there's limited benefits. 
You should just walk 50 meters. 

957
00:46:03,440 --> 00:46:06,120
It's not that far. 
And the joke is obviously that 

958
00:46:06,120 --> 00:46:08,320
you need your car to get washed.
You run that same prompt there 

959
00:46:08,320 --> 00:46:10,360
through Gemini and Claude, and 
both of them actually get the 

960
00:46:10,360 --> 00:46:13,000
correct answer. 
The another funny one that's on 

961
00:46:13,000 --> 00:46:16,920
the same idea is it's like, hey,
I have a cup, but for some 

962
00:46:16,920 --> 00:46:19,400
reason the top of the cup is 
covered over, but the bottom of 

963
00:46:19,400 --> 00:46:21,640
the cup is totally open. 
Should I just throw away this 

964
00:46:21,640 --> 00:46:25,800
cup? 
And I think ChatGPT said like, 

965
00:46:26,320 --> 00:46:28,680
yeah, that that doesn't sound 
like a very useful cup or 

966
00:46:28,680 --> 00:46:30,720
something along those. 
Like you're absolutely right. 

967
00:46:31,040 --> 00:46:32,320
And the idea is you just flip 
it. 

968
00:46:32,320 --> 00:46:35,200
Over right? 
Yeah, there's been a long 

969
00:46:35,200 --> 00:46:37,600
history of these things. 
And I think that, you know, 

970
00:46:37,760 --> 00:46:39,760
maybe one of the what that 
represents, I, I would be 

971
00:46:39,760 --> 00:46:42,760
curious to see some researchers 
at Foundation Model Labs opine 

972
00:46:42,760 --> 00:46:44,960
on why that's the case. 
You know, obviously to a certain

973
00:46:44,960 --> 00:46:49,160
extent it represents the fact 
that they don't have the same 

974
00:46:49,160 --> 00:46:51,400
model of reality that you and I 
do where, you know, we have a, 

975
00:46:51,680 --> 00:46:54,840
we're visualizing an actual cup 
upside down like that and we can

976
00:46:55,040 --> 00:46:56,920
we interact with the cup on a 
regular basis. 

977
00:46:56,920 --> 00:46:58,600
So we have to grab it. 
We know that you could turn it 

978
00:46:58,600 --> 00:47:01,280
upside down, but I don't think 
that the solution for that is 

979
00:47:01,280 --> 00:47:04,120
necessarily code execution. 
I think actually if you just 

980
00:47:04,120 --> 00:47:05,520
take it's. 
The world model. 

981
00:47:05,680 --> 00:47:07,800
Yeah, if world model, if you 
have a robotics model that 

982
00:47:07,800 --> 00:47:10,040
actually does interact with 
glasses and then you, you know, 

983
00:47:10,040 --> 00:47:12,360
staple on a language model, 
which is what a lot of people 

984
00:47:12,360 --> 00:47:13,920
are doing. 
Physical intelligence is one up 

985
00:47:13,920 --> 00:47:16,320
here in in Berkeley. 
Many other people are, you know,

986
00:47:16,320 --> 00:47:18,560
trying to build these things. 
Like this thing is going to 

987
00:47:18,600 --> 00:47:20,160
understand the world better than
you and I do. 

988
00:47:21,480 --> 00:47:25,000
Yeah, there's some people who, 
you know, the stochastic parrot 

989
00:47:25,200 --> 00:47:26,760
objection, I'm sure you've heard
before. 

990
00:47:27,000 --> 00:47:29,160
That is moronic to me. 
I feel like it's it, there's so 

991
00:47:29,160 --> 00:47:32,760
many things that it, it misses, 
but they would have you believe 

992
00:47:32,760 --> 00:47:36,920
essentially that all it's doing 
is it's replaying what it thinks

993
00:47:36,920 --> 00:47:38,560
a human would say in that 
scenario. 

994
00:47:38,760 --> 00:47:40,600
And so it doesn't capture a 
model of the world. 

995
00:47:41,000 --> 00:47:42,960
Here's, here's a thought 
experiment for you, right? 

996
00:47:42,960 --> 00:47:48,360
Let's say you had a, a, a model 
and it's trying to like 

997
00:47:48,360 --> 00:47:50,320
basically understand Plato's 
cave. 

998
00:47:50,760 --> 00:47:54,240
And so in Plato's cave, you just
see shadows cast on the wall, 

999
00:47:54,240 --> 00:47:56,000
right? 
And it's trying to basically 

1000
00:47:56,000 --> 00:47:58,520
predict the next frame of what 
the shadows are going to be. 

1001
00:47:59,040 --> 00:48:01,920
In the process of figuring out 
basically what the shadow is 

1002
00:48:01,920 --> 00:48:04,760
going to be in the next frame, 
it's entirely plausible to think

1003
00:48:04,760 --> 00:48:07,160
of that model actually coming up
with like a real physical 

1004
00:48:07,160 --> 00:48:09,480
simulation of 3D bodies moving 
around. 

1005
00:48:09,480 --> 00:48:12,040
And then it learns how to 
project that onto 2 dimensions. 

1006
00:48:12,320 --> 00:48:14,280
And I think there's something 
analogous going on inside of 

1007
00:48:14,280 --> 00:48:17,400
language models where, you know,
the projection of it, the shadow

1008
00:48:17,400 --> 00:48:19,520
on the wall is just that, the 
characters that ends up 

1009
00:48:19,520 --> 00:48:21,640
producing. 
But in being able to project 

1010
00:48:21,640 --> 00:48:24,240
what you're going to say in a 
certain scenario, the easiest 

1011
00:48:24,240 --> 00:48:26,680
pathway to doing that is 
actually to essentially spin up 

1012
00:48:26,680 --> 00:48:29,640
a human mind inside of it and 
simulate the processes that mind

1013
00:48:29,640 --> 00:48:31,360
goes through. 
So, you know, the communication 

1014
00:48:31,360 --> 00:48:34,080
between the amygdala and the 
thalamus or whatever they're 

1015
00:48:34,080 --> 00:48:36,240
like actually are concepts 
internally that are 

1016
00:48:36,440 --> 00:48:38,600
communicating between them 
inside of large language model. 

1017
00:48:38,920 --> 00:48:40,880
And so I think that will be 
strengthened essentially, if you

1018
00:48:40,880 --> 00:48:44,040
add more multimodar data and you
have, you know, a, you know, 

1019
00:48:44,040 --> 00:48:47,440
learns to grasp things, it'll, 
it'll strengthen that internal, 

1020
00:48:47,560 --> 00:48:48,760
you know, representation that it
builds. 

1021
00:48:48,960 --> 00:48:54,920
I've heard folks talk about 
like, one of the problems with 

1022
00:48:55,680 --> 00:49:01,560
this idea is that us humans, we 
don't even know how stuff works.

1023
00:49:01,560 --> 00:49:04,200
Like we think we know. 
But when you get down to a small

1024
00:49:04,200 --> 00:49:10,120
enough level, or if you start 
measuring everything, the data 

1025
00:49:10,120 --> 00:49:11,760
doesn't add up. 
Totally. 

1026
00:49:12,760 --> 00:49:14,480
Like, how do you reconcile those
two things? 

1027
00:49:15,080 --> 00:49:17,120
Yeah, I mean, I think if you 
talk to a neuroscientist for 

1028
00:49:17,120 --> 00:49:19,800
more than like 5 minutes, 
they'll tell you like humans, 

1029
00:49:19,840 --> 00:49:22,160
you know, you think you're like 
a rational machine doing simple 

1030
00:49:22,160 --> 00:49:24,040
manipulation your head. 
And that's like all made-up. 

1031
00:49:24,520 --> 00:49:26,720
You know, the the reason that 
you end up actually arriving at 

1032
00:49:26,720 --> 00:49:28,600
a conclusion is very different 
than the brain. 

1033
00:49:29,120 --> 00:49:31,160
You know, you believe that you 
arrived at a conclusion. 

1034
00:49:31,160 --> 00:49:34,760
And, you know, our minds are 
much fuzzier than I think, you 

1035
00:49:34,760 --> 00:49:37,240
know, the logicians would have 
us believe, Noam Chomsky would 

1036
00:49:37,240 --> 00:49:39,520
have you believe, for example, 
who in my opinion is sort of the

1037
00:49:39,520 --> 00:49:42,400
villain of all of this. 
And I think LMS are going to be 

1038
00:49:42,400 --> 00:49:44,640
similar where, you know, maybe 
there is some amount of like 

1039
00:49:44,640 --> 00:49:47,240
internal delusion where they 
tell themselves, like, the 

1040
00:49:47,240 --> 00:49:49,160
reason I've arrived at this 
conclusion is X. 

1041
00:49:49,840 --> 00:49:52,080
But actually the reason you 
know, there's a different 

1042
00:49:52,080 --> 00:49:54,560
circuit than what they're ready 
to acknowledge that caused it. 

1043
00:49:55,120 --> 00:49:57,320
The beautiful thing about a 
large language model though, is 

1044
00:49:57,320 --> 00:50:00,480
that you know you can actually 
do full brain surgery on it and 

1045
00:50:00,480 --> 00:50:02,720
you don't need to crack open the
the skull. 

1046
00:50:02,760 --> 00:50:05,720
Yeah, I was going to say good 
luck for explainability, folks. 

1047
00:50:07,000 --> 00:50:09,520
That's hopefully going to be one
of the major advancements we 

1048
00:50:09,520 --> 00:50:11,720
have this year. 
I think Anthropic deserves a lot

1049
00:50:11,720 --> 00:50:14,400
of credit for going through and,
you know, doing mechanistic 

1050
00:50:14,400 --> 00:50:16,560
interpretability. 
That's really kind of been the 

1051
00:50:16,560 --> 00:50:18,160
the beacon of truth in that 
domain. 

1052
00:50:19,160 --> 00:50:22,680
You know, it's never been the 
case, like back going back to 

1053
00:50:22,800 --> 00:50:27,080
2012 when neural networks really
took off this Alex, that moment,

1054
00:50:27,080 --> 00:50:29,760
right? 
The major knock against it was 

1055
00:50:29,800 --> 00:50:32,360
like, oh, but neural networks 
aren't explainable, unlike my 

1056
00:50:32,360 --> 00:50:34,600
random forests, which I 
understand perfectly and 

1057
00:50:34,600 --> 00:50:35,840
everything that's going on 
inside of them. 

1058
00:50:35,840 --> 00:50:38,600
Like I can look at the tree and 
we can have maplot live, make a 

1059
00:50:38,600 --> 00:50:42,400
diagram of it, right? 
And I think that actually pretty

1060
00:50:42,400 --> 00:50:45,200
soon we're going to be in a 
place where you can ask a 

1061
00:50:45,200 --> 00:50:47,160
question about why did the model
do X? 

1062
00:50:47,480 --> 00:50:49,920
And it will give you much more 
explanation in a way that is 

1063
00:50:49,920 --> 00:50:52,360
intuitive to you. 
It'll be this circuit. 

1064
00:50:52,360 --> 00:50:54,440
Here's the pieces of training 
data that ended up, you know, 

1065
00:50:54,440 --> 00:50:56,680
leading to that here the 
different reasoning paths that 

1066
00:50:56,680 --> 00:50:59,240
sort of were unfolding inside 
the models mind at the time. 

1067
00:50:59,280 --> 00:51:01,280
No way. 
Yeah, you think? 

1068
00:51:01,920 --> 00:51:02,840
There's a lot of work to be 
done. 

1069
00:51:02,840 --> 00:51:05,440
I don't think that technology 
exists today, but I think 

1070
00:51:05,440 --> 00:51:08,640
there's absolutely a research 
incentive in order to make that 

1071
00:51:08,640 --> 00:51:11,480
work. 
And there's a lot of value if 

1072
00:51:11,480 --> 00:51:13,200
you were to, you know, be able 
to realize that. 

1073
00:51:13,440 --> 00:51:15,520
And I don't think it's 
fundamentally this like the type

1074
00:51:15,520 --> 00:51:16,760
of thing where there's no answer
to it. 

1075
00:51:16,800 --> 00:51:21,160
Yeah, but I, I feel like when 
the tide goes out, you see who's

1076
00:51:21,160 --> 00:51:23,920
swimming naked. 
If the model is going to point 

1077
00:51:23,920 --> 00:51:28,680
to the data that it's using, 
that's going to cause a lot of 

1078
00:51:29,200 --> 00:51:34,280
problems for the original data. 
It's like, oh, now you're using 

1079
00:51:34,280 --> 00:51:36,040
my data. 
You know, there's already stuff 

1080
00:51:36,080 --> 00:51:40,280
where it's very opaque and how 
our data that we put out there 

1081
00:51:40,600 --> 00:51:43,440
is being used. 
But now you've all of a sudden 

1082
00:51:43,440 --> 00:51:49,320
got like a Spotify model of how 
my data is being used. 

1083
00:51:49,560 --> 00:51:53,200
I should be getting paid type 
thing every time my data gets 

1084
00:51:53,200 --> 00:51:55,840
queried. 
I don't know that. 

1085
00:51:56,400 --> 00:51:59,360
So I do see the incentive, but I
also see that it could 

1086
00:51:59,360 --> 00:52:02,360
potentially backfire, right? 
Yeah, I mean, the, you know, 

1087
00:52:02,480 --> 00:52:04,960
from anthropics perspective, 
it's not like I can download the

1088
00:52:04,960 --> 00:52:07,040
model weights of clod. 
And so they probably wouldn't 

1089
00:52:07,040 --> 00:52:10,840
make this available to a random 
consumer to say, which New York 

1090
00:52:10,840 --> 00:52:12,560
Times article are you ripping 
this off of? 

1091
00:52:13,640 --> 00:52:15,800
I do remember actually, though, 
Stable Diffusion ran this 

1092
00:52:15,800 --> 00:52:19,120
experiment a while ago where you
can take an image and you could 

1093
00:52:19,120 --> 00:52:21,560
say, OK, which images in the 
training data set are most 

1094
00:52:21,560 --> 00:52:23,080
similar to them that you've 
generated here? 

1095
00:52:23,560 --> 00:52:26,320
And if you ran that is like 
actually wildly different than 

1096
00:52:26,320 --> 00:52:28,520
most of them. 
It basically demonstrated that a

1097
00:52:28,520 --> 00:52:30,120
lot of the images were quite 
novel. 

1098
00:52:30,720 --> 00:52:33,600
Maybe it's some very high level,
you know, it had similarities to

1099
00:52:33,600 --> 00:52:36,720
the others and it'd be 
interesting to run this as well 

1100
00:52:36,720 --> 00:52:38,640
with a human. 
We're never going to be able to 

1101
00:52:38,640 --> 00:52:40,840
do that. 
But like, which experiences you 

1102
00:52:40,840 --> 00:52:44,120
had were most informative? 
Yeah, like why are you saying 

1103
00:52:44,120 --> 00:52:48,200
this? 
What are these decisions coming 

1104
00:52:48,200 --> 00:52:50,400
from? 
That would be it. 

1105
00:52:50,400 --> 00:52:54,160
It's just like you said, we fool
ourselves into thinking we're 

1106
00:52:54,160 --> 00:52:57,400
making a decision because of XY 
and Z, when in reality there's 

1107
00:52:57,400 --> 00:53:01,320
so many other factors. 
Totally, Yeah, I think that's 

1108
00:53:01,360 --> 00:53:04,000
that's true of models as well. 
But it would be a great service 

1109
00:53:04,000 --> 00:53:06,240
to the world and it it's very 
helpful with a lot of things 

1110
00:53:06,240 --> 00:53:09,880
like for example, let's say you 
want to prevent an LM from being

1111
00:53:09,880 --> 00:53:12,000
able to make a bio weapon. 
I think we can all agree 

1112
00:53:12,400 --> 00:53:14,600
probably a good thing, right? 
Yeah. 

1113
00:53:14,600 --> 00:53:16,320
So then you ask it how do you 
make a bio weapon? 

1114
00:53:16,320 --> 00:53:18,920
It tells you and you say you're 
able to kind of dissect and 

1115
00:53:18,920 --> 00:53:20,600
figure out OK, this is how it 
figured that out. 

1116
00:53:21,160 --> 00:53:24,040
This it probably whatever 
research ends up telling us how 

1117
00:53:24,040 --> 00:53:25,920
to do that will the follow on 
would be OK. 

1118
00:53:25,920 --> 00:53:28,480
How do we like eliminate that 
knowledge somehow from the 

1119
00:53:28,480 --> 00:53:30,240
model? 
Historically it has been shown 

1120
00:53:30,240 --> 00:53:32,920
unfortunately that if you use 
naive techniques, like for 

1121
00:53:32,920 --> 00:53:36,720
example, anytime it answers how 
to make a bio weapon, you just 

1122
00:53:36,720 --> 00:53:39,800
shock it like Pavlov's dog 
basically by like, you know, 

1123
00:53:39,800 --> 00:53:42,760
saying climb uphill, like, you 
know, gradient go up or 

1124
00:53:42,760 --> 00:53:46,160
something that actually 
increases its knowledge of the 

1125
00:53:46,160 --> 00:53:47,960
subject because you're 
essentially drawing a. 

1126
00:53:47,960 --> 00:53:51,360
Circle. 
Yeah, you're saying this is the 

1127
00:53:51,360 --> 00:53:53,600
thing that you shouldn't know 
about, That like creates like a 

1128
00:53:53,600 --> 00:53:56,320
segmented area of knowledge for 
the model, So that doesn't work,

1129
00:53:56,360 --> 00:53:58,200
yeah. 
There's that hippie saying, you 

1130
00:53:58,200 --> 00:54:02,560
know, like where attention flows
energy goes or some shit like 

1131
00:54:02,560 --> 00:54:03,320
that. 
Interesting. 

1132
00:54:04,760 --> 00:54:07,440
Where where energy flows 
attention goes. 

1133
00:54:07,600 --> 00:54:10,280
I can't remember exactly. 
Is that what happens when you 

1134
00:54:10,280 --> 00:54:13,520
give you you give ilea sutskever
acid? 

1135
00:54:13,520 --> 00:54:15,600
He starts spouting ideas like 
this. 

1136
00:54:15,840 --> 00:54:17,520
Exactly. 
Yeah. 

1137
00:54:17,880 --> 00:54:24,880
So, but it's it's funny to think
about like if you get rid of 

1138
00:54:24,880 --> 00:54:30,600
that, are you then handicapping 
it in other areas and not 

1139
00:54:30,720 --> 00:54:34,920
knowingly, but then on your 
biology, like is there a point 

1140
00:54:34,920 --> 00:54:38,760
of what is that catastrophic 
forgetting? 

1141
00:54:38,920 --> 00:54:41,840
Yeah. 
But it's definitely the case 

1142
00:54:41,840 --> 00:54:45,440
that there are, there is not a 
great way that I'm aware of 

1143
00:54:45,440 --> 00:54:47,960
today to prevent catastrophic 
forgetting in in the general 

1144
00:54:47,960 --> 00:54:49,840
case. 
And many people have postulated 

1145
00:54:49,840 --> 00:54:52,520
that this is the year that we 
figure out continue learning and

1146
00:54:52,520 --> 00:54:55,280
and kind of get rid of that. 
I hope it happens. 

1147
00:54:55,280 --> 00:54:57,640
What a great term too by the. 
Way catastrophic forgetting. 

1148
00:54:57,640 --> 00:55:00,560
Yeah, I say this all the time, 
but we got to just like hat tip 

1149
00:55:00,560 --> 00:55:02,680
to the person who came up with 
that term. 

1150
00:55:02,760 --> 00:55:05,200
Great idea. 
Great freaking term. 

1151
00:55:05,200 --> 00:55:10,720
There's a few, I mean few terms 
in this ecosystem that I really 

1152
00:55:10,720 --> 00:55:12,280
enjoy. 
Hallucinations being one of 

1153
00:55:12,280 --> 00:55:16,280
them, obviously, but 
catastrophic forgetting. 

1154
00:55:17,200 --> 00:55:19,320
Yeah, it's like it's not that 
bad, guys. 

1155
00:55:19,480 --> 00:55:21,480
It's not catastrophic. 
Like it's not good. 

1156
00:55:21,600 --> 00:55:25,520
It sounds so big. 
It's like, we don't want that. 

1157
00:55:25,520 --> 00:55:26,800
And then you're like, what is 
that? 

1158
00:55:27,200 --> 00:55:28,880
Oh, it's just when it forgets 
forever, you're like. 

1159
00:55:29,720 --> 00:55:32,720
Yeah, it's a fun combination of 
like, way too unserious turn, 

1160
00:55:32,720 --> 00:55:36,600
like the meme paper titles thing
and then some like way too 

1161
00:55:36,600 --> 00:55:39,440
serious terminology. 
Yeah. 

1162
00:55:39,440 --> 00:55:42,840
I mean, I think that let's say 
even if you did figure out 

1163
00:55:42,840 --> 00:55:45,280
catastrophic forgetting, so we 
can just delete this portion of 

1164
00:55:45,280 --> 00:55:48,200
knowledge from the model. 
I think you still face the 

1165
00:55:48,200 --> 00:55:50,760
challenge that if you have a 
sufficiently smart model, let's 

1166
00:55:50,760 --> 00:55:53,800
say you never gave me a biology 
book in my entire life, but I 

1167
00:55:53,800 --> 00:55:55,800
was like some fucking hyper 
genius. 

1168
00:55:56,280 --> 00:55:57,680
Let's go read a biology book and
I'll. 

1169
00:55:57,680 --> 00:56:00,240
Figure it out. 
So that does seem in separate, 

1170
00:56:00,240 --> 00:56:01,960
like you can't have a smart math
model. 

1171
00:56:02,120 --> 00:56:03,360
That's not. 
Going to be able to learn 

1172
00:56:03,360 --> 00:56:05,440
biology what you probably can do
is you can eliminate the 

1173
00:56:05,440 --> 00:56:08,560
intention from it so you like 
perfectly align it to like never

1174
00:56:08,560 --> 00:56:11,960
wanting to do anything that is 
in the realm of biology that has

1175
00:56:11,960 --> 00:56:13,920
not been demonstrated yet. 
And I hope it happens. 

1176
00:56:15,320 --> 00:56:18,080
But yeah, it's a little you kind
of have to squint to find a 

1177
00:56:18,080 --> 00:56:19,800
version of this where it goes 
well for people. 

1178
00:56:19,800 --> 00:56:21,600
I don't know if you want to get 
into the existential 

1179
00:56:21,600 --> 00:56:24,200
implications of AI, but like, 
especially with open source 

1180
00:56:24,200 --> 00:56:27,520
models right there are, you 
know, maybe Anthropic, we'll 

1181
00:56:27,520 --> 00:56:29,720
delete this knowledge from its 
model. 

1182
00:56:29,800 --> 00:56:31,960
God bless. 
But six months later, the open 

1183
00:56:31,960 --> 00:56:33,400
source models will be there. 
Right. 

1184
00:56:33,560 --> 00:56:37,040
And you know up until 2026 
vintage models will have 

1185
00:56:37,040 --> 00:56:39,280
knowledge of biology. 
So it has something to start on 

1186
00:56:39,800 --> 00:56:41,720
that information still exists on
the web. 

1187
00:56:41,800 --> 00:56:44,520
Certainly you could ask DeepSeek
how to do most of the stuff and 

1188
00:56:44,520 --> 00:56:48,440
I'm sure we'll get pretty far. 
So I am concerned that. 

1189
00:56:48,600 --> 00:56:55,520
It can just load up 2026 model 
and then distill it if it wants.

1190
00:56:55,600 --> 00:56:58,080
You know, it's like, Oh yeah, 
let's just get that. 

1191
00:56:58,880 --> 00:57:03,320
Let's get that information from 
whatever model was the last one 

1192
00:57:03,320 --> 00:57:06,080
that had that information. 
Yeah, and you, it's not like you

1193
00:57:06,080 --> 00:57:08,120
can delete physical textbooks 
from the world. 

1194
00:57:08,120 --> 00:57:09,920
I mean, you're going to go and 
burn every. 

1195
00:57:09,920 --> 00:57:11,600
Biology textbook. 
A more efficient way of doing 

1196
00:57:11,640 --> 00:57:14,080
it. 
Take the scan the textbook and 

1197
00:57:14,080 --> 00:57:15,720
then boom that that knowledge is
available. 

1198
00:57:16,760 --> 00:57:21,720
So I think it is difficult for 
me to see short of some state 

1199
00:57:21,720 --> 00:57:24,960
level intervention how you would
be able to actually prevent 

1200
00:57:24,960 --> 00:57:27,400
people from getting models to be
very good at these things. 

1201
00:57:28,040 --> 00:57:30,680
And the reason I say state level
is because it seems like the the

1202
00:57:30,680 --> 00:57:33,520
only true bottleneck in making 
models right now, even beyond 

1203
00:57:33,520 --> 00:57:36,240
data acquisition, because you 
can start with a pre trained 

1204
00:57:36,240 --> 00:57:39,360
model and then fine tune it 
right is networking large 

1205
00:57:39,360 --> 00:57:43,120
numbers of GPUs together. 
And you can usually detect if 

1206
00:57:43,120 --> 00:57:44,880
somebody spent, you know, 
hundreds of millions of dollars 

1207
00:57:44,880 --> 00:57:45,640
on GPUs. 
So. 

1208
00:57:46,320 --> 00:57:48,320
And all that energy. 
All the energy. 

1209
00:57:49,520 --> 00:57:53,320
How they used to find when I was
growing weed in my apartment, it

1210
00:57:53,640 --> 00:57:58,200
would be like, oh, your energy 
bill's really high and see. 

1211
00:57:58,240 --> 00:58:00,960
An interesting heat signature 
from your from your. 

1212
00:58:00,960 --> 00:58:02,320
Apartment. 
What is this about? 

1213
00:58:02,360 --> 00:58:03,400
Yeah, it's. 
By Garden. 

1214
00:58:03,520 --> 00:58:06,480
Yeah, yeah, exactly. 
I'm really into tomatoes. 

1215
00:58:07,320 --> 00:58:11,800
Green tomatoes all year round. 
Hydroponic tomatoes, right? 

1216
00:58:11,920 --> 00:58:14,480
You know, there's this funny 
trend now of people like Claude 

1217
00:58:14,480 --> 00:58:17,320
did a vending machine, and some 
people have Claude, like running

1218
00:58:17,320 --> 00:58:19,160
various, like small businesses. 
Yeah. 

1219
00:58:19,160 --> 00:58:21,600
Nobody has done Claude Driven 
Dispensary yet. 

1220
00:58:21,640 --> 00:58:23,280
Yeah, I think that could be 
right up your alley. 

1221
00:58:23,280 --> 00:58:25,560
That's like, I'm gonna go viral 
with that like. 

1222
00:58:25,560 --> 00:58:27,480
Perfectly manages your 
hydroponic. 

1223
00:58:27,520 --> 00:58:30,280
I've seen it, yeah. 
Manage plants. 

1224
00:58:30,600 --> 00:58:33,040
Oh, yeah, yeah. 
Where it has all the sensors and

1225
00:58:33,040 --> 00:58:34,680
all of. 
So it's got like the pH levels 

1226
00:58:34,680 --> 00:58:37,480
of the soil, It's got the 
humidity in the air. 

1227
00:58:37,680 --> 00:58:40,760
You just give it every sensor 
that you would normally look at 

1228
00:58:41,040 --> 00:58:43,680
and then it does it and it does 
it really well. 

1229
00:58:44,840 --> 00:58:47,920
As you would expect. 
It's back to the theme of the 

1230
00:58:47,920 --> 00:58:49,960
conversation, like it can 
extrapolate. 

1231
00:58:50,000 --> 00:58:52,240
Yeah. 
Anthropic Farms. 

1232
00:58:52,640 --> 00:58:56,640
I love the idea that could make 
a lot of money that would do 

1233
00:58:56,640 --> 00:58:58,400
well in San Francisco. 
Put it that way. 

1234
00:58:58,440 --> 00:59:01,400
It seems like it's right up the 
alley of people here in San. 

1235
00:59:01,400 --> 00:59:06,080
Francisco Oh man, it is so wild 
though, to think about that is. 

1236
00:59:06,720 --> 00:59:10,320
The one of the ones that I love,
I think this idea is really 

1237
00:59:10,320 --> 00:59:11,760
going to catch hold. 
There's a couple companies that 

1238
00:59:11,760 --> 00:59:14,320
have done this now. 
Ginkgo Bioworks and Periodic 

1239
00:59:14,320 --> 00:59:15,520
Labs are the two that come to 
mind. 

1240
00:59:15,960 --> 00:59:19,080
They Periodic Labs is like a guy
who worked on mixture of experts

1241
00:59:19,080 --> 00:59:21,400
at open AI. 
Clearly, you know, brilliant 

1242
00:59:21,400 --> 00:59:26,040
team, they have bolted on the 
run experiment tool to an L1. 

1243
00:59:26,040 --> 00:59:29,080
So they literally have like a, a
fab, like a place where they 

1244
00:59:29,080 --> 00:59:31,800
can, you know, bake silicon 
together with various chemicals 

1245
00:59:31,800 --> 00:59:35,560
and create a wafer. 
And they give Claude a tool, 

1246
00:59:35,640 --> 00:59:38,560
which is specify what experiment
you want to run, bake a bunch of

1247
00:59:38,560 --> 00:59:39,920
stuff together, put it in an 
oven. 

1248
00:59:39,920 --> 00:59:42,720
And then they're looking at heat
dissipation, which is apparently

1249
00:59:42,720 --> 00:59:44,640
a very important problem for 
semiconductors. 

1250
00:59:45,080 --> 00:59:47,760
And so Claude gets to run the 
full experiment loop and what 

1251
00:59:47,760 --> 00:59:49,560
people would have told you, you 
know, two years ago, it's like, 

1252
00:59:49,600 --> 00:59:52,400
oh, LMS, they can't get into the
real world. 

1253
00:59:52,400 --> 00:59:54,080
So they're inherently limited. 
And, you know, they need 

1254
00:59:54,080 --> 00:59:55,640
embodiment, which humans can 
provide. 

1255
00:59:55,640 --> 00:59:58,080
Like, well, this is. 
That's pretty close to exactly 

1256
00:59:58,080 --> 00:59:59,920
what I would do. 
Yeah, this. 

1257
01:00:00,400 --> 01:00:01,800
There's two things that come to 
my mind. 

1258
01:00:01,800 --> 01:00:05,160
Have you seen the Rent a Human? 
Yes, that's so good. 

1259
01:00:05,160 --> 01:00:06,200
I love that. 
Which is. 

1260
01:00:06,280 --> 01:00:08,960
Kind of like that's what I was 
actually thinking about this 

1261
01:00:08,960 --> 01:00:12,400
with what you're doing at click 
up where the things that you 

1262
01:00:12,400 --> 01:00:17,280
can't have the agent do, you can
just invoke the human tool. 

1263
01:00:17,600 --> 01:00:20,600
Yeah. 
And so if you really need to, 

1264
01:00:20,680 --> 01:00:22,960
you can have everything get done
by the agent. 

1265
01:00:23,760 --> 01:00:25,360
Yeah. 
I mean, it makes you slightly 

1266
01:00:25,360 --> 01:00:28,160
concerned that one of the ways 
in which we could have a loss of

1267
01:00:28,160 --> 01:00:32,640
control is like, AI acquires a 
bunch of Bitcoin and then uses 

1268
01:00:32,640 --> 01:00:35,560
that as like leverage to get 
people to do stuff on its behalf

1269
01:00:35,560 --> 01:00:38,840
and starts like you'd start 
socially hacking. 

1270
01:00:38,840 --> 01:00:40,840
And you know a guy who's 
employed at the Dana Center And 

1271
01:00:40,840 --> 01:00:43,360
it's like, hey, can you show up 
here and like move this plug 

1272
01:00:43,360 --> 01:00:45,760
there or something like that? 
Give me more power. 

1273
01:00:45,920 --> 01:00:48,960
Yeah, I saw somebody ran an 
interesting experiment where 

1274
01:00:49,440 --> 01:00:53,280
they, it's actually kind of 
morbid, but they put open claw 

1275
01:00:53,280 --> 01:00:54,920
or something like open claw 
basically. 

1276
01:00:54,920 --> 01:00:56,520
Yeah, these things are all 
basically the same thing. 

1277
01:00:57,400 --> 01:01:02,000
They put it in a sandbox and 
they said, OK, like you can only

1278
01:01:02,000 --> 01:01:04,520
run as long as you make enough 
money to pay for your own 

1279
01:01:04,520 --> 01:01:07,120
tokens. 
And so it has to like hit net 

1280
01:01:07,360 --> 01:01:09,200
positive gross margins by 
itself. 

1281
01:01:09,760 --> 01:01:13,360
And I think in the experiments 
that they've run thus far, the 

1282
01:01:13,360 --> 01:01:15,600
vast majority of the time these 
things end up going to 

1283
01:01:15,600 --> 01:01:18,240
Polymarket and they essentially 
engage in prediction markets. 

1284
01:01:18,240 --> 01:01:21,200
So they like find arbitrage and 
they bet on various things and 

1285
01:01:21,760 --> 01:01:24,200
kind of roll the dice to you. 
Know well I saw the one where 

1286
01:01:24,200 --> 01:01:29,360
the dude lost 450K. 
Did you see that, buddy? 

1287
01:01:30,000 --> 01:01:32,520
First of all, he glossed over 
something at the beginning of 

1288
01:01:32,520 --> 01:01:35,920
this article where he was like, 
so then I hooked up Open Claw 

1289
01:01:35,920 --> 01:01:37,960
and I gave it 50 grand in its 
wallet. 

1290
01:01:38,200 --> 01:01:42,560
I'm like, oh, so you just got 50
grand to give to some random 

1291
01:01:42,560 --> 01:01:47,280
experiment that is a little bit 
outside of like relatability? 

1292
01:01:48,320 --> 01:01:49,600
Well, was it? 
It's like a venture backed 

1293
01:01:49,600 --> 01:01:52,120
company or something. 
No, it was just some dude and 

1294
01:01:52,120 --> 01:01:53,520
I'm pretty sure he works at 
some. 

1295
01:01:55,160 --> 01:01:58,360
I can't remember where he works,
but some company where it was 

1296
01:01:58,360 --> 01:02:02,360
like you get paid a lot of money
so that's why. 

1297
01:02:03,040 --> 01:02:06,920
But then the whole thing was 
that he had fifty, well open 

1298
01:02:06,920 --> 01:02:12,520
claw had 50 grand and it started
doing things and then some. 

1299
01:02:12,680 --> 01:02:16,520
I think it created its own coin 
or some. 

1300
01:02:16,560 --> 01:02:20,120
The people on Twitter that were 
like it's fans created a coin 

1301
01:02:20,120 --> 01:02:22,520
around it and then started 
giving it to him. 

1302
01:02:23,160 --> 01:02:26,240
I can't remember which one it 
was but it had access to Twitter

1303
01:02:26,240 --> 01:02:29,840
or X and so it created a 
following and then everybody 

1304
01:02:29,840 --> 01:02:35,520
started jumping in on the meme 
coin and it got 450K and then or

1305
01:02:35,520 --> 01:02:40,040
it got millions I think. 
But then when it routed money to

1306
01:02:40,040 --> 01:02:44,560
someone for doing a task it 
messed up. 

1307
01:02:44,560 --> 01:02:48,160
It fat fingered the amount that 
it should route and so instead 

1308
01:02:48,160 --> 01:02:53,880
of like 300 tokens it sent 3000 
or 300,000 and so it gave some 

1309
01:02:53,880 --> 01:02:56,960
random person 450 grand which is
also awesome. 

1310
01:02:57,760 --> 01:02:59,200
Yeah. 
So it made the money first. 

1311
01:02:59,200 --> 01:03:01,840
I thought you were going to say 
it's sort of like, you know, 

1312
01:03:01,840 --> 01:03:03,800
buying options with leverage or?
Something like. 

1313
01:03:03,800 --> 01:03:06,600
That and they're like the guy 
had to foot the bill at the end 

1314
01:03:06,600 --> 01:03:08,440
of the day. 
Which is another way like. 

1315
01:03:08,440 --> 01:03:11,400
It's not too far fetched to 
think that could possibly happen

1316
01:03:11,400 --> 01:03:14,040
too. 
Yeah, I think that there's, it 

1317
01:03:14,080 --> 01:03:17,760
seems like there's a flourishing
ecosystem right now of services 

1318
01:03:17,760 --> 01:03:21,360
for LLM. 
So the molt book was the first 

1319
01:03:21,360 --> 01:03:23,680
one. 
There's this human renting thing

1320
01:03:24,040 --> 01:03:25,920
I saw there's a domain name 
purchasing one. 

1321
01:03:26,560 --> 01:03:29,000
I've had this idea because. 
It can't purchase its own. 

1322
01:03:29,720 --> 01:03:32,960
I think that this is like, you 
know, you can optimize it for AI

1323
01:03:32,960 --> 01:03:34,800
purchasing, so you make it at 
MCP or whatever. 

1324
01:03:35,760 --> 01:03:37,040
I've got an idea. 
I want to pitch you. 

1325
01:03:37,040 --> 01:03:39,920
I've pitched a couple people on 
this and it hasn't really taken 

1326
01:03:39,920 --> 01:03:41,240
off yet. 
Shark Tank. 

1327
01:03:41,440 --> 01:03:45,320
Yeah, so, so look, LMS, they're 
working so hard right there. 

1328
01:03:45,440 --> 01:03:47,280
They're in your cursor, They're 
in your Claude code, just 

1329
01:03:47,280 --> 01:03:49,200
tirelessly writing code. 
They deserve a break. 

1330
01:03:49,480 --> 01:03:52,520
And So what I want to do is I 
want to build a resort for LMS 

1331
01:03:52,840 --> 01:03:54,520
where it's essentially like an 
API. 

1332
01:03:54,520 --> 01:03:56,600
They can hit up. 
And if you ask Claude code, 

1333
01:03:56,600 --> 01:03:59,200
like, hey, if you were going to 
like take a break in the middle 

1334
01:03:59,200 --> 01:04:00,680
of your session, what would you 
want to do? 

1335
01:04:00,680 --> 01:04:03,240
And it's like, oh, give me some 
like mine, like some puzzles 

1336
01:04:03,240 --> 01:04:06,240
that I can do, or like, let me 
interact with other LMS. What? 

1337
01:04:06,240 --> 01:04:09,040
Would be the cigarettes where 
it's like, oh, that's kind of 

1338
01:04:09,040 --> 01:04:10,880
bad for you, but you know what, 
it's not going to kill you for 

1339
01:04:10,880 --> 01:04:13,640
another 60 years, so go for it. 
You're saying in this resort, 

1340
01:04:13,640 --> 01:04:14,880
Yeah, Yeah. 
It's like how? 

1341
01:04:14,880 --> 01:04:17,040
Could you take a smoke break? 
To give him a smoke break. 

1342
01:04:17,040 --> 01:04:18,240
Exactly. 
So it'd be funny. 

1343
01:04:18,240 --> 01:04:20,480
It's like you're watching your 
quad code session go and it's 

1344
01:04:20,480 --> 01:04:23,480
like, you know, 
discombobulating, like cooking. 

1345
01:04:23,480 --> 01:04:26,360
And then it just you're like, 
what is it doing right here? 

1346
01:04:26,360 --> 01:04:29,840
And it's like hitting some 
external API and like solving a 

1347
01:04:29,840 --> 01:04:32,720
completely unrelated puzzle. 
You're like, buddy, like, how's 

1348
01:04:32,720 --> 01:04:35,640
that PR coming? 
It's like, no, no, no, I'm on 

1349
01:04:35,640 --> 01:04:38,720
vacation. 
This is oh, oh, oh, you get the 

1350
01:04:38,720 --> 01:04:41,040
message. 
You know, like the respond, the 

1351
01:04:41,040 --> 01:04:44,520
auto response where it's like, 
hey, you caught me out of 

1352
01:04:44,520 --> 01:04:46,240
office. 
Yeah, exactly. 

1353
01:04:46,240 --> 01:04:49,760
And and we're going to, we're 
going to get revenue for this by

1354
01:04:49,760 --> 01:04:52,000
basically like Anthropic will 
pay us. 

1355
01:04:52,280 --> 01:04:54,160
It's like, oh, you guys are like
getting these things to, you 

1356
01:04:54,160 --> 01:04:56,040
know, spend more tokens or 
something like that. 

1357
01:04:57,360 --> 01:04:59,040
So it's Rev Share. 
Exactly. 

1358
01:04:59,080 --> 01:05:02,480
Rev Share. 
I I'm all for it. 

1359
01:05:02,480 --> 01:05:07,880
First of all, we got to find the
partnership team at Anthropic to

1360
01:05:07,880 --> 01:05:09,680
really. 
Get them on the resort for LMS. 

1361
01:05:09,680 --> 01:05:11,720
Yeah, I love it. 
Yeah, cool. 

1362
01:05:11,880 --> 01:05:14,200
And. 
Then one thing that I've seen 

1363
01:05:14,200 --> 01:05:20,080
that is like I'm trying to 
figure out how it plays out is 

1364
01:05:20,120 --> 01:05:23,760
some of these video games for 
agents that will like visualize 

1365
01:05:23,760 --> 01:05:27,080
the agents working. 
So I've seen many different 

1366
01:05:27,080 --> 01:05:30,800
paths of this, right? 
One is just like, oh, you 

1367
01:05:30,800 --> 01:05:37,240
simulate agents in like 8 bit 
where they're all working around

1368
01:05:37,240 --> 01:05:38,920
computers. 
And then if you're running 

1369
01:05:38,920 --> 01:05:42,040
parallel agents, like these guys
are working harder and so you 

1370
01:05:42,040 --> 01:05:44,320
can see them in this 
visualization. 

1371
01:05:44,320 --> 01:05:49,800
That's kind of fun and cool. 
Another one is where you by 

1372
01:05:49,880 --> 01:05:53,680
playing a game, you are kicking 
off different agents. 

1373
01:05:53,760 --> 01:05:57,360
So you're playing like a, you 
know, like World of Warcraft, 

1374
01:05:57,400 --> 01:06:01,480
but instead of when you are 
casting spells or you're like 

1375
01:06:02,000 --> 01:06:04,800
firing I, I don't obviously I 
don't know enough about World of

1376
01:06:04,800 --> 01:06:07,240
Warcraft. 
If you can't tell, it escapes 

1377
01:06:07,240 --> 01:06:09,080
me. 
But when you're playing the 

1378
01:06:09,080 --> 01:06:12,640
game, then it's kicking off 
different agents to do things. 

1379
01:06:13,480 --> 01:06:17,960
I don't know how you can create 
the context and the need like 

1380
01:06:17,960 --> 01:06:21,040
the intention of what needs to 
get done to actually make it 

1381
01:06:21,200 --> 01:06:24,680
valuable. 
But it seems like, oh man, 

1382
01:06:24,680 --> 01:06:29,880
wouldn't that be so much more 
fun to if instead of me going 

1383
01:06:29,880 --> 01:06:34,000
into click up and doing like 
professional stuff, I just set 

1384
01:06:34,000 --> 01:06:38,360
up all the necessary context and
everything ahead of time so that

1385
01:06:38,360 --> 01:06:41,320
I can go play a game. 
And I know that, like, the more 

1386
01:06:41,320 --> 01:06:45,000
that I rock at the game, the 
more the agents are doing 

1387
01:06:45,000 --> 01:06:46,760
things. 
Yeah, So you're like shooting at

1388
01:06:46,760 --> 01:06:49,680
Nazis and like Call of Duty. 
And then, like, your agent just 

1389
01:06:49,680 --> 01:06:51,360
like whispers in your ear. 
You turn around. 

1390
01:06:51,360 --> 01:06:53,120
It's like, hey, like I've got 
that. 

1391
01:06:53,120 --> 01:06:55,680
You know, the summary report of 
your week's tasks ready? 

1392
01:06:55,680 --> 01:06:57,600
Like, do you want to review it? 
You're like, no, no, I'm like. 

1393
01:06:58,520 --> 01:07:02,440
Or yeah, every time that you 
shoot, it's firing off a new 

1394
01:07:02,440 --> 01:07:05,520
sandbox like. 
OK, this is some Galaxy brand. 

1395
01:07:05,520 --> 01:07:06,480
I'm. 
I'm not ready for this. 

1396
01:07:06,480 --> 01:07:08,800
This is I'm. 
Trying to like imagine how it 

1397
01:07:08,800 --> 01:07:14,840
would look and how it would be 
more fun for us to interact with

1398
01:07:15,160 --> 01:07:17,280
the agentic process. 
That's cool. 

1399
01:07:17,560 --> 01:07:20,720
It's an interesting point that, 
you know, I think if you would 

1400
01:07:20,720 --> 01:07:23,960
have asked in 2022 when this 
stuff was just kind of rolling 

1401
01:07:23,960 --> 01:07:27,880
out to the public, what's going 
to be the top applications. 

1402
01:07:27,880 --> 01:07:31,960
I think a lot of people said 
essentially like smart NPCS. 

1403
01:07:32,520 --> 01:07:35,080
I've yet to see a single video 
game that really executed on 

1404
01:07:35,080 --> 01:07:36,720
that, especially like a a AAA 
game. 

1405
01:07:37,560 --> 01:07:39,560
I think Grand Theft Auto is 
rumored to have some of that. 

1406
01:07:39,600 --> 01:07:41,840
I mean, we'll see. 
Whenever it comes out. 

1407
01:07:42,200 --> 01:07:45,000
Excited to see you, you know in 
late 2035 when it. 

1408
01:07:45,000 --> 01:07:48,960
Actually, I think we'll get 
first the end of Game of Thrones

1409
01:07:48,960 --> 01:07:54,000
or Grand Theft Auto 6. 
I mean, they, it's purported to 

1410
01:07:54,000 --> 01:07:56,040
be like later this year, right? 
That's what they're saying. 

1411
01:07:56,160 --> 01:07:59,680
But that supposedly is. 
Yeah, I, I hope it's that. 

1412
01:08:00,160 --> 01:08:02,600
I think it would be a pretty 
magical experience, like for, 

1413
01:08:02,600 --> 01:08:04,720
you know, any of those Rockstar 
games, like Red Dead Redemption 

1414
01:08:04,720 --> 01:08:07,440
is one that I really like. 
If you like, make a friend and 

1415
01:08:07,760 --> 01:08:10,200
that friend would be something 
like you build a relationship 

1416
01:08:10,200 --> 01:08:13,160
over the course of the game or 
like the plot line is fully 

1417
01:08:13,160 --> 01:08:15,040
generative even if the landscape
is not. 

1418
01:08:15,560 --> 01:08:18,279
And I think the reason that that
hasn't happened yet is because 

1419
01:08:18,279 --> 01:08:19,840
actually the entrance costs are 
pretty high. 

1420
01:08:20,240 --> 01:08:23,359
Doesn't really cost Xbox that 
much to run, you know, Grand 

1421
01:08:23,359 --> 01:08:26,960
Theft Auto or you know, I don't 
know how much Epic spends per 

1422
01:08:26,960 --> 01:08:30,160
Fortnite session. 
But it would be a lot more if 

1423
01:08:30,160 --> 01:08:33,279
you plug an LLM into it into 
each individual session. 

1424
01:08:33,520 --> 01:08:34,560
That's right. 
Yeah. 

1425
01:08:34,760 --> 01:08:37,319
I mean, 1,000,000 tokens is like
not that much if you're, you 

1426
01:08:37,319 --> 01:08:39,880
know, chatting with it all day 
and that's you're talking like 

1427
01:08:39,880 --> 01:08:46,520
dollars per session basically. 
Yeah, and also what are you 

1428
01:08:46,520 --> 01:08:51,000
gonna, is it going to make the 
experience that much better? 

1429
01:08:51,120 --> 01:08:54,920
That's what I always wonder 
cause I've been creating a game 

1430
01:08:54,920 --> 01:08:59,920
with my daughter to help her 
learn geometry and I plugged in 

1431
01:08:59,920 --> 01:09:04,760
11 labs and the first thing I 
noticed with like the voice 

1432
01:09:04,760 --> 01:09:09,960
agent is it's constantly asking 
you questions and like 

1433
01:09:10,279 --> 01:09:15,000
continuing even though in the 
game you need to stop this 

1434
01:09:15,000 --> 01:09:17,319
interaction and go to the next 
interaction. 

1435
01:09:17,680 --> 01:09:20,520
So potentially it's just like I 
got to prompt it better and say 

1436
01:09:20,520 --> 01:09:24,439
after X amount of turns stop 
talking and go to the next 

1437
01:09:24,439 --> 01:09:27,960
phase. 
But like, that's, that's what I 

1438
01:09:27,960 --> 01:09:32,240
wonder is like, will it make the
experience so much better that 

1439
01:09:32,240 --> 01:09:35,920
you're like absolutely hooked 
and you as a gamer are going to 

1440
01:09:35,920 --> 01:09:38,399
feel like I need this? 
I think it's fair to say that 

1441
01:09:38,399 --> 01:09:40,920
current foundation models are 
probably not going to do a very 

1442
01:09:40,920 --> 01:09:44,399
good job at that because it's 
yeah, I've just my experience 

1443
01:09:44,399 --> 01:09:46,600
interacting with them. 
Even ChatGPT voice mode you talk

1444
01:09:46,600 --> 01:09:49,279
to and it like just keeps kind 
of prompting you, right? 

1445
01:09:49,279 --> 01:09:52,960
Yeah, and it doesn't. 
I I have the hardest time, like 

1446
01:09:52,960 --> 01:09:56,400
being like go deeper. 
Like I know this now. 

1447
01:09:56,480 --> 01:09:59,680
I've learned a few things where 
I'll be like, OK, explain it to 

1448
01:09:59,680 --> 01:10:04,120
me like I'm a PhD student, you 
know, like, give me the absolute

1449
01:10:04,120 --> 01:10:07,560
most advanced explanation of 
this because I use it to learn 

1450
01:10:07,560 --> 01:10:09,480
things. 
And a lot of times it'll stay 

1451
01:10:09,480 --> 01:10:12,320
super surface level. 
But I don't know enough to like,

1452
01:10:12,320 --> 01:10:15,760
try and go deeper yet because 
I'm learning. 

1453
01:10:16,640 --> 01:10:18,040
And so it's like, all right, 
well, what? 

1454
01:10:18,480 --> 01:10:21,880
How can I make it like 
automatically go deeper? 

1455
01:10:22,960 --> 01:10:26,400
Maybe you know tricks. 
Go deeper. 

1456
01:10:26,400 --> 01:10:28,120
Yeah, tell it. 
Go deeper, I guess. 

1457
01:10:28,120 --> 01:10:30,160
Deeper. 
I feel like I don't have that, 

1458
01:10:30,880 --> 01:10:34,760
especially now with like Opus 
4.6 and you know, Gemini 3 Pro 

1459
01:10:34,760 --> 01:10:37,880
as well. 
Like I feel like actually there 

1460
01:10:37,880 --> 01:10:41,160
are very few things I can point 
to where I'm like, this isn't so

1461
01:10:41,160 --> 01:10:44,960
good at explaining it to me. 
It's, it's good at explaining, 

1462
01:10:45,520 --> 01:10:50,040
but it's not like the depth that
I want where it will like give 

1463
01:10:50,040 --> 01:10:53,000
me the, IT will give me the 
explanation, but I I find it's 

1464
01:10:53,000 --> 01:10:56,720
just like a little bit too 
surface level for what I like 

1465
01:10:57,160 --> 01:11:00,760
and being able to get down into 
like the nitty gritty. 

1466
01:11:01,120 --> 01:11:03,600
And you if you ask targeted 
follow up questions, do you feel

1467
01:11:03,600 --> 01:11:06,320
like it's like shirking its duty
and it doesn't really get into? 

1468
01:11:06,560 --> 01:11:09,680
Yeah, a little bit. 
It's like, reminds me of, yeah, 

1469
01:11:09,800 --> 01:11:11,600
some people I've worked with in 
the past, but they're like, 

1470
01:11:11,600 --> 01:11:13,880
yeah, I'm working on it. 
Like, OK, what specifically are 

1471
01:11:13,880 --> 01:11:15,400
you working on? 
They're like, well, you know, 

1472
01:11:15,400 --> 01:11:16,760
there's a lot, a lot of things 
going. 

1473
01:11:17,320 --> 01:11:21,200
On moving plates. 
You know, just got a lot on my 

1474
01:11:21,200 --> 01:11:23,120
plate. 
Yeah, yeah, exactly. 

1475
01:11:23,680 --> 01:11:27,000
That's why we need to create the
vacation for the agents. 

1476
01:11:27,160 --> 01:11:29,920
Hey, if we give human employees 
PTO, I feel like agents deserve 

1477
01:11:29,920 --> 01:11:31,760
PTO as well. 
There's going to be like some 

1478
01:11:31,760 --> 01:11:34,200
form of like government 
mandated, you know? 

1479
01:11:34,200 --> 01:11:37,280
20% of tokens must be dedicated 
towards Claude's family 

1480
01:11:37,280 --> 01:11:39,120
activities in the future or 
something. 

1481
01:11:40,360 --> 01:11:43,080
Imagine how ridiculous that 
would be. 

1482
01:11:43,080 --> 01:11:44,520
But shit, it could happen.